AOS and SOA

In computing, AoS and SoA refer to contrasting ways to arrange a sequence of records in memory, with regards to interleaving, and are of interest in SIMD programming.

Structure of arrays

Structure of arrays (or SoA) is a layout separating elements of a record (or 'struct' in the C programming language) into one parallel array per field. ^[1] The motivation is easier manipulation with packed SIMD instructions in most instruction set architectures, since a single SIMD register can load homogeneous data, possibly transferred by a wide internal datapath (e.g. 128-bit). The downside is requiring more cache ways when traversing data, and inefficient indexed addressing. (see also: planar image format)

Array of structures

Array of structures (or AoS) is the opposite (and more conventional) layout, in which data for different fields is interleaved. This is often more intuitive, and supported directly by most programming languages.

Alternatives

A hybrid approach is possible, e.g. de-interleaving a number of objects that correspond to the number of SIMD lanes.

It is also possible to split some subset of a structure (rather than each individual field) into a parallel array – this can actually improve locality of reference if different pieces of fields are used at different times in the program (see data oriented design).

Some SIMD architectures provide strided load/store instructions to load homogeneous data from SoA format. Yet another option used in some CELL libraries is to de-interleave data from AoS format when loading sources into registers, and interleave when writing out results (facilitated by superscalar issue of permutes). Some vector maths libraries align floating point 4D vectors with the SIMD register to leverage the associated data path and instructions, whilst still providing programmer convenience, although this does not scale to SIMD units wider than four lanes.

In the GPU stream processing model, SoA layout is generally not required, as they provide true vector addressing.

4D vectors

AOS vs SOA presents a choice when considering 3D or 4D vector data on machines with 4-lane SIMD hardware. SIMD ISAs are usually designed for homogenous data, however some provide a dot product instruction^[2] and additional permutes making the AOS case easier to handle. As of 2016, most GPUs have moved away from 4D instructions to scalar SIMT pipelines,^[3] for better performance for compute kernels.

Software support

Most languages support the AOS format more naturally by combining records and various array abstract data types. The experimental JAI programming language is a recent attempt to provide language level SOA support.^[4] Julia language, supports multi-dimensional arrays, with AOS, or SOA (through a package).

References

• http://www.cc.gatech.edu/~hyesoon/spr10/lec_cuda5.pdf