NUMA and SIMD

1.8 NUMA

A two-socket server is two computers sharing one address space. Each socket has its own cores, L3, memory controllers and DRAM, and the sockets are joined by a link (UPI on Intel, Infinity Fabric on AMD). Any core can read any address, but a read from the other socket’s DRAM must cross the link. This is non-uniform memory access, or NUMA.

numactl --hardware reports NUMA distances as relative weights, not nanoseconds. In practice, local DRAM costs about 80–100 ns and remote DRAM about 150–200 ns. A line held Modified by a core on the other socket costs more still, and the link’s bandwidth is shared by all cross-socket traffic.

Linux places memory by first touch. Allocating only reserves addresses; the physical page is chosen when a thread first writes to it, on that thread’s current node. The classic bug follows. The main thread, on node 0, allocates and zeroes a 2 GB order table. The hot thread runs on node 1, and every one of its misses is remote. Nothing in the code looks wrong, and memory latency has roughly doubled.

Thread migration does similar damage at runtime. A hot thread moved to another core leaves its warm L1 and L2 behind and pays hundreds of misses to rebuild them. If it crosses sockets, all its memory becomes remote too.

The remedy is to pin the hot thread to one core and allocate and initialise its data from that thread. For a whole process, numactl --cpunodebind=1 --membind=1 ./quoter does both. Check which socket your network card hangs off with cat /sys/class/net/<iface>/device/numa_node and run the hot thread there. Chapter 2 covers pinning.

1.9 SIMD, briefly

SIMD (single instruction, multiple data) instructions work on wide registers that hold several values at once. With AVX2, one instruction compares 32 bytes against 32 others or adds four pairs of u64. AVX-512 registers are 64 bytes wide. When the same operation applies across many elements, SIMD divides the instruction count by the vector width.

In a trading system it pays for bulk byte work. Checksums over packets are one case. Scanning for a delimiter, such as the SOH byte between FIX fields, is another: compare 32 bytes in one instruction, turn the result into a 32-bit mask, and find the first match with trailing_zeros, as the memchr crate does. Parsing fixed-width ASCII numbers is a third: eight digits convert in a handful of shift, multiply and mask steps instead of the chain from section 1.6. Batch book snapshots are a fourth.

It rarely pays in the decision loop. Quoting logic compares a few values, branches and touches a few lines; there is nothing wide to vectorise, and moving scalars into vector registers costs more than it saves. On older Intel servers, heavy AVX-512 code could also lower the core’s clock for a while.

Rust gives you two routes. Autovectorisation is free: LLVM can vectorise suitable slice loops and reductions. A search that exits at the first match may need an explicit SIMD routine; the SoA layout alone does not guarantee vectorisation. Look for ymm or zmm registers in the assembly. The default target assumes only SSE2, so build with -C target-cpu=native to get AVX2. Explicit intrinsics in std::arch::x86_64, such as _mm256_cmpeq_epi8, give control, but the caller must establish CPU support, using runtime feature detection or an appropriate build target before entering an AVX2-enabled function.