Parsing in place without allocation

Reading in place

Zero-copy parsing reads each field where it lies in the receive buffer. The plain safe way to read one is u64::from_le_bytes(buf[16..24].try_into().unwrap()). The slice index checks the bounds, with a compare and a branch to a panic, and the compiler removes such checks only where it can prove them redundant. Converting the prefix once to a &[u8; 48] establishes a length bound for constant offsets. This safe parser also checks the header and side:

#[derive(Debug, PartialEq)]
struct Update {
    instrument: u32,
    sequence: u64,
    price: u64,
    qty: u64,
    side: u8,
}

fn parse_update(buf: &[u8]) -> Option<Update> {
    let b: &[u8; 48] = buf.get(..48)?.try_into().ok()?;
    if u16::from_le_bytes(b[0..2].try_into().ok()?) != 48
        || u16::from_le_bytes(b[2..4].try_into().ok()?) != 1
        || b[40] > 1
    {
        return None;
    }
    Some(Update {
        instrument: u32::from_le_bytes(b[4..8].try_into().ok()?),
        sequence: u64::from_le_bytes(b[8..16].try_into().ok()?),
        price: u64::from_le_bytes(b[16..24].try_into().ok()?),
        qty: u64::from_le_bytes(b[24..32].try_into().ok()?),
        side: b[40],
    })
}

let mut frame = [0u8; 49]; // one-byte offset exercises unaligned input
let b = &mut frame[1..];
b[0..2].copy_from_slice(&48u16.to_le_bytes());
b[2..4].copy_from_slice(&1u16.to_le_bytes());
b[4..8].copy_from_slice(&42u32.to_le_bytes());
b[8..16].copy_from_slice(&7u64.to_le_bytes());
b[16..24].copy_from_slice(&6812345u64.to_le_bytes());
b[24..32].copy_from_slice(&2u64.to_le_bytes());
assert_eq!(parse_update(b), Some(Update {
    instrument: 42, sequence: 7, price: 6812345, qty: 2, side: 0,
}));
assert!(parse_update(&b[..47]).is_none());
b[40] = 2;
assert!(parse_update(b).is_none());
b[40] = 0;
b[2] = 9;
assert!(parse_update(b).is_none());

The caller retains bytes after the first 48 for the next message and checks instrument validity, scaling and sequence continuity before updating state. Constant-offset conversions can compile to plain loads; inspect the target assembly rather than promising a fixed instruction count.

Byte order depends on who defined the field. Protocol headers are big-endian, the internet’s convention: IP and TCP, a TLS record’s length, a WebSocket frame’s extended length. Most exchange binary formats, SBE by default among them, are little-endian, because both ends are x86 or AArch64 and a little-endian field needs no byte swap there. A binary feed over WebSocket meets both orders in one path: from_be_bytes for the frame length and from_le_bytes for the price. A big-endian load is a mov and a bswap, or one movbe, so the only cost of byte order is getting it wrong.

Alignment is the subtler trap. A u64’s alignment is 8, meaning its address should be a multiple of 8. The hardware barely cares: x86-64 and AArch64 load an unaligned u64 at full speed unless it straddles two cache lines. Rust cares absolutely: a reference to a u64, or to a struct holding one, at a misaligned address is undefined behaviour even if it is never read, because the compiler may generate code that assumes the alignment. And messages in a stream land wherever the previous one ended, after a 2-byte length, a 4-byte WebSocket header or a 5-byte TLS header. So casting buf.as_ptr() to a RawUpdate pointer and making a reference can be invalid for alignment and for insufficient length or invalid field values. repr(C) does not discharge any of those obligations.

std::ptr::read_unaligned copies a value from any address into an aligned local, one unaligned mov on x86, but leaves the length check, and the promise that every bit pattern is a valid value, to the caller inside unsafe. The zerocopy crate makes the overlay safe. Its derives prove at compile time that any bytes form a valid value of the type (FromBytes, which rules out bool, char, references and most enums), that its alignment is 1 (Unaligned), that its layout is known (KnownLayout), and that nothing in it can change behind a shared reference (Immutable). Its byte-order wrappers can represent integer fields at alignment 1, with .get() converting byte order. Derive the required traits and assert the complete wire type’s size is 48, including explicit reserved bytes.

ref_from_prefix checks that the slice holds at least 48 bytes and returns a reference into the buffer, along with the rest of the slice. Nothing is copied, each field access is a load at a fixed offset, and the const assertion breaks the build if the struct’s size ever changes.

Why allocation is the real cost

The convenient JSON path is serde_json::from_slice into a struct with String fields, and each String is an allocation and a copy; a serde_json::Value is a map for every object and a String for every key and string value. The worked example has six non-empty string values, so a matching owned struct needs storage for each. A Value also owns keys and object maps, whose allocation counts depend on features, capacity growth and implementation. Chapter 2 priced them: 20–50 ns each on the allocator’s fast path, and about as much again for each free, so six of each add 240–600 ns to every message. On the slow path, when the arena has run dry, an allocation costs a system call and page faults, at a moment the allocator chooses, and bursts of messages are what drive it there.

The scan is a steady cost that cannot be removed while the format is text. The allocations are where the tail comes from, nothing in the code shows them, and they can be removed. A deserialised struct can use &'a str fields, such as price: &'a str, with #[serde(borrow)] on its nested data field. That borrows the unescaped strings from the input buffer instead of allocating owned Strings.

This version allocates nothing. A string that contains an escape cannot be borrowed, because its bytes differ from its value, and serde_json then returns an error instead of quietly allocating; a Cow<'a, str> field marked #[serde(borrow)] borrows when it can and allocates only in that case. The price stays text until a short loop over its digits turns it into integer ticks, with no f64 along the way.

When JSON cannot be avoided and the rate is high, simd-json, the Rust port of Geoff Langdale and Daniel Lemire’s simdjson, finds the structural characters 64 bytes at a time with vector instructions and parses in place, writing unescaped strings back into the input so that it can lend even escaped ones. Its gains are largest on large documents, so on 200-byte messages it is worth measuring on the venue’s own traffic; it still reads every byte.

What it costs

The binary read touches 48 bytes, one cache line or two, already in L1 because recv has just written them there. Five fields take five loads, which the compiler merges where it can. There are two branches, on the length and on the type, both always predicted, and nothing is allocated. With the framing check around it and the fields stored where the strategy reads them, the step costs 10–20 ns, which is 40–80 cycles at 4 GHz and 1–2% of the budget.

The JSON parse of the same update, the 161-byte object below, spans three or four cache lines and must be scanned, classified and converted; optimised parsers can handle several bytes per step. On top of those come nine keys compared against the expected names, two integers of 10 and 19 digits converted, fifteen strings scanned to their closing quotes and checked for escapes, and the price and quantity converted again from text: several hundred branches, of which a few are mispredicted wherever this message differs from the last (a quantity one digit longer, a side that flips), each miss throwing away 15–20 cycles. With owned strings, add six allocations and six frees. In all the parse costs 1–3 µs, which is 4,000–12,000 cycles at 4 GHz, one to three budgets, and a hundred times the binary read. Borrowing removes the allocations and moves the cost towards the bottom of that range; it removes nothing else. A tight benchmark that parses the same message over and over, with everything warm in the caches and the predictor, can come in under 1 µs; a running feed handler, whose parser shares L1 and the predictor with the rest of the program, sits inside the range.

There is an indirect cost as well. The binary read is a dozen instructions; the JSON parser is several kilobytes of code and tables, and the allocator’s free lists add more, all competing for the L1 that also holds the strategy’s code and the book.

Worked example: one depth update, two encodings

A feed thread on core 2 receives depth updates from a venue that publishes the same stream in two encodings: the 48-byte binary update above, at 10–20 ns, and this 161-byte JSON object on one line (excluding the trailing newline), at an illustrative 1–3 µs:

{"type":"depth","symbol":"BTCUSDT","seq":1234567890,"ts":1720000000000000000,"data":{"price":"68123.45000000","qty":"2.00000000","side":"bid","venue":"EXAMPLE"}}

Map the symbol to the binary instrument id at setup, and use the same integer tick and lot scales in both parsers. Both paths must validate equivalent fields for a fair comparison.

On an ordinary day the venue sends 2,000 updates a second. JSON costs 2,000 × 1–3 µs = 2–6 ms of the core each second, 0.2–0.6% of it, and binary 20–40 µs. Throughput is not the problem; latency is. Every JSON update puts 1–3 µs between the last bit on the wire and the strategy’s decision, one to three budgets spent before the strategy has looked at the price, and every binary update 10–20 ns.

In a burst the venue sends 20,000 a second, and JSON still takes only 2–6% of the core. The average misleads, because a market move sends updates in clumps, and one recv can return ten at once, the tenth waiting behind the nine parsed before it: 9 × 1–3 µs = 9–27 µs before its own parse starts in JSON, against 90–180 ns in binary. Its completion includes one more parse. If 40 updates arrive in 100 µs, one every 2.5 µs, a JSON parser at 3 µs a message needs 120 µs for them, the queue grows through the whole burst, and the last update is handled about 22 µs after it arrived; the binary parser spends 0.4–0.8 µs on all 40. With owned strings the burst also makes 120,000 allocations and as many frees a second, each a chance to hit the allocator’s slow path at the worst moment.

Seeing it

perf stat -e cycles,instructions,branches,branch-misses ./parse --binary, over ten million messages replayed from a capture, gives the per-message cost when each count is divided by ten million: a few tens of cycles and instructions per message and almost no misses. With --json it shows 4,000–12,000 cycles, thousands of instructions and several hundred branches per message, and a few misses that together cost about as much as the whole binary read.

cargo asm shows what the compiler produced for parse_update and its caller. Look for constant-offset loads and the required validation branches, with no allocation or byte-scanning loop. LLVM may combine adjacent loads into vector moves; the exact sequence depends on target, compiler and caller.

A counting allocator, a GlobalAlloc that wraps the system allocator and bumps an atomic on every alloc, turns “the parse allocates nothing” into a check: warm up, sample the counter, parse a batch, and assert the counter is unchanged. Count alloc_zeroed and realloc too, and keep unrelated threads outside the measured interval.

It passes with the binary parser and with the borrowed JSON struct. It fails with the owned struct, by six allocations per message, and with serde_json::Value, by an implementation-dependent amount that must be measured.

perf record -g ./parse --json, then perf report, shows where the JSON time goes: serde_json’s string scanning (parse_str in its SliceRead), the derived Deserialize code matching keys to fields, parse_integer for the two numbers, and, with owned strings, malloc, free and memcpy. The binary parse barely registers.