Polling and parsing labs

Lab 4.3 — poll vs wait (ch04-network/src/bin/pollwait.rs)

The binary is an echo client and server for 100-byte messages, the size of the chapter’s depth update, over UDP, or over TCP with --tcp. The server runs on the second machine as --echo and never changes: a non-blocking recv loop with SO_BUSY_POLL on an isolated core of a box held out of deep C-states, so that only the client’s way of waiting differs between runs. With --gap <µs> the server holds each reply that long, spinning, and the client subtracts the gap from each round trip, so that it waits between messages as a feed thread waits between updates. The client, pinned to an isolated core with --core <n>, makes 100,000 round trips after warm-up and prints p50, p99, max and its own CPU time. --blocking waits in a blocking recv. --epoll registers its one socket with mio, waits in Poll::poll with no timeout and drains the socket to WouldBlock. --spin loops on a non-blocking recv, and --busy-poll does the same with SO_BUSY_POLL at 50 µs. --c0 holds /dev/cpu_dma_latency at 0 for the run, Chapter 2’s tool; without it the client’s box idles as deeply as its governor likes. Run all four modes with and without --c0 at --gap 2000, a quiet feed of 500 messages a second, then --blocking once more without --c0 at --gap 0.

Reason it out. The server’s half and the wire are the same in every run: the ladder’s busy-polled round trip less one busy-polled receive, about 5–12 µs. Each mode adds its own receive, which the waiting section’s worked example prices. --busy-poll adds about 3 µs at p50 and 5–6 µs at p99, for round trips of 8–15 µs, and --c0 changes nothing, because the poll and the walk run on a core that never idles. --spin with --c0 adds 1–2 µs more, for the interrupt and the cross-core transfer. Without --c0 it is not immune: the softirq that walks its packets runs on a housekeeping core that now sleeps between them, and many round trips pay that core’s exit from C6, 100–200 µs. --epoll with --c0 adds the card and the kernel’s receive path, 8–10 µs at p50 and 20–40 µs at p99, for round trips of 13–22 µs and 25–50 µs, near the bottom of each range on two otherwise idle boxes; --blocking saves the epoll_wait, about 0.5 µs. Without --c0, a 2 ms gap lets the governor choose its deepest state, and both sleeping modes add 100–200 µs: a p50 of 105–210 µs, a p99 of several hundred, and a max in milliseconds whenever the interrupt’s core slept too. The --gap 0 run should sit near the --c0 numbers, because a core idle for 15–30 µs is never sent deep: quiet sets a sleeping reader’s tail. The CPU cost runs the other way. The sleeping modes work for 5–20 µs in every 2 ms, 1% at most, and top -H shows the thread near 0%. The spinning modes show 100% at any gap, a CPU-second a second under perf stat -e task-clock, and at 100–300 ns an empty poll --spin makes 3–10 million polls a second, about 7,000–20,000 per round trip, --busy-poll somewhat fewer. Against --epoll with --c0, that core buys 5–7 µs at p50 and about 15–35 µs at p99; against a core allowed to sleep, 100–200 µs on every message.

Pitfalls. A server that changes between runs puts its own wake-ups into every number. --c0 holds every core of the box out of deep states, the interrupt’s core included, which is why --spin differs with and without it; turbostat prints each core’s residency in each C-state, and a BIOS that caps C-states makes the two halves of the lab alike. Keep coalescing off on both cards, or every mode but --busy-poll pays rx-usecs, a true lesson that hides this one, and steer the interrupt as in Lab 4.1. A lost datagram would hang a blocking client, so every message carries a sequence number and every wait a 100 ms timeout; report timed-out/lost messages separately alongside the successful-latency histogram; silently excluding them would hide the worst outcomes. Investigate repeated losses using NIC, softnet_stat and RcvbufErrors counters; they need not all be caused by a full socket queue. voluntary_ctxt_switches should rise by one a round trip for the sleeping modes and not at all for the spinning ones. Besides the send, strace -c should show one recvfrom a round trip for --blocking, one epoll_wait and two recvfrom, one of them EAGAIN, for --epoll, and only recvfrom for the spinning modes. When busy polling works, TcpExtBusyPollRxPackets rises, for UDP as well, and the processed count in softnet_stat moves from the interrupt core’s row to the client core’s. Write down, for each mode with and without --c0, the p50, p99 and max, the thread’s share of its core, and the context switches and system calls per round trip; the empty polls a second for the spinning modes; and the --gap 0 run beside its twin.

Lab 4.4 — parse it (ch04-network/src/bin/parse.rs)

At startup the binary generates 256 depth updates from a fixed seed, with prices, quantities, sides and sequence numbers varying as a feed’s do, and writes each twice into buffers allocated once. The first copy is the chapter’s 48-byte #[repr(C)] update: length, type and instrument in the first eight bytes, then the sequence number, the price in ticks, the quantity in lots and the venue’s timestamp as little-endian 8-byte integers, then a side byte and seven bytes of padding. The second is the same fields in JSON, shaped like the chapter’s 161-byte example. The binary records where each message starts, so that the timed loop does no framing, then parses 10 million messages, cycling through the 256 after a million of warm-up, and hands each result to std::hint::black_box. --binary uses the safe parser above, or an equivalently validated zerocopy overlay, and copies out the five fields the strategy uses. Both encoders write explicit reserved bytes, not uninitialised Rust padding. --json deserialises with serde_json::from_slice into an owned struct with six String fields and turns the price and quantity into integer ticks and lots. --json-borrowed does the same into a struct of &str fields borrowed from the buffer, and --json-value parses into a serde_json::Value and reads the same fields out of it. The binary prints nanoseconds per message, the measured loop’s wall time over 10 million, and allocations per message from a counting allocator. For process-wide perf stat, counts include startup and warm-up: divide by all 11 million parses only as an approximation, or scope counters to the timed 10-million-message interval. Do not divide whole-process counts by just the timed work.

Reason it out. --binary is a length check, a type check, five loads and five stores: a dozen or two instructions, branches that always go the same way, and no allocation. Expect the chapter’s 10–20 ns, 40–80 cycles, or a little less, since in a loop the core overlaps one message’s loads with the next’s; the instruction count, which no overlap changes, is the steadier figure. --json-borrowed walks about 161 bytes, compares nine keys, converts 29 digits, scans fifteen strings and turns two decimal strings into integers: thousands of instructions and several hundred branches a message, a few of them missed at 15–20 cycles each. The part’s 1–3 µs is the parse in a running handler, and this loop is the warm benchmark that it notes can come in under 1 µs, so expect under 1 µs. --json adds six allocations and six frees at 20–50 ns each on the allocator’s fast path, 240–600 ns more, which puts it around 1 µs. --json-value also owns nine keys and two object maps. Map nodes, growth and feature flags make its allocation count implementation-dependent; measure it. Predict zero for the binary and borrowed paths and roughly six owned-string allocations for this unescaped fixture, then explain any difference rather than asserting a fixed performance ratio. --json less --json-borrowed, divided by six, is the price of one allocation and its free: 40–100 ns, or somewhat less in a loop this warm.

Pitfalls. Without black_box the compiler can delete the binary parse and report a fraction of a nanosecond. One message parsed 10 million times lets the branch predictor learn its every digit; 256 varied ones keep the few misses a real feed causes. serde_json’s default object map and its preserve_order alternative can allocate differently, so record feature flags alongside allocation counts. A borrowed struct fails, rather than allocates, on a string with an escape in it; the generated messages have none, and a real feed’s may. The allocator sets the price of every allocation, so record which one is linked. cargo asm should show the required header and side validation and constant-offset loads; extra checks may be correct, and instruction counts depend on compiler and target. strace -c should show the same few dozen calls for every mode, all at startup; a count that grows with the messages is the allocator going to the kernel. Write down, per message and for each mode, the nanoseconds, cycles, instructions, branches, branch misses and allocations, the ratio to --binary, and the derived price of an allocation and its free.