Stage intervals and expected results

Four timestamps and five histograms

Five points are marked on each message’s path, all in ticks of the one invariant TSC:

1. t0, arrival: in paced mode the scheduled arrival tick, computed and not read; in flat mode now() immediately before decode.
2. t1, decode done: read on core 2 just before try_push.
3. t2, popped: read on core 3 inside the closure drain calls for each message.
4. t3, decision: read on core 3 once per batch, after the quoter has run, just before the order try_push. A batch whose top did not move still ends in a decision, to send nothing, and t3 is read at the same point.
5. t4, gateway popped: read on core 6 as each order leaves the output ring.

So the path reads the clock at four points, t1 to t4, and in paced mode computes the fifth. A difference between two cores’ readings, t2 on core 3 minus t1 on core 2, means something only because both cores read one synchronised counter, the condition set out in the discussion of the clock, and pinning keeps each reading on the core it names.

Each is one Histogram<u64> with bounds of 1 ns to 1 s and 3 significant figures, about 170 KB, allocated before the first message and recorded outside the timed region: the strategy thread records a batch’s samples after it has read t3, and the gateway after it has read t4. Ticks become nanoseconds as they are recorded, by the factor calibrate() measured at startup. Each histogram is reported as count, p50, p99, p99.9 and max, with the sequence number of the message that set the max, except out_hop, since the StampedOrder that crosses to core 6 holds t3 and the order and no sequence number; beside them, the worst 100 total samples are kept with their sequence numbers in a preallocated array, so the slow messages can be found in the replay file and run again on their own.

Three reads on the path and four records a message cost 100–150 ns against a p50 of about 300 ns, as the ladder of a measurement showed, which is too much to leave inside the number being reported. So t1 and t2, the two reads made on the path for every message, are compiled in only with the stages feature. The default build gives the headline, total, from t0 and the one t3 per batch; t4 is read on core 6 after the decision, where it cannot touch total; and the stages build’s total set against the default build’s is what the stage timers cost.

One property of the numbers follows from the loop’s shape. The strategy drains the ring and publishes once, the data-structures chapter’s rule, so orders are emitted after the whole batch has been applied, and total for the first message of a batch includes its batch-mates’ apply time: when a drain finds 30 messages, the first waits for 29 pops, checks and applies before its decision exists, a few nanoseconds each in the default build and 20–30 ns more each in the stages build, whose fenced t2 read stops the core overlapping them. That is real latency, not an artefact of measuring. It is the trade the data-structures chapter made on purpose, one line transfer per batch instead of one per message, and think and total show its price.

What done looks like

A finished capstone prints the five histograms, the worst 100, and counters that must read zero: sequence gaps, orders dropped on a full output ring, intervals that came out backwards, allocations after warm-up from the counting allocator, and, for the threads on cores 2, 3 and 6 after warm-up, page faults, context switches and migrations. A counter that is not zero means the histograms describe a different program from the one intended.

The numbers below are predictions, to be written down before the first run and measured against. They assume a 4 GHz x86 machine with cores 2 and 3 sharing an L3, the hot threads pinned and isolated, memory locked, the performance governor, and a paced run at 100,000 messages a second.

Decode is the networking chapter’s 10–20 ns on bytes already in cache, and its p99 is a record that starts a new line or page of the file. Hop is the ring’s transfers from core 2 to core 3, the head counter and then the slot, plus the wait until the spinning strategy next looks; its p99 is a slot that straddles two lines, or a poll caught in the middle of a pause. Think is the book chapter’s apply and quote plus the 20–30 ns of the clock read that ends the interval; its p99 is the 1% of updates that move the best price and walk the bitmap, and the bursts, where a message waits for its batch-mates. For each message, the three consecutive stage intervals add to its total. Their medians and other percentiles do not generally add; measure the total distribution separately. At p99 it is the bursts, the last of 30 messages waiting behind 29 decodes on core 2 and the first of a batch waiting for the others’ applies on core 3. At p99.9 it is the machine, a line evicted by something else or an interrupt that isolation did not remove. Its max is whatever the platform still does: tens of microseconds once pinned, isolated and locked, up to a millisecond when the scheduler is free to put something else on the core. out_hop is the hop’s trip in the other direction, to a gateway spinning on its own ring.