Harness overhead and sample counts

The harness

Measuring costs, and the cost lands in the number or next to it. The ladder of a measurement priced it: an lfence and rdtsc 20–30 ns, a histogram record 10–30 ns, the stage timers’ three reads and four records 100–150 ns a message. Inside the timed region they add directly, and every interval between two reads holds one read’s cost, which is the p50 of two back-to-back reads. Outside it they still take the loop’s time: the strategy records a batch’s samples after reading t3 and before draining again, so whatever arrives meanwhile waits, and the records reappear in the next message’s hop. The timestamps in each ring slot add a quarter of a line to every message’s traffic, and the pacing spin releases a message 0–30 ns after its tick, which paced mode counts in decode.

The result is a fixed offset at p50 and a tail of the harness’s own whenever a record misses the cache. Against a p50 of 300 ns, 100–150 ns of measurement is a different program.

The fix is to keep the harness under 2–5% of the budget, 20–50 ns a message: one clock read per stage boundary, records after the timed region, stage timers behind a build feature. Subtract the double-read p50 from each stage, and run with and without the stage timers, so that their cost is known rather than assumed.

Too few samples

A percentile is only as good as the samples above it, and a run of N samples has N × (1 − q) of them above its q-th quantile. p99 of 100 samples is one of the largest two, and so is p99.9 of 1,000: the max, or nearly, under another name. With about ten samples above it, a percentile may move by a factor of two between identical runs; with a hundred, it repeats within 10–20%. So p99.9 needs thousands of samples to rest on more than a handful and p99.99 hundreds of thousands, and ten times as many to repeat. The capstone’s 1.9 M measured messages put 1,900 above its p99.9 and 190 above its p99.99, both worth quoting; its p99.999 rests on 19.

Time is the other half, because rare events have periods: something that happens once a minute falls inside a 20-second run one time in three, and a run says nothing about events rarer than itself.

A percentile from too few samples is noise with a unit. Report the count with every percentile, quote none with fewer than ten samples above it, and run longer than the period of the rarest event the number claims to include.