Designing and checking the feed loop

How the program behaves

1. Poll, never wait. The hot thread’s loop is a non-blocking recv and the work on what it returns, and on an empty socket it calls recv again: no blocking wait, sleep or scheduler yield. A thread that sleeps pays 5–20 µs to wake, and 100–200 µs after a quiet spell, on the packet that ends the quiet.
2. One recv per batch of messages, into one buffer the thread owns, allocated at startup. Each complete message is parsed where it lies, with from_le_bytes or a zerocopy overlay, and the partial tail is moved to the front for the next recv to append to. The recv is the only copy, and nothing is allocated. Where the venue requires TLS, the records are decrypted in user space in the same thread, just ahead of the framer: measure kTLS against that path if its offloads or copy savings may help. Count allocations for the exact TLS API and version; a library name alone does not establish hot-path allocation behaviour.
3. Binary where the venue offers it: 10–20 ns for a 48-byte fixed layout, against 1–3 µs in a running program, and an allocation per owned string field, for the same fields as 200 bytes of JSON through serde_json. A few venues now offer a binary encoding of the same streams. Where only JSON is offered, it is parsed into borrowed &str fields, so that at least the allocations go.
4. Connections are warm: opened, authenticated and subscribed at startup, kept alive with the venue’s heartbeats, each with a spare pre-connected beside it. An idle spare may still need authentication, subscription and book recovery; a subscribed and sequence-aligned spare can make failover a switch, not by 140–210 ms of handshakes to a venue 35 ms away and a congestion window of 10 segments. net.ipv4.tcp_slow_start_after_idle=0 keeps a quiet order connection’s window from shrinking back towards that. Reconnects run on a housekeeping thread with randomised, growing delays.
5. One writer per socket. The order gateway alone writes the order socket, fed by single-producer single-consumer rings from the strategies; two writers would contend for its lock and could splice two messages into one corrupt stream.
6. Sends are batched into one write where the protocol allows: two orders that leave together go out in one call and one segment, from one buffer or several with write_vectored. MSG_MORE, or TCP_CORK set and cleared, is used only around a single message written in several calls; a cork left on holds data for up to 200 ms.

How it is checked

1. strace -c -p <tid>, or perf trace -s -t <tid> with far less overhead, on each hot thread after warm-up shows one recvfrom or recvmsg per batch and nothing else, the empty polls counted as EAGAIN errors, and on the gateway one write per batch of orders besides. An epoll_wait, futex, nanosleep, mmap or brk among them is a wait or an allocation that has crept in.
2. voluntary_ctxt_switches in /proc/<pid>/task/<tid>/status stays flat on every hot thread; each increment is a wake-up paid.
3. /proc/net/softnet_stat shows the second column, packets dropped, at zero, and the third, time squeezes, not growing on the hot core’s row. The card’s missed-frame counters in ethtool -S stay flat.
4. ss -ti records sender-side retransmissions and the actual rto; do not derive it by adding RTT to 200 ms. On an incoming feed these sender counters do not diagnose the venue’s retransmissions; inspect receive-side out-of-order counters and captures too.
5. /proc/interrupts, read twice ten seconds apart, grows only on the hot queue’s row in the hot core’s column, and in the columns of cores 0–2.
6. The wire-to-decision histogram, from the card’s receive stamp to the strategy’s decision, has its p99 within a few microseconds of its p50 on a quiet feed. A p99 tens of microseconds higher is a wake-up, a coalescing hold or an interrupt on the wrong core; a p99.9 in tens of milliseconds is a lost segment being recovered, and one near 200 ms a retransmission timeout.

The table

Three caveats

The kernel stack is right for everything that is not the hot socket, and most of a trading system is not: the control connections, the logs shipped off the box, the monitoring, ssh. They keep the defaults, and every rule above applies to the hot sockets and the hot card alone.

A busy-polling thread on a shared core is preempted like any other thread, and then its packets wait behind it, in the ring and in the socket’s queue, for as long as it is away. Everything above assumes each hot thread has a core of its own.

And 70 ms of fibre makes 15 µs of kernel invisible. Measure where the time goes, from the wire stamp to the decision and from the decision to the venue’s acknowledgement, before removing the stack or tuning it; to a venue across a sea, where the machine sits and how TCP recovers from loss are worth more than every setting in this section.