Low-Latency Systems Fundamentals Draft
Low-Latency Systems Fundamentals
About this book
A program receives a message, updates some state, and sends a response. The code may fit on a screen. Explaining the time it takes means looking beyond that code: where its data lives, when its thread gets to run, what it shares with other threads, and how bytes reach it from the network.
This book follows those connections. It builds a model of the machine beneath a Rust program so you can look at a frequently executed piece of code, its hot path, and explain where the time goes. The aim is to make performance decisions you can justify with a prediction and test with a measurement.
Trading systems provide a running example because they bring these concerns together: messages arrive, shared state changes, and responses have deadlines. The same questions arise in services, messaging systems, and other software where response time matters.
Who this book is for
The examples use Rust, but you do not need to know Rust to read this book. Familiarity with basic programming concepts is enough to get started. You do not need a background in high-frequency trading, computer architecture, operating systems, or networking. The opening chapter introduces the vocabulary and mechanisms used throughout the book, including the market terminology behind the examples.
If you already know some of that material, use it as a refresher. Later chapters return to the same ideas in more detail and connect them to decisions in code.
The path through the book
The book contains eight chapters, numbered 0–7, each divided into short readings:
Chapter 0 — The machine from the top builds the foundations: CPUs, memory, processes, threads, networks, markets, and measurement.
Chapter 1 — Memory and the CPU follows data through caches and memory, then examines layout, branches, and the cost of sharing cache lines.
Chapter 2 — The Linux platform explains how system calls, scheduling, interrupts, page faults, and allocation affect a running program.
Chapter 3 — Sharing between threads develops the reasoning behind ownership, atomics, memory ordering, and communication between cores.
Chapter 4 — Talking between machines follows the network path through sockets, protocols, TLS, message framing, and parsing.
Chapter 5 — Shaping the data develops allocation-free data structures, dense lookups, bitmaps, and batched ring buffers.
Chapter 6 — The book builds a local order book, follows snapshot and stream recovery, and connects the book to a toy market maker.
Chapter 7 — The number explains clocks, distributions, and measurement pitfalls, then brings the preceding chapters together in the final trading loop.
How to read and work through it
On a first reading, follow the chapters in order. Each adds a layer to the model developed before it. Read one short section at a time, then use the labs and review questions to check whether you can explain the behaviour yourself. The reading map gives a more detailed view of the topics and exercises.
The habit behind every lab is predict first, then measure. Write down what you expect, run the experiment, and explain the difference. When the result surprises you, revisit the mechanism and the measurement setup before changing the code.
Write the hot-path code in the labs by hand. Use tools such as perf stat, perf record, Criterion, and the timing techniques introduced in the text to test your reasoning. Report latency distributions, including p50, p99, and the maximum observed value, with enough detail about the setup to make the result interpretable. The labs describe programs to build and experiments to run. Companion code is maintained separately from the book; the chapter paths identify the corresponding crates.
Reading the numbers
The text distinguishes latency estimates and predictions from reported measurements. Each figure belongs to its stated setup and provides a scale for reasoning; measure on your own machine before applying it to a design. Unless a section says otherwise, examples assume Linux on x86-64, 64-byte cache lines, and a 4 GHz core.
Memory sizes in page and cache calculations use binary units (KiB, MiB, GiB), including examples that retain older KB/MB/GB labels. Transfer rates use decimal GB/s. Record your own hardware and configuration when comparing a measurement with an example.
Start reading
Begin with Chapter 0 — The machine from the top. Keep one question in mind as you read: what has to happen between this line of code and the machine completing the work?
Further reading provides references for exploring each topic in more depth.