Processes and threads

0.5 What the operating system does to your program

Your program never has the machine to itself. Linux, the kernel, owns the hardware and lends it out, and Chapter 2 is about the cost of that lending. The words you need:

Kernel and user space. The core runs in one of two modes. In user mode, where your program runs, it can touch only its own memory and normally accesses devices through the kernel; explicitly mapped device memory is an exception used by kernel bypass. In kernel mode, the operating system can do anything. The boundary is enforced by the hardware, so the only way for a program to get something from the kernel is to ask.

System calls. The asking is a system call, or syscall: read, write, mmap, clock_gettime, sched_setaffinity. Each one switches the core into kernel mode, runs kernel code, and switches back. The switch and the kernel’s bookkeeping cost hundreds of nanoseconds to a few microseconds, and the caches the kernel touched are no longer holding your data when you return. A hot loop makes no syscalls at all; strace -c counts the ones you did not know you were making.

Processes and threads. A process is a running program with its own virtual address space; two processes cannot see each other’s memory unless they arrange to share some. A thread is one stream of execution inside a process; all the threads of a process share its memory, which is what makes Chapter 3’s problems possible. The kernel keeps a record for each thread and decides which core runs it.

The scheduler and preemption. There are usually more runnable threads than cores. The scheduler gives each thread a slice of time on a core, then interrupts it and runs another; your thread can be paused at any instruction, for tens of microseconds or more, without any sign in your code. It can also be moved to a different core, leaving its warm caches behind. Pinning a thread to one core (taskset, sched_setaffinity) and keeping other work off that core (isolcpus, nohz_full) are how Chapter 2 removes those pauses.

Interrupts. Devices and timers interrupt whatever a core is doing to run a kernel handler: a packet arrived, a timer tick, a disk finished. Each one evicts a little of your cache and delays your thread by a few microseconds. The kernel lets you steer most interrupts to cores you are not using.

Virtual memory and page faults. Section 0.4 introduced pages. When your program allocates memory, the kernel only promises it; the first write to each 4 KB page traps into the kernel to find a physical page and zero it, which costs microseconds per page. That is why Chapter 2 touches memory at startup, locks it with mlockall, and uses 2 MB huge pages so that one translation covers 512 times more memory.

Allocation. Vec::new() starts empty without allocating; growing a Vec or creating a non-zero-sized Box asks the allocator for memory, and the allocator sometimes asks the kernel for more with mmap or brk. On the hot path a program allocates nothing: everything is sized and allocated once at startup.

0.6 Threads sharing memory

Two threads of one process can read and write the same bytes, and nothing stops them doing so at the same moment. Chapter 3 is about making that safe and cheap; here is the vocabulary.

A data race involves conflicting accesses by different threads to the same location, at least one writing and at least one non-atomic, without synchronisation ordering them. Atomic increments racing with each other are not data races. counter += 1 from two threads is three steps each, load, add, store, and the steps interleave, so two increments can produce one. Rust’s compiler refuses to compile plain shared mutation across threads; the tools it offers instead are a Mutex, which lets one thread in at a time, and atomics, single operations the hardware guarantees are indivisible: AtomicU64::fetch_add(1) is one increment however many threads call it.

A mutex is simple and slow in the wrong place: an uncontended lock and unlock cost about 20–40 ns, and a contended one puts the loser to sleep through a syscall and wakes it microseconds later. Atomics avoid the sleep, but a line that several cores write still bounces between their caches; Chapter 1, section 1.5, shows that each bounce costs 50–100 ns.

Reordering is the surprise. The compiler may move loads and stores around if a single thread could not tell the difference, and the core does the same at run time. A thread that writes data and then sets a flag may, from another core’s point of view, set the flag first. The memory orderings on atomics (Relaxed, Acquire, Release, SeqCst) are the way to say which reorderings are forbidden, and Chapter 3 explains how they order accesses. The trap for x86 users is that Intel and AMD hardware reorders very little, so a wrong program passes its tests; the compiler still reorders, and ARM servers reorder a great deal.

The pattern the book keeps returning to is producer and consumer: one thread, the producer, writes items into a queue, and another, the consumer, takes them out. With one of each (single-producer, single-consumer, SPSC) the queue needs only two atomic counters and no locks. A thread can also just wait by spinning, re-reading a value in a loop until it changes; that burns a core but answers in nanoseconds, where sleeping and being woken costs microseconds. Hot paths spin.