Waiting, idle states and wake-ups

2.8 Waiting: the idle path

When a thread has nothing to do it can tell the kernel so, and almost every thread on the box does. That is blocking, the fourth service: the kernel parks the thread, gives the core to someone else or to nobody, and brings the thread back when what it waited for arrives. Everything about the design assumes that waking a little late is fine, because for a thousand ordinary threads it is. This section follows what blocking sets in motion, what the wake-up costs, and why the cost is highest exactly when the wait was longest.

What blocking sets in motion

Blocking is a system call (futex, epoll_wait, recv without MSG_DONTWAIT, nanosleep), after which the scheduler marks the thread sleeping and picks another. If nothing else is runnable it runs the idle task, a kernel thread whose job is to execute an instruction (mwait or hlt) that halts the core until an interrupt arrives. How deeply it halts is a C-state. C0 is running. C1 stops the clock and comes back in about 2 µs. Deeper states cut voltage and, at C6, flush the private caches and power the core down, taking 100–200 µs to come back on server parts, with L1 and L2 empty afterwards. The exit latency of each state is listed in /sys/devices/system/cpu/cpu<n>/cpuidle/state*/latency, in microseconds. A general-purpose box is idle most of the time, so the deeper the better for power and heat, and the cpuidle governor predicts how long each idle spell will last and picks the deepest state it expects to pay for. The prediction is right for a web server. For a thread waiting for the next message it is wrong in the worst direction: the longer the quiet spell, the deeper the core goes, and the slower the thread wakes for the message that ends it.

Frequency goes the same way. The core’s clock is governed by cpufreq (the performance, powersave and schedutil governors) or, on recent Intel parts, by the hardware itself with hints from the kernel; either way an idle core has been clocked down and takes tens of microseconds to ramp back after it wakes, and every cycle count in this book assumes it has.

Timers add a delay of their own. The kernel keeps a high-resolution timer for every nanosleep, every epoll_wait timeout and the like, and to avoid waking a core for each one separately it is allowed to delay a thread’s timed wake-up by that thread’s timer slack, commonly 50 µs for a normal thread and not applied to real-time scheduling policies, so that nearby wake-ups coalesce. A 1 µs sleep can therefore last roughly 50–60 µs in a common setup, but slack is permission to delay, not a mandatory 50 µs minimum; scheduling can also delay it further. See timer slack.

Below all of this sits firmware. The system management interrupt (SMI) is taken by the BIOS, not the kernel, for thermal and power management, error logging and legacy device emulation; the kernel cannot intercept or mask it and learns of it only afterwards, from a count the CPU keeps in a model-specific register, and it steals 10–200 µs from whichever core it lands on, idle or not.

What a wake-up costs

Put the pieces in order. The data arrives; the waker, a network softirq on some core or a thread on another, marks your thread runnable and sends the reschedule interrupt to its core; the core leaves its C-state; the scheduler runs and picks your thread; your thread resumes, into cold caches if the state was deep. 5–20 µs is a good day. 100–200 µs after a quiet spell is normal, and the quiet spell is exactly when the next message is the one that matters. Sleeping instead of blocking is worse, since the timer slack sits on top: a sleeping poll loop can add tens of microseconds. prctl(PR_SET_TIMERSLACK, 1) reduces allowed coalescing to 1 ns but does not remove syscall, scheduling or idle-exit costs.

Not waiting

There is one way to wait that costs none of this: do not tell the kernel. A thread that checks the socket, or the ring buffer’s index, or the NIC’s descriptor ring, in a loop of its own, sees new data within one iteration, sub-microsecond, and stays runnable. Ring polling can remain in user space, while an ordinary non-blocking socket receive still makes a syscall on every poll. Its core never idles, so it never enters a C-state, never pays the wake-up, and keeps its caches warm through the quiet spell. The price is one core at 100% forever, its power and heat, and the slowing of its hyperthread sibling. For a thousand ordinary threads that is a bad trade, which is why the kernel’s default is the other one; for one thread whose reaction time is the product, the trade reverses, because the alternative costs ten to two hundred budgets at the moment it matters most.

spin_loop compiles to pause, which tells the core that the loop is a spin, so it stops speculating ahead on the loads and yields issue slots to the sibling; the instruction itself took about 10 cycles on Intel parts before Skylake-SP and about 140 since, so it belongs once per empty poll and not between every two reads of a hot flag. yield_now does not belong in the spin at all: sched_yield is a system call and an invitation to the scheduler.

The spinning thread’s own core stays awake by construction. Every other core still idles between its work, and anything the hot thread needs from them, a wake-up, a reply, an interrupt taken on its behalf, arrives with their C-state’s exit latency attached. The kernel’s power-management quality-of-service interface exists for drivers that cannot tolerate slow wake-ups: a limit in microseconds, and no core enters an idle state with a longer exit latency while the limit is held. /dev/cpu_dma_latency is that interface opened from user space, and writing zero requests a zero-microsecond idle-exit limit. That constrains the idle governor; it is not a portable guarantee of one particular polling state. Newer hardware offers a middle way between spinning and halting, umonitor/umwait and tpause on Intel from Sapphire Rapids and wfe on ARM: lighter waits whose support and wake-up behaviour depend on the CPU and OS policy. ARM wfe waits for an event and is not a drop-in monitored-address equivalent of x86 umwait; using either requires a target-specific protocol.