In What if the prefetcher is wrong for my code?, I described the switch and the experiments I wanted to run. I now have measurements from CPython, Valkey sorted sets, and SPY quote lookups. On this machine, the right prefetcher settings depend on the workload and where its memory lives.

The hardware prefetcher is my favorite example, because it looks like pure upside. A modern Intel core watches your memory accesses and, when it thinks it sees a pattern, pulls the next lines into cache before you ask for them. When the guess is right, the data is waiting and the load is free. There are four of these predictors on the Xeon this runs on: an L2 stream prefetcher, an L2 adjacent-line prefetcher, and two at L1, the DCU next-line and DCU IP-history prefetchers. All of them are on by default, and most of the time nobody thinks about them.

Here is the part that is easy to forget: a prefetch spends two scarce resources. The first is cache capacity. The cache is finite, so every line the prefetcher speculatively pulls in has to take the place of a line that was already there — and if the guess was wrong, you evicted something still live and turned a future hit into a miss. That is the opportunity cost of a bad prefetch: not the fetch itself, but the thing you threw away to make room for it. The second is memory bandwidth. Every speculative fetch, right or wrong, is real traffic on the memory bus, and that bus is shared — among the cores of a socket, and more narrowly still across the link to another socket. When it is saturated, a wasted prefetch doesn't merely cost you a cache line; it takes bandwidth straight from the demand loads that are actually stalling your program. So a bad prefetch can hurt in two different ways — polluting a cache that had no room to spare, or burning a bus that had no bandwidth to spare — and which one bites depends on whether your bottleneck is capacity or throughput.

For a workload with long, linear, predictable scans, that trade is a bargain. For a workload that chases pointers, or streams data past exactly once, or walks a hash table in a deliberately unpredictable order, the predictor can be systematically wrong, and then it is not a subsidy but a tax you cannot see on any profiler that only counts your instructions.

These knobs are available in the BIOS on some machines. The settings apply to the whole machine and require a reboot to change. A/B testing is possible, but rebooting between settings makes each experiment slow and cumbersome. I wanted to measure their effect and write a kernel patch that lets each process choose its own prefetcher settings. That means exposing another hardware control through the kernel, with a per-process interface in /proc.

So I built it. It is a small patch to a mainline Linux kernel. Each task carries a mask of which prefetchers it wants disabled; the kernel writes Intel's MSR_MISC_FEATURE_CONTROL (0x1A4) at context switch, so the setting follows the process onto whatever CPU it lands on. Userspace controls it through a directory that appears under every process:

ls /proc/self/prefetch_disable/
# l1_ip  l1_stream  l2_adjacent  l2_stream

echo 1 > /proc/self/prefetch_disable/l1_stream   # turn the L1 next-line prefetcher off
echo 1 > /proc/self/prefetch_disable/l1_ip        # and the L1 IP-history one
cat /proc/self/prefetch_disable/l2_stream         # 0 == still enabled

Each file is a 1 (that prefetcher disabled for this task) or a 0 (enabled). A process can set its own, or, with the right permissions, another process's, which matters when the thing you want to slow down is not the thing holding the shell.

file prefetcher
l2_stream L2 hardware stream prefetcher
l2_adjacent L2 adjacent cache-line (128-byte pair)
l1_stream L1 data-cache next-line prefetcher
l1_ip L1 data-cache IP-history prefetcher

There is one genuinely interesting wrinkle. The register is per-core, and a core has two hyperthreads that share it. So "this process wants the L2 prefetcher off" is not a question one thread can answer alone. I resolved it the conservative way: a prefetcher is disabled on the core if either sibling asks for it to be off. That is the right default for isolating an effect, and it is exactly the configuration you want when you deliberately place two processes on the two threads of one core and watch them interfere. To keep the whole thing cheap enough to leave on, each core caches the value it has already programmed, so the actual register only gets written when the effective setting changes, not on every context switch.

Alongside it I keep an unmodified kernel, identical in every other way, as the baseline. That is the only honest way to separate three different things: the cost of the feature existing, the cost of using it, and the effect of the prefetchers themselves.

First, does the switch actually work?

Before trusting a single result I made the knob prove itself on a workload that must love prefetching: a sequential scan of an array far larger than the 30 MB last-level cache. On one core:

sequential scan, 512 MB bandwidth
all prefetchers on 9.0 GB/s
L2 stream off 3.7 GB/s
all off 2.45 GB/s

Turning the prefetchers off makes a sequential scan 3.7× slower, exactly as it should — the demand loads now eat the full memory latency instead of finding the next line already waiting. Reading MSR 0x1A4 on the core confirmed the bits flip while the process runs and clear the instant it exits. The switch bites. Now the interesting part: the workloads where the answer is not obvious.

The interpreter: it doesn't matter at all

I started with CPython, because "make Python faster" is a perennial itch. I ran the official pyperformance suite — richards, chaos, raytrace, float, spectral_norm, nqueens, fannkuch, nbody, go — across all sixteen on/off combinations of the four prefetchers.

Nothing. Every benchmark stayed within about ±1% of the all-on baseline, and the tiny deltas scattered in both directions — the signature of noise, not signal. The reason is dull and total: these interpreter workloads have small working sets that live in L1 and L2, so the prefetcher has nothing to guess about and nothing to pollute. The default is simply irrelevant. If you were hoping the CPU's speculation was secretly holding your interpreter back, it isn't.

One cautionary note from this pass, because I nearly fooled myself: my first run spread the sixteen cases across four NUMA nodes and produced a tidy −4% "win." It was an artifact — the baseline had landed on the busiest node. Re-running all sixteen on one quiet core erased it. On effects this small, the confounds are bigger than the thing you're measuring.

Valkey sorted sets: the prefetcher is a rounding error next to layout

Something with a real working set, then. I loaded 5 million members into a Valkey (the Redis fork) sorted set — a skiplist, all pointer-chasing — and hammered it with ZRANGEBYSCORE ... LIMIT 0 128 range scans, toggling the server's prefetchers by PID.

Prefetch on was about 14% faster than off. A pleasant, ordinary result: a skiplist's bottom level is a sorted linked list and the scan walks it in order, so there's enough sequentiality for the prefetcher to help.

Except that "14%" turned out to be a story about how I built the set. I had inserted the members in score order, which laid the nodes out roughly sequentially in memory. When I rebuilt the identical set by inserting in shuffled order — same contents, scattered nodes — the prefetch benefit collapsed to under 3%, and the whole thing got 34% slower regardless of the prefetch setting. Churning the allocator made it worse still.

how the 5M zset was built throughput prefetch worth
sorted insert ~41,000 op/s +14%
shuffled insert ~27,000 op/s +3%
churned ~26,000 op/s +3%

The prefetcher was worth at most 14%. Insertion order was worth 37%. The new per-process control let me measure how much prefetching helped each layout. Changing the insertion order changed both the overall throughput and the benefit from prefetching.

Market data: same bytes, opposite answers

The real payoff came from a workload I care about: seeking through a day of SPY quotes with the massive-speedup.dev library. I used a compiled binary quote database — fixed 83-byte records, sorted by timestamp, mmapped straight out of a RAM-backed tmpfs so disk was never in the picture — and pinned the data to one NUMA node so I could also ask a second question: does the answer change when the memory lives across the QPI bus, on another socket?

That 83-byte record is the detail that matters. A cache line is 64 bytes, so a record never fits in one: every record straddles at least two lines, and one that starts late enough in a line straddles three. Read a record and you need the line it sits in and its successor — and, about a quarter of the time, the line after that too.

First workload: read one random record at a time, through the database's Python API. Here the prefetcher helps, and the reason is exactly the straddle — the L1 next-line prefetcher fetches the record's second line before you ask for it. And how much it helps depends entirely on where the memory is:

random record read, 12 cores prefetch worth
data local to the socket ~+10%
data across the QPI bus +19% (the L1 next-line prefetcher alone: +11%)

Locally, the second-line miss is cheap (~90 ns), so prefetching it saves little. Across the socket boundary that miss is expensive, so the next-line prefetcher hides a real stall and is worth twice as much. Same code, same data — the prefetcher's value scaled with how far away the memory was.

Second workload: a random binary search over the same records, written in C++ so there was no interpreter overhead to hide behind. Each query makes ~21 dependent, scattered probes, and with no Python in the way, 24 threads on a socket actually saturate the memory link. Now the prefetcher is a footgun:

random binary search, across QPI throughput vs. all-on
all prefetchers on 7.71 M query/s —
L2 stream off 9.42 M query/s +22%
all off 9.64 M query/s +25%

Turning the L2 stream prefetcher off made it 22% faster. On its early hops a binary search jumps unpredictably across the whole file, so the streamer speculates lines that are never used — and on a saturated cross-socket link that wasted traffic steals directly from the demand loads. Locally, where there's bandwidth to spare, disabling it changed nothing (+1.5%). Add the second hyperthread on each core and the win grew to +26%, because more threads deepen the saturation. (These are 5-run means; run-to-run noise was under 1%, so the effects aren't close to noise.)

So here is the same machine, the same SPY quotes, giving flatly opposite advice:

workload regime verdict who did it
CPython (pyperformance) fits in cache doesn't matter (±1%) —
Valkey zset scan sorted skiplist keep on (+14%), but layout is 37% —
SPY random record read 83 B record straddles a line, latency keep on (+11% across QPI) L1 next-line prefetcher (hero)
SPY random binary search scattered, bandwidth-saturated turn off (+22% across QPI) L2 stream prefetcher (footgun)
synthetic strided scan sparse, bandwidth-saturated turn off (+62%) L2 stream prefetcher (footgun)

The verdict depends on three things at once: your access pattern, whether you're bottlenecked on latency or bandwidth, and — the one people forget — whether the memory is local or a socket away. The L2 stream prefetcher is a reliable bandwidth-waster whenever you're saturated and not truly streaming. The L1 next-line prefetcher is a quiet hero whenever your records cross cache lines and latency is what hurts. Different units, different verdicts, and the right setting for one process is the wrong setting for the process beside it.

Which is why it has to be per-process

That last sentence is the argument for building this as a /proc knob instead of leaving it in the BIOS. On a shared machine the binary searcher wants the L2 stream prefetcher off while it pounds remote memory, and the record reader running right next to it wants the L1 next-line prefetcher on. A global, reboot-to-change BIOS switch cannot satisfy both. A per-process switch, set from userspace and applied at context switch, can — and only on the cores actually touching the far-away memory.

Try it yourself

The kernel patch and the little userspace helper live here, built against Linux v7.3-rc1: distribution.goblinreactor.com/linux-prefetch-control/ — grab prefetch-control.tar.zst (the patch, prefetchctl.c, and a README).

# unpack
zstd -d prefetch-control.tar.zst && tar xf prefetch-control.tar

# apply to a mainline tree and turn the feature on
git clone https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git
cd linux && git checkout v7.3-rc1
git am ../prefetch-control/0001-x86-prefetch-*.patch
./scripts/config --enable X86_PREFETCH_CTL          # default y, under Processor type and features
make -j"$(nproc)" && sudo make modules_install install && sudo reboot

# then flip the switches — 1 = disabled, 0 = enabled — per process:
echo 1 > /proc/self/prefetch_disable/l2_stream      # off for this shell only
gcc -O2 -o prefetchctl prefetch-control/prefetchctl.c
./prefetchctl exec l2_stream -- ./mybench           # run a benchmark with it off
./prefetchctl disable l2_stream "$(pgrep -n victim)"  # or aim it at another process

A prefetcher is disabled on a core if either hyperthread asks for it; the mask follows the process across context switches; and it survives exec but not fork — so wrap the process you're actually measuring, not its launcher.

The honest caveats: this is one machine, a four-socket Ivy Bridge Xeon, and prefetcher behavior differs across microarchitectures. Every run is pinned to fixed cores with the clock frequency locked, memory placed deliberately on a chosen NUMA node, and every mask confirmed live by reading the MSR — because, as the interpreter pass reminded me, on effects this size the setup will invent a result if you let it.

But the thing I set out to check holds up. The prefetcher default is a guess. For my interpreter it's a guess that costs nothing either way. For my sorted set it's a small help riding on top of a much larger choice I didn't know I was making. For reading records it's quietly saving me, more so the farther the data is. And for searching them it's quietly taxing me, up to 22%. The only way to know which of those is happening to your code is to flip the switch and measure — so I built the switch.