I’ve been digging into a corner of GCC, the compiler behind much of the Linux world.

This is a story about a negative result that turned out to be wrong—and how close I came to publishing it.

The Part of the Compiler I’m Touching

When a program loops over an array, the CPU has to keep track of where it is.

GCC has a pass called ivopts—induction variable optimization—whose job is to decide how that bookkeeping should work: how many counters to maintain and what each one should count.

Get it right, and the loop stays tight.

Get it wrong, and you pay on every iteration.

The compiler cannot generate every possible version and benchmark each one. Compilation itself has to remain reasonably fast. Instead, GCC scores the alternatives using a cost formula and chooses the winner.

That formula is what I’m studying.

In the source, the main calculation lives in a function called iv_ca_cost. Close decisions are settled by roughly twenty lines in cheaper_cost_pair.

My Theory

Compilers are full of formulas like this: educated guesses written years ago and tuned by hand.

My theory was simple:

What if the search is good, but the scoring formula is wrong?

What if GCC reliably finds the best answer according to a formula that does not actually predict execution speed?

So I built a harness that does what the compiler cannot afford to do: enumerate every candidate and measure all of them.

I tested roughly 55,000 loops drawn from GCC’s own test suite and the LLVM benchmark suite.

The First Answer, Which Was Wrong

The search turned out to be excellent.

It finds the best-scoring candidate 99.9% of the time.

I built a more sophisticated search anyway. It found better-scoring answers for five loops and produced no measurable performance improvement.

I also checked whether the existing formula at least minimized code size.

It already did.

Every result pointed in the same direction: there was nothing to win here.

I was ready to publish the work as a negative result and move on.

Why That Was a False Lead

All of those measurements came from an Intel desktop machine.

That machine was lying.

A busy consumer desktop is a terrible instrument for precision benchmarking. Clock speeds change constantly. Each physical core shares resources with a hyperthreaded sibling. A browser was open. Other activity came and went.

Depending on the benchmark, my measurement noise ranged from 1% to 19%.

Anything smaller was effectively invisible.

Worse, noise does not merely hide real effects. It manufactures fake ones.

Three separate times, I found a result, became suspicious, increased the repetition count, and watched it disappear.

A 10% speedup became 0.02%.

A 1% regression became a 0.3% improvement.

Each result had looked completely convincing at first.

What Better Instruments Showed

I moved the experiment to an ARM server in the cloud: no hyperthreading, no clock games, and no competing workload.

The noise dropped to 0.007%—roughly a thousandfold improvement.

And there it was.

On one matrix multiplication loop, the candidate GCC’s formula ranks sixth runs 17% faster than the candidate it ranks first.

That might have been an ARM-specific quirk, so I repeated the experiment on a quiet, idle 48-core Xeon.

It belonged to the same broad architecture family as the desktop that had shown me nothing, but turbo was disabled and the machine had no competing load.

Its noise floor was 0.05%.

The result appeared again.

On that machine, the candidate GCC ranks fourth runs 4.8% faster than its first choice.

That is 96 times the measured noise floor. This was not a marginal call.

The original desktop had not revealed a different answer.

It had simply been a bad ruler.

The Mechanism, Which Is the Satisfying Part

On the Xeon, three alternative candidates all execute roughly 12% fewer instructions than GCC’s chosen version.

One is 4.8% faster.

The other two are slightly slower.

They retire about the same number of instructions, yet their execution times differ by roughly 5%.

The difference is how many of those instructions the processor can execute in parallel.

For the winning candidate, instructions per cycle rise from about 2.6 to 3.1.

GCC’s formula works largely by counting instructions.

It cannot distinguish these candidates, even in principle.

The formula was not merely mistuned.

It was measuring the wrong thing.

What I Took Away

Two things.

The obvious lesson is that a benchmark machine is a scientific instrument.

An uncalibrated instrument produces confident nonsense.

I nearly published a negative result caused by hyperthreading, changing clock speeds, background activity, and a browser tab.

The less obvious lesson is that I was wrong repeatedly in a single day—and each time, it was because I stopped measuring too early.

The solution was never to become cleverer.

It was to rerun the experiments I was already sure about.

The work continues.

Now there is something real to chase.