#debugging

The bug that wasn't in the mesh

“Write your root cause as a sentence you'd be willing to have your own logs contradict. Mine did, and the cure was 30 seconds of debug logging — not another rebuild.”

The City Walk: Path Loss ~4 in Dense Urban — and the Cache That Wouldn't Die

“The chip was telling the truth the whole time. The firmware just wouldn't repeat it.”

Hunting a connection leak the soak test wouldn't explain

“When a leak resists reasoning, stop reasoning and instrument it. A spawned-vs-finished counter found in minutes what hours of reading the reap path missed.”

Four days into the soak, the RAM was still climbing

“The worst CPU offenders had zero active peers. That's the fingerprint of dead-state accumulation.”

Flamegraphing your way out of "this can't possibly be right"

“Comments lie. Flamegraphs don't.”

When 18 nodes pegged my 80-core box at 100%

“Three obvious wins got the host from 100% to 35%. Still wrong, just less wrong.”