The disk has two speeds
Ask someone how fast a disk is and they will give you one number. That number is a lie of averaging. A disk does not have a speed. It has two, separated by roughly three orders of magnitude, and your access pattern decides which machine you actually bought.
The record player
A spinning drive is a record player where the needle can also write. The platter turns constantly. An arm swings the head to the right track, then you wait for the right spot to rotate under it, then you read whatever passes beneath. Getting there costs real, mechanical time: around five to ten milliseconds of arm swing plus a few more of rotation. Reading what is already passing under the head costs almost nothing, because the platter was moving anyway.
Now do the arithmetic that reorganizes your intuition. Ten milliseconds per reposition means about a hundred random touches per second. Touch four kilobytes each time and the disk delivers less than half a megabyte per second. The same disk, streaming one continuous run, delivers well over a hundred megabytes per second. Same hardware, same day, a thousandfold difference, and the only variable is whether you made the arm move.
Two bars, one budget
Picture it as two bars on a chart. A long bar for sequential work. A sliver for random work. Every storage system ever built is a strategy for keeping its workload on the long bar.
Kafka is the purest example: it only ever appends to files and only ever reads forward, so the arm parks and glides. That is why a message broker built on cheap disks embarrassed systems that avoided disks entirely. Log-structured storage engines make the same vow for databases: absorb writes in memory, dump them to disk in big sorted runs, and pay the reorganization cost later, in bulk, sequentially. Even your columnar analytics formats are long-bar strategies: scan a compressed column straight through instead of hopping between rows.
The sliver bar is where careers go to suffer. Any design that touches the disk per record, in an order the disk did not choose, is buying the slow machine at full price: per-message database inserts, per-event index updates, small-file storms on object stores. The symptoms show up as mysterious ceilings. The cause is almost always an arm being asked to dance.
But SSDs, though
Solid state drives removed the arm, and the gap shrank. It did not close. Flash still reads in pages and erases in large blocks, controllers still reward predictable access, operating systems still prefetch aggressively when you scan and cannot help you when you hop. Random access on a good SSD is merely tens of times worse than sequential instead of a thousand times worse. The law survived the hardware that taught it, which is the mark of a real law.
And the law has cousins up the whole ladder. RAM rewards sequential access because of cache lines and prefetchers. Object stores reward big reads because of per-request overhead. Networks reward streaming because of connection setup. Everywhere you look, there is a fast lane for predictable bulk movement and a toll booth for scattered little touches.
The takeaway
Before you tune anything that touches storage, ask the two-speed question: is this workload gliding or hopping? If it hops, no configuration flag will save you; the fix is structural, and it usually looks like batching, buffering, sorting, or logging. The systems that feel magically fast are not doing less work. They are doing the same work in the order the machine wanted all along.
One level down. The deep version of this lives in the private atlas: 003, Kafka’s storage engine. The two-speed law, applied end to end: segments, the page cache, the sparse index, and sendfile. Private atlas, not deployed with this site.