Exactly once is a habit, not a feature
Every streaming platform gets asked for the same guarantee: process each event exactly once. And every honest platform gives the same answer, quietly, in the fine print. You cannot buy exactly once. You can only build it, and the building is a habit practiced on every write path you own.
Why the pure promise is impossible
Picture the smallest possible consumer loop: read a batch, process it, then record how far you got. Now crash it at every point and watch what happens. Crash after processing but before recording progress, and on restart you will process those events again. Record progress before processing, crash in the middle, and the events you skipped are gone forever. There is no third place to put the progress marker that removes both windows, because the marker and the work live in different systems and cannot commit as one atom.
So the real menu has two items. At most once: record first, lose things on a bad day. At least once: record last, repeat things on a bad day. For anything that matters, you choose at least once, because a duplicate is a problem while a hole is a lie.
Making repeats harmless
Having chosen repetition, the craft is making repetition invisible. The word is idempotence: an operation you can apply twice with the same result as once. Setting a value is idempotent. Adding to a counter is not. Inserting a row is not. Inserting a row keyed by a natural event identifier, where a second attempt becomes an update of identical values, is.
That is the entire trick, and it reorganizes how you design sinks. Give every event a stable key. Write with merge semantics on that key, not blind appends. Let the storage layer absorb the duplicate instead of asking the pipeline to never produce one. When people say a system is effectively exactly once, this is what they mean: at least once delivery wearing idempotent writes, so the repeats happen and nobody can tell.
The checkpoint is a bookmark, not a diary
The progress marker deserves respect too. Keep it tiny: a position, not a copy of the work. Commit it after the work is durable, never before. Store it somewhere that survives the worker, because the whole point is that workers are disposable. If your marker is a bookmark in an ordered stream, restarting is just reopening the book a page early and rereading a paragraph you already know.
The same habit, new costume
Here is why this essay is not only about pipelines. An AI agent working through a multi-step task has exactly the shape of that consumer loop: do a step, then record that it happened. Crash between the two and the step replays. If your steps are idempotent, replay is a shrug. If they are not, your agent double-sends the email, double-writes the file, double-charges the card. The reliability discipline that streaming engineers drilled for a decade transfers to agents without changing a single principle, and the teams that know the habit are shipping the agents that survive restarts.
The takeaway
Stop asking systems to promise exactly once and start building the two halves yourself: retry freely, and make retries harmless. It is less a feature than a posture. Once you hold it, crashes stop being incidents and become weather.