← essays shrey patel →

Prompts are contracts

Shrey Patel and Jay Patel, Coconut Labs · Published 2026-08-05 · Last updated 2026-08-08 · 6 min · Applied AI

The difference between an AI demo and an AI system is almost never the model. It is the discipline around the prompt. Demos treat prompts as incantations: magic words, tweaked until the output looks right once. Systems treat prompts as contracts: versioned interfaces with schemas, failure modes, and tests. The mindset shift costs nothing and changes everything downstream.

What a contract mindset means

A contract says: given input of this shape, I return output of this shape, and here is what happens when I cannot. Applied to a prompt, that means the output format is specified, not suggested. It means the edge cases are enumerated: what should come back for empty input, absurd input, input that tries to hijack the instructions? And it means the prompt lives in version control with a changelog, because a prompt that silently changed is an API that silently changed, and everyone downstream deserves to know.

input incl. hostile text the contract versioned prompt + schema edge cases enumerated valid output parses, always parse fails → repair loop, logged failure zoo on watch: drift · bleed · injection · refusal · invented fields · verbosity creep thirty golden cases run on every change, like unit tests.
a prompt with a schema, a version, and a test suite. an interface, not an incantation.

The failure taxonomy

Prompts fail in ways regular code does not, and naming the ways is half the defense. Format drift: the model returns almost your schema, with a field renamed or a stray sentence prepended. Instruction bleed: your formatting rules leak into the content itself. Injection: the input contains text that reads like instructions, and the model obeys the wrong master. Refusal: a benign request pattern-matches to something the model declines. Hallucinated fields: the schema is honored and filled with confident inventions. Verbosity creep: answers that grow ever longer because nothing pushed back. Keep a private zoo of real examples of each; nothing sharpens a review like recognizing a species on sight.

Three ways to enforce structure

Ranked by strength. Schema-constrained decoding, where the provider guarantees the output parses against your schema, is the strongest and should be your default when available. Tool-call shaped output, where you define the response as a function signature, gets you most of the way with wide support. Parse-and-repair, where you validate the response and loop back with the error on failure, is the honest fallback and a good safety net under either of the first two. Log every repair. A rising repair rate is a smoke alarm for drift somewhere upstream.

The test suite prompts deserve

Thirty cases is enough to start: twenty realistic, five hard, five adversarial. Run them on every prompt change the way you run unit tests on every commit. The first time a one-word edit that improved fluency breaks two edge cases, and your golden set catches it before production does, the practice pays for itself permanently.

The takeaway

You already know how to do all of this. It is schema design, interface versioning, and regression testing, applied to a component that happens to speak English. Treat the prompt like the interface it is, and the most unreliable part of your stack becomes just another dependency with a contract and a test suite. Treat it like magic words, and you will be debugging vibes at midnight.