®Model-backed features have an unusual failure mode: they look finished long before they are reliable. A prompt that works on the six examples someone tried in a playground will meet inputs it has never seen within an hour of launch, and nobody on the team will be able to say whether the last change made things better or worse.
Write the test set first
Before the interface, collect fifty to a few hundred real inputs — support tickets, search queries, documents, whatever the feature will actually receive — and agree what a good output looks like for each. This is tedious and it is the single highest-leverage hour in the project.
- Include the awkward cases: empty input, hostile input, the longest document anyone will paste
- Score what you care about — factual accuracy, format compliance, refusal behaviour, tone
- Keep a held-out slice you do not tune against, so you can tell improvement from overfitting
- Version it alongside the code; a prompt change is a code change
Then every decision becomes evidence-based
With a suite in place, questions that were previously arguments become measurements: whether the cheaper model is good enough for this step, whether the longer prompt actually helps, whether retrieval improved after the chunking change. Teams without one relitigate the same opinions every sprint.
Guardrails are product design, not paranoia
- Decide what happens when the model is slow, unavailable or wrong — and design that state properly, because users will see it
- Constrain outputs where the downstream system needs a shape; validate before you trust
- Scope tool permissions tightly, and log every action an agent takes on a user’s behalf
- Put a human checkpoint where the cost of being wrong is asymmetric
Show sources when the answer matters. A visible citation is both a quality signal and an honest admission that the model is summarising something, not knowing it.
Budget latency and cost like any other resource
Token cost and response time are product constraints, not billing details. Set a target for both, instrument them per feature, and cache the calls that repeat. A feature that is excellent and takes eleven seconds gets used once; the same feature streaming its first tokens in under a second gets used daily.
What good looks like six months in
The team can change the prompt on a Friday, because the suite says whether it helped. The model provider can be swapped without a rewrite, because the integration sits behind one interface. Quality is a number on a dashboard rather than a feeling. That is an AI product; everything before it is a demo with good lighting.