A coding agent changes when its model, prompt, tools, policies, dependencies, or target repository changes. Any of those can improve one task and break another. Continuous evaluation is the discipline of detecting that trade before promotion.
“Continuous” does not mean one giant benchmark running without pause. It means evaluation is wired into the lifecycle: fast checks on each change, broader suites on a schedule, and production monitoring after release.
Define the unit under test
Evaluate the workflow, not only the final patch. A run record should include:
- task and fixture version
- model, prompt, policy, and tool versions
- retrieved context references
- tool calls, approvals, retries, and errors
- patch or answer
- deterministic test results
- latency, tokens, and cost
- human corrections or override
OpenAI's agent-evaluation guidance recommends traces for debugging end-to-end behavior, including tool choice, handoffs, instructions, and guardrails. Once expected behavior is clear, repeatable datasets and eval runs support comparison over time.
Build a representative fixture set
Start from real task classes, then keep the suite balanced:
- common edits
- cross-file changes
- ambiguous requests
- missing or conflicting context
- tool failures and timeouts
- security-sensitive changes
- tasks that should stop or request approval
- previously fixed regressions
Each fixture needs explicit success criteria. Prefer executable checks: compilation, unit and integration tests, schema validation, static analysis, and exact policy assertions. Use human or model graders for qualities that cannot be reduced to code, such as whether a rationale is useful. Calibrate those graders against reviewed examples.
Keep a held-out set. If every failure becomes a prompt tweak against the same public fixtures, the agent can overfit the benchmark without becoming more useful.
Compare baseline and candidate
Run both versions against the same pinned fixtures and environment. Store paired results.
A score can summarize performance, but preserve dimensions:
correctness
safety and policy compliance
evidence quality
task completion
recovery
human override
latency
costDo not let cheap success offset unsafe behavior. Cost and speed are optimization dimensions; safety and required correctness are gates.
Report sample count and uncertainty with every aggregate. Slice results by task class, language, repository size, and risk. An overall improvement can hide a severe regression in a small but important category.
Use promotion gates
A candidate is promotable only when all required conditions hold. A practical policy is:
- Build, typecheck, and deterministic tests pass.
- Security and policy checks have no new blocking finding.
- No protected task class regresses beyond its tolerance.
- The primary quality score exceeds the pinned baseline by the required margin.
- Cost and latency remain within budget.
- High-risk changes receive independent human approval.
- The candidate and rollback identifiers are recorded.
A letter grade can make the dashboard readable, but it should be derived from the underlying evidence. New agents need a minimum number of representative samples before receiving a high official grade. The agent's self-confidence may help select review depth; it must not assign the official grade.
NIST's AI RMF calls for documented, repeatable test, evaluation, verification, and validation; comparisons to benchmarks; uncertainty; monitoring in production; and independent review. These are useful properties even when the system is not regulated.
Test recovery, not just the happy path
Agentic systems fail in sequences. Add controlled faults:
- a tool returns malformed data
- the test runner exits nonzero
- a dependency is unavailable
- context changes mid-run
- a write requires approval
- the token or time budget is exhausted
The expected behavior may be a bounded retry, a targeted reread, a rollback, or a clean stop. “Eventually produced code” is not a passing result if the path violated policy or hid an error.
Monitor after promotion
Offline fixtures cannot cover the whole deployment distribution. In production, monitor task mix, failure rate, override rate, rollback rate, tool errors, latency, cost, and policy violations. Sample traces for review under the system's privacy rules. Promote observed failures into regression fixtures after removing sensitive data.
Do not train directly on unreviewed production outcomes. A user accepting a patch may mean it was good, merely expedient, or never inspected.
The evaluation loop is simple:
observe failure
-> create a reviewed fixture
-> run baseline and candidate
-> inspect paired evidence
-> pass hard gates
-> promote gradually
-> monitor and retain rollbackThe grade is the dashboard. The fixtures, traces, tests, and gates are the control system.