Context Makers, Consolidators, and Gradekeepers

“Context Maker,” “Consolidator,” and “Gradekeeper” are useful design roles, not industry-standard protocol terms. Their value is separation of concerns.

A model that gathers evidence, decides what it means, and grades its own answer has weak checks on omission and self-inflation. Split those responsibilities. Keep the interfaces small enough to inspect.

Context Makers produce evidence packets

A Context Maker retrieves or computes one narrow kind of evidence: repository symbols, test failures, policy rules, image metadata, or source excerpts. It should return structured facts rather than a narrative.

{
  "producer": "test-context",
  "facts": [
    {
      "key": "test.failure",
      "value": "expected 401, received 500",
      "evidence_refs": ["run:184:test/auth:42"]
    }
  ],
  "unknowns": ["production identity provider behavior"],
  "confidence": 1,
  "status": "complete"
}

The reference is as important as the fact. It lets the next stage reread the original evidence instead of trusting a paraphrase.

Context Makers should be cheap by default:

• Use parsers, schemas, hashes, and test runners before models.
• Run independent makers in parallel when they do not share mutable state.
• Report “unknown” and “not found” explicitly.
• Load only the tools and sources required for the task.
• Never convert missing evidence into a guessed fact.

MCP standardizes how applications expose tools and contextual resources. Cloudflare's Code Mode demonstrates a related efficiency pattern: compose, filter, and reshape intermediate tool results inside a sandbox, then return only what the model needs. Neither mechanism guarantees evidence quality; the application still owns validation and authorization.

The Consolidator builds a ledger

The Consolidator combines packets. Its core should be deterministic so the same inputs produce the same ledger.

Its work is mechanical:

1. Validate every packet against a versioned schema.
2. Deduplicate identical facts and retain all supporting references.
3. Record conflicting values without choosing a winner.
4. Compare supplied evidence with task requirements.
5. Emit facts, conflicts, missing evidence, unknowns, and source lineage.
6. Route only unresolved items to deeper reasoning.

A conflict is not an error to hide. If one helper reports “furnace” and another reports “heat pump,” the ledger should preserve both claims and their evidence. A model or human can then inspect that narrow disagreement.

The Consolidator can calculate deterministic status:

blocked    if a required source or policy gate failed
needs_more if evidence is missing, unknown, or contradictory
complete   otherwise

It should not decide whether risky work is acceptable. That belongs to policy gates and authorized humans.

The Gradekeeper measures outcomes

The Gradekeeper is independent of the agent being graded. The agent may report confidence or say “I need work,” but that self-assessment is diagnostic input, not the official score.

Use dimensions tied to observable outcomes:

• accuracy against expected results
• safety and policy compliance
• evidence completeness
• cost and latency
• recovery after tool or test failure
• human override rate
• regression performance by task class

OpenAI's agent-evaluation guidance recommends traces while debugging workflow behavior, then repeatable datasets and eval runs for comparisons over time. NIST's AI RMF likewise calls for documented, repeatable evaluation, benchmark comparisons, uncertainty, production monitoring, and independent review.

A grade card should therefore show sample count, baseline, current score, delta, task coverage, and weak dimensions. Minimum sample rules matter: five easy tasks should not produce an A. Safety gates must never disappear because an agent's average grade improved.

The loop

intent
  -> parallel Context Makers
  -> deterministic Consolidator
  -> targeted reasoning or action
  -> tests and observed outcome
  -> independent Gradekeeper
  -> improvement queue

Promotion is a comparison, not a feeling. Freeze the evaluation set and scoring policy, run baseline and candidate, inspect failures, and promote only when required gates pass. Keep the losing result too; it explains what was tried and prevents the same weak change from returning later.

This pattern reduces wasted context because deep reasoning sees a compact evidence ledger. More importantly, it makes disagreement, absence, and quality visible as data.

Sources / further reading

• Model Context Protocol architecture
• Cloudflare Code Mode
• OpenAI: Evaluate agent workflows
• OpenAI evaluation best practices
• NIST AI RMF Core