How to Replace Clay With Codex, Part 3: Write Evidence-First Worker Contracts

Part 2 built the orchestrator that can move work without losing its place. That system is still unsafe if a worker can turn missing evidence into plausible text. The next layer is a machine-checkable contract for what goes into each worker, what may come back, and what happens when the answer is unknown.

A worker prompt is not a contract

"Research this company and write a useful personalization" describes an aspiration. It does not specify which sources are permitted, which evidence must be returned, how conflicts are represented, or whether an unsupported answer should be blank.

A contract gives both sides something testable:

• The parent knows exactly what it is asking for.
• The worker knows the source and output boundaries.
• The validator can reject malformed or unsupported results.
• The orchestrator can route known failure types without reading prose.
• A cheaper or newer model can be substituted without weakening acceptance rules.

The prompt may explain the task. The contract defines whether the result is usable.

Version the request envelope

Every request should identify the job and the exact contract version. Include only the inputs the worker needs. Do not pass an entire spreadsheet, an unbounded browser session, or access to unrelated destinations.

{
  "schema_version": "research-request-v3",
  "job_id": "run-001:lead-924:company-signal-v3",
  "subject": {
    "subject_id": "lead-924",
    "company_domain": "example.com"
  },
  "task": "find_latest_allowed_company_signal",
  "allowed_sources": ["company_website", "approved_social_profile"],
  "required_evidence": 1,
  "max_sources": 4,
  "retrieval_deadline": "2026-07-16T12:05:00Z",
  "output_schema": "research-result-v3"
}

A schema version is part of the job identity. If you add a field, change a status meaning, or alter the source policy, create a new version and migrate deliberately. Do not let old workers and new validators silently disagree.

Make the response strict enough to reject

JSON is only a container. A valid JSON object can still omit evidence, invent a status, or add an unexpected field. Use a JSON Schema with required properties, enums, length constraints, and no unevaluated properties.

{
  "$schema": "https://json-schema.org/draft/2020-12/schema",
  "type": "object",
  "required": [
    "schema_version", "job_id", "subject_id", "status",
    "selected_value", "evidence", "confidence", "error"
  ],
  "properties": {
    "schema_version": {"const": "research-result-v3"},
    "job_id": {"type": "string", "minLength": 1},
    "subject_id": {"type": "string", "minLength": 1},
    "status": {
      "enum": [
        "VERIFIED", "UNVERIFIED", "CONFLICT", "NOT_APPLICABLE",
        "RETRYABLE_ERROR", "PERMANENT_ERROR", "HELD_FOR_REVIEW"
      ]
    },
    "selected_value": {"type": ["string", "null"]},
    "evidence": {"type": "array", "items": {"$ref": "#/$defs/evidence"}},
    "confidence": {"$ref": "#/$defs/confidence"},
    "error": {"type": ["object", "null"]}
  },
  "unevaluatedProperties": false
}

JSON Schema Draft 2020-12 provides the vocabulary for describing and validating this structure. OpenAI's current API reference also supports JSON Schema response formats with strict adherence on compatible models, while noting that strict mode supports a subset of JSON Schema. Keep provider formatting support separate from your own full application validation.

Evidence is an array, not a footnote

A source URL alone is weak provenance. The page may change, redirect, or contain many unrelated claims. Store enough material to reconstruct why the worker selected the value.

{
  "evidence_id": "ev-01",
  "source_url": "https://example.com/news/launch",
  "source_type": "company_website",
  "retrieved_at": "2026-07-16T12:03:18Z",
  "content_hash": "sha256:...",
  "excerpt": "The company launched its enterprise product in June 2026.",
  "supports": ["selected_value"],
  "derivation": "direct_statement"
}

The W3C PROV model treats provenance as information about the entities, activities, and people involved in producing data. That framing is useful here: preserve the source object, retrieval activity, transformation step, worker version, and accepted result. Provenance is what makes a later audit possible.

Confidence needs rules, not vibes

Do not ask the model to invent a confidence percentage. A number such as 0.93 looks precise but says nothing about why the result should be trusted. Define confidence from observable conditions.

Return both a level and a basis array, such as `direct_primary_source`, `identity_match`, and `published_within_90_days`. The parent should recompute or verify the level instead of accepting the label on faith.

Use typed outcomes for every failure path

A blank string can mean no result, blocked access, invalid input, a timeout, or a policy hold. Those conditions need different next actions.

• UNVERIFIED. The worker completed the allowed search but found no qualifying evidence. Do not retry automatically.
• CONFLICT. Qualifying sources disagree. Preserve all evidence and hold the row.
• NOT_APPLICABLE. The requested fact does not apply to this subject.
• RETRYABLE_ERROR. A timeout, explicit rate limit, or temporary dependency failure prevented completion.
• PERMANENT_ERROR. The input or requested operation violates a stable contract condition.
• HELD_FOR_REVIEW. The result may be valid but cannot advance without an approval or policy decision.

RFC 9457 defines a machine-readable problem-details format for HTTP APIs with fields such as `type`, `title`, `status`, `detail`, and `instance`. You can use the same idea inside worker errors: one stable code for routing, one human detail for diagnosis, and one job or attempt identity for traceability.

Fail closed when evidence is unavailable

The hardest contract rule is also the simplest: if the required evidence cannot be verified from an allowed source, do not replace it with a convenient source or a plausible answer. Return `UNVERIFIED` and a null value.

We saw the effect of this rule in one internal 20-row operating QA batch. The first strict audit produced 9 PASS and 11 REVISE. The source contract then required one approved public source class only. If that source blocked access or did not expose the signal, the row had to remain explicitly unverified. After targeted research and another full check, 18 populated rows passed and 2 rows remained blank holds.

Preserve conflicts instead of averaging them away

Evidence-first systems must also keep disagreement visible. In a separate internal reconciliation, daily opportunity events summed to 124 while the campaign-level aggregate reported 120. Neither count should be silently rewritten to match the other.

The correct contract result is `CONFLICT`, with both values, their source identities, the aggregation method, retrieval time, and a note that the difference remains unresolved. A downstream graphic or report can then use one declared source of truth while disclosing the reconciliation difference.

Separate the tool contract from the model contract

A worker often calls a browser, scraper, search API, or database before the model sees any text. Give each tool its own contract.

• The fetcher returns final URL, status, content type, retrieval time, hash, and raw artifact reference.
• The parser returns extracted text, parser version, discarded sections, and parse warnings.
• The model receives only the bounded evidence and returns the result schema.
• The validator checks schema, source policy, evidence support, and business rules.
• The parent records acceptance or a typed rejection. It does not ask the model to grade itself.

This separation prevents a successful model call from masking a failed fetch or a broken parser.

Route cheaper models under the same acceptance contract

Not every row needs the strongest model. Deterministic extraction, formatting, or straightforward classification may be eligible for a cheaper worker. The cost decision belongs in a declared routing policy, not inside the worker.

{
  "task": "classify_allowed_signal",
  "eligible_profiles": ["economy-v2", "standard-v3"],
  "default_profile": "economy-v2",
  "escalate_when": ["CONFLICT", "LOW_CONFIDENCE", "SCHEMA_REPAIR_EXHAUSTED"],
  "acceptance_contract": "research-result-v3",
  "parent_revalidates_evidence": true
}

Run every eligible profile through the same fixtures and acceptance checks. If the cheaper model cannot meet the evidence and schema contract for that task class, it is not cheaper after rework.

Test the contract with negative fixtures

A contract test suite should contain failures on purpose:

• A valid direct source and a supported value.
• No evidence from an allowed source.
• A disallowed source that contains the desired answer.
• Two allowed sources with conflicting values.
• A source URL that resolves to a different subject.
• An excerpt that does not support the selected value.
• Malformed JSON, an extra field, a missing required field, and an unknown enum.
• A retryable timeout and a permanent invalid-input error.
• A cheaper model result that sounds correct but lacks evidence.

The suite passes only when each fixture reaches the expected typed state. A prose answer that looks reasonable is not a passing result.

Definition of done for Part 3

1. Every request and response names an explicit schema version.
2. Allowed source classes and minimum evidence are part of the request.
3. The result schema has required fields, bounded enums, and no silent extras.
4. Every accepted claim points to preserved evidence and provenance.
5. Confidence follows observable rules the parent can verify.
6. Unknown, conflicting, retryable, permanent, and approval-held outcomes remain distinct.
7. Unsupported values remain null instead of becoming plausible text.
8. Every model profile passes the same positive and negative contract tests.

What Part 4 will build

The workers can now return results that are structured, sourced, and safe to reject. Continue to Part 4, the production control layer, for approval gates, observability, cost limits, immutable delivery snapshots, CRM and sequencer handoff, destination readback, rollback, and reconciliation.

Frequently asked questions

What does Clay software do?

Clay coordinates data providers, web research, AI transformations, table logic, and destination integrations. Replacing it means recreating the orchestration and control behavior you use, not merely sending a prompt to a model.

Is there a free version of Clay?

Clay's current plan details can change, so check its official pricing page for the latest offer. A self-built Codex workflow is not automatically free either. You still pay for models, enrichment providers, hosting, monitoring, and engineering time.

What is an AI worker contract?

It is a versioned agreement that defines the worker's allowed inputs, sources, output schema, evidence requirements, confidence semantics, error types, and acceptance tests.

Should an AI agent return confidence scores?

Use bounded levels derived from observable evidence conditions. Avoid unsupported percentages. The parent should verify the basis instead of trusting a self-reported number.

What should happen when no allowed source verifies the answer?

Return an explicit `UNVERIFIED` status, keep the selected value null, preserve the attempted-source evidence, and route the row according to policy. Do not substitute a disallowed source.

Can I use cheaper models for outbound research?

Yes, for task classes where they pass the same evidence, schema, and negative-fixture tests. The parent contract must remain unchanged, and high-risk or conflicting cases should escalate.

Does structured output prevent hallucinations?

No. Structured output can enforce shape. It does not prove that a claim is true or supported. Evidence validation and source-policy checks are still required.