The CSV becomes dangerous when it forgets its history
One file came from Apollo. Another came from Prospeo. A researcher added website evidence. Hunter verified some addresses. A spreadsheet formula removed duplicates. Then somebody downloaded final-final-v3.csv and uploaded it to a sequencer.
The file may work. It cannot answer basic operational questions: Which provider supplied this title? Was the email verified before or after the role changed? Did two similar names merge? Was this company excluded in the current ICP version? Has this person already been contacted for another client?
A lead-list source of truth is the system that can answer those questions. It is the foundation underneath the Apollo, Prospeo, and Hunter waterfall and any serious Codex or Claude Code Clay replacement.
Store observations before selected values
Do not overwrite one job_title cell whenever a new provider replies. Store each observation as a separate record:
lead_id | field | value | source | source_record_id observed_at | retrieved_at | confidence | raw_artifact_hash
A selection process then chooses the current value and records the rule. For example, a current employer value from a recently fetched official team page may outrank an older provider record. A verified work email may outrank a guessed pattern. A provider-specific confidence score should not be compared directly with another provider's scale unless you have defined the translation.
Resolve companies before people
Company identity is usually more stable than person identity. Normalize the root domain, preserve the entered website, and record aliases, subsidiaries, and regional entities separately. Do not collapse every related domain into one company without a review rule.
Then resolve people using strong keys. A normalized work email can be strong when current and verified. Provider IDs are useful inside their provider. LinkedIn URLs can help but can change or be missing. Name plus company domain is a useful composite, not an infallible identity.
Use append-only events for important state changes
A mutable status column is convenient but incomplete. Add an event ledger for acceptance, rejection, enrichment, verification, review, suppression, export, upload, and campaign readback.
event_id | lead_id | event_type | event_time | actor rule_version | artifact_id | previous_state | new_state | reason
This lets you reconstruct why a lead entered a campaign even after the current record changes. It also prevents a later cleanup from erasing evidence that the lead was already delivered.
Separate four kinds of truth
- Source truth: what the provider, website, or file actually returned.
- Identity truth: which records represent the same person or company.
- Decision truth: which value and qualification state the current rules select.
- Delivery truth: which exact snapshot was exported, uploaded, accepted, rejected, or already present downstream.
This separation matters when a sequencer reports fewer accepted rows than the local export. The correct next step is provider readback using stable keys, not assuming the upload succeeded because the API returned a success message.
Give Codex bounded jobs
Codex or Claude Code can ingest new provider files, validate schemas, normalize domains, compute candidate matches, attach provenance, generate conflict reports, and create delivery snapshots. Keep irreversible merges and high-risk suppressions behind review rules.
Workers should process independent slices or unique companies. One parent process should own the schema, global dedupe index, event ledger, and final merge. This avoids two workers simultaneously creating different canonical records for the same person.
The minimum viable schema
companies: canonical company ID, normalized domain, name, status.people: canonical person ID, normalized name, current company ID.source_records: raw provider or file record plus artifact hash.field_observations: every sourced value and timestamp.selected_fields: current winner, selection rule, and decision time.lead_events: append-only state transitions.suppressions: scope, reason, source, start, and expiry.delivery_snapshots: exact exported rows, campaign, and readback result.
A spreadsheet can implement a small version if tabs and keys are protected. SQLite or a managed database becomes easier once concurrency, history, and repeated campaigns grow. The principle is more important than the product.
Operational proof from live workflows
Ink Persuasion's recent client operations use the same controls repeatedly: dated output folders, deterministic audits, source URLs, deduplication before writeback, provider readback after enrollment, and separate local evidence for accepted and rejected records. In one monitored workflow, the system verified that accepted rows were present in the target campaign and list and that no expected rows were missing.
This is an anonymized operating pattern, not a claim that a specific database design guarantees campaign performance. Its value is narrower: provider readback and append-only evidence prevent "the script ran" from being confused with "the intended rows are present."
Run six launch checks
- Every selected field has a source and retrieval time.
- Every person and company has a stable canonical key.
- Every merge can be explained or reviewed.
- Every accepted lead points to the ICP and rule version that accepted it.
- Every email has a current verification state and every suppression has been applied.
- Every delivery has an immutable snapshot and downstream readback.
Source-of-truth rule: never make the final CSV more authoritative than the evidence and decisions that produced it.
FAQ
What is a single source of truth?
A single source of truth is the governed system that preserves source observations, resolves identities, records selected values and rules, and exposes the current approved state without erasing history.
What is an example of a single source of truth for lead data?
One example is a database with canonical person and company IDs, immutable provider records, field-level provenance, conflict decisions, suppressions, and exact campaign delivery snapshots.
What is the difference between a system of record and a source of truth?
A system of record is the authoritative home for a defined data domain. A source of truth is the trusted state used for a decision. In a lead workflow, several systems of record can feed one governed source of truth.
Can Google Sheets be the source of truth?
For a small, single-writer workflow, yes, if it uses stable IDs, protected schemas, append-only history, and clear snapshots. It becomes fragile with concurrent writers and complex identity resolution.
Should old provider values be deleted?
No. Retain them as observations with timestamps. Change the selected value and record why the new source won.
What is the best dedupe key?
There is no universal single key. Use a hierarchy of strong identifiers and route ambiguous matches to review. Names alone are not enough.
