Provenance per row.
Every claim records its source, batch, ingest time, confidence, and reconciliation history. Auditable end to end.
The provenance row
Every claim attached to an entity carries five governance fields:
These five fields travel with the claim through Iceberg, into NebulaGraph, and out to partner exports. Customer-facing exports include them on request.
Reconciliation
When two sources disagree (say, IRS BMF vs Delivr Companies on a company's primary address), both claims are kept. Reconciliation produces a third row marked as "consensus" or "preferred", with a pointer to the underlying claims and the rule that selected it. The losing claim is not deleted — it stays auditable.
| Claim type | How resolved |
|---|---|
| Company name | State SOS > IRS BMF > SAM.gov > commercial |
| Company address | SAM.gov > State SOS > commercial > bidstream |
| Domain to company | WHOIS + Common Crawl + commercial; weighted |
| Industry (NAICS) | SAM.gov > IRS > commercial |
Batch IDs and rebuilds
Each continual refresh produces a single batch ID (build-YYYY-wWW). The batch ID is stamped on every row added or changed in that build. Manifests for prior builds are retained; an export can be reproduced from any preserved manifest.
- Manifests retained 13 months by default
- Iceberg snapshots retained 90 days for compute reads, longer cold-tier copies in Glacier
- Deletions create a new snapshot; prior snapshots are sealed against the deletion request
Auditability
Customers under MNDA can request a row-level provenance dump for any export. The dump lists every claim that contributed, the source loader, the batch ID, and the reconciliation outcome.
Roles and stewards
- Data steward: owns source onboarding, classification, and label assignment
- Privacy ops: owns deletion, opt-out, and consumer-rights operations
- Engineering: owns pipeline, schema, and build manifests
- Customer success: owns export-level provenance disclosures