A data contract is an explicit, versioned agreement between the team that produces data (usually a service owner) and the teams that consume it (analytics.
Data contracts formalize the interface between data producers and data consumers, treating schemas, semantics, freshness, and quality as a first-class contract owned by the producing team. The shift is cultural before it is technical: an event schema stops being 'a thing analytics figured out from what the app emitted' and becomes 'a thing the app team explicitly published and promised to maintain.' Implementation runs from lightweight (a YAML file in the repo and a CI check) to heavyweight (a contract registry with runtime enforcement in the event bus). Most teams should start lightweight and only add machinery when they've felt the pain that machinery solves.
A minimal contract specifies: (1) Schema — field names, types, nullability, allowed values, primary key. Use a schema language your ecosystem supports natively (Protobuf, Avro, JSON Schema, or dbt YAML for warehouse tables). (2) Semantics — what each field actually means, in prose. 'user_id' is not self-explanatory; is it the auth ID, the profile ID, is it stable across account merges? (3) SLAs — freshness (this table is refreshed hourly by 15 past), completeness (95% of events land within 5 minutes), correctness (referential integrity to which upstream tables). (4) Ownership — a named team and an on-call, not a person. (5) Deprecation policy — how much notice before a breaking change; typically 30-90 days.
Where the contract lives: in the producing service's repo, next to the code that emits the data. This makes the producing team the owner-of-record and forces schema changes through their code review. Enforcement: (a) at build time, a CI check validates the contract file is well-formed and semantically valid. (b) at test time, unit tests verify the service emits data conforming to the contract. (c) at deploy time, a compatibility check compares the new contract to the deployed contract and fails on breaking changes (unless explicitly marked as a versioned major bump). Tools: Buf for Protobuf, Confluent Schema Registry for Avro/Kafka, dbt contracts for warehouse models, or a homegrown YAML + Python validator for smaller setups.
Consumers subscribe to specific contract versions, not to raw tables or streams. When a producer publishes a new version, consumers get a signal (PR opened against their repo, Slack alert, dashboard) with the diff and the deprecation deadline. Consumer-side testing: a dbt or SQL check that runs against production data hourly and alerts when the observed data violates the contract (nulls where non-null promised, values outside enum). This catches producer regressions that slipped past their own tests. For ML pipelines, wire contract violations to feature-freshness monitors — an ML model consuming stale or malformed features is a silent failure mode.
Phase 1 (weeks 1-4): pick the 3-5 highest-pain data assets (the ones that break dashboards most often), write contracts for them collaboratively with the producing team, add consumer-side checks only. This proves value without demanding producer changes. Phase 2 (months 2-4): move enforcement upstream to the producer's CI, add contract review to their PR process, kill the shadow schemas consumers were maintaining. Phase 3 (months 5-12): expand to the long tail, retire the pre-contract 'raw' pipelines, publish the contract registry as an internal product. Do not announce a company-wide 'all data must have contracts by end of quarter' policy — it produces low-quality contracts everyone ignores.
Data contracts require producing teams to accept ongoing accountability for data quality they may have historically considered analytics's problem. This works when: (a) leadership backs it explicitly and includes contract SLAs in team scorecards, (b) analytics/ML teams reciprocate by giving producers a clear escalation path and not adding new consumer dependencies without notice, (c) breaking-change violations have a real consequence — postmortem, follow-up work, not just a Slack apology. Without those, contracts become documentation that ages in a wiki.
Investor directory · Fundraising library · Articles A–Z · Company funding database