datacontract-cli

The decision

Your AI feature runs on data another team owns, so when they change it, who notices first, you or your customer? This one is about writing that promise down, testing it, and deciding who owns it.

The problem it removes

Most of the production AI failures I've debugged came down to a data change nobody announced, like a new category, a field that became optional without anyone saying so, or a location stored as one city when the posting listed three. The model keeps running and gets a bit worse every day, and the bill lands on whoever is on call and whoever trusted the output. Fixing it takes one conversation between two teams, but that conversation rarely happens because nobody owns it.

What I ran

My job-search pipeline holds 1,106 postings from four sources, and last week it saved a role listed in three cities as one US city, which I only caught by reading the page.

So I wrote a contract for that table with 43 checks, plus one business rule saying roles I add by hand have to be workable from the Netherlands. Against the backup from before my fix it flagged exactly that row, and against today's data it flagged something I'd missed, because my fix had added a new value to the source field that nothing downstream knew about. So yes, I'm the other team too.

I also got two lessons for free: my first version of the rule let the bug through, because the check for the CET timezone also matched "Princeton", and a rule tied to a field I'd misread failed on every row. The whole suite runs in about a second.

Where it sits in the stack

Between the teams that produce data and everything that uses it, from pipelines and feature stores to retrieval indexes and prompts. It reads the data where it already lives (Postgres, Snowflake, Databricks, BigQuery, S3, plain files) and leaves it there, and the contract itself is a YAML file in the open ODCS standard, kept next to the code, so you can switch tools later and keep the contract.

The first 30 days

Week 1. Pick the dataset your most visible AI feature depends on and generate a starting contract from it, which takes seconds, then get the producing and consuming teams in a room for an hour and write down what each field promises. That hour is the actual work.

Weeks 2 to 4. Run the tests in CI on every change the producer makes and on a schedule against production, and send failures to the producing team with the AI lead copied in.

Skills. SQL and YAML, with no new platform and no new hires. Running cost is one CI job.

Working at day 30 means it has caught at least one silent change before release, and both teams agree who fixes what.

Risk

Coverage. There's no SQLite connector, so I had to export the table to Parquet first, which means you should check your sources against the list before you commit.

Severity. Its breaking-change check rated "this field can now be empty" as informational and exits cleanly even when it finds errors, so if you want a CI gate on contract changes, add a small wrapper and have a person review the diff.

Maintainers. There are 158 contributors, but three people do most of the work and also sell a hosted product the CLI can publish to. The CLI is MIT licensed and works fully on its own, and the format is an open standard under the Linux Foundation, so lock-in is low.

The real risk is people. A contract nobody owns turns into a stale YAML file.

Build, adopt or buy

Build. Most teams already have a few SQL checks in a pipeline somewhere, and what they're missing is the agreement itself, one versioned file both teams can read with a diff that shows what changed. Rebuilding that yourself is months of glue code for no gain.

Buy. Data observability platforms spot anomalies after the fact and add lineage and alerting, which is worth it at scale, and they sit well alongside a contract, since the contract says what should be true and the platform notices when things drift.

Adopt the CLI for the promise itself, and revisit a platform once you have more than a handful of datasets under contract.

Verdict

I'd adopt it. It's about a week of work with no new hires, it runs where your data already lives, and the format will outlive the tool, but most of the value comes from two teams finally writing down what they promise each other. Give every contract a named owner on the producing side, have the AI lead sign off on changes, and review its breaking-change ratings yourself, because it will happily call a breaking change informational.