Catching JSON Drift in CI with Schema Snapshots

API responses change. Fields get added, types shift from string to number, nullable properties appear without warning. If your code depends on a specific shape, these silent changes cause bugs that only surface in production.

Snapshot testing catches visual regressions in UI code. The same idea works for JSON structure. Generate a schema from your API response, commit it, and fail the build when the shape changes.

The problem with value-based snapshots

Jest and similar tools can snapshot entire JSON responses, but value-based snapshots are noisy. A timestamp changes, a counter increments, and your snapshot fails for reasons that have nothing to do with structure. Teams end up updating snapshots reflexively, which defeats the purpose.

Schema snapshots strip out values entirely. They capture only field names, types, nesting, and optionality. A new user signing up does not break the snapshot. A field changing from string to number does.

Generating a schema snapshot

Using the @maisondigital/jsontoschema CLI, you can generate a schema from any JSON file:

npx @maisondigital/jsontoschema response.json > snapshots/users-endpoint.schema

The output looks like this:

{
  id: number,
  email: string,
  name: string,
  role: string,
  team: string | null,
  created_at: string,
  preferences: {
    theme: string,
    notifications: boolean
  }
}

Commit this file alongside your tests.

Adding a CI check

A simple shell script can compare the current schema against the committed snapshot:

#!/bin/bash
# check-schema.sh

ENDPOINT="https://api.example.com/users/1"
SNAPSHOT="snapshots/users-endpoint.schema"

curl -s "$ENDPOINT" | npx @maisondigital/jsontoschema > /tmp/current.schema

if ! diff -q "$SNAPSHOT" /tmp/current.schema > /dev/null 2>&1; then
  echo "Schema drift detected:"
  diff "$SNAPSHOT" /tmp/current.schema
  exit 1
fi

echo "Schema unchanged."

Wire this into your CI pipeline as a step. When the diff is non-empty, the build fails and the output shows exactly which fields changed.

What this catches

Schema snapshots detect structural changes that matter:

A field was removed or renamed
A type changed (e.g., number to string)
A previously required field became nullable
A new nested object appeared inside an existing field
An array item gained or lost properties

They ignore changes that do not matter: different values, different array lengths, updated timestamps.

When to update the snapshot

When the diff is intentional, update the snapshot and commit it. The diff in your pull request serves as documentation. Reviewers can see exactly how the API shape changed and whether the consuming code was updated to match.

npx @maisondigital/jsontoschema response.json > snapshots/users-endpoint.schema
git add snapshots/users-endpoint.schema
git commit -m "update schema snapshot: users endpoint added team field"

This creates an audit trail of structural changes over time.

Scaling to multiple endpoints

For APIs with many endpoints, generate one schema file per route. Name them by endpoint path:

snapshots/
  users-list.schema
  users-detail.schema
  projects-list.schema
  projects-tasks.schema

A loop in CI can check them all in seconds. Schemas are small files. The overhead is negligible.

Tradeoffs

This approach works best for APIs you consume but do not control. For your own APIs, contract testing tools like Pact offer more features. Schema snapshots fill the gap for third-party APIs, internal services without formal contracts, and quick validation during development.

They are not a replacement for integration tests. They tell you the shape changed. Your tests tell you whether your code handles the new shape.

Convert a sample response into a schema snapshot with the converter.