How to catch breaking API changes before merge (not after a customer pages you)

Most breaking changes don't look like breaking changes in the diff. Nobody opens a PR titled "break the free-tier rate limit." They open a PR titled "refactor limiter config," and three weeks later a customer's integration starts getting throttled at 50 requests instead of 100. The change was reviewed. It was approved. It was correct, line by line. It just quietly contradicted a rule nobody had written down.

Catching that beforemerge — not after a customer pages you — is a different problem than schema diffing, and most teams don't have anything pointed at it.

What actually counts as a breaking change

Engineers usually mean one of three things, and tooling coverage drops off sharply as you go down the list:

1. Shape breaks. A field is removed, renamed, or retyped; a 200 becomes a 204. These are the easy ones — OpenAPI diffing and schema linters catch them in CI today.
2. Behavioral breaks. The shape is identical but the behaviorchanged: a limit moved, a default flipped, an endpoint that used to require auth now doesn't. The response still validates against the spec. Schema tooling sees nothing.
3. Rule breaks. The change contradicts a business rule the company actually promised — a plan cap, a refund ceiling, a data-retention window. These live in validation code, middleware, and half-remembered tickets, not in the spec at all.

Category 1 is solved. Categories 2 and 3 are where the expensive incidents come from, and they're exactly the categories a spec diff can't see — because the spec never encoded the rule in the first place.

Why tests and spec diffs miss them

The standard answer is "write a test." The problem is that the test only exists if someone already knew the rule was load-bearing. The rules that break in production are precisely the ones nobody thought to protect — the implicit ones. You can't write a regression test for a requirement you've never made explicit.

• Contract tests (Pact, etc.)protect the shapes both sides agreed on. They don't encode "free tier is capped at 100 req/min" unless someone hand-wrote that expectation.
• OpenAPI diff / Spectral compare specs. If the rule was never in the spec, the diff is clean.
• Unit tests cover the cases the author imagined. The dangerous change is the one that looked too boring to test.

The gap isn't a missing tool. It's a missing list— the set of rules a change should be checked against. That's what an API rule registry is, and it's the thing a pre-merge check needs to read from.

What a pre-merge rule check should actually do

A useful check runs where reviewers already look — as a status on the PR — and answers one question: does this change threaten a rule we've already decided to enforce? When it does, it should show:

1. The rule at risk,in plain English, not a line number. "Free-tier requests are capped at 100/min."
2. The provenance — the ticket that authorized the rule, so the reviewer can tell intent from accident in one glance.
3. The owner, so if the change is intentional, the right person signs off instead of the rule silently dying.

The goal is not to block merges. It's to make the implicit rule visible at the one moment a human is already deciding whether the change is safe. Most of the value is in the sentence "heads up — this touches a rule," surfaced before the merge button, not after the postmortem.

How Stoney checks a PR before merge

Stoney installs as a GitHub App and builds the rule registry from your code, PRs, and Jira tickets — no SDK, no annotations, no test files to write. Once the registry is live, every PR gets a pre-merge check: Stoney maps the diff to the rules it could affect and posts a status when a change threatens one, with the rule, the ticket that authorized it, and the owner to page. If a change contradicts a rule outright, that surfaces as a contradiction for someone to resolve, rather than merging quietly.

Because the check reads from rules with provenance — not from a spec someone has to keep current by hand — it covers the behavioral and rule-level breaks that schema diffing structurally can't. (For the mechanics, see the documentation.)

Getting started

The free tier covers one repository and a rolling history window — enough to install the GitHub App, synthesize rules from a real codebase, and watch a PR get checked against them before deciding whether to enforce. Setup is under five minutes and requires no code:

1. Install the Stoney GitHub App on the repo you want watched.
2. Describe your product in one sentence — this steers rule-synthesis quality more than anything else.
3. Approve the draft rules that describe real requirements, then open a PR and watch the check run.

See pricing or start free. For the production-side counterpart — catching a rule that broke after a change shipped — read API drift detection explained.