The familiar week
Walk through the lifecycle of a typical upstream change from the consumer side. A provider announces that a field is renamed, a unit is changing, or an old version is being retired. The announcement lands in an inbox, a changelog feed, or a Deprecation header nobody is logging.
Eventually someone notices. They search the codebase for the affected calls, update the client code, adjust the mocks so the tests compile, and open a pull request. Review checks that the diff looks reasonable. CI checks that the tests pass. The change merges.
Notice what nobody checked: whether the system behaves the same way it did before, for the requests it actually receives. Review checks intent. Tests check the scenarios someone wrote down, usually against fixtures that describe the old API. Neither one is evidence that production behavior is unchanged.
Three jobs, only two of them have tools
Handling an upstream API change is really three separate jobs:
- Detection. Knowing that a change happened and what it is. Specification diffing, changelog feeds and SDK release notes cover a lot of this. See OpenAPI diff.
- Remediation. Changing the consuming code. Codemods, updated SDKs and now AI coding assistants make the patch itself faster to write than it has ever been.
- Verification. Proving the patched system still does what it did before, with the same inputs, in every repository that was touched.
The first two jobs have mature or rapidly improving tooling. The third is mostly done by hope, a staging deploy and a careful engineer reading logs after release. That gap is the reason migrations feel risky even when the code change is small.
Why faster patches make the gap wider
It is tempting to assume that better code generation shrinks the problem. It does the opposite. When a patch takes minutes instead of hours to write, the bottleneck moves to the question of whether the patch is right. A team can now produce migration pull requests faster than it can confidently review them.
Generated patches are also very good at satisfying the visible checks. They update the types, fix the compile errors and adjust the fixtures. If the fixtures encode the old behavior, the generated code will happily agree with them. Passing tests become a weaker signal, not a stronger one, as the cost of producing a passing diff falls.
What verification should actually mean
We think API change verification should meet a specific bar:
- It uses real inputs. Recorded production traffic, sanitised, rather than fixtures someone invented. Real traffic contains the edge cases nobody thought to test.
- It compares whole responses. Status code, headers, body and timing, compared between the old code and the new code. A field that silently changed meaning shows up as a diff.
- It runs on every affected repository. A breaking change rarely touches one service. Verification that covers one repository and assumes the rest is not verification.
- It gates the merge. A divergence is a blocking result, not a dashboard someone may look at later.
The underlying technique is not new. Traffic replay testing has been used for years by teams migrating databases and rewriting services. What has been missing is applying it specifically and routinely to API and SDK changes, as part of the pull request rather than as a special project.
Why it deserves a category, not a script
Plenty of teams have built a replay harness once, for one migration, and then let it rot. That pattern is the strongest argument for treating verification as a product category rather than a one-off script. The hard parts are the same every time: capturing traffic safely, scrubbing secrets and personal data, handling non-deterministic fields like timestamps and generated IDs, isolating side effects, and deciding which differences are expected.
Those problems are general. Solving them once, well, is more valuable than every consuming team solving them badly under deadline. The same thing happened to specification diffing and to contract testing: both started as internal scripts and became standard practice once tooling made them cheap.
Verification also changes how the other two jobs feel. When there is a reliable check that behavior did not move, detection becomes less anxious and remediation can be more aggressive. Teams can take upstream changes promptly instead of letting them pile up until a forced API migration arrives with a sunset date attached.
What we are building
Releo is our attempt to make that loop complete: detect the breaking change, write the migration pull request in every affected repository, and replay recorded traffic against the old and new code so that any divergence blocks the merge. You can read how it works. We are in private beta, and the waitlist is open at early access.
Frequently asked questions
What is API change verification?
API change verification is the practice of proving that a system behaves the same after it adopts an upstream API or SDK change. It typically replays real requests against the old and new code and compares full responses, rather than relying only on unit tests and fixtures.
How is verification different from contract testing?
Contract testing checks that a provider still honours the interactions consumers declared. Verification checks the consumer side after a change: that the migrated code produces the same observable behavior on real inputs. They are complementary.
Related guides
- What is an API breaking change?
- Traffic replay testing, explained
- How to migrate to a new API version safely