Undocumented changes

Why we look at the diff, and what kinds of silent changes we surface.

An undocumented change is a modification we detect in the git diff that the release notes don't mention. These are the changes that quietly break production weeks after a routine dependency update.

Why they happen #

Maintainers are human. Common patterns:

  • Squashed PR where the description didn't make it into CHANGELOG.md.
  • Fix labeled as "internal" that turned out to be public-API-adjacent.
  • Refactor where one signature change snuck in alongside a bunch of cleanup.
  • Release-notes auto-generated from commits, but the relevant commit message was generic ("misc fixes").

What we look for #

The diff analysis is prompted to identify, among other things:

  • Signature changes — parameters added, removed, or retyped.
  • Removed or renamed exports — symbols disappearing from a module's public surface.
  • Default-value changes — a function that used to default timeout=30 now defaults timeout=10.
  • Behavioral changes — same signature, different observable effect (e.g. error path now returns null instead of throwing).
  • Schema changes — for libraries with serialized state, JSON shape changes.

Each finding cites the file and line range so you can verify it against the diff yourself.

How we surface them #

  • Per-release: in the summary, undocumented changes appear as a separate section with severity tags.
  • In aggregate: undocumented changes are weighted heavily in the risk score, so a release with several of them will land in the High band even if churn is modest.
  • In alerts: lockfile alerts call out undocumented changes specifically — the kind of thing you want in a notification, not buried in a dashboard.

Limitations #

  • We only see what's in the diff. Behavior changes that happen entirely at runtime (e.g. via dependency updates inside the library) aren't visible to us.
  • We can't run the code. If a function looks unchanged but its dependency changed semantics, we won't catch it from the diff alone.
  • Generated code can produce noisy diffs. We try to skip obvious generated-file patterns, but expect occasional noise on libraries that ship generated bindings.