On this page#
What changes when an agent does this#
The bottleneck moves from writing changelog prose to proving that every claim matches the repository. An agent can inspect a large change set and draft quickly, but speed is irrelevant if it invents impact, misses a breaking change, or converts an implementation detail into a user-facing promise.
The OpenAI Codex CLI documentation describes an agent that can inspect a repository, edit files, and run commands in a local development workflow. That capability makes changelog generation practical, but it does not make the output authoritative. The repository remains the evidence; the agent is only the analyst and drafter.
A trustworthy workflow separates four jobs:
- Define the comparison boundary.
- Extract candidate changes from repository evidence.
- Classify changes by user impact.
- Write and verify the final entry.
Do not ask for all four in one prompt. A single “write the changelog” instruction encourages the agent to smooth over uncertainty. Polished prose then hides weak evidence.
The key invariant is simple: every published statement must trace back to an inspected diff, commit, test, documentation change, or explicitly supplied release decision. If no evidence supports a claim, remove it or mark it for human resolution.
This is also why a changelog is not a commit summary. Commits describe how work happened. A changelog describes what a user must know now.
Photo by Christina Morillo on Pexels.
The prompt chain#
Use a staged prompt chain that forces evidence collection before writing. Each stage should produce an inspectable artifact, and later stages should consume that artifact rather than silently repeating repository analysis.
Stage 1: Establish the boundary#
Start by defining exactly what belongs in the candidate release. This prevents the agent from reading unrelated history, treating local work as released work, or comparing the wrong endpoints.
Inspect the repository without editing files.
Compare <BASE> with <TARGET>. Identify:
- changed user-facing behavior
- fixes with observable user impact
- breaking changes
- configuration, API, or CLI changes
- dependency changes only when they alter user behavior
- documentation changes that reveal a behavior change
Exclude refactors, formatting, test-only changes, and internal cleanup unless they
change an external contract.
Return an evidence table with:
- candidate change
- affected users
- evidence location
- confidence
- unresolved question
Do not draft changelog prose.<BASE> and <TARGET> must be concrete repository references chosen by a human. Never let the agent guess the release boundary. The Codex CLI documentation explains the tool’s repository-oriented workflow, but your release policy still has to define what counts as included.
Stage 2: Challenge the candidates#
The second prompt should attack the evidence table. Its job is to remove claims that merely sound plausible and surface changes requiring release-owner judgment.
Review the evidence table against the inspected comparison.
For each candidate:
- confirm that the evidence supports the claimed user impact
- separate observed behavior from inferred behavior
- identify compatibility risks
- flag missing migration instructions
- reject claims supported only by commit-message wording
- identify duplicate or overlapping entries
Do not edit files. Return a revised table and a rejection list with reasons.This prompt explicitly distinguishes observation from inference. That distinction matters because source changes can imply an outcome without proving the full user-visible behavior.
For versioning decisions, use Semantic Versioning as the shared vocabulary: a public API is the contract exposed to consumers, and incompatible changes to that contract require different treatment from backward-compatible additions or fixes. Do not let the agent assign release significance until a human has confirmed what the project considers public API.
Stage 3: Draft from approved evidence#
Only approved candidates should reach the drafting stage. This reduces hallucination by preventing the agent from reopening the whole repository and discovering convenient new “facts” while writing.
Draft a changelog entry using only the approved evidence table.
Rules:
- lead with user-visible outcomes
- use one bullet per distinct change
- state breaking changes directly
- include migration action only when supported by approved evidence
- omit implementation details unless users must act on them
- do not add dates, version numbers, performance claims, or compatibility claims
- preserve unresolved items as explicit TODOs
After the draft, provide a claim-to-evidence checklist.This is where the prose becomes concise, but concision must not erase conditions. “Fixed authentication” is too broad if the evidence supports only one failure path.
Teams that want a reusable review sequence can adapt PairFoundry’s Review and Repair workflow. The important part is not the prompt wording; it is the enforced separation between evidence, judgment, drafting, and approval.
Stage 4: Perform a hostile final review#
The final prompt should assume the draft is unsafe until proven otherwise. A friendly copy edit is inadequate because the main risk is unsupported meaning, not awkward language.
Audit the draft as a skeptical release maintainer.
For every sentence:
- identify its evidence
- mark any broader wording than the evidence permits
- flag hidden breaking changes
- flag missing scope or conditions
- flag promises about performance, compatibility, or future behavior
- flag internal implementation details presented as user benefits
Return:
1. blocking findings
2. non-blocking wording improvements
3. a minimally corrected draft
Do not resolve TODOs by guessing.This review can improve the draft, but it cannot approve it. Approval belongs to someone accountable for the release.
The gate#
The mandatory gate is a human claim-by-claim review before the changelog is merged or published. The reviewer must compare the draft with the evidence checklist, confirm the release boundary, and decide every compatibility or migration question the agent flagged.
Stop at the gate if any of these remain:
- A claim has no concrete evidence location.
- The affected-user scope is inferred rather than demonstrated.
- A breaking change lacks explicit migration guidance.
- The public API boundary is disputed.
- A TODO survives in publishable text.
- The stated versioning impact has not been approved.
This is where Semantic Versioning helps, but it does not make the decision for you. The specification defines relationships between public API changes and version increments; the project still owns the definition of its public API.
The reviewer should also read the rendered changelog, not only the patch. Formatting can merge bullets, hide warnings, or make a migration note appear optional.
If the team is still building this review discipline, use the free PairFoundry foundations track before automating publication. Automation without a clear approval boundary merely makes incorrect releases easier to produce.
Photo by panumas nikhomkhai on Pexels.
What it gets wrong#
The agent most often fails by converting incomplete repository evidence into complete-sounding product language. You can detect this by asking whether each noun, scope word, causal claim, and compatibility statement is directly supported.
| Failure mode | Warning sign | Required response | |---|---|---| | Commit-message laundering | The draft repeats “fix,” “support,” or “improve” without inspecting behavior | Verify against the diff and tests | | Scope inflation | “All,” “always,” or an unqualified feature name replaces a narrow case | Restore the actual condition | | Breaking-change burial | Compatibility impact appears as a minor note | Promote it and require migration guidance | | Refactor promotion | Internal restructuring becomes a claimed user benefit | Remove it unless behavior changed | | Causal invention | The draft explains why a bug occurred without evidence | State only the verified outcome | | Versioning overreach | The agent selects significance from file names or diff size | Require human API-boundary review | | Duplicate narration | One change appears under several implementation components | Merge around the user-visible outcome |
Another common error is omission. Agents favor obvious code changes and may underweight configuration defaults, removed flags, altered error behavior, or documentation that exposes a previously unstated contract.
The official Codex CLI documentation tells you what the tool can do; it does not supply your repository’s release policy, public API definition, or acceptable evidence standard. Treating official tool capability as workflow governance is a category error.
For related operational patterns, see PairFoundry’s workflow articles or the full pack overview. Neither replaces the project-specific gate.
Rollback#
Rollback should remove only the agent-produced changelog work while preserving unrelated repository changes. The cleanest method is to start from a dedicated branch or isolated worktree, capture the initial state, and keep the changelog edit in its own reviewable commit.
Before the agent writes:
- Confirm the intended branch and comparison boundary.
- Record whether the working tree already contains unrelated changes.
- Isolate the task from those changes.
- Require the agent to edit only the agreed changelog files.
- Review the resulting diff before committing.
If the draft is wrong before commit, discard only the identified changelog edits. If it has its own unshared commit, reverse that commit without rewriting unrelated history. If it has already been shared, use the team’s normal history-preserving reversal process and publish a corrected entry when readers may have seen the false claim.
Do not ask the agent to “clean everything up.” That instruction has an unsafe scope. Name the exact files or commit being reversed, inspect the rollback diff, and rerun the human gate.
Photo by Christina Morillo on Pexels.
Related reading#
- Codex CLI docs — where the agent helps and where it quietly hurts
- Amazon mandating AI code review, with the guardrails that keep it honest
FAQ#
Should the agent choose the release version too?#
No. It can identify evidence relevant to versioning, but a human must confirm the public API boundary and release significance. Use Semantic Versioning as the rule set, not as permission for the agent to guess project policy.
How is this different from asking Codex CLI to summarize recent commits?#
A commit summary follows development history; this workflow builds a user-facing release claim from inspected evidence. Commit messages are inputs to investigate, not facts to publish.
What breaks when several engineers use this workflow?#
Boundaries drift, evidence tables conflict, and concurrent drafts duplicate changes. Assign one release boundary, one approved evidence set, and one accountable reviewer. Merge candidate evidence before merging prose.
What if the generated changelog is already committed?#
Inspect whether the commit contains only changelog work. Reverse that isolated commit through the team’s normal history-preserving process. If it mixes unrelated work, separate the exact changelog changes instead of reverting the entire commit blindly.
When should we not use an agent for this?#
Do not use it when the repository cannot reveal the relevant behavior, the release boundary is unknown, or nobody can approve compatibility claims. In those cases, the agent can format supplied facts, but it should not infer the release narrative.