Automating Release Notes with goose in CI

At Entire, a developer replaced hours of manual release note drafting with a CI pipeline powered by goose, an AI agent. The workflow, documented in a DEV Community post, tackles a problem far messier than the typical single-repo release: aggregating changes across multiple projects with different release cadences, feature flags, and privacy constraints.

The Initial Approach and Its Failures

The first workflow, "Release Notes to Slack," mirrored a setup the developer built at Block. That earlier system worked cleanly: after a release, GitHub Actions checked out the new tag, compared it to the previous one, and fed the diff to goose. The agent organized commits into features, bug fixes, and improvements, then posted to Discord.

At Entire, that approach collapsed. A "Dispatch" (Entire's release notes) spans multiple projects: the Entire CLI, entire.io, EntireDB, external agent integrations, and open-source libraries like go-git, go-nuts, git-sync, and ForgeMark. Each ships on different schedules. The CLI has stable and nightly channels. Feature flags gate public visibility.

The core problem wasn't collecting changes—GitHub APIs handle that. It was determining what was safe to announce. A private repo might contain code for a feature already public, while a merged public PR could still be hidden behind a flag. The pipeline needed to answer: Has this shipped? Is it live for everyone? Is it stable or nightly? Does it matter to readers? Did a community contributor help? Is anything sensitive?

Learning from Bad Assumptions

The commit history of the first workflow reads like a record of assumptions failing. Initially, the workflow ran across every repository and aggregated Slack output. Then it stopped relying on release tags and used the previous Dispatch publication date as the reporting window. Next, a recipe turned raw notes into a draft in Marvin's (the CEO's) voice.

Deployment edge cases emerged. Merged code didn't mean public code. The workflow added filtering to include only public work, labeling nightly changes. For entire.io, which uses continuous delivery and feature flags, it added PostHog checks to see if a flag was active before treating a merged PR as public.

Contributor attribution came next, followed by experiments with output formats: Slack messages, threads, Markdown tables, PDFs, and eventually plain Markdown. The developer realized "writing release notes" was actually several smaller problems stacked together.

Different repositories needed different goose recipes. Entire.io needed rules for continuous delivery and feature flags. Tagged projects needed rules for stable releases, nightlies, and unreleased merges. Go-git needed community attribution rules. The workflow worked but became tangled—data collection, release checks, contributor matching, writing style, and formatting were all mixed.

V2: Separating Collection from Writing

The second iteration, "Release Summary to Slack V2," fixed the architecture by separating data collection from writing. Instead of asking goose to wander through repositories, the workflow gathers facts first.

It starts by finding the date of the last published Dispatch. A matrix job runs across every repository to collect releases, merged PRs, titles, descriptions, labels, authors, links, feature flag definitions, current PostHog status, commit authors, and organization membership data.

The workflow then compiles two files. The Source Brief contains the releases, PRs, links, labels, feature flags, and repository metadata goose is allowed to use as facts. The Style Reference contains excerpts from the three most recent Dispatches as examples of tone, structure, and length.

The recipe makes the boundary explicit: previous Dispatches are for style only, so goose can't pull old changes into the new draft. The recipe also defines a JSON response schema. Goose must return a payload with the Markdown draft, and the workflow validates it. If goose returns conversational preamble, malformed JSON, or an empty response, the build fails.

The Recipe and the Skill

The system relies on two components: a goose recipe and a Dispatch skill. The recipe controls the run—which files to read, which sources to trust, which tools are available, and the output format. The skill contains reusable editorial rules: which projects belong in each section, how to filter private or flagged work, how to separate nightly features from stable releases, how to explain value instead of repeating commit messages, and how to capture Marvin's tone without recycling intros.

The skill was created using a recursive technique. The developer built a "session-to-skill" skill that searches Entire-tracked sessions and checkpoints for repeated workflows, then turns useful parts into instructions. It doesn't copy transcripts; it pulls out decisions, corrections, commands, checks, and preferences worth repeating.

The developer applied session-to-skill to sessions where they wrote previous Dispatches. Those sessions captured the editing history: asking for every bullet to explain the benefit, rejecting corny intros, correcting project coverage, checking contributor credits, and removing non-public work. That became the source material for the Dispatch skill.

The Division of Labor

Privacy Still Needs a Human

The workflow can collect evidence but shouldn't make final privacy decisions. For entire.io, it checks feature flag states in PostHog. If a flag is disabled, internal-only, or mid-rollout, the recipe tells goose to leave that work out. For other projects, if the workflow can't confirm something is public, it omits it and flags for manual review.

The developer doesn't auto-publish. An awkward sentence is easy to fix; announcing an unreleased feature or crediting the wrong contributor damages trust. The generated Markdown goes to Slack for review first.

The Current Workflow

  1. GitHub Actions finds the last published Dispatch date.
  2. A matrix job collects activity, feature flags, and contributor metadata across repositories.
  3. The workflow compiles a single Source Brief.
  4. It adds the latest Dispatches as Style References and loads the Dispatch skill.
  5. Goose runs with the recipe and generates the Markdown draft.
  6. The workflow validates the JSON response and uploads the draft to Slack.
  7. A human reviews, edits, and publishes.

What Actually Got Automated

The developer didn't automate editorial judgment. They automated research, data collection, recurring checks, and the first draft. Before, writing a Dispatch meant hours opening tabs, reading PRs, and reconstructing the week. Now goose brings source material together, applies editorial rules, and gives a draft to polish.

This approach—separating data collection from generation, using structured inputs, and encoding editorial rules in a skill—is a pattern worth stealing for any team that produces regular technical communications or reports. The key takeaway: AI agents in CI are most effective when given clean, structured facts and explicit rules, not just a prompt and a repository.