Packing for Berlin

Agents write code faster than a team can review it. We don't solve that with smarter agents, but with stricter guardrails.

Our agents now take a GitHub issue all the way to a running deployment. We finished it on a Friday afternoon. By Sunday evening seven pull requests had gone through it. Here is what the agents do on their own, where we still step in, and why the guardrails matter more than the agents.

Jorrit wrote on LinkedIn that Baseflow goes to Berlin once we have fully transformed the way we work with AI. I read that post twice. Then I looked at the pipeline we built for one of our internal tools over the past few days, and I can already see us there.

This is what that pipeline looks like today. Every agent in it is Claude Code, started inside our GitHub Actions workflows by Anthropic's open-source claude-code-action. It is early, it is opinionated, and it already ships code to our cluster without anyone opening an editor.

One issue, one proposal, one approval

Everything starts with a GitHub issue. The moment it is opened, a proposal agent reads it, looks through the codebase and writes back a short plan: what changes, which files, roughly how many lines. A person reads that plan and replies in the thread. Too broad, wrong file, missed a case: the agent takes the comment and comes back with a revised proposal, as often as needed. Only when the plan fits do you give the go-ahead.

That approval is the first place where we deliberately keep a person at the wheel. An agent that starts coding before anyone has agreed on the shape of the change wastes tokens at best, and opens a pull request nobody asked for at worst.

Once approved, an implementation agent takes the issue, writes the change on a branch and opens a pull request. When it gets stuck it says so on the PR and asks. A developer answers in the thread, and the agent continues from there.

Review by agents, with a hard cap

Every pull request first runs the unit tests and a Docker build. Only when both pass does the review agent start. It runs the code review skill from the open-source BMAD Method, which reviews in three passes: a blind hunter that looks for bugs without reading the issue, an edge-case hunter that walks every branch and boundary, and an acceptance auditor that checks the change against what was asked. Findings are sorted into three piles: patch now, defer to a follow-up, or reject.

When something needs patching, a fix agent commits the fix and the review runs again. This is where our favourite guardrail lives. We allow two automated fix rounds. If the third review still finds problems, the pipeline stops and asks for a human reviewer, because two agents that disagree can keep each other busy forever and nobody should pay for that.

When the review is clean and the change stays within our budget of roughly 300 lines, the agent approves, squash-merges, closes the linked issue and starts the build. From there it is boring in the best possible way: CI publishes a versioned image, FluxCD sees the new tag in the registry and rolls it out to the cluster within half an hour. Nobody touches a deployment by hand.

One weekend in, in numbers

Most of this landed on Friday afternoon, and then the weekend started. The pipeline did not notice. It merged its first pull request on Friday, two more on Saturday, and on Sunday the review agent filed seven follow-up issues and the cluster got four more deployments, with us approving proposals from the couch.

• 7 pull requests reviewed, merged and deployed by the pipeline
• 1 pull request held back for a person and never merged
• 7 issues filed by the review agent, four of them already shipped
• <2 h from a fresh issue to running on the cluster

All numbers come from the repository's GitHub history and FluxCD's deploy commits. The four shipped issues are Sunday's four deployments.

The one that never got merged is our favourite. The first attempt at a data-freshness indicator came back as 620 added lines across 18 files, and its review fix changed the auth middleware for every HTMX request in the app. The review agent fixed its own two findings, then still declined: too big, too large a blast radius, a developer should look at this. We closed it, tightened the scope rules for the agents and opened a fresh issue. The second attempt was 58 lines, needed one fix round, and was merged and running on the cluster within two hours.

The proposal loop works the way you would want it to. On the concurrent-sync issue the first plan silently skipped a sync that was already running. One comment asked for a disabled button that says "Syncing" instead. One revised proposal, one "looks good, implement", and the change was live the same afternoon.

Guardrails we are glad we added

• Two fix rounds, then a developer. The cap is counted from the commits on the PR, so no agent can talk its way past it. So far two PRs needed one round each, and none reached the cap.
• Small changes only. Around 300 changed lines per PR. A bigger idea comes back as a proposal to split it, and a bigger PR is never auto-merged.
• Single-shot runs. Each agent run has one job and a fixed turn budget. No background work, no waiting, no half-finished verdicts.
• Merge only on green. Tests, build, review and every external check. If the pipeline cannot read a check, it declines to merge rather than guess.
• Human signals win. A work-in-progress title, a requested reviewer or a change request switches auto-merge off for that PR.
• The laziest solution that works. Every agent runs ponytail, Dietrich Gebert's open-source skill, which pushes towards the standard library before custom code and one line before fifty. We added our own scope rules on top: no drive-by refactoring, and discovered bugs become follow-up issues instead of scope creep.

Deferred findings become tomorrow's proposals

💡 Whatever the review agent defers is filed as a new issue, labelled as suggested by an agent. Each one triggers a fresh proposal. The backlog grooms itself, and a person still says yes to every item before any code is written.

This closes the loop for us. Review findings that were too big for the current PR used to disappear into a comment thread. Now they show up as a plan the next morning, with a size estimate and a reason.

Early days, honestly

The pipeline is a few days old and we have already rewritten parts of it. GitHub's own "closes #12" automation silently skipped three merges, so we now close issues explicitly. The review agent once ended its run while its own helper agents were still working, so the three review passes now run one after the other. Each of those fixes was itself a small PR through the same pipeline, which is a nice way to test your own guardrails.

There is plenty left to refine: the quality of the proposals, what a review is allowed to cost, and how we bring the same flow to client repositories with different rules. We will keep sharing what we learn along the way.

Start with your own rules

Take the last pull request your team held back and write down why. Too big, too risky, a check that failed: every reason you write down is a rule an agent can follow. A reason that only lives in someone's head is one an agent cannot apply.

Already running agents in your pipeline? I'd love to hear how you approach it.