Skip to content
MikeDaDev's Blog

All Code Is Slop: how the Grok Bot team ships 11,000 PRs a month

Tung Bui Thanh (MikeDaDev)13 mins read

Shallow focus photography of potted plants
On this page

You open a pull request an agent wrote. It runs, the tests are green, and something feels off on every screen: a setTimeout(0) to "wait for state to update", a long comment explaining why this shortcut is fine here, and an import that drags a module across a layer it should never cross. You fix it, tell the agent "don't do that again", and a week later you meet the same pattern in another PR.

With a handful of PRs a day, careful review covers it. The Grok Bot team at SpaceX shipped 11,000 PRs to production in a single month. Nobody reads that much code. So how do they keep quality up?

That is the subject of Lauren Tan's talk "All Code Is Slop (Except Mine)" at AI Engineer Code 2026 (mid November 2026, about 17 minutes). This post summarises what she said, checks every number against the transcript, and adds my own take: what a Java/Spring team can use right away, and where to read with a little scepticism.

# who Lauren Tan is

Lauren Tan, "poteto" online, is a principal engineer on the Grok Bot team at SpaceX SI (xAI became part of SpaceX in February 2026). Before that she worked at Netflix, then at Meta on the React Compiler core team, then at Cursor. She also wrote pstack, a set of coding-agent skills in Cursor's official plugin collection.

That background matters. This is not someone announcing that AI replaces programmers. It is a compiler engineer making a very engineering-shaped argument: to trust your agents, invest in their environment, rather than trusting them blindly.

# the thesis: it is about trust, not volume

Lauren opens with a number: last month she personally shipped 3,000 PRs to production. Before agents, volume was never the measure; only a few outliers earned the label "coding machine", like Bret Taylor rewriting Google Maps over a weekend. With agents, anyone can be a coding machine.

So, she argues, the job now is keeping (or even raising) code quality while productivity explodes. People call this a "software factory". She frames it more simply as trust: not blind trust, but investing in the agent's environment until you can trust it to do the right thing. Getting there took her six months, to the point where she ships thousands of PRs, many of them while she sleeps.

# the inverted pyramid: where to spend the effort

The backbone of the talk is an inverted pyramid that ranks where to invest:

The top two layers are deterministic: the codebase itself, and mechanical checks such as lint rules, type systems, compilers and tests. Use them to force agents onto the right path by default. The third layer, verification, finds bugs and checks whether the agent understood your intent. Prose that needs judgment, such as rules, skills and AGENTS.md, comes last.

This order cuts against a common habit: the agent gets something wrong, so you add a line to the rules file. Lauren's suggestion is to ask, every time you correct an agent: can this correction become a deterministic check? A line in AGENTS.md may or may not be read, and may or may not be understood. A lint rule runs every time and gives the same answer.

# the codebase is a garden

The reason code sits at the top is simple: agents generate output from their input, and your codebase is the largest input they have.

In 2024, she recalls, she reviewed about three of her colleagues' PRs in a morning; today it is closer to 300. Every engineer is now a team of hundreds or thousands of agents, and code arrives almost as a continuous stream, like something alive. She calls this "organic architecture": your constraints and guardrails are the concrete foundations, and the code agents write grows around them like plants around a building.

The consequence is unpleasant: every workaround, anti-pattern, deprecated API or bit of tech debt in the repo looks like an established pattern, and the next agent copies it. Weeds in one corner spread across the whole garden. The engineer's job starts to look like gardening: keep the codebase clean so agents always copy the good parts.

# three examples from Dune

To put this into practice the team built Dune, a client framework for Grok Bot. Lauren says it is "not really that interesting technically", roughly Next.js for Electron apps. The interesting part is that Dune encodes years of desktop-app lessons as constraints and lint rules.

useEffect is banned in product code. Anything that needs a lot of nuance to use well tends to produce slop, and React's useEffect is the classic case (some joke you need a PhD to use it). Agents are bad at it. So the team banned it in product code and allows it only inside a few Dune APIs. A whole class of bugs disappeared, and review no longer has to look for it.

Context through directory structure. In Cursor, another Electron app, agents often imported main-process code into the renderer, which caused long tasks and jank. Grok Bot has two top-level directories literally named electron-main and electron-renderer, and imports between them are blocked by a dependency-graph analyser plus a few lint rules. The names alone tell the agent where the code runs.

Almost all comments are banned. This is the most controversial part. At Cursor, agents loved imperative timers such as setTimeout, requestAnimationFrame and requestIdleCallback as quick fixes. The root cause was an architectural clash between VS Code's imperative style and React's declarative one: the architecture itself made both the app and the agents worse. Timers created and destroyed in bulk triggered frequent garbage collection and visible lag. Worse, agents wrote plenty of comments explaining why the quick fix was justified, the next agent read those comments, and the timers spread everywhere.

Her lesson: agents use Markdown (comments, docs, rules) as a place to justify not doing the right thing, or they over-rotate on a small correction and treat it as an unbreakable law. Comments also cannot be enforced. So Grok Bot bans about 99% of comments and relies on lint rules whose error messages point at the root-cause fix.

The general principle: every correction becomes either a deterministic check or a design change that removes the whole class of problem. Do that long enough and your review checklist shrinks until there is nothing left to review, and agents merge most of their own PRs.

# try it in your own codebase

Both Dune constraints take minutes to set up with existing tools. I ran the examples below with ESLint 9 and dependency-cruiser 16. The important part is the error message: the agent reads it as instructions, so it has to say how to do it right, not only that something is wrong.

// eslint.config.mjs: only src/dune may use useEffect
const banUseEffect = {
  'no-restricted-imports': ['error', {
    paths: [{
      name: 'react',
      importNames: ['useEffect', 'useLayoutEffect'],
      message: 'useEffect is banned in product code. Derive state during render, or use a Dune API (useSubscription, useMutation).',
    }],
  }],
  'no-restricted-syntax': ['error', {
    selector: "MemberExpression[object.name='React'][property.name=/^use(Layout)?Effect$/]",
    message: 'React.useEffect is banned in product code. Use a Dune API instead.',
  }],
}
 
export default [
  { files: ['src/**/*.js'], languageOptions: { sourceType: 'module' }, rules: banUseEffect },
  { files: ['src/dune/**/*.js'], rules: { 'no-restricted-imports': 'off', 'no-restricted-syntax': 'off' } },
]
// .dependency-cruiser.cjs: renderer and main may not import each other
module.exports = {
  forbidden: [
    {
      name: 'renderer-must-not-import-main',
      severity: 'error',
      comment: 'Main-process code in the renderer blocks the UI thread. Talk to main over IPC.',
      from: { path: '^src/electron-renderer/' },
      to: { path: '^src/electron-main/' },
    },
    {
      name: 'main-must-not-import-renderer',
      severity: 'error',
      from: { path: '^src/electron-main/' },
      to: { path: '^src/electron-renderer/' },
    },
  ],
  options: { doNotFollow: { path: 'node_modules' } },
}

On a Java backend the idea is identical and the tool is ArchUnit. Say an agent takes a shortcut and lets the order module call billing's internal repository instead of going through BillingApi:

import com.tngtech.archunit.core.domain.JavaClasses;
import com.tngtech.archunit.core.importer.ClassFileImporter;
import com.tngtech.archunit.lang.ArchRule;
import static com.tngtech.archunit.lang.syntax.ArchRuleDefinition.noClasses;
 
public class ModuleBoundaryCheck {
  // Modules talk through their api package; internal packages stay private to their module.
  static final ArchRule ORDER_MUST_NOT_REACH_INTO_BILLING = noClasses()
      .that().resideInAPackage("com.shop.order..")
      .should().dependOnClassesThat().resideInAPackage("com.shop.billing.internal..")
      .because("billing.internal is private to billing: call BillingApi (billing.api) instead");
 
  public static void main(String[] args) {
    JavaClasses classes = new ClassFileImporter().importPackages("com.shop");
    ORDER_MUST_NOT_REACH_INTO_BILLING.check(classes);
  }
}

With ArchUnit 1.3.0 the rule catches all three violations (constructor, field, method call) with line numbers, and the because(...) text appears right in the failure message, which is exactly what an agent needs to fix it on its own. In a real project you would put it in a JUnit test, or use Spring Modulith's ApplicationModules.verify() if your modules already follow that layout.

# verification: read evidence, not code

The third layer is what lets agents act more autonomously and lets people trust the result. The team's code review has shrunk into looking at verification artifacts: videos, screenshots, diagrams, formal verification results.

The basic form is runtime verification: give the agent a way to run the app itself and use the debugging tools people use, such as the Chrome DevTools Protocol. Two ideas are worth copying:

  • Feature map: a sitemap of every feature from the user's point of view. When you say "the thing in the left sidebar", the agent does not have to guess; you share a vocabulary.
  • Custom CLIs and drivers inside skills: the agent reuses the same tool every time, which saves tokens, and the tool itself is testable.

The team also uses formal verification: TLA+ to model-check major flows such as notifications across devices and platforms, and Lean to prove that their custom virtualization library does not flicker or jitter. On top of that, Bombadil, a property-based testing library for UIs, has already caught a few bugs. Lauren is clear that none of this is a silver bullet, just another layer, even though it has found quite a few bugs already.

She quotes a tweet from someone named Cindy: six months ago Cindy could not trust her agents because subtle bugs kept getting merged; today 70% of Cindy's daily tokens go to verification. Lauren says that matches her own experience, most of her tokens go to verification too. The key point: none of these techniques are new, they used to be too expensive to write and maintain. Agents bring that cost down, which is why she calls this a "golden age of testing and verification". To make review easier she also wrote a statechart skill in pstack that draws state machines from the user's point of view, to read next to the videos and screenshots.

# the results, and how to read them

According to Lauren, after this investment:

Metric (self-reported by the team)Value
PRs shipped to production last monthabout 11,000
Stable client releasesabout one a day
People with at least one PR in Octoberabout 227, many of them drive-by contributors
Revert ratesteady at about 0.5%, lower on a relative basis
App performancestartup, typing, opening and switching chats all faster than at launch

The outer loop is automated too: whenever a release is cut, Grok Bot spins up a Cursor project that fuzzes the whole app like a real user, opens a fix PR for each bug it finds, and auto-merges it. She says this works because the codebase is good enough that PRs tend to be "good by default".

I find the approach convincing, but the numbers deserve a sober reading:

  • These are self-reported numbers from a keynote, with no technical write-up or independent data attached yet.
  • A 0.5% revert rate is not a defect rate. It only counts PRs bad enough to roll back. Small bugs fixed by a follow-up PR do not show up, and at 11,000 PRs, 0.5% is still about 55 reverts a month.
  • Agent PRs tend to be small. 11,000 agent PRs are not the same unit as 11,000 human PRs, so do not compare productivity directly.
  • Grok Bot is SpaceX's own product, and the end of the talk pitches it as the tool for building the outer loop. That does not make anything wrong, it is just context worth knowing.

I would not copy the 99% comment ban as is. Lauren targets the comments agents use to justify a shortcut, and that point is right. But a comment explaining a business "why", such as a legal constraint or a decision made with a partner, still helps the next human reader. A reasonable middle ground: move the "rules" that currently live in comments into lint rules or tests, and record architectural decisions in ADRs, where someone reviews them.

# what to do next week

Lauren's closing advice is short: before you build a software factory and automate everything, fix every agent correction in the codebase first. Add a lint rule to block new occurrences of an anti-pattern, then clean up the old ones. Make verification solid. If your inner coding loop produces slop, automating it only produces more slop, and you end up throwing more and more tokens at the problem.

If I had to pick three things to start with:

  1. Open your AGENTS.md (or CLAUDE.md, .cursor/rules), find the lines that say "never do X", and try to turn each into a lint rule or ArchUnit test with a helpful error message.
  2. Pick the anti-pattern agents copy most in your repo, block new occurrences with a rule, then clean up the existing ones so agents have no bad examples left to copy.
  3. Give the agent a way to run and check the app on its own: a start script, a few CLI commands to drive features, and screenshots or logs attached to every PR as evidence.

The original video is only about 17 minutes and worth watching in full. For more on building an agent's environment, see the earlier posts on agentic AI architecture and ADLC, the agentic development life cycle.

References:


Just personal notes, shared in the hope they are useful. If you found this helpful, feel free to pass it on to your friends and colleagues!

Happy coding ๐Ÿ˜Ž ๐Ÿ‘๐Ÿป ๐Ÿš€ ๐Ÿ”ฅ.