Grok Bot team says it shipped 11,000 pull requests to production in a month
Lauren Tan said SpaceXSI’s Grok Bot team maintains the quality of coding agents’ code by turning corrections given to agents into automated checks. About 0.5 per cent of pull requests are reverted, Tan said.
Lauren Tan (poteto) described in a talk at the AI Engineer conference how SpaceXSI’s Grok Bot team keeps code quality high enough that, by Tan’s account, most product code no longer needs thorough human review. The video of the talk was published on 14 November. Tan described Grok Bot as an Electron-based desktop app for launching and steering Cursor’s cloud agents.
Corrections become automated checks
Tan starts from the premise that an agent also copies workarounds and outdated solutions from the codebase. When the team corrects the agent, the correction does not become a new line in an instructions file such as agents.md. Instead, the team turns it into an automated check or improves the interface or architecture, because a check in code works repeatably, Tan said.
Dune, the team’s own framework for Electron apps, includes lint rules: checks that reject code using a banned pattern. React’s useEffect, which Tan said agents use poorly, is banned in product code. Code for Electron’s main process and renderer process sits in separate directories, and neither may import the other’s code.
The team has also banned 99 per cent of code comments. Tan gave an example from Cursor: agents used comments to justify quick fixes built on timers, which degraded the app’s performance. Now the lint rule’s error message directs the agent to fix the root cause.
Tan advises starting with a lint rule that blocks new bad patterns, then cleaning up the old ones. Automating a way of working that produces bad code only produces more bad code, Tan said. It took Tan six months to reach the point of being able to ship thousands of pull requests to production.
Agents verify their own work
Review has shrunk to looking at videos, screenshots and diagrams, Tan said. The agent runs the app itself with debugging tools such as the Chrome DevTools Protocol. A feature map lists the app’s functions for the agent from the user’s point of view.
The team is also experimenting with formal verification, the mathematical checking of a program’s properties. The team checks algorithms with Lean and key user flows, such as notifications across multiple devices, with TLA+. Tan said the method has already found several bugs but is no silver bullet, and the experiment continues. Most of Tan’s tokens go on verification.
The results are the team’s own figures
Last month the team shipped 11,000 pull requests, or proposed code changes, to production, Tan said. In October, about 227 people contributed at least one pull request. A new stable version is released about once a day.
About 0.5 per cent of pull requests are consistently reverted, a lower share than before, Tan said. The talk’s transcript does not say which period the share is compared with, and the figures have not been independently verified. Tan said that since launch the app has become faster at starting up, typing, and opening and switching conversations. The transcript gives no measurements.
After each new version is released, Grok Bot starts a Cursor project that uses the app as a user would. The project fixes the bugs it finds and merges the fixes automatically. The codebase is in such good shape that pull requests are usually good from the start, Tan said.
Tan’s pstack plugin is available on GitHub in Cursor’s plugins repository. It includes a State Chart skill that draws state machines from the user’s point of view.
Read more: How an agent works · Agent map