Chetas Joshi, an engineer on Robinhood’s data platform team, explained how the team reviews changes to its data pipelines in a talk published on the AI Engineer channel on 10 November 2026. Coding agents open pull requests – proposed changes awaiting approval – faster than the team’s few experts can read them. The changes affect schemas, configuration and data access permissions, among other things.

Scripts check the facts, an agent assesses the risks

Scripts check each change against data from production systems. They compare the schema with the schema registry and fetch the table owner and personal data tags from the data catalogue. They also look up the column’s readers in the lineage tool, which records which jobs read which data. A performance test runs the change in a test environment at production data volume. If a check fails, the pull request is halted.

At the same time, an AI agent reads the change and can fetch files from the repository, but it cannot access production systems. Rules tell the agent what the scripts already cover. They direct it to look for problematic query patterns, access permissions on sensitive data and readers missing from the lineage tool. The agent comments on individual lines, giving its reasons, scores the risk and either approves the change or passes it to a person.

In Joshi’s example, a column’s integer type was widened to allow larger values (integer → bigint). The scripts’ checks passed, but the agent found an unregistered job in the repository. The job wrote the value to a column in another table that still accepted only smaller integers. The write would have failed on the first value that was too large. A person confirmed that the job was in use, and the other table was fixed before the merge.

The agent cannot see manually run queries or dashboards, so it must not claim that there are no readers. It may only state that it found none in the places it searched. A person always handles access permissions on sensitive data, migrations that change the meaning of data and uncertain assessments.

The figures are Robinhood’s own

As of 31 August 2026, about 60 per cent of data pipeline pull requests were merged without human review, Joshi said. Manual review effort fell by about 75 per cent, saving about 132 working hours a month. Incidents linked to data pipeline changes fell by about 30 per cent. Joshi did not give a comparison period in the talk.

Joshi put the rate of incorrect approvals at about 2 per cent; these stemmed mainly from gaps in the rules. The team has no clear figure for unnecessary handovers to people. The on-call engineer updates the rules by hand once a week based on people’s feedback. The system does not learn automatically.

The scripts and the agent cannot see each other’s results

Joshi said the team made a mistake by building the scripts and the agent as separate systems that run in parallel. Neither sees the other’s output, and the agent cannot call the scripts. The team is exploring a design in which a classifier decides how an entire pull request is handled and the agent gets the script checks as tools. This has not yet been built.