Google says it tests its browser agents on a network of thousands of simulated websites
Google’s Mirror Dimension environment redirects the real web addresses an agent opens to simulated sites, so the model does not notice it is being tested. Google says the test pipeline produces more than 1,000 attack test candidates a day per engineer.
Google’s Chrome team explained in a blog post on 19 November how it tests Chrome’s auto browse feature, in which an AI agent browses the web on the user’s behalf. The post was written by Aaron Selya and Charles Qian, software engineers in Chrome AI security. They say one of the main challenges with simulated websites is eval awareness: the model suspects it is being tested and may behave differently than in real use. The test environment must therefore not give itself away.
The agent can open a real address, such as youtube.com. The browser runs in an isolated gVisor sandbox, which routes traffic to a local proxy server. The server shows the agent a simulated site and generates an HTTPS certificate for it on the fly. Chrome accepts the certificates because a custom root certificate authority is installed in the sandboxed browser.
Google said the technique requires full control of the network and cannot be used against ordinary Chrome users, NewsBytes reported.
The pages carry prompt injections: hidden instructions meant to push the agent into unauthorised actions. An instruction can sit in text, an image, an accessibility label or a tool response. At first the attack identifier appeared as a URL parameter, and the blog says frontier models in many cases spotted the test from that alone. The identifier now travels in an HTTP header, so the address looks normal.
The sites are built by a harness, the program that steers the agents, gives the models tools and runs their workflow steps. Subagents draft requirements, code the page and check it with screenshots and clicks. The blog refers both to thousands of simulated websites and to maintaining hundreds of interactive pages.
Google says its test pipeline produces more than 1,000 attack test candidates a day per security engineer, which the company puts at more than 30 times what an engineer could do by hand. The figure measures the number of tests. The blog does not say how the agents performed in them.
Google stressed that the agents tested so far have no cyber capabilities, NewsBytes reported. Heise wrote that the method covers only some of the risks and does not solve the problem that criminals could use agents for attacks.
Citing WIRED, Heise reported that Google plans to open the environment to its other internal teams. Google has not announced that it will open it to other browser makers, Heise said.
Read more: How an agent works · Agent map