Nvidia announced the news in a blog post on 10 November, alongside KubeCon + CloudNativeCon, the conference for Kubernetes developers. The Cloud Native Computing Foundation (CNCF), part of the Linux Foundation, is taking on OpenShell as a Sandbox project. Organisations that build OpenShell into their own products will steer the project from now on, Nvidia said.

OpenShell monitors AI agents, which are programs that work with the help of an AI model. When an agent uses tools, calls an inference service or runs workloads, OpenShell applies deterministic rules, meaning rules that always work the same way. Users can connect their own compute resources, inference services, credential store and monitoring tools. Hundreds of organisations use OpenShell, and it has received tens of thousands of contributions in less than a year, the company said.

Agent Substrate saves a waiting agent’s state

Nvidia is also contributing to Agent Substrate, a runtime led by Google. The CNCF has accepted the open-source environment as a Sandbox project. An agent spends most of its time waiting for a human prompt, a tool result or a model response, Nvidia’s technical blog says. Kubernetes runs programs in pods, its smallest deployable units. If every session has its own pod, idle sessions hold on to memory. If the pod is deleted between steps, a fresh start takes several seconds and the agent loses its state.

Agent Substrate keeps a small pool of worker pods ready and shares a large number of agents among them. When an agent goes idle, Agent Substrate snapshots its state out of the sandbox (gVisor or microVM), together with its memory and file system. The worker pod is freed. When the next call arrives, Agent Substrate restores the state into any free pod without Kubernetes’ usual start-up cycle. Nvidia said Agent Substrate is designed to run ten times as many agents as standard containers on the same capacity. The blog presents no measurements.

Nvidia has added queueing to Agent Substrate: when all worker pods are busy, a request waits for a limited time instead of ending in an error. MicroVM sandboxes gained per-container CPU and memory limits. GPU support is being built on Kubernetes’ Dynamic Resource Allocation mechanism. Saving GPU state is so far only a proposal. It would solve the problem that saved state can usually be restored only to the same GPU model, and sometimes only with the same driver version.

Nvidia’s Dynamo inference framework can give Agent Substrate information such as an estimate of time-to-first-token latency and the load on the GPU pool. Based on this, Agent Substrate can keep an agent running, suspend it during a long wait or wake it when a response is nearly ready. The authors wrote that the interfaces are still changing, GPU support is being rebuilt and part of the Dynamo integration is experimental. Nvidia and Google are also working on making OpenShell and Agent Substrate work together.

Inference services start in seconds

Nvidia also released three open-source tools for operating GPU clusters on Kubernetes. NVIDIA Kubernetes Engine and NVIDIA Kubernetes Recipes are in public preview. NVIDIA Snapshot starts an inference service from a saved copy without reloading the model.

Nvidia’s blog describes partner tests on HGX B300 systems. In Crusoe’s test, a GLM-5.3 model service was ready in 31 seconds, down from about eight minutes previously. Crusoe itself has contributed to Snapshot’s development. In Saturn Cloud’s measurements, vLLM start-up times fell from 2–8.5 minutes to 6–34 seconds.

Agent Substrate’s code and discussion are in the agent-substrate/substrate repository.