NVIDIA Vera CPUs Deliver 2x AI Throughput Boost for Modal’s Agentic AI Sandboxes
NVIDIA’s Vera CPU is beginning to reshape how companies view Arm-based processors for demanding AI infrastructure. While Intel and AMD x86 chips have long dominated server environments, Modal’s latest testing shows that NVIDIA’s Arm-based Vera platform can offer a major advantage for agentic AI workloads, especially when systems are under heavy load.
Modal recently received its first NVIDIA Vera CPU servers at its San Francisco office. Until now, the company had relied entirely on x86-based systems powered by Intel and AMD processors. Moving to Vera was not a simple plug-and-play upgrade. Because NVIDIA Vera uses an Arm-based architecture, Modal had to make significant changes to its infrastructure, container images, and build systems before it could properly deploy and evaluate the new hardware.
That effort appears to have paid off. According to Modal’s testing, NVIDIA Vera delivered up to a 2x increase in AI throughput for agentic AI sandboxes compared with an equivalent Intel Xeon Granite Rapids system. Modal described the result as highly compelling, particularly for dense workloads where many containers are running at the same time.
The key advantage comes down to how Vera handles contention. In AI infrastructure, performance is not only about how fast a single task can finish. It is also about how many tasks a system can complete at the same time without slowing down.
Latency refers to the time it takes for a task to start and finish. Throughput measures how many tasks are completed within a given period. In most systems, improving one can hurt the other. If too many workloads are packed onto a CPU, throughput may initially rise, but latency often gets worse once the processor becomes fully loaded.
This tradeoff is a major concern for platforms like Modal, where workloads can be unpredictable and spiky. The company wants to pack more containers onto each host machine to make better use of its infrastructure, but overpacking can create performance problems. If a CPU becomes overloaded, individual tasks begin competing for resources, causing slowdowns.
To test NVIDIA Vera in a realistic environment, Modal created a set of representative workloads for agentic AI sandboxes. The company also built a benchmark called Agentless Agent Bench, which includes 50 Terminal-Bench oracle solutions and verifiers. This allowed Modal to evaluate both straight-line task completion and concurrent throughput under load.
In early tests without system load, NVIDIA Vera already showed an improvement over x86 CPUs. However, those results mostly reflected latency in a clean environment. The more important test came when Modal pushed the system with many concurrent workloads.
That is where Vera separated itself from Intel and AMD alternatives. Under load, Vera maintained its pace while competing x86 systems began to hit throughput limits. Modal found that Vera could deliver up to twice the task throughput per core compared with an Intel Xeon Granite Rapids processor.
The gap became even larger against AMD EPYC in one specific high-concurrency test. With 150 concurrent containers and 124 logical CPUs enabled on each system, Vera delivered more than 6x the task throughput. This result highlights how well the architecture performs when many AI sandbox workloads are running at once.
Modal credits Vera’s high memory bandwidth as a major reason for the performance uplift. More memory bandwidth allows the CPU to handle more container workloads per core without throttling. While Intel and AMD systems reached throughput plateaus earlier, Vera continued scaling as more CPU resources were added. This suggests that the platform may have even more headroom than Modal was able to test on the available server configuration.
For agentic AI, this matters a great deal. These workloads often involve many small, interactive, and concurrent tasks rather than a single long-running process. Sandboxes may need to run code, verify results, interact with tools, and process multiple requests at once. A CPU that can maintain performance under contention can improve infrastructure efficiency, reduce bottlenecks, and allow more users or agents to run simultaneously.
Modal’s experience also shows that the shift from x86 to Arm in AI infrastructure is becoming more practical. The transition still requires engineering work, especially for companies with existing x86-based systems and container pipelines. But the performance gains may justify the effort for businesses focused on dense AI workloads and large-scale agentic computing.
NVIDIA Vera’s biggest strength is not just raw speed in a lightly loaded benchmark. Its advantage becomes clear when the system is stressed by many simultaneous containers. Instead of slowing down as jobs accumulate, Vera continues processing tasks efficiently, making it a strong fit for environments where throughput, scaling, and predictable performance are critical.
For Modal, the move to NVIDIA Vera CPUs delivered a clear result: a major throughput boost for agentic AI sandboxes and better scaling under heavy contention. As AI platforms continue to grow more complex and more concurrent, Arm-based server CPUs like NVIDIA Vera could become an increasingly attractive alternative to traditional Intel and AMD x86 processors.
