Four NVIDIA Tesla V100 GPUs Beat Newer AI Hardware in Local LLM Benchmarks
NVIDIA’s Volta architecture may have launched back in 2017, but it is proving that older data-center GPUs can still be surprisingly powerful for modern AI workloads. In a recent local AI experiment, a four-GPU setup using Tesla V100 accelerators delivered impressive performance against newer systems, including Apple’s M5 Ultra and a pair of DGX Spark units.
The test shows that, with the right modifications, used enterprise hardware can still be a serious option for running large language models locally, especially for users who want high memory bandwidth and strong compute performance without relying on monthly cloud AI subscriptions.
The Tesla V100 remains a capable AI GPU thanks to its 32GB of HBM2 memory and 900GB/s of memory bandwidth. Those specifications still matter a lot for AI inference, particularly when running larger models or handling long prompts. While the V100 is no longer a new product, its high-bandwidth memory and data-center design allow it to remain competitive in certain AI tasks.
However, building a four-V100 system is not as simple as plugging in consumer graphics cards. YouTuber Jun Yamog managed to bring four Tesla V100 GPUs together using an NVLink carrier board capable of providing 300GB/s of bandwidth. The setup uses six NVLink 2.0 connections, helping the GPUs communicate at much higher speeds than a basic PCIe-only configuration would allow.
Cooling was another major challenge. Each Tesla V100 can generate around 300W of heat, so standard cooling would not be enough for a compact multi-GPU setup. To solve this, Jun modified a heatsink originally designed for NVIDIA’s A100 GPU, milling it down so it could fit the V100. Once adapted, the large cooler was able to handle the heat output from each chip effectively.
The entire carrier board was then connected using PCIe x8 through passthrough adapters. While the system required extra work, custom cooling, and specialized hardware, the final result was a powerful local AI machine built around older NVIDIA data-center GPUs.
In benchmarks using Qwen3.8-Flash-Next with 4-bit quantization, the four Tesla V100 GPUs performed extremely well. For longer prompt processing, the quad-V100 setup was around 40 percent faster than a dual DGX Spark configuration and roughly 230 percent faster than Apple’s M5 Ultra.
That gap against Apple Silicon may not be entirely due to raw hardware performance. Jun noted that the M5 Ultra result could be affected by software optimization limitations. Even so, the V100 system’s performance highlights how much value remains in older NVIDIA AI accelerators when they are paired with the right software and interconnect setup.
For shorter prompts, the four Tesla V100 GPUs maintained a similar lead over the DGX Spark setup and were around 163 percent faster than the M5 Ultra. In token generation tests for shorter prompts, the V100 cluster reached 100.6 tokens per second, making it 121 percent faster than the two DGX Spark units. The M5 Ultra, based on separately reported results, reached 44.5 tokens per second.
The benchmark comparison also included a system with eight RTX PRO 6000 GPUs, giving viewers a wider look at how different AI hardware configurations stack up. Still, one of the most interesting takeaways is that the Tesla V100, despite its age, remains highly relevant for local AI and LLM workloads.
This is not the first time the V100 has been repurposed for modern AI use. Other enthusiasts have combined Tesla V100 cards with newer GPUs, such as the RTX 4080, to gain more VRAM and memory bandwidth for running larger AI models. In some cases, that additional memory capacity can make the difference between running a model locally and not being able to load it at all.
For AI hobbyists, developers, and researchers interested in building a local AI workstation, used Tesla V100 GPUs may be worth considering. The 32GB version can often be found on the used market for far less than modern high-end professional AI hardware. Buying one or two units can significantly expand the range of models that can be run locally, especially when memory bandwidth and VRAM capacity are the main limitations.
Of course, there are trade-offs. Tesla V100 cards are not designed like standard gaming GPUs. They may require server-style cooling, adapters, custom mounting, or specialized carrier boards depending on the configuration. Power consumption is also high, and the setup may not be ideal for users who want a simple plug-and-play experience.
Still, for those comfortable with hardware modifications, the results are hard to ignore. A properly configured Tesla V100 system can deliver strong AI inference performance, handle demanding LLM workloads, and compete with much newer hardware in certain scenarios.
The experiment is a reminder that AI performance is not only about buying the newest chip. Memory bandwidth, VRAM capacity, software support, and multi-GPU communication can all have a major impact. In the right setup, NVIDIA’s 2017-era Volta hardware can still punch far above its age, making the Tesla V100 one of the more interesting used GPUs for local AI enthusiasts today.
