NVIDIA has solidified its position as a leader in the data center market by confirming that its anticipated Blackwell platform is now operational and on schedule for global distribution within the year. This announcement casts aside any concerns of potential delays and reinforces NVIDIA’s commitment to innovation in the realm of data center, cloud, and AI solutions.
The Blackwell lineup is more than a single chip; it represents a comprehensive platform that includes a variety of components engineered to meet the demanding requirements of modern AI applications. The Blackwell family encompasses the Blackwell GPU, Grace CPU, NVLINK Switch Chip, Bluefield-3, ConnectX-7, ConnectX-8, Spectrum-4, and Quantum-3.
To emphasize the technological prowess behind the Blackwell series, NVIDIA has showcased images of the various trays within the lineup, highlighting the engineering acumen that goes into crafting next-generation data center platforms.
The Blackwell generation has been meticulously designed to handle the complexities of current AI tasks, including performance optimization for expansive language models such as the 405B Llama-3.1 from Meta. As these large language models (LLMs) expand to incorporate more parameters, the need for increased computational power and reduced latency becomes critical. While constructing a larger GPU with ample memory could host an entire model, utilizing a multi-GPU setup is key to achieving the low latency required for efficient token generation.
In a multi-GPU configuration, each GPU must communicate and relay the results of its calculations to every other GPU at each processing layer. This necessitates a high bandwidth communication network between GPUs, and NVIDIA has risen to the challenge with its NVSwitch. The NVSwitch enhances inference throughput by utilizing an interconnect bandwidth of 900 GB/s, significantly improving communication between GPUs and reducing the number of hops required for data transmission.
The Blackwell GPU itself is a marvel, consisting of two reticle-limited GPUs fused into one package and packing a powerful punch with 208 billion transistors, built on TSMC’s 4NP node. It boasts 20 petaFLOPS of FP4 AI capability, an 8 TB/s memory bandwidth, 8-site HBM3e memory, and a 1.8 TB/s bidirectional NVLINK bandwidth that facilitates a high-speed connection to the Grace CPU.
By capitalizing on the reticle limit of the GPU, NVIDIA achieves the highest communication density, lowest latency, and optimal energy efficiency obtainable in chip design. With the Blackwell upgrade, the NVLINK Switch’s capability has been doubled to 1.8 TB/s, accommodating a full bi-directional bandwidth of 7.2 TB/s across 72 ports and further expanding the NVLINK’s reach to 72 GPUs within the GB200 NVL72 racks.
In an effort to enhance performance and efficiency, NVIDIA is also exploring innovative liquid-cooling systems, such as the warm water direct-to-chip method. This approach promises to boost cooling efficiency, reduce operational costs, extend server life, and even allow for heat reuse, potentially cutting down data center power costs by 28%.
The Blackwell platform, when integrated with NVIDIA CUDA software, paves the way for a new era of AI applications, transcending boundaries across a myriad of use cases and industries. The GB200 NVL72 product is a testament to this, bringing together 72 Blackwell GPUs and 36 Grace CPUs in a liquid-cooled, multi-node system that redefines AI system design standards. With NVLink’s all-to-all GPU communication, it enables unprecedented throughput and minimal latency ideal for generative AI tasks.
Moreover, NVIDIA has revealed the first AI-generated image using FP4 compute, demonstrating that models with reduced precision can maintain quality at accelerated speeds. Even as precision diminishes (transitioning from FP16 to FP4), the image quality largely remains preserved. Such advancements in reduced-precision AI computation are part of NVIDIA’s Quasar Quantization system research initiatives.
Leveraging AI to further its hardware design, NVIDIA uses generative AI to produce optimized Verilog code, a hardware description language that fuels the design and verification of processors like Blackwell, and aids in the acceleration of future chip architectures. NVIDIA’s annual product release cycle promises even more advancements, with the Blackwell Ultra GPU being anticipated next year, followed by Rubin/Rubin Ultra GPUs in subsequent years.
The technological evolution led by NVIDIA’s Blackwell platform not only showcases the company’s relentless push towards the frontier of AI and computational power but also affirms its commitment to providing cutting-edge solutions to meet the ever-growing challenges of AI-infused industries.
