Samsung’s 8-layer HBM3E memory chips have successfully completed quality assessments and are ready to be integrated into NVIDIA’s AI accelerators to deliver heightened performance and energy efficiency. The collaboration signals a significant advancement in memory technology for data-intensive applications, with NVIDIA’s AI chips poised to benefit from this innovation.
The fifth-generation HBM (High Bandwidth Memory), known as HBM3E, represents a notable leap over the currently used HBM3 in NVIDIA’s Hopper GPUs. While HBM3 provides a 1024-bit interface and transfer rates of 6.4 Gb/s, HBM3E promises to push the data rate up to 9.6 Gb/s. Consequently, this upgrade will enhance memory bandwidth to over 1200GB/s from HBM3’s 819GB/s, an essential boon for applications in AI, machine learning, and data analytics which demand substantial memory bandwidth alongside reduced power consumption.
Despite previous rumors concerning Samsung’s HBM3E chips being rejected by NVIDIA due to potential issues with heat and power consumption, Samsung addressed these concerns and reiterated that their chips meet the essential requirements for AI processors in terms of efficiency and temperature regulation.
The official partnership between Samsung and NVIDIA is anticipated to be confirmed soon, with expectations that Samsung will commence the supply of these chips to NVIDIA starting in the fourth quarter of 2024. The HBM3E technology’s adoption is projected to account for up to 60% of the overall HBM chip sales by the same quarter, with the demand for HBM chips forecasted to have an annual growth rate of 82% through to 2027.
While Samsung has achieved recent success in their HBM3E testing with NVIDIA, competitor SK Hynix is not far behind, moving towards shipping their 12-layer HBM3E chips, some of which are already allocated for NVIDIA’s existing H200 and forthcoming Blackwell B100 GPUs. SK Hynix has been a prominent player in the HBM3 market, reaching key production milestones ahead of schedule, indicating a hotly contested space in the HBM industry.
Micron, another significant player, has already begun mass production of its own HBM3E solutions, offering yet another option for NVIDIA. As such, the HBM landscape is becoming increasingly competitive, with numerous suppliers vying for dominance and the chance to power the next generation of advanced AI processing capabilities.
As we move forward, the thirst for more robust and efficient memory solutions like Samsung’s HBM3E illustrates the tech industry’s relentless pursuit of innovation, chiefly to cater to the growing complexities and computational demands of AI and data-centric workloads.
