Samsung’s Next-Gen HBM Roadmap Points to Smarter Memory for AI, With 3D ZHBM as the End Goal
Samsung is laying out an ambitious future for high-bandwidth memory, and the message is clear: the HBM base die is no longer just a passive interface layer. With advanced logic processes, it can become an active, intelligent part of AI computing systems.
The company’s latest vision focuses on how HBM can evolve to meet the exploding bandwidth, capacity, and power-efficiency demands of modern AI accelerators. As GPUs, TPUs, and other XPUs require faster access to larger pools of data, traditional memory architectures are running into limits. Samsung’s answer is a three-stage roadmap that starts with improving the base die, moves toward feature-rich custom HBM, and ultimately leads to ZHBM, a true 3D memory architecture that stacks DRAM directly on top of compute silicon.
Understanding the Role of the HBM Base Die
HBM, or High Bandwidth Memory, is built from stacked DRAM dies connected through TSVs, or through-silicon vias. These vertical connections allow data to move quickly between layers. A typical HBM stack includes core dies, which contain the DRAM cells, and a base die at the bottom, which manages interface, signaling, testing, and other support functions.
In current designs, the base die acts mainly as the communication bridge between the DRAM stack and the compute die. That compute die could be a GPU, TPU, AI accelerator, or another type of high-performance processor. Data moves from the HBM stack to the XPU through a high-speed interface, enabling the massive bandwidth required for AI training and inference.
However, as AI workloads grow, this architecture faces major pressure. Each new HBM generation is expected to deliver much higher bandwidth, larger capacity, and better energy efficiency. HBM4 is already expected to exceed 3 TB/s of bandwidth per stack, while HBM4E could move into the 4 TB/s range. HBM5 is expected to push bandwidth even further while also increasing capacity beyond 60 GB per stack.
The challenge is that bandwidth does not scale for free. TSV density, pitch limitations, I/O count, interface speed, power consumption, and physical area all become obstacles. Samsung believes the solution is to modernize the HBM base die using advanced logic process technology.
Why Advanced Logic Processes Matter for HBM
As the process gap between memory base dies and compute processors becomes smaller, Samsung sees an opportunity to bring more logic capability into the HBM stack itself. The company has already applied advanced DRAM and logic technologies, including D1c and 4nm-class processes, to HBM4 development with the goal of reducing power consumption and shrinking active area.
This shift is important because it changes what the base die can do. Instead of simply supporting data movement, the base die can begin to host more complex functions that previously lived on the XPU. This opens the door to custom HBM designs with smaller interfaces, lower power draw, improved thermals, and more available silicon area on the main processor.
Samsung’s roadmap breaks this transformation into three major phases.
Phase 1: Reclaiming Valuable XPU Area
The first stage focuses on reducing the amount of silicon area consumed by traditional memory interface blocks on the compute die. In standard HBM systems, the PHY interface takes up valuable space and consumes significant power. Samsung’s custom HBM approach replaces the traditional HBM PHY with a more compact die-to-die interface built using advanced logic processes.
This creates several benefits. The interface footprint becomes smaller, the communication path becomes shorter, and energy efficiency improves because data does not need to travel as far. By moving certain memory-related functions into the HBM base die, Samsung estimates that 5% to 10% of XPU area could potentially be reclaimed. That extra area could then be used for more compute cores, cache, or AI acceleration logic, potentially improving performance by 10% to 20% depending on the design.
Thermals are a major concern when more logic is added to the base die. Shorter interconnects are efficient, but dense logic can create hotspots. To address this, Samsung has introduced Heat Path Block technology, designed to reduce peak temperature by more than 35%. This thermal-management approach is especially important for custom HBM4 designs, where the base die begins taking on a more active role.
One of the first major functions Samsung is considering moving from the XPU to custom HBM is the memory controller. This is a practical candidate because it can reduce area on the main processor without creating an unmanageable thermal burden.
Samsung is also exploring SRAM-based fine-grained repair schemes inside the base die. By using unused base die space for repair resources, the memory system could support cell-level fail-address decoding and redirection. This could improve yield, reliability, and flexibility in advanced HBM products.
Phase 2: Expanding HBM Functionality
The second phase moves beyond area savings and begins turning the HBM base die into a more capable platform. Samsung refers to this broader concept as advanced HBM, or AHBM.
In this stage, the base die could integrate advanced reliability, availability, and serviceability features. Real-time telemetry sensors could monitor voltage, temperature, aging, and other operating conditions inside the HBM stack. This would give system designers much better visibility into memory behavior, helping improve stability and performance in demanding AI servers.
On-die testing is another important feature. By embedding more test capability into the base die, Samsung could improve fault detection, manufacturing coverage, and long-term reliability. This matters greatly for AI data centers, where memory failures can disrupt expensive training runs or large-scale inference workloads.
Another major idea in this phase is memory expansion. AI models are becoming larger, and context windows are growing rapidly. This creates huge demand for memory capacity, especially for KV cache storage in large language model workloads. Samsung proposes using unused base die area to integrate a memory extension controller and PHY. This could allow HBM to connect with additional external memory, such as LPDDR or other HBM resources, increasing effective capacity near the XPU.
Samsung is also exploring selective processing elements inside the HBM base die. These would not replace the main accelerator, but they could perform limited near-memory computation. By handling certain operations closer to the data, the system could reduce data movement, lower bandwidth pressure, and cut power consumption. For AI workloads, where moving data often consumes more energy than computation itself, this could be a major advantage.
Phase 3: ZHBM and True 3D Memory-Compute Integration
The final stage of Samsung’s roadmap is the most transformative: ZHBM.
Unlike today’s HBM implementations, which typically use a 2.5D interposer to place memory stacks next to the compute die, ZHBM is designed to stack HBM vertically on top of the XPU. This creates a much tighter connection between DRAM and compute logic, reducing physical distance, improving bandwidth, and lowering I/O power.
By eliminating the conventional 2.5D interposer, ZHBM could simplify the data path and make memory access far more efficient. Samsung’s concept uses distributed I/O connections across the stack, allowing signals to travel shorter distances instead of being routed through large horizontal interfaces.
The expected benefits are significant. Compared with a standard HBM4E-style approach, Samsung claims ZHBM could deliver up to 70% better DRAM power efficiency and more than 2.3 times the bandwidth. The company also points to potential savings of up to 100W per DRAM module, which could free additional thermal and power headroom for the GPU or other XPU beneath it.
In one example, Samsung describes a configuration with four ZHBM stacks placed over an XPU. This kind of arrangement could provide enormous bandwidth while allowing more of the system power budget to be used for compute performance.
To make ZHBM possible, Samsung is working on advanced packaging methods such as wafer-on-wafer bonding and hybrid bonding. These technologies are essential for achieving the ultra-high I/O density needed to connect DRAM and compute silicon in a unified 3D structure.
What This Means for AI Hardware
Samsung’s HBM roadmap is not just about making memory faster. It is about changing the role of memory in AI systems.
Today’s AI accelerators are often limited by memory bandwidth, memory capacity, and data movement power. As models grow larger and workloads become more complex, simply adding faster memory interfaces is not enough. The industry needs architectures that bring memory and compute closer together.
Custom HBM and advanced HBM represent the first steps. By moving memory controllers, telemetry, repair functions, testing logic, extension controllers, and selective processing elements into the base die, Samsung can reduce pressure on the XPU and improve overall system efficiency.
ZHBM takes the idea further by physically stacking DRAM on compute. If successful, this could mark a major shift in AI accelerator design, enabling higher bandwidth, lower latency, better power efficiency, and more compact packaging.
The Bigger Picture
Samsung’s three-phase plan shows how HBM could evolve from a high-speed memory stack into a smart, tightly integrated partner for next-generation AI processors.
Phase 1 focuses on reclaiming XPU area, reducing PHY footprint, improving thermals, and offloading memory-control functions.
Phase 2 adds intelligence to the base die through monitoring, testing, memory expansion, repair features, and near-memory processing.
Phase 3 brings full 3D integration with ZHBM, placing DRAM directly on top of compute to reduce I/O power and dramatically increase bandwidth.
For AI servers, data centers, and high-performance computing platforms, these changes could help overcome some of the biggest bottlenecks in modern system design. More bandwidth, more capacity, lower power, and better thermal efficiency are exactly what future AI workloads will require.
Samsung’s vision suggests that the future of HBM will not be defined only by taller DRAM stacks or faster interfaces. Instead, the biggest breakthrough may come from turning the base die into an intelligent logic layer and eventually merging memory and compute into a true 3D architecture.
