SanDisk Explores 3D NAND Stacking to Tackle AI Memory Bottlenecks
The explosive growth of artificial intelligence is putting enormous pressure on the memory industry. As AI accelerators, GPUs, and high-performance computing systems become more powerful, they are also demanding faster access to larger pools of data. That demand is exposing one of the biggest challenges in modern chip design: memory is struggling to keep up with compute.
For years, DRAM has been the go-to solution for high-speed memory, while NAND flash has been favored for large-capacity storage at lower cost. But the AI era is changing the equation. Faster processors are only useful if they can be fed with data quickly enough, and today’s memory technologies each come with trade-offs.
High Bandwidth Memory, or HBM, has become essential for advanced AI chips because it delivers much higher bandwidth than traditional DRAM. However, HBM is expensive, limited in capacity, difficult to manufacture at scale, and currently facing supply constraints. Even as new generations improve speed and density, demand from AI hardware makers continues to outpace availability.
NAND flash, on the other hand, offers much larger capacity at a lower cost. The downside is that it is much slower than DRAM and typically sits farther away from the processor, which increases latency. That makes it less suitable for the immediate, high-speed data access required by AI workloads.
SanDisk is now looking at ways to combine the strengths of both technologies more effectively.
The company has already discussed a concept known as High-Bandwidth Flash, or HBF. This approach borrows ideas from HBM by stacking multiple layers of NAND flash and connecting them through Through-Silicon Vias, also known as TSVs. These vertical connections allow the stacked memory layers to communicate more efficiently, creating a single high-capacity memory package.
While current HBM stacks commonly offer capacities in the range of tens of gigabytes, HBF could potentially scale far higher, with capacities reaching up to several terabytes per stack. That could make it especially attractive for future AI and data-intensive workloads where massive datasets need to remain close to the processor.
However, SanDisk’s latest patent points to an even more ambitious idea: placing NAND flash directly beneath the main compute chip.
The patent describes a 3D-stacked architecture where a NAND flash memory tile using CBA, or CMOS Bonded Array technology, is positioned under a compute tile. That compute tile could be a large AI accelerator, graphics processor, or another type of high-performance processor. In this design, HBM would still be placed on the same interposer around the processor, but it would not be the only memory resource.
The idea is to let each memory type do what it does best.
HBM would handle the most urgent, bandwidth-sensitive tasks that require extremely fast access. Meanwhile, the NAND-based memory tile would support larger read and write operations, offering much greater capacity closer to the processor than traditional storage solutions. By placing NAND directly beneath or very near the compute tile, the design could reduce data movement, improve efficiency, and lower overall system cost and power consumption.
In simpler terms, SanDisk appears to be exploring a future where AI chips are not forced to rely only on expensive HBM for every memory task. Instead, they could combine fast HBM with dense NAND flash in a tightly integrated package, creating a more balanced memory hierarchy.
This could be important for AI training, inference, scientific computing, graphics workloads, and large-scale data processing. These applications often need both extremely fast memory and huge capacity. Today, balancing those needs is difficult. Too much reliance on HBM increases cost and supply pressure, while leaning too heavily on conventional NAND introduces latency and performance limits.
A processor-on-NAND design could help close that gap.
The patent describes a processing core where a multi-core processor is integrated directly with high-bandwidth, high-capacity non-volatile memory. The non-volatile memory may include a large NAND memory tile bonded with CMOS logic. The entire structure can be placed on an interposer, with HBM stacks arranged around one or more sides of the processor and memory tile.
That packaging strategy could give future chips a more flexible memory system. The processor would have access to ultra-fast HBM for immediate data needs, while also benefiting from a much larger pool of nearby NAND flash for heavier data storage and movement.
Still, this is not a product announcement. It is a patent, and there are many challenges to solve before such a design could become commercially viable.
Power consumption would be a major concern. Stacking compute and memory closely together can create thermal challenges, especially when the processor is a large GPU or AI accelerator. Manufacturing cost is another hurdle, since combining advanced compute silicon, HBM, and NAND flash in one complex package would require sophisticated assembly and strong yields.
There are also questions about performance consistency, software support, memory management, reliability, and how such a system would fit into existing AI server architectures. Even if the concept is technically promising, turning it into a mass-produced product would take time.
Even so, the patent is significant because it shows where the memory industry may be heading. As AI hardware continues to scale, simply making processors faster is not enough. The future of performance will depend heavily on how efficiently data can move between compute engines and memory.
SanDisk’s approach suggests that NAND flash may play a much bigger role in high-performance computing than it does today. Rather than remaining a slower, distant storage layer, NAND could become part of a tightly integrated memory architecture designed specifically for AI workloads.
The broader trend is clear: memory innovation is becoming just as important as processor innovation. HBM helped unlock the current wave of AI accelerators, but capacity limits, cost, and supply constraints are pushing companies to search for new solutions. High-bandwidth NAND concepts, 3D stacking, and processor-on-memory designs could become key technologies in the next generation of AI chips.
For now, SanDisk’s patent offers a glimpse into one possible future. It outlines a strategy where DRAM and NAND are not competing technologies, but complementary parts of a more advanced memory system. HBM would provide speed, NAND would provide capacity, and advanced packaging would bring both closer to the compute engine.
If this type of architecture can overcome manufacturing, thermal, and cost challenges, it could reshape how AI accelerators and GPUs are built in the years ahead. The race to solve the AI memory bottleneck is only beginning, and SanDisk’s stacked NAND vision could become an important part of that story.
