SanDisk’s High-Bandwidth Flash Pitch Raises Questions Over AI Memory Claims
SanDisk’s latest Investor Day presentation was meant to highlight the company’s vision for next-generation AI memory. Instead, parts of the presentation sparked criticism among hardware enthusiasts and AI infrastructure watchers, who argued that some comparisons between High-Bandwidth Flash and High-Bandwidth Memory appeared incomplete or overly favorable.
At the center of the discussion is High-Bandwidth Flash, or HBF, a developing memory technology that SanDisk is working on with SK hynix. The idea is straightforward: AI systems need more memory capacity, especially as large language models grow larger and context windows expand. HBM is fast, but it is expensive and limited in capacity because it sits close to the GPU package. HBF aims to offer much higher capacity at potentially lower cost, but there are important technical trade-offs that were not fully addressed in the presentation.
Why AI models need so much memory
Modern AI models rely heavily on two major components: attention layers and feed-forward networks.
The attention layer helps a model understand relationships between words or tokens in a prompt. For example, if a user says, “Paris is the capital of France,” the attention mechanism helps the model connect “Paris,” “capital,” and “France.” This process creates and stores temporary information known as KV cache. The longer the context window, the more KV cache is required.
The feed-forward network, often called the FFN, contains much of the model’s learned knowledge in the form of weights. These weights help the model interpret meaning, make associations, and produce useful responses.
Today, both model weights and KV cache are typically stored in HBM. That is one reason AI accelerators with large HBM capacity are so valuable. However, increasing HBM capacity is not simple. Because HBM stacks are physically integrated near the GPU, adding more memory often means using more GPUs or more advanced packaging, both of which can rapidly increase costs.
This is where HBF enters the conversation.
What High-Bandwidth Flash is trying to solve
High-Bandwidth Flash is designed to stack NAND flash dies in a way that resembles how HBM stacks DRAM. The structure uses Through-Silicon Vias to connect layers of NAND, along with a controller logic die that manages access to the storage array.
SanDisk has described a future HBF solution offering around 512GB of storage, with bandwidth ranging from roughly 0.4TB/s to 3TB/s. The company is targeting a commercial launch around the 2028 to 2029 timeframe.
On paper, that sounds promising. AI hardware needs more memory capacity, and HBF could offer a way to store more model data close to accelerators without relying entirely on expensive HBM stacks.
But the challenge is speed.
NAND flash is far slower than DRAM. SRAM can operate at around the nanosecond level, DRAM is typically measured in tens to hundreds of nanoseconds, while NAND access latency can be measured in microseconds. That means NAND can be roughly 1,000 times slower than DRAM for individual reads.
HBF attempts to overcome this limitation through massive parallelism. Instead of relying on a single fast read, the controller schedules thousands of NAND reads at the same time. By combining many slow reads in parallel, HBF can achieve much higher total bandwidth than conventional NAND storage.
That approach could make HBF useful for some AI workloads, particularly those that benefit from large-capacity memory pools. However, it does not eliminate all of NAND’s weaknesses.
The biggest concern: write endurance
One of the most important criticisms aimed at SanDisk’s presentation was the lack of detail around HBF write endurance.
NAND flash has a limited number of write cycles. Unlike DRAM, which can be rewritten continuously in normal operation, NAND wears out over time as data is written and erased. This is a fundamental characteristic of flash memory.
That matters because KV cache is not static. During AI inference, KV cache can change constantly as the model processes new tokens. If HBF is expected to handle workloads involving frequent writes, write endurance becomes a critical issue.
Without clear information on how many write cycles HBF can tolerate, how wear leveling will work, or whether the technology is intended mainly for read-heavy workloads, it is difficult to judge whether HBF can truly replace or supplement HBM for key AI tasks.
Critics argue that leaving out write endurance data makes the comparison incomplete.
Questions over the HBM comparison
Another point of criticism involves the way SanDisk compared HBF with HBM.
SanDisk’s presentation reportedly used 192GB of HBM per GPU and bandwidth of around 12.8TB/s. However, because HBF is not expected to arrive commercially until around 2028 or 2029, some observers argue that the comparison should be made against future HBM standards, not today’s or near-term configurations.
By that time, advanced HBM variants such as 16-high HBM4E may be more relevant. Such memory could offer significantly higher bandwidth, potentially around 32TB/s in certain configurations. If that becomes the industry benchmark, then HBF’s projected bandwidth would look less competitive than SanDisk’s slides may suggest.
In other words, HBF may offer more capacity, but HBM is still expected to maintain a major advantage in raw bandwidth and latency.
Quantization assumptions also raised eyebrows
SanDisk’s presentation also used bf16 as part of its AI model memory comparison. That choice drew criticism because many modern AI inference deployments increasingly use lower-precision formats such as fp8 or fp4.
This matters because precision affects memory usage. A large model stored in bf16 requires far more memory than the same model stored in fp8 or fp4. If a comparison assumes bf16 while real-world deployments are moving toward smaller formats, the memory pressure shown in the presentation may appear larger than it would be in practice.
For example, a model with hundreds of billions of parameters could occupy very different memory footprints depending on whether it is stored in bf16, fp8, or fp4. This directly affects the argument for how much HBF would be needed and how it compares to HBM-based systems.
HBF may still have a future in AI hardware
Despite the criticism, High-Bandwidth Flash is not without merit. The AI industry is clearly searching for new ways to expand memory capacity while controlling costs. HBM is powerful, but it is expensive and difficult to scale indefinitely. As models grow and inference workloads increase, memory innovation will become just as important as GPU performance.
HBF could become valuable in scenarios where large capacity matters more than ultra-low latency or extreme write endurance. It may also serve as an additional memory tier rather than a direct HBM replacement. In such a design, HBM could handle the fastest and most write-intensive operations, while HBF could store larger read-heavy data sets closer to the accelerator than traditional SSD storage.
That type of tiered AI memory architecture could make sense, especially for future data centers running large models at scale.
However, for HBF to be taken seriously as a major AI memory solution, the industry will need transparent answers on several key points:
How durable is HBF under sustained write-heavy workloads?
Can it handle KV cache efficiently, or is it better suited for model weights?
How will its bandwidth compare with future HBM standards available at the same time?
What will latency look like in real-world AI inference?
How much cost advantage will it actually deliver?
Until those questions are answered, HBF remains an intriguing but unproven technology.
SanDisk’s presentation may have succeeded in drawing attention to High-Bandwidth Flash, but it also highlighted how carefully AI memory claims need to be examined. Capacity alone is not enough. Bandwidth, latency, endurance, cost, and workload suitability all matter.
HBF could play an important role in the future of AI servers, especially as demand for larger memory pools continues to rise. But based on the current discussion, it is not yet clear that it can challenge HBM in the areas where HBM is strongest. For now, HBF looks less like a direct replacement for high-end GPU memory and more like a possible complementary tier in future AI infrastructure.
