Running a massive Large Language Model with 400 billion parameters usually demands workstation-class hardware and an enormous amount of memory. Even after compression or quantization, models of this size can still need around 200GB of RAM just to operate. That’s why the idea of running a 400B LLM on a smartphone sounds unrealistic at first glance.
Yet a new demonstration shows an iPhone 17 Pro doing exactly that—running a 400B model locally on-device. It’s an eye-opening proof of concept, but it comes with major caveats, and it only works thanks to a few smart technical workarounds.
The biggest limitation is speed. In the demo, the iPhone 17 Pro generates text at roughly 0.6 tokens per second, which works out to about one word every 1.5 to 2 seconds. That’s slow enough to test anyone’s patience, especially if you’re used to instant responses from cloud-based AI assistants. It’s the kind of performance that makes the difference between “it runs” and “it’s usable” very clear.
The magic behind this achievement comes from an open-source project called Flash-MoE. The iPhone 17 Pro has 12GB of LPDDR5X RAM, nowhere near enough to load a full 400B-parameter model into memory. Instead, Flash-MoE relies on storage streaming, using the phone’s SSD to feed model data directly to the GPU as needed rather than keeping everything in RAM at once.
There’s another key trick: MoE stands for Mixture of Experts. Rather than activating all 400 billion parameters for every generated token, an MoE model only uses a subset of the network each time it predicts the next word. That reduces the active compute and memory pressure per step—one of the reasons a model with a headline parameter count this large can be coaxed into running on much smaller devices.
Why does this matter if it’s still painfully slow? Because it highlights where on-device AI is headed. If developers can keep improving techniques like model streaming, expert routing, and further optimization, running larger and more capable local language models on smartphones becomes more realistic over time.
On-device LLMs also bring real-world benefits that many users care about. Local processing can offer stronger privacy because prompts and responses don’t need to be sent to remote servers. It can also work without an internet connection, which is useful for travel, connectivity dead zones, or anyone who simply prefers offline tools. The tradeoff is that workloads this heavy can hit battery life hard, especially when the GPU and storage are being pushed continuously.
To be clear, quantized models are already a popular strategy for shrinking AI to fit on consumer hardware, but even a quantized 400B model would typically still demand far more memory than a smartphone can provide. That’s what makes this demo notable: it’s not a typical “download and run” scenario—it’s a clever engineering approach to prove what’s possible.
The takeaway is simple. Yes, a 400B-parameter Large Language Model can be made to run on a smartphone like the iPhone 17 Pro, but it currently runs at a pace that’s difficult to recommend for everyday use. Still, as a demonstration of on-device AI potential, it’s a strong signal that the gap between mobile hardware and advanced local language models is starting to narrow—one optimization at a time.
