A 145-billion-parameter AI model now runs on a laptop

On-device AI has long forced a trade-off: a model could be capable but too demanding for the hardware, or small enough to run but weak on complex work. One team says it has changed that equation. According to the report, Edge0-145B and Edge0-120B were released as open source along with a unified inference framework and deployment tools. The team claims the 145-billion-parameter model is the first open-source on-device model of that size that can run on a personal laptop.

The real bottleneck was memory and data movement. Model weights sit in storage and must be read during generation, competing with the operating system and other apps for limited resources. As the article explains, the team designed the architecture and the inference engine together to cut that load. Sharing large matrices across groups of experts reduces how many weights must be stored and moved, and only about 10 billion of the model's 145 billion parameters take part in each inference step.

The team previously reported that its 35-billion-parameter phone release ran at more than 37 tokens per second using roughly 2.5 GB of memory. For the new model, its laptop comparison cites more than 80 tokens per second and 28 GB of peak memory, though these are different models and different hardware and should not be read as a direct comparison.

Why does this matter? Until now, using powerful models mostly meant relying on the cloud, with provider fees, data-handling constraints and a dependence on connectivity. According to the report, the team is opening up the weights, framework and app so developers can reproduce results, fine-tune the models and build them into their own products.