All stories
Yapay Zekâ

DeepSeek V3 reimplemented in 190 lines and trained on a MacBook Air

DeepSeek V3 reimplemented in 190 lines and trained on a MacBook Air

Complex language model architectures usually hide behind several hundred lines of configuration and abstraction. According to the post, one developer did the opposite this time: he reduced the architecture from the DeepSeek-V3 technical report to 190 lines of pure JAX and trained it from scratch on a MacBook Air. There is no Flax, Equinox or PyTorch; parameters live in plain dicts, and every part of the model sits in a single file as functions you can read top to bottom.

The training data is as unusual as the code. The write-up says the model was trained on around 200 detective and crime novels that are public domain in the United States, including Dashiell Hammett's books and texts gathered from Project Gutenberg. The developer is explicit that this is an architecture demo, not a strong language model. The goal is not conversation ability but making the model's inner workings visible.

The most interesting result concerns the expert balancing mechanism. According to the report, validation loss at step 2000 was nearly identical with balancing on and off: 3.921 versus 3.923. That supports, at small scale, the paper's claim that the bias term evens out the experts without costing anything. Still, the developer warns that you should not expect exactly the same numbers on your own hardware, since math on Metal is not bit-for-bit deterministic.

So who benefits? People learning about model architectures, those who want to experiment with JAX, and anyone who likes understanding complex systems through simple code have a good starting point here. The project is shared under an open source licence, and anyone can run it on their own Mac or on CPU. One caveat is also noted: the published checkpoint is a Python pickle that can run code when loaded, so it should only be taken from sources you trust.