There's a running joke in software engineering that Doom — the 1993 shooter — runs on everything. Since its source code was opened up, people have ported it to pregnancy tests, wristwatches, graphing calculators, gym equipment, thermostats, printers, and vapes. It's a gag, but it's also one of the most instructive traditions in our field: a standing demonstration that with enough understanding of the hardware, software thought to require "real" computers runs happily on almost nothing.
I've come to believe the same tradition is quietly playing out with language models — and that it matters far more than the trillion-parameter arms race getting all the headlines.
The existence proof nobody asked for
Here's my favorite demonstration of the genre: a working GPT-style language model running on a Commodore 64. The 1982 home computer. Roughly 38 KB of usable RAM, an 8-bit processor, floppy disks for storage.
Absurd? Consider that the Apollo 11 guidance computer had <em>less</em> on most specs than a C64, and it helped land humans on the moon. Compute constraints have always been negotiable for engineers willing to actually engineer.
The recipe for the C64 feat is a miniature masterclass in the craft. Train a tiny transformer — around 133,000 parameters, a nano-GPT-style architecture — on a narrow corpus (in this case, examples of Commodore BASIC code, where a modest number of <em>high-quality</em> samples beats a mountain of mediocre ones; training takes minutes, even on a CPU). Then leave the Python comfort zone entirely: quantize the checkpoint from 16-bit floats down to 8-bit integers, because the chip only speaks bytes. Reimplement the layers — attention and all — in C and 6502 assembly, with fixed-point math functions written by hand. And when the 131 KB model still won't fit in 38 KB of RAM, page the weight matrices back and forth from floppy disk.
The result is a slow, rudimentary code assistant on forty-year-old hardware. Commercially useless, obviously. But as an existence proof it's devastating to a whole set of industry assumptions — because if <em>that</em> hardware can run a specialized language model, then the phone in your pocket is a supercomputer.
Your "commodity hardware" is embarrassingly capable
That's the real punchline. Put the C64, the Apollo computer, and a mid-range Android phone in the same comparison table and the modern phone is so absurdly overpowered it's almost unfair. And unlike the C64, you don't have to write assembly to use it: mature frameworks now exist for compiling and serving models on phones and other edge devices, complete with quantization tooling and OpenAI-style APIs. You can run not just a model but an entire agentic stack — model, tools, retrieval — on a device that fits in your hand, with no network connection anywhere in sight.
This is why I've stopped being impressed when a major platform announces, with great fanfare, an assistant that "works offline" on their flagship phones. The enthusiasm baffles me slightly — this capability has been sitting there for anyone willing to build with it. The means exist today, on ordinary devices, for anyone shipping a specialized model.
Whatever your target hardware, the work always reduces to the same three disciplines. <strong>Quantization</strong>: moving weights to a smaller precision format without wrecking quality — and <em>verifying</em> that, by comparing weight distributions between original and quantized models and testing against the KPIs you actually care about, not vibes. <strong>The architecture-versus-weights decision</strong>: whether you need a pretrained model's general knowledge at all, or just its skeleton trained on your own data (more on that heresy in another post). And <strong>genuinely understanding the destination hardware</strong> — because in industrial reality, the accelerator landscape is a zoo of chips that aren't made by the one famous GPU vendor, and squeezing low latency out of them means knowing what's underneath.
Notice something about those three skills: none of them is "prompt engineering," and none of them depends on which framework is fashionable this year. They're grounded in the unglamorous fundamentals — numerical precision, memory hierarchies, the mathematics under the model. The frameworks will churn; PyTorch today, something else tomorrow. The fundamentals are the transferable asset, and I worry that a generation of engineers fluent in high-level libraries is skipping them.
Constraints are a feature
Here's the deeper thing I've absorbed from all this. The scaling era trained us to treat constraints as problems money solves: model too slow, buy GPUs; context too small, rent a bigger endpoint. The Doom tradition — and its language-model descendants — teaches the opposite reflex: a constraint is a design input. The C64 port didn't succeed despite the 38 KB of RAM; the 38 KB is precisely what forced the quantization discipline, the floppy-paging scheme, the fixed-point math. Every one of those techniques transfers directly to hardware people actually deploy on: sensors, controllers, wearables, vehicles, factory equipment.
Meanwhile, real operating environments look a lot more like the C64 than like a lab: a factory floor with strict network segmentation and no route to the internet; emergency workers in places with no coverage; battery budgets where an app that burns half your charge doing something trivial is simply unacceptable. In those settings, "just call the cloud API" isn't an architecture — it's a wish. A small model that lives <em>on</em> the device, tuned to its hardware, working offline at low latency and low power, isn't the budget option there. It's the only option that works.
So my advice, especially to engineers early in their careers: spend some time under the hood. Port something small to something absurd. Learn what int8 actually does to a weight distribution, what your target chip actually supports, where the memory actually goes. The industry's center of gravity is drifting from renting enormous intelligence in the cloud toward owning small, specialized intelligence at the edge — and when it arrives, the engineers who understood the hardware will be the ones everyone else calls.
If we can land on the moon with kilobytes, and run a transformer on a Commodore 64, the ceiling on your commodity hardware is a lot higher than the industry wants you to believe.
