- DeepSeek open-sourced DeepInfer, the inference software behind its V4.1-Flash model, under the Apache 2.0 license. The README calls it an offline-inference release.
- WebPulse's view: with the engine free, the open question becomes who runs it, on what hardware, with what controls.
- Leaders should ask which AI workloads are billed per request, who would operate a self-hosted stack, and how model files are vetted.
The engine is now public. The garage is not.
A free engine does not give you a racing team. That is the useful way to read DeepSeek's latest release.
DeepSeek has open-sourced DeepInfer, the framework it says powers its large-scale production services, and licensed the code under Apache 2.0. The Decoder reported on October 20 that DeepInfer is the serving software that lets the V4.1-Flash model answer quickly at scale. The Decoder reports that code for future models will also ship as open source.
Inference is the stage where a trained model answers real requests. Releasing the engine means anyone can study how one large provider runs it.
How the engine works
DeepSeek's README lists several techniques. Each targets a different source of waste.
The first is a guess-and-check method for writing answers. The model drafts a few words ahead, and the system verifies them. The README says confidence estimates decide when to schedule that verification. The default maximum draft length is 5 tokens, which are small chunks of text.
The second is scheduling. The README says the batch scheduler runs on the CPU while the GPUs are working, so the chips are not left waiting. That is on by default.
The third is how the work is split. The README lists parallelism for the attention step and for mixture-of-experts layers, where a model routes each piece of text to specialist sub-networks. It also names optimized kernels, the low-level routines that run on the chip, including FlashMLA and DeepGEMM.
The README also describes a trick that cuts work in the prompt-reading stage. The system replays only a bounded suffix of the input, for example 140 tokens in the default V4.1 settings.
What it needs to run
The benchmark setup matters for anyone planning around this. DeepSeek's published numbers come from a split setup: four SM100 GPUs reading prompts and 64 writing answers. Prefill is the model reading your prompt. Decode is the model writing its answer.
That is a large cluster, not a spare server. The software is free. The hardware and the people who run it are not.
The release is also incomplete by its own account. DeepSeek describes it as the offline-inference part of a serving stack. It covers core features, but a live service needs components that are not included. Near that notice the README names prefill and decode disaggregation, described as mostly implemented, and load balancing across replicas.
According to The Decoder, DeepSeek is also releasing deepseek-recipe. It is meant to make a self-hosted deployment handle tokens the way DeepSeek's own API does, so the two behave alike.
Why machine-to-machine work should care
Several features in the README suit software that calls AI, rather than people typing into a chat window.
Structured outputs let a developer constrain a model to a regex, a JSON schema or a grammar. That makes the answer something other software can read reliably.
The README also describes batch-invariant inference and deterministic sampling. With fixed settings and a fixed seed, identical inputs produce identical outputs. Both are listed as NVIDIA-only. In WebPulse's view, repeatable output is the kind of property an auditor is likely to ask about when an AI system touches a decision.
Where the risk moves
This is WebPulse's argument, not a finding from the sources. When the engine is free, the open questions shift to operations. Who patches it? Who watches it? Who decides what model files are allowed in?
Per the README, the software fetches a default model, deepseek-ai/DeepSeek-V4.1-Flash-1017, into the Hugging Face cache the first time it runs. An environment variable can point it at a local copy instead. Either way, someone in your organisation must vet what is pulled onto machines that handle your data.
The Decoder frames the release as pressure on Western providers such as OpenAI and Anthropic. It says the move could also be politically motivated. DeepSeek's stated goal, quoted by the outlet from X, is to "help the community lower compute costs, reduce energy use, and make AI more affordable." Neither the outlet nor the README gives cost figures for self-hosting, so any savings remain unproven for your workloads.
Questions to put to your team
Start with spend. Which AI workloads are billed per request today, and what do they cost each month?
Then ask about capability. Do we have GPU capacity near the scale DeepSeek benchmarked, or would we buy it? Who would be paged when the serving layer fails?
Finally, ask about control. What is our process for approving model files and their licenses? Can we show an auditor that the same input gives the same output?
The engine is now public. What it costs you depends on the team and hardware you put around it.
Produced by the WebPulse Newsroom with AI assistance from the original reporting credited below, and checked against that source by our editorial review. How we use AI.
Original reporting: DeepSeek (deepseek-ai).
