DeepSeek Open-Sources DeepInfer Serving Engine Claiming 155T Daily Tokens

DeepSeek open-sourced DeepInfer, the production inference framework powering its DeepSeek-V4.1-Flash model, under an Apache-2.0 license. According to the Chinese AI company, the stack serves up to 155T tokens a day from a single cluster and earns about $38 per GPU per hour at off-peak prices, though those figures are self-reported and revenue per GPU-hour does not measure net profit or full unit economics. Built as a customization of the open-source vLLM project with credit to SGLang and TRT-LLM, DeepInfer supports distributed inference and speculative decoding. It also enables prompt caching for up to 72 hours, outlasting the 12-hour window OpenAI recently announced. Timed to the opening day of the PyTorch Conference, the release exposes production serving infrastructure in a space where prior advances largely came from closed-source startups like Fireworks, Baseten, and Modal, following a similar push when Google, Red Hat, and IBM shipped the llm-d 1.0 serving layer. The open-source codebase is not a complete turnkey replica of DeepSeek's internal service and currently targets only one model. The company withheld its optimized KV-cache management, disaggregation, and load balancing systems.