# deepseek-ai/DeepInfer

> Offline inference framework for running DeepSeek models at large scale: optimized kernels, MoE/Attention parallelism, deterministic sampling and structured outputs. Aimed at teams deploying DeepSeek-V4.1-Flash on NVIDIA SM100 or Huawei Ascend 950.

- Magnitude: 9.3 out of 10 — Breakout
- Stars: 6,533 total · +4,651 stars measured 2026-10-20, 09:29–12:24 UTC, ≈ 6,854 by evening
- Star trust: star growth looks organic
- Category: Language models · Language: Python · License: Apache-2.0 · Created: 2026-10-16 · Last push: 2026-10-20
- GitHub: https://github.com/deepseek-ai/DeepInfer · Page: https://gitnova.dev/en/r/deepseek-ai/DeepInfer

## Useful for

- Run offline inference of DeepSeek-V4.1-Flash across a multi-GPU cluster
- Constrain generation to a JSON schema or grammar via structured outputs
- Get reproducible results across batch sizes with batch-invariant mode

## Why it’s here

- Star-counter measurements on 2026-10-20 (UTC), 09:29–12:24: 1882 → 6533 stars (+4651). This is the change over that interval.
- Estimated end-of-day forecast: about +6,854 stars, using observed gains and the previous day.
- The repository is 4 days old and already has 6,533 stars. With less than two weeks of history, there's no usual pace to compare the spike against yet.
- Top new repositories this week: #2.
- Top new repositories this month: #23.
- Hacker News: “DeepSeek just open-sourced its inference engine” — 684 points, 3 h ago.
- About 280 forks a day — people are taking the code.

## Star trust

Star growth looks organic.
- Hardly anyone watches for updates: 5 watchers for 6,533 stars.

Star-trust labels are heuristics based on the repository’s behavior, not a check of every stargazer.

## Numbers

- Forks: 526
- Issues and pull requests: 38
- Watchers: 5
- Average over the last week: 1,371 per day
- Usual pace: too little history (under two weeks)
- Stars in the last hour (measured): 1,288

## Stars per day, last 5 days (oldest → newest, today is partial)

2026-10-16 … 2026-10-20: 0, 0, 0, 3, 4651

## Hacker News

- DeepSeek just open-sourced its inference engine — 684 points, 238 comments: https://news.ycombinator.com/item?id=50163277

## Spotted in now

- Top new repositories this week: #2
- Top new repositories this month: #23
- Top new repositories in the last 4 months: #97
- Spotted on Hacker News

## Similar by description

1. **Wallawalla47/Infernix** — 0.8 · Steady · Language models · C++ · +1 star measured 2026-10-20, 00:32–12:35 UTC, ≈ 2 by evening
   C++/CUDA single-GPU inference engine for Qwen models on RTX 50-series cards, with offloaded experts, prefix caching and OpenAI/Anthropic-compatible APIs. Aimed at running large Qwen checkpoints locally, including agentic coding workloads.
   Full card: https://gitnova.dev/en/r/Wallawalla47/Infernix.md
2. **sgl-project/sglang** — 3.1 · Breakout · Language models · Python · +36 stars measured 2026-10-20, 00:25–12:28 UTC, ≈ 90 by evening · unusual star pattern (heuristic)
   SGLang is an open-source inference framework for fast serving of large language and multimodal models, optimized for agentic workloads, RL rollouts, and large-scale deployment.
   Full card: https://gitnova.dev/en/r/sgl-project/sglang.md

---

Magnitude (0–10) measures how fast and how unusually interest in a repository is growing right now. It is not a quality score.
Days are UTC. “So far today” is a fact; “expected by the end of the day” is a forecast.
Summaries and use cases are written by an LLM (DeepSeek V4.1 Flash) from the README and may be inaccurate: verify specific claims (benchmarks, speed, hardware) in the repository itself.
Data as of 2026-10-20 12:38 UTC, updated every 30 minutes.