# NVIDIA/local-inference

> Flare is a local inference server for NVIDIA SM120/SM121 GPUs aimed at long conversations, coding agents and repeated tool use. It compiles models with PyTorch and serves them from a Rust runtime to use workstation memory and compute more efficiently.

- Magnitude: 2.6 out of 10 — Cooling
- Stars: 683 total · +14 stars measured 2026-10-27, 00:39–18:54 UTC, ≈ 18 by evening
- Star trust: star growth looks organic
- Category: Language models · Language: Python · License: Apache-2.0 · Created: 2026-10-20 · Last push: 2026-10-27
- GitHub: https://github.com/NVIDIA/local-inference · Page: https://gitnova.dev/en/r/NVIDIA/local-inference

## Useful for

- Serve a local LLM on an NVIDIA SM120/SM121 workstation for long coding sessions
- Run repeated tool calls against a locally hosted model without a Python runtime
- Prepare and serve a supported model package with the Flare CLI

## Why it’s here

- Star-counter measurements on 2026-10-27 (UTC), 00:39–18:54: 669 → 683 stars (+14). This is the change over that interval.
- Estimated end-of-day forecast: about +17 stars, using observed gains and the previous day.
- Interest is fading: the two-day pace is 12% of the previous 5 days.
- The repository is 7 days old and already has 683 stars. With less than two weeks of history, there's no usual pace to compare the spike against yet.
- Hacker News: “Nvidia Flare Local Inference” — 5 points, 6 days ago.
- Recent forks include notable developers: @gmh5225 (1,299 followers), @msuiche (439 followers).

## Star trust

Star growth looks organic.

Star-trust labels are heuristics based on the repository’s behavior, not a check of every stargazer.

## Numbers

- Forks: 73
- Issues and pull requests: 114
- Watchers: 3
- Average over the last week: 69 per day
- Usual pace: too little history (under two weeks)
- Stars in the last hour (measured): 0

## Stars per day, last 8 days (oldest → newest, today is partial)

2026-10-20 … 2026-10-27: 213, 343, 58, 11, 21, 18, 14, 14

## Magnitude by day, last 7 days

10-21 5.8, 10-22 6.6, 10-23 4.2, 10-24 3.0, 10-25 2.9, 10-26 2.7, 10-27 2.8

## Hacker News

- Nvidia Flare Local Inference — 5 points, 0 comments: https://news.ycombinator.com/item?id=50176267

## Similar by description

1. **vllm-project/vllm** — 3.7 · Steady · Language models · Python · +75 stars measured 2026-10-27, 00:36–18:51 UTC, ≈ 95 by evening
   A high-throughput and memory-efficient inference and serving engine for large language models, using PagedAttention for KV-cache management. Supports 200+ model architectures and an OpenAI-compatible API.
   Full card: https://gitnova.dev/en/r/vllm-project/vllm.md

---

Magnitude (0–10) measures how fast and how unusually interest in a repository is growing right now. It is not a quality score.
Days are UTC. “So far today” is a fact; “expected by the end of the day” is a forecast.
Summaries and use cases are written by an LLM (DeepSeek V4.1 Flash) from the README and may be inaccurate: verify specific claims (benchmarks, speed, hardware) in the repository itself.
Data as of 2026-10-27 19:04 UTC, updated every 30 minutes.