# ValerioDolci/capy-flash

> A C++ inference engine that runs the large Qwen3.8-Flash-Next MoE model (125B) in NVFP4 on two 16 GB consumer GPUs: routed experts are offloaded to host RAM and computed on the CPU, the rest runs tensor-parallel. It also serves the dense Qwen3.8-27B and exposes an OpenAI/Anthropic-compatible API.

- Magnitude: 1.7 out of 10 — Steady
- Stars: 58 total · +0 stars measured 2026-10-20, 01:05–03:58 UTC, ≈ 6 by evening
- Star trust: star growth looks organic
- Category: Language models · Language: C++ · License: Apache-2.0 · Created: 2026-09-24 · Last push: 2026-10-19
- GitHub: https://github.com/ValerioDolci/capy-flash · Page: https://gitnova.dev/en/r/ValerioDolci/capy-flash

## Useful for

- Serve Qwen3.8-Flash-Next locally behind an OpenAI-compatible API on two RTX 5070 Ti
- Run agentic conversations with prefix cache and idle sessions evicted to RAM
- Serve the dense Qwen3.8-27B on the same two GPUs for concurrent requests

## Why it’s here

- Star-counter measurements on 2026-10-20 (UTC), 01:05–03:58: 58 → 58 stars (+0). This is the change over that interval.
- Estimated end-of-day forecast: about +6 stars, using observed gains and the previous day.
- The repository is 26 days old and already has 58 stars.
- Hacker News: “Show HN: Capy-flash – a 125B MoE model on two 16 GB consumer GPUs” — 8 points, 3 days ago.

## Star trust

Star growth looks organic.

Star-trust labels are heuristics based on the repository’s behavior, not a check of every stargazer.

## Numbers

- Forks: 11
- Issues and pull requests: 11
- Watchers: 1
- Average over the last week: 5 per day
- Usual pace: 1 per day
- Stars in the last hour (measured): 0
- Latest release: v0.6.0 (2026-10-19)

## Stars per day, last 27 days (oldest → newest, today is partial)

2026-09-24 … 2026-10-20: 0, 2, 3, 1, 3, 1, 2, 0, 3, 2, 0, 4, 0, 0, 3, 0, 0, 2, 2, 1, 0, 7, 7, 1, 6, 8, 0

## Hacker News

- Show HN: Capy-flash – a 125B MoE model on two 16 GB consumer GPUs — 8 points, 2 comments: https://news.ycombinator.com/item?id=50118153

## Spotted in now

- Spotted on Hacker News

## Similar by description

1. **Wallawalla47/Infernix** — 0.6 · Steady · Language models · C++ · +0 stars measured 2026-10-20, 00:32–04:00 UTC, ≈ 1 by evening
   C++/CUDA single-GPU inference engine for Qwen models on RTX 50-series cards, with offloaded experts, prefix caching and OpenAI/Anthropic-compatible APIs. Aimed at running large Qwen checkpoints locally, including agentic coding workloads.
   Full card: https://gitnova.dev/en/r/Wallawalla47/Infernix.md
2. **yibie/strata-mlx** — 0.4 · Steady · Language models · Python · +0 stars measured 2026-10-20, 00:32–04:01 UTC, ≈ 1 by evening
   An unofficial MLX inference engine for Apple Silicon that loads Strata's GSQ-RCO GGUF files and speaks its line protocol, so Strata's server and web UI can drive a 125B MoE model on a Mac.
   Full card: https://gitnova.dev/en/r/yibie/strata-mlx.md
3. **Niko1221/Strata** — 4.6 · Steady · Language models · C++ · +150 stars measured 2026-10-20, 00:23–03:53 UTC, ≈ 814 by evening
   A C++ local inference engine that runs the 125B MoE model Qwen3.8-Flash-Next on a regular PC with one NVIDIA GPU (12-24 GB) and 64 GB RAM, exposing an OpenAI/Anthropic-compatible API on localhost.
   Full card: https://gitnova.dev/en/r/Niko1221/Strata.md
4. **FlashML-org/FreeToken** — 2.2 · Steady · Language models · Python · +12 stars measured 2026-10-20, 00:29–03:56 UTC, ≈ 42 by evening
   A Mixture-of-Experts inference engine for running frontier-scale open-weight MoE models (290B+) on consumer hardware — GPUs, CPUs, and host memory. Aimed at users who want to run frontier models locally.
   Full card: https://gitnova.dev/en/r/FlashML-org/FreeToken.md
5. **mw00/project-maya** — 2.7 · Steady · Language models · C++ · +20 stars measured 2026-10-20, 00:27–03:55 UTC, ≈ 80 by evening
   Local engine, server and dashboard to run the 321B MoE model GLM-5.3-Flash on one or two NVIDIA GPUs, offloading experts to RAM and NVMe.
   Full card: https://gitnova.dev/en/r/mw00/project-maya.md

---

Magnitude (0–10) measures how fast and how unusually interest in a repository is growing right now. It is not a quality score.
Days are UTC. “So far today” is a fact; “expected by the end of the day” is a forecast.
Summaries and use cases are written by an LLM (DeepSeek V4.1 Flash) from the README and may be inaccurate: verify specific claims (benchmarks, speed, hardware) in the repository itself.
Data as of 2026-10-20 04:02 UTC, updated every 30 minutes.