# FlashML-org/FreeToken

> A Mixture-of-Experts inference engine for running frontier-scale open-weight MoE models (290B+) on consumer hardware — GPUs, CPUs, and host memory. Aimed at users who want to run frontier models locally.

- Magnitude: 1.6 out of 10 — Steady
- Stars: 14,812 total · +0 stars measured 2026-10-29, 00:09–10:50 UTC, ≈ 5 by evening
- Star trust: star growth looks organic
- Category: Language models · Language: Python · License: Apache-2.0 · Created: 2026-07-20 · Last push: 2026-10-29
- GitHub: https://github.com/FlashML-org/FreeToken · Homepage: flashml.ai · Page: https://gitnova.dev/en/r/FlashML-org/FreeToken

## Useful for

- Run a large MoE model locally on a gaming PC via the GUI app or CLI.
- Connect a local model to coding agents (Codex, Claude Code) through the OpenAI/Anthropic-compatible API.
- Re-allocate VRAM between expert cache and KV cache without restarting the engine.

## Why it’s here

- Star-counter measurements on 2026-10-29 (UTC), 00:09–10:50: 14812 → 14812 stars (+0). This is the change over that interval.
- Estimated end-of-day forecast: about +5 stars, using observed gains and the previous day.
- Interest is fading: the two-day pace is 44% of the previous week and a half.
- Top new repositories in the last 4 months: #37.

## Star trust

Star growth looks organic.

Star-trust labels are heuristics based on the repository’s behavior, not a check of every stargazer.

## Numbers

- Forks: 1,501
- Issues and pull requests: 839
- Watchers: 97
- Average over the last week: 20 per day
- Usual pace: 41 per day
- Stars in the last hour (measured): 0
- Latest release: nightly (2026-09-09)

## Stars per day, last 30 days (oldest → newest, today is partial)

2026-09-30 … 2026-10-29: 47, 58, 36, 18, 54, 30, 32, 29, 36, 36, 45, 38, 45, 39, 27, 33, 17, 21, 25, 35, 45, 35, 20, 28, 26, 24, 19, 17, 18, 0

## Magnitude by day, last 14 days

10-16 2.4, 10-17 2.0, 10-18 2.1, 10-19 2.0, 10-20 2.5, 10-21 2.4, 10-22 2.3, 10-23 2.0, 10-24 2.2, 10-25 2.1, 10-26 1.8, 10-27 1.8, 10-28 1.9, 10-29 1.8

## Spotted in now

- Top new repositories in the last 4 months: #37

## Similar by description

1. **vllm-project/vllm** — 3.2 · Steady · Language models · Python · +47 stars measured 2026-10-29, 00:05–10:45 UTC, ≈ 100 by evening
   A high-throughput and memory-efficient inference and serving engine for large language models, using PagedAttention for KV-cache management. Supports 200+ model architectures and an OpenAI-compatible API.
   Full card: https://gitnova.dev/en/r/vllm-project/vllm.md
2. **ValerioDolci/capy-flash** — 0.8 · Steady · Language models · C++ · +1 star measured 2026-10-29, 00:11–10:53 UTC, ≈ 2 by evening
   A C++ inference engine that runs the large Qwen3.8-Flash-Next MoE model (125B) in NVFP4 on two 16 GB consumer GPUs: routed experts are offloaded to host RAM and computed on the CPU, the rest runs tensor-parallel. It also serves the dense…
   Full card: https://gitnova.dev/en/r/ValerioDolci/capy-flash.md
3. **mw00/project-maya** — 2.6 · Steady · Language models · Cuda · +17 stars measured 2026-10-29, 00:05–10:47 UTC, ≈ 45 by evening
   Local engine, server and dashboard to run the 321B MoE model GLM-5.3-Flash on one or two NVIDIA GPUs, offloading experts to RAM and NVMe.
   Full card: https://gitnova.dev/en/r/mw00/project-maya.md
4. **ggml-org/llama.cpp** — 3.3 · Steady · Language models · C++ · +48 stars measured 2026-10-29, 00:04–10:45 UTC, ≈ 113 by evening
   LLM and VLM inference implemented in C/C++ with no dependencies, running on CPUs and GPUs across many hardware backends. Enables local model execution with minimal setup and quantization.
   Full card: https://gitnova.dev/en/r/ggml-org/llama.cpp.md

---

Magnitude (0–10) measures how fast and how unusually interest in a repository is growing right now. It is not a quality score.
Days are UTC. “So far today” is a fact; “expected by the end of the day” is a forecast.
Summaries and use cases are written by an LLM (DeepSeek V4.1 Flash) from the README and may be inaccurate: verify specific claims (benchmarks, speed, hardware) in the repository itself.
Data as of 2026-10-29 10:55 UTC, updated every 30 minutes.