# trymirai/uzu

> A high-performance AI model inference engine written in Rust with APIs for Rust, Python, Swift, and TypeScript. It lets you run LLMs locally inside your app with zero latency and no inference costs.

- Magnitude: 2.8 out of 10 — Peaking
- Stars: 2,525 total · +32 stars measured 2026-10-22, 00:01–04:52 UTC, ≈ 97 by evening
- Star trust: star growth looks organic
- Category: Language models · Language: Rust · License: MIT · Created: 2025-06-23 · Last push: 2026-10-22
- GitHub: https://github.com/trymirai/uzu · Homepage: https://trymirai.com · Page: https://gitnova.dev/en/r/trymirai/uzu

## Useful for

- Run a local LLM chat inside a desktop or mobile app
- Embed model inference into a Swift app on Apple devices with unified memory
- Download and run a Qwen model from a Python script without an external API

## Why it’s here

- Star-counter measurements on 2026-10-22 (UTC), 00:01–04:52: 2493 → 2525 stars (+32). This is the change over that interval.
- Estimated end-of-day forecast: about +97 stars, using observed gains and the previous day.
- Over the last two days the pace is 1.5× that of the previous week and a half.
- GitHub Trending Rust today: #9, +15 stars.

## Star trust

Star growth looks organic.

Star-trust labels are heuristics based on the repository’s behavior, not a check of every stargazer.

## Numbers

- Forks: 145
- Issues and pull requests: 997
- Watchers: 17
- Average over the last week: 39 per day
- Usual pace: 36 per day
- Stars in the last hour (measured): 5
- Latest release: 0.6.11 (2026-10-20)

## Stars per day, last 30 days (oldest → newest, today is partial)

2026-09-23 … 2026-10-22: 1, 1, 0, 0, 2, 4, 0, 1, 2, 2, 0, 1, 0, 0, 0, 34, 255, 47, 29, 53, 60, 39, 16, 16, 17, 11, 40, 25, 64, 32

## Magnitude by day, last 2 days

10-21 1.6, 10-22 2.8

## Spotted in now

- GitHub Trending Rust today: #9, +15 stars

## More in this category

1. **deepseek-ai/DeepInfer** — 9.8 · Breakout · Language models · Python · +603 stars measured 2026-10-22, 00:37–04:49 UTC, ≈ 3,064 by evening
   Offline inference framework for running DeepSeek models at large scale: optimized kernels, MoE/Attention parallelism, deterministic sampling and structured outputs. Aimed at teams deploying DeepSeek-V4.1-Flash on NVIDIA SM100 or Huawei…
   Full card: https://gitnova.dev/en/r/deepseek-ai/DeepInfer.md
2. **vllm-project/vllm** — 7.3 · Breakout · Language models · Python · +101 stars measured 2026-10-22, 00:37–04:49 UTC, ≈ 436 by evening
   A high-throughput and memory-efficient inference and serving engine for large language models, using PagedAttention for KV-cache management. Supports 200+ model architectures and an OpenAI-compatible API.
   Full card: https://gitnova.dev/en/r/vllm-project/vllm.md
3. **google-pytorch/torch_tpu** — 6.7 · Early signal · Language models · C++ · +10 stars measured 2026-10-22, 00:37–04:49 UTC, ≈ 128 by evening
   TorchTPU is a PyTorch backend for Google TPUs that lets you run PyTorch models and operations on TPU devices, with custom ATen kernels, an XLA compilation cache, and distributed training support.
   Full card: https://gitnova.dev/en/r/google-pytorch/torch_tpu.md
4. **NVIDIA/flare-local-inference** — 6.5 · Early signal · Language models · Python · +37 stars measured 2026-10-22, 00:37–04:49 UTC, ≈ 273 by evening
   Flare is a local inference server for NVIDIA SM120/SM121 GPUs aimed at long conversations, coding agents and repeated tool use. It compiles models with PyTorch and serves them from a Rust runtime to use workstation memory and compute more…
   Full card: https://gitnova.dev/en/r/NVIDIA/flare-local-inference.md
5. **eagle-hq/EAGLE-4** — 6.5 · Early signal · Language models · Python · +22 stars measured 2026-10-22, 00:37–04:49 UTC, ≈ 222 by evening
   Official inference harness for EAGLE-4 speculative decoding, adding a flexible drafter plugin to SGLang, vLLM and TokenSpeed that adapts to batch size, hardware and workload. It speeds up serving of LLMs such as Qwen3.8-27B while…
   Full card: https://gitnova.dev/en/r/eagle-hq/EAGLE-4.md

---

Magnitude (0–10) measures how fast and how unusually interest in a repository is growing right now. It is not a quality score.
Days are UTC. “So far today” is a fact; “expected by the end of the day” is a forecast.
Summaries and use cases are written by an LLM (DeepSeek V4.1 Flash) from the README and may be inaccurate: verify specific claims (benchmarks, speed, hardware) in the repository itself.
Data as of 2026-10-22 05:00 UTC, updated every 30 minutes.