syv-ai/qwen38-27b-rtx3090

Qwen3.8-27B on a single RTX 3090 with vLLM: ~1,000 tok/s at 64 concurrent (int8 tensor-core GEMMs, fp16 DeltaNet state), ~114 tok/s single-user at default sampling / ~124 greedy (MTP drafts, own-output draft vocab, calibrated int4 lm_head, split-KV verify attention), 150k-262k context; patches, requant scripts, benchmarks

Language modelsPython#kv-cache#llm-inference#local-llm#quantization#qwen

1.9 Steady Magnitude out of 10. Star growth looks organic. Data as of September 14, 2026.

Open on Seismograph Open on GitHub

About the project

A ready-made setup for serving Qwen3.8-27B on a single 24 GB consumer RTX 3090 with vLLM: 150k context, OpenAI-compatible API, single-user and batch modes, plus patches, requant scripts and benchmarks.

Useful for

Run a local OpenAI-compatible Qwen3.8-27B server on a single RTX 3090
Pick batch mode for an API backend with dozens of concurrent requests
Enable DFlash2 and prefix cache to speed up answers that quote the prompt

README summarized by DeepSeek V4.1 Flash. Details may be inaccurate.

Why it’s trending

22 stars today.
Interest is fading: the two-day pace is 58% of the previous week and a half.
The repository is 29 days old and already has 1,349 stars.
Top new repositories this month: #48.

Stars per day

Bars are daily stars, the line is the usual pace. Red marks spike days.

Numbers

Total stars: 1,349
Stars in a day: 22
Forks: 190
Issues and pull requests: 108
Watchers: 17
Language: Python
License: Apache-2.0
Created: August 15, 2026
Last push: September 13, 2026

Star trust

Growth looks organic: forks and discussion are in line with active projects, and stars arrive unevenly, the way people give them.

These are heuristics, not a verdict: we judge by the repository’s behavior, not by a list of stargazers.

Spotted in

September 13, 2026Top new repositories this month: #48

README badge

Paste it into your README — the badge shows the current magnitude and links to this page.

[![Seismograph](https://gitnova.dev/badge/syv-ai/qwen38-27b-rtx3090.svg?lang=en)](https://gitnova.dev/en/r/syv-ai/qwen38-27b-rtx3090)

Similar projects

6.1
JustVugg/colibri
A pure-C, zero-dependency inference engine for running large MoE models (744B–2.8T parameters) on consumer hardware by treating VRAM, RAM, and storage as a single multitier hierarchy and streaming experts from disk.

BreakoutLanguage modelsC+1,048 stars in a day
5.7
asgeirtj/system_prompts_leaks
A collection of extracted system prompts from Anthropic, OpenAI, Google, xAI and others — the hidden instructions chatbots receive before a user's first message.

BreakoutLanguage modelsJavaScript+428 stars in a day
5.1
kennethwolters/litelm
A lightweight litellm alternative: routes LLM calls across 19 providers and translates message formats in ~2,900 lines with two dependencies, without proxy, caching, or cost tracking.

Early signalLanguage modelsPython+9 stars in a day
5.0
MiaAI-Lab/DeepSeek-v4.1-Flash-EXL3-2x-DGX-Sparks
A local EXL3 2.9 bpw checkpoint of DeepSeek-V4.1-Flash (196 GiB, 39 shards) served via an OpenAI-compatible vLLM API on a 2x NVIDIA GB10 (DGX Spark) kit with tensor-parallel 2. Includes DSpark speculative decoding and…

Early signalLanguage modelsPython+81 stars in a day
4.8
unclecode/crawl4ai
Open-source web crawler and scraper that turns pages into clean, LLM-ready Markdown for RAG, agents and data pipelines. Runs via Python API, CLI and Docker with no API keys.

BreakoutLanguage modelsPython+411 stars in a day
4.5
Edge0-AI/Edge0
An open-source streaming MoE inference framework: expert weights are offloaded from SSD on demand while a trained prerouter predicts routing ahead of time. Runs on Apple Silicon via MLX and ships with two ready-to-run…

CoolingLanguage modelsPython+118 stars in a day