Language models: what’s taking off on GitHub
The fastest-growing repositories in “Language models”: plain-language summaries, use cases and a check on star quality. Updated every 30 minutes.
-
6.1
JustVugg/colibri
A pure-C, zero-dependency inference engine for running large MoE models (744B–2.8T parameters) on consumer hardware by treating VRAM, RAM, and storage as a single multitier hierarchy and streaming experts from disk.
-
5.7
asgeirtj/system_prompts_leaks
A collection of extracted system prompts from Anthropic, OpenAI, Google, xAI and others — the hidden instructions chatbots receive before a user's first message.
-
5.1
kennethwolters/litelm
A lightweight litellm alternative: routes LLM calls across 19 providers and translates message formats in ~2,900 lines with two dependencies, without proxy, caching, or cost tracking.
-
5.0
MiaAI-Lab/DeepSeek-v4.1-Flash-EXL3-2x-DGX-Sparks
A local EXL3 2.9 bpw checkpoint of DeepSeek-V4.1-Flash (196 GiB, 39 shards) served via an OpenAI-compatible vLLM API on a 2x NVIDIA GB10 (DGX Spark) kit with tensor-parallel 2. Includes DSpark speculative decoding and…
-
4.8
unclecode/crawl4ai
Open-source web crawler and scraper that turns pages into clean, LLM-ready Markdown for RAG, agents and data pipelines. Runs via Python API, CLI and Docker with no API keys.
-
4.5
Edge0-AI/Edge0
An open-source streaming MoE inference framework: expert weights are offloaded from SSD on demand while a trained prerouter predicts routing ahead of time. Runs on Apple Silicon via MLX and ships with two ready-to-run…
-
4.4
huggingface/transformers
Model-definition framework for state-of-the-art pretrained ML models (text, vision, audio, multimodal) for inference and training, compatible with most training and inference engines.
-
4.2
microsoft/markitdown
A Python utility from Microsoft for converting PDFs, Office documents, images, audio, HTML, and other formats to Markdown, aimed at LLM and text-analysis pipelines. It preserves document structure (headings, lists,…
-
3.8
arikchakma/gpu-time
An experimental neural parser for English time expressions that turns text into dates, time ranges, and RFC 5545 recurrence rules, running locally on CPU or WebGPU without sending input to a server.
-
3.5
FireRedTeam/FireRedAudio
A general-purpose 9B-parameter audio language model with decoupled representations for understanding and generation: ASR, audio QA, zero-shot and instruct TTS, speech editing, and hour-long recordings.
-
3.5
ggml-org/llama.cpp
LLM and VLM inference implemented in C/C++ with no dependencies, running on CPUs and GPUs across many hardware backends. Enables local model execution with minimal setup and quantization.
-
3.4
FareedKhan-dev/kimi-k3-in-c
A portable C99 inference engine that runs the 2.78-trillion-parameter Kimi K3 model on a single CPU within 8.24 GB of RAM, with no BLAS, framework, or GPU. The model streams from disk, and memory only affects speed.
-
3.4
AlexsJones/llmfit
A Rust tool that inspects your CPU, RAM, GPU and VRAM and recommends open-source LLMs that will actually run on your hardware, scoring fit, speed, quality and context. Ships with a TUI, CLI, web dashboard and REST API.
-
3.2
deeplethe/utopia
Utopia is an open-source enterprise knowledge platform: a bitemporal knowledge graph, RAG search and agent workflows on a single Rust binary plus PostgreSQL. It deploys offline, for companies that need a controlled…
-
3.1
MiaAI-Lab/DeepSeek-v4.1-Flash-DGX-Sparks
Scripts and patches to serve DeepSeek-V4.1-Flash with SGLang across a 3–4 node NVIDIA DGX Spark cluster, using MXFP4/FP8, speculative decoding and an OpenAI-compatible endpoint.
-
3.1
NVIDIA-NeMo/Switchyard
An LLM request router that picks the cheapest model capable of handling each call while preserving OpenAI and Anthropic API compatibility. Runs as a NeMo Relay or LiteLLM plugin, an embeddable library, or a standalone…
-
3.0
xinxuxin/keystone-bench
A clinical chat benchmark: it tests whether an assistant's action changes when one decisive fact in a conversation is removed, replaced, or contradicted, and whether the physician rubric stays applicable.
-
2.9
RyanCodrai/turbovec
A Rust vector index with Python bindings built on the TurboQuant algorithm for compressed embeddings and fast nearest-neighbor search without training.
-
2.9
FlashML-org/FreeToken
A Mixture-of-Experts inference engine for running frontier-scale open-weight MoE models (290B+) on consumer hardware — GPUs, CPUs, and host memory. Aimed at users who want to run frontier models locally.
-
2.7
maximpri/mlxtop
A terminal monitor for local LLMs on Mac: shows running models, memory usage, GPU activity and generation speed. Works with oMLX, MLX-LM, Ollama, llama.cpp and other runtimes.
-
2.6
deepseek-ai/deepseek-recipe
A collection of Rust libraries and Python bindings that convert API requests in different formats (Messages, Chat Completions, Responses) into the Conversation format, encode them into prompts for DeepSeek V4/V4.1…
-
2.6
firecrawl/anydoc
Rust library that converts office documents (Word, PowerPoint, Excel, OpenDocument, RTF, EPUB, CSV, PDF) into clean Markdown, with Node.js, Python and browser bindings. Built to make documents LLM-ready.
-
2.5
baidu/Unlimited-OCR
Baidu's OCR model for document parsing: extracts text from single images, multi-page files and PDFs in one pass, supporting long context up to 32768 tokens.
-
2.5
gauravapiscean/agentic-kv-cache
A prefix-cache simulator for LLM serving that replays real Claude Code and Mooncake traces to test whether published policies beat LRU. The author finds LRU is harder to beat than the papers claim.
-
2.4
jundot/omlx
Local LLM inference server for Apple Silicon with continuous batching and tiered KV caching (RAM + SSD), managed from the macOS menu bar. OpenAI API compatible, suited for running local models in tools like Claude Code.
-
2.4
0xBakeer/deepseek-v41-flash-spark
A recipe for serving DeepSeek-V4.1-Flash (510 GB, 15,360 FP4 experts) on a single NVIDIA DGX Spark: a hot set of experts stays resident while the rest stream from NVMe via O_DIRECT, plus DSpark speculative decoding and…
-
2.3
deepseek-ai/DeepSelect
A high-performance CUDA implementation of the TopK kernel used in DeepSeek Sparse Attention (DSA) and the sampler, speeding up torch.topk by 2-20x. Targeted at developers running DeepSeek V3.2/V4 model inference on GPUs.
-
2.3
Tencent/WeMM-Embedding
A family of universal multimodal embedding models from Tencent's WeChat Vision team (2B, 4B, 9B) producing unified vectors for text, images, videos, visual documents, and interleaved multimodal inputs. Supports…
-
2.2
youssofal/MTPLX
Run local LLMs on Apple Silicon with native multi-token prediction (MTP) speculative decoding and no external drafter; speeds up inference 1.6–2.2x while keeping the same output distribution.
-
1.9
syv-ai/qwen38-27b-rtx3090
A ready-made setup for serving Qwen3.8-27B on a single 24 GB consumer RTX 3090 with vLLM: 150k context, OpenAI-compatible API, single-user and batch modes, plus patches, requant scripts and benchmarks.
-
1.7
modular/modular
Open-source components of the Modular Platform for AI development and deployment: the Mojo language compiler and standard library, plus the MAX framework with an inference server and model pipelines.
-
1.5
Albert-Weasker/niubigeo
Open-source tool for tracking brand visibility and competitors in AI answers: given a domain, it shows how models describe your product, whom they recommend, and which sources they cite.
-
1.4
MoonshotAI/Kimi-K3
Kimi K3 is an open-weight native multimodal agentic MoE language model with 2.8T parameters and a 1M-token context window, understanding text, images and video. It targets long-horizon coding, agentic knowledge work…
-
1.4
RH-RunningHub/MiniMax-H3-MultiGPU-Lightning
Multi-GPU inference acceleration recipe for MiniMax H3 video generation using step distillation, SageAttention2, Cache-DiT and torch.compile on SGLang, up to ~12x on 8x RTX 6000D.
-
1.2
vllm-project/vllm-ascend
A hardware plugin for vLLM that runs LLM inference on Ascend NPUs (Atlas A2/A3). It enables deployment of Transformer, MoE, embedding and multimodal models on Huawei Ascend hardware.
-
1.2
deepseek-ai/DeepSpec
Full-stack pipeline for training and evaluating draft models for speculative decoding: data preparation, training against a target-model cache, and acceptance measurement on benchmarks. Supports DSpark, DFlash and…
-
1.1
drumih/turbo-fieldfare
A custom Swift + Metal runtime for running Gemma 4 26B-A4B on Apple Silicon Macs: keeps ~2 GB in memory and streams experts from SSD, so it works even on 8 GB RAM machines.
-
1.1
StarTrail-org/PixelRAG
PixelRAG is a visual RAG system that renders web pages, PDFs and images into screenshot tiles and retrieves over the images directly, preserving tables, charts and layout lost when parsing to text.
-
1.0
tomnio/rubric
A library for schema-first structured extraction from LLMs: it puts a Zod schema on the request, parses and validates JSON, and reasks the model on failure until it conforms or retries run out.
-
1.0
argonautlabsai/deltafin
A Rust fork of the Deltafin engine that runs the full 2.8T-parameter Kimi K3 MoE on Apple Silicon by streaming expert weights from four SSDs, with a benchmark package and ARGODRIVE tooling.
-
0.9
nayakbhupen/Spnda
A library for epistemic uncertainty quantification and LLM hallucination detection via Exact-Match Normalized Entropy (R_sc), requiring no NLI models or GPU and running ~90,000x faster than Semantic Entropy.
-
0.7
pathwaycom/arc-task-gen
Generates ARC-AGI-1-style tasks distribution-matched to the public eval set. Used to evaluate models on fresh problems they are unlikely to have encountered before.
-
0.5
donvito/local-evals
A local-first app for evaluating LLMs on document→JSON, text→JSON, and tool-call tasks via OpenAI-compatible APIs or OpenRouter, showing inputs, outputs, scores, and logs side by side.
-
0.3
Staatsgeheim/MathKernel
Evidence-aware multi-engine mathematics kernel for LLMs (SymPy, Z3, Lean, mpmath) that computes exactly, symbolically, formally and in certified intervals while tracking trust levels and provenance. Usable as a Python…
-
0.3
ngocdaobao/A-Survey-on-Looped-Transformers
A survey of research on looped transformers — recurrent-depth architectures for language models. It organizes approaches by loop topology, loop-count policy, weight sharing and memory strategy.
-
0.0
yureii1996/cek-probe-model
A set of Python scripts for testing OpenAI-compatible model endpoints: it fetches the model list, runs simple probes, and prints a summary, including an architecture mismatch warning.
-
0.0
SpinnerAppreciate/glm-flash-offline-client
Standalone Windows client for running the GLM-5.3-Flash model locally: chat works without cloud and without sending data outside. The repository is the project's documentation hub; the client itself ships as a single…
-
0.0
Binaire-0101/FRZi-inference
FRZi-inference is an inference engine for running machine learning models. It is designed to execute pre-trained models and obtain predictions.