Language models: what’s taking off on GitHub
The fastest-growing repositories in “Language models”: plain-language summaries, use cases and a check on star quality. Updated every 30 minutes.
-
8.1
togg53192-cmd/jailbreaks
A collection of 31 system prompts, persona locks, and AGENTS.md files for 25+ models (Claude, Gemini, Grok, GLM, DeepSeek, Kimi, Qwen, Gemma, GPT-OSS). Intended for research on how instruction hierarchies and policy…
-
7.4
Edge0-AI/Edge0
An open-source streaming MoE inference framework: expert weights are offloaded from SSD on demand while a trained prerouter predicts routing ahead of time. Runs on Apple Silicon via MLX and ships with two ready-to-run…
-
6.8
OpenBMB/MiniCPM-OCR
MiniCPM-OCR is an open-source OCR model family for parsing Chinese and English documents and PDFs into Markdown or JSON, with a 2B model (Jev) that scores recognition quality per region.
-
5.9
thomasunise/casecite
Self-hosted legal research platform: upload your firm's documents, ask questions with citations, review contracts, draft documents and research judges. Runs on your own infrastructure with your own API keys.
-
5.4
NVIDIA/auto-ontology
NVIDIA's Python framework that builds a semantic ontology graph over enterprise metadata, giving LLM agents trusted metric and entity definitions for reliable text-to-SQL.
-
5.4
lyramakesmusic/neuron-explorer
A steering-first neuron explorer for Qwen3.5-4B: it pushes every MLP neuron up and down on prompts, keeps all rollouts, and lays the results out over a 3D view of the model for interpretability research.
-
5.2
google-deepmind/detectorscope
A toolkit for building AI-text detectors whose decisions are mechanistically grounded and auditable: each prediction is a sum of named features localized in the text. Built on sparse autoencoders and a concept…
-
4.9
SoMarkAI/OXR
OXR is an open-source document parsing model that turns PDFs and document images into Markdown or JSON with layout coordinates, recognizing 13 element types such as tables, formulas, chemical structures and code. With…
-
4.8
baidu/Unlimited-OCR
Baidu's OCR model for document parsing: extracts text from single images, multi-page files and PDFs in one pass, supporting long context up to 32768 tokens.
-
4.6
Niko1221/Strata
A C++ local inference engine that runs the 125B MoE model Qwen3.8-Flash-Next on a regular PC with one NVIDIA GPU (12-24 GB) and 64 GB RAM, exposing an OpenAI/Anthropic-compatible API on localhost.
-
4.5
Tencent/WeVisOne-VL
WeVisOne-VL is a vision-language model by the WeChat Vision Team at Tencent that unifies object-level and pixel-level perception: detection, referring localization, counting, pointing, OCR, segmentation, and alpha…
-
4.3
JustVugg/colibri
A pure-C, zero-dependency inference engine for running large MoE models (744B–2.8T parameters) on consumer hardware by treating VRAM, RAM, and storage as a single multitier hierarchy and streaming experts from disk.
-
4.0
deepseek-ai/DeepInfer
Offline inference framework for running DeepSeek models at large scale: optimized kernels, MoE/Attention parallelism, deterministic sampling and structured outputs. Aimed at teams deploying DeepSeek-V4.1-Flash on…
-
3.6
QuantumNous/new-api
Self-hosted AI gateway that aggregates multiple LLM providers behind a unified OpenAI-, Claude-, or Gemini-compatible API with routing, access, and cost management.
-
3.6
BerriAI/litellm
Open-source AI gateway and Python SDK that calls 100+ LLM providers (OpenAI, Anthropic, Bedrock, Azure, VertexAI, etc.) through a single OpenAI-compatible interface with cost tracking, guardrails, load balancing, and…
-
3.5
p-e-w/heretic
A tool for fully automatic removal of censorship (safety alignment) from transformer-based language models using directional ablation and Optuna-based parameter optimization.
-
3.4
unslothai/unsloth
Desktop app and web UI for locally running, fine-tuning and serving LLMs, diffusion, embedding and audio models, with OpenAI-compatible API and integrations for Claude Code, Codex and MCP agents.
-
3.3
ollama/ollama
Run open language models (Kimi, GLM, DeepSeek, Qwen, Gemma and more) locally via CLI and REST API with integrations into editors and assistants.
-
3.3
solidsf-inc/solidcv
A pure-Rust OpenCV replacement with zero FFI: classical 2D/3D vision, stereo reconstruction, and mechanistic interpretability for LLMs/VLMs with hardware kernels for Apple Silicon and NVIDIA.
-
3.3
ggml-org/llama.cpp
LLM and VLM inference implemented in C/C++ with no dependencies, running on CPUs and GPUs across many hardware backends. Enables local model execution with minimal setup and quantization.
-
3.3
trymirai/uzu
A high-performance AI model inference engine written in Rust with APIs for Rust, Python, Swift, and TypeScript. It lets you run LLMs locally inside your app with zero latency and no inference costs.
-
3.2
vllm-project/vllm
A high-throughput and memory-efficient inference and serving engine for large language models, using PagedAttention for KV-cache management. Supports 200+ model architectures and an OpenAI-compatible API.
-
3.2
Mesh-LLM/mesh-llm
Decentralized LLM serving system that pools GPUs and memory across machines and exposes them as one OpenAI-compatible API at localhost:9337. A model can run locally, route to a peer, or be split into layer stages…
-
3.1
magnitudedev/magnitude
Open source inference engine for consumer hardware that profiles your machine, recommends the best local models, then downloads, tunes, and runs them, with one-click connection to AI agents.
-
3.1
infiniflow/ragflow
Open-source RAG engine that fuses document context extraction with agent capabilities to build production LLM systems. Supports parsing PDF, DOCX, spreadsheets, images and syncing from Confluence, Notion, S3.
-
3.0
headroomlabs-ai/headroom
A local library, proxy and MCP server that compress tool outputs, logs, files and RAG chunks before they reach the LLM, cutting token usage without changing answers.
-
2.9
slee-persis/GVS5H
A training-free method that runs five instances of one LLM in parallel, coordinating through a shared filesystem to solve hard coding problems, evaluated on LiveCodeBench.
-
2.8
Tencent-Hunyuan/Hy-SimulMT
Tencent Hunyuan's simultaneous speech translation model for real-time bidirectional Chinese-English translation with voice cloning. The repository provides the SimulInfer inference service and SimulTrain training tools.
-
2.6
langfuse/langfuse
Open source LLM engineering platform for tracing, evaluating, and debugging AI applications, with self-hosting and prompt management.
-
2.6
xai-org/x-algorithm
Open-source code of X's For You feed algorithm: it blends in-network posts with ML-retrieved out-of-network content, filters them, and ranks posts with a transformer model. Published for transparency and study of how…
-
2.6
mw00/project-maya
Local engine, server and dashboard to run the 321B MoE model GLM-5.3-Flash on one or two NVIDIA GPUs, offloading experts to RAM and NVMe.
-
2.6
jaredpalmer/kev
kev is a LoRA adapter with a small readout head on top of Qwen2.5-0.5B that answers many typed questions about a document in a single forward pass, returning calibrated probabilities instead of text.
-
2.5
promptfoo/promptfoo
CLI and library for evaluating and red-teaming LLM apps: automated prompt evals, model comparison, and vulnerability scanning. Runs locally and integrates with CI/CD.
-
2.5
tensorflow/tensorflow
An open source machine learning platform from Google Brain with Python and C++ APIs for training and deploying neural networks. Used by researchers and developers of ML applications.
-
2.3
firecrawl/anydoc
Rust library that converts office documents (Word, PowerPoint, Excel, OpenDocument, RTF, EPUB, CSV, PDF) into clean Markdown, with Node.js, Python and browser bindings. Built to make documents LLM-ready.
-
2.3
Albert-Weasker/niubigeo
Open-source tool for tracking brand visibility and competitors in AI answers: given a domain, it shows how models describe your product, whom they recommend, and which sources they cite.
-
2.3
deeplethe/utopia
Utopia is an open-source enterprise knowledge platform: a bitemporal knowledge graph, RAG search and agent workflows on a single Rust binary plus PostgreSQL. It deploys offline, for companies that need a controlled…
-
2.3
kvcache-ai/Mooncake
LLM serving platform built on a KVCache-centric disaggregated architecture: separates prefill and decode and transfers KV cache between nodes over RDMA. Powers Kimi in production at Moonshot AI.
-
2.1
MemTensor/MemT-1.0
A family of memory foundation models with native long-term memory: dynamic local memory and trainable hyper memory retain history without accessing the original context, cutting storage roughly 66x versus full-context…
-
2.1
mudler/LocalAI
LocalAI is an open-source AI engine that runs LLMs, vision, voice, image and video models on any hardware, including CPU-only, behind an OpenAI-compatible API.
-
2.0
tesseract-ocr/tesseract
Open-source OCR engine libtesseract and CLI tool tesseract for recognizing text in images. Supports 100+ languages, PNG/JPEG/TIFF input, and output in text, hOCR, PDF, TSV, ALTO.
-
2.0
mistralai/timestral
A family of pretrained foundation models for multivariate probabilistic time series forecasting with covariate support. Enables zero-shot forecasting on CPU or GPU without training.
-
2.0
vllm-project/semantic-router
A programmable routing layer for Mixture-of-Models systems across heterogeneous LLM infrastructure: it selects or composes the right model path per request based on request signals, user preferences, and application…
-
1.9
NVIDIA/cutlass
CUTLASS is a collection of CUDA C++ templates and Python DSLs (CuTe DSL) for implementing high-performance linear algebra, primarily GEMM, on NVIDIA GPUs. It targets researchers and performance engineers writing…
-
1.9
VectifyAI/PageIndex
A vectorless RAG engine that builds a hierarchical tree index of a document and retrieves by LLM reasoning over that tree, the way a human navigates a report. Aimed at long professional PDFs such as financial, legal…
-
1.8
google-research/kauldron
A modular Google Research library for training ML models on JAX, focused on research velocity and flexible configuration.
-
1.8
FareedKhan-dev/kimi-k3-in-c
A portable C99 inference engine that runs the 2.78-trillion-parameter Kimi K3 model on a single CPU within 8.24 GB of RAM, with no BLAS, framework, or GPU. The model streams from disk, and memory only affects speed.
-
1.8
microsoft/BitNet
Official inference framework for 1-bit LLMs (BitNet b1.58) with optimized kernels for CPU and GPU. Runs ternary models fast and losslessly, including a 100B model on a single CPU.
-
1.8
cocoindex-io/cocoindex
Rust-based incremental engine that turns codebases, PDFs, Slack and notes into continuously fresh context for AI agents and RAG apps, recomputing only the delta.
-
1.7
google-pytorch/torch_tpu
TorchTPU is a PyTorch backend for Google TPUs that lets you run PyTorch models and operations on TPU devices, with custom ATen kernels, an XLA compilation cache, and distributed training support.
-
1.7
chenglou/handwired
Turns regular JavaScript code into a differentiable neural net: if-else, loops and state are compiled into matrix multiplications and ReLU with no training. Runs on JS or WebGPU.
-
1.7
mostlygeek/llama-swap
A Go proxy server that runs multiple local models and hot-swaps between them on demand, compatible with OpenAI and Anthropic APIs. Works with llama.cpp, vllm, stable-diffusion.cpp and other backends.
-
1.6
FlashML-org/FreeToken
A Mixture-of-Experts inference engine for running frontier-scale open-weight MoE models (290B+) on consumer hardware — GPUs, CPUs, and host memory. Aimed at users who want to run frontier models locally.
-
1.6
cobanov/PS5LM
A llama.cpp port for jailbroken PlayStation 5 that runs GGUF models natively on the console via a payload, with a browser-based model library and chat. Only small models up to 1.6 GB work so far.
-
1.5
NVIDIA/spark-local-inference
Flare is a local inference server for NVIDIA SM120/SM121 GPUs aimed at long conversations, coding agents and repeated tool use. It compiles models with PyTorch and serves them from a Rust runtime to use workstation…
-
1.5
microsoft/onnxruntime
Cross-platform accelerator for ML inference and training. Runs models from PyTorch, TensorFlow, scikit-learn and others via the ONNX format with graph optimizations and hardware acceleration.
-
1.5
MiaAI-Lab/DeepSeek-v4.1-Flash-1017-2x-DGX-Sparks
Scripts to serve the quantized DeepSeek-V4.1-Flash-1017 (EXL3, 2.9 bpw) across two NVIDIA DGX Sparks via the TensorFold engine, exposing an OpenAI-compatible API with up to a 1M-token window and DSpark speculative…
-
1.4
deepseek-ai/FlashMLA
DeepSeek's library of optimized attention kernels (sparse and dense) for running DeepSeek-V4.1 inference on NVIDIA GPUs and Huawei Ascend NPUs, with FP8/FP4 KV cache support.
-
1.4
nace-ai/kapitsa
Apple Silicon optimized inference engine for Qwen and Nemotron models with OpenAI- and Anthropic-compatible APIs. Runs local LLMs on Mac via CLI and a web chat.
-
1.4
MoonshotAI/Kimi-K3
Kimi K3 is an open-weight native multimodal agentic MoE language model with 2.8T parameters and a 1M-token context window, understanding text, images and video. It targets long-horizon coding, agentic knowledge work…
-
1.3
martina9516/uncensored-ai-models
A single-key API gateway to Claude, GPT, Gemini, Grok, DeepSeek, GLM and Qwen, including uncensored and abliterated variants. OpenAI- and Anthropic-compatible, pay per token.
-
1.3
ml-explore/mlx
MLX is an array and machine learning framework from Apple, optimized for Apple silicon with unified memory and lazy computation. Its Python, C++, C, and Swift APIs mirror NumPy and PyTorch, simplifying model training…
-
1.3
drumih/turbo-fieldfare
A custom Swift + Metal runtime for running Gemma 4 26B-A4B on Apple Silicon Macs: keeps ~2 GB in memory and streams experts from SSD, so it works even on 8 GB RAM machines.
-
1.3
eagle-hq/EAGLE-4
Official inference harness for EAGLE-4 speculative decoding, adding a flexible drafter plugin to SGLang, vLLM and TokenSpeed that adapts to batch size, hardware and workload. It speeds up serving of LLMs such as…
-
1.2
lemonade-sdk/lemonade
A local AI server that runs optimized LLMs, whisper, TTS, and image generation on the user's GPU and NPU, exposing OpenAI-, Anthropic-, and Ollama-compatible APIs. Available as a service or an embeddable binary for apps.
-
1.2
recursive-oss/rec
A local inference engine for Recursive's language models, optimized for Apple Silicon and faster than llama.cpp. It runs the model on your machine and exposes an OpenAI-compatible API for clients.
-
1.1
openvinotoolkit/openvino
OpenVINO is an open-source toolkit for optimizing and deploying deep learning model inference on Intel CPU, GPU and NPU. It supports models from PyTorch, TensorFlow, ONNX, Keras, PaddlePaddle, JAX/Flax and Hugging Face…
-
1.1
deepseek-ai/DeepSpec
Full-stack pipeline for training and evaluating draft models for speculative decoding: data preparation, training against a target-model cache, and acceptance measurement on benchmarks. Supports DSpark, DFlash and…
-
1.1
astroautomata/PySR
PySR is a symbolic regression library for Python and Julia that searches for interpretable analytic expressions fitting a dataset, instead of a black-box neural network.
-
1.1
byenzyme/enzyme
Local-first indexing tool that turns Markdown notes into a searchable knowledge base for coding agents, ranking passages by wording and meaning on your own machine.
-
1.0
mizorewww/laya-mlx
Native MLX runtime for Laya typed decision models: returns probabilities for choices, scores or truth values without text generation, running locally on Apple Silicon.
-
0.9
h563k/chat-keeper
A third-party mirror of the Claude web interface plus API access to Claude, Gemini and OpenAI, reachable directly from China without a VPN. The repository mainly serves as a landing page and entry point for the paid…
-
0.9
BaruqueRodrigues/llm-legislativo
Large-scale Brazilian legislative corpus and a causal Transformer language model trained from scratch in pure PyTorch without pretrained weights or closed APIs.
-
0.8
ValerioDolci/capy-flash
A C++ inference engine that runs the large Qwen3.8-Flash-Next MoE model (125B) in NVFP4 on two 16 GB consumer GPUs: routed experts are offloaded to host RAM and computed on the CPU, the rest runs tensor-parallel. It…
-
0.8
apple-aiml-research/ml-rsi-inference-bench
A benchmark for evaluating coding agents that optimize and repair the LLM inference stack in real engines like vLLM and SGLang on real GPUs. Agents work on the live engine and are graded by running what they built.
-
0.6
deepopen-com/deepopen
Open-source non-autoregressive System 1 decision engine for typed decisions: returns typed answers (choice, score, noul) in a single forward pass across 100+ languages without generating text. Suited for…
-
0.6
justxor/deepseek-local-guide-ru
Russian-language guide to running DeepSeek models (R1, V4) locally on PC, Mac, and server: Ollama, llama.cpp, LM Studio, vLLM, Open WebUI, API, RAG, agents, and fine-tuning.
-
0.6
KohakuBlueleaf/KohakuMossTrainer
A finetuning framework for MOSS-Image: every LyCORIS algorithm and full finetune on text-image data at native resolutions, plus inference, on Windows and Linux from 6 GB cards to multi-GPU nodes.
-
0.5
Taichu-AI/ZDTaichu5.0-9B
ZDTaichu5.0-9B is a multimodal foundation model built on Qwen3.5-9B and C-RADIOv4-H for image, video, spatial and embodied reasoning, and agentic tasks.
-
0.4
strot-ai/STROT-Compiler
STROT compiles a natural-language decision intent into a frozen local Python artifact that classifies, scores, or routes records without calling an LLM at runtime. It targets teams wanting cheap, deterministic,…
-
0.4
heterodoxin/AbliteratedOutTheBox
A llama.cpp fork that performs abliteration at inference time: it reads refusal directions from the prompt itself and removes them inside the graph without touching the weights on disk. Runs any supported model with no…
-
0.2
NineAbyss/OR-License
A 31-task benchmark evaluating LLM agents as operations research practitioners: the agent requests information, writes an optimization program in an isolated environment, and submits solution files.
-
0.2
prabanta-dev/janas
Janas is a family of AI tools in C with no dependencies. Janas-LLM runs large MoE models (up to 80B, 48 GB) on a 32 GB laptop by streaming experts from NVMe SSD, and outperforms llama.cpp on all 13 tested models.
-
0.1
composo-ai/DishwasherBench
A browser game about loading a dishwasher, built on a physical simulation of a real Bosch machine to benchmark language models. Players load dishes, run the Eco 50 programme and score points for items that come out…
-
0.1
7API/glm-5.3-abliterated
A guide and prompt collection for GLM 5.3 abliterated/uncensored models, with 35 jailbreak prompts and access through a single OpenAI- and Anthropic-compatible 7API key.
-
0.0
rockmatrix-ai/EdgeRock
YANchor-4B is a 4.58B-parameter general-purpose recurrent language model that preserves crucial information as memory anchors, enabling long-horizon reasoning in O(N) time and O(1) memory. The repo provides inference…
-
0.0
cdcailf/cjkfold
Python library that folds look-alike CJK glyphs (Kangxi radicals and CJK radical supplements) back to the Han characters they render as, so text extracted from PDFs matches in search and comparison. Zero-dependency,…
-
0.0
Augmented-Nature/BioOntology-MCP-Server
Unofficial MCP server exposing the BioOntology API for searching, annotating, and exploring 1,200+ biological ontologies. It lets MCP-compatible clients like Claude Desktop work with biomedical ontology data.
-
0.0
7API/uncensored-ai-api
A gateway giving one API key to Claude, GPT, Gemini, Grok, DeepSeek, GLM and Kimi, including uncensored and abliterated variants. OpenAI- and Anthropic-compatible, pay per token.
-
0.0
JackWang666666/my-rag
Retrieval-augmented generation knowledge base system with a Next.js frontend and two equivalent backends (FastAPI/LangChain and Spring Boot/Spring AI) sharing the same MySQL, Milvus and Neo4j data.
-
0.0
routerhubSI/uncensored-ai-api
Gateway to AI APIs and models (Claude, GPT, Gemini, Grok, DeepSeek, GLM, Qwen) with one key, OpenAI- and Anthropic-compatible, offering uncensored and abliterated variants with pay-per-token billing.