NVIDIA/TensorRT-LLM
TensorRT LLM provides users with an easy-to-use Python API to define Large Language Models (LLMs) and supports state-of-the-art optimizations to perform inference efficiently on NVIDIA GPUs. TensorRT LLM also contains components to create Python and C++ runtimes that orchestrate the inference execution in a performant way.
About the project
NVIDIA library for optimizing inference of large language models and visual generative models on GPUs, with a Python API, specialized kernels, and an efficient C++/Python runtime.
Useful for
- Deploy a high-throughput LLM inference server on NVIDIA GPUs
- Speed up video and image generation with quantization and optimized kernels
- Configure speculative decoding and KV cache to reduce latency
README summarized by DeepSeek V4.1 Flash. Details may be inaccurate.
Why it’s trending
- 0 stars so far today, about 5 expected by the end of the day.
- Over the last two days the pace is 1.6× that of the previous week and a half.
- GitHub Trending Python today: #5, +16 stars.
- About 6 forks a day — people are taking the code.
Stars per day
Bars are daily stars, the line is the usual pace. Red marks spike days.
Numbers
- Total stars
- 14,663
- Stars in a day
- 6
- Forks
- 2,761
- Issues and pull requests
- 19,205
- Watchers
- 123
- Language
- Python
- Latest release
- v1.3.0rc27 · September 18, 2026
- Created
- August 16, 2023
- Last push
- September 19, 2026
Star trust
Growth looks organic: forks and discussion are in line with active projects, and stars arrive unevenly, the way people give them.
These are heuristics, not a verdict: we judge by the repository’s behavior, not by a list of stargazers.
Spotted in
- September 19, 2026GitHub Trending Python today: #5, +16 stars
Share
README badge
Paste it into your README — the badge shows the current magnitude and links to this page.
[](https://gitnova.dev/en/r/NVIDIA/TensorRT-LLM)Similar projects
-
7.7
TheoLeeCJ/SemIf
An open reproduction of the Jev-style semantic decision interface: reads typed option probabilities directly from a 4B model's logits without generating text. Runs on a single RTX 3090.
-
7.4
TianyuCodings/NanoJev
A nano replica of Jev built on Qwen3-0.6B that outputs probability distributions over candidates in a single forward pass with no token decoding, plus a training pipeline and game demos (maze, Snake).
-
6.4
Continuum-AI-Corp/OrcaBonsai-27B-Uncensored
A tool for runtime refusal ablation in the compressed Ternary Bonsai 2 27B LLM: it projects the residual stream to remove the refusal direction without modifying or re-quantizing weights. Runs on Apple Silicon via MLX.
-
6.1
jaredpalmer/kev
kev is a LoRA adapter with a small readout head on top of Qwen2.5-0.5B that answers many typed questions about a document in a single forward pass, returning calibrated probabilities instead of text.
-
6.0
githubnext/localjev
A local TypeScript/Bun HTTP service implementing a Jev-compatible POST /v1/systemone API on top of an OpenAI-compatible Chat Completions endpoint (DiffusionGemma via oMLX). It acts as a bridge for typed decisions…
-
6.0
higgsfield-ai/higgsfield
Fault-tolerant GPU orchestration and ML framework for distributed training of models with billions to trillions of parameters, including LLMs. Manages node access, experiment queueing, and GitHub Actions integration.