TensorRT-LLM
github.com
NVIDIA library for optimizing LLM inference.
Quick Information
- Category:
- MLOps & Infrastructure
- Pricing:
- Free
- Platforms:
- Linux, Python
- Free Plan:
- Yes
- API Available:
- Yes
- Last Verified:
- October 2026
What Is TensorRT-LLM?
TensorRT-LLM is NVIDIA's library for optimizing large language model inference on its GPUs. It is free to download, and your cost is the NVIDIA hardware you run it on. The latest version is unclear from the sources I found. PyPI lists 1.2.1 as the latest release, dated April 20, 2026, but its history includes later pre-releases, the newest being 1.4.0rc0 from October 13, 2026, and a GitHub releases page shows v1.4.0rc1 around October 22. NVIDIA's release notes cover version 1.3, which deprecates the TRITON MoE backend as of September 2026, and describe 1.0 as the point where the PyTorch-based architecture and LLM API became stable. So the most recent stable release is not clear from these results.
On licensing, Wikipedia describes TensorRT as proprietary with some companion components under Apache 2.0 and treats TensorRT-LLM as a separate repository, but does not state the TensorRT-LLM licence, so I cannot confirm Apache 2.0 here. Check the LICENSE file in NVIDIA/TensorRT-LLM. My search returned nothing on NVIDIA Dynamo's relationship to it. TensorRT-LLM tends to deliver strong performance on NVIDIA hardware but ties you to that vendor, and a separate NVIDIA Edge LLM project exists for physical AI.
TensorRT-LLM suits teams standardised on NVIDIA GPUs who want maximum inference performance. Compare vLLM, Triton Inference Server and ONNX Runtime. Browse the MLOps & Infrastructure tools directory for more.
What Can You Use TensorRT-LLM For?
Optimized LLM inference
NVIDIA GPUs.
Production serving with Triton
Backend.
TensorRT-LLM Features
- PyTorch-based architecture — Stable since 1.0.
- Release candidates — 1.4.0 in preview.
- LLM API — Python.
Who Is TensorRT-LLM For?
- NVIDIA GPU operators — Performance.
- Inference engineers — Fine-grained tuning.
TensorRT-LLM Pros & Limitations
What We Like
- Strong NVIDIA performance
- Free
- Active releases
Limitations
- NVIDIA only
- Latest stable version unclear
- Licence not confirmed here
TensorRT-LLM Pricing
| Plan | Price | Main Features |
|---|---|---|
| Library | Free to download (licence not confirmed here) | Requires NVIDIA GPUs |
Pricing was last verified October 2026. Check the official website for the latest pricing.
AI Listing Jungle Ranking
- Global Rank:
- #1824
- Overall Score:
- 0.0/100
- Monthly Users:
- –
- Rank Change:
- ▼ 256
This ranking reflects AI Listing Jungle's own independent scoring methodology and is not an endorsement by TensorRT-LLM or its company. How Our Rankings Work
Frequently Asked Questions
Is it free?
Yes to download.
Latest stable?
PyPI shows 1.2.1; 1.4.0 pre-releases exist.
Runs on AMD?
It targets NVIDIA GPUs.
Licence?
Check the repository.
Our Take
Top performance if you are all-in on NVIDIA.
Verify the licence and version.
TensorRT-LLM Alternatives
TensorRT-LLM Comparisons
Ready to try TensorRT-LLM?
Visit the official website to get started.
Visit Official WebsiteAI Listing Jungle is an independent AI tools directory. Product names, logos, and trademarks belong to their respective owners.