vLLM

The high-throughput open-source engine for serving LLMs in production: “easy, fast, and cheap LLM serving for everyone”.

Rank
#7 of 66#2 of 17 in Infrastructure
Adoption score
91 / 10010.5 points above the category average
Entry price
FreeConfirmed 15 Oct 2026
Open source
YesCan be self-hosted

Our assessment

vLLM is one of the most widely used engines for self-hosted LLM serving. Its PagedAttention memory management and continuous batching let one GPU serve many concurrent requests efficiently, and it exposes an OpenAI-compatible API, so applications written against a hosted provider can point at your own deployment with little change.

Breadth is the other reason to choose it: 200+ model architectures, quantization formats, speculative decoding, tensor and pipeline parallelism, and hardware support well beyond NVIDIA. The trade-off is operational: it is a serving engine, not a product, so you own the GPUs, scaling, monitoring and upgrades on a fast release cadence.

Pick it when you serve open models to real traffic and care about throughput per GPU. For a single developer running models locally, Ollama is far simpler. If you would rather not operate GPUs at all, hosted inference from Together AI, Fireworks AI or Groq runs comparable models for you.

Best for
Production serving of open models with high concurrency and an OpenAI-compatible API.
Watch out for
You operate the infrastructure; rapid releases mean regular upgrade work.
Consider instead
Ollama (free) for local development · Together AI or Fireworks AI for hosted inference.

How it compares in Infrastructure

History

Tracked since Oct 13, 2026.

SnapshotRankAdoptionEntry pricePrice check
Nov 20, 2026#7 same91FreeConfirmed
Nov 12, 2026#7 same91FreeConfirmed
Oct 29, 2026#7 same91FreeConfirmed
Oct 15, 2026#7 same91FreeConfirmed
Oct 13, 2026#7 91FreeConfirmed

Key facts

CategoryInfrastructure: inference serving
MakervLLM Project (UC Berkeley Sky Computing Lab origin)
LicenseApache-2.0
Self-hostYes
LanguagesPython, Rust
Repository createdFeb 2023
GitHub stars93.8k (Oct 15, 2026)
Latest releasev0.31.0 (Oct 5, 2026)
Core techniquesPagedAttention, continuous batching, prefix caching
APIsOpenAI-compatible server; Anthropic Messages API; gRPC
HardwareNVIDIA and AMD GPUs; x86, ARM, PowerPC CPUs; plugins for TPU, Gaudi, Apple Silicon
Models200+ architectures from Hugging Face

Compare vLLM

Get an email when the index changes

Price changes, new tools, acquisitions and shutdowns, sent when the data is updated.