Michael Goin's contributions
Article
5 steps to triage vLLM performance
Learn how to improve the performance of your vLLM deployments with a diagnostic workflow that isolates latency issues and server saturation. Discover the key metrics to monitor and techniques to alleviate memory pressure.
Article
vLLM with torch.compile: Efficient LLM inference on PyTorch
Learn how to optimize PyTorch code with minimal effort using torch.compile, a just-in-time compiler that generates optimized kernels automatically.
Article
Structured outputs in vLLM: Guiding AI responses
Learn how to control the output of vLLM's AI responses with structured outputs. Discover how to define choice lists, JSON schemas, regex, and more.
Article
LLM Compressor: Optimize LLMs for low-latency deployments
LLM Compressor bridges the gap between model training and efficient deployment via quantization and sparsity, enabling cost-effective, low-latency inference.
Article
How we optimized vLLM for DeepSeek-R1
Explore inference performance improvements that help vLLM serve DeepSeek AI models more efficiently in this technical deep dive.
Article
vLLM V1: Accelerating multimodal inference for large language models
Explore how vLLM's new multimodal AI inference capabilities enhance performance, scalability, and flexibility across diverse hardware platforms.
Article
Distributed inference with vLLM
Explore how distributed inference works within vLLM in this recap of Neural Magic's vLLM Office Hours with Michael Goin and Murali Andoorveedu, a vLLM committer from CentML.
Article
vLLM brings FP8 inference to the open source community
Explore the integration of FP8 in vLLM. Learn how to receive up to a 2x reduction in latency on NVIDIA GPUs with minimal accuracy degradation.
5 steps to triage vLLM performance
Learn how to improve the performance of your vLLM deployments with a diagnostic workflow that isolates latency issues and server saturation. Discover the key metrics to monitor and techniques to alleviate memory pressure.
vLLM with torch.compile: Efficient LLM inference on PyTorch
Learn how to optimize PyTorch code with minimal effort using torch.compile, a just-in-time compiler that generates optimized kernels automatically.
Structured outputs in vLLM: Guiding AI responses
Learn how to control the output of vLLM's AI responses with structured outputs. Discover how to define choice lists, JSON schemas, regex, and more.
LLM Compressor: Optimize LLMs for low-latency deployments
LLM Compressor bridges the gap between model training and efficient deployment via quantization and sparsity, enabling cost-effective, low-latency inference.
How we optimized vLLM for DeepSeek-R1
Explore inference performance improvements that help vLLM serve DeepSeek AI models more efficiently in this technical deep dive.
vLLM V1: Accelerating multimodal inference for large language models
Explore how vLLM's new multimodal AI inference capabilities enhance performance, scalability, and flexibility across diverse hardware platforms.
Distributed inference with vLLM
Explore how distributed inference works within vLLM in this recap of Neural Magic's vLLM Office Hours with Michael Goin and Murali Andoorveedu, a vLLM committer from CentML.
vLLM brings FP8 inference to the open source community
Explore the integration of FP8 in vLLM. Learn how to receive up to a 2x reduction in latency on NVIDIA GPUs with minimal accuracy degradation.
