Yuan Tang

Learn how to optimize deployment of vLLM for various traffic shapes, including high-concurrency chat, long-context RAG, high-throughput batch, and distributed AI-grid.

Learn about the three optimization levers for distributed AI inference: prefill/decode disaggregation, KV cache strategy, and speculative decoding.

Learn about the five-dimensional design space in modern LLM serving, including tensor, pipeline, expert, data, and context parallelism.

Learn how to combine KServe and llm-d to optimize generative AI inference, improve performance, and reduce infrastructure costs. This article demonstrates the integration architecture and provides practical guidance for AI platform teams.

Discover the advantages of vLLM, an open source inference server that speeds up generative AI applications by making better use of GPU memory.

Discover the benefits of KServe, a highly scalable machine learning deployment tool for Kubernetes.