Greg Pereira

Learn how to optimize deployment of vLLM for various traffic shapes, including high-concurrency chat, long-context RAG, high-throughput batch, and distributed AI-grid.

Learn about the three optimization levers for distributed AI inference: prefill/decode disaggregation, KV cache strategy, and speculative decoding.

Learn about the five-dimensional design space in modern LLM serving, including tensor, pipeline, expert, data, and context parallelism.

Discover the advantages of vLLM, an open source inference server that speeds up generative AI applications by making better use of GPU memory.