Yuchen Fama's contributions
Article
Benchmarking tiered KV cache offloading on H100 GPUs in OpenShift AI 3.5
Optimize long-context AI inference with Red Hat AI 3.5 KV cache offloading, which boosts throughput by 5.8x and cuts TTFT from 60s to 10s on NVIDIA H100 GPUs.
Article
Deploying distributed AI inference: Blueprints & troubleshooting
Learn how to optimize deployment of vLLM for various traffic shapes, including high-concurrency chat, long-context RAG, high-throughput batch, and distributed AI-grid.
Article
Optimizing distributed AI inference: Advanced deployment patterns
Learn about the three optimization levers for distributed AI inference: prefill/decode disaggregation, KV cache strategy, and speculative decoding.
Article
Designing distributed AI inference: Core concepts and scaling dimensions
Learn about the five-dimensional design space in modern LLM serving, including tensor, pipeline, expert, data, and context parallelism.
Article
From local prototype to enterprise production: Private speech transcription with Whisper and Red Hat AI
Learn how to run OpenAI's Whisper model through vLLM on Apple Silicon, giving you an OpenAI-compatible endpoint on localhost. Then, discover how to take this architecture into production using Red Hat AI Inference Server.
Benchmarking tiered KV cache offloading on H100 GPUs in OpenShift AI 3.5
Optimize long-context AI inference with Red Hat AI 3.5 KV cache offloading, which boosts throughput by 5.8x and cuts TTFT from 60s to 10s on NVIDIA H100 GPUs.
Deploying distributed AI inference: Blueprints & troubleshooting
Learn how to optimize deployment of vLLM for various traffic shapes, including high-concurrency chat, long-context RAG, high-throughput batch, and distributed AI-grid.
Optimizing distributed AI inference: Advanced deployment patterns
Learn about the three optimization levers for distributed AI inference: prefill/decode disaggregation, KV cache strategy, and speculative decoding.
Designing distributed AI inference: Core concepts and scaling dimensions
Learn about the five-dimensional design space in modern LLM serving, including tensor, pipeline, expert, data, and context parallelism.
From local prototype to enterprise production: Private speech transcription with Whisper and Red Hat AI
Learn how to run OpenAI's Whisper model through vLLM on Apple Silicon, giving you an OpenAI-compatible endpoint on localhost. Then, discover how to take this architecture into production using Red Hat AI Inference Server.
