Alberto Perdomo

Optimize long-context AI inference with Red Hat OpenShift AI 3.5 KV cache offloading to increase throughput and reduce time to first token (TTFT) across NVIDIA H100 GPU deployments.

This performance analysis compares KServe's SLO-driven KEDA autoscaling approach against Knative's concurrency-based autoscaling for vLLM inference.

Walk through how to set up KServe autoscaling by leveraging the power of vLLM, KEDA, and the custom metrics autoscaler operator in Open Data Hub.

Learn about the Red Hat OpenShift AI model fine-tuning stack and how to run performance and scale validation.