AI inference

Optimize LLM inference performance with a 71% latency reduction on NVIDIA H200 GPUs. Learn how to cut TTFT from 995ms to 287ms with smarter request routing.

Track AI inference costs by department using Red Hat OpenShift AI’s built-in MaaS gateway API keys, Perses dashboards, and MLflow tracing, with no manual instrumentation required.

Implement a regression detector for Red Hat AI Inference, catching performance regressions in CI/CD before release. Learn how in this guide.

Connect AI coding agents to Red Hat Developer Hub to enforce governance rules, prevent compliance violations, and ground architectural choices in your catalog.

Learn how to reduce Llama 3.1 8B Instruct model size and improve performance with W8A8 INT8 quantization.

Automate enterprise-wide RAG pipelines with Red Hat OpenShift AI's AutoRAG, speeding up AI optimization for your business.

Manage AI infrastructure costs and governance with Red Hat AI 3.4's Model-as-a-Service.

Learn how to install and configure OpenCode, an open source AI coding assistant, for local development with Ollama, OpenVINO, or Red Hat AI.

Improve large language model inference speed with Speculators 0.6.0's FastMTP-style fine-tuning.

Cut Llama 3.1 8B VRAM by 46% without losing accuracy. Master the mechanics of INT8 W8A8 quantization, SmoothQuant, and GPTQ using llm-compressor.

Learn how P-EAGLE in Speculators v0.6.0 uses parallel drafting to reduce LLM latency. Train and deploy custom draft models with this step-by-step guide.

Evaluate AI agents on Red Hat OpenShift AI with IBM CLEAR and EvalHub. Learn how to automate analysis for recurring failure patterns in AI agent execution.

Learn how the vLLM LoRA dynamic loading flaw enables data theft and how to enforce zero trust defenses using Red Hat OpenShift AI and cluster management.

Learn how to automate RAG document processing with Red Hat OpenShift AI for production-ready parsing, embedding, and querying of PDF documents.

Headed to Devoxx Belgium 2026? Visit the Red Hat Developer booth on-site to speak to our expert technologists.

Learn how to prioritize mixed workloads with Red Hat AI Inference 3.5's GPU-shared flow control

Learn how to optimize self-hosted LLM cost per token. Cut GPU spending and maximize real-world throughput with autoscaling, right-sizing, and vLLM tuning.

Explore LLM inference on Kubernetes using Red Hat AI on EKS. Trace requests from the Envoy gateway through the EPP scheduler down to individual vLLM pods.

Optimize LLM deployment with Neural Navigator on Red Hat OpenShift AI, reducing cost overruns and latency spikes.

Confidently deploy LLMs with Red Hat support: Learn how to determine if your model is supported by Red Hat's vLLM community.

See how a team upgraded Red Hat OpenShift AI 3.3.2 three to four times faster with an AI coding assistant, reducing engineering effort by 60%.

Reduce observability costs with Red Hat OpenShift AI summarizer, bridging the interpretation gap for cloud-native architectures.

Explore Kubernetes resources for Red Hat AI Inference on Amazon EKS, enabling intelligent routing for your model serving.

Stop guessing RAG settings. Discover how AutoRAG uses fast evaluation sweeps to optimize chunking and retrieval precision for small LLMs on your data.

Learn how to run isolated Llama 3.1 8B workloads on a single NVIDIA H100 GPU using OpenShift, Kubernetes dynamic resource allocation, and NVIDIA MIG.