AI inference

Optimize long-context AI inference with Red Hat AI 3.5 KV cache offloading, which boosts throughput by 5.8x and cuts TTFT from 60s to 10s on NVIDIA H100 GPUs.

Learn how to integrate the Jev API for bounded semantic judgments by executing your first call and interpreting confidence scores.

Discover how Atos fine-tuned SLMs on OpenShift AI to outperform frontier models on structured extraction tasks using only 3 GPUs.

Learn how to benchmark local LLMs with Red Hat AI Inference and Red Hat Enterprise Linux by deploying vLLM and testing performance on your hardware.

Learn How hierarchical orchestration scales multi-agent systems by replacing passive merging with active coordination to prevent attention dilution.

Double Nemotron 3.5 Super VL inference throughput on NVIDIA Hopper GPUs. Learn how FP8 quantization and DSpark speculative decoding boost speed and accuracy.

Learn how to choose between vLLM and SGLang for production LLM inference by comparing PagedAttention, RadixAttention, and independent benchmark results.

Explore how Red Hat AI Inference 3.5 MaaS brings multi-tenant API governance, API key authentication, and token rate limiting to LLM endpoints on AKS.

Learn how to add NeMo Guardrails to LangGraph agents on Red Hat OpenShift AI using a proxy pattern to inspect requests without modifying agent code.

Compare Jev decision models with Red Hat guardrails to evaluate AI safety performance, cost, and latency for enterprise implementation.

Discover how to build cost-ranked fraud detection models in Python, combine predictive ML with LLM triage notes, and scale using Red Hat OpenShift AI.

Optimize LLM inference performance with a 71% latency reduction on NVIDIA H200 GPUs. Learn how to cut TTFT from 995ms to 287ms with smarter request routing.

Track AI inference costs by department using Red Hat OpenShift AI’s built-in MaaS gateway API keys, Perses dashboards, and MLflow tracing, with no manual instrumentation required.

Implement a regression detector for Red Hat AI Inference, catching performance regressions in CI/CD before release. Learn how in this guide.

Connect AI coding agents to Red Hat Developer Hub to enforce governance rules, prevent compliance violations, and ground architectural choices in your catalog.

Learn how to reduce Llama 3.1 8B Instruct model size and improve performance with W8A8 INT8 quantization.

Automate enterprise-wide RAG pipelines with Red Hat OpenShift AI's AutoRAG, speeding up AI optimization for your business.

Manage AI infrastructure costs and governance with Red Hat AI 3.4's Model-as-a-Service.

Learn how to install and configure OpenCode, an open source AI coding assistant, for local development with Ollama, OpenVINO, or Red Hat AI.

Improve large language model inference speed with Speculators 0.6.0's FastMTP-style fine-tuning.

Cut Llama 3.1 8B VRAM by 46% without losing accuracy. Master the mechanics of INT8 W8A8 quantization, SmoothQuant, and GPTQ using llm-compressor.

Learn how P-EAGLE in Speculators v0.6.0 uses parallel drafting to reduce LLM latency. Train and deploy custom draft models with this step-by-step guide.

Evaluate AI agents on Red Hat OpenShift AI with IBM CLEAR and EvalHub. Learn how to automate analysis for recurring failure patterns in AI agent execution.

Learn how the vLLM LoRA dynamic loading flaw enables data theft and how to enforce zero trust defenses using Red Hat OpenShift AI and cluster management.

Learn how to automate RAG document processing with Red Hat OpenShift AI for production-ready parsing, embedding, and querying of PDF documents.