Red Hat Developer Blog

Here's our most recent blog content. Explore our featured monthly resource as well as our most recently published items. Don't miss the chance to learn more about our contributors.

Subscribe to the feed

View all blogs & articles

Content type
Product
Topics
Article Featured image for Deploy Llama 3 8B with vLLM blog.
Jun 18, 2024

Deploy Llama 3 8B with vLLM

Mark Kurtz

Llama 3's advancements, particularly at 8 billion parameters, make AI more...

Article Featured image for LLM inference with Marlin.

Learn about Marlin, a mixed-precision matrix multiplication kernel that...

Article Featured image for How Well Do Quantized Models Handle Long-Context Tasks?

4-bit and 8-bit quantized LLMs excel in long-context tasks, retaining over...

Article Featured image for AI/ML

Sparse fine-tuning in combination with sparsity-aware inference software,...

Article featured image for SparseGPT.

Compress large language models (LLMs) with SparseGPT to make your machine...