Rahul Tuli

Speculators standardizes speculative decoding for large language models, with a unified Hugging Face format, vLLM integration, and more.

Discover how to deploy compressed, fine-tuned models for efficient inference with the new Axolotl and LLM Compressor integration.

LLM Compressor bridges the gap between model training and efficient deployment via quantization and sparsity, enabling cost-effective, low-latency inference.