Rahul Tuli's contributions
Article
Optimize vLLM speculative decoding with FastMTP heads
Improve large language model inference speed with Speculators 0.6.0's FastMTP-style fine-tuning.
Article
Speculators: Standardized, production-ready speculative decoding
Speculators standardizes speculative decoding for large language models, with a unified Hugging Face format, vLLM integration, and more.
Article
Axolotl meets LLM Compressor: Fast, sparse, open
Discover how to deploy compressed, fine-tuned models for efficient inference with the new Axolotl and LLM Compressor integration.
Article
LLM Compressor: Optimize LLMs for low-latency deployments
LLM Compressor bridges the gap between model training and efficient deployment via quantization and sparsity, enabling cost-effective, low-latency inference.
Article
Optimize vLLM speculative decoding with FastMTP heads
Improve large language model inference speed with Speculators 0.6.0's FastMTP-style fine-tuning.
Article
Speculators: Standardized, production-ready speculative decoding
Speculators standardizes speculative decoding for large language models, with a unified Hugging Face format, vLLM integration, and more.
Article
Axolotl meets LLM Compressor: Fast, sparse, open
Discover how to deploy compressed, fine-tuned models for efficient inference with the new Axolotl and LLM Compressor integration.
Article
LLM Compressor: Optimize LLMs for low-latency deployments
LLM Compressor bridges the gap between model training and efficient deployment via quantization and sparsity, enabling cost-effective, low-latency inference.
