Helen Zhao

Double Nemotron 3.5 Super VL inference throughput on NVIDIA Hopper GPUs. Learn how FP8 quantization and DSpark speculative decoding boost speed and accuracy.

Learn how P-EAGLE in Speculators v0.6.0 uses parallel drafting to reduce LLM latency. Train and deploy custom draft models with this step-by-step guide.

Speculators v0.5.0 introduces DFlash support, enabling single-pass draft token generation with block diffusion for more efficient speculative decoding workflows. The release also adds unified online and offline training through vLLM’s native hidden states extraction system, improving training flexibility, version stability, and production readiness.

Speculators standardizes speculative decoding for large language models, with a unified Hugging Face format, vLLM integration, and more.

LLM Compressor 0.7.0 brings Hadamard transforms for better accuracy, mixed-precision FP4/FP8, and calibration-free block quantization for efficient compression.