Kai Xu's contributions
Article
Sculpting subspaces: How we solved continual learning in LLMs
Discover how the adaptive SVD approach enables LLMs to continually learn and adapt without forgetting previously acquired knowledge.
Article
Lessons on reproducing R1-like reasoning in small LLMs
Learn about an efficient inference scaling method that can improve your model's reasoning ability and performance at runtime while saving on compute costs.
Article
On reasoning versus inference-time scaling
Progress in small LLM reasoning: Our Qwen-32B model, using particle filtering, now surpasses o1-preview on Math500.
Article
Granite, LIMO, and small LLM reasoning
On reproducing R1-like reasoning in small LLMs: LIMO dataset ineffective for Llama/Granite; synthetic data generation shows promise but fine-tuning is tricky.
Article
How particle filtering makes small LLMs think big
An update on reproducing R1-like reasoning in small LLMs: Granite models show big gains with particle filtering, outperforming GPT-4o on benchmarks.
Article
Sculpting subspaces: How we solved continual learning in LLMs
Discover how the adaptive SVD approach enables LLMs to continually learn and adapt without forgetting previously acquired knowledge.
Article
Lessons on reproducing R1-like reasoning in small LLMs
Learn about an efficient inference scaling method that can improve your model's reasoning ability and performance at runtime while saving on compute costs.
Article
On reasoning versus inference-time scaling
Progress in small LLM reasoning: Our Qwen-32B model, using particle filtering, now surpasses o1-preview on Math500.
Article
Granite, LIMO, and small LLM reasoning
On reproducing R1-like reasoning in small LLMs: LIMO dataset ineffective for Llama/Granite; synthetic data generation shows promise but fine-tuning is tricky.
Article
How particle filtering makes small LLMs think big
An update on reproducing R1-like reasoning in small LLMs: Granite models show big gains with particle filtering, outperforming GPT-4o on benchmarks.
