SmolLM2: When Smol Goes Big -- Data-Centric Training of a Small Language Model
Paper
β’
2502.02737
β’
Published
β’
61
CompVis
university
AI & ML interests
Generative approaches for visual synthesis, Invertible deep models for explainable AI, Deep metric and representation learning, self-supervised learning paradigms
Recent Activity
View all activity
CompVis's activity
anton-lΒ
authored
a
paper
about 1 hour ago
taohuΒ
updated
a
model
about 4 hours ago
lvwerraΒ
authored
a
paper
about 4 hours ago
taohuΒ
published
a
model
about 5 hours ago
stellaathenaΒ
authored
a
paper
8 days ago
lvwerraΒ
authored
a
paper
21 days ago
stellaathenaΒ
authored
a
paper
21 days ago
Post
8287
Google drops Gemini 2.0 Flash Thinking
a new experimental model that unlocks stronger reasoning capabilities and shows its thoughts. The model plans (with thoughts visible), can solve complex problems with Flash speeds, and more
now available in anychat, try it out: akhaliq/anychat
a new experimental model that unlocks stronger reasoning capabilities and shows its thoughts. The model plans (with thoughts visible), can solve complex problems with Flash speeds, and more
now available in anychat, try it out: akhaliq/anychat
- 2 replies
YfZΒ
authored
7
papers
about 2 months ago
Towards Aligned Layout Generation via Diffusion Model with Aesthetic Constraints
Paper
β’
2402.04754
β’
Published
LLaVA-Read: Enhancing Reading Ability of Multimodal Language Models
Paper
β’
2407.19185
β’
Published
β’
1
ARTIST: Improving the Generation of Text-rich Images by Disentanglement
Paper
β’
2406.12044
β’
Published
MMR: Evaluating Reading Ability of Large Multimodal Models
Paper
β’
2408.14594
β’
Published
TextLap: Customizing Language Models for Text-to-Layout Planning
Paper
β’
2410.12844
β’
Published
LoRA-Contextualizing Adaptation of Large Multimodal Models for Long Document Understanding
Paper
β’
2411.01106
β’
Published
β’
4
SUGAR: Subject-Driven Video Customization in a Zero-Shot Manner
Paper
β’
2412.10533
β’
Published
β’
5
Post
2378
Introducing ππ
π’π§πππππ‘: the best public math pre-training dataset with 50B+ tokens!
HuggingFaceTB/finemath
Math remains challenging for LLMs and by training on FineMath we see considerable gains over other math datasets, especially on GSM8K and MATH.
We build the dataset by:
π οΈ carefully extracting math data from Common Crawl;
π iteratively filtering and recalling high quality math pages using a classifier trained on synthetic annotations to identify math reasoning and deduction.
We conducted a series of ablations comparing the performance of Llama-3.2-3B-Base after continued pre-training on FineMath and observe notable gains compared to the baseline model and other public math datasets.
We hope this helps advance the performance of LLMs on math and reasoning! π
Weβre also releasing all the ablation models as well as the evaluation code.
HuggingFaceTB/finemath-6763fb8f71b6439b653482c2
HuggingFaceTB/finemath
Math remains challenging for LLMs and by training on FineMath we see considerable gains over other math datasets, especially on GSM8K and MATH.
We build the dataset by:
π οΈ carefully extracting math data from Common Crawl;
π iteratively filtering and recalling high quality math pages using a classifier trained on synthetic annotations to identify math reasoning and deduction.
We conducted a series of ablations comparing the performance of Llama-3.2-3B-Base after continued pre-training on FineMath and observe notable gains compared to the baseline model and other public math datasets.
We hope this helps advance the performance of LLMs on math and reasoning! π
Weβre also releasing all the ablation models as well as the evaluation code.
HuggingFaceTB/finemath-6763fb8f71b6439b653482c2
fix website link
1
#2 opened about 2 months ago
by
qwertyforce
Add pipeline tag, enable download stats
3
#1 opened about 2 months ago
by
nielsr
kliyerΒ
updated
a
model
about 2 months ago
Add pipeline tag, enable download stats
3
#1 opened about 2 months ago
by
nielsr