SmolLM2: When Smol Goes Big -- Data-Centric Training of a Small Language Model
Paper
•
2502.02737
•
Published
•
60
Data Is Better Together
community
AI & ML interests
Building better datasets together
Recent Activity
View all activity
data-is-better-together's activity
plaguss
authored
a
paper
about 1 hour ago
guipenedo
authored
a
paper
about 1 hour ago
gabrielmbmb
authored
a
paper
about 1 hour ago
burtenshaw
authored
a
paper
about 1 hour ago
davanstrien
updated
a
dataset
about 3 hours ago
davidberenstein1957
posted
an
update
about 5 hours ago
Post
349
Agentic RAG: Applied, visual, and step-by-step! 🐾
Get familiar with the Agents and tools, not the bells and whistles!
Retrieve - Augment and now GENERATE.
part 3: https://huggingface.co/blog/davidberenstein1957/ai-blueprint-agentic-rag-part-3-generate
Get familiar with the Agents and tools, not the bells and whistles!
Retrieve - Augment and now GENERATE.
part 3: https://huggingface.co/blog/davidberenstein1957/ai-blueprint-agentic-rag-part-3-generate
davidberenstein1957
posted
an
update
1 day ago
Post
1718
Anyone can create free hosted tools for their AI agents! 🔥
Agentic RAG stack part 2 - augment
Augment retrieval results by reranking optimises content without increasing time too much
part2: https://huggingface.co/blog/davidberenstein1957/ai-blueprint-agentic-rag-part-2-augment
code: https://github.com/huggingface/ai-blueprint
Agentic RAG stack part 2 - augment
Augment retrieval results by reranking optimises content without increasing time too much
part2: https://huggingface.co/blog/davidberenstein1957/ai-blueprint-agentic-rag-part-2-augment
code: https://github.com/huggingface/ai-blueprint
davanstrien
updated
a
dataset
2 days ago
davidberenstein1957
posted
an
update
2 days ago
Post
1742
Creating an agentic RAG stack on the Hugging Face Hub - part 1 - retrieval (1/5).
🚀 Web apps and microservices included!
Chunk, embed and index documents at a huge scale without overhead.
Blog: https://huggingface.co/blog/davidberenstein1957/ai-blueprint-agentic-rag-part-1-retrieve
🚀 Web apps and microservices included!
Chunk, embed and index documents at a huge scale without overhead.
Blog: https://huggingface.co/blog/davidberenstein1957/ai-blueprint-agentic-rag-part-1-retrieve
Post
1750
We have been cooking a couple of fine-tuning runs on CogVideoX with finetrainers, smol datasets, and LoRA to generate cool video effects like crushing, dissolving, etc.
We are also releasing a LoRA extraction utility from a fully fine-tuned checkpoint. I know that kind of stuff has existed since eternity, but the quality on video models was nothing short of spectacular. Below are some links:
* Models and datasets: https://huggingface.co/finetrainers
* finetrainers: https://github.com/a-r-r-o-w/finetrainers
* LoRA extraction: https://github.com/huggingface/diffusers/blob/main/scripts/extract_lora_from_model.py
We are also releasing a LoRA extraction utility from a fully fine-tuned checkpoint. I know that kind of stuff has existed since eternity, but the quality on video models was nothing short of spectacular. Below are some links:
* Models and datasets: https://huggingface.co/finetrainers
* finetrainers: https://github.com/a-r-r-o-w/finetrainers
* LoRA extraction: https://github.com/huggingface/diffusers/blob/main/scripts/extract_lora_from_model.py
- 1 reply
davidberenstein1957
posted
an
update
7 days ago
Post
1532
tldr; Parquet is awesome, DuckDB too!
Datasets on the Hugging Face Hub rely on parquet files. We can interact with these files using DuckDB as a fast in-memory database system. One of DuckDB’s features is vector similarity search which can be used with or without an index.
blog:
https://huggingface.co/learn/cookbook/vector_search_with_hub_as_backend
Datasets on the Hugging Face Hub rely on parquet files. We can interact with these files using DuckDB as a fast in-memory database system. One of DuckDB’s features is vector similarity search which can be used with or without an index.
blog:
https://huggingface.co/learn/cookbook/vector_search_with_hub_as_backend
davanstrien
posted
an
update
8 days ago
Post
1683
Why choose between strong LLM reasoning and efficient models?
Use DeepSeek to generate high-quality training data, then distil that knowledge into ModernBERT answerdotai/ModernBERT-base for fast, efficient classification.
Blog post: https://danielvanstrien.xyz/posts/2025/deepseek/distil-deepseek-modernbert.html
Use DeepSeek to generate high-quality training data, then distil that knowledge into ModernBERT answerdotai/ModernBERT-base for fast, efficient classification.
Blog post: https://danielvanstrien.xyz/posts/2025/deepseek/distil-deepseek-modernbert.html
davanstrien
posted
an
update
9 days ago
Post
1823
Updated the ColPali Query Generator Space
davanstrien/ColPali-Query-Generator to use
Qwen/Qwen2.5-VL-7B-Instruct.
Given an input image, it generates several queries along with explanations to justify them. This approach can generate synthetic data for fine-tuning ColPali models.
Given an input image, it generates several queries along with explanations to justify them. This approach can generate synthetic data for fine-tuning ColPali models.
Post
1898
We have authored a post to go over the state of video generation in the Diffusers ecosystem 🧨
We cover the models supported, the knobs of optims our users can fire, fine-tuning, and more 🔥
5-6GBs for HunyuanVideo, sky is the limit 🌌 🤗
https://huggingface.co/blog/video_gen
We cover the models supported, the knobs of optims our users can fire, fine-tuning, and more 🔥
5-6GBs for HunyuanVideo, sky is the limit 🌌 🤗
https://huggingface.co/blog/video_gen
davanstrien
posted
an
update
10 days ago
Post
1981
🌍 Big step for multilingual AI data!
The Hugging Face community has rated educational content in languages spoken by 1.6 billion people! New additions:
• Japanese
• Italian
• Old High German
Learn more and contribute: https://huggingface.co/blog/davanstrien/fineweb2-community
These ratings can help enhance training data for major world languages.
The Hugging Face community has rated educational content in languages spoken by 1.6 billion people! New additions:
• Japanese
• Italian
• Old High German
Learn more and contribute: https://huggingface.co/blog/davanstrien/fineweb2-community
These ratings can help enhance training data for major world languages.
- 1 reply
fix config names
#11 opened 10 days ago
by
davanstrien
davidberenstein1957
posted
an
update
10 days ago
Post
1741
Let's uncover the post-training dataset from DeepSeek-R1 with Magpie!
Pass pre-query tokens
We can get realistic examples!
Gist: https://gist.github.com/davidberenstein1957/3f20046ce57395a6aba13f8b4e956b59
Pass pre-query tokens
<｜begin▁of▁sentence｜>User: , let the model generate the rest.We can get realistic examples!
Gist: https://gist.github.com/davidberenstein1957/3f20046ce57395a6aba13f8b4e956b59
-
- 6 replies
burtenshaw
posted
an
update
10 days ago
Post
2796
Manic few days in open source AI, with game changing development all over the place. Here's a round up of the resources:
- The science team at @huggingface reproduced and open source the seek r1. https://github.com/huggingface/open-r1
- @qwen released a series of models with 1 million token context! https://qwenlm.github.io/blog/qwen2.5-1m/
- SmolVLM got even smaller with completely new variants at 256m and 500m https://huggingface.co/blog/smolervlm
There's so much you could do with these developments. Especially combining them together into agentic applications or fine-tuning them on your use case.
- The science team at @huggingface reproduced and open source the seek r1. https://github.com/huggingface/open-r1
- @qwen released a series of models with 1 million token context! https://qwenlm.github.io/blog/qwen2.5-1m/
- SmolVLM got even smaller with completely new variants at 256m and 500m https://huggingface.co/blog/smolervlm
There's so much you could do with these developments. Especially combining them together into agentic applications or fine-tuning them on your use case.
-
- 1 reply
burtenshaw
posted
an
update
13 days ago
Post
1034
Hey 👋
I'm helping out on some community research to learn about the AI community. If you want to join in the conversation, head over here where I started a community discussion on the most influential model since BERT.
OSAIResearchCommunity/README#2
I'm helping out on some community research to learn about the AI community. If you want to join in the conversation, head over here where I started a community discussion on the most influential model since BERT.
OSAIResearchCommunity/README#2
burtenshaw
posted
an
update
13 days ago
Post
1703
📣 Teachers and Students! Here's a handy quiz app if you're preparing your own study material.
TLDR, It's a quiz that uses a dataset to make questions and save answers
Here's how it works:
- make a dataset of multiple choice questions
- duplicate the space add set the dataset repo
- log in and do the quiz
- submit the questions to create a new dataset
I made this to get ready for the agents course, but I hope it's useful for you projects too!
quiz app burtenshaw/dataset_quiz
dataset with questions burtenshaw/exam_questions
agents course we're working on https://huggingface.co/agents-course
TLDR, It's a quiz that uses a dataset to make questions and save answers
Here's how it works:
- make a dataset of multiple choice questions
- duplicate the space add set the dataset repo
- log in and do the quiz
- submit the questions to create a new dataset
I made this to get ready for the agents course, but I hope it's useful for you projects too!
quiz app burtenshaw/dataset_quiz
dataset with questions burtenshaw/exam_questions
agents course we're working on https://huggingface.co/agents-course