SmolLM2: When Smol Goes Big -- Data-Centric Training of a Small Language Model
Paper
•
2502.02737
•
Published
•
54
Blog-explorers
AI & ML interests
None defined yet.
Recent Activity
View all activity
blog-explorers's activity
burtenshaw
authored
a
paper
40 minutes ago
[Support] Community Articles
80
#5 opened 11 months ago
by
victor
[Support] Community Articles
80
#5 opened 11 months ago
by
victor
Post
1320
🚀 Reproducing DeepSeek R1 for Text-to-Graph Extraction
I’ve been working on replicating DeepSeek R1, focusing on zero-shot text-to-graph extraction—a challenging task where LMs extract entities and relations from text based on predefined types.
🧠 Key Insight:
Language models struggle when constrained by entity/relation types. Supervised training alone isn’t enough, but reinforcement learning (RL), specifically Guided Reward Policy Optimization (GRPO), shows promise.
💡 Why GRPO?
It trains the model to generate structured graphs, optimizing multiple reward functions (format, JSON validity, and extraction accuracy).
It allows the model to learn from both positive and hard negative examples dynamically.
RL can be fine-tuned to emphasize relation extraction improvements.
📊 Early Results:
Even with limited training, F1 scores consistently improved, and we saw clear benefits from RL-based optimization. More training = better performance!
🔬 Next Steps:
We’re scaling up experiments with larger models and high-quality data. Stay tuned for updates! Meanwhile, check out one of our experimental models here:
Ihor/Text2Graph-R1-Qwen2.5-0.5b
📔 Learn more details from the blog post: https://medium.com/p/d8b648d9f419
Feel free to share your thoughts and ask questions!
I’ve been working on replicating DeepSeek R1, focusing on zero-shot text-to-graph extraction—a challenging task where LMs extract entities and relations from text based on predefined types.
🧠 Key Insight:
Language models struggle when constrained by entity/relation types. Supervised training alone isn’t enough, but reinforcement learning (RL), specifically Guided Reward Policy Optimization (GRPO), shows promise.
💡 Why GRPO?
It trains the model to generate structured graphs, optimizing multiple reward functions (format, JSON validity, and extraction accuracy).
It allows the model to learn from both positive and hard negative examples dynamically.
RL can be fine-tuned to emphasize relation extraction improvements.
📊 Early Results:
Even with limited training, F1 scores consistently improved, and we saw clear benefits from RL-based optimization. More training = better performance!
🔬 Next Steps:
We’re scaling up experiments with larger models and high-quality data. Stay tuned for updates! Meanwhile, check out one of our experimental models here:
Ihor/Text2Graph-R1-Qwen2.5-0.5b
📔 Learn more details from the blog post: https://medium.com/p/d8b648d9f419
Feel free to share your thoughts and ask questions!
- 2 replies
Post
3700
🔥 THE WAIT IS OVER... HAI-SER IS HERE! 🔥
Yo fam, this ain't just another AI drop— this is the FUTURE of emotional intelligence! 🚀
Introducing HAI-SER, powered by Structured Emotional Reasoning (SER), the next-level AI that doesn’t just understand your words—it feels you, analyzes your emotions, and helps you navigate life’s toughest moments. 💡
💥 What makes HAI-SER a game-changer?
🔹 Emotional Vibe Check – Gets the mood, energy, and what’s really going on 🎭
🔹 Mind-State Analysis – Breaks down your thoughts, beliefs, and patterns 🤯
🔹 Root Cause Deep-Dive – Unpacks the WHY behind your emotions 💡
🔹 Impact Check – Sees how it’s affecting your life and mental health 💔
🔹 Safety Check – Prioritizes your well-being and crisis management 🚨
🔹 Healing Game Plan – Custom strategies to help you bounce back 💪
🔹 Growth Potential – Turns struggles into opportunities for self-improvement 📈
🔹 How to Approach – Teaches you and others how to communicate and heal 🤝
🔹 Personalized Response – Not just generic advice—real talk, tailored to YOU 💯
No more robotic AI responses. No more surface-level advice. HAI-SER gets deep, analyzing emotions with precision and giving real, actionable support.
This ain’t just AI—this is your digital therapist, life coach, and hype squad all in one. Whether it’s mental health, career struggles, relationships, or personal growth, HAI-SER has your back.
🚀 The future of emotionally intelligent AI is HERE.
Are you ready? 🔥💯
HelpingAI/HAI-SER
Yo fam, this ain't just another AI drop— this is the FUTURE of emotional intelligence! 🚀
Introducing HAI-SER, powered by Structured Emotional Reasoning (SER), the next-level AI that doesn’t just understand your words—it feels you, analyzes your emotions, and helps you navigate life’s toughest moments. 💡
💥 What makes HAI-SER a game-changer?
🔹 Emotional Vibe Check – Gets the mood, energy, and what’s really going on 🎭
🔹 Mind-State Analysis – Breaks down your thoughts, beliefs, and patterns 🤯
🔹 Root Cause Deep-Dive – Unpacks the WHY behind your emotions 💡
🔹 Impact Check – Sees how it’s affecting your life and mental health 💔
🔹 Safety Check – Prioritizes your well-being and crisis management 🚨
🔹 Healing Game Plan – Custom strategies to help you bounce back 💪
🔹 Growth Potential – Turns struggles into opportunities for self-improvement 📈
🔹 How to Approach – Teaches you and others how to communicate and heal 🤝
🔹 Personalized Response – Not just generic advice—real talk, tailored to YOU 💯
No more robotic AI responses. No more surface-level advice. HAI-SER gets deep, analyzing emotions with precision and giving real, actionable support.
This ain’t just AI—this is your digital therapist, life coach, and hype squad all in one. Whether it’s mental health, career struggles, relationships, or personal growth, HAI-SER has your back.
🚀 The future of emotionally intelligent AI is HERE.
Are you ready? 🔥💯
HelpingAI/HAI-SER
- 5 replies
[Support] Community Articles
80
#5 opened 11 months ago
by
victor
davanstrien
posted
an
update
8 days ago
Post
1682
Why choose between strong LLM reasoning and efficient models?
Use DeepSeek to generate high-quality training data, then distil that knowledge into ModernBERT answerdotai/ModernBERT-base for fast, efficient classification.
Blog post: https://danielvanstrien.xyz/posts/2025/deepseek/distil-deepseek-modernbert.html
Use DeepSeek to generate high-quality training data, then distil that knowledge into ModernBERT answerdotai/ModernBERT-base for fast, efficient classification.
Blog post: https://danielvanstrien.xyz/posts/2025/deepseek/distil-deepseek-modernbert.html
davanstrien
posted
an
update
9 days ago
Post
1823
Updated the ColPali Query Generator Space
davanstrien/ColPali-Query-Generator to use
Qwen/Qwen2.5-VL-7B-Instruct.
Given an input image, it generates several queries along with explanations to justify them. This approach can generate synthetic data for fine-tuning ColPali models.
Given an input image, it generates several queries along with explanations to justify them. This approach can generate synthetic data for fine-tuning ColPali models.
davanstrien
posted
an
update
10 days ago
Post
1981
🌍 Big step for multilingual AI data!
The Hugging Face community has rated educational content in languages spoken by 1.6 billion people! New additions:
• Japanese
• Italian
• Old High German
Learn more and contribute: https://huggingface.co/blog/davanstrien/fineweb2-community
These ratings can help enhance training data for major world languages.
The Hugging Face community has rated educational content in languages spoken by 1.6 billion people! New additions:
• Japanese
• Italian
• Old High German
Learn more and contribute: https://huggingface.co/blog/davanstrien/fineweb2-community
These ratings can help enhance training data for major world languages.
- 1 reply
KennyUTC
authored
2
papers
10 days ago
burtenshaw
posted
an
update
10 days ago
Post
2794
Manic few days in open source AI, with game changing development all over the place. Here's a round up of the resources:
- The science team at @huggingface reproduced and open source the seek r1. https://github.com/huggingface/open-r1
- @qwen released a series of models with 1 million token context! https://qwenlm.github.io/blog/qwen2.5-1m/
- SmolVLM got even smaller with completely new variants at 256m and 500m https://huggingface.co/blog/smolervlm
There's so much you could do with these developments. Especially combining them together into agentic applications or fine-tuning them on your use case.
- The science team at @huggingface reproduced and open source the seek r1. https://github.com/huggingface/open-r1
- @qwen released a series of models with 1 million token context! https://qwenlm.github.io/blog/qwen2.5-1m/
- SmolVLM got even smaller with completely new variants at 256m and 500m https://huggingface.co/blog/smolervlm
There's so much you could do with these developments. Especially combining them together into agentic applications or fine-tuning them on your use case.
-
- 1 reply
burtenshaw
posted
an
update
13 days ago
Post
1034
Hey 👋
I'm helping out on some community research to learn about the AI community. If you want to join in the conversation, head over here where I started a community discussion on the most influential model since BERT.
OSAIResearchCommunity/README#2
I'm helping out on some community research to learn about the AI community. If you want to join in the conversation, head over here where I started a community discussion on the most influential model since BERT.
OSAIResearchCommunity/README#2
burtenshaw
posted
an
update
13 days ago
Post
1703
📣 Teachers and Students! Here's a handy quiz app if you're preparing your own study material.
TLDR, It's a quiz that uses a dataset to make questions and save answers
Here's how it works:
- make a dataset of multiple choice questions
- duplicate the space add set the dataset repo
- log in and do the quiz
- submit the questions to create a new dataset
I made this to get ready for the agents course, but I hope it's useful for you projects too!
quiz app burtenshaw/dataset_quiz
dataset with questions burtenshaw/exam_questions
agents course we're working on https://huggingface.co/agents-course
TLDR, It's a quiz that uses a dataset to make questions and save answers
Here's how it works:
- make a dataset of multiple choice questions
- duplicate the space add set the dataset repo
- log in and do the quiz
- submit the questions to create a new dataset
I made this to get ready for the agents course, but I hope it's useful for you projects too!
quiz app burtenshaw/dataset_quiz
dataset with questions burtenshaw/exam_questions
agents course we're working on https://huggingface.co/agents-course
burtenshaw
posted
an
update
14 days ago
Post
2320
AI was built on side projects!
KennyUTC
authored
2
papers
15 days ago
burtenshaw
posted
an
update
15 days ago
Post
3699
🚧 Work in Progress! 🚧
👷♀️ We're working hard on getting the official agents course ready for the 50,000 students that have signed up.
If you want to contribute to the discussion, I started these community posts. Looking forward to hearing from you:
- smolagents unit in the agents course - agents-course/README#7
- LlamaIndex Unit in the agents course - agents-course/README#6
- LangChain and LangGraph unit in the agents course - agents-course/README#5
- Real world use cases in the agents course - agents-course/README#8
👷♀️ We're working hard on getting the official agents course ready for the 50,000 students that have signed up.
If you want to contribute to the discussion, I started these community posts. Looking forward to hearing from you:
- smolagents unit in the agents course - agents-course/README#7
- LlamaIndex Unit in the agents course - agents-course/README#6
- LangChain and LangGraph unit in the agents course - agents-course/README#5
- Real world use cases in the agents course - agents-course/README#8
duyguislakoglu
authored
a
paper
20 days ago
burtenshaw
posted
an
update
20 days ago
Post
1384
Playing with agents, and I reckon Gradio spaces make the perfect agent tools! So I wrote this guide using Gradio and smolagents:
https://huggingface.co/blog/burtenshaw/gradio-spaces-agent-tools
https://huggingface.co/blog/burtenshaw/gradio-spaces-agent-tools