SmolLM2: When Smol Goes Big -- Data-Centric Training of a Small Language Model
Paper
โข
2502.02737
โข
Published
โข
54
FeeL (Feedback Loop)
non-profit
AI & ML interests
Human Feedback and LLMs
Recent Activity
View all activity
feel-fl's activity
burtenshawย
authored
a
paper
40 minutes ago
davidberenstein1957ย
posted
an
update
about 5 hours ago
Post
310
Agentic RAG: Applied, visual, and step-by-step! ๐พ
Get familiar with the Agents and tools, not the bells and whistles!
Retrieve - Augment and now GENERATE.
part 3: https://huggingface.co/blog/davidberenstein1957/ai-blueprint-agentic-rag-part-3-generate
Get familiar with the Agents and tools, not the bells and whistles!
Retrieve - Augment and now GENERATE.
part 3: https://huggingface.co/blog/davidberenstein1957/ai-blueprint-agentic-rag-part-3-generate
davidberenstein1957ย
posted
an
update
1 day ago
Post
1663
Anyone can create free hosted tools for their AI agents! ๐ฅ
Agentic RAG stack part 2 - augment
Augment retrieval results by reranking optimises content without increasing time too much
part2: https://huggingface.co/blog/davidberenstein1957/ai-blueprint-agentic-rag-part-2-augment
code: https://github.com/huggingface/ai-blueprint
Agentic RAG stack part 2 - augment
Augment retrieval results by reranking optimises content without increasing time too much
part2: https://huggingface.co/blog/davidberenstein1957/ai-blueprint-agentic-rag-part-2-augment
code: https://github.com/huggingface/ai-blueprint
davidberenstein1957ย
posted
an
update
2 days ago
Post
1714
Creating an agentic RAG stack on the Hugging Face Hub - part 1 - retrieval (1/5).
๐ Web apps and microservices included!
Chunk, embed and index documents at a huge scale without overhead.
Blog: https://huggingface.co/blog/davidberenstein1957/ai-blueprint-agentic-rag-part-1-retrieve
๐ Web apps and microservices included!
Chunk, embed and index documents at a huge scale without overhead.
Blog: https://huggingface.co/blog/davidberenstein1957/ai-blueprint-agentic-rag-part-1-retrieve
davidberenstein1957ย
updated
a
dataset
2 days ago
davidberenstein1957ย
posted
an
update
7 days ago
Post
1532
tldr; Parquet is awesome, DuckDB too!
Datasets on the Hugging Face Hub rely on parquet files. We can interact with these files using DuckDB as a fast in-memory database system. One of DuckDBโs features is vector similarity search which can be used with or without an index.
blog:
https://huggingface.co/learn/cookbook/vector_search_with_hub_as_backend
Datasets on the Hugging Face Hub rely on parquet files. We can interact with these files using DuckDB as a fast in-memory database system. One of DuckDBโs features is vector similarity search which can be used with or without an index.
blog:
https://huggingface.co/learn/cookbook/vector_search_with_hub_as_backend
davidberenstein1957ย
posted
an
update
10 days ago
Post
1741
Let's uncover the post-training dataset from DeepSeek-R1 with Magpie!
Pass pre-query tokens
We can get realistic examples!
Gist: https://gist.github.com/davidberenstein1957/3f20046ce57395a6aba13f8b4e956b59
Pass pre-query tokens
<๏ฝbeginโofโsentence๏ฝ>User: , let the model generate the rest.We can get realistic examples!
Gist: https://gist.github.com/davidberenstein1957/3f20046ce57395a6aba13f8b4e956b59
-
- 6 replies
burtenshawย
posted
an
update
10 days ago
Post
2794
Manic few days in open source AI, with game changing development all over the place. Here's a round up of the resources:
- The science team at @huggingface reproduced and open source the seek r1. https://github.com/huggingface/open-r1
- @qwen released a series of models with 1 million token context! https://qwenlm.github.io/blog/qwen2.5-1m/
- SmolVLM got even smaller with completely new variants at 256m and 500m https://huggingface.co/blog/smolervlm
There's so much you could do with these developments. Especially combining them together into agentic applications or fine-tuning them on your use case.
- The science team at @huggingface reproduced and open source the seek r1. https://github.com/huggingface/open-r1
- @qwen released a series of models with 1 million token context! https://qwenlm.github.io/blog/qwen2.5-1m/
- SmolVLM got even smaller with completely new variants at 256m and 500m https://huggingface.co/blog/smolervlm
There's so much you could do with these developments. Especially combining them together into agentic applications or fine-tuning them on your use case.
- 1 reply
burtenshawย
posted
an
update
13 days ago
Post
1034
Hey ๐
I'm helping out on some community research to learn about the AI community. If you want to join in the conversation, head over here where I started a community discussion on the most influential model since BERT.
OSAIResearchCommunity/README#2
I'm helping out on some community research to learn about the AI community. If you want to join in the conversation, head over here where I started a community discussion on the most influential model since BERT.
OSAIResearchCommunity/README#2
burtenshawย
posted
an
update
13 days ago
Post
1703
๐ฃ Teachers and Students! Here's a handy quiz app if you're preparing your own study material.
TLDR, It's a quiz that uses a dataset to make questions and save answers
Here's how it works:
- make a dataset of multiple choice questions
- duplicate the space add set the dataset repo
- log in and do the quiz
- submit the questions to create a new dataset
I made this to get ready for the agents course, but I hope it's useful for you projects too!
quiz app burtenshaw/dataset_quiz
dataset with questions burtenshaw/exam_questions
agents course we're working on https://huggingface.co/agents-course
TLDR, It's a quiz that uses a dataset to make questions and save answers
Here's how it works:
- make a dataset of multiple choice questions
- duplicate the space add set the dataset repo
- log in and do the quiz
- submit the questions to create a new dataset
I made this to get ready for the agents course, but I hope it's useful for you projects too!
quiz app burtenshaw/dataset_quiz
dataset with questions burtenshaw/exam_questions
agents course we're working on https://huggingface.co/agents-course
burtenshawย
posted
an
update
14 days ago
Post
2320
AI was built on side projects!
burtenshawย
posted
an
update
15 days ago
Post
3699
๐ง Work in Progress! ๐ง
๐ทโโ๏ธ We're working hard on getting the official agents course ready for the 50,000 students that have signed up.
If you want to contribute to the discussion, I started these community posts. Looking forward to hearing from you:
- smolagents unit in the agents course - agents-course/README#7
- LlamaIndex Unit in the agents course - agents-course/README#6
- LangChain and LangGraph unit in the agents course - agents-course/README#5
- Real world use cases in the agents course - agents-course/README#8
๐ทโโ๏ธ We're working hard on getting the official agents course ready for the 50,000 students that have signed up.
If you want to contribute to the discussion, I started these community posts. Looking forward to hearing from you:
- smolagents unit in the agents course - agents-course/README#7
- LlamaIndex Unit in the agents course - agents-course/README#6
- LangChain and LangGraph unit in the agents course - agents-course/README#5
- Real world use cases in the agents course - agents-course/README#8
davidberenstein1957ย
posted
an
update
16 days ago
Post
1863
The RAG's in the bag!
You can now use the Synthetic Data Generator with your own domain-specific seed data to generate a dataset for fine-tuning retrieval or reranking models.
GitHub: https://buff.ly/49IDSmd
Spaces: https://buff.ly/3Y1S99z
Blog: https://huggingface.co/blog/sdiazlor/fine-tune-modernbert-for-rag-with-synthetic-data
You can now use the Synthetic Data Generator with your own domain-specific seed data to generate a dataset for fine-tuning retrieval or reranking models.
GitHub: https://buff.ly/49IDSmd
Spaces: https://buff.ly/3Y1S99z
Blog: https://huggingface.co/blog/sdiazlor/fine-tune-modernbert-for-rag-with-synthetic-data
- 1 reply
davidberenstein1957ย
updated
a
Space
17 days ago
davidberenstein1957ย
posted
an
update
20 days ago
Post
1240
You can now use the "Synthetic Data Generator" at a much larger scale with your preferred inference engine: Ollama, vLLM, TGI, and serverless inference! ๐ฅ
Install, configure, launch!
Space: argilla/synthetic-data-generator
Examples: https://github.com/argilla-io/synthetic-data-generator/tree/main/examples
Install, configure, launch!
Space: argilla/synthetic-data-generator
Examples: https://github.com/argilla-io/synthetic-data-generator/tree/main/examples
burtenshawย
posted
an
update
20 days ago
Post
1384
Playing with agents, and I reckon Gradio spaces make the perfect agent tools! So I wrote this guide using Gradio and smolagents:
https://huggingface.co/blog/burtenshaw/gradio-spaces-agent-tools
https://huggingface.co/blog/burtenshaw/gradio-spaces-agent-tools
burtenshawย
posted
an
update
22 days ago
Post
41232
Weโre launching a FREE and CERTIFIED course on Agents!
We're thrilled to announce the launch of the Hugging Face Agents course on Learn! This interactive, certified course will guide you through building and deploying your own AI agents.
Here's what you'll learn:
- Understanding Agents: We'll break down the fundamentals of AI agents, showing you how they use LLMs to perceive their environment (observations), reason about it (thoughts), and take actions. Think of a smart assistant that can book appointments, answer emails, or even write code based on your instructions.
- Building with Frameworks: You'll dive into popular agent frameworks like LangChain, LlamaIndex and smolagents. These tools provide the building blocks for creating complex agent behaviors.
- Real-World Applications: See how agents are used in practice, from automating SQL queries to generating code and summarizing complex documents.
- Certification: Earn a certification by completing the course modules, implementing a use case, and passing a benchmark assessment. This proves your skills in building and deploying AI agents.
Audience
This course is designed for anyone interested in the future of AI. Whether you're a developer, data scientist, or simply curious about AI, this course will equip you with the knowledge and skills to build your own intelligent agents.
Enroll today and start building the next generation of AI agent applications!
https://bit.ly/hf-learn-agents
We're thrilled to announce the launch of the Hugging Face Agents course on Learn! This interactive, certified course will guide you through building and deploying your own AI agents.
Here's what you'll learn:
- Understanding Agents: We'll break down the fundamentals of AI agents, showing you how they use LLMs to perceive their environment (observations), reason about it (thoughts), and take actions. Think of a smart assistant that can book appointments, answer emails, or even write code based on your instructions.
- Building with Frameworks: You'll dive into popular agent frameworks like LangChain, LlamaIndex and smolagents. These tools provide the building blocks for creating complex agent behaviors.
- Real-World Applications: See how agents are used in practice, from automating SQL queries to generating code and summarizing complex documents.
- Certification: Earn a certification by completing the course modules, implementing a use case, and passing a benchmark assessment. This proves your skills in building and deploying AI agents.
Audience
This course is designed for anyone interested in the future of AI. Whether you're a developer, data scientist, or simply curious about AI, this course will equip you with the knowledge and skills to build your own intelligent agents.
Enroll today and start building the next generation of AI agent applications!
https://bit.ly/hf-learn-agents
- 23 replies
davidberenstein1957ย
posted
an
update
23 days ago
Post
2100
๐ฆ What? The Hub as a vector search backend!
code: https://gist.github.com/davidberenstein1957/f0157a471ec59d9dd44ae6957f1d52ec
build on DuckDB: https://huggingface.co/docs/hub/en/datasets-duckdb
code: https://gist.github.com/davidberenstein1957/f0157a471ec59d9dd44ae6957f1d52ec
build on DuckDB: https://huggingface.co/docs/hub/en/datasets-duckdb
davidberenstein1957ย
posted
an
update
about 1 month ago
Post
1941
Fine-tune a SmolLM on domain-specific synthetic data from a LLM
Blog: https://huggingface.co/blog/davidberenstein1957/fine-tune-a-smollm-on-synthetic-data-of-llm
Blog: https://huggingface.co/blog/davidberenstein1957/fine-tune-a-smollm-on-synthetic-data-of-llm
- 1 reply
davidberenstein1957ย
posted
an
update
about 1 month ago
Post
2012
Fine-tuning ModernBERT for text classification using synthetic data generation
From prompt to model in 3 steps.
1 dataset description
20 minutes of generating
60 minutes of fine-tuning on my Macbook Pro
Tutorial: https://nbsanity.com/static/552eb50cbd91bedb4e5b73fddca2664a/fine-tune-modernbert-classifier.html
From prompt to model in 3 steps.
1 dataset description
20 minutes of generating
60 minutes of fine-tuning on my Macbook Pro
Tutorial: https://nbsanity.com/static/552eb50cbd91bedb4e5b73fddca2664a/fine-tune-modernbert-classifier.html