SmolLM2: When Smol Goes Big -- Data-Centric Training of a Small Language Model
Paper
β’
2502.02737
β’
Published
β’
61
Unofficial Mistral Community
community
AI & ML interests
Unofficial org for community upload of Mistral's Open Source models.
Recent Activity
mistral-community's activity
reach-vbΒ
authored
a
paper
about 1 hour ago
Getting shape mismatch while loading saved Pixtral model
2
#24 opened 2 days ago
by
ss007
Post
1332
GPT4chan Series Release
GPT4chan is a series of models I trained on v2ray/4chan dataset, which is based on lesserfield/4chan-datasets. The dataset contains mostly posts from 2023. Not every board is included, for example, /pol/ is NOT included. To see which boards are included, visit v2ray/4chan.
This release contains 2 models sizes, 8B and 24B. The 8B model is based on meta-llama/Llama-3.1-8B and the 24B model is based on mistralai/Mistral-Small-24B-Base-2501.
Why I made these models? Because for a long time after the original gpt-4chan model, there aren't any serious fine-tunes on 4chan datasets. 4chan is a good data source since it contains coherent replies and nice topics. It's fun to talk to an AI generated version of 4chan and get instant replies, and without the need to actually visit 4chan. You can also sort of analyze the content and behavior of 4chan posts by probing the model's outputs.
Disclaimer: The GPT4chan models should only be used for research purposes, the outputs they generated do not represent the view of me on the subjects. Moderate the responses before sending it online.
Model links:
Full model:
- v2ray/GPT4chan-8B
- v2ray/GPT4chan-24B
Adapter:
- v2ray/GPT4chan-8B-QLoRA
- v2ray/GPT4chan-24B-QLoRA
AWQ:
- v2ray/GPT4chan-8B-AWQ
- v2ray/GPT4chan-24B-AWQ
FP8:
- v2ray/GPT4chan-8B-FP8
GPT4chan is a series of models I trained on v2ray/4chan dataset, which is based on lesserfield/4chan-datasets. The dataset contains mostly posts from 2023. Not every board is included, for example, /pol/ is NOT included. To see which boards are included, visit v2ray/4chan.
This release contains 2 models sizes, 8B and 24B. The 8B model is based on meta-llama/Llama-3.1-8B and the 24B model is based on mistralai/Mistral-Small-24B-Base-2501.
Why I made these models? Because for a long time after the original gpt-4chan model, there aren't any serious fine-tunes on 4chan datasets. 4chan is a good data source since it contains coherent replies and nice topics. It's fun to talk to an AI generated version of 4chan and get instant replies, and without the need to actually visit 4chan. You can also sort of analyze the content and behavior of 4chan posts by probing the model's outputs.
Disclaimer: The GPT4chan models should only be used for research purposes, the outputs they generated do not represent the view of me on the subjects. Moderate the responses before sending it online.
Model links:
Full model:
- v2ray/GPT4chan-8B
- v2ray/GPT4chan-24B
Adapter:
- v2ray/GPT4chan-8B-QLoRA
- v2ray/GPT4chan-24B-QLoRA
AWQ:
- v2ray/GPT4chan-8B-AWQ
- v2ray/GPT4chan-24B-AWQ
FP8:
- v2ray/GPT4chan-8B-FP8
JustinLin610Β
authored
a
paper
9 days ago
JustinLin610Β
authored
a
paper
10 days ago
Post
6906
AI is not a zero-sum game. Open-source AI is the tide that lifts all boats!
RaushanTurganbayΒ
updated
a
model
10 days ago
We are working on creating a single 22b from this model
21
#5 opened 10 months ago
by
rombodawg
mrfakenameΒ
posted
an
update
14 days ago
Post
1073
Iβm excited to introduce a new leaderboard UI + keyboard shortcuts on the TTS Arena!
The refreshed UI for the leaderboard is smoother and (hopefully) more intuitive. You can now view models based on a simpler win-rate percentage and exclude closed models.
In addition, the TTS Arena now supports keyboard shortcuts. This should make voting much more efficient as you can now vote without clicking anything!
In both the normal Arena and Battle Mode, press "r" to select a random text, Cmd/Ctrl + Enter to synthesize, and "a"/"b" to vote! View more details about keyboard shortcuts by pressing "?" (Shift + /) on the Arena.
Check out all the new updates on the TTS Arena:
TTS-AGI/TTS-Arena
The refreshed UI for the leaderboard is smoother and (hopefully) more intuitive. You can now view models based on a simpler win-rate percentage and exclude closed models.
In addition, the TTS Arena now supports keyboard shortcuts. This should make voting much more efficient as you can now vote without clicking anything!
In both the normal Arena and Battle Mode, press "r" to select a random text, Cmd/Ctrl + Enter to synthesize, and "a"/"b" to vote! View more details about keyboard shortcuts by pressing "?" (Shift + /) on the Arena.
Check out all the new updates on the TTS Arena:
TTS-AGI/TTS-Arena
- 1 reply
Is the chat template correct? (issue for vLLM)
7
#22 opened about 1 month ago
by
MichaelAI23
JustinLin610Β
authored
a
paper
15 days ago
danielhanchenΒ
posted
an
update
17 days ago
Post
2342
I uploaded DeepSeek R1 GGUFs!
unsloth/DeepSeek-R1-Distill-Llama-8B-GGUF
unsloth/DeepSeek-R1-Distill-Llama-70B-GGUF
2bit for MoE: unsloth/DeepSeek-R1-GGUF
unsloth/DeepSeek-R1-Zero-GGUF
More at unsloth/deepseek-r1-all-versions-678e1c48f5d2fce87892ace5
unsloth/DeepSeek-R1-Distill-Llama-8B-GGUF
unsloth/DeepSeek-R1-Distill-Llama-70B-GGUF
2bit for MoE: unsloth/DeepSeek-R1-GGUF
unsloth/DeepSeek-R1-Zero-GGUF
More at unsloth/deepseek-r1-all-versions-678e1c48f5d2fce87892ace5
JustinLin610Β
authored
a
paper
23 days ago
JustinLin610Β
authored
a
paper
24 days ago
danielhanchenΒ
posted
an
update
27 days ago
Post
4634
We fixed many bugs in Phi-4 & uploaded fixed GGUF + 4-bit versions! β¨
Our fixed versions are even higher on the Open LLM Leaderboard than Microsoft's!
GGUFs: unsloth/phi-4-GGUF
Dynamic 4-bit: unsloth/phi-4-unsloth-bnb-4bit
You can also now finetune Phi-4 for free on Colab: https://colab.research.google.com/github/unslothai/notebooks/blob/main/nb/Phi_4-Conversational.ipynb
Read our blogpost for more details on bug fixes etc: https://unsloth.ai/blog/phi4
Our fixed versions are even higher on the Open LLM Leaderboard than Microsoft's!
GGUFs: unsloth/phi-4-GGUF
Dynamic 4-bit: unsloth/phi-4-unsloth-bnb-4bit
You can also now finetune Phi-4 for free on Colab: https://colab.research.google.com/github/unslothai/notebooks/blob/main/nb/Phi_4-Conversational.ipynb
Read our blogpost for more details on bug fixes etc: https://unsloth.ai/blog/phi4
JustinLin610Β
authored
a
paper
27 days ago
danielhanchenΒ
posted
an
update
30 days ago
Post
3133
Deepseek V3, including GGUF + bf16 versions are now uploaded!
Includes 2, 3, 4, 5, 6 and 8-bit quantized versions.
GGUFs: unsloth/DeepSeek-V3-GGUF
bf16: unsloth/DeepSeek-V3-bf16
Min. hardware requirements to run: 48GB RAM + 250GB of disk space for 2-bit.
See how to run them with examples and the full collection: unsloth/deepseek-v3-all-versions-677cf5cfd7df8b7815fc723c
Includes 2, 3, 4, 5, 6 and 8-bit quantized versions.
GGUFs: unsloth/DeepSeek-V3-GGUF
bf16: unsloth/DeepSeek-V3-bf16
Min. hardware requirements to run: 48GB RAM + 250GB of disk space for 2-bit.
See how to run them with examples and the full collection: unsloth/deepseek-v3-all-versions-677cf5cfd7df8b7815fc723c
JustinLin610Β
authored
a
paper
about 1 month ago
Post
4252
Cool to see
@ylecun
joining the top 10 of most followed on HF!
(and leaderboard by @mvaloatto is here: mvaloatto/TCTF)
(and leaderboard by @mvaloatto is here: mvaloatto/TCTF)
- 2 replies