SmolLM2: When Smol Goes Big -- Data-Centric Training of a Small Language Model
Paper
•
2502.02737
•
Published
•
58
Need4Speed
company
AI & ML interests
None defined yet.
Recent Activity
View all activity
need-for-speed's activity
loubnabnl
authored
a
paper
about 1 hour ago
lvwerra
authored
a
paper
about 4 hours ago
lvwerra
authored
a
paper
21 days ago
Post
2330
Are we the only providers of INT4 quantized models for Llama 3.2 VL?
OPEA/Llama-3.2-90B-Vision-Instruct-int4-sym-inc
OPEA/Llama-3.2-11B-Vision-Instruct-int4-sym-inc
OPEA/Llama-3.2-90B-Vision-Instruct-int4-sym-inc
OPEA/Llama-3.2-11B-Vision-Instruct-int4-sym-inc
- 3 replies
Post
1817
AutoRound has demonstrated strong results even at 2-bit precision for VLM models like QWEN2-VL-72B. Check it out here:
OPEA/Qwen2-VL-72B-Instruct-int2-sym-inc.
-
- 4 replies
Post
341
This week, OPEA Space released several new INT4 models, including:
nvidia/Llama-3.1-Nemotron-70B-Instruct-HF
allenai/OLMo-2-1124-13B-Instruct
THUDM/glm-4v-9b
AIDC-AI/Marco-o1
and several others.
Let us know which models you'd like prioritized for quantization, and we'll do our best to make it happen!
https://huggingface.co/OPEA
nvidia/Llama-3.1-Nemotron-70B-Instruct-HF
allenai/OLMo-2-1124-13B-Instruct
THUDM/glm-4v-9b
AIDC-AI/Marco-o1
and several others.
Let us know which models you'd like prioritized for quantization, and we'll do our best to make it happen!
https://huggingface.co/OPEA
-
-
- 3 replies
Haihao
authored
a
paper
2 months ago
Post
985
OPEA space just releases nearly 20 int4 models, for example, QWQ-32B-Preview,
Llama-3.2-11B-Vision-Instruct, Qwen2.5, Llama3.1, etc. Check out https://huggingface.co/OPEA
Llama-3.2-11B-Vision-Instruct, Qwen2.5, Llama3.1, etc. Check out https://huggingface.co/OPEA
Post
2226
Making SmolLM2 reproducible: open-sourcing our training & evaluation toolkit 🛠️ https://github.com/huggingface/smollm/
- Pre-training code with nanotron
- Evaluation suite with lighteval
- Synthetic data generation using distilabel (powers our new SFT dataset HuggingFaceTB/smoltalk)
- Post-training scripts with TRL & the alignment handbook
- On-device tools with llama.cpp for summarization, rewriting & agents
Apache 2.0 licensed. V2 pre-training data mix coming soon!
Which other tools should we add next?
- Pre-training code with nanotron
- Evaluation suite with lighteval
- Synthetic data generation using distilabel (powers our new SFT dataset HuggingFaceTB/smoltalk)
- Post-training scripts with TRL & the alignment handbook
- On-device tools with llama.cpp for summarization, rewriting & agents
Apache 2.0 licensed. V2 pre-training data mix coming soon!
Which other tools should we add next?
ofirzaf
authored
2
papers
3 months ago
orenpereg
authored
5
papers
3 months ago
Term Set Expansion based on Multi-Context Term Embeddings: an End-to-end Workflow
Paper
•
1807.10104
•
Published
•
1
ABSApp: A Portable Weakly-Supervised Aspect-Based Sentiment Extraction System
Paper
•
1909.05608
•
Published
Cross-Domain Aspect Extraction using Transformers Augmented with Knowledge Graphs
Paper
•
2210.10144
•
Published
Efficient Few-Shot Learning Without Prompts
Paper
•
2209.11055
•
Published
•
3
Accelerating Speculative Decoding using Dynamic Speculation Length
Paper
•
2405.04304
•
Published
•
2
lvwerra
authored
a
paper
3 months ago
Haihao
authored
3
papers
4 months ago