File size: 283 Bytes
de149e0 7317a5a d53b07a de149e0 7317a5a de149e0 7317a5a |
1 2 3 4 5 6 7 8 9 |
---
library_name: transformers
datasets:
- HuggingFaceFW/fineweb-edu
license: mit
---
This is a GPT-2 (7.3B) model trained in llm.c for 100B tokens with cosine LR on Fineweb-Edu.
A lot more detailed info and observations are here: https://x.com/Yuchenj_UW/status/1817223820589375752 |