Update README.md
Browse files
README.md
CHANGED
@@ -19,26 +19,10 @@ datasets:
|
|
19 |
> OLMoE-1B-7B is a Mixture-of-Experts LLM with 1B active and 7B total parameters released in September 2024 (0924). It yields state-of-the-art performance among models with a similar cost (1B) and is competitive with much larger models like Llama2-13B. OLMoE is 100% open-source.
|
20 |
|
21 |
This information and more can also be found on the [**OLMoE GitHub repository**](https://github.com/allenai/OLMoE).
|
22 |
-
|
23 |
- **Paper**: (Soon)
|
24 |
-
|
25 |
-
- **
|
26 |
-
|
27 |
-
- [Code](https://github.com/allenai/OLMo/tree/Muennighoff/MoE)
|
28 |
-
- [Data](https://huggingface.co/datasets/allenai/OLMoE-mix-0924)
|
29 |
-
- [Logs](https://wandb.ai/ai2-llm/olmoe/reports/OLMoE-1B-7B-0924--Vmlldzo4OTcyMjU3)
|
30 |
-
|
31 |
-
- **SFT (Supervised Fine-Tuning)**
|
32 |
-
- [Checkpoints](https://huggingface.co/allenai/OLMoE-1B-7B-0924-SFT)
|
33 |
-
- [Code](https://github.com/allenai/open-instruct/tree/olmoe-sft)
|
34 |
-
- [Data](https://hf.co/datasets/allenai/tulu-v3.1-mix-preview-4096-OLMoE)
|
35 |
-
- [Logs](https://github.com/allenai/OLMoE/blob/main/logs/olmoe-sft-logs.txt)
|
36 |
-
|
37 |
-
- **DPO/KTO (Direct Preference Optimization/Kahneman-Tversky Optimization)**
|
38 |
-
- [Checkpoints](https://huggingface.co/allenai/OLMoE-1B-7B-0924-Instruct)
|
39 |
-
- [Preference Data](https://hf.co/datasets/allenai/ultrafeedback_binarized_cleaned)
|
40 |
-
- [DPO code](https://github.com/allenai/open-instruct/tree/olmoe-sft), [KTO code](https://github.com/Muennighoff/kto/blob/master/kto.py)
|
41 |
-
- [Logs](https://github.com/allenai/OLMoE/blob/main/logs/olmoe-dpo-logs.txt)
|
42 |
|
43 |
# Use
|
44 |
|
|
|
19 |
> OLMoE-1B-7B is a Mixture-of-Experts LLM with 1B active and 7B total parameters released in September 2024 (0924). It yields state-of-the-art performance among models with a similar cost (1B) and is competitive with much larger models like Llama2-13B. OLMoE is 100% open-source.
|
20 |
|
21 |
This information and more can also be found on the [**OLMoE GitHub repository**](https://github.com/allenai/OLMoE).
|
|
|
22 |
- **Paper**: (Soon)
|
23 |
+
- **Pretraining** [Checkpoints](https://hf.co/allenai/OLMoE-1B-7B-0924), [Code](https://github.com/allenai/OLMo/tree/Muennighoff/MoE), [Data](https://huggingface.co/datasets/allenai/OLMoE-mix-0924) and [Logs](https://wandb.ai/ai2-llm/olmoe/reports/OLMoE-1B-7B-0924--Vmlldzo4OTcyMjU3).
|
24 |
+
- **SFT (Supervised Fine-Tuning)** [Checkpoints](https://huggingface.co/allenai/OLMoE-1B-7B-0924-SFT), [Code](https://github.com/allenai/open-instruct/tree/olmoe-sft), [Data](https://hf.co/datasets/allenai/tulu-v3.1-mix-preview-4096-OLMoE) and [Logs](https://github.com/allenai/OLMoE/blob/main/logs/olmoe-sft-logs.txt).
|
25 |
+
- **DPO/KTO (Direct Preference Optimization/Kahneman-Tversky Optimization)**, [Checkpoints](https://huggingface.co/allenai/OLMoE-1B-7B-0924-Instruct), [Preference Data](https://hf.co/datasets/allenai/ultrafeedback_binarized_cleaned), [DPO code](https://github.com/allenai/open-instruct/tree/olmoe-sft), [KTO code](https://github.com/Muennighoff/kto/blob/master/kto.py) and [Logs](https://github.com/allenai/OLMoE/blob/main/logs/olmoe-dpo-logs.txt).
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
26 |
|
27 |
# Use
|
28 |
|