natolambert commited on
Commit
e4c66e0
·
verified ·
1 Parent(s): ff42442

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +3 -19
README.md CHANGED
@@ -19,26 +19,10 @@ datasets:
19
  > OLMoE-1B-7B is a Mixture-of-Experts LLM with 1B active and 7B total parameters released in September 2024 (0924). It yields state-of-the-art performance among models with a similar cost (1B) and is competitive with much larger models like Llama2-13B. OLMoE is 100% open-source.
20
 
21
  This information and more can also be found on the [**OLMoE GitHub repository**](https://github.com/allenai/OLMoE).
22
-
23
  - **Paper**: (Soon)
24
-
25
- - **Pretraining**
26
- - [Checkpoints](https://hf.co/allenai/OLMoE-1B-7B-0924)
27
- - [Code](https://github.com/allenai/OLMo/tree/Muennighoff/MoE)
28
- - [Data](https://huggingface.co/datasets/allenai/OLMoE-mix-0924)
29
- - [Logs](https://wandb.ai/ai2-llm/olmoe/reports/OLMoE-1B-7B-0924--Vmlldzo4OTcyMjU3)
30
-
31
- - **SFT (Supervised Fine-Tuning)**
32
- - [Checkpoints](https://huggingface.co/allenai/OLMoE-1B-7B-0924-SFT)
33
- - [Code](https://github.com/allenai/open-instruct/tree/olmoe-sft)
34
- - [Data](https://hf.co/datasets/allenai/tulu-v3.1-mix-preview-4096-OLMoE)
35
- - [Logs](https://github.com/allenai/OLMoE/blob/main/logs/olmoe-sft-logs.txt)
36
-
37
- - **DPO/KTO (Direct Preference Optimization/Kahneman-Tversky Optimization)**
38
- - [Checkpoints](https://huggingface.co/allenai/OLMoE-1B-7B-0924-Instruct)
39
- - [Preference Data](https://hf.co/datasets/allenai/ultrafeedback_binarized_cleaned)
40
- - [DPO code](https://github.com/allenai/open-instruct/tree/olmoe-sft), [KTO code](https://github.com/Muennighoff/kto/blob/master/kto.py)
41
- - [Logs](https://github.com/allenai/OLMoE/blob/main/logs/olmoe-dpo-logs.txt)
42
 
43
  # Use
44
 
 
19
  > OLMoE-1B-7B is a Mixture-of-Experts LLM with 1B active and 7B total parameters released in September 2024 (0924). It yields state-of-the-art performance among models with a similar cost (1B) and is competitive with much larger models like Llama2-13B. OLMoE is 100% open-source.
20
 
21
  This information and more can also be found on the [**OLMoE GitHub repository**](https://github.com/allenai/OLMoE).
 
22
  - **Paper**: (Soon)
23
+ - **Pretraining** [Checkpoints](https://hf.co/allenai/OLMoE-1B-7B-0924), [Code](https://github.com/allenai/OLMo/tree/Muennighoff/MoE), [Data](https://huggingface.co/datasets/allenai/OLMoE-mix-0924) and [Logs](https://wandb.ai/ai2-llm/olmoe/reports/OLMoE-1B-7B-0924--Vmlldzo4OTcyMjU3).
24
+ - **SFT (Supervised Fine-Tuning)** [Checkpoints](https://huggingface.co/allenai/OLMoE-1B-7B-0924-SFT), [Code](https://github.com/allenai/open-instruct/tree/olmoe-sft), [Data](https://hf.co/datasets/allenai/tulu-v3.1-mix-preview-4096-OLMoE) and [Logs](https://github.com/allenai/OLMoE/blob/main/logs/olmoe-sft-logs.txt).
25
+ - **DPO/KTO (Direct Preference Optimization/Kahneman-Tversky Optimization)**, [Checkpoints](https://huggingface.co/allenai/OLMoE-1B-7B-0924-Instruct), [Preference Data](https://hf.co/datasets/allenai/ultrafeedback_binarized_cleaned), [DPO code](https://github.com/allenai/open-instruct/tree/olmoe-sft), [KTO code](https://github.com/Muennighoff/kto/blob/master/kto.py) and [Logs](https://github.com/allenai/OLMoE/blob/main/logs/olmoe-dpo-logs.txt).
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
26
 
27
  # Use
28