Update README.md
Browse files
README.md
CHANGED
|
@@ -24,6 +24,10 @@ pipeline_tag: text-generation
|
|
| 24 |
|
| 25 |
**DNA 1.0 8B Instruct** is a <u>state-of-the-art (**SOTA**)</u> bilingual language model based on Llama architecture, specifically optimized for Korean language understanding and generation, while also maintaining strong English capabilities. The model was developed through a sophisticated process involving model merging via spherical linear interpolation (**SLERP**) with Llama 3.1 8B Instruct, and underwent knowledge distillation (**KD**) using Llama 3.1 405B as the teacher model. It was extensively trained through continual pre-training (**CPT**) with a high-quality Korean dataset. The training pipeline was completed with supervised fine-tuning (**SFT**) and direct preference optimization (**DPO**) to align with human preferences and enhance instruction-following abilities.
|
| 26 |
|
|
|
|
|
|
|
|
|
|
|
|
|
| 27 |
DNA 1.0 8B Instruct was fine-tuned on approximately 7B tokens of carefully curated data and has undergone extensive instruction tuning to enhance its ability to follow complex instructions and engage in natural conversations.
|
| 28 |
|
| 29 |
- **Developed by:** Dnotitia Inc.
|
|
@@ -39,12 +43,6 @@ DNA 1.0 8B Instruct was fine-tuned on approximately 7B tokens of carefully curat
|
|
| 39 |
<p>Try DNA-powered Mnemos Assistant! <a href="https://request-demo.dnotitia.ai/">Beta Open →</a></p>
|
| 40 |
</div>
|
| 41 |
|
| 42 |
-
## Training Procedure
|
| 43 |
-
|
| 44 |
-
<p align="center">
|
| 45 |
-
<img src="assets/training-procedure.png" width="600" style="margin: 40px auto;">
|
| 46 |
-
</p>
|
| 47 |
-
|
| 48 |
## Evaluation
|
| 49 |
|
| 50 |
We evaluated DNA 1.0 8B Instruct against other prominent language models of similar size across various benchmarks, including Korean-specific tasks and general language understanding metrics. More details will be provided in the upcoming <u>Technical Report</u>.
|
|
|
|
| 24 |
|
| 25 |
**DNA 1.0 8B Instruct** is a <u>state-of-the-art (**SOTA**)</u> bilingual language model based on Llama architecture, specifically optimized for Korean language understanding and generation, while also maintaining strong English capabilities. The model was developed through a sophisticated process involving model merging via spherical linear interpolation (**SLERP**) with Llama 3.1 8B Instruct, and underwent knowledge distillation (**KD**) using Llama 3.1 405B as the teacher model. It was extensively trained through continual pre-training (**CPT**) with a high-quality Korean dataset. The training pipeline was completed with supervised fine-tuning (**SFT**) and direct preference optimization (**DPO**) to align with human preferences and enhance instruction-following abilities.
|
| 26 |
|
| 27 |
+
<p align="center">
|
| 28 |
+
<img src="assets/training-procedure.png" width="600" style="margin: 40px auto;">
|
| 29 |
+
</p>
|
| 30 |
+
|
| 31 |
DNA 1.0 8B Instruct was fine-tuned on approximately 7B tokens of carefully curated data and has undergone extensive instruction tuning to enhance its ability to follow complex instructions and engage in natural conversations.
|
| 32 |
|
| 33 |
- **Developed by:** Dnotitia Inc.
|
|
|
|
| 43 |
<p>Try DNA-powered Mnemos Assistant! <a href="https://request-demo.dnotitia.ai/">Beta Open →</a></p>
|
| 44 |
</div>
|
| 45 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 46 |
## Evaluation
|
| 47 |
|
| 48 |
We evaluated DNA 1.0 8B Instruct against other prominent language models of similar size across various benchmarks, including Korean-specific tasks and general language understanding metrics. More details will be provided in the upcoming <u>Technical Report</u>.
|