|
--- |
|
license: other |
|
license_name: yi-license |
|
license_link: LICENSE |
|
datasets: |
|
- jondurbin/airoboros-3.1 |
|
--- |
|
|
|
|
|
|
|
# Instruction tune of Yi-34b-200k with Airoboros-3.1 (fp16) |
|
|
|
|
|
## Overview |
|
|
|
This is [larryvrh/Yi-34B-200K-Llamafied](https://huggingface.co/larryvrh/Yi-34B-200K-Llamafied), with instruction tuning performed with Jon Durbin's [jondurbin/airoboros-3.1](https://huggingface.co/datasets/jondurbin/airoboros-3.1) dataset. That base model is [01-ai/Yi-34B-200k](https://huggingface.co/01-ai/Yi-34B-200k), but using llama2 model definitions and tokenizer to remove any remote code requirements. |
|
|
|
**This is a (merged) QLoRA fine-tune (rank 64)**. |
|
|
|
The finetune was performed with 1x RTX 6000 Ada (~80 hours to this checkpoint). Prompts were truncated to 4096 tokens (for speed and VRAM headroom). |
|
|
|
I have done very little testing with this model, so feedback on real world performance is appreciated! |
|
|
|
## How to Use |
|
|
|
Use as you would any other Hugging Face fp16 llama-2 model. |
|
|
|
## Prompting: |
|
|
|
Model was trained with llama-2 chat prompt format. See [jondurbin/airoboros-l2-13b-3.1.1](https://huggingface.co/jondurbin/airoboros-l2-13b-3.1.1) model card for details. |
|
|
|
|
