README.md

metadata

base_model: mistralai/Mixtral-8x22B-v0.1
tags:
  - Mixtral
  - instruct
  - finetune
  - chatml
  - gpt4
  - synthetic data
  - distillation
language:
  - en
license: apache-2.0
datasets:
  - teknium/OpenHermes-2.5

OpenHermes 2.5 - Mixtral 8x22B

Mixtral 8x22B full SFTed on OpenHermes 2.5 dataset (https://huggingface.co/datasets/teknium/OpenHermes-2.5).

Evaluations are still being ran. Download the model from branches 4th-epoch and 3rd-epoch.

Prompt format is ChatML. Refer to https://huggingface.co/teknium/OpenHermes-2.5-Mistral-7B for examples.

Research supported by Google's TPU Research Cloud.