Trained on a flavorful melange of the WizardLM, Airoboros, and Wizard Vicuna datasets.
This model was trained using both linear and NTK-aware RoPE scaling in tandem. When loading, ensure that compress_pos_emb (or scale) is set to 2, and alpha_value is set to 4. Both values must be set.
Expect context length of up to 8192 to work for sure. It will probably maintain coherence into the ~12k range, but I have not tested that.
Prompt format is vicuna 1.1:
<whatever nonsense system prompt you want>
USER: ...
ASSISTANT: ...
- Downloads last month
- 7
Inference Providers
NEW
This model is not currently available via any of the supported third-party Inference Providers, and
the model is not deployed on the HF Inference API.