GGUFs for MoE-Girl.

Note: Until KoboldCPP merges the latest llama.cpp version, these won't work with it. Use vLLM instead until then, see their page on GGUF

These now work with KoboldCPP 1.76!

Downloads last month: 200

GGUF

Model size

6.92B params

Architecture

olmoe

2-bit

3-bit

4-bit

5-bit

6-bit

8-bit

View +1 file

Inference Providers NEW

This model is not currently available via any of the supported third-party Inference Providers, and HF Inference API was unable to determine this model's library.

Model tree for allura-org/MoE-Girl-1BA-7BT-GGUF

Base model

allenai/OLMoE-1B-7B-0924

Finetuned

allura-org/MoE-Girl-1BA-7BT

Quantized

(3)

this model