falcon-40b-instruct quantized with GPTQ using the script in https://github.com/huggingface/text-generation-inference/pull/438

group size: 128
act order: true
nsamples: 128
dataset: wikitext2

Downloads last month: 14

Safetensors

Model size

6.53B params

Tensor type

I64

I32

FP16

Inference Providers NEW

Text Generation

This model is not currently available via any of the supported third-party Inference Providers, and HF Inference API does not yet support model repos that contain custom code.