Install: https://github.com/KONAKONA666/q8_kernels
Usage
import torch
from q8_kernels.models.T5EncoderFP8 import T5EncoderModelFP8
text_encoder = T5EncoderModelFP8.from_pretrained(
"konakona/t5xxl_encoder_fp8", torch_dtype=torch.bfloat16
)
Needed dX for token training(PTI and textual inversion) in LTX.
dX is in bf16, calculations in FP8
Needs Ada GPU
- Downloads last month
- 49
Inference Providers
NEW
This model is not currently available via any of the supported third-party Inference Providers, and
HF Inference API was unable to determine this model鈥檚 pipeline type.