calcuis

/

lumina-gguf

like 8

upgraded your comfyui as well as the node to the latest version? code support just added since

you could try the gemma-2-2b-fp16 version here; tested, works fine

Tried upgrading, still don't work. I can (try) paste log here.

Good day.
Is there's a way to make model work with quantized Gemma 2? When I run it, it just says - "ClipLoaderGGUF Unknown architecture: 'gemma2'"
I tried running finetuned version with Q_8 quant.
Also, Lumina was Q_8 too.

oh, i see; did you quantize the gemma-2-2b-fp16 to fp8 or gguf or etc.? it won't work in that case; still sorting out right away; use the fp16 safetensors, should work

No, I downloaded already quantized finetune by bartowski (abliterated, Q_8). Send a log?

never test the one (q_8) you mentioned; we don't think it works actually; any link/source; the log won't help

Alright. Thanks for help

since it was for the text-generation; different format; it works for llama.cpp related connector(s) but doesn't work as text encoder for image model

Same issue. I have it working fine with the fp16 safetensors file, but I'd like to be able to use it with a gguf. I don't see why that wouldn't be possible.

it's possible but takes time to re-format it

It would be awesome to see how this version might work with it:
https://huggingface.co/bartowski/gemma-2-2b-it-abliterated-GGUF

I'd guess the weights are similar enough that it should be fine. But if 9b could also work, that would be great to see, although it seems like that might not be possible.

It would be awesome to see how this version might work with it:
https://huggingface.co/bartowski/gemma-2-2b-it-abliterated-GGUF

I'd guess the weights are similar enough that it should be fine. But if 9b could also work, that would be great to see, although it seems like that might not be possible.

I will do some tests to see if this will work. I wonder if this will make the model generate more NSFW images.

It would be awesome to see how this version might work with it:
https://huggingface.co/bartowski/gemma-2-2b-it-abliterated-GGUF

I'd guess the weights are similar enough that it should be fine. But if 9b could also work, that would be great to see, although it seems like that might not be possible.

I will do some tests to see if this will work. I wonder if this will make the model generate more NSFW images.

Nope. It doesn't work. Unfortunately.

code support added; but might need to wait for the upper level upgrade since gemma2 is not a typical model, the prompt will keep looping inside

It would be awesome to see how this version might work with it:
https://huggingface.co/bartowski/gemma-2-2b-it-abliterated-GGUF

I'd guess the weights are similar enough that it should be fine. But if 9b could also work, that would be great to see, although it seems like that might not be possible.

I will do some tests to see if this will work. I wonder if this will make the model generate more NSFW images.

In theory it could. Issue is that in order to make custom Gemma 2 models work, you need to set specific LLM settings (meaning you need to be able to set temperature, top_k and so on) cause otherwise those models are usually useless. There are models that work "as they are", but its pretty rare. Regular abliterated and unfixed model is mostly just lobotomized and impossible to work with.