Alpha-VLLM

/

Lumina-Image-2.0

like 150

Follow

Alpha-VLLM 106

i am getting the same error, any leads?

Based on this pr, specifically the mention of the 1.5x (6144 * 1.5 == 9216) and the link to the working spaces demo, I found the transformer config (locally for me ~/.cache/huggingface/hub/{model name}/snapshots/{hash}/transformer/config.json) and set the value to 1.5 and I was able to get past this error.

However, I got a new one that I suspect may be related to the second comment in the PR description, but I'm unsure where to replicate the changes mentioned.

sorry for the confusion. I have updated the diffuser pr and upload new hf checkpoint to fix this issue

erm... you said you have "uploaded new hf checkpoints", but the files actually on hf have not changed since I posted this.
The pth files I just downloaded have the same checksum as the ones I downloaded yesterday.

@ppbrown Thank you very much for your interest in our project. Please check the transformer folder, as we have already updated it.

I'm also confused, the config file still has null instead of 1.5 which is what seems to work for the fork@branch of diffusers.

In any case, I was able to get an image generated by commenting out the cfg_trunc_ratio argument in the pipe fn call from the README example which seems to solve the second issue I was having related to l_effective_img_len index error.

Ah, I see. Only the subfolders need to be updated, if I use the "pipe" libraries.
okay... but its still broken.

I removed diffusers, and re-ran
pip install git+https://github.com/painebenjamin/diffusers.git@lumina2

Installing collected packages: diffusers
Successfully installed diffusers-0.33.0.dev0

I then ran the code... I watched it download "new" models... but I still get

ValueError: Trying to set a tensor of shape torch.Size([9216, 2304]) in "weight" (which has shape torch.Size([6144, 2304])), this looks incorrect.

PS: I tried the "comment out cfg_trunc_ratio ", but it still threw same error for me.

https://github.com/painebenjamin/diffusers.git@lumina2 is not the official implementation, you should try this one: https://github.com/zhuole1025/diffusers

Ah, good to know.

in that case, can I suggest that

1. the README here gets updated with that info

2. https://huggingface.co/Alpha-VLLM/Lumina-Image-2.0/discussions/1
   get updated with that info

Also... when I try using the URL you just mentioned... I now get the same issue mentioned in that OTHER thread:

"Traceback (most recent call last):
File "/data/Lumina-Image-2.0/./lumina_hf", line 4, in
from diffusers import Lumina2Text2ImgPipeline
ImportError: cannot import name 'Lumina2Text2ImgPipeline' from 'diffusers' (/data/Lumina-Image-2.0/git-lumina/lib/python3.10/site-packages/diffusers/init.py)"

and if I tack "@lumina2"on the end of that one... I get yet OTHER errors.

Could a dev please try running the script given in the README here from a fresh environment, fix all the errors, and then update the instructions to be accurate, pretty-please?
I'll mention that at minimum, the requirements.txt in github needs to be updated, since it is missing "normalization"

Or better yet, create a "requirements.txt" file here, if the modules are actually different from the github style implementation?

We apologize for the confusion. We are currently preparing detailed instructions for setting up the environment and will notify you once they are ready.

@0xbarnji @fareh01
Currently, the official Diffusers repository has not merged our code yet. However, you can directly install Diffusers by running:

pip install git+https://github.com/zhuole1025/diffusers@lumina2

Once installed, you can use the inference code we provided, and it will run normally.

@0xbarnji @fareh01
Currently, the official Diffusers repository has not merged our code yet. However, you can directly install Diffusers by running:

pip install git+https://github.com/zhuole1025/diffusers@lumina2

Once installed, you can use the inference code we provided, and it will run normally.

Working for me. Thanks :-)

@poluramus could you please detail exactly how you are using it?

as for me, I did the following RIGHT NOW:

 (save the code in https://huggingface.co/Alpha-VLLM/Lumina-Image-2.0/blob/main/README.md to test.py)
python -m venv venv
. venv/bin/activate
pip install torch
pip install git+https://github.com/zhuole1025/diffusers@lumina2

$ python test.py
Traceback (most recent call last):
  File "/data/Lumina_hf/test.py", line 2, in <module>
    from diffusers import Lumina2Text2ImgPipeline
ImportError: cannot import name 'Lumina2Text2ImgPipeline' from 'diffusers' (/data/Lumina_hf/lumina_hf/lib/python3.10/site-packages/diffusers/__init__.py)

@poluramus could you please detail exactly how you are using it?

as for me, I did the following RIGHT NOW:

 (save the code in https://huggingface.co/Alpha-VLLM/Lumina-Image-2.0/blob/main/README.md to test.py)
python -m venv venv
. venv/bin/activate
pip install torch
pip install git+https://github.com/zhuole1025/diffusers@lumina2

$ python test.py
Traceback (most recent call last):
  File "/data/Lumina_hf/test.py", line 2, in <module>
    from diffusers import Lumina2Text2ImgPipeline
ImportError: cannot import name 'Lumina2Text2ImgPipeline' from 'diffusers' (/data/Lumina_hf/lumina_hf/lib/python3.10/site-packages/diffusers/__init__.py)

I can't really since i did too much tests to try to make it work 3 or 4 days ago... But you can try to install the packages from the requirements.txt from this page https://huggingface.co/Alpha-VLLM/Lumina-Image-2.0/discussions/1. But use the diffuser mention abave insead: pip install git+https://github.com/zhuole1025/diffusers@lumina2

Also, cuda wasn't working at the time with the settup, so i installed torch using https://pytorch.org/get-started/locally/. I'm not sure if i did something wrong :-D

I'll try with a fresh new conda env later, see if I can make it work quickly. Let me know if you can get it.

Comfyui: https://comfyanonymous.github.io/ComfyUI_examples/lumina2/