Note Will describe music mood

Note Optional: will separate different audio tracks; used here to get song voice only which is then passed through whisper

Note Whisper will transcribe lyrics

meta-llama/Llama-2-7b

Text Generation • Updated Apr 17, 2024 • 4.23k

Note Llama is the major part: will use LP-Music-Cap + optional lyrics transcription to write an image description that should match your music input, according to the previous steps

stabilityai/stable-diffusion-xl-base-1.0

Text-to-Image • Updated Oct 30, 2023 • 2.76M • • 6.26k

Note Llama just gave an image description, use it to generate an image with SDXL model