Persistent RuntimeError: all inputs to slice must contain the same amount of layers

#39
by Blazgo - opened

I keep getting RuntimeError when merging models!! I tried so many things! slerp with qwen and calme... llama and qwen, qwen 1m with TIES... I don't get it! I want to ideally merge:

Qwen/Qwen2.5-14B-Instruct-1M
MaziyarPanahi/calme-3.2-instruct-78b
meta-llama/Llama-3.1-70B-Instruct
Steelskull/L3.3-MS-Nevoria-70b
Sakalti/ultiima-72B

Ik thats a lot of models, and I don't mind dropping one or two. But I can't even merge 2!! Also I want to merge them to make sure that the best model is selected by task (e.g. ultiima is good at math so choose it for math questions).

Sign up or log in to comment