sometimesanotion PRO

Okay, this has become a major component of how I build model_stocks that keep IFEVAL high even while merging distantly related models, and this is the reason for some TIES merges to "qwenvergify" models you might have seen.

Here's the basic idea:
https://www.arcee.ai/blog/use-mergekit-to-extract-lora-adapters-from-any-fine-tuned-model

But not as many models are inter-compatible for LoRAs as you'd expect, because there are minor variations in size among some important finetunes. I get the train tracks to a standard width, as it were, and make them intercompatible with the "qwenvergify" TIES merges between two models, weight 1.0 for the model of interest and weight 0.0 for any Qwenvergence or Lamarck model for the tiny bit of infill. You now have all models intercompatible for what is akin to a super-high-precision DELLA merge of the most significant parts of the model, the most IFEVAL-preserving parts of the model. A rank 512 adapter extracts around 30% of the most defining aspects of the model, but captures around 90% of its distinct performance. A rank 128 adapter captures around 8% of the model, but about 70% of its distinct performance.

I arrived at this while thinking about the implication of @rombodawg 's "Continuous Fine Tuning" strategy, and reading I-forget-which-arxiv-paper and I really need to find that again. It's like the opposite side of the coin from how rombodawg uses it. I use it at the beginning to get a large model_stock started. He uses it to extract most of your merge at the end and apply it to a target model to avoid catastrophic forgetting.

There. Now you know the methodology behind my merge YAML that produced https://huggingface.co/sometimesanotion/Qwenvergence-14B-v13-Prose-DS - or, the model that calls itself "Qwenconceited-14B-v13-DeepSuffering". 😆

Adapters from a strong IFEVAL+BBH model applied to the majority of the models in the model_stock merge, in a mixture of rank sizes between 32 and 128, get them on the same page for core operation. Applying a Virtuoso or Chocolatine-based LoRA to just any model out there could cause instability, but the model_stock smooths many varying levels of adapter merges out.

That's enough for you to digest for now, and @rombodawg might be interested to know he inspired such a different strategy from anything he's shared.

You can reach me on Discord, my username is as you'd expect.

Once I show you how Qwentinuum broke the barrier and finally got stabilized, and made Vimarckoso v3, you'll see why I'm being a little careful. It takes multiple steps to reliably tame weighty breadcrumbs merges, and I'm using Makefiles to make sure nothing gets skipped. That's not so easily posted to a modelcard! If people misuse parts of my recipe, especially with more CoT models out there, we'll get spammed with a lot of unstable models.

But the rewards of getting it right!

@Inschrift-Spruch-Raum , I have a treat for you. I'm glad you liked v12 of Qwenvergence Prose - but I've struck gold. 13 just might be your lucky number!
https://huggingface.co/sometimesanotion/Qwenvergence-14B-v13-Prose-DS

I've really been pondering that, and it's almost certainly because of the blend of R1 and Krystalan/DRT-o1-14B. We have two different CoT lineages feeding into one model - wonderful, until it's not! DRT is a bit hard to give up. I think this is where we finally have done all we can do with merging, however fancy, and get down to fine-tuning, because if DRT and DS's influences sync up, it'll be magic.

I've spilled some of the beans in little separate doses, because I've hoped to prompt people to fill in the blanks with unique ideas rather than inspire a lot of copypasta. There's a lot of stuff that is just unique to my own workflow, but there's also some reaaaaally long and detailed YAML.

I do feel that what happens between the model_stock + LoRAs and the SLERP+TIES has been loosely described. It really is just a bit of general info about which layers influence what metric, like multiple gradients overlaid. I tend to keep densities under 40 or even 30, because if there's a strong core model, each extra model needs to leave headroom for the others.

Hit me up, though, I'm particularly grateful for your contribution!

While Arcee beats Lamarck 0.7 and
tempesthenno-ppo-ckpt40 for IFEVAL, BBH, and MATH, you score 23.55% higher on GPQA, 1.96% higher on MUSR, and 2.49% higher on MUSR than Virtuoso Small v2.

Plus, I'm thinking you fine-tune for use cases Arcee and I don't.

@Inschrift-Spruch-Raum is right. My estimates based off of comparator are only partially correct. Are percentile calculations on the leaderboard changing? Regardless, this graph shows why nobody needs to give up on their models, especially when each one's making a specialized contribution. Diversity is a benefit to us all.

I really like how this class of models proves that MUSR isn't just what you get when you throw IFEVAL into a blender. 😆

Wow! I have to check what I saw in comparator and based estimates off of. There's no doubt that Virtuoso Small v2 is a great model, and I'm already working on a Qwenvergence based on it. It's as awesome at IFEVAL and BBH as I'd thought.

Qwenvergence is the model_stock that makes the bases to blend in varying proportions across Lamarck's layers. Yet, it's not mere raw material. I'm getting really outstanding results out of Qwenvergence-14B-v12-Prose-DS's successor which includes Virtuoso Small v2. It's playing very nicely with the other components!

My high-benchmarking merges have included Virtuoso v1 at nearly every stage, and I am now creating a new generation switching in V2 where apt.

Feedback from finetuners suggests my minimal compute and Arcee's MergeKit have given them a shortcut to great results. Smart merging really is energy efficient. Thank you for helping us push the limits!

Any model of yours made to a purpose beyond benchmarks has a reason unto itself. Your tempesthenno-ppo-ckpt40 does neat things. I've also found surprise pops in benchmarks for merges when two models that have similar scores arrive at it in different and complementary ways.

Not gonna lie, my merge strategy for Lamarck v0.8 was made with the expectation of 3-4 models with different strengths, and the combination of IFEVAL, BBH, MATH, and CoT in Virtuoso-Small-v2 is forcing me to look hard at that.

You helped this project get started, validating merge methods. Thank you!