Sthenno's picture

Sthenno

sthenno

AI & ML interests

To contact me: [email protected]

Recent Activity

liked a model about 5 hours ago
Sakalti/Saka-14B
liked a model 1 day ago
sometimesanotion/KytheraMix-7B-v0.2
liked a dataset 1 day ago
open-thoughts/OpenThoughts-114k
View all activity

Organizations

MLX Community's profile picture Hugging Face Discord Community's profile picture sthenno-com's profile picture

sthenno's activity

reacted to sometimesanotion's post with ๐Ÿ‘ 5 days ago
view post
Post
3093
**Update** Either I had some wrong numbers plugged in to estimate benchmark numbers from comparator, or the benchmark changed. Virtuoso Small v2 at 41.07 average is still very impressive, especially for writing draft copy for business purposes, while Lamarck remains a chatty generalist-reasoning model.

I've felt confident that 14B Qwen finetunes and merges could break the 42.0 average, and Arcee **came close** with https://huggingface.co/arcee-ai/Virtuoso-Small-2. Congratulations to @arcee-ai !

Just two months ago, it was easy to think that 14B had plateaued, that you could have high IFEVAL or high MUSR/MATH/GPQA at 14B, but not both. That barrier is completely shattered. I see a pathway to even better, and Virtuoso Small 2 is a big part of why. Very impressive work. This community would expect no less from Arcee.

Just look at this graph! Keep in mind, my merges here build on the first Virtuoso Small, and *-DS merges build on DeepSeek R1. There are some impressive merges in the pipe!
ยท
replied to sometimesanotion's post 5 days ago
view reply

Congratulations as well! When I first saw the evaluation results for Virtuoso-Small-2, I quickly abandoned the release of "miscii-14b-0130". Although BBH and IFEval were once strengths of the miscii series - I admit that within my limited personal technical capabilities, I was indeed beaten by @arcee-ai ;)

reacted to sometimesanotion's post with ๐Ÿš€ 5 days ago
view post
Post
3093
**Update** Either I had some wrong numbers plugged in to estimate benchmark numbers from comparator, or the benchmark changed. Virtuoso Small v2 at 41.07 average is still very impressive, especially for writing draft copy for business purposes, while Lamarck remains a chatty generalist-reasoning model.

I've felt confident that 14B Qwen finetunes and merges could break the 42.0 average, and Arcee **came close** with https://huggingface.co/arcee-ai/Virtuoso-Small-2. Congratulations to @arcee-ai !

Just two months ago, it was easy to think that 14B had plateaued, that you could have high IFEVAL or high MUSR/MATH/GPQA at 14B, but not both. That barrier is completely shattered. I see a pathway to even better, and Virtuoso Small 2 is a big part of why. Very impressive work. This community would expect no less from Arcee.

Just look at this graph! Keep in mind, my merges here build on the first Virtuoso Small, and *-DS merges build on DeepSeek R1. There are some impressive merges in the pipe!
ยท