Spaces:
Running
on
CPU Upgrade
Resubmitting a model to use `chat_template` doesn't re-evaluate, but does change `chat_template` column
I tried to re-evaluate rombodawg/Rombos-LLM-V2.5-Qwen-72b so that it can be properly compared to Qwen/Qwen2.5-72B-Instruct but it didn't re-evaluate, and now it incorrectly states that it evaluated using it. Can this be fixed, and can the model properly be resubmitted for evaluation using the chat_template?
this looks like a potential problem i mentioned or guessed might happen here (i'm sorry, i might have not been clear about it that time, maybe that problem is a thing?),
or is this due to recent new UI and internal code changes to the leaderboard?
Yep looks like a bug we'll need to fix - it's normal that you can't re-evaluate by resubmitting the same model ,(it's to avoid spamming the leaderboard), as indicated in the FAQ, but it's an issue that the request file changes.
If maybe one of you has some time to take a look that would be great