Yuichi Tateno

Sorry, there was a mistake in the measurement script.
When I measured it again, I got a result of 99.1% for 512dim.

I'll rewrite the article later. Thank you for pointing this out.

I also find this very strange.

In the case of 512 dims, clustering tasks etc. are good, so there is a possibility that there is a bias in the data between [256:512] for specific information.

Maybe, because the batch size is large at 6144, there is a possibility that a bias has occurred by chance towards the end of the learning.

This is a fantastic approach!

I trained a Static Embedding Japanese model (static-embedding-japanese) by incorporating a large amount of Japanese datasets, and when we compared it using the Japanese Multilingual Text Embedding Benchmark (JMTEB), I were able to achieve scores that were only slightly lower than mE5-small.

• https://huggingface.co/hotchpotch/static-embedding-japanese

JMTEB results

Thank you for publishing such an excellent article.