update model card README.md

README.md

@@ -1,28 +1,65 @@
 ---
-language: multilingual
-datasets:
-- mc4
-
-license: apache-2.0
+tags:
+- generated_from_trainer
+metrics:
+- bleu
+model-index:
+- name: mt5-base-en-ru
+  results: []
 ---
 
-[Google's mT5](https://github.com/google-research/multilingual-t5)
+<!-- This model card has been generated automatically according to the information the Trainer had access to. You
+should probably proofread and complete it, then remove this comment. -->
+
+# mt5-base-en-ru
+
+This model was trained from scratch on the None dataset.
+It achieves the following results on the evaluation set:
+- Loss: 0.8401
+- Bleu: 12.2867
+- Gen Len: 17.8712
+
+## Model description
+
+More information needed
+
+## Intended uses & limitations
+
+More information needed
 
-mT5 is pretrained on the [mC4](https://www.tensorflow.org/datasets/catalog/c4#c4multilingual) corpus, covering 101 languages:
+## Training and evaluation data
 
-Afrikaans, Albanian, Amharic, Arabic, Armenian, Azerbaijani, Basque, Belarusian, Bengali, Bulgarian, Burmese, Catalan, Cebuano, Chichewa, Chinese, Corsican, Czech, Danish, Dutch, English, Esperanto, Estonian, Filipino, Finnish, French, Galician, Georgian, German, Greek, Gujarati, Haitian Creole, Hausa, Hawaiian, Hebrew, Hindi, Hmong, Hungarian, Icelandic, Igbo, Indonesian, Irish, Italian, Japanese, Javanese, Kannada, Kazakh, Khmer, Korean, Kurdish, Kyrgyz, Lao, Latin, Latvian, Lithuanian, Luxembourgish, Macedonian, Malagasy, Malay, Malayalam, Maltese, Maori, Marathi, Mongolian, Nepali, Norwegian, Pashto, Persian, Polish, Portuguese, Punjabi, Romanian, Russian, Samoan, Scottish Gaelic, Serbian, Shona, Sindhi, Sinhala, Slovak, Slovenian, Somali, Sotho, Spanish, Sundanese, Swahili, Swedish, Tajik, Tamil, Telugu, Thai, Turkish, Ukrainian, Urdu, Uzbek, Vietnamese, Welsh, West Frisian, Xhosa, Yiddish, Yoruba, Zulu.
+More information needed
 
-**Note**: mT5 was only pre-trained on mC4 excluding any supervised training. Therefore, this model has to be fine-tuned before it is useable on a downstream task.
+## Training procedure
 
-Pretraining Dataset: [mC4](https://www.tensorflow.org/datasets/catalog/c4#c4multilingual)
+### Training hyperparameters
 
-Other Community Checkpoints: [here](https://huggingface.co/models?search=mt5)
+The following hyperparameters were used during training:
+- learning_rate: 0.0004
+- train_batch_size: 14
+- eval_batch_size: 3
+- seed: 42
+- gradient_accumulation_steps: 10
+- total_train_batch_size: 140
+- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
+- lr_scheduler_type: linear
+- num_epochs: 5
 
-Paper: [mT5: A massively multilingual pre-trained text-to-text transformer](https://arxiv.org/abs/2010.11934)
+### Training results
 
-Authors: *Linting Xue, Noah Constant, Adam Roberts, Mihir Kale, Rami Al-Rfou, Aditya Siddhant, Aditya Barua, Colin Raffel* 
+| Training Loss | Epoch | Step  | Validation Loss | Bleu    | Gen Len |
+|:-------------:|:-----:|:-----:|:---------------:|:-------:|:-------:|
+| 1.1469        | 1.0   | 11019 | 1.0554          | 10.1262 | 17.8974 |
+| 1.0112        | 2.0   | 22038 | 0.9529          | 10.9674 | 17.8698 |
+| 0.937         | 3.0   | 33057 | 0.8913          | 11.6301 | 17.8687 |
+| 0.8809        | 4.0   | 44076 | 0.8545          | 11.9517 | 17.8833 |
+| 0.8501        | 5.0   | 55095 | 0.8401          | 12.2867 | 17.8712 |
 
 
-## Abstract
+### Framework versions
 
-The recent "Text-to-Text Transfer Transformer" (T5) leveraged a unified text-to-text format and scale to attain state-of-the-art results on a wide variety of English-language NLP tasks. In this paper, we introduce mT5, a multilingual variant of T5 that was pre-trained on a new Common Crawl-based dataset covering 101 languages. We describe the design and modified training of mT5 and demonstrate its state-of-the-art performance on many multilingual benchmarks. All of the code and model checkpoints used in this work are publicly available.
+- Transformers 4.16.2
+- Pytorch 1.10.2+cu113
+- Datasets 1.18.3
+- Tokenizers 0.11.0