my_fine_tuned_t5_small_model

This model is a fine-tuned version of huggingface-course/mt5-finetuned-amazon-en-es on the None dataset. It achieves the following results on the evaluation set:

• Loss: 2.1069
• Rouge1: 26.742
• Rouge2: 7.9622
• Rougel: 20.4423
• Rougelsum: 20.7426

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 0.001
• train_batch_size: 8
• eval_batch_size: 8
• seed: 42
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• lr_scheduler_warmup_steps: 3000
• num_epochs: 10

Training results

Training Loss	Epoch	Step	Validation Loss	Rouge1	Rouge2	Rougel	Rougelsum
4.428	1.0	13	2.8216	19.1382	5.389	15.1855	15.1893
4.299	2.0	26	2.6894	19.6236	5.4931	15.4001	15.5122
4.0999	3.0	39	2.5552	22.2435	6.6021	17.2465	17.4679
3.8541	4.0	52	2.4394	23.3478	7.1578	18.4112	18.6265
3.6964	5.0	65	2.3568	25.092	7.4595	19.3078	19.5198
3.5481	6.0	78	2.3018	26.0743	8.0058	19.9644	20.2044
3.2977	7.0	91	2.2207	26.8852	7.9185	20.5406	20.7554
3.2768	8.0	104	2.1832	26.3885	7.9643	20.3863	20.6338
3.2047	9.0	117	2.1403	26.598	7.883	20.3585	20.4868
3.0813	10.0	130	2.1069	26.742	7.9622	20.4423	20.7426

Framework versions

• Transformers 4.41.2
• Pytorch 2.3.1+cu121
• Datasets 2.19.2
• Tokenizers 0.19.1