mt5-smallmt5-finetuned-on-en-yor-BBC-news

This model is a fine-tuned version of google/mt5-small on the None dataset. It achieves the following results on the evaluation set:

• Loss: 2.9640
• Rouge1: 31.9414
• Rouge2: 10.6476
• Rougel: 27.9541
• Rougelsum: 27.974

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 5.6e-05
• train_batch_size: 8
• eval_batch_size: 8
• seed: 42
• optimizer: Use adamw_torch with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
• lr_scheduler_type: linear
• num_epochs: 8

Training results

Training Loss	Epoch	Step	Validation Loss	Rouge1	Rouge2	Rougel	Rougelsum
3.547	1.0	1588	3.2871	30.6764	9.756	26.925	26.9536
3.6705	2.0	3176	3.1250	30.387	9.6758	26.7739	26.8351
3.4985	3.0	4764	3.0648	31.2651	10.2554	27.4377	27.4828
3.3861	4.0	6352	3.0187	31.0368	10.0801	27.1596	27.1924
3.3116	5.0	7940	3.0051	31.9967	10.6125	28.0438	28.08
3.2603	6.0	9528	2.9801	31.7684	10.6465	27.8597	27.8956
3.2242	7.0	11116	2.9681	31.6692	10.4338	27.7238	27.7438
3.2007	8.0	12704	2.9640	31.9414	10.6476	27.9541	27.974

Framework versions

• Transformers 4.47.1
• Pytorch 2.5.1+cu121
• Datasets 3.2.0
• Tokenizers 0.21.0