en_to_kjven_translator

This model is a fine-tuned version of t5-base on the None dataset. It achieves the following results on the evaluation set:

• Loss: 0.7384
• Bleu: 24.3137
• Gen Len: 18.018

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 2e-05
• train_batch_size: 32
• eval_batch_size: 32
• seed: 42
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• num_epochs: 16
• mixed_precision_training: Native AMP

Training results

Training Loss	Epoch	Step	Validation Loss	Bleu	Gen Len
1.0945	1.0	2993	0.9622	21.6701	17.9817
0.9915	2.0	5986	0.8925	22.5111	17.9861
0.9306	3.0	8979	0.8543	22.9079	18.0025
0.889	4.0	11972	0.8257	23.2629	17.9987
0.8616	5.0	14965	0.8053	23.4706	18.0027
0.8412	6.0	17958	0.7886	23.6554	18.0092
0.811	7.0	20951	0.7769	23.8119	18.0106
0.8024	8.0	23944	0.7675	23.9582	18.0114
0.7898	9.0	26937	0.7583	24.0002	18.012
0.7802	10.0	29930	0.7538	24.0943	18.0156
0.7645	11.0	32923	0.7491	24.1756	18.0168
0.7565	12.0	35916	0.7445	24.2229	18.0158
0.7553	13.0	38909	0.7410	24.2709	18.0159
0.7454	14.0	41902	0.7399	24.2593	18.0205
0.7424	15.0	44895	0.7388	24.3007	18.0174
0.7461	16.0	47888	0.7384	24.3137	18.018

Framework versions

• Transformers 4.28.1
• Pytorch 2.0.0+cu118
• Datasets 2.11.0
• Tokenizers 0.13.3