metadata
license: apache-2.0
tags:
- generated_from_trainer
metrics:
- bleu
model-index:
- name: en_to_kjven_translator
results: []
en_to_kjven_translator
This model is a fine-tuned version of t5-small on the None dataset. It achieves the following results on the evaluation set:
- Loss: 0.9847
- Bleu: 20.6579
- Gen Len: 18.0017
Model description
More information needed
Intended uses & limitations
More information needed
Training and evaluation data
More information needed
Training procedure
Training hyperparameters
The following hyperparameters were used during training:
- learning_rate: 2e-05
- train_batch_size: 16
- eval_batch_size: 16
- seed: 42
- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
- lr_scheduler_type: linear
- num_epochs: 30
- mixed_precision_training: Native AMP
Training results
| Training Loss | Epoch | Step | Validation Loss | Bleu | Gen Len |
|---|---|---|---|---|---|
| 1.5437 | 1.0 | 6105 | 1.3910 | 17.168 | 17.9611 |
| 1.4232 | 2.0 | 12210 | 1.2885 | 18.0224 | 17.9691 |
| 1.3474 | 3.0 | 18315 | 1.2305 | 18.5705 | 17.9751 |
| 1.32 | 4.0 | 24420 | 1.1890 | 18.942 | 17.981 |
| 1.2733 | 5.0 | 30525 | 1.1597 | 19.2091 | 17.976 |
| 1.2347 | 6.0 | 36630 | 1.1324 | 19.4233 | 17.9852 |
| 1.2168 | 7.0 | 42735 | 1.1134 | 19.5744 | 17.9872 |
| 1.1951 | 8.0 | 48840 | 1.0967 | 19.7628 | 17.9841 |
| 1.1694 | 9.0 | 54945 | 1.0813 | 19.8682 | 17.9898 |
| 1.1539 | 10.0 | 61050 | 1.0681 | 20.0338 | 17.989 |
| 1.1447 | 11.0 | 67155 | 1.0566 | 20.1025 | 17.9912 |
| 1.1164 | 12.0 | 73260 | 1.0477 | 20.2067 | 17.9904 |
| 1.1157 | 13.0 | 79365 | 1.0390 | 20.251 | 17.9946 |
| 1.1145 | 14.0 | 85470 | 1.0310 | 20.3136 | 17.9952 |
| 1.0927 | 15.0 | 91575 | 1.0252 | 20.3417 | 17.9934 |
| 1.0837 | 16.0 | 97680 | 1.0185 | 20.407 | 17.9909 |
| 1.0776 | 17.0 | 103785 | 1.0136 | 20.4439 | 17.9955 |
| 1.0753 | 18.0 | 109890 | 1.0084 | 20.4762 | 17.9956 |
| 1.0569 | 19.0 | 115995 | 1.0039 | 20.4643 | 17.9995 |
| 1.0676 | 20.0 | 122100 | 1.0001 | 20.5041 | 18.002 |
| 1.0593 | 21.0 | 128205 | 0.9977 | 20.5265 | 18.0009 |
| 1.0555 | 22.0 | 134310 | 0.9952 | 20.5503 | 18.0018 |
| 1.0302 | 23.0 | 140415 | 0.9927 | 20.5968 | 18.0005 |
| 1.045 | 24.0 | 146520 | 0.9906 | 20.6097 | 18.0012 |
| 1.0407 | 25.0 | 152625 | 0.9889 | 20.6209 | 18.0015 |
| 1.0282 | 26.0 | 158730 | 0.9867 | 20.6371 | 18.0018 |
| 1.024 | 27.0 | 164835 | 0.9862 | 20.6404 | 18.0022 |
| 1.0411 | 28.0 | 170940 | 0.9851 | 20.6558 | 18.0018 |
| 1.0292 | 29.0 | 177045 | 0.9848 | 20.6559 | 18.0016 |
| 1.0149 | 30.0 | 183150 | 0.9847 | 20.6579 | 18.0017 |
Framework versions
- Transformers 4.27.4
- Pytorch 2.0.0+cu118
- Datasets 2.11.0
- Tokenizers 0.13.3