distilbert_oscarth_0020

This model is a fine-tuned version of distilbert-base-uncased on an unknown dataset. It achieves the following results on the evaluation set:

• Train Loss: 1.4909
• Validation Loss: 1.4161
• Epoch: 19

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• optimizer: {'name': 'AdamWeightDecay', 'learning_rate': 2e-05, 'decay': 0.0, 'beta_1': 0.9, 'beta_2': 0.999, 'epsilon': 1e-07, 'amsgrad': False, 'weight_decay_rate': 0.01}
• training_precision: float32

Training results

Train Loss	Validation Loss	Epoch
4.1327	2.9983	0
2.7813	2.4562	1
2.4194	2.2066	2
2.2231	2.0562	3
2.0894	1.9450	4
1.9905	1.8621	5
1.9148	1.7941	6
1.8508	1.7363	7
1.7976	1.6909	8
1.7509	1.6488	9
1.7126	1.6124	10
1.6764	1.5835	11
1.6450	1.5521	12
1.6175	1.5282	13
1.5919	1.5045	14
1.5679	1.4833	15
1.5476	1.4627	16
1.5271	1.4498	17
1.5098	1.4270	18
1.4909	1.4161	19

Framework versions

• Transformers 4.20.1
• TensorFlow 2.8.2
• Datasets 2.3.2
• Tokenizers 0.12.1