output

This model is a fine-tuned version of distilbert/distilbert-base-uncased on an unknown dataset. It achieves the following results on the evaluation set:

• Loss: 0.1131
• F1: 0.9809

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 4e-05
• train_batch_size: 32
• eval_batch_size: 32
• seed: 42
• optimizer: Use adamw_torch_fused with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
• lr_scheduler_type: linear
• num_epochs: 5

Training results

Training Loss	Epoch	Step	Validation Loss	F1
0.3479	0.0856	50	0.1687	0.9652
0.1711	0.1712	100	0.2640	0.9259
0.1464	0.2568	150	0.1211	0.9762
0.099	0.3425	200	0.0968	0.9803
0.0991	0.4281	250	0.1052	0.9768
0.1037	0.5137	300	0.0974	0.9759
0.096	0.5993	350	0.0838	0.9812
0.0773	0.6849	400	0.1089	0.9765
0.0824	0.7705	450	0.1058	0.9809
0.1142	0.8562	500	0.0911	0.9794
0.0984	0.9418	550	0.0816	0.9803
0.0821	1.0274	600	0.0891	0.9835
0.0761	1.1130	650	0.0953	0.9791
0.0706	1.1986	700	0.1002	0.9835
0.0745	1.2842	750	0.0872	0.9827
0.0831	1.3699	800	0.1102	0.9794
0.0828	1.4555	850	0.0895	0.9833
0.0656	1.5411	900	0.0969	0.9812
0.0859	1.6267	950	0.0856	0.9841
0.0608	1.7123	1000	0.0873	0.9797
0.0751	1.7979	1050	0.0818	0.9835
0.0561	1.8836	1100	0.0808	0.9833
0.0753	1.9692	1150	0.0839	0.9838
0.0758	2.0548	1200	0.0955	0.9824
0.0413	2.1404	1250	0.1082	0.9827
0.0572	2.2260	1300	0.0906	0.9821
0.0597	2.3116	1350	0.0847	0.9830
0.0506	2.3973	1400	0.0870	0.9838
0.0629	2.4829	1450	0.0843	0.9815
0.0404	2.5685	1500	0.1024	0.9830
0.0476	2.6541	1550	0.0809	0.9833
0.0556	2.7397	1600	0.0895	0.9830
0.0803	2.8253	1650	0.0840	0.9824
0.0856	2.9110	1700	0.0812	0.9841
0.0475	2.9966	1750	0.0955	0.9827
0.05	3.0822	1800	0.0849	0.9824
0.0409	3.1678	1850	0.0958	0.9812
0.0368	3.2534	1900	0.1069	0.9821
0.0423	3.3390	1950	0.1012	0.9809
0.0444	3.4247	2000	0.0959	0.9809
0.0456	3.5103	2050	0.0923	0.9806
0.0518	3.5959	2100	0.0985	0.9821
0.029	3.6815	2150	0.1144	0.9818
0.0453	3.7671	2200	0.0975	0.9809
0.0452	3.8527	2250	0.1031	0.9815
0.0369	3.9384	2300	0.0918	0.9827
0.038	4.0240	2350	0.0907	0.9830
0.0329	4.1096	2400	0.1058	0.9821
0.0249	4.1952	2450	0.1093	0.9818
0.0467	4.2808	2500	0.1006	0.9803
0.0328	4.3664	2550	0.1100	0.9824
0.0314	4.4521	2600	0.1150	0.9815
0.0274	4.5377	2650	0.1147	0.9821
0.0377	4.6233	2700	0.1173	0.9824
0.0378	4.7089	2750	0.1169	0.9827
0.0228	4.7945	2800	0.1141	0.9818
0.0281	4.8801	2850	0.1138	0.9809
0.0362	4.9658	2900	0.1131	0.9809

Framework versions

• Transformers 4.48.0
• Pytorch 2.5.1+cu121
• Datasets 3.2.0
• Tokenizers 0.21.0