Model save

README.md

@@ -17,8 +17,8 @@ should probably proofread and complete it, then remove this comment. -->
 
 This model is a fine-tuned version of [AnhNam/Luong_modernBert_ft](https://huggingface.co/AnhNam/Luong_modernBert_ft) on the None dataset.
 It achieves the following results on the evaluation set:
-- Loss: 0.4593
-- F1: 0.7823
+- Loss: 0.4084
+- F1: 0.8571
 
 ## Model description
 
@@ -37,37 +37,29 @@ More information needed
 ### Training hyperparameters
 
 The following hyperparameters were used during training:
-- learning_rate: 1e-06
-- train_batch_size: 4
-- eval_batch_size: 4
+- learning_rate: 5e-06
+- train_batch_size: 2
+- eval_batch_size: 2
 - seed: 42
 - gradient_accumulation_steps: 2
-- total_train_batch_size: 8
+- total_train_batch_size: 4
 - optimizer: Use adamw_torch with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
 - lr_scheduler_type: linear
 - lr_scheduler_warmup_ratio: 0.1
-- num_epochs: 15
+- num_epochs: 7
 - mixed_precision_training: Native AMP
 
 ### Training results
 
-| Training Loss | Epoch | Step | Validation Loss | F1     |
-|:-------------:|:-----:|:----:|:---------------:|:------:|
-| 0.3443        | 1.0   | 7    | 0.5216          | 0.5071 |
-| 0.434         | 2.0   | 14   | 0.5117          | 0.5071 |
-| 1.1626        | 3.0   | 21   | 0.5010          | 0.5071 |
-| 1.6782        | 4.0   | 28   | 0.4867          | 0.7016 |
-| 0.4249        | 5.0   | 35   | 0.4777          | 0.7016 |
-| 0.9495        | 6.0   | 42   | 0.4716          | 0.7016 |
-| 0.5363        | 7.0   | 49   | 0.4685          | 0.7823 |
-| 1.0944        | 8.0   | 56   | 0.4669          | 0.7823 |
-| 1.1263        | 9.0   | 63   | 0.4637          | 0.7823 |
-| 0.8751        | 10.0  | 70   | 0.4626          | 0.7823 |
-| 0.4535        | 11.0  | 77   | 0.4611          | 0.7823 |
-| 0.5308        | 12.0  | 84   | 0.4596          | 0.7823 |
-| 0.8087        | 13.0  | 91   | 0.4595          | 0.7823 |
-| 0.4981        | 14.0  | 98   | 0.4592          | 0.7823 |
-| 0.698         | 15.0  | 105  | 0.4593          | 0.7823 |
+| Training Loss | Epoch  | Step | Validation Loss | F1     |
+|:-------------:|:------:|:----:|:---------------:|:------:|
+| 0.1646        | 1.0    | 14   | 0.5255          | 0.5578 |
+| 0.1971        | 2.0    | 28   | 0.4755          | 0.7016 |
+| 0.1588        | 3.0    | 42   | 0.4503          | 0.7823 |
+| 0.0806        | 4.0    | 56   | 0.4234          | 0.8571 |
+| 0.0221        | 5.0    | 70   | 0.4139          | 0.8571 |
+| 0.0196        | 6.0    | 84   | 0.4086          | 0.8571 |
+| 0.2718        | 6.5185 | 91   | 0.4084          | 0.8571 |
 
 
 ### Framework versions

model.safetensors

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:5121332c4dadc233a3803a901dbbd66175b1c1a9c0eb300d4ea4cbb1baf31540
+oid sha256:aa3a335bc79db5daac4a97ca4bc2be4a3380b99fcdc36ece33a21bd34b082c77
 size 598439784

special_tokens_map.json

@@ -8,7 +8,7 @@
   },
   "mask_token": {
     "content": "[MASK]",
-    "lstrip": false,
+    "lstrip": true,
     "normalized": false,
     "rstrip": false,
     "single_word": false

tokenizer.json

@@ -309,7 +309,7 @@
       "id": 50284,
       "content": "[MASK]",
       "single_word": false,
-      "lstrip": false,
+      "lstrip": true,
       "rstrip": false,
       "normalized": false,
       "special": true

tokenizer_config.json

@@ -258,7 +258,7 @@
     },
     "50284": {
       "content": "[MASK]",
-      "lstrip": false,
+      "lstrip": true,
       "normalized": false,
       "rstrip": false,
       "single_word": false,

training_args.bin

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:620208894b9c71fe522efafb2b9d6d4edfb2047d9a050bae37d79c30b4ef9537
+oid sha256:b0e12b5090f6aaace50fbd1fc6f402df8012510172f678ec40598bf9476dbebd
 size 5432