Hugging Face's logo

baby-dev
/
test-default-02

PEFT
Safetensors
phi3
axolotl
Generated from Trainer
custom_code
Model card Files Community
baby-dev commited on
Commit
c02d777
·
verified ·
1 Parent(s): f17570e

End of training

Browse files
Files changed (5) hide show
  1. README.md +8 -8
  2. adapter_config.json +2 -2
  3. adapter_model.bin +1 -1
  4. adapter_model.safetensors +1 -1
  5. training_args.bin +1 -1
README.md CHANGED
@@ -47,7 +47,7 @@ eval_max_new_tokens: 128
47
  eval_steps: 50
48
  eval_table_size: null
49
  evals_per_epoch: null
50
- flash_attention: false
51
  fp16: false
52
  fsdp: null
53
  fsdp_config: null
@@ -115,7 +115,7 @@ xformers_attention: null
115
 
116
  This model is a fine-tuned version of [migtissera/Tess-v2.5-Phi-3-medium-128k-14B](https://huggingface.co/migtissera/Tess-v2.5-Phi-3-medium-128k-14B) on the None dataset.
117
  It achieves the following results on the evaluation set:
118
- - Loss: 0.0860
119
 
120
  ## Model description
121
 
@@ -149,12 +149,12 @@ The following hyperparameters were used during training:
149
 
150
  | Training Loss | Epoch | Step | Validation Loss |
151
  |:-------------:|:------:|:----:|:---------------:|
152
- | No log | 0.0002 | 1 | 0.8770 |
153
- | 0.3826 | 0.0114 | 50 | 0.1017 |
154
- | 0.275 | 0.0229 | 100 | 0.0823 |
155
- | 0.2988 | 0.0343 | 150 | 0.1188 |
156
- | 0.2608 | 0.0457 | 200 | 0.0651 |
157
- | 0.212 | 0.0572 | 250 | 0.0860 |
158
 
159
 
160
  ### Framework versions
 
47
  eval_steps: 50
48
  eval_table_size: null
49
  evals_per_epoch: null
50
+ flash_attention: true
51
  fp16: false
52
  fsdp: null
53
  fsdp_config: null
 
115
 
116
  This model is a fine-tuned version of [migtissera/Tess-v2.5-Phi-3-medium-128k-14B](https://huggingface.co/migtissera/Tess-v2.5-Phi-3-medium-128k-14B) on the None dataset.
117
  It achieves the following results on the evaluation set:
118
+ - Loss: 0.0855
119
 
120
  ## Model description
121
 
 
149
 
150
  | Training Loss | Epoch | Step | Validation Loss |
151
  |:-------------:|:------:|:----:|:---------------:|
152
+ | No log | 0.0002 | 1 | 0.8763 |
153
+ | 0.3649 | 0.0114 | 50 | 0.1069 |
154
+ | 0.3225 | 0.0229 | 100 | 0.0854 |
155
+ | 0.2397 | 0.0343 | 150 | 0.0699 |
156
+ | 0.2604 | 0.0457 | 200 | 0.0639 |
157
+ | 0.1548 | 0.0572 | 250 | 0.0855 |
158
 
159
 
160
  ### Framework versions
adapter_config.json CHANGED
@@ -20,9 +20,9 @@
20
  "rank_pattern": {},
21
  "revision": null,
22
  "target_modules": [
23
- "qkv_proj",
24
- "down_proj",
25
  "o_proj",
 
 
26
  "gate_up_proj"
27
  ],
28
  "task_type": "CAUSAL_LM",
 
20
  "rank_pattern": {},
21
  "revision": null,
22
  "target_modules": [
 
 
23
  "o_proj",
24
+ "down_proj",
25
+ "qkv_proj",
26
  "gate_up_proj"
27
  ],
28
  "task_type": "CAUSAL_LM",
adapter_model.bin CHANGED
@@ -1,3 +1,3 @@
1
  version https://git-lfs.github.com/spec/v1
2
- oid sha256:630869cadff03125917e33a5248a12e4989cd4f0ad37eccaa428f0b8035335d9
3
  size 445760970
 
1
  version https://git-lfs.github.com/spec/v1
2
+ oid sha256:dd38428b4bd026e5819ea60f1163fd767c9da28ec19852e3b1857d4d963a2754
3
  size 445760970
adapter_model.safetensors CHANGED
@@ -1,3 +1,3 @@
1
  version https://git-lfs.github.com/spec/v1
2
- oid sha256:4d9bf86f014a05dc67164438701b9dbf80ba0c37e3b9f2060c6abef0ab7c1641
3
  size 445688440
 
1
  version https://git-lfs.github.com/spec/v1
2
+ oid sha256:d9433966f7a757fab46a74e0770742134b969279fd289c9cef84a841b09e5f1b
3
  size 445688440
training_args.bin CHANGED
@@ -1,3 +1,3 @@
1
  version https://git-lfs.github.com/spec/v1
2
- oid sha256:fe8e44f29f7af913bf292c53b5d6728d1a582d30e6e936e7aded8c2fcec71489
3
  size 6776
 
1
  version https://git-lfs.github.com/spec/v1
2
+ oid sha256:68d8d840fb8858e3bd7f466aded7dbd9ced6a79dd57555c86a327c72822c75c1
3
  size 6776