End of training

README.md

@@ -69,7 +69,7 @@ max_steps: 10
 micro_batch_size: 1
 mlflow_experiment_name: /tmp/45fb2d361254b178_train_data.json
 model_type: AutoModelForCausalLM
-num_epochs: 1
+num_epochs: 4
 optimizer: adamw_bnb_8bit
 output_dir: miner_id_24
 pad_to_sequence_len: true
@@ -104,7 +104,7 @@ xformers_attention: null
 
 This model is a fine-tuned version of [NousResearch/CodeLlama-7b-hf](https://huggingface.co/NousResearch/CodeLlama-7b-hf) on the None dataset.
 It achieves the following results on the evaluation set:
-- Loss: 1.6888
+- Loss: 1.3355
 
 ## Model description
 
@@ -139,9 +139,15 @@ The following hyperparameters were used during training:
 | Training Loss | Epoch  | Step | Validation Loss |
 |:-------------:|:------:|:----:|:---------------:|
 | 18.8184       | 0.0007 | 1    | 2.6565          |
-| 19.4363       | 0.0022 | 3    | 2.6460          |
-| 19.7239       | 0.0044 | 6    | 2.4790          |
-| 20.1204       | 0.0067 | 9    | 1.6888          |
+| 24.6349       | 0.0015 | 2    | 2.6549          |
+| 19.4332       | 0.0022 | 3    | 2.6461          |
+| 19.745        | 0.0030 | 4    | 2.6241          |
+| 20.1983       | 0.0037 | 5    | 2.5784          |
+| 19.7283       | 0.0044 | 6    | 2.4844          |
+| 16.9993       | 0.0052 | 7    | 2.3075          |
+| 19.1259       | 0.0059 | 8    | 2.0358          |
+| 20.2162       | 0.0067 | 9    | 1.7064          |
+| 11.843        | 0.0074 | 10   | 1.3355          |
 
 
 ### Framework versions

adapter_model.bin

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:6f1d49b888a23cc7e9999fb09f6e1e06ff7013a51409c80261ec3bd88e9cbc4f
+oid sha256:a0c9202d9dc89c3b0d506c40bdab2e7d5a41568f48807e519a611cb587c2a320
 size 40138058