End of training

README.md

@@ -1,11 +1,15 @@
 ---
 base_model: sanchit-gandhi/Mistral-7B-v0.1-6-layer
 tags:
+- alignment-handbook
+- trl
+- sft
+- generated_from_trainer
 - trl
 - sft
 - generated_from_trainer
 datasets:
-- generator
+- stingning/ultrachat
 model-index:
 - name: sanchit-gandhi/Mistral-7B-v0.1-6-layer
   results: []
@@ -16,7 +20,7 @@ should probably proofread and complete it, then remove this comment. -->
 
 # sanchit-gandhi/Mistral-7B-v0.1-6-layer
 
-This model is a fine-tuned version of [sanchit-gandhi/Mistral-7B-v0.1-6-layer](https://huggingface.co/sanchit-gandhi/Mistral-7B-v0.1-6-layer) on the generator dataset.
+This model is a fine-tuned version of [sanchit-gandhi/Mistral-7B-v0.1-6-layer](https://huggingface.co/sanchit-gandhi/Mistral-7B-v0.1-6-layer) on the stingning/ultrachat dataset.
 It achieves the following results on the evaluation set:
 - Loss: 1.0042
 

config.json

@@ -21,6 +21,6 @@
   "tie_word_embeddings": false,
   "torch_dtype": "bfloat16",
   "transformers_version": "4.40.1",
-  "use_cache": false,
+  "use_cache": true,
   "vocab_size": 32000
 }

wandb/debug-internal.log

@@ -37480,3 +37480,9 @@
 2024-04-25 23:47:32,219 DEBUG   HandlerThread:156911 [handler.py:handle_request():146] handle_request: keepalive
 2024-04-25 23:47:32,264 INFO    Thread-12 :156911 [dir_watcher.py:_on_file_modified():288] file/dir modified: /fsx/sanchit/distil-zephyr-1.5b-ssft-ultrachat/wandb/run-20240425_134518-etajcxpg/files/output.log
 2024-04-25 23:47:33,304 DEBUG   HandlerThread:156911 [handler.py:handle_request():146] handle_request: status_report
+2024-04-25 23:47:36,269 INFO    Thread-12 :156911 [dir_watcher.py:_on_file_modified():288] file/dir modified: /fsx/sanchit/distil-zephyr-1.5b-ssft-ultrachat/wandb/run-20240425_134518-etajcxpg/files/output.log
+2024-04-25 23:47:37,260 DEBUG   HandlerThread:156911 [handler.py:handle_request():146] handle_request: keepalive
+2024-04-25 23:47:38,271 INFO    Thread-12 :156911 [dir_watcher.py:_on_file_modified():288] file/dir modified: /fsx/sanchit/distil-zephyr-1.5b-ssft-ultrachat/wandb/run-20240425_134518-etajcxpg/files/output.log
+2024-04-25 23:47:38,974 DEBUG   HandlerThread:156911 [handler.py:handle_request():146] handle_request: status_report
+2024-04-25 23:47:41,480 DEBUG   HandlerThread:156911 [handler.py:handle_request():146] handle_request: internal_messages
+2024-04-25 23:47:42,275 INFO    Thread-12 :156911 [dir_watcher.py:_on_file_modified():288] file/dir modified: /fsx/sanchit/distil-zephyr-1.5b-ssft-ultrachat/wandb/run-20240425_134518-etajcxpg/files/output.log

wandb/run-20240425_134518-etajcxpg/files/output.log

@@ -18858,3 +18858,19 @@ Training completed. Do not forget to share your model on huggingface.co/models =
 [INFO|tokenization_utils_base.py:2488] 2024-04-25 23:47:31,220 >> tokenizer config file saved in ./tokenizer_config.json
 [INFO|tokenization_utils_base.py:2497] 2024-04-25 23:47:31,221 >> Special tokens file saved in ./special_tokens_map.json
 [INFO|modelcard.py:450] 2024-04-25 23:47:31,303 >> Dropping the following result as it does not have all the necessary fields:
+{'task': {'name': 'Causal Language Modeling', 'type': 'text-generation'}, 'dataset': {'name': 'generator', 'type': 'generator', 'config': 'default', 'split': 'train', 'args': 'default'}}
+events.out.tfevents.1714088841.ip-26-0-167-177.156194.1: 100%|██████████| 364/364 [00:00<00:00, 2.49kB/s]
+run-etajcxpg.wandb: 100%|██████████| 9.70M/9.70M [00:00<00:00, 42.3MB/s]
+Upload 2 LFS files: 100%|██████████| 2/2 [00:00<00:00,  5.07it/s]       | 0.00/364 [00:00<?, ?B/s]
+2024-04-25 23:47:35 - INFO - __main__ - Model saved to ./
+2024-04-25 23:47:35 - INFO - __main__ - Pushing to hub...
+[INFO|modelcard.py:450] 2024-04-25 23:47:35,964 >> Dropping the following result as it does not have all the necessary fields:
+{'task': {'name': 'Causal Language Modeling', 'type': 'text-generation'}, 'dataset': {'name': 'stingning/ultrachat', 'type': 'stingning/ultrachat', 'config': 'default', 'split': 'train', 'args': 'default'}}
+[INFO|configuration_utils.py:471] 2024-04-25 23:47:35,968 >> Configuration saved in ./config.json
+[INFO|trainer.py:3305] 2024-04-25 23:47:35,970 >> Saving model checkpoint to ./
+[INFO|configuration_utils.py:471] 2024-04-25 23:47:35,972 >> Configuration saved in ./config.json
+[INFO|configuration_utils.py:697] 2024-04-25 23:47:35,973 >> Configuration saved in ./generation_config.json
+[INFO|modeling_utils.py:2590] 2024-04-25 23:47:40,802 >> Model weights saved in ./model.safetensors
+[INFO|tokenization_utils_base.py:2488] 2024-04-25 23:47:40,804 >> tokenizer config file saved in ./tokenizer_config.json
+[INFO|tokenization_utils_base.py:2497] 2024-04-25 23:47:40,806 >> Special tokens file saved in ./special_tokens_map.json
+[INFO|modelcard.py:450] 2024-04-25 23:47:40,849 >> Dropping the following result as it does not have all the necessary fields:

wandb/run-20240425_134518-etajcxpg/logs/debug-internal.log

@@ -37480,3 +37480,9 @@
 2024-04-25 23:47:32,219 DEBUG   HandlerThread:156911 [handler.py:handle_request():146] handle_request: keepalive
 2024-04-25 23:47:32,264 INFO    Thread-12 :156911 [dir_watcher.py:_on_file_modified():288] file/dir modified: /fsx/sanchit/distil-zephyr-1.5b-ssft-ultrachat/wandb/run-20240425_134518-etajcxpg/files/output.log
 2024-04-25 23:47:33,304 DEBUG   HandlerThread:156911 [handler.py:handle_request():146] handle_request: status_report
+2024-04-25 23:47:36,269 INFO    Thread-12 :156911 [dir_watcher.py:_on_file_modified():288] file/dir modified: /fsx/sanchit/distil-zephyr-1.5b-ssft-ultrachat/wandb/run-20240425_134518-etajcxpg/files/output.log
+2024-04-25 23:47:37,260 DEBUG   HandlerThread:156911 [handler.py:handle_request():146] handle_request: keepalive
+2024-04-25 23:47:38,271 INFO    Thread-12 :156911 [dir_watcher.py:_on_file_modified():288] file/dir modified: /fsx/sanchit/distil-zephyr-1.5b-ssft-ultrachat/wandb/run-20240425_134518-etajcxpg/files/output.log
+2024-04-25 23:47:38,974 DEBUG   HandlerThread:156911 [handler.py:handle_request():146] handle_request: status_report
+2024-04-25 23:47:41,480 DEBUG   HandlerThread:156911 [handler.py:handle_request():146] handle_request: internal_messages
+2024-04-25 23:47:42,275 INFO    Thread-12 :156911 [dir_watcher.py:_on_file_modified():288] file/dir modified: /fsx/sanchit/distil-zephyr-1.5b-ssft-ultrachat/wandb/run-20240425_134518-etajcxpg/files/output.log