End of training

README.md

@@ -1,12 +1,16 @@
 ---
 base_model: sanchit-gandhi/Mistral-7B-v0.1-6-layer
 tags:
+- alignment-handbook
+- trl
+- sft
+- generated_from_trainer
 - trl
 - sft
 - alignment-handbook
 - generated_from_trainer
 datasets:
-- generator
+- stingning/ultrachat
 model-index:
 - name: sanchit-gandhi/Mistral-7B-v0.1-6-layer
   results: []
@@ -17,7 +21,7 @@ should probably proofread and complete it, then remove this comment. -->
 
 # sanchit-gandhi/Mistral-7B-v0.1-6-layer
 
-This model is a fine-tuned version of [sanchit-gandhi/Mistral-7B-v0.1-6-layer](https://huggingface.co/sanchit-gandhi/Mistral-7B-v0.1-6-layer) on the generator dataset.
+This model is a fine-tuned version of [sanchit-gandhi/Mistral-7B-v0.1-6-layer](https://huggingface.co/sanchit-gandhi/Mistral-7B-v0.1-6-layer) on the stingning/ultrachat dataset.
 It achieves the following results on the evaluation set:
 - Loss: 1.0042
 

config.json

@@ -21,6 +21,6 @@
   "tie_word_embeddings": false,
   "torch_dtype": "bfloat16",
   "transformers_version": "4.40.1",
-  "use_cache": false,
+  "use_cache": true,
   "vocab_size": 32000
 }

wandb/debug-internal.log

@@ -86,3 +86,9 @@
 2024-04-25 23:49:36,063 DEBUG   HandlerThread:212584 [handler.py:handle_request():146] handle_request: stop_status
 2024-04-25 23:49:36,063 DEBUG   SenderThread:212584 [sender.py:send_request():406] send_request: stop_status
 2024-04-25 23:49:37,798 INFO    Thread-12 :212584 [dir_watcher.py:_on_file_modified():288] file/dir modified: /fsx/sanchit/distil-zephyr-1.5b-ssft-ultrachat/wandb/run-20240425_234916-ozdw63qu/files/output.log
+2024-04-25 23:49:39,684 DEBUG   HandlerThread:212584 [handler.py:handle_request():146] handle_request: status_report
+2024-04-25 23:49:39,800 INFO    Thread-12 :212584 [dir_watcher.py:_on_file_modified():288] file/dir modified: /fsx/sanchit/distil-zephyr-1.5b-ssft-ultrachat/wandb/run-20240425_234916-ozdw63qu/files/output.log
+2024-04-25 23:49:40,801 INFO    Thread-12 :212584 [dir_watcher.py:_on_file_modified():288] file/dir modified: /fsx/sanchit/distil-zephyr-1.5b-ssft-ultrachat/wandb/run-20240425_234916-ozdw63qu/files/output.log
+2024-04-25 23:49:41,802 INFO    Thread-12 :212584 [dir_watcher.py:_on_file_modified():288] file/dir modified: /fsx/sanchit/distil-zephyr-1.5b-ssft-ultrachat/wandb/run-20240425_234916-ozdw63qu/files/output.log
+2024-04-25 23:49:42,804 INFO    Thread-12 :212584 [dir_watcher.py:_on_file_modified():288] file/dir modified: /fsx/sanchit/distil-zephyr-1.5b-ssft-ultrachat/wandb/run-20240425_234916-ozdw63qu/files/output.log
+2024-04-25 23:49:45,208 DEBUG   HandlerThread:212584 [handler.py:handle_request():146] handle_request: status_report

wandb/run-20240425_234916-ozdw63qu/files/output.log

@@ -37,3 +37,21 @@ Training completed. Do not forget to share your model on huggingface.co/models =
 [INFO|tokenization_utils_base.py:2488] 2024-04-25 23:49:35,768 >> tokenizer config file saved in ./tokenizer_config.json
 [INFO|tokenization_utils_base.py:2497] 2024-04-25 23:49:35,769 >> Special tokens file saved in ./special_tokens_map.json
 [INFO|modelcard.py:450] 2024-04-25 23:49:35,816 >> Dropping the following result as it does not have all the necessary fields:
+{'task': {'name': 'Causal Language Modeling', 'type': 'text-generation'}, 'dataset': {'name': 'generator', 'type': 'generator', 'config': 'default', 'split': 'train', 'args': 'default'}}
+events.out.tfevents.1714088955.ip-26-0-167-177.211869.0: 100%|██████████| 5.07k/5.07k [00:00<00:00, 36.5kB/s]
+events.out.tfevents.1714088965.ip-26-0-167-177.211869.1: 100%|██████████| 364/364 [00:00<00:00, 2.20kB/s]
+training_args.bin: 100%|██████████| 4.98k/4.98k [00:00<00:00, 25.1kB/s]
+run-etajcxpg.wandb: 100%|██████████| 9.71M/9.71M [00:00<00:00, 30.9MB/s]
+Upload 4 LFS files: 100%|██████████| 4/4 [00:00<00:00,  8.49it/s]███████| 364/364 [00:00<00:00, 3.15kB/s]
+2024-04-25 23:49:40 - INFO - __main__ - Model saved to ./
+[INFO|modelcard.py:450] 2024-04-25 23:49:40,249 >> Dropping the following result as it does not have all the necessary fields:
+{'task': {'name': 'Causal Language Modeling', 'type': 'text-generation'}, 'dataset': {'name': 'stingning/ultrachat', 'type': 'stingning/ultrachat', 'config': 'default', 'split': 'train', 'args': 'default'}}
+[INFO|configuration_utils.py:471] 2024-04-25 23:49:40,253 >> Configuration saved in ./config.json
+[INFO|trainer.py:3305] 2024-04-25 23:49:40,254 >> Saving model checkpoint to ./
+[INFO|configuration_utils.py:471] 2024-04-25 23:49:40,255 >> Configuration saved in ./config.json
+[INFO|configuration_utils.py:697] 2024-04-25 23:49:40,257 >> Configuration saved in ./generation_config.json
+2024-04-25 23:49:40 - INFO - __main__ - Pushing to hub...
+[INFO|modeling_utils.py:2590] 2024-04-25 23:49:45,207 >> Model weights saved in ./model.safetensors
+[INFO|tokenization_utils_base.py:2488] 2024-04-25 23:49:45,209 >> tokenizer config file saved in ./tokenizer_config.json
+[INFO|tokenization_utils_base.py:2497] 2024-04-25 23:49:45,211 >> Special tokens file saved in ./special_tokens_map.json
+[INFO|modelcard.py:450] 2024-04-25 23:49:45,255 >> Dropping the following result as it does not have all the necessary fields:

wandb/run-20240425_234916-ozdw63qu/logs/debug-internal.log

@@ -86,3 +86,9 @@
 2024-04-25 23:49:36,063 DEBUG   HandlerThread:212584 [handler.py:handle_request():146] handle_request: stop_status
 2024-04-25 23:49:36,063 DEBUG   SenderThread:212584 [sender.py:send_request():406] send_request: stop_status
 2024-04-25 23:49:37,798 INFO    Thread-12 :212584 [dir_watcher.py:_on_file_modified():288] file/dir modified: /fsx/sanchit/distil-zephyr-1.5b-ssft-ultrachat/wandb/run-20240425_234916-ozdw63qu/files/output.log
+2024-04-25 23:49:39,684 DEBUG   HandlerThread:212584 [handler.py:handle_request():146] handle_request: status_report
+2024-04-25 23:49:39,800 INFO    Thread-12 :212584 [dir_watcher.py:_on_file_modified():288] file/dir modified: /fsx/sanchit/distil-zephyr-1.5b-ssft-ultrachat/wandb/run-20240425_234916-ozdw63qu/files/output.log
+2024-04-25 23:49:40,801 INFO    Thread-12 :212584 [dir_watcher.py:_on_file_modified():288] file/dir modified: /fsx/sanchit/distil-zephyr-1.5b-ssft-ultrachat/wandb/run-20240425_234916-ozdw63qu/files/output.log
+2024-04-25 23:49:41,802 INFO    Thread-12 :212584 [dir_watcher.py:_on_file_modified():288] file/dir modified: /fsx/sanchit/distil-zephyr-1.5b-ssft-ultrachat/wandb/run-20240425_234916-ozdw63qu/files/output.log
+2024-04-25 23:49:42,804 INFO    Thread-12 :212584 [dir_watcher.py:_on_file_modified():288] file/dir modified: /fsx/sanchit/distil-zephyr-1.5b-ssft-ultrachat/wandb/run-20240425_234916-ozdw63qu/files/output.log
+2024-04-25 23:49:45,208 DEBUG   HandlerThread:212584 [handler.py:handle_request():146] handle_request: status_report