Training in progress, step 93, checkpoint

last-checkpoint/README.md

@@ -0,0 +1,202 @@
+---
+base_model: NousResearch/CodeLlama-7b-hf
+library_name: peft
+---
+
+# Model Card for Model ID
+
+<!-- Provide a quick summary of what the model is/does. -->
+
+
+
+## Model Details
+
+### Model Description
+
+<!-- Provide a longer summary of what this model is. -->
+
+
+
+- **Developed by:** [More Information Needed]
+- **Funded by [optional]:** [More Information Needed]
+- **Shared by [optional]:** [More Information Needed]
+- **Model type:** [More Information Needed]
+- **Language(s) (NLP):** [More Information Needed]
+- **License:** [More Information Needed]
+- **Finetuned from model [optional]:** [More Information Needed]
+
+### Model Sources [optional]
+
+<!-- Provide the basic links for the model. -->
+
+- **Repository:** [More Information Needed]
+- **Paper [optional]:** [More Information Needed]
+- **Demo [optional]:** [More Information Needed]
+
+## Uses
+
+<!-- Address questions around how the model is intended to be used, including the foreseeable users of the model and those affected by the model. -->
+
+### Direct Use
+
+<!-- This section is for the model use without fine-tuning or plugging into a larger ecosystem/app. -->
+
+[More Information Needed]
+
+### Downstream Use [optional]
+
+<!-- This section is for the model use when fine-tuned for a task, or when plugged into a larger ecosystem/app -->
+
+[More Information Needed]
+
+### Out-of-Scope Use
+
+<!-- This section addresses misuse, malicious use, and uses that the model will not work well for. -->
+
+[More Information Needed]
+
+## Bias, Risks, and Limitations
+
+<!-- This section is meant to convey both technical and sociotechnical limitations. -->
+
+[More Information Needed]
+
+### Recommendations
+
+<!-- This section is meant to convey recommendations with respect to the bias, risk, and technical limitations. -->
+
+Users (both direct and downstream) should be made aware of the risks, biases and limitations of the model. More information needed for further recommendations.
+
+## How to Get Started with the Model
+
+Use the code below to get started with the model.
+
+[More Information Needed]
+
+## Training Details
+
+### Training Data
+
+<!-- This should link to a Dataset Card, perhaps with a short stub of information on what the training data is all about as well as documentation related to data pre-processing or additional filtering. -->
+
+[More Information Needed]
+
+### Training Procedure
+
+<!-- This relates heavily to the Technical Specifications. Content here should link to that section when it is relevant to the training procedure. -->
+
+#### Preprocessing [optional]
+
+[More Information Needed]
+
+
+#### Training Hyperparameters
+
+- **Training regime:** [More Information Needed] <!--fp32, fp16 mixed precision, bf16 mixed precision, bf16 non-mixed precision, fp16 non-mixed precision, fp8 mixed precision -->
+
+#### Speeds, Sizes, Times [optional]
+
+<!-- This section provides information about throughput, start/end time, checkpoint size if relevant, etc. -->
+
+[More Information Needed]
+
+## Evaluation
+
+<!-- This section describes the evaluation protocols and provides the results. -->
+
+### Testing Data, Factors & Metrics
+
+#### Testing Data
+
+<!-- This should link to a Dataset Card if possible. -->
+
+[More Information Needed]
+
+#### Factors
+
+<!-- These are the things the evaluation is disaggregating by, e.g., subpopulations or domains. -->
+
+[More Information Needed]
+
+#### Metrics
+
+<!-- These are the evaluation metrics being used, ideally with a description of why. -->
+
+[More Information Needed]
+
+### Results
+
+[More Information Needed]
+
+#### Summary
+
+
+
+## Model Examination [optional]
+
+<!-- Relevant interpretability work for the model goes here -->
+
+[More Information Needed]
+
+## Environmental Impact
+
+<!-- Total emissions (in grams of CO2eq) and additional considerations, such as electricity usage, go here. Edit the suggested text below accordingly -->
+
+Carbon emissions can be estimated using the [Machine Learning Impact calculator](https://mlco2.github.io/impact#compute) presented in [Lacoste et al. (2019)](https://arxiv.org/abs/1910.09700).
+
+- **Hardware Type:** [More Information Needed]
+- **Hours used:** [More Information Needed]
+- **Cloud Provider:** [More Information Needed]
+- **Compute Region:** [More Information Needed]
+- **Carbon Emitted:** [More Information Needed]
+
+## Technical Specifications [optional]
+
+### Model Architecture and Objective
+
+[More Information Needed]
+
+### Compute Infrastructure
+
+[More Information Needed]
+
+#### Hardware
+
+[More Information Needed]
+
+#### Software
+
+[More Information Needed]
+
+## Citation [optional]
+
+<!-- If there is a paper or blog post introducing the model, the APA and Bibtex information for that should go in this section. -->
+
+**BibTeX:**
+
+[More Information Needed]
+
+**APA:**
+
+[More Information Needed]
+
+## Glossary [optional]
+
+<!-- If relevant, include terms and calculations in this section that can help readers understand the model or model card. -->
+
+[More Information Needed]
+
+## More Information [optional]
+
+[More Information Needed]
+
+## Model Card Authors [optional]
+
+[More Information Needed]
+
+## Model Card Contact
+
+[More Information Needed]
+### Framework versions
+
+- PEFT 0.14.0

last-checkpoint/adapter_config.json

@@ -0,0 +1,37 @@
+{
+  "alpha_pattern": {},
+  "auto_mapping": null,
+  "base_model_name_or_path": "NousResearch/CodeLlama-7b-hf",
+  "bias": "none",
+  "eva_config": null,
+  "exclude_modules": null,
+  "fan_in_fan_out": null,
+  "inference_mode": true,
+  "init_lora_weights": true,
+  "layer_replication": null,
+  "layers_pattern": null,
+  "layers_to_transform": null,
+  "loftq_config": {},
+  "lora_alpha": 16,
+  "lora_bias": false,
+  "lora_dropout": 0.05,
+  "megatron_config": null,
+  "megatron_core": "megatron.core",
+  "modules_to_save": null,
+  "peft_type": "LORA",
+  "r": 8,
+  "rank_pattern": {},
+  "revision": null,
+  "target_modules": [
+    "o_proj",
+    "q_proj",
+    "up_proj",
+    "down_proj",
+    "k_proj",
+    "gate_proj",
+    "v_proj"
+  ],
+  "task_type": "CAUSAL_LM",
+  "use_dora": false,
+  "use_rslora": false
+}

last-checkpoint/adapter_model.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:b77cad872e4bcc0e7430f7ae5e5dd49663565ee18e3d3f00e8e2a69232aaefe9
+size 80013120

last-checkpoint/optimizer.pt

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:07e5a97fc87827d4ac283ef085c485e13587e49c599f0bc37667e3ec5c9fd3b5
+size 41119636

last-checkpoint/rng_state.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:e43e7b805b054b95bdd6a42492a7a566a708443ec3c2c635b33190dba7252c59
+size 14244

last-checkpoint/scheduler.pt

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:b893c9226071507bd706a0d6a7a997c6693067f1b6d62a3307e8595b10559486
+size 1064

last-checkpoint/special_tokens_map.json

@@ -0,0 +1,30 @@
+{
+  "bos_token": {
+    "content": "<s>",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  },
+  "eos_token": {
+    "content": "</s>",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  },
+  "pad_token": {
+    "content": "</s>",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  },
+  "unk_token": {
+    "content": "<unk>",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  }
+}

last-checkpoint/tokenizer.model

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:45ccb9c8b6b561889acea59191d66986d314e7cbd6a78abc6e49b139ca91c1e6
+size 500058

last-checkpoint/tokenizer_config.json

@@ -0,0 +1,42 @@
+{
+  "add_bos_token": true,
+  "add_eos_token": false,
+  "add_prefix_space": null,
+  "added_tokens_decoder": {
+    "0": {
+      "content": "<unk>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "1": {
+      "content": "<s>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "2": {
+      "content": "</s>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    }
+  },
+  "bos_token": "<s>",
+  "chat_template": "{% if not add_generation_prompt is defined %}{% set add_generation_prompt = false %}{% endif %}{% set loop_messages = messages %}{% for message in loop_messages %}{% set content = '<|start_header_id|>' + message['role'] + '<|end_header_id|>\n\n'+ message['content'] | trim + '<|eot_id|>' %}{% if loop.index0 == 0 %}{% set content = bos_token + content %}{% endif %}{{ content }}{% endfor %}{% if add_generation_prompt %}{{ '<|start_header_id|>assistant<|end_header_id|>\n\n' }}{% endif %}",
+  "clean_up_tokenization_spaces": false,
+  "eos_token": "</s>",
+  "legacy": true,
+  "model_max_length": 1000000000000000019884624838656,
+  "pad_token": "</s>",
+  "sp_model_kwargs": {},
+  "tokenizer_class": "LlamaTokenizer",
+  "unk_token": "<unk>",
+  "use_default_system_prompt": false
+}

last-checkpoint/trainer_state.json

@@ -0,0 +1,692 @@
+{
+  "best_metric": null,
+  "best_model_checkpoint": null,
+  "epoch": 0.25016812373907193,
+  "eval_steps": 93,
+  "global_step": 93,
+  "is_hyper_param_search": false,
+  "is_local_process_zero": true,
+  "is_world_process_zero": true,
+  "log_history": [
+    {
+      "epoch": 0.0026899798251513113,
+      "grad_norm": 1.8751003742218018,
+      "learning_rate": 2e-05,
+      "loss": 2.0835,
+      "step": 1
+    },
+    {
+      "epoch": 0.005379959650302623,
+      "grad_norm": 2.948612928390503,
+      "learning_rate": 4e-05,
+      "loss": 2.7117,
+      "step": 2
+    },
+    {
+      "epoch": 0.008069939475453935,
+      "grad_norm": 2.65693998336792,
+      "learning_rate": 6e-05,
+      "loss": 2.2257,
+      "step": 3
+    },
+    {
+      "epoch": 0.010759919300605245,
+      "grad_norm": 3.0478532314300537,
+      "learning_rate": 8e-05,
+      "loss": 2.5187,
+      "step": 4
+    },
+    {
+      "epoch": 0.013449899125756557,
+      "grad_norm": 5.911397457122803,
+      "learning_rate": 0.0001,
+      "loss": 2.3052,
+      "step": 5
+    },
+    {
+      "epoch": 0.01613987895090787,
+      "grad_norm": 6.985674858093262,
+      "learning_rate": 0.00012,
+      "loss": 2.2497,
+      "step": 6
+    },
+    {
+      "epoch": 0.01882985877605918,
+      "grad_norm": 12.322802543640137,
+      "learning_rate": 0.00014,
+      "loss": 2.3409,
+      "step": 7
+    },
+    {
+      "epoch": 0.02151983860121049,
+      "grad_norm": 5.827511787414551,
+      "learning_rate": 0.00016,
+      "loss": 2.4239,
+      "step": 8
+    },
+    {
+      "epoch": 0.0242098184263618,
+      "grad_norm": 4.404603481292725,
+      "learning_rate": 0.00018,
+      "loss": 2.2837,
+      "step": 9
+    },
+    {
+      "epoch": 0.026899798251513115,
+      "grad_norm": 5.033199310302734,
+      "learning_rate": 0.0002,
+      "loss": 1.9387,
+      "step": 10
+    },
+    {
+      "epoch": 0.029589778076664425,
+      "grad_norm": 5.6907196044921875,
+      "learning_rate": 0.00019999623426388962,
+      "loss": 1.7416,
+      "step": 11
+    },
+    {
+      "epoch": 0.03227975790181574,
+      "grad_norm": 6.009058952331543,
+      "learning_rate": 0.00019998493733917384,
+      "loss": 1.1876,
+      "step": 12
+    },
+    {
+      "epoch": 0.03496973772696705,
+      "grad_norm": 7.73223352432251,
+      "learning_rate": 0.00019996611007667742,
+      "loss": 1.6472,
+      "step": 13
+    },
+    {
+      "epoch": 0.03765971755211836,
+      "grad_norm": 9.206978797912598,
+      "learning_rate": 0.00019993975389437038,
+      "loss": 1.3337,
+      "step": 14
+    },
+    {
+      "epoch": 0.04034969737726967,
+      "grad_norm": 8.522892951965332,
+      "learning_rate": 0.00019990587077726128,
+      "loss": 1.107,
+      "step": 15
+    },
+    {
+      "epoch": 0.04303967720242098,
+      "grad_norm": 6.441080093383789,
+      "learning_rate": 0.0001998644632772477,
+      "loss": 1.3421,
+      "step": 16
+    },
+    {
+      "epoch": 0.04572965702757229,
+      "grad_norm": 10.297110557556152,
+      "learning_rate": 0.00019981553451292396,
+      "loss": 1.542,
+      "step": 17
+    },
+    {
+      "epoch": 0.0484196368527236,
+      "grad_norm": 8.839176177978516,
+      "learning_rate": 0.0001997590881693464,
+      "loss": 1.3331,
+      "step": 18
+    },
+    {
+      "epoch": 0.05110961667787491,
+      "grad_norm": 7.464210510253906,
+      "learning_rate": 0.00019969512849775565,
+      "loss": 0.6544,
+      "step": 19
+    },
+    {
+      "epoch": 0.05379959650302623,
+      "grad_norm": 8.324406623840332,
+      "learning_rate": 0.00019962366031525664,
+      "loss": 0.6081,
+      "step": 20
+    },
+    {
+      "epoch": 0.05648957632817754,
+      "grad_norm": 7.356775760650635,
+      "learning_rate": 0.00019954468900445566,
+      "loss": 0.8796,
+      "step": 21
+    },
+    {
+      "epoch": 0.05917955615332885,
+      "grad_norm": 4.948946475982666,
+      "learning_rate": 0.00019945822051305507,
+      "loss": 0.6637,
+      "step": 22
+    },
+    {
+      "epoch": 0.06186953597848016,
+      "grad_norm": 4.626830577850342,
+      "learning_rate": 0.00019936426135340528,
+      "loss": 0.9787,
+      "step": 23
+    },
+    {
+      "epoch": 0.06455951580363148,
+      "grad_norm": 5.476314067840576,
+      "learning_rate": 0.0001992628186020143,
+      "loss": 1.1509,
+      "step": 24
+    },
+    {
+      "epoch": 0.06724949562878278,
+      "grad_norm": 4.901749134063721,
+      "learning_rate": 0.00019915389989901474,
+      "loss": 1.2591,
+      "step": 25
+    },
+    {
+      "epoch": 0.0699394754539341,
+      "grad_norm": 5.085122108459473,
+      "learning_rate": 0.00019903751344758848,
+      "loss": 0.7551,
+      "step": 26
+    },
+    {
+      "epoch": 0.0726294552790854,
+      "grad_norm": 7.4879231452941895,
+      "learning_rate": 0.0001989136680133488,
+      "loss": 0.8804,
+      "step": 27
+    },
+    {
+      "epoch": 0.07531943510423672,
+      "grad_norm": 4.756115913391113,
+      "learning_rate": 0.00019878237292368013,
+      "loss": 0.7665,
+      "step": 28
+    },
+    {
+      "epoch": 0.07800941492938802,
+      "grad_norm": 4.777318954467773,
+      "learning_rate": 0.0001986436380670357,
+      "loss": 1.0634,
+      "step": 29
+    },
+    {
+      "epoch": 0.08069939475453934,
+      "grad_norm": 4.6226325035095215,
+      "learning_rate": 0.00019849747389219272,
+      "loss": 0.5563,
+      "step": 30
+    },
+    {
+      "epoch": 0.08338937457969066,
+      "grad_norm": 4.621855735778809,
+      "learning_rate": 0.0001983438914074654,
+      "loss": 0.6377,
+      "step": 31
+    },
+    {
+      "epoch": 0.08607935440484196,
+      "grad_norm": 4.654213905334473,
+      "learning_rate": 0.00019818290217987587,
+      "loss": 0.7768,
+      "step": 32
+    },
+    {
+      "epoch": 0.08876933422999328,
+      "grad_norm": 4.971796989440918,
+      "learning_rate": 0.00019801451833428312,
+      "loss": 0.7793,
+      "step": 33
+    },
+    {
+      "epoch": 0.09145931405514458,
+      "grad_norm": 4.303219318389893,
+      "learning_rate": 0.00019783875255246973,
+      "loss": 0.7334,
+      "step": 34
+    },
+    {
+      "epoch": 0.0941492938802959,
+      "grad_norm": 4.168240547180176,
+      "learning_rate": 0.0001976556180721867,
+      "loss": 0.7872,
+      "step": 35
+    },
+    {
+      "epoch": 0.0968392737054472,
+      "grad_norm": 3.7110626697540283,
+      "learning_rate": 0.00019746512868615656,
+      "loss": 0.9156,
+      "step": 36
+    },
+    {
+      "epoch": 0.09952925353059852,
+      "grad_norm": 4.882535457611084,
+      "learning_rate": 0.00019726729874103448,
+      "loss": 1.1328,
+      "step": 37
+    },
+    {
+      "epoch": 0.10221923335574983,
+      "grad_norm": 7.1945624351501465,
+      "learning_rate": 0.00019706214313632784,
+      "loss": 1.173,
+      "step": 38
+    },
+    {
+      "epoch": 0.10490921318090114,
+      "grad_norm": 8.771895408630371,
+      "learning_rate": 0.00019684967732327396,
+      "loss": 0.9978,
+      "step": 39
+    },
+    {
+      "epoch": 0.10759919300605246,
+      "grad_norm": 5.988046169281006,
+      "learning_rate": 0.00019662991730367663,
+      "loss": 0.7832,
+      "step": 40
+    },
+    {
+      "epoch": 0.11028917283120376,
+      "grad_norm": 12.11080551147461,
+      "learning_rate": 0.00019640287962870062,
+      "loss": 1.593,
+      "step": 41
+    },
+    {
+      "epoch": 0.11297915265635508,
+      "grad_norm": 8.772387504577637,
+      "learning_rate": 0.00019616858139762534,
+      "loss": 0.7483,
+      "step": 42
+    },
+    {
+      "epoch": 0.11566913248150638,
+      "grad_norm": 8.320154190063477,
+      "learning_rate": 0.000195927040256557,
+      "loss": 0.7829,
+      "step": 43
+    },
+    {
+      "epoch": 0.1183591123066577,
+      "grad_norm": 9.61361026763916,
+      "learning_rate": 0.00019567827439709954,
+      "loss": 1.014,
+      "step": 44
+    },
+    {
+      "epoch": 0.121049092131809,
+      "grad_norm": 5.479886531829834,
+      "learning_rate": 0.00019542230255498454,
+      "loss": 0.5207,
+      "step": 45
+    },
+    {
+      "epoch": 0.12373907195696032,
+      "grad_norm": 18.779264450073242,
+      "learning_rate": 0.0001951591440086602,
+      "loss": 1.5535,
+      "step": 46
+    },
+    {
+      "epoch": 0.12642905178211164,
+      "grad_norm": 11.512892723083496,
+      "learning_rate": 0.00019488881857783935,
+      "loss": 1.3748,
+      "step": 47
+    },
+    {
+      "epoch": 0.12911903160726296,
+      "grad_norm": 16.13918685913086,
+      "learning_rate": 0.00019461134662200668,
+      "loss": 1.1999,
+      "step": 48
+    },
+    {
+      "epoch": 0.13180901143241425,
+      "grad_norm": 11.946172714233398,
+      "learning_rate": 0.00019432674903888548,
+      "loss": 0.7801,
+      "step": 49
+    },
+    {
+      "epoch": 0.13449899125756556,
+      "grad_norm": 10.913107872009277,
+      "learning_rate": 0.0001940350472628637,
+      "loss": 0.4945,
+      "step": 50
+    },
+    {
+      "epoch": 0.13718897108271688,
+      "grad_norm": 7.555224895477295,
+      "learning_rate": 0.00019373626326337946,
+      "loss": 1.7617,
+      "step": 51
+    },
+    {
+      "epoch": 0.1398789509078682,
+      "grad_norm": 8.530354499816895,
+      "learning_rate": 0.0001934304195432668,
+      "loss": 2.1801,
+      "step": 52
+    },
+    {
+      "epoch": 0.1425689307330195,
+      "grad_norm": 6.264369487762451,
+      "learning_rate": 0.0001931175391370605,
+      "loss": 1.514,
+      "step": 53
+    },
+    {
+      "epoch": 0.1452589105581708,
+      "grad_norm": 5.179037094116211,
+      "learning_rate": 0.00019279764560926142,
+      "loss": 1.746,
+      "step": 54
+    },
+    {
+      "epoch": 0.14794889038332212,
+      "grad_norm": 4.9991536140441895,
+      "learning_rate": 0.00019247076305256176,
+      "loss": 1.4403,
+      "step": 55
+    },
+    {
+      "epoch": 0.15063887020847344,
+      "grad_norm": 5.004469871520996,
+      "learning_rate": 0.00019213691608603047,
+      "loss": 2.0104,
+      "step": 56
+    },
+    {
+      "epoch": 0.15332885003362476,
+      "grad_norm": 5.55224609375,
+      "learning_rate": 0.00019179612985325908,
+      "loss": 1.5966,
+      "step": 57
+    },
+    {
+      "epoch": 0.15601882985877605,
+      "grad_norm": 4.276426315307617,
+      "learning_rate": 0.00019144843002046806,
+      "loss": 1.3126,
+      "step": 58
+    },
+    {
+      "epoch": 0.15870880968392737,
+      "grad_norm": 6.414350509643555,
+      "learning_rate": 0.0001910938427745737,
+      "loss": 1.4917,
+      "step": 59
+    },
+    {
+      "epoch": 0.16139878950907868,
+      "grad_norm": 4.202977657318115,
+      "learning_rate": 0.000190732394821216,
+      "loss": 1.0693,
+      "step": 60
+    },
+    {
+      "epoch": 0.16408876933423,
+      "grad_norm": 4.08539342880249,
+      "learning_rate": 0.00019036411338274703,
+      "loss": 1.0895,
+      "step": 61
+    },
+    {
+      "epoch": 0.16677874915938132,
+      "grad_norm": 3.5492026805877686,
+      "learning_rate": 0.00018998902619618116,
+      "loss": 0.9263,
+      "step": 62
+    },
+    {
+      "epoch": 0.1694687289845326,
+      "grad_norm": 3.8257992267608643,
+      "learning_rate": 0.00018960716151110554,
+      "loss": 1.3858,
+      "step": 63
+    },
+    {
+      "epoch": 0.17215870880968392,
+      "grad_norm": 3.3236279487609863,
+      "learning_rate": 0.00018921854808755294,
+      "loss": 1.0358,
+      "step": 64
+    },
+    {
+      "epoch": 0.17484868863483524,
+      "grad_norm": 3.356065511703491,
+      "learning_rate": 0.00018882321519383534,
+      "loss": 0.8704,
+      "step": 65
+    },
+    {
+      "epoch": 0.17753866845998656,
+      "grad_norm": 4.296020984649658,
+      "learning_rate": 0.00018842119260433982,
+      "loss": 1.0503,
+      "step": 66
+    },
+    {
+      "epoch": 0.18022864828513785,
+      "grad_norm": 5.693877696990967,
+      "learning_rate": 0.00018801251059728604,
+      "loss": 0.6922,
+      "step": 67
+    },
+    {
+      "epoch": 0.18291862811028917,
+      "grad_norm": 5.999386310577393,
+      "learning_rate": 0.0001875971999524458,
+      "loss": 0.824,
+      "step": 68
+    },
+    {
+      "epoch": 0.18560860793544048,
+      "grad_norm": 4.900099754333496,
+      "learning_rate": 0.000187175291948825,
+      "loss": 0.7182,
+      "step": 69
+    },
+    {
+      "epoch": 0.1882985877605918,
+      "grad_norm": 4.141096115112305,
+      "learning_rate": 0.0001867468183623077,
+      "loss": 0.4006,
+      "step": 70
+    },
+    {
+      "epoch": 0.19098856758574312,
+      "grad_norm": 6.806368350982666,
+      "learning_rate": 0.00018631181146326305,
+      "loss": 0.8936,
+      "step": 71
+    },
+    {
+      "epoch": 0.1936785474108944,
+      "grad_norm": 2.945824146270752,
+      "learning_rate": 0.0001858703040141148,
+      "loss": 0.5178,
+      "step": 72
+    },
+    {
+      "epoch": 0.19636852723604573,
+      "grad_norm": 6.985172271728516,
+      "learning_rate": 0.00018542232926687383,
+      "loss": 0.8644,
+      "step": 73
+    },
+    {
+      "epoch": 0.19905850706119704,
+      "grad_norm": 5.231998920440674,
+      "learning_rate": 0.0001849679209606338,
+      "loss": 1.0585,
+      "step": 74
+    },
+    {
+      "epoch": 0.20174848688634836,
+      "grad_norm": 4.8978705406188965,
+      "learning_rate": 0.00018450711331903006,
+      "loss": 0.7828,
+      "step": 75
+    },
+    {
+      "epoch": 0.20443846671149965,
+      "grad_norm": 5.309878826141357,
+      "learning_rate": 0.00018403994104766212,
+      "loss": 0.7666,
+      "step": 76
+    },
+    {
+      "epoch": 0.20712844653665097,
+      "grad_norm": 5.227763652801514,
+      "learning_rate": 0.00018356643933147986,
+      "loss": 0.8396,
+      "step": 77
+    },
+    {
+      "epoch": 0.20981842636180228,
+      "grad_norm": 4.239619731903076,
+      "learning_rate": 0.00018308664383213344,
+      "loss": 0.6439,
+      "step": 78
+    },
+    {
+      "epoch": 0.2125084061869536,
+      "grad_norm": 5.731531620025635,
+      "learning_rate": 0.00018260059068528762,
+      "loss": 1.3371,
+      "step": 79
+    },
+    {
+      "epoch": 0.21519838601210492,
+      "grad_norm": 4.006597518920898,
+      "learning_rate": 0.00018210831649790018,
+      "loss": 0.5272,
+      "step": 80
+    },
+    {
+      "epoch": 0.2178883658372562,
+      "grad_norm": 4.0596699714660645,
+      "learning_rate": 0.00018160985834546475,
+      "loss": 0.4416,
+      "step": 81
+    },
+    {
+      "epoch": 0.22057834566240753,
+      "grad_norm": 4.053659915924072,
+      "learning_rate": 0.00018110525376921862,
+      "loss": 0.4781,
+      "step": 82
+    },
+    {
+      "epoch": 0.22326832548755884,
+      "grad_norm": 4.120569705963135,
+      "learning_rate": 0.00018059454077331527,
+      "loss": 0.8082,
+      "step": 83
+    },
+    {
+      "epoch": 0.22595830531271016,
+      "grad_norm": 6.420701503753662,
+      "learning_rate": 0.00018007775782196214,
+      "loss": 0.5476,
+      "step": 84
+    },
+    {
+      "epoch": 0.22864828513786148,
+      "grad_norm": 13.603157997131348,
+      "learning_rate": 0.00017955494383652365,
+      "loss": 0.7857,
+      "step": 85
+    },
+    {
+      "epoch": 0.23133826496301277,
+      "grad_norm": 7.784971237182617,
+      "learning_rate": 0.00017902613819258985,
+      "loss": 0.7705,
+      "step": 86
+    },
+    {
+      "epoch": 0.23402824478816409,
+      "grad_norm": 5.966789722442627,
+      "learning_rate": 0.00017849138071701092,
+      "loss": 0.9065,
+      "step": 87
+    },
+    {
+      "epoch": 0.2367182246133154,
+      "grad_norm": 9.451849937438965,
+      "learning_rate": 0.0001779507116848976,
+      "loss": 0.998,
+      "step": 88
+    },
+    {
+      "epoch": 0.23940820443846672,
+      "grad_norm": 6.712332725524902,
+      "learning_rate": 0.00017740417181658788,
+      "loss": 1.1464,
+      "step": 89
+    },
+    {
+      "epoch": 0.242098184263618,
+      "grad_norm": 4.188553333282471,
+      "learning_rate": 0.00017685180227458003,
+      "loss": 0.6356,
+      "step": 90
+    },
+    {
+      "epoch": 0.24478816408876933,
+      "grad_norm": 9.703150749206543,
+      "learning_rate": 0.00017629364466043273,
+      "loss": 1.0548,
+      "step": 91
+    },
+    {
+      "epoch": 0.24747814391392065,
+      "grad_norm": 6.08168363571167,
+      "learning_rate": 0.00017572974101163165,
+      "loss": 0.7252,
+      "step": 92
+    },
+    {
+      "epoch": 0.25016812373907193,
+      "grad_norm": 10.167409896850586,
+      "learning_rate": 0.00017516013379842337,
+      "loss": 0.658,
+      "step": 93
+    },
+    {
+      "epoch": 0.25016812373907193,
+      "eval_loss": 0.9310864210128784,
+      "eval_runtime": 10.8326,
+      "eval_samples_per_second": 14.493,
+      "eval_steps_per_second": 7.293,
+      "step": 93
+    }
+  ],
+  "logging_steps": 1,
+  "max_steps": 372,
+  "num_input_tokens_seen": 0,
+  "num_train_epochs": 2,
+  "save_steps": 93,
+  "stateful_callbacks": {
+    "TrainerControl": {
+      "args": {
+        "should_epoch_stop": false,
+        "should_evaluate": false,
+        "should_log": false,
+        "should_save": true,
+        "should_training_stop": false
+      },
+      "attributes": {}
+    }
+  },
+  "total_flos": 3.029473590194995e+16,
+  "train_batch_size": 2,
+  "trial_name": null,
+  "trial_params": null
+}

last-checkpoint/training_args.bin

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:827f5a2b3225b16377a974c3d1db141246397228e535d09bc318973ba999772a
+size 6840