Training in progress, step 150, checkpoint

last-checkpoint/README.md

@@ -0,0 +1,202 @@
+---
+base_model: samoline/fae73c60-74aa-4322-9e1c-f682bc8ef18c
+library_name: peft
+---
+
+# Model Card for Model ID
+
+<!-- Provide a quick summary of what the model is/does. -->
+
+
+
+## Model Details
+
+### Model Description
+
+<!-- Provide a longer summary of what this model is. -->
+
+
+
+- **Developed by:** [More Information Needed]
+- **Funded by [optional]:** [More Information Needed]
+- **Shared by [optional]:** [More Information Needed]
+- **Model type:** [More Information Needed]
+- **Language(s) (NLP):** [More Information Needed]
+- **License:** [More Information Needed]
+- **Finetuned from model [optional]:** [More Information Needed]
+
+### Model Sources [optional]
+
+<!-- Provide the basic links for the model. -->
+
+- **Repository:** [More Information Needed]
+- **Paper [optional]:** [More Information Needed]
+- **Demo [optional]:** [More Information Needed]
+
+## Uses
+
+<!-- Address questions around how the model is intended to be used, including the foreseeable users of the model and those affected by the model. -->
+
+### Direct Use
+
+<!-- This section is for the model use without fine-tuning or plugging into a larger ecosystem/app. -->
+
+[More Information Needed]
+
+### Downstream Use [optional]
+
+<!-- This section is for the model use when fine-tuned for a task, or when plugged into a larger ecosystem/app -->
+
+[More Information Needed]
+
+### Out-of-Scope Use
+
+<!-- This section addresses misuse, malicious use, and uses that the model will not work well for. -->
+
+[More Information Needed]
+
+## Bias, Risks, and Limitations
+
+<!-- This section is meant to convey both technical and sociotechnical limitations. -->
+
+[More Information Needed]
+
+### Recommendations
+
+<!-- This section is meant to convey recommendations with respect to the bias, risk, and technical limitations. -->
+
+Users (both direct and downstream) should be made aware of the risks, biases and limitations of the model. More information needed for further recommendations.
+
+## How to Get Started with the Model
+
+Use the code below to get started with the model.
+
+[More Information Needed]
+
+## Training Details
+
+### Training Data
+
+<!-- This should link to a Dataset Card, perhaps with a short stub of information on what the training data is all about as well as documentation related to data pre-processing or additional filtering. -->
+
+[More Information Needed]
+
+### Training Procedure
+
+<!-- This relates heavily to the Technical Specifications. Content here should link to that section when it is relevant to the training procedure. -->
+
+#### Preprocessing [optional]
+
+[More Information Needed]
+
+
+#### Training Hyperparameters
+
+- **Training regime:** [More Information Needed] <!--fp32, fp16 mixed precision, bf16 mixed precision, bf16 non-mixed precision, fp16 non-mixed precision, fp8 mixed precision -->
+
+#### Speeds, Sizes, Times [optional]
+
+<!-- This section provides information about throughput, start/end time, checkpoint size if relevant, etc. -->
+
+[More Information Needed]
+
+## Evaluation
+
+<!-- This section describes the evaluation protocols and provides the results. -->
+
+### Testing Data, Factors & Metrics
+
+#### Testing Data
+
+<!-- This should link to a Dataset Card if possible. -->
+
+[More Information Needed]
+
+#### Factors
+
+<!-- These are the things the evaluation is disaggregating by, e.g., subpopulations or domains. -->
+
+[More Information Needed]
+
+#### Metrics
+
+<!-- These are the evaluation metrics being used, ideally with a description of why. -->
+
+[More Information Needed]
+
+### Results
+
+[More Information Needed]
+
+#### Summary
+
+
+
+## Model Examination [optional]
+
+<!-- Relevant interpretability work for the model goes here -->
+
+[More Information Needed]
+
+## Environmental Impact
+
+<!-- Total emissions (in grams of CO2eq) and additional considerations, such as electricity usage, go here. Edit the suggested text below accordingly -->
+
+Carbon emissions can be estimated using the [Machine Learning Impact calculator](https://mlco2.github.io/impact#compute) presented in [Lacoste et al. (2019)](https://arxiv.org/abs/1910.09700).
+
+- **Hardware Type:** [More Information Needed]
+- **Hours used:** [More Information Needed]
+- **Cloud Provider:** [More Information Needed]
+- **Compute Region:** [More Information Needed]
+- **Carbon Emitted:** [More Information Needed]
+
+## Technical Specifications [optional]
+
+### Model Architecture and Objective
+
+[More Information Needed]
+
+### Compute Infrastructure
+
+[More Information Needed]
+
+#### Hardware
+
+[More Information Needed]
+
+#### Software
+
+[More Information Needed]
+
+## Citation [optional]
+
+<!-- If there is a paper or blog post introducing the model, the APA and Bibtex information for that should go in this section. -->
+
+**BibTeX:**
+
+[More Information Needed]
+
+**APA:**
+
+[More Information Needed]
+
+## Glossary [optional]
+
+<!-- If relevant, include terms and calculations in this section that can help readers understand the model or model card. -->
+
+[More Information Needed]
+
+## More Information [optional]
+
+[More Information Needed]
+
+## Model Card Authors [optional]
+
+[More Information Needed]
+
+## Model Card Contact
+
+[More Information Needed]
+### Framework versions
+
+- PEFT 0.13.2

last-checkpoint/adapter_config.json

@@ -0,0 +1,34 @@
+{
+  "alpha_pattern": {},
+  "auto_mapping": null,
+  "base_model_name_or_path": "samoline/fae73c60-74aa-4322-9e1c-f682bc8ef18c",
+  "bias": "none",
+  "fan_in_fan_out": null,
+  "inference_mode": true,
+  "init_lora_weights": true,
+  "layer_replication": null,
+  "layers_pattern": null,
+  "layers_to_transform": null,
+  "loftq_config": {},
+  "lora_alpha": 128,
+  "lora_dropout": 0.3,
+  "megatron_config": null,
+  "megatron_core": "megatron.core",
+  "modules_to_save": null,
+  "peft_type": "LORA",
+  "r": 64,
+  "rank_pattern": {},
+  "revision": null,
+  "target_modules": [
+    "up_proj",
+    "v_proj",
+    "q_proj",
+    "down_proj",
+    "o_proj",
+    "gate_proj",
+    "k_proj"
+  ],
+  "task_type": "CAUSAL_LM",
+  "use_dora": false,
+  "use_rslora": false
+}

last-checkpoint/adapter_model.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:831a3e01b34c2529de75bfb28518bb658ddb626c9b6afb9d82fc09b7a0bd447c
+size 411094576

last-checkpoint/added_tokens.json

@@ -0,0 +1,5 @@
+{
+  "<|endofprompt|>": 100276,
+  "<|im_end|>": 100265,
+  "<|im_start|>": 100264
+}

last-checkpoint/optimizer.pt

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:b9d7d1b98e5380c6fe0eaa615849ead5e75647a603da6b69903a53122387b2f2
+size 209193332

last-checkpoint/rng_state.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:7adb5297a759a1832155916e7ad694104cf5c644cb7a9d48a6e6e254da29c525
+size 14244

last-checkpoint/scheduler.pt

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:bcf6c2743f0632628c20d16fde51d82be9077fac7fbc2c9011927c3124f1796c
+size 1064

last-checkpoint/special_tokens_map.json

@@ -0,0 +1,30 @@
+{
+  "bos_token": {
+    "content": "<|endoftext|>",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  },
+  "eos_token": {
+    "content": "<|endoftext|>",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  },
+  "pad_token": {
+    "content": "<|endoftext|>",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  },
+  "unk_token": {
+    "content": "<|endoftext|>",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  }
+}

last-checkpoint/tokenizer_config.json

@@ -0,0 +1,70 @@
+{
+  "add_prefix_space": false,
+  "added_tokens_decoder": {
+    "100257": {
+      "content": "<|endoftext|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "100258": {
+      "content": "<|fim_prefix|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "100259": {
+      "content": "<|fim_middle|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "100260": {
+      "content": "<|fim_suffix|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "100264": {
+      "content": "<|im_start|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "100265": {
+      "content": "<|im_end|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "100276": {
+      "content": "<|endofprompt|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    }
+  },
+  "bos_token": "<|endoftext|>",
+  "chat_template": "{% if not add_generation_prompt is defined %}{% set add_generation_prompt = false %}{% endif %}{% set loop_messages = messages %}{% for message in loop_messages %}{% set content = '<|start_header_id|>' + message['role'] + '<|end_header_id|>\n\n'+ message['content'] | trim + '<|eot_id|>' %}{% if loop.index0 == 0 %}{% set content = bos_token + content %}{% endif %}{{ content }}{% endfor %}{% if add_generation_prompt %}{{ '<|start_header_id|>assistant<|end_header_id|>\n\n' }}{% endif %}",
+  "clean_up_tokenization_spaces": false,
+  "eos_token": "<|endoftext|>",
+  "extra_special_tokens": {},
+  "model_max_length": 8192,
+  "pad_token": "<|endoftext|>",
+  "tokenizer_class": "GPT2Tokenizer",
+  "unk_token": "<|endoftext|>"
+}

last-checkpoint/trainer_state.json

@@ -0,0 +1,1108 @@
+{
+  "best_metric": 0.9891857504844666,
+  "best_model_checkpoint": "miner_id_24/checkpoint-150",
+  "epoch": 0.0019940113193375893,
+  "eval_steps": 150,
+  "global_step": 150,
+  "is_hyper_param_search": false,
+  "is_local_process_zero": true,
+  "is_world_process_zero": true,
+  "log_history": [
+    {
+      "epoch": 1.329340879558393e-05,
+      "grad_norm": 1.9388657808303833,
+      "learning_rate": 5e-06,
+      "loss": 1.3911,
+      "step": 1
+    },
+    {
+      "epoch": 1.329340879558393e-05,
+      "eval_loss": 1.8459720611572266,
+      "eval_runtime": 3461.4381,
+      "eval_samples_per_second": 36.602,
+      "eval_steps_per_second": 9.151,
+      "step": 1
+    },
+    {
+      "epoch": 2.658681759116786e-05,
+      "grad_norm": 2.8200104236602783,
+      "learning_rate": 1e-05,
+      "loss": 1.3589,
+      "step": 2
+    },
+    {
+      "epoch": 3.988022638675179e-05,
+      "grad_norm": 2.7405104637145996,
+      "learning_rate": 1.5e-05,
+      "loss": 1.3274,
+      "step": 3
+    },
+    {
+      "epoch": 5.317363518233572e-05,
+      "grad_norm": 2.6861093044281006,
+      "learning_rate": 2e-05,
+      "loss": 1.2136,
+      "step": 4
+    },
+    {
+      "epoch": 6.646704397791965e-05,
+      "grad_norm": 2.4807560443878174,
+      "learning_rate": 2.5e-05,
+      "loss": 1.2634,
+      "step": 5
+    },
+    {
+      "epoch": 7.976045277350358e-05,
+      "grad_norm": 3.396902322769165,
+      "learning_rate": 3e-05,
+      "loss": 1.1798,
+      "step": 6
+    },
+    {
+      "epoch": 9.305386156908751e-05,
+      "grad_norm": 2.9391775131225586,
+      "learning_rate": 3.5e-05,
+      "loss": 1.2212,
+      "step": 7
+    },
+    {
+      "epoch": 0.00010634727036467144,
+      "grad_norm": 2.992940902709961,
+      "learning_rate": 4e-05,
+      "loss": 1.1953,
+      "step": 8
+    },
+    {
+      "epoch": 0.00011964067916025536,
+      "grad_norm": 3.2214736938476562,
+      "learning_rate": 4.5e-05,
+      "loss": 1.1323,
+      "step": 9
+    },
+    {
+      "epoch": 0.0001329340879558393,
+      "grad_norm": 3.6011428833007812,
+      "learning_rate": 5e-05,
+      "loss": 1.1466,
+      "step": 10
+    },
+    {
+      "epoch": 0.00014622749675142322,
+      "grad_norm": 3.9849705696105957,
+      "learning_rate": 5.500000000000001e-05,
+      "loss": 1.0944,
+      "step": 11
+    },
+    {
+      "epoch": 0.00015952090554700716,
+      "grad_norm": 3.5957984924316406,
+      "learning_rate": 6e-05,
+      "loss": 1.171,
+      "step": 12
+    },
+    {
+      "epoch": 0.00017281431434259108,
+      "grad_norm": 2.5957956314086914,
+      "learning_rate": 6.500000000000001e-05,
+      "loss": 1.074,
+      "step": 13
+    },
+    {
+      "epoch": 0.00018610772313817502,
+      "grad_norm": 3.4556539058685303,
+      "learning_rate": 7e-05,
+      "loss": 1.0507,
+      "step": 14
+    },
+    {
+      "epoch": 0.00019940113193375893,
+      "grad_norm": 3.1671648025512695,
+      "learning_rate": 7.500000000000001e-05,
+      "loss": 1.0411,
+      "step": 15
+    },
+    {
+      "epoch": 0.00021269454072934287,
+      "grad_norm": 2.902188539505005,
+      "learning_rate": 8e-05,
+      "loss": 1.0133,
+      "step": 16
+    },
+    {
+      "epoch": 0.00022598794952492681,
+      "grad_norm": 2.787954568862915,
+      "learning_rate": 8.5e-05,
+      "loss": 1.0352,
+      "step": 17
+    },
+    {
+      "epoch": 0.00023928135832051073,
+      "grad_norm": 2.525041103363037,
+      "learning_rate": 9e-05,
+      "loss": 0.9911,
+      "step": 18
+    },
+    {
+      "epoch": 0.00025257476711609464,
+      "grad_norm": 2.6477138996124268,
+      "learning_rate": 9.5e-05,
+      "loss": 0.9524,
+      "step": 19
+    },
+    {
+      "epoch": 0.0002658681759116786,
+      "grad_norm": 2.82763409614563,
+      "learning_rate": 0.0001,
+      "loss": 1.0342,
+      "step": 20
+    },
+    {
+      "epoch": 0.0002791615847072625,
+      "grad_norm": 3.421452760696411,
+      "learning_rate": 9.999866555428618e-05,
+      "loss": 0.9806,
+      "step": 21
+    },
+    {
+      "epoch": 0.00029245499350284644,
+      "grad_norm": 2.983229398727417,
+      "learning_rate": 9.999466228837451e-05,
+      "loss": 0.8953,
+      "step": 22
+    },
+    {
+      "epoch": 0.00030574840229843035,
+      "grad_norm": 3.4214861392974854,
+      "learning_rate": 9.998799041595064e-05,
+      "loss": 0.9724,
+      "step": 23
+    },
+    {
+      "epoch": 0.0003190418110940143,
+      "grad_norm": 3.5156090259552,
+      "learning_rate": 9.997865029314463e-05,
+      "loss": 0.9613,
+      "step": 24
+    },
+    {
+      "epoch": 0.00033233521988959824,
+      "grad_norm": 3.8509676456451416,
+      "learning_rate": 9.996664241851197e-05,
+      "loss": 0.913,
+      "step": 25
+    },
+    {
+      "epoch": 0.00034562862868518215,
+      "grad_norm": 3.1751909255981445,
+      "learning_rate": 9.995196743300692e-05,
+      "loss": 1.1327,
+      "step": 26
+    },
+    {
+      "epoch": 0.0003589220374807661,
+      "grad_norm": 3.8788204193115234,
+      "learning_rate": 9.993462611994832e-05,
+      "loss": 0.7996,
+      "step": 27
+    },
+    {
+      "epoch": 0.00037221544627635004,
+      "grad_norm": 3.27899432182312,
+      "learning_rate": 9.991461940497786e-05,
+      "loss": 0.9915,
+      "step": 28
+    },
+    {
+      "epoch": 0.00038550885507193395,
+      "grad_norm": 3.78037691116333,
+      "learning_rate": 9.989194835601048e-05,
+      "loss": 1.0367,
+      "step": 29
+    },
+    {
+      "epoch": 0.00039880226386751786,
+      "grad_norm": 4.236947536468506,
+      "learning_rate": 9.986661418317759e-05,
+      "loss": 0.9108,
+      "step": 30
+    },
+    {
+      "epoch": 0.00041209567266310183,
+      "grad_norm": 3.913956642150879,
+      "learning_rate": 9.983861823876231e-05,
+      "loss": 1.0729,
+      "step": 31
+    },
+    {
+      "epoch": 0.00042538908145868575,
+      "grad_norm": 4.501612186431885,
+      "learning_rate": 9.980796201712734e-05,
+      "loss": 1.0856,
+      "step": 32
+    },
+    {
+      "epoch": 0.00043868249025426966,
+      "grad_norm": 4.474135875701904,
+      "learning_rate": 9.977464715463524e-05,
+      "loss": 0.897,
+      "step": 33
+    },
+    {
+      "epoch": 0.00045197589904985363,
+      "grad_norm": 5.405597686767578,
+      "learning_rate": 9.973867542956104e-05,
+      "loss": 1.1942,
+      "step": 34
+    },
+    {
+      "epoch": 0.00046526930784543754,
+      "grad_norm": 3.8748316764831543,
+      "learning_rate": 9.97000487619973e-05,
+      "loss": 1.1122,
+      "step": 35
+    },
+    {
+      "epoch": 0.00047856271664102146,
+      "grad_norm": 3.826747417449951,
+      "learning_rate": 9.965876921375165e-05,
+      "loss": 1.097,
+      "step": 36
+    },
+    {
+      "epoch": 0.0004918561254366054,
+      "grad_norm": 3.8462743759155273,
+      "learning_rate": 9.961483898823678e-05,
+      "loss": 0.8577,
+      "step": 37
+    },
+    {
+      "epoch": 0.0005051495342321893,
+      "grad_norm": 4.05244255065918,
+      "learning_rate": 9.956826043035268e-05,
+      "loss": 0.9462,
+      "step": 38
+    },
+    {
+      "epoch": 0.0005184429430277732,
+      "grad_norm": 4.053645610809326,
+      "learning_rate": 9.951903602636166e-05,
+      "loss": 0.9335,
+      "step": 39
+    },
+    {
+      "epoch": 0.0005317363518233572,
+      "grad_norm": 4.824429988861084,
+      "learning_rate": 9.946716840375551e-05,
+      "loss": 1.0454,
+      "step": 40
+    },
+    {
+      "epoch": 0.0005450297606189411,
+      "grad_norm": 5.300111770629883,
+      "learning_rate": 9.94126603311153e-05,
+      "loss": 0.9345,
+      "step": 41
+    },
+    {
+      "epoch": 0.000558323169414525,
+      "grad_norm": 4.619444847106934,
+      "learning_rate": 9.935551471796358e-05,
+      "loss": 1.0301,
+      "step": 42
+    },
+    {
+      "epoch": 0.000571616578210109,
+      "grad_norm": 4.485474586486816,
+      "learning_rate": 9.92957346146091e-05,
+      "loss": 0.9127,
+      "step": 43
+    },
+    {
+      "epoch": 0.0005849099870056929,
+      "grad_norm": 5.240790843963623,
+      "learning_rate": 9.923332321198395e-05,
+      "loss": 1.1571,
+      "step": 44
+    },
+    {
+      "epoch": 0.0005982033958012768,
+      "grad_norm": 5.170898914337158,
+      "learning_rate": 9.916828384147331e-05,
+      "loss": 1.0133,
+      "step": 45
+    },
+    {
+      "epoch": 0.0006114968045968607,
+      "grad_norm": 6.1428446769714355,
+      "learning_rate": 9.910061997473752e-05,
+      "loss": 1.0003,
+      "step": 46
+    },
+    {
+      "epoch": 0.0006247902133924447,
+      "grad_norm": 5.351384162902832,
+      "learning_rate": 9.903033522352687e-05,
+      "loss": 0.999,
+      "step": 47
+    },
+    {
+      "epoch": 0.0006380836221880286,
+      "grad_norm": 5.867308616638184,
+      "learning_rate": 9.895743333948874e-05,
+      "loss": 1.0887,
+      "step": 48
+    },
+    {
+      "epoch": 0.0006513770309836126,
+      "grad_norm": 7.773362159729004,
+      "learning_rate": 9.888191821396744e-05,
+      "loss": 1.1315,
+      "step": 49
+    },
+    {
+      "epoch": 0.0006646704397791965,
+      "grad_norm": 9.756021499633789,
+      "learning_rate": 9.880379387779637e-05,
+      "loss": 1.2055,
+      "step": 50
+    },
+    {
+      "epoch": 0.0006779638485747804,
+      "grad_norm": 3.411668062210083,
+      "learning_rate": 9.872306450108292e-05,
+      "loss": 1.4152,
+      "step": 51
+    },
+    {
+      "epoch": 0.0006912572573703643,
+      "grad_norm": 2.6563198566436768,
+      "learning_rate": 9.863973439298597e-05,
+      "loss": 1.1962,
+      "step": 52
+    },
+    {
+      "epoch": 0.0007045506661659482,
+      "grad_norm": 2.5267629623413086,
+      "learning_rate": 9.855380800148572e-05,
+      "loss": 1.1744,
+      "step": 53
+    },
+    {
+      "epoch": 0.0007178440749615322,
+      "grad_norm": 3.2284536361694336,
+      "learning_rate": 9.846528991314639e-05,
+      "loss": 0.9512,
+      "step": 54
+    },
+    {
+      "epoch": 0.0007311374837571162,
+      "grad_norm": 2.0135741233825684,
+      "learning_rate": 9.837418485287127e-05,
+      "loss": 1.0179,
+      "step": 55
+    },
+    {
+      "epoch": 0.0007444308925527001,
+      "grad_norm": 2.1273906230926514,
+      "learning_rate": 9.828049768365068e-05,
+      "loss": 0.9856,
+      "step": 56
+    },
+    {
+      "epoch": 0.000757724301348284,
+      "grad_norm": 2.432741165161133,
+      "learning_rate": 9.818423340630228e-05,
+      "loss": 1.0059,
+      "step": 57
+    },
+    {
+      "epoch": 0.0007710177101438679,
+      "grad_norm": 2.2927942276000977,
+      "learning_rate": 9.808539715920414e-05,
+      "loss": 1.2061,
+      "step": 58
+    },
+    {
+      "epoch": 0.0007843111189394518,
+      "grad_norm": 2.312389850616455,
+      "learning_rate": 9.798399421802056e-05,
+      "loss": 0.8977,
+      "step": 59
+    },
+    {
+      "epoch": 0.0007976045277350357,
+      "grad_norm": 2.4386355876922607,
+      "learning_rate": 9.78800299954203e-05,
+      "loss": 0.8272,
+      "step": 60
+    },
+    {
+      "epoch": 0.0008108979365306198,
+      "grad_norm": 1.9699569940567017,
+      "learning_rate": 9.777351004078783e-05,
+      "loss": 1.087,
+      "step": 61
+    },
+    {
+      "epoch": 0.0008241913453262037,
+      "grad_norm": 2.6284430027008057,
+      "learning_rate": 9.766444003992703e-05,
+      "loss": 1.1052,
+      "step": 62
+    },
+    {
+      "epoch": 0.0008374847541217876,
+      "grad_norm": 2.237579822540283,
+      "learning_rate": 9.755282581475769e-05,
+      "loss": 0.9417,
+      "step": 63
+    },
+    {
+      "epoch": 0.0008507781629173715,
+      "grad_norm": 2.1193315982818604,
+      "learning_rate": 9.743867332300478e-05,
+      "loss": 0.9962,
+      "step": 64
+    },
+    {
+      "epoch": 0.0008640715717129554,
+      "grad_norm": 2.354325294494629,
+      "learning_rate": 9.732198865788047e-05,
+      "loss": 1.0707,
+      "step": 65
+    },
+    {
+      "epoch": 0.0008773649805085393,
+      "grad_norm": 2.586552858352661,
+      "learning_rate": 9.72027780477588e-05,
+      "loss": 0.9973,
+      "step": 66
+    },
+    {
+      "epoch": 0.0008906583893041232,
+      "grad_norm": 2.397660255432129,
+      "learning_rate": 9.708104785584323e-05,
+      "loss": 0.9432,
+      "step": 67
+    },
+    {
+      "epoch": 0.0009039517980997073,
+      "grad_norm": 2.3507635593414307,
+      "learning_rate": 9.695680457982713e-05,
+      "loss": 1.0085,
+      "step": 68
+    },
+    {
+      "epoch": 0.0009172452068952912,
+      "grad_norm": 2.0967140197753906,
+      "learning_rate": 9.683005485154677e-05,
+      "loss": 0.9675,
+      "step": 69
+    },
+    {
+      "epoch": 0.0009305386156908751,
+      "grad_norm": 2.5464537143707275,
+      "learning_rate": 9.67008054366274e-05,
+      "loss": 0.871,
+      "step": 70
+    },
+    {
+      "epoch": 0.000943832024486459,
+      "grad_norm": 2.32565975189209,
+      "learning_rate": 9.656906323412217e-05,
+      "loss": 0.9763,
+      "step": 71
+    },
+    {
+      "epoch": 0.0009571254332820429,
+      "grad_norm": 2.7051308155059814,
+      "learning_rate": 9.643483527614372e-05,
+      "loss": 1.1479,
+      "step": 72
+    },
+    {
+      "epoch": 0.0009704188420776268,
+      "grad_norm": 2.812052011489868,
+      "learning_rate": 9.629812872748901e-05,
+      "loss": 0.8656,
+      "step": 73
+    },
+    {
+      "epoch": 0.0009837122508732107,
+      "grad_norm": 2.9575998783111572,
+      "learning_rate": 9.615895088525677e-05,
+      "loss": 0.9026,
+      "step": 74
+    },
+    {
+      "epoch": 0.0009970056596687947,
+      "grad_norm": 2.4681830406188965,
+      "learning_rate": 9.601730917845797e-05,
+      "loss": 1.0127,
+      "step": 75
+    },
+    {
+      "epoch": 0.0010102990684643786,
+      "grad_norm": 2.636371374130249,
+      "learning_rate": 9.587321116761938e-05,
+      "loss": 0.9537,
+      "step": 76
+    },
+    {
+      "epoch": 0.0010235924772599625,
+      "grad_norm": 4.0426554679870605,
+      "learning_rate": 9.57266645443799e-05,
+      "loss": 0.9712,
+      "step": 77
+    },
+    {
+      "epoch": 0.0010368858860555464,
+      "grad_norm": 2.898866653442383,
+      "learning_rate": 9.557767713108009e-05,
+      "loss": 0.8165,
+      "step": 78
+    },
+    {
+      "epoch": 0.0010501792948511305,
+      "grad_norm": 3.4302549362182617,
+      "learning_rate": 9.542625688034449e-05,
+      "loss": 0.9472,
+      "step": 79
+    },
+    {
+      "epoch": 0.0010634727036467144,
+      "grad_norm": 2.756509304046631,
+      "learning_rate": 9.527241187465734e-05,
+      "loss": 0.7932,
+      "step": 80
+    },
+    {
+      "epoch": 0.0010767661124422984,
+      "grad_norm": 2.6443912982940674,
+      "learning_rate": 9.511615032593096e-05,
+      "loss": 0.8982,
+      "step": 81
+    },
+    {
+      "epoch": 0.0010900595212378823,
+      "grad_norm": 2.9568185806274414,
+      "learning_rate": 9.49574805750675e-05,
+      "loss": 0.8393,
+      "step": 82
+    },
+    {
+      "epoch": 0.0011033529300334662,
+      "grad_norm": 3.2217254638671875,
+      "learning_rate": 9.479641109151373e-05,
+      "loss": 1.0166,
+      "step": 83
+    },
+    {
+      "epoch": 0.00111664633882905,
+      "grad_norm": 2.9364144802093506,
+      "learning_rate": 9.463295047280891e-05,
+      "loss": 0.9075,
+      "step": 84
+    },
+    {
+      "epoch": 0.001129939747624634,
+      "grad_norm": 3.6510021686553955,
+      "learning_rate": 9.446710744412595e-05,
+      "loss": 0.9151,
+      "step": 85
+    },
+    {
+      "epoch": 0.001143233156420218,
+      "grad_norm": 3.259711742401123,
+      "learning_rate": 9.429889085780557e-05,
+      "loss": 0.7931,
+      "step": 86
+    },
+    {
+      "epoch": 0.0011565265652158018,
+      "grad_norm": 3.346482276916504,
+      "learning_rate": 9.41283096928839e-05,
+      "loss": 0.8724,
+      "step": 87
+    },
+    {
+      "epoch": 0.0011698199740113858,
+      "grad_norm": 3.270686626434326,
+      "learning_rate": 9.395537305461311e-05,
+      "loss": 0.7726,
+      "step": 88
+    },
+    {
+      "epoch": 0.0011831133828069697,
+      "grad_norm": 3.747311592102051,
+      "learning_rate": 9.378009017397542e-05,
+      "loss": 0.8455,
+      "step": 89
+    },
+    {
+      "epoch": 0.0011964067916025536,
+      "grad_norm": 3.821293592453003,
+      "learning_rate": 9.360247040719039e-05,
+      "loss": 0.757,
+      "step": 90
+    },
+    {
+      "epoch": 0.0012097002003981375,
+      "grad_norm": 4.054718017578125,
+      "learning_rate": 9.342252323521545e-05,
+      "loss": 0.9279,
+      "step": 91
+    },
+    {
+      "epoch": 0.0012229936091937214,
+      "grad_norm": 3.5439743995666504,
+      "learning_rate": 9.324025826323994e-05,
+      "loss": 0.7374,
+      "step": 92
+    },
+    {
+      "epoch": 0.0012362870179893056,
+      "grad_norm": 4.841712951660156,
+      "learning_rate": 9.305568522017227e-05,
+      "loss": 0.7237,
+      "step": 93
+    },
+    {
+      "epoch": 0.0012495804267848895,
+      "grad_norm": 5.552839279174805,
+      "learning_rate": 9.286881395812066e-05,
+      "loss": 0.8263,
+      "step": 94
+    },
+    {
+      "epoch": 0.0012628738355804734,
+      "grad_norm": 5.980641841888428,
+      "learning_rate": 9.267965445186733e-05,
+      "loss": 1.141,
+      "step": 95
+    },
+    {
+      "epoch": 0.0012761672443760573,
+      "grad_norm": 5.712211608886719,
+      "learning_rate": 9.248821679833596e-05,
+      "loss": 1.0009,
+      "step": 96
+    },
+    {
+      "epoch": 0.0012894606531716412,
+      "grad_norm": 4.681507110595703,
+      "learning_rate": 9.229451121605279e-05,
+      "loss": 0.8064,
+      "step": 97
+    },
+    {
+      "epoch": 0.0013027540619672251,
+      "grad_norm": 5.275393009185791,
+      "learning_rate": 9.209854804460121e-05,
+      "loss": 0.9145,
+      "step": 98
+    },
+    {
+      "epoch": 0.001316047470762809,
+      "grad_norm": 5.1414361000061035,
+      "learning_rate": 9.190033774406977e-05,
+      "loss": 0.92,
+      "step": 99
+    },
+    {
+      "epoch": 0.001329340879558393,
+      "grad_norm": 6.979901313781738,
+      "learning_rate": 9.16998908944939e-05,
+      "loss": 0.9631,
+      "step": 100
+    },
+    {
+      "epoch": 0.0013426342883539769,
+      "grad_norm": 3.0721938610076904,
+      "learning_rate": 9.149721819529119e-05,
+      "loss": 1.3745,
+      "step": 101
+    },
+    {
+      "epoch": 0.0013559276971495608,
+      "grad_norm": 2.620919704437256,
+      "learning_rate": 9.129233046469022e-05,
+      "loss": 1.2247,
+      "step": 102
+    },
+    {
+      "epoch": 0.0013692211059451447,
+      "grad_norm": 2.184368371963501,
+      "learning_rate": 9.108523863915314e-05,
+      "loss": 1.0819,
+      "step": 103
+    },
+    {
+      "epoch": 0.0013825145147407286,
+      "grad_norm": 1.9781068563461304,
+      "learning_rate": 9.087595377279192e-05,
+      "loss": 1.1311,
+      "step": 104
+    },
+    {
+      "epoch": 0.0013958079235363125,
+      "grad_norm": 1.648494005203247,
+      "learning_rate": 9.066448703677828e-05,
+      "loss": 1.0057,
+      "step": 105
+    },
+    {
+      "epoch": 0.0014091013323318964,
+      "grad_norm": 1.681408405303955,
+      "learning_rate": 9.045084971874738e-05,
+      "loss": 0.9557,
+      "step": 106
+    },
+    {
+      "epoch": 0.0014223947411274806,
+      "grad_norm": 1.9565364122390747,
+      "learning_rate": 9.023505322219536e-05,
+      "loss": 0.9797,
+      "step": 107
+    },
+    {
+      "epoch": 0.0014356881499230645,
+      "grad_norm": 1.8517160415649414,
+      "learning_rate": 9.001710906587064e-05,
+      "loss": 1.1434,
+      "step": 108
+    },
+    {
+      "epoch": 0.0014489815587186484,
+      "grad_norm": 2.314161539077759,
+      "learning_rate": 8.9797028883159e-05,
+      "loss": 0.9643,
+      "step": 109
+    },
+    {
+      "epoch": 0.0014622749675142323,
+      "grad_norm": 2.426609516143799,
+      "learning_rate": 8.957482442146272e-05,
+      "loss": 0.9958,
+      "step": 110
+    },
+    {
+      "epoch": 0.0014755683763098162,
+      "grad_norm": 2.338745355606079,
+      "learning_rate": 8.935050754157344e-05,
+      "loss": 0.8958,
+      "step": 111
+    },
+    {
+      "epoch": 0.0014888617851054001,
+      "grad_norm": 2.2527472972869873,
+      "learning_rate": 8.912409021703913e-05,
+      "loss": 0.8262,
+      "step": 112
+    },
+    {
+      "epoch": 0.001502155193900984,
+      "grad_norm": 2.3767216205596924,
+      "learning_rate": 8.889558453352492e-05,
+      "loss": 0.9135,
+      "step": 113
+    },
+    {
+      "epoch": 0.001515448602696568,
+      "grad_norm": 2.25586199760437,
+      "learning_rate": 8.866500268816803e-05,
+      "loss": 0.9426,
+      "step": 114
+    },
+    {
+      "epoch": 0.0015287420114921519,
+      "grad_norm": 2.027083158493042,
+      "learning_rate": 8.84323569889266e-05,
+      "loss": 1.0912,
+      "step": 115
+    },
+    {
+      "epoch": 0.0015420354202877358,
+      "grad_norm": 2.3341987133026123,
+      "learning_rate": 8.819765985392296e-05,
+      "loss": 0.964,
+      "step": 116
+    },
+    {
+      "epoch": 0.0015553288290833197,
+      "grad_norm": 2.9502508640289307,
+      "learning_rate": 8.79609238107805e-05,
+      "loss": 0.9789,
+      "step": 117
+    },
+    {
+      "epoch": 0.0015686222378789036,
+      "grad_norm": 2.1710007190704346,
+      "learning_rate": 8.772216149595513e-05,
+      "loss": 0.8787,
+      "step": 118
+    },
+    {
+      "epoch": 0.0015819156466744875,
+      "grad_norm": 2.401892900466919,
+      "learning_rate": 8.748138565406081e-05,
+      "loss": 0.9684,
+      "step": 119
+    },
+    {
+      "epoch": 0.0015952090554700715,
+      "grad_norm": 2.7512049674987793,
+      "learning_rate": 8.72386091371891e-05,
+      "loss": 0.8961,
+      "step": 120
+    },
+    {
+      "epoch": 0.0016085024642656556,
+      "grad_norm": 2.9185006618499756,
+      "learning_rate": 8.699384490422331e-05,
+      "loss": 0.8282,
+      "step": 121
+    },
+    {
+      "epoch": 0.0016217958730612395,
+      "grad_norm": 2.936816930770874,
+      "learning_rate": 8.674710602014671e-05,
+      "loss": 0.8772,
+      "step": 122
+    },
+    {
+      "epoch": 0.0016350892818568234,
+      "grad_norm": 3.143145799636841,
+      "learning_rate": 8.649840565534513e-05,
+      "loss": 0.9349,
+      "step": 123
+    },
+    {
+      "epoch": 0.0016483826906524073,
+      "grad_norm": 2.681953191757202,
+      "learning_rate": 8.624775708490402e-05,
+      "loss": 0.9328,
+      "step": 124
+    },
+    {
+      "epoch": 0.0016616760994479912,
+      "grad_norm": 3.1768240928649902,
+      "learning_rate": 8.59951736878998e-05,
+      "loss": 0.8815,
+      "step": 125
+    },
+    {
+      "epoch": 0.0016749695082435752,
+      "grad_norm": 3.0723233222961426,
+      "learning_rate": 8.574066894668573e-05,
+      "loss": 0.9168,
+      "step": 126
+    },
+    {
+      "epoch": 0.001688262917039159,
+      "grad_norm": 3.1787002086639404,
+      "learning_rate": 8.548425644617224e-05,
+      "loss": 0.9053,
+      "step": 127
+    },
+    {
+      "epoch": 0.001701556325834743,
+      "grad_norm": 3.2121903896331787,
+      "learning_rate": 8.522594987310184e-05,
+      "loss": 0.8593,
+      "step": 128
+    },
+    {
+      "epoch": 0.001714849734630327,
+      "grad_norm": 2.9943299293518066,
+      "learning_rate": 8.49657630153185e-05,
+      "loss": 0.9997,
+      "step": 129
+    },
+    {
+      "epoch": 0.0017281431434259108,
+      "grad_norm": 3.535433292388916,
+      "learning_rate": 8.47037097610317e-05,
+      "loss": 0.9114,
+      "step": 130
+    },
+    {
+      "epoch": 0.0017414365522214947,
+      "grad_norm": 3.203751564025879,
+      "learning_rate": 8.443980409807512e-05,
+      "loss": 0.9689,
+      "step": 131
+    },
+    {
+      "epoch": 0.0017547299610170786,
+      "grad_norm": 3.2908623218536377,
+      "learning_rate": 8.417406011315998e-05,
+      "loss": 1.0035,
+      "step": 132
+    },
+    {
+      "epoch": 0.0017680233698126626,
+      "grad_norm": 2.752976179122925,
+      "learning_rate": 8.390649199112315e-05,
+      "loss": 0.9917,
+      "step": 133
+    },
+    {
+      "epoch": 0.0017813167786082465,
+      "grad_norm": 3.3330466747283936,
+      "learning_rate": 8.363711401417e-05,
+      "loss": 1.0635,
+      "step": 134
+    },
+    {
+      "epoch": 0.0017946101874038306,
+      "grad_norm": 3.2648167610168457,
+      "learning_rate": 8.336594056111197e-05,
+      "loss": 0.8795,
+      "step": 135
+    },
+    {
+      "epoch": 0.0018079035961994145,
+      "grad_norm": 3.277704954147339,
+      "learning_rate": 8.309298610659916e-05,
+      "loss": 0.8441,
+      "step": 136
+    },
+    {
+      "epoch": 0.0018211970049949984,
+      "grad_norm": 3.603541374206543,
+      "learning_rate": 8.281826522034764e-05,
+      "loss": 0.9423,
+      "step": 137
+    },
+    {
+      "epoch": 0.0018344904137905823,
+      "grad_norm": 2.947814464569092,
+      "learning_rate": 8.254179256636179e-05,
+      "loss": 0.8583,
+      "step": 138
+    },
+    {
+      "epoch": 0.0018477838225861663,
+      "grad_norm": 3.5735108852386475,
+      "learning_rate": 8.226358290215151e-05,
+      "loss": 1.0053,
+      "step": 139
+    },
+    {
+      "epoch": 0.0018610772313817502,
+      "grad_norm": 4.395263195037842,
+      "learning_rate": 8.198365107794457e-05,
+      "loss": 0.908,
+      "step": 140
+    },
+    {
+      "epoch": 0.001874370640177334,
+      "grad_norm": 4.384109973907471,
+      "learning_rate": 8.17020120358939e-05,
+      "loss": 0.9973,
+      "step": 141
+    },
+    {
+      "epoch": 0.001887664048972918,
+      "grad_norm": 3.42045521736145,
+      "learning_rate": 8.141868080927996e-05,
+      "loss": 0.8164,
+      "step": 142
+    },
+    {
+      "epoch": 0.001900957457768502,
+      "grad_norm": 3.931617259979248,
+      "learning_rate": 8.113367252170844e-05,
+      "loss": 0.9282,
+      "step": 143
+    },
+    {
+      "epoch": 0.0019142508665640858,
+      "grad_norm": 4.341536045074463,
+      "learning_rate": 8.084700238630283e-05,
+      "loss": 0.9588,
+      "step": 144
+    },
+    {
+      "epoch": 0.0019275442753596697,
+      "grad_norm": 4.990645885467529,
+      "learning_rate": 8.055868570489247e-05,
+      "loss": 0.9038,
+      "step": 145
+    },
+    {
+      "epoch": 0.0019408376841552537,
+      "grad_norm": 5.111573219299316,
+      "learning_rate": 8.026873786719573e-05,
+      "loss": 0.9121,
+      "step": 146
+    },
+    {
+      "epoch": 0.0019541310929508376,
+      "grad_norm": 4.322947978973389,
+      "learning_rate": 7.997717434999861e-05,
+      "loss": 0.8981,
+      "step": 147
+    },
+    {
+      "epoch": 0.0019674245017464215,
+      "grad_norm": 4.9193010330200195,
+      "learning_rate": 7.968401071632855e-05,
+      "loss": 0.9212,
+      "step": 148
+    },
+    {
+      "epoch": 0.0019807179105420054,
+      "grad_norm": 7.261848449707031,
+      "learning_rate": 7.938926261462366e-05,
+      "loss": 1.1311,
+      "step": 149
+    },
+    {
+      "epoch": 0.0019940113193375893,
+      "grad_norm": 9.099236488342285,
+      "learning_rate": 7.909294577789766e-05,
+      "loss": 0.8936,
+      "step": 150
+    },
+    {
+      "epoch": 0.0019940113193375893,
+      "eval_loss": 0.9891857504844666,
+      "eval_runtime": 3478.2034,
+      "eval_samples_per_second": 36.426,
+      "eval_steps_per_second": 9.106,
+      "step": 150
+    }
+  ],
+  "logging_steps": 1,
+  "max_steps": 450,
+  "num_input_tokens_seen": 0,
+  "num_train_epochs": 1,
+  "save_steps": 150,
+  "stateful_callbacks": {
+    "EarlyStoppingCallback": {
+      "args": {
+        "early_stopping_patience": 2,
+        "early_stopping_threshold": 0.0
+      },
+      "attributes": {
+        "early_stopping_patience_counter": 0
+      }
+    },
+    "TrainerControl": {
+      "args": {
+        "should_epoch_stop": false,
+        "should_evaluate": false,
+        "should_log": false,
+        "should_save": true,
+        "should_training_stop": false
+      },
+      "attributes": {}
+    }
+  },
+  "total_flos": 4.713236627088998e+16,
+  "train_batch_size": 8,
+  "trial_name": null,
+  "trial_params": null
+}

last-checkpoint/training_args.bin

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:094fd035c00a172fbe559c7af64aef705843919114fbd179c7cf916b6451f253
+size 6840