Training in progress, step 99, checkpoint

last-checkpoint/README.md

@@ -0,0 +1,202 @@
+---
+base_model: furiosa-ai/mlperf-gpt-j-6b
+library_name: peft
+---
+
+# Model Card for Model ID
+
+<!-- Provide a quick summary of what the model is/does. -->
+
+
+
+## Model Details
+
+### Model Description
+
+<!-- Provide a longer summary of what this model is. -->
+
+
+
+- **Developed by:** [More Information Needed]
+- **Funded by [optional]:** [More Information Needed]
+- **Shared by [optional]:** [More Information Needed]
+- **Model type:** [More Information Needed]
+- **Language(s) (NLP):** [More Information Needed]
+- **License:** [More Information Needed]
+- **Finetuned from model [optional]:** [More Information Needed]
+
+### Model Sources [optional]
+
+<!-- Provide the basic links for the model. -->
+
+- **Repository:** [More Information Needed]
+- **Paper [optional]:** [More Information Needed]
+- **Demo [optional]:** [More Information Needed]
+
+## Uses
+
+<!-- Address questions around how the model is intended to be used, including the foreseeable users of the model and those affected by the model. -->
+
+### Direct Use
+
+<!-- This section is for the model use without fine-tuning or plugging into a larger ecosystem/app. -->
+
+[More Information Needed]
+
+### Downstream Use [optional]
+
+<!-- This section is for the model use when fine-tuned for a task, or when plugged into a larger ecosystem/app -->
+
+[More Information Needed]
+
+### Out-of-Scope Use
+
+<!-- This section addresses misuse, malicious use, and uses that the model will not work well for. -->
+
+[More Information Needed]
+
+## Bias, Risks, and Limitations
+
+<!-- This section is meant to convey both technical and sociotechnical limitations. -->
+
+[More Information Needed]
+
+### Recommendations
+
+<!-- This section is meant to convey recommendations with respect to the bias, risk, and technical limitations. -->
+
+Users (both direct and downstream) should be made aware of the risks, biases and limitations of the model. More information needed for further recommendations.
+
+## How to Get Started with the Model
+
+Use the code below to get started with the model.
+
+[More Information Needed]
+
+## Training Details
+
+### Training Data
+
+<!-- This should link to a Dataset Card, perhaps with a short stub of information on what the training data is all about as well as documentation related to data pre-processing or additional filtering. -->
+
+[More Information Needed]
+
+### Training Procedure
+
+<!-- This relates heavily to the Technical Specifications. Content here should link to that section when it is relevant to the training procedure. -->
+
+#### Preprocessing [optional]
+
+[More Information Needed]
+
+
+#### Training Hyperparameters
+
+- **Training regime:** [More Information Needed] <!--fp32, fp16 mixed precision, bf16 mixed precision, bf16 non-mixed precision, fp16 non-mixed precision, fp8 mixed precision -->
+
+#### Speeds, Sizes, Times [optional]
+
+<!-- This section provides information about throughput, start/end time, checkpoint size if relevant, etc. -->
+
+[More Information Needed]
+
+## Evaluation
+
+<!-- This section describes the evaluation protocols and provides the results. -->
+
+### Testing Data, Factors & Metrics
+
+#### Testing Data
+
+<!-- This should link to a Dataset Card if possible. -->
+
+[More Information Needed]
+
+#### Factors
+
+<!-- These are the things the evaluation is disaggregating by, e.g., subpopulations or domains. -->
+
+[More Information Needed]
+
+#### Metrics
+
+<!-- These are the evaluation metrics being used, ideally with a description of why. -->
+
+[More Information Needed]
+
+### Results
+
+[More Information Needed]
+
+#### Summary
+
+
+
+## Model Examination [optional]
+
+<!-- Relevant interpretability work for the model goes here -->
+
+[More Information Needed]
+
+## Environmental Impact
+
+<!-- Total emissions (in grams of CO2eq) and additional considerations, such as electricity usage, go here. Edit the suggested text below accordingly -->
+
+Carbon emissions can be estimated using the [Machine Learning Impact calculator](https://mlco2.github.io/impact#compute) presented in [Lacoste et al. (2019)](https://arxiv.org/abs/1910.09700).
+
+- **Hardware Type:** [More Information Needed]
+- **Hours used:** [More Information Needed]
+- **Cloud Provider:** [More Information Needed]
+- **Compute Region:** [More Information Needed]
+- **Carbon Emitted:** [More Information Needed]
+
+## Technical Specifications [optional]
+
+### Model Architecture and Objective
+
+[More Information Needed]
+
+### Compute Infrastructure
+
+[More Information Needed]
+
+#### Hardware
+
+[More Information Needed]
+
+#### Software
+
+[More Information Needed]
+
+## Citation [optional]
+
+<!-- If there is a paper or blog post introducing the model, the APA and Bibtex information for that should go in this section. -->
+
+**BibTeX:**
+
+[More Information Needed]
+
+**APA:**
+
+[More Information Needed]
+
+## Glossary [optional]
+
+<!-- If relevant, include terms and calculations in this section that can help readers understand the model or model card. -->
+
+[More Information Needed]
+
+## More Information [optional]
+
+[More Information Needed]
+
+## Model Card Authors [optional]
+
+[More Information Needed]
+
+## Model Card Contact
+
+[More Information Needed]
+### Framework versions
+
+- PEFT 0.13.2

last-checkpoint/adapter_config.json

@@ -0,0 +1,33 @@
+{
+  "alpha_pattern": {},
+  "auto_mapping": null,
+  "base_model_name_or_path": "furiosa-ai/mlperf-gpt-j-6b",
+  "bias": "none",
+  "fan_in_fan_out": null,
+  "inference_mode": true,
+  "init_lora_weights": true,
+  "layer_replication": null,
+  "layers_pattern": null,
+  "layers_to_transform": null,
+  "loftq_config": {},
+  "lora_alpha": 16,
+  "lora_dropout": 0.05,
+  "megatron_config": null,
+  "megatron_core": "megatron.core",
+  "modules_to_save": null,
+  "peft_type": "LORA",
+  "r": 8,
+  "rank_pattern": {},
+  "revision": null,
+  "target_modules": [
+    "q_proj",
+    "fc_in",
+    "out_proj",
+    "v_proj",
+    "k_proj",
+    "fc_out"
+  ],
+  "task_type": "CAUSAL_LM",
+  "use_dora": false,
+  "use_rslora": false
+}

last-checkpoint/adapter_model.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:af6267ce90a0965cb692e568ce50202ab53b4c422985295b1188138cd16e8253
+size 66103768

last-checkpoint/added_tokens.json

@@ -0,0 +1,146 @@
+{
+  "<|extratoken_100|>": 50356,
+  "<|extratoken_101|>": 50357,
+  "<|extratoken_102|>": 50358,
+  "<|extratoken_103|>": 50359,
+  "<|extratoken_104|>": 50360,
+  "<|extratoken_105|>": 50361,
+  "<|extratoken_106|>": 50362,
+  "<|extratoken_107|>": 50363,
+  "<|extratoken_108|>": 50364,
+  "<|extratoken_109|>": 50365,
+  "<|extratoken_10|>": 50266,
+  "<|extratoken_110|>": 50366,
+  "<|extratoken_111|>": 50367,
+  "<|extratoken_112|>": 50368,
+  "<|extratoken_113|>": 50369,
+  "<|extratoken_114|>": 50370,
+  "<|extratoken_115|>": 50371,
+  "<|extratoken_116|>": 50372,
+  "<|extratoken_117|>": 50373,
+  "<|extratoken_118|>": 50374,
+  "<|extratoken_119|>": 50375,
+  "<|extratoken_11|>": 50267,
+  "<|extratoken_120|>": 50376,
+  "<|extratoken_121|>": 50377,
+  "<|extratoken_122|>": 50378,
+  "<|extratoken_123|>": 50379,
+  "<|extratoken_124|>": 50380,
+  "<|extratoken_125|>": 50381,
+  "<|extratoken_126|>": 50382,
+  "<|extratoken_127|>": 50383,
+  "<|extratoken_128|>": 50384,
+  "<|extratoken_129|>": 50385,
+  "<|extratoken_12|>": 50268,
+  "<|extratoken_130|>": 50386,
+  "<|extratoken_131|>": 50387,
+  "<|extratoken_132|>": 50388,
+  "<|extratoken_133|>": 50389,
+  "<|extratoken_134|>": 50390,
+  "<|extratoken_135|>": 50391,
+  "<|extratoken_136|>": 50392,
+  "<|extratoken_137|>": 50393,
+  "<|extratoken_138|>": 50394,
+  "<|extratoken_139|>": 50395,
+  "<|extratoken_13|>": 50269,
+  "<|extratoken_140|>": 50396,
+  "<|extratoken_141|>": 50397,
+  "<|extratoken_142|>": 50398,
+  "<|extratoken_143|>": 50399,
+  "<|extratoken_14|>": 50270,
+  "<|extratoken_15|>": 50271,
+  "<|extratoken_16|>": 50272,
+  "<|extratoken_17|>": 50273,
+  "<|extratoken_18|>": 50274,
+  "<|extratoken_19|>": 50275,
+  "<|extratoken_1|>": 50257,
+  "<|extratoken_20|>": 50276,
+  "<|extratoken_21|>": 50277,
+  "<|extratoken_22|>": 50278,
+  "<|extratoken_23|>": 50279,
+  "<|extratoken_24|>": 50280,
+  "<|extratoken_25|>": 50281,
+  "<|extratoken_26|>": 50282,
+  "<|extratoken_27|>": 50283,
+  "<|extratoken_28|>": 50284,
+  "<|extratoken_29|>": 50285,
+  "<|extratoken_2|>": 50258,
+  "<|extratoken_30|>": 50286,
+  "<|extratoken_31|>": 50287,
+  "<|extratoken_32|>": 50288,
+  "<|extratoken_33|>": 50289,
+  "<|extratoken_34|>": 50290,
+  "<|extratoken_35|>": 50291,
+  "<|extratoken_36|>": 50292,
+  "<|extratoken_37|>": 50293,
+  "<|extratoken_38|>": 50294,
+  "<|extratoken_39|>": 50295,
+  "<|extratoken_3|>": 50259,
+  "<|extratoken_40|>": 50296,
+  "<|extratoken_41|>": 50297,
+  "<|extratoken_42|>": 50298,
+  "<|extratoken_43|>": 50299,
+  "<|extratoken_44|>": 50300,
+  "<|extratoken_45|>": 50301,
+  "<|extratoken_46|>": 50302,
+  "<|extratoken_47|>": 50303,
+  "<|extratoken_48|>": 50304,
+  "<|extratoken_49|>": 50305,
+  "<|extratoken_4|>": 50260,
+  "<|extratoken_50|>": 50306,
+  "<|extratoken_51|>": 50307,
+  "<|extratoken_52|>": 50308,
+  "<|extratoken_53|>": 50309,
+  "<|extratoken_54|>": 50310,
+  "<|extratoken_55|>": 50311,
+  "<|extratoken_56|>": 50312,
+  "<|extratoken_57|>": 50313,
+  "<|extratoken_58|>": 50314,
+  "<|extratoken_59|>": 50315,
+  "<|extratoken_5|>": 50261,
+  "<|extratoken_60|>": 50316,
+  "<|extratoken_61|>": 50317,
+  "<|extratoken_62|>": 50318,
+  "<|extratoken_63|>": 50319,
+  "<|extratoken_64|>": 50320,
+  "<|extratoken_65|>": 50321,
+  "<|extratoken_66|>": 50322,
+  "<|extratoken_67|>": 50323,
+  "<|extratoken_68|>": 50324,
+  "<|extratoken_69|>": 50325,
+  "<|extratoken_6|>": 50262,
+  "<|extratoken_70|>": 50326,
+  "<|extratoken_71|>": 50327,
+  "<|extratoken_72|>": 50328,
+  "<|extratoken_73|>": 50329,
+  "<|extratoken_74|>": 50330,
+  "<|extratoken_75|>": 50331,
+  "<|extratoken_76|>": 50332,
+  "<|extratoken_77|>": 50333,
+  "<|extratoken_78|>": 50334,
+  "<|extratoken_79|>": 50335,
+  "<|extratoken_7|>": 50263,
+  "<|extratoken_80|>": 50336,
+  "<|extratoken_81|>": 50337,
+  "<|extratoken_82|>": 50338,
+  "<|extratoken_83|>": 50339,
+  "<|extratoken_84|>": 50340,
+  "<|extratoken_85|>": 50341,
+  "<|extratoken_86|>": 50342,
+  "<|extratoken_87|>": 50343,
+  "<|extratoken_88|>": 50344,
+  "<|extratoken_89|>": 50345,
+  "<|extratoken_8|>": 50264,
+  "<|extratoken_90|>": 50346,
+  "<|extratoken_91|>": 50347,
+  "<|extratoken_92|>": 50348,
+  "<|extratoken_93|>": 50349,
+  "<|extratoken_94|>": 50350,
+  "<|extratoken_95|>": 50351,
+  "<|extratoken_96|>": 50352,
+  "<|extratoken_97|>": 50353,
+  "<|extratoken_98|>": 50354,
+  "<|extratoken_99|>": 50355,
+  "<|extratoken_9|>": 50265,
+  "[PAD]": 50400
+}

last-checkpoint/optimizer.pt

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:176213d2a27e0e814f99b90054da282e323511b836c29cfbf0bcd6dd995c8583
+size 33939412

last-checkpoint/rng_state.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:4eea88962c2c003f5022f62ceea4f10f2c62dd1dbbbf421a89ac4036ef0279d8
+size 14244

last-checkpoint/scheduler.pt

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:e80bd13ccd86381c43d36a50b9b8744dacf3b87fffadb3e5cb79368ce9e10822
+size 1064

last-checkpoint/special_tokens_map.json

@@ -0,0 +1,30 @@
+{
+  "bos_token": {
+    "content": "<|endoftext|>",
+    "lstrip": false,
+    "normalized": true,
+    "rstrip": false,
+    "single_word": false
+  },
+  "eos_token": {
+    "content": "<|endoftext|>",
+    "lstrip": false,
+    "normalized": true,
+    "rstrip": false,
+    "single_word": false
+  },
+  "pad_token": {
+    "content": "<|endoftext|>",
+    "lstrip": false,
+    "normalized": true,
+    "rstrip": false,
+    "single_word": false
+  },
+  "unk_token": {
+    "content": "<|endoftext|>",
+    "lstrip": false,
+    "normalized": true,
+    "rstrip": false,
+    "single_word": false
+  }
+}

last-checkpoint/tokenizer_config.json

@@ -0,0 +1,1177 @@
+{
+  "add_bos_token": false,
+  "add_prefix_space": false,
+  "added_tokens_decoder": {
+    "50256": {
+      "content": "<|endoftext|>",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "50257": {
+      "content": "<|extratoken_1|>",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50258": {
+      "content": "<|extratoken_2|>",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50259": {
+      "content": "<|extratoken_3|>",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50260": {
+      "content": "<|extratoken_4|>",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50261": {
+      "content": "<|extratoken_5|>",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50262": {
+      "content": "<|extratoken_6|>",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50263": {
+      "content": "<|extratoken_7|>",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50264": {
+      "content": "<|extratoken_8|>",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50265": {
+      "content": "<|extratoken_9|>",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50266": {
+      "content": "<|extratoken_10|>",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50267": {
+      "content": "<|extratoken_11|>",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50268": {
+      "content": "<|extratoken_12|>",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50269": {
+      "content": "<|extratoken_13|>",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50270": {
+      "content": "<|extratoken_14|>",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50271": {
+      "content": "<|extratoken_15|>",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50272": {
+      "content": "<|extratoken_16|>",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50273": {
+      "content": "<|extratoken_17|>",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50274": {
+      "content": "<|extratoken_18|>",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50275": {
+      "content": "<|extratoken_19|>",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50276": {
+      "content": "<|extratoken_20|>",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50277": {
+      "content": "<|extratoken_21|>",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50278": {
+      "content": "<|extratoken_22|>",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50279": {
+      "content": "<|extratoken_23|>",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50280": {
+      "content": "<|extratoken_24|>",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50281": {
+      "content": "<|extratoken_25|>",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50282": {
+      "content": "<|extratoken_26|>",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50283": {
+      "content": "<|extratoken_27|>",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50284": {
+      "content": "<|extratoken_28|>",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50285": {
+      "content": "<|extratoken_29|>",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50286": {
+      "content": "<|extratoken_30|>",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50287": {
+      "content": "<|extratoken_31|>",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50288": {
+      "content": "<|extratoken_32|>",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50289": {
+      "content": "<|extratoken_33|>",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50290": {
+      "content": "<|extratoken_34|>",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50291": {
+      "content": "<|extratoken_35|>",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50292": {
+      "content": "<|extratoken_36|>",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50293": {
+      "content": "<|extratoken_37|>",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50294": {
+      "content": "<|extratoken_38|>",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50295": {
+      "content": "<|extratoken_39|>",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50296": {
+      "content": "<|extratoken_40|>",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50297": {
+      "content": "<|extratoken_41|>",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50298": {
+      "content": "<|extratoken_42|>",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50299": {
+      "content": "<|extratoken_43|>",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50300": {
+      "content": "<|extratoken_44|>",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50301": {
+      "content": "<|extratoken_45|>",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50302": {
+      "content": "<|extratoken_46|>",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50303": {
+      "content": "<|extratoken_47|>",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50304": {
+      "content": "<|extratoken_48|>",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50305": {
+      "content": "<|extratoken_49|>",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50306": {
+      "content": "<|extratoken_50|>",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50307": {
+      "content": "<|extratoken_51|>",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50308": {
+      "content": "<|extratoken_52|>",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50309": {
+      "content": "<|extratoken_53|>",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50310": {
+      "content": "<|extratoken_54|>",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50311": {
+      "content": "<|extratoken_55|>",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50312": {
+      "content": "<|extratoken_56|>",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50313": {
+      "content": "<|extratoken_57|>",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50314": {
+      "content": "<|extratoken_58|>",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50315": {
+      "content": "<|extratoken_59|>",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50316": {
+      "content": "<|extratoken_60|>",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50317": {
+      "content": "<|extratoken_61|>",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50318": {
+      "content": "<|extratoken_62|>",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50319": {
+      "content": "<|extratoken_63|>",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50320": {
+      "content": "<|extratoken_64|>",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50321": {
+      "content": "<|extratoken_65|>",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50322": {
+      "content": "<|extratoken_66|>",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50323": {
+      "content": "<|extratoken_67|>",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50324": {
+      "content": "<|extratoken_68|>",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50325": {
+      "content": "<|extratoken_69|>",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50326": {
+      "content": "<|extratoken_70|>",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50327": {
+      "content": "<|extratoken_71|>",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50328": {
+      "content": "<|extratoken_72|>",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50329": {
+      "content": "<|extratoken_73|>",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50330": {
+      "content": "<|extratoken_74|>",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50331": {
+      "content": "<|extratoken_75|>",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50332": {
+      "content": "<|extratoken_76|>",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50333": {
+      "content": "<|extratoken_77|>",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50334": {
+      "content": "<|extratoken_78|>",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50335": {
+      "content": "<|extratoken_79|>",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50336": {
+      "content": "<|extratoken_80|>",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50337": {
+      "content": "<|extratoken_81|>",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50338": {
+      "content": "<|extratoken_82|>",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50339": {
+      "content": "<|extratoken_83|>",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50340": {
+      "content": "<|extratoken_84|>",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50341": {
+      "content": "<|extratoken_85|>",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50342": {
+      "content": "<|extratoken_86|>",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50343": {
+      "content": "<|extratoken_87|>",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50344": {
+      "content": "<|extratoken_88|>",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50345": {
+      "content": "<|extratoken_89|>",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50346": {
+      "content": "<|extratoken_90|>",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50347": {
+      "content": "<|extratoken_91|>",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50348": {
+      "content": "<|extratoken_92|>",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50349": {
+      "content": "<|extratoken_93|>",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50350": {
+      "content": "<|extratoken_94|>",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50351": {
+      "content": "<|extratoken_95|>",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50352": {
+      "content": "<|extratoken_96|>",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50353": {
+      "content": "<|extratoken_97|>",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50354": {
+      "content": "<|extratoken_98|>",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50355": {
+      "content": "<|extratoken_99|>",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50356": {
+      "content": "<|extratoken_100|>",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50357": {
+      "content": "<|extratoken_101|>",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50358": {
+      "content": "<|extratoken_102|>",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50359": {
+      "content": "<|extratoken_103|>",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50360": {
+      "content": "<|extratoken_104|>",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50361": {
+      "content": "<|extratoken_105|>",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50362": {
+      "content": "<|extratoken_106|>",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50363": {
+      "content": "<|extratoken_107|>",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50364": {
+      "content": "<|extratoken_108|>",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50365": {
+      "content": "<|extratoken_109|>",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50366": {
+      "content": "<|extratoken_110|>",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50367": {
+      "content": "<|extratoken_111|>",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50368": {
+      "content": "<|extratoken_112|>",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50369": {
+      "content": "<|extratoken_113|>",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50370": {
+      "content": "<|extratoken_114|>",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50371": {
+      "content": "<|extratoken_115|>",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50372": {
+      "content": "<|extratoken_116|>",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50373": {
+      "content": "<|extratoken_117|>",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50374": {
+      "content": "<|extratoken_118|>",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50375": {
+      "content": "<|extratoken_119|>",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50376": {
+      "content": "<|extratoken_120|>",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50377": {
+      "content": "<|extratoken_121|>",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50378": {
+      "content": "<|extratoken_122|>",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50379": {
+      "content": "<|extratoken_123|>",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50380": {
+      "content": "<|extratoken_124|>",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50381": {
+      "content": "<|extratoken_125|>",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50382": {
+      "content": "<|extratoken_126|>",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50383": {
+      "content": "<|extratoken_127|>",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50384": {
+      "content": "<|extratoken_128|>",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50385": {
+      "content": "<|extratoken_129|>",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50386": {
+      "content": "<|extratoken_130|>",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50387": {
+      "content": "<|extratoken_131|>",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50388": {
+      "content": "<|extratoken_132|>",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50389": {
+      "content": "<|extratoken_133|>",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50390": {
+      "content": "<|extratoken_134|>",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50391": {
+      "content": "<|extratoken_135|>",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50392": {
+      "content": "<|extratoken_136|>",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50393": {
+      "content": "<|extratoken_137|>",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50394": {
+      "content": "<|extratoken_138|>",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50395": {
+      "content": "<|extratoken_139|>",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50396": {
+      "content": "<|extratoken_140|>",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50397": {
+      "content": "<|extratoken_141|>",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50398": {
+      "content": "<|extratoken_142|>",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50399": {
+      "content": "<|extratoken_143|>",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50400": {
+      "content": "[PAD]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    }
+  },
+  "bos_token": "<|endoftext|>",
+  "chat_template": "{% if not add_generation_prompt is defined %}{% set add_generation_prompt = false %}{% endif %}{% set loop_messages = messages %}{% for message in loop_messages %}{% set content = '<|start_header_id|>' + message['role'] + '<|end_header_id|>\n\n'+ message['content'] | trim + '<|eot_id|>' %}{% if loop.index0 == 0 %}{% set content = bos_token + content %}{% endif %}{{ content }}{% endfor %}{% if add_generation_prompt %}{{ '<|start_header_id|>assistant<|end_header_id|>\n\n' }}{% endif %}",
+  "clean_up_tokenization_spaces": true,
+  "eos_token": "<|endoftext|>",
+  "errors": "replace",
+  "from_slow": true,
+  "model_max_length": 1919,
+  "pad_token": "<|endoftext|>",
+  "padding_side": "left",
+  "tokenizer_class": "GPT2Tokenizer",
+  "unk_token": "<|endoftext|>"
+}

last-checkpoint/trainer_state.json

@@ -0,0 +1,742 @@
+{
+  "best_metric": null,
+  "best_model_checkpoint": null,
+  "epoch": 0.04327395913014971,
+  "eval_steps": 50,
+  "global_step": 99,
+  "is_hyper_param_search": false,
+  "is_local_process_zero": true,
+  "is_world_process_zero": true,
+  "log_history": [
+    {
+      "epoch": 0.0004371106982843405,
+      "grad_norm": 1.0747889280319214,
+      "learning_rate": 5e-05,
+      "loss": 12.2248,
+      "step": 1
+    },
+    {
+      "epoch": 0.0004371106982843405,
+      "eval_loss": 3.090005874633789,
+      "eval_runtime": 63.9939,
+      "eval_samples_per_second": 15.064,
+      "eval_steps_per_second": 7.532,
+      "step": 1
+    },
+    {
+      "epoch": 0.000874221396568681,
+      "grad_norm": 1.063541293144226,
+      "learning_rate": 0.0001,
+      "loss": 12.6966,
+      "step": 2
+    },
+    {
+      "epoch": 0.0013113320948530216,
+      "grad_norm": 1.083733081817627,
+      "learning_rate": 0.00015,
+      "loss": 11.8971,
+      "step": 3
+    },
+    {
+      "epoch": 0.001748442793137362,
+      "grad_norm": 1.1510637998580933,
+      "learning_rate": 0.0002,
+      "loss": 12.5537,
+      "step": 4
+    },
+    {
+      "epoch": 0.0021855534914217026,
+      "grad_norm": 1.5311155319213867,
+      "learning_rate": 0.00025,
+      "loss": 13.4675,
+      "step": 5
+    },
+    {
+      "epoch": 0.0026226641897060432,
+      "grad_norm": 1.697422981262207,
+      "learning_rate": 0.0003,
+      "loss": 11.6525,
+      "step": 6
+    },
+    {
+      "epoch": 0.0030597748879903834,
+      "grad_norm": 1.9460680484771729,
+      "learning_rate": 0.00035,
+      "loss": 10.7923,
+      "step": 7
+    },
+    {
+      "epoch": 0.003496885586274724,
+      "grad_norm": 1.9320839643478394,
+      "learning_rate": 0.0004,
+      "loss": 10.2103,
+      "step": 8
+    },
+    {
+      "epoch": 0.003933996284559064,
+      "grad_norm": 2.0210471153259277,
+      "learning_rate": 0.00045000000000000004,
+      "loss": 10.7568,
+      "step": 9
+    },
+    {
+      "epoch": 0.004371106982843405,
+      "grad_norm": 2.5222268104553223,
+      "learning_rate": 0.0005,
+      "loss": 10.1545,
+      "step": 10
+    },
+    {
+      "epoch": 0.004808217681127745,
+      "grad_norm": 5.3654961585998535,
+      "learning_rate": 0.0004998442655654946,
+      "loss": 11.1142,
+      "step": 11
+    },
+    {
+      "epoch": 0.0052453283794120864,
+      "grad_norm": 6.164823532104492,
+      "learning_rate": 0.0004993772562876909,
+      "loss": 10.6837,
+      "step": 12
+    },
+    {
+      "epoch": 0.005682439077696427,
+      "grad_norm": 3.8814218044281006,
+      "learning_rate": 0.0004985995540019955,
+      "loss": 9.5134,
+      "step": 13
+    },
+    {
+      "epoch": 0.006119549775980767,
+      "grad_norm": 2.6187520027160645,
+      "learning_rate": 0.0004975121276286136,
+      "loss": 10.3129,
+      "step": 14
+    },
+    {
+      "epoch": 0.006556660474265108,
+      "grad_norm": 2.1759867668151855,
+      "learning_rate": 0.0004961163319653958,
+      "loss": 9.9295,
+      "step": 15
+    },
+    {
+      "epoch": 0.006993771172549448,
+      "grad_norm": 1.982545256614685,
+      "learning_rate": 0.0004944139059999286,
+      "loss": 8.5633,
+      "step": 16
+    },
+    {
+      "epoch": 0.007430881870833789,
+      "grad_norm": 1.8032101392745972,
+      "learning_rate": 0.000492406970742972,
+      "loss": 9.2849,
+      "step": 17
+    },
+    {
+      "epoch": 0.007867992569118128,
+      "grad_norm": 1.65458345413208,
+      "learning_rate": 0.0004900980265859448,
+      "loss": 9.5214,
+      "step": 18
+    },
+    {
+      "epoch": 0.00830510326740247,
+      "grad_norm": 1.7164701223373413,
+      "learning_rate": 0.0004874899501857477,
+      "loss": 10.1914,
+      "step": 19
+    },
+    {
+      "epoch": 0.00874221396568681,
+      "grad_norm": 1.529309630393982,
+      "learning_rate": 0.00048458599088080736,
+      "loss": 9.3715,
+      "step": 20
+    },
+    {
+      "epoch": 0.009179324663971151,
+      "grad_norm": 1.4430363178253174,
+      "learning_rate": 0.0004813897666428053,
+      "loss": 9.4226,
+      "step": 21
+    },
+    {
+      "epoch": 0.00961643536225549,
+      "grad_norm": 1.3935481309890747,
+      "learning_rate": 0.00047790525956913543,
+      "loss": 9.6574,
+      "step": 22
+    },
+    {
+      "epoch": 0.010053546060539832,
+      "grad_norm": 1.3008925914764404,
+      "learning_rate": 0.0004741368109217071,
+      "loss": 9.1926,
+      "step": 23
+    },
+    {
+      "epoch": 0.010490656758824173,
+      "grad_norm": 1.3502999544143677,
+      "learning_rate": 0.00047008911571827283,
+      "loss": 9.3747,
+      "step": 24
+    },
+    {
+      "epoch": 0.010927767457108512,
+      "grad_norm": 1.3965394496917725,
+      "learning_rate": 0.00046576721688302105,
+      "loss": 8.7614,
+      "step": 25
+    },
+    {
+      "epoch": 0.011364878155392853,
+      "grad_norm": 1.3983211517333984,
+      "learning_rate": 0.0004611764989637205,
+      "loss": 9.6498,
+      "step": 26
+    },
+    {
+      "epoch": 0.011801988853677194,
+      "grad_norm": 1.3375810384750366,
+      "learning_rate": 0.0004563226814232444,
+      "loss": 8.3323,
+      "step": 27
+    },
+    {
+      "epoch": 0.012239099551961534,
+      "grad_norm": 1.4064358472824097,
+      "learning_rate": 0.0004512118115138315,
+      "loss": 9.0801,
+      "step": 28
+    },
+    {
+      "epoch": 0.012676210250245875,
+      "grad_norm": 1.4555835723876953,
+      "learning_rate": 0.0004458502567429631,
+      "loss": 9.4093,
+      "step": 29
+    },
+    {
+      "epoch": 0.013113320948530216,
+      "grad_norm": 1.4439679384231567,
+      "learning_rate": 0.00044024469694024196,
+      "loss": 9.3942,
+      "step": 30
+    },
+    {
+      "epoch": 0.013550431646814555,
+      "grad_norm": 1.4487768411636353,
+      "learning_rate": 0.00043440211593515554,
+      "loss": 8.4057,
+      "step": 31
+    },
+    {
+      "epoch": 0.013987542345098896,
+      "grad_norm": 1.420520544052124,
+      "learning_rate": 0.0004283297928560951,
+      "loss": 9.1814,
+      "step": 32
+    },
+    {
+      "epoch": 0.014424653043383237,
+      "grad_norm": 1.587709903717041,
+      "learning_rate": 0.0004220352930614672,
+      "loss": 9.7101,
+      "step": 33
+    },
+    {
+      "epoch": 0.014861763741667578,
+      "grad_norm": 1.5516473054885864,
+      "learning_rate": 0.00041552645871420013,
+      "loss": 9.4094,
+      "step": 34
+    },
+    {
+      "epoch": 0.015298874439951917,
+      "grad_norm": 1.475528597831726,
+      "learning_rate": 0.00040881139901138467,
+      "loss": 9.301,
+      "step": 35
+    },
+    {
+      "epoch": 0.015735985138236257,
+      "grad_norm": 1.5390028953552246,
+      "learning_rate": 0.00040189848008122475,
+      "loss": 9.5242,
+      "step": 36
+    },
+    {
+      "epoch": 0.016173095836520598,
+      "grad_norm": 1.6369234323501587,
+      "learning_rate": 0.00039479631455988334,
+      "loss": 10.05,
+      "step": 37
+    },
+    {
+      "epoch": 0.01661020653480494,
+      "grad_norm": 1.6309175491333008,
+      "learning_rate": 0.0003875137508612103,
+      "loss": 9.3983,
+      "step": 38
+    },
+    {
+      "epoch": 0.01704731723308928,
+      "grad_norm": 1.6779584884643555,
+      "learning_rate": 0.00038005986215272055,
+      "loss": 10.1454,
+      "step": 39
+    },
+    {
+      "epoch": 0.01748442793137362,
+      "grad_norm": 1.7082078456878662,
+      "learning_rate": 0.0003724439350515571,
+      "loss": 9.3163,
+      "step": 40
+    },
+    {
+      "epoch": 0.017921538629657962,
+      "grad_norm": 1.6987196207046509,
+      "learning_rate": 0.0003646754580545226,
+      "loss": 9.0876,
+      "step": 41
+    },
+    {
+      "epoch": 0.018358649327942303,
+      "grad_norm": 1.550484299659729,
+      "learning_rate": 0.000356764109716594,
+      "loss": 9.5422,
+      "step": 42
+    },
+    {
+      "epoch": 0.01879576002622664,
+      "grad_norm": 1.5593546628952026,
+      "learning_rate": 0.00034871974659264783,
+      "loss": 9.1657,
+      "step": 43
+    },
+    {
+      "epoch": 0.01923287072451098,
+      "grad_norm": 1.5249624252319336,
+      "learning_rate": 0.0003405523909574206,
+      "loss": 8.5582,
+      "step": 44
+    },
+    {
+      "epoch": 0.019669981422795323,
+      "grad_norm": 1.469465970993042,
+      "learning_rate": 0.0003322722183190025,
+      "loss": 8.2051,
+      "step": 45
+    },
+    {
+      "epoch": 0.020107092121079664,
+      "grad_norm": 1.5793440341949463,
+      "learning_rate": 0.0003238895447414211,
+      "loss": 9.9744,
+      "step": 46
+    },
+    {
+      "epoch": 0.020544202819364005,
+      "grad_norm": 1.4703450202941895,
+      "learning_rate": 0.0003154148139921102,
+      "loss": 8.5032,
+      "step": 47
+    },
+    {
+      "epoch": 0.020981313517648346,
+      "grad_norm": 1.504827857017517,
+      "learning_rate": 0.00030685858453027663,
+      "loss": 9.1006,
+      "step": 48
+    },
+    {
+      "epoch": 0.021418424215932683,
+      "grad_norm": 1.5373276472091675,
+      "learning_rate": 0.0002982315163523742,
+      "loss": 8.431,
+      "step": 49
+    },
+    {
+      "epoch": 0.021855534914217024,
+      "grad_norm": 1.5321044921875,
+      "learning_rate": 0.000289544357711076,
+      "loss": 8.8328,
+      "step": 50
+    },
+    {
+      "epoch": 0.021855534914217024,
+      "eval_loss": 2.2413060665130615,
+      "eval_runtime": 64.0607,
+      "eval_samples_per_second": 15.048,
+      "eval_steps_per_second": 7.524,
+      "step": 50
+    },
+    {
+      "epoch": 0.022292645612501365,
+      "grad_norm": 1.5985313653945923,
+      "learning_rate": 0.0002808079317242896,
+      "loss": 8.8504,
+      "step": 51
+    },
+    {
+      "epoch": 0.022729756310785706,
+      "grad_norm": 1.531156301498413,
+      "learning_rate": 0.0002720331228909005,
+      "loss": 8.0988,
+      "step": 52
+    },
+    {
+      "epoch": 0.023166867009070048,
+      "grad_norm": 1.497868537902832,
+      "learning_rate": 0.00026323086353004075,
+      "loss": 8.4299,
+      "step": 53
+    },
+    {
+      "epoch": 0.02360397770735439,
+      "grad_norm": 1.6191630363464355,
+      "learning_rate": 0.0002544121201607822,
+      "loss": 9.1572,
+      "step": 54
+    },
+    {
+      "epoch": 0.02404108840563873,
+      "grad_norm": 1.6276392936706543,
+      "learning_rate": 0.00024558787983921783,
+      "loss": 8.837,
+      "step": 55
+    },
+    {
+      "epoch": 0.024478199103923067,
+      "grad_norm": 1.6775784492492676,
+      "learning_rate": 0.0002367691364699592,
+      "loss": 9.3404,
+      "step": 56
+    },
+    {
+      "epoch": 0.024915309802207408,
+      "grad_norm": 1.6100305318832397,
+      "learning_rate": 0.00022796687710909964,
+      "loss": 8.4332,
+      "step": 57
+    },
+    {
+      "epoch": 0.02535242050049175,
+      "grad_norm": 1.423418402671814,
+      "learning_rate": 0.00021919206827571036,
+      "loss": 8.1461,
+      "step": 58
+    },
+    {
+      "epoch": 0.02578953119877609,
+      "grad_norm": 1.5356935262680054,
+      "learning_rate": 0.00021045564228892402,
+      "loss": 9.4439,
+      "step": 59
+    },
+    {
+      "epoch": 0.02622664189706043,
+      "grad_norm": 1.4723604917526245,
+      "learning_rate": 0.00020176848364762578,
+      "loss": 8.8865,
+      "step": 60
+    },
+    {
+      "epoch": 0.026663752595344772,
+      "grad_norm": 1.5057430267333984,
+      "learning_rate": 0.00019314141546972343,
+      "loss": 9.0139,
+      "step": 61
+    },
+    {
+      "epoch": 0.02710086329362911,
+      "grad_norm": 1.4735465049743652,
+      "learning_rate": 0.00018458518600788986,
+      "loss": 8.995,
+      "step": 62
+    },
+    {
+      "epoch": 0.02753797399191345,
+      "grad_norm": 1.5528627634048462,
+      "learning_rate": 0.00017611045525857898,
+      "loss": 9.5315,
+      "step": 63
+    },
+    {
+      "epoch": 0.027975084690197792,
+      "grad_norm": 1.510013222694397,
+      "learning_rate": 0.0001677277816809975,
+      "loss": 8.2992,
+      "step": 64
+    },
+    {
+      "epoch": 0.028412195388482133,
+      "grad_norm": 1.492727279663086,
+      "learning_rate": 0.00015944760904257942,
+      "loss": 8.7207,
+      "step": 65
+    },
+    {
+      "epoch": 0.028849306086766474,
+      "grad_norm": 1.5560027360916138,
+      "learning_rate": 0.0001512802534073522,
+      "loss": 9.5337,
+      "step": 66
+    },
+    {
+      "epoch": 0.029286416785050815,
+      "grad_norm": 1.4782696962356567,
+      "learning_rate": 0.00014323589028340596,
+      "loss": 8.5763,
+      "step": 67
+    },
+    {
+      "epoch": 0.029723527483335156,
+      "grad_norm": 1.452972412109375,
+      "learning_rate": 0.00013532454194547733,
+      "loss": 8.0966,
+      "step": 68
+    },
+    {
+      "epoch": 0.030160638181619494,
+      "grad_norm": 1.4597505331039429,
+      "learning_rate": 0.00012755606494844294,
+      "loss": 7.6456,
+      "step": 69
+    },
+    {
+      "epoch": 0.030597748879903835,
+      "grad_norm": 1.5495052337646484,
+      "learning_rate": 0.00011994013784727947,
+      "loss": 9.4187,
+      "step": 70
+    },
+    {
+      "epoch": 0.031034859578188176,
+      "grad_norm": 1.5275565385818481,
+      "learning_rate": 0.00011248624913878966,
+      "loss": 9.2429,
+      "step": 71
+    },
+    {
+      "epoch": 0.03147197027647251,
+      "grad_norm": 1.5431679487228394,
+      "learning_rate": 0.0001052036854401166,
+      "loss": 9.3511,
+      "step": 72
+    },
+    {
+      "epoch": 0.03190908097475686,
+      "grad_norm": 1.5252043008804321,
+      "learning_rate": 9.810151991877531e-05,
+      "loss": 8.7401,
+      "step": 73
+    },
+    {
+      "epoch": 0.032346191673041196,
+      "grad_norm": 1.4787598848342896,
+      "learning_rate": 9.118860098861537e-05,
+      "loss": 8.6475,
+      "step": 74
+    },
+    {
+      "epoch": 0.03278330237132554,
+      "grad_norm": 1.529118537902832,
+      "learning_rate": 8.44735412857999e-05,
+      "loss": 8.7036,
+      "step": 75
+    },
+    {
+      "epoch": 0.03322041306960988,
+      "grad_norm": 1.525482177734375,
+      "learning_rate": 7.79647069385328e-05,
+      "loss": 9.5446,
+      "step": 76
+    },
+    {
+      "epoch": 0.03365752376789422,
+      "grad_norm": 1.4786763191223145,
+      "learning_rate": 7.167020714390501e-05,
+      "loss": 9.1046,
+      "step": 77
+    },
+    {
+      "epoch": 0.03409463446617856,
+      "grad_norm": 1.5982285737991333,
+      "learning_rate": 6.559788406484446e-05,
+      "loss": 9.4103,
+      "step": 78
+    },
+    {
+      "epoch": 0.0345317451644629,
+      "grad_norm": 1.5381639003753662,
+      "learning_rate": 5.975530305975807e-05,
+      "loss": 8.6963,
+      "step": 79
+    },
+    {
+      "epoch": 0.03496885586274724,
+      "grad_norm": 1.5134705305099487,
+      "learning_rate": 5.414974325703686e-05,
+      "loss": 8.9013,
+      "step": 80
+    },
+    {
+      "epoch": 0.03540596656103158,
+      "grad_norm": 1.559717059135437,
+      "learning_rate": 4.8788188486168616e-05,
+      "loss": 9.1272,
+      "step": 81
+    },
+    {
+      "epoch": 0.035843077259315924,
+      "grad_norm": 1.4342595338821411,
+      "learning_rate": 4.367731857675569e-05,
+      "loss": 7.396,
+      "step": 82
+    },
+    {
+      "epoch": 0.03628018795760026,
+      "grad_norm": 1.5284779071807861,
+      "learning_rate": 3.882350103627952e-05,
+      "loss": 8.7514,
+      "step": 83
+    },
+    {
+      "epoch": 0.036717298655884606,
+      "grad_norm": 1.4775699377059937,
+      "learning_rate": 3.423278311697897e-05,
+      "loss": 8.3527,
+      "step": 84
+    },
+    {
+      "epoch": 0.037154409354168944,
+      "grad_norm": 1.5286551713943481,
+      "learning_rate": 2.9910884281727225e-05,
+      "loss": 9.1249,
+      "step": 85
+    },
+    {
+      "epoch": 0.03759152005245328,
+      "grad_norm": 1.509277105331421,
+      "learning_rate": 2.586318907829291e-05,
+      "loss": 9.0266,
+      "step": 86
+    },
+    {
+      "epoch": 0.038028630750737626,
+      "grad_norm": 1.4688650369644165,
+      "learning_rate": 2.209474043086457e-05,
+      "loss": 8.2366,
+      "step": 87
+    },
+    {
+      "epoch": 0.03846574144902196,
+      "grad_norm": 1.5496207475662231,
+      "learning_rate": 1.861023335719475e-05,
+      "loss": 9.1925,
+      "step": 88
+    },
+    {
+      "epoch": 0.03890285214730631,
+      "grad_norm": 1.5909758806228638,
+      "learning_rate": 1.5414009119192633e-05,
+      "loss": 8.7692,
+      "step": 89
+    },
+    {
+      "epoch": 0.039339962845590645,
+      "grad_norm": 1.529322862625122,
+      "learning_rate": 1.25100498142523e-05,
+      "loss": 8.8621,
+      "step": 90
+    },
+    {
+      "epoch": 0.03977707354387499,
+      "grad_norm": 1.53654944896698,
+      "learning_rate": 9.901973414055187e-06,
+      "loss": 8.0944,
+      "step": 91
+    },
+    {
+      "epoch": 0.04021418424215933,
+      "grad_norm": 1.5465720891952515,
+      "learning_rate": 7.593029257027956e-06,
+      "loss": 9.0611,
+      "step": 92
+    },
+    {
+      "epoch": 0.040651294940443665,
+      "grad_norm": 1.5916392803192139,
+      "learning_rate": 5.5860940000714015e-06,
+      "loss": 8.6493,
+      "step": 93
+    },
+    {
+      "epoch": 0.04108840563872801,
+      "grad_norm": 1.5334380865097046,
+      "learning_rate": 3.8836680346041594e-06,
+      "loss": 8.2758,
+      "step": 94
+    },
+    {
+      "epoch": 0.04152551633701235,
+      "grad_norm": 1.5617411136627197,
+      "learning_rate": 2.487872371386424e-06,
+      "loss": 9.7541,
+      "step": 95
+    },
+    {
+      "epoch": 0.04196262703529669,
+      "grad_norm": 1.5655155181884766,
+      "learning_rate": 1.4004459980045125e-06,
+      "loss": 8.9262,
+      "step": 96
+    },
+    {
+      "epoch": 0.04239973773358103,
+      "grad_norm": 1.612404227256775,
+      "learning_rate": 6.22743712309054e-07,
+      "loss": 9.412,
+      "step": 97
+    },
+    {
+      "epoch": 0.04283684843186537,
+      "grad_norm": 1.687268614768982,
+      "learning_rate": 1.557344345054501e-07,
+      "loss": 9.9603,
+      "step": 98
+    },
+    {
+      "epoch": 0.04327395913014971,
+      "grad_norm": 1.4794572591781616,
+      "learning_rate": 0.0,
+      "loss": 8.6769,
+      "step": 99
+    }
+  ],
+  "logging_steps": 1,
+  "max_steps": 99,
+  "num_input_tokens_seen": 0,
+  "num_train_epochs": 1,
+  "save_steps": 300,
+  "stateful_callbacks": {
+    "TrainerControl": {
+      "args": {
+        "should_epoch_stop": false,
+        "should_evaluate": false,
+        "should_log": false,
+        "should_save": true,
+        "should_training_stop": true
+      },
+      "attributes": {}
+    }
+  },
+  "total_flos": 2.8519729975148544e+16,
+  "train_batch_size": 2,
+  "trial_name": null,
+  "trial_params": null
+}

last-checkpoint/training_args.bin

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:c0c9943b50430682b61831099ea7d294642979741c2346544ba10d5c016524eb
+size 6712