add readme

README.md

@@ -0,0 +1,236 @@
+---
+license: apache-2.0
+license_link: https://huggingface.co/Qwen/Qwen2.5-7B-Instruct-1M/blob/main/LICENSE
+language:
+- en
+pipeline_tag: text-generation
+base_model: Qwen/Qwen2.5-7B
+tags:
+- chat
+library_name: transformers
+---
+
+# Qwen2.5-7B-Instruct-1M
+<a href="https://chat.qwenlm.ai/" target="_blank" style="margin: 2px;">
+    <img alt="Chat" src="https://img.shields.io/badge/%F0%9F%92%9C%EF%B8%8F%20Qwen%20Chat%20-536af5" style="display: inline-block; vertical-align: middle;"/>
+</a>
+
+## Introduction
+
+Qwen2.5-1M is the long-context version of the Qwen2.5 series models, supporting a context length of up to 1M tokens. Compared to the Qwen2.5 128K version, Qwen2.5-1M demonstrates significantly improved performance in handling long-context tasks while maintaining its capability in short tasks.
+
+The model has the following features:
+- Type: Causal Language Models
+- Training Stage: Pretraining & Post-training
+- Architecture: transformers with RoPE, SwiGLU, RMSNorm, and Attention QKV bias
+- Number of Parameters: 7.61B
+- Number of Paramaters (Non-Embedding): 6.53B
+- Number of Layers: 28
+- Number of Attention Heads (GQA): 28 for Q and 4 for KV
+- Context Length: Full 1,010,000 tokens and generation 8192 tokens
+  - We recommend deploying with our custom vLLM, which introduces sparse attention and length extrapolation methods to ensure efficiency and accuracy for long-context tasks. For specific guidance, refer to [this section](#processing-ultra-long-texts).
+  - You can also use the previous framework that supports Qwen2.5 for inference, but accuracy degradation may occur for sequences exceeding 262,144 tokens.
+
+For more details, please refer to our [blog](https://qwenlm.github.io/blog/qwen2.5-1m/), [GitHub](https://github.com/QwenLM/Qwen2.5), and [Documentation](https://qwen.readthedocs.io/en/latest/).
+
+## Requirements
+
+The code of Qwen2.5 has been in the latest Hugging face `transformers` and we advise you to use the latest version of `transformers`.
+
+With `transformers<4.37.0`, you will encounter the following error:
+```
+KeyError: 'qwen2'
+```
+
+## Quickstart
+
+Here provides a code snippet with `apply_chat_template` to show you how to load the tokenizer and model and how to generate contents.
+
+```python
+from transformers import AutoModelForCausalLM, AutoTokenizer
+
+model_name = "Qwen/Qwen2.5-7B-Instruct"
+
+model = AutoModelForCausalLM.from_pretrained(
+    model_name,
+    torch_dtype="auto",
+    device_map="auto"
+)
+tokenizer = AutoTokenizer.from_pretrained(model_name)
+
+prompt = "Give me a short introduction to large language model."
+messages = [
+    {"role": "system", "content": "You are Qwen, created by Alibaba Cloud. You are a helpful assistant."},
+    {"role": "user", "content": prompt}
+]
+text = tokenizer.apply_chat_template(
+    messages,
+    tokenize=False,
+    add_generation_prompt=True
+)
+model_inputs = tokenizer([text], return_tensors="pt").to(model.device)
+
+generated_ids = model.generate(
+    **model_inputs,
+    max_new_tokens=512
+)
+generated_ids = [
+    output_ids[len(input_ids):] for input_ids, output_ids in zip(model_inputs.input_ids, generated_ids)
+]
+
+response = tokenizer.batch_decode(generated_ids, skip_special_tokens=True)[0]
+```
+
+### Processing Ultra Long Texts
+
+To enhance processing accuracy and efficiency for long sequences, we have developed an advanced inference framework based on vLLM, incorporating sparse attention and length extrapolation. This approach significantly improves model generation performance for sequences exceeding 256K tokens and achieves a 3 to 7 times speedup for sequences up to 1M tokens.
+
+Here we provide step-by-step instructions for deploying the Qwen2.5-1M models with our framework.
+
+#### 1. System Preparation
+
+To achieve the best performance, we recommend using GPUs with Ampere or Hopper architecture, which support optimized kernels.
+
+Ensure your system meets the following requirements:
+
+- **CUDA Version**: 12.1 or 12.3
+- **Python Version**: >=3.9 and <=3.12
+
+**VRAM Requirements:**
+
+- For processing 1 million-token sequences:
+  - **Qwen2.5-7B-Instruct-1M**: At least 120GB VRAM (total across GPUs).
+  - **Qwen2.5-14B-Instruct-1M**: At least 320GB VRAM (total across GPUs).
+
+If your GPUs do not have sufficient VRAM, you can still use Qwen2.5-1M for shorter tasks.
+
+#### 2. Install Dependencies
+
+For now, you need to clone the vLLM repository from our custom branch and install it manually. We are working on getting our branch merged into the main vLLM project.
+
+```bash
+git clone -b dev/dual-chunk-attn [email protected]:QwenLM/vllm.git
+cd vllm
+pip install -e . -v
+```
+
+
+#### 3. Launch vLLM
+
+vLLM supports offline inference or launch an openai-like server.
+
+**Example of Offline Inference**
+
+```python
+from transformers import AutoTokenizer
+from vllm import LLM, SamplingParams
+
+# Initialize the tokenizer
+tokenizer = AutoTokenizer.from_pretrained("Qwen/Qwen2.5-7B-Instruct-1M")
+
+# Pass the default decoding hyperparameters of Qwen2.5-7B-Instruct
+# max_tokens is for the maximum length for generation.
+sampling_params = SamplingParams(temperature=0.7, top_p=0.8, repetition_penalty=1.05, max_tokens=512)
+
+# Input the model name or path. See below for parameter explanation (after the example of openai-like server).
+llm = LLM(model="Qwen/Qwen2.5-7B-Instruct-1M",
+    tensor_parallel_size=4,
+    max_model_len=1010000,
+    enable_chunked_prefill=True,
+    max_num_batched_tokens=131072,
+    enforce_eager=True,
+    # quantization="fp8", # Enabling FP8 quantization for model weights can reduce memory usage.
+)
+
+# Prepare your prompts
+prompt = "Tell me something about large language models."
+messages = [
+    {"role": "system", "content": "You are Qwen, created by Alibaba Cloud. You are a helpful assistant."},
+    {"role": "user", "content": prompt}
+]
+text = tokenizer.apply_chat_template(
+    messages,
+    tokenize=False,
+    add_generation_prompt=True
+)
+
+# generate outputs
+outputs = llm.generate([text], sampling_params)
+
+# Print the outputs.
+for output in outputs:
+    prompt = output.prompt
+    generated_text = output.outputs[0].text
+    print(f"Prompt: {prompt!r}, Generated text: {generated_text!r}")
+```
+
+**Example of Openai-like Server**
+
+```bash
+vllm serve Qwen/Qwen2.5-7B-Instruct-1M \
+  --tensor-parallel-size 4 \
+  --max-model-len 1010000 \
+  --enable-chunked-prefill --max-num-batched-tokens 131072 \
+  --enforce-eager \
+  --max-num-seqs 1
+
+# --quantization fp8  # Enabling FP8 quantization for model weights can reduce memory usage.
+```
+
+Then you can use curl or python to interact with the deployed model.
+
+**Parameter Explanations:**
+
+- **`--tensor-parallel-size`**
+  - Set to the number of GPUs you are using. Max 4 GPUs for the 7B model, and 8 GPUs for the 14B model.
+  
+- **`--max-model-len`**
+  - Defines the maximum input sequence length. Reduce this value if you encounter Out of Memory issues.
+
+- **`--max-num-batched-tokens`**
+  - Sets the chunk size in Chunked Prefill. A smaller value reduces activation memory usage but may slow down inference. 
+  - Recommend 131072 for optimal performance.
+
+- **`--max-num-seqs`**
+  - Limits concurrent sequences processed. 
+
+You can also refer to our [Documentation](https://qwen.readthedocs.io/en/latest/deployment/vllm.html) for usage of vLLM.
+
+#### Troubleshooting:
+
+1. Encountering the error: "The model's max sequence length (xxxxx) is larger than the maximum number of tokens that can be stored in the KV cache."
+
+    The VRAM reserved for the KV cache is insufficient. Consider reducing the ``max_model_len`` or increasing the ``tensor_parallel_size``. Alternatively, you can reduce ``max_num_batched_tokens``, although this may significantly slow down inference.
+
+2. Encountering the error: "torch.OutOfMemoryError: CUDA out of memory."
+
+    The VRAM reserved for activation weights is insufficient. You can try setting ``gpu_memory_utilization`` to 0.85 or lower, but be aware that this might reduce the VRAM available for the KV cache.
+
+3. Encountering the error: "Input prompt (xxxxx tokens) + lookahead slots (0) is too long and exceeds the capacity of the block manager."
+
+    The input is too lengthy. Consider using a shorter sequence or increasing the ``max_model_len``.
+
+## Evaluation & Performance
+
+Detailed evaluation results are reported in this [📑 blog](https://qwenlm.github.io/blog/qwen2.5-1m/) and our [technical report](https://qianwen-res.oss-cn-beijing.aliyuncs.com/Qwen2.5-1M/Qwen2_5_1M_Technical_Report.pdf).
+
+## Citation
+
+If you find our work helpful, feel free to give us a cite.
+
+```
+@misc{qwen2.5-1m,
+    title = {Qwen2.5-1M: Deploy Your Own Qwen with Context Length up to 1M Tokens},
+    url = {https://qwenlm.github.io/blog/qwen2.5-1m/},
+    author = {Qwen Team},
+    month = {January},
+    year = {2025}
+}
+
+@article{qwen2.5,
+      title={Qwen2.5 Technical Report}, 
+      author={An Yang and Baosong Yang and Beichen Zhang and Binyuan Hui and Bo Zheng and Bowen Yu and Chengyuan Li and Dayiheng Liu and Fei Huang and Haoran Wei and Huan Lin and Jian Yang and Jianhong Tu and Jianwei Zhang and Jianxin Yang and Jiaxi Yang and Jingren Zhou and Junyang Lin and Kai Dang and Keming Lu and Keqin Bao and Kexin Yang and Le Yu and Mei Li and Mingfeng Xue and Pei Zhang and Qin Zhu and Rui Men and Runji Lin and Tianhao Li and Tianyi Tang and Tingyu Xia and Xingzhang Ren and Xuancheng Ren and Yang Fan and Yang Su and Yichang Zhang and Yu Wan and Yuqiong Liu and Zeyu Cui and Zhenru Zhang and Zihan Qiu},
+      journal={arXiv preprint arXiv:2412.15115},
+      year={2024}
+}
+```

config.json

@@ -24,5 +24,10 @@
   "transformers_version": "4.47.1",
   "use_cache": true,
   "use_sliding_window": false,
-  "vocab_size": 152064
+  "vocab_size": 152064,
+  "dual_chunk_attention_config": {
+    "chunk_size": 262144,
+    "local_size": 8192,
+    "original_max_position_embeddings": 262144
+  }
 }

sparse_attention_config.json

@@ -0,0 +1 @@
+[{"0": ["vertical_and_slash", 9192, 14288, 0.6920984983444214], "1": ["vertical_and_slash", 1000, 6096, 0.9862403869628906], "2": ["vertical_and_slash", 3048, 8144, 0.8198256492614746], "3": ["vertical_and_slash", 11340, 16336, 0.9518619179725647], "4": ["vertical_and_slash", 5096, 10192, 0.739506721496582], "5": ["vertical_and_slash", 1000, 6096, 0.9902973771095276], "6": ["vertical_and_slash", 30, 800, 0.9972315430641174], "7": ["vertical_and_slash", 1000, 6096, 0.9914392828941345], "8": ["vertical_and_slash", 1000, 6096, 0.950901210308075], "9": ["vertical_and_slash", 3048, 8144, 0.8771697878837585], "10": ["vertical_and_slash", 13388, 18384, 0.8314550518989563], "11": ["vertical_and_slash", 1000, 6096, 0.9330742359161377], "12": ["vertical_and_slash", 1000, 6096, 0.9362020492553711], "13": ["vertical_and_slash", 3048, 8144, 0.8856230974197388], "14": ["vertical_and_slash", 9192, 14288, 0.6533831357955933], "15": ["vertical_and_slash", 13388, 18384, 0.8002538084983826], "16": ["vertical_and_slash", 1000, 6096, 0.9781253933906555], "17": ["vertical_and_slash", 3048, 8144, 0.8541072010993958], "18": ["vertical_and_slash", 9192, 14288, 0.6799413561820984], "19": ["vertical_and_slash", 3048, 8144, 0.8869160413742065], "20": ["vertical_and_slash", 1000, 6096, 0.9579253196716309], "21": ["vertical_and_slash", 1000, 6096, 0.9464394450187683], "22": ["vertical_and_slash", 3048, 8144, 0.8140186071395874], "23": ["vertical_and_slash", 1000, 6096, 0.9577401876449585], "24": ["vertical_and_slash", 1000, 6096, 0.9639880061149597], "25": ["vertical_and_slash", 1000, 6096, 0.9603379368782043], "26": ["vertical_and_slash", 1000, 6096, 0.9592313170433044], "27": ["vertical_and_slash", 1000, 6096, 0.9610494375228882]}, {"0": ["vertical_and_slash", 1000, 6096, 0.9999175667762756], "1": ["vertical_and_slash", 11692, 8292, 0.5583158731460571], "2": ["vertical_and_slash", 11340, 16336, 0.9962033033370972], "3": ["vertical_and_slash", 1000, 6096, 0.9998870491981506], "4": ["vertical_and_slash", 1000, 6096, 0.9992745518684387], "5": ["vertical_and_slash", 1000, 6096, 0.9997513890266418], "6": ["vertical_and_slash", 13388, 18384, 0.8439157009124756], "7": ["vertical_and_slash", 1000, 6096, 0.9102388024330139], "8": ["vertical_and_slash", 3048, 8144, 0.8252120018005371], "9": ["vertical_and_slash", 9192, 14288, 0.5839092135429382], "10": ["vertical_and_slash", 1000, 6096, 0.9527733325958252], "11": ["vertical_and_slash", 1000, 6096, 0.9262151122093201], "12": ["vertical_and_slash", 1000, 6096, 0.9747101664543152], "13": ["vertical_and_slash", 9192, 14288, 0.693157434463501], "14": ["vertical_and_slash", 1000, 6096, 0.9503929615020752], "15": ["vertical_and_slash", 1000, 6096, 0.9907119870185852], "16": ["vertical_and_slash", 5096, 10192, 0.7641042470932007], "17": ["vertical_and_slash", 1000, 6096, 0.9974478483200073], "18": ["vertical_and_slash", 3048, 8144, 0.864757776260376], "19": ["vertical_and_slash", 1000, 6096, 0.9005423188209534], "20": ["vertical_and_slash", 1000, 6096, 0.934573769569397], "21": ["vertical_and_slash", 9192, 14288, 0.5810629725456238], "22": ["vertical_and_slash", 5096, 10192, 0.7144602537155151], "23": ["vertical_and_slash", 1000, 6096, 0.9564751386642456], "24": ["vertical_and_slash", 1000, 6096, 0.9916792511940002], "25": ["vertical_and_slash", 3048, 8144, 0.8495540618896484], "26": ["vertical_and_slash", 9192, 14288, 0.6452760696411133], "27": ["vertical_and_slash", 1000, 6096, 0.9232876300811768]}, {"0": ["vertical_and_slash", 9744, 6244, 0.9077025055885315], "1": ["vertical_and_slash", 5548, 2148, 0.8882887959480286], "2": ["vertical_and_slash", 3500, 100, 0.9153436422348022], "3": ["vertical_and_slash", 3500, 100, 0.9312531352043152], "4": ["vertical_and_slash", 9192, 14288, 0.6427125334739685], "5": ["vertical_and_slash", 3500, 100, 0.9152134656906128], "6": ["vertical_and_slash", 5548, 2148, 0.8578056693077087], "7": ["vertical_and_slash", 1000, 6096, 0.932381808757782], "8": ["vertical_and_slash", 3048, 8144, 0.8542991280555725], "9": ["vertical_and_slash", 1000, 6096, 0.9245713949203491], "10": ["vertical_and_slash", 3048, 8144, 0.8989690542221069], "11": ["vertical_and_slash", 1000, 6096, 0.9269236326217651], "12": ["vertical_and_slash", 5096, 10192, 0.7965206503868103], "13": ["vertical_and_slash", 5096, 10192, 0.7843445539474487], "14": ["vertical_and_slash", 9192, 14288, 0.5634094476699829], "15": ["vertical_and_slash", 5096, 10192, 0.7882068157196045], "16": ["vertical_and_slash", 5096, 10192, 0.7812356352806091], "17": ["vertical_and_slash", 9192, 14288, 0.6630747318267822], "18": ["vertical_and_slash", 11692, 8292, 0.695554792881012], "19": ["vertical_and_slash", 9192, 14288, 0.6496447920799255], "20": ["vertical_and_slash", 9192, 14288, 0.6239688992500305], "21": ["vertical_and_slash", 3048, 8144, 0.8381555676460266], "22": ["vertical_and_slash", 9192, 14288, 0.5935300588607788], "23": ["vertical_and_slash", 1000, 6096, 0.9077279567718506], "24": ["vertical_and_slash", 5096, 10192, 0.7705077528953552], "25": ["vertical_and_slash", 3048, 8144, 0.8881292343139648], "26": ["vertical_and_slash", 15436, 20432, 0.654614269733429], "27": ["vertical_and_slash", 1000, 6096, 0.9183178544044495]}, {"0": ["vertical_and_slash", 9192, 14288, 0.6299600005149841], "1": ["vertical_and_slash", 1000, 6096, 0.9939385652542114], "2": ["vertical_and_slash", 1000, 6096, 0.9255406856536865], "3": ["vertical_and_slash", 1000, 6096, 0.9815036058425903], "4": ["vertical_and_slash", 1000, 6096, 0.9068899154663086], "5": ["vertical_and_slash", 1000, 6096, 0.9562771916389465], "6": ["vertical_and_slash", 1000, 6096, 0.9499258399009705], "7": ["vertical_and_slash", 1000, 6096, 0.9981131553649902], "8": ["vertical_and_slash", 9192, 14288, 0.47260501980781555], "9": ["vertical_and_slash", 1000, 6096, 0.9922735691070557], "10": ["vertical_and_slash", 1000, 6096, 0.9986152052879333], "11": ["vertical_and_slash", 1000, 6096, 0.9634198546409607], "12": ["vertical_and_slash", 1000, 6096, 0.9510660171508789], "13": ["vertical_and_slash", 1000, 6096, 0.9214046597480774], "14": ["vertical_and_slash", 5096, 10192, 0.7424683570861816], "15": ["vertical_and_slash", 1000, 6096, 0.9193776845932007], "16": ["vertical_and_slash", 11692, 8292, 0.6805881261825562], "17": ["vertical_and_slash", 5096, 10192, 0.7827474474906921], "18": ["vertical_and_slash", 9192, 14288, 0.6499208211898804], "19": ["vertical_and_slash", 3048, 8144, 0.8205469846725464], "20": ["vertical_and_slash", 1000, 6096, 0.9113396406173706], "21": ["vertical_and_slash", 3048, 8144, 0.8239019513130188], "22": ["vertical_and_slash", 1000, 6096, 0.953626275062561], "23": ["vertical_and_slash", 3048, 8144, 0.8758710622787476], "24": ["vertical_and_slash", 1000, 6096, 0.9802020788192749], "25": ["vertical_and_slash", 1000, 6096, 0.9784566760063171], "26": ["vertical_and_slash", 1000, 6096, 0.9459996819496155], "27": ["vertical_and_slash", 1000, 6096, 0.9493811726570129]}, {"0": ["vertical_and_slash", 3048, 8144, 0.8282588720321655], "1": ["vertical_and_slash", 3048, 8144, 0.8945374488830566], "2": ["vertical_and_slash", 1000, 6096, 0.9859223961830139], "3": ["vertical_and_slash", 1000, 6096, 0.9584572911262512], "4": ["vertical_and_slash", 1000, 6096, 0.9155289530754089], "5": ["vertical_and_slash", 1000, 6096, 0.9548629522323608], "6": ["vertical_and_slash", 3048, 8144, 0.8657681345939636], "7": ["vertical_and_slash", 1000, 6096, 0.9909697771072388], "8": ["vertical_and_slash", 3048, 8144, 0.8176682591438293], "9": ["vertical_and_slash", 1000, 6096, 0.9185464978218079], "10": ["vertical_and_slash", 3048, 8144, 0.8876745104789734], "11": ["vertical_and_slash", 1000, 6096, 0.9989773631095886], "12": ["vertical_and_slash", 100, 800, 1.0], "13": ["vertical_and_slash", 13840, 10340, 0.9865253567695618], "14": ["vertical_and_slash", 30, 800, 0.9833630919456482], "15": ["vertical_and_slash", 100, 800, 0.9375], "16": ["vertical_and_slash", 1000, 6096, 0.9603279829025269], "17": ["vertical_and_slash", 9192, 14288, 0.5568011403083801], "18": ["vertical_and_slash", 30, 800, 0.9944095611572266], "19": ["vertical_and_slash", 100, 800, 0.98828125], "20": ["vertical_and_slash", 30, 800, 0.9845749139785767], "21": ["vertical_and_slash", 1000, 6096, 0.9226506948471069], "22": ["vertical_and_slash", 30, 800, 0.9778012633323669], "23": ["vertical_and_slash", 30, 800, 0.9957334399223328], "24": ["vertical_and_slash", 5096, 10192, 0.7061529159545898], "25": ["vertical_and_slash", 30, 800, 0.9853928685188293], "26": ["vertical_and_slash", 3048, 8144, 0.8172721862792969], "27": ["vertical_and_slash", 30, 800, 0.9829623103141785]}, {"0": ["vertical_and_slash", 1000, 6096, 0.9093767404556274], "1": ["vertical_and_slash", 30, 800, 0.9735046625137329], "2": ["vertical_and_slash", 1000, 6096, 0.9702109098434448], "3": ["vertical_and_slash", 30, 800, 0.9810881614685059], "4": ["vertical_and_slash", 1000, 6096, 0.956884503364563], "5": ["vertical_and_slash", 15436, 20432, 0.5826653242111206], "6": ["vertical_and_slash", 9192, 14288, 0.6588539481163025], "7": ["vertical_and_slash", 1000, 6096, 0.9873548150062561], "8": ["vertical_and_slash", 1000, 6096, 0.9430211186408997], "9": ["vertical_and_slash", 5096, 10192, 0.7686501145362854], "10": ["vertical_and_slash", 1000, 6096, 0.9063356518745422], "11": ["vertical_and_slash", 9192, 14288, 0.6081218123435974], "12": ["vertical_and_slash", 1000, 6096, 0.9467446208000183], "13": ["vertical_and_slash", 3048, 8144, 0.8317232728004456], "14": ["vertical_and_slash", 1000, 6096, 0.9639446139335632], "15": ["vertical_and_slash", 3048, 8144, 0.8432199358940125], "16": ["vertical_and_slash", 1000, 6096, 0.9603729844093323], "17": ["vertical_and_slash", 9192, 14288, 0.5282297134399414], "18": ["vertical_and_slash", 9192, 14288, 0.5313483476638794], "19": ["vertical_and_slash", 19532, 24528, 0.6716174483299255], "20": ["vertical_and_slash", 9192, 14288, 0.5943314433097839], "21": ["vertical_and_slash", 9192, 14288, 0.5765253901481628], "22": ["vertical_and_slash", 30, 800, 0.9723267555236816], "23": ["vertical_and_slash", 30, 800, 0.9864580631256104], "24": ["vertical_and_slash", 100, 800, 0.9375], "25": ["vertical_and_slash", 30, 800, 0.9841305613517761], "26": ["vertical_and_slash", 100, 800, 0.9375], "27": ["vertical_and_slash", 100, 800, 0.94140625]}, {"0": ["vertical_and_slash", 30, 800, 0.9887828826904297], "1": ["vertical_and_slash", 30, 800, 0.991061270236969], "2": ["vertical_and_slash", 30, 800, 0.9967905282974243], "3": ["vertical_and_slash", 30, 800, 0.9886236786842346], "4": ["vertical_and_slash", 30, 800, 0.995130717754364], "5": ["vertical_and_slash", 30, 800, 0.980113685131073], "6": ["vertical_and_slash", 30, 800, 0.9969208836555481], "7": ["vertical_and_slash", 1000, 6096, 0.960637092590332], "8": ["vertical_and_slash", 1000, 6096, 0.9405831694602966], "9": ["vertical_and_slash", 1000, 6096, 0.9176963567733765], "10": ["vertical_and_slash", 3048, 8144, 0.819543182849884], "11": ["vertical_and_slash", 3048, 8144, 0.849609911441803], "12": ["vertical_and_slash", 1000, 6096, 0.9364283084869385], "13": ["vertical_and_slash", 3500, 100, 0.9509801864624023], "14": ["vertical_and_slash", 30, 800, 0.9800994396209717], "15": ["vertical_and_slash", 30, 800, 0.9964476227760315], "16": ["vertical_and_slash", 30, 800, 0.9962098598480225], "17": ["vertical_and_slash", 3048, 8144, 0.8155508637428284], "18": ["vertical_and_slash", 5096, 10192, 0.969977855682373], "19": ["vertical_and_slash", 30, 800, 0.9929144978523254], "20": ["vertical_and_slash", 30, 800, 0.9956931471824646], "21": ["vertical_and_slash", 30, 800, 0.9900340437889099], "22": ["vertical_and_slash", 30, 800, 0.9817578196525574], "23": ["vertical_and_slash", 5096, 10192, 0.7279469966888428], "24": ["vertical_and_slash", 30, 800, 0.977938711643219], "25": ["vertical_and_slash", 30, 800, 0.9940697550773621], "26": ["vertical_and_slash", 30, 800, 0.9888582825660706], "27": ["vertical_and_slash", 30, 800, 0.9776446223258972]}, {"0": ["vertical_and_slash", 6274, 6944, 0.9849215745925903], "1": ["vertical_and_slash", 100, 800, 0.91796875], "2": ["vertical_and_slash", 3048, 8144, 0.8702616691589355], "3": ["vertical_and_slash", 1000, 6096, 0.9742972254753113], "4": ["vertical_and_slash", 30, 800, 0.9930776953697205], "5": ["vertical_and_slash", 30, 800, 0.9923169016838074], "6": ["vertical_and_slash", 11340, 16336, 0.7224187850952148], "7": ["vertical_and_slash", 1000, 6096, 0.9252712726593018], "8": ["vertical_and_slash", 1000, 6096, 0.987740695476532], "9": ["vertical_and_slash", 1000, 6096, 0.9918652176856995], "10": ["vertical_and_slash", 3048, 8144, 0.8553299307823181], "11": ["vertical_and_slash", 30, 800, 0.9839231967926025], "12": ["vertical_and_slash", 1000, 6096, 0.9922063946723938], "13": ["vertical_and_slash", 30, 800, 0.9844004511833191], "14": ["vertical_and_slash", 7244, 12240, 0.9108995199203491], "15": ["vertical_and_slash", 5096, 10192, 0.7733249068260193], "16": ["vertical_and_slash", 100, 800, 0.9921875], "17": ["vertical_and_slash", 1000, 6096, 0.9894708395004272], "18": ["vertical_and_slash", 6344, 6944, 0.96484375], "19": ["vertical_and_slash", 5096, 10192, 0.7390482425689697], "20": ["vertical_and_slash", 30, 800, 0.9494436979293823], "21": ["vertical_and_slash", 1000, 6096, 0.9951373934745789], "22": ["vertical_and_slash", 1000, 6096, 0.9152458906173706], "23": ["vertical_and_slash", 1000, 6096, 0.9692571759223938], "24": ["vertical_and_slash", 1000, 6096, 0.9701493978500366], "25": ["vertical_and_slash", 1000, 6096, 0.983892023563385], "26": ["vertical_and_slash", 3048, 8144, 0.8999871611595154], "27": ["vertical_and_slash", 1000, 6096, 0.9546129703521729]}, {"0": ["vertical_and_slash", 1000, 6096, 0.9800829291343689], "1": ["vertical_and_slash", 1000, 6096, 0.9934086799621582], "2": ["vertical_and_slash", 1000, 6096, 0.9579113721847534], "3": ["vertical_and_slash", 1000, 6096, 0.910209059715271], "4": ["vertical_and_slash", 3048, 8144, 0.8276802897453308], "5": ["vertical_and_slash", 1000, 6096, 0.9891079068183899], "6": ["vertical_and_slash", 1000, 6096, 0.9916367530822754], "7": ["vertical_and_slash", 1000, 6096, 0.9807681441307068], "8": ["vertical_and_slash", 1000, 6096, 0.996056318283081], "9": ["vertical_and_slash", 1000, 6096, 0.9689919948577881], "10": ["vertical_and_slash", 1000, 6096, 0.9973182678222656], "11": ["vertical_and_slash", 1000, 6096, 0.9486907720565796], "12": ["vertical_and_slash", 1000, 6096, 0.9872133135795593], "13": ["vertical_and_slash", 1000, 6096, 0.9248751997947693], "14": ["vertical_and_slash", 500, 700, 0.9737105965614319], "15": ["vertical_and_slash", 30, 800, 0.9953463673591614], "16": ["vertical_and_slash", 500, 700, 0.996398389339447], "17": ["vertical_and_slash", 7596, 4196, 0.9452576041221619], "18": ["vertical_and_slash", 500, 700, 0.9803798794746399], "19": ["vertical_and_slash", 30, 800, 0.9829242825508118], "20": ["vertical_and_slash", 30, 800, 0.9800650477409363], "21": ["vertical_and_slash", 1000, 6096, 0.9978428483009338], "22": ["vertical_and_slash", 15436, 20432, 0.6166174411773682], "23": ["vertical_and_slash", 13388, 18384, 0.8101190328598022], "24": ["vertical_and_slash", 1000, 6096, 0.9721275568008423], "25": ["vertical_and_slash", 7244, 12240, 0.9621387720108032], "26": ["vertical_and_slash", 5096, 10192, 0.761796236038208], "27": ["vertical_and_slash", 15436, 20432, 0.735977292060852]}, {"0": ["vertical_and_slash", 1000, 6096, 0.9662788510322571], "1": ["vertical_and_slash", 1000, 6096, 0.933112621307373], "2": ["vertical_and_slash", 3048, 8144, 0.845474123954773], "3": ["vertical_and_slash", 11340, 16336, 0.9396570324897766], "4": ["vertical_and_slash", 5548, 2148, 0.8758905529975891], "5": ["vertical_and_slash", 1000, 6096, 0.9595933556556702], "6": ["vertical_and_slash", 1000, 6096, 0.9791407585144043], "7": ["vertical_and_slash", 1000, 6096, 0.9448844194412231], "8": ["vertical_and_slash", 15436, 20432, 0.7103061079978943], "9": ["vertical_and_slash", 1000, 6096, 0.985871434211731], "10": ["vertical_and_slash", 7596, 4196, 0.7926585078239441], "11": ["vertical_and_slash", 5096, 10192, 0.9791630506515503], "12": ["vertical_and_slash", 1000, 6096, 0.9933391809463501], "13": ["vertical_and_slash", 11340, 16336, 0.9010775089263916], "14": ["vertical_and_slash", 1000, 6096, 0.9843372702598572], "15": ["vertical_and_slash", 3048, 8144, 0.8298346996307373], "16": ["vertical_and_slash", 1000, 6096, 0.9661977887153625], "17": ["vertical_and_slash", 1000, 6096, 0.90715092420578], "18": ["vertical_and_slash", 1000, 6096, 0.9170965552330017], "19": ["vertical_and_slash", 9192, 14288, 0.6205919981002808], "20": ["vertical_and_slash", 1000, 6096, 0.9696729779243469], "21": ["vertical_and_slash", 1000, 6096, 0.9067583084106445], "22": ["vertical_and_slash", 7596, 4196, 0.7575717568397522], "23": ["vertical_and_slash", 7244, 12240, 0.9475517868995667], "24": ["vertical_and_slash", 1000, 6096, 0.966336727142334], "25": ["vertical_and_slash", 5548, 2148, 0.8904582262039185], "26": ["vertical_and_slash", 1000, 6096, 0.9011759757995605], "27": ["vertical_and_slash", 1000, 6096, 0.9367696046829224]}, {"0": ["vertical_and_slash", 1000, 6096, 0.9443923234939575], "1": ["vertical_and_slash", 7244, 12240, 0.9806238412857056], "2": ["vertical_and_slash", 13840, 10340, 0.957985520362854], "3": ["vertical_and_slash", 11340, 16336, 0.9818342924118042], "4": ["vertical_and_slash", 500, 700, 0.9885478019714355], "5": ["vertical_and_slash", 1000, 6096, 0.9833213090896606], "6": ["vertical_and_slash", 7596, 4196, 0.7089235782623291], "7": ["vertical_and_slash", 1000, 6096, 0.9972561597824097], "8": ["vertical_and_slash", 1000, 6096, 0.995736837387085], "9": ["vertical_and_slash", 1000, 6096, 0.9955088496208191], "10": ["vertical_and_slash", 1000, 6096, 0.9935648441314697], "11": ["vertical_and_slash", 1000, 6096, 0.9156711101531982], "12": ["vertical_and_slash", 1000, 6096, 0.9951316118240356], "13": ["vertical_and_slash", 1000, 6096, 0.9924508929252625], "14": ["vertical_and_slash", 1000, 6096, 0.9403036832809448], "15": ["vertical_and_slash", 30, 800, 0.9910191297531128], "16": ["vertical_and_slash", 500, 700, 0.9839996099472046], "17": ["vertical_and_slash", 3500, 100, 0.9900570511817932], "18": ["vertical_and_slash", 1000, 6096, 0.901885449886322], "19": ["vertical_and_slash", 1000, 6096, 0.9504903554916382], "20": ["vertical_and_slash", 1000, 6096, 0.9650601744651794], "21": ["vertical_and_slash", 10840, 10940, 0.990660548210144], "22": ["vertical_and_slash", 30, 800, 0.9931259155273438], "23": ["vertical_and_slash", 6274, 6944, 0.9908908605575562], "24": ["vertical_and_slash", 1000, 6096, 0.9198976755142212], "25": ["vertical_and_slash", 30, 800, 0.9933606386184692], "26": ["vertical_and_slash", 6274, 6944, 0.9890289306640625], "27": ["vertical_and_slash", 30, 800, 0.9929149746894836]}, {"0": ["vertical_and_slash", 3048, 8144, 0.8870794177055359], "1": ["vertical_and_slash", 30, 800, 0.9871358275413513], "2": ["vertical_and_slash", 1000, 6096, 0.9092150330543518], "3": ["vertical_and_slash", 1000, 6096, 0.9849765300750732], "4": ["vertical_and_slash", 500, 700, 0.9693071246147156], "5": ["vertical_and_slash", 30, 800, 0.9693368673324585], "6": ["vertical_and_slash", 1000, 6096, 0.9419503211975098], "7": ["vertical_and_slash", 30, 800, 0.9812013506889343], "8": ["vertical_and_slash", 15436, 20432, 0.6496729254722595], "9": ["vertical_and_slash", 100, 800, 0.94140625], "10": ["vertical_and_slash", 1000, 6096, 0.9741862416267395], "11": ["vertical_and_slash", 4196, 4896, 0.95703125], "12": ["vertical_and_slash", 15788, 12388, 0.46300187706947327], "13": ["vertical_and_slash", 10440, 11040, 0.96875], "14": ["vertical_and_slash", 30, 800, 0.9830393195152283], "15": ["vertical_and_slash", 30, 800, 0.9927415251731873], "16": ["vertical_and_slash", 1000, 6096, 0.9534035325050354], "17": ["vertical_and_slash", 100, 800, 0.984375], "18": ["vertical_and_slash", 30, 800, 0.9904463291168213], "19": ["vertical_and_slash", 6274, 6944, 0.9873848557472229], "20": ["vertical_and_slash", 6274, 6944, 0.991168737411499], "21": ["vertical_and_slash", 7244, 12240, 0.9426537752151489], "22": ["vertical_and_slash", 3048, 8144, 0.8975484371185303], "23": ["vertical_and_slash", 5096, 10192, 0.7350636124610901], "24": ["vertical_and_slash", 11340, 16336, 0.9229165315628052], "25": ["vertical_and_slash", 1000, 6096, 0.9412928223609924], "26": ["vertical_and_slash", 5096, 10192, 0.9706910252571106], "27": ["vertical_and_slash", 5096, 10192, 0.9476442933082581]}, {"0": ["vertical_and_slash", 1000, 6096, 0.9863660931587219], "1": ["vertical_and_slash", 4196, 4896, 0.95703125], "2": ["vertical_and_slash", 5548, 2148, 0.8480055332183838], "3": ["vertical_and_slash", 500, 700, 0.9538192749023438], "4": ["vertical_and_slash", 4196, 4896, 0.953125], "5": ["vertical_and_slash", 6344, 6944, 0.98046875], "6": ["vertical_and_slash", 5096, 10192, 0.7229321599006653], "7": ["vertical_and_slash", 17936, 14436, 0.7735021114349365], "8": ["vertical_and_slash", 1000, 6096, 0.9491978883743286], "9": ["vertical_and_slash", 1000, 6096, 0.9595749974250793], "10": ["vertical_and_slash", 1000, 6096, 0.942698061466217], "11": ["vertical_and_slash", 1000, 6096, 0.9691566228866577], "12": ["vertical_and_slash", 3048, 8144, 0.8794195055961609], "13": ["vertical_and_slash", 1000, 6096, 0.9725112915039062], "14": ["vertical_and_slash", 1000, 6096, 0.9814248085021973], "15": ["vertical_and_slash", 3048, 8144, 0.8134639859199524], "16": ["vertical_and_slash", 5096, 10192, 0.7067453265190125], "17": ["vertical_and_slash", 9192, 14288, 0.5412349700927734], "18": ["vertical_and_slash", 1000, 6096, 0.9959503412246704], "19": ["vertical_and_slash", 1000, 6096, 0.9272978901863098], "20": ["vertical_and_slash", 11340, 16336, 0.934908926486969], "21": ["vertical_and_slash", 1000, 6096, 0.9556578397750854], "22": ["vertical_and_slash", 1000, 6096, 0.9684078097343445], "23": ["vertical_and_slash", 1000, 6096, 0.9632892608642578], "24": ["vertical_and_slash", 13388, 18384, 0.8790978789329529], "25": ["vertical_and_slash", 1000, 6096, 0.9878668785095215], "26": ["vertical_and_slash", 1000, 6096, 0.9301976561546326], "27": ["vertical_and_slash", 1000, 6096, 0.9775260090827942]}, {"0": ["vertical_and_slash", 1000, 6096, 0.9639821648597717], "1": ["vertical_and_slash", 1000, 6096, 0.9714324474334717], "2": ["vertical_and_slash", 1000, 6096, 0.9500700235366821], "3": ["vertical_and_slash", 1000, 6096, 0.9642037153244019], "4": ["vertical_and_slash", 1000, 6096, 0.94118332862854], "5": ["vertical_and_slash", 1000, 6096, 0.9665160179138184], "6": ["vertical_and_slash", 5548, 2148, 0.8518802523612976], "7": ["vertical_and_slash", 1000, 6096, 0.9963601231575012], "8": ["vertical_and_slash", 30, 800, 0.9910691380500793], "9": ["vertical_and_slash", 100, 800, 0.9765625], "10": ["vertical_and_slash", 30, 800, 0.9925505518913269], "11": ["vertical_and_slash", 9192, 14288, 0.6851534247398376], "12": ["vertical_and_slash", 1000, 6096, 0.9318822622299194], "13": ["vertical_and_slash", 30, 800, 0.9992240071296692], "14": ["vertical_and_slash", 7244, 12240, 0.9547796845436096], "15": ["vertical_and_slash", 1000, 6096, 0.9696962237358093], "16": ["vertical_and_slash", 1000, 6096, 0.9316620826721191], "17": ["vertical_and_slash", 1000, 6096, 0.9258195161819458], "18": ["vertical_and_slash", 3048, 8144, 0.8255325555801392], "19": ["vertical_and_slash", 3048, 8144, 0.8840956687927246], "20": ["vertical_and_slash", 1000, 6096, 0.9662702083587646], "21": ["vertical_and_slash", 1000, 6096, 0.9504241943359375], "22": ["vertical_and_slash", 1000, 6096, 0.9732783436775208], "23": ["vertical_and_slash", 1000, 6096, 0.9553895592689514], "24": ["vertical_and_slash", 1000, 6096, 0.9737565517425537], "25": ["vertical_and_slash", 1000, 6096, 0.9609088897705078], "26": ["vertical_and_slash", 1000, 6096, 0.9755902886390686], "27": ["vertical_and_slash", 1000, 6096, 0.9681479930877686]}, {"0": ["vertical_and_slash", 19532, 24528, 0.6649974584579468], "1": ["vertical_and_slash", 5096, 10192, 0.7836548686027527], "2": ["vertical_and_slash", 11340, 16336, 0.937842607498169], "3": ["vertical_and_slash", 9192, 14288, 0.47755563259124756], "4": ["vertical_and_slash", 11340, 16336, 0.9645791053771973], "5": ["vertical_and_slash", 11340, 16336, 1.0000194311141968], "6": ["vertical_and_slash", 13388, 18384, 0.8536562919616699], "7": ["vertical_and_slash", 13840, 10340, 0.9791404604911804], "8": ["vertical_and_slash", 22032, 18532, 0.680722177028656], "9": ["vertical_and_slash", 3048, 8144, 0.8969497084617615], "10": ["vertical_and_slash", 3048, 8144, 0.8135265707969666], "11": ["vertical_and_slash", 11340, 16336, 0.9405107498168945], "12": ["vertical_and_slash", 9192, 14288, 0.6148377656936646], "13": ["vertical_and_slash", 8392, 8992, 0.87890625], "14": ["vertical_and_slash", 7244, 12240, 0.9699380993843079], "15": ["vertical_and_slash", 7244, 12240, 0.9827472567558289], "16": ["vertical_and_slash", 1000, 6096, 0.9853057861328125], "17": ["vertical_and_slash", 1000, 6096, 0.9891751408576965], "18": ["vertical_and_slash", 1000, 6096, 0.9121230244636536], "19": ["vertical_and_slash", 7244, 12240, 0.935899555683136], "20": ["vertical_and_slash", 3048, 8144, 0.8554543852806091], "21": ["vertical_and_slash", 3048, 8144, 0.8143891096115112], "22": ["vertical_and_slash", 5096, 10192, 0.7405089139938354], "23": ["vertical_and_slash", 9292, 14288, 0.8643857836723328], "24": ["vertical_and_slash", 1000, 6096, 0.916479229927063], "25": ["vertical_and_slash", 1000, 6096, 0.9841439127922058], "26": ["vertical_and_slash", 1000, 6096, 0.9474544525146484], "27": ["vertical_and_slash", 7144, 12240, 0.8183215260505676]}, {"0": ["vertical_and_slash", 1000, 6096, 0.9937507510185242], "1": ["vertical_and_slash", 9292, 14288, 0.8320319652557373], "2": ["vertical_and_slash", 1000, 6096, 0.9843978881835938], "3": ["vertical_and_slash", 1000, 6096, 0.99434894323349], "4": ["vertical_and_slash", 3048, 8144, 0.8445190787315369], "5": ["vertical_and_slash", 1000, 6096, 0.9977291226387024], "6": ["vertical_and_slash", 1000, 6096, 0.9971616864204407], "7": ["vertical_and_slash", 6344, 6944, 0.92578125], "8": ["vertical_and_slash", 30, 800, 0.9918319582939148], "9": ["vertical_and_slash", 30, 800, 0.9931505918502808], "10": ["vertical_and_slash", 500, 700, 0.9629374146461487], "11": ["vertical_and_slash", 30, 800, 0.9984971880912781], "12": ["vertical_and_slash", 500, 700, 0.9874173402786255], "13": ["vertical_and_slash", 30, 800, 0.9872177839279175], "14": ["vertical_and_slash", 8292, 8992, 0.6796875], "15": ["vertical_and_slash", 7244, 12240, 0.9822348356246948], "16": ["vertical_and_slash", 4196, 4896, 0.984375], "17": ["vertical_and_slash", 1000, 6096, 0.9580352902412415], "18": ["vertical_and_slash", 13388, 18384, 0.8853479027748108], "19": ["vertical_and_slash", 6344, 6944, 0.953125], "20": ["vertical_and_slash", 13388, 18384, 0.8276364803314209], "21": ["vertical_and_slash", 500, 700, 0.9783065915107727], "22": ["vertical_and_slash", 1000, 6096, 0.9267724752426147], "23": ["vertical_and_slash", 3048, 8144, 0.8524566888809204], "24": ["vertical_and_slash", 9192, 14288, 0.663494348526001], "25": ["vertical_and_slash", 3048, 8144, 0.8196530342102051], "26": ["vertical_and_slash", 3048, 8144, 0.8980101346969604], "27": ["vertical_and_slash", 4596, 4796, 0.9499697089195251]}, {"0": ["vertical_and_slash", 11340, 16336, 0.9402349591255188], "1": ["vertical_and_slash", 10440, 11040, 0.9453125], "2": ["vertical_and_slash", 3048, 8144, 0.8996480703353882], "3": ["vertical_and_slash", 1000, 6096, 0.9078497886657715], "4": ["vertical_and_slash", 11340, 16336, 0.958014965057373], "5": ["vertical_and_slash", 5096, 10192, 0.796527087688446], "6": ["vertical_and_slash", 7244, 12240, 0.9736052751541138], "7": ["vertical_and_slash", 11340, 16336, 0.9348691701889038], "8": ["vertical_and_slash", 1000, 6096, 0.9979962706565857], "9": ["vertical_and_slash", 15436, 20432, 0.7974388003349304], "10": ["vertical_and_slash", 1000, 6096, 0.9191967248916626], "11": ["vertical_and_slash", 5096, 10192, 0.7944039702415466], "12": ["vertical_and_slash", 11340, 16336, 0.7239288091659546], "13": ["vertical_and_slash", 15436, 20432, 0.6403535008430481], "14": ["vertical_and_slash", 13388, 18384, 0.8503284454345703], "15": ["vertical_and_slash", 9292, 14288, 0.8853875994682312], "16": ["vertical_and_slash", 11340, 16336, 0.9519768953323364], "17": ["vertical_and_slash", 13840, 10340, 0.9313420653343201], "18": ["vertical_and_slash", 7244, 12240, 0.9588450789451599], "19": ["vertical_and_slash", 5096, 10192, 0.7538775205612183], "20": ["vertical_and_slash", 17936, 14436, 0.7871273756027222], "21": ["vertical_and_slash", 4596, 4796, 0.983285665512085], "22": ["vertical_and_slash", 7244, 12240, 0.9609666466712952], "23": ["vertical_and_slash", 1000, 6096, 0.974759042263031], "24": ["vertical_and_slash", 1000, 6096, 0.9698718190193176], "25": ["vertical_and_slash", 1000, 6096, 0.9898294806480408], "26": ["vertical_and_slash", 1000, 6096, 0.9256659150123596], "27": ["vertical_and_slash", 7244, 12240, 0.9485111832618713]}, {"0": ["vertical_and_slash", 9292, 14288, 0.8835593461990356], "1": ["vertical_and_slash", 100, 800, 0.98046875], "2": ["vertical_and_slash", 1000, 6096, 0.9635430574417114], "3": ["vertical_and_slash", 1000, 6096, 0.9747232794761658], "4": ["vertical_and_slash", 1000, 6096, 0.9661608934402466], "5": ["vertical_and_slash", 6344, 6944, 0.953125], "6": ["vertical_and_slash", 1000, 6096, 0.9636247754096985], "7": ["vertical_and_slash", 13388, 18384, 0.8315137028694153], "8": ["vertical_and_slash", 13388, 18384, 0.8079983592033386], "9": ["vertical_and_slash", 3048, 8144, 0.8652046322822571], "10": ["vertical_and_slash", 5096, 10192, 0.7678346633911133], "11": ["vertical_and_slash", 4196, 4896, 0.9921875], "12": ["vertical_and_slash", 1000, 6096, 0.9621175527572632], "13": ["vertical_and_slash", 1000, 6096, 0.9095380902290344], "14": ["vertical_and_slash", 15888, 12388, 0.8682130575180054], "15": ["vertical_and_slash", 5096, 10192, 0.7981836199760437], "16": ["vertical_and_slash", 4196, 4896, 0.94921875], "17": ["vertical_and_slash", 3048, 8144, 0.8423125743865967], "18": ["vertical_and_slash", 11340, 16336, 0.9536053538322449], "19": ["vertical_and_slash", 19532, 24528, 0.6965113878250122], "20": ["vertical_and_slash", 11340, 16336, 0.765586256980896], "21": ["vertical_and_slash", 1000, 6096, 0.9789958000183105], "22": ["vertical_and_slash", 1000, 6096, 0.941737174987793], "23": ["vertical_and_slash", 1000, 6096, 0.9770992398262024], "24": ["vertical_and_slash", 13388, 18384, 0.8724013566970825], "25": ["vertical_and_slash", 1000, 6096, 0.9776409268379211], "26": ["vertical_and_slash", 3048, 8144, 0.8984430432319641], "27": ["vertical_and_slash", 1000, 6096, 0.9618942141532898]}, {"0": ["vertical_and_slash", 30, 800, 0.9397658705711365], "1": ["vertical_and_slash", 1000, 6096, 0.9678259491920471], "2": ["vertical_and_slash", 500, 700, 0.9607874751091003], "3": ["vertical_and_slash", 30, 800, 0.9745917320251465], "4": ["vertical_and_slash", 30, 800, 0.9659082889556885], "5": ["vertical_and_slash", 6274, 6944, 0.947303056716919], "6": ["vertical_and_slash", 30, 800, 0.9196206331253052], "7": ["vertical_and_slash", 30, 800, 1.000028133392334], "8": ["vertical_and_slash", 100, 800, 0.96484375], "9": ["vertical_and_slash", 500, 700, 0.9743742346763611], "10": ["vertical_and_slash", 11340, 16336, 0.7517896294593811], "11": ["vertical_and_slash", 1000, 6096, 0.9228399991989136], "12": ["vertical_and_slash", 1000, 6096, 0.9122115969657898], "13": ["vertical_and_slash", 4196, 4896, 0.96875], "14": ["vertical_and_slash", 1000, 6096, 0.9384160041809082], "15": ["vertical_and_slash", 1000, 6096, 0.955692708492279], "16": ["vertical_and_slash", 15436, 20432, 0.7190804481506348], "17": ["vertical_and_slash", 500, 700, 0.9635783433914185], "18": ["vertical_and_slash", 1000, 6096, 0.9114483594894409], "19": ["vertical_and_slash", 3048, 8144, 0.8773453831672668], "20": ["vertical_and_slash", 1000, 6096, 0.9607734084129333], "21": ["vertical_and_slash", 500, 700, 0.9821328520774841], "22": ["vertical_and_slash", 7244, 12240, 0.9674672484397888], "23": ["vertical_and_slash", 5096, 10192, 0.7276496887207031], "24": ["vertical_and_slash", 1000, 6096, 0.97364741563797], "25": ["vertical_and_slash", 1000, 6096, 0.9671147465705872], "26": ["vertical_and_slash", 1000, 6096, 0.9272445440292358], "27": ["vertical_and_slash", 4196, 4896, 0.96875]}, {"0": ["vertical_and_slash", 4126, 4896, 1.0000450611114502], "1": ["vertical_and_slash", 3048, 8144, 0.8740337491035461], "2": ["vertical_and_slash", 9744, 6244, 0.9127500057220459], "3": ["vertical_and_slash", 1000, 6096, 0.9917009472846985], "4": ["vertical_and_slash", 3048, 8144, 0.8467518091201782], "5": ["vertical_and_slash", 5096, 10192, 0.9295762181282043], "6": ["vertical_and_slash", 13388, 18384, 0.8826475143432617], "7": ["vertical_and_slash", 30, 800, 0.9837709069252014], "8": ["vertical_and_slash", 30, 800, 0.9920428991317749], "9": ["vertical_and_slash", 30, 800, 0.9717660546302795], "10": ["vertical_and_slash", 6274, 6944, 0.9047616124153137], "11": ["vertical_and_slash", 5096, 10192, 0.7814681529998779], "12": ["vertical_and_slash", 500, 700, 0.9585937857627869], "13": ["vertical_and_slash", 30, 800, 0.9965728521347046], "14": ["vertical_and_slash", 3048, 8144, 0.831355631351471], "15": ["vertical_and_slash", 11340, 16336, 0.9474310874938965], "16": ["vertical_and_slash", 11340, 16336, 0.9529578685760498], "17": ["vertical_and_slash", 11340, 16336, 0.9589705467224121], "18": ["vertical_and_slash", 7244, 12240, 0.9103161692619324], "19": ["vertical_and_slash", 11340, 16336, 0.9817567467689514], "20": ["vertical_and_slash", 11340, 16336, 0.9756876230239868], "21": ["vertical_and_slash", 7244, 12240, 0.9710824489593506], "22": ["vertical_and_slash", 11340, 16336, 0.9776445627212524], "23": ["vertical_and_slash", 11340, 16336, 0.9071230888366699], "24": ["vertical_and_slash", 1000, 6096, 0.9814715385437012], "25": ["vertical_and_slash", 1000, 6096, 0.9802733659744263], "26": ["vertical_and_slash", 1000, 6096, 0.9048901200294495], "27": ["vertical_and_slash", 7244, 12240, 0.948653519153595]}, {"0": ["vertical_and_slash", 30, 800, 0.9610792398452759], "1": ["vertical_and_slash", 30, 800, 0.9808367490768433], "2": ["vertical_and_slash", 3048, 8144, 0.8854068517684937], "3": ["vertical_and_slash", 6344, 6944, 0.95703125], "4": ["vertical_and_slash", 100, 800, 0.95703125], "5": ["vertical_and_slash", 3048, 8144, 0.8755441904067993], "6": ["vertical_and_slash", 100, 800, 0.92578125], "7": ["vertical_and_slash", 100, 800, 0.921875], "8": ["vertical_and_slash", 9192, 14288, 0.6833025813102722], "9": ["vertical_and_slash", 5096, 10192, 0.7133266925811768], "10": ["vertical_and_slash", 1000, 6096, 0.9820693731307983], "11": ["vertical_and_slash", 1000, 6096, 0.9388710856437683], "12": ["vertical_and_slash", 3048, 8144, 0.8800644278526306], "13": ["vertical_and_slash", 4196, 4896, 0.98828125], "14": ["vertical_and_slash", 1000, 6096, 0.9447485208511353], "15": ["vertical_and_slash", 1000, 6096, 0.9788766503334045], "16": ["vertical_and_slash", 1000, 6096, 0.9929926991462708], "17": ["vertical_and_slash", 7144, 12240, 0.8362777233123779], "18": ["vertical_and_slash", 3048, 8144, 0.8786005973815918], "19": ["vertical_and_slash", 7244, 12240, 0.9762356281280518], "20": ["vertical_and_slash", 1000, 6096, 0.9894784688949585], "21": ["vertical_and_slash", 1000, 6096, 0.9231916666030884], "22": ["vertical_and_slash", 1000, 6096, 0.9846755266189575], "23": ["vertical_and_slash", 1000, 6096, 0.9899857640266418], "24": ["vertical_and_slash", 1000, 6096, 0.9508718848228455], "25": ["vertical_and_slash", 1000, 6096, 0.9378311038017273], "26": ["vertical_and_slash", 13388, 18384, 0.895405650138855], "27": ["vertical_and_slash", 1000, 6096, 0.969482421875]}, {"0": ["vertical_and_slash", 1000, 6096, 0.9123888611793518], "1": ["vertical_and_slash", 7244, 12240, 0.9673317074775696], "2": ["vertical_and_slash", 7244, 12240, 0.9695746302604675], "3": ["vertical_and_slash", 1000, 6096, 0.9790319204330444], "4": ["vertical_and_slash", 11340, 16336, 0.9740945100784302], "5": ["vertical_and_slash", 11340, 16336, 0.9472271800041199], "6": ["vertical_and_slash", 1000, 6096, 0.9376558661460876], "7": ["vertical_and_slash", 1000, 6096, 0.9759783148765564], "8": ["vertical_and_slash", 100, 800, 0.984375], "9": ["vertical_and_slash", 100, 800, 0.9921875], "10": ["vertical_and_slash", 3500, 100, 0.9970564246177673], "11": ["vertical_and_slash", 9744, 6244, 0.9889124035835266], "12": ["vertical_and_slash", 100, 800, 0.9765625], "13": ["vertical_and_slash", 11340, 16336, 0.9012805819511414], "14": ["vertical_and_slash", 1000, 6096, 0.939403235912323], "15": ["vertical_and_slash", 30, 800, 0.9993904829025269], "16": ["vertical_and_slash", 7244, 12240, 0.9905298352241516], "17": ["vertical_and_slash", 5096, 10192, 0.7282559275627136], "18": ["vertical_and_slash", 1000, 6096, 0.9881381988525391], "19": ["vertical_and_slash", 1000, 6096, 0.9709164500236511], "20": ["vertical_and_slash", 1000, 6096, 0.9852944016456604], "21": ["vertical_and_slash", 1000, 6096, 0.9899225234985352], "22": ["vertical_and_slash", 11340, 16336, 0.9815132021903992], "23": ["vertical_and_slash", 1000, 6096, 0.9717292785644531], "24": ["vertical_and_slash", 7244, 12240, 0.9921572208404541], "25": ["vertical_and_slash", 11340, 16336, 0.9181587100028992], "26": ["vertical_and_slash", 1000, 6096, 0.9939910769462585], "27": ["vertical_and_slash", 7244, 12240, 0.9894195199012756]}, {"0": ["vertical_and_slash", 5096, 10192, 0.9363827705383301], "1": ["vertical_and_slash", 11340, 16336, 0.9809366464614868], "2": ["vertical_and_slash", 1000, 6096, 0.9730969071388245], "3": ["vertical_and_slash", 13388, 18384, 0.8695287704467773], "4": ["vertical_and_slash", 11340, 16336, 0.9124827980995178], "5": ["vertical_and_slash", 11340, 16336, 0.7494962215423584], "6": ["vertical_and_slash", 13388, 18384, 0.8308557868003845], "7": ["vertical_and_slash", 11340, 16336, 0.957757830619812], "8": ["vertical_and_slash", 1000, 6096, 0.9518009424209595], "9": ["vertical_and_slash", 1000, 6096, 0.9622867703437805], "10": ["vertical_and_slash", 1000, 6096, 0.979856550693512], "11": ["vertical_and_slash", 1000, 6096, 0.9948192834854126], "12": ["vertical_and_slash", 7244, 12240, 0.9975912570953369], "13": ["vertical_and_slash", 7244, 12240, 0.9956138730049133], "14": ["vertical_and_slash", 1000, 6096, 0.9788697957992554], "15": ["vertical_and_slash", 6344, 6944, 1.0], "16": ["vertical_and_slash", 7244, 12240, 0.9795857667922974], "17": ["vertical_and_slash", 11340, 16336, 0.9933436512947083], "18": ["vertical_and_slash", 1000, 6096, 0.9908619523048401], "19": ["vertical_and_slash", 7244, 12240, 0.9774454832077026], "20": ["vertical_and_slash", 1000, 6096, 0.9626350402832031], "21": ["vertical_and_slash", 9292, 14288, 0.823958158493042], "22": ["vertical_and_slash", 11340, 16336, 0.9598273634910583], "23": ["vertical_and_slash", 1000, 6096, 0.9597938656806946], "24": ["vertical_and_slash", 11340, 16336, 0.9751371145248413], "25": ["vertical_and_slash", 7244, 12240, 0.9830336570739746], "26": ["vertical_and_slash", 7244, 12240, 0.9900059103965759], "27": ["vertical_and_slash", 11340, 16336, 0.9930396676063538]}, {"0": ["vertical_and_slash", 4196, 4896, 1.0], "1": ["vertical_and_slash", 1000, 6096, 0.9948315024375916], "2": ["vertical_and_slash", 11340, 16336, 0.9978001117706299], "3": ["vertical_and_slash", 1000, 6096, 0.9911418557167053], "4": ["vertical_and_slash", 1000, 6096, 0.9911423921585083], "5": ["vertical_and_slash", 1000, 6096, 0.9994310140609741], "6": ["vertical_and_slash", 1000, 6096, 0.9933038949966431], "7": ["vertical_and_slash", 1000, 6096, 0.9815270304679871], "8": ["vertical_and_slash", 1000, 6096, 0.9188183546066284], "9": ["vertical_and_slash", 7244, 12240, 0.949191689491272], "10": ["vertical_and_slash", 10440, 11040, 0.99609375], "11": ["vertical_and_slash", 11340, 16336, 0.9600533246994019], "12": ["vertical_and_slash", 11340, 16336, 0.9197850823402405], "13": ["vertical_and_slash", 11340, 16336, 0.9567902684211731], "14": ["vertical_and_slash", 7244, 12240, 0.9746363759040833], "15": ["vertical_and_slash", 7244, 12240, 0.9832977652549744], "16": ["vertical_and_slash", 1000, 6096, 0.9724404811859131], "17": ["vertical_and_slash", 1000, 6096, 0.9983720183372498], "18": ["vertical_and_slash", 7244, 12240, 0.9574311375617981], "19": ["vertical_and_slash", 11340, 16336, 0.9586833715438843], "20": ["vertical_and_slash", 7244, 12240, 0.9386707544326782], "21": ["vertical_and_slash", 7596, 4196, 0.9758472442626953], "22": ["vertical_and_slash", 1000, 6096, 0.9404810070991516], "23": ["vertical_and_slash", 9744, 6244, 0.9920862913131714], "24": ["vertical_and_slash", 1000, 6096, 0.9855319261550903], "25": ["vertical_and_slash", 1000, 6096, 0.9087734818458557], "26": ["vertical_and_slash", 1000, 6096, 0.9804069995880127], "27": ["vertical_and_slash", 1000, 6096, 0.9705759882926941]}, {"0": ["vertical_and_slash", 1000, 6096, 0.9644368290901184], "1": ["vertical_and_slash", 30, 800, 0.9986089468002319], "2": ["vertical_and_slash", 1000, 6096, 0.9745039343833923], "3": ["vertical_and_slash", 1000, 6096, 0.9963008761405945], "4": ["vertical_and_slash", 7596, 4196, 0.9790559411048889], "5": ["vertical_and_slash", 1000, 6096, 0.9741643071174622], "6": ["vertical_and_slash", 1000, 6096, 0.9873979687690735], "7": ["vertical_and_slash", 7596, 4196, 0.9851264357566833], "8": ["vertical_and_slash", 9744, 6244, 0.9803036451339722], "9": ["vertical_and_slash", 1000, 6096, 0.9777622818946838], "10": ["vertical_and_slash", 1000, 6096, 0.9912298321723938], "11": ["vertical_and_slash", 1000, 6096, 0.962704598903656], "12": ["vertical_and_slash", 7596, 4196, 0.9825693368911743], "13": ["vertical_and_slash", 5096, 10192, 0.7750788331031799], "14": ["vertical_and_slash", 1000, 6096, 0.9972183704376221], "15": ["vertical_and_slash", 1000, 6096, 0.9945282340049744], "16": ["vertical_and_slash", 9744, 6244, 0.9972888231277466], "17": ["vertical_and_slash", 1000, 6096, 0.9963790774345398], "18": ["vertical_and_slash", 1000, 6096, 0.9650697112083435], "19": ["vertical_and_slash", 7596, 4196, 0.9987449645996094], "20": ["vertical_and_slash", 5096, 10192, 0.9960426092147827], "21": ["vertical_and_slash", 7244, 12240, 0.9878199100494385], "22": ["vertical_and_slash", 1000, 6096, 0.9935839176177979], "23": ["vertical_and_slash", 7244, 12240, 0.9975365400314331], "24": ["vertical_and_slash", 1000, 6096, 0.968522310256958], "25": ["vertical_and_slash", 7244, 12240, 0.9933074116706848], "26": ["vertical_and_slash", 7244, 12240, 0.9941068887710571], "27": ["vertical_and_slash", 1000, 6096, 0.9910241365432739]}, {"0": ["vertical_and_slash", 1000, 6096, 0.9869405627250671], "1": ["vertical_and_slash", 9744, 6244, 0.9560899138450623], "2": ["vertical_and_slash", 9744, 6244, 0.9869239330291748], "3": ["vertical_and_slash", 3500, 100, 0.9870764017105103], "4": ["vertical_and_slash", 1000, 6096, 0.9273780584335327], "5": ["vertical_and_slash", 7596, 4196, 0.9860992431640625], "6": ["vertical_and_slash", 9744, 6244, 0.9839791059494019], "7": ["vertical_and_slash", 1000, 6096, 0.9863566756248474], "8": ["vertical_and_slash", 7244, 12240, 0.9857996106147766], "9": ["vertical_and_slash", 1000, 6096, 0.9804321527481079], "10": ["vertical_and_slash", 7244, 12240, 0.9676904082298279], "11": ["vertical_and_slash", 7596, 4196, 0.9908879399299622], "12": ["vertical_and_slash", 7244, 12240, 0.9486064314842224], "13": ["vertical_and_slash", 7244, 12240, 0.9652743339538574], "14": ["vertical_and_slash", 30, 800, 0.9901286363601685], "15": ["vertical_and_slash", 5096, 10192, 0.7799577116966248], "16": ["vertical_and_slash", 3048, 8144, 0.8891290426254272], "17": ["vertical_and_slash", 7596, 4196, 0.9794012308120728], "18": ["vertical_and_slash", 30, 800, 0.951410710811615], "19": ["vertical_and_slash", 30, 800, 0.9806299209594727], "20": ["vertical_and_slash", 3048, 8144, 0.8039274215698242], "21": ["vertical_and_slash", 1000, 6096, 0.9898089170455933], "22": ["vertical_and_slash", 1000, 6096, 0.986038863658905], "23": ["vertical_and_slash", 7244, 12240, 0.9948249459266663], "24": ["vertical_and_slash", 1000, 6096, 0.9429303407669067], "25": ["vertical_and_slash", 11340, 16336, 0.978610098361969], "26": ["vertical_and_slash", 1000, 6096, 0.9908936023712158], "27": ["vertical_and_slash", 1000, 6096, 0.9936961531639099]}, {"0": ["vertical_and_slash", 7244, 12240, 0.9856012463569641], "1": ["vertical_and_slash", 1000, 6096, 0.9747336506843567], "2": ["vertical_and_slash", 7596, 4196, 0.9834318161010742], "3": ["vertical_and_slash", 1000, 6096, 0.9862018823623657], "4": ["vertical_and_slash", 1000, 6096, 0.9937860369682312], "5": ["vertical_and_slash", 1000, 6096, 0.9886960387229919], "6": ["vertical_and_slash", 7596, 4196, 0.9887931942939758], "7": ["vertical_and_slash", 3048, 8144, 0.8064029216766357], "8": ["vertical_and_slash", 500, 700, 0.9941562414169312], "9": ["vertical_and_slash", 1000, 6096, 0.9482210278511047], "10": ["vertical_and_slash", 1000, 6096, 0.933103084564209], "11": ["vertical_and_slash", 500, 700, 0.9964345097541809], "12": ["vertical_and_slash", 1000, 6096, 0.9534383416175842], "13": ["vertical_and_slash", 1000, 6096, 0.9467671513557434], "14": ["vertical_and_slash", 7244, 12240, 0.9333358407020569], "15": ["vertical_and_slash", 11340, 16336, 0.9803599715232849], "16": ["vertical_and_slash", 1000, 6096, 0.9709724187850952], "17": ["vertical_and_slash", 1000, 6096, 0.9853048324584961], "18": ["vertical_and_slash", 1000, 6096, 0.9850086569786072], "19": ["vertical_and_slash", 7244, 12240, 0.9774631261825562], "20": ["vertical_and_slash", 1000, 6096, 0.9743291139602661], "21": ["vertical_and_slash", 3048, 8144, 0.8560544848442078], "22": ["vertical_and_slash", 3048, 8144, 0.8773953914642334], "23": ["vertical_and_slash", 1000, 6096, 0.9476152062416077], "24": ["vertical_and_slash", 1000, 6096, 0.9344698786735535], "25": ["vertical_and_slash", 11340, 16336, 0.9541553854942322], "26": ["vertical_and_slash", 1000, 6096, 0.9854223728179932], "27": ["vertical_and_slash", 1000, 6096, 0.9799894690513611]}, {"0": ["vertical_and_slash", 1000, 6096, 0.9208858609199524], "1": ["vertical_and_slash", 1000, 6096, 0.9677489995956421], "2": ["vertical_and_slash", 1000, 6096, 0.9397966861724854], "3": ["vertical_and_slash", 1000, 6096, 0.9308995008468628], "4": ["vertical_and_slash", 1000, 6096, 0.9643043279647827], "5": ["vertical_and_slash", 1000, 6096, 0.9657536149024963], "6": ["vertical_and_slash", 3048, 8144, 0.8598212599754333], "7": ["vertical_and_slash", 10370, 11040, 0.9467752575874329], "8": ["vertical_and_slash", 1000, 6096, 0.9844503998756409], "9": ["vertical_and_slash", 1000, 6096, 0.9382181763648987], "10": ["vertical_and_slash", 10370, 11040, 0.972872257232666], "11": ["vertical_and_slash", 10370, 11040, 0.9927771687507629], "12": ["vertical_and_slash", 10370, 11040, 0.9328325390815735], "13": ["vertical_and_slash", 10370, 11040, 0.9946746826171875], "14": ["vertical_and_slash", 1000, 6096, 0.9508710503578186], "15": ["vertical_and_slash", 1000, 6096, 0.9318876266479492], "16": ["vertical_and_slash", 1000, 6096, 0.9340384602546692], "17": ["vertical_and_slash", 7244, 12240, 0.9182395339012146], "18": ["vertical_and_slash", 1000, 6096, 0.9607776403427124], "19": ["vertical_and_slash", 1000, 6096, 0.9068328738212585], "20": ["vertical_and_slash", 1000, 6096, 0.9481955170631409], "21": ["vertical_and_slash", 1000, 6096, 0.9357800483703613], "22": ["vertical_and_slash", 3048, 8144, 0.8631651997566223], "23": ["vertical_and_slash", 7244, 12240, 0.9462882876396179], "24": ["vertical_and_slash", 3048, 8144, 0.808625340461731], "25": ["vertical_and_slash", 1000, 6096, 0.9675469398498535], "26": ["vertical_and_slash", 1000, 6096, 0.9622009992599487], "27": ["vertical_and_slash", 3048, 8144, 0.8711419701576233]}]

tokenizer_config.json

@@ -198,7 +198,7 @@
   "clean_up_tokenization_spaces": false,
   "eos_token": "<|im_end|>",
   "errors": "replace",
-  "model_max_length": 131072,
+  "model_max_length": 1010000,
   "pad_token": "<|endoftext|>",
   "split_special_tokens": false,
   "tokenizer_class": "Qwen2Tokenizer",