diff --git "a/logs.txt" "b/logs.txt" new file mode 100644--- /dev/null +++ "b/logs.txt" @@ -0,0 +1,741 @@ +/opt/conda/envs/py310/bin/python -m mlc_llm gen_config /models/Qwen2-0.5B-Instruct --quantization q4f16_1 --conv-template chatml --output /models/mlc-delivery/hf/mlc-ai/Qwen2-0.5B-Instruct-q4f16_1-MLC +[2024-06-06 23:37:44] INFO auto_config.py:116: Found model configuration: /models/Qwen2-0.5B-Instruct/config.json +[2024-06-06 23:37:44] INFO auto_config.py:154: Found model type: qwen2. Use `--model-type` to override. +[2024-06-06 23:37:44] INFO qwen2_model.py:49: context_window_size not found in config.json. Falling back to max_position_embeddings (32768) +[2024-06-06 23:37:44] INFO qwen2_model.py:66: prefill_chunk_size defaults to 2048 +[2024-06-06 23:37:44] INFO config.py:107: Overriding max_batch_size from 1 to 80 +[2024-06-06 23:37:44] INFO gen_config.py:143: [generation_config.json] Setting bos_token_id: 151643 +[2024-06-06 23:37:44] INFO gen_config.py:143: [generation_config.json] Setting pad_token_id: 151643 +[2024-06-06 23:37:44] INFO gen_config.py:143: [generation_config.json] Setting eos_token_id: [151645, 151643] +[2024-06-06 23:37:44] INFO gen_config.py:143: [generation_config.json] Setting repetition_penalty: 1.1 +[2024-06-06 23:37:44] INFO gen_config.py:143: [generation_config.json] Setting temperature: 0.7 +[2024-06-06 23:37:44] INFO gen_config.py:143: [generation_config.json] Setting top_p: 0.8 +[2024-06-06 23:37:44] INFO gen_config.py:157: Not found tokenizer config: /models/Qwen2-0.5B-Instruct/tokenizer.model +[2024-06-06 23:37:44] INFO gen_config.py:155: Found tokenizer config: /models/Qwen2-0.5B-Instruct/tokenizer.json. Copying to /models/mlc-delivery/hf/mlc-ai/Qwen2-0.5B-Instruct-q4f16_1-MLC/tokenizer.json +[2024-06-06 23:37:44] INFO gen_config.py:155: Found tokenizer config: /models/Qwen2-0.5B-Instruct/vocab.json. Copying to /models/mlc-delivery/hf/mlc-ai/Qwen2-0.5B-Instruct-q4f16_1-MLC/vocab.json +[2024-06-06 23:37:44] INFO gen_config.py:155: Found tokenizer config: /models/Qwen2-0.5B-Instruct/merges.txt. Copying to /models/mlc-delivery/hf/mlc-ai/Qwen2-0.5B-Instruct-q4f16_1-MLC/merges.txt +[2024-06-06 23:37:44] INFO gen_config.py:157: Not found tokenizer config: /models/Qwen2-0.5B-Instruct/added_tokens.json +[2024-06-06 23:37:44] INFO gen_config.py:155: Found tokenizer config: /models/Qwen2-0.5B-Instruct/tokenizer_config.json. Copying to /models/mlc-delivery/hf/mlc-ai/Qwen2-0.5B-Instruct-q4f16_1-MLC/tokenizer_config.json +[2024-06-06 23:37:44] INFO gen_config.py:216: Detected tokenizer info: {'token_postproc_method': 'byte_level', 'prepend_space_in_encode': False, 'strip_space_in_decode': False} +[2024-06-06 23:37:44] INFO gen_config.py:32: [System default] Setting presence_penalty: 0.0 +[2024-06-06 23:37:44] INFO gen_config.py:32: [System default] Setting frequency_penalty: 0.0 +[2024-06-06 23:37:44] INFO gen_config.py:223: Dumping configuration file to: /models/mlc-delivery/hf/mlc-ai/Qwen2-0.5B-Instruct-q4f16_1-MLC/mlc-chat-config.json +/opt/conda/envs/py310/bin/python -m mlc_llm convert_weight /models/Qwen2-0.5B-Instruct --quantization q4f16_1 --output /models/mlc-delivery/hf/mlc-ai/Qwen2-0.5B-Instruct-q4f16_1-MLC +[2024-06-06 23:37:45] INFO auto_config.py:116: Found model configuration: /models/Qwen2-0.5B-Instruct/config.json +[2024-06-06 23:37:47] INFO auto_device.py:79: Found device: cuda:0 +[2024-06-06 23:37:48] INFO auto_device.py:88: Not found device: rocm:0 +[2024-06-06 23:37:50] INFO auto_device.py:88: Not found device: metal:0 +[2024-06-06 23:37:52] INFO auto_device.py:79: Found device: vulkan:0 +[2024-06-06 23:37:52] INFO auto_device.py:79: Found device: vulkan:1 +[2024-06-06 23:37:52] INFO auto_device.py:79: Found device: vulkan:2 +[2024-06-06 23:37:52] INFO auto_device.py:79: Found device: vulkan:3 +[2024-06-06 23:37:53] INFO auto_device.py:88: Not found device: opencl:0 +[2024-06-06 23:37:53] INFO auto_device.py:35: Using device: cuda:0 +[2024-06-06 23:37:53] INFO auto_weight.py:71: Finding weights in: /models/Qwen2-0.5B-Instruct +[2024-06-06 23:37:53] INFO auto_weight.py:137: Not found Huggingface PyTorch +[2024-06-06 23:37:54] INFO auto_weight.py:161: Found source weight format: huggingface-safetensor. Source configuration: /models/Qwen2-0.5B-Instruct/model.safetensors.index.json +[2024-06-06 23:37:54] INFO auto_weight.py:107: Using source weight configuration: /models/Qwen2-0.5B-Instruct/model.safetensors.index.json. Use `--source` to override. +[2024-06-06 23:37:54] INFO auto_weight.py:111: Using source weight format: huggingface-safetensor. Use `--source-format` to override. +[2024-06-06 23:37:54] INFO auto_config.py:154: Found model type: qwen2. Use `--model-type` to override. +[2024-06-06 23:37:54] INFO qwen2_model.py:49: context_window_size not found in config.json. Falling back to max_position_embeddings (32768) +[2024-06-06 23:37:54] INFO qwen2_model.py:66: prefill_chunk_size defaults to 2048 +Traceback (most recent call last): + File "/opt/conda/envs/py310/lib/python3.10/runpy.py", line 196, in _run_module_as_main + return _run_code(code, main_globals, None, + File "/opt/conda/envs/py310/lib/python3.10/runpy.py", line 86, in _run_code + exec(code, run_globals) + File "/opt/conda/envs/py310/lib/python3.10/site-packages/mlc_llm/__main__.py", line 64, in + main() + File "/opt/conda/envs/py310/lib/python3.10/site-packages/mlc_llm/__main__.py", line 37, in main + cli.main(sys.argv[2:]) + File "/opt/conda/envs/py310/lib/python3.10/site-packages/mlc_llm/cli/convert_weight.py", line 88, in main + convert_weight( + File "/opt/conda/envs/py310/lib/python3.10/site-packages/mlc_llm/interface/convert_weight.py", line 181, in convert_weight + _convert_args(args) + File "/opt/conda/envs/py310/lib/python3.10/site-packages/mlc_llm/interface/convert_weight.py", line 145, in _convert_args + tvmjs.dump_ndarray_cache( + File "/opt/conda/envs/py310/lib/python3.10/site-packages/tvm/contrib/tvmjs.py", line 272, in dump_ndarray_cache + for k, origin_v in param_generator: + File "/opt/conda/envs/py310/lib/python3.10/site-packages/mlc_llm/interface/convert_weight.py", line 122, in _param_generator + loader = LOADER[args.source_format]( + File "/opt/conda/envs/py310/lib/python3.10/site-packages/mlc_llm/loader/huggingface_loader.py", line 99, in __init__ + check_parameter_usage(extern_param_map, set(self.torch_to_path.keys())) + File "/opt/conda/envs/py310/lib/python3.10/site-packages/mlc_llm/loader/utils.py", line 33, in check_parameter_usage + raise ValueError( +ValueError: The following extern parameters do not exist in the weight files: + lm_head.weight +Weight conversion with arguments: + --config /models/Qwen2-0.5B-Instruct/config.json + --quantization GroupQuantize(name='q4f16_1', kind='group-quant', group_size=32, quantize_dtype='int4', storage_dtype='uint32', model_dtype='float16', linear_weight_layout='NK', quantize_embedding=True, quantize_final_fc=True, num_elem_per_storage=8, num_storage_per_group=4, max_int_value=7) + --model-type qwen2 + --device cuda:0 + --source /models/Qwen2-0.5B-Instruct/model.safetensors.index.json + --source-format huggingface-safetensor + --output /models/mlc-delivery/hf/mlc-ai/Qwen2-0.5B-Instruct-q4f16_1-MLC +Start storing to cache /models/mlc-delivery/hf/mlc-ai/Qwen2-0.5B-Instruct-q4f16_1-MLC +/home/rickzhou/miniconda3/envs/mlc/bin/python -m mlc_llm gen_config /ssd2/models/Qwen2-0.5B-Instruct --quantization q4f16_1 --conv-template chatml --output /models/mlc-delivery/hf/mlc-ai/Qwen2-0.5B-Instruct-q4f16_1-MLC +[2024-06-06 22:33:19] INFO auto_config.py:116: Found model configuration: /ssd2/models/Qwen2-0.5B-Instruct/config.json +[2024-06-06 22:33:19] INFO auto_config.py:154: Found model type: qwen2. Use `--model-type` to override. +[2024-06-06 22:33:19] INFO qwen2_model.py:50: context_window_size not found in config.json. Falling back to max_position_embeddings (32768) +[2024-06-06 22:33:19] INFO qwen2_model.py:67: prefill_chunk_size defaults to 2048 +[2024-06-06 22:33:19] INFO config.py:107: Overriding max_batch_size from 1 to 80 +[2024-06-06 22:33:19] INFO gen_config.py:143: [generation_config.json] Setting bos_token_id: 151643 +[2024-06-06 22:33:19] INFO gen_config.py:143: [generation_config.json] Setting pad_token_id: 151643 +[2024-06-06 22:33:19] INFO gen_config.py:143: [generation_config.json] Setting eos_token_id: [151645, 151643] +[2024-06-06 22:33:19] INFO gen_config.py:143: [generation_config.json] Setting repetition_penalty: 1.1 +[2024-06-06 22:33:19] INFO gen_config.py:143: [generation_config.json] Setting temperature: 0.7 +[2024-06-06 22:33:19] INFO gen_config.py:143: [generation_config.json] Setting top_p: 0.8 +[2024-06-06 22:33:19] INFO gen_config.py:157: Not found tokenizer config: /ssd2/models/Qwen2-0.5B-Instruct/tokenizer.model +[2024-06-06 22:33:19] INFO gen_config.py:155: Found tokenizer config: /ssd2/models/Qwen2-0.5B-Instruct/tokenizer.json. Copying to /models/mlc-delivery/hf/mlc-ai/Qwen2-0.5B-Instruct-q4f16_1-MLC/tokenizer.json +[2024-06-06 22:33:19] INFO gen_config.py:155: Found tokenizer config: /ssd2/models/Qwen2-0.5B-Instruct/vocab.json. Copying to /models/mlc-delivery/hf/mlc-ai/Qwen2-0.5B-Instruct-q4f16_1-MLC/vocab.json +[2024-06-06 22:33:19] INFO gen_config.py:155: Found tokenizer config: /ssd2/models/Qwen2-0.5B-Instruct/merges.txt. Copying to /models/mlc-delivery/hf/mlc-ai/Qwen2-0.5B-Instruct-q4f16_1-MLC/merges.txt +[2024-06-06 22:33:19] INFO gen_config.py:157: Not found tokenizer config: /ssd2/models/Qwen2-0.5B-Instruct/added_tokens.json +[2024-06-06 22:33:19] INFO gen_config.py:155: Found tokenizer config: /ssd2/models/Qwen2-0.5B-Instruct/tokenizer_config.json. Copying to /models/mlc-delivery/hf/mlc-ai/Qwen2-0.5B-Instruct-q4f16_1-MLC/tokenizer_config.json +[2024-06-06 22:33:19] INFO gen_config.py:216: Detected tokenizer info: {'token_postproc_method': 'byte_level', 'prepend_space_in_encode': False, 'strip_space_in_decode': False} +[2024-06-06 22:33:19] INFO gen_config.py:32: [System default] Setting presence_penalty: 0.0 +[2024-06-06 22:33:19] INFO gen_config.py:32: [System default] Setting frequency_penalty: 0.0 +[2024-06-06 22:33:19] INFO gen_config.py:223: Dumping configuration file to: /models/mlc-delivery/hf/mlc-ai/Qwen2-0.5B-Instruct-q4f16_1-MLC/mlc-chat-config.json +/home/rickzhou/miniconda3/envs/mlc/bin/python -m mlc_llm convert_weight /ssd2/models/Qwen2-0.5B-Instruct --quantization q4f16_1 --output /models/mlc-delivery/hf/mlc-ai/Qwen2-0.5B-Instruct-q4f16_1-MLC +[2024-06-06 22:33:21] INFO auto_config.py:116: Found model configuration: /ssd2/models/Qwen2-0.5B-Instruct/config.json +[2024-06-06 22:33:22] INFO auto_device.py:79: Found device: cuda:0 +[2024-06-06 22:33:22] INFO auto_device.py:79: Found device: cuda:1 +[2024-06-06 22:33:23] INFO auto_device.py:88: Not found device: rocm:0 +[2024-06-06 22:33:24] INFO auto_device.py:88: Not found device: metal:0 +[2024-06-06 22:33:25] INFO auto_device.py:79: Found device: vulkan:0 +[2024-06-06 22:33:25] INFO auto_device.py:79: Found device: vulkan:1 +[2024-06-06 22:33:25] INFO auto_device.py:79: Found device: vulkan:2 +[2024-06-06 22:33:26] INFO auto_device.py:88: Not found device: opencl:0 +[2024-06-06 22:33:26] INFO auto_device.py:35: Using device: cuda:0 +[2024-06-06 22:33:26] INFO auto_weight.py:71: Finding weights in: /ssd2/models/Qwen2-0.5B-Instruct +[2024-06-06 22:33:26] INFO auto_weight.py:137: Not found Huggingface PyTorch +[2024-06-06 22:33:26] INFO auto_weight.py:144: Found source weight format: huggingface-safetensor. Source configuration: /ssd2/models/Qwen2-0.5B-Instruct/model.safetensors.index.json +[2024-06-06 22:33:26] INFO auto_weight.py:107: Using source weight configuration: /ssd2/models/Qwen2-0.5B-Instruct/model.safetensors.index.json. Use `--source` to override. +[2024-06-06 22:33:26] INFO auto_weight.py:111: Using source weight format: huggingface-safetensor. Use `--source-format` to override. +[2024-06-06 22:33:26] INFO auto_config.py:154: Found model type: qwen2. Use `--model-type` to override. +[2024-06-06 22:33:26] INFO qwen2_model.py:50: context_window_size not found in config.json. Falling back to max_position_embeddings (32768) +[2024-06-06 22:33:26] INFO qwen2_model.py:67: prefill_chunk_size defaults to 2048 +Weight conversion with arguments: + --config /ssd2/models/Qwen2-0.5B-Instruct/config.json + --quantization GroupQuantize(name='q4f16_1', kind='group-quant', group_size=32, quantize_dtype='int4', storage_dtype='uint32', model_dtype='float16', linear_weight_layout='NK', quantize_embedding=True, quantize_final_fc=True, num_elem_per_storage=8, num_storage_per_group=4, max_int_value=7) + --model-type qwen2 + --device cuda:0 + --source /ssd2/models/Qwen2-0.5B-Instruct/model.safetensors.index.json + --source-format huggingface-safetensor + --output /models/mlc-delivery/hf/mlc-ai/Qwen2-0.5B-Instruct-q4f16_1-MLC +Start storing to cache /models/mlc-delivery/hf/mlc-ai/Qwen2-0.5B-Instruct-q4f16_1-MLC + 0%| | 0/170 [00:00