Spaces:
Running
on
CPU Upgrade
Running
on
CPU Upgrade
Update utils.py
Browse files
utils.py
CHANGED
@@ -64,17 +64,20 @@ SUBMIT_INTRODUCTION = """# Submit on Science Leaderboard Introduction
|
|
64 |
## ⚠ Please note that you need to submit the json file with following format:
|
65 |
|
66 |
```json
|
67 |
-
|
68 |
-
|
69 |
-
|
70 |
-
|
71 |
-
|
72 |
-
|
73 |
-
|
74 |
-
|
|
|
|
|
|
|
|
|
75 |
```
|
76 |
-
|
77 |
-
|
78 |
"""
|
79 |
|
80 |
|
|
|
64 |
## ⚠ Please note that you need to submit the json file with following format:
|
65 |
|
66 |
```json
|
67 |
+
[
|
68 |
+
{
|
69 |
+
"question_id": 123,
|
70 |
+
"question": "abc",
|
71 |
+
"options": ["abc", "xyz", ...],
|
72 |
+
"answer": "ABC",
|
73 |
+
"answer_index": 1,
|
74 |
+
"category": "abc,
|
75 |
+
"pred": "B",
|
76 |
+
"model_outputs": ""
|
77 |
+
}, ...
|
78 |
+
]
|
79 |
```
|
80 |
+
You can generate an output file in the above format using the evaluation script provided in our GitHub repository. For your convenience, the script and detailed instructions are available at GitHub: https://github.com/TIGER-AI-Lab/MMLU-Pro. After generating the file, please send us an email at [email protected], attaching the output file.
|
|
|
81 |
"""
|
82 |
|
83 |
|