Commit
·
19cfca2
1
Parent(s):
d5d0b9d
commit files to HF hub
Browse files
README.md
CHANGED
@@ -13,6 +13,8 @@ tags:
|
|
13 |
datasets:
|
14 |
- h2oai/h2ogpt-oig-oasst1-instruct-cleaned-v1
|
15 |
- h2oai/openassistant_oasst1_h2ogpt
|
|
|
|
|
16 |
---
|
17 |
# h2oGPT Model Card
|
18 |
## Summary
|
@@ -20,9 +22,9 @@ datasets:
|
|
20 |
H2O.ai's `h2ogpt-oig-oasst1-512-6.9b` is a 6.9 billion parameter instruction-following large language model licensed for commercial use.
|
21 |
|
22 |
- Base model: [EleutherAI/pythia-6.9b](https://huggingface.co/EleutherAI/pythia-6.9b)
|
23 |
-
- Fine-tuning dataset: [h2oai/h2ogpt-oig-oasst1-instruct-cleaned-v1](https://huggingface.co/datasets/h2oai/h2ogpt-oig-oasst1-instruct-cleaned-v1) and [h2oai/openassistant_oasst1_h2ogpt](https://huggingface.co/datasets/h2oai/openassistant_oasst1_h2ogpt)
|
24 |
- Data-prep and fine-tuning code: [H2O.ai GitHub](https://github.com/h2oai/h2ogpt)
|
25 |
-
- Training logs: [zip](https://huggingface.co/h2oai/h2ogpt-oig-oasst1-512-6.9b/blob/main/pythia-6.9b.h2ogpt-oig-oasst1-instruct-cleaned-v1.json.1_epochs.5fc91911bc2bfaaf3b6c2de577c4b0ae45a07a4a.7.zip) and [zip](https://huggingface.co/h2oai/h2ogpt-oig-oasst1-512-6.9b/blob/main/h2ogpt-oig-oasst1-512-6.9b.h2oaiopenassistant_oasst1_h2ogpt.2_epochs.e35e2e06e0af2f7dceac2e16e3646c90ccce4ec0.1.zip)
|
26 |
|
27 |
## Chatbot
|
28 |
|
@@ -134,22 +136,22 @@ GPTNeoXConfig {
|
|
134 |
Model validation results using [EleutherAI lm-evaluation-harness](https://github.com/EleutherAI/lm-evaluation-harness).
|
135 |
|
136 |
|
137 |
-
[eval source code](https://github.com/h2oai/h2ogpt/issues/
|
138 |
|
139 |
| Task |Version| Metric |Value | |Stderr|
|
140 |
|-------------|------:|--------|-----:|---|-----:|
|
141 |
-
|
|
142 |
-
|
|
143 |
-
|
|
144 |
-
|
|
145 |
-
|
|
146 |
-
|
|
147 |
-
|
|
148 |
-
|
|
149 |
-
| | |acc_norm|0.
|
150 |
-
|
|
151 |
-
|
|
152 |
-
|
|
153 |
|
154 |
|
155 |
## Disclaimer
|
|
|
13 |
datasets:
|
14 |
- h2oai/h2ogpt-oig-oasst1-instruct-cleaned-v1
|
15 |
- h2oai/openassistant_oasst1_h2ogpt
|
16 |
+
- h2oai/h2ogpt-fortune2000-personalized
|
17 |
+
- h2oai/h2ogpt-oig-oasst1-instruct-cleaned-v3
|
18 |
---
|
19 |
# h2oGPT Model Card
|
20 |
## Summary
|
|
|
22 |
H2O.ai's `h2ogpt-oig-oasst1-512-6.9b` is a 6.9 billion parameter instruction-following large language model licensed for commercial use.
|
23 |
|
24 |
- Base model: [EleutherAI/pythia-6.9b](https://huggingface.co/EleutherAI/pythia-6.9b)
|
25 |
+
- Fine-tuning dataset: [h2oai/h2ogpt-oig-oasst1-instruct-cleaned-v1](https://huggingface.co/datasets/h2oai/h2ogpt-oig-oasst1-instruct-cleaned-v1) and [h2oai/openassistant_oasst1_h2ogpt](https://huggingface.co/datasets/h2oai/openassistant_oasst1_h2ogpt) and [h2oai/h2ogpt-fortune2000-personalized](https://huggingface.co/datasets/h2oai/h2ogpt-fortune2000-personalized) and [h2oai/h2ogpt-oig-oasst1-instruct-cleaned-v3](https://huggingface.co/datasets/h2oai/h2ogpt-oig-oasst1-instruct-cleaned-v3)
|
26 |
- Data-prep and fine-tuning code: [H2O.ai GitHub](https://github.com/h2oai/h2ogpt)
|
27 |
+
- Training logs: [zip](https://huggingface.co/h2oai/h2ogpt-oig-oasst1-512-6.9b/blob/main/pythia-6.9b.h2ogpt-oig-oasst1-instruct-cleaned-v1.json.1_epochs.5fc91911bc2bfaaf3b6c2de577c4b0ae45a07a4a.7.zip) and [zip](https://huggingface.co/h2oai/h2ogpt-oig-oasst1-512-6.9b/blob/main/h2ogpt-oig-oasst1-512-6.9b.h2oaiopenassistant_oasst1_h2ogpt.2_epochs.e35e2e06e0af2f7dceac2e16e3646c90ccce4ec0.1.zip) and [zip](https://huggingface.co/h2oai/h2ogpt-oig-oasst1-512-6.9b/blob/main/h2ogpt-oig-oasst1-512-6.9b.h2oaih2ogpt-oig-oasst1-instruct-cleaned-v3.1_epochs.e48f9debb0d2bd8d866fa5668bbbb51c317c553c.1.zip)
|
28 |
|
29 |
## Chatbot
|
30 |
|
|
|
136 |
Model validation results using [EleutherAI lm-evaluation-harness](https://github.com/EleutherAI/lm-evaluation-harness).
|
137 |
|
138 |
|
139 |
+
[eval source code](https://github.com/h2oai/h2ogpt/issues/125#issue-1702311702)
|
140 |
|
141 |
| Task |Version| Metric |Value | |Stderr|
|
142 |
|-------------|------:|--------|-----:|---|-----:|
|
143 |
+
|arc_easy | 0|acc |0.6591|± |0.0097|
|
144 |
+
| | |acc_norm|0.6178|± |0.0100|
|
145 |
+
|arc_challenge| 0|acc |0.3174|± |0.0136|
|
146 |
+
| | |acc_norm|0.3558|± |0.0140|
|
147 |
+
|openbookqa | 0|acc |0.2540|± |0.0195|
|
148 |
+
| | |acc_norm|0.3580|± |0.0215|
|
149 |
+
|winogrande | 0|acc |0.6069|± |0.0137|
|
150 |
+
|piqa | 0|acc |0.7486|± |0.0101|
|
151 |
+
| | |acc_norm|0.7546|± |0.0100|
|
152 |
+
|hellaswag | 0|acc |0.4843|± |0.0050|
|
153 |
+
| | |acc_norm|0.6388|± |0.0048|
|
154 |
+
|boolq | 1|acc |0.6193|± |0.0085|
|
155 |
|
156 |
|
157 |
## Disclaimer
|