Update README.md
Browse files
README.md
CHANGED
@@ -14,8 +14,7 @@ tags:
|
|
14 |
|
15 |
This lora was trained on 250k post and response pairs from 43 different fincial, investing, and crypto subreddits. It is not an instruct model, it is designed to generate a reply to a reddit text post. It was an experiment in fine tuning for specific tasks. **Use it responsibly**
|
16 |
|
17 |
-
* Training
|
18 |
-
* Dataset and tools for building the dataset will be released soon.
|
19 |
|
20 |
## Training Details
|
21 |
|
@@ -25,7 +24,7 @@ Base Model: llama 7b
|
|
25 |
|
26 |
* 1 note worthy change I will mention now, is this was trained with casualLM rather than seq2seq like a number of the other instruct models have been. I can't explain why they used seq2seq for data collators, other than that's what alpaca lora originally used. Llama as a generative model was trained for casualLM so to me it makes sense to use that when fine tuning.
|
27 |
|
28 |
-
|
29 |
|
30 |
### Training Hyperparams
|
31 |
|
@@ -59,7 +58,9 @@ Base Model: llama 7b
|
|
59 |
| crypto_currency | 186 | 0.694596 | 1.101901 | 0.026738 |
|
60 |
| StocksAndTrading | 93 | 0.184637 | 1.704545 | 0.019066 |
|
61 |
|
62 |
-
|
|
|
|
|
63 |
|
64 |
|
65 |
## Usage
|
|
|
14 |
|
15 |
This lora was trained on 250k post and response pairs from 43 different fincial, investing, and crypto subreddits. It is not an instruct model, it is designed to generate a reply to a reddit text post. It was an experiment in fine tuning for specific tasks. **Use it responsibly**
|
16 |
|
17 |
+
* Training, dataset and tools available here: <https://github.com/getorca/ProfitsBot_V0_OLLM>
|
|
|
18 |
|
19 |
## Training Details
|
20 |
|
|
|
24 |
|
25 |
* 1 note worthy change I will mention now, is this was trained with casualLM rather than seq2seq like a number of the other instruct models have been. I can't explain why they used seq2seq for data collators, other than that's what alpaca lora originally used. Llama as a generative model was trained for casualLM so to me it makes sense to use that when fine tuning.
|
26 |
|
27 |
+
training code is availble here: <https://github.com/getorca/ProfitsBot_V0_OLLM/tree/main/training>
|
28 |
|
29 |
### Training Hyperparams
|
30 |
|
|
|
58 |
| crypto_currency | 186 | 0.694596 | 1.101901 | 0.026738 |
|
59 |
| StocksAndTrading | 93 | 0.184637 | 1.704545 | 0.019066 |
|
60 |
|
61 |
+
the dataset is available here: <https://huggingface.co/datasets/winddude/reddit_finance_43_250k>
|
62 |
+
|
63 |
+
code for recreating the dataset is here: <https://github.com/getorca/ProfitsBot_V0_OLLM/tree/main/ds_builder>
|
64 |
|
65 |
|
66 |
## Usage
|