Update README.md
Browse files
README.md
CHANGED
|
@@ -10,4 +10,7 @@ tags:
|
|
| 10 |
- chatbot
|
| 11 |
- peft
|
| 12 |
- qlora
|
| 13 |
-
---
|
|
|
|
|
|
|
|
|
|
|
|
| 10 |
- chatbot
|
| 11 |
- peft
|
| 12 |
- qlora
|
| 13 |
+
---
|
| 14 |
+
|
| 15 |
+
This is a finetuned model I have made using QLora (PEFT) with `lora_r` = 64 (help decide the matrice size where the weights will change) and `lora_alpha` = 16 (defines the intensity with which the lora will change the paramters)
|
| 16 |
+
which are the majorly important hyperparameters of QLora which I learnt recently with the help of Krish Naik.
|