Update README.md
Browse files
README.md
CHANGED
|
@@ -23,18 +23,31 @@ metrics:
|
|
| 23 |
</div>
|
| 24 |
|
| 25 |
|
| 26 |
-
<div align="center">
|
| 27 |
-
|
| 28 |
-
|
| 29 |
-
|
| 30 |
-
|
| 31 |
-
|
| 32 |
-
|
| 33 |
-
|
| 34 |
-
|
| 35 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 36 |
</div>
|
| 37 |
|
|
|
|
| 38 |
## Abstract
|
| 39 |
State-of-the-art advances in LLMs have pushed NLP to its limits, where even complex tasks, such as code generation, can be automated. This paper describes the deepSQL-R1-distill-8B, a fine-tuned and quantized model variant of the DeepSeek-R1 model architecture and specifically optimized for text-to-SQL conversion. Fine-tuning was performed using Unsloth, one of the most efficient frameworks for fine-tuning LLMs, in combination with Parameter-Efficient Fine-Tuning and the SFTTrainer framework. This allows domain-specific adaptation with minimal resource consumption. The approach fine-tunes curated datasets by LoRA, ensuring a more parameter-efficient and lower-memory-consuming model. Besides this, we investigate reinforcement learning techniques to further enhance the model's ability in generating accurate and contextually appropriate SQL queries. Combination of 8-bit quantization, LoRA, Unsloth, and reinforcement learning places deepSQL-R1-distill-8B as one of the cutting-edge solutions for automatic SQL code generation in real-world applications. Addressing major challenges in computational efficiency, domain-specific adaptation, and reinforcement-based refinement, this model is leading the way toward a more intuitive and resource-effective way of interacting with relational databases.
|
| 40 |
|
|
@@ -194,43 +207,50 @@ _ = model.generate(**inputs, streamer=text_streamer, max_new_tokens=350)
|
|
| 194 |
```
|
| 195 |
|
| 196 |
## Author
|
| 197 |
-
<div align="center">
|
| 198 |
-
<
|
| 199 |
-
<
|
| 200 |
-
|
| 201 |
-
|
| 202 |
-
<
|
| 203 |
-
|
| 204 |
-
|
| 205 |
-
<
|
| 206 |
-
|
| 207 |
-
|
| 208 |
-
<
|
| 209 |
-
|
| 210 |
-
|
| 211 |
-
<
|
| 212 |
-
|
| 213 |
-
|
| 214 |
-
<
|
| 215 |
-
|
| 216 |
-
|
| 217 |
-
<
|
| 218 |
-
|
| 219 |
-
|
| 220 |
-
<
|
| 221 |
-
|
| 222 |
-
|
| 223 |
-
<
|
| 224 |
-
|
| 225 |
-
</a>
|
| 226 |
-
</p>
|
| 227 |
</div>
|
| 228 |
|
| 229 |
|
|
|
|
| 230 |
## Usages Services
|
| 231 |
-
<
|
| 232 |
-
|
| 233 |
-
|
| 234 |
-
|
| 235 |
-
|
| 236 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 23 |
</div>
|
| 24 |
|
| 25 |
|
| 26 |
+
<div align="center" style="line-height: 1;">
|
| 27 |
+
<a href="https://huggingface.co/imsanjoykb/deepSQL-R1-distill-8B" target="_blank" style="margin: 2px;">
|
| 28 |
+
<img alt="Hugging Face Model" src="https://img.shields.io/badge/HuggingFace-Model-FF6F00?style=for-the-badge&logo=huggingface&logoColor=white" style="display: inline-block; vertical-align: middle;">
|
| 29 |
+
</a>
|
| 30 |
+
<a href="https://drive.google.com/file/d/145PP-oW50OMS1bYJaYuUphfufpsuOGWl/view?usp=sharing" target="_blank" style="margin: 2px;">
|
| 31 |
+
<img alt="Open In Colab" src="https://img.shields.io/badge/Open%20in%20Colab-FF6F00%2F000000?style=for-the-badge&logo=googlecolab&logoColor=white&labelColor=FF6F00" style="display: inline-block; vertical-align: middle;">
|
| 32 |
+
</a>
|
| 33 |
+
<a href="https://www.kaggle.com/code/imsanjoykb/inference-deepsql-r1-distill-8b" target="_blank" style="margin: 2px;">
|
| 34 |
+
<img alt="Kaggle Notebook" src="https://img.shields.io/badge/Kaggle-Notebook-20BEFF?style=for-the-badge&logo=kaggle&logoColor=white" style="display: inline-block; vertical-align: middle;">
|
| 35 |
+
</a>
|
| 36 |
+
<a href="https://github.com/imsanjoykb/deepSQL-R1-distill-8B" target="_blank" style="margin: 2px;">
|
| 37 |
+
<img alt="GitHub Repo" src="https://img.shields.io/badge/GitHub-Repo-181717?style=for-the-badge&logo=github&logoColor=white" style="display: inline-block; vertical-align: middle;">
|
| 38 |
+
</a>
|
| 39 |
+
<a href="https://huggingface.co/spaces/imsanjoykb/deepSQL-R1-distill-8B" target="_blank" style="margin: 2px;">
|
| 40 |
+
<img alt="Gradio App" src="https://img.shields.io/badge/Chat%20App-Gradio-0084FF?style=for-the-badge&logo=gradio&logoColor=white" style="display: inline-block; vertical-align: middle;">
|
| 41 |
+
</a>
|
| 42 |
+
<a href="https://colab.research.google.com/drive/1ze7qAQnjppZKfxNVBXXlOBTM6xFWEYrJ?usp=sharing" target="_blank" style="margin: 2px;">
|
| 43 |
+
<img alt="Gradio-Colab" src="https://img.shields.io/badge/Gradio-Colab-0084FF?style=for-the-badge&logo=gradio&labelColor=F9AB00" style="display: inline-block; vertical-align: middle;">
|
| 44 |
+
</a>
|
| 45 |
+
<a href="https://arxiv.org/abs/Your_Paper_ID" target="_blank" style="margin: 2px;">
|
| 46 |
+
<img alt="arXiv Paper" src="https://img.shields.io/badge/arXiv-Preprint-B31B1B?style=for-the-badge&logo=arxiv&logoColor=white" style="display: inline-block; vertical-align: middle;">
|
| 47 |
+
</a>
|
| 48 |
</div>
|
| 49 |
|
| 50 |
+
|
| 51 |
## Abstract
|
| 52 |
State-of-the-art advances in LLMs have pushed NLP to its limits, where even complex tasks, such as code generation, can be automated. This paper describes the deepSQL-R1-distill-8B, a fine-tuned and quantized model variant of the DeepSeek-R1 model architecture and specifically optimized for text-to-SQL conversion. Fine-tuning was performed using Unsloth, one of the most efficient frameworks for fine-tuning LLMs, in combination with Parameter-Efficient Fine-Tuning and the SFTTrainer framework. This allows domain-specific adaptation with minimal resource consumption. The approach fine-tunes curated datasets by LoRA, ensuring a more parameter-efficient and lower-memory-consuming model. Besides this, we investigate reinforcement learning techniques to further enhance the model's ability in generating accurate and contextually appropriate SQL queries. Combination of 8-bit quantization, LoRA, Unsloth, and reinforcement learning places deepSQL-R1-distill-8B as one of the cutting-edge solutions for automatic SQL code generation in real-world applications. Addressing major challenges in computational efficiency, domain-specific adaptation, and reinforcement-based refinement, this model is leading the way toward a more intuitive and resource-effective way of interacting with relational databases.
|
| 53 |
|
|
|
|
| 207 |
```
|
| 208 |
|
| 209 |
## Author
|
| 210 |
+
<div align="center" style="line-height: 1;">
|
| 211 |
+
<a href="mailto:[email protected]" target="_blank" style="margin: 2px;">
|
| 212 |
+
<img alt="Email" src="https://img.shields.io/badge/Gmail-D14836?style=for-the-badge&logo=gmail&logoColor=white" style="display: inline-block; vertical-align: middle;">
|
| 213 |
+
</a>
|
| 214 |
+
<a href="https://imsanjoykb.github.io/" target="_blank" style="margin: 2px;">
|
| 215 |
+
<img alt="Portfolio" src="https://img.shields.io/badge/Portfolio-8B89CC?style=for-the-badge&logo=protonmail&logoColor=white" style="display: inline-block; vertical-align: middle;">
|
| 216 |
+
</a>
|
| 217 |
+
<a href="https://www.linkedin.com/in/imsanjoykb/" target="_blank" style="margin: 2px;">
|
| 218 |
+
<img alt="Linkedin" src="https://img.shields.io/badge/LinkedIn-0077B5?style=for-the-badge&logo=linkedin&logoColor=white" style="display: inline-block; vertical-align: middle;">
|
| 219 |
+
</a>
|
| 220 |
+
<a href="https://orcid.org/0009-0001-6265-841X" target="_blank" style="margin: 2px;">
|
| 221 |
+
<img alt="ORCID" src="https://img.shields.io/badge/ORCID-0000--002--182-green?style=for-the-badge&logo=orcid&logoColor=white" style="display: inline-block; vertical-align: middle;">
|
| 222 |
+
</a>
|
| 223 |
+
<a href="https://github.com/imsanjoykb/" target="_blank" style="margin: 2px;">
|
| 224 |
+
<img alt="Github" src="https://img.shields.io/badge/GitHub-100000?style=for-the-badge&logo=github&logoColor=white" style="display: inline-block; vertical-align: middle;">
|
| 225 |
+
</a>
|
| 226 |
+
<a href="https://medium.com/@imsanjoykb" target="_blank" style="margin: 2px;">
|
| 227 |
+
<img alt="Medium" src="https://img.shields.io/badge/Medium-000000?style=for-the-badge&logo=medium&logoColor=white" style="display: inline-block; vertical-align: middle;">
|
| 228 |
+
</a>
|
| 229 |
+
<a href="https://www.kaggle.com/imsanjoykb" target="_blank" style="margin: 2px;">
|
| 230 |
+
<img alt="Kaggle" src="https://img.shields.io/badge/Kaggle-20BEFF?style=for-the-badge&logo=Kaggle&logoColor=white" style="display: inline-block; vertical-align: middle;">
|
| 231 |
+
</a>
|
| 232 |
+
<a href="https://instagram.com/imsanjoykb/" target="_blank" style="margin: 2px;">
|
| 233 |
+
<img alt="Instagram" src="https://img.shields.io/badge/Instagram-E4405F?style=for-the-badge&logo=instagram&logoColor=white" style="display: inline-block; vertical-align: middle;">
|
| 234 |
+
</a>
|
| 235 |
+
<a href="https://discord.com/channels/@imsanjoykb" target="_blank" style="margin: 2px;">
|
| 236 |
+
<img alt="Discord" src="https://img.shields.io/badge/Discord-7289DA?style=for-the-badge&logo=discord&logoColor=white" style="display: inline-block; vertical-align: middle;">
|
| 237 |
+
</a>
|
|
|
|
|
|
|
| 238 |
</div>
|
| 239 |
|
| 240 |
|
| 241 |
+
|
| 242 |
## Usages Services
|
| 243 |
+
<div align="center" style="line-height: 1;">
|
| 244 |
+
<a href="#" target="_blank" style="margin: 2px;">
|
| 245 |
+
<img src="https://ia801209.us.archive.org/26/items/github.com-unslothai-unsloth_-_2023-12-03_15-21-29/cover.jpg" alt="Unsloth" width="100" style="display: inline-block; vertical-align: middle;">
|
| 246 |
+
</a>
|
| 247 |
+
<a href="#" target="_blank" style="margin: 2px;">
|
| 248 |
+
<img src="https://wandb.ai/logo.png" alt="Weights & Biases" width="100" style="display: inline-block; vertical-align: middle;">
|
| 249 |
+
</a>
|
| 250 |
+
<a href="#" target="_blank" style="margin: 2px;">
|
| 251 |
+
<img src="https://huggingface.co/front/assets/huggingface_logo.svg" alt="Hugging Face" width="100" style="display: inline-block; vertical-align: middle;">
|
| 252 |
+
</a>
|
| 253 |
+
<a href="#" target="_blank" style="margin: 2px;">
|
| 254 |
+
<img src="https://images.saasworthy.com/tr:w-160,h-0,c-at_max,e-sharpen-1/gradio_43063_logo_1681283997_9ue7l.jpg" alt="Gradio" width="100" style="display: inline-block; vertical-align: middle;">
|
| 255 |
+
</a>
|
| 256 |
+
</div>
|