uploaded readme

README.md

@@ -0,0 +1,92 @@
+Quantization made by Richard Erkhov.
+
+[Github](https://github.com/RichardErkhov)
+
+[Discord](https://discord.gg/pvy7H8DZMG)
+
+[Request more models](https://github.com/RichardErkhov/quant_request)
+
+
+vicuna-7b-v1.3-attention-sparsity-20 - GGUF
+- Model creator: https://huggingface.co/wang7776/
+- Original model: https://huggingface.co/wang7776/vicuna-7b-v1.3-attention-sparsity-20/
+
+
+| Name | Quant method | Size |
+| ---- | ---- | ---- |
+| [vicuna-7b-v1.3-attention-sparsity-20.Q2_K.gguf](https://huggingface.co/RichardErkhov/wang7776_-_vicuna-7b-v1.3-attention-sparsity-20-gguf/blob/main/vicuna-7b-v1.3-attention-sparsity-20.Q2_K.gguf) | Q2_K | 2.36GB |
+| [vicuna-7b-v1.3-attention-sparsity-20.IQ3_XS.gguf](https://huggingface.co/RichardErkhov/wang7776_-_vicuna-7b-v1.3-attention-sparsity-20-gguf/blob/main/vicuna-7b-v1.3-attention-sparsity-20.IQ3_XS.gguf) | IQ3_XS | 2.6GB |
+| [vicuna-7b-v1.3-attention-sparsity-20.IQ3_S.gguf](https://huggingface.co/RichardErkhov/wang7776_-_vicuna-7b-v1.3-attention-sparsity-20-gguf/blob/main/vicuna-7b-v1.3-attention-sparsity-20.IQ3_S.gguf) | IQ3_S | 2.75GB |
+| [vicuna-7b-v1.3-attention-sparsity-20.Q3_K_S.gguf](https://huggingface.co/RichardErkhov/wang7776_-_vicuna-7b-v1.3-attention-sparsity-20-gguf/blob/main/vicuna-7b-v1.3-attention-sparsity-20.Q3_K_S.gguf) | Q3_K_S | 2.75GB |
+| [vicuna-7b-v1.3-attention-sparsity-20.IQ3_M.gguf](https://huggingface.co/RichardErkhov/wang7776_-_vicuna-7b-v1.3-attention-sparsity-20-gguf/blob/main/vicuna-7b-v1.3-attention-sparsity-20.IQ3_M.gguf) | IQ3_M | 2.9GB |
+| [vicuna-7b-v1.3-attention-sparsity-20.Q3_K.gguf](https://huggingface.co/RichardErkhov/wang7776_-_vicuna-7b-v1.3-attention-sparsity-20-gguf/blob/main/vicuna-7b-v1.3-attention-sparsity-20.Q3_K.gguf) | Q3_K | 3.07GB |
+| [vicuna-7b-v1.3-attention-sparsity-20.Q3_K_M.gguf](https://huggingface.co/RichardErkhov/wang7776_-_vicuna-7b-v1.3-attention-sparsity-20-gguf/blob/main/vicuna-7b-v1.3-attention-sparsity-20.Q3_K_M.gguf) | Q3_K_M | 3.07GB |
+| [vicuna-7b-v1.3-attention-sparsity-20.Q3_K_L.gguf](https://huggingface.co/RichardErkhov/wang7776_-_vicuna-7b-v1.3-attention-sparsity-20-gguf/blob/main/vicuna-7b-v1.3-attention-sparsity-20.Q3_K_L.gguf) | Q3_K_L | 3.35GB |
+| [vicuna-7b-v1.3-attention-sparsity-20.IQ4_XS.gguf](https://huggingface.co/RichardErkhov/wang7776_-_vicuna-7b-v1.3-attention-sparsity-20-gguf/blob/main/vicuna-7b-v1.3-attention-sparsity-20.IQ4_XS.gguf) | IQ4_XS | 3.4GB |
+| [vicuna-7b-v1.3-attention-sparsity-20.Q4_0.gguf](https://huggingface.co/RichardErkhov/wang7776_-_vicuna-7b-v1.3-attention-sparsity-20-gguf/blob/main/vicuna-7b-v1.3-attention-sparsity-20.Q4_0.gguf) | Q4_0 | 3.56GB |
+| [vicuna-7b-v1.3-attention-sparsity-20.IQ4_NL.gguf](https://huggingface.co/RichardErkhov/wang7776_-_vicuna-7b-v1.3-attention-sparsity-20-gguf/blob/main/vicuna-7b-v1.3-attention-sparsity-20.IQ4_NL.gguf) | IQ4_NL | 3.58GB |
+| [vicuna-7b-v1.3-attention-sparsity-20.Q4_K_S.gguf](https://huggingface.co/RichardErkhov/wang7776_-_vicuna-7b-v1.3-attention-sparsity-20-gguf/blob/main/vicuna-7b-v1.3-attention-sparsity-20.Q4_K_S.gguf) | Q4_K_S | 3.59GB |
+| [vicuna-7b-v1.3-attention-sparsity-20.Q4_K.gguf](https://huggingface.co/RichardErkhov/wang7776_-_vicuna-7b-v1.3-attention-sparsity-20-gguf/blob/main/vicuna-7b-v1.3-attention-sparsity-20.Q4_K.gguf) | Q4_K | 3.8GB |
+| [vicuna-7b-v1.3-attention-sparsity-20.Q4_K_M.gguf](https://huggingface.co/RichardErkhov/wang7776_-_vicuna-7b-v1.3-attention-sparsity-20-gguf/blob/main/vicuna-7b-v1.3-attention-sparsity-20.Q4_K_M.gguf) | Q4_K_M | 3.8GB |
+| [vicuna-7b-v1.3-attention-sparsity-20.Q4_1.gguf](https://huggingface.co/RichardErkhov/wang7776_-_vicuna-7b-v1.3-attention-sparsity-20-gguf/blob/main/vicuna-7b-v1.3-attention-sparsity-20.Q4_1.gguf) | Q4_1 | 3.95GB |
+| [vicuna-7b-v1.3-attention-sparsity-20.Q5_0.gguf](https://huggingface.co/RichardErkhov/wang7776_-_vicuna-7b-v1.3-attention-sparsity-20-gguf/blob/main/vicuna-7b-v1.3-attention-sparsity-20.Q5_0.gguf) | Q5_0 | 4.33GB |
+| [vicuna-7b-v1.3-attention-sparsity-20.Q5_K_S.gguf](https://huggingface.co/RichardErkhov/wang7776_-_vicuna-7b-v1.3-attention-sparsity-20-gguf/blob/main/vicuna-7b-v1.3-attention-sparsity-20.Q5_K_S.gguf) | Q5_K_S | 4.33GB |
+| [vicuna-7b-v1.3-attention-sparsity-20.Q5_K.gguf](https://huggingface.co/RichardErkhov/wang7776_-_vicuna-7b-v1.3-attention-sparsity-20-gguf/blob/main/vicuna-7b-v1.3-attention-sparsity-20.Q5_K.gguf) | Q5_K | 4.45GB |
+| [vicuna-7b-v1.3-attention-sparsity-20.Q5_K_M.gguf](https://huggingface.co/RichardErkhov/wang7776_-_vicuna-7b-v1.3-attention-sparsity-20-gguf/blob/main/vicuna-7b-v1.3-attention-sparsity-20.Q5_K_M.gguf) | Q5_K_M | 4.45GB |
+| [vicuna-7b-v1.3-attention-sparsity-20.Q5_1.gguf](https://huggingface.co/RichardErkhov/wang7776_-_vicuna-7b-v1.3-attention-sparsity-20-gguf/blob/main/vicuna-7b-v1.3-attention-sparsity-20.Q5_1.gguf) | Q5_1 | 4.72GB |
+| [vicuna-7b-v1.3-attention-sparsity-20.Q6_K.gguf](https://huggingface.co/RichardErkhov/wang7776_-_vicuna-7b-v1.3-attention-sparsity-20-gguf/blob/main/vicuna-7b-v1.3-attention-sparsity-20.Q6_K.gguf) | Q6_K | 5.15GB |
+| [vicuna-7b-v1.3-attention-sparsity-20.Q8_0.gguf](https://huggingface.co/RichardErkhov/wang7776_-_vicuna-7b-v1.3-attention-sparsity-20-gguf/blob/main/vicuna-7b-v1.3-attention-sparsity-20.Q8_0.gguf) | Q8_0 | 6.67GB |
+
+
+
+
+Original model description:
+---
+inference: false
+license: apache-2.0
+---
+# Overview
+This model has been pruned to 20% sparsity using the [Wanda pruning method](https://arxiv.org/abs/2306.11695) on attention layers. This method requires no retraining or weight updates and still achieves competitive performance. A link to the base model can be found [here](https://huggingface.co/lmsys/vicuna-7b-v1.3).
+
+
+# Vicuna Model Card
+
+## Model Details
+
+Vicuna is a chat assistant trained by fine-tuning LLaMA on user-shared conversations collected from ShareGPT.
+
+- **Developed by:** [LMSYS](https://lmsys.org/)
+- **Model type:** An auto-regressive language model based on the transformer architecture.
+- **License:** Non-commercial license
+- **Finetuned from model:** [LLaMA](https://arxiv.org/abs/2302.13971).
+
+### Model Sources
+
+- **Repository:** https://github.com/lm-sys/FastChat
+- **Blog:** https://lmsys.org/blog/2023-03-30-vicuna/
+- **Paper:** https://arxiv.org/abs/2306.05685
+- **Demo:** https://chat.lmsys.org/
+
+## Uses
+
+The primary use of Vicuna is research on large language models and chatbots.
+The primary intended users of the model are researchers and hobbyists in natural language processing, machine learning, and artificial intelligence.
+
+## How to Get Started with the Model
+
+- Command line interface: https://github.com/lm-sys/FastChat#vicuna-weights.  
+- APIs (OpenAI API, Huggingface API): https://github.com/lm-sys/FastChat/tree/main#api.  
+
+## Training Details
+
+Vicuna v1.3 is fine-tuned from LLaMA with supervised instruction fine-tuning.
+The training data is around 125K conversations collected from ShareGPT.com.
+See more details in the "Training Details of Vicuna Models" section in the appendix of this [paper](https://arxiv.org/pdf/2306.05685.pdf).
+
+## Evaluation
+
+Vicuna is evaluated with standard benchmarks, human preference, and LLM-as-a-judge. See more details in this [paper](https://arxiv.org/pdf/2306.05685.pdf) and [leaderboard](https://huggingface.co/spaces/lmsys/chatbot-arena-leaderboard).
+
+## Difference between different versions of Vicuna
+See [vicuna_weights_version.md](https://github.com/lm-sys/FastChat/blob/main/docs/vicuna_weights_version.md)
+