|
--- |
|
base_model: meta-llama/Llama-3.2-1B |
|
library_name: peft |
|
tags: |
|
- code |
|
- llm |
|
- Evolution_Learning_Network |
|
- qlora |
|
- llama |
|
--- |
|
|
|
# Evolution Learning Network (ELN) with QLoRA and Genetic Algorithms For LLM |
|
|
|
## Overview |
|
|
|
This project implements an **Evolution Learning Network (ELN)** to fine-tune transformer-based models like LLaMA using a combination of **Quantized Low-Rank Adaptation (QLoRA)** and **Genetic Algorithms (GA)**. The primary objective is to evolve a population of models across multiple generations to optimize for performance (fitness) and specialization, while maintaining diversity. |
|
|
|
### Key Features |
|
- Efficient model fine-tuning using **QLoRA**. |
|
- Evolutionary strategies, including **random mutations** and fitness-based selection. |
|
- Hardware-efficient training with **4-bit quantization**. |
|
- Comprehensive experiment tracking with **WandB**. |
|
- Diversity maintenance through **LoRA weight fingerprinting**. |
|
|
|
--- |
|
|
|
## Model Details |
|
|
|
### Base Model |
|
- **Name**: [meta-llama/Llama-3.2-1B](https://huggingface.co/meta-llama/Llama-3.2-1B) (can be replaced with any Hugging Face model). |
|
- **Architecture**: Transformer-based causal language model. |
|
|
|
### Quantization Configuration |
|
- **Quantization Type**: 4-bit using `bitsandbytes` (`bnb_4bit`). |
|
- **Parameters**: |
|
- Compute Type: `torch.float16` |
|
- Quantization Type: `"nf4"` (Nonlinear quantization). |
|
- Double Quantization: Enabled. |
|
- Nested Quantization: Enabled. |
|
|
|
### LoRA (Low-Rank Adaptation) |
|
- **Dimensions (r)**: 8 |
|
- **Alpha (Scaling)**: 16 |
|
- **Target Modules**: Query and Value projections (`q_proj`, `v_proj`). |
|
- **Dropout**: 0.05 |
|
- **Task Type**: Causal Language Modeling (`CAUSAL_LM`). |
|
|
|
### Training Strategy |
|
- **Optimizer**: `paged_adamw_8bit` for memory-efficient updates. |
|
- **Precision**: Mixed precision (`fp16`) for faster training. |
|
|
|
--- |
|
|
|
## Hyperparameters |
|
|
|
### General Parameters |
|
- **Generations**: 10 |
|
- **Population Size**: 4 |
|
- **Dataset Size**: 2000 samples per split (adjustable for larger datasets). |
|
|
|
### Training |
|
- **Batch Size**: 8 |
|
- **Gradient Accumulation**: 16 steps. |
|
- **Learning Rate**: `2e-4` |
|
- **Epochs per Model**: 2 |
|
|
|
### Mutations |
|
- **Mutation Rate**: 10% (probability per parameter). |
|
- **Mutation Scale**: Noise added with a standard deviation of 0.02. |
|
|
|
--- |
|
|
|
## Dataset Details |
|
|
|
### Source |
|
- **Name**: WikiText ([wikitext-2-raw-v1](https://huggingface.co/datasets/Salesforce/wikitext/viewer/wikitext-2-raw-v1) for larger datasets). |
|
- **Splits**: |
|
- `train` → Model training. |
|
- `validation` → General task evaluation. |
|
- `test` → Specific task evaluation. |
|
|
|
### Tokenization |
|
- **Tokenizer**: Hugging Face `AutoTokenizer`. |
|
- **Max Token Length**: 128 tokens. |
|
- **Padding**: Fixed to `"max_length"`. |
|
|
|
--- |
|
|
|
## Results |
|
|
|
### Summary |
|
- **Total Generations**: 10 |
|
- **Best Fitness Achieved**: 0.4772 |
|
- **Final Population Diversity**: 0.0011 |
|
|
|
### Evolution History (Highlights) |
|
| Generation | Best Fitness | Avg Fitness | Diversity | Best Specialization | |
|
|------------|--------------|-------------|-----------|---------------------| |
|
| 1 | 0.4096 | 0.4023 | 0.00097 | 0.9967 | |
|
| 5 | 0.4727 | 0.4722 | 0.00099 | 0.9968 | |
|
| 10 | 0.4772 | 0.4768 | 0.00106 | 0.9972 | |
|
|
|
--- |
|
|
|
## Hardware & Framework |
|
|
|
### Hardware |
|
- Multi-GPU support with `torch.nn.parallel.DistributedDataParallel` or `Accelerator`. |
|
- Logs GPU/CPU usage with `psutil` and `torch.cuda`. |
|
|
|
### Frameworks & Libraries |
|
- **Transformers**: Hugging Face model and tokenizer handling. |
|
- **Datasets**: Data loading and processing. |
|
- **WandB**: Experiment tracking and visualization. |
|
- **BitsAndBytes**: 4-bit quantization. |
|
- **PEFT**: LoRA-based fine-tuning. |
|
|
|
--- |
|
|
|
## Future Work |
|
- Explore larger population sizes and more generations for enhanced diversity. |
|
- Experiment with other datasets to generalize findings. |
|
- Integrate additional mutation strategies for broader exploration. |
|
|
|
--- |
|
|
|
## Citation |
|
Remaining |
|
|
|
--- |
|
> Code to run locally |
|
|
|
```python |
|
from peft import PeftModel |
|
from transformers import AutoModelForCausalLM |
|
|
|
base_model = AutoModelForCausalLM.from_pretrained("meta-llama/Llama-3.2-1B") |
|
model = PeftModel.from_pretrained(base_model, "diabolic6045/ELN-AOC-CAIN") |
|
``` |
|
### Framework versions |
|
|
|
- PEFT 0.14.0 |
|
|
|
~ [diabolic6045](https://huggingface.co/diabolic6045) |
