syubraj/MedicalChat-Phi-3.5-mini-instruct

🏥 Phi-3.5 Mini Instruct Medical Chat (LoRA Adapter)

The MedicalChat-Phi-3.5-mini-instruct- fine-tuned model is designed to simulate doctor-patient conversations, offering medical consultations and suggestions based on patient queries. However, its accuracy may be limited in real-world scenarios, as the training dataset was relatively small.

🔍 Model Overview

• 🧑‍⚕️ Developed by: [syubraj]
• 📜 Model Type: Causal Language Model (CausalLM)
• 🗣️ Language: English
• 📜 License: MIT License
• 🛠️ Fine-tuned From: microsoft/phi-3.5-mini-instruct

---

🚀 How to Get Started

Install Required Packages

!pip install transformers peft accelerate torch bitsandbytes

Load and Use the Model

Since inference requires more memory, the model will be loaded and inference will be performed on the CPU.

from transformers import AutoTokenizer, AutoModelForCausalLM
import torch

device = "cuda" if torch.cuda.is_available() else "cpu"
print(f"Using device: {device}")

# Clear CUDA memory before using the model
torch.cuda.empty_cache()

model_name = "syubraj/MedicalChat-Phi-3.5-mini-instruct"

try:
    model = AutoModelForCausalLM.from_pretrained(model_name, trust_remote_code = True)
    tokenizer = AutoTokenizer.from_pretrained(model_name)
    print("Model and Tokenizer loaded successfully.")
except Exception as e:
    raise ValueError(f"Error loading Model and Tokenizer: {e}")

def generate_response(user_query: str, system_message: str = None, max_length: int = 1024) -> str:
    """
    Generates a response based on the given user query.

    :param user_query: The user's input message.
    :param system_message: Custom system instruction (optional, defaults to medical assistant).
    :param max_length: Max tokens to generate.
    :return: Generated assistant response.
    """
    if not user_query.strip():
        return "Error: User query cannot be empty."

    if system_message is None:
        system_message = ("You are a trusted AI-powered medical assistant. "
                          "Analyze patient queries carefully and provide accurate, professional, and empathetic responses. "
                          "Prioritize patient safety, adhere to medical best practices, and recommend consulting a healthcare provider when necessary.")

    prompt = f"<|system|> {system_message} <|end|> <|user|> {user_query} <|end|> <|assistant|>"

    try:
        inputs = tokenizer(prompt, return_tensors="pt").to(device)
        outputs = model.generate(**inputs, max_length=max_length)

    # Decode response
        response = tokenizer.decode(outputs[0], skip_special_tokens=True)
        return response.split("<|assistant|>")[-1].strip().split("<|end|>")[0].strip()
    except Exception as e:
        return f"Error generating response: {e}"
    
    return response if response else "Error: No response generated."

user_input = "Hi Doctor, How to increase my height? I am 23 years old."
response = generate_response(user_input)
print("Assistant Response:", response)

---

📝 Example Output

Assistant Response: Hello! I understand that you're looking to potentially increase your height, and I'll do my best to provide you with helpful information.

Firstly, it's important to clarify that after puberty, the growth plates in our bones, known as epiphyseal plates, have typically fused, meaning our height generally stabilizes. For most adults, especially those who have reached their late teens or early twenties, further height increase is unlikely.

However, there are a few factors that can influence your height:

1. Genetics: Your genes play a significant role in determining your height. If your parents are tall, you might be taller too.

2. Nutrition: Proper nutrition is essential for growth. Ensure you're getting a balanced diet with enough protein, calcium, and vitamins.

3. Posture: Sometimes, poor posture can make you appear shorter. Practicing good posture might help you look taller.

4. Sleep: Adequate sleep is crucial for growth hormones to work effectively.

5. Exercise: Regular exercise can help maintain a healthy posture and spine alignment, which might help you appear taller.

Here are some tips that might help you look taller:

• Stand Straight: Keep your shoulders back and your back straight. This can make you appear taller.

• Practice Good Posture: Be mindful of your posture throughout the day.

• Exercise: Engage in exercises that strengthen your core, back, and leg muscles.

• Shoe Lifts: Some people find that wearing shoe lifts can make them appear taller. However, this is more of a visual effect and won't actually increase your height.

• Physical Therapy: If you have concerns about your posture or spinal alignment, a physical therapist can provide exercises and treatments to help.

Remember, while these tips might help you look taller, they won't change your actual height. If you're concerned about your height for medical reasons, such as a growth hormone deficiency, it's best to consult with a healthcare provider.

I hope this information is helpful! If you have any more questions or need further assistance, feel free to ask.

---

📊 Training Details

Step	Training Loss	Validation Loss
50	15.751700	1.201298
100	4.741700	1.170077
150	4.734700	1.158106
200	4.521200	1.146289
250	4.638000	1.135715
300	3.985500	1.138858
350	4.079600	1.135848
400	4.299300	1.136855
450	4.298400	1.127821
500	3.974000	1.125197
550	3.747800	1.158925
600	3.638500	1.157699
650	3.661100	1.162100
700	3.460500	1.160023
750	3.640400	1.159106

---

📚 Training Data

A total of 1,000 entries were used for training, while 200 entries were reserved for testing.

📂 Dataset: syubraj/medical-chat-phi-3.5-instruct-1k

📂 Preprocessed Dataset: syubraj/ruslanmv_medicalChat_phi3.5_instruct

---

🛠️ Training Hyperparameters

Parameter	Value
output_dir	./results
per_device_train_batch_size	1
per_device_eval_batch_size	1
gradient_accumulation_steps	4
logging_steps	50
num_train_epochs	3
save_steps	50
save_total_limit	2
eval_strategy	"steps"
eval_steps	50
learning_rate	2e-4
weight_decay	0.01
lr_scheduler_type	"cosine"
warmup_steps	25
fp16	True (Mixed Precision)
push_to_hub	False
report_to	"wandb" (WandB Logging)
run_name	"phi3.5-finetune-nf4"

---

💻 Hardware Used

🖥️ P100 - Kaggle Free GPU

🛠 Framework Versions:

• PEFT 0.14.0

---

🎯 Thank you for using this model! Feel free to reach out for any improvements. 🚀