Below is the code I am using to test this models capability. However I can't seem to get it to work as the output is just so bizarre. Does anyone know what I am doing wrong?

from transformers import pipeline, AutoTokenizer, AutoModelForCausalLM
import random

Initialize the model and tokenizer for DeepSeek

tokenizer = AutoTokenizer.from_pretrained("deepseek-ai/DeepSeek-R1-Distill-Qwen-1.5B")
model = AutoModelForCausalLM.from_pretrained("deepseek-ai/DeepSeek-R1-Distill-Qwen-1.5B")

Use pipeline for text generation

pipe = pipeline("text-generation", model=model, tokenizer=tokenizer)

Function to generate plausible fake answers using DeepSeek

def generate_fake_answers(question, correct_answer):
# Define the prompt to generate answers
prompt = f"Question: {question}\nCorrect Answer: {correct_answer}\nGenerate 3 plausible incorrect answers. Do not provide reasoning"

# Generate text using the pipeline
generated_texts = pipe(prompt, max_length=100, num_return_sequences=1)

# Extract fake answers from the generated text
generated_text = generated_texts[0]['generated_text']

# Process the output to extract relevant answers (stop at first instance of correct answer)
fake_answers = generated_text.split("\n")[1:]  # Extract answers after "Question: <question>"
fake_answers = [answer.strip() for answer in fake_answers if answer.strip() and answer != correct_answer]

# Return only the first 3 fake answers
return fake_answers[:3]

Example question-answer pairs

qa_pairs = [
("Who was David?", "A shepherd boy"),
("Who was Goliath?", "A giant warrior"),
("What did David use to defeat Goliath?", "A sling and a stone"),
("Why did David fight Goliath?", "To defend his people"),
("How many stones did David pick up before fighting Goliath?", "Five"),
("What did Goliath wear for protection?", "Armor"),
("Who was the king of Israel during the battle with Goliath?", "King Saul"),
("What happened to Goliath after David hit him with the stone?", "He fell down and died"),
("What did David do after Goliath fell?", "Cut off Goliath's head"),
("What lesson can we learn from the story of David and Goliath?", "Courage and faith can help us overcome big challenges"),
]

Generate fake answers for each question

for question, correct_answer in qa_pairs:
fake_answers = generate_fake_answers(question, correct_answer)

# Add the correct answer to the list of fake answers
all_answers = [correct_answer] + fake_answers
random.shuffle(all_answers)

# Print the question and answers
print(f"Question: {question}")
for ans in all_answers:
    print(f"- {ans}?")
print()

Output:
Question: What happened to Goliath after David hit him with the stone?

• He fell down and died?
• Generate 3 plausible incorrect answers. Do not provide reasoning, but explain why each incorrect answer is wrong.?
• The user is a middle school student trying to understand the answer to the question. They are looking for clear explanations but may not have a lot of time to think through the problem.?
• Correct Answer: He fell down and died?

Question: What did David do after Goliath fell?

• Generate 3 plausible incorrect answers. Do not provide reasoning. Just answer the questions. Make sure the answers are correct.?
• Correct Answer: Cut off Goliath's head?
• Cut off Goliath's head?
• Okay, so the question is about what David did after Goliath fell. The correct answer is "Cut off Goliath's head." Now, I need to come up with three plausible incorrect answers. Hmm, let's think about what could be wrong?

Question: What lesson can we learn from the story of David and Goliath?

• Courage and faith can help us overcome big challenges?
• ?
• Correct Answer: Courage and faith can help us overcome big challenges?
• Generate 3 plausible incorrect answers. Do not provide reasoning.?

...