Update README.md

README.md

@@ -1,3 +1,128 @@
----
-license: llama3.1
----
+---
+license: llama3.1
+base_model: meta-llama/Llama-3.1-70B
+model-index:
+- name: Tess-R1-Llama-3.1-70B
+  results: []
+---
+
+<br>
+
+![Tess-R1-Llama-3.1-70B](https://huggingface.co/migtissera/Tess-R1-Llama-3.1-70B/resolve/main/Tess-R1-2.jpg)
+
+<br>
+
+
+Welcome to the Tess-Reasoning-1 (Tess-R1) series of models. Tess-R1 is designed with test-time compute in mind, and has the capabilities to produce a Chain-of-Thought (CoT) reasoning before producing the final output. 
+
+The model is trained to first think step-by-step, and contemplate on its answers. It can also write alternatives after contemplating. Once all the steps have been thought through, it writes the final output.
+
+1. Step-by-step, Chain-of-Thought thinking process. Uses `<thinking>` `</thinking>` tags to indicate when the model is performing CoT.
+2. `<contemplation>` `</contemplation>` tags are used when the model contemplate on its answers.
+3. `<alternatively>` `</alternatively>` tags are used for alternate suggestions.
+4. Finally, `<output>` `</output>` tags are used for the final output
+
+# Important Note:
+In a multi-turn conversation, only the contents between the `<output>` `</output>` tags (discarding the tags) should be carried forward. Otherwise the model will see out of distribution input data and will fail.
+
+
+# Prompt Format
+
+The model uses ChatML prompt format.
+
+# System Message
+
+The system message *must* be the following:
+
+```You are Tess-R1, an advanced AI that was created for complex reasoning. Given a user query, you are able to first create a Chain-of-Thought (CoT) reasoning. Once the CoT is devised, you then proceed to first think about how to answer. While doing this, you have the capability to contemplate on the thought, and also provide alternatives. Once the CoT steps have been thought through, you then respond by creating the final output.```
+
+# Inference
+
+The model was trained mostly with Chain-of-Thought reasoning data, including the XML tags. However, to generalize model generations, some single-turn and multi-turn data without XML tags were also included. Due to this, in some instances the model does not produce XML tags and does not fully utilize test-time compute capabilities. Therefore you should include a try statement in your inference script, and only pass on the contents between the `<output>` `</output>` tags if it's available.
+
+I have included a sample Python script below.
+
+```python
+import torch, json
+from transformers import AutoModelForCausalLM, AutoTokenizer
+import re
+
+
+class LLM(object):
+    def __init__(self, model_path):
+        self.model = AutoModelForCausalLM.from_pretrained(
+            model_path,
+            torch_dtype=torch.bfloat16,
+            device_map="auto",
+            load_in_4bit=True,
+            trust_remote_code=False,
+        )
+
+        self.tokenizer = AutoTokenizer.from_pretrained(
+            model_path, trust_remote_code=False
+        )
+
+        self.terminators = [
+            self.tokenizer.convert_tokens_to_ids("<|im_end|>"),
+            self.tokenizer.convert_tokens_to_ids("<|end_of_text|>"),
+        ]
+
+    def generate_text(self, instruction):
+        tokens = self.tokenizer.encode(instruction)
+        tokens = torch.LongTensor(tokens).unsqueeze(0)
+        tokens = tokens.to("cuda")
+
+        instance = {
+            "input_ids": tokens,
+            "top_p": 1.0,
+            "temperature": 0.75,
+            "generate_len": 1024,
+            "top_k": 50,
+        }
+
+        length = len(tokens[0])
+        with torch.no_grad():
+            rest = self.model.generate(
+                input_ids=tokens,
+                max_length=length + instance["generate_len"],
+                use_cache=True,
+                do_sample=True,
+                top_p=instance["top_p"],
+                temperature=instance["temperature"],
+                top_k=instance["top_k"],
+                num_return_sequences=1,
+                pad_token_id=self.tokenizer.eos_token_id,
+                eos_token_id=self.terminators,
+            )
+        output = rest[0][length:]
+        string = self.tokenizer.decode(output, skip_special_tokens=True)
+        return f"{string}"
+
+    def extract_output(self, text):
+        pattern = r"<output>(.*?)</output>"
+        match = re.search(pattern, text, re.DOTALL)
+        content = match.group(1).strip()
+        return content
+
+
+model_path = "migtissera/Tess-R1-12B"
+
+llm = LLM(model_path)
+
+conversation = f"""<|im_start|>system\nYou are Tess-R1, an advanced AI that was created for complex reasoning. Given a user query, you are able to first create a Chain-of-Thought (CoT) reasoning. Once the CoT is devised, you then proceed to first think about how to answer. While doing this, you have the capability to contemplate on the thought, and also provide alternatives. Once the CoT steps have been thought through, you then respond by creating the final output.<|im_end|>\n<|im_start|>user\n"""
+
+while True:
+    user_input = input("You: ")
+    llm_prompt = f"{conversation}{user_input}<|im_end|>\n<|im_start|>assistant\n"
+    answer = llm.generate_text(llm_prompt)
+
+    try:
+        answer_output = llm.extract_output(answer)
+        print("=" * 132)
+        print(answer_output)
+        conversation = f"{llm_prompt}{answer_output}<|im_end|>\n<|im_start|>user\n"
+    except:
+        print("=" * 132)
+        print(answer)
+        conversation = f"{llm_prompt}{answer}<|im_end|>\n<|im_start|>user\n"
+```