File size: 818 Bytes
6bc90ef
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
```python
import torch
from transformers import LlamaTokenizer, LlamaForCausalLM
import transformers
tokenizer = LlamaTokenizer.from_pretrained(
       'ocisd4/openllama-zh',
        add_bos_token=False,
        add_eos_token=False,
        use_auth_token=True,
        use_fast=False)

model = LlamaForCausalLM.from_pretrained('ocisd4/openllama-zh', device_map='auto',use_auth_token=True)


prompt = '關於華碩的傳說'
input_ids = tokenizer(prompt, return_tensors="pt").input_ids

generation_output = model.generate(
    input_ids=input_ids, max_new_tokens=256,
    do_sample=True, top_k=40, top_p=0.95, temperature=0.7, repetition_penalty=1.08,
)

print(tokenizer.decode(generation_output[0]))
```

The is a 7B pretrain model, train from openllama pretrain weight, context size=2048

**keep updating new model**