Hugging Face
Models
Datasets
Spaces
Posts
Docs
Enterprise
Pricing
Log In
Sign Up
Spaces:
yhavinga
/
dutch-tokenizer-arena
like
8
Running
App
Files
Files
Community
1
Fetching metadata from the HF Docker repository...
55df72d
dutch-tokenizer-arena
/
vocab
/
code_davinci_002
/
__init__.py
xu-song
add grok mixtral
480ae5d
10 months ago
raw
Copy download link
history
blame
Safe
115 Bytes
import
copy
from
vocab.text_davinci_003
import
tokenizer
as
base_tokenizer
tokenizer = copy.copy(base_tokenizer)