Hugging Face
Models
Datasets
Spaces
Posts
Docs
Enterprise
Pricing
Log In
Sign Up
Spaces:
yhavinga
/
dutch-tokenizer-arena
like
7
Running
App
Files
Files
Community
1
Fetching metadata from the HF Docker repository...
c78da21
dutch-tokenizer-arena
3 contributors
History:
79 commits
yhavinga
Add Llama tokenizer creation for Dutch, English, Code, Markdown and TeX.
c78da21
9 months ago
css
add compression leaderboard
9 months ago
images
update
over 1 year ago
js
add grok mixtral
10 months ago
patcher
add compression leaderboard
9 months ago
stats
Add Llama tokenizer creation for Dutch, English, Code, Markdown and TeX.
9 months ago
utils
Add Llama tokenizer creation for Dutch, English, Code, Markdown and TeX.
9 months ago
vocab
Add Llama tokenizer creation for Dutch, English, Code, Markdown and TeX.
9 months ago
.gitattributes
Safe
2.32 kB
add grok mixtral
10 months ago
.gitignore
Safe
171 Bytes
update
over 1 year ago
README.md
Safe
25.5 kB
Hack some Dutch tokenizers into it
about 1 year ago
app.py
Safe
565 Bytes
Add Llama tokenizer creation for Dutch, English, Code, Markdown and TeX.
9 months ago
app_compression.py
Safe
4.75 kB
Add Llama tokenizer creation for Dutch, English, Code, Markdown and TeX.
9 months ago
app_playground.py
Safe
9.81 kB
update
9 months ago
config.py
Safe
1.72 kB
Add Llama tokenizer creation for Dutch, English, Code, Markdown and TeX.
9 months ago
examples.py
Safe
3.16 kB
add compression leaderboard
9 months ago
requirements.txt
Safe
96 Bytes
Pin protobuf dep
11 months ago
util.py
Safe
8.15 kB
Hack some Dutch tokenizers into it
about 1 year ago
