67 3 10

Régis Pierrard

regisss

regisss

AI & ML interests

None yet

Recent Activity

new activity 17 days ago

regisss/bridgetower-newyorker-a100-8x:Adding `safetensors` variant of this model

upvoted an article 21 days ago

Introducing multi-backends (TRT-LLM, vLLM) support for Text Generation Inference

posted an update about 2 months ago

Nice to see day 1 support of Falcon 3 on Gaudi with Optimum Habana! 👉 https://www.intel.com/content/www/us/en/developer/articles/technical/intel-ai-solutions-support-falcon-3-fdn-models.html

View all activity

Organizations

regisss's activity

New activity in regisss/bridgetower-newyorker-a100-8x 17 days ago

Adding `safetensors` variant of this model

#2 opened 19 days ago by

SFconvertbot

upvoted an article 21 days ago

Article

Introducing multi-backends (TRT-LLM, vLLM) support for Text Generation Inference

22 days ago

• 63

posted an update about 2 months ago

Post

996

Nice to see day 1 support of Falcon 3 on Gaudi with Optimum Habana!

👉 https://www.intel.com/content/www/us/en/developer/articles/technical/intel-ai-solutions-support-falcon-3-fdn-models.html

New activity in Habana/mamba 2 months ago

Upload 2 files

#2 opened 2 months ago by

zzhang37

New activity in AIEnergyScore/README 2 months ago

Expanding AI Energy Score Research to other GPUs

#1 opened 2 months ago by

PranavViswanath

New activity in Habana/mamba 2 months ago

Upload 2 files

#1 opened 2 months ago by

zzhang37

updated a model 2 months ago

Habana/stable-diffusion-xl

Updated Dec 2, 2024

updated a dataset 4 months ago

regisss/benchmarks

Preview • Updated Oct 21, 2024 • 18

reacted to onekq's post with 🔥 4 months ago

Post

1862

I'm now working on finetuning of coding models. If you are GPU-hungry like me, you will find quantized models very helpful. But quantization for finetuning and inference are different and incompatible. So I made two collections here.

Inference (GGUF, via Ollama, CPU is enough)
onekq-ai/ollama-ready-coding-models-67118c3cfa1af2cf04a926d6

Finetuning (Bitsandbytes, QLora, GPU is needed)
onekq-ai/qlora-ready-coding-models-67118771ce001b8f4cf946b2

For quantization, the inference models are far more popular on HF than finetuning models. I use https://huggingface.co/QuantFactory to generate inference models (GGUF), and there are a few other choices.

But there hasn't been such a service for finetuning models. DIY isn't too hard though. I made a few myself and you can find the script in the model cards. If the original model is small enough, you can even do it on a free T4 (available via Google Colab).

If you know a (small) coding model worthy of quantization, please let me know and I'd love to add it to the collections.

posted an update 4 months ago

Post

1415

Interested in performing inference with an ONNX model?⚡️

The Optimum docs about model inference with ONNX Runtime is now much clearer and simpler!

You want to deploy your favorite model on the hub but you don't know how to export it to the ONNX format? You can do it in one line of code as follows:

from optimum.onnxruntime import ORTModelForSequenceClassification

# Load the model from the hub and export it to the ONNX format
model_id = "distilbert-base-uncased-finetuned-sst-2-english"
model = ORTModelForSequenceClassification.from_pretrained(model_id, export=True)

Check out the whole guide 👉 https://huggingface.co/docs/optimum/onnxruntime/usage_guides/models