3 3

Sylvain Delahaies

sylvain471

https://sdelahaies.fr

sylvain471

AI & ML interests

None yet

Recent Activity

updated a model about 4 hours ago

sylvain471/Llama-3.2-1B-math-orca-qlora-10k-ep1-merged

published a model about 4 hours ago

sylvain471/Llama-3.2-1B-math-orca-qlora-10k-ep1-merged

upvoted an article 8 days ago

Run ComfyUI workflows for free on Spaces

View all activity

Organizations

sylvain471's activity

updated a model about 4 hours ago

sylvain471/Llama-3.2-1B-math-orca-qlora-10k-ep1-merged

Updated about 4 hours ago

published a model about 4 hours ago

sylvain471/Llama-3.2-1B-math-orca-qlora-10k-ep1-merged

Updated about 4 hours ago

upvoted an article 8 days ago

Article

Run ComfyUI workflows for free on Spaces

Jan 14, 2024

• 50

reacted to Jaward's post with 👍 22 days ago

Post

1869

minimal single script implementation of knowledge distillation in LLMs. In this implementation, we use GPT-2 (124M) as student model and GPT-2 Medium (340M) as teacher via reverse Kullback-Leibler (KL) divergence, trained on a small chunk of openwebtext.

Code: https://github.com/Jaykef/ai-algorithms/blob/main/llm_knowledge_distillation.ipynb