locally running ideas

#20
by gopi87 - opened

hey guys feel free to share your set up to run deepseek locally.

please list out:
*Total cost
*cpu,gpu,board etc..
*qunat version examble (q6 )
*t/sec

  • your feedback on your setup

I also want to run on my notebook. any configuration needed? or need the server grade machine to run it.

@henrycwf i ust bought the new server around 1K$

q2 ud k xl dynamic quant

256gp ddr4 ram
2*xeon 2680 v4

added two oudated gpu
gt 1030
gt 730

getting only cpu mode ~1.8 t/sec
with gpu also same since both gpu is worst and outdated

i need to spend bit more money like ~500$ for gpu to get proper speed to work with coding task.

@gopi87 , Thanks for your info.

I tried to search the host from Azure VMs. But, only few modals include GPU. Not sure what is the good choice for hosting the model.

Sign up or log in to comment