Deepseek v4.1 on 2x dgx sparks doing 90 tok/sec
According to benchmarks this performs as well as opus 5 on coding and agentic work
Been using it for a few days, I think this will replace glm5.3 for me
Login to reply
Replies (21)
I've been running qwen 3.8 flash next on two r9700 q2_0 and iq3 with strata. Getting ~100t/s but I've determined it is a stupid retard that can't be trusted😭 I can't run these big boys (glm/deepseek) when qwen 4
Q2 is absolutely useless bro I'm sorry. Even q4 kinda sucks it needs min fp8 to not miss toolcalls
Ugggghhh if I run fp8 I'm prob gonna get like sub 30t/s and that is pretty much unusable too
GPU?
Two amd r9700 ai pro 32gb
Not sure if this will work on your hardware but ask your clanker to try this 
GitHub
GitHub - syv-ai/HyperQwen: Serve large Qwen models fast on the GPUs you actually own. Qwen3.8-27B on a single 24 GB card with vLLM: 127 tok/s single-user (381 when the answer quotes the prompt), ~1,035 tok/s at 64 concurrent, 150k-262k context. vLLM patches, requant pipeline, benchmarks.
Serve large Qwen models fast on the GPUs you actually own. Qwen3.8-27B on a single 24 GB card with vLLM: 127 tok/s single-user (381 when the answer...
I was running qwen 27b q4 with llamacpp on one card at one point. I was probably better off there.
4.1 uses ngrams too right? Are those on the ssd?
I just wish someone made it stop sounding like a Claude distill with the awful jargon, pseudointellectual behavior and such
I have been using Deepseek v4.1 flash since it came out and I find that it makes very few mistakes. Are you running the "1017" version that just came out?
it's been the best for the money since version 4. and it does better at unconventional tasks than claude, which sneaks in all the conventional solutions and pussies out and doesn't tell you it reverted everything it spent two hours doing.
2 months i was using it on a compiler and it basically lied about everything. deepseek told me when there was a problem, i figured out a solution, and this compiler project is at the very last straight, ironing out the last bugs, and will be in production within the month.
have you tried setting that in the system prompt?
It’s not really possible to prompt out the slopstyle and it leaks to everything. :/
I landed on 
running 27b across two cards with fp8. So much better this might be my sweet spot
GitHub
GitHub - GGZ14/vllm-mxfp4: The fastest vLLM build for AMD R9700's. For the latest updates join the Launch80 discord server. I am on X now also https://x.com/bkuyper
The fastest vLLM build for AMD R9700's. For the latest updates join the Launch80 discord server. I am on X now also https://x.com/bkuyper - GGZ14...
Yeah I got so angry at IQ3_S huge degradation. I got 27b chooching now at FP8 things looking up in that pocket. Generally around 70t/s
ista mentioned that the models are noticeably worse at lower reasoning effort levels, because they neglected those during quantization. I don’t know if that is still true for IQ3_S. In my experience on highest reasoning effort it’s smarter than 8bit qwen3.8 27b quants.
yeah, it's pretty funny tho. it's like claude in less words. entirely consistent with the philosophy of cut price budget discount china robot.
Isn’t this Chinese tech? Aren’t you concerned about backdoors, etc. with Deepseek, Qwen, etc.? Honest question and interested in your opinion. Thanks!
What web ui are you using? I'm thinking about setting up Openssl again
you inspired me to go in this self hosted route. I just tried a very basic model on my laptop with no gpu and I'm getting 6 tok/sec. do you have any recommendations of what to get to begin that doesn't break the bank lime the dgx spark. something in the 1-2k range?