RTX PRO 4000 Blackwell SFF vs Arc Pro B70: one long day of tuning Qwen3.8-27B
The short version. I spent 2026-11-19 benchmarking the two GPUs in my MS-02 Ultra, an RTX PRO 4000 Blackwell SFF and an Arc Pro B70 on Thunderbolt 5. Both run the same local coding model, Qwen3.8-27B (Huihui abliterated), for my agent workers. By the end of the day the RTX had changed in three ways:
- It runs a newer llama.cpp plus one unmerged pull request, #31802. Together they made deep-context decode 31% to 56% faster.
- It runs a 105 W power limit, set through an in-memory VBIOS override.
- It runs a +180 MHz core offset.
Against the B70, on the same model with both cards live:
- The RTX now processes prompts 16% to 42% faster at every context length. It ties on decode at 64K and leads at 128K.
- The B70 still wins quick, short chats.
The setup
| RTX lane | B70 lane | |
|---|---|---|
| GPU | NVIDIA RTX PRO 4000 Blackwell SFF, 24 GB GDDR7 | Intel Arc Pro B70, 32 GB, over Thunderbolt 5 |
| Power and clocks | 105 W (stock 70 W) via an in-memory VBIOS override, core +180 MHz, memory +3599 | 275 W stock cap, core pinned at 2800 MHz, VRAM at 21,000 Mbps (stock 19,000) |
| Model file | Huihui Qwen3.8-27B abliterated, NVFP4 + MTP head | Huihui Qwen3.8-27B abliterated, Q5_K_S + MTP head |
| Runtime | llama.cpp b12532 + PR #31802, CUDA 13.5 | llama.cpp b12464, SYCL |
| Serving | 262K context, q4_0 KV cache, MTP draft 2, two slots sharing one cache | same |
Host: Minisforum MS-02 Ultra, Core Ultra 9 285HX, Debian 13.
What I did, in order
Each test window had a plan and a pick rule written down before the first number came in. Anything I considered deploying had to pass four output-quality gates:
- an exact-copy check at 33K tokens
- a needle-in-a-haystack check at 128K
- a repetition canary
- a “stop when told” check
- llama.cpp b11804 → b12532. Prompt processing rose 32% at 64K and 45% at 128K. Decode was flat, and every quality gate passed. Deployed. This build includes #32407, which reorders flash-attention reads for better L2 cache hits.
- Concurrency and speculative decoding. Two slots give 1.44 to 1.50x the throughput of one, so two stay. MTP draft depth 2 stays: draft 3 tied at 64K and lost at 128K.
- KV cache and model. My KL-divergence test could not tell q4_0 from q8_0 KV, so q4_0 at 262K stays. Stock and Huihui passed the same quality gates, so Huihui stays.
- ubatch, MMQ precision, CUDA graph options, P-core pinning. None of them cleared the bar. A 512 ubatch came closest, at +4.7% prompt speed.
- Power, 70 to 120 W in 5 W steps. This is the first chart.
- Clock offsets. A VRAM copy test found no errors up to +4800 on the memory offset. +4200 bought about 2% decode, just under my pre-set bar, so memory stays at +3599. Core +180 bought 3% prompt speed and was adopted. +300 added less than 1% more.
- Three llama.cpp pull requests. #32520 (kv_rows in the vector FA kernel) and #32610 (few-row NVFP4 MMQ) changed nothing at my settings. #31802 (fused q4_0/q8_0 KV dequantization inside the MMA flash-attention kernels) was the big one.
- Quality pass on #31802. Every gate passed. Its 200-token canary output was byte-identical to the build without it, and it uses about 1 GB less VRAM. Deployed.
- Head to head with the B70, both lanes live on the same model.
Power: the knee is around 105 W

- Short-prompt decode goes from 31.5 tok/s at 70 W to 41.0 at 90 W, then 43.6 at 105 W and 44.2 at 120 W.
- Decode at 64K follows the same curve.
- Prompt processing keeps climbing: 1112 tok/s at 90 W, 1306 at 120 W.
- The 120 W step peaked at 78 °C. A 10-minute soak there ran clean: 35 requests, none failed.
I picked 105 W. Past that, every extra 5 W buys under 1% decode, and this card draws all its power through the PCIe slot. A more aggressive fan curve went live mid-sweep, so the steps from 90 W up ran a little cooler than those below.
What the software bought

| At 105 W | 64K prompt / decode (tok/s) | 128K prompt / decode (tok/s) |
|---|---|---|
| b12532 | 1229 / 30.1 | 895 / 22.4 |
| b12532 + #31802 | 1188 / 39.4 | 860 / 35.0 |
Two caveats on #31802:
- It costs 3 to 4% of prompt speed on a single request. It pays that back with two requests in flight: a 64K read beside a busy slot went from 928 to 1001 tok/s.
- It is an unreviewed upstream pull request. I keep the plain b12532 image for a one-line rollback.
Head to head with the B70

| Test (tok/s unless noted) | RTX PRO 4000 SFF | Arc Pro B70 |
|---|---|---|
| Short-prompt decode | 44.5 | 49.0 |
| 16K prompt / decode | 1757 / 47.5 | 1236 / 53.2 |
| 64K prompt / decode | 1265 / 42.2 | 1009 / 40.4 |
| 128K prompt / decode | 909 / 37.5 | 785 / 33.4 |
| Two short jobs at once, combined decode | 67 to 72 | 74 |
| Two 16K reads at once, time to finish both | 26.8 s | 35.9 s |
| Two 64K reads at once, time to finish both | 120.5 s | 180.5 s |
Both lanes ran the same requests in alternating order, with nothing else on them.
- Prompt speed: the RTX reads 42% faster at 16K, 25% at 64K and 16% at 128K.
- Decode: the B70 is ahead up to 16K, they are about even at 64K, and the RTX leads at 128K. Speculative decoding makes decode depend on how many draft tokens land, and that varied from 48% to 75% per request here. So treat decode gaps under about 10% as a direction, not a number.
- Two requests at once: whichever request finishes reading first decodes at only 2 to 5 tok/s until the other finishes reading. Staggering big reads matters more than either card.
Two slots, in tokens per second

| Two at once (tok/s) | RTX PRO 4000 SFF | Arc Pro B70 |
|---|---|---|
| Two short chats, output from both | 67 to 72 | 74 |
| Two 16K reads, all tokens | 1263 | 943 |
| Two 64K reads, all tokens | 1110 | 741 |
| A chat while the other slot reads 64K | 10.3 | 8.7 |
One request vs two at once

| Two at once vs one | RTX PRO 4000 SFF | Arc Pro B70 |
|---|---|---|
| Short chats, output tok/s | 44.5 → 69.6 (1.57x) | 49.0 → 73.9 (1.51x) |
| 16K reads, tokens/s | 1133 → 1263 (1.11x) | 924 → 943 (1.02x) |
| 64K reads, tokens/s | 1139 → 1110 (0.97x) | 917 → 741 (0.81x) |
“All tokens” counts the prompt and the output of both requests, divided by the time until both finish. Two slots pay off on short chats. On 64K reads the RTX breaks even and the B70 loses ground, so big reads are better staggered than run together.
Against stock and published numbers

| Card and setup | Short-prompt decode (tok/s) | Source |
|---|---|---|
| RTX, stock card: 70 W, no offsets, October build, MTP 1 | 32.8 | my bench, 2026-10-26 |
| RTX, today: 105 W, tuned, b12532 + #31802, MTP 2 | 44.5 | this post |
| RTX, clean llama.cpp master with MTP | 45.4 | NVIDIA forum |
| RTX, custom runtime with chained MTP | 55.4 | NVIDIA forum |
| RTX, 70 W, DFlash2, 8K prompt | 35.0 | piszczek.pl |
| RTX, 120 W + memory and core offsets, DFlash2, 8K prompt | 58.5 | piszczek.pl |
| B70, today: llama.cpp SYCL, Q5_K_S, MTP 2 | 49.0 | this post |
| B70, llama.cpp SYCL, Q4_K_M, MTP 3, ~1K context | 38.5 | llama.cpp #28721 |
| B70, llama.cpp SYCL, Q4_K_XL, MTP 2 | 28.4 | jonathanmann.tech |
| B70, vLLM XPU, GPTQ Int4, MTP 4 | 84.6 | jonathanmann.tech |
- Against my own stock-card bench from October, the RTX lane is 36% faster on short prompts. That bench used an older harness with MTP draft 1, so treat the gap as approximate. It comes from power, the memory offset, newer llama.cpp and MTP draft 2 together.
- On a clean llama.cpp build my RTX lane lands where the forum’s does, 44.5 against 45.4. The bigger published RTX numbers use a different drafter or a chained-MTP runtime. I tested that chained-MTP patch set on this card in October: +31% short decode, but 14 to 16% slower prompt processing and output that no longer matched stock. I did not adopt it.
- My B70 lane beats every published llama.cpp B70 number I found. vLLM on XPU is much faster on a short prompt, at 84.6, but its author measured it with one request at a time. Two overlapping requests with MTP on could crash it, so it doesn’t fit my two-slot setup today.

Depth is where the difference shows. The only published depth scan I found is for the B70, in llama.cpp issue #28721. On SYCL it falls from 38.5 tok/s at about 1K to 15.8 at 64K. My B70 lane holds 40.4 at 64K and 33.4 at 128K, and the RTX holds 42.2 and 37.5.
None of these are like-for-like. Quants, prompts, drafters and runtimes all differ, so read the gaps as directions.
What I learned
- On this card, past about 105 W the watts buy prompt speed and almost no decode.
- The biggest single win of the day was software: one kernel fix for quantized KV caches in flash attention.
- A bare
docker compose upbrought my lane back on the wrong model, because the compose defaults name the stock file. Benchmarks now refuse to run unless the lane reports the model it should serve. - Measure the same requests on both cards, on the same model, in alternating order. Anything less compares setups, not cards.
Caveats
- One card of each kind, one day of runs. Medians of 2 to 3 samples for the deep tests.
- The RTX runs above its rated power through the PCIe slot. That is my choice and my risk.
-
31802 is unmerged upstream code.
- Numbers are for this model, quantization and context policy. A different quant or a single-slot setup will move them.
Sources
- llama.cpp pull requests: #31802, #32520, #32610, #27173, #32407
- llama.cpp issue #28721, Arc Pro B70 decode at depth on SYCL and Vulkan
- NVIDIA Developer Forums, Qwen3.8-27B at 256K on a 24 GB Blackwell target GPU
- Jonathan Mann, How Well Does Qwen3.8-27B Run on the Intel Arc Pro B70?
- Michał Piszczek, RTX PRO 4000 Blackwell SFF at 120 W
- My earlier posts on RTX PRO 4000 SFF power scaling and the Arc Pro B70 on Linux over Thunderbolt 5