RTX PRO 4000 Blackwell SFF vs Arc Pro B70: one long day of tuning Qwen3.8-27B

RTX PRO 4000 Blackwell SFF vs Arc Pro B70: prompt processing and decode, Qwen3.8-27B

RTX PRO 4000 Blackwell SFF vs Arc Pro B70: one long day of tuning Qwen3.8-27B

The short version. I spent 2026-11-19 benchmarking the two GPUs in my MS-02 Ultra, an RTX PRO 4000 Blackwell SFF and an Arc Pro B70 on Thunderbolt 5. Both run the same local coding model, Qwen3.8-27B (Huihui abliterated), for my agent workers. By the end of the day the RTX had changed in three ways:

  • It runs a newer llama.cpp plus one unmerged pull request, #31802. Together they made deep-context decode 31% to 56% faster.
  • It runs a 105 W power limit, set through an in-memory VBIOS override.
  • It runs a +180 MHz core offset.

Against the B70, on the same model with both cards live:

  • The RTX now processes prompts 16% to 42% faster at every context length. It ties on decode at 64K and leads at 128K.
  • The B70 still wins quick, short chats.

The setup

RTX lane B70 lane
GPU NVIDIA RTX PRO 4000 Blackwell SFF, 24 GB GDDR7 Intel Arc Pro B70, 32 GB, over Thunderbolt 5
Power and clocks 105 W (stock 70 W) via an in-memory VBIOS override, core +180 MHz, memory +3599 275 W stock cap, core pinned at 2800 MHz, VRAM at 21,000 Mbps (stock 19,000)
Model file Huihui Qwen3.8-27B abliterated, NVFP4 + MTP head Huihui Qwen3.8-27B abliterated, Q5_K_S + MTP head
Runtime llama.cpp b12532 + PR #31802, CUDA 13.5 llama.cpp b12464, SYCL
Serving 262K context, q4_0 KV cache, MTP draft 2, two slots sharing one cache same

Host: Minisforum MS-02 Ultra, Core Ultra 9 285HX, Debian 13.

What I did, in order

Each test window had a plan and a pick rule written down before the first number came in. Anything I considered deploying had to pass four output-quality gates:

  • an exact-copy check at 33K tokens
  • a needle-in-a-haystack check at 128K
  • a repetition canary
  • a “stop when told” check
  1. llama.cpp b11804 → b12532. Prompt processing rose 32% at 64K and 45% at 128K. Decode was flat, and every quality gate passed. Deployed. This build includes #32407, which reorders flash-attention reads for better L2 cache hits.
  2. Concurrency and speculative decoding. Two slots give 1.44 to 1.50x the throughput of one, so two stay. MTP draft depth 2 stays: draft 3 tied at 64K and lost at 128K.
  3. KV cache and model. My KL-divergence test could not tell q4_0 from q8_0 KV, so q4_0 at 262K stays. Stock and Huihui passed the same quality gates, so Huihui stays.
  4. ubatch, MMQ precision, CUDA graph options, P-core pinning. None of them cleared the bar. A 512 ubatch came closest, at +4.7% prompt speed.
  5. Power, 70 to 120 W in 5 W steps. This is the first chart.
  6. Clock offsets. A VRAM copy test found no errors up to +4800 on the memory offset. +4200 bought about 2% decode, just under my pre-set bar, so memory stays at +3599. Core +180 bought 3% prompt speed and was adopted. +300 added less than 1% more.
  7. Three llama.cpp pull requests. #32520 (kv_rows in the vector FA kernel) and #32610 (few-row NVFP4 MMQ) changed nothing at my settings. #31802 (fused q4_0/q8_0 KV dequantization inside the MMA flash-attention kernels) was the big one.
  8. Quality pass on #31802. Every gate passed. Its 200-token canary output was byte-identical to the build without it, and it uses about 1 GB less VRAM. Deployed.
  9. Head to head with the B70, both lanes live on the same model.

Power: the knee is around 105 W

RTX PRO 4000 SFF decode and prompt processing vs power limit

  • Short-prompt decode goes from 31.5 tok/s at 70 W to 41.0 at 90 W, then 43.6 at 105 W and 44.2 at 120 W.
  • Decode at 64K follows the same curve.
  • Prompt processing keeps climbing: 1112 tok/s at 90 W, 1306 at 120 W.
  • The 120 W step peaked at 78 °C. A 10-minute soak there ran clean: 35 requests, none failed.

I picked 105 W. Past that, every extra 5 W buys under 1% decode, and this card draws all its power through the PCIe slot. A more aggressive fan curve went live mid-sweep, so the steps from 90 W up ran a little cooler than those below.

What the software bought

Prompt-processing gains from b12532 and decode gains from PR #31802

At 105 W 64K prompt / decode (tok/s) 128K prompt / decode (tok/s)
b12532 1229 / 30.1 895 / 22.4
b12532 + #31802 1188 / 39.4 860 / 35.0

Two caveats on #31802:

  • It costs 3 to 4% of prompt speed on a single request. It pays that back with two requests in flight: a 64K read beside a busy slot went from 928 to 1001 tok/s.
  • It is an unreviewed upstream pull request. I keep the plain b12532 image for a one-line rollback.

Head to head with the B70

RTX PRO 4000 SFF vs Arc Pro B70, single and concurrent requests

Test (tok/s unless noted) RTX PRO 4000 SFF Arc Pro B70
Short-prompt decode 44.5 49.0
16K prompt / decode 1757 / 47.5 1236 / 53.2
64K prompt / decode 1265 / 42.2 1009 / 40.4
128K prompt / decode 909 / 37.5 785 / 33.4
Two short jobs at once, combined decode 67 to 72 74
Two 16K reads at once, time to finish both 26.8 s 35.9 s
Two 64K reads at once, time to finish both 120.5 s 180.5 s

Both lanes ran the same requests in alternating order, with nothing else on them.

  • Prompt speed: the RTX reads 42% faster at 16K, 25% at 64K and 16% at 128K.
  • Decode: the B70 is ahead up to 16K, they are about even at 64K, and the RTX leads at 128K. Speculative decoding makes decode depend on how many draft tokens land, and that varied from 48% to 75% per request here. So treat decode gaps under about 10% as a direction, not a number.
  • Two requests at once: whichever request finishes reading first decodes at only 2 to 5 tok/s until the other finishes reading. Staggering big reads matters more than either card.

Two slots, in tokens per second

RTX PRO 4000 SFF vs Arc Pro B70 with two requests at once, tokens per second

Two at once (tok/s) RTX PRO 4000 SFF Arc Pro B70
Two short chats, output from both 67 to 72 74
Two 16K reads, all tokens 1263 943
Two 64K reads, all tokens 1110 741
A chat while the other slot reads 64K 10.3 8.7

One request vs two at once

One request vs two at once on each card, tokens per second

Two at once vs one RTX PRO 4000 SFF Arc Pro B70
Short chats, output tok/s 44.5 → 69.6 (1.57x) 49.0 → 73.9 (1.51x)
16K reads, tokens/s 1133 → 1263 (1.11x) 924 → 943 (1.02x)
64K reads, tokens/s 1139 → 1110 (0.97x) 917 → 741 (0.81x)

“All tokens” counts the prompt and the output of both requests, divided by the time until both finish. Two slots pay off on short chats. On 64K reads the RTX breaks even and the B70 loses ground, so big reads are better staggered than run together.

Against stock and published numbers

Short-prompt decode: our two lanes vs the stock card and published Qwen3.8-27B numbers

Card and setup Short-prompt decode (tok/s) Source
RTX, stock card: 70 W, no offsets, October build, MTP 1 32.8 my bench, 2026-10-26
RTX, today: 105 W, tuned, b12532 + #31802, MTP 2 44.5 this post
RTX, clean llama.cpp master with MTP 45.4 NVIDIA forum
RTX, custom runtime with chained MTP 55.4 NVIDIA forum
RTX, 70 W, DFlash2, 8K prompt 35.0 piszczek.pl
RTX, 120 W + memory and core offsets, DFlash2, 8K prompt 58.5 piszczek.pl
B70, today: llama.cpp SYCL, Q5_K_S, MTP 2 49.0 this post
B70, llama.cpp SYCL, Q4_K_M, MTP 3, ~1K context 38.5 llama.cpp #28721
B70, llama.cpp SYCL, Q4_K_XL, MTP 2 28.4 jonathanmann.tech
B70, vLLM XPU, GPTQ Int4, MTP 4 84.6 jonathanmann.tech
  • Against my own stock-card bench from October, the RTX lane is 36% faster on short prompts. That bench used an older harness with MTP draft 1, so treat the gap as approximate. It comes from power, the memory offset, newer llama.cpp and MTP draft 2 together.
  • On a clean llama.cpp build my RTX lane lands where the forum’s does, 44.5 against 45.4. The bigger published RTX numbers use a different drafter or a chained-MTP runtime. I tested that chained-MTP patch set on this card in October: +31% short decode, but 14 to 16% slower prompt processing and output that no longer matched stock. I did not adopt it.
  • My B70 lane beats every published llama.cpp B70 number I found. vLLM on XPU is much faster on a short prompt, at 84.6, but its author measured it with one request at a time. Two overlapping requests with MTP on could crash it, so it doesn’t fit my two-slot setup today.

Decode as context grows: our two lanes vs the published Arc Pro B70 depth scan

Depth is where the difference shows. The only published depth scan I found is for the B70, in llama.cpp issue #28721. On SYCL it falls from 38.5 tok/s at about 1K to 15.8 at 64K. My B70 lane holds 40.4 at 64K and 33.4 at 128K, and the RTX holds 42.2 and 37.5.

None of these are like-for-like. Quants, prompts, drafters and runtimes all differ, so read the gaps as directions.

What I learned

  • On this card, past about 105 W the watts buy prompt speed and almost no decode.
  • The biggest single win of the day was software: one kernel fix for quantized KV caches in flash attention.
  • A bare docker compose up brought my lane back on the wrong model, because the compose defaults name the stock file. Benchmarks now refuse to run unless the lane reports the model it should serve.
  • Measure the same requests on both cards, on the same model, in alternating order. Anything less compares setups, not cards.

Caveats

  • One card of each kind, one day of runs. Medians of 2 to 3 samples for the deep tests.
  • The RTX runs above its rated power through the PCIe slot. That is my choice and my risk.
  • 31802 is unmerged upstream code.

  • Numbers are for this model, quantization and context policy. A different quant or a single-slot setup will move them.

Sources