headroom

What runs on your hardware, and how comfortably.

Estimates from a roofline model calibrated against published measurements. Treat them as a band, not a promise. Model catalog generated .

Setup

4.7M downloads/mo

Most-downloaded first, by Hugging Face downloads over the 30 days before the catalog was generated on 2026-08-21 — a snapshot, not a live count.

US launch list price: $3,999 for the represented machine. Before tax. Checked 2026-08-16.

How gpt-oss ships. Experts at 4-bit, attention untouched.

Dequantizes a quantized checkpoint to FP16 before the matmul, so a card’s low-precision peak is out of reach.

Usage

32K

The window each user gets: prompt plus everything generated so far. Multiplied by the number of users, this is what sizes the KV cache.

1

How many sequences are in flight at once. Each one holds its own cache, so this multiplies memory directly.

8K

How much of that context is already filled when generation starts. Part of the context, not extra — it sets how long you wait for the first token.

KV precision

How many bits each cached token costs. Narrower shrinks the cache and can cost quality, the same trade quantization makes for weights.

1x

How many of this machine to shard the model across. llama.cpp / Ollama runs whole layers on each device in turn, so this buys capacity, not speed — one device’s bandwidth is the ceiling however many you add.

Memory budgetper device

63 GiB / 120 GiB

  • Weights61 GiB
  • KV cache1.1 GiB
  • Overhead0.6 GiB
  • Free57 GiB

Verdicts

Capacity

57 GiBfree

Fits

Room to grow — 128K context would still fit at this concurrency.

Decode

36tok/s per user

Fast

Bound by weight bandwidth. Lower quantization or faster memory is what moves this.

Time to first token

5.7 s

Noticeable

1438 tok/s prompt processing, bound by compute on the linear layers.

What you could do with it

6 of 7 workloads

  • YesInteractive chat35 tok/s, 0.7s to first token on a short message.
  • YesInline code completion0.4s to first token stays inside the window where a suggestion helps.
  • TightCoding agentUsable, but 29s to read a 16K turn against the 47.5K already in the cache is longer than a brisk step allows.
  • NoRAG / document Q&A31s to read a 32K document before answering.
  • TightLong-context analysisUsable, but it holds 128K, short of the 128.5K a full window needs with room to answer in — it reads the 64K it can hold in 83s.
  • YesBatch / offline29 tok/s end to end — latency does not matter here, only the total.
  • YesMulti-user serving4 users at 14 tok/s each, 56 tok/s aggregate, 5.3s to first token.

What this machine should run

Every model in the catalog, at every format and runtime DGX Spark (GB10) can use — ranked.

One person, back and forth, reading as it types.

Ranked by verdict first, then by parameter count, then by how little the weights are compressed, then by decode rate. Parameter count is a proxy for capability and not a measurement of it — Headroom knows what runs, not what is good.

The format that grades best, and the widest of those — narrowing only as far as the verdict requires. Only formats the runtime can actually load.

Swept 105 model and runtime pairings on DGX Spark (GB10) with a 16-bit cache at one user, each graded at the prompt its own workload sends. Change the hardware, the cache or the user count above and the list moves.

Run it

The flags for this exact placement — the layer split, the cache precision and the window above, in each launcher’s own spelling.

llama.cpp (llama-server)

llama-server \
  -m <path to your gpt-oss 120B MXFP4 (expert-only) .gguf> \
  -c 32768 \
  -ngl 37 \
  -ctk f16 -ctv f16
  • -c is the whole cache, not one user's window: llama.cpp hands each of the -np slots n_ctx / n_parallel, so 32,768 tokens for each of 1 is 32,768 here.
  • -ngl 37 is all 36 layers plus one: llama.cpp counts the output tensor a position past the repeating blocks, so 36 would keep the output tensor and leave layer 0 on the host.
  • -m takes a path on your own disk, which no catalog can supply — the placeholder is the one thing here you are meant to replace.

llama-server serves; it does not measure. The llama-bench command below is the measurement form of this same placement.

Flags checked against upstream documentation on 2026-08-01.

Ollama

(
set -e
(set -C; cat > headroom-gpt-oss-120b-mxfp4-expert-only.Modelfile) <<'EOF'
FROM <path to your gpt-oss 120B MXFP4 (expert-only) .gguf>
PARAMETER num_ctx 32768
EOF

ollama create headroom-gpt-oss-120b-mxfp4-expert-only -f headroom-gpt-oss-120b-mxfp4-expert-only.Modelfile && ollama run headroom-gpt-oss-120b-mxfp4-expert-only
)
  • Ollama's Modelfile has no parameter for the GPU layer count — num_ctx and num_predict are documented, num_gpu is not — so the 36-layer split above is the one thing this surface cannot be told. Ollama decides it. Use llama-server if you need to pin it.
  • num_ctx is per request here, unlike llama-server's -c, which is the whole cache across slots.
  • FROM takes a path on your own disk, absolute or relative to the Modelfile.
  • Written to headroom-gpt-oss-120b-mxfp4-expert-only.Modelfile rather than to Modelfile, under set -C, so this cannot overwrite one you already have — and chained with && so a refusal there does not go on to run the old file's settings.

Ollama ships no benchmark client. It runs llama.cpp, so the llama-bench command in this panel measures the same engine on the same GGUF — that is the form to submit.

Flags checked against upstream documentation on 2026-08-01.

llama.cpp (llama-bench)

llama-bench measures; it does not serve. The llama-server command above is the serving form of this same placement.

llama-bench \
  -m <path to your gpt-oss 120B MXFP4 (expert-only) .gguf> \
  -p 8192 \
  -n 0 \
  -ngl 37 \
  -ctk f16 -ctv f16 \
  -o md

llama-bench \
  -m <path to your gpt-oss 120B MXFP4 (expert-only) .gguf> \
  -p 0 \
  -n 128 \
  -d 32640 \
  -ngl 37 \
  -ctk f16 -ctv f16 \
  -o md
  • Two runs, because -p and -n are separate tests: the generation one does not inherit the prompt as cache depth, so a single command would measure decoding from an empty cache. The second run fills the cache to 32,640 tokens and then generates 128 from there, which puts it at the 32,768-token window the decode figure above is charged against.
  • The prompt length is this scenario's own, which is what makes the first result comparable with the figure above rather than with llama-bench's default of 512. The second run generates only 128 tokens, and deliberately: decode is a steady-state per-token rate, so asking for more does not sharpen the measurement — the depth is what decides the number, and a window's worth of generation is minutes of wall clock per repetition, five times over.
  • -ngl 37 is all 36 layers plus one: llama.cpp counts the output tensor a position past the repeating blocks, so 36 would keep the output tensor and leave layer 0 on the host.

Flags checked against upstream documentation on 2026-08-01.

Check these numbers

Every figure on this page is a model, calibrated on two measured machines and asserted within ±30%. Paste your own llama-bench output and see both numbers side by side.

How much room is leftcontext against concurrent users

1 of 56 comfortable

Colour the field by

Headroom left after weights, cache and overhead. The ramp runs between this grid’s own extremes, so it says which way the region falls off rather than by how much — the table has the figures.

  • You are here The context and concurrency the Bench above is set to. The table marks the same cell.
  • less roommore roomEvery cell that fits, graded against the others on this grid rather than against an absolute scale.
  • Comfortable Fits with room, types fast enough to read along, and starts answering promptly. At 1 user, up to 2K of context; the table marks each one.
  • Tight Runs, but near the ceiling, slow to type, or slow to start — the table says which.
  • Will not run Past what this hardware can hold.