Local Host Lab · Tools

Sources and further reading

Every public number on the ladder, cost-performance and calculator pages, with where it came from and when it was read. Measured numbers from this notebook are not listed here; each links to its own entry.

All sources read 2026-10-13 unless a date says otherwise

Part of the Local Host Lab notebook, for the ladder, the cost-performance table and the calculator.

Benchmark results

• llama.cpp discussion 4167: Performance of llama.cpp on Apple Silicon M-series Llama 2 7B at F16, Q8_0 and Q4_0, pp512 and tg128 per chip, with stated bandwidth. Body last updated 2026-10-01. M1 to M4 rows use build 8e672ef (November 2023), M5 rows build c1d0e7a (August 2026).
• llama.cpp discussion 15013: Performance of llama.cpp on NVIDIA CUDA Llama 2 7B Q4_0, tg128, with and without flash attention, one row per submitted GPU. Last updated 2026-10-01.
• llama.cpp discussion 15021: Performance of llama.cpp on AMD ROCm (HIP) Same protocol for AMD GPUs and APUs. Last updated 2026-10-02.
• llama.cpp discussion 10879: Performance of llama.cpp with Vulkan Same model across vendors on the Vulkan backend. Last updated 2026-10-10. Used for the RX 7900 XTX and MI50 Vulkan figures.
• llama.cpp discussion 15396: guide to running gpt-oss with llama.cpp gpt-oss-20b and gpt-oss-120b in MXFP4, tg128, on NVIDIA and Apple hardware. Created 2025-08-18; results from builds 6123 to 6316 (August 2025).
• llama.cpp benches/dgx-spark/dgx-spark.md DGX Spark results for six models at increasing context depth, tg32, plus batched decoding. Run 2026-02-05, build 7941, CUDA 13.0. Linked from discussion 16578.
• kyuz0, AMD Strix Halo toolboxes: benchmark results Ryzen AI Max+ 395 with 128 GB, many models and five backends (ROCm and Vulkan variants), pp512 and tg128. Generated 2026-05-18, llama.cpp builds 9187 and 9193.
• LMSYS, NVIDIA DGX Spark in-depth review Launch-day results under Ollama and SGLang, 2025-10-13. Cited only for the Ollama gpt-oss-20b figure, to show the backend effect.

Hardware specifications

• NVIDIA product pages: RTX 5090, RTX 5080, RTX 5060 family, RTX 3090, RTX 3080, RTX PRO 6000 Blackwell Memory size and board power. The GeForce pages do not list bandwidth; bandwidth for the RTX 50 and 40 series is from the Wikipedia RTX 50 series and RTX 40 series tables, and the RTX 3080's 760 GB/s is its 320-bit bus at 19 Gbps.
• NVIDIA V100 datasheet and NVIDIA A100 datasheet V100: 900 GB/s, 250 W (PCIe). A100 80GB: 1,935 GB/s PCIe, 2,039 GB/s SXM.
• NVIDIA DGX Spark hardware documentation 128 GB LPDDR5X, 273 GB/s, 240 W power adapter.
• AMD product pages: Radeon RX 7900 XTX, Radeon AI PRO R9700, Instinct MI50 32GB, Ryzen AI Max+ 395 AMD does not state the Ryzen AI Max+ 395's bandwidth; 256 GB/s is its 256-bit LPDDR5X-8000 bus, the figure Framework also gives.
• Intel Arc Pro B60 specifications 24 GB, 456 GB/s, 200 W.
• Apple: Mac Studio (2026) specifications, Mac mini (2026) specifications, Mac Studio (2025) technical specifications Bandwidth depends on the chip variant: M5 Max 460 GB/s (32-core GPU) or 614 GB/s (40-core GPU, required above 36 GB); M5 Ultra 1.2 TB/s; M4 Max 410 or 546 GB/s; M3 Ultra 819 GB/s. Power is the whole machine's maximum continuous rating.

Model architectures and formats

• The config.json and model card of each calculator preset, on Hugging Face: Qwen/Qwen3.8-27B, Qwen/Qwen3.6-35B-A3B, Qwen/Qwen3.5-122B-A10B, Qwen/Qwen3-30B-A3B, Qwen/Qwen3-235B-A22B, openai/gpt-oss-20b, openai/gpt-oss-120b, meta-llama/Llama-3.1-8B-Instruct, meta-llama/Llama-3.1-70B-Instruct, google/gemma-3-27b-it, mistralai/Mistral-Small-3.2-24B-Instruct-2506, zai-org/GLM-4.5-Air, zai-org/GLM-5.3-Flash, deepseek-ai/DeepSeek-V4-Flash, deepseek-ai/DeepSeek-V3.2, moonshotai/Kimi-K2-Instruct. Layers, KV heads, head dimension, attention pattern, and total and active parameters from the model card. The gated Meta and Google repositories were read through byte-identical public mirrors.
• llama.cpp tools/quantize/quantize.cpp Bits per weight for the IQ formats as stated; for the k-quant mixes, derived from the Llama-3-8B file sizes listed there.
• unsloth/Qwen3.6-27B-GGUF File sizes, including the 15.44 GB IQ4_XS file used as the 27B reference.

Prices

• videocardprices.com GPU price index Street prices for the RTX 5090 ($6,300), RTX 5080 ($1,489.99) and RX 7900 XTX ($1,050), as of 2026-10-12.
• vramglass.com Median eBay asking prices: RTX 3090 $1,749 (110 listings), RTX 4090 $3,699 (73), RTX PRO 6000 $22,500 (11). Asking prices, not completed sales.
• amcompute.com GPU appraisals Used price ranges over the 12 months to August 2026: V100 PCIe 32GB $760 to $830, MI50 32GB $500 to $540.
• Apple Store configuration pages for the Mac Studio M5 Max 128GB ($5,399), Mac Studio M5 Ultra 256GB ($9,499) and Mac mini M5 Pro 64GB ($3,199).
• Framework Desktop DIY configurator Ryzen AI Max+ 395 with 128 GB at $3,449, shown out of stock.
• Newegg listing for the ASRock Arc Pro B60 24GB $649.99.
• TechnoEdge, 2026-10-08 (in Japanese) DGX Spark 128GB raised from $4,699 to $6,950 on NVIDIA's own store, shown out of stock; a 64 GB version from partners at $4,999 from 2026-10-23. Launch price was $3,999.
• Hosted prices: OpenRouter model list and its per-model endpoint lists (Qwen3.8-27B, gpt-oss-120b, gpt-oss-20b), DeepSeek API pricing, Alibaba Cloud Model Studio pricing, Z.ai pricing Medians are over all providers listed for the model; OpenRouter's headline price is one provider's.
• US EIA, Electric Power Monthly, table 5.6.A Average residential price, July 2026 (preliminary): 18.31 cents per kWh for the US, 19.22 for Illinois. Released 2026-09-24.

Further reading

• kipply, Transformer inference arithmetic (2022) Why decoding is memory-bound, and the KV cache and FLOP arithmetic behind the calculator.
• Horace He, Making deep learning go brrrr from first principles (2022) Compute-bound, memory-bound and overhead-bound work, the three regimes that show up in the calibration table.
• Databricks, LLM inference performance engineering: best practices (2023) Model bandwidth utilisation, batching, and the latency and throughput trade.
• LocalScore Crowd-sourced local benchmark results by accelerator. Undated, and some recent cards show results far below every other source, so it is not used for any number here.