Cost-performance, per gigabyte and per token
What local hardware costs per gigabyte of memory, per gigabyte per second of bandwidth, and per million generated tokens, set against what hosted providers charge for the same open models. Prices move every month, so every one carries the date it was read.
Part of the Local Host Lab notebook. The same arithmetic, with your own numbers, is in the calculator; the hardware itself is laid out on the ladder.
Assumptions
- Electricity at $0.1831 per kWh, the US average residential price for July 2026 (EIA, Electric Power Monthly, table 5.6.A, preliminary, read 2026-10-13).
- Power while decoding taken as the board or system power rating, which overstates it; measured draw is lower on most machines.
- Hardware amortised over 3 years of 8 hours of decoding a day, with no resale value. The rest of the computer, cooling and the operator's time are left out.
- One stream at a time. Decode speed is the bandwidth ceiling times 0.75, the low end of the 0.75 to 0.82 measured for dense models in this notebook (calibration). Batching many requests multiplies throughput; see the last section.
- The reference model is a dense 27B at about 4.25 bits per weight: Qwen3.6-27B IQ4_XS, a 15.44 GB file, as measured in entries 12 and 13. On 16 GB cards it fits only with a quantised KV cache and a moderate context.
The table
| Hardware | PriceUSD, kind | $ / GiB | $ / (GB/s) | 27B decodetok/s, est. | Per 1M tokenshardware + power | At 24 h/day |
|---|---|---|---|---|---|---|
| RTX 5080 16GB16 GiB · 960 GB/s · 360 W | $1,489.99retail | 93 | 1.55 | 47 | $1.41 | $0.73 |
| RTX 3090 24GB24 GiB · 936 GB/s · 350 W | $1,749eBay asking | 73 | 1.87 | 45 | $1.61 | $0.80 |
| RTX 4090 24GB24 GiB · 1,008 GB/s · 450 W | $3,699eBay asking | 154 | 3.67 | 49 | $2.86 | $1.27 |
| Radeon RX 7900 XTX 24GB24 GiB · 960 GB/s · 355 W | $1,050retail | 44 | 1.09 | 47 | $1.10 | $0.63 |
| Intel Arc Pro B60 24GB24 GiB · 456 GB/s · 200 W | $649.99retail | 27 | 1.43 | 22 | $1.39 | $0.77 |
| RTX 5090 32GB32 GiB · 1,792 GB/s · 575 W | $6,300retail | 197 | 3.52 | 87 | $2.63 | $1.10 |
| Tesla V100 32GB PCIe (2017)32 GiB · 900 GB/s · 250 W | $795used | 25 | 0.88 | 44 | $0.87 | $0.48 |
| Instinct MI50 32GB (2018)32 GiB · 1,000 GB/s · 300 W | $520used | 16 | 0.52 | 49 | $0.65 | $0.43 |
| RTX PRO 6000 Blackwell 96GB96 GiB · 1,792 GB/s · 600 W | $22,500eBay asking | 234 | 12.56 | 87 | $8.55 | $3.08 |
| NVIDIA DGX Spark 128GB128 GiB · 273 GB/s · 240 W | $6,950list | 54 | 25.46 | 13 | $17.54 | $6.46 |
| AMD Ryzen AI Max+ 395, 128GB (Framework Desktop)128 GiB · 256 GB/s · 120 W | $3,449list | 27 | 13.47 | 12 | $9.29 | $3.42 |
| Mac Studio M5 Max 128GB (2026)128 GiB · 614 GB/s · 480 W | $5,399list | 42 | 8.79 | 30 | $6.56 | $2.73 |
| Mac mini M5 Pro 64GB (2026)64 GiB · 307 GB/s · 155 W | $3,199list | 50 | 10.42 | 15 | $7.33 | $2.80 |
| Mac Studio M5 Ultra 256GB (2026)256 GiB · 1,200 GB/s · 480 W | $9,499list | 37 | 7.92 | 58 | $5.59 | $2.14 |
Highlighted rows are machines measured in this notebook. Price kinds: list is the vendor's own price, retail a current store listing or street-price index, used an appraiser's range and eBay asking the median of current listings, the two that vary most; each links to its source, with the full description on hover, all read on or just before 2026-10-13. Electricity is between $0.29 and $0.92 per million tokens on every row, so on the cheapest hardware it is a third to a half of the total. The rated power used here overstates it: two V100s drew 262 to 281 W while decoding in entry 03, against a rating of 500 W.