A hundred million job postings
131.7 million US job postings from 2021 to 2023, held locally, being labelled task by task with an open-weight model on ten cards. The engineering answer turned out to be one line long: make the model think and then emit almost nothing.
Labour statistics about technology adoption are usually built from surveys: a sample of firms, a questionnaire, an occupational code. The postings themselves are a census of what employers asked for, in their own words, and until recently there was no way to read them at that scale.
The shape of the corpus
| Year | Postings |
|---|---|
| 2021 | 44,001,303 |
| 2022 | 48,817,572 |
| 2023 | 38,887,797 |
| Total | 131,706,672 |
330 GB of raw parquet, held locally so that nothing has to leave the machine to be read. The body text was measured on a 20,000-posting sample from 2022:
At roughly four characters to the token in English, the working figure is about 1,000 input tokens per posting, plus a hundred or two of instruction. That number is the whole basis of every estimate below, and it is measured rather than assumed.
Ten workers
The annotator is Qwen3.6 35B-A3B: a mixture of experts with 35B total parameters of which about 3B are active per token, quantised to 4 bits. That combination decides the entire engineering picture.
All 35B of expert weights have to be resident, which is 18 to 20 GB at 4 bits, so a 32 GB card holds one complete instance and no tensor parallelism is needed. But only 3B of weights are read per generated token, so decoding runs at the speed of a small dense model rather than a 35B one. The result is not a cluster. It is ten independent workers, eight V100s on a shared academic cluster plus one Grace-Blackwell desktop and one consumer card, each taking a slice of the data and working through it in order. Aggregate throughput is simply their sum, which is the least clever and most reliable way to parallelise a job whose data splits perfectly.
| Per card, llama.cpp, single stream | Rate | Basis |
|---|---|---|
| Decode, writing the answer | ~100 tok/s | measured, with speculative decoding on |
| Prefill, reading the posting | ~2,000 tok/s | estimated; not the bottleneck |
Concurrency here is one or two streams, not dozens, which means decode never gets the batching relief a high-throughput serving stack would give it. Speculative decoding is what buys the 100 tokens per second, and it is mutually exclusive with large batches, so the low-concurrency path is at least self-consistent. The consequence is that output length is paid for linearly and in full.
The arithmetic
One posting costs its input tokens divided by prefill speed, plus its output tokens divided by decode speed. Reading is fixed at about 1,100 tokens. Writing is whatever the task asks for, and it is the only free variable.
| Task shape | Output tokens | Postings / day | Full corpus |
|---|---|---|---|
| One classification label | ~30 | 800,000 | 5 months |
| Structured JSON, a few fields | ~150 | 320,000 | 13 months |
| Extraction with a short reason | ~500 | 120,000 | 3 years |
| Extraction with full reasoning | ~800 | 80,000 | 4.5 years |
Daily figures include about 20% real-world overhead. The span from the top row to the bottom is a factor of ten, and it is produced entirely by how much the model is asked to write. Nothing else available to tune here moves the number that far: truncating the input to the first 512 tokens saves half the reading on a side of the ledger that was never the bottleneck.
Which leaves two honest positions and no third one. Either the task fits in a short label and the full census is a few months of occupancy, or the task needs the model to explain itself and the study should be designed on a stratified sample of a few hundred thousand, finishing in a day or two. The middle option, full coverage with rich output, is a multi-year commitment of every card in the building.
The answer sheet problem
All of the above concerns the output side. The measurement that changed how I think about this sits on the input side, and I did not go looking for it.
The task was matching postings to the O*NET intermediate work activities, which means presenting the model with 332 candidate activities and asking which apply. Measured against a hosted model on real sampled bodies:
The candidate list is 89% of the prompt. The document being studied is 11% of it. That ratio is paid once per posting per run, and at a hundred million postings it is not a rounding error, it is the study's entire budget.
It is recoverable in this instance. The list is stable across every call, so putting it first as a system message lets the provider's prefix cache absorb it after the first request. But the recovery is a property of this particular deployment rather than of the design, and the general shape does not go away: a classification schema written for human coders, handed to a model verbatim, once per document, forever.
This is why I no longer think the binding constraint on population-scale empirical work is the price of the model. Two studies with identical scientific content can differ tenfold in cost according to how the schema is shaped and where it sits in the prompt, and there is currently no methods literature that would tell an author they had chosen the expensive shape.
The design rule
Stated as bluntly as it deserves:
- Sample before you optimise anything else. A stratified draw of a million postings turns five months into a day. The first question is always whether the research question truly needs the census, and frequently it does not.
- Make the model think and emit almost nothing. Reasoning that stays internal is close to free here; reasoning that gets written down is the ten-times penalty.
- Put the stable part of the prompt first. Anything constant across calls belongs where a cache can find it.
- Run ten thousand first, and look at them. Not to re-estimate the timing, which is already measured, but to fix the real average output length and to check by eye whether the labels are any good. A fast pipeline producing bad labels is the most expensive outcome available.