Self-driving laboratories
A definition-first census of laboratory automation: 149,364 papers pulled by citation and keyword expansion, 125,757 of them classified one by one against a two-axis test, 1,909 accepted as relevant. Ninety-seven percent of what it finds never calls itself a self-driving laboratory.
The substantive question is which laboratory experiments are already automated, at the level of the unit operation rather than the field. Answering it needs a corpus of work that automates experiments, and that corpus turns out to be very hard to assemble by the usual means.
The design
Papers are judged against a definition, not a name. The test has two axes, how much of the experimental loop is under software control and how much of it is physically automated, and a paper is accepted only if it clears a threshold on both. Nothing in the test asks whether the authors describe their system as a self-driving laboratory.
Around that classifier sits a fixed-point iteration. A round takes the papers accepted last time as seeds, expands from them along two legs, classifies everything new, and hands its acceptances to the next round:
- Citation expansion. Every backward reference and every forward citation of every seed, uncapped.
- Keyword expansion. An n-gram gate run mechanically over the seed set, with every parameter frozen from the previous round and a regression check that has to reproduce the earlier round's query count before the new queries are allowed through.
Classification runs on gpt-5-mini through the batch API with reasoning on, and the round-2 requests are byte-identical in structure to round-1's, verified by hashing the prompt framework. Candidates are papers that are unclassified, carry an abstract of at least 200 characters, and are not already seeds, so the two rounds share no classified document at all.
What a census yields
Round one built a pool of 30,422, classified 27,459 and accepted 994. Round two used those 994 as seeds, pulled 118,942 more papers, classified 98,298 and accepted a further 915. De-duplicated, the pool is 138,967 logical papers and the two rounds accept 1,709 between them.
The ratio is the part worth looking at directly.
Read the other way: to find 975 papers, the design paid to read 98,298. That is a bad trade under any budget where reading is expensive, and a perfectly ordinary one where it is not. The entire method rests on that inversion.
The silent majority
Of 845 de-duplicated relevant papers, 820 contain no self-driving-laboratory brand phrase at all. The phrase list was built mechanically, as the 42 surface variants of the 34 canonical queries the round-1 keyword gate produced, rather than assembled by hand to suit the result. A hand-built, more generous list still leaves 79% silent.
These are not marginal papers. Run-to-run control in semiconductor fabrication. Reinforcement learning for single-molecule manipulation. Nonlinear model-predictive control of fed-batch fermentation. Automated stem-cell production. Evolutionary algorithms in photonics. The technique is spreading across fields under local names, and a review organised around the term cannot reach any of it.
The two recall channels are close to complementary: citation expansion alone found 399, generalised non-brand keywords alone found 337, and only 84 were reachable both ways. Drop either leg and roughly half the accepted set disappears.
Per-leg quality runs the other way from per-leg volume. The keyword leg contributed 93,971 net new papers at an 18% SDL-related rate; backward citations contributed 15,371 at 20%; forward citations contributed 9,600 at 34%. The cheapest leg to run is the worst one, which is also the only leg a conventional search uses.
Knowing when to stop
A fixed-point iteration needs a stopping rule, and the absolute counts actively mislead here: round two accepted 915 papers against round one's 994, which looks like no progress toward convergence at all. Two scale-invariant diagnostics say otherwise.
| Diagnostic | Round 1 | Round 2 |
|---|---|---|
| Relevant accepted per seed | 6.04 | 0.98 |
| Forward citations already in pool | n/a | 74% |
| New papers per backward citation edge | n/a | 1,423 of 30,295 |
Yield per seed fell roughly sixfold. Three quarters of the forward citation neighbourhood was already inside the pool, and the backward neighbourhood was almost entirely exhausted. The absolute count stayed high only because the seed base grew sevenfold, from 143 to 994, and because the keyword leg inflated its query set from 34 to 411.
The noise was duplicates, not a loose threshold
That keyword inflation looked at first like a threshold set too generously. It was not.
Running a null model over the round-2 gate showed that document frequency is the only support measure that stays put as the seed set grows: the FDR-controlled threshold sits at 2 whether the seed set is 143 papers or 1,137, while seed count and log-likelihood ratio both drift. And the noisy queries that survived were not statistical accidents. A phrase like foster knowledge exchange reached a document frequency of 9 because those nine records are the same paper: the journal version, its preprint, and seven supplementary files.
Across the whole pool, 149,364 rows collapse to 138,967 logical papers, about 7%. No threshold can remove that kind of noise, because the noise is real repetition of a real phrase. The fix is de-duplication before the support count is taken, plus a bootstrap stability requirement, which is the one criterion that separated genuine queries (selected in every resample) from noise (0.35 to 0.76).
This is the kind of failure I expect to be generic to census-scale corpus work. The pipeline was statistically sound and the defect was in the record layer underneath it, where nothing in the method description would ever have looked.