Fengyuan Zhu

Research Assistant Professor  ·  Siebel School of Computing and Data Science  ·  University of Illinois Urbana-Champaign

I work on the cost side of AI: what a real task costs to run on a model, and what becomes possible in empirical research once that cost falls. Every page below carries the measurements it is built on.

01The cost side of AI

The price of the model is not the bottleneck

Cost is settled after the model is chosen: by how the hardware is scheduled, by how many times the same context gets read again, and by a per-token price that ranks models in the wrong order. Three notes, all measured rather than quoted.

How far behind are open weights? Twenty months of a public model leaderboard, rebuilt from web archives and rescored on a single ruler. Six points behind at the top, with no trend. At the price anyone actually pays, an open-weight model strictly dominates 313 of the 398 models on sale. What running a model actually costs Three field notes on utilisation, on repetition inside an agent loop, and on why dollars per million tokens picks the wrong model. Same workload, same two 2017 cards, same bill: 27.55 tokens per second at one concurrent request, 435.62 at 128. Serving modern models on 2017 silicon Four experiments asking how far nine-year-old GPUs go against models released this year, run on a shared academic cluster. A 744B mixture-of-experts at one bit on eight V100s. 720p video, with audio, in five minutes on two of them.
02Reading the whole population

Sampling and keyword search were workarounds for expensive reading

Annotation now costs little enough to read a population document by document. Once that is true, the two oldest design assumptions in empirical work, draw a sample and search for a keyword, stop being necessary.

Reading the whole population Four populations labelled in full rather than sampled, what full coverage costs on real hardware, and where that cost actually goes. 149,364 papers, 100M+ job postings, and US computing dissertations. Of the laboratory-automation papers the census accepts, 97% never call themselves that. Where CS graduates go US computing dissertations linked person by person to what their authors did next. The career panel does not know what anyone studied; the dissertations do. Shared privately, password required. Compare fields and cohorts The same linkage, driven by hand. Pick a degree, a graduating cohort and a field, and read off where those graduates went, what the panel estimates they were paid, and how many were still in academia five years on. Shared privately, password required.
03Three dimensions and vision

The same cost question, in a different medium

What a real place costs to reconstruct from public data, to stream to an ordinary browser, and to generate from nothing.

The Illinois main quad, from lidar A 450 by 600 metre crop of the main quad as a gaussian splat you can fly around in the browser. Shape comes from a public airborne lidar survey, not from photographs of the site. 17.9 million gaussians, cut into 285 tiles and streamed by distance. Rebuilding a campus from public data How that model is built and streamed. Public lidar and public aerial photography only, no Street View, which is a design constraint rather than a disclaimer. Density came from cutting the scene into 20 pieces, not from the densification knob, which bought 37% finer spacing for 5× the gaussians. 167 MB ships. What a finished second of generated film costs A 43 second animated short, picture and sound generated together, made in a week of evenings on one desk machine. 8,779 seconds of compute for 41 seconds of picture on the pass where nothing went wrong. Rewriting one performance cost three quarters of that again.
04Getting agents to people

The barrier to an agent is the installation, not the model

An agent can do far more than a chat window because it has a real computer. Chat reached everyone because it is a URL; agents have not, because each one has to be installed, keyed and driven from a terminal. So put the computer in the cloud and hand out a link.

Agent Hub: one link into the agent world A cloud workspace for AI agents on top of JupyterHub. One workspace per person, nothing to install, files that persist, work that outlives the tab. Built from open-source components, with a seven-minute demo recorded live and a short white paper. Every credential lives in one gateway and every call lands in one ledger; a container holds nothing but a token that says whose it is. 14 checks run against the live site.
Who is writing this

I am a Research Assistant Professor in the Siebel School of Computing and Data Science at the University of Illinois Urbana-Champaign.

I work on the adoption of AI. What these models can do has run well ahead of what anyone actually uses them for, and that gap is the thing I want to explain. A large part of it is cost. So the questions I keep asking are what a real task costs to run, how far that cost can be pushed down, and what becomes possible once it falls. The part worth caring about is not the same work done more cheaply. It is the work nobody attempted at the old price.

Those are questions about numbers rather than opinions, which is why most of what I do is measurement. I take a cost that usually gets quoted secondhand, run it on hardware and data I can account for, and report what came out, including when the careful method loses to the naive one.

I came to this from the interface side, and it turned out to be the same question. My doctorate at the University of Toronto was on interaction between a phone and a headset, and the plain finding was that people did not want the headset, because the phone already did the job. What stops a good technology from being used is the thing I keep running into.