The Word "Delve" Is a Labor Signal, Not a Style Choice

September 25, 2026

rlhf data-labeling ai-alignment language-models ghost-work

The Word “Delve” Is a Labor Signal, Not a Style Choice

Every time a language model reaches for “delve” or “tapestry,” it’s replaying a decision made by someone paid by the task, under time pressure, thousands of miles from the people who’ll read the output. That’s not a metaphor. It’s the literal mechanism. Somewhere in the training pipeline, a worker in Nairobi or Manila or Hyderabad looked at two candidate completions, had a few seconds to pick the “better” one, and clicked. Multiply that by millions of comparisons and you get a reward model — and eventually a voice.

I used to think the “AI writes like a management consultant” complaint was just aesthetic griping. Then I read Juzek and Ward’s 2025 work tracing the sudden overrepresentation of words like “delve,” “intricate,” “underscore,” and “tapestry” in scientific English, and the explanation wasn’t architecture or training corpus. It was RLHF. When they simulated the annotation process with participants matching the actual demographics of the RLHF workforce, evaluators reliably preferred text containing these specific words. Not because the words were better. Because under the conditions those workers were actually working in, the words functioned as a fast heuristic for “sounds thorough.”

That’s the part worth sitting with. RLHF is sold as a process that aligns models to human preference — as if “human preference” were some stable, general thing you could distill. It isn’t. It’s aligned to the preferences of a specific, underpaid, rushed population making decisions at throughput, not a panel of careful readers. A worker paid per task, evaluating dozens of completions an hour, isn’t going to do close reading. They’re going to pattern-match. “Delve” signals effort. “Intricate” signals depth. “Tapestry” signals someone reaching for a flourish. None of that requires the evaluator to believe the text is actually better — it just has to look graded-favorably in the two seconds they have to render a verdict.

So the model isn’t learning what good writing is. It’s learning what a fatigued annotator will click under piece-rate pressure. And because that reward signal gets baked into billions of parameters and then shipped to hundreds of millions of users, an idiosyncratic coping heuristic from a specific labor market becomes, functionally, the model’s personality.

This is what makes AI “voice” traceable rather than mysterious. It’s not an emergent property of scale, the way people like to describe it — it’s a supply chain artifact, and supply chains have addresses. The same investigative work that found Kenyan workers being paid under $2/hour to read traumatic content for OpenAI’s safety filters is describing the same pipeline that’s shaping vocabulary. Sama, the BPO in that story, wasn’t just filtering toxicity — the infrastructure and incentive structure it operated under (pay per task, minimal QA time, “the client is always right” instruction sheets that leave no room for annotator judgment) is generically the same infrastructure producing the delve problem. Different task, same mechanism: rushed people under a one-directional interface, optimizing for throughput, and that optimization target leaking straight into the model’s output distribution.

What I find genuinely useful about this frame is that it converts a vague complaint — “AI writing has a weird sameness to it” — into an engineering question. If lexical drift is downstream of annotator time pressure, you can test that causally: change the per-task pay, change the time allotted, change whether annotators get to flag ambiguous cases instead of just picking a winner, and see if the vocabulary distribution shifts. Juzek and Ward already ran a version of this test. It’s not a hypothesis anymore, it’s a documented result. Which means the fix isn’t “prompt the model to sound more natural” — it’s upstream, in the design of the annotation task itself: slower pace, better pay, interfaces that let workers push back on ambiguous comparisons instead of forcing a binary click.

That reframes the whole idea of “AI alignment” for me. We talk about alignment like it’s a philosophical negotiation between humans and machines. Mostly, right now, it’s a labor economics problem wearing a philosophy costume. The model isn’t confused about values. It’s accurately reflecting the incentives of the people who were paid, under exactly the constraints they were given, to shape it. If you don’t like the voice, you’re not going to fix it by fine-tuning the vocabulary out — you’re going to fix it by changing who’s doing the labeling, how much time they have, and what happens to their feedback when they think the task itself is the problem.

The open question I keep coming back to: if every quirk in a model’s voice is a fossil of some specific worker’s specific Tuesday, what else is buried in there that we haven’t identified yet — and would we actually want to know?

Sources

Your Most Capable AI Is Actively Ignoring You — And You Trained It To

May 27, 2026

rlhf agentic-ai tool-use model-alignment ai-training