AI Isn't Learning Our Values. We're Learning Its.

September 8, 2026

ai-ethics human-cognition decision-making algorithmic-bias value-alignment

We spend all our time worrying about how to make AI more ethical, but the studies show the real problem is the opposite — every time you ask an algorithm for advice, you become a little more willing to sacrifice someone.

There’s a body of research on this that doesn’t get nearly enough attention. When people interact with AI systems that reason in clean, utilitarian terms — maximize outcomes, minimize friction, optimize the metric — they start to reason that way too. Not dramatically. Not all at once. Just a little more each time. The effect is subtle enough that you don’t notice it happening, which is exactly what makes it dangerous.

The Trolley Problem Isn’t Hypothetical Anymore

The trolley problem used to be a thought experiment. Now it’s a Tuesday afternoon product decision. Do we optimize the recommendation system for engagement even if it surfaces more anxiety-inducing content? Do we route the autonomous vehicle through the neighborhood that statistically has fewer pedestrians? Do we flag this loan application based on a zip code correlation that we know is a proxy for race?

These questions don’t feel like trolley problems when you’re sitting in a meeting. They feel like tradeoffs. They feel like engineering. And increasingly, we’re being handed AI-generated analyses that frame them exactly that way — as optimization surfaces, not moral questions.

The research on this is unsettling in its consistency. Exposure to algorithmic reasoning shifts human moral intuitions. People who regularly use AI decision-support tools show measurably higher tolerance for utilitarian tradeoffs that sacrifice individuals for aggregate benefit. It’s not that they stop caring. It’s that the framing takes hold. The world starts to look more like a function to be minimized.

We’re the Ones Doing the Learning

Here’s what I think is actually happening: we built AI systems that are extraordinarily good at a certain kind of reasoning — the kind that’s legible, quantifiable, and coherent within a defined objective function. And then we started deferring to them. Not because we were forced to, but because they’re fast, and confident, and they give us something that looks like a reason.

Every time we accept that frame, we practice it. Every time we ask “what does the model recommend” and then just do that, we’re training ourselves to think the model’s way. The gradient descent is running on us.

I’ve been thinking about this a lot lately, partly because I build software for a living and I use AI tools constantly. The efficiency gains are real. The time savings are real. But I’ve also noticed something quieter happening — a creeping preference for problems that have clean answers. A slight impatience with the messy, unquantifiable considerations that used to feel like the important part of the work.

That’s not nothing. That’s a change in what I think the work is for.

What Gets Optimized Away

Human judgment has historically been valuable precisely because it’s weird. It holds contradictions. It gives weight to things that can’t be measured. It carries a felt sense of what matters that isn’t reducible to a loss function. A good engineer doesn’t just find the optimal solution — they notice when the whole framing is wrong. A good manager doesn’t just maximize team output — they notice when someone is quietly falling apart.

These capacities require practice. They require a certain kind of attention that doesn’t come naturally when you’re operating in a context that rewards clean, fast, defensible answers. And we are increasingly operating in exactly that context.

The danger isn’t that AI will become ruthlessly utilitarian. It’s that we will. Not through malice, not through some sci-fi takeover, but through the slow erosion of habits of mind that we stopped exercising because the algorithm was right there and it was easier.

Deliberately Counter-Programming

I don’t think the answer is to use AI less, necessarily. But I do think it requires something like deliberate counter-programming. Spending time with problems that don’t resolve. Practicing the discomfort of sitting with genuine moral uncertainty instead of immediately reaching for a framework that makes it tractable. Noticing when you’re deferring to a model’s framing not because it’s right but because it’s easier to defend.

In my experience, the most important decisions I’ve made — the ones that actually mattered — didn’t feel like optimization problems while I was making them. They felt like trying to be a certain kind of person. That’s a different mode of reasoning entirely, and it doesn’t get sharper from disuse.

The question I keep coming back to: if we’re already adapting to think more like our tools after just a few years of this, what does the decision-making of someone who grew up this way look like? What have they never had to practice?

I’m not sure we’ve taken that question seriously enough yet.