AI took the how. Now it’s coming for the what — and judgment built through experience, intuition with scars, is what survives
There’s a moment, working with AI, when you stop knowing exactly what you’re doing. Not because you get lost technically, but because your relationship with the tool changes completely. That’s where something cracks. Not in the technology, but in our role.
For the past few days I’ve been building an agent that evaluates the quality of other agents, and when I needed to share it to keep iterating, I pushed it to GitHub. I’m a Product Designer with no technical background. The most technical thing I’d ever done was wrestling with HTML to build ugly corporate emails. But I went in with Claude beside me, interpreting the terminal in real time. And it worked. Watching it not just write the code I asked for, but make decisions about architecture and configuration on the fly, I physically felt the AI cross a line.
That GitHub repo with my name on it is just the symptom. What’s interesting is what the symptom forced me to think about.
The obedient tool is over
For years we heard something reasonable, almost reassuring. Machines would handle the tactical, people would stay in charge of judgment. Ben Thompson, over at Stratechery, drew that map with precision. He defines AGI as an AI you can hand a task to and trust to complete it, one that solves the how. And he reserves the what for something much further out, superintelligence, “the ability to come up with the tasks in the first place.” On that map, human judgment stays safe until a future no one has seen. That’s why it was reassuring.
And for a while it felt that way. Chatting with LLMs helped me draft, structure, move faster. Jakob Nielsen called it the first new UI paradigm in 60 years. We finally declared the what to the machine, and it worked out the how. I aimed. The AI fired. There was something comfortable in that narrative. AI as a powerful but obedient tool, and me deciding where to point it.
And it was easy to believe, because it mirrored something we already knew. In any career, delegating the how was always a sign of moving up. It was the first thing you let go of as you grew, the thing you handed to someone more junior while you climbed toward the strategic. The what, on the other hand, had to be earned. It was the prize, and the responsibility, for having fought enough battles. AI seemed to fit that model perfectly. One more delegate, an impossibly fast one. What the model didn’t account for was the delegate starting to climb the ladder.
That stage is over. And a lot of conversations about AI still operate there, as if the only thing at stake were productivity.
What came next is what I call the crisis of the what. The moment AI stopped executing instructions and started taking part in the decisions that generate them.
For years we assumed the how was negotiable, but the what stayed ours. What to build, for whom, with what judgment. That boundary is giving way, and this is where I part ways with Thompson’s map. In his scheme, the what only comes into play with superintelligence. In my daily work it’s in play now, with nothing remotely like superintelligence in sight. And it’s giving way quietly. Nobody handed the what to AI in a strategy meeting. It’s taking it little by little, through small decisions that look like pure execution, from inside the work we always treated as tactical.
When AI started deciding
Something shifted over the past few months, and I saw it while building tools for myself. Small personal projects in HTML, a chord-sheet app for my sheet music, a Spider Solitaire game for my wife, an automation to file my medical expense claims in one click. Things that a couple of years ago would have taken me weeks, or that I simply wouldn’t have attempted. Now they come together in under 30 minutes.
But it wasn’t only the speed. At some point I started noticing the AI was doing more than executing. It analyzed structures, proposed paths, caught problems before I saw them, interpreted intent I hadn’t made explicit. An agent no longer needs you to spell out the how step by step. And in that process, it inevitably starts brushing up against the what. Weighing in on what’s worth doing. Prioritizing without anyone asking it to.
Here’s the interesting part, because it’s exactly where these systems hit their limit. Ethan Mollick, a professor at Wharton, and his co-authors gave that edge a name: the jagged frontier. In a study with Boston Consulting Group they showed that AI isn’t even. It shines at a complex task and stumbles on a simpler one, with no pattern you can anticipate. The frontier has moved since that study, and the jaggedness, so far, persists. Mollick reads it as a productivity problem. Learn the shape of the frontier and split the work according to where each thing falls. I see something different. The teeth of that frontier don’t land just anywhere. They line up with the edge of the what, right where judgment begins.
And there’s a deeper reason. Today’s models are, in essence, probabilistic systems. They predict the most likely next thing based on patterns in existing data. That makes them extraordinarily good at finding the reasonable, the expected, what already has precedent. But the most interesting human decisions run the other way. They go against the obvious. They decide not to build something that technically could be built. They sense something’s wrong before they can explain why.
Think about a problem-resolution flow in a complex product. A model trained to optimize sees a complaint and proposes the statistically most efficient resolution, closing the ticket with a one-click refund. It’s the logical move, the expected one. But a person with judgment reads the frustration between the lines and does the opposite of what the statistics suggest. They add deliberate friction, designed so the person feels the product is taking responsibility for the error. They slow the automation down, because they understand that here the person doesn’t just want their money back fast. They need to know someone acknowledged the failure.
The AI chases the highest probability. Human intuition knows when that probability is a design mistake.
Yann LeCun, one of the three researchers who won the Turing Award for laying the foundations of deep learning, keeps insisting that LLMs are, essentially, statistical machines. He explains it with a simple analogy. There are students who memorize and do well on familiar tests, and others who understand the problem and can move through new territory. Today’s models, he says, are the first kind.
A study on clinical reasoning published in Scientific Reports found the same thing by another route. The researchers deliberately triggered the Einstellung effect, that mental fixation that makes us apply the familiar solution even when the problem has changed. The models fell into the trap systematically, where human doctors performed far better. With one aggravating detail. The models defended their wrong answers with complete confidence.
These systems fail precisely where human judgment is most needed. In the rare, atypical contexts, where the right answer is improbable. And that, at least so far, seems independent of how much they improve.
There’s a technical debate about how far these models will go, and LeCun is arguing just one side of it. On the other, Dario Amodei, CEO of Anthropic, has argued that scaling laws still hold and that many current limitations will be solved with more compute and better architectures. He’s probably right about a good part of it. But that discussion is about the ceiling of the technology, and I’m interested in something else…
What happens in the meantime, in everyday work, where these models are already making decisions?
There’s no pessimism here. I say it because it’s a conversation a lot of people in tech are having in private, while in public we keep pushing old narratives to describe roles that have already changed.
What survives the crisis of the what
It’s worth drawing the line carefully, because not everything falls on the same side.
AI is extraordinary with the probable. What already has precedent, what someone wrote down somewhere, the pattern that repeats. Give it a known problem and it finds the most efficient way out almost effortlessly.
What’s left for us is the other thing. The atypical, what no one documented, the decision to go against the rule when the context calls for it. Not finding the most efficient way out. Knowing when that way out is a mistake.
That’s the hard part to name, so better a concrete example.
Picture yourself at a red light, wanting to cross. Without judgment, the rule is simple. Wait for green. But almost nobody works that way. You read the surroundings. Whether there are cars, how far off they are, whether the road is clear, how much time you have. Those variables are the part of judgment that’s pure context, and context is something models are getting better at handling. Bigger windows, more specialized agents, more information integrated in real time. That part is already up for negotiation, and it’ll probably keep shifting. The line of what’s documentable isn’t fixed.
But crossing on red takes more than reading variables. It’s also knowing you can’t take the risk today because you’re carrying something important. Or that something in the air, hard to name, tells you no. That “something” isn’t irrational. It’s context accumulated slowly, undocumented anywhere, that changes the weight of each variable depending on who you are, where you’re standing, and what’s at stake in that specific moment.
None of this is new, either. Donald Schön described it more than forty years ago in The Reflective Practitioner. Competent professionals deploy, in practice, a kind of knowing that never passed through theory first. They perform well in situations that are uncertain, unique, loaded with conflict, exactly where technical rules fall short. And when you ask them to explain how they did it, the explanation arrives late and incomplete. Schön called it knowing-in-action. I’ve been calling it judgment. But there’s a consequence he never got to see. What never let itself be turned into rules is also, so far, what most resists being turned into training data.
Careful, though. Not every gut feeling is sound judgment. We humans also get it wrong with confidence, and sometimes what we call intuition is just bias with good PR. The difference is in the track record.
Kahneman and Klein studied it from opposite corners. One distrusted intuition, the other defended it. And they agreed on two conditions for an intuition to be worth trusting. That the environment is regular and gives reliable signals, and that you’ve had enough practice and feedback to read them. The track record is just that. The two conditions working together.
Judgment is built by getting it right and getting it wrong in real contexts, with consequences that land on you. Intuition, in a way, is scar tissue.
Nassim Taleb calls it having skin in the game. You don’t truly understand a risk until you carry its consequences. I agree, but I’m pointing at something different. Skin in the game looks forward, at what you have at stake right now. Scars look backward. They’re what’s left once you’ve already paid. One is the stake. The other, the mark.
And something else goes beyond the one deciding. It lives in the one receiving the decision. People aren’t that regular environment intuition depended on. We aren’t predictable. The same feature that delighted someone yesterday annoys them today. The same bug that one user forgives, another experiences as a betrayal. Not for lack of data, but because mood, context, and each person’s history change the meaning of things in real time.
Anyone paying attention could throw Kahneman and Klein back at me. If people aren’t a regular environment, what intuition could be worth anything there? My answer: a different one. Not the one that predicts the reaction, but the one that recognizes when the pattern stopped applying. It’s trained the same way, with practice, feedback, and consequences that land on you. What changes is that the goal becomes detecting the exception.
Judgment, unlike data, is formed by having paid the cost of being wrong.
A model trained to predict the most probable is, by design, blind to that part. And designing products means, in large part, working with that unpredictability. Not erasing it, but understanding it, anticipating it sometimes, respecting it almost always.
That’s where the human part survives. More than just reading the surroundings, it’s the hierarchy you assign to each thing. Intuition over data. The understanding of a system no one ever wrote down in any Confluence. Hallway conversations. The social, political, or historical context of the people who use the products. The tensions between teams. What someone didn’t say in the meeting. At the end of the day, the unpredictability of human experience.
Building is trivial. Deciding isn’t.
This is the idea that matters most to me. And, probably, the one that’s most overused in the current discussion.
At a moment when building anything is easier and cheaper than ever, what separates a product worth making from one that isn’t comes down less to execution and more to the decision. What to build, for whom, in what order, on what assumptions, how far, when to stop, what to leave out. Those questions aren’t answered with speed. They’re answered with judgment.
The paradox is that the more execution gets automated, the more strategic the human part that doesn’t. There’s an economic principle that explains it. Joel Spolsky named it commoditize your complement. When the price of an input falls, the value shifts to its complement. Thompson has been applying it to AI. But almost nobody completes the equation for what’s coming.
When execution is the input that gets commoditized, judgment is the complement that gets expensive.
If everyone can execute at AI speed, the competitive edge shifts to where AI doesn’t reach yet.
And I’m not the only one looking that way. Recently, even the CEOs who’d most loudly announced the mass replacement of jobs walked the message back. Amodei himself went from saying AI could wipe out half of entry-level office jobs to arguing that automating 90% of a job makes the remaining 10% expand and become the new focus. Whatever their interests — and with IPOs on the horizon, there are plenty — the reframing is telling. Even from the companies building these models, the conversation is moving from “AI replaces” to “AI expands what’s left as human.”
What’s left
This zone we’ve isolated, the one that goes against the obvious, that reads the undocumented, that decides when not to build, I think it’s going to be one of the foundational skills for those of us working on digital products in the age of AI. Not because AI won’t advance, but because while we learn to use it deeply, to tame the beast, the human focus has to be exactly there. On what makes us indispensable.
Within design, this conversation was already surfacing. UX Collective, in The State of UX report, warned that once design systems connect to code and AI can build interfaces on the fly, the designer’s role as an intermediary loses weight. I share the diagnosis, though I’m more interested in the question it opens. If the intermediary disappears, what’s left? For me, exactly this zone. If drawing the screen is automatic, the value lands on whoever decides which screen to draw. And above all, which one not to.
The market, still without a clear way to name this, is already figuring out where it should live. A year ago I wrote about UX Prompting — my own, admittedly personal way of naming something I could feel before I could explain it. A year later, that hunch turned into job postings. Companies are starting to look for roles like AI Product Designer or AI Interaction Designer. The names vary, the emphases too. But the same thing shows up in all of them, almost without meaning to.
Someone who carries the judgment when the machine can already build almost anything.
I go back to that agent that evaluates the quality of other agents, terminal open, Claude making decisions I hadn’t asked it to make. I wasn’t scared of what the AI could do. What gave me vertigo was realizing my work was no longer what I thought it was.
Maybe the crisis of the what was never, deep down, a crisis of AI. It’s ours. We spent entire careers treating execution as the toll and decision as the prize. Now that executing is no longer scarce, it’s time to prove we deserved the prize.
The how changed. The what changed too. And only now are we discovering what our work really was.
A note on language: English isn’t my first language. I wrote this in Chilean Spanish and leaned on AI to bring it into a register that travels. The voice is still mine, the accent just had to change.
The crisis of the what was originally published in UX Collective on Medium, where people are continuing the conversation by highlighting and responding to this story.
