AI Doesn’t Kill Skills — It Moves Them Up a Layer

A new academic paper shows that AI users scored 17% lower on a quiz designed to measure how much they’d learned while completing a task. AI had smoothed out many of the rough edges that produce real learning, like debugging code. They completed the task, but came out the other side knowing measurably less than colleagues who didn’t use AI.

If you manage people who use AI tools, this sounds alarming.

But it isn’t—or at least not for the reasons you think.

Here’s what the (excellent!) paper actually found. A group of 52 experienced Python developers were randomized into two groups: one used AI to help them complete a task, the other did not. The AI group hit fewer errors along the way, which also meant there was less opportunity to learn from correcting those errors. If you never debug your mistakes, you never really learn to debug, and historically debugging is one of the most valuable skills a developer can cultivate.

But there’s some real nuance in the paper beyond that headline finding.

Digging deeper, the authors identified six interaction patterns in how developers used AI, and the learning outcomes were starkly different.

Three patterns tanked learning. Delegation—hand the AI the whole task and paste in the result. Progressive reliance—start doing the work yourself, then gradually shift to letting the AI write all the code. Iterative debugging—run into an error, ask the AI to fix it, run into another, ask again, never understanding what went wrong.

Three patterns preserved it. Conceptual inquiry—ask the AI questions instead of asking it to do things. This group scored highest and, interestingly, was also the fastest among the high-scoring patterns. Hybrid code-explanation—ask for code and an explanation together, then actually read the explanation. Generation-then-comprehension—let the AI write the code, then manually copy it and ask follow-up questions to understand what it did.

So despite the scary title, the pattern isn’t ultimately “AI helps or hurts.” It’s “how you use it determines what you learn.” The paper knows this—it’s right there in the data. But the headline finding is the scary number, not the nuanced pattern.

The layer the paper didn’t measure

Here’s what happened when I had an AI agent build a Gmail MCP integration from scratch: I learned essentially nothing about the implementation—I couldn’t tell you how the OAuth flow works, couldn’t debug the API calls. By the paper’s measure, I atrophied.

But I learned a lot about something else: how to scope a project so an agent can execute it. How to break work into pieces that are independently verifiable. How to write a spec tight enough that the output is useful and loose enough that I’m not just dictating code. How to evaluate results when I can’t read every line.

The paper measured skills at the execution layer. It didn’t measure what was forming at the layer above.

This has happened before!

Every wave of programming abstraction—assembly to C, C to Python, Python to frameworks—changed which skills mattered without reducing whether skills mattered. Nobody argues that Python developers are less skilled than assembly programmers. They’re skilled at different things, at a different layer.

The pattern: each compression step lets you say less while the system fills in more. The skills shift from “how do I implement this” to “how do I specify this” to “how do I evaluate whether this is right.” The new layer isn’t optional or lesser. It’s just what the work requires.

AI agents are the next compression step, and they’ve made the leap all the way to natural human language (rather than, say, compiled libraries) but the dynamic is the same.

The management transition

This is the same structural shift that happens when an individual contributor is promoted to a manager. You stop learning how to do the work and start learning how to direct and evaluate it. Nobody calls that “skill atrophy”—they call it a career path. (I wrote recently about how agents are forcing us to rediscover organizational behavior—this is the individual version of that same shift.)

The VP of Engineering who can’t implement OAuth anymore isn’t atrophied. They’ve shifted their primary skills to a layer where their judgment about estimates, architecture trade-offs, and team capability is what matters. They need enough understanding to smell when something’s wrong—but “enough” is not “can do it themselves.”

What the paper actually caught

Look at those six patterns again. The three that tanked learning—delegation, progressive reliance, iterative debugging—have something in common: the developer simply offloaded work and moved on. The three that preserved learning—conceptual inquiry, hybrid explanation, generation-then-comprehension—all involve the developer doing cognitive work on top of the AI output, actively engaged with guiding and evaluating it.

The paper didn’t catch AI killing skills. It caught people building the new skills needed to work more effectively a layer above writing the code.

The developers who learned the most were the ones who stayed engaged—who asked questions, who read the explanations, who treated the AI output as something to understand rather than something to blindly accept. They were doing the work at the new layer above, even if nobody was measuring it.

(To me this sounds an awful lot like the way a new manager has to stop measuring their worth by individual output and start learning to actively direct and evaluate someone else’s.)

The skills (and the learning) just moved up a layer. They didn’t disappear.