Skill Loss and GenAI - Make It a Choice, Not an Accident

In my own development work, I have a ruleset for Claude around duplicate code. Building a new feature, Claude changed 1500+ lines of code. It confidently reported that it had checked for duplicate code. It had not. While reviewing the code, I found a function yearPeriod() that had been duplicated. Visually, I found three instances of the function. Claude eventually reported that it had written five.

I caught it because I still know what duplicated code looks like, and because I still read the diff. That is the skill in question.

GenAI-related skill loss is happening wherever we hand a task to a GenAI. If GenAI was perfect at doing the tasks we hand it, that might be ok. It isn’t.

If GenAI is right 95% of the time, how do we maintain the skill to recognize the cases where there is a mistake? Even harder, how do we train the next generation to have a sense of correctness in the first place?

All is not lost. GenAI-induced skill loss isn’t inevitable. We can make deliberate choices about what to abandon and what to keep sharp.

Background

As far back as Plato, people have been thinking about deskilling. In Plato’s Phaedrus, Socrates warned that writing would erode memory and weaken our ability to think for ourselves.

In regulated fields like medicine and flying, this is a well recognized problem. When we automate a task and don’t practice it, we lose the skill. In these cases practitioners are expected to take regular refreshers or training to maintain their certifications. In the case of flying, they even spend time in a simulator practicing emergency procedures to deal with problems that they will hopefully never encounter.

What makes it worse in the world of GenAI is that the automation appears to be good, but it isn’t reliable. We’re losing our existing skills. And the people still forming those skills, the next generation, may have it worse.

AI Generated Artifacts Appear Good

When you prompt a GenAI to do some work, it will do the work and confidently report back that it did the job. The code will look right, obey all the conventions, and yet it’s full of unknowns.

GenAI is not perfect. It produces bugs at about the same rate (or worse) as the human code it was trained on, and the flaws are structural rather than something a review will reliably catch: we went through the fundamental flaws in GenAI code quality in an earlier article. Further, as John Ferguson Smart recently demonstrated,¹ the GenAI reported that it had built all of the requested features in a spec. Yet it missed two. Worse, independent reviewers looked at the code and thought it looked good.

From my own experience, even a small task can result in 1000+ lines of diffs. If the changes are just renames or other simple refactoring, the review process is still manageable.² The research shows that as the Cognitive Complexity increases, the number of defects in the code increases.³ And at the same time, as the Cognitive Complexity increases, it’s harder to review the code.⁴⁵ Even worse, the LLMs are happy to mix simple refactorings with complex changes.⁶⁷

Critical Thinking Matters More

The more we use GenAI to handle the low level drudge work, the harder the work that remains becomes. Our cognitive load is going up. This is part of why the promised speed keeps failing to show up: we looked at the research on that in Using GenAI to Code? Not So Fast.

With GenAI generating potentially useful code, we need to review something we didn’t write. Something with a style and patterns we may not be familiar with.

Does the code do the thing I asked for? Does it cover the edge cases and exceptional conditions? Does it fit the architecture of the system? Are there defects and security flaws?

Human first programming is about understanding a problem. Building a model of the problem in our heads and then implementing the model. GenAI skips the model building and goes straight to code. When trying to review the code, we need to build a model at the same time as seeing the code for the first time. This is both a tremendous cognitive burden and a new skill.

What the Research Tells Us

One recent study has some clues: How AI Impacts Skill Formation⁸ (NB the paper was published by Anthropic and the authors are Anthropic Fellows).

The study authors recruited 52 mostly-junior developers who already knew Python to use a new async Python library (Trio) they didn’t know. They were assigned a programming task with the library and then divided into two groups.

The control group (No AI): participants were prohibited from using AI assistance and had to complete the tasks entirely on their own.

The treatment group (AI Assistance): participants had access to a chat-based AI assistant and were encouraged to use the AI assistant to complete the task as quickly as possible.

After the task, the developers were assigned a quiz. The quiz was a test of their understanding of the new library that they had worked with. The no AI group did noticeably better than the AI-use group. That headline gap needs a caveat though: the no-AI group also worked without autocomplete or any other tooling, and under the same time pressure, so the raw difference can’t be cleanly pinned on AI alone. Within the AI-use group, there was still another divide: the people who used the AI to pose conceptual questions and ask for explanations, but then wrote their code independently, did better than the group that simply relied on the AI to generate the code.

So it’s not AI use alone that is causing the loss of skills, it’s the lack of engagement when using the AI. The more cognitively engaged you are, the stronger the skill formation. This is where our kids have the challenge: they’re not maintaining a skill they already have, they’re trying to form it. Lean on the GenAI to do the thinking and they don’t learn.

Why the difference?

• Struggling through errors builds knowledge.⁹ We have a mental model of the problem, something fails, and we’re challenged to update that model.
• Delegation is a trap: the more that we delegate to the AI, the less we learn.

What Can We Afford to Lose?

Make explicit choices, don’t let it happen by accident. What am I personally willing to give up in terms of skills? In your team, ask what it is that you could afford to give up.

I’m asking this in two areas: as the owner of Agile Pain Relief, and as the developer of an expense tracking app.

In my own case:

• Dealing with databases. I spent 16 years of my career working with database-related apps (Andyne and then DataBeacon). Today, I’m building an expense management app. I understand the table model, but I’m not fussed about the SQL.
• Astro. Our website is built with Astro and content-collection Zod schemas. Again, I can understand what we have, but I couldn’t build it from scratch.
• CSS. Even more than content collections, CSS makes me nervous.
• Understanding some of the plumbing for internal systems. For example, I don’t need to understand the code that uploads my social media posts. I still write the posts, they just exist on my computer as plain text. The tooling allows me to focus on the writing and not fiddly web interfaces.

Those I can afford to give up. What really matters to me is the deep thinking and the human side of the work:

• Writing
• Research
• Analysis work
• Talking to people in the industry

What Is Critical for a Team Building a Software Product?

The same question scales up. For a team shipping software products:

• Understanding the problem, deep enough to spot poor AI assumptions
• Empathy for the customer and end-user
• Understanding the architecture
• Being able to verify that the generated code is what we expected
• Verifying the behaviour of the system
• Deep enough understanding of the whole thing to be able to diagnose problems
• Judgement on quality, security, performance, and the rest

And perhaps most importantly: critical thinking, constantly questioning the results of the generated system. Is it really what it claims to be?

What to Do to Offset Skill Loss and Learn New Skills

We can learn from medicine and aviation:

• Practice our skills, especially under difficult circumstances
• Self testing. Quiz yourself in a subject area, or get the GenAI to generate questions for you
• Lean on the GenAI to answer conceptual questions and give explanations

We will all need self-discipline to make this work. Taking the time away from the GenAI to keep our skills sharp, and to help train the next generation.

When you use a GenAI, probe its output for the mistakes that it doesn’t tell you about. What are the edge cases? What assumptions did it make? What did it not consider, such as error handling or accessibility?

I try to practice this constantly, and I can tell you this is cognitively draining. Keeping your guard up, questioning constantly, and looking for evidence to support decisions and claims made by AI.

Deskilling and the knowledge cliff are a full module of our Surviving the AI Tsunami course, an evidence-based, cohort-based course for Agile teams.

Conclusion

The more often the GenAI works, the more tempting it is to assume that it always works. That assumption is exactly what erodes the skill.

If we want to maintain our skills, learn new things, and welcome new people to our field, we need to operate with the reverse assumption. Assume the GenAI has made a mistake. Treat it as an adversary. It is rolling dice rather than thinking, which is the argument we made in GenAI vs Human Intelligence: a Reality Check.

Practice the skills you intend to maintain. Make it a choice, not an accident.

Most importantly, working with GenAI is intense. It carries a human cost well beyond the code. Take time away from your computer. Talk to other people in the real world. Get outside and experience nature.

Footnotes

1. John Ferguson Smart, “ATDD-driven AI.” https://johnfergusonsmart.com/atdd-driven-ai/ ↩

2. Kim et al. (2008). https://doi.org/10.1109/ICSM.2008.4658077 ↩

3. Muñoz Barón et al. (2020). https://doi.org/10.1145/3382494.3410636 ↩

4. Tao et al. (2012). https://doi.org/10.1145/2393596.2393608 ↩

5. McIntosh et al. (2015). https://doi.org/10.1007/s10664-015-9381-9 ↩

6. Murphy-Hill et al. (2012). https://doi.org/10.1109/TSE.2011.41 ↩

7. Opu et al. (2025). https://doi.org/10.48550/arXiv.2505.08263 ↩

8. Judy Hanwen Shen and Alex Tamkin, “How AI Impacts Skill Formation.” https://arxiv.org/abs/2601.20245v2 ↩

9. Elizabeth L. Bjork and Robert A. Bjork, “Making Things Hard on Yourself, But in a Good Way: Creating Desirable Difficulties to Enhance Learning” (2011). https://bjorklab.psych.ucla.edu/wp-content/uploads/sites/13/2016/04/EBjork_RBjork_2011.pdf ↩