The AI Confidence Trap

Confidence using GenAI isn't the same as AI literacy. In fact, the research keeps showing that more confidence can lead to worse practices.

A new study (Abdelghani et al., 2026) puts data behind what we've been seeing since 2023.

Researchers in France gave 63 middle schoolers unrestricted access to ChatGPT for six science investigation tasks, then tracked how well students asked questions, evaluated AI responses, and followed up. The results paint a clear picture of what happens when students engage with GenAI without literacy.

💡 Students couldn't tell good AI outputs from bad ones. 71.4% of the responses experts flagged as low-quality were rated by students as clear and useful.

💡 Only 22% of students asked follow-up questions at all across the six tasks. Even when students recognized an AI answer was bad, they only followed up 31.8% of the time.

💡 The finding that stuck out to us the most: the students with the most positive attitudes toward AI performed the worst. Comfort with the tool was inversely related to skill with the tool.

So what separated the stronger students? Metacognition and what the researchers describe as "healthy skepticism," which included critical perceptions of AI's fairness, usefulness, and the effort required to use it well. We've seen this critical disposition develop when learners build knowledge of how GenAI actually works and pair it with purposeful, ongoing practice.

Without that foundational knowledge and the right mindsets, students (and educators) get trapped in what the researchers call "the illusion of understanding." They feel like they're using GenAI effectively even when they lack the understanding to do so.

This is exactly why our SEE GenAI Literacy Framework focuses on building the foundational knowledge and critical mindsets that together shape safe, ethical, and effective practices, including the kind of output evaluation, self-regulation, and reflective use this study shows students need.

Building GenAI literacy requires intentional learning. But until schools prioritize it, most young people are figuring this out on their own.

Check out our free GenAI Literacy 101 for Students course as a place to start.