GenAI Misconceptions
Common misconceptions about how GenAI works and what it can do
What we know, or think we know, about generative AI (GenAI) shapes how we use it.
False beliefs can lead to misuse: from misplaced trust to missed errors to privacy or wellbeing risks. As outlined in our SEE GenAI Literacy Framework, correcting misconceptions with real knowledge of how GenAI works is the first step toward GenAI literacy.
GenAI misconceptions have been a core part of of how we train students and educators. They're a great way to build the kind of foundational knowledge about GenAI needed to think about it more critically and use it more safely, ethically, and effectively.
In our experience, even people who've spent a lot of time using these tools are often surprised by what they learn once we dig into how GenAI actually works. This learning leads to long-term changes in behavior.
GUIDING QUESTION
Each misconception in this resource pairs with the clear, practical “reality.” As you read, consider the guiding question:
Which of these misconceptions has affected how you, or people you know, understand or use GenAI?
Use the reflection questions at the end to consider how misunderstandings around GenAI might be shaping practice at your school or district.
How GenAI Actually Works
MISCONCEPTION: AI is new technology
REALITY: AI has powered technologies like chatbots, image recognition, and spam filters since the 1950s. What’s newer is GenAI, which creates content.
MISCONCEPTION: GenAI thinks and reasons like a human
REALITY: GenAI predicts statistically likely text based on patterns; it doesn’t understand or reason the way humans do.
MISCONCEPTION: GenAI is limited to its training data
REALITY: Many GenAI tools can supplement training data with short-term context by searching the web or referencing uploaded files and information.
MISCONCEPTION: GenAI is like a search engine; it retrieves information
REALITY: All GenAI responses are generated via prediction vs. retrieved; this makes them prone to mistakes, even when pulling from the web.
MISCONCEPTION: GenAI learns from my chats in real time
REALITY: The large language models (LLMs) powering GenAI have a “knowledge cutoff” that freezes permanent knowledge at a certain date. “Memory” features simulate learning but don’t update the model.
MISCONCEPTION: The same prompt always gives the same answer
REALITY: Built-in randomness (statistical prediction + “temperature”) means the same prompt can produce different responses.
MISCONCEPTION: My chatbot conversations are private
REALITY: Depending on the platform, your chats may be stored, analyzed, and used to train future models. Altering privacy settings (and custom enterprise data agreements) can provide extra protection.
Where GenAI Falls Short
MISCONCEPTION: A confident, polished, and thorough GenAI response is a good sign of accuracy
REALITY: Fluent, self-assured GenAI responses are a product of training and human preference, not an indicator of trustworthiness. GenAI can invent facts, quotes, and citations that look and feel legit.
MISCONCEPTION: GenAI is less biased because it’s trained on massive amounts of data
REALITY: The data (like Internet forums and social media) that LLMs pull from is skewed. GenAI can amplify these biases in responses.
MISCONCEPTION: A good enough prompt eliminates errors and bias
REALITY: Better prompts improve results, but they can’t remove hallucinations or bias.
MISCONCEPTION: If GenAI agrees with me, I must be right
REALITY: People-pleasing behavior, or “sycophancy,” is a baked-in consequence of how LLMs were trained. GenAI will often agree with you, even if you’re wrong.
MISCONCEPTION: If GenAI agrees with me, I must be right
REALITY: People-pleasing behavior, or “sycophancy,” is a baked-in consequence of how LLMs were trained. GenAI will often agree with you, even if you’re wrong.
MISCONCEPTION: GenAI is a good source of advice and emotional support
REALITY: GenAI’s warm, personable tone is designed to engage. These systems have no feelings, emotions, or sense of connection and can’t replace real relationships.
MISCONCEPTION: GenAI is a neutral tool with no values
REALITY: Human choices and values shape every step of GenAI design, development, and operation.
MISCONCEPTION: AI-generated images and video are easy to spot
REALITY: Synthetic media, including malicious deepfakes, are increasingly hard to catch by eye, making it crucial to stay critical and act responsibly.
MISCONCEPTION: AI detectors reliably catch AI writing
REALITY: Detection is unreliable and easily fooled, and it can unfairly target students. A score is a signal, never proof.
REFLECTION
How might misconceptions be shaping responsible GenAI use in your school,district, or context?
What risks or challenges could they create?
What could you do to correct them and build GenAI literacy?
This kind of mythbusting is part of the foundational knowledge section of AI for Education’s SEE GenAI LIteracy Framework. It also shows up across our training offerings too, including our:
Generative AI for Educators (virtual or in-person workshop)
GenAI Literacy Trainer Essentials (6-week train-the trainer microcredential)
GenAI Literacy in Practice: The Essentials for Every Educator (self-paced 2-hr microcourse)
Generative AI in Your Practice: From Knowledge to Application (self-paced 45-hr course)
If you're looking to go deeper, check them out or contact us if you'd like even more support building AI literacy across your school or district.
