Artificial intelligence can make a student look more capable before it makes them more knowledgeable. A polished answer appears in seconds; the assignment improves; the learner may even feel more engaged. But the real test comes later, when the chatbot is gone and the student must explain the idea, retrieve it from memory or use it in a new setting.
That distinction sits at the center of a new American Psychological Association report on educational technology, released in September 2026. Its warning is not that digital tools are inherently harmful. It is that visible activity—clicks, time on task, rapid responses and completed work—is a poor substitute for durable learning. The better question is whether knowledge transfers beyond the screen.
Recent evidence complicates any simple verdict on generative AI. Some students appear to use it as cognitive scaffolding; others use it as cognitive substitution. The technology may be the same. The learning process is not.
A finished product can hide an unfinished mind
The OECD's PISA 2025 results, published this month, show how widespread AI-assisted learning has become: 46% of students across OECD countries reported using AI chatbots at least weekly to help them learn. The headline finding is uncomfortable. Students who used AI for tasks such as summarising, drafting or research scored about 20 points lower in science than non-users—roughly the gap the OECD equates with a year of schooling.
That is an association, not proof that AI caused lower scores. Students who are struggling may turn to chatbots more often; schools differ in how they teach AI literacy; family resources and prior achievement matter. Indeed, the same report found that regular general-purpose users who also had opportunities to develop AI literacy at school tended to score slightly higher. The pattern therefore argues against both panic and complacency.
Psychology offers a useful concept here: cognitive offloading. Humans routinely shift mental work onto notebooks, calculators, maps and other people. Offloading can free limited working memory for harder reasoning. It can also leave less to remember when the external aid disappears. A review of intention offloading makes clear that external tools are neither automatically good nor bad; their effect depends on what mental work they replace and what they enable.
Generative AI raises the stakes because it can offload not merely storage or calculation, but the production of explanations. A student who asks for a finished paragraph may bypass the very acts—selecting, organizing and connecting ideas—that build understanding.
The evidence improves when AI behaves like a tutor
Experimental studies suggest a more hopeful possibility. In a 2025 randomized trial involving 194 Harvard undergraduates, a carefully designed AI physics tutor produced more than double the median learning gains of an active-learning class, while students spent a median of 49 minutes with the tutor versus about 60 minutes in class. The study, published in *Scientific Reports*, did not test a generic “give me the answer” chatbot. Instructors built question-specific prompts, scaffolding and feedback into the system.
The result is promising but bounded. It covered two lessons in one course, relied on immediate post-tests and did not measure long-term retention. The AI condition also differed from the classroom condition in pace, location and presentation. It shows that a well-designed tutor can work—not that any chatbot, used in any way, will improve learning.
A newer randomized experiment released as a preprint in July 2026 sharpens the distinction. Undergraduates learned an unfamiliar topic and wrote an analytical essay with or without access to generative AI, then completed unaided assessments immediately and one week later. AI access raised immediate knowledge-test scores by 0.27 standard deviations, and the gains persisted a week later. Yet the strongest delayed benefits appeared among “augmentation” users, who asked AI to explain concepts, rather than “automation” users, who delegated text production. Because the paper has not yet completed peer review, its estimates should remain provisional.
These findings fit a broader research framework on generative learning. Learners understand material more deeply when they generate explanations, visualise relationships, test themselves and connect new information to prior knowledge. Mere activity is not enough. The mind has to construct meaning.
The decisive question is who generates the explanation
The practical lesson is not “never use AI” or “learn faster with AI.” It is to preserve the parts of learning that require generation and retrieval. Before consulting a chatbot, a learner can attempt the problem or state what remains confusing. During the exchange, AI can be asked for a hint, a counterexample, a critique or a question—not a finished submission. Afterwards, the learner can close the tool and explain the idea unaided, then apply it to a new case.
Those steps turn the chatbot from a production engine into a form of guided practice. They also make failure visible. If the student cannot reconstruct the reasoning without the screen, the fluent answer created an illusion of mastery rather than mastery itself.
The best current evidence therefore points to a design principle, not a blanket judgment: AI helps learning when it increases the learner's share of sense-making. When it removes that work, performance can rise while understanding stays behind.
