This article is part of "Minds in the Machine Age" — a companion series to Between Brain & Binary. Where that series traced the history of AI and mental health, this series examines the psychology of living alongside intelligent machines today.
← Back to Series Overview | Between Brain & Binary →
← Previous: The Empathy Gap: Why AI Can Simulate But Not Feel | → Next: Decision Fatigue in the Algorithmic Age
Watch a child using a well-designed adaptive learning app for the first time and it is genuinely impressive. Problems calibrate to their level. Correct answers are celebrated. Wrong answers redirect without penalty. The interface is warm, responsive, and patient in a way that a classroom of thirty students rarely permits. The child is engaged. You can see it. And engagement, we have been told for decades, is the precondition for learning.
The relationship between engagement and actual learning turns out to be more complicated than it looks.
The Promise and Where It Came From
The dream of personalised learning is not new. In 1984, Benjamin Bloom published what became one of the most cited papers in educational research — a study demonstrating that students who received one-on-one tutoring performed, on average, two standard deviations above students in conventional classroom instruction. Bloom called this the 2 Sigma problem: the challenge of finding a scalable alternative to individual tutoring that could produce remotely comparable results.
Adaptive learning technology arrived, a few decades later, as the proposed solution. The logic was compelling: an algorithm could track each student's performance in real time, identify gaps, adjust difficulty, deliver targeted practice, and move at the learner's pace — approximating the personalised attention of a tutor at a fraction of the cost and at any scale.
Platforms like DreamBox, Knewton, and the early iterations of Khan Academy's adaptive exercises were built on this premise. So were hundreds of others, backed by substantial investment and considerable enthusiasm from schools, districts, and governments seeking to modernise education. The EdTech sector grew into a multi-billion-dollar industry, and the language of personalisation became embedded in educational policy worldwide.
What the Research Actually Shows
The evidence, when examined carefully, is more nuanced than the marketing.
Intelligent Tutoring Systems (ITS) — the most sophisticated category of adaptive learning technology — have the strongest research base. A 2011 meta-analysis by Kurt VanLehn, published in Educational Psychologist, found that well-designed ITS produced effect sizes roughly comparable to human tutoring for specific, procedural domains: mathematics, physics problem-solving, basic programming. For structured content with clear right and wrong answers, adaptive systems can be genuinely effective.
John Hattie's Visible Learning project, which synthesised thousands of meta-analyses on educational interventions, placed computer-assisted instruction as moderately effective — meaningful, but well below the effect sizes associated with quality feedback, strong teacher-student relationships, and deliberate practice with expert guidance.
The Education Endowment Foundation, which rigorously reviews EdTech evidence for UK schools, has repeatedly found that enthusiasm for adaptive learning technology significantly outpaces the evidence for it — particularly for reading comprehension, higher-order thinking, and subjects where the learning goals are less easily quantified.
The pattern that emerges is consistent: adaptive systems work well when the learning goal is procedural and discrete (practise this skill until mastery), and poorly when the learning goal is conceptual, relational, or requires genuine intellectual struggle (understand this idea, apply it in a new context, argue for a position).
Where It Falls Short
The deeper limitation of most adaptive learning systems is in what they are actually measuring. Engagement is easy to track: time on task, click rates, completion percentages. Learning is far harder. Short-term performance on embedded quizzes is not the same as durable understanding, transferable knowledge, or the capacity to apply what has been learned to genuinely novel problems.
Several studies have found that students using adaptive platforms show strong in-system performance while demonstrating limited transfer to external assessments. They learn to do well on the system's questions without developing the underlying cognitive structures that would generalise beyond it. This is the adaptive learning equivalent of teaching to the test — optimising for measurable proxies rather than the underlying goal.
There is also a motivation paradox embedded in well-designed gamification. Systems that reward correct answers and make the experience frictionless may actually undermine the development of intrinsic motivation — the durable, self-directed curiosity that is ultimately what we want education to cultivate. When learning is always smooth, always encouraging, always immediately gratifying, it becomes ill-suited preparation for the experiences of real intellectual work: the confusion, the false starts, the slow satisfaction of genuine understanding.
Mastery-based learning — the pedagogical principle that students should not advance until they have genuinely consolidated what came before — is sound. But what "mastery" means to an adaptive algorithm (getting a certain percentage of questions right in a session) and what it means to a skilled teacher (flexible, transferable, deep understanding) are not always the same thing.
What Good Looks Like
This is not an argument against using adaptive technology in schools. It is an argument for deploying it honestly, as the supplement it is rather than the solution it has been sold as.
Adaptive platforms can free up teacher time by handling repetitive practice and flagging individual gaps — if teachers are then able to use that time for the high-value interactions the research consistently identifies as most impactful: direct feedback, discussion, guided inquiry, the kind of teaching that requires a human who can read a room.
The 2 Sigma problem that Benjamin Bloom identified remains largely unsolved. What one-on-one tutoring actually delivers — moment-to-moment responsiveness to a specific learner's reasoning, not just their answer — is still something a human tutor does better than any algorithm currently deployed at scale.
The promise of adaptive learning is real. But the gap between the promise and the current reality is also real, and it is worth being honest about — particularly with the parents, educators, and policymakers making decisions about where to invest.
Technology does not teach. It can support teaching. The difference is not semantic.
References (APA style):
Bloom, B. S. (1984). The 2 sigma problem: The search for methods of group instruction as effective as one-to-one tutoring. Educational Researcher, 13(6), 4–16.
VanLehn, K. (2011). The relative effectiveness of human tutoring, intelligent tutoring systems, and other tutoring systems. Educational Psychologist, 46(4), 197–221.
Hattie, J. (2009). Visible Learning: A Synthesis of Over 800 Meta-Analyses Relating to Achievement. Routledge.
Education Endowment Foundation. (2019). Using Digital Technology to Improve Learning: Guidance Report. educationendowmentfoundation.org.uk
AI Disclosure: Research and organization for this article were assisted by AI tools; all factual claims and citations were independently verified against primary academic sources, and the analysis and conclusions are the author's own. The featured image was generated using Canva AI.
← Previous: The Empathy Gap: Why AI Can Simulate But Not Feel | → Next: Decision Fatigue in the Algorithmic Age
Related reading from Between Brain & Binary: Digital Psychiatry and the Web: Minds Online
Comments
Post a Comment