Myth: AI tutors and AI scoring tools have been shown to help students learn

Some papers about AI tutors and AI tools that score student work test how the AI performs: its answers, its scores or its feedback. Often they do not measure whether students learned more. Across all topics, 58 studies in our tracker are labeled Technical test as of 2026-10-26, which means the study did not test whether students learned. Some examples:

• MisconceptDialogue: Evaluation Criteria Shape Measured Success in LLM Math Tutoring (arXiv, a site for papers posted before expert review, 2026-10-26), labeled Technical test
• The learner who does not learn: when optimizing a pedagogical metric degrades LLM tutoring (arXiv, a site for papers posted before expert review, 2026-10-08), labeled Technical test
• Machine-human agreement in marking undergraduate assignments is specific to model, prompt and institution (arXiv, a site for papers posted before expert review, 2026-10-13), labeled Technical test

What to do instead

A technical test can tell you how the AI performed on the tasks tested. It cannot tell you whether students learn more with it. For that, look for a study with real learners and a learning outcome.

These counts come from our tracker on 2026-10-26 and change as new items are added. Labels come from our automatic first pass and are provisional until a person checks them. See the tracker, the Labels page and the Method page.