Picked by rule: the highest evidence levels first (studies with real learners), then newest.
Level 3: Controlled comparisonLearners: Adult learnersOutcome: Learning or achievementPreprintPreregisteredShort exposureStudy type: Randomized controlled trial
2026-09-26 · arXiv (back catalog) · Study (preprint)
Chatbots deliver real-time fact-checks, but whether a chatbot correction leaves anything behind once the chatbot is gone - metalearning, distinct from correcting misbeliefs - is untested. We report a preregistered, three-country randomized experiment (USA, India, Singapore; N ~ 2,200) on out-of-context image...
Why this label for Chatbot Engagement Does Not Always Beget Metalearning: Evidence from Three Count
- Level 3: Controlled comparison: Both graders agree; rules version missed it. Consensus level confirmed.
- Review status: Confirmed against the published method (AI-assisted review)
- Grader A: R5 randomized participants across four conditions with a one-week unaided discernment retest; R7 preprint cap.
- Grader B: R5: preregistered randomized experiment with N~2,200 real adults and a discernment (learning) outcome at one-week retest gives Level 3; R7 preprint and single-session caps hold it at 3, and as a preprint it cannot be a Level-4 candidate.
- Rule version: METHODOLOGY v0.3, pipeline v1, rules rules-v0.4
Rated blind by two AI graders against our published method. Confirmed against the published method (AI-assisted review) Method
Level 3: Controlled comparisonLearners: Higher-ed studentsOutcome: Learning or achievementPreprintStudy type: Quasi-experimental
2026-07-23 · arXiv (back catalog) · Study (preprint)
The availability of generative AI (GenAI) has raised concerns that college students substitute GenAI for cognitive effort, earning high grades without learning. If this "GenAI substitution hypothesis" is true, grades should rise disproportionately in GenAI-susceptible courses--those relying more on assessments like...
Why this label for Generative AI Availability, Grades, and Student Satisfaction at a Large Universi
- Level 3: Controlled comparison: Both graders agree; rules version missed it. Consensus level confirmed.
- Review status: Confirmed against the published method (AI-assisted review)
- Grader A: R5 design ceiling: the difference-in-differences analysis compares courses before and after AI availability using baseline administrative data.
- Grader B: R5 design ceiling: difference-in-differences quasi-experiment on administrative data (138,386 students) with a comparison and a pre-period gives Level 3; R7 preprint cap is 3; grades are a proxy for learning, so review for R6 indirectness
- Rule version: METHODOLOGY v0.3, pipeline v1, rules rules-v0.4
- Cognitive load: matched 'cognitive effort'
Rated blind by two AI graders against our published method. Confirmed against the published method (AI-assisted review) Method
Level 3: Controlled comparisonLevel-4 candidate (held at 3: preprint, needs full-text human check)Outcome: Learning or achievementPreprintStudy type: Randomized controlled trial
2026-04-06 · arXiv (back catalog) · Study (preprint)
People often optimize for long-term goals in collaboration: A mentor or companion doesn't just answer questions, but also scaffolds learning, tracks progress, and prioritizes the other person's growth over immediate results. In contrast, current AI systems are fundamentally short-sighted collaborators -- optimized for...
Why this label for AI Assistance Reduces Persistence and Hurts Independent Performance
- Level 3: Controlled comparison (human override): Label correction only, level unchanged. The arXiv abstract describes participants only as 'people' and does not say how long they used the AI, so the automated labels 'Learners: Adult learners' and 'Short exposure' are not supported by the source. Removed to match the corrected explainer.
- Changed from: Level 3: Controlled comparison
- Grader A: R5 design ceiling: randomized controlled trials assign human participants and measure unassisted performance; Level 4 verification and uncertainty information are not reported.
- Grader B: R5 design: series of RCTs (N=1,222) on unassisted performance and persistence; R7 preprint and short-exposure caps hold at max Level 3; human review needed before any upgrade.
- Rule version: METHODOLOGY v0.3, pipeline v1, rules rules-v0.4
- Cognitive load: matched 'cognitive offloading'
- Reading and writing: matched 'reading comprehension'
Rated blind by two AI graders against our published method. Human rating change 2026-10-24, approved by Matt, Oct 24 2026 Method
Level 2: CorrelationalProvisional (rules only)
2026-10-23 · Frontiers in Education · Study
This study reconceptualizes AI Governance Readiness (AIGR) as a form of institutional capital, representing an institution's capacity to establish, coordinate, and operationalize responsible AI governance arrangements in ways associated with student retention intentions through capability orchestration rather than...
Why this label for AI governance readiness as institutional capital: a capability-orchestration fra
- Level 2: Correlational: provisional rules-only rating, pending grader consensus and review: R5 design: correlational: Associational analysis of real-participant data.
- Policy and governance: matched 'governance'
Provisional label from fixed rules applied to the collected text; research items are re-rated by our review pipeline. Method
Level 2: CorrelationalStudy type: Quasi-experimental
2026-10-19 · Frontiers in Education · Study
Effective communication is fundamental to high-quality healthcare. Communication skills training is a core component of medical curricula, yet opportunities to practice with standardized patients (SPs) are limited, costly, and can elicit anxiety in novice learners.
Why this label for Use of a GenAI-patient for medical communication training for undergraduate medi
- Level 2: Quasi-experiment, n=383, groups allocated by class schedule; outcomes are self-reported nervousness, preparedness and self-efficacy, so the perception-only cap holds it at Level 2.
- Review status: Rated by one AI grader from the abstract, on Matt's delegation; not yet human-verified
Rated by one AI grader from the abstract, on Matt's delegation; not yet human-verified. Method
