Level 3: Controlled comparisonLearners: Higher-ed studentsOutcome: Learning or achievementPreprintStudy type: Quasi-experimental
2026-07-23 · arXiv (back catalog) · Study (preprint)
The availability of generative AI (GenAI) has raised concerns that college students substitute GenAI for cognitive effort, earning high grades without learning. If this "GenAI substitution hypothesis" is true, grades should rise disproportionately in GenAI-susceptible courses--those relying more on assessments like...
Why this label for Generative AI Availability, Grades, and Student Satisfaction at a Large Universi
- Level 3: Controlled comparison: Both graders agree; rules version missed it. Consensus level confirmed.
- Review status: Confirmed against the published method (AI-assisted review)
- Grader A: R5 design ceiling: the difference-in-differences analysis compares courses before and after AI availability using baseline administrative data.
- Grader B: R5 design ceiling: difference-in-differences quasi-experiment on administrative data (138,386 students) with a comparison and a pre-period gives Level 3; R7 preprint cap is 3; grades are a proxy for learning, so review for R6 indirectness
- Rule version: METHODOLOGY v0.3, pipeline v1, rules rules-v0.4
- Cognitive load: matched 'cognitive effort'
Graded blind by two AI graders against our published method. Confirmed against the published method (AI-assisted review) Method
Level 3: Controlled comparisonLevel-4 candidate (held at 3: preprint, needs full-text human check)Outcome: Learning or achievementPreprintStudy type: Randomized controlled trial
2026-04-06 · arXiv (back catalog) · Study (preprint)
People often optimize for long-term goals in collaboration: A mentor or companion doesn't just answer questions, but also scaffolds learning, tracks progress, and prioritizes the other person's growth over immediate results. In contrast, current AI systems are fundamentally short-sighted collaborators -- optimized for...
Why this label for AI Assistance Reduces Persistence and Hurts Independent Performance
- Level 3: Controlled comparison (human override): Label correction only, level unchanged. The arXiv abstract describes participants only as 'people' and does not say how long they used the AI, so the automated labels 'Learners: Adult learners' and 'Short exposure' are not supported by the source. Removed to match the corrected explainer.
- Changed from: Level 3: Controlled comparison
- Grader A: R5 design ceiling: randomized controlled trials assign human participants and measure unassisted performance; Level 4 verification and uncertainty information are not reported.
- Grader B: R5 design: series of RCTs (N=1,222) on unassisted performance and persistence; R7 preprint and short-exposure caps hold at max Level 3; human review needed before any upgrade.
- Rule version: METHODOLOGY v0.3, pipeline v1, rules rules-v0.4
- Cognitive load: matched 'cognitive offloading'
- Reading and writing: matched 'reading comprehension'
Graded blind by two AI graders against our published method. Human grade change 2026-10-24, approved by Matt, Oct 24 2026 Method
Level 3: Controlled comparisonProvisional (rules only)
2026-03-02 · ERIC (peer reviewed) · Study
This study investigated whether wakeful targeted memory reactivation (TMR) during short rest intervals improves motor learning. Participants were randomly assigned to three groups and performed a sequential key-press task under each condition: (1) TMR[subscript regular] group: auditory cues played at the same speed as...
Why this label for Wakeful Targeted Memory Reactivation during Short Rest Periods Modulates Early M
- Level 3: Controlled comparison: provisional rules-only grade, pending grader consensus and review: R5 design: randomized: Randomized design with real participants.
Provisional label from fixed rules applied to the collected text; research items are re-graded by our review pipeline. Method
Level 2: CorrelationalLearners: Higher-ed studentsOutcome: Behavior or engagementPreprintPreregisteredPerception onlyStudy type: Cohort / case-control
2026-10-07 · arXiv (back catalog) · Study (preprint)
AI learning tools are rapidly entering classrooms, but evidence about whether they help students learn is mixed and rests mostly on test scores. Comparatively less research addresses whether the use of AI changes students' live learning behaviors in class.
Why this label for How assigned AI use before class shapes active student engagement in class
- Level 2: Correlational: Both graders agree; rules version missed it. Consensus level confirmed.
- Review status: Confirmed against the published method (AI-assisted review)
- Grader A: R7 hard cap: randomized assignment is reported, but the measured outcome is classroom engagement rather than learning.
- Grader B: R7 engagement-only cap: a preregistered randomized field experiment (n=759 MBA students) that would otherwise be Level 3, but outcomes are class contributions and self-reported perceptions, with no learning measure, so max 2
- Rule version: METHODOLOGY v0.3, pipeline v1, rules rules-v0.4
- Motivation and engagement: matched 'student engagement'
Graded blind by two AI graders against our published method. Confirmed against the published method (AI-assisted review) Method
Level 2: CorrelationalLearners: Higher-ed studentsOutcome: Learning or achievementPreprintPreregisteredSmall or impreciseStudy type: Cohort / case-control
2026-09-21 · arXiv (back catalog) · Study (preprint)
Students increasingly study alongside generative artificial intelligence (AI), yet unguided access to fluent answers invites cognitive offloading, and there is little evidence on which configurations of AI tutoring produce learning. Two design margins are usually bundled together: pedagogical structure (how the tutor...
Why this label for When AI Tutors Speak: Evidence from a Randomized Field Experiment
- Level 2: Correlational: Consensus level confirmed against the methodology.
- Review status: Confirmed against the published method (AI-assisted review)
- Grader A: R5 randomized 86 MBA students with a learning outcome starts at Level 3; R6 imprecision lowers it because no interval or power information is reported.
- Grader B: R5: a preregistered randomized field experiment with a learning outcome starts at Level 3; R6 imprecision (n=86, under 100, and only p-values with no interval or power information) lowers it to Level 2.
- Rule version: METHODOLOGY v0.3, pipeline v1, rules rules-v0.4
- Cognitive load: matched 'cognitive offloading'
- Tutoring and personalization: matched 'Tutors'
Graded blind by two AI graders against our published method. Confirmed against the published method (AI-assisted review) Method
Level 2: CorrelationalLearners: K-12 studentsOutcome: Learning or achievementPreprintShort exposureResearcher-made measure
2026-07-14 · arXiv (back catalog) · Study (preprint)
This study proposes a semi-automated system for generating dialogue-based lessons using Large Language Models (LLMs) and Text-to-Speech (TTS) technology, and exploratorily examines its educational potential via a practical quasi-experiment. The system augments rather than replaces educators through a three-stage...
Why this label for A Semi-Automated System for Generating Dialogue-Based TTS Lessons Using Large La
- Level 2: Correlational: Change to 2. Within-subject comparison of three lesson formats with 245 high school students; no baseline equivalence or counterbalancing reported, so 2.
- Review status: Confirmed against the published method (AI-assisted review)
- Grader A: R5 quasi-experiment lacks an equivalent baseline comparison because lesson formats were given in fixed order with different content.
- Grader B: R5/R6: a fixed-order within-subject quasi-experiment with 245 high school students. Format is confounded with content and there is no equivalent comparison, so the ceiling is Level 2. The measurement downgrade (researcher-made outcomes with no reliability reported) lowers it to Level 1.
- Rule version: METHODOLOGY v0.3, pipeline v1, rules rules-v0.4
- Cognitive load: matched 'cognitive load'
Graded blind by two AI graders against our published method. Confirmed against the published method (AI-assisted review) Method
Level 2: CorrelationalLearners: K-12 studentsOutcome: BehaviorPreprintStudy type: Cohort / case-control
2026-06-25 · arXiv (back catalog) · Study (preprint)
Generative AI (GAI) creates new opportunities for collaborative problem-solving (CPS), yet its role in shaping student interaction remains unclear. To address this gap, we conducted a six-week quasi-experimental study with 201 fifth-grade students in two conditions: with and without GAI.
Why this label for Beyond Information Retrieval: Generative AI as an Epistemic Arbiter to Enhance C
- Level 2: Correlational: Consensus level confirmed against the methodology.
- Review status: Confirmed against the published method (AI-assisted review)
- Grader A: R5 design ceiling: quasi-experimental comparison without reported baseline equivalence; R7 caps engagement outcomes at 2.
- Grader B: R7 cap: a six-week quasi-experiment with 201 fifth graders and a control group, but the only outcome is collaborative behavior and no baseline data is reported, so behavior/engagement-only is capped at Level 2
- Rule version: METHODOLOGY v0.3, pipeline v1, rules rules-v0.4
Graded blind by two AI graders against our published method. Confirmed against the published method (AI-assisted review) Method
Level 2: CorrelationalLearners: Higher-ed studentsOutcome: Learning or achievementPreprintSmall or impreciseStudy type: Cohort / case-control
2026-06-21 · arXiv (back catalog) · Study (preprint)
Pretesting - attempting problems before instruction - supports learning by activating prior knowledge and sharpening attention to subsequent instruction. Recent work suggests that adaptive AI-assisted pretesting can yield further advantages, particularly for tasks requiring higher-order reasoning, yet it remains...
Why this label for Do Gains from Generative AI-Enabled Adaptive Pretesting Persist? Evidence from a
- Level 2: Correlational: Randomized retention study, but sample size not reported; held at 2 until n is verified.
- Review status: Confirmed against the published method (AI-assisted review)
- Grader A: R5 design ceiling: undergraduates were randomly assigned to study conditions and assessed on posttest performance.
- Grader B: R5 RCT (undergraduates randomly assigned, seven-week retention, learning outcome) gives Level 3; R6 imprecision lowers it to 2 because n and intervals are not reported (RCT without n, needs human review)
- Rule version: METHODOLOGY v0.3, pipeline v1, rules rules-v0.4
- Retrieval practice: matched 'retrieval practice'
- Spacing and interleaving: matched 'spaced retrieval'
Graded blind by two AI graders against our published method. Confirmed against the published method (AI-assisted review) Method
Level 2: CorrelationalLearners: Not reportedOutcome: Learning or achievementPreprintSmall or impreciseStudy type: Cohort / case-control
2026-06-16 · arXiv (back catalog) · Study (preprint)
Motivational factors such as self-efficacy and how favorably students feel toward practice play a crucial role in shaping learning, particularly in technology-supported environments. Yet, educational interventions often overlook how these factors interact with practice format.
Why this label for Self-Efficacy and Favorability Shape Learning from Tutoring Systems and Paper Pr
- Level 2: Correlational: Counterbalanced within-subject design with learning outcome; no sample size or interval reported, so held at 2.
- Review status: Confirmed against the published method (AI-assisted review)
- Grader A: R5 design ceiling: counterbalanced within-subject comparison uses matched problem sets and baseline self-efficacy, but Level 4 checks are unavailable.
- Grader B: R5: the headline finding is a moderation link between self-efficacy or favorability and learning gains (correlational, Level 2) inside a counterbalanced within-subject ITS vs paper comparison; n and intervals are not reported, so it stays at 2 pending review (accepted at EC-TEL 2026, not matched)
- Rule version: METHODOLOGY v0.3, pipeline v1, rules rules-v0.4
- Motivation and engagement: matched 'students' motivation'
- Tutoring and personalization: matched 'Tutoring'
Graded blind by two AI graders against our published method. Confirmed against the published method (AI-assisted review) Method
Level 2: CorrelationalProvisional (rules only)
2026-06-04 · ERIC (peer reviewed) · Study
Dialogic peer feedback in synchronous online discussions provides multiple educational benefits; however, its implementation is often constrained by students' cognitive and interactional limitations. To address this issue, this study investigated the effects of argumentation scaffold and role assignment on students'...
Why this label for Fostering Dialogic Peer Feedback in Synchronous Online Discussions: The Effect o
- Level 2: Correlational: provisional rules-only grade, pending grader consensus and review: R5 design: comparison without baseline (F5): Comparison groups but no baseline equivalence reported.
- Feedback: matched 'Peer Feedback'
- Metacognition and self-regulation: matched 'metacognition'
Provisional label from fixed rules applied to the collected text; research items are re-graded by our review pipeline. Method
Level 2: CorrelationalProvisional (rules only)
2026-04-18 · ERIC (peer reviewed) · Study
This study investigated the rehabilitative effects of a 12-week Tai Chi intervention on attentional bias and prefrontal cortex (PFC) activation in men with methamphetamine use disorder (MUD). Forty-eight male participants were randomized into a Tai Chi group or a moderate-intensity treadmill-running control group, of...
Why this label for Effects of Tai Chi on Attentional Bias in Men with Methamphetamine Use Disorder
- Level 2: Correlational: provisional rules-only grade, pending grader consensus and review: R5 design: comparison without baseline (F5): Comparison groups but no baseline equivalence reported.
Provisional label from fixed rules applied to the collected text; research items are re-graded by our review pipeline. Method
Level 1: Descriptive or exploratoryStudy type: Cross-sectional
2026-10-19 · Frontiers in Education · Study
The advent of generative artificial intelligence (AI) is transforming academic production and confronting the social sciences with the challenge of preserving critical thinking in the face of mechanically generated content. This research examines the perceptions and practices of social science lecturers and students...
Why this label for Perceptions, practices and governance of generative AI in university social scie
- Level 1: Mixed-methods: interviews, non-probabilistic survey of 200 students, policy content analysis. Perception outcomes.
- Review status: Graded by one AI grader from the abstract, on Matt's delegation; not yet human-verified
Graded by one AI grader from the abstract, on Matt's delegation; not yet human-verified. Method
Level 1: Descriptive or exploratoryProvisional (rules only)
2026-10-19 · arXiv physics.ed-ph · Study (preprint)
This chapter examines the use of connectivism as a pedagogical approach in preservice physics teacher education through a classroom experience conducted in a Contemporary Physics course at the Universidad Pedagógica y Tecnológica de Colombia (UPTC). The study involved 21 preservice teachers organized into four...
Why this label for Connectivism as an Innovative Didactic Experience in Preservice Physics Teacher
- Level 1: Descriptive or exploratory: provisional rules-only grade, pending grader consensus and review: R5 design: descriptive (F3): Descriptive, qualitative, pilot or deployment study with real participants.
- Preprint: not peer reviewed; capped at Level 3
Provisional label from fixed rules applied to the collected text; research items are re-graded by our review pipeline. Method
Level 1: Descriptive or exploratoryLearners: Higher-ed studentsOutcome: Behavior or engagementPreprintPerception onlyStudy type: Case study / case series
2026-09-22 · arXiv (back catalog) · Study (preprint)
This paper presents a multi-source classroom study conducted during a 10-week quarter in data science courses at Drexel University. We first investigate the behaviors of student engagement with large language models (LLMs) using four surveys across three data science courses.
Why this label for Student Use of LLMs and the Limits of AI-Generated Question Difficulty in Data S
- Level 1: Descriptive or exploratory: Consensus level confirmed against the methodology.
- Review status: Confirmed against the published method (AI-assisted review)
- Grader A: R5 classroom surveys describe student use and perceptions; response data also test LLM question ratings without a learner comparison.
- Grader B: R5: surveys and an item-difficulty correlational analysis in a 10-week course with no comparison group are descriptive (Level 1); the R7 perception/engagement cap also applies to learning claims.
- Rule version: METHODOLOGY v0.3, pipeline v1, rules rules-v0.4
- Retrieval practice: matched 'retrieval practice'
- Motivation and engagement: matched 'student engagement'
Graded blind by two AI graders against our published method. Confirmed against the published method (AI-assisted review) Method
Level 1: Descriptive or exploratoryLearners: Higher-ed studentsOutcome: Learning or achievementPreprintSmall or impreciseShort exposureResearcher-made measureStudy type: Case study / case series
2026-09-18 · arXiv (back catalog) · Study (preprint)
Generative AI can improve students' programming performance, but successful task completion may not reflect what they retain. We examined performance, retention, cognitive load, and ownership in a controlled between-subjects experiment with 59 undergraduate computer science students, 55 were retained for analysis.
Why this label for Your Programming Students' Cognition with ChatGPT: Higher Performance, Lower Ret
- Level 1: Descriptive or exploratory: Between-subjects experiment, 55 analyzed, no baseline equivalence and no interval; downgraded to 1.
- Review status: Confirmed against the published method (AI-assisted review)
- Grader A: R5 controlled comparison lacks explicit random assignment or baseline data; R6 imprecision lowers Level 2 for 55 analyzed students without an interval or power report.
- Grader B: R5: a controlled between-subjects experiment with a recall (learning) outcome is Level 3; R6 imprecision (n=55, no intervals) lowers it to Level 2. R7 short-exposure cap also noted.
- Rule version: METHODOLOGY v0.3, pipeline v1, rules rules-v0.4
- Cognitive load: matched 'cognitive load'
Graded blind by two AI graders against our published method. Confirmed against the published method (AI-assisted review) Method
Level 1: Descriptive or exploratoryLearners: Higher-ed studentsOutcome: Perception or attitudePreprintPerception onlySmall or impreciseStudy type: Cross-sectional
2026-09-16 · arXiv (back catalog) · Study (preprint)
Generative artificial intelligence (GenAI) is changing how students learn, yet the roles of course context, cognitive reliance, evaluation literacy, and early reliance remain underexplored. Using survey responses from 118 students across 12 AI-related courses at our institution, we examined differences in GenAI use...
Why this label for The Uneven Impact of Generative AI on Student Learning: Examining the Roles of R
- Level 1: Descriptive or exploratory: Survey of 118 students on perceived learning; perception-only descriptive evidence.
- Review status: Confirmed against the published method (AI-assisted review)
- Grader A: R5 multivariable survey associations support Level 2; R7 perceived learning and use do not establish learning achievement.
- Grader B: R5: a cross-sectional survey of 118 students with regression on self-reported perceived learning is a survey (Level 1); the R7 perception-only cap also applies.
- Rule version: METHODOLOGY v0.3, pipeline v1, rules rules-v0.4
- Reading and writing: matched 'Literacy'
Graded blind by two AI graders against our published method. Confirmed against the published method (AI-assisted review) Method
Level 1: Descriptive or exploratoryLearners: Adult learnersOutcome: Learning or achievementPeer-reviewed (journal reference listed)Small or impreciseShort exposureResearcher-made measureStudy type: Case study / case series
2026-09-01 · arXiv (back catalog) · Study
Does unrestricted AI access bypass the cognitive effort required for learning, or does it streamline knowledge acquisition? This paper reports on a study where we compare three designs for user-AI interaction in a learning context: (1) an unrestricted conversational bot like ChatGPT, (2) a pedagogically constrained...
Why this label for Socrates went Nuclear: Comparing Interaction Strategies for AI systems in a Lear
- Level 1: Descriptive or exploratory: Consensus level confirmed against the methodology.
- Review status: Confirmed against the published method (AI-assisted review)
- Grader A: R5 design ceiling permits Level 2 for a pre/post comparison without stated equivalence; R6 imprecision and measurement lower it to Level 1 for 50 learners and an unvalidated immediate test.
- Grader B: R5 three-condition comparison with pre-test, capped at 3; R7 short-exposure cap applies; R6 imprecision (n=50, d and p only, no interval) lowers to 2; R6 measurement (researcher-made immediate post-test, no reliability) lowers to 1; construct flag: the authors say the chatbot's gains reflect answer retrieval rather than deeper learning
- Rule version: METHODOLOGY v0.3, pipeline v1, rules rules-v0.4
- Cognitive load: matched 'cognitive effort'
- Tutoring and personalization: matched 'tutoring'
Graded blind by two AI graders against our published method. Confirmed against the published method (AI-assisted review) Method
Level 1: Descriptive or exploratoryLearners: Higher-ed studentsOutcome: Learning or achievementPreprintResearcher-made measureSmall or impreciseStudy type: Case study / case series
2026-08-25 · arXiv (back catalog) · Study (preprint)
Worked examples are an important part of introductory programming, but reading their expert explanations is passive. Self explanation, students explaining the problem and its solution to themselves with subgoal level analysis, converts passive reading into an active study of worked example, yet it is hard to scale...
Why this label for Self-Explanation Tutor for Active Study of CS1 Worked Examples
- Level 1: Descriptive or exploratory: Consensus level confirmed against the methodology.
- Review status: Confirmed against the published method (AI-assisted review)
- Grader A: R5 design ceiling: the course deployment reports student revisions and learning evidence without a stated comparison group.
- Grader B: R4 mixed study, so only the real-learner arm is graded (the LLM-vs-human-rater arm is T); that arm is a single-group course deployment, so R5 sets Level 1
- Rule version: METHODOLOGY v0.3, pipeline v1, rules rules-v0.4
- Worked examples and explicit instruction: matched 'Worked Examples'
- Tutoring and personalization: matched 'Tutor'
Graded blind by two AI graders against our published method. Confirmed against the published method (AI-assisted review) Method
Level 1: Descriptive or exploratoryLearners: Higher-ed studentsOutcome: Learning or achievementPreprintSmall or impreciseResearcher-made measureShort exposureStudy type: Case study / case series
2026-07-17 · arXiv (back catalog) · Study (preprint)
Generative AI (GenAI) is increasingly used by students for programming explanation, debugging, and assignment support. Yet unrestricted large language model (LLM) tutors can hallucinate, contradict course policy, reveal complete solutions, and foster passive dependence.
Why this label for EduGuard: A Safe RAG-Based LLM Tutor for Programming Education
- Level 1: Descriptive or exploratory: Consensus level confirmed against the methodology.
- Review status: Confirmed against the published method (AI-assisted review)
- Grader A: R5 controlled pilot with 10 learners has a Level 2 ceiling; R6 imprecision lowers it to Level 1.
- Grader B: R4 mixed study graded only on the real-learner arm. That arm is a counterbalanced pilot with 10 undergraduates and an immediate post-test only, and pilots are Level 1 under sec. 6. The benchmark part would be T.
- Rule version: METHODOLOGY v0.3, pipeline v1, rules rules-v0.4
- Tutoring and personalization: matched 'Tutor'
Graded blind by two AI graders against our published method. Confirmed against the published method (AI-assisted review) Method
Level 1: Descriptive or exploratoryLearners: Higher-ed studentsOutcome: Learning or achievementPreprintSmall or impreciseShort exposureResearcher-made measureStudy type: Case study / case series
2026-07-04 · arXiv (back catalog) · Study (preprint)
Writing is cognitively demanding and anxiety-provoking for English as a Foreign Language (EFL) learners, especially under time pressure. This paper presents The Brand War, a web-based gamified writing application combining competitive game mechanics with iterative GPT-4.1-powered formative feedback for undergraduate...
Why this label for The Brand War: A Gamified AI-Feedback System for Time-Limited EFL Writing
- Level 1: Descriptive or exploratory: Consensus level confirmed against the methodology.
- Review status: Confirmed against the published method (AI-assisted review)
- Grader A: R5 design ceiling: the 29-student exploratory, single-session study lacks an equivalent comparison group.
- Grader B: R5 design ceiling: exploratory single-session, single-group within-student study (n=29) with AI-scored revisions; user vs non-user comparison is self-selected and not significant
- Rule version: METHODOLOGY v0.3, pipeline v1, rules rules-v0.4
- Feedback: matched 'formative feedback'
Graded blind by two AI graders against our published method. Confirmed against the published method (AI-assisted review) Method
Level 1: Descriptive or exploratoryLearners: K-12 studentsOutcome: Learning or achievementPreprintSmall or impreciseShort exposureStudy type: Case study / case series
2026-06-26 · arXiv (back catalog) · Study (preprint)
Generative AI systems (GenAI) are increasingly used by students as learning companions, yet little is known about how they regulate these tools in open-ended learning settings, where the goal is to improve understanding for later independent performance. This study examined Grade-9 students' use of a general-purpose...
Why this label for Changes in Help-Seeking Strategies Predict unaided Performance during AI-based M
- Level 1: Descriptive or exploratory: Consensus level confirmed against the methodology.
- Review status: Confirmed against the published method (AI-assisted review)
- Grader A: R6 imprecision: the controlled association uses 89 learners with computable trajectories and reports no interval or power information.
- Grader B: R5 correlational design with pre-test control (Level 2 ceiling) on Grade-9 students, lowered one level by R6 imprecision (analytic n=95/89, no interval reported)
- Rule version: METHODOLOGY v0.3, pipeline v1, rules rules-v0.4
- Metacognition and self-regulation: matched 'self-regulated'
- Tutoring and personalization: matched 'tutor'
Graded blind by two AI graders against our published method. Confirmed against the published method (AI-assisted review) Method
Level 1: Descriptive or exploratoryLearners: Higher-ed studentsOutcome: Perception or attitudePreprintSmall or impreciseStudy type: Case study / case series
2026-06-26 · arXiv (back catalog) · Study (preprint)
Large language models can complete most of the assignments in an introductory artificial intelligence course. This paper is an experience report on redesigning one such course, CSS~382 at the University of Washington Bothell, in response.
Why this label for Teaching Intro AI When the Tools Can Do the Homework: A Course Redesign and a St
- Level 1: Descriptive or exploratory: Both graders agree; rules version missed it. Consensus level confirmed.
- Review status: Confirmed against the published method (AI-assisted review)
- Grader A: R5 design ceiling: the single-cohort course experience report describes student artifacts and objections without a comparison group.
- Grader B: R5 design ceiling: single-cohort experience report on a course redesign, with no outcome measurement or comparison
- Rule version: METHODOLOGY v0.3, pipeline v1, rules rules-v0.4
Graded blind by two AI graders against our published method. Confirmed against the published method (AI-assisted review) Method
Level 1: Descriptive or exploratoryLearners: K-12 studentsOutcome: Learning or achievementPreprintResearcher-made measureStudy type: Case study / case series
2026-06-26 · arXiv (back catalog) · Study (preprint)
Research indicates students require customized written composition feedback to enhance critical thinking and writing competence, yet teachers face barriers to delivering timely personalized guidance due to heavy workloads. To address this gap, this study developed the Writing Improvement and Smart Evaluation Agent...
Why this label for Using AI-Generated Feedback to Improve Critical Thinking and Writing Proficiency
- Level 1: Descriptive or exploratory: Both graders agree; rules version missed it. Consensus level confirmed.
- Review status: Confirmed against the published method (AI-assisted review)
- Grader A: R5 design ceiling: three-month single-group intervention; no comparison group reported.
- Grader B: R5 design ceiling: a three-month intervention with 260 sixth graders, but no comparison group is described, so it is a single-group pre/post (Level 1)
- Rule version: METHODOLOGY v0.3, pipeline v1, rules rules-v0.4
- Feedback: matched 'AI-Generated Feedback'
Graded blind by two AI graders against our published method. Confirmed against the published method (AI-assisted review) Method
Level 1: Descriptive or exploratoryLearners: Adult learnersOutcome: Learning or achievementPreprintSmall or impreciseShort exposureStudy type: Cross-sectional
2026-05-07 · arXiv (back catalog) · Study (preprint)
Advance in technology offer the potential for future adoption of a combination of virtual reality (VR) and real-time adaptivity to enhance training and education. Providing a valid neuro-ergonomic measure of cognitive load can enable an adaptive training regime to continuously adjust tas difficulty to an optimal level...
Why this label for Leveraging fNIRS to Evaluate Workload for Adaptive Training in Virtual Reality
- Level 1: Descriptive or exploratory: Consensus level confirmed against the methodology.
- Review status: Confirmed against the published method (AI-assisted review)
- Grader A: R5 design ceiling: 36 people complete one training task and retention test, with no comparison group reported.
- Grader B: R5 design ceiling: single-session study validating an fNIRS workload measure (n=36); the outcome is the cognitive-load measure, not a learning comparison
- Rule version: METHODOLOGY v0.3, pipeline v1, rules rules-v0.4
- Cognitive load: matched 'cognitive load'
Graded blind by two AI graders against our published method. Confirmed against the published method (AI-assisted review) Method
Level 1: Descriptive or exploratoryLearners: Higher-ed studentsOutcome: Behavior or engagementPreprintSmall or impreciseResearcher-made measureStudy type: Case study / case series
2026-04-16 · arXiv (back catalog) · Study (preprint)
Generative AI (GenAI) holds significant promise for advancing educational equity among ethnic minority students by broadening access to learning resources and mitigating linguistic barriers. However, these benefits are counterbalanced by the risk of cognitive laziness, whereby students may treat GenAI as an answer...
Why this label for Scaffolding Critical Engagement with GenAI: Transforming Ethnic Minority Prepara
- Level 1: Descriptive or exploratory: Consensus level confirmed against the methodology.
- Review status: Confirmed against the published method (AI-assisted review)
- Grader A: R5 design ceiling: design-based research with 78 students, discourse analysis, and paired pre/post self-efficacy measures has no comparison group.
- Grader B: R5 design ceiling: design-based research with a single group (n=78) and paired pre/post t-tests = Level 1; outcomes are discourse and self-efficacy, not achievement; accepted at AIED 2026.
- Rule version: METHODOLOGY v0.3, pipeline v1, rules rules-v0.4
Graded blind by two AI graders against our published method. Confirmed against the published method (AI-assisted review) Method
Level 1: Descriptive or exploratoryProvisional (rules only)
2026-03-30 · ERIC (peer reviewed) · Study
Background: Children with attention deficit hyperactivity disorder (ADHD) are vulnerable to the psychological impact of traumatic events due to inherent difficulties in emotion regulation and executive functioning. This study examines the predictive roles of attachment styles and family functioning on depressive,...
Why this label for Attachment Style and Family Functioning as Predictors of Trauma Symptoms in Chil
- Level 1: Descriptive or exploratory: provisional rules-only grade, pending grader consensus and review: R5 design: descriptive (F3): Descriptive, qualitative, pilot or deployment study with real participants.
Provisional label from fixed rules applied to the collected text; research items are re-graded by our review pipeline. Method
Level 1: Descriptive or exploratoryProvisional (rules only)
2026-02-17 · ERIC (peer reviewed) · Study
Question-asking is central to human communication, cognitive development, and learning. While prior research has examined questioning in controlled, non-longitudinal settings, little is known about how the originality and complexity of the questions people ask change over time or how these abilities respond to...
Why this label for Knowledge Reshapes Inquiry by Changing Question Asking Ability and Impacting Aca
- Level 1: Descriptive or exploratory: provisional rules-only grade, pending grader consensus and review: R5 design: descriptive (F3): Descriptive, qualitative, pilot or deployment study with real participants.
- Assessment and grading: matched 'exam'
Provisional label from fixed rules applied to the collected text; research items are re-graded by our review pipeline. Method
Level 1: Descriptive or exploratoryLearners: Higher-ed studentsOutcome: Learning or achievementPreprintSmall or impreciseShort exposureResearcher-made measureStudy type: Case study / case series
2025-07-08 · arXiv (back catalog) · Study (preprint)
Retrieval practice is a well-established pedagogical technique known to significantly enhance student learning and knowledge retention. However, generating high-quality retrieval practice questions is often time-consuming and labor intensive for instructors, especially in rapidly evolving technical subjects.
Why this label for Enhancing Student Learning with LLM-Generated Retrieval Practice Questions: An E
- Level 1: Descriptive or exploratory: Both graders agree; rules version missed it. Consensus level confirmed.
- Review status: Confirmed against the published method (AI-assisted review)
- Grader A: R5 places the week-to-week comparison without equivalence data at Level 2; R6 imprecision lowers it for about 60 students with no interval or power information.
- Grader B: R5: a week with LLM-generated retrieval questions vs a week without (~60 students, no equivalence data) is Level 2, then the R6 imprecision downgrade (n<100, no interval) gives 1; one-week exposure and a researcher-made quiz
- Rule version: METHODOLOGY v0.3, pipeline v1, rules rules-v0.4
- Retrieval practice: matched 'Retrieval Practice'
Graded blind by two AI graders against our published method. Confirmed against the published method (AI-assisted review) Method
Technical testLearners: Not a learning studyOutcome: Model performanceStudy type: Technical test (no real learners)
2026-10-13 · arXiv search: AI + education · Study (preprint)
The rapid advancement of image-to-text extraction systems and large language models (LLMs) poses new challenges for educational assessment by making assignment materials easy to transcribe and solve with minimal human effort. To address this problem, we present AIRRI (Artificial Intelligence Resilient Reading...
Why this label for AIRRI: Towards a Human-Readable, Machine-Resistant Reading Interface
- Technical test: R4: OCR degradation tests; readability check is not a learning outcome.
- Review status: Graded by one AI grader from the abstract, on Matt's delegation; not yet human-verified
Graded by one AI grader from the abstract, on Matt's delegation; not yet human-verified. Method
Technical testLearners: Not a learning studyOutcome: Model performanceStudy type: Technical test (no real learners)
2026-10-07 · arXiv search: AI + education · Study (preprint)
As Large Language Models (LLMs) are increasingly adopted for automated grading and feedback in higher education, their structured outputs, including multi-dimensional rubric scores, detailed feedback comments, and improvement suggestions, create an appearance of thorough analytic evaluation. This study examines...
Why this label for Beyond Score Accuracy: Examining the Diagnostic Quality of LLM-Generated Structu
- Technical test: R4: LLM grading vs instructor scores on an existing dataset; outcome is model performance.
- Review status: Graded by one AI grader from the abstract, on Matt's delegation; not yet human-verified
Graded by one AI grader from the abstract, on Matt's delegation; not yet human-verified. Method
Technical testLearners: Higher-ed studentsOutcome: Model performancePreprintStudy type: Technical test (no real learners)
2026-10-07 · arXiv (back catalog) · Study (preprint)
As Large Language Models (LLMs) are increasingly adopted for automated grading and feedback in higher education, their structured outputs, including multi-dimensional rubric scores, detailed feedback comments, and improvement suggestions, create an appearance of thorough analytic evaluation. This study examines...
Why this label for Beyond Score Accuracy: Examining the Diagnostic Quality of LLM-Generated Structu
- Technical test: Two independent AI graders gave the same level.
- Review status: Confirmed against the published method (AI-assisted review): both graders agreed
- Grader A: R4 real-learner gate: student responses provide test data, while the outcomes measure LLM grading and feedback quality.
- Grader B: R4 real-learner gate: compares LLM grades and feedback with instructors on existing student responses; the only outcome is model output quality
- Rule version: METHODOLOGY v0.3, pipeline v1, rules rules-v0.4
Graded blind by two AI graders against our published method. Confirmed against the published method (AI-assisted review): both graders agreed Method
Technical testProvisional (rules only)Study type: Technical test (no real learners)
2026-10-01 · arXiv physics.ed-ph · Study (preprint)
Multiple-choice questions are a staple of educational assessment due to their efficiency, but their diagnostic ability is inherently limited. Traditional psychometric techniques infer misconceptions by fitting models to predetermined data patterns, making them ineffective for analysing small, novel, or...
Why this label for Determining the degree of randomness in multiple-choice question distractors
- Technical test: provisional rules-only grade, pending grader consensus and review: R4 real-learner gate (F1/F2): Outputs of models, simulations or audits; any humans act as raters, references or data sources, not learners.
- Preprint: not peer reviewed; capped at Level 3
Provisional label from fixed rules applied to the collected text; research items are re-graded by our review pipeline. Method
Technical testLearners: Adults not in a learning role (raters)Outcome: Model performancePreprintResearcher-made measureStudy type: Technical test (no real learners)
2026-09-22 · arXiv (back catalog) · Study (preprint)
This study aimed to automate hierarchical diagnostic reasoning (HDR), a constructive method for evaluating practical judgment skills, by developing and testing an evaluation process using a large language model. HDR is a descriptive task that measures higher-order cognitive skills by requiring students to identify and...
Why this label for Automating Constructive Assessment with Large Language Models: Toward Scalable a
- Technical test: Two independent AI graders gave the same level.
- Review status: Confirmed against the published method (AI-assisted review): both graders agreed
- Grader A: R4 the reported results validate automated problem generation, scoring, and feedback against ratings, not learner outcomes.
- Grader B: R4 real-learner gate: the outcomes are LLM item-generation validity, human-AI scoring agreement and rater judgments of feedback, which is model performance only; no student learning outcome.
- Rule version: METHODOLOGY v0.3, pipeline v1, rules rules-v0.4
Graded blind by two AI graders against our published method. Confirmed against the published method (AI-assisted review): both graders agreed Method
Technical testLearners: Simulated or AI-onlyOutcome: Model performancePreprintVendor-linkedStudy type: Technical test (no real learners)
2026-08-01 · arXiv (back catalog) · Study (preprint)
MagicSchool's K-12 AI product suite is used by millions of teachers and serves millions of teacher and student messages each month. Our team monitors output along four priority dimensions--student safety, tone and instructional role, pedagogical value, and structural output quality--and tracks how often it fails on...
Why this label for When Evaluators Cry Wolf: Lessons from Production LLM-as-Judge Evaluation in Edu
- Technical test: Two independent AI graders gave the same level.
- Review status: Confirmed against the published method (AI-assisted review): both graders agreed
- Grader A: R4 real-learner gate: the reported outcomes evaluate model judges on synthetic cases, not student learning.
- Grader B: R4 real-learner gate: LLM-as-judge evaluators are tuned on synthetic datasets, and the only outcome is evaluator precision; the authors are from the vendor, MagicSchool.
- Rule version: METHODOLOGY v0.3, pipeline v1, rules rules-v0.4
Graded blind by two AI graders against our published method. Confirmed against the published method (AI-assisted review): both graders agreed Method
Technical testLearners: Adults not in a learning role (raters)Outcome: Perception or attitudePreprintShort exposurePerception onlySmall or impreciseStudy type: Technical test (no real learners)
2026-08-01 · arXiv (back catalog) · Study (preprint)
This innovative practice full paper presents BoilerSketch, a TA-supervised, diagram-first GenAI practice and tablet interface for providing structured visual explanations in CS1 and early CS2 support settings. Large early computing courses routinely face a support bottleneck during labs and office hours because many...
Why this label for BoilerSketch: A TA-Supervised, Diagram-First GenAI Practice for Structured Diagr
- Technical test: Tool paper; no learner outcomes reported.
- Review status: Confirmed against the published method (AI-assisted review)
- Grader A: R4 real-learner gate: instructional staff evaluated a prototype; no learner outcomes were studied.
- Grader B: R5 design ceiling: one 45-minute expert evaluation with a post-use survey of 21 instructional staff, with no comparison group, so descriptive Level 1 and no learner outcome
- Rule version: METHODOLOGY v0.3, pipeline v1, rules rules-v0.4
- Tutoring and personalization: matched 'tutoring'
Graded blind by two AI graders against our published method. Confirmed against the published method (AI-assisted review) Method
Technical testLearners: Simulated or AI-onlyOutcome: Model performancePreprintSimulated learnersStudy type: Technical test (no real learners)
2026-07-10 · arXiv (back catalog) · Study (preprint)
Social desirability and impression management are pervasive sources of response distortion in human personality assessment, yet their effects on Large Language Models (LLMs) remain underexplored. This study investigates whether contemporary LLMs systematically modulate the expression of Dark Triad traits...
Why this label for Faking Good and Faking Bad in LLMs: Response Distortion Across Dark Triad Person
- Technical test: Two independent AI graders gave the same level.
- Review status: Confirmed against the published method (AI-assisted review): both graders agreed
- Grader A: R4 real-learner gate: all experimental responses come from LLMs.
- Grader B: R4 real-learner gate: psychometric prompting of seven LLMs, with no human participants.
- Rule version: METHODOLOGY v0.3, pipeline v1, rules rules-v0.4
Graded blind by two AI graders against our published method. Confirmed against the published method (AI-assisted review): both graders agreed Method
Technical testLearners: Higher-ed studentsOutcome: Model performancePreprintStudy type: Technical test (no real learners)
2026-06-22 · arXiv (back catalog) · Study (preprint)
Large language models (LLMs) are increasingly used for estimating item difficulty in educational assessment. However, it remains unclear whether such estimates reflect how learners actually experience difficulty.
Why this label for The Easy Trap: Why LLMs Underestimate Misconception-Driven Difficulty
- Technical test: Two independent AI graders gave the same level.
- Review status: Confirmed against the published method (AI-assisted review): both graders agreed
- Grader A: R4 real-learner gate: student responses serve as reference data; the evaluated outcome is LLM difficulty-rating performance.
- Grader B: R4 real-learner gate: the only outcome is how well LLM difficulty ratings match item difficulty; student responses (n=770) are only the ground truth
- Rule version: METHODOLOGY v0.3, pipeline v1, rules rules-v0.4
Graded blind by two AI graders against our published method. Confirmed against the published method (AI-assisted review): both graders agreed Method
Technical testLearners: Simulated or AI-onlyOutcome: Model performancePreprintSimulated learnersStudy type: Technical test (no real learners)
2026-05-12 · arXiv (back catalog) · Study (preprint)
Large language models (LLMs) can fluently generate student-like responses, making them attractive as simulated students for training and evaluating AI tutors and human educators. Yet such simulators are typically evaluated by output similarity to real students, not by whether they behave like students with coherent...
Why this label for Simulating Students or Sycophantic Problem Solving? On Misconception Faithfulnes
- Technical test: Two independent AI graders gave the same level.
- Review status: Confirmed against the published method (AI-assisted review): both graders agreed
- Grader A: R4 real-learner gate: all tested students are LLM simulators.
- Grader B: R4 real-learner gate: an evaluation and post-training of LLM simulated students, with no real learners
- Rule version: METHODOLOGY v0.3, pipeline v1, rules rules-v0.4
- Tutoring and personalization: matched 'tutors'
Graded blind by two AI graders against our published method. Confirmed against the published method (AI-assisted review): both graders agreed Method
Technical testLearners: Simulated or AI-onlyOutcome: Model performancePreprintStudy type: Technical test (no real learners)
2026-05-12 · arXiv (back catalog) · Study (preprint)
Automatic generation of educational materials using large language models (LLMs) is becoming increasingly common, but assigning difficulty levels to such materials still requires substantial human effort. LLM-as-a-Judge has therefore attracted attention, yet disagreement with human raters remains a major challenge.
Why this label for Predicting Disagreement with Human Raters in LLM-as-a-Judge Difficulty Assessmen
- Technical test: Two independent AI graders gave the same level.
- Review status: Confirmed against the published method (AI-assisted review): both graders agreed
- Grader A: R4 real-learner gate: experiments assess model difficulty ratings against human raters, with no learners.
- Grader B: R4 real-learner gate: predicts LLM-judge disagreement with human difficulty ratings; the outcome is model AUC only; EDM poster acceptance not matched
- Rule version: METHODOLOGY v0.3, pipeline v1, rules rules-v0.4
Graded blind by two AI graders against our published method. Confirmed against the published method (AI-assisted review): both graders agreed Method
Technical testLearners: Simulated or AI-onlyOutcome: Model performancePreprintOpen data or materialsStudy type: Technical test (no real learners)
2026-05-10 · arXiv (back catalog) · Study (preprint)
Large language models are increasingly used in K-12 education, but existing benchmarks mainly test exam question answering rather than understanding how curriculum knowledge is structured and visually presented. We call this capability curriculum cognition.
Why this label for K12-KGraph: A Curriculum-Aligned Knowledge Graph for Benchmarking and Training E
- Technical test: Two independent AI graders gave the same level.
- Review status: Confirmed against the published method (AI-assisted review): both graders agreed
- Grader A: R4 real-learner gate: evaluates educational LLMs on curriculum benchmarks, not K-12 students.
- Grader B: R4 real-learner gate: curriculum knowledge-graph benchmark and fine-tuning corpus for LLMs; K-12 content but no learners took part
- Rule version: METHODOLOGY v0.3, pipeline v1, rules rules-v0.4
Graded blind by two AI graders against our published method. Confirmed against the published method (AI-assisted review): both graders agreed Method
Technical testLearners: Simulated or AI-onlyOutcome: Model performancePreprintStudy type: Technical test (no real learners)
2026-05-08 · arXiv (back catalog) · Study (preprint)
Automated short answer scoring (ASAS) is shifting from discriminative, fine-tuned models to large language models (LLMs) used in few-shot settings. This paradigm leverages LLMs broad world knowledge and ease of deployment, but limited task-specific data may reduce alignment on complex scoring tasks.
Why this label for Quality-Conditioned Agreement in Automated Short Answer Scoring: Mid-Range Degra
- Technical test: Two independent AI graders gave the same level.
- Review status: Confirmed against the published method (AI-assisted review): both graders agreed
- Grader A: R4 real-learner gate: student answers are existing test data for automated scoring, not a real-learner arm.
- Grader B: R4 real-learner gate: automated scoring agreement on existing student responses; the outcome is model-human agreement, not learner outcomes
- Rule version: METHODOLOGY v0.3, pipeline v1, rules rules-v0.4
Graded blind by two AI graders against our published method. Confirmed against the published method (AI-assisted review): both graders agreed Method
Technical testLearners: Adults not in a learning role (raters)Outcome: Model performancePeer-reviewed (journal reference listed)Simulated learnersStudy type: Technical test (no real learners)
2025-08-08 · arXiv (back catalog) · Study
Generative Artificial Intelligence (GenAI) holds a potential to advance existing educational technologies with capabilities to automatically generate personalised scaffolds that support students' self-regulated learning (SRL). While advancements in large language models (LLMs) promise improvements in the adaptability...
Why this label for Towards Reliable Generative AI-Driven Scaffolding: Reducing Hallucinations and E
- Technical test: Two independent AI graders gave the same level.
- Review status: Confirmed against the published method (AI-assisted review): both graders agreed
- Grader A: R4 real-learner gate: evaluation datasets and expert judgments assess generated scaffolds, without measuring student learning.
- Grader B: R4 real-learner gate: LLM-generated SRL scaffolds evaluated by multi-agent and LLM-as-judge systems against expert ratings; no learner outcomes (Computers & Education).
- Rule version: METHODOLOGY v0.3, pipeline v1, rules rules-v0.4
- Metacognition and self-regulation: matched 'Self-Regulated'
Graded blind by two AI graders against our published method. Confirmed against the published method (AI-assisted review): both graders agreed Method
CommentaryStudy type: Expert opinion / commentary
2026-10-19 · Frontiers in Education · Study
Female academic leaders remain systemically underrepresented in senior positions across higher education institutions globally, and this disparity is particularly pronounced across Asian contexts. Drawing on the authors’ collective experience as female academic leaders at IMU University a private health professions...
Why this label for Female academic leadership in private higher education: gender dynamics and lead
- Commentary: Perspective article drawing on authors' experience (R3).
- Review status: Graded by one AI grader from the abstract, on Matt's delegation; not yet human-verified
Graded by one AI grader from the abstract, on Matt's delegation; not yet human-verified. Method
CommentaryProvisional (rules only)
2026-10-15 · arXiv physics.ed-ph · Study (preprint)
This study investigates integrative thinking in undergraduate computational physics design essays (CPDEs) within an engineering design-based introductory physics laboratory. Integrative thinking refers to the interaction of science, mathematics, computation, design, and metacognitive reflection during problem solving.
Why this label for Integrative Thinking in Computational Physics Design Essays
- Commentary: provisional rules-only grade, pending grader consensus and review: R3 genre gate (F7): Self-described conceptual, legal, policy or reflective work with no learner data.
- Preprint: not peer reviewed; capped at Level 3
- Metacognition and self-regulation: matched 'metacognitive'
- Assessment and grading: matched 'rubric'
Provisional label from fixed rules applied to the collected text; research items are re-graded by our review pipeline. Method
CommentaryStudy type: Expert opinion / commentary
2026-10-13 · arXiv search: AI + education · Report / framework
Human-AI Interaction is commonly evaluated by what an artificial agent enables during use, yet interaction can also alter the human who acts next. We introduce Residual Human States, a trajectory-based framework for representing the potentially multidimensional human state carried forward after interaction with AI,...
Why this label for What Human-AI Interaction Leaves Behind: Residual Human States - A Trajectory-Ba
- Commentary (human override): The paper introduces a conceptual framework (Residual Human States) and illustrates it with four cases, most drawn from previously published or separate studies, rather than reporting one new graded study. Our genre rule places framework work in Commentary.
- Changed from: Level 1: Descriptive or exploratory
Human grade change 2026-10-21, approved by Matthew Rhoads, Ed.D. (approved in chat, 2026-10-21). Method
CommentaryProvisional (rules only)
2026-10-06 · arXiv physics.ed-ph · Study (preprint)
The moving-coil meter provides a simple context in which to model deterministic electromechanical dynamics, thermal fluctuations, and electronic readout. The movement is a linear electromechanical system in which electrical resistance and intrinsic mechanical damping act as two independent dissipative channels, each...
Why this label for Thermal fluctuations and electronic readout using a moving-coil movement
- Commentary: provisional rules-only grade, pending grader consensus and review: R3 genre gate: No empirical learner data described; conceptual, framework, policy or argument paper.
- Preprint: not peer reviewed; capped at Level 3
Provisional label from fixed rules applied to the collected text; research items are re-graded by our review pipeline. Method
CommentaryStudy type: Not a study
2026-09-24 · EdTech Magazine K-12 · News
When schools talk about students using artificial intelligence, the conversation often begins with assignments. Are students using AI to brainstorm, summarize or complete work they should be doing themselves?
Why this label for The Hidden Side of Student AI Use: Mental Health, Access and Risk
- Commentary: news, commentary or policy with no study
Provisional label from fixed rules applied to the collected text; research items are re-graded by our review pipeline. Method
CommentaryProvisional (rules only)
2026-09-15 · Educational Administration Quarterly (SAGE) · Study
This paper argues for the systematic integration of African Indigenous Knowledge Systems (AIKS) - anchored in the normative philosophy of Ubuntu - into K - 12 education policy across Africa. Historically, the dominance of Eurocentric epistemologies has ...
Why this label for Reimagining K - 12 Schooling for Cognitive Justice and Equity in Africa
- Commentary: provisional rules-only grade, pending grader consensus and review: R3 genre gate: No empirical learner data described; conceptual, framework, policy or argument paper.
- Policy and governance: matched 'education policy'
Provisional label from fixed rules applied to the collected text; research items are re-graded by our review pipeline. Method
CommentaryStudy type: Expert opinion / commentary
2026-09-05 · Carl Hendrick (learning science) · Commentary
Roundup of new research on explaining answers, building knowledge, reading instruction, social media and the conditions under which AI helps students learn.
Why this label for New Research on Learning Science: The September Dispatch
- Commentary: news summary of others' work; mentions 'New Research', graded C until the cited study is itself graded
- Reading and writing: matched 'reading instruction'
Provisional label from fixed rules applied to the collected text; research items are re-graded by our review pipeline. Method
CommentaryLearners: Not reportedOutcome: Not reportedPreprintStudy type: Expert opinion / commentary
2026-09-03 · arXiv (back catalog) · Study (preprint)
Contributions: A reflection model suitable for the era of Generative Artificial Intelligence (GenAI) is introduced. The proposed model is an integrated model that extracts features from various existing models and also incorporates technological aspects of GenAI.
Why this label for The 5P Reflection Model for Education in the Generative Artificial Intelligence
- Commentary: Both graders agree; rules version missed it. Consensus level confirmed.
- Review status: Confirmed against the published method (AI-assisted review)
- Grader A: R3 genre gate: the paper proposes a reflection model from existing literature without reporting learner outcomes.
- Grader B: R3 genre gate: it is labelled design-based research, but it analyses existing reflection models and proposes a new one; no participants or outcome data
- Rule version: METHODOLOGY v0.3, pipeline v1, rules rules-v0.4
Graded blind by two AI graders against our published method. Confirmed against the published method (AI-assisted review) Method
CommentaryStudy type: Not a study
2026-08-26 · EdSurge · News
Educators must shift their focus from preventing cognitive offloading to preserving the critical thinking skills AI threatens to replace.
Why this label for Protecting Student Cognition in the Age of AI
- Commentary: news, commentary or policy with no study
- Cognitive load: matched 'cognitive offloading'
Provisional label from fixed rules applied to the collected text; research items are re-graded by our review pipeline. Method
CommentaryLearners: TeachersOutcome: Perception or attitudePreprintReport (grey literature)Study type: Expert opinion / commentary
2026-08-19 · arXiv (back catalog) · Study (preprint)
On 28 July 2026, the SIGCSE Virtual 2026 Working Group on computing assessment and generative AI held an open online workshop attended by 73 computing educators. The first hour ran seven strategy rooms, one per approach to adapting (or deliberately preserving) computing assessment in the AI era: open-ended and...
Why this label for AI Can Do Your Homework. Now What? Report from an Online Workshop on Computing A
- Commentary: Workshop report, not a study.
- Review status: Confirmed against the published method (AI-assisted review)
- Grader A: R5 descriptive ceiling: workshop report records 73 educators' accounts without a controlled learning study.
- Grader B: R3 genre gate: a workshop report recording what 73 computing educators discussed; it has no study design and no measured outcome.
- Rule version: METHODOLOGY v0.3, pipeline v1, rules rules-v0.4
- Motivation and engagement: matched 'student motivation'
Graded blind by two AI graders against our published method. Confirmed against the published method (AI-assisted review) Method
CommentaryLearners: Adult learnersOutcome: Not reportedPeer-reviewed (journal reference listed)Study type: Expert opinion / commentary
2026-08-19 · arXiv (back catalog) · Study
Systemic change across the digital research landscape is required to reduce the environmental impact of digital research, but while many researchers and technical professionals are motivated to act, they often lack the skills required to translate motivation into lasting organisational change. We present an...
Why this label for Turning interest into institutional change: teaching advocacy for sustainable re
- Commentary: Course description for researchers with no evaluation; conceptual.
- Review status: Confirmed against the published method (AI-assisted review)
- Grader A: R5 pilot ceiling: a short course pilot is mentioned, but no learner outcome or comparison is reported.
- Grader B: R3 genre gate: describes an open course and its design; the short pilot reports no data or outcomes, so there is no study to grade.
- Rule version: METHODOLOGY v0.3, pipeline v1, rules rules-v0.4
Graded blind by two AI graders against our published method. Confirmed against the published method (AI-assisted review) Method
CommentaryLearners: Not reportedOutcome: Not reportedPreprintStudy type: Expert opinion / commentary
2026-08-05 · arXiv (back catalog) · Study (preprint)
As generative AI permeates computing instruction, the emergent design challenge is to configure each tool's pedagogical role, authority, and accountability for the instructional work it performs. We argue for instructional governance by design: governance should be encoded in a teaching tool's interaction model,...
Why this label for Instructional Governance by Design: A Framework for AI in Computing Education
- Commentary: Two independent AI graders gave the same level.
- Review status: Confirmed against the published method (AI-assisted review): both graders agreed
- Grader A: R3 genre gate: the paper explicitly develops a position and governance framework.
- Grader B: R3 genre gate: a position paper and framework illustrated with case descriptions, with no learner outcome data.
- Rule version: METHODOLOGY v0.3, pipeline v1, rules rules-v0.4
Graded blind by two AI graders against our published method. Confirmed against the published method (AI-assisted review): both graders agreed Method
CommentaryLearners: Not reportedOutcome: Not reportedPreprintStudy type: Expert opinion / commentary
2026-07-27 · arXiv (back catalog) · Study (preprint)
Spaced repetition systems (SRS) have demonstrated robust effects on long-term retention, yet existing tools reduce the flashcard interaction to a single binary gesture: flip and self-rate. This impoverished interaction model fails to leverage decades of cognitive science evidence on retrieval practice, and requires...
Why this label for Memdora: Designing Cognitively-Grounded Flashcard Interactions for AI-Powered Sp
- Commentary: Both graders agree; rules version missed it. Consensus level confirmed.
- Review status: Confirmed against the published method (AI-assisted review)
- Grader A: R3 genre gate: the paper presents a system and design rationale but reports no evaluation of learners.
- Grader B: R3 genre gate: system design and rationale paper for a flashcard app, with no evaluation with learners reported
- Rule version: METHODOLOGY v0.3, pipeline v1, rules rules-v0.4
- Retrieval practice: matched 'Flashcard'
- Spacing and interleaving: matched 'Spaced Repetition'
Graded blind by two AI graders against our published method. Confirmed against the published method (AI-assisted review) Method
CommentaryLearners: Not reportedOutcome: Not reportedPreprintStudy type: Expert opinion / commentary
2026-07-22 · arXiv (back catalog) · Study (preprint)
Generative AI is changing a basic premise of educational assessment: that submitted work can reliably evidence the human capacities a credential claims to certify. The challenge is not simply whether students use AI, but what remains inferable about learning when some cognitive work has been delegated to a system.
Why this label for What Does the Credential Still Certify? Cognitive Stewardship for AI-Mediated Ed
- Commentary: Both graders agree; rules version missed it. Consensus level confirmed.
- Review status: Confirmed against the published method (AI-assisted review)
- Grader A: R3 genre gate: the paper develops an assessment framework and audits university policies rather than studying learners.
- Grader B: R3 genre gate: conceptual framework plus an LLM-coded audit of 30 university policy documents; no learners, so no learner-level evidence
- Rule version: METHODOLOGY v0.3, pipeline v1, rules rules-v0.4
Graded blind by two AI graders against our published method. Confirmed against the published method (AI-assisted review) Method
CommentaryLearners: Higher-ed studentsOutcome: Behavior or engagementPeer-reviewed (journal reference listed)Report (grey literature)Study type: Expert opinion / commentary
2026-07-06 · arXiv (back catalog) · Study
This paper examines the psychological determinants that shape academically honest and dishonest uses of generative artificial intelligence (GenAI) in higher education. Rather than treating academic misconduct as a purely technological problem, the study conceptualizes academic integrity as a psychologically mediated...
Why this label for Psychological Determinants of Academic Integrity in the Use of Generative AI in
- Commentary: Two independent AI graders gave the same level.
- Review status: Confirmed against the published method (AI-assisted review): both graders agreed
- Grader A: R3 genre gate: the paper is a narrative review and conceptual synthesis.
- Grader B: R3 genre gate: self-described focused narrative review and conceptual synthesis of a purposive corpus of 16 sources, not a systematic review
- Rule version: METHODOLOGY v0.3, pipeline v1, rules rules-v0.4
Graded blind by two AI graders against our published method. Confirmed against the published method (AI-assisted review): both graders agreed Method
CommentaryStudy type: Expert opinion / commentary
2026-06-08 · Carl Hendrick (learning science) · Commentary
The most-cited AI-in-education paper gets retracted, retrieval practice hits a wall under high cognitive load, and a little knowledge turns out to be worse than none at all.
Why this label for The Monthly Dispatch - What's New in Learning Science? - June 2026
- Commentary: news, commentary or policy with no study
- Retrieval practice: matched 'retrieval practice'
- Cognitive load: matched 'cognitive load'
Provisional label from fixed rules applied to the collected text; research items are re-graded by our review pipeline. Method
CommentaryProvisional (rules only)
2026-05-18 · ERIC (peer reviewed) · Study
Teacher shortage is a global problem. One strategy adopted in several countries to help address the shortfall of teachers is to employ career-change teachers, sometimes through alternative entry pathways, which allow aspiring teachers to teach in a school, while simultaneously working towards their teacher...
Why this label for It's Not about the Money: STEM Professionals' Motivations to Become Secondary Sc
- Commentary: provisional rules-only grade, pending grader consensus and review: R3 genre gate: No empirical learner data described; conceptual, framework, policy or argument paper.
Provisional label from fixed rules applied to the collected text; research items are re-graded by our review pipeline. Method
CommentaryStudy type: Expert opinion / commentary
2026-05-06 · Carl Hendrick (learning science) · Commentary
New evidence on the limits of retrieval practice, phone bans that don't move test scores, ChatGPT as cognitive crutch, and why early vocabulary growth tells us surprisingly little about later reading
Why this label for The Monthly Dispatch - What’s New in Learning Science - May 2026
- Commentary: news, commentary or policy with no study
- Retrieval practice: matched 'retrieval practice'
Provisional label from fixed rules applied to the collected text; research items are re-graded by our review pipeline. Method
CommentaryProvisional (rules only)
2026-04-21 · ERIC (peer reviewed) · Study
It has long been recognized that expert decision making entails both fast, intuitive and slower, deliberative processes. The enduring debate has to do with their relative roles.
Why this label for Intuition and Deliberation in Elite Expertise
- Commentary: provisional rules-only grade, pending grader consensus and review: R3 genre gate: No empirical learner data described; conceptual, framework, policy or argument paper.
Provisional label from fixed rules applied to the collected text; research items are re-graded by our review pipeline. Method
CommentaryStudy type: Expert opinion / commentary
2026-04-11 · Carl Hendrick (learning science) · Commentary
Retrieval practice under scrutiny, feedback's missing mechanism, AI that actually teaches, and the quiet erosion of cognitive effort
Why this label for The Monthly Dispatch - What’s New in Learning Science - April 2026
- Commentary: news, commentary or policy with no study
- Retrieval practice: matched 'Retrieval practice'
- Cognitive load: matched 'cognitive effort'
Provisional label from fixed rules applied to the collected text; research items are re-graded by our review pipeline. Method
CommentaryStudy type: Expert opinion / commentary
2026-03-14 · Carl Hendrick (learning science) · Commentary
The Cognitive Price of AI at Work, and the Learning Science Case for Optimism
Why this label for AI Brain Fry, Workslop and the Ironies of Automation
- Commentary: news, commentary or policy with no study
Provisional label from fixed rules applied to the collected text; research items are re-graded by our review pipeline. Method
CommentaryStudy type: Expert opinion / commentary
2026-03-08 · Carl Hendrick (learning science) · Commentary
New studies on retrieval practice, seductive details, engagement myths, reading instruction, and the cognitive costs of modern technology
Why this label for The Monthly Dispatch - What's New in Learning Science? - March 2026
- Commentary: news summary of others' work; mentions 'studies', graded C until the cited study is itself graded
- Retrieval practice: matched 'retrieval practice'
- Reading and writing: matched 'reading instruction'
Provisional label from fixed rules applied to the collected text; research items are re-graded by our review pipeline. Method
CommentaryProvisional (rules only)
2026-02-25 · ERIC (peer reviewed) · Study
Who is more creative: highor low-socioeconomic status (SES) individuals? This question is the focus of intense debates within both the scientific community and society at large, since creativity has been linked to innovation, productivity, and wealth generation.
Why this label for Reading Ability Conflates SES Creativity Gaps
- Commentary: provisional rules-only grade, pending grader consensus and review: R3 genre gate: No empirical learner data described; conceptual, framework, policy or argument paper.
Provisional label from fixed rules applied to the collected text; research items are re-graded by our review pipeline. Method
Not yet classified
Listed for the February 2027 issue · Computers & Education (Elsevier) · Study
Publisher listing (abstract not yet available): Journal article in Computers & Education, Volume 257 by Wenjun Zhong, Jianghua Luo, Shuhui Yu et al. Issue: February 2027. Abstract on the publisher page.
Why this label for Navigating AI affective support: The cognitive-affective pathway from appraisal
- Not yet classified: publisher listing with no abstract yet
Provisional label from fixed rules applied to the collected text; research items are re-graded by our review pipeline. Method
Not yet classified
Listed for the February 2027 issue · Learning and Instruction (Elsevier) · Study
Publisher listing (abstract not yet available): Journal article in Learning and Instruction, Volume 107 by Kelsey E. Schenck, Mitchell J. Nathan. Issue: February 2027. Abstract on the publisher page.
Why this label for An integrated cognitive-affective spatial system for geometry
- Not yet classified: publisher listing with no abstract yet
Provisional label from fixed rules applied to the collected text; research items are re-graded by our review pipeline. Method
Not yet classified
Listed for the December 2026 issue · Teaching and Teacher Education (Elsevier) · Study
Publisher listing (abstract not yet available): Journal article in Teaching and Teacher Education, Volume 184 by Jiayi Wang, Chun Chen, Yuran Chen et al. Issue: December 2026. Abstract on the publisher page.
Why this label for Teachers’ motivation for coaching: A latent profile analysis of predictors and o
- Not yet classified: publisher listing with no abstract yet
Provisional label from fixed rules applied to the collected text; research items are re-graded by our review pipeline. Method
Not yet classified
Listed for the February 2027 issue · Learning and Instruction (Elsevier) · Study
Publisher listing (abstract not yet available): Journal article in Learning and Instruction, Volume 107 by Zamzami Zainuddin, Tianchong Wang, Riyan Hidayat et al. Issue: February 2027. Abstract on the publisher page.
Why this label for From faux flipped to gamified flipped classrooms: scaffolding and cognitive load
- Not yet classified: publisher listing with no abstract yet
- Cognitive load: matched 'cognitive load'
Provisional label from fixed rules applied to the collected text; research items are re-graded by our review pipeline. Method
