Level 2: CorrelationalLearners: Higher-ed studentsOutcome: Behavior or engagementPreprintPreregisteredPerception only
2026-10-07 · arXiv (back catalog) · Study (preprint)
AI learning tools are rapidly entering classrooms, but evidence about whether they help students learn is mixed and rests mostly on test scores. Comparatively less research addresses whether the use of AI changes students' live learning behaviors in class.
Why this label for How assigned AI use before class shapes active student engagement in class
- Level 2: Correlational: Both graders agree; rules version missed it. Consensus level confirmed.
- Review status: Confirmed against the published method (AI-assisted review)
- Grader A: R7 hard cap: randomized assignment is reported, but the measured outcome is classroom engagement rather than learning.
- Grader B: R7 engagement-only cap: a preregistered randomized field experiment (n=759 MBA students) that would otherwise be Level 3, but outcomes are class contributions and self-reported perceptions, with no learning measure, so max 2
- Rule version: METHODOLOGY v0.3, pipeline v1, rules rules-v0.4
- Motivation and engagement: matched 'student engagement'
Graded blind by two AI graders against our published method. Confirmed against the published method (AI-assisted review) Method
Technical testProvisional (rules only)
2026-10-05 · arXiv search: AI + education · Study (preprint)
We present an AI-based web-oriented tool designed to support formative feedback for oral presentation slides in higher education: AISSA. It allows students to upload their slides before presentation and automatically receive rubric-based quantitative scores and qualitative feedback generated by large language models...
Why this label for AISSA Demo: AI-based Student Slides Analysis Tool for Automated Grading and Feed
- Technical test: provisional rules-only grade, pending grader consensus and review: R4 real-learner gate (F1/F2): Outputs of models, simulations or audits; any humans act as raters, references or data sources, not learners.
- Preprint: not peer reviewed; capped at Level 3
- Feedback: matched 'formative feedback'
- Motivation and engagement: matched 'student engagement'
- Assessment and grading: matched 'Grading'
Provisional label from fixed rules applied to the collected text; research items are re-graded by our review pipeline. Method
Level 1: Descriptive or exploratoryLearners: Higher-ed studentsOutcome: Behavior or engagementPreprintPerception only
2026-09-22 · arXiv (back catalog) · Study (preprint)
This paper presents a multi-source classroom study conducted during a 10-week quarter in data science courses at Drexel University. We first investigate the behaviors of student engagement with large language models (LLMs) using four surveys across three data science courses.
Why this label for Student Use of LLMs and the Limits of AI-Generated Question Difficulty in Data S
- Level 1: Descriptive or exploratory: Consensus level confirmed against the methodology.
- Review status: Confirmed against the published method (AI-assisted review)
- Grader A: R5 classroom surveys describe student use and perceptions; response data also test LLM question ratings without a learner comparison.
- Grader B: R5: surveys and an item-difficulty correlational analysis in a 10-week course with no comparison group are descriptive (Level 1); the R7 perception/engagement cap also applies to learning claims.
- Rule version: METHODOLOGY v0.3, pipeline v1, rules rules-v0.4
- Retrieval practice: matched 'retrieval practice'
- Motivation and engagement: matched 'student engagement'
Graded blind by two AI graders against our published method. Confirmed against the published method (AI-assisted review) Method
CommentaryLearners: TeachersOutcome: Perception or attitudePreprintReport (grey literature)
2026-08-19 · arXiv (back catalog) · Study (preprint)
On 28 July 2026, the SIGCSE Virtual 2026 Working Group on computing assessment and generative AI held an open online workshop attended by 73 computing educators. The first hour ran seven strategy rooms, one per approach to adapting (or deliberately preserving) computing assessment in the AI era: open-ended and...
Why this label for AI Can Do Your Homework. Now What? Report from an Online Workshop on Computing A
- Commentary: Workshop report, not a study.
- Review status: Confirmed against the published method (AI-assisted review)
- Grader A: R5 descriptive ceiling: workshop report records 73 educators' accounts without a controlled learning study.
- Grader B: R3 genre gate: a workshop report recording what 73 computing educators discussed; it has no study design and no measured outcome.
- Rule version: METHODOLOGY v0.3, pipeline v1, rules rules-v0.4
- Motivation and engagement: matched 'student motivation'
Graded blind by two AI graders against our published method. Confirmed against the published method (AI-assisted review) Method
Level 1: Descriptive or exploratoryLearners: K-12 studentsOutcome: Behavior or engagementPeer-reviewed (journal reference listed)Vendor-linkedPerception only
2026-08-07 · arXiv (back catalog) · Study
Many AI tutors leverage large language models (LLMs) today. Given that LLMs are opaque black boxes, robust evaluation and live experimentation to measure the impact of every change are essential.
Why this label for Methodologies for Improving the Quality of AI Tutoring in K-12 Education
- Level 1: Descriptive or exploratory: Consensus level confirmed against the methodology.
- Review status: Confirmed against the published method (AI-assisted review)
- Grader A: R5 descriptive ceiling: product metrics and experiments are described without an explicit learner-assignment design or learning outcome.
- Grader B: R5 design ceiling: a vendor (Khan Academy) describes its metrics and unspecified experiments without design or sample details, so it is descriptive; R7 engagement-only cap also applies.
- Rule version: METHODOLOGY v0.3, pipeline v1, rules rules-v0.4
- Motivation and engagement: matched 'student engagement'
- Tutoring and personalization: matched 'Tutoring'
Graded blind by two AI graders against our published method. Confirmed against the published method (AI-assisted review) Method
Level 2: CorrelationalLearners: Not reportedOutcome: Learning or achievementPreprintSmall or imprecise
2026-06-16 · arXiv (back catalog) · Study (preprint)
Motivational factors such as self-efficacy and how favorably students feel toward practice play a crucial role in shaping learning, particularly in technology-supported environments. Yet, educational interventions often overlook how these factors interact with practice format.
Why this label for Self-Efficacy and Favorability Shape Learning from Tutoring Systems and Paper Pr
- Level 2: Correlational: Counterbalanced within-subject design with learning outcome; no sample size or interval reported, so held at 2.
- Review status: Confirmed against the published method (AI-assisted review)
- Grader A: R5 design ceiling: counterbalanced within-subject comparison uses matched problem sets and baseline self-efficacy, but Level 4 checks are unavailable.
- Grader B: R5: the headline finding is a moderation link between self-efficacy or favorability and learning gains (correlational, Level 2) inside a counterbalanced within-subject ITS vs paper comparison; n and intervals are not reported, so it stays at 2 pending review (accepted at EC-TEL 2026, not matched)
- Rule version: METHODOLOGY v0.3, pipeline v1, rules rules-v0.4
- Motivation and engagement: matched 'students' motivation'
- Tutoring and personalization: matched 'Tutoring'
Graded blind by two AI graders against our published method. Confirmed against the published method (AI-assisted review) Method
