Blog / Weekly roundup · 2026-11-09

Weekly roundup, November 9, 2026 - an AI tutor that lowered grades, a big effect with a wide range, and new school AI controls

By Matthew Rhoads, Ed.D.

This roundup covers news and research released between late September and November 5, 2026. Each item shows its own publication date. Evidence levels below are provisional: they were applied by hand from our published rules, and none of these items has been through our two-grader review yet. Levels describe study design and who took part. They are not endorsements or official ratings.

The short version

Two of this week's items point the same way. In a randomized university trial, grades fell in the closest comparison after an AI tutor was added. In a Stanford study described in the news, the advantage of an AI tool vanished once students lost it. A third study found no clear exam benefit from AI-written feedback. A large meta-analysis still reports a big average benefit, but its own numbers show why a single average should not drive a purchase. On the news side, Google is changing how its AI tools are switched on for K-12 student accounts, and the U.S. Department of Education has proposed renaming "artificial intelligence" in its grant priorities.

Research

A course AI tutor did not raise grades, and in the closest comparison it lowered them

The Effects of Course-Integrated AI Tutoring on Student Performance and Engagement: A Randomized University Trial, Liu and colleagues, EdWorkingPapers (Annenberg Institute at Brown University), October 2026.

Provisional Level 3: Controlled comparison. Labels: Preprint (working paper, not peer reviewed), Learners: Higher-ed students.

What they did. 2,379 undergraduates and 30 instructors at a large U.S. public university took part in fall 2025. Instructors, not individual students, were randomly assigned so that whole sections either had access to an AI tutor built into the course site or did not.

What they found. In the comparison limited to sections of the same course, students with tutor access ended with final grades 0.37 standard deviations lower. Recorded participation in the course site's activities dropped by 0.90 standard deviations. In the full sample, which mixed different courses, the paper reports no significant evidence of grade harm. Only about 15% of students offered the tutor used it at least once, and the authors report that, descriptively, final grades in tutor sections were similar for users and non-users. Estimated grade losses were larger for first-generation students.

Limitations. It is a working paper. The analysis plan was registered after data collection but before analysis. Low use means the effect may come from how the tutor changed the class environment, not from tutoring itself. It is one university and one semester.

What it means in practice. Adding an AI tutor to a course is not automatically neutral. If you pilot one, track participation in the activities you already assign, not just tool usage.

AI feedback on handwritten math: good practice problems, no clear exam effect

Evaluating LLM-Generated Feedback on Handwritten Mathematics: Feedback Quality and Subsequent Exam Performance, Vanhoyweghen and colleagues, arXiv, posted October 20, 2026.

Provisional Level 2: Correlational. Labels: Preprint, Learners: Higher-ed students, Small or imprecise.

What they did. At a university in Brussels, GPT-5.1 wrote feedback and two practice exercises on students' handwritten answers to weekly bonus tests in an English-taught cohort (343 students) and a Dutch-taught cohort (236). On two tests, feedback was withheld from some students in a crossover design. Six reviewers rated 157 feedback messages.

What they found. 95.5% of reviewer ratings for the practice exercises were strong, compared with 41.1% for diagnostic specificity and 33.4% for clarity and tone. Feedback for the lowest-scoring answers was rated less actionable. On linked exam questions, the four comparisons showed no statistically significant differences after adjustment.

Limitations. Our rules start a randomized design at Level 3, but we lowered it one level. The exam comparisons only included students who attended both crossover tests, and each group had about 31 to 52 students. Attendance was voluntary. It is a preprint.

What it means in practice. AI can draft useful practice problems. The weakest spot was where help matters most: feedback for the lowest-scoring answers was rated less actionable, possibly because those students wrote very little. Those students may still need a teacher's first step.

A meta-analysis reports a large average effect, and a range that includes zero

A meta-analysis of the effects of generative artificial intelligence on learning, cognitive, and linguistic outcomes, Khodjaev, Masalimova, Chistyakov and Ay, Frontiers in Education, October 21, 2026.

Provisional Level 3: Controlled comparison (peer-reviewed meta-analysis; our rules cap reviews here until a person checks the included studies).

What they did. The authors pooled 68 effect sizes from 58 studies on generative AI and learning, cognitive and language outcomes.

What they found. The pooled effect was large (g = 0.864). The authors also report that heterogeneity was high (I² about 88%) and that the 95% prediction interval ran from -0.213 to 1.942. That means a new setting could plausibly see no benefit, or harm. A small-study bias test was significant, and corrected estimates were roughly g = 0.26 to 0.45.

Limitations. 11 of the 68 effect sizes come from single-group pre/post designs, which our method treats as Level 1 evidence. About 90% of effect sizes come from settings the authors coded as vertical-collectivist cultures, and 73.5% from higher education or adults. Only 7 come from primary-age learners.

What it means in practice. When a vendor cites a big average effect, ask for the prediction interval and the small-study correction. Here those two numbers tell a more cautious story than the headline.

Stanford researchers describe AI help that disappeared when the tool did

What happens to learning when AI does the thinking for us?, Stanford Report, November 4, 2026.

No study cited (N). Labels: News summary of others' work. The article describes a study but does not link it, so we cannot rate the study yet.

What the article reports. Guilherme Lichand and Sachin Allums gave Brazilian middle schoolers different levels of access to an AI tool across two creative tasks. When students had the tool for the first task and then lost it, its advantage disappeared, and their creative problem solving fell below that of students who never had it. Lichand said the effects were concentrated among students who used the tool more on the first task. Separately, the article reports that Lily Fesler of Stanford's SCALE Initiative and her team reviewed over 800 papers and focused on 20 high-quality causal studies. In that review, general-purpose chatbots were the most harmful tools, while harms were less clear for tools with more scaffolding.

Limitations. No sample size, effect sizes or paper link appear in the article, so the size of the effect is unknown.

What it means in practice. This points the same way as the AI tutor trial above, though the designs differ. Before adopting a tool, ask what students will be able to do without it.

Who shapes state AI guidance for K-12 schools

U.S. State Guidance on AI in K-12 Education: A Comparative Content and Citation Analysis, Memon, Gueorguieva, Kumar, Wang, West and Weber (University of Washington), arXiv, posted November 4, 2026.

Descriptive document analysis, no learners. Our tracker currently shows a provisional rules label of Technical test for this paper. That label is pending grader review, because the paper analyzes policy documents rather than testing AI models. Labels: Preprint.

What they did. The team coded AI guidance documents from 35 states and Puerto Rico and traced 5,351 references to 328 sources.

What they found. Data privacy and academic integrity each appear in 97% of documents. Workforce readiness and personalized learning each appear in 86%, but the authors found few specifics on capacity building and accountability. TeachAI is cited by 78% of documents, and some states reproduced its model guidance nearly word for word. No state lists parents as contributors, and only five include students.

Limitations. Each topic was coded as present or absent, so the coding does not capture depth. It uses each state's latest document only. The first author helped write Washington's guidance, which the paper discloses.

What it means in practice. If your district is writing or updating an AI policy, the gaps this paper names are a useful checklist: named roles, budget, vendor evaluation, and parent and student voice.

Research beyond AI

Asking teachers to show high expectations made feedback more detailed, with small bias effects

The Effect of Promoting High Expectations on Teachers' Feedback to Students, Copur-Gencturk, Thacker and Cimpian, Educational Researcher, published online October 17, 2026. Summary: USC Rossier, October 30, 2026.

Provisional Level 3: Controlled comparison. Labels: Peer-reviewed. Participants are teachers, and the outcome is the feedback they wrote, not student learning.

What they did. The researchers randomly assigned 963 U.S. teachers to give feedback on the same math solutions either as they normally would or while emphasizing high standards and the student's potential to meet them. Each solution was shown with an image of a randomly assigned White or Black girl or boy.

What they found. Prompted teachers gave more detailed feedback on strengths and weaknesses. They also commented more on ability and effort. USC reports that prompted teachers praised math ability nearly four times as often (8.1% vs. 2.1% of responses), and questioned ability or effort more often, though under 3% of responses. Black girls received negative ability feedback in 3.3% of prompted responses vs. 0.6% unprompted.

Limitations. Teachers gave feedback online, in a simulated classroom, to students they did not know. The study did not measure what students learned.

What it means in practice. A "high expectations" prompt alone is not enough. Pair it with a norm to comment on the student's mathematical reasoning and give a concrete next step, not judgments about ability or effort.

In the news

Google will switch Gemini tools and Search off for K-12 student accounts until schools turn them on

Designing responsible AI for learning, Google, November 3, 2026. Related: Google's AI in Education report, November 2026.

Not yet classified (?). Labels: Vendor-linked. Google's post links its own studies, and its report cites a Sierra Leone study. We have not rated those studies yet.

Google says access to Gemini, Gemini Notebook, Gemini in Classroom and Google Search will be entirely off for K-12 student accounts until administrators explicitly turn them on. It says more details will come in the coming months before the change rolls out. Guided Learning, which prompts students step by step instead of giving answers, is now available in Google Classroom, and a Search pilot is planned in select U.S. schools. Google's report credits its Guided Learning trial in Sierra Leone with a 0.26 standard deviation improvement and notes that students who started further ahead in math benefited more.

What it means in practice. Districts will need to decide product by product what to enable, and write down why. Treat the learning-gain figures as company-reported until independent studies are available.

The Education Department proposes "super intelligence" in place of "artificial intelligence" in grant priorities

Proposed Priorities and Definitions: Secretary's Supplemental Priorities and Definitions, U.S. Department of Education, Federal Register, November 5, 2026.

Commentary (C). A policy notice, not a study.

The notice proposes replacing "Artificial Intelligence (AI)" with "Super Intelligence (SI)" throughout its supplemental grant priorities, following a September 29, 2026 executive order. SI is defined as the technologies already covered by the federal legal definition of artificial intelligence. The notice also proposes new parts that would support building evidence on whether and how these tools improve student outcomes. Comments are due December 7, 2026.

What it means in practice. The proposed change renames the term but keeps the same underlying definition of the technology. Grant writers should watch for the final wording, and the new evidence-building parts may favor proposals that include a real evaluation.

How we rated these

Every item links to its original source. Levels are provisional and were applied by hand from our published rules. They will be updated if our two-grader review changes them. Read our method.

Get Navigating Education free

You're subscribing to Navigating Education, Dr. Matt Rhoads' free weekly newsletter on the science of learning, AI and practical teaching. It goes out Tuesdays. Each issue includes an Evidence Signal section.