How Unrestricted AI Tutoring Enhances Student Learning
Fischer, Rau & Rilke: "AI Tutoring Enhances Student Learning Without Crowding Out Reading Effort" CRC Discussion Paper No. 557
A recent study by Mira Fischer (WZB Berlin), Holger A. Rau (Georg-August-Universität Göttingen), and Rainer Michael Rilke (WHU - Otto Beisheim School of Management) provides new causal evidence on the integration of generative artificial intelligence into higher education, focusing on how different modes of access shape student learning outcomes. Through a randomized laboratory experiment involving 334 university students preparing for an incentivized economics exam, the researchers compared a control group using only textbook materials against two groups with access to an AI tutor based on GPT-4. The AI-tutor groups differed by whether access was unrestricted or only given after reading the textbook for 10 minutes. The results indicate that while AI access generally raises test performance by 0.23 standard deviations relative to the control, the specific design of restricted vs. immediate is a critical determinant of success.
Specifically, the most striking finding is that students do not benefit from a compulsory independent study period before using AI tools to prevent a “crowding out” of learning effort. Students with unrestricted, continuous access to the AI tutor outperformed those with restricted access, by 0.21 standard deviations. The behavioral data reveals that unrestricted access encouraged a more effective “scaffolding” approach—where students build understanding incrementally by consulting AI when encountering difficulties rather than studying in isolation first. More precisely, students of this group gradually integrated the AI tutor into their study process, using shorter and more frequent prompts as their understanding developed. In contrast, students in the restricted group exhibited intensive bursts of prompting immediately upon gaining access, a pattern that appeared to disrupt the flow of learning and led to less effective knowledge acquisition.
Interestingly, the benefits of the AI tutor were not uniform across the student population. Heterogeneity analysis showed that learning gains were concentrated among students with lower baseline subject knowledge and those with stronger self-regulation skills, such as lower tendencies toward procrastination and distraction. This suggests that AI tutors are most effective when they serve as adaptive support for goal-directed learners.
Link (pdf): AI Tutoring Enhances Student Learning Without Crowding Out Reading Effort
