Artificial intelligence is rapidly transforming education in both concerning and beneficial ways. On the positive side of Ledger, new research shows how AI can help instructors improve how they engage with students via cutting-edge tools that provide feedback on class interactions. increase.
The M-Powering Teachers tool provides feedback with example interactions from the class to illustrate supportive speaking patterns. Click image to enlarge. (Image credit: Courtesy of Dora Demszky)
New Stanford-led study published May 8 in peer-reviewed journal Education evaluation and policy analysis, found that automated feedback tools improved instructors’ use of practices called uptake, in which teachers recognize, repeat, and build on student contributions. The findings also provided evidence that the tool improved assignment completion rates and overall satisfaction with the course among students.
For instructors looking to improve their practice, this tool is a low-cost complement to traditional classroom observation. This does not require instructional coaches or other professionals to observe teacher behavior and compile a set of recommendations.
“We know from past research that timely and specific feedback can improve teaching, but having someone sit in the teacher’s classroom and give feedback every time is not scalable or feasible.” says Dora Demsky, assistant professor at the Stanford Graduate School of Education. (GSE) and first author of the study. “We wanted to see if automated tools could support teacher professional development in a scalable and cost-effective manner, and this is the first study to show that.”
Promoting effective educational practices
Recognizing that existing methods for providing personalized feedback required considerable resources, Demszky and his colleagues set out to create a low-cost alternative. Leveraging recent advances in natural language processing (NLP), a branch of AI that helps computers read and interpret human language, analyze transcripts of class sessions to identify speech patterns, We have developed a tool that can provide consistent and automated feedback.
This research focused on identifying teachers’ understanding of student contributions. Demszky said: “But it’s also widely believed to be difficult for teachers to improve.”
Researchers trained a tool called M-Powering Teachers (M stands for machine, as in machine learning) to detect how specific a teacher’s response was to what a student said. Made with student ideas. The tool can also provide feedback on teachers’ questioning practices, such as which questions elicited significant responses from students and the ratio of teacher-to-student talk time.
The research team used the tool in the Spring 2021 session of Code in Place at Stanford University. This is a free online course currently in its third year. In a five-week program based on the popular introductory computer science course at Stanford University, hundreds of volunteer instructors teach learners around the world basic programming in small sections with his 1:10 teacher-to-student ratio. I teach
Code in Place instructors come from all kinds of backgrounds, from recent undergraduates taking courses to professional computer programmers working in industry. Eager to introduce beginners to the world of coding, many instructors approach this opportunity with little or no teaching experience.
Volunteer instructors received basic training, clear lesson goals, and session outlines to prepare them for the role, and many welcomed the opportunity to receive auto-fill in the sessions, said study co-authors. Chris Piech, Assistant Professor of Computer Science Education at Stanford University, said. He is also the co-founder of Code in Place.
“We place great importance in education on the importance of timely feedback to students, but when do teachers receive such feedback?” he said. “Maybe the principal walks into your class and sits with you. Plus, you get feedback from day one on the job.”
Instructors received feedback from the tool via the app a few days after each class so they could reflect on it before the next session. Concrete examples of conversations from the class were included to illustrate supportive speech patterns, using targeted and unbiased language.
Researchers found that, on average, instructors who reviewed the feedback subsequently increased their understanding and use of questions, with the greatest change seen in the third week of the course. Students’ learning and satisfaction with the course also increased in the group where the instructor received feedback compared to the control group. Because Code in Place does not administer end-of-course exams, researchers used optional assignment completion rates and course surveys to measure student learning and satisfaction.
Test with other settings
A subsequent study by Demszky and one of the study’s co-authors, Jing Liu (PhD ’18), investigated tool use among instructors who worked one-on-one with high school students in an online mentoring program. bottom. Researchers who plan to present their findings at the Learning at Scale conference in July 2023 found, on average, that using the tool improved mentors’ uptake of student contributions by 10% and mentors’ 5% reduction in conversation time, improving the student experience. Relative optimism about their academic future, not just the program.
Demszky is currently conducting research on the use of face-to-face tools in classrooms at K-12 schools, and noted the difficulty of producing high-quality transcriptions obtained from virtual environments. “Classroom audio quality is not good and it is not easy to separate voices,” she said. “With transcripts, natural language processing can do so many things, but we need good transcripts.”
She stressed that the tool was not designed for monitoring or evaluation purposes, but rather to assist teachers’ professional development by giving them an opportunity to reflect on their practice. She likens it to a fitness tracker that provides information for the user’s own benefit.
And she said the tool is designed to complement other professional development resources rather than replace human feedback.
The research was co-authored by Dan Jurafsky, professor of linguistics and computer science at Stanford University, and Heather C. Hill, professor at the Harvard Graduate School of Education, along with Dora Demszky, Jing Liu, and Chris Piech.
