This guest post is by Manaz R. Charania, PhD, former Senior Research Fellow in Education at the Christensen Institute. It was originally published on the Christensen Institute blog and is republished with permission.
What's been dubbed the craziest college admissions season in history is also proving to be a natural experiment for the American education system.
Recent test-free and test-blind admissions programs have the potential to greatly expand eligibility for admission to selective universities. Yet, despite well-intentioned efforts to break down systemic inequalities in admissions, universities are returning to standardized assessment systems in the hope that they can more accurately predict which students will thrive in their environment and graduate on time. While returning to known methods may be more efficient, it also risks perpetuating the existing inequalities that universities themselves are so doggedly trying to eliminate.
The patchwork of admissions testing policies in college admissions highlights larger challenges and opportunities for both K-12 and higher education. The aims and goals of the education system continue to change, but the ways schools define and measure student success have not kept pace. This disconnect exacerbates higher education's current admissions challenges. To catch up, measurement in education must go beyond using one set of scores (the SAT) to predict another set of scores (post-secondary success) and generate data that improves opportunities for all learners in and out of the classroom. Emerging technologies such as artificial intelligence can help.
Artificial intelligence to personalize evaluations rather than standardize them
Technology has long shaped schools’ approach to assessment. In the early 2000s, I had first-hand experience with how large school districts make decisions about adopting edtech and deploying AI-enabled personalized learning. At the time, the adoption of adaptive learning and diagnostic solutions like DreamBox, i-Ready, IXL, and even NWEA Map was exploding across the nation. These edtech tools were seen as breakthrough technologies that provided classrooms with real-time reporting and analytics to track and adjust instruction while students were playing. Since then, online learning has continued to break the boundaries of traditional monolithic approaches to K-12 teaching and learning. The integration of Digital Promise’s digital equity and safety work with Getting Smart on the evolution of AI-enabled innovations that shape teaching and learning is a testament to how far we’ve come.
Generative AI has the potential to significantly enhance existing approaches, but it also has the potential to disrupt them. Used appropriately, AI has the potential to usher in a generation of assessments that reduce over-reliance on standardization and embrace more personalized, unbiased approaches.
This poses a special challenge for system leaders: How can we use AI to help us measure the things we know are important but haven't yet done a good job of measuring? How can we use AI to individualize rather than standardize assessments, and equitably support the success of all students in and out of the classroom?
The answer lies in expanding efforts in at least three areas: learner-centered assessment, integrated and invisible assessment, and disaggregated data.
1. Develop learner-centered assessments aligned with learner-centered systems
The skills that make us human are the skills that a learner-centered framework encourages. They are also the skills that technology will find very difficult, if not impossible, to replicate reliably. Instead of putting our energy into teaching kids what robots can do, we need to focus on teaching them what only humans can do.
For example, Quill.org provides low-income students with AI-powered literacy tutors to help them become better writers, readers, and critical thinkers. The tutors provide students with real-time guidance and feedback on literacy activities that combine nonfiction reading and informational writing. Additionally, Quill's new Reading for Evidence tool gives students the opportunity to demonstrate their comprehension of nonfiction texts by writing arguments based on feedback from Quill's AI tools on how to strengthen the logic, evidence, and syntax of their answers. As a result, students, especially under-resourced communities and multilingual learners who may benefit from additional support, receive the feedback they need equitably.
AI-powered literacy tools also have the potential to strengthen students’ historical thinking and, therefore, their capacity for civic discourse, an increasingly necessary skill for all. For example, Thinking Nation, a nonprofit dedicated to improving social studies education, recently switched from paying teachers to grade essays based on assessment criteria to an AI chatbot. The chatbot is trained on the same assessment criteria and provides students with instant feedback on their ability to critically evaluate historical texts. This frees up teachers’ bandwidth, empowers students to speak up, and engages learners in the art of negotiation and debate, activities that foster students’ ability to show empathy, understanding, and respect in order to enact individual and collective action.
2. Moving from pen-and-paper assessment to integrated, invisible assessment
Assessment methods that are woven into the fabric of learning and invisible to students provide another opportunity to leverage AI to transform how we measure student progress. Stealth assessments have been a lifeline for most families during COVID, especially during the first year when school doors were closed. Stealth assessments have also been shown to reduce test anxiety and increase student engagement. This type of assessment provides endless opportunities to measure higher order thinking skills. For example, video game-based assessments are particularly attractive as a means to help cultivate skills unique to the human brain and increase engagement.
A recent survey by Gallup and the Walton Family Foundation found that less than half of Gen Z students enrolled in middle and high school are motivated to attend school. Only about half say they do something interesting at school every day. Part of the reason for this increase in absenteeism is a narrow focus on curriculum and an emphasis on high-stakes testing as the primary way to measure students' knowledge and skills.
To combat the decline in student engagement, programs like Labster are on a mission to democratize access to education by empowering remote students to participate in virtual science labs. Students join this virtual community and receive simulated, real-world learning with real-time feedback, at their own time and pace. The shift from pen and paper to real-world simulations has not only increased student engagement, but also sparked interest in STEM-related careers.
3. Segment your data to shift focus from the average student to all students
From an equity perspective, norm-referenced tests (essentially all standardized tests) are particularly problematic. First, they are almost always ill-suited for students with limited English proficiency or who speak dialects other than typical American English. Also, the format of these tests may be biased because they reflect traditional Western values. These values may be embedded in the logic of the questions and expectations for speed of completion. Those with access to resources may be able to circumvent these challenges by using private tutoring or test preparation services.
By employing analytic techniques that leverage AI and enable disaggregated data, we can shift the focus from dominant groups to view every child – black, Hispanic, low-income, immigrant, English language learner, special needs student – from an asset-based lens by understanding their expertise and strengths relative to their reference group. The joint effort by the Carnegie Foundation for the Advancement of Teaching and the ETS Research Institute holds great promise for this important shift from standardization to adaptive personalization in assessment.
Effectively leveraging AI also requires changes to the computational tools used. One promising approach to ensure that authentic, game-based assessments provide meaningful insights is to leverage evidence-centered assessment designs. This design includes a student model that describes the traits, skills, or abilities being assessed, a task model that describes the activities students undertake to generate evidence of building those traits, and an evidence model that describes the variables and statistical methods used to link evidence to those traits. These features are particularly useful for computer-based simulations that can be automated with AI to uncover desired student-level outcomes.
A call for investment in evaluation research and development to reduce inequality
For students who are not adequately served by the limited perspective that standardized tests provide, especially those who do not predict success outside of the classroom, empowering AI-driven assessments could be a game changer.
These new approaches have immense disruptive potential. At first glance, the growing number of AI-powered assessment opportunities may seem “lower quality” compared to the tried-and-true standardized assessments that dominate the education market today. However, they are: Non-consumable Rating, The only option is to not measure these outcomes at all.
But to ensure that AI-powered assessments don’t reinforce the status quo, weaken relationships, or exacerbate inequities, R&D funding must be invested to make these disruptive approaches take root and ensure assessment criteria are created fairly, transparently, and aligned with the programs that exist to support students. As we’ve suggested before, our inability to foster deep learning among peer organizations is hindering our ability to scale solutions that could have the greatest impact on student learning, skill mastery, and advancement. With more knowledge sharing and nuanced insights readily accessible, system leaders across K-12 can be in a position to rapidly test and scale what works for who, under what conditions.
As schools continue to develop roadmaps and policies for making the most of technology and technology integration tools, this creates an opportunity for educators, policymakers, and technologists to collaborate and leverage the opportunities AI provides with students and their families to reimagine what personalized assessment looks like when students need it most.
Related:
Classification of AI use in education
The role of AI in the future of learning
For more news about AI in education, check out eSN's Digital Learning Hub.
