Using AI to Increase Fairness & Accuracy
The news is filled with stories that describe the potential for Artificial Intelligence (AI) to improve education. There are also numerous stories regarding the risks of implementing AI systems that do not have fairness and accuracy embedded in their design. While we hear almost daily numerous stories of bias and incorrect information (i.e., hallucinations) provided by AI, we don’t hear as much about solutions.
Building on its history of innovation in education, Smarter Balanced has announced a new initiative with IBM Consulting that is designed to contribute to the field by iterating on existing industry guidelines on a principled use of AI in educational measurement. These new principles and guidelines leverage a multidisciplinary panel sharing their expertise to enhance fairness and accuracy. This initiative will include:
AI PRINCIPLES AND GUIDELINES
We will form a multidisciplinary panel that will:
- Develop measurement-focused principles and guidelines for the use of Artificial Intelligence (AI) to enhance accessibility, accuracy, and fairness while reducing bias in educational measurement;
- Help problem-solve issues with the use of AI based on case studies and new technology;
- Promote responsible use of AI in educational measurement.
Define the North Star and Provide Foundation
Activities
Partner and establish AI committee to inform strategy and vision
AI strategy document including guiding principles
Outreach and engagement to bring on diverse and inclusive expertise
Why?
- Define the North Star for AI in educational measurement
- Provide leadership to the educational measurement community
- Create specific, quantifiable success metrics for each AI principle
- Embed principles of fairness, accuracy, accessibility, transparency, and strategies to minimize bias in the use of AI for educational measurement
- Provide ongoing leadership to the educational measurement community
- Ensure we engage in discussions with diverse perspectives
AI PROOF OF CONCEPT
We will create a proof-of-concept using AI for language translations in math assessments, aiming to improve accessibility and demonstrate a principled use of AI in educational measurement for Smarter Balanced members and students.
This work simultaneously iterates on existing industry guidelines while co-developing improved AI principles and guidelines.
Prove the Effectiveness of AI and Provide Vision
Activities
AI model
Implementation roadmap and next steps
Script for evaluating model based on model evaluation framework
Why?
- Demonstrate how guiding principles may support the implementation of AI in a responsible manner
- Demonstrate how a responsible use of AI can enhance accessibility resources for students
- Establish a foundational infrastructure to support further enhancements to the Smarter Balanced assessment system
Initial phases of work to be completed by May 2024.
For continued updates, follow along on: