What it evaluates

The AI reads through discussion transcripts and scores students on specific criteria—like how well they apply concepts (Bloom's Taxonomy), articulate a reasoned position with evidence, or engage with their peers' ideas.

How it scores

For each discussion, the AI generates a score (0-6 for Bloom's, 0-4 for Discussion Quality rubrics) along with detailed justification. Importantly, students see not just their score, but also:

• The AI's reasoning for that score
• Key arguments the AI identified in their contributions
• Direct citations from the transcript supporting those arguments

Context matters

We inject assignment-specific information—learning objectives, key concepts, and assignment descriptions—so the AI focuses only on what's relevant to that particular discussion, not everything students said. Before any assignment goes live, instructors review these learning objectives with our instructional designers to ensure they align with what's being taught in class. This alignment process ensures the AI evaluates students on criteria that actually matter for the course.

In addition to rubric-based scoring, the AI system also synthesizes patterns across groups

When multiple groups complete the same assignment, the system categorizes topics by themes and aggregates findings at the assignment level, answering questions like:

• "Are my students making connections between the discussion and what I've been teaching them in class?"
• "What was the most divisive topic in the discussion?"
• "Were there any debates that would serve as a good jumping off point for an in-class lecture?"
• "What are my students struggling to understand?"
• "Can you suggest an in-class activity that will help my students better understand what they're struggling with?"

Part of a bigger picture

The AI evaluation is just one component of a student's grade. By default, it accounts for 30% of the final score, with the rest coming from quiz performance (40%), attendance (15%), and completion (15%). Instructors have full control to adjust these weightings or even exclude the AI evaluation entirely.

Consensus-based scoring

We don't rely on a single AI evaluation. Instead, we run the AI multiple times and only accept scores that appear consistently across at least 7 evaluations. If we can't reach consensus after up to 27 attempts, the system flags the assignment for human review by our team.

Full transparency

Students can see exactly how the AI arrived at their score—the reasoning, their identified arguments, and the specific parts of the transcript that support the evaluation. This transparency helps students learn from the feedback and allows them to identify any potential errors.

Human oversight

We actively monitor AI results through an internal dashboard. When issues arise, our team investigates. We're also launching a grading adjudicator system that will allow students to contest grades, with streamlined workflows for instructors to review and resolve concerns.

Instructor control

Built for substance, not shortcuts

The AI works best when discussions involve articulating positions, applying concepts, and engaging with ideas. We design experiences that invite substantive discussion rather than simple fact recall, and we help ensure evaluation criteria match the discussion format.

Continuous improvement

We're currently testing multiple AI models against exemplar discussions to ensure we're using the best technology available. Our approach evolves as the field advances.

The bottom line

AI is a powerful tool for evaluating discussions at scale, but it's not infallible. That's why we've made it transparent, contestable, flexible, and just one part of a holistic assessment approach. Instructors maintain full agency over how—and whether—to use it.

Text transcription, not video storage

The AI analyzes text transcripts of what students say during discussions. We do not save or store video files of students' faces or the video feed itself. Once the discussion is transcribed, students' video and audio streams are not retained.

Personal information in discussions

The system filters what instructors see. Instructors don't receive full transcripts—they only see snippets that are relevant to the assignment and learning objectives in the student session results. If instructors want to explore additional context or quotes, they can query for that information through the insights tool. Full transcripts are only accessible to the internal Breakout Learning team for quality monitoring and troubleshooting purposes.