How the Prediction Engine Works

This section covers the technology, data sources, and reliability of the model.

Q: What is the difference between my “Assessment Based Pass Probability” and my “Comprehensive Exam Readiness Pass Probability”?

A: Your Assessment Based Pass Probability is an assessment-specific metric: it tells you how many questions you got right on this specific exam and how that compares to the national average for those test questions. Your Comprehensive Exam Readiness Pass Probability is a platform-specific metric: it uses your Mock score plus your study habits to estimate the likelihood of you passing the real Board Exam. 

Q: What exactly is “Behavioral Forensics”?

A: Standard scoring only looks at what you answered (Correct vs. Incorrect) at this point in time. Behavioral Forensics looks at how you answered. The model analyzes over 144 different data points, including your pacing (are you rushing?), your doubt (how often do you switch a correct answer to a wrong one?), and your stamina (does your accuracy drop after an hour of studying?). This allows us to predict exam readiness earlier and more accurately than a raw Mock Assessment accuracy alone.

Q: Does my performance from months ago count as much as my performance yesterday?

A: No. The model applies “Recency Weighting.” It understands that you are learning and evolving. Behavior and scoring from your most recent study sessions are weighted more heavily than data from the beginning of the year. This ensures the prediction reflects your current capability, not who you were six months ago.

Q: Does the model treat all questions equally?

A: No. The model evaluates questions based on Item Difficulty. Getting a “High Difficulty” question wrong is expected and penalized less. However, missing “Low Difficulty” questions (questions that most students get right) is a stronger negative signal, as it indicates a foundational knowledge gap.

Interpreting Your Behaviors

This section explains the specific “flags” that might lower or raise a student’s probability.

Q: What does the model consider “Panic”?

A: “Panic” is detected when a student rushes through a sequence of questions much faster than the average reading time allows, often occurring specifically during Mocks or after getting a difficult question wrong. This is a high-risk signal because it indicates a loss of composure that can be disastrous on the real exam.

Q: Students often second-guess themselves and change their answers. Does the model penalize this?

A: It depends on the result. The model measures a “Doubt Factor”—specifically tracking how often you switch a correct answer to an incorrect one. Frequent “Right-to-Wrong” switching is a behavioral risk signal that can lower your Pass Probability, even if your overall average is okay, because it indicates a lack of confidence.

Q: What do you mean by “Stamina Drops” or “Post-Mistake Resilience”?

A: The exam is a marathon, not a sprint. The model specifically measures Post-Mistake Resilience by calculating your accuracy immediately after you make an error to see if you maintain your focus. Additionally, it tracks Stamina Drops—how much your accuracy decays when facing highly complex questions or toward the end of a long study block. If your performance drops off significantly in either scenario, the model views this as a risk for the long-duration Board Exam.

Q: Does the time of day I study actually affect my prediction?

A: Yes. The model detects circadian and seasonal patterns in your study history. It analyzes whether you have a consistent daily routine or if your study habits are erratic. In our dataset, we find that students who demonstrate a stable, predictable study pattern often perform better on their board exams than those with unpredictable schedules.

Q: A student crammed for 12 hours a day the week before the Mock. Why didn’t their probability go up?

A: The model includes a “Cramming” feature that compares your maximum daily volume against your typical median volume. Sustainable, spaced repetition is a stronger predictor of passing than sudden, unsustainable bursts of activity. If the model detects a “Cramming” pattern, it may view that high volume as a risk factor rather than a strength.

Troubleshooting Confusing Results

This section addresses common “Why is my score X?” scenarios.

Q: A student got a higher accuracy/pass probability on the Mock Assessment than their classmate, but the student’s Comprehensive Exam Readiness Pass Probability is lower. Why?

A: The model does not look only at your Mock Assessment Accuracy. It looks at your entire study history. If your probability is lower, it may be detecting risk factors such as:

• Inconsistent study habits (cramming vs. daily practice).
• Rushing through questions (answering faster than is realistic to read/comprehend).
• Weak performance on “Easy” questions, which significantly impacts scaling.
• A “Panic” pattern where accuracy drops sharply after a string of difficult questions.

Q: A student accidentally left a test open overnight. Did it ruin their “Pacing” score?

A: No. The model is smart enough to detect anomalies. Extremely long duration outliers (like leaving a tab open for 12 hours) are generally filtered out of the “Pacing” algorithm so they do not artificially inflate your time-per-question metrics.

Q: A student experienced an internet glitch during their  Mock Assessment. Will this lower their probability of passing?

A: Generally, no. If a technical issue caused a single erratic data point (like a 5-minute answer time for one question), the model’s outlier detection usually smooths this out. However, if the glitch caused you to leave many questions unanswered, your raw mock assessment accuracy would drop, which would lower your probability.

Q: A student has a high overall QBank average, but their prediction is low. Why?

A: If a student had very high scores early in the year but their recent performance has trended downward or plateaued while the material got harder, the model picks up on this negative momentum. A downward trend is a stronger predictor of risk than a high historical average.

Q: What if a student used TrueLearn heavily several months ago but stopped for a few months?

A: The model’s “Recency Weighting” means it will penalize that gap. Your “Consistency” metric will have dropped, and the model effectively treats your old knowledge as “decayed”. You will likely see a lower probability until you re-establish a consistent study streak and prove you have retained that information.

Next Steps & Logistics

Q: Does this prediction mean a student is destined to fail?

A: Absolutely not. The prediction is a snapshot of their current trajectory. It is a “check engine light,” not a final judgment. If your probability is low, it is a signal to adjust your study habits—focus on consistency, slow down your pacing, or review core weaknesses—to improve your odds before exam day.

Q: Will a student’s predicted probability update as they continue to answer QBank questions?

A: No. The prediction is a fixed benchmark calculated at the moment they submit your Mock Assessment. It reflects their Mock assessment accuracy combined with the “Behavioral Forensics” (consistency, stamina, pacing) they demonstrated up to that point in time. Because study habits can fluctuate, this prediction is valid only for that specific date. Currently, to see an updated probability that reflects new study momentum, they will need to take a subsequent Mock Assessment if it is available.

Q: A student took the Mock Assessment in January, but their exam is in June. Is the prediction still valid?

A: The prediction is most accurate close to the time the assessment was taken. It is not valid indefinitely. If a student takes the Mock in January, the prediction reflects your readiness as if you were taking the Board Exam that day. It does not account for the studying they will do between now and June.

Q: What specific habits should a student change to improve their prediction next time?

A: While every student is different, the model rewards:

• Stamina: Maintaining accuracy even at the end of long question blocks.
• Consistency: Studying in regular intervals rather than “Cramming” (large volume spikes followed by inactivity).
• Confidence: Reducing uncertainty (the frequency of changing right answers to wrong answers).
• Pacing: Avoiding “Panic” behavior, where you rush through questions faster than is possible to read them.

Q: Can my professors or advisors see this prediction?

A: Yes, if you have an institutional subscription. This tool is designed as a partnership between you and your institution. Your advisors see these predictions to help them identify who needs extra support. If you see a low probability, it is a great prompt to schedule a meeting with your academic support team to discuss the specific behaviors (like pacing or anxiety) the model has flagged.

Q: Does the model account for a specific exam date?

A: The prediction is a snapshot in time. It evaluates your readiness as if you were taking the exam today and does not project future learning or “guess” how much you will improve over the coming weeks.
As illustrated in Figure 1 (AUC-ROC), while the advanced Comprehensive Exam Readiness model is capable of identifying risk up to 12 months prior to the exam, its accuracy naturally peaks the closer you get to your actual test date.
Based on the model’s performance data shown in Figure 1, here is how you should interpret your prediction depending on your timeline:

• 6 to 12 months away: A lower probability is completely normal. At this stage, the Advanced model is already providing predictive value (operating between 0.6 and 0.7 AUC). Use this early snapshot as a baseline to identify weak spots and course-correct bad habits.
• 2 to 4 months away: The prediction enters a zone of high reliability, crossing the 0.8 AUC threshold. This is the critical window to aggressively target the behavioral risks or knowledge gaps identified in your Insights.
• 1 month or less: The model’s accuracy peaks at its highest level (exceeding 0.9 AUC). A low probability in this immediate window is an urgent call to action, and you should strongly consider consulting with your advisor before sitting for the exam.