Student Guide

What do you need to know?

Here’s what you need to know about the current version of the COMBANK Level 1 Assessment score prediction model and how to use and interpret its predictions when considering whether you are prepared to sit for the real-world COMLEX-USA® Level 1 standardized exam.

What predictions does the score prediction model provide you?

When you complete TrueLearn’s TrueLearn’s COMBANK Level 1 Assessment, you are provided with an Individual Assessment Summary Report that shows a score prediction for how you may perform on their real-world exam. This prediction is represented as a probability, or likelihood, of passing. This probability is a percentage that ranges from 0 percent to 100 percent.

Please note that students who completed a COMBANK Level 1 Mock Assessment before February 1, 2025, will be shown a predicted outcome of “Pass” or “Fail” using a previous version of the score prediction model.

How should you use and interpret the model’s predictions?

When using the model’s prediction to help you identify if you are prepared to sit for the actual exam, there are several important factors to keep in mind. 

1. The COMBANK Level 1 Assessment’s score prediction model provides an estimated probability of you passing the real-world COMLEX-USA® Level 1 standardized exam based on your performance on the mock assessment. It is designed as a decision-support tool to help you assess your exam readiness. However, the predicted probability should not be interpreted as a guarantee of success or failure, but rather as an indicator of likelihood based on historical test-taking patterns. Also, while the model has been trained on relevant test performance data, it may not account for unexpected variables or changes in exam difficulty over time. The score prediction model is most effective when used as a tool for measuring your readiness to sit for the real-world exam, close in time after completing the mock assessment.

2. The table below provides additional information about your performance on the COMBANK Level 1 Assessment, which relates to the score prediction model’s predicted probability of passing the real-world COMLEX-USA® Level 1 standardized exam. Table 1 lists some general guidelines for how your predicted probability of passing may be interpreted with regard to your readiness to sit for the real-world exam. Remember, the score prediction model is intended to support, not replace, human judgment in assessing exam readiness. Decisions about exam readiness should always incorporate broader context and expert guidance rather than relying solely on the model’s output.

Table 1. Guidelines for usage and interpretation of predicted probability for TrueLearn’s COMBANK Level 1 Assessment score prediction model.

3. The mock assessment is most predictive of your real-world performance on the actual COMLEX-USA® Level 1 standardized exam when the assessment is taken as close as possible in time to when you will sit for the actual exam and under conditions that are representative of the actual exam. Among the data used to train the score prediction model, over 90% of the students included in the training set took their real-world COMLEX-USA® Level 1 standardized exam within two months after completing the mock assessment. Practically, this means the model’s predictions for whether you will pass or fail their real-world exam are most applicable and accurate within a two-month period of time after completing the mock assessment. While the model’s predictions are still useful if you take your real-world standardized exam more than two months after completing your mock assessment, the predictions are less accurate and more prone to error. It is because of this detail related to the training data used to build the model that makes it so important for you to take their exam as close as possible to when you complete the mock assessment, and ideally no more than two months out, so that the accuracy of the model’s predictions is maintained.

4. The score prediction model was created using a collection of historical data that brings together students’ mock assessment scores and real-world exam scores (pass vs. fail). Ideally, these data would include a similar distribution of high and low performers on the mock assessment and real-world exam. However, in practice, few students score on the very low or high end of the score distribution of the mock assessment, and most students pass their real-world exam on their first attempt. The collection of historical data used to build the score prediction model reflects this reality. Ultimately, this impacts the accuracy of the model’s predictions in two very important ways that are critical to understand. First, on average, the score prediction model is most accurate for students who score within the middle of the distribution of scores of the mock assessment, which is between 45% and 76%. Second, on average, the model is less accurate at predicting whether a student will fail the real-world exam (i.e., having a low probability of passing) relative to the alternative; that is, passing the real-world exam (i.e., having a high probability of passing). In both of these cases, accuracy is lost because the model has not “seen” or “learned” from as many examples of students who scored very high (or low) on the mock assessment or failed the real-world exam on the first attempt. When interpreting the model’s predictions, be careful to review your mock assessment score, with a special focus on scores that are very low or very high, and recognize that the model is, on average, less accurate at correctly predicting when you will fail.

5. The prediction about your performance on the actual COMLEX-USA^® Level 1 exam is just that – a prediction. It is not fate, and the prediction may be wrong. There are many factors that influence the predicted outcome, such as how you felt when you took the mock assessment and the mock assessment’s method of delivery (and how its delivery differed from the real exam). Additionally, no predictive model is perfect, and all predictions are associated with some error. You should consider other measures of test readiness, in addition to the prediction, when deciding on whether you are prepared to sit for the actual exam.

6. Do not forget to consider all of the other performance metrics provided by the assessment summary reports. The prediction for whether you may pass or fail the real-world exam is only one data point. For example, carefully review how you performed on different topic categories and identify competencies where they are weak and require more study. You should also consider how you felt while taking the mock assessment. Did you feel confident as you completed the assessment? Were you surprised by or unsure of its content? Use this self-reflection as an opportunity to identify areas for improvement.

Why did we develop a new predictive model?

TrueLearn is committed to providing a world-class product that prepares students to become strong lifelong learners inside and outside of the classroom. A key part of our data-centric approach is to communicate comparative benchmarks for students to use to gauge their preparedness. It is critical for their preparation that students know where they stand and how they will perform, and TrueLearn’s COMBANK Level 1 Assessment provides a benchmark for students (and faculty) to use to assess their likely real-world performance on the actual COMLEX-USA® Level 1 standardized exam.

The score prediction model for TrueLearn’s COMBANK Level 1 Assessment was developed by leveraging historical performance data comparing students’ scores on the mock assessment to their actual performance on the real-world standardized exam. 

Internal testing found that the updated model more accurately predicts how students will perform on the actual exam compared to the previous approach used for score prediction. This increased accuracy will help students better determine their level of preparedness to sit for the real-world exam.

How was the model developed, and how does it perform?

The new model for the COMBANK Level 1 Assessment was developed using a predictive modeling approach. Data for a sample of students who completed TrueLearn’s COMBANK Level 1 Assessment and sat for the actual standardized exam were used to train the predictive model. The data included the students’ percentage correct score on the COMBANK Level 1 Assessment and the most recent score on the actual standardized exam taken after completing the mock assessment. Several additional performance metrics are also used by the model. 

The model was trained and validated using a best-in-class 10-fold cross-validation modeling approach. Several different linear and non-linear models were investigated and the final model selected demonstrated the best performance on several key validation metrics, including out-of-sample (OOS) R-squared (R²) and correlation (r) between students’ actual scores on the real-world COMLEX-USA Level 1 exam and the model’s predicted scores for how students may perform on the actual exam.

How will the model be maintained and improved over time?

TrueLearn is committed to providing a world-class learning and exam preparation resource, and we consider the predictive models that we develop as being a key feature of that product. We conduct quarterly reviews of the COMBANK Level 1 Assessment’s score prediction model’s performance to identify deficiencies and ensure it continues to provide accurate predictions. Additionally, we continually strive to enhance and improve the model by leveraging new and additional data and experimenting with different modeling approaches.

If you have any questions about the COMBANK Level 1 Assessment’s score prediction model, please reach out to the TrueLearn Customer Support team.

*COMLEX-USA® Level 1 is a trademark of the National Board of Osteopathic Medical Examiners® (NBOME®). This content is not endorsed or approved by NBOME.