On January 10, 2022, TrueLearn launched a new predictive model for the COMLEX-USA Level 1 mock assessment. The new model improves the accuracy of translating student performance on TrueLearn’s COMLEX-USA Level 1 mock assessment to predicted performance on the actual standardized exam.
The new predictive model went into effect for all COMLEX-USA Level 1 mock assessments completed on or after January 10, 2022.
What do you need to know?
Here’s what you need to know about the release of the new COMLEX-USA Level 1 predictive model and how to use and interpret its predictions when considering whether you are prepared to sit for the actual standardized exam.
| #1 The new COMLEX-USA Level 1 predictive model launched on January 10, 2022 for all mock assessments completed on or after this date. |
| #2 The individual assessment summary report provides you with a prediction about whether you will “Pass” or “Fail” the actual COMLEX-USA Level 1 exam based on your mock assessment score. |
| #3 For the most accurate predictions, treat the mock assessment as seriously as you would the actual exam. This means that you should take the assessment under conditions that most closely mimic the real-world exam. Since the prediction is based on how you performed when you completed the mock assessment, the results can fluctuate. If feasible, complete the assessment as close as possible in time to when you will sit for the actual exam. Also, if time permits, review your results and compare them to your study plan. In addition to reviewing your strengths and weaknesses, reflect on your studying and testing behaviors that may impact your performance. |
| #4 A prediction is not fate; it is an indicator for how you may do on the real-world exam given your performance on the mock assessment at the time of testing. In addition to the prediction, it is important to consider your preparedness to sit for the real-world exam in conjunction with other measures of test readiness. |
What predictions does the model provide you?
When you complete TrueLearn’s COMLEX-USA Level 1 mock assessment, you are provided with an individual assessment summary report that shows a prediction for whether you may “Pass” or “Fail” the actual standardized exam.
| TrueLearn’s COMLEX-USA Level 1 mock assessment is the most predictive when the assessment is administered as close as possible in time to when you will sit for the actual standardized exam and under conditions that most closely mimic the real-world exam. You should take the mock assessment in a proctored or controlled setting and complete it in one sitting to simulate a similar experience to the real exam. The more representative of the real-world standardized exam your test-taking experience is when completing the mock assessment, the more predictive and accurate the mock assessment will be of your performance on the actual exam. |
How should you use and interpret the model’s predictions?
When using the model’s prediction to identify whether you are prepared to sit for the actual exam, there are several important factors to keep in mind.
- The mock assessment is most predictive of your real-world performance on the actual standardized exam when the assessment is taken as close as possible in time to when you will sit for the actual exam and under conditions that are representative of the actual exam.
- The prediction about your performance on the actual COMLEX-USA Level 1 exam is just that – a prediction. It is not fate, and the prediction may be wrong. There are many factors that influence the predicted outcome, such as how you felt when you took the mock assessment and the mock assessment’s method of delivery (and how its delivery differed from the real exam). Additionally, no predictive model is perfect, and all predictions are associated with some error. Remember to consult with your faculty advisor and consider other measures of test readiness, in addition to the prediction, when deciding on whether you are prepared to sit for the actual exam.
- Do not forget to consider all of the other performance metrics provided by the individual assessment summary report. The prediction for whether you may pass or fail the real-world exam is only one data point. For example, carefully review how you performed on different topic categories and identify competencies where you are weak and require more study. Also, consider how you felt while taking the mock assessment. Did you feel confident as you completed the assessment? Were you surprised or unsure of its content? Use this self-reflection as an opportunity to identify areas for improvement.
Why did we develop a new predictive model?
TrueLearn is committed to providing a world-class product that prepares students to become strong life-long learners inside and outside of the classroom. A key part of our data-centric approach is to communicate comparative benchmarks for students to use to gauge their preparedness. Knowing where you stand and how you will perform is critical for your preparation, and TrueLearn’s COMLEX-USA Level 1 mock assessment provides a benchmark for you to use to assess your likely real-world performance on the actual standardized exam.
The new predictive model for TrueLearn’s COMLEX-USA Level 1 mock assessment was developed by leveraging historical performance data comparing students’ scores on the assessment to their actual performance on the real-world standardized exam.
Our own internal testing found that this new model more accurately predicts how students will perform on the actual exam compared to the previous approach used for score prediction. This increased accuracy will help students better understand their level of preparedness to sit for the real-world exam.
How was the model developed?
The new model for the COMLEX-USA Level 1 mock assessment was developed using a predictive modeling approach. Data for a sample of students that completed TrueLearn’s COMLEX-USA Level 1 mock assessment and sat for the actual standardized exam were used to train the predictive model. The data included the students’ percentage correct score on the COMLEX-USA Level 1 mock assessment and the most recent score on the actual standardized exam taken after completing the mock assessment. Several additional performance metrics are also used by the model.
The model was trained and validated using a best-in-class 10-fold cross-validation modeling approach. Several different linear and non-linear models were investigated and the final model selected demonstrated the best performance on several key validation metrics, including out-of-sample (OOS) R-squared (R2) and correlation (r) between students’ actual scores on the real-world COMLEX-USA Level 1 exam and the model’s predicted scores for how students may perform on the actual exam.
How will the model be maintained and improved over time?
TrueLearn is committed to providing a world-class preparation product and we consider the predictive models that we develop and deploy as being a key feature of that product. We conduct quarterly reviews of the COMLEX-USA Level 1 score prediction model’s performance to identify deficiencies and ensure it continues to provide accurate predictions. Additionally, we continually strive to enhance and improve the model by leveraging more data and experimenting with different modeling approaches.
| Help us improve the accuracy of the model’s predictions! The most important ingredient for building accurate predictive models is data. You can help us to improve the accuracy of the COMLEX-USA Level 1 predictive model by sharing your score report with TrueLearn after you sit for the real-world standardized exam. Please upload your score report from within your TrueLearn account by navigating to your “Settings”, selecting “Subscriptions”, and then “Upload Score”. |
If you have any questions about the COMLEX-USA Level 1 score prediction model, please reach out to the TrueLearn Customer Support team.