Mean Scaled MBE versus Pass Rates
According to the November 2006 edition of NCBE's Bar Examiner periodical, "[i]f an individual of average proficiency sits for the bar with a particularly bright candidate pool, this individual’s raw written scores will remain lower than they would have been in previous sittings with less able peers. But the equating of the MBE will take into account that this is a particularly bright candidate pool and that the individual in question is in fact of average ability. The individual’s written test scores will then be scaled to account for the difference in the candidate pool, and his written test scores will be brought into alignment with his demonstrated level of ability. Scaled essay scores lead to total bar examination scores that eliminate contextual issues and that accurately reflect individual proficiency. Therefore, the higher the MBE average for an exam, the higher the scale. Scaling written scores to the MBE does not change the rank-ordering of examinees on either test. A person who had the 83rd best MBE score and the 23rd best essay score will still have the 83rd best MBE score and the 23rd best essay score after scaling." (emphasis added)
I decided to examine whether the higher the MBE average for an exam, the higher the pass rate. I concluded that generally, the mean MBE scaled score is positively correlated with pass rates, so examinees can expect a higher overall pass rate if there is a higher mean MBE scaled score. The below graphs examine the national mean scaled MBE score versus overall pass rates for July and February exams from 1996 to 2013. The graphs are based on data contained in the NCBEX Bar Examination and Admission Statistics. Please note that these pass rate statistics include a few jurisdictions that do not use the MBE (i.e. Louisiana and formerly Washington). This can skew the results because the pass rates for these jurisdictions are not tied to the MBE. However, based on these graphs, there is still a positive correlation between mean scaled MBE score and overall pass rates for all periods except for July 1998, July 1999, July 2005 and July 2009.
What is more interesting is that recent MBE mean scores have resulted in lower pass rates as compared to past MBE mean scores. For example, the February 2013 mean MBE scaled score of 138 resulted in an overall pass rate of 57.7%. However, the February 1999 mean MBE scaled MBE score of 137.9 resulted in a higher overall pass rate of 58.9%. For a very small difference in mean scaled MBE score, the pass rate difference was rather dramatic. Based on an average of 23,000 examinees nationwide sitting for a February administration, this means that an extra 282 examinees failed February 2013 as compared to February 1999, despite having a slightly higher mean MBE scaled score. In another example, the July 2013 mean MBE scaled score of 144.3 resulted in an overall pass rate of 72.1%. However, there are a number of past July administrations with lower mean MBE scores and higher pass rates. For example, the July 2011 mean MBE scaled MBE score of 143.8 resulted in an overall pass rate of 72.7%; the July 2010 mean MBE scaled MBE score of 143.6 resulted in an overall pass rate of 72.8%; and the July 2007 mean MBE scaled MBE score of 143.7 resulted in an overall pass rate of 72.9%. Whether this trend is due to a perceived decline in essay quality, I don't know, and only time will tell.
If you have any additional questions or comments, please email me at joe@seperac.com.
Click Here to go back to the site.