Seperac.com July 2010 Essay/MPT Comparison Report Sample

To help examinees better understand the NY bar exam essays/MPT, I created an Essay/MPT comparison report. This analysis examines a collection of essays/MPTs and looks through them for matching words in phrases (minimum of 3 words). The reports contain the document text with the matching phrases underlined. The reports also show PDFs of the two essays you selected side-by-side. Put simply, examinees learn by example - reviewing a collection of graded essays helps you better understand the New York essays/MPT. I find this essay/MPT comparison to be incredibly useful to examinees. For example:

You can compare an essay to any of the other examinees essays or the above average answers.
You will see exactly what NYBOLE considers to be a passing essay. You can then compare your essay to passing essays (for example, above a 46.81 scaled score on the July 2010 exam).
You can compare an essay to the best essay, the worst essay, or essays with scores with similar scores (and sometimes exactly the same score).
You can learn how much (or how little) is written for high scoring essays.
In the Text comparison, you can see exactly what words an essay used that the comparison essay also used.
In the PDF comparison, you can compare the the style, layout, penmanship, neatness of essays.
You can compare handwritten essays to typed essays.
You can divine other useful information (for example, for the July 2010 exam, I have essays from two examinees in seat # sequence. It is likely that these essays were graded by the same grader one after the other. Comparing the essays/grades may give a better understanding of what the grader was thinking when the grader assigned the grades).

I feel this analysis is invaluable for examinees to discover "what works" versus "what doesn't work." For example, you can use the Text comparison to compare high scoring essays/MPTs to the released above average answers to see what phrases both essays shared. I am in the process of examining the essays to see if I can determine (with reasonable confidence) if anything other than an essay's content has a bearing on the essay's score. For example, does penmanship, neatness, headings, or mis-spellings have any effect on an examinee's final score?

In the Essay/MPT analysis, each and every essay is compared to every other essay. There are three columns on the report - a "Matching Words" column, a "Text Comparison" column, and a "PDF Comparison" column.

The "Matching Words" column reports the number of perfectly matching words that have been marked in the pair of documents. It includes too-short phrases that require bridging over non-matching words in order to count as matching between the two documents. Each "Matching Words" row item has 3 subparts: (a) the number of matching words; (b) what percentage of the first document is accounted for by these matching words; and (c) what percentage of the second document is accounted for by these matching words. Finding commonality in the matching words of passing essays can help examinees develop a vocabulary for those issues.

The "Text Comparison" column shows the text matches between the two essays you select. In the reports, perfect matches are indicated by red-underlined words and bridging, but non-matching words are indicated by green-italicized-underlined words. The matching phrases are links. If you click on a matching phrase, you will be taken to the equivalent phrase in the other document of the pair.

The "PDF Comparison" column shows the PDFs of the two essays you select side-by-side.

To maintain the anonymity of the examinees, all identifying information has been redacted and each examinee is assigned a random 3-digit ID. In the tables, the hyperlink naming convention operates as follows:
Exam-Essay-Score-Typed/Written-ID

For example, the naming convention "July 2010-Essay1-Score 33.92-Typed-ID 052" means that this is an essay from the July 2010 exam, it was written in response to Essay Question 1, the scaled score of the essay was 33.92, the examinee typed the essay, and the randomly generated ID of the candidate was 052.

If the essay is an above average answer released by NY BOLE, in place of an ID, the following appears:
NYBOLE-Above Average Answer 1
NYBOLE-Above Average Answer 2

Please note that these released above average answers do not have scores, but it would be safe to say these essays received a score between 65-80.

I suggest examinees send a full day comparing and contrasting these essays/MPTs, time permitting. The more you learn about what comprises a good essay, the better you will do on future essays. You will also see instances of essay unreliability - where one essay appears to be better than another but receives a lower score. Sometimes this is due to the reader's subjective opinion, and sometimes it is due to grading unreliability. If you identify a pair of essays that illustrate what you perceive to be an egregious case of grading unreliability, please email me at joe@seperac.com.

For the July 2010 exam, for each essay/MPT, there are 1,596 comparisons based on 55 examinee essays and the two released above average answers. The essays are sorted in score order - the lowest scoring essays are at the top and the highest scoring essays are at the bottom. These essays will give you a good idea of what is required to write a "passing" essay on a July exam.

In the following sample, there are 10 comparisons based on 3 examinee essays and the two released above average answers.


 


Matching WordsMPT Text comparisonMPT PDF comparison
289 [21%,19%]July 2010-MPT 01-NYBOLE-Above Average Answer 2 vs. July 2010-MPT 01-NYBOLE-Above Average Answer 1July 2010-MPT 01-NYBOLE-Above Average Answer 2 vs. July 2010-MPT 01-NYBOLE-Above Average Answer 1
130 [20%,8%]July 2010-MPT 01-Score 29.53-Written-ID 001 vs. July 2010-MPT 01-NYBOLE-Above Average Answer 1July 2010-MPT 01-Score 29.53-Written-ID 001 vs. July 2010-MPT 01-NYBOLE-Above Average Answer 1
107 [16%,7%]July 2010-MPT 01-Score 29.53-Written-ID 001 vs. July 2010-MPT 01-NYBOLE-Above Average Answer 2July 2010-MPT 01-Score 29.53-Written-ID 001 vs. July 2010-MPT 01-NYBOLE-Above Average Answer 2
188 [32%,12%]July 2010-MPT 01-Score 45.53-Written-ID 015 vs. July 2010-MPT 01-NYBOLE-Above Average Answer 1July 2010-MPT 01-Score 45.53-Written-ID 015 vs. July 2010-MPT 01-NYBOLE-Above Average Answer 1
152 [26%,11%]July 2010-MPT 01-Score 45.53-Written-ID 015 vs. July 2010-MPT 01-NYBOLE-Above Average Answer 2July 2010-MPT 01-Score 45.53-Written-ID 015 vs. July 2010-MPT 01-NYBOLE-Above Average Answer 2
45 [7%,7%]July 2010-MPT 01-Score 45.53-Written-ID 015 vs. July 2010-MPT 01-Score 29.53-Written-ID 001July 2010-MPT 01-Score 45.53-Written-ID 015 vs. July 2010-MPT 01-Score 29.53-Written-ID 001
190 [19%,13%]July 2010-MPT 01-Score 60.97-Typed-ID 043 vs. July 2010-MPT 01-NYBOLE-Above Average Answer 1July 2010-MPT 01-Score 60.97-Typed-ID 043 vs. July 2010-MPT 01-NYBOLE-Above Average Answer 1
247 [25%,18%]July 2010-MPT 01-Score 60.97-Typed-ID 043 vs. July 2010-MPT 01-NYBOLE-Above Average Answer 2July 2010-MPT 01-Score 60.97-Typed-ID 043 vs. July 2010-MPT 01-NYBOLE-Above Average Answer 2
48 [4%,7%]July 2010-MPT 01-Score 60.97-Typed-ID 043 vs. July 2010-MPT 01-Score 29.53-Written-ID 001July 2010-MPT 01-Score 60.97-Typed-ID 043 vs. July 2010-MPT 01-Score 29.53-Written-ID 001
122 [12%,21%]July 2010-MPT 01-Score 60.97-Typed-ID 043 vs. July 2010-MPT 01-Score 45.53-Written-ID 015July 2010-MPT 01-Score 60.97-Typed-ID 043 vs. July 2010-MPT 01-Score 45.53-Written-ID 015