Must Read Nbme Grading Shelf

[LDMD2B]

I THINK SMU NEEDS TO READ THE RULES ABOUT THE SHELF FORM THE NBME, I THINK THEY ARE ABOUT TO MAKE A HUGE GRADING ERROR. READ THE RED BELOW


The National Board of Medical Examiners (NBME) reports student scores from its
subject (shelf) exams in what the NBME calls a Subject Test Score (STS). While
reported as a two-digit score, the STS is not a percent score. Consequently, the STS
should not be treated as such in calculating a student’s final percent score in a course or
clerkship.
The Subject Test Score is a standard score. What happens when raw scores (the actual
number of items answered correctly on an exam – e.g., 51 would be a student’s raw score
if she answered 51 of 55 test items correctly) or percent scores (e.g., 51 of 55 would be
93%) are standardized? The mean and standard deviation are calculated from ALL
exam scores. We now can tell a student how much above or below the mean she is for
the group taking the exam. Thus, if T1 Susan Smith were 1.17 standard deviations above
the mean on her Physiology exam, we would say her standard score is 1.17.
If an exam does not yield scores that are normally distributed (typically the case with
local medical school exams), the value of knowing a student’s standard score is limited.
We know that Ms. Smith scored higher than most of her classmates, but we cannot
interpret her score in normal distribution terms.
However, standard scores for an NBME shelf exam are calculated from a large
population certain to yield a normal distribution of scores. If Susan Smith were a T3 in
Surgery, her standard score of 1.17 on the Surgery shelf exam might tell us that she is at
the 88th percentile, or that she scored better than 88 percent of the national group on
which the standard scores was based.
While psychometricians prefer to report standard scores in standard deviation units,
this is not necessary. At its origin USMLE Step standard scores had a mean of 200 and a
standard deviation of 20. A student with a score of 220 was one standard deviation above
the mean or at the 84th percentile.
The NBME subject (shelf) exams are reported as standard scores using a mean of 70 and
a standard deviation of 8. This unfortunate choice of a two-digit scale can be easily
confused with percents. The take-home message is: DO NOT TREAT A SUBJECT
TEST SCORE FROM A SHELF EXAM AS A PERCENT. This is an avoidable
measurement error, and the resulting course or clerkship final “percent” score will be
both invalid and a disservice to students.
What NOT TO DO:
• Medical schools sometime use two methods to “convert” shelf exam standard
scores to percents. One approach is to change the mean STS of 70 to a typical
local test mean percent score, say 80%, and use the STS standard deviation of
8 to adjust scores above and below the mean. A related approach is simply to
multiply the STS score by some factor (e.g., 1.15 or 1.25). Both methods are
psychometrically unsound practices that should be avoided.
What TO DO:
• The NBME will provide students’ actual percent scores from a shelf exam
(if you are insistent). You will find that the distribution of shelf exam percent
scores is lower than your local exam scores. It is defensible to adjust shelf
exam percent scores to conform to local standards.
• Some medical schools use the percentiles paralleling STS standard scores as
cutoffs for the Honors grade. For example, to obtain an Honors in Surgery,
you must (1) successfully complete all required procedures, (2) be rated
superior on all faculty/resident clinical evaluations, AND (3) be at or above
the 90th percentile on the NBME shelf exam.
This approach has the advantage of controlling the number of Honors grades
in a clerkship. Further, the NBME will send quarterly norms for your shelf
exam, allowing you to use a changing standard score cutoff to determine
Honors as the year progresses. These quarterly norms reduce the advantage of
taking a clerkship later in the year when the student will have benefited from
her experiences in prior clerkships.

[AzDaku]

Thanks for the info, have you thought about emailing it to everybody at SMU?

[Jeep23Guy]

Someone hasn't been paying attention AT ALL.

SMU knows how they score the NBME exams. They are scored pretty much the EXACT SAME WAY AS STEP 1.

Like I was saying, they know how the exams are scored and the grades will obviously be curved (as they are at most US med schools that use the shelf exam as a final).

[AzDaku]

Someone hasn't been paying attention AT ALL.

SMU knows how they score the NBME exams. They are scored pretty much the EXACT SAME WAY AS STEP 1.

Like I was saying, they know how the exams are scored and the grades will obviously be curved (as they are at most US med schools that use the shelf exam as a final).

it would be great if somebody would explain this to all the faculty because last semester i did not get a curve for taking the shelf lol.

[Jeep23Guy]

Last semester we didn't have the subject shelf exams, so there was no curve involved...

[Laurie S]

I thought last semester's exam was extra credit?

[UFTim]

I thought last semester's exam was extra credit?

You're 100% correct. It was extra credit last semester

[Jeep23Guy]

Yes, but last semester we had the COMPREHENSIVE shelf and not the subject shelf exams...

[onelifetolive]

So is this issue corrected at SMU or what ?????

[desai29]

I feel every foreign school should have COMPREHENSIVE shelf nor subject shelf exams

[Junito]

So is this issue corrected at SMU or what ?????

I don't think this will change. The faculty want to guage the performance of students and their faculty with these subject shelf exams...It would be ideal to have students take the exam, and that the exam not be calculated into the final grade. If this were the case not many students would show up for the exam. They would rather study for the final exam... The school then tried to use a carrot (extra points) to encourage students to take the comp, and most people showed up.

Now it is using the shelf exams as part of the grades and well, that is not a new concept. At Ross the subject shelf exams were 15% of your grade. But you had to take only one, and it was a mystery which one you were taking till you sat down to take the test, hence the term "mystery shelf" at Ross. I say get rid of the subject shelf exams and bring back the comp.

[LDMD2B]

I wish the comp exam was back. NOT because of the extra points, they can keep them, but because I want to know how I improved. Every semester I have been improving my comp grade it would of been nice to see it now. Having it as a part of your overall grade is a bad idea, you guys will see in a few weeks. The shelf will bring everyones grades down. If you are passing a class witha 71, you will fail because the chance of getting above a 60 (with out a curve) on the shelf is slim to none from talking to people who took it. Ross does give one shelf, but they also curve your final course grade. SMU if not like Ross but lately with the policy changes they are becoming more like Ross.:evil:

The one thing I don't agree is how the grading system always changes, that means one semester you can get a 68 and get 4 extra for shelf and pass. Or this semester you get a 70 in a class and you FAIL because of the shelf. Last year when the school went from subject shelf to comp, I was a new SMU student and I went and talked with Dr.N , he told me that most people were getting in the 20's and 30's on the subject shelf at smu, and thus I should only expect to get 2 or 3 extra credit points. But in my opinion the comp is a bit easier because it covers your strong and weak points, a subject specific is just that, specific and its hard. I 2 days you will all see what I am talking about.:rolleyes:

[AzDaku]

I wish the comp exam was back. NOT because of the extra points, they can keep them, but because I want to know how I improved. Every semester I have been improving my comp grade it would of been nice to see it now. Having it as a part of your overall grade is a bad idea, you guys will see in a few weeks. The shelf will bring everyones grades down. If you are passing a class witha 71, you will fail because the chance of getting above a 60 (with out a curve) on the shelf is slim to none from talking to people who took it. Ross does give one shelf, but they also curve your final course grade. SMU if not like Ross but lately with the policy changes they are becoming more like Ross.:evil:

The one thing I don't agree is how the grading system always changes, that means one semester you can get a 68 and get 4 extra for shelf and pass. Or this semester you get a 70 in a class and you FAIL because of the shelf. Last year when the school went from subject shelf to comp, I was a new SMU student and I went and talked with Dr.N , he told me that most people were getting in the 20's and 30's on the subject shelf at smu, and thus I should only expect to get 2 or 3 extra credit points. But in my opinion the comp is a bit easier because it covers your strong and weak points, a subject specific is just that, specific and its hard. I 2 days you will all see what I am talking about.:rolleyes:

Wassup hommie, i hear ya man. I totally agree that sometimes that this school is turning into Ross. But you have to remember Ross has been around for years and they must be doing something right. I am not defending Ross, but simply stating that they doing something right out there. As per our school, we are still babies, and we are still trying to create an identity for ourselves. If SMU, like in the past, decided to pass everybody, our Step 1 scores would reflect their actions, and our school would be looked down upon, and we might loose liscencing as the resulting consequence. Everybody should be happy that they are in Medical School, who cares about the stuff that occurs with administration. Just do your best, and in the end your hard work will pay off.

[Laurie S]

It looks like AUC is getting rid of their shelf exams. :shock:

Middle of page 4-under "Chance Count" name

http://www.valuemd.com/images/misc/subscribed.gif I Passed!!! (http://www.valuemd.com/american-university-caribbean-auc/109648-i-passed.html) (http://www.valuemd.com/images/misc/multipage.gif 1 (http://www.valuemd.com/american-university-caribbean-auc/109648-i-passed.html) 2 (http://www.valuemd.com/american-university-caribbean-auc/109648-i-passed-2.html) 3 (http://www.valuemd.com/american-university-caribbean-auc/109648-i-passed-3.html) ... Last Page (http://www.valuemd.com/american-university-caribbean-auc/109648-i-passed-5.html))

[AzDaku]

i could not find it in the forum, but that guy was really excited that he passed the exam. It took him 7 semesters to pass basic science, mad props to him lol.

[microphage]

It looks like AUC is getting rid of their shelf exams. :shock:

Middle of page 4-under "Chance Count" name

http://www.valuemd.com/images/misc/subscribed.gif I Passed!!! (http://www.valuemd.com/american-university-caribbean-auc/109648-i-passed.html) (http://www.valuemd.com/images/misc/multipage.gif 1 (http://www.valuemd.com/american-university-caribbean-auc/109648-i-passed.html) 2 (http://www.valuemd.com/american-university-caribbean-auc/109648-i-passed-2.html) 3 (http://www.valuemd.com/american-university-caribbean-auc/109648-i-passed-3.html) ... Last Page (http://www.valuemd.com/american-university-caribbean-auc/109648-i-passed-5.html))

seems more like they're making it optional... I'm willing to bet >85 percent of the people will still take it. Another 10 percent will oversleep and the rest will complain that they didn't have enough time to study.

[IMG2006]

When I was in **, we did not take shelf exams, but I think it is a very good idea with some provisions.

1) You need to take them and have the REAL NBME score reported to you. The school should then curve accordingly for a grade. The problem here is that there are so many lazy b@$t@rad$ in our school that a large percentage will "ride the curve" to a passing grade. Thus, they don't prep at all and will inflate your score, which you really need to be spot on. I've said it once, and I'll say it again... the problem with foreign schools is overwhelmingly with the students and not with the schools. You can't turn chicken crap into chicken salad.

2) These tests are the best measure you will have to gauge your ability to take the questions on the USMLE for that subject. I have seen people with high GPAs fail the USMLE time and time again from Carib schools. Meanwhile, someone from the middle of the pack who has spent the whole time getting ready for the real deal passes with a great score. I feel that if more of us... well... now, you... placed an emphasis on self study then the abismal passing rates would climb dramatically. Make sure that you suppliment each class with a board directed text. Take each class for the material on the shelf and USMLE, not for your course grades. I can't tell you how little PDs care about basic science grades... they almost make fun of them. The material on the boards, however, is derived from real-life, must know material... they take these scores very seriously.

3) In clinicals, we took the shelf after every core. We never got the NBME score, and the grades were obviously curved... very curved.... warped...bent, skewed in fact. I remember taking one exam months after my core and then after a few electives. I had no time to study for it. I walked out concerned about whether or not I had passed. When I got my score... let's just say it was impossibly high... as in the A+++ range. That didn't tell me a thing regarding my readiness for step 2.

The harder our schools are the better. If something makes you work like you've never worked in your life, then you had best just get in line. It's the only way we'll ever have any credibility. We can be "just good enough" you need to make sure that you ALWAYS know the answer... that you are the best student your attending has ever had. You need to show that you have made a change in your life that makes you fit to be a doctor and not some beach bum reject... If you hate the sterotype, then shred it.

[Laurie S]

seems more like they're making it optional... I'm willing to bet >85 percent of the people will still take it. Another 10 percent will oversleep and the rest will complain that they didn't have enough time to study.

Micro,

You are wise beyond your years. :)

[IMG2006]

... and because someone is probably pissed that I said what I just said, let me give you an example of why I said it.

A few years ago, an really nasty article was written about a Spartan grad who intubated a woman's esophagus resulting in brain injury from hypoxia. The article said nothing about the fact that this was a common intern mistake, or that his attending should have been looking over his shoulder... the article didn't even mention that this happens all the time, and that is why interns have to be checked off after having done numerous intubation... the article went on and on about how carib schools turn out dangerous doctors who mustn't be inflicted on the public anymore.

You aren't going to be afforded the opportunity to make the common mistakes... you're going to have to be number one. The minute you slip up, no matter how easy it was, or how many did it before you, someone will think it is because you got nothing from school but a tan and an MD.
Sooo.... if people in your class are having trouble with the shelf... start study groups. Sleep less and read more... get your game face on.

[AzDaku]

When I was in **, we did not take shelf exams, but I think it is a very good idea with some provisions.

1) You need to take them and have the REAL NBME score reported to you. The school should then curve accordingly for a grade. The problem here is that there are so many lazy b@$t@rad$ in our school that a large percentage will "ride the curve" to a passing grade. Thus, they don't prep at all and will inflate your score, which you really need to be spot on. I've said it once, and I'll say it again... the problem with foreign schools is overwhelmingly with the students and not with the schools. You can't turn chicken crap into chicken salad.

2) These tests are the best measure you will have to gauge your ability to take the questions on the USMLE for that subject. I have seen people with high GPAs fail the USMLE time and time again from Carib schools. Meanwhile, someone from the middle of the pack who has spent the whole time getting ready for the real deal passes with a great score. I feel that if more of us... well... now, you... placed an emphasis on self study then the abismal passing rates would climb dramatically. Make sure that you suppliment each class with a board directed text. Take each class for the material on the shelf and USMLE, not for your course grades. I can't tell you how little PDs care about basic science grades... they almost make fun of them. The material on the boards, however, is derived from real-life, must know material... they take these scores very seriously.

3) In clinicals, we took the shelf after every core. We never got the NBME score, and the grades were obviously curved... very curved.... warped...bent, skewed in fact. I remember taking one exam months after my core and then after a few electives. I had no time to study for it. I walked out concerned about whether or not I had passed. When I got my score... let's just say it was impossibly high... as in the A+++ range. That didn't tell me a thing regarding my readiness for step 2.

The harder our schools are the better. If something makes you work like you've never worked in your life, then you had best just get in line. It's the only way we'll ever have any credibility. We can be "just good enough" you need to make sure that you ALWAYS know the answer... that you are the best student your attending has ever had. You need to show that you have made a change in your life that makes you fit to be a doctor and not some beach bum reject... If you hate the sterotype, then shred it.

RIGHT ON IMG2006. It discouraging on the amount of discrepancies they are between schools and students in the schools. I just wish at times the deans of the schools would read this forum and change the ciriculum that way in order to benefit the students and not their bank accounts.

[Laurie S]

Who's pissed?

[Jeep23Guy]

Who's pissed?

I just did and I feel a lot better, thanks.

[IMG2006]

RIGHT ON IMG2006. It discouraging on the amount of discrepancies they are between schools and students in the schools. I just wish at times the deans of the schools would read this forum and change the ciriculum that way in order to benefit the students and not their bank accounts.

I hear you on that brother... The model I think would do all of us the most good is to admit basically anyone with a dream and a clean reccord. They might just shock you... After that... the slaughter needs to begin, and the students who get cut shouldn't be able to retake classes 3 or 4 times. I think anyone could fail a class... MAYBE two... after that, the schools need to be thinking about whether or not the ******** alarm is functional. Typically though, schools will string this poor guy along for another 20-30 grand until he fails the boards or worse... graduates.... What then? Now, you've got some schmuck with ruined credit, no future and your name all over him. Not only that, the cut needs to happen during basic sciences and not in clinicals... that's why I am a huge fan of step 1 being required to go into clinicals. It's better for everyone. If you're not going to pass, then better to learn earlier than later. If you did pass, then it's better not to have some idiot with your school's name on his lab coat putzing up the rotation...


why am I speaking yiddish?

Not only that, this way the schools could pack out the ** classes, make their money, and still have a good reputation in the clinical world. I think we've actually entered a new realm in economics here which are causing unique problems that prevent this though.
There are now so many schools out there, that they are competing for tuition dollars instead of reputations. BAD BAD BAD.... If you got a pamphlet from every carib school, you'd go blind these days. So prospects often don't know where to look. The harder schools generate flunkies who ramble about politics and unfair grading... which leads to a ballooning of apps to another school. If you could somehow take the top 1/3rd of all carib schools and roll them into 2-3 good schools, man.... you'd have a stellar program. SGU seems to have done this by steadily raising admissions... which gave birth to the next 20 schools full of people who didn't get in there or know to apply there. I think SMU really has a shot to be THE school in the carib, but the students are going to have to play along and toe the line for it to happen.

[JTP73]

IMG2006: Your points are well taken. I agree that it would be more humane to slaughter in the beginning rather than in the middle or beyond. To damage one's credit in the process of a mid-season slaughter and then to brand your initials all over their *** is reprehensible. Thanks for the advice -- it is well taken.

[camden]

ITA , but more of a 'weedout' school doesn't translate in better students or USMLE preparation.

JUst when you thought having the shelf as part of your grade- now 20 PERCENT- was a bad idea, take for example, the NEW policy of grading pt-doctor for next semester.

I know what people are going to say "wow- great", but how is it that someone reading off the powerpoints, changing the schedule and structure of the class around a few times, and then giving us some weird non-USMLE style test we have to study for helping us be better doctors/students?

Its trying to cram us with over 30 credits- i smell a 'weedout' tactic- if we were to really LEARN something, we should emulate SGU and have students get experience in local hospitals and clinics in the lower semesters.

This new policy is better for the pocketbooks again and not for the students - It has been said before , but the new 'investors/ownership' of the school seems to want to emulate Ross and not SGU...

[IMG2006]

Where I was going with the shelf is that if you can't pass those... you're not going to pass the boards... 20% or not.....

With the PD thing... I'm on board with you 100%. They either need to make this a softball last semester for you guys who have made it through the culling, so that you can study for step 1, or space this out over a wide area where it is a building process that isn't very stressful.


I had this talk when I was down there with a then professor, now dean, who was very critical of that program. He called it medical drama lessons and made an excellent point. No one will care how great your exam skills are if you're never going to get a license because you fail the board. I remeber being tested on what side of the bed you should stand when taking a history... I can remember thinking, "has this guy lost his freaking mind? Aren't I here to learn something useful..." If that doesn't sum up the pee-pee poor focus of the whole program, then I don't know what does. It's the right side, by the way... I got it wrong. Now, I stand on the left just to tap-dance on the grave of my ** career.


I also agree with you 100% about trying to emulate SGU... you know why they're so good? Because they emulate the schools in the US and Canada... given the fact that these are the best schools the world has ever known, I question the wisdom in trying to do something new before you've perfected or even aproximated the standard. However, I think the first step in getting to that point is going to have to include a Ross phase so that we can generate enough good clinical people to get a good rep, so that we'll have more residents in better spots, and later a better image that will raise the admissions standards to near or equal those of many US schools-The SGU phase.

[LDMD2B]

I start SMU in Jan 2005, in the past year SMU changed a lot, some for the good and some for the bad. The grading system is the worse I have ever seen between Maine and Cayman. How can you have two campus and two diffrent grading. At the end of the day the transcripts do not say Maine it say SMU cayman. Yet in cayman you get extra credit, you have 3 exams NOT 6. Califonia was right on the money when they talked about Maine and Cayman, how hard is it to have the same system in place. You would think a year and a half after the Ca report, SMU would of fixed the problems. Instead of being more a like, we are more apart. Shelf worth 10% instead of 20%, 6 class exams vs 3, and it keeps going and going:bored:

[onelifetolive]

I think it is very very important to have a representing student body like a Student Council to bridge communication between the administration and the students

[Nimmuk]

[quote=IMG2006]Where I was going with the shelf is that if you can't pass those... you're not going to pass the boards... 20% or not..... quote]

Well said. There are many people talking about we only hav 2-6 days to study for the shelfs where in reality we had the entire semester....

[Junito]

I start SMU in Jan 2005, in the past year SMU changed a lot, some for the good and some for the bad. The grading system is the worse I have ever seen between Maine and Cayman. How can you have two campus and two diffrent grading. At the end of the day the transcripts do not say Maine it say SMU cayman. Yet in cayman you get extra credit, you have 3 exams NOT 6. Califonia was right on the money when they talked about Maine and Cayman, how hard is it to have the same system in place. You would think a year and a half after the Ca report, SMU would of fixed the problems. Instead of being more a like, we are more apart. Shelf worth 10% instead of 20%, 6 class exams vs 3, and it keeps going and going:bored:

I have to agree with you on that one. During our 4th semester (last Summer) my class made a big stink about this. Was anything done during the semester? Nothing...There was a rumor started around the Maine campus that there would be no extra credit for the Comp for both campuses (which proved to be false). Incidentally the shelf was worth extra points for the next semester.

[IMG2006]

At the end of the day the transcripts do not say Maine it say SMU cayman. Califonia was right on the money when they talked about Maine and Cayman, how hard is it to have the same system in place. :bored:

The transcripts DO reflect if you went to Maine. The course numbers are MD101 for cayman and MMD101 for Maine... No PD will know what that means though.

Overall... the Maine program has hurt SMU from the CA standpoint. Which is not to say that you won't get a great education there. That caused some confusion and was a reason CA used to deny us. They also talked about students in ME saying that the education was far better. Idiots.....The list of states requiring CA is growing. TN added that this past year.

[SillyDoc]

The benefit you get from the Maine campus is not some letters on a transcript, it is the satisfaction and challenge of difficult exams that can be overcome with diligent study.

[LDMD2B]

yeah well at the end of the day the school should have ONE grading system for all, should not matter if you go to cayman or maine, its the same school. Yes Ca is growing, that was the meeting they had this past year with nbme. Sillydoc there is no satisfaction in Maine, medschool is ment to be hard not be crammed. I don't feel as if I got a great education in Maine. Some teachers were great some the worse I ever had. In cayman the teachers are not great but you do have the time to properly study. Here in Maine its the other way around you can say. Having a dozen exams is not a good thing when the school makes you cram. No person can properly master a 150 pages a week properly. Yes I can read and spit it out the next day, but ask anyone 2 weeks later and they dont remember anything. Yes you can say US schools do it, but no US school has a trimaster, our semester is 3 months theirs is 4.5 months, thats a huge diffrence when you study the same pharm or path they do. It is all about money, thats why we have trimasters.