Sunday, October 24, 2010

ABIM Board Recertification Exam: Threat Level Green

Two Thursdays ago, I took the American Board of Internal Medicine (ABIM) recertification examination in gastroenterology (GI).   Whistleblower readers have already digested some of my musings on this event.  The good news is that there was no penalty for incorrect answers.  The bad news is that I submitted many incorrect answers.  Every one of these standardized tests that we all take becomes a mind game, where the examinee (us) tries to penetrate the psyche of the test makers (them).  We’ve all been there.   We torture ourselves between what we think is the right answer, and what we think that the questioners think is the right answer.   Sometimes, I thought that the ‘correct’ answer on the list is out of date, which confused me.  Or, what I felt was the truly correct answer, wasn’t included in the answer choices.  For example, I am a very conservative practitioner, who often advises observation, rather than tossing patients into the diagnostic arena.  In a few of the exam’s clinical management questions, I would have chosen ‘wait and see’, but on the test I was forced to scope, scan or operate. 
I didn’t invest even one nanosecond to prepare, which I think was the proper strategy.  I’ll find out in a few months. This is clearly a test that one cannot study for.
Many of the questions were reasonable.  Physicians’ define reasonable as an easy question that we know the answer to.  Unfair questions are all the rest.  Many questions do not relate to typical GI community practice, or they ask the examinee to resurrect an arcane medical fact, that was last known to us during the heady memorization days of medical school. 
Years ago, I submitted sample GI questions to the ABIM, at their request, to be considered in the internal medicine board certification exam.  This wasn’t an easy task, and I recall it took considerable effort to create just a few questions.  Here are some of the hurdles.  Is the question’s content reasonable for an internist?  Should the question test medical knowledge or judgment?  Is the clinical narrative realistic?  Is sufficient information included to lead the examinee to the correct answer?  Should I insert an irrelevant piece of data to distract the test taker, as occurs in the real world?  Has the question been sanitized of ambiguity so that examinee doesn’t develop bulging neck veins and a paroxysmal surge in blood pressure?
Is an exam question reasonable if you could find the answer by consulting a reference, as we do in everyday practice?  Of course, we do not need conventional text books or journals  any longer, now that we have 24/7 access to the world’s most authoritative medical resource – Wikipedia.  I think that internet use should be permitted during the exam to replicate actual medical practice.   Do physicians have to know the right answer immediately or simply know how to find it?
I think that creating exam questions is a useful exercise for medical students and residents, and even for practicing physicians.   Try it yourself.  Devise a few board questions and show them to colleagues. Be prepared for some critical reactions.  It’s not as easy as it looks.
However, many medical standardized exam questions are downright silly, and are of marginal clinical relevance.  On every one of these tests that I’ve ever taken, there are always a slew of patient vignettes that are literally foreign to American physicians.  Patients on these exams do a lot more foreign travel than my real patients do.
Here’s a sample question, which is solely the product of my imagination.  I will never divulge, even under hypnosis or interrogation, any of the copyrighted materials on the GI board exam.  I do not want the ABIM enforcers to snatch me in the middle of the night and escort me on a perp walk, cuffed and shackled, while reporters blind me as their flash cameras detonate.  

A patient returns home from Southeast Asia and develops a series of wacky symptoms. Chartreuse colored fluids are being ejected from various bodily orifices. He has a rash in the right nostril (see Figure A) and noticed some tingling in the ring fingers of both hands. For the last week, he has been speaking in rhyme and has developed a craving for guacamole. There is stridor present, which is quite ‘inspirational’. (Click Audio) Labs are notable for the abnormal trace element levels listed below.

Which of the following diseases that you’ve never heard of is the likely diagnosis?
The most impressive aspect of the experience was the security.  When I arrived, I had to show 2 forms of identification.  The testing administrator didn’t simply glance at them, he studied them.  I had my palm vein pattern recorded and my photograph taken.  I was advised that I would be monitored by video and audio devices.  I wondered if an examinee read questions softly aloud to himself, and was caught on audio, that a trap door might snap open underneath him.  Poof, another physician prematurely retiring! Every time I entered the sacred exam room, I had to empty my pockets and roll up my sleeves.   You could not enter or leave the room without a photo ID check and a palm vein scan.  When I returned for the final session after lunch, the proctor nabbed me trying to sneak my wristwatch into the exam room, which I forgot to remove.   “It was an accident,” I protested.  Was my medical career over?  Fortunately, this major ethical lapse did not constitute ‘irregular behavior’.  I’m sure after this incident, that I never left her site. 
Of course, I agreed to the Pledge of Honesty, and was given a list of admonitions afterwards that promised to send me to the gulag if I gave any test info away.
I am not here to criticize these security personnel, but to praise them.  I felt a lot more secure there than I do at the airport.   In fact, the Homeland Security Advisory System assigned a Low threat level in my testing center.   There were no free peanuts, but there was peace of mind.

9 comments:

A. Bailey said...

What? You weren't allowed to pull UpToDate on your IPhone?

Did any of the answers involve leeches, arsenicals, or long wax candles?

A. Bailey said...

And slnce I know as a fact that some of your more thoughtful contributors are out on the cyclocross course, I will make a more thoughtful comment.

Right now I'm working my way through the Gastrointestinal Endoscopy Self-Assessment Program. Before I answer any question, I try to put it into one of several categories:

1. Straightforward questions. Fortunately these are the majority of the GESAP.

2. Straightforward questions about pretty unusual stuff that they really want you to learn. I'm guessing you had to answer at least 40 questions about cystic neoplasms of the pancreas. If I were taking the test I would have memorized that stuff if it took all night using flash cards to do it. As it is now, after I read UpToDate I give the local pancreatologist a call and see what he thinks.

3. Agenda-driven questions. Last time I took the GESAP, if I didn't know the answer to something, I would chose Endoscopic Ultrasound and it would almost always be right. Now I think they think they overplayed their hand on that one, so if one of the answers is Endoscopic ULtrasound, it is almost always the wrong one.

4. "Got me there" sort of questions, like how many Giardia may dance on the head of a pinworm. I don't mind missing those questions as long as there aren't too many of them.

Anonymous said...

I did my fmaily medicine boards in 09 to recertify. At least 80% of the questions were about thinks I rarely, if ever see in private clinical practice. The cases were 1-2 sentences and very short. The good news is I got a day off from the office. The bad news is that being board certified does not equate to higher fees from the health insurance companies, who could care less about my recertification.

Michael Kirsch, M.D. said...

@A. Bailey, didn't know you were a fellow scoper. Have you been at it long?

doc said...

I wonder if doing the 100 hours CME/projects for recert is tougher than the exam itself.

Michael Kirsch, M.D. said...

How many hours of educational value were contained in the 100 CME hours? Not enough.

Dr. Val said...

I just took my PM&R board exam. I had the same experience - some really weird and irrelevant questions. Some were about Ob/Gyn and infectious disease. Um... Not really my field. Though I'd find the right answer on UpToDate if I needed to... ;-)

Anonymous said...

And still, as of 12/20/2010, we don't know whether we passed. What's with that? It's multiple choice, computerized test.

Anonymous said...

Thanks for all the insight

Add this