Summary

Objectives: To validate and evaluate the reliability and realism of incorporating simulation-based OSCE into the Israeli Board Examination in Anesthesia.

Methods: Validation was performed before the exam regarding Content Validity using the modified Delphi technique by members of the Task Force of the Israeli Board Examination Committee in Anesthesiology.

Results: The examination has been administered six times in the past 3 years to a total of 145 examinees. The pass rate ranged from 62% (trauma) to 91% (regional anesthesia). The mean inter-rater correlations for the total score (all items), for the Critical checklist items score, and for the Global (General) rating were 0.89, 0.86 and 0.76, respectively. The inter-correlations between the five OSCE stations scores were significant (P < 0.01) only between Trauma & Ventilation for the Total score (r = 0.32, n=63), and between Resuscitation & Regional and OR-crisis for the Global score (r = 0.42 and 0.27, n=64 and 104, respectively). The correlation between the OSCE examination score and the success rate at each of the eight different clinical domains of the oral board examination did not reach statistical significance. Most participants (70–90%) found the difficulty level of the examination stations reasonable to very easy. All major errors, which were identified in the initial two exam periods, disappeared later in the next two exam periods.

Conclusions: The exam has gradually progressed from being an optional part of the oral board examination to a prerequisite component of this test. Other anesthesiology programs or medical professions can adopt the model described here.