Feasibility Study of Self-Directed Mock Oral Examinations in the Assessment of General Surgery Resident Clinical Decision-Making

Self-Directed Mock Oral Examination

Authors

  • Terrance Peng University of California Los Angeles David Geffen School of Medicine, Department of Surgery, Los Angeles, CA, USA
  • Ivy A. Huang University of California Los Angeles David Geffen School of Medicine, Department of Surgery, Los Angeles, CA, USA
  • Marilee Fisher University of California Los Angeles David Geffen School of Medicine, Department of Surgery, Los Angeles, CA, USA
  • James X. Wu University of California Los Angeles David Geffen School of Medicine, Department of Surgery, Los Angeles, CA, USA
  • Justin P. Wagner University of California Los Angeles David Geffen School of Medicine, Department of Surgery, Los Angeles, CA, USA
  • Areti Tillou University of California Los Angeles David Geffen School of Medicine, Department of Surgery, Los Angeles, CA, USA
  • Formosa Chen University of California Los Angeles David Geffen School of Medicine, Department of Surgery, Los Angeles, CA, USA

Keywords:

General Surgery Residency, Education, Medical, Graduate, Educational Measurement, Board Examination Scores, Clinical Competence

Abstract

Background: This feasibility study describes the implementation of a novel self-directed mock oral examination (SD-MOE), which is administered in the absence of faculty examiners, and explores its potential as an alternative to the conventional mock oral examination (C-MOE) to prepare trainees for the American Board of Surgery Certifying Exam (ABSCE).

Methods: A multi-institutional MOE was administered, during which all examinees completed the C-MOE and were presented the option to additionally participate in the SD-MOE. Exam performance metrics, interrater reliability, and participant feedback were compared between formats.

Results: The C-MOE included 62 faculty examiners and 55 examinees, while the SD-MOE included 25 faculty examiners and 41 examinees. There was no difference in the frequency of technical issues between formats. Moderate interrater reliability was observed for the C-MOE (k =.45 [0.14-0.72]) and SD-MOE (k =.49 [-0.25-1.0]). Faculty examiner satisfaction was significantly lower with the SD-MOE compared to the C-MOE (65% vs. 93%, P < .01). PGY-level was significantly associated with mean case score (PGY-3: 2.45; PGY-4: 2.58; PGY-5: 2.70; P =.01), proportion of questions passed (PGY-3: 81.7%; PGY-4: 87.3%; PGY-5: 90.1%; P < .01), and pass rate (PGY-3: 58.3%; PGY-4: 82.4%; PGY-5: 95.8%; P = .04) in the C-MOE, but not the SD-MOE.

Conclusion: The SD-MOE was successfully administered with minimal logistical challenges during a multi-institutional MOE. Although the lack of direct examiner-examinee interaction is a notable departure from the ABSCE with impacts on performance metrics, the SD-MOE can be considered as a supplemental educational tool and C-MOE alternative for resource-limited institutions.

Author Biographies

Terrance Peng, University of California Los Angeles David Geffen School of Medicine, Department of Surgery, Los Angeles, CA, USA

Resident-Physician
Department of Surgery

Ivy A. Huang, University of California Los Angeles David Geffen School of Medicine, Department of Surgery, Los Angeles, CA, USA

Resident-Physician
Department of Surgery

Marilee Fisher, University of California Los Angeles David Geffen School of Medicine, Department of Surgery, Los Angeles, CA, USA

Medical Student
David Geffen School of Medicine

James X. Wu, University of California Los Angeles David Geffen School of Medicine, Department of Surgery, Los Angeles, CA, USA

Assistant Professor of Surgery
Department of Surgery

Justin P. Wagner, University of California Los Angeles David Geffen School of Medicine, Department of Surgery, Los Angeles, CA, USA

Assistant Professor of Surgery
Department of Surgery

Areti Tillou, University of California Los Angeles David Geffen School of Medicine, Department of Surgery, Los Angeles, CA, USA

Professor of Surgery
Vice Chair for Education
Department of Surgery

Formosa Chen, University of California Los Angeles David Geffen School of Medicine, Department of Surgery, Los Angeles, CA, USA

Associate Professor of Surgery
UCLA Health, Department of Surgery

References

Higgins RM, Deal RA, Rinewalt D, et al. The utility of mock oral examinations in preparation for the American Board of Surgery certifying examination. The American Journal of Surgery. 2016;211(2):416-420. doi:10.1016/j.amjsurg.2015.09.008

Fingeret AL, Arnell T, McNelis J, Statter M, Dresner L, Widmann W. Sequential Participation in a Multi-Institutional Mock Oral Examination Is Associated With Improved American Board of Surgery Certifying Examination First-Time Pass Rate. Journal of Surgical Education. 2016;73(6):e95-e103. doi:10.1016/j.jsurg.2016.06.016

Guzman E, Babakhani A, Maker VK. Improving Outcomes on the ABS Certifying Examination: Can Monthly Mock Orals Do It? Journal of Surgical Education. 2008;65(6):441-444. doi:10.1016/j.jsurg.2008.04.005

Nguyen J, Liu A, McKenney M, Elkbuli A. Predictive Factors of First Time Pass Rate on the American Board of Surgery Certification in General Surgery Exams: A Systematic Review. Journal of Surgical Education. Published online February 2021:S1931720421000209. doi:10.1016/j.jsurg.2021.01.020

Maker VK, Zahedi MM, Villines D, Maker AV. Can We Predict Which Residents Are Going to Pass/Fail the Oral Boards? Journal of Surgical Education. 2012;69(6):705-713. doi:10.1016/j.jsurg.2012.08.009

Meyerson SL, Lipnick S, Hollinger E. The Usage of Mock Oral Examinations for Program Improvement. Journal of Surgical Education. 2017;74(6):946-951. doi:10.1016/j.jsurg.2017.05.003

Kimbrough MK, Thrush CR, Smeds MR, Cobos RJ, Harris TJ, Bentley FR. National Landscape of General Surgery Mock Oral Examination Practices: Survey of Residency Program Directors. Journal of Surgical Education. 2018;75(6):e54-e60. doi:10.1016/j.jsurg.2018.07.012

Strickland C, Jensen A, McArthur T. Does the Oral “Mock Board” Examination Still Have a Role as a Training Tool? Academic Radiology. 2017;24(11):1463-1467. doi:10.1016/j.acra.2017.06.012

Huang IA, Lu Y, Wagner JP, et al. Multi-institutional virtual mock oral examinations for general surgery residents in the era of COVID-19. Am J Surg. 2021;221(2):429-430. doi:10.1016/j.amjsurg.2020.11.002

Shebrain S, Nava K, Munene G, Shattuck C, Collins J, Sawyer R. Virtual Surgery Oral Board Examinations in the Era of COVID-19 Pandemic. How I Do It! J Surg Educ. Published online September 16, 2020. doi:10.1016/j.jsurg.2020.09.012

Zemela MS, Malgor RD, Smith BK, Smeds MR. Feasibility and acceptability of virtual mock oral examinations for senior vascular surgery trainees and implications for the certifying exam. Annals of Vascular Surgery. Published online April 7, 2021. doi:10.1016/j.avsg.2021.03.005

Smeds MR, Thrush C, Kimbrough MK, Moursi MM. Use of Mock Oral Examinations in Vascular Surgery Training Programs: A Nationwide Survey. Journal of Surgical Research. 2018;232:94-98. doi:10.1016/j.jss.2018.06.030

Peters GW, Decker RH, Park HS, Yu JB, Evans SB. Resident attitudes and benefits of mock oral board examinations in radiation oncology. BMC Med Educ. 2020;20(1):203. doi:10.1186/s12909-020-02106-4

House JB, Dooley-Hash S, Kowalenko T, et al. Prospective Comparison of Live Evaluation and Video Review in the Evaluation of Operator Performance in a Pediatric Emergency Airway Simulation. J Grad Med Educ. 2012;4(3):312-316. doi:10.4300/JGME-D-11-00123.1

Patrawalla P, Eisen LA, Shiloh A, et al. Development and Validation of an Assessment Tool for Competency in Critical Care Ultrasound. J Grad Med Educ. 2015;7(4):567-573. doi:10.4300/JGME-D-14-00613.1

Scaffidi MA, Grover SC, Carnahan H, et al. A prospective comparison of live and video-based assessments of colonoscopy performance. Gastrointestinal Endoscopy. 2018;87(3):766-775. doi:10.1016/j.gie.2017.08.020

Jacobsohn E, Klock PA, Avidan M, Oral Examinations Group. Poor inter-rater reliability on mock anesthesia oral examinations. Can J Anesth. 2006;53(7):659-668. doi:10.1007/BF03021624

Burchard KW, Rowland-Morin PA, Coe NP, Garb JL. A surgery oral examination: interrater agreement and the influence of rater characteristics. Academic Medicine. 1995;70(11):1044-1046.

Waterfield J, West B. Inclusive Assessment in Higher Education: A Resource for Change. Published online 2006. https://www.plymouth.ac.uk/uploads/production/document/path/3/3026/Space_toolkit.pdf

Ramella D. Oral Exams: A Deeply Neglected Tool for Formative Assessment in Chemistry. In: Blaser M, Clark T, Lamont L, Stewart JJ, eds. ACS Symposium Series. Vol 1340. American Chemical Society; 2019:79-89. doi:10.1021/bk-2019-1340.ch006

Weston TJ. The Validity of Oral Accommodation in Testing. Published online 2002. doi:10.1037/e540302012-001

Lu Y, Miranda R, Quach C, et al. Standardized Multi-Institutional Mock Oral Examination: A Feasible and Valuable Educational Experience for General Surgery Residents. Journal of Surgical Education. 2020;77(6):1568-1576. doi:10.1016/j.jsurg.2020.05.015

Subhas G, Yoo S, Chang YJ, et al. Benefits of mock oral examinations in a multi-institutional consortium for board certification in general surgery training. Am Surg. 2009;75(9):817-821.

Theobold AS. Oral Exams: A More Meaningful Assessment of Students’ Understanding. Journal of Statistics and Data Science Education. 2021;29(2):156-159. doi:10.1080/26939169.2021.1914527

Huxham M, Campbell F, Westwood J. Oral versus written assessments: a test of student performance and attitudes. Assessment & Evaluation in Higher Education. 2012;37(1):125-136. doi:10.1080/02602938.2010.515012

Kozol R, Giles M, Voytovich A. The value of videotape in mock oral board examinations. Current Surgery. 2004;61(5):511-514. doi:10.1016/j.cursur.2004.06.004

Ruiz TL, Sellers B, Devarakonda A, Wehrle CJ, Arora TK. A Novel Mock Oral Curriculum for Senior Surgery Residents: Results of a Pilot Study. Journal of Surgical Research. 2022;277:92-99. doi:10.1016/j.jss.2022.03.027

Downloads

Published

2026-06-28

How to Cite

Peng, T., Huang, I. A., Fisher, M., Wu, J. X., Wagner, J. P., Tillou, A., & Chen, F. (2026). Feasibility Study of Self-Directed Mock Oral Examinations in the Assessment of General Surgery Resident Clinical Decision-Making: Self-Directed Mock Oral Examination. International Journal of Medical Students. Retrieved from https://ijms.info/IJMS/article/view/3978

Issue

Forthcoming

Section

Original Article

Categories

License

Copyright (c) 2026 Terrance Peng, Ivy A. Huang, Marilee Fisher, James X. Wu, Justin P. Wagner, Areti Tillou, Formosa Chen

Creative Commons License

This work is licensed under a Creative Commons Attribution 4.0 International License.

  1. The Author retains copyright in the Work, where the term “Work” shall include all digital objects that may result in subsequent electronic publication or distribution.
  2. Upon acceptance of the Work, the author shall grant to the Publisher the right of first publication of the Work.
  3. The Author shall grant to the Publisher and its agents the nonexclusive perpetual right and license to publish, archive, and make accessible the Work in whole or in part in all forms of media now or hereafter known under a Creative Commons Attribution 4.0 International License or its equivalent, which, for the avoidance of doubt, allows others to copy, distribute, and transmit the Work under the following conditions:
    1. Attribution—other users must attribute the Work in the manner specified by the author as indicated on the journal Web site; with the understanding that the above condition can be waived with permission from the Author and that where the Work or any of its elements is in the public domain under applicable law, that status is in no way affected by the license.
    2. The Author is able to enter into separate, additional contractual arrangements for the nonexclusive distribution of the journal's published version of the Work (e.g., post it to an institutional repository or publish it in a book), as long as there is provided in the document an acknowledgment of its initial publication in this journal.
    3. Authors are permitted and encouraged to post online a prepublication manuscript (but not the Publisher’s final formatted PDF version of the Work) in institutional repositories or on their Websites prior to and during the submission process, as it can lead to productive exchanges, as well as earlier and greater citation of published work. Any such posting made before acceptance and publication of the Work shall be updated upon publication to include a reference to the Publisher-assigned DOI (Digital Object Identifier) and a link to the online abstract for the final published Work in the Journal.
    4. Upon Publisher’s request, the Author agrees to furnish promptly to Publisher, at the Author’s own expense, written evidence of the permissions, licenses, and consents for use of third-party material included within the Work, except as determined by Publisher to be covered by the principles of Fair Use.
    5. The Author represents and warrants that:
      1. the Work is the Author’s original work;
      2. the Author has not transferred, and will not transfer, exclusive rights in the Work to any third party;
      3. the Work is not pending review or under consideration by another publisher;
      4. the Work has not previously been published;
      5. the Work contains no misrepresentation or infringement of the Work or property of other authors or third parties; and
      6. the Work contains no libel, invasion of privacy, or other unlawful matter.
    6. The Author agrees to indemnify and hold Publisher harmless from the Author’s breach of the representations and warranties contained in Paragraph 6 above, as well as any claim or proceeding relating to Publisher’s use and publication of any content contained in the Work, including third-party content.

Enforcement of copyright

The IJMS takes the protection of copyright very seriously.

If the IJMS discovers that you have used its copyright materials in contravention of the license above, the IJMS may bring legal proceedings against you seeking reparation and an injunction to stop you using those materials. You could also be ordered to pay legal costs.

If you become aware of any use of the IJMS' copyright materials that contravenes or may contravene the license above, please report this by email to contact@ijms.org

 

Infringing material

If you become aware of any material on the website that you believe infringes your or any other person's copyright, please report this by email to contact@ijms.org