Canon Medical Systems USA

Moving Towards Modern Medical Education and Training

Pt. 3B: Competency or Passing the Boards? Every patient wants an expert.

Anthony Mancuso M.D. - Professor and Chairman of the Department of Radiology - University of Florida - College of Medicine | November 22, 2017

Anthony Mancuso M.D. has a mission to modernize post graduate medical education.  He has  spent the last eight years developing a competency based curriculum and evaluation, based on modern learning theory, with his team at the University of Florida.  They have leveraged extraordinary Learning Technologies to deliver this platform anywhere in the world. In his essay series, Moving Towards Modern Medical Education and Training, Dr. Mancuso will examine in detail: the specific pathway to this adherence to modern learning, educational theory, and the outcome of the application of those principles in this sphere of medical education.

In this installment in the series we will review of the current state of post graduate medical education and training, and the related methods of competency evaluation. The focus will be on three key components of our education and training model and, in particular, how they might affect our unwritten social contract 1 to prove to our patients that our approach to education creates a safe and effective system of professional behavior and competency on their behalf. Those three elements include:
 

1. Curriculum development
2. The testing rubrics that are proxies for establishing competency
3. What we define today as “competency”   

With regard to the last of these three topics, we must understand the complex relationship between mastery, competency and what parameters constitute an acceptable rate of error in those individuals considered to have become competent/expert in the current system. We must also admit, freely, that it is unreasonable to believe that any single individual can perform at an expert level in all aspects of diagnostic imaging.
 
The Accreditation Council for Graduate Medical Education (ACGME) in their “Envelope of Expectations” indicates a trainee in Diagnostic Radiology should reach a level of “Competent” late in the 2nd year to beginning of the 3rd year (Figure 3B-1).
 


Figure 3B-1. Expectation of level of competency achieved by diagnostic radiology residents at each phase of training and practice.
 
Beyond that ACGME outgrowth Milestone – related, specified goal, there is a progressive expectation by the end of the 4th year that the graduating resident become “Proficient” (Figure 3B-1).1  It might be difficult for a patient to understand the difference between “Competent” and “Proficient” since neither terms are objectively defined or established by the current curriculum requirements or sufficient evaluation rubrics, either during or following, the completion of training; so where is the real measure of these goals? 
 
Beyond those “Expectations” plateaus, there is an aspiration expressed, in this milestone surrogate ACGME Envelope of Expectations, that the practicing radiologist attain an “Expert” level of competency sometime shortly after graduation; that expert status to be assured by a process of Maintenance Of Certification (MOC) (Figure 3B-1).What is the guarantee to the patient that the curriculum and evaluation methodologies in the MOC process somehow creates or assures “Expert” professional behavior. Truthfully, there is no such guarantee. Establishing expert level behavior/competency is a process that requires intense self-discipline and study, referred to by some as “Deliberate Practice2-4 This “Deliberate Practice” discipline must initially persist for at least several years beyond the training, preferably in an environment that provides feedback and accountability for progress in order to become a true expert. So this aspiration of attaining expert level professional results really becomes a matter of personal responsibility rather than one imposed and measured by an external authority.

Realistically, our patients want to believe that we are truly expert as we engage in medical decision-making on their behalf. In order to fulfill this expectation, we must have a never-ending curriculum and method of self-evaluation and self-discovery that inspires and requires “Deliberate Practice” that will make true expertise possible with sufficient effort. “Deliberate Practice” is what makes the difference between someone who plays an instrument occasionally and produces, perhaps, pleasant music and someone who practices enough to be part of an orchestra, or preferably, first chair and/or a concert soloist.2-4  For those who cannot, or are satisfied not to become expert, we owe our patients an accessible expert reasoning and problem solving support mechanism as Dr. Weed suggested in the 1960’s. 5-9 For those who aspire to routine expert level performance we owe an educational system that can help them to develop expert competency and confirm it has been attained and maintained and, in fact, grows with time.
 
The terms adopted by the ACGME to describe the “levels” of professional development in their “Envelope of Expectations” are not defined (Figure 3B-1). As a result, the “Envelope of Expectations” is open to a good deal of subjective interpretation of what constitutes reasonable progression of professional growth during training. The related ACGME Milestones do not constitute a curriculum lesson plan; they are more of a framework, conceptual plan. There are no specific, progressive, evaluation rubrics to monitor the mastery of specific competencies during training. The current system enforces and certifies progression in learning based rather simply on subjective evaluations that mark each year of training. The only specified, objective evaluation rubric of competency within the “Envelope of Expectations” is the ABR core exam. However, the core exam does not measure “real life” professional radiology skill set competency. Therefore, the “Competencies”, “Milestones” and “Envelope of Expectations” are essentially progressively complex ACGME mission statements. As such these policy statements do not specifically inform the teaching methodology/theory that is necessary to choose and exploit effective adult learning tools that are appropriate to each level of necessary professional growth in training in the ACGME “Envelope”.

Eventually an effective curriculum must set the stage for lifelong “Deliberate Practice” to make us all experts for our patients. Our patients believe we are experts. Expert level competence is certainly what they desire in those who engage in medical decision-making on their behalf. Expert professional behavior is what we should aspire to and achieve to deserve their trust in our professionalism. So, as we become more informed as educators, there is another layer of understanding and curriculum development that we owe our trainees and patients in pursuit of this necessary aspirational goal of eventual expert level competence. The central question in pursuit of a competency based curriculum becomes: What can we do to align, well known and well documented adult learning principles with the level of desired professional educational development to create an accountable system of true competency based training and evaluation? We must to take on this task. To do this we must seriously consider the state of the trainee as they move through the reasonable levels of training and experience represented in the ACGME “Envelope of Expectations”.
 
We can begin that exploration by endeavoring to understand what actually constitutes competency. Dreyfus and Dreyfus provide us with a nomenclature for, and definition of, the levels of competence that can lead us to expert level development.10   These levels match up pretty well with the “stages” in the ACGME Envelope of Expectation. The Dreyfus model defines these levels as situations. This situational awareness can then be matched with parameters with an appropriate application of professional experiential and other adult learning theory.11-15  The five Dreyfus levels are:
 

1. Novice: Rule-based behavior, strongly limited and inflexible
2. Experienced Beginner: Incorporates aspects of the situation
3. Practitioner: Acting consciously from long-term goals and plans
4. Knowledgeable practitioner: Sees the situation as a whole and acts from personal conviction
5. Expert: Has an intuitive understanding of the situation and zooms in on the central aspects

As we consider the following arguments and data to be presented in this essay it may become apparent that what is missing in our diagnostic radiology post graduate educational system is a proper structured beginning point at the Novice level. The specific question being: Do we teach the fundamentals of observation and data gathering from images in a manner that is reproducible and reliable? All else interpretive flows from that fundamental skill. Failing that the remainder of the educational experience will produce suboptimal results.

In the next installment we will review the historic error rates in positive studies for radiology trainees and the resulting “education opportunity gaps” they create.  We will also explore how the "education opportunity gaps" present a challenge to acheiving expert performance.

Coming next:
Part 3C – The challenge to achieving expert performance: reducing the historic 30% error rate in positive studies.
 


Anthony A. Mancuso, MD

Dr. Mancuso graduated from the University of Miami School of Medicine in 1973 and completed a Residency and fellowship training in Diagnostic Radiology, including 2 years of subspecialty Neuroradiology training, at UCLA Health. He joined the faculty at UCLA Health where he was fortunate to have a founding member of organized neuroradiology in the United States, Dr. William Hanafee, as his friend and lifelong mentor. Dr. Mancuso owes much of the professional development in his career to Dr. Hanafee both with regard to his dedication to development of effective educational methodology and a devotion to discovery of practices that make a positive impact on patient care. He is a Past President of the American Society of Head and Neck Radiology and Senior Member of the American Society of Neuroradiology.

In 1983, Dr. Mancuso joined the faculty at the UF Health to direct the development of the MRI clinical and clinical research program. In 2000, he became Chairman of the Department of Radiology and remains in that position currently. He is also the President of the Florida Clinical Practice Association at UF Health.

Dr. Mancuso is an acknowledged international expert in the area of ENT radiology having been recognized for his achievements by Gold Medals from the American and European Societies of Head and Neck Radiology and a Presidential Citation from the American Society of Head and Neck chirurgery. He has over 170 refereed publications most in the area of Head and Neck Radiology, and has written several books, most recently, a comprehensive 3 volume text covering the clinical practice of head and neck imaging.

Dr. Mancuso's current research interests have been in developing novel methodologies for radiology education, exploiting foundational and modern learning techniques and merging those techniques with IT tools that make personalized, asynchronous delivery of an effective Radiology curriculum finally possible. His clinical research interest is now focused on the development of advanced brain MRI utilizing DTI and fMRI for the evaluation of traumatic brain injury and a wide range of neuropsychiatric disorders.
 

References:

1-Starr, Paul (1982). The Social Transformation of American Medicine. Basic Books. pp. 514 pages. ISBN 0-465-07935-0.
2-Ericsson, K. A. (2004). Deliberate practice and the acquisition and maintenance of expert performance in medicine and related domains. Academic Medicine, 79(10 Suppl), S70–S81.Physicians’ learning in practice 93
 3-Ericsson, K. A. (2006). The Influence of experience and deliberate practice on the development of superior expert performance. In K. A. Ericsson, N. Charness, P. J. Feltovich, & R. R. Hoffman (Eds.), The Cambridge handbook of expertise and expert performance (pp. 683–704). New York, NY: Cambridge University Press.
 4-Ericsson, K. A., Krampe, R. T., & Tesch-Ro¨mer, C. (1993). The role of deliberate practice in the acquisition of expert performance. Psychological Review, 100(3), 363–406.
 5-Weed, L. L. (1964-06-01). "MEDICAL RECORDS, PATIENT CARE, AND MEDICAL EDUCATION". Irish Journal of Medical Science. 462: 271–282.
 6-Weed, L. L. (1968-03-14). "Medical records that guide and teach". The New England Journal of Medicine. 278 (11): 593–600. doi:10.1056/NEJM196803142781105. ISSN 0028-4793. PMID 5637758.
7-Weed, L. L. (1968-03-21). "Medical records that guide and teach". The New England Journal of Medicine. 278 (12): 652–657 concl. doi:10.1056/NEJM196803212781204. ISSN 0028-4793. PMID 5637250.
8-Weed LL. Medical records, medical education, and patient care: the Problem-Oriented Medical Record as a basic tool. 1970. Cleveland (OH): Press of Case Western Reserve University.
9-Jacobs L. Interview with Lawrence Weed, MD—the father of the problem-oriented medical record looks ahead [editorial]. Perm J 2009 Summer; 13(3):84–9.
10-Stuart E. Dreyfus The Five-Stage Model of Adult Skill Acquisition
Bulletin of Science Technology & Society 2004 24: 177.  The online version of this article can be found at
http://bst.sagepub.com/content/24/3/177
 11-Knowles, M. (1984). The Adult Learner: A Neglected Species (3rd Ed.). Houston, TX: Gulf Publishing.
12-Kolb, D. A. (1984). Experiential learning: Experience as the source of learning and development (Vol. 1). Englewood Cliffs, NJ: Prentice-Hall.
13-Bloom, B., Englehart, M. Furst, E., Hill, W., & Krathwohl, D. (1956). Taxonomy of educational objectives: The classification of educational goals. Handbook I: Cognitive domain. New York, Toronto: Longmans, Green.
14-Krathwohl, D. R. Methods of Educational & Social Science Research: An Integrated Approach. 1st Ed. 1993, 2nd Ed. 1998, New York: Longman, also Long Grove, IL: Waveland Press; 3rd Ed 2009, Waveland Press
 15- Understanding by Design® book (Wiggins & McTighe, 1998