LAURA SPINNEWIJN Shared decision-making revisited The nexus of learning, culture, and behavior change
LAURA SPINNEWIJN Shared decision-making revisited The nexus of learning, culture, and behavior change
Colofon Provided by thesis specialist Ridderprint, ridderprint.nl Layout en design: Hans Schaapherder, persoonlijkproefschrift.nl Illustratie cover: Laura Spinnewijn © 2024, Laura Spinnewijn, Amsterdam, The Netherlands All rights reserved. No parts of this thesis may be reproduced or transmitted in any form or by any means without prior written permission of the author.
Contents CHAPTER 1 General introduction 7 PART I Training residents in shared decision-making 19 CHAPTER 2 Assessing the educational quality of shared decision-making interventions for residents: a systematic review 21 CHAPTER 3 Structurally collecting patient feedback on trainee skills: a pilot study in Obstetrics and Gynecology 43 PART II Exploring determinants of clinician (dis-)engagement in shared decision-making 59 CHAPTER 4 Knowing what the patient wants: a hospital ethnography studying physician culture in shared decision-making in the Netherlands 61 CHAPTER 5 Is it fun or is it hard? Studying physician-related attributes of shared decision-making by ranking case vignettes 79 CHAPTER 6 Unravelling clinicians’ shared decision-making adoption: a framework analysis through the lens of Diffusion of Innovations theory 95 CHAPTER 7 General discussion 113 ADDENDUM English summary 134 Nederlandse samenvatting (Dutch summary) 138 Author contributions 142 Supplementary files 144 Dankwoord 163 Curriculum Vitae 165
CHAPTER 1 General introduction
8 Chapter 1 Throughout my professional career, I have navigated through diverse and contrasting environments, starting as a schoolteacher, immersing myself in the educational sciences, and then transitioning to becoming a doctor after embarking on a new path as a medical student. These distinct experiences have sparked an enduring ambition to integrate knowledge from these different backgrounds. As my journey progressed, I undertook an educational internship at the same faculty where I pursued my medical training. During this transformative phase, I encountered an inspiring thesis titled “Unravelling Learning by Doing” by Pim Teunissen, significantly impacting my view on medical education. This thesis illuminated the connection between practical medical experience during junior doctors’ specialization training and the shaping of their normative and behavioral patterns. [1] Additional research I encountered highlighted that regardless of well-trained skills acquired during medical training, many of these abilities eroded once medical students advanced in training or entered medical practice. [2, 3] These findings made me realize that, despite sincere training efforts for students, seemingly desired physician skills often fail to translate into actual medical practice. This phenomenon appears also to encompass shared decision-making, which constitutes the primary focus of this thesis. [4] Although it has been underscored consistently as a crucial virtue for virtuous doctors over the last three decades, and (future) doctors are increasingly taught its concepts throughout their medical training, [5] this emphasis has not translated into the intended level of success in daily care.
9 General introduction Introduction In 1997, Charles and others presented a pioneering and enduring definition of shared decisionmaking (SDM), which continues to encapsulate this concept’s essence to the present day. [4] In this definition, shared decision-making entails the active participation of both a physician and a patient, where the exchange of information is coupled with collaborative efforts to establish a consensus regarding the most favorable treatment approach. Ultimately, this process results in an agreement on the course of treatment to be enacted. [4] Contemporary research has explored the implementation of SDM, shedding light on a range of issues. Some studies defined the essential competencies that doctors require to apply SDM principles adeptly. [6] Others investigated barriers that impede successful implementation, [7] or reviewed tools, like decision aids, to facilitate this intricate process. [8] A wide range of training interventions has been developed and reviewed, from their inclusion in medical curricula to ongoing development initiatives. [9] Moreover, research underscores the potential benefits of SDM, including enhanced treatment adherence, increased satisfaction among both patients and healthcare professionals, and potentially improved disease-related results, [10] though empirical support for the latter remains limited. [11] However, despite these dedicated efforts, the full realization of SDM’s potential within standard patient care has yet to be attained. For example, practitioners frequently display inconsistencies or insufficiencies in using decision aids, [12] and significant treatment determinations, such as chemotherapy choices, still lack the requisite level of collaborative discussion and preference formation. [13] Furthermore, certain physicians even resist the necessary changes to integrate SDM into their practice effectively. [14] Elwyn and others aptly point out that advocating for SDM as the morally right approach does not necessarily guarantee its consistent application in practice. [11] They express the need for more extensive research into SDM, particularly in dimensions that have received inadequate attention, such as its impact on healthcare professionals. Furthermore, they highlight the imperative of cultivating a new cultural norm within medical practice, centered on ‘deliberation and collaboration’, and the transformation from former paternalistic approaches to a strategy of ‘coaching patients’ in the decision-making process. [11] In this thesis, I will explore the underlying causes for SDM’s unrealized potential within medical care settings, specifically focusing on its application within gynecology. Drawing insights from the social sciences, I aim to uncover the reasons behind the gap between SDM’s ideals and its actual implementation in this field. While existing research has identified substantial barriers impeding the integration of SDM, it has fallen short in thoroughly examining the underlying mechanisms responsible for inadequate application. To draw a parallel with medical terminology, prior studies have explored the symptoms and diagnosis but have not fully elucidated the pathophysiology of SDM’s unsuccessful integration in practice. Gynecology is an apt focal 1
10 Chapter 1 point due to its intrinsic focus on addressing personal and sensitive health concerns, which strongly align with the principles of patient autonomy and involvement advocated by SDM. This dissertation primarily targets two main groups: medical residents, doctors in specialized training to become medical specialists, and established medical specialists. The significance of studying residents lies in their formative stage of professional identity development, during which they implicitly or explicitly assimilate the principles shaping their utilization of SDM. In contrast, medical specialists are a crucial focus group as well, as they represent both the current state of practice and the potential areas for improvement regarding SDM. It is worth noting that this thesis acknowledges the essential role patients play in SDM, even though it does not explicitly focus on their role. As explained later in this introduction, the dissertation is structured into two distinct sections. The first section delves into SDM training initiatives for residents, while the second section investigates underlying factors contributing to clinicians’ ongoing challenges in fully embracing SDM to the desired level. I will provide a brief background on medical workplace learning, culture, and behavior change to provide context for this division and shed light on the rationale and significance behind these two focal points. This background section will lay the foundation for articulating my research aims and questions. Background Information Theories on learning within the medical workplace Their educational environment significantly shifts as medical students transition from undergraduate to clinical training. They evolve from being recipients of classroom instruction to active learners immersed in the clinical workplace. Numerous educational experts have expounded on the distinctions between classroom-based and workplace learning. In the following section, I will provide succinct insights into the perspectives of notable experts and their relevant educational theories. These insights will provide a theoretical foundation for educational principles in training residents, enhancing the understanding of how to design and evaluate training initiatives tailored to their distinctive training needs. In his work, John Dewey, a prominent figure in educational philosophy, emphasized the pivotal role of workplace experience in formal training. He advocated for the integration of education into real-life situations, stressing that education should empower students to apply their knowledge meaningfully, fostering a deep and comprehensive understanding of the world. [15] In his theory of adult education and learning, Malcolm Knowles similarly stressed the importance of practical experiences. [16] Moreover, Knowles drew a clear distinction between children’s and adult learning, underlining a shift from a teacher-centered approach to a learner-centered one. While children’s learning often revolves around a fixed curriculum and relies on external motivators to fulfill immediate needs, adult learning is characterized by experiential, active,
11 General introduction and self-directed processes. [16] Hence, treating adult learning in the same manner as children’s learning is inappropriate, as the two follow significantly different paradigms. David Kolb incorporated the concept of experiential learning in his theory of learning, presenting a dynamic learning cycle characterized by four interconnected stages: (1) concrete experience, (2) reflective observation, (3) abstract conceptualization, and (4) active experimentation. [17] This cycle forms a continuous and iterative process, facilitating a comprehensive understanding of the learning experience. In Kolb’s framework, guidance plays a pivotal role in facilitating the learning process and enhancing the effectiveness of experiential learning. This guidance can come from various sources, including teachers, mentors, or peers, enabling learners to maximize their experiences, navigate the learning cycle stages, and promote deeper reflection, conceptualization, and thoughtful experimentation, thereby enhancing their understanding and skills. [17] Kolb’s strong emphasis on experiential and reflective learning continues to hold significance in contemporary medical education. [18-21] Research has shown that present-day medical education curricula have indeed integrated experiential learning and reflection as essential components. [22] Nevertheless, questions arise regarding whether these educational principles are effectively applied in SDM training, considering that a gap persists in the transfer of SDM skills following training. [5] Moreover, it is crucial to recognize that even with high-quality SDM training initiatives, achieving the desired learning outcomes can be challenging. The decline in professional skills that residents experience when they transition into practice, as described earlier in this chapter, [2, 3] might similarly apply to SDM training initiatives. These unintended or even undesirable training results have been acknowledged before and are often called ‘informal learning’. Informal learning is characterized by the spontaneous and self-directed acquisition of knowledge and skills through day-to-day experiences and interaction. [23] It closely relates to the ‘hidden curriculum’ concept, which encompasses unspoken and implicit lessons, values, and behaviors unintentionally transmitted alongside formal education. [24, 25] Therefore, when considering training initiatives for residents, it is essential to consider both formal learning within structured programs and informal learning within residents’ workplace and, thus, social environment. Consequently, it is crucial to delve into the dynamics of this social environment, focusing on its intricate connection with culture before further exploring behavior change. The social environment, culture, and behavior change Recognizing the pivotal role of the social environment in workplace learning, Etienne Wenger and Jean Lave introduced the concept of ‘communities of practice’. They defined these as social environments comprised of individuals who share common interests and knowledge domains and engage in activities relevant to their field. [26] Within these communities, learners actively 1
12 Chapter 1 participate in authentic tasks, collaborate with others, and acquire both explicit knowledge and tacit understanding intrinsic to the practices of that community. [26] There is a reciprocal connection between communities of practice and culture. While various definitions of culture exist, they generally encompass shared beliefs, values, norms, symbols, practices, and knowledge that shape how individuals within a society perceive and interact with their environment. [14, 27] Culture significantly shapes how individuals interact within their social context, while the social environment, represented by communities of practice, plays a vital role in culture’s formation, transmission, and evolution. [28, 29] This dynamic interplay underscores the profound influence of culture and the social environment as they continually shape and inform each other, emphasizing the bidirectional nature of their influence as they coexist and evolve. Considering culture becomes crucial when contemplating practice change. In this context, the field of medical anthropology stands out as the social science explicitly dedicated to examining the interplay of culture within the field of medicine. Several anthropologists have highlighted the relationship between culture and medical practices, emphasizing how cultural influences shape medical approaches. [30-33] This adds depth to our understanding of the hidden curriculum and informal learning, shedding light on the impact of culture on the practice of medicine and aiding in comprehending current attitudes and behaviors related to SDM. However, it is important to note that understanding these influences does not automatically lead to behavior change. This scenario can be likened to receiving a precise medical diagnosis but still facing uncertainty about the best treatment. Recognizing the substantial influence of the hidden curriculum, it is evident that attaining the desired level of proficiency in SDM practice cannot be solely achieved through the education of residents. The cultural elements that shape junior doctors’ and more experienced clinicians’ social environments must also be addressed, as they significantly contribute to whether the intended learning outcomes are met. Subsequently, it becomes imperative to give due consideration to establishing behavior change. Regrettably, adopting behavior change theory as a guiding framework for changing medical practice tends to be marginalized, often overshadowed by a predominant emphasis on quality improvement models and tools. [34] Integrating these theories in this thesis can significantly enhance our understanding and enable the development of more potent strategies for effecting transformative change within the medical field. Aims of this thesis The persistent gap between teaching desirable skills and their practical application remains a concern. Motivated by these concerns, this thesis explores the intricacies surrounding this issue, with a specific emphasis on the skill of SDM among residents and medical specialists.
13 General introduction The research team involved, including myself, is dedicated to unraveling the underlying factors contributing to these disparities and identifying pathways to bridge the gap between medical education and SDM’s real-world application. The following overarching research questions are addressed: 1. What are effective approaches to teaching shared decision-making to residents? 2. What factors determine physicians’ engagement (or disengagement) in shared decisionmaking? a. How does physician culture impact shared decision-making use? b. How do individual beliefs, cognitive processes and contextual factors shape the decision to adopt or reject shared decision-making? To address these questions, this thesis adopts a structured format with two distinct segments, each dedicated to addressing one of the two themes outlined subsequently. In Part I, an evaluation of SDM training initiatives tailored for residents is undertaken. In Part II, an exploration into the prevailing determinants influencing physicians’ engagement with SDM is carried out. Outline thesis PART I – Training residents in shared decision-making In Chapter 2, we evaluate the educational quality of interventions to improve residents’ SDM skills. An innovative and comprehensive framework is employed, scrutinizing the influence of training characteristics on the overall educational quality. Chapter 3 presents a pilot study on gathering patient feedback concerning residents’ SDM skills. The chapter also investigates the utility of this collected feedback for reflection and reflective learning. PART II – Exploring determinants of clinician (dis-)engagement in shared decision-making In Chapter 4, an exploration into the relationship between physician culture and SDM is undertaken through the application of a hospital ethnography, interviews, and the utilization of the renowned ‘Theory of Practice’ by French sociologist Bourdieu as a lens to elucidate cultural dynamics. Chapter 5 investigates provider-related attributes concerning SDM in contrast to other tasks. Chapter 6 employs the Diffusion of Innovation theory to uncover underlying beliefs, cognitions, and contextual challenges influencing the decision to adopt or reject SDM. In Chapter 7, the main findings of the dissertation are highlighted and explained. It offers a critical reflection, followed by insights into potential avenues for future research. The chapter also delves into the strengths and limitations of this dissertation and concludes by presenting practice recommendations. 1
14 Chapter 1 Figure 1: Structure and outline of the thesis.
15 General introduction References 1. Teunissen, P.W., Unravelling learning by doing. 2009, VU: Amsterdam. 2. Newton, B.W., et al., Is there hardening of the heart during medical school? Acad Med, 2008. 83(3): p. 244-9. 3. Bombeke, K., et al., Help, I’m losing patient-centredness! Experiences of medical students and their teachers. Med Educ, 2010. 44(7): p. 66273. 4. Charles, C., A. Gafni, and T. Whelan, Shared decision-making in the medical encounter: what does it mean? (or it takes at least two to tango). Soc Sci Med, 1997. 44(5): p. 681-92. 5. Singh Ospina, N., et al., Educational programs to teach shared decision making to medical trainees: A systematic review. Patient Education and Counseling, 2020. 103(6): p. 1082-1094. 6. Legare, F., et al., Core competencies for shared decision making training programs: insights from an international, interdisciplinary working group. J Contin Educ Health Prof, 2013. 33(4): p. 267-73. 7. Legare, F. and H.O. Witteman, Shared decision making: examining key elements and barriers to adoption into routine clinical practice. Health Aff (Millwood), 2013. 32(2): p. 276-84. 8. Stacey, D., et al., Decision aids for people facing health treatment or screening decisions. Cochrane Database Syst Rev, 2017. 4: p. CD001431. 9. Coates, D. and T. Clerke, Training Interventions to Equip Health Care Professionals With Shared Decision-Making Skills: A Systematic Scoping Review. J Contin Educ Health Prof, 2020. 40(2): p. 100-119. 10. Hauser, K., et al., Outcome-Relevant Effects of Shared Decision Making. Dtsch Arztebl Int, 2015. 112(40): p. 665-71. 11. Elwyn, G., D.L. Frosch, and S. Kobrin, Implementing shared decision-making: consider all the consequences. Implement Sci, 2016. 11: p. 114. 12. Wyatt, K.D., et al., Shared Decision Making in Pediatrics: A Systematic Review and Meta-analysis. Acad Pediatr, 2015. 15(6): p. 573-83. 13. Henselmans, I., et al., Shared decision making about palliative chemotherapy: A qualitative observation of talk about patients’ preferences. Palliat Med, 2017. 31(7): p. 625-633. 14. Welch, S., Understand physician culture to facilitate change. Begin by engaging their scientific and competitive qualities. Healthc Exec, 2010. 25(3): p. 92-5. 15. Dewey, J., Experience and education. The Kappa Delta Pi lecture series. 1938, New York,: The Macmillan company. xii, p., 2 l., 116 p. 16. Knowles, M., The Modern Practice of Adult Education : From Pedagogy to Andragogy. 1980: CAMBRIDGE. 17. Kolb, D.A., Experiential Learning. 1984: Englwood Cliffs. 18. Schö n, D.A., The reflective practitioner : how professionals think in action. Harper torchbooks TB 5126. 1983, New York: Basic Books. x, 374 p. : ill. 19. Moon, J.A. and Dawsonera, A handbook of reflective and experiential learning : theory and practice. 1st edition. ed. 2004, London: RoutledgeFalmer. viii, 252 p. 20. Boud, D., et al., Reflection : turning experience into learning. 1985, London ; New York: RoutledgeFalmer. 21. Argyris, C., D.A. Schö n, and S. Institute of Advanced Architectural, Theory in practice : increasing professional effectiveness. 1st ed. Jossey-Bass higher and adult education series. 1974, San Francisco: Jossey-Bass Publishers. xiv, 224 pages. 22. Epstein, R.M. and E.M. Hundert, Defining and assessing professional competence. Jama-Journal of the American Medical Association, 2002. 287(2): p. 226-235. 23. Eraut, M., Informal learning in the workplace. Studies in Continuing Education, 2004. 26(2): p. 247-73. 24. Hafferty, F.W., Beyond curriculum reform: Confronting medicine’s hidden curriculum. Academic Medicine, 1998. 73(4): p. 403-407. 25. Hafferty, F.W. and R. Franks, The Hidden Curriculum, Ethics Teaching, and the Structure of Medical-Education. Academic Medicine, 1994. 69(11): p. 861-871. 26. Lave, J. and E. Wenger, Situated learning : legitimate peripheral participation. Learning in doing. 1991, Cambridge England ; New York: Cambridge University Press. 138 p. 27. Kleinman, A., L. Eisenberg, and B. Good, Culture, illness, and care: clinical lessons from anthropologic and cross-cultural research. Ann Intern Med, 1978. 88(2): p. 251-8. 28. Bourdieu, P., Outline of a theory of practice. Cambridge studies in social anthropology 16. 1977, Cambridge ; New York: Cambridge University Press. viii, 248 p. 1
16 Chapter 1 29. Giddens, A., The constitution of society : outline of the theory of structuration. 1984, Cambridge Cambridgeshire: Polity Press. xxxvii, 402 p. 30. Hahn, R.A. and A. Kleinman, Biomedical Practice and Anthropological Theory - Frameworks and Directions. Annual Review of Anthropology, 1983. 12: p. 305-333. 31. Good, B., Medicine, rationality, and experience : an anthropological perspective. The Lewis Henry Morgan lectures. 1994, Cambridge ; New York: Cambridge University Press. xvii, 242 p. 32. van der Geest, S. and K. Finkler, Hospital ethnography: introduction. Soc Sci Med, 2004. 59(10): p. 1995-2001. 33. Singer, M. and H.A. Baer, Introducing medical anthropology : a discipline in action. 2007, Lanham, MD: AltaMira Press. v, 246 p. : ill. 34. Hilton, C.E., Behaviour change, the itchy spot of healthcare quality improvement: How can psychology theory and skills help to scratch the itch? Health Psychol Open, 2023. 10(2): p. 20551029231198938.
17 General introduction 1
PART I Training residents in shared decision-making
CHAPTER 2 Assessing the educational quality of shared decision-making interventions for residents: a systematic review Spinnewijn L Scheele F Braat D Aarts J Patient Educ Couns. 2024 Feb 3:108187 This chapter is accepted for publication and published online ahead of print. Permanent link: https://doi.org/10.1016/j.pec.2024.108187
22 Chapter 2 Abstract Objectives Many studies on educational interventions to enhance residents’ shared decision-making (SDM) skills show limited improvement in SDM skills and clinical outcomes. One plausible explanation for these suboptimal results is the insufficient emphasis on the educational quality of training interventions. Methods This review evaluates interventions’ educational quality using an evaluation framework based on a previous study on effective skills transfer and a well-known SDM model. A systematic review was conducted, searching three databases until December 13, 2022. We assessed study quality by calculating MERSQI scores, examined the levels of study effects based on Kirkpatrick’s model, and applied our evaluation framework to assess the interventions’ educational quality. Given the heterogeneity among the studies, a meta-analysis was not feasible. Results Twenty-six studies were included. Role-play and feedback were common training characteristics (65% and 54% of interventions). Only four studies (15%) met our framework’s high educational quality threshold. No correlation was found between MERSQI scores and educational quality. Conclusions This review is a valuable attempt to assess the educational quality of SDM interventions beyond measuring study outcomes. Practice implications Future evaluation frameworks should consider study results, training characteristics, and training content. Our framework offers a sound basis for such an evaluation framework.
23 Assessing SDM interventions’ educational quality Introduction There is an ongoing appreciation for patient-centered medicine. Shared decision-making (SDM) plays a crucial role as part of this approach. [1] SDM can be defined as an interactive process in which healthcare professionals and patients collaborate to make informed decisions about the patient’s health. [2] The potential benefits of SDM include reducing overuse, underuse, and misuse of healthcare resources, as well as improving patient satisfaction. [3-5] Furthermore, ethically, SDM is supported by the belief that patients have the right to be informed about available treatment options and should participate actively in decisions regarding their health. [3, 6] Despite the promising prospects of SDM, widespread implementation in routine care is still needed. [6, 7] Various barriers hinder the adoption of SDM, including healthcare professionals’ lack of knowledge and familiarity with the approach. [8, 9] Educational interventions have the potential to overcome these barriers and enhance SDM practice. [10, 11] Particularly noteworthy is the significance of SDM education in medical residency training. Recent research indicates that residents are less likely to engage in SDM compared to more experienced physicians, often favoring paternalistic approaches. [12] Moreover, existing training programs demonstrate poor effects, as there is limited evidence of their impact on trainees’ skills and clinical outcomes. [13] Therefore, a detailed exploration of SDM training for residents becomes of particular interest. These study results are disturbing yet non-surprising. First, it is hard to gain solid evidence of the effectiveness of SDM training programs, as outcome measures are very heterogeneous. [10, 13, 14] Second, intended effects like behavioral and practice change are complex aims, and numerous influences determine their success. [15] Therefore, a comprehensive approach is necessary to evaluate training interventions for residents, extending beyond assessing study outcomes alone. This comprehensive approach encompasses examining the characteristics and content quality of the training interventions. However, established standards for assessing training quality remain elusive, and published descriptions of training interventions often lack precision. For example, many journals do not require a checklist for describing training interventions, despite the availability of several checklists. [16-18] To address these gaps, we have developed a novel approach to evaluating the educational properties of SDM training interventions. Our evaluation framework is based on the outcomes of a comprehensive overview of systematic reviews on the transfer of effective communication skills, which identified minimal criteria for successful skills transfer. [19] Although no validated or published set of criteria exists specifically for SDM interventions, our framework draws from this well-referenced overview, which has been cited in over 100 studies evaluating training initiatives, including those focusing on SDM-related skills. Focusing on a concise set of critical teaching characteristics allows our evaluation framework to better compare previously published interventions, even when they do not meet the publication standards outlined in the earlier mentioned checklists. [16-18] Furthermore, we have incorporated four key SDM training elements in our evaluation framework, derived from leading authors in the SDM literature, to 2
24 Chapter 2 comprehensively evaluate interventions’ training content. [6, 20, 21] These key elements involve four steps in SDM consultations: (1) setting the stage, (2) explaining options, (3) discussing preferences and (4) making a decision. This study contributes to the existing literature by going beyond traditional outcome measures and aiming to differentiate between interventions that are likely to effectively transfer SDM knowledge and skills to medical residents. We sought to answer the following research question: To what extent do current SDM skills training interventions for medical residents involve effective teaching characteristics or strategies, as proposed in our evaluation framework? Additionally, we explored the relationship between our evaluation framework and study quality, as well as whether studies incorporating criteria from our framework demonstrate greater success in improving SDM. Method We performed a systematic review and reported our results following the PRISMA statement. [22] Study eligibility To be eligible for inclusion in this study, the selected studies had to meet specific criteria. They needed to report on an educational intervention that focused on teaching medical residents’ particular elements of shared decision-making (SDM). If the educational interventions were targeted at a broader audience (such as other healthcare professionals), they were included only if medical residents were explicitly part of the study population. For this study, medical residents were defined as physicians who had completed medical school and were actively engaged in medical specialty training. Only studies that reported on training effects were considered for inclusion in our analysis. Both observational and randomized designs were included in the review. To ensure a specific focus on SDM-related skills improvement, articles that did not explicitly state their intention to enhance SDM-related skills were excluded. Moreover, we limited our analysis to studies published in the English language. No other exclusion criteria were applied during the selection process. Search strategy and study selection First, PubMed databases were searched. The search was built with the help of an experienced librarian. The following search criteria were used: Medical residents AND Educational intervention AND Shared decision-making. To ensure comprehensive coverage, a combination of various terms and keywords was employed. Articles were searched up to December 31, 2021. An updated search was performed on December 13, 2022, to capture any additional relevant articles. Coarse searches in Embase and ERIC databases found no additional eligible studies. The complete search query is provided in Appendix A.
25 Assessing SDM interventions’ educational quality Following this search, we utilized Covidence, a secure online data management program, for article upload and rating tracking. [23] The data management program removed duplicates. The eligibility of the studies was checked independently and in duplicate by two researchers (LS, JA) in two rounds, starting with the title and abstract and followed by full-text screening. Articles that did not match the eligibility criteria were excluded. Any discrepancies between the researchers’ judgements were discussed until they reached an agreement. Duplicates were removed. Two researchers (LS, JA) independently assessed study eligibility in two rounds: title/abstract screening and full-text screening. Non-eligible articles were excluded. Discrepancies were resolved through discussion. Manual reference list searches and reviews of related systematic reviews yielded no new results. [1, 13, 14, 24, 25] Data extraction and synthesis The extracted data were recorded on a predefined data extraction sheet, which was developed and pilot-tested by the two researchers (LS, JA) responsible for the extraction process. Only published data were utilized for the purpose of data extraction and subsequent analysis. A comprehensive list of the items from our data extraction sheet is detailed in Appendix B. Quality assessment We conducted assessments of both the study quality and the quality of the intervention itself. Initially, two researchers (LS, JA) independently scored all the quality items. Subsequently, any discrepancies in the quality ratings were discussed until a consensus was reached. Study quality To evaluate the study quality and risk of bias, we utilized the Medical Education Research Study Quality Instrument (MERSQI), a 10-item instrument that assesses the quality of educational interventions across six domains: study design, sampling, type of data, data analysis, validity of the evaluation instrument, and outcome measures. [26, 27] Each domain has a maximum score of 3. The total MERSQI scores can range from 5 to 18, with higher scores indicating higher study quality. The MERSQI instrument can be applied to non-qualitative research reports, including non-comparative study designs. [28] In order to establish a metric for study quality, we utilized the total MERSQI scores, recognizing that higher-quality study scores can only be meaningfully determined when compared to a comparable sample. [28] To provide context, we calculated the median score within our sample and established this score as the threshold for higher quality. Additionally, we separately reported the Kirkpatrick levels of evaluation. The Kirkpatrick Model comprises four levels: (1) reaction, (2) learning, (3) behavior, and (4) results. [29, 30] We opted to include these levels, as they provide insights into how the included studies measured the impact of their interventions. Evaluation framework We developed a comprehensive framework for evaluating training quality, encompassing items that assess both the training content, focusing on the four key elements of SDM, and five critical training characteristics essential for effective skills transfer. 2
26 Chapter 2 Training content: SDM elements addressed in training In our assessment of training content, we evaluated whether the training addressed key SDM elements derived from the ‘talk model’ or ‘three talk model by Elwyn and others. [20, 21] This model integrates various previously published SDM models and comprises ‘choice talk,’ ‘option talk,’ and ‘decision talk’. [20, 21] Stiggelbout and others further refined this model into four steps. [6] The first step involves the clinician ‘setting the stage’ at the beginning of the conversation by informing the patient about the decision-making process and their active role. The second step pertains ‘discussing options’, which corresponds to Elwyn’s ‘option talk’, and encompasses the explanation of the advantages and disadvantages of each available option. The third and fourth steps align with Elwyn’s ‘decision talk’, with the third step focusing on ‘discussing preferences,’ where the healthcare professional actively elicits and discusses the patient’s preferences, and the fourth element concerning ‘making a decision,’ which can be either a joint decision or dependent on the patient’s preference for their role in decisionmaking. [6] Our evaluation framework followed this four-step approach, resulting in the four SDM elements in our evaluation framework. Training characteristics: Critical elements for effective skills-transfer addressed in training To define the criteria for high-quality training characteristics conducive to effective skillstransfer for our evaluation framework, we relied on the results of the previously mentioned overview of systematic reviews by Berkhof and others. [19] According to this overview, effective training programs should be learner-centered, incorporate participants’ practice examples or experiences, and include opportunities for skill practice. Ideally, the training duration should be a whole day or longer. Effective training methods include role-play, feedback on practical training components, and small group discussions. [19] Therefore, our evaluation framework incorporated five quality criteria for effective skills transfer: 1) training duration of approximately a whole day or longer, 2) learner-centered training, and the use of 3) role-play, 4) feedback and 5) small group discussions as training methods. Although not explicitly incorporated into our evaluation framework, study results will report on four additional training characteristics from Berkhof’s study. These characteristics were added to the study results to enhance the overall understanding of the training interventions, although their effectiveness in skill transfer has not been proven. [19] The extra training descriptors encompassed (1) any form of discussion, (2) an oral presentation, (3) modelling (e.g., demonstration of SDM skills), and (4) the provision of written information. Data analysis For the purpose of reporting, study characteristics were systematically charted and presented in a tabular format. Due to heterogeneity in study designs and reported effects, a meta-analysis was not conducted. [31] Instead, we used descriptive statistics in a narrative synthesis to summarize the study results. We examined the relationship between study quality and effective teaching strategies by calculating the Spearman’s correlation coefficient between MERSQI scores and the count of effective training elements. We also compared studies meeting our criteria for high
27 Assessing SDM interventions’ educational quality quality with other studies in terms of MERSQI scores and study characteristics. The research group agreed on a threshold of four out of five critical training characteristics and three out of four essential SDM elements to define high-quality studies. Results The process of study selection is outlined in Figure 1. The team agreed on a final list of 26 articles, all meeting our research’s inclusion criteria and describing unique educational SDM interventions aimed at residents. Table 1 provides a summary of general study and training intervention characteristics, including the computed quality scores and the applied evaluation framework. Figure 1: PRISMA flow chart of article selection. n means the number of articles. 2
28 Chapter 2 Table 1: General study characteristics, training characteristics, quality measures and the evaluation framework. Ref: [7, 11, 32-55] General study characteristics Quality measures First author, year Country Study design Clinical domain Participants # Reporting of outcome Risk of bias/quality score§ (MERSQI) Level of outcome* (Kirkpatrick) Ajayi, 2019 [32] US NCBA Internal and palliative medicine 30 Self-reported by participants 9.5 2,3 Alexander, 2006 [33] US CBA Internal and palliative medicine 56 Objective assessment by observers 12.5 2 Amell, 2022 [7] US NCBA Internal medicine 48 Self-reported by participants; objective assessment by observers 11.5 2 Bhatt, 2016 [34] Ireland NCBA Surgery 57 Self-reported by participants 7.5 1,2 Bieber, 2006 [35] GER CBA; overall study RCT Internal medicine 13 Self-reported by participants; patient experiences 12 2 Bossen, 2022 [36] NL NCBA and CBA Orthopedic surgery 9 (19)# Self-reported by participants; patient experiences; objective measurements of decision aid use 13.5 2,4 Chesney, 2018 [37] Canada NCBA General surgery 18 Self-reported by participants; objective assessment by observers 12 1,2,3 Dion, 2016 [38] Canada NCBA Family medicine 247 Objective assessment through general examination 10 2 Geessink, 2017 [39] NL NCBA Geriatric oncology 5 (11)# Self-reported by participants; patient experiences 14.5 4 Geiger, 2017 [40] GER RCT Multiple1) 38 Self-reported by participants; objective assessment by observers; patient ratings 16 3 Grad, 2022 [41] Canada NCBA Family medicine 73 Self-reported by participants 11 2
29 Assessing SDM interventions’ educational quality The evaluation framework† Other training characteristics Training content: SDM elements addressed Training characteristics: Critical elements for effective skills-transfer Additional elements, Berkhof et al. [19] Training group size SDM training duration Setting the stage Explaining options Discussing preferences Making a decision ≥ Whole-day training⁰ Learner centered? ‡ Role-play Feedback Small group discussion Discussion Oral presentation Modelling Written information NNYNNNYYYYYNN4-5 90 minutes plus a general introduction N N Y N N Y Y Y Y Y Y N N Unclear,“small groups” 3.5 hours Y Y Y Y N Y Y Y Y Y Y Y N 7-9,role-playin ‘smaller groups 100 minutes NNNN NNYNNYNYN18-20 2 hours YNNN YNYNNNYNNUnclear 18 hours, but unclear what precisely about SDM and what not YYYY NNYNNNYNY19 4 hours plus 1530 minutes of e-learning Y Y Y N N N Y Y N N Y Y N Unclear, role-play with 2-3 2 hours N Y YY NNNNNNN Y YN/A,on-line 2 to 3 hours YYYY YYYYNYYYNUnclear 8 hours plus training on the job YYYY NNNYNNNYYUnclear Unclear NYYN NNYNNNYNYUnclear 2 hours [continued on next page] 2
30 Chapter 2 Table 1: [continued] General study characteristics Quality measures First author, year Country Study design Clinical domain Participants # Reporting of outcome Risk of bias/ quality score§ (MERSQI) Level of outcome* (Kirkpatrick) Harman, 2019 [42] US NCBA Internal medicine, pediatrics Unclear Assessment by (peer-) observers 11.5 3 Henselmans, 2019 [43] NL RCT Medical oncology 14 (31)# Self-reported by participants; objective assessment by rating videos 16 1,2,3 Kanzaria, 2020 [44] US PTO Emergency medicine 28 Self-reported by participants, objective assessment by observers 6.5 2 Légaré, 2012 [11] Canada RCT Family medicine 250 Self-reported by participants; patient experiences 14.5 3,4 Ritter, 2019 [45] SUI CBA Internal medicine 27 Self-reported by participants, objective assessment by observers 12.5 2,3 Robertson, 2021 [46] US RCT Anesthesio-logy 60 Self-reported by participants, objective assessment by observers 13 1,2 Roter, 2012 [47] US NCBA; patients in RCT Family medicine 9 (29)# Self-reported by participants, patient experiences 11.5 3 Rusiecki, 2018 [48] US NCBA Internal medicine, pediatrics 36 Self-reported by participants; objective assessment by observers and in a knowledge test 12.5 2,3 Sherman, 2005 [49] US RCT Pediatrics 27 Self-reported by participants 12 2,3 Simmons, 2016 [50] US PTO Internal medicine, pediatrics 130 Self-reported by participants, objective measurement of instrument use 8 1
31 Assessing SDM interventions’ educational quality The evaluation framework† Other training characteristics Training content: SDM elements addressed Training characteristics: Critical elements for effective skills-transfer Additional elements, Berkhof et al. [19] Training group size SDM training duration Setting the stage Explaining options Discussing preferences Making a decision ≥ Whole-day training⁰ Learner centered? ‡ Role-play Feedback Small group discussion Discussion Oral presentation Modelling Written information NYYNNYYYNYYYNUnclear 45-90 minutes plus multiple feedback moments YYYY YYYYNNYYY3-6 10 hours Y Y Y Y N N Y Y Y Y Y Y N15,role-play with 2 1 hour plus 20 minutes assessment NYYN NNNNNYYYYUnclear 4 hours YYYY NNYNNYYNYUnclear 2 hours N Y YN NNN Y NN Y Y NN/A,on-line 40-45 minutes video plus feedback OSCE NNYY NYNNNNNY YN/A,on-line Unclear, traineedependent YYYY YNYYYYYNN4-6 6 hours, plus practice in reallife N Y Y N N N N N Y Y Y Y N 13-14,however, with ‘small group discussion’ 1 hour YYYY NNYNNYYYYUnclear 1 or 2 hours [continued on next page] 2
32 Chapter 2 Table 1: [continued] General study characteristics Quality measures First author, year Country Study design Clinical domain Participants # Reporting of outcome Risk of bias/ quality score§ (MERSQI) Level of outcome* (Kirkpatrick) Stacey, 2012 [51] Canada NCBA Oncology, palliative medicine 11 Self-reported by participants, objective assessment by observers 9.5 1,2 Sullivan, 2010 [52] US RCT Internal medicine 213 Self-reported by participants 12 1,3 Volk, 2014 [53] US PTO Multiple2) 12 (49)# Self-reported by participants 8.5 1,2 Worthington, 2020 [54] US NCBA Internal medicine, pediatrics 58 Self-reported by participants 7.5 1,2 Yuen, 2013 [55] US PTO Internal medicine 33 Self-reported by participants 7.5 1,2 Abbreviations: United States (US); the Netherlands (NL); Germany (GER); Switzerland (SUI) shared decisionmaking (SDM); controlled before and after study (CBA); non-controlled before and after study (NCBA); post-test only study (PTO); Randomized controlled trial (RCT); not applicable (N/A); objective structured clinical examination (OSCE) # Between brackets, the total number of trained health professionals in the study is mentioned, including the number of residents; only applicable when the study includes healthcare professionals other than residents in the training * Levels of Kirkpatrick: 1 = level 1 ‘reactions’; 2 = level 2 ‘learning’; 3 = level 3 ‘behavior’; 4 = level 4 ‘results’ § Higher quality MERSQI scores above the median score (> 11.75) are represented in green † Scores meeting the quality criterion according to our evaluation framework (Y) are represented in green ⁰ Whole day training is defined as reaching a minimum of approximately 6 to 7 hours of training time ‡ Learner-centered means practical in nature, e.g., using learner practice examples/reflecting learners’ practice, not solely predetermined training content or teacher-centered 1) Specialties/units involved: stroke unit, multiple sclerosis outpatient ward, stem cell transplantation unit, dentistry, radiation oncology, surgery, neurosurgery, gynecology 2) Primary care specialties involved: family medicine, internal medicine, infectious diseases, preventive medicine, and clinical psychology The studies printed in italic and bold represent studies with at least four critical training characteristics and three essential SDM elements described in their training content, thus meeting all thresholds of the evaluation framework
33 Assessing SDM interventions’ educational quality The evaluation framework† Other training characteristics Training content: SDM elements addressed Training characteristics: Critical elements for effective skills-transfer Additional elements, Berkhof et al. [19] Training group size SDM training duration Setting the stage Explaining options Discussing preferences Making a decision ≥ Whole-day training⁰ Learner centered? ‡ Role-play Feedback Small group discussion Discussion Oral presentation Modelling Written information N Y Y Y N N Y Y Y Y Y Y Y11,role-play with 3 3 hours Y Y NY NNN Y NNN Y YN/A,on-line Unclear, traineedependent Y Y YY NNNNNN Y Y NN/A,on-line Unclear YYYY NNNYNYYYNUnclear 1 hour N Y Y Y N N Y Y Y Y Y Y N33,role-play with 6 4 hours plus online PowerPoint Study characteristics and study quality It should be noted that certain studies evaluated both doctor-aimed training and patient interventions, such as decision aids. However, our results section does not include findings regarding these patient interventions, as they were beyond the scope of our review. MERSQI scores ranged from 6.5 to 16, with a mean score of 11.27 and a median score of 11.75. 50% of studies reached MERSQI scores of 12 or higher, reflecting higher study quality. Outcome measures and levels of Kirkpatrick Among the included studies, seventeen (65%) utilized objective outcome measures such as knowledge tests, observations, or patient-reported outcomes/experiences. However, the majority of studies relied on self-reported outcomes from the participants. Subjective measures were included as primary or secondary outcomes in twenty-three studies (88%). Due to the substantial heterogeneity in study design, intervention design, assessment instruments (subjective vs. objective), and outcome measures, a systematic comparison of study outcomes through meta-analysis was not feasible. While many studies reported statistically significant improvements, the definition of what constituted a relevant outcome varied significantly. The heterogeneity in outcome measures is also reflected in the different levels of the Kirkpatrick model. Eleven studies (42%) reported potential changes in daily practice (Kirkpatrick level 3). Among them, only four studies utilized tests or structured observations to assess these 2
34 Chapter 2 behavioral changes. [40, 42, 43, 48] The remaining seven studies relied on self-reported outcomes from participants or patient-reported outcome measures. [11, 32, 37, 45, 47, 49, 52] Three studies (12%) described the impact of the intervention on practice results (Kirkpatrick level 4). [11, 36, 39] One study examined patient satisfaction and the use of a decision aid as outcomes, [36] while another study focused on patient-reported experiences. [39] The third study reported changes in antibiotic use, although the measurement relied on self-reports from study participants rather than objective assessment. [11] Training content and characteristics SDM elements In terms of training content, five interventions (19%) addressed three out of four SDM elements, while 11 interventions (42%) incorporated all four SDM elements in their training. One study did not clearly specify whether any SDM element was addressed. Training characteristics The majority of training sessions lasted up to half a day, with 11 studies (42%) reporting a duration of up to two hours. Only four interventions (15%) had a training duration exceeding six hours. Six interventions (23%) were considered learner-centered, incorporating trainees’ actual practice experiences into the training content. The sizes of the training groups varied, with several studies lacking explicit information on group size. In some cases, training did not involve groups at all, as four studies (15%) solely utilized online training. Role-play opportunities were provided in 17 interventions (65%), while 14 interventions (54%) included feedback as part of the training. Seven interventions (27%) utilized small-group discussions as a training method, as they provided training in small groups and incorporated discussions. Educational quality versus outcome The correlation between the MERSQI score and the number of critical characteristics in the educational interventions was very low (Spearman’s correlation coefficient = 0.089). Four studies (15%) met the threshold for high quality according to our evaluation framework. [7, 39, 43, 48] These studies, highlighted in Table 1 with bold lettering, consisted of three non-controlled before and after (NCBA) studies and one randomized controlled trial (RCT). These four studies had substantially higher MERSQI scores, with an average score of 13.63 (compared to 11.27 in all studies) and included the highest-scoring study with a MERSQI score of 16. Due to the heterogeneity of reported outcome measures and Kirkpatrick levels (ranging from 1 to 4), no conclusions could be drawn regarding the relationship between quality and outcome.
www.ridderprint.nlRkJQdWJsaXNoZXIy MTk4NDMw