van Der P.M

van Der P.M
Are you van Der P.M?

Claim your profile, edit publications, add additional information:

Contact Details

Name
van Der P.M
Affiliation
Location

Publications Authored By van Der P.M

Objective-To determine the perceived importance of specific competencies in professional veterinary practice and education among veterinarians in several countries. Design-Survey-based prospective study. Sample-1,137 veterinarians in 10 countries.Read More

Procedures-Veterinarians were invited via email to participate in the study. A framework of 18 competencies grouped into 7 domains (veterinary expertise, communication, collaboration, entrepreneurship, health and welfare, scholarship, and personal development) was used. Respondents rated the importance of each competency for veterinary professional practice and for veterinary education by use of a 9-point Likert scale in an online questionnaire. Quantitative statistical analyses were performed to assess the data. Results-All described competencies were perceived as having importance (with overall mean ratings [all countries] ≥ 6.45/9) for professional practice and education. Competencies related to veterinary expertise had the highest ratings (overall mean, 8.33/9 for both professional practice and education). For the veterinary expertise, entrepreneurship, and scholarship domains, substantial differences (determined on the basis of statistical significance and effect size) were found in importance ratings among veterinarians in different countries. Conclusions and Clinical Relevance-Results indicated a general consensus regarding the importance of specific types of competencies in veterinary professional practice and education. Further research into the definition of competencies essential for veterinary professionals is needed to help inform an international dialogue on the subject.

Abstract Background: Virtual patients (VPs) are increasingly used to train clinical reasoning. So far, no validated evaluation instruments for VP design are available. Aims: We examined the validity of an instrument for assessing the perception of VP design by learners.Read More

Methods: Three sources of validity evidence were examined: (i) Content was examined based on theory of clinical reasoning and an international VP expert team. (ii) The response process was explored in think-aloud pilot studies with medical students and in content analyses of free text questions accompanying each item of the instrument. (iii) Internal structure was assessed by exploratory factor analysis (EFA) and inter-rater reliability by generalizability analysis. Results: Content analysis was reasonably supported by the theoretical foundation and the VP expert team. The think-aloud studies and analysis of free text comments supported the validity of the instrument. In the EFA, using 2547 student evaluations of a total of 78 VPs, a three-factor model showed a reasonable fit with the data. At least 200 student responses are needed to obtain a reliable evaluation of a VP on all three factors. Conclusion: The instrument has the potential to provide valid information about VP design, provided that many responses per VP are available.

2014Oct
Adv Health Sci Educ Theory Pract
Adv Health Sci Educ Theory Pract 2014 Oct 15. Epub 2014 Oct 15.
Department of Primary and Community Care, Radboud University Nijmegen Medical Centre, ELG 117, Postbus 9101, 6500 HB, Nijmegen, The Netherlands, greetje.sagasser@radboudumc.nl.

Self-regulated learning is essential for professional development and lifelong learning. As self-regulated learning has many inaccuracies, the need to support self-regulated learning has been recommended. Supervisors can provide such support.Read More

In a prior study trainees reported on the variation in received supervisor support. This study aims at exploring supervisors' perspectives. The aim is to explore how supervisors experience self-regulated learning of postgraduate general practitioners (GP) trainees and their role in this, and what helps and hinders them in supervising. In a qualitative study using a phenomenological approach, we interviewed 20 supervisors of first- and third-year postgraduate GP trainees. Supervisors recognised trainee activity in self-regulated learning and adapted their coaching style to trainee needs, occasionally causing conflicting emotions. Supervisors' beliefs regarding their role, trainees' role and the usefulness of educational interventions influenced their support. Supervisors experienced a relation between patient safety, self-regulated learning and trainee capability to learn. Supervisor training was helpful to exchange experience and obtain advice. Supervisors found colleagues helpful in sharing supervision tasks or in calibrating judgments of trainees. Busy practice occasionally hindered the supervisory process. In conclusion, supervisors adapt their coaching to trainees' self-regulated learning, sometimes causing conflicting emotions. Patient safety and entrustment are key aspects of the supervisory process. Supervisors' beliefs about their role and trainees' role influence their support. Supervisor training is important to increase awareness of these beliefs and the influence on their behaviour, and to improve the use of educational instruments. The results align with findings from other (medical) education, thereby illustrating its relevance.

2014Sep
Acad Med
Acad Med 2014 Sep;89(9):1259-66
Dr. Fokkema is a physician and PhD student, St. Lucas Andreas Hospital, Amsterdam, the Netherlands. Dr. Scheele is professor, VU University Medical Center, Amsterdam, the Netherlands, and a gynecologist and residency program director, St. Lucas Andreas Hospital, Amsterdam, the Netherlands. Dr. Westerman is a researcher, School of Medical Sciences, VU University Medical Center, Amsterdam, the Netherlands, and a resident in internal medicine, St. Lucas Andreas Hospital, Amsterdam, the Netherlands. Dr. van Exel is associate professor, Institute of Health Policy and Management, Erasmus University Rotterdam, Rotterdam, the Netherlands. Dr. Scherpbier is professor and dean, Faculty of Health, Medicine and Life Sciences, Maastricht University, Maastricht, the Netherlands. Dr. van der Vleuten is professor, Department of Educational Development and Research, Faculty of Health, Medicine and Life Sciences, Maastricht University, Maastricht, the Netherlands. Dr. Dörr, deceased, was professor, Department of Education and Teaching, Leiden University Medical Center, Leiden, the Netherlands, and a gynecologist, Medical Centre Haaglanden, Den Haag, the Netherlands. Dr. Teunissen is a resident in obstetrics-gynecology, VU University Medical Center, Amsterdam, the Netherlands, and associate professor, Department of Educational Development and Research, Faculty of Health, Medicine and Life Sciences, Maastricht University, Maastricht, the Netherlands.

Anticipating users' perceptions of the effects an innovation will have in daily practice prior to implementation may lead to a more optimal innovation process. In this study, the authors aimed to identify the kinds of perceptions that exist concerning the effects of workplace-based assessment (WBA), an innovation that is widely used in medical education, among its users.
In 2012, the authors used Q methodology to ascertain the principal user perceptions of effects of WBA in practice.Read More

Participating obstetrics-gynecology residents and attending physicians (including residency program directors) at six hospitals in the Netherlands performed individual Q sorts to rank 36 statements concerning WBA and WBA tools according to their level of agreement. The authors conducted by-person factor analysis to uncover patterns in the rankings of the statements. They used the statistical results and participant comments about their sorts to interpret and describe distinct perceptions.
The analysis of 65 Q sorts (completed by 22 residents and 43 attendings) identified five distinct user perceptions regarding the effects of WBA in practice, which the authors labeled enthusiasm, compliance, effort, neutrality, and skepticism. These perceptions were characterized by differences in views on three main issues: the intended goals of the innovation, its applicability (ease of applying it to practice), and its actual impact.
User perceptions of the effects of innovations in medical education can be typified and should be anticipated. This study's insights into five principal user perceptions can support the design and implementation of innovations in medical education.

Abstract Background: Evidence tailored to sub-Saharan Africa on outcomes of innovations in medical education is needed to encourage and advance their implementation in this region. Aim: To investigate preparedness for practice of students and graduates from an innovative and a conventional medical curriculum in a sub-Saharan African context. Methods: Using mixed methods we compared junior doctors and fifth-year students from two Mozambican medical schools: one with an innovative problem- and community-based curriculum and one with a conventional lecture- and discipline-based curriculum.Read More

A questionnaire on professional competencies was administered, semi-structured interviews were conducted, and work diaries were collected. The findings were integrated in a conceptual model. Results: Six areas of tension between global health care ideals and local health care practice emerged from the data that challenged doctors' motivation and preparedness for practice. Four elements of the innovative curriculum equipped students and graduates with skills, attitudes and competencies to better cope with these tensions. Students and graduates from the innovative curriculum rated significantly higher levels on various competencies and expressed more satisfaction with the curriculum and its usefulness for their work. Conclusion: An innovative problem- and community-based curriculum can improve sub-Saharan African doctors' motivation and preparedness to tackle the challenges of health care practice in this region.

2014Sep
Phys Ther
Phys Ther 2014 Sep 18. Epub 2014 Sep 18.
M.W.G. Nijhuis-van der Sanden, PhD, Radboud University Medical Center, Scientific Institute for Quality of Healthcare (IQ Healthcare).

Clinical practice guidelines (CPGs) are not readily implemented in clinical practice. One of the impeding factors is that physical therapists (PTs) do not hold realistic perceptions of their adherence to CPGs. Peer Assessment (PA) is a implementation strategy that aims at improving guideline adherence by enhancing reflective practice, awareness of professional performance, and attainment of personal goals.Read More


To compare the effectiveness of PA with the usual Case Discussion (CD) strategy on adherence to CPGs for physical therapy management in patients with upper extremity complaints.
Single-masked cluster-randomized controlled trial with pre- post-test design.
Twenty Communities of Practice (n=149 PTs) were randomly assigned to PA or CD, both consisting of four sessions over six months. PA and CD groups worked on identical clinical cases relevant to the guidelines. PA focused on individual performance observed and evaluated by peers; CD focused on discussion.
Guideline adherence, measured with clinical vignettes; Reflective practice, with the Self Reflection and Insight Scale (SRIS); Awareness of performance, via the correlation between perceived and assessed improvement; Attainment of personal goals, with written commitments to change.
PA improved more on guideline adherence than CD. Effect:22.52, 95% CI:2.38 - 42.66, p=.03. SRIS scores did not differ between groups. Awareness of performance was greater for PA(r=0.36) than for CD(r=0.08). Effect:14.73,95% CI:2.78-26.68, p=.01. PA was more effective in attaining personal goals: Effect: 0.50, 95% CI: 0.04 - 0.96, p=.03.
Limited validity of clinical vignettes as a proxy measure of clinical practice.
PA was more effective in improving adherence to CPGs than CD. Personal feedback may have contributed to its effectiveness. Future research should address the role of the group coach.

2014Sep
Acad Med
Acad Med 2014 Sep 23. Epub 2014 Sep 23.
Ms. Gingerich is a PhD candidate, School of Health Professions Education, Maastricht University, Maastricht, Netherlands, and research associate, Northern Medical Program, University of Northern British Columbia, Prince George, British Columbia, Canada. Dr. van der Vleuten is scientific director, School of Health Professions Education, Maastricht University, Maastricht, Netherlands. Dr. Eva is acting director and senior scientist, Centre for Health Education Scholarship, University of British Columbia, Vancouver, British Columbia, Canada. Dr. Regehr is associate director of research and senior scientist, Centre for Health Education Scholarship, University of British Columbia, Vancouver, British Columbia, Canada.

Social judgment research suggests that rater unreliability in performance assessments arises from raters' differing inferences about the performer and the underlying reasons for the performance observed. These varying social judgments are not entirely idiosyncratic but, rather, tend to partition into a finite number of distinct subgroups, suggesting some "signal" in the "noise" of interrater variability. The authors investigated the proportion of variance in Mini-CEX ratings attributable to such partitions of raters' social judgments about residents.Read More


In 2012 and 2013, physicians reviewed video-recorded patient encounters for seven residents, completed a Mini-CEX, and described their social judgments of the residents. Additional participants sorted these descriptions, which were analyzed using latent partition analysis (LPA). The best-fitting set of partitions for each resident served as an independent variable in a one-way ANOVA to determine the proportion of variance explained in Mini-CEX ratings.
Forty-eight physicians rated at least one resident (34 assessed all seven). The seven sets of social judgments were sorted by 14 participants. Across residents, 2 to 5 partitions (mode: 4) provided a good LPA fit, suggesting that subgroups of raters were making similar social judgments, while different causal explanations for each resident's performance existed across subgroups. The partitions accounted for 9% to 57% of the variance in Mini-CEX ratings across residents (mean = 32%).
These findings suggest that multiple "signals" do exist within the "noise" of interrater variability in performance-based assessment. It may be valuable to understand and exploit these multiple signals rather than try to eliminate them.

2014Aug
Med Educ
Med Educ 2014 Aug;48(8):761-7
Department of Educational Development and Research, Maastricht University, Maastricht, the Netherlands.

Medical education research is thriving. In recent decades, numbers of journals and publications have increased enormously, as have the number and size of medical education meetings around the world. The aim of this paper is to shed some light on the origins of this success.Read More

My central argument is that dialogue between education practice (and its teachers) and education research (and its researchers) is indispensable.
To illustrate how I have come to this perspective, I discuss two crucial developments of personal import to myself. The first is the development of assessment theory informed by both research findings and insights emerging from implementations conducted in collaboration with teachers and learners. The second is the establishment of a department of education that includes many members from the medical domain.
Medical education is thriving because it is shaped and nourished within a community of practice of collaborating teachers, practitioners and researchers. This obviates the threat of a fissure between education research and education practice. The values of this community of practice - inclusiveness, openness, supportiveness, nurture and mentorship - are key elements for its sustainability. In pacing the development of our research in a manner that maintains this synergy, we should be mindful of the zone of proximal development of our community of practice.

2014Jul
Med Educ
Med Educ 2014 Jul;48(7):713-23
Department of Clinical Neurological Sciences, Schulich School of Medicine and Dentistry, University of Western Ontario, London, ON, Canada.

Feedback should facilitate learning, but within medical education it often fails to deliver on its promise. To better understand why feedback is challenging, we explored the unique perspectives of doctors who had also trained extensively in sport or music, aiming to: (i) distinguish the elements of the response to feedback that are determined by the individual learner from those determined by the learning culture, and (ii) understand how these elements interact in order to make recommendations for improving feedback in medical education.
Using a constructivist grounded theory approach, we conducted semi-structured interviews with 27 doctors or medical students who had high-level training and competitive or performance experience in sport (n = 15) or music (n = 12).Read More

Data were analysed iteratively using constant comparison. Key themes were identified and their relationships critically examined to derive a conceptual understanding of feedback and its impact.
We identified three essential sources of influence on the meaning that feedback assumed: the individual learner; the characteristics of the feedback, and the learning culture. Individual learner traits, such as motivation and orientation toward feedback, appeared stable across learning contexts. Similarly, certain feedback characteristics, including specificity, credibility and actionability, were valued in sport, music and medicine alike. Learning culture influenced feedback in three ways: (i) by defining expectations for teachers and teacher-learner relationships; (ii) by establishing norms for and expectations of feedback, and (iii) by directing teachers' and learners' attention toward certain dimensions of performance. Learning culture therefore neither creates motivated learners nor defines 'good feedback'; rather, it creates the conditions and opportunities that allow good feedback to occur and learners to respond.
An adequate understanding of feedback requires an integrated approach incorporating both the individual and the learning culture. Our research offers a clear direction for medicine's learning culture: normalise feedback; promote trusting teacher-learner relationships; define clear performance goals, and ensure that the goals of learners and teachers align.

2014Jun
Perspect Med Educ
Perspect Med Educ 2014 Jun;3(3):190-203
Department of Educational Development and Research, Faculty of Health, Medicine and Life Sciences, Maastricht University, PO Box 616, 6200 MD, Maastricht, The Netherlands, j.frambach@maastrichtuniversity.nl.

There is a growing need for research on culture, cultural differences and cultural effects of globalization in medical education, but these are complex phenomena to investigate. Socio-cultural activity theory seems a useful framework to study cultural complexity, because it matches current views on culture as a dynamic process situated in a social context, and has been valued in diverse fields for yielding rich understandings of complex issues and key factors involved. This paper explains how activity theory can be used in (cross-)cultural medical education research.Read More

We discuss activity theory's theoretical background and principles, and we show how these can be applied to the cultural research practice by discussing the steps involved in a cross-cultural study that we conducted, from formulating research questions to drawing conclusions. We describe how the activity system, the unit of analysis in activity theory, can serve as an organizing principle to grasp cultural complexity. We end with reflections on the theoretical and practical use of activity theory for cultural research and note that it is not a shortcut to capture cultural complexity: it is a challenge for researchers to determine the boundaries of their study and to analyze and interpret the dynamics of the activity system.

2014Jun
Adv Health Sci Educ Theory Pract
Adv Health Sci Educ Theory Pract 2014 Jun 14. Epub 2014 Jun 14.
Department of Intensive Care Medicine, Maastricht University Medical Centre+, P. Debyelaan 25, 6202 AZ, Maastricht, The Netherlands, w.van.mook@mumc.nl.

Although several examples of frameworks dealing with students' unprofessional behaviour are available, guidance on how to deal locally or regionally with dysfunctional residents is limited (Hickson et al. in Acad Med 82(11):1040-1048, 2007b; Leape and Fromson in Ann Intern Med 144(2):107-115, 2006). Any 'rules' are mostly unwritten, and often emerge by trial and error within the specialty training programme (Stern and Papadakis in N Engl J Med 355(17):1794-1799, 2006).Read More

It is nevertheless of utmost importance that objectives, rules and guidelines comparable to those existing in undergraduate training (Project Team Consilium Abeundi van Luijk in Professional behaviour: teaching, assessing and coaching students. Final report and appendices. Mosae Libris, 2005; van Mook et al. in Neth J Crit Care 16(4):162-173, 2010a) are developed for postgraduate training. And that implicit rules are made explicit. This article outlines a framework based on the lessons learned from contemporary postgraduate medical training programmes.

2014Jun
BMC Med Educ
BMC Med Educ 2014 13;14:117. Epub 2014 Jun 13.
HAN University of Applied Sciences, Department Allied Health Studies, Kapittelweg 33, 5425 EN Nijmegen, The Netherlands. marjo.maas@han.nl.

Peer Assessment (PA) in health professions education encourages students to develop a critical attitude towards their own and their peers' performance. We designed a PA task to assess students' clinical skills (including reasoning, communication, physical examination and treatment skills) in a role-play that simulated physical therapy (PT) practice. Students alternately performed in the role of PT, assessor, and patient.Read More

Oral face-to-face feedback was provided as well as written feedback and scores.This study aims to explore the impact of PA on the improvement of clinical performance of undergraduate PT students.
The PA task was analyzed and decomposed into task elements. A qualitative approach was used to explore students' perceptions of the task and the task elements. Semi-structured interviews with second year students were conducted to explore the perceived impact of these task elements on performance improvement. Students were asked to select the elements perceived valuable, to rank them from highest to lowest learning value, and to motivate their choices. Interviews were transcribed verbatim and analyzed, using a phenomenographical approach and following template analysis guidelines. A quantitative approach was used to describe the ranking results.
Quantitative analyses showed that the perceived impact on learning varied widely. Performing the clinical task in the PT role, was assigned to the first place (1), followed by receiving expert feedback (2), and observing peer performance (3). Receiving peer feedback was not perceived the most powerful task element.Qualitative analyses resulted in three emerging themes: pre-performance, true-performance, and post-performance triggers for improvement. Each theme contained three categories: learning activities, outcomes, and conditions for learning.Intended learning activities were reported, such as transferring prior learning to a new application context and unintended learning activities, such as modelling a peer's performance. Outcomes related to increased self-confidence, insight in performance standards and awareness of improvement areas. Conditions for learning referred to the quality of peer feedback.
PA may be a powerful tool to improve clinical performance, although peer feedback is not perceived the most powerful element. Peer assessors in undergraduate PT education use idiosyncratic strategies to assess their peers' performance.

2014Apr
Clin Anat
Clin Anat 2014 Apr 19;27(3):296-303. Epub 2013 Dec 19.
Department of Educational Development and Research, School of Health Professions Education, Maastricht University, Maastricht, The Netherlands; Department of Anatomy, Radboud University Medical Centre Nijmegen, Nijmegen, The Netherlands.

Eight factors are claimed to have a negative influence on anatomical knowledge of medical students: (1) teaching by nonmedically qualified teachers, (2) the absence of a core anatomy curriculum, (3) decreased use of dissection as a teaching tool, (4) lack of teaching anatomy in context, (5) integrated curricula (problem-based learning or systems-based curricula), (6) inadequate assessment of anatomical knowledge, (7) decreased anatomy teaching time, and (8) neglect of vertical integration of anatomy teaching. A recent review revealed a lack of evidence underpinning any of the claims owing to the poor quality of papers, and recommendations were made for education and research on teaching in context and the implementation of vertical integration and of assessment strategies. In this article, we will describe the alleged factors fully, revealing additional recommendations for improving anatomy education by promoting recognition for teaching in institutions, by enhancing the professional recognition of anatomists through the implementation of a national postgraduate training program, and by encouraging anatomists to participate in educational research.Read More

Consistent identification and prevention of failure for at-risk medical students is challenging, failing courses is costly to all stakeholders, and there is need for further research into duration, timing and structure of interventions to help students in difficulty.
To verify the value of a new exam two weeks into medical school as a predictor of failure, and explore the requirements for a preventative intervention.
Students who failed the two-week exam were invited to a series of large-group workshops and small-group follow-up meetings.Read More

Participants' subsequent exam performance was compared with non-participants.
About 71% of students who performed poorly in the new exam subsequently failed a course. Attendance at the workshops made no difference to short- or long-term pass rates. Attendance at more than three follow-up small group sessions significantly improved pass rates two semesters later, and was influenced by teacher experience.
Close similarity between predictor task and target task is important for accurate prediction of failure. Consideration should be given to dose effect and class size in the prevention of failure of at-risk students, and we recommend a systemic approach to intervention/remediation programmes, involving a whole semester of mandatory, weekly small group meetings with experienced teachers.

2013Dec
Behav Res Methods
Behav Res Methods 2013 Dec;45(4):1058-72
Department of Educational Development and Research, Maastricht University, P. O. Box 616, 6200 MD, Maastricht, The Netherlands, jimmie.leppink@maastrichtuniversity.nl.

According to cognitive load theory, instructions can impose three types of cognitive load on the learner: intrinsic load, extraneous load, and germane load. Proper measurement of the different types of cognitive load can help us understand why the effectiveness and efficiency of learning environments may differ as a function of instructional formats and learner characteristics. In this article, we present a ten-item instrument for the measurement of the three types of cognitive load.Read More

Principal component analysis on data from a lecture in statistics for PhD students (n = 56) in psychology and health sciences revealed a three-component solution, consistent with the types of load that the different items were intended to measure. This solution was confirmed by a confirmatory factor analysis of data from three lectures in statistics for different cohorts of bachelor students in the social and health sciences (ns = 171, 136, and 148), and received further support from a randomized experiment with university freshmen in the health sciences (n = 58).

2013Dec
Med Educ
Med Educ 2013 Dec;47(12):1164-74
Educational Development and Research, Maastricht University, Maastricht, the Netherlands.

Although work-based assessments (WBA) may come closest to assessing habitual performance, their use for summative purposes is not undisputed. Most criticism of WBA stems from approaches to validity consistent with the quantitative psychometric framework. However, there is increasing research evidence that indicates that the assumptions underlying the predictive, deterministic framework of psychometrics may no longer hold.Read More

In this discussion paper we argue that meaningfulness and appropriateness of current validity evidence can be called into question and that we need alternative strategies to assessment and validity inquiry that build on current theories of learning and performance in complex and dynamic workplace settings.
Drawing from research in various professional fields we outline key issues within the mechanisms of learning, competence and performance in the context of complex social environments and illustrate their relevance to WBA. In reviewing recent socio-cultural learning theory and research on performance and performance interpretations in work settings, we demonstrate that learning, competence (as inferred from performance) as well as performance interpretations are to be seen as inherently contextualised, and can only be under-stood 'in situ'. Assessment in the context of work settings may, therefore, be more usefully viewed as a socially situated interpretive act.
We propose constructivist-interpretivist approaches towards WBA in order to capture and understand contextualised learning and performance in work settings. Theoretical assumptions underlying interpretivist assessment approaches call for a validity theory that provides the theoretical framework and conceptual tools to guide the validation process in the qualitative assessment inquiry. Basic principles of rigour specific to qualitative research have been established, and they can and should be used to determine validity in interpretivist assessment approaches. If used properly, these strategies generate trustworthy evidence that is needed to develop the validity argument in WBA, allowing for in-depth and meaningful information about professional competence.

E-learning resources, such as virtual patients (VPs), can be more effective when they are integrated in the curriculum. To gain insights that can inform guidelines for the curricular integration of VPs, we explored students' perceptions of scenarios with integrated and non-integrated VPs aimed at promoting clinical reasoning skills.
During their paediatric clerkship, 116 fifth-year medical students were given at least ten VPs embedded in eight integrated scenarios and as non-integrated add-ons.Read More

The scenarios differed in the sequencing and alignment of VPs and related educational activities, tutor involvement, number of VPs, relevance to assessment and involvement of real patients. We sought students' perceptions on the VP scenarios in focus group interviews with eight groups of 4-7 randomly selected students (n = 39). The interviews were recorded, transcribed and analysed qualitatively.
The analysis resulted in six themes reflecting students' perceptions of important features for effective curricular integration of VPs: (i) continuous and stable online access, (ii) increasing complexity, adapted to students' knowledge, (iii) VP-related workload offset by elimination of other activities, (iv) optimal sequencing (e.g.: lecture--1 to 2 VP(s)--tutor-led small group discussion--real patient) and (V) optimal alignment of VPs and educational activities, (vi) inclusion of VP topics in assessment.
The themes appear to offer starting points for the development of a framework to guide the curricular integration of VPs. Their impact needs to be confirmed by studies using quantitative controlled designs.

2013Nov

Despite acknowledgement that the Canadian Medical Educational Directives for Specialists (CanMEDS) framework covers the relevant competencies of physicians, many educators and medical professionals struggle to translate the CanMEDS roles into comprehensive training programmes for specific specialties.
To gain insight into the applicability of the CanMEDS framework to guide the design of educational programmes for specific specialties by exploring stakeholders' perceptions of specialty specific competencies and examining differences between those competencies and the CanMEDS framework.
This case study is a sequel to a study among ObsGyn specialists.Read More

It explores the perspectives of patients, midwives, nurses, general practitioners, and hospital boards on gynaecological competencies and compares these with the CanMEDS framework.
Clinical expertise, reflective practice, collaboration, a holistic view, and involvement in practice management were perceived to be important competencies for gynaecological practice. Although all the competencies were covered by the CanMEDS framework, there were some mismatches between stakeholders' perceptions of the importance of some competencies and their position in the framework.
The CanMEDS framework appears to offer relevant building blocks for specialty specific postgraduate training, which should be combined with the results of an exploration of specialty specific competencies to arrive at a postgraduate curriculum that is in alignment with professional practice.

2013Nov
BMC Med Educ
BMC Med Educ 2013 11;13:123. Epub 2013 Sep 11.
Quality Improvement in Veterinary Education, Faculty of Veterinary Medicine, Utrecht University, Utrecht, The Netherlands. G.J.Bok@uu.nl.

In competency-based medical education emphasis has shifted towards outcomes, capabilities, and learner-centeredness. Together with a focus on sustained evidence of professional competence this calls for new methods of teaching and assessment. Recently, medical educators advocated the use of a holistic, programmatic approach towards assessment.Read More

Besides maximum facilitation of learning it should improve the validity and reliability of measurements and documentation of competence development. We explored how, in a competency-based curriculum, current theories on programmatic assessment interacted with educational practice.
In a development study including evaluation, we investigated the implementation of a theory-based programme of assessment. Between April 2011 and May 2012 quantitative evaluation data were collected and used to guide group interviews that explored the experiences of students and clinical supervisors with the assessment programme. We coded the transcripts and emerging topics were organised into a list of lessons learned.
The programme mainly focuses on the integration of learning and assessment by motivating and supporting students to seek and accumulate feedback. The assessment instruments were aligned to cover predefined competencies to enable aggregation of information in a structured and meaningful way. Assessments that were designed as formative learning experiences were increasingly perceived as summative by students. Peer feedback was experienced as a valuable method for formative feedback. Social interaction and external guidance seemed to be of crucial importance to scaffold self-directed learning. Aggregating data from individual assessments into a holistic portfolio judgement required expertise and extensive training and supervision of judges.
A programme of assessment with low-stakes assessments providing simultaneously formative feedback and input for summative decisions proved not easy to implement. Careful preparation and guidance of the implementation process was crucial. Assessment for learning requires meaningful feedback with each assessment. Special attention should be paid to the quality of feedback at individual assessment moments. Comprehensive attention for faculty development and training for students is essential for the successful implementation of an assessment programme.

2013Nov
BMC Med Educ
BMC Med Educ 2013 27;13:132. Epub 2013 Sep 27.
Study Skills Centre, Room 202, Main Arts Library, College Road, Bangor University, Bangor, Gwynedd LL57 2DF, UK. kalwinston@gmail.com.

Previous work has shown that a programme that draws on a blend of theories makes a positive difference to outcomes for students who fail and repeat their first semester at medical school. Exploration of student and teacher perspectives revealed that remediation of struggling medical students can be achieved through a cognitive apprenticeship within a small community of inquiry. This community needs expert teachers capable of performing a unique combination of roles (facilitator, nurturing mentor, disciplinarian, diagnostician and role model), with high levels of teaching presence and practical wisdom.Read More

Yet, despite participants' convergent opinions on the elements of effective remediation, significant differences were found between outcomes of students working with experienced and inexperienced teachers. The current study explores the actual practice of teachers on this remediation course, aiming to exemplify elements of our theory of remediation and explore differences between teachers.
Since it is in the classroom context that the interactions that constitute the complex process of remediation emerge, this practice-based research has focused on direct observation of classroom teaching. Nineteen hours of small group sessions were recorded and transcribed. Drawing on ethnography and sociocultural discourse analysis, selected samples of talk-in-context demonstrate how the various elements of remediation play out in practice, highlighting aspects that are most effective, and identifying differences between experienced and novice teachers.
Long-term student outcomes are strongly correlated to teacher experience (r, 0.81). Compared to inexperienced teachers, experienced teachers provide more challenging, disruptive facilitation, and take a dialogic stance that encourages more collaborative group dynamics. They are more expert at diagnosing cognitive errors, provide frequent metacognitive time-outs and make explicit links across the curriculum.
Remediation is effective in small groups where dialogue is used for collaborative knowledge construction and social regulation. This requires facilitation by experienced teachers who attend to details of both content and process, and use timely interventions to foster curiosity and the will to learn. These teachers should actively challenge students' language use, logical inconsistencies and uncertainties, problematize their assumptions, and provide a metacognitive regulatory voice that can generate attitudinal shifts and nurture the development of independent critical thinkers.

2013Nov
Teach Learn Med
Teach Learn Med 2013 ;25 Suppl 1:S17-25
a National Board of Medical Examiners , Philadelphia , Pennsylvania , USA.
2013Oct
Adv Health Sci Educ Theory Pract
Adv Health Sci Educ Theory Pract 2013 Oct 28;18(4):659-72. Epub 2012 Sep 28.
Department of Educational Research and Development, Maastricht University, P.O. Box 616, 6200 MD, Maastricht, The Netherlands, f.vanblankenstein@gmail.com.

Although elaboration has been investigated frequently, there is little evidence for the beneficial effect of elaboration in problem-based learning. A controlled experiment tested the effect of elaboration during problem-based discussion on recall. Sixty-seven students observed a video-recorded, problem-based discussion.Read More

In one experimental condition, a tutor in the video encouraged participants to elaborate by asking elaborative questions. In a second condition, the tutor asked superficial questions. After the discussion, all participants studied a text with relevant new information. Elaborative questions had no significant effect on recall of idea units from the text, p = .39, η(2) = .01. High-ability students outperformed low-ability students, p = .04, η(2) = .07, but this effect did not interact with the experimental treatment, p = .22, η(2) = .02. Suggestions for further research are presented.

2013Oct
J Res Med Sci
J Res Med Sci 2013 Oct;18(10):887-91
Department of Educational Development and Research, School of Health Professions Education, Maastricht University, Maastricht, The Netherlands.

One of the methods used for standard setting is the borderline regression method (BRM). This study aims to assess the reliability of BRM when the pass-fail standard in an objective structured clinical examination (OSCE) was calculated by averaging the BRM standards obtained for each station separately.
In nine stations of the OSCE with direct observation the examiners gave each student a checklist score and a global score.Read More

Using a linear regression model for each station, we calculated the checklist score cut-off on the regression equation for the global scale cut-off set at 2. The OSCE pass-fail standard was defined as the average of all station's standard. To determine the reliability, the root mean square error (RMSE) was calculated. The R (2) coefficient and the inter-grade discrimination were calculated to assess the quality of OSCE.
The mean total test score was 60.78. The OSCE pass-fail standard and its RMSE were 47.37 and 0.55, respectively. The R (2) coefficients ranged from 0.44 to 0.79. The inter-grade discrimination score varied greatly among stations.
The RMSE of the standard was very small indicating that BRM is a reliable method of setting standard for OSCE, which has the advantage of providing data for quality assurance.

2013Sep
Acad Med
Acad Med 2013 Sep;88(9):1346-56
Department of Educational Development and Research, Faculty of Health, Medicine and Life Sciences, Maastricht University, Maastricht, the Netherlands. mjippes@gmail.com

Because successful change implementation depends on organizational readiness for change, the authors developed and assessed the validity of a questionnaire, based on a theoretical model of organizational readiness for change, designed to measure, specifically, a medical school's organizational readiness for curriculum change (MORC).
In 2012, a panel of medical education experts judged and adapted a preliminary MORC questionnaire through a modified Delphi procedure. The authors administered the resulting questionnaire to medical school faculty involved in curriculum change and tested the psychometric properties using exploratory and confirmatory factor analysis, and generalizability analysis.Read More


The mean relevance score of the Delphi panel (n = 19) reached 4.2 on a five-point Likert-type scale (1 = not relevant and 5 = highly relevant) in the second round, meeting predefined criteria for completing the Delphi procedure. Faculty (n = 991) from 131 medical schools in 56 countries completed MORC. Exploratory factor analysis yielded three underlying factors-motivation, capability, and external pressure-in 12 subscales with 53 items. The scale structure suggested by exploratory factor analysis was confirmed by confirmatory factor analysis. Cronbach alpha ranged from 0.67 to 0.92 for the subscales. Generalizability analysis showed that the MORC results of 5 to 16 faculty members can reliably evaluate a school's organizational readiness for change.
MORC is a valid, reliable questionnaire for measuring organizational readiness for curriculum change in medical schools. It can identify which elements in a change process require special attention so as to increase the chance of successful implementation.

2013Aug
Med Teach
Med Teach 2013 Aug 30;35(8):661-70. Epub 2013 Apr 30.
Department of Educational Development and Research FHML, Maastricht University, POB 616, NL-6200 MD Maastricht, The Netherlands. mjippes@gmail.com

Earlier studies suggested national culture to be a potential barrier to curriculum reform in medical schools. In particular, Hofstede's cultural dimension 'uncertainty avoidance' had a significant negative relationship with the implementation rate of integrated curricula.
However, some schools succeeded to adopt curriculum changes despite their country's strong uncertainty avoidance.Read More

This raised the question: 'How did those schools overcome the barrier of uncertainty avoidance?'
Austria offered the combination of a high uncertainty avoidance score and integrated curricula in all its medical schools. Twenty-seven key change agents in four medical universities were interviewed and transcripts analysed using thematic cross-case analysis.
Initially, strict national laws and limited autonomy of schools inhibited innovation and fostered an 'excuse culture': 'It's not our fault. It is the ministry's'. A new law increasing university autonomy stimulated reforms. However, just this law would have been insufficient as many faculty still sought to avoid change. A strong need for change, supportive and continuous leadership, and visionary change agents were also deemed essential.
In societies with strong uncertainty avoidance strict legislation may enforce resistance to curriculum change. In those countries opposition by faculty can be overcome if national legislation encourages change, provided additional internal factors support the change process.

2013Aug
Med Educ
Med Educ 2013 Aug;47(8):842-50
Department of Clinic Neurological Sciences, Schulich School of Medicine and Dentistry, University of Western Ontario, London, Ontario, Canada. chris.watling@schulich.uwo.ca

Research on medical learning has tended to focus on the individual learner, but a sufficient understanding of the learning process requires that attention also be paid to the essential influence of the cultural context within which learning takes place. In this study, we undertook a comparative examination of two learning cultures - those of music and medicine - in order to unearth assumptions about learning that are taken for granted within the medical culture.
We used a constructivist grounded theory approach to explore experiences of learning within the two cultures.Read More

We conducted nine focus groups (two with medical students, three with residents, four with music students) and four individual interviews (with one clinician-educator, one music educator and two doctor-musicians), for a total of 37 participants. Analysis occurred alongside and informed data collection. Themes were identified iteratively using constant comparisons.
Cultural perspectives diverged in terms of where learning should occur, what learning outcomes are desired, and how learning is best facilitated. Whereas medicine valued learning by doing, music valued learning by lesson. Whereas medical learners aimed for competence, music students aimed instead for ever-better performance. Whereas medical learners valued their teachers for their clinical skills more than for their teaching abilities, the opposite was true in music, in which teachers' instructional skills were paramount. Self-assessment challenged learners in both cultures, but medical learners viewed self-assessment as a skill they could develop, whereas music students recognised that external feedback would always be required.
This comparative analysis reveals that medicine and music make culturally distinct assumptions about teaching and learning. The contrasts between the two cultures illuminate potential vulnerabilities in the medical learning culture, including the risks inherent in its competence-focused approach and the constraints it places on its own teachers. By highlighting these vulnerabilities, we provide a stimulus for reimagining and renewing medicine's educational practices.

2013Jul
Med Educ
Med Educ 2013 Jul;47(7):734-44
School of Medicine, Keele University, Keele, UK. c.j.harrison@keele.ac.uk

There is little research into how to deliver summative assessment student feedback effectively. The main aims of this study were to clarify how students engage with feedback in this context and to explore the roles of learning-related characteristics and previous and current performance.
A website was developed to deliver feedback about the objective structural clinical examination (OSCE) in various formats: station by station or on skills across stations.Read More

In total, 138 students (in the third year out of five) completed a questionnaire about goal orientation, motivation, self-efficacy, control of learning beliefs and attitudes to feedback. Individual website usage was analysed over an 8-week period. Latent class analyses were used to identify profiles of students, based on their use of different aspects of the feedback website. Differences in learning-related student characteristics between profiles were assessed using analyses of variance (anovas). Individual website usage was related to OSCE performance.
In total, 132 students (95.7%) viewed the website. The number of pages viewed ranged from two to 377 (median 102). Fifty per cent of students engaged comprehensively with the feedback, 27% used it in a minimal manner, whereas a further 23% used it in a more selective way. Students who were comprehensive users of the website scored higher on the value of feedback scale, whereas students who were minimal users scored higher on extrinsic motivation. Higher performing students viewed significantly more web pages showing comparisons with peers than weaker students did. Students who just passed the assessment made least use of the feedback.
Higher performing students appeared to use the feedback more for positive affirmation than for diagnostic information. Those arguably most in need engaged least. We need to construct feedback after summative assessment in a way that will more effectively engage those students who need the most help.

2013Jun

Danish and Dutch new consultants' perceptions regarding the transition to consultant were compared to gain insight into this period, particularly the influence of contextual factors concerning the organisation of specialty training and health care therein. Preparation for medical and generic competencies, perceived intensity and burnout were compared. Additionally, effects of differences in working conditions and cultural dimensions were explored.Read More


All consultants registered in the Netherlands in 2007-2009 (n = 2643) and Denmark in 2007-2010 (n = 1336) received in June 2010 and April 2011, respectively, a survey about their preparation for medical and generic competencies, perceived intensity and burnout. Power analysis resulted in required sample sizes of 542. Descriptive statistics and independent t-tests were used for analysis.
Data were available of 792 new consultants in the Netherlands and 677 Danish new consultants. Compared to their Dutch counterparts, Danish consultants perceived specialty training and the transition less intensely, reported higher levels of preparation for generic competencies and scored lower on burnout.
The importance of contextual aspects in the transition is underscored and shows that Denmark appears to succeed better in aligning training with practice. Regulations regarding working hours and progressive independence of trainees appear to facilitate the transition.

2013Jun
Med Educ
Med Educ 2013 Jun;47(6):585-94
Department of Clinical Neurological Sciences, Schulich School of Medicine and Dentistry, University of Western Ontario, London, Ontario, Canada. chris.watling@schulich.uwo.ca

Although feedback is widely considered essential to learning, its actual influence on learners is variable. Research on responsivity to feedback has tended to focus on individual rather than social or cultural influences on learning. In this study, we explored how feedback is handled within different professional cultures, and how the characteristics and values of a profession shape learners' responses to feedback.Read More


Using a constructivist grounded theory approach, we conducted 12 focus groups and nine individual interviews (with a total of 50 participants) across three cultures of professional training in, respectively, music, teacher training and medicine. Constant comparative analysis for recurring themes was conducted iteratively.
Each of the three professional cultures created a distinct context for learning that influenced how feedback was handled. Despite these contextual differences, credibility and constructiveness emerged as critical constants, identified by learners across cultures as essential for feedback to be perceived as meaningful. However, the definitions of credibility and constructiveness were distinct to each professional culture and the cultures varied considerably in how effectively they supported the occurrence of feedback with these critical characteristics.
Professions define credibility and constructiveness in culturally specific ways and create contexts for learning that may either facilitate or constrain the provision of meaningful feedback. Comparison with other professional cultures may offer strategies for creating a productive feedback culture within medical education.

2013Apr
Anat Sci Educ
Anat Sci Educ 2013 May-Jun;6(3):149-56. Epub 2012 Oct 26.
Faculty of Medicine, Department of Anatomy, Umm Al-Qura University, Makkah, Kingdom of Saudi Arabia. hsshaibah@uqu.edu.sa

Traditionally, an anatomy practical examination is conducted using a free response format (FRF). However, this format is resource-intensive, as it requires a relatively large time investment from anatomy course faculty in preparation and grading. Thus, several interventions have been reported where the response format was changed to a selected response format (SRF).Read More

However, validity evidence from those interventions has not proved entirely adequate for the practical anatomy examination, and thus, further investigation was required. In this study, the validity evidence of SRF was examined using multiple choice questions (MCQs) constructed according to different levels of Bloom's taxonomy in comparison with the traditional free response format. A group of 100 medical students registered in a gross anatomy course volunteered to be enrolled in this study. The experimental MCQ examinations were part of graded midterm and final steeplechase practical examination. Volunteer students were instructed to complete the practical examinations twice, once in each of two separate examination rooms. The two separate examinations consisted of a traditional free response format and MCQ format. Scores from the two examinations (FRF and MCQ) displayed a strong correlation, even with higher level Bloom's taxonomy questions. In conclusion, the results of this study provide empirical evidence that the SRF (MCQ) response format is a valid method and can be used as an alternative to the traditional FRF steeplechase examination.

2013Apr
Med Educ
Med Educ 2013 Apr;47(4):408-16
Department of Education, St Lucas Andreas Hospital, Amsterdam, the Netherlands. mchwesterman@gmail.com

The supervision of specialty registrars during on-call shifts is essential to ensure the quality of both health care and medical education, but has been identified as a major novelty and stressor for new consultants in the transition from specialty training. There is a paucity of research on how consultants deal with their new supervisory roles and which factors influence this process. These issues are addressed in a prospective study designed to gather insights that can inform measures to ensure the provision of high-quality supervision and specialty training.Read More


A longitudinal qualitative study was performed in the Netherlands. Semi-structured interviews were conducted with new consultants. The study was guided by an interpretative phenomenological approach until saturation was reached. At 3-month intervals between July 2011 and March 2012, eight novice consultants in internal medicine were interviewed three times each about their supervisory role while on call. Interviews focused on their preparation for the role in training, the actions they took to master the role, and their progression over time.
Three interrelated domains of relevant factors emerged from the data: preparedness; personal characteristics, and contextual characteristics. Preparedness referred to the extent to which new consultants were prepared by training to take full responsibility for registrars' actions while supervising them from a distance. Personal characteristics, such as coping strategies and views on supervision, guided consultants' development as supervisors. Essential to this process were contextual characteristics, especially those concerning the extent to which the consultant knew the registrar, was familiar with departmental procedures, and had access to support from colleagues.
New consultants should be prepared for their supervisory role by training and by being given a proper introduction to their workplace. The former requires progressive independence and exposure to supervisory tasks during specialty training; the latter requires an induction programme to enable new consultants to familiarise themselves with the departmental environment and the registrars they will be supervising.

2013Apr

Insight into the transition from specialist registrar to hospital consultant is needed to better align specialty training with starting as a consultant and to facilitate this transition.
This study investigates whether preparedness regarding medical and generic competencies, perceived intensity, and social support are associated with burnout among new consultants.
A population-based study among all 2643 new consultants in the Netherlands (all specialties) was conducted in June 2010.Read More

A questionnaire covering preparedness for practice, intensity of the transition, social support, and burnout was used. Structural equation modelling was used for statistical analysis.
Data from a third of the population were available (32% n = 840) (43% male/57% female). Preparation in generic competencies received lower ratings than in medical competencies. A total of 10% met the criteria for burnout and 18% scored high on the emotional exhaustion subscale. Perceived lack of preparation in generic competencies correlated with burnout (r = 0.15, p < 0.001). No such relation was found for medical competencies. Furthermore, social support protected against burnout.
These findings illustrate the relevance of generic competencies for new hospital consultants. Furthermore, social support facilitates this intense and stressful stage within the medical career.

2013Apr
Med Teach
Med Teach 2013 ;35 Suppl 1:S31-8
College of Medicine, King Saud bin Abdulaziz University for Health Sciences, P.O. Box 57374, Riyadh 11574, Kingdom of Saudi Arabia. halkadri@gmail.com

We have performed this research to assess the effect of work-place based assessment (WBA) practice on medical students' learning approaches.
The research was conducted at the King Saud bin Abdulaziz University for Health Sciences, College of Medicine from 1 March to 31 July 2012. We conducted a qualitative, phenomenological research utilizing semi-structured individual interviews with medical students exposed to WBA.Read More

The audio-taped interviews were transcribed verbatim, analyzed, and themes were identified. We preformed investigators' triangulation, member checking with clinical supervisors and we triangulated the data with a similar research performed prior to the implementation of WBA.
WBA results in variable learning approaches. Based on several affecting factors; clinical supervisors, faculty-given feedback, and assessment function, students may swing between surface, deep and effort and achievement learning approaches. Students' and supervisors' orientations on the process of WBA, utilization of peer feedback and formative rather than summative assessment facilitate successful implementation of WBA and lead to students' deeper approaches to learning. Interestingly, students and their supervisors have contradicting perceptions to WBA.
A change in culture to unify students' and supervisors' perceptions of WBA, more accommodation of formative assessment, and feedback may result in students' deeper approach to learning.

2013Mar
Anat Sci Educ
Anat Sci Educ 2013 Mar-Apr;6(2):114-24. Epub 2012 Aug 15.
Department of Educational Development and Research, School of Health Professions Education, Maastricht University, Maastricht, The Netherlands. e.bergman@anat.umcn.nl

Anatomy education often consists of a combination of lectures and laboratory sessions, the latter frequently including surface anatomy. Studying surface anatomy enables students to elaborate on their knowledge of the cadaver's static anatomy by enabling the visualization of structures, especially those of the musculoskeletal system, move and function in a living human being. A recent development in teaching methods for surface anatomy is body painting, which several studies suggest increases both student motivation and knowledge acquisition.Read More

This article focuses on a teaching approach and is a translational contribution to existing literature. In line with best evidence medical education, the aim of this article is twofold: to briefly inform teachers about constructivist learning theory and elaborate on the principles of constructive, collaborative, contextual, and self-directed learning; and to provide teachers with an example of how to implement these learning principles to change the approach to teaching surface anatomy. Student evaluations of this new approach demonstrate that the application of these learning principles leads to higher student satisfaction. However, research suggests that even better results could be achieved by further adjustments in the application of contextual and self-directed learning principles. Successful implementation and guidance of peer physical examination is crucial for the described approach, but research shows that other options, like using life models, seem to work equally well. Future research on surface anatomy should focus on increasing the students' ability to apply anatomical knowledge and defining the setting in which certain teaching methods and approaches have a positive effect.

2013Mar
Med Educ
Med Educ 2013 Mar;47(3):271-81
Department of Education, St Lucas Andreas Hospital, Amsterdam, the Netherlands. joanne.fokkema@gmail.com

Many studies have examined how educational innovations in postgraduate medical education (PGME) impact on teaching and learning, but little is known about effects in the clinical workplace outside the strictly education-related domain. Insights into the full scope of effects may facilitate the implementation and acceptance of innovations because expectations can be made more realistic, and difficulties and pitfalls anticipated. Using workplace-based assessment (WBA) as a reference case, this study aimed to determine which types of effect are perceived by users of innovations in PGME.Read More


Focusing on WBA as a recent instance of innovation in PGME, we conducted semi-structured interviews to explore perceptions of the effects of WBA in a purposive sample of Dutch trainees and (lead) consultants in surgical and non-surgical specialties. Interviews conducted in 2011 with 17 participants were analysed thematically using template analysis. To support the exploration of effects outside the domain of education, the study design was informed by theory on the diffusion of innovations.
Six domains of effects of WBA were identified: sentiments (affinity with the innovation and emotions); dealing with the innovation; specialty training; teaching and learning; workload and tasks, and patient care. Users' affinity with WBA partly determined its effects on teaching and learning. Organisational support and the match between the innovation and routine practice were considered important to minimise additional workload and ensure that WBA was used for relevant rather than easily assessable training activities. Dealing with WBA stimulated attention for specialty training and placed specialty training on the agenda of clinical departments.
These outcomes are in line with theoretical notions regarding innovations in general and may be helpful in the implementation of other innovations in PGME. Given the substantial effects of innovations outside the strictly education-related domain, individuals designing and implementing innovations should consider all potential effects, including those identified in this study.

2013Mar
Med Educ
Med Educ 2013 Mar;47(3):282-91
Quality Improvement in Veterinary Education, Faculty of Veterinary Medicine, Utrecht University, Utrecht, the Netherlands. g.j.bok@uu.nl

Why and how do students seek feedback on their performance in the clinical workplace and which factors influence this? These questions have remained largely unanswered in research into workplace learning during clinical clerkships. Research on feedback has focused mainly on feedback providers. Whether and how feedback recipients actively seek feedback are under-examined issues.Read More

Research in organisational psychology has proposed a mechanism whereby feedback seeking is influenced by motives and goal orientation mediated by the perceived costs and benefits of feedback. Building on a recently published model of resident doctors' feedback-seeking behaviour, we conducted a qualitative study to explore students' feedback-seeking behaviours in the clinical workplace.
Between April and June 2011, we conducted semi-structured face-to-face interviews with veterinary medicine students in Years 5 and 6 about their feedback-seeking behaviour during clinical clerkships. In the interviews, 14 students were asked about their goals and motives for seeking feedback, the characteristics of their feedback-seeking behaviour and factors influencing that behaviour. Using template analysis, we coded the interview transcripts and iteratively reduced and displayed the data until agreement on the final template was reached.
The students described personal and interpersonal factors to explain their reasons for seeking feedback. The factors related to intentions and the characteristics of the feedback provider, and the relationship between the feedback seeker and provider. Motives relating to image and ego, particularly when students thought that feedback might have a positive effect on image and ego, influenced feedback-seeking behaviour and could induce specific behaviours related to students' orientation towards particular sources of feedback, their orientation towards particular topics for and timing of feedback, and the frequency and method of feedback-seeking behaviour.
This study shows that during clinical clerkships, students actively seek feedback according to personal and interpersonal factors. Perceived costs and benefits influenced this active feedback-seeking behaviour. These results may contribute towards the optimising and developing of meaningful educational opportunities during clerkships.

2013Mar
Perspect Med Educ
Perspect Med Educ 2013 Mar 15. Epub 2013 Mar 15.
Department of Primary Care and Community Care, Radboud University Nijmegen Medical Centre, PO Box 9101, Huispostnummer 117 6500 HB, Nijmegen, the Netherlands, e.pelgrim@elg.umcn.nl.
2013Feb
Minim Invasive Ther Allied Technol
Minim Invasive Ther Allied Technol 2013 Feb 10;22(1):26-32. Epub 2012 May 10.
Catharina Hospital Eindhoven, Eindhoven, the Netherlands.

There is growing pressure from the government and the public to define proficiency standards for surgical skills. Aim of this study was to estimate the reliability of the Program for Laparoscopic Urological Skills (PLUS) assessment and to set a certification standard for second-year urological residents.
Fifty participants were assessed on performance time and performance quality to investigate the reliability of the PLUS assessment.Read More

Generalisability coefficient of 0.8, on a scale of 0 to 1.0, was considered to indicate good reliability for assessment purposes. Pass/fail standards were based on laparoscopic experience: Novices, intermediates, and experts (>100 procedures). The pass/fail standards were investigated for the PLUS performances of 33 second-year urological residents.
Fifteen novices, twenty-three intermediates and twelve experts were included. An inter-trial reliability of >0.80 was reached with two trials for each task. Inter-rater reliability of the quality measurements was 0.79 for two judges. Pass/fail scores were determined for the novice/intermediate boundary and the intermediate/expert boundary. Pass rates for second-year residents were 63.64% and 9.09%, respectively.
The PLUS assessment is reliable for setting a certification standard for second-year urological residents that serves as a starting point for residents to proceed to the next level of laparoscopic competency.

2013Feb
Patient Educ Couns
Patient Educ Couns 2013 Feb 12;90(2):184-92. Epub 2012 Jul 12.
Maastricht University, Maastricht, The Netherlands. v.vdeertwegh@maastrichtuniversity.nl

In order to reduce the inconsistencies of findings and the apparent low transfer of communication skills from training to medical practice, this narrative review identifies some main gaps in research on medical communication skills training and presents insights from theories on learning and transfer to broaden the view for future research.
Relevant literature was identified using Pubmed, GoogleScholar, Cochrane database, and Web of Science; and analyzed using an iterative procedure.
Research findings on the effectiveness of medical communication training still show inconsistencies and variability.Read More

Contemporary theories on learning based on a constructivist paradigm offer the following insights: acquisition of knowledge and skills should be viewed as an ongoing process of exchange between the learner and his environment, so called lifelong learning. This process can neither be atomized nor separated from the context in which it occurs. Four contemporary approaches are presented as examples.
The following shift in focus for future research is proposed: beyond isolated single factor effectiveness studies toward constructivist, non-reductionistic studies integrating the context.
Future research should investigate how constructivist approaches can be used in the medical context to increase effective learning and transition of communication skills.

2013Feb
Patient Educ Couns
Patient Educ Couns 2013 Feb 29;90(2):213-9. Epub 2012 Oct 29.
CAHPRI School for Public Health and Primary Care Research, Maastricht University, Maastricht, Netherlands. Wemke.Veldhuijzen@maastrichtuniversity.nl

To explore characteristics of written communication guidelines that enhance the success of training aimed at the application of the recommendations in the guidelines.
Seven mixed focus groups were held consisting of communication skill teachers and communication skill learners and three groups with only learners. Analysis was done in line with principles of grounded theory.Read More


Five key attributes of guidelines for communication skill training were identified: complexity, level of detail, format and organization, type of information, and trustworthiness/validity. The desired use of these attributes is related to specific educational purposes and learners' expertise. The low complexity of current communication guidelines is appreciated, but seems ad odds with the wish for more valid communication guidelines.
Which guideline characteristics are preferred by users depends on the expertise of the learners and the educational purpose of the guideline.
Communication guidelines can be improved by modifying the key attributes in line with specific educational functions and learner expertise. For example: the communication guidelines used in GP training in the Netherlands, seem to offer an oversimplified model of doctor patient communication. This model may be suited for undergraduate learning, but does not meet the validity demands of physicians in training.

2012Nov
Med Teach
Med Teach 2012 4;34(11):e732-42. Epub 2012 Jun 4.
Ross University School of Medicine, PO Box 266, Roseau, Commonwealth of Dominica. kwinston@rossmed.edu.dm

Previous work identified complex ingredients of a remediation programme for at-risk medical students: mandatory, stable, facilitated small groups promote both cognitive and affective developments, with improved self-regulation, metacognition and reflection resulting in significant performance gains.
We explore the teachers' role in this intervention, aiming to expand and deepen understanding of remediation methods in medical education.
Extensive qualitative data from student surveys and in-depth teacher interviews, along with quantitative student performance data, produced a rich description of remediation processes.Read More


Remediation should support emotional needs and foster cognitive and metacognitive skills for self-regulation and critical thinking. Teachers of remediation need to motivate, critique, challenge and advise their learners, applying teaching and contextual expertise in a constructivist, student-centred environment that fosters curiosity and joy for learning. Teachers of remediation can mediate these processes through embodiment of five core roles: facilitator, nurturing mentor, disciplinarian, diagnostician and modeller of desired skills, attitudes and behaviours.
Remediation of struggling medical students can be achieved through a cognitive apprenticeship within a small community of inquiry that motivates and challenges the students. This community needs teachers capable of performing a unique combination of roles that demands high levels of teaching presence and practical wisdom.

2012Nov
BMC Med Educ
BMC Med Educ 2012 20;12:79. Epub 2012 Aug 20.
Department of Medical Education, MMC Academy, Máxima Medical Center, De Run 4600, 5500 MB, Veldhoven, the Netherlands. W.Koops@student.maastrichtuniversity.nl

Medical students in clerkship are continuously confronted with real and relevant patient problems. To support clinical problem solving skills, students perform a Critical Appraisal of a Topic (CAT) task, often resulting in a paper. Because such a paper may contain errors, students could profit from discussion with peers, leading to paper revision.Read More

Active peer discussion by a Computer Supported Collaborative Learning (CSCL) environment show positive medical students perceptions on subjective knowledge improvement. High students' activity during discussions in a CSCL environment demonstrated higher task-focussed discussion reflecting higher levels of knowledge construction. However, it remains unclear whether high discussion activity influences students' decisions revise their CAT paper. The aim of this research is to examine whether students who revise their critical appraisal papers after discussion in a CSCL environment show more task-focussed activity and discuss more intensively on critical appraisal topics than students who do not revise their papers.
Forty-seven medical students, stratified in subgroups, participated in a structured asynchronous online discussion of individual written CAT papers on self-selected clinical problems. The discussion was structured by three critical appraisal topics. After the discussion, the students could revise their paper. For analysis purposes, all students' postings were blinded and analysed by the investigator, unaware of students characteristics and whether or not the paper was revised. Postings were counted and analysed by an independent rater, Postings were assigned into outside activity, non-task-focussed activity or task-focussed activity. Additionally, postings were assigned to one of the three critical appraisal topics. Analysis results were compared by revised and unrevised papers.
Twenty-four papers (51.6%) were revised after the online discussion. The discussions of the revised papers showed significantly higher numbers of postings, more task-focussed activities, and more postings about the two critical appraisal topics: "appraisal of the selected article(s)", and "relevant conclusion regarding the clinical problem".
A CSCL environment can support medical students in the execution and critical appraisal of authentic tasks in the clinical workplace. Revision of CAT papers appears to be related to discussions activity, more specifically reflecting high task-focussed activity of critical appraisal topics.

2012Nov
BMC Med Educ
BMC Med Educ 2012 18;12:97. Epub 2012 Oct 18.
Department of Primary Care and Community Care, Radboud University Nijmegen Medical Centre, Postbus 9101 Huispostnummer 117, Nijmegen HB 6500, The Netherlands. e.pelgrim@elg.umcn.nl

Research has shown that narrative feedback, (self) reflections and a plan to undertake and evaluate improvements are key factors for effective feedback on clinical performance. We investigated the quantity of narrative comments comprising feedback (by trainers), self-reflections (by trainees) and action plans (by trainer and trainee) entered on a mini-CEX form that was modified for use in general practice training and to encourage trainers and trainees to provide narrative comments. In view of the importance of specificity as an indicator of feedback quality, we additionally examined the specificity of the comments.Read More


We collected and analysed modified mini-CEX forms completed by GP trainers and trainees. Since each trainee has the same trainer for the duration of one year, we used trainer-trainee pairs as the unit of analysis. We determined for all forms the frequency of the different types of narrative comments and rated their specificity on a three-point scale: specific, moderately specific, not specific. Specificity was compared between trainee-trainer pairs.
We collected 485 completed modified mini-CEX forms from 54 trainees (mean of 8.8 forms per trainee; range 1-23; SD 5.6). Trainer feedback was more frequently provided than trainee self-reflections, and action plans were very rare. The comments were generally specific, but showed large differences between trainee-trainer pairs.
The frequency of self-reflection and action plans varied, all comments were generally specific and there were substantial and consistent differences between trainee-trainer pairs in the specificity of comments. We therefore conclude that feedback is not so much determined by the instrument as by the users. Interventions to improve the educational effects of the feedback procedure should therefore focus more on the users than on the instruments.

2012Nov
Med Educ
Med Educ 2012 Nov;46(11):1087-98
Centre for Teaching and Learning, Stellenbosch University, Stellenbosch, South Africa. fjc@sun.ac.za

We previously developed a model of the pre-assessment learning effects of consequential assessment and started to validate it. The model comprises assessment factors, mechanism factors and learning effects. The purpose of this study was to continue the validation process.Read More

For stringency, we focused on a subset of assessment factor-learning effect associations that featured least commonly in a baseline qualitative study. Our aims were to determine whether these uncommon associations were operational in a broader but similar population to that in which the model was initially derived.
A cross-sectional survey of 361 senior medical students at one medical school was undertaken using a purpose-made questionnaire based on a grounded theory and comprising pairs of written situational tests. In each pair, the manifestation of an assessment factor was varied. The frequencies at which learning effects were selected were compared for each item pair, using an adjusted alpha to assign significance. The frequencies at which mechanism factors were selected were calculated.
There were significant differences in the learning effect selected between the two scenarios of an item pair for 13 of this subset of 21 uncommon associations, even when a p-value of < 0.00625 was considered to indicate significance. Three mechanism factors were operational in most scenarios: agency; response efficacy, and response value.
For a subset of uncommon associations in the model, the role of most assessment factor-learning effect associations and the mechanism factors involved were supported in a broader but similar population to that in which the model was derived. Although model validation is an ongoing process, these results move the model one step closer to the stage of usefully informing interventions. Results illustrate how factors not typically included in studies of the learning effects of assessment could confound the results of interventions aimed at using assessment to influence learning.

2012Oct
BMC Med Educ
BMC Med Educ 2012 6;12:67. Epub 2012 Aug 6.
Department of Primary and Community Care, Radboud University Nijmegen Medical Centre, Radboud, The Netherlands. g.sagasser@elg.umcn.nl

Self-regulation is essential for professional development. It involves monitoring of performance, identifying domains for improvement, undertaking learning activities, applying newly learned knowledge and skills and self-assessing performance. Since self-assessment alone is ineffective in identifying weaknesses, learners should seek external feedback too.Read More

Externally regulated educational interventions, like reflection, learning portfolios, assessments and progress meetings, are increasingly used to scaffold self-regulation.The aim of this study is to explore how postgraduate trainees regulate their learning in the workplace, how external regulation promotes self-regulation and which elements facilitate or impede self-regulation and learning.
In a qualitative study with a phenomenologic approach we interviewed first- and third-year GP trainees from two universities in the Netherlands. Twenty-one verbatim transcripts were coded. Through iterative discussion the researchers agreed on the interpretation of the data and saturation was reached.
Trainees used a short and a long self-regulation loop. The short loop took one week at most and was focused on problems that were easy to resolve and needed minor learning activities. The long loop was focused on complex or recurring problems needing multiple and planned longitudinal learning activities. External assessments and formal training affected the long but not the short loop. The supervisor had a facilitating role in both loops. Self-confidence was used to gauge competence.Elements influencing self-regulation were classified into three dimensions: personal (strong motivation to become a good doctor), interpersonal (stimulation from others) and contextual (organizational and educational features).
Trainees did purposefully self-regulate their learning. Learning in the short loop may not be visible to others. Trainees should be encouraged to actively seek and use external feedback in both loops. An important question for further research is which educational interventions might be used to scaffold learning in the short loop. Investing in supervisor quality remains important, since they are close to trainee learning in both loops.

2012Sep
BMC Med Educ
BMC Med Educ 2012 17;12:20. Epub 2012 Apr 17.
Department of Educational Development and Research, Maastricht University, Maastricht, The Netherlands. Joost.dijkstra@maastrichtuniversity.nl

An assessment programme, a purposeful mix of assessment activities, is necessary to achieve a complete picture of assessee competence. High quality assessment programmes exist, however, design requirements for such programmes are still unclear. We developed guidelines for design based on an earlier developed framework which identified areas to be covered.Read More

A fitness-for-purpose approach defining quality was adopted to develop and validate guidelines.
First, in a brainstorm, ideas were generated, followed by structured interviews with 9 international assessment experts. Then, guidelines were fine-tuned through analysis of the interviews. Finally, validation was based on expert consensus via member checking.
In total 72 guidelines were developed and in this paper the most salient guidelines are discussed. The guidelines are related and grouped per layer of the framework. Some guidelines were so generic that these are applicable in any design consideration. These are: the principle of proportionality, rationales should underpin each decisions, and requirement of expertise. Logically, many guidelines focus on practical aspects of assessment. Some guidelines were found to be clear and concrete, others were less straightforward and were phrased more as issues for contemplation.
The set of guidelines is comprehensive and not bound to a specific context or educational approach. From the fitness-for-purpose principle, guidelines are eclectic, requiring expertise judgement to use them appropriately in different contexts. Further validation studies to test practicality are required.

2012Aug
Adv Health Sci Educ Theory Pract
Adv Health Sci Educ Theory Pract 2012 Aug 28;17(3):339-55. Epub 2011 Jun 28.
Skillslab, Faculty of Health Medicine and Life Sciences, Maastricht University, The Netherlands. r.duvivier@maastrichtuniversity.nl

Lack of published studies on students' practice behaviour of physical examination skills outside timetabled training sessions inspired this study into what activities medical students undertake to improve their skills and factors influencing this. Six focus groups of a total of 52 students from Years 1-3 using a pre-established interview guide. Interviews were recorded, transcribed and analyzed using qualitative methods.Read More

The interview guide was based on questionnaire results; overall response rate for Years 1-3 was 90% (n = 875). Students report a variety of activities to improve their physical examination skills. On average, students devote 20% of self-study time to skill training with Year 1 students practising significantly more than Year 3 students. Practice patterns shift from just-in-time learning to a longitudinal selfdirected approach. Factors influencing this change are assessment methods and simulated/real patients. Learning resources used include textbooks, examination guidelines, scientific articles, the Internet, videos/DVDs and scoring forms from previous OSCEs. Practising skills on fellow students happens at university rooms or at home. Also family and friends were mentioned to help. Simulated/real patients stimulated students to practise of physical examination skills, initially causing confusion and anxiety about skill performance but leading to increased feelings of competence. Difficult or enjoyable skills stimulate students to practise. The strategies students adopt to master physical examination skills outside timetabled training sessions are self-directed. OSCE assessment does have influence, but learning takes place also when there is no upcoming assessment. Simulated and real patients provide strong incentives to work on skills. Early patient contacts make students feel more prepared for clinical practice.

2012Aug
Postgrad Med J
Postgrad Med J 2012 Aug 17;88(1042):443-50. Epub 2012 May 17.
Departments of Intensive Care and Internal Medicine Maastricht University Medical Centre, P. Debyelaan 25, 6202 AZ Maastricht, The Netherlands. w.van.mook@mumc.nl

To determine whether analysis of unsolicited healthcare complaints specifically focusing on unprofessional behaviour can provide additional information from the patients' perspective.
A qualitative study with content analysis of healthcare complaints and associated judgements using complaints filed from 2004 to 2009 at the complaints committee of a tertiary-referral centre. Subsequent comparison of the resulting categories of poor professionalism to categories perceived relevant by physicians in a previous study was performed.Read More


137 complaints (98%) yielded 46 different unprofessional behaviours grouped into 18 categories. The element 'perceived medical complications and error' occurred most commonly (n=77), followed by 'having to wait for care' and 'insufficient or unclear clarification' (n=52, n=48, respectively). The combined non-cognitive elements of professionalism (especially aspects of communication) were far more prominently discussed than cognitive issues (knowledge/skills) related to medical error. Most categories of professionalism elements were considered important by physicians but, nevertheless, were identified in patient complaints analysis. Some issues (eg, 'altruism', 'appearance', 'keeping distance/respecting boundaries with patients') were not perceived as problematic by patients and/or relatives, while mentioned by physicians. Conversely, eight categories of poor professionalism revealed from complaint analysis (eg, 'having to wait for care', 'lack of continuity of care' and 'lack of shared decision making') were not considered essential by physicians.
The vast majority of unprofessional behaviour identified related to non-cognitive, professionalism aspects of care. Complaints pertaining to unsatisfactory communication were especially noticeable. Incongruence is noted between the physicians' and the patients' perception of actual care.

2012Aug
Med Educ
Med Educ 2012 Aug;46(8):738-47
Department of Educational Development and Research, Faculty of Health, Medicine and Life Sciences, Maastricht University, Maastricht, the Netherlands. j.frambach@maastrichtuniversity.nl

Medical schools worldwide are increasingly switching to student-centred methods such as problem-based learning (PBL) to foster lifelong self-directed learning (SDL). The cross-cultural applicability of these methods has been questioned because of their Western origins and because education contexts and learning approaches differ across cultures.
This study evaluated PBL's cross-cultural applicability by investigating how it is applied in three medical schools in regions with different cultures in, respectively, East Asia, the Middle East and Western Europe.Read More

Specifically, it investigated how students' cultural backgrounds impact on SDL in PBL and how this impact affects students.
A qualitative, cross-cultural, comparative case study was conducted in three medical schools. Data were collected through 88 semi-structured, in-depth interviews with Year 1 and 3 students, tutors and key persons involved in PBL, 32 observations of Year 1 and 3 PBL tutorials, document analysis, and contextual information. The data were thematically analysed using the template analysis method. Comparisons were made among the three medical schools and between Year 1 and 3 students across and within the schools.
The cultural factors of uncertainty and tradition posed a challenge to Middle Eastern students' SDL. Hierarchy posed a challenge to Asian students and achievement impacted on both sets of non-Western students. These factors were less applicable to European students, although the latter did experience some challenges. Several contextual factors inhibited or enhanced SDL across the cases. As students grew used to PBL, SDL skills increased across the cases, albeit to different degrees.
Although cultural factors can pose a challenge to the application of PBL in non-Western settings, it appears that PBL can be applied in different cultural contexts. However, its globalisation does not postulate uniform processes and outcomes, and culturally sensitive alternatives might be developed.

2012Aug
Med Teach
Med Teach 2012 ;34(9):683-97
Department of Educational Development and Research, Maastricht University, The Netherlands.

There has been increasing use and significance of progress testing in medical education. It is used in many ways and with several formats to reflect the variety of curricula and assessment purposes. These developments have occurred alongside a recognised sensitivity for error variance inherent in multiple choice tests from which challenges to its validity and reliability have arisen.Read More

This Guide presents a generic, systemic framework to help identify and explore improvements in the quality and defensibility of progress test data. The framework draws on the combined experience of the Dutch consortium, an individual medical school in the United Kingdom, and the bulk of the progress test literature to date. It embeds progress testing as a quality-controlled assessment tool for improving learning, teaching and the demonstration of educational standards. The paper describes strengths, highlights constraints and explores issues for improvement. These may assist in the establishment of potential or new progress testing in medical education programmes. They can also guide the evaluation and improvement of existing programmes.

2012Jul

Conceptions of medical teachers regarding learning and teaching affect their teaching practice. Therefore conceptions should be addressed in faculty development.
To facilitate this, we constructed the Conceptions Of Learning and Teaching (COLT) instrument.Read More


The COLT was adapted based on experts' comments during a meeting and interviews, followed by a Delphi procedure (Part I). It was administered to teachers from two Dutch medical schools with different traditions in student-centred education (Part II; N=646). The data were analyzed using confirmatory factor analysis and reliability analysis.
324 Teachers (50.2%) completed the questionnaire. Confirmatory factor analysis did not confirm the underlying theoretical model, but an alternative model demonstrated a good fit. This led to an instrument with eighteen items reflecting three underlying factors: 'teacher centredness', 'appreciation of active learning', and 'orientation to professional practice'. We found significant differences in COLT scores between the faculty of the two medical schools.
The COLT appears to be a construct valid tool resulting in reliable scores of teachers' conceptions of learning and teaching, in student-centred medical education. Two of the three factors are new and may be specific for student-centred medical education. The COLT may be a promising tool to improve faculty development.

2012Jun
BMC Med Educ
BMC Med Educ 2012 16;12. Epub 2012 Mar 16.
Centre for Teaching and Learning, Stellenbosch University, Matieland 7602, South Africa. fjc@sun.ac.za

No validated model exists to explain the learning effects of assessment, a problem when designing and researching assessment for learning. We recently developed a model explaining the pre-assessment learning effects of summative assessment in a theory teaching context. The challenge now is to validate this model.Read More

The purpose of this study was to explore whether the model was operational in a clinical context as a first step in this process.
Given the complexity of the model, we adopted a qualitative approach. Data from in-depth interviews with eighteen medical students were subject to content analysis. We utilised a code book developed previously using grounded theory. During analysis, we remained alert to data that might not conform to the coding framework and open to the possibility of deploying inductive coding. Ethical clearance and informed consent were obtained.
The three components of the model i.e., assessment factors, mechanism factors and learning effects were all evident in the clinical context. Associations between these components could all be explained by the model. Interaction with preceptors was identified as a new subcomponent of assessment factors. The model could explain the interrelationships of the three facets of this subcomponent i.e., regular accountability, personal consequences and emotional valence of the learning environment, with previously described components of the model.
The model could be utilized to analyse and explain observations in an assessment context different to that from which it was derived. In the clinical setting, the (negative) influence of preceptors on student learning was particularly prominent. In this setting, learning effects resulted not only from the high-stakes nature of summative assessment but also from personal stakes, e.g. for esteem and agency. The results suggest that to influence student learning, consequences should accrue from assessment that are immediate, concrete and substantial. The model could have utility as a planning or diagnostic tool in practice and research settings.

2012Jun
Med Educ
Med Educ 2012 Jun;46(6):593-603
Department of Clinical Neurological Sciences, Schulich School of Medicine and Dentistry, University of Western Ontario, London, Ontario, Canada. chris.watling@schulich.uwo.ca

Regulatory focus theory posits the existence of two systems of self-regulation underlying human motivation: promotion focus, which is concerned with aspirations and accomplishments, and prevention focus, which is concerned with obligations and responsibilities. It has been proposed that regulatory focus theory may help to explain learners' variable responses to feedback, predicting that positive feedback is motivating under promotion focus, whereas negative feedback is motivating under prevention focus. We aimed to explore this link between regulatory focus theory and response to feedback using data collected in a naturalistic setting.Read More


In a constructivist grounded theory study, we interviewed 22 early-career academic doctors about experiences they perceived as influential in their learning. Although feedback emerged as important, responses to feedback were highly variable. To better understand how feedback becomes (or fails to become) influential, we used the theoretical framework of regulatory focus to re-examine all descriptions of experiences of receiving and responding to feedback.
Feedback could be influential or non-influential, regardless of its sign (positive or negative). In circumstances in which the individual's regulatory focus was readily determined, such as in choosing a career (promotion) or preparing for a high-stakes examination (prevention), the apparent influence of feedback was consistent with the prediction of regulatory focus theory. However, we encountered many challenges in applying regulatory focus theory to real feedback scenarios, including the frequent presence of a mixed regulatory focus, the potential for regulatory focus to change over time, and the competing influences of other factors, such as the perceived credibility of the source or content of the feedback.
Regulatory focus theory offers a useful, if limited, construct for exploring learners' responses to feedback in the clinical setting. The insights and predictions it offers must be considered in light of the motivational complexity of clinical learning tasks and of other factors influencing the impact of feedback.

2012Jun
Med Educ
Med Educ 2012 Jun;46(6):604-12
Department of Primary Care and Community Care, Radboud University Nijmegen Medical Centre, Nijmegen, the Netherlands. e.pelgrim@elg.umcn.nl

Feedback in workplace-based clinical settings often relies on expert trainers' judgements of directly observed trainee performance. There is ample literature on effective feedback, but in practice trainees in workplace-based training are not regularly observed. We aimed to examine external conditions that impact feedback in observational workplace-based assessment (WBA).Read More


Interviews were conducted and the resulting data analysed using a qualitative, phenomenological approach. Between October 2009 and January 2010, we interviewed 22 postgraduate general practice trainees at two institutions in the Netherlands. Three researchers analysed the transcripts of the interviews.
A three-step scheme emerged from the data. Feedback as part of WBA is of greater benefit to trainees if: (i) observation and feedback are planned by the trainee and trainer; (ii) the content and delivery of the feedback are adequate, and (iii) the trainee uses the feedback to guide his or her learning by linking it to learning goals. Negative emotions reported by almost all trainees in relation to observation and feedback led to different responses. Some trainees avoided observation, whereas others overcame their apprehension and actively sought observation and feedback. Active trainers were able to help trainees overcome their fears. Four types of trainer-trainee pairs were distinguished according to their engagement in observation and feedback. External requirements set by training institutions may stimulate inactive trainers and trainees.
In line with the literature, our results emphasise the importance of the content of feedback and the way it is provided, as well as the importance of its incorporation in trainees' learning. Moreover, we highlight the step before the actual feedback itself. The way arrangements for feedback are made appears to be important to feedback in formative WBA. Finally, we outline several factors that influence the success or failure of feedback but precede the process of observation and feedback.

2012Apr
Acad Med
Acad Med 2012 Apr;87(4):463-75
Department of Psychology, Erasmus University, Rotterdam, The Netherlands. schmidt@fsw.eur.nl

Studies that compare the effects of problem-based and conventional medical curricula on student performance generally show no significant differences. However, curriculum comparison studies are at best quasi-experimental, so they are prone to forms of selection bias. The purpose of this study was to reanalyze data from such comparisons by controlling for two potential biases: differential student attrition and differential exposure.Read More


The authors reanalyzed 104 previously published comparisons involving a single, problem-based medical school in the Netherlands (Maastricht University's medical school), using student attrition and study duration data from this school and the schools with which it was compared. The authors removed bias by reequalizing the comparison groups in terms of attrition and study duration.
The uncorrected data showed no differences between problem-based and conventional curricula: Mean effect sizes as expressed by Cohen d were 0.02 for medical knowledge and 0.07 for diagnostic reasoning. However, the reanalysis demonstrated medium-level effect sizes favoring the problem-based curriculum. After corrections for attrition and study duration, the mean effect size for knowledge acquisition was 0.31 and for diagnostic reasoning was 0.51.
Effects of the Maastricht problem-based curriculum were masked by differential attrition and differential exposure in the original studies. Because this school has been involved in many studies included in influential literature reviews published in the past 20 years, the authors' findings have implications for the assessment of the value of problem-based learning put forward by these reviews.

2012Apr
Med Educ
Med Educ 2012 Apr;46(4):390-8
Department of Education, St Lucas Andreas Hospital, Amsterdam, the Netherlands. joanne.fokkema@gmail.com

  Consultants in charge of postgraduate medical education (PGME) in hospital departments ('lead consultants') are responsible for the implementation of educational change. Although difficulties in innovating in medical education are described in the literature, little is known about how lead consultants approach educational change.
  This study was conducted to explore lead consultants' approaches to educational change in specialty training and factors influencing these approaches.Read More


  From an interpretative constructivist perspective, we conducted a qualitative exploratory study using semi-structured interviews with a purposive sample of 16 lead consultants in the Netherlands between August 2010 and February 2011. The study design was based on the research questions and notions from corporate business and social psychology about the roles of change managers. Interview transcripts were analysed thematically using template analysis.
  The lead consultants described change processes with different stages, including cause, development of content, and the execution and evaluation of change, and used individual change strategies consisting of elements such as ideas, intentions and behaviour. Communication is necessary to the forming of a strategy and the implementation of change, but the nature of communication is influenced by the strategy in use. Lead consultants differed in their degree of awareness of the strategies they used. Factors influencing approaches to change were: knowledge, ideas and beliefs about change; level of reflection; task interpretation; personal style, and department culture.
  Most lead consultants showed limited awareness of their own approaches to change. This can lead them to adopt a rigid approach, whereas the ability to adapt strategies to circumstances is considered important to effective change management. Interventions and research should be aimed at enhancing the awareness of lead consultants of approaches to change in PGME.

2012Mar
Adv Health Sci Educ Theory Pract
Adv Health Sci Educ Theory Pract 2012 Mar 3;17(1):39-53. Epub 2011 Apr 3.
Centre for Teaching and Learning, Stellenbosch University, Private Bag X1, Matieland, 7602, South Africa. fjc@sun.ac.za

It has become axiomatic that assessment impacts powerfully on student learning. However, surprisingly little research has been published emanating from authentic higher education settings about the nature and mechanism of the pre-assessment learning effects of summative assessment. Less still emanates from health sciences education settings.Read More

This study explored the pre-assessment learning effects of summative assessment in theoretical modules by exploring the variables at play in a multifaceted assessment system and the relationships between them. Using a grounded theory strategy, in-depth interviews were conducted with individual medical students and analyzed qualitatively. Respondents' learning was influenced by task demands and system design. Assessment impacted on respondents' cognitive processing activities and metacognitive regulation activities. Individually, our findings confirm findings from other studies in disparate non-medical settings and identify some new factors at play in this setting. Taken together, findings from this study provide, for the first time, some insight into how a whole assessment system influences student learning over time in a medical education setting. The findings from this authentic and complex setting paint a nuanced picture of how intricate and multifaceted interactions between various factors in an assessment system interact to influence student learning. A model linking the sources, mechanism and consequences of the pre-assessment learning effects of summative assessment is proposed that could help enhance the use of summative assessment as a tool to augment learning.

2012Mar
Adv Health Sci Educ Theory Pract
Adv Health Sci Educ Theory Pract 2012 Mar 1;17(1):81-93. Epub 2011 May 1.
Department of Intensive Care, Maastricht University Medical Centre, P. Debyelaan 25, 6202 AZ, Maastricht, The Netherlands. w.van.mook@mumc.nl

Although other web-based approaches to assessment of professional behaviour have been studied, no publications studying the potential advantages of a web-based instrument versus a classic, paper-based method have been published to date. This study has two research goals: it focuses on the quantity and quality of comments provided by students and their peers (two researchers independently scoring comments as correct and incorrect in relation to five commonly used feedback rules (and resulting in an aggregated score of the five scores) on the one, and on the feasibility, acceptability and perceived usefulness of the two approaches on the other hand (using a survey). The amount of feedback was significantly higher in the web-based group than in the paper based group for all three categories (dealing with work, others and oneself).Read More

Regarding the quality of feedback, the aggregated score for each of the three categories was not significantly different between the two groups, neither for the interim, nor for the final assessment. Some, not statistically significant, but nevertheless noteworthy trends were nevertheless noted. Feedback in the web-based group was more often unrelated to observed behaviour for several categories for both the interim and final assessment. Furthermore, most comments relating to the category 'Dealing with oneself' consisted of descriptions of a student's attendance, thereby neglecting other aspects of personal functioning. The survey identified significant differences between the groups for all questionnaire items regarding feasibility, acceptability and perceived usefulness in favour of the paper-based form. The use of a web-based instrument for professional behaviour assessment yielded a significantly higher number of comments compared to the traditional paper-based assessment. Unfortunately, the quality of the feedback obtained by the web-based instrument as measured by several generally accepted feedback criteria did not parallel this increase.

2012Mar
Med Teach
Med Teach 2012 ;34 Suppl 1:S42-50
College of Medicine, King Saud bin Abdulaziz University for Health Sciences, PO Box 57374, Riyadh 11574, Kingdom of Saudi Arabia. halkadri@gmail.com

"Assessment steers students' learning" is a statement that has been used repeatedly without solid evidence in the literature. This manuscript aims to evaluate the published literatures on the effect of teaching learning environment in particular, the implemented assessment on students' learning approaches. A literature review was performed on the effect of assessment on students learning approaches.Read More

English language literatures were searched in Pubmed, PsycINFO, and Medline without restriction to type or date of publication. Reviewing the literature, the most prominent identified theme was assessment function characterized in summative and formative assessment and general effect of assessment on students' learning approaches. The literature review has pointed clearly to the complexity of the relationship between learning environment, students' perceptions of assessment demands, and students' approaches to learning. Many factors (extrinsic and intrinsic) were theoretically proposed to mediate students' approaches to learning in response to their assessment. However, few of these factors were researched in the published literature. Formative assessment is likely to contribute to students' deep approach to learning while summative is likely to contribute to their surface approach. However, these effects are not definite and further research about the complex relationship between assessment and students' learning is required.

2012Mar
Perspect Med Educ
Perspect Med Educ 2012 Mar 10;1(1):24-30. Epub 2012 Mar 10.
Flinders Innovation in Clinical Education, Flinders University, Adelaide, Australia ; Department of Educational Development and Research, Maastricht University, Maastricht, the Netherlands.

Progress testing is gaining ground rapidly after having been used almost exclusively in Maastricht and Kansas City. This increased popularity is understandable considering the intuitive appeal longitudinal testing has as a way to predict future competence and performance. Yet there are also important practicalities.Read More

Progress testing is longitudinal assessment in that it is based on subsequent equivalent, yet different, tests. The results of these are combined to determine the growth of functional medical knowledge for each student, enabling more reliable and valid decision making about promotion to a next study phase. The longitudinal integrated assessment approach has a demonstrable positive effect on student learning behaviour by discouraging binge learning. Furthermore, it leads to more reliable decisions as well as good predictive validity for future competence or retention of knowledge. Also, because of its integration and independence of local curricula, it can be used in a multi-centre collaborative production and administration framework, reducing costs, increasing efficiency and allowing for constant benchmarking. Practicalities include the relative unfamiliarity of faculty with the concept, the fact that remediation for students with a series of poor results is time consuming, the need to embed the instrument carefully into the existing assessment programme and the importance of equating subsequent tests to minimize test-to-test variability in difficulty. Where it has been implemented-collaboratively-progress testing has led to satisfaction, provided the practicalities are heeded well.

2012Feb
Med Educ
Med Educ 2012 Feb;46(2):192-200
Department of Clinical Neurological Sciences, Schulich School of Medicine and Dentistry, University of Western Ontario, London, Ontario, Canada. chris.watling@schulich.uwo.ca

How learners interpret their clinical experiences to create meaningful learning has not been well studied. We explored experiences considered by doctors to be influential in their learning in order to better understand this process.
Using a grounded theory approach, we interviewed 22 academic doctors who had been in practice for ≤ 5 years.Read More

Participants were asked to reflect on experiences they considered to have been influential during their training. Constant comparative analysis for emerging themes was conducted iteratively with data collection.
A model of clinical learning emerged in which the clinical work itself is central. As they observe and participate in clinical work, learners can attend to a variety of sources of information that facilitate the interpretation of the experience and the construction of knowledge from it. These 'learning cues' include feedback, role models, clinical outcomes, patient or family responses, and comparisons with peers. The integration of a cue depends on the learner's judgement of its credibility. Certain cues, such as clinical outcomes or feedback from patients, are seen as innately credible, whereas other cues, particularly feedback from supervisors, are subjected to critical judgement.
Learners make complex judgements regarding the credibility of information about clinical performance. Credibility judgements influence the learning that arises from the clinical experience. Further understanding of how such judgements are made could guide educators in providing credible information to learners.

2012Feb
Med Educ Online
Med Educ Online 2012 17;17:11204. Epub 2012 Feb 17.
College of Medicine, King Saud bin Abdulaziz University for Health Sciences, Riyadh, Saudi Arabia. halkadri@gmail.com

Self-assessment is recognized as a necessary skill for lifelong learning. It is widely reported to offer numerous advantages to the learner. The research evaluated the impact of students' and supervisors' self-assessment and feedback training on students' perceptions and practices of self-assessment.Read More

Moreover, it evaluated the effect of self-assessment process on students' study strategies within a community of clinical practice.
We conducted a qualitative phenomenological study from May 2008 to December 2009. We held 37 semi-structured individual interviews with three different cohorts of undergraduate medical students until we reached data saturation. The cohorts were exposed to different contexts while experiencing their clinical years' assessment program. In the interviews, students' perceptions and interpretations of 'self-assessment practice' and 'supervisor-provided feedback' within different contexts and the resulting study strategies were explored.
The analysis of interview data with the three cohorts of students yielded three major themes: strategic practice of self-assessment, self-assessment and study strategies, and feedback and study strategies. It appears that self-assessment is not appropriate within a summative context, and its implementation requires cultural preparation. Despite education and orientation on the two major components of the self-assessment process, feedback was more effective in enhancing deeper study strategies.
This research suggests that the theoretical advantages linked to the self-assessment process are a result of its feedback component rather than the practice of self-assessment isolated from feedback. Further research exploring the effects of different contextual and personal factors on students' self-assessment is needed.

2012Feb
Med Teach
Med Teach 2012 ;34(3):226-31
Department of Educational Development and Research, Faculty of Health, Medicine and Life Sciences, Maastricht University, The Netherlands. e.driessen@maastrichtuniversity.nl

The differences of learning experiences in the workplace put challenges on the assessment: the assessment programme should be aligned with the general competency framework of the curriculum and also fit to the differences in learning contexts of the workplace. We used van der Vleuten's programmatic assessment model to develop a workplace-based assessment programme for final year clerkships. We aimed to design a programme that stimulates learning, supports assessment decision, is feasible and non-bureaucratic.Read More

The first experiences with the programme show that students think that the programme has high learning value and the assessment is sufficiently robust. Many of the commonly reported weaknesses of work-based assessment (not a good fit with the educational context, too complex, too bureaucratic and too much work) were not mentioned by the students.

2012Jan
Med Educ
Med Educ 2012 Jan;46(1):38-48
Flinders Innovation in Clinical Education, Flinders University, South Australia, Australia. Lambert.schuwirth@flinders.edu.au

Programmatic assessment is a notion that implies that the strength of the assessment process results from a careful combination of various assessment instruments. Accordingly, no single instrument is superior to another, but each has its own strengths, weaknesses and purpose in a programme. Yet, in terms of psychometric methods, a one-size-fits-all approach is often used.Read More

Kane's views on validity as represented by a series of arguments provide a useful framework from which to highlight the value of different widely used approaches to improve the quality and validity of assessment procedures.
In this paper we discuss four inferences which form part of Kane's validity theory: from observations to scores; from scores to universe scores; from universe scores to target domain, and from target domain to construct. For each of these inferences, we provide examples and descriptions of approaches and arguments that may help to support the validity inference.
As well as standard psychometric methods, a programme of assessment makes use of various other arguments, such as: item review and quality control, structuring and examiner training; probabilistic methods, saturation approaches and judgement processes, and epidemiological methods, collation, triangulation and member-checking procedures. In an assessment programme each of these can be used.

2011Oct
J Vet Med Educ
J Vet Med Educ 2011 ;38(3):262-9
Utrecht University, Utrecht, Netherlands. G.J.Bok@uu.nl

Changing demands from society and the veterinary profession call for veterinary medical curricula that can deliver veterinarians who are able to integrate specific and generic competencies in their professional practice. This requires educational innovation directed by an integrative veterinary competency framework to guide curriculum development. Given the paucity of relevant information from the veterinary literature, a qualitative multi-method study was conducted to develop and validate such a framework.Read More

A competency framework was developed based on the analysis of focus group interviews with 54 recently graduated veterinarians and clients and subsequently validated in a Delphi procedure with a panel of 29 experts, representing the full range and diversity of the veterinary profession. The study resulted in an integrated competency framework for veterinary professionals, which consists of 16 competencies organized in seven domains: veterinary expertise, communication, collaboration, entrepreneurship, health and welfare, scholarship, and personal development. Training veterinarians who are able to use and integrate the seven domains in their professional practice is an important challenge for today's veterinary medical schools. The Veterinary Professional (VetPro) framework provides a sound empirical basis for the ongoing debate about the direction of veterinary education and curriculum development.

2011Sep
Med Educ
Med Educ 2011 Sep;45(9):927-38
Department of Medicine, Uniformed Services University of the Health Sciences (USUHS), 4301 JonesBridge Road, Bethesda, MD 20814, USA. sdurning@usuhs.mil

Prior work has found that a doctor's clinical reasoning performance varies on a case-by-case (situation) basis; this is often referred to as 'context specificity'. To explore the influence of context on diagnostic and therapeutic clinical reasoning, we constructed a series of videotapes to which doctors were asked to respond, modifying different contextual factors (patient, doctor, setting). We explored how these contextual factors, as displayed by videotape encounters, may have influenced the clinical reasoning of board-certified internists (experts).Read More

Our purpose was to clarify the influence of context on reasoning, to build upon education theory and to generate implications for education practice.
Qualitative data about experts were gathered from two sources: think-aloud protocols reflecting concurrent thought processes that occurred while board-certified internists viewed videotape encounters, and free-text responses to queries that explicitly asked these experts to comment on the influence of selected contextual factors on their clinical reasoning processes. These data sources provided both actual performance data (think-aloud responses) and opinions on reflection (free-text answers) regarding the influence of context on reasoning. Results for each data source were analysed for emergent themes and then combined into a unified theoretical model.
Several themes emerged from our data and were broadly classified as components influencing the impact of contextual factors, mechanisms for addressing contextual factors, and consequences of contextual factors for patient care. Themes from both data sources had good overlap, indicating that experts are somewhat cognisant of the potential influences of context on their reasoning processes; notable exceptions concerned the themes of missed key findings, balancing of goals and the influence of encounter setting, which emerged in the think-aloud but not the free-text analysis.
Our unified model is consistent with the tenets of cognitive load, situated cognition and ecological psychology theories. A number of potentially modifiable influences on clinical reasoning were identified. Implications for doctor training and practice are discussed.

2011Sep
Med Teach
Med Teach 2011 ;33(10):783-97
Flinders University, Adelaide 5001, South Australia, Australia. lambert.schuwirth@flinders.edu.au

There are no scientific theories that are uniquely related to assessment in medical education. There are many theories in adjacent fields, however, that can be informative for assessment in medical education, and in the recent decades they have proven their value. In this AMEE Guide we discuss theories on expertise development and psychometric theories, and the relatively young and emerging framework of assessment for learning.Read More

Expertise theories highlight the multistage processes involved. The transition from novice to expert is characterised by an increase in the aggregation of concepts from isolated facts, through semantic networks to illness scripts and instance scripts. The latter two stages enable the expert to recognise the problem quickly and form a quick and accurate representation of the problem in his/her working memory. Striking differences between experts and novices is not per se the possession of more explicit knowledge but the superior organisation of knowledge in his/her brain and pairing it with multiple real experiences, enabling not only better problem solving but also more efficient problem solving. Psychometric theories focus on the validity of the assessment - does it measure what it purports to measure and reliability - are the outcomes of the assessment reproducible. Validity is currently seen as building a train of arguments of how best observations of behaviour (answering a multiple-choice question is also a behaviour) can be translated into scores and how these can be used at the end to make inferences about the construct of interest. Reliability theories can be categorised into classical test theory, generalisability theory and item response theory. All three approaches have specific advantages and disadvantages and different areas of application. Finally in the Guide, we discuss the phenomenon of assessment for learning as opposed to assessment of learning and its implications for current and future development and research.

2011Aug
Adv Health Sci Educ Theory Pract
Adv Health Sci Educ Theory Pract 2011 Aug 19;16(3):297-310. Epub 2010 Nov 19.
Department of Medical Education, University of KwaZulu-Natal, KwaZulu, South Africa. singaram@ukzn.ac.za

Collaborative approaches such as Problem Based Learning (PBL) may provide the opportunity to bring together diverse students but their efficacy in practice and the complications that arise due to the mixed ethnicity needs further investigation. This study explores the key advantages and problems of heterogeneous PBL groups from the students' and teachers' opinions. Focus groups were conducted with a stratified sample of second year medical students and their PBL teachers.Read More

We found that students working in heterogeneous groupings interact with students with whom they don't normally interact with, learn a lot more from each other because of their differences in language and academic preparedness and become better prepared for their future professions in multicultural societies. On the other hand we found students segregating in the tutorials along racial lines and that status factors disempowered students and subsequently their productivity. Among the challenges was also that academic and language diversity hindered student learning. In light of these the recommendations were that teachers need special diversity training to deal with heterogeneous groups and the tensions that arise. Attention should be given to create 'the right mix' for group learning in diverse student populations. The findings demonstrate that collaborative heterogeneous learning has two sides that need to be balanced. On the positive end we have the 'ideology' behind mixing diverse students and on the negative the 'practice' behind mixing students. More research is needed to explore these variations and their efficacy in more detail.

2011Aug
Eval Program Plann
Eval Program Plann 2011 Aug 5;34(3):206-16. Epub 2011 Mar 5.
Utrecht University, The Netherlands. l.baartman@tue.nl

The goal of this article is to contribute to the validation of a self-evaluation method, which can be used by schools to evaluate the quality of their Competence Assessment Program (CAP). The outcomes of the self-evaluations of two schools are systematically compared: a novice school with little experience in competence-based education and assessment, and an innovative school with extensive experience. The self-evaluation was based on 12 quality criteria for CAPs, including both validity and reliability, and criteria stressing the importance of the formative function of assessment, such as meaningfulness and educational consequences.Read More

In each school, teachers, management and examination board participated. Results show that the two schools use different approaches to assure assessment quality. The innovative school seems to be more aware of its own strengths and weaknesses, to have a more positive attitude towards teachers, students, and educational innovations, and to explicitly involve stakeholders (i.e., teachers, students, and the work field) in their assessments. This school also had a more explicit vision of the goal of competence-based education and could design its assessments in accordance with these goals.

2011Aug
Med Educ
Med Educ 2011 Aug;45(8):807-17
Department of Medicine, Faculty of School of Medicine, Uniformed Services University of the Health Sciences, 4301 Jones Bridge Road, Bethesda, MD 20814, USA.

This study aimed to investigate the relationship between the authenticity of instructional formats and outcome measures within a pre-clerkship clinical reasoning course.
We conducted a randomised, prospective, crossover study with Year 2 medical students taking a pre-clerkship clinical reasoning course. Students were randomised to small groups and exposed to three formats of differing instructional authenticity (paper case, DVD presentation, standardised patient [SP] presentation) across three subject areas (abdominal pain, anaemia, polyuria).Read More

Three student cohorts were taught using one instructional format per subject area so that each cohort received a different instructional format for each of the three subject areas. Outcome measures (objective structured clinical examination, video quiz, written examination) were selected to determine the effect of each instructional format on the clinical reasoning of students.
Increasingly authentic instructional formats did not significantly improve clinical reasoning performance across all outcome measures and subject areas. However, the results of the video quiz showed significant differences in the anaemia subject area between students who had been instructed using the paper case and live SP-based formats (scores of 47.4 and 57.6, respectively; p = 0.01) and in the abdominal pain subject area, in which students instructed using the DVD format scored higher than students instructed using either the paper case or SP-based formats (scores of 41.6, 34.9 and 31.2, respectively; p=0.002).
Increasing the authenticity of instructional formats does not appear to significantly improve clinical reasoning performance in a pre-clerkship course. Medical educators should balance increases in authenticity with factors such as cognitive load, subject area and learner experience when designing new instructional formats.

2011Jun
Med Educ
Med Educ 2011 Jun;45(6):578-84
EMGO Institute for Health and Care Research, Vrije Universiteit University Medical Centre, Amsterdam, The Netherlands. marcel.reinders@vumc.nl

Training in and assessment of consultation skills are high on the agenda of vocational training institutes for postgraduate training. There is a need to establish valid and reliable instruments to assess consultation skills in authentic settings. We investigated the number of assessors and observations needed to achieve reliable assessments of the consultation skills of general practice trainees (GPTs) using a communication instrument (MAAS-Global) and either standardised patient (SP) encounters or videotaped real patient (RP) encounters.Read More


Eight teachers at the Vrije Universiteit (VU) University Medical Centre in Amsterdam attended a training course on the use of the MAAS-Global instrument, which they subsequently used to assess the consultation skills of 53 GPTs in 176 videotaped consultations (102 with SPs, 74 with RPs). All consultations were randomly allocated and assessed by two teachers independently. The reliability of the ratings was estimated using generalisability theory.
It was easier to obtain acceptable reliability using RP consultations than SP consultations. Two assessors and five consultations were required to achieve minimal reliability (generalisability coefficient 0.7) with RPs, whereas three assessors and 30 consultations were needed to achieve minimal reliability with SPs.
Inter-observer and context variability in the assessment of the consultation skills of GPTs remains high. To achieve acceptable levels of reliability, large samples of observations are required in both formats, but, interestingly, RP encounters require a smaller sample than SP encounters.

2011Jun
Med Educ
Med Educ 2011 Jun;45(6):636-47
Office of Continuing Medical Education, Faculty of Medicine, Dalhousie University, Halifax, Nova Scotia, Canada. joan.sargeant@dal.ca

Conceptualisations of self-assessment are changing as its role in professional development comes to be viewed more broadly as needing to be both externally and internally informed through activities that enable access to and the interpretation and integration of data from external sources. Education programmes use various activities to promote learners' reflection and self-direction, yet we know little about how effective these activities are in 'informing' learners' self-assessments.
This study aimed to increase understanding of the specific ways in which undergraduate and postgraduate learners used learning and assessment activities to inform self-assessments of their clinical performance.Read More


We conducted an international qualitative study using focus groups and drawing on principles of grounded theory. We recruited volunteer participants from three undergraduate and two postgraduate programmes using structured self-assessment activities (e.g. portfolios). We asked learners to describe their perceptions of and experiences with formal and informal activities intended to inform self-assessment. We conducted analysis as a team using a constant comparative process.
Eighty-five learners (53 undergraduate, 32 postgraduate) participated in 10 focus groups. Two main findings emerged. Firstly, the perceived effectiveness of formal and informal assessment activities in informing self-assessment appeared to be both person- and context-specific. No curricular activities were considered to be generally effective or ineffective. However, the availability of high-quality performance data and standards was thought to increase the effectiveness of an activity in informing self-assessment. Secondly, the fostering and informing of self-assessment was believed to require credible and engaged supervisors.
Several contextual and personal conditions consistently influenced learners' perceptions of the extent to which assessment activities were useful in informing self-assessments of performance. Although learners are not guaranteed to be accurate in their perceptions of which factors influence their efforts to improve performance, their perceptions must be taken into account; assessment strategies that are perceived as providing untrustworthy information can be anticipated to have negligible impact.

Despite its ubiquitous use over the past 4 decades, there is no structured, formal method with which to assess polypectomy.
To develop and validate a new method with which to assess competency in polypectomy.
Polypectomy underwent task deconstruction, and a structured checklist and global assessment scale were developed (direct observation of polypectomy skills [DOPyS]).Read More

Sixty bowel cancer screening polypectomy videos were randomly chosen for analysis and were scored independently by 7 expert assessors by using DOPyS. Each parameter and the global rating were scored from 1 to 4 (scores ≥3 = competency). The scores were analyzed by using generalizability theory (G theory).
Multicenter.
Fifty-nine of the 60 videos were assessable and scored. The majority of the assessors agreed across the pass/fail divide for the global assessment scale in 58 of 59 (98%) polyps. For G-theory analysis, 47 of the 60 videos were analyzed. G-theory analysis suggested that DOPyS is a reliable assessment tool, provided that it is used by 2 assessors to score 5 polypectomy videos all performed by 1 endoscopist. DOPyS scores obtained in this format would reflect the endoscopist's competence.
Small sample and polyp size.
This study is the first attempt to develop and validate a tool designed specifically for the assessment of technical skills in performing polypectomy. G-theory analysis suggests that DOPyS could reliably reflect an endoscopist's competence in performing polypectomy provided a requisite number of assessors and cases were used.

Medical curricula should focus on the future of health care. Contemporary competency frameworks for curriculum design such as Canadian Medical Education Directions for Specialists (CanMEDS), ACGME and Tomorrow's Doctors share this vision by stressing generic competencies.
The objective of this study was to investigate how well a contemporary competency framework fits in with clinicians' perspectives on future health care.Read More


Using a strategic planning approach, a semi-structured open-ended questionnaire on the future of their profession was sent to 102 Dutch gynecologists. Through inductive analysis, a future perspective and its needed competencies were identified and compared to the CanMEDS framework.
The 62 responses showed content validity for the CanMEDS roles. Additionally, two roles were identified: advanced technology user and entrepreneur. Within the role Communicator, the focus will change through more active patient participation. The roles Collaborator and Manager are predicted to change in focus because of an increase of complex interdisciplinary teamwork and leadership roles.
By studying the Dutch gynecologists' perspective of the future in a strategic planning approach, two additional roles and focus areas within a contemporary competency framework were identified. The perspective of clinicians on future health care provides valuable messages on how to design future-proof curricula.

2011May
Med Educ
Med Educ 2011 May;45(5):438-9
Department of Educational Research and Development, Faculty of Health, Medicine and Life Sciences, Maastricht University, The Netherlands. e.driessen@educ.unimaas.nl
2011May
Med Teach
Med Teach 2011 ;33(6):478-85
Department of Educational Development and Research, Maastricht University, The Netherlands. l.schuwirth@maastrichtuniversity.nl

In assessment a considerable shift in thinking has occurred from assessment of learning to assessment for learning. This has important implications for the conceptual framework from which to approach the issue of assessment, but also with respect to the research agenda. The main conceptual changes pertain to programmes of assessment.Read More

This has led to a broadened perspective on the types of construct assessment tries to capture, the way information from various sources is collected and collated, the role of human judgement and the variety of psychometric methods to determine the quality of the assessment. Research into the quality of assessment programmes, how assessment influences learning and teaching, new psychometric models and the role of human judgement is much needed.

2011Apr
Med Teach
Med Teach 2011 28;33(5):403-9. Epub 2011 Feb 28.
Department of Anatomy/Embryology, Faculty of Health, Medicine and Life Sciences, Maastricht University, The Netherlands. e.bergman@ae.unimaas.nl

Publications in a variety of journals have described the problem of medical students' decreasing anatomical knowledge. Interestingly, the number of people making this assertion is growing, despite a lack of empirical evidence that today's medical graduates actually know less about anatomy than medical students in the past. Nevertheless, many people are claiming that students' anatomical knowledge is impaired due to negative effects from several factors, including teaching by non-medically qualified teachers, diminished use of cadaver dissection as a teaching tool and neglect of vertical integration of anatomy teaching.Read More


To find empirical evidence for the factors claimed to have an influence on anatomical knowledge of students.
A literature search.
There is a lack of sufficient quantity and quality of information within the existing literature to support any of the claims, but the gathered literature did reveal some fascinating insights which are discussed.
Anatomy education should be made as effective as possible, as nobody will deny that medical students cannot do without anatomical knowledge. Because of promising findings in the areas of teaching in context, vertical integration and assessment strategies, it is recommended that future research into anatomy education should focus on these factors.

2011Apr
Med Educ
Med Educ 2011 Apr;45(4):329-38
Department of Neurology and Neurosurgery, McGill University, Montreal, Quebec, Canada. stuart.lubarsky@mcgill.ca

Script concordance test (SCT) scores are intended to reflect respondents' competence in interpreting clinical data under conditions of uncertainty. The validity of inferences based on SCT scores has not been rigorously established.
This study was conducted in order to develop a structured validity argument for the interpretation of test scores derived through use of the script concordance method.Read More


We searched the PubMed, EMBASE and PsycINFO databases for articles pertaining to script concordance testing. We then reviewed these articles to evaluate the construct validity of the script concordance method, following an established approach for analysing validity data from five categories: content; response process; internal structure; relations to other variables, and consequences.
Content evidence derives from clear guidelines for the creation of authentic, ill-defined scenarios. High internal consistency reliability supports the internal structure of SCT scores. As might be expected, SCT scores correlate poorly with assessments of pure factual knowledge, in which correlations for more advanced learners are lower. The validity of SCT scores is weakly supported by evidence pertaining to examinee response processes and educational consequences.
Published research generally supports the use of SCT to assess the interpretation of clinical data under conditions of uncertainty, although specifics of the validity argument vary and require verification in different contexts and for particular SCTs. Our review identifies potential areas of further validity inquiry in all five categories of evidence. In particular, future SCT research might explore the impact of the script concordance method on teaching and learning, and examine how SCTs integrate with other assessment methods within comprehensive assessment programmes.

2011Mar
BMC Med Educ
BMC Med Educ 2011 6;11:101. Epub 2011 Dec 6.
Skillslab, Faculty of Health Medicine and Life Sciences, Maastricht University, Maastricht, The Netherlands. r.duvivier@maastrichtuniversity.nl

The role of deliberate practice in medical students' development from novice to expert was examined for preclinical skill training.
Students in years 1-3 completed 34 Likert type items, adapted from a questionnaire about the use of deliberate practice in cognitive learning. Exploratory factor analysis and reliability analysis were used to validate the questionnaire.Read More

Analysis of variance examined differences between years and regression analysis the relationship between deliberate practice and skill test results.
875 students participated (90%). Factor analysis yielded four factors: planning, concentration/dedication, repetition/revision, study style/self reflection. Student scores on 'Planning' increased over time, score on sub-scale 'repetition/revision' decreased. Student results on the clinical skill test correlated positively with scores on subscales 'planning' and 'concentration/dedication' in years 1 and 3, and with scores on subscale 'repetition/revision' in year 1.
The positive effects on test results suggest that the role of deliberate practice in medical education merits further study. The cross-sectional design is a limitation, the large representative sample a strength of the study. The vanishing effect of repetition/revision may be attributable to inadequate feedback. Deliberate practice advocates sustained practice to address weaknesses, identified by (self-)assessment and stimulated by feedback. Further studies should use a longitudinal prospective design and extend the scope to expertise development during residency and beyond.

There is a paucity of research into the processes surrounding the transition from residency to the position of attending physician. This report retrospectively investigates the question: Are attending physicians adequately prepared and trained to perform the tasks and duties of their new position? This study aimed at formulating a conceptual framework that captures the transition and is applicable beyond discipline- or location-specific boundaries.
Individual semistructured interviews were conducted and analyzed using a qualitative, grounded theory approach.Read More

Between January and May 2009, 14 physicians were interviewed who had commenced an attending post in internal medicine or obstetrics-gynecology between six months and two years earlier, within the Netherlands. Interviews focused on the attendings' perceptions of the transition, their socialization within the new organization, and the preparation they had received during residency training. The interview transcripts were openly coded, and through constant comparison, themes emerged. The research team discussed the results until full agreement was reached.
A conceptual framework emerged from the data, consisting of three themes interacting in a longitudinal process. The framework describes how novel disruptive elements (first theme) due to the transition from resident to attending physician are perceived and acted on (second theme), and how this directs new attendings' personal development (third theme).
The conceptual framework finds support in transition psychology and notions from organizational socialization literature. It provides insight into the transition from resident to attending physician that can inform measures to smooth the intense transition.

2010Nov
Med Teach
Med Teach 2010 ;32(11):891-8
Department of Intensive Care, Maastricht University Medical Centre, P. Debeijelaan 25, 6202 AZ Maastricht, The Netherlands. W.van.mook@mumc.nl

Given the changes in society we are experiencing, the increasing focus on patient-centred care and acknowledgement that medical education including professionalism issues needs to continue not only in the residency programmes but also throughout the doctors career, is not surprising. Although most of the literature on professionalism pertains to learning and teaching professionalism issues, addressing unprofessional behaviour and related patient safety issues forms an alternative or perhaps complementary approach. This article describes the possibility of selecting applicants for a medical school based on personality characteristics, the attention to professional lapses in contemporary undergraduate training, as well as the magnitude, aetiology, surveillance and methods of dealing with reports of unprofessional behaviour in postgraduate education and CME.Read More

Teaching and assessment of professional behaviour (PB) has been receiving increasing attention in the educational literature and educational practice. Although the focus tends to be summative aspects, it seems perfectly feasible to combine formative and summative approaches in one procedural approach.
Although, many examples of frameworks of professionalism and PB can be found in the literature, most originate from North America, and only few are designed in other continents.Read More

This article presents the framework for PB that is used at Maastricht medical school, the Netherlands.
The approach to PB used in the Dutch medical schools is described with special attention to 4 years (2005-2009) of experience with PB education in the first 3 years of the 6-year undergraduate curriculum of Maastricht medical school. Future challenges are identified.
The adages 'Assessment drives learning' and 'They do not respect what you do not inspect' [Cohen JJ. 2006. Professionalism in medical education, an American perspective: From evidence to accountability. Med Educ 40, 607-617] suggest that formative and summative aspects of PB assessment can be combined within an assessment framework. Formative and summative assessments do not represent contrasting but rather complementary approaches. The Maastricht medical school framework combines the two approaches, as two sides of the same coin.

2010Oct
Med Educ
Med Educ 2010 Oct;44(10):953-61
Department of Medicine, Uniformed Services University, Bethesda, Maryland 20814-4712, USA. sdurning@usuhs.mil

The practice of medicine involves many stakeholders (or participant groups such as patients, doctors and trainees). Based on their respective goals, perceptions and understandings, and on what is being measured, these stakeholders may have dramatically different viewpoints of the same event. There are many ways to characterise what occurred in a clinical encounter; these include an oral presentation (faculty perspective), a written note (trainee perspective), and the patient's perspective.Read More

In the present study, we employed two established theories as frameworks with the purpose of assessing the extent to which different views of the same clinical encounter (a three-component, Year 2 medical student objective structured clinical examination [OSCE] station) are similar to or differ from one another.
We performed univariate comparisons between the individual items on each of the three components of the OSCE: the standardised patient (SP) checklist (patient perspective); the post-encounter form (trainee perspective), and the oral presentation rating form (faculty perspective). Confirmatory factor analysis (CFA) of the three-component station was used to assess the fit of the three-factor (three-viewpoint) model. We also compared tercile performance across these three views as a form of extreme groups analysis.
Results from the CFA yielded a measurement model with reasonable fit. Moderate correlations between the three components of the station were observed. Individual trainee performance, as measured by tercile score, varied across components of the station.
Our work builds on research in fields outside medicine, with results yielding small to moderate correlations between different perspectives (and measurements) of the same event (SP checklist, post-encounter form and oral presentation rating form). We believe obtaining multiple perspectives of the same encounter provides a more valid measure of a student's clinical performance.

2010Oct
Med Educ
Med Educ 2010 Oct;44(10):1038-47
Department of Academic Success, Ross University School of Medicine, Roseau, Dominica. kwinston@rossmed.edu.dm

A mandatory remedial programme for students who repeat their first semester at medical school has resulted in large gains in academic performance and greatly reduced attrition. Here, we explore the students' views of this in order to clarify understanding of optimal remediation practice.
Using a mixed-methods approach, quantitative and qualitative data were gathered from student surveys (n=333) and three in-depth focus groups.Read More

Results were analysed for emergent themes.
Remedial programmes for at-risk medical students should be mandatory, but should respect students' identity as repeaters. Attitude and motivation are key, and working in stable groups provides essential emotional and cognitive support. The learning environment needs to foster changes in students' ways of thinking and their development as flexible, reflective learners. These endeavours require support from honest teachers with rigorous expectations and good facilitation skills.
Successful remediation needs to challenge students' conceptions of learning, works best in groups with skilled facilitators, and must take into account a blend of cognitive and affective factors and the complex interplay between learner and environment. Given a carefully designed programme, at-risk medical students can learn to make effective and lasting changes to their approach to study, and their views of learning can come to converge with influential ideas in the education literature.

2010Aug
GMS Z Med Ausbild
GMS Z Med Ausbild 2010 22;27(2):Doc34. Epub 2010 Apr 22.
Maastricht University, Medicine and Life Sciences, Faculty of Health, Department of Educational Development and Research, Maastricht, Netherland.

Medical education research has changed over the years from merely descriptive studies towards justification or curriculum comparison studies and, nowadays, towards a slow introduction of more clarification studies. In clarification studies quantitative and qualitative methods are used to explain why or how educational interventions work or do not work. This shift is described in this paper.Read More

In addition, it is explained how research into workplace learning and assessment has impacted developments in educational practice. Finally, it is argued that the participation of teachers within the medical domain in conducting and disseminating research should be cherished, because they play a crucial role in ensuring that medical education research is applied in educational practice.

2010Jul
Med Teach
Med Teach 2010 ;32(7):e263-9
University College Arteveldehogeschool Ghent, Belgium. mieke.embo@arteveldehs.be

Clinical workplaces are hectic and dynamic learning environments, which require students to take charge of their own learning. Competency development during clinical internships is a continuous process that is facilitated and guided by feedback. Limited feedback, lack of supervision and problematic assessment of clinical competencies make the development of learning instruments to support self-directed learning necessary.Read More


To explore students' perceptions about a newly introduced integrated feedback and assessment instrument to support self-directed learning in clinical practice. Students collected feedback from clinical supervisors and wrote it on a competency-based format. This feedback was used for self-assessment, which had to be completed before the final assessment.
Four focus group discussions were conducted with second and last year Midwifery students. Focus groups were audiotaped, transcribed verbatim and analysed in a thematic way using ATLAS.ti for qualitative data analysis.
The analysis of the transcripts suggested that integrating feedback and assessment supports participation and active involvement in learning by collecting, writing, asking, reading and rereading feedback. Under the condition of training and dedicated time, these learning activities stimulate reflection and facilitate the development of strategies for improvement. The integration supports self-assessment and formative assessment but the value for summative assessment is contested. The quality of feedback and empowerment by motivated supervisors are essential to maximise the learning effects.
The integrated Midwifery Assessment and Feedback Instrument is a valuable tool for supporting formative learning and assessment in clinical practice, but its effect on students' self-directed learning depends on the feedback and support from supervisors.

2010Jun
Med Teach
Med Teach 2010 ;32(6):491-5
Department of Educational Development and Research, Maastricht University, The Netherlands. a.muijtjens@educ.unimaas.nl

The potential richness of the feedback for learners and teachers is one of the educational advantages of progress tests (PTs). Every test administration yields information on a student's knowledge level in each sub-domain of the test (cross-sectional information), and it adds a next point to the corresponding knowledge growth curve (longitudinal information). Traditional paper-based feedback has severe limitations and requires considerable effort from the learners to give meaning to the data.Read More

We reasoned that the PT data should be flexibly accessible in all pathways and with any available comparison data, according to the personal interest of the learner. For that purpose, a web-based tool (Progress test Feedback, the ProF system) was developed. This article presents the principles and features of the generated feedback and shows how it can be used. In addition to enhancement of the feedback, the ProF database of longitudinal PT-data also provides new opportunities for research on knowledge growth, and these are currently being explored.

2010Mar
Med Teach
Med Teach 2010 ;32(3):e133-7
Department of Medical Education, College of Health Sciences, University of KwaZulu-Natal, PBX Congella 4013, South Africa. singaram@ukzn.ac.za

Tutorial group effectiveness is essential for the success of learning in problem-based learning (PBL). Less effective and dysfunctional groups compromise the quality of students learning in PBL.
This article aims to report on the reliability and validity of an instrument aimed at measuring tutorial group effectiveness in PBL.Read More


The items within the instrument are clustered around motivational and cognitive factors based on Slavin's theoretical framework. A confirmatory factor analysis (CFA) was carried out to estimate the validity of the instrument. Furthermore, generalizability studies were conducted and alpha coefficients were computed to determine the reliability and homogeneity of each factor.
The CFA indicated that a three-factor model comprising 19 items showed a good fit with the data. Alpha coefficients per factor were high. The findings of the generalizability studies indicated that at least 9-10 student responses are needed in order to obtain reliable data at the tutorial group level.
The instrument validated in this study has the potential to provide faculty and students with diagnostic information and feedback about student behaviors that enhance and hinder tutorial group effectiveness.

2010Mar
Med Teach
Med Teach 2010 ;32(3):236-43
Department of Academic Success, Ross University School of Medicine, Commonwealth of Dominica,West Indies. kwinston@rossmed.edu.dm

Many medical schools provide academic support programmes to aid increasing numbers of students from diverse backgrounds. There have been calls for research into successful intervention programmes, and for detailed descriptions of how they work.
To explore the efficacy of a mandatory intervention programme for at-risk medical students.Read More


Students who failed and then repeated first semester were required to participate in a cognitive skills programme, following a syllabus based on principles drawn from both educational experience and multi-disciplinary theory and practice. Performance of programme participants was compared to the performance of students who repeated prior to the mandatory programme.
Of the participants (n = 216), 91% passed their repeat semester, compared to 58% (n = 715) for controls (p < 0.0001). This significant effect persisted for progression through the school for the subsequent three semesters (p < 0.0005).
A mandatory programme that draws on a blend of theories and research-proven techniques can make a positive difference to the outcomes for at-risk medical students.

2010Feb
Intensive Care Med
Intensive Care Med 2010 Feb 22;36(2):296-303. Epub 2009 Sep 22.
Department of Intensive Care, Maastricht University Medical Centre, P. Debeyelaan 25, 6202 AZ, Maastricht, The Netherlands. w.van.mook@mumc.nl

The emphasis on the importance of professionalism in a recent CoBaTrICE-IT paper was impressive. However, insight into the elements of professionalism as perceived relevant for intensivists from the fellows' view, and how these are taught and learned, is limited.
A nationwide study was performed in 2007-2008.Read More

All ICM fellows (n = 90) were sent a questionnaire containing the following questions regarding training in professionalism (7-point Likert scale (1 = very inadequate, 7 = very adequate)): which are the elements perceived to be important in intensivists'' daily practice (38 items, cat. I)? Which methods of learning and teaching are recognised (16 items, cat. II)? Which methods of teaching and learning are considered especially useful (16 items, cat. III)? Finally, the perceived quantity and quality of formal and informal learning methods, as well as the responsible organisational body was studied. Data were analysed using SPSS 15.0.
Response was 75.5 % (n = 68), mean age 34 years. Regarding Elements, scores on virtually all items were high. The factor 'striving for excellence' explained half the variance. Two other aspects, 'Teamwork' and 'Dealing with ethical dilemmas', were identified. Regarding Methods, three dimensions, 'formal curriculum'', 'private and academic experiences' and 'role modelling', proved important. The factor 'formal curriculum' explained most of the variance. Regarding Usefulness the same factors, now mainly explained by the factor Private and academic experiences, emerged with variance. In both categories the items 'observations in daily practice' and 'watching television programmes like ER and House' were the highest- and lowest-scoring items (5.99 and 5.81, and 2.69 and 2.49, respectively). Mean scores regarding the quantity of formal and informal teaching were 4.06 and 4.58 (range 1.841 and 1.519). For the quality of teaching, the figures were 4.22 and 4.52 (range 1.659 and 1.560, respectively). 54 suggestions for improvement of teaching were documented. The need for some form of formal teaching of professionalism aspects as well as for feedback was most frequently mentioned (n = 19 and 16). The local training centres are considered and should remain pivotal for teaching professionalism issues (n = 17 and 28).
Almost all elements of professionalism were considered relevant to intensivists' daily practice. Although formal teaching methods regarding professionalism aspects are easily recognised in daily practice, learning by personal experiences and informal ways quantitatively plays a more important, and more valued role. Qualitative comments, nevertheless, stress the need for providing and receiving (solicited and unsolicited) feedback, thereby requesting expansion of formal teaching methods. The local training centres (should continue to) play a major role in teaching professionalism, although an additional role for the (inter)national intensive care organisations remains.

2010Feb
Med Teach
Med Teach 2010 ;32(2):154-60
Center for Research and Innovation in Medical Education, University of Groningen and University Medical Center Groningen, Groningen, The Netherlands. j.cohenschotanus@med.umcg.nl

Teachers involved in test development usually prefer criterion-referenced standard setting methods using panels. Since expert panels are costly, standards are often set by a pre-fixed percentage of questions answered correctly or norm-referenced methods aimed at ranking examinees.
To discuss the (dis)advantages of commonly used criterion and norm-referenced methods and present a new compromise method: standards based on a fixed cut-off score using the best scoring students as reference point.Read More


Historical data from 54 Maastricht (norm-referenced) and 52 Groningen (criterion-referenced) tests were used to demonstrate huge discrepancies and variability in cut-off scores and failure rates. Subsequently, the compromise model - known as Cohen's method - was applied to the Groningen tests.
The Maastricht norm-referenced method led to a large variation in required cut-off scores (15-46%), but a stable failure rate (about 17%). The Groningen method with a conventional, pre-fixed standard of 60% led to a large variation in failure rates (17-97%). The compromise method reduced variation in required cut-off scores as well as failure rates.
Both the criterion and norm-referenced standards, used in practice, have disadvantages. The proposed compromise model reduces the disadvantages of both methods and is considered more acceptable. Last but not least, compared to standard setting methods using panels, this method is affordable.

2009Dec
Eur. J. Intern. Med.
Eur J Intern Med 2009 Dec 3;20(8):e148-52. Epub 2009 Oct 3.
Department of Intensive Care Medicine and Internal Medicine, Maastricht University Medical Centre, Maastricht, The Netherlands.

Given the changes in society we are experiencing, the increasing focus on patient centred care and acknowledgment that medical education needs to continue not only in the residency programmes but throughout the doctors career, is not surprising. This article describes the attention currently paid to professionalism in the residency programmes, differences in perception of professionalism between patients, faculty, students and residents, differences in professionalism issues in the different educational phases, as well as their consequences for training and assessment regarding professionalism. Continuous medical education in professionalism is thereafter briefly discussed.Read More

2009Dec
Eur. J. Intern. Med.
Eur J Intern Med 2009 Dec 20;20(8):e153-7. Epub 2009 Sep 20.
Department of Intensive Care and Internal Medicine, Maastricht University Medical Centre, Maastricht, The Netherlands. w.van.mook@mumc.nl

There is general agreement that professionalism and professional behaviour should be (formatively and summatively) assessed, but consensus on how this should be done is still lacking. After discussing some of the remaining issues and questions regarding professionalism assessment, this article discusses the importance of qualitative comments to the assessment of professional behaviour, focuses on the currently most frequently used tools, as well as stresses the need for triangulation (combining) of these tools.Read More

2009Sep
Eur. J. Intern. Med.
Eur J Intern Med 2009 Sep 20;20(5):e105-11. Epub 2009 Feb 20.
Department of Intensive Care and Internal Medicine, Maastricht University Medical Centre, Maastricht, The Netherlands. w.van.mook@mumc.nl

This paper is the fourth article in a series on Professionalism and provides an overview of current methods used for teaching and learning about professionalism. The questions "whether" and "how" professionalism can be placed in the formal medical school curricula are addressed, and the informal learning related to professionalism reviewed.Read More

2009Sep
Ned Tijdschr Geneeskd
Ned Tijdschr Geneeskd 2009 ;153:A542
Maastricht Universitair Medisch Centrum, Maastricht, The Netherlands. w.van.mook@mumc.nl

Assessment of professional behaviour (PB) is increasingly receiving attention in undergraduate and postgraduate medical training. Its aim is to prevent later inappropriate behaviour and to strengthen appropriate behaviour by self-reflection and feedback. Assessment should start early in the curriculum and be repeated frequently, preferably by different assessors in differing educational situations, and with longitudinal follow-up.Read More

The consequences of the assessment as well as any agreements reached, must be clearly documented. If remediation of inappropriate behaviour is unsuccessful, a consilium abeundi, i.e. a recommendation to leave the programme, should be discussed with the student. The Dutch Higher Education and Scientific Research Act (WHW) does not provide for denying students access to educational activities and exams after completing the first year. However, the new Higher Education and Research Act (WHOO), which has yet not been implemented, will provide for obligatory cessation of studies.

2009Jul
Eur. J. Intern. Med.
Eur J Intern Med 2009 Jul 21;20(4):e90-5. Epub 2008 Dec 21.
Department of Intensive Care and Internal Medicine, Maastricht University Medical Centre, Maastricht, The Netherlands. w.van.mook@mumc.nl

Core medical knowledge has been assessed for over fifty years and technical and communication skills for at least twenty. The assessment of professionalism however has failed to achieve recognition within this time frame. The interest in the assessment of professionalism and professional behaviour thus is a fairly recent development.Read More

This article will firstly clarify how professional behaviour assessment relates to other assessment methods using the framework proposed by Miller6. Thereafter a brief overview will be provided of the current "tool box" of methods available to assess professionalism. Data on the validity, reliability, feasibility, acceptability and educational utility of these "tools" as derived from published evidence will be reviewed. Subsequently a general overview of the way forward in the assessment of professionalism and professional behaviour will be given.

2009Jul
Eur. J. Intern. Med.
Eur J Intern Med 2009 Jul 24;20(4):e96-e100. Epub 2009 Jan 24.
Department of Intensive Care and Internal Medicine, Maastricht University Medical Centre, Maastricht, The Netherlands. w.van.mook@mumc.nl

Recommendations in the literature concerning measures to address the challenges to professionalism have converged on the establishment of an education community, on a structured curriculum dealing with professionalism, on developing programs for role modelling and mentoring, and on attention to the assessment of professional conduct. The interventions in the field of medical education appear central among these efforts, since it is during medical school that the template for professional conduct in medicine is primarily learned. This article attempts to provide a more in-depth discussion of the goals, purposes and current factors influencing teaching and learning professional behaviour in the medical school curriculum and the residency programs.Read More