Funded mini-projects
(522) Enhancing medical student performance in objective structured clinical examinations.
N.B. The information below is authored by the mini-project applicants, not by staff of the subject centre.
This text represents the views and opinions of the mini-project team
only, not those of the subject centre or its affiliates.
Principal investigator
Kirsty Forrest, University of Leeds
Full list of project partners
Dr K Forrest
Consultant Anaesthetist
Academic Unit of Anaesthesia
University of Leeds
The General Infirmary at Leeds
Great George Street
Leeds
LS1 3EX
k.forrest@leeds.ac.uk
0113 3926360
Professor Phil Hopkins
Academic Unit of Anaesthesia
University of Leeds
The General Infirmary at Leeds
Great George Street
Leeds
LS1 3EX
Professor Ian Maynard
Sport Psychologist (fBASES, C. Psychol.)
Director: Research Centre for Sport and Exercise Science
Collegiate Crescent Campus
Sheffield Hallam University
Sheffield
S10 2BP
Dr. Joanne Butt
Sport Psychologist (BASES)
Faculty of Health and Wellbeing
Collegiate Crescent Campus
Sheffield Hallam University
Sheffield
S10 2BP
Topic
This is a MINI-PROJECT proposal
Background
Competitive and stressful situations occur throughout all people’s lives where performance can be affected. Sometimes these can be substandard performances such as failing a driving test, exam or not winning the race! We propose a collaboration, between Sheffield Hallam University and the Academic Unit of Anaesthesia at Leeds University, to look at a large scale project as to whether a mental skills training programme, based on the performance enhancement tools used in sport psychology (e.g., goal setting, imagery, concentration, anxiety management and self confidence), delivered to medical students has an effect on their performance in an objective structured clinical examination (OSCE) situation. Is it possible to enhance OSCE performance-using the techniques of Sports Psychology?
Sports psychology is increasingly popular, with well known and successful athletes using the tools provided by psychologists as part of their preparation. Sports Psychology aims to improve understanding of the physical and mental processes which influence behaviour and performance in sport and physical activity. It is widely accepted that at the top level of performance there is little difference in the skill level of the participants. In the sport setting, research has shown that a major factor to distinguish between successful and unsuccessful athletes is often the ability to cope with psychological pressure (Gould et al., 2002; Gould et al., 1981; Vealey, 1994).
Research has shown improved performance in both amateurs and expert athletes with the help of sports psychologists. These mental processes are a 'painless' avenue for assisting athletes to improve. Task-relevant content, positive self-statements and mood words are some of the steps taught and employed by athletes during a competitive performance. Each athlete integrates these steps into a meaningful dialogue practiced (rehearsed), performed, and evaluated for effect and improvement. The inclusion of the processes in training programmes is not difficult, and practice constitutes 'sporting homework'. Businesses are adopting similar training programme structures to improve performance and create success (e.g., Jones, 2002).
Medical students are one of the most frequently tested groups in higher education. Their clinical competence is a complex construct requiring multiple, mixed, and higher order methods of assessment to support valid interpretations. Advances in education assessment have led to the OSCE gaining popularity as a flexible, standardized approach commonplace in the practical assessment of basic clinical competence. OSCE’s use criterion based referencing in determining a student’s grade - by comparing his or her achievements with clearly stated criteria for learning outcomes and standards for particular levels of performance. Unfortunately it has come to our attention that while some students have an adequate level of knowledge as assessed by other forms of examination such as MCQs and EMQs, they have problems in performance examinations such as OSCE’s.
Proposed activities
Ethical approval was obtained for the pilot and is ongoing for the larger project.
Information from the pilot study has informed the group sizes needed for the larger study. A power calculation (Vincent, 1999) revealed that the numbers of students required per group is 17-20 (Power 0.8, alpha 0.05). The data used was the cognitive anxiety intensity variable because research consistently indicates a strong and negative relationship with performance.
Recruitment
All the medical students in the 4th year 2006 - 2007 will be invited to take part in the study. Those that volunteer will be randomly assigned to receive either the training in May/June 2007 or the training in 2008.
The intervention
The proposed intervention programme is a multimodal intervention based on stress management training (SMT) (Smith, 1980). The programme will be conducted over a 6-week period and will include two intervention sessions each week being delivered. The proposed plan is outlined below.
Table 1. Multimodal Intervention (approx. 12-Session).
Week 1 SMT 1 • Applied Relaxation (AR) phase 1: Progressive muscle relaxation.
• 20 minutes of taped instruction (practice twice a day for 3 days).
• Complete the sessions without the taped instructions by end of week 1
• Cognitive restructuring session: Taking Control (controlling the controllables)
Week 2 SMT 2 • AR phase 2: Remove the tension instructions from the procedure (practice twice/day).
• Introduce the role of mental processes in coping with stress
• AR 3: Conditioning the word ‘relax’ on exhalation (practice twice/day)
• Task: Identify negative thoughts/expectations in preparation for next session.
Week 3 SMT 3 • AR phase 3: Practice use of cue controlled relaxation – word ‘relax’ – to control emotional responses induced through imagined stressful ‘OSCE’ related situations.
• Cognitive Restructuring: negative thoughts (pre-identified), negative thought stopping, reframing negative thoughts to positive.
• Attentional Focus strategies/cue controlled self-talk.
Week 4 SMT 4 • Practice stress-reducing mental statements to control emotional reactions induced through imagination
• By end of stage 4 – the two types of coping responses are combined into an “Integrated coping response” – ties in both the self statements and the muscular relaxation (into the breathing cycle).
• Practice techniques 4-5 time/day (short periods)
• Self Confidence Session: Building the bubble of self-belief
Week 5 SMT 5 • Continued practice of integrated response (develop the integrated coping response).
• End of week 5 – integrated breathing technique during medicine practice/procedures.
• Cognitive Restructuring session: Rationalization/Planning for the unexpected
• “What if?” Mental Action Plan development.
Week 6 SMT 6 • Practice integrated coping response in imagined ‘OSCE’ situations.
• (Now possible to use integrated coping response during actual OSCE exam)
The analysis of the data
Measurable outputs include the number of failed OSCE stations, total OSCE scores, and the number of domain fails and subtotals of those scores. Variables include: Gender and attendance at intervention profile - high/medium/low. This will be quantitatively analysed by looking at absolute differences between groups and performing ANOVA for covariant interaction. In addition, multiple regression analyses with follow-up canonical correlation analysis will be conducted to look at the relationship between anxiety and self- confidence variables (i.e., predictor variables) and performance, and to see the strength of these predictive factors.
Presentation of the data
The results will be widely disseminated throughout the medical education community and if successful the implementation of a wider training programme for students.
Proposed outcomes
Main aims and objectives
1. To identify key areas for consideration when preparing performance training for medical students.
2. To gain an understanding of the mental skills used by undergraduate medical students to optimize their performance in OSCE examinations.
3. To show that skills of sports psychology can be transferred to another domain i.e. medicine and achieve an improvement in exam performance.
A pilot study was performed in July 2006 (n=14). This was to inform the larger study and help clarify the logistics and the outcome of the training. The volunteer medical students received 4 hours of mental skills training over 2 days. They filled in an anxiety questionnaire pre and post intervention and their overall OSCE result as a percentage was obtained.
Results of the dependent t-tests of the questionnaire revealed significant changes from pre to post intervention on student's levels of cognitive and somatic anxiety and self confidence. Specifically, levels of cognitive (t=2.85; df = 14; p<.05) and somatic anxiety (t=3.63; df = 14; p<.01) decreased from pre to post intervention. In the case of self confidence, students levels of self confidence significantly increased (t=-4.26; df = 14; p<.01) from pre to post intervention.
Results were also obtained on student's directional interpretation of anxiety symptoms in relation to whether they perceived these symptoms to be positive or negative toward OSCE performance. Although changes in anxiety direction were not significant, observation of mean scores indicated that student's cognitive (decreased M=-11.9 to -7.6) and somatic anxiety (decreases M=-9.0 to -5.5) direction scores decreased from pre to post intervention. This decrease in direction scores indicated that student's interpreted their anxiety symptoms to be less debilitating toward performance in the OSCE post intervention than prior to the intervention.
In the case of performance, regression analysis revealed that student’s levels of self confidence post intervention was significantly related to OSCE performance. In addition, it is also important to note that cognitive anxiety levels were negatively related to OSCE performance. In conclusion, the results indicate that the anxiety management and self confidence intervention employed was successful in reducing students levels of cognitive and somatic anxiety while increasing their self confidence prior to taking the OSCE. Finally, it is important to note that these skills taught as part of the intervention will also be important to help students cope in their future professional endeavours (beyond just passing exams).
Material for dissemination:
1. Statistical evidence that the intervention works (or not).
2. The development of a training programme for medical students to enhance performance in OSCE’s.
Benefits to your institution:
1. The students will be better prepared for exams
2. The students will also be better prepared for the future when performance is measured and assessed.
3. Collaboration between institutions leading to high quality research and strategic partnerships.
4. Provide a stimulating and innovative environment for knowledge transfer between professions.
5. Support dissemination of good practice to a wider audience
Numbers affected: Initially, 40-60 students, however if successful all medical students within the institution.
Expertise of grant holder and project team
Dr Forrest has been involved in the education of medical students for 5 years, both as a lecturer and honorary senior lecturer. Recognition of the work is included in her job plan with 2 programmed activities. Dr Forrest has received a year developmental university teaching fellowship (UTF) in 2004 that included project management. This was subsequently upgraded to a full 3 year UTF in 2006. The project for this has included developing simulation training in acute care for 5th year medical students. Dr Forrest was also awarded a HEFCE funded TIF for £20,000 to develop opportunistic teaching.
Professor Hopkins is Head of the Academic Unit of Anaesthesia at the University of Leeds. He has 20 years of experience in undergraduate and postgraduate medical education as teacher and programme manager. He has been awarded research grants from the Wellcome Trust, MRC, British Heart Foundation and Department of Health, amongst others, and has been widely published in current anaesthetic literature.
As Director of the Research Centre for Sport and Exercise Science (SHU), Prof. Ian Maynard has been responsible for managing funding and grant applications awarded to the centre over the past 3 years (1.4 million annual turnover budget). In addition, Prof. Ian Maynard has been involved in several grant awarding projects involving performance enhancement in elite sport. For example:
(a) World Class Performance (WCP) Plan SSSP – 60k/year (Royal Yachting Association) (1990-to date)
(b) WCP (Diving) – 20k/year (1990-to date)
(c) Equipment bid as part of the National Sports Medicine Institute (N.S.M.I) network of centres of excellence, 2000 (in collaboration with the University of Sheffield & NHS trusts) (98k).
Prof. Ian Maynard and Dr. J. Butt (SHU) have published research in the area of competitive anxiety and/stress management interventions (see examples below). In addition to both authors being involved in delivering sport psychology services to a variety of athletes, they also deliver performing under pressure consulting workshops to business organisations (SHU Wellness Initiative Programme).
Maynard, I.W, Smith, M.J, & Warwick-Evans, L. (1995). The effects of a cognitive intervention strategy on competitive state anxiety and performance in semi-professional soccer players. Journal of Sport and Exercise Psychology. Vol. 17, (4), pp. 428-446.
Maynard, I.W, Hemmings, B, Greenlees, I.A, Warwick-Evans, L & Stanton, N. (1998). Stress management in sport : A comparison of unimodal and multimodal interventions. Anxiety, Stress and Coping : An International Journal. Vol. 11, pp. 225-246.
Butt, J., Weinberg, R.S., Horn, T.S. (2003). The Intensity and Directional
Interpretation of Anxiety: Fluctuations Throughout Competition and Relationship to Performance. The Sport Psychologist, 17.
Similar work
The university teaching fellowship awarded to Dr Forrest is being used to fund a qualitative project to inform the current mental skills medical students use to perform in OSCEs. Initial theme analysis has revealed the major topics of anxiety, confidence, preparation, feedback, experience and practice.
There is a growing trend in the field of Sport Psychology (see references below) to develop performance enhancement knowledge as well as to test the generalisability of sport-based findings by studying other performance situations (e.g., military, business organizations, artists)(Gould, 2002). However, the effectiveness of psychological skills training upon medical students OSCE performance is an unexplored area.
Hays, K.F. (2002). The enhancement of performance excellence among performing artists. Journal of Applied Sport Psychology, 14, (4), p.299-312.
Jones, G. (2002). Performance excellence: A personal perspective on the link between sport and business. Journal of Applied Sport Psychology, 14, (4), p.268-281.
Contact details
Grant holder: Kirsty Forrest,
University of Leeds
Amount awarded: £5,000
Subject centre project contact: Megan Quentin-Baxter
Reports
