How to assess WPBAs: Guidance for assessors

When in secondary care/other non-general practice placements – CSR, CbD and MiniCEX

• The Clinical Supervisors Report should be completed by the named Clinical Supervisor, who needs to have met the educator requirements of the GMC. The RCGP has produced guidance as to how Local Education Providers (LEPs) can meet the requirements set out in the GMC’s “Excellence by Design” (PDF file, 1.3 MB)
• It would be expected that at least one CbD and one mini-CEX be completed by the doctor completing the CSR.  
• Other assessments should be conducted by a doctor who is at a level of ST4 or above. 
• None of the assessors should be a peer, or anyone at the same or lower level of training.

Exceptionally, and only with the express permission of the Head of School, other assessors might be considered appropriate. 

When in GP placements - CbDs and CATS

• All assessments should be completed by a GP Educational Supervisor or an approved Clinical Supervisor who needs to have met the educator requirements of the GMC.  
• Assessments should be conducted by more than one such person in each post and MUST be conducted by more than one such person over their time in training. This allows for triangulation of evidence and is recommended by the GMC.  
• If all other avenues have been exhausted and the Postgraduate School is aware of a lack of Educational Supervisors / named Clinical Supervisors then the triangulation of evidence can be achieved by Training Programme Directors or other GPs who have been trained and remain updated in the use of WPBA. 
• Where assessments are conducted in settings outside the practice (e.g. OOH), then whilst the format can be used by the relevant supervising clinician, if they have not met the educator requirements of the GMC, then the assessment will not contribute to the minimum mandatory evidence.  

Registrars must make every effort to work with their supervisors in a timely fashion to ensure that assessments and reports are completed.  However, if there is persistent lack of engagement by the supervisor, the registrar should inform their Training Programme Director at the time and record this in their Portfolio.  Similarly, if the registrar is failing to engage in providing cases for WPBA assessments despite encouragement, then the Educational Supervisor should record this in Educators' Notes.

Clinical Examination and Procedural Skills –throughout GP training

These examinations must be observed by a professional trained to perform the examination to a level where they would be expected to be able to identify any abnormalities. It would also be expected that the relevant professional be competent in giving feedback. If this is another doctor they must be at ST4 level or above. If the colleague is another health professional - such as a specialist nurse - they must confirm their role and training so that the Educational Supervisor can be satisfied that they have been appropriately trained.

GMC requirements for assessors

The following table sets out how Local Education Providers (LEPs) can meet the requirements set out in the GMC’s “Excellence by Design” (PDF file, 1.3 MB).

Introduction

Workplace Based Assessment (WPBA) comprises one third of the MRCGP qualification, providing a framework for evaluating a GP registrar’s progress in those areas of professional practice best tested in the workplace.

This document provides guide to what is required from each GP Workplace Based Assessment that might take place in secondary/non-primary care.

Throughout GP specialty training, GP registrars collect evidence relating to 13 areas of professional capabilities and record it in their Portfolio. This evidence is used to inform six-monthly reviews and - at the end of training - to make a judgement about their readiness for independent practice.

WPBA that may be undertaken in Secondary Care:

• Case Based Discussion (CbD)
• Mini Clinical Evaluation Exercise (MiniCEX) (only used in secondary care)
• Multi-Source Feedback (MSF)
• Clinical Examination and Procedural Skills (CEPS) 
• Clinical Supervisors Report (CSR)
• Learning Log – via Portfolio
• Personal Development Plan (PDP) – via Portfolio

WPBA that are only undertaken in Primary Care:

• Patient Satisfaction Questionnaire (PSQ)
• Consultation Observation Tool (COT) and Audio-COT
• Care Assessment Tool (CAT) which includes Case Based Discussions (CbD)

WPBA Capabilities

The WPBA component of the MRCGP exam is designed to test competence in key areas derived from the core RCGP curriculum statement ‘Being a GP’.

WPBA capabilities framework

GP Curriculum

The core GP Curriculum is entitled ‘Being a General Practitioner’. It is broken down into topic guides relating to professional issues, life stages and clinical areas. Further information relating to the curriculum can be found on the RCGP website and it is recommended that this is viewed to increase your understanding of these domains.

In WPBA, Clinical Experience Groups map to the GP Curriculum and cover the range of patients that should be covered in GP training. 

Calibrating the standard required in assessments

Registrars in a non-primary care / hospital setting are rated in comparison to other registrars at the same stage of training or to comparable specialty registrars.

The only exception to this is the assessment of competence in CEPS (Clinical Examination and Procedural Skills) where the standard is that of an independent practitioner carrying out this GP-focussed examination or procedure, whether the registrar is in a primary care or non-primary care post.

All assessors are asked to define the level of complexity of the case as low, medium, or high and to link the case to the relevant Clinical Experience Groups.

Below, are the grades for the assessments and descriptors for the ‘assessment of performance levels’. In addition, as part of the CSR, the Clinical Supervisor is also asked an entrustable question relating to the level of supervision the registrar has needed during that post.

Word descriptors for the grades within each Capability have been written to support both the registrar and the supervisors. These should be used alongside the assessment until you become familiar with their content.

Non-primary care assessment grades for the capability area being assessed:

• Significantly below expectations 
• Below expectation
• Meets expectations 
• Above expectations
• Not Applicable (MiniCEX only)

Assessment of performance levels for the CbD and MiniCEX

Based on this observation, please rate the registrars’ overall performance:

• Below the level expected prior to starting on a GP Training programme 
• Below the level expected of a GP registrar working in the current clinical post
• At the level expected of a GP registrar working in the current clinical post
• Above the level expected of a GP registrar working in the current clinical post

Assessment of Performance for Clinical Examination and Procedural Skills

Based on this observation, please rate the registrars’ overall performance:

• Unable to perform the procedure appropriately
• Able to perform the procedure but needs direct supervision and /or assistance
• Able to perform the procedure with minimal supervision or assistance 
• Competent to perform the procedure unsupervised

CSR levels of supervision for use in non-primary care placements

If levels 1 or 2 are selected the Clinical Supervisor will be required to clarify their reasons for this choice and it is expected that they would then also contact the registrar’s GP Educational Supervisor and / or their local GP Associate Dean/Training Programme Director to inform them of their concerns

Level 1: Supervision definition: Cannot be left without direct supervision. 

Limited to observing care and/or seeing patients alone but not allowed to let patients leave the building or complete an episode of care before review by the supervisor.

Level 2: Supervision definition: Requires more supervision than expected in their clinical role. 

Requires direct supervision by named supervisor. The registrar may provide clinical care, but the supervisor (or in their absence a delegated supervisor) is physically within the building and is immediately available if required to provide direct supervision on specific cases and non-immediate review of all cases.

Level 3: Supervision definition: Requires expected levels of supervision in their clinical role 

Requires indirect supervision by the named supervisor. The registrar may provide clinical care when the supervisor is at a distance (urgent/unscheduled care, home visits) but is available by means of telephone to provide advice, and available to attend jointly if required to provide direct supervision. The registrar does not need to have every case reviewed but a regular review of random or selected cases takes place at routine intervals.

Level 4 (ST3 only): Supervision definition: Requires no supervision in their clinical role

It is expected that GP registrars will only reach Level 4 at the end of their training.

Indicators of Potential Underperformance

The competency framework, developed for the MRCGP from the GP curriculum, is a series of word descriptors that describe ‘positive’ behaviours that doctors display in practice. This framework has been augmented by selectively adding a number of ‘negative’ behaviours and placing them alongside the themes in the framework to which they are particularly (but not exclusively) related. These behavioural descriptors are intended as an additional interpretative tool to make it easier to recognise underperformance, to do so early in training and this material may be used to give constructive feedback to the registrar. Further information on these indicators can be found on the RCGP website.

The roles of the Clinical Supervisor and Educational supervisors

Clinical supervisor

The Clinical Supervisor (CS) should (normally) be a consultant that a GP ST1 or ST2 registrar should remain attached to throughout the post. Though a CS usually has access to the Portfolio, on rare occasions they will not. For those Clinical supervisors who do not have access to the Portfolio, the registrar will send you a ‘ticket code’ with a link to the assessment form via an email in advance of your planned meeting or agreed assessment, to enable formative discussion and joint completion of the assessment forms.

All doctors who are named GP Clinical Supervisor, must have attended appropriate clinical supervisor training. The Clinical Supervisor is expected to:

• Be the named supervisor for the duration of the post
• Undertake an initial induction meeting reviewing the learning needs of the GP registrar and agreeing an educational plan for the post. The registrar needs to complete a mandatory placement planning meeting learning log entry in their Portfolio after the meeting.
• Complete at least one WPBA for the registrar and ideally one CbD and 1 miniCEX.
• Complete a Clinical Supervisors Report (CSR) on the Portfolio at the end of the post (access to the assessment and report forms are sent via a ‘ticket code’ as described above when the supervisor doesn’t have access to the registrar's Portfolio).

Placement planning meeting

Within the first few weeks of each new placement a CS should meet with the registrar for a placement planning meeting. This is an opportunity to ensure that the registrar has appropriate learning objectives for the placement. At the placement planning meeting it is recommended that dates are agreed for a mid-post review and for a final review to complete and discuss the CS report.

Educational Supervisor

The registrar’s Educational Supervisor (ES) will be a GP who will have access to the Portfolio. This GP will provide continuity and support and at least six-monthly reviews which are summarised in the Educational Supervisor Reports (ESRs) throughout training.

Case Based Discussion (CbD)

How to manage and assess a registrar in a CBD

A Case Based Discussion (CbD) is a structured interview between educator and registrar; the assessor is required to stick to the ‘here and now’ and what the registrar actually did, rather than asking ‘what if…’ questions. The cases should be ones that the registrar managed independently (it is not appropriate to have got advice from another doctor for the case and then to be assessed on actions which were not independent).

Before starting

• Protected time is needed for CbDs.
• The registrar is responsible for selecting the case, organising a time for the assessment with the identified, approved assessor and completing any preparatory work.
• For each CbD interview, the registrar should select a case and share details of the clinical entry and a medical summary with the supervisor at least three days before the discussion.
• The registrar needs to have prepared the case against the capabilities they believe the case covered or capabilities they wish to discuss. It is expected the registrar will have covered 3 to 4 capability areas.
• When a case is presented to you for a CbD, it is helpful if you are familiar with it beforehand.
• Case selection is very important: encourage your registrar to present cases that focus on the capabilities they have not covered in previous CbDs or those that have previously been flagged as needing development during this post. Cases where there was an element of uncertainty or where a conflict in decision-making are particularly good ones to choose.
• The ‘CbD question maker’ can be used to help work out questions you would want to use on each capability domain you are assessing (some assessors may want to use the word descriptors to generate their own questions).
• It is important for the progress of the registrar that the discussion is used to guide further development by offering structured feedback. The CbD discussion should normally take no longer than 30 minutes in total which allows about 20 minutes for feedback together with any recommendations for change (10 minutes feedback).

The CbD process

• Ask the registrar to introduce the case briefly (2-3 mins). Another useful starting point is to ask them 3 things:
  1. what issues did you feel the case raised?
  2. what issues did you feel needed resolving?
  3. what bits did you find challenging/difficult?
• Clarify matters of fact e.g. ‘what did you mean when you told them….?’
• Take notes as they talk: especially of some of the things they say that relate to the capability domains you want to assess; explore those later.
• Set the agenda: review the capability domains selected by the registrar. It may be appropriate to select different capability domains that you planned to look at having considered the presentation that the registrar has made.
  • ‘Today, we're going to use this case to explore three capability domains.
    The first is.... and the second.... and finally.  ’
• Before asking the set of questions relating to a specific capability, signpost the capability being assessed.
  • ‘Okay, let’s move on. The next set of questions I am going to ask you relate to the capability…’
• Do not ask hypothetical questions like ‘What if’ scenarios. CbDs are testing what they actually did and why and not what they would do in an ideal world. You can ask "What is your next step?”
• It is good practice to ask for evidence of what they did
  • ‘So, how did you actually phrase that? What did you actually say? What was the response to that?’
• And / Or ask for justification of what they did
  • ‘So, what made you do that?’
  • ‘Why did you choose that option above all the others?’
  • ‘Did you use any guidance, protocols or evidence to guide you?’, ‘Where did you look for that?’, ‘What did it say?’
• Once you’ve got enough evidence to rate the registrar on that capability, move onto the next one. To make the assessment decision for each capability, you should encourage the registrar to share as much as they can until they seem to have no further insights to add.
• If a particular capability is causing concern, stay with it and give it the time it deserves to enable you to draw out what the registrar needs to do in order to improve.
• Don’t teach during the assessment phase of the process. Some supervisors assess and teach at the same time. This disrupts the process, shifts the balance in terms of the time devoted to assessment and to teaching, and can result in cases taking an awfully long time to complete. You can have a deeper discussion when the assessment bit is over; it is appropriate to flag that you intend to return to teach on this area later during the discussion.
• Continuously monitor registrar’s verbal and non-verbal cues. CbDs are an opportunity to tease out a true picture of what they did and why. That’s unlikely to happen if the discussion is adversarial.

When you are giving feedback

• At the end of each case, ask the registrar how they felt they performed.
• A judgement of the level of performance demonstrated by the registrar should be recorded on the marking grid along with recommendations for further development. Remember, your feedback should be specific and descriptive. ‘Let’s look at each capability area we covered... practising holistically – how did you feel that went for you?’
• For each capability state what they did well and what they could improve. The RCGP capability descriptors will help you to do this.
• You might wish to ask the registrar:
  • ‘What’s the most important things you’re going to take away from today’s session?’ ‘What do you need to follow up?’
• Now record it in their Portfolio using the ticket code provided in advance by the registrar if the CS does not have personal access to Portfolio. It is best to do this jointly so that what is written is shared and therefore owned, thought the final responsibility for what is written lies with the assessor.  The formative written feedback is reviewed by the registrar and their educational supervisor when completing their six-monthly educational supervisor review. Please capture your discussions within the assessment tool to help facilitate the registrar’s reviews.

MiniCEX (Clinical Evaluation Exercise)

The Clinical Evaluation Exercise (miniCEX) assesses clinical skills, attitudes and behaviours in a non-primary care setting. The miniCEX should provide a 15-minute snapshot of how the registrar interacts with patients in a non-primary care setting.

Each miniCEX should represent a different clinical problem. It is helpful to vary the types of cases that are assessed using miniCEX so that the registrars’ competence is reviewed on different challenges. One miniCEX should be assessed by the clinical supervisor.

Other assessments can be completed by a doctor who is at a level of ST4 or above, or SAS equivalent. The observer should not be a GP registrar or specialty registrar at a similar stage in training.

The assessor should give the registrar immediate feedback and then provide a contemporaneous report, rating the registrar and capturing the feedback within the MiniCEX form in the Portfolio. Some assessors will have full access to Portfolio but in non GP settings many will have to be sent a ticket code by the registrar to enable the report to be completed. When assessors have provided more detailed written feedback on the MiniCEX this has been very helpful evidence for the ESR.

When in a hospital setting registrars are rated in comparison to other registrars at the same stage.

Clinical Examination and Procedural Skills (CEPS)

The development and assessment of Clinical Examination and Procedural Skills (CEPS) is an extremely important part of GP training. Competence in these skills is integral to good clinical practice. Registrars need to gather evidence of their clinical skills through several different assessments and reflective log entries.

The registrar must be observed performing these examinations. The assessor should record their observation on the CEPS evidence form on the registrar’s Portfolio. (The registrar can provide a ‘ticket code’ via an email if the assessor does not have personal access.) It is the individual patient who determines what is intimate or invasive for them and this will be determined by a number of possible factors including their prior experiences, their religion and their cultural background.

The registrar is expected to document their performance in CEPS in their learning log and /or discuss their learning needs during placement planning meetings with their Clinical Supervisor and at their 6 monthly reviews with their Educational Supervisor. The range of examinations, procedures and the number of observations will depend on the registrar’s particular requirements and the professional judgement of the registrar’s clinical and educational supervisors. For example, the registrar may recognise that their learning needs are joint examinations, the examination of the eye or doing newborn baby checks. It is expected that a registrar completes a range of CEPS in each post that are relevant to that post – for example, if a registrar is working in paediatrics then it would be appropriate for them to provide evidence of completing a baby check.

If the registrar is performing below the level expected in this domain, it is important for the assessor to be specific about why this conclusion has been reached and what steps the registrar can take to rectify the situation.

Unlike other hospital assessments the standard that the assessor is assessing against is that of an independent fully qualified GP. As well as the technical aspects of examination and the ability to recognise abnormal physical signs, it includes the choice of examination best suited to the clinical context. For example, a competent GP very rarely performs an extensive neurological examination but will perform a limited neurological examination as determined by the history taken from the patient.

Clinical Supervisor's Report (CSR)

The Clinical Supervisor's Report (CSR) is a short, structured report completed by the Clinical Supervisor towards the end of the registrars’ placement. It is an opportunity for the CS to give the registrar feedback on their performance. A well completed CSR is also a valuable source of evidence for each capability in the Educational Supervisor Review and for ARCP panels.

The CSR makes a clear link between each section and the relevant GP capabilities and includes an overall assessment by the Clinical Supervisor (CS) of the level of supervision that the registrar has required.

Who carries out the CSR?

The Clinical Supervisor is responsible for writing the report although it is appropriate and usual for the Clinical Supervisor to discuss the CSR with colleagues to inform the final report. In addition to this gathering of information from colleagues it is expected that the Clinical Supervisor  will have carried out at least one of the mandatory Work Place Based Assessments personally (CBD/MiniCEX) prior to each CSR. Where there are particular concerns about the registrar's progress and there is more than one experienced CS working in the department or practice, it is appropriate for there to be more than one CSR written for a single period of training.

What does the form review?

Each of the questions covers a particular area of practice, for example Professionalism. There follows a description of how this is likely to be observed in the working environment. Professionalism, for example, includes being respectful, diligent and self-directed in the registrar’s approach to patients and others, developing resilience and making appropriate ethical decisions. Each question will automatically be linked to specific Capabilities in the Portfolio (e.g. Maintaining Performance Learning and Teaching, Ethics, Fitness to practice).

Word descriptors have been written to support the grading and feedback for each question which are available as an appendix here

The CS is also asked to make an assessment of the level of supervision required compared to the expected level of performance for a GP registrar at this stage. There are 4 levels of supervision and if more supervision than would be expected is required, or the registrar cannot be left without supervision, then an additional comment box will appear asking for further details.

Finally, in line with all other specialties there is a question about whether the registrar has been involved Significant Events and what the outcome has been if so.

Short Placements (for example 3 months or less)

It is particularly important that a CSR is completed if the registrar has had a short placement so that there is an assessment of engagement and learning in the post. (It would also be expected that there should be pro rata assessments (CBD/ MiniCEX/ COT) for these placements).

Multi-Source Feedback (MSF)

Registrars are required to complete one cycle of Multi-Source Feedback (MSF) during their ST1 and ST2 years. The registrar must get responses from at least ten people who know the registrar’s work well; they should come from a range of roles and include people with a range of seniority. Ideally the registrar should have responses from 5 clinical and 5 non-clinical roles. The registrar’s educational supervisor reviews the responses, meets with the registrar to give feedback then releases the information to the registrar.

People completing an MSF are asked to rate the registrar on their professional behaviour and clinical performance. A free text box asks for examples / evidence with suggestions of areas for consideration. Raters are encouraged to be honest in their feedback and to provide examples where possible in the feedback boxes as this greatly improves the value of the feedback.