SMJ
2003 48(2): 27-31
Alan
A. Connacher, QIan
K. Ritchie, +Gellisse Bagnall, #Diane Quinn
Department
of Medicine, Perth Royal Infirmary, Perth, QDepartment
of Orthopaedic Surgery, Stirling Royal Infirmary, Stirling, +NHS
Education for Scotland – West Region and #NHS Education for
Scotland – East Region.
All authors are on the faculty of Supporting Clinicians on Training in Scotland (SCOTS)
Author
responsible for correspondence: Dr Alan A. Connacher
Correspondence
to: Dr Alan A. Connacher, SCOTS Director, Department of Medicine, Perth Royal
Infirmary, Perth PH1 1NX.
email:
aconnacher@pri.tuht.scot.nhs.uk
Consultant
Appraisal is here to stay. It has been introduced rapidly for hospital doctors
throughout the UK but with Scottish packaging for those working in the NHS in
Scotland. It is one component of the clinical governance and quality agenda that
has expanded within the NHS in recent years. The drivers for its introduction
have come from the general public and from politicians in the wake of
high-profile legal cases including Shipman and Bristol. However, Consultant
Appraisal cannot be expected to weed out all the bad apples and identify and
manage all poor performance. Indeed
these are not the issues that Consultant Appraisal should deal with.
Well-established mechanisms already exist which should be invoked at any
time to deal with matters of conduct or performance, as and when they arise. It
would be wholly inappropriate to wait for an annual appraisal before dealing
with such potentially serious issues.
Appraisal
is about reflecting on recent work activity, identifying strengths and
weaknesses, setting reasonable and achievable objectives and looking to the
future for the benefit of the appraisee and their employing organisation.
However, in business and industry there is wide variation on how these
principles have been applied, with at one extreme the process being very
management orientated with little concern for confidentiality and sometimes with
a direct link to remuneration. In hospital medicine appraisal is not a new
concept and has been widely used with trainees, but only occasionally for senior
staff. There is widespread agreement in medical education circles that appraisal
should be towards the opposite end of the spectrum.1 It should be
focussed on the appraisee, facilitate reflection, encourage personal
development, be confidential and have no link to remuneration.
It
is also necessary to comment on how educational or peer appraisal in medicine
contrasts with assessment since there is still confusion even at the highest
supervisory and management levels. Assessment involves making judgements, which
may be interim or final, about an individual against externally defined
criteria.1 There are well-recognised difficulties over
standardisation, validity and reproducibility. The assessment is often a test or
examination, may be done by a number of individuals and usually results in a
score, a ranking or a pass/fail adjudication that can become public knowledge.
Therefore the result of an assessment can be either a significant barrier or a
welcome opening to career progression. None of this applies to appraisal which
should be incremental, evolutionary year-on-year and should always focus on the
appraisee’s personal development plans.
Despite
the momentum now associated with Consultant Appraisal, it remains apt to ask
what evidence exists to demonstrate that it is worthwhile? There is certainly
very little evidence to show that regular systematic appraisal of senior
hospital medical staff is beneficial to the individual or to the employing
Trust. But there is strong evidence from across all employment sectors that
giving feedback and setting appropriate objectives results in improved employee
performance.2 A recent study has shown that the greater the extent
and sophistication of appraisal within hospitals is a predictor for low patient
mortality.3 The value of appraisal has been shown to depend on
whether the appraiser is skilled in conducting appraisal interviews and is
supportive, focussed on the future and participative.4,5 Further,
appraisal that is undertaken badly has a negative effect on performance,45 so
the selection of appraisers is crucial to success.
Finally,
and arguably most persuasively, the General Medical Council has indicated that
annual Consultant Appraisal is likely to be the usual mechanism for hospital
doctors working in the NHS to gather information towards part or all of the five
yearly process of Revalidation.
Over
the past eighteen months there have been intensive efforts made to pave the way
for Consultant Appraisal, particularly through the provision of training courses
for ‘local experts’, appraisers and appraisees. There has been substantial
input from the faculty of the Supporting Clinicians On Training in Scotland
(SCOTS) initiative. This review outlines these processes and discusses the
lessons that have been learnt along the way.
The
British Medical Association and the four UK Departments of Health reached an
agreement on Consultant Appraisal by early 2001 and guidance was issued in
Scotland under NHS Circulars, initially for all consultant staff, 6
and then specifically for consultants in Public Health Medicine.7 The
process has been described as compulsory, mandatory and as a contractual
requirement. This seems to be a reasonable interpretation given the clear
responsibility for implementation given to Chief Executives as part of clinical
governance, the proposed intimate link to revalidation and the direction to
inform discretionary point and distinction award committees of any
non-participating consultants.
The
guidance states that it is necessary to use standard documentation, but the
templates given are not user-friendly and contain areas of duplication. It may
be possible for amendments to be made before too long. Every consultant is
encouraged to compile an appraisal folder to keep all paperwork relevant to
Consultant Appraisal. The process is predicated on the existence of job plans
that can be reviewed annually, although it is well known that there are large
numbers of consultants who do not have an up-to-date and accurate job plan.
The
template which is intended for inclusion in the appraisal folder is a series of
six forms. Form 1 records background details pertaining to the individual and
their career and professional status. Form 2 records details of current medical
activities in NHS and other sectors, it relates to the job plan, encourages a
wide look at all professional activities and can log issues of resource. Form 3
is the substance of the documentation. It is the record of reference
documentation supporting the appraisal and is structured in line with the
GMC’s Good Medical Practice.8 The seven headings are Good
Medical Care, Maintaining Good Medical Practice, Working Relationships with
Colleagues, Relations with Patients, Teaching and Training, Probity and Health.
It then records the supporting documentation for Management Activity and
Research. It concludes with a review of the previous year’s Personal
Development Plan.
Form
4 is a summary of the appraisal discussion again based on the seven areas
mentioned above, with an additional section to record any other points that
arise. There is then a Personal Development Plan to complete that should
identify key development objectives for the following year. This form should be
completed by the appraiser, probably at the appraisal meeting. The contents need
to be agreed by both appraisee and appraiser, and signed off to this effect by
both parties as a true record of the appraisal discussion. It is this document
that will be passed on to the Trust Chief Executive.
Form 5 is about personal and organisational effectiveness. It encourages
appraisees to describe their effectiveness both on a personal level and within
the context of their NHS workplace, with a view to informing future job plan
reviews. Supporting documentation is to be listed on the form. Unfortunately
there is considerable overlap with Form 3. Form 6 provides an opportunity, if
required, to record a fuller, more detailed account of the appraisal
discussion than is recorded on Form 4 to help with the next appraisal. It is
however confidential and the contents are not passed on to the Trust Chief
Executive. There is a caution to avoid commenting on third parties and not using
the form as a means of documenting concerns about poor performance of colleagues
since the latter should be dealt with under separate procedures.
In
response to the agreement on Consultant Appraisal between the UK Health
Departments and the BMA and the subsequent Circulars, a meeting was held with
the BMA and the SEHD in March 2001. The BMA suggested that training should be
delivered by consultants for consultants. Trusts were each asked to provide
nominations for a Core Group, and about 70 names were received from various
backgrounds which included Trust Managers, Clinical Directors and Lead
Clinicians, Trust Consultant Appraisal Steering Group Members and some
individuals with a special interest in or past experience of appraisal. This
Core Group first met at a Programme Design Session in August 2001.
At
this meeting key issues were discussed and these included
·
The time and resource needed to undertake Consultant Appraisal.
·
How would information be gathered and recorded?
·
What could be the role of Information Technology in the process?
·
Can the process be started in a relatively simple form to be built on
over time?
·
The weaknesses of the template that must be used (Forms 1-6).
·
The need for preparatory work to be undertaken to update Job Plans.
·
Who does the appraising?
It
was agreed that it would be useful to have a brief printed guide to Consultant
Appraisal.9 This was formulated by a smaller Design Group and then
commissioned and distributed by the Scottish Executive (SE).
The
group also realised that in order for the initiative to succeed, training was of
paramount importance, not only for the Core Group but also for all consultants
who would become involved, be they appraisers or appraisees. However, it was
seen as a priority for the Core Group to be trained so that they could develop a
level of expertise that would be useful back at their individual Trusts.
In
the Spring of 2002, two-day Core Group Training Programmes were run on four
occasions in Glasgow and Edinburgh. The Core Group had by this time been
expanded to about 140 members, most of whom attended for the training. The
format was varied, interaction and participation were encouraged and the content
covered both the knowledge-based needs and the interpersonal skills pertinent to
successful appraisal interviewing.
Before
the meeting took place attendees were asked by questionnaire what issues were
thought to be important and what were the areas of potential difficulty (Table
1). Interestingly, in the 116 questionnaires returned, only 32 intimated
that they had experience of regular appraisal and most had this in a management
or university role. Further there were 18 individuals undertaking appraisals who
had never been appraised themselves. These courses were generally well received
with most participants more confident in their role as local expert and better
equipped to make a contribution to Trust Consultant Appraisal Steering Groups.
HR
Staff from the Strategic Change Unit of the Scottish Executive delivered these
two-hour sessions for any Trust on as many occasions as were requested. The
sessions ran during the Spring of 2002 which, for most of the Consultants
present, was very early in the whole process. Not surprisingly, the responses
were mixed because there were a number of anxieties to be discussed and
questions to be answered. The format of these meetings was a presentation and
discussion. The presentation covered the background, process, documentation and
the link with GMC Revalidation. The issues raised during discussion at these
sessions were logged and collated. Interesting examples include:
·
How will time for the process be created within the current working week?
·
There will be knock-on effects for waiting times and lists.
·
There is poor overall availability of current job plans.
·
How many individuals can one person appraise?
·
Are there financial resources to support the process?
·
Are there any potential legal implications for appraisers?
·
There is a need for IT systems to provide information for the process.
·
Will it be possible to keep an electronic portfolio e.g. Cyber Medical
College?
·
There is a need for readily
available training courses for all.
With
time the discussions have matured, some questions have been answered but other
questions remain!
The
Supporting Clinicians On Training in Scotland (SCOTS) initiative is a well
established group run through NHS Education for Scotland and with close links to
the three Scottish Royal Colleges. In recent years it has successfully delivered
training courses for Educational Supervisors that have included appraisal skills
development. With this background it was agreed that SCOTS would develop a
one-day course on Consultant Appraisal to be made available to appraisers and
appraisees throughout the country. The courses have been arranged with
individual Trusts and delivered by a small faculty group of experienced
Consultants and Education and Training Staff from each of the Scottish
Deaneries.
The
aims for these courses were
·
to examine terminology and definitions
·
to explore questions, fears and anxieties
·
to scrutinise the documentation
·
to emphasise good appraisal skills
·
to allow each participant to experience the appraisal process.
This
was achieved using a range of educational methods, including large group
discussions, small group work and role-play. There was close focus on GMC Good
Medical Practice and the link to Revalidation.
All
but two of the Trusts in Scotland have requested courses to be run for their
Consultants. Between April and December 2002 sixty-three courses were provided
and 1300 Consultants attended. The courses have been very well received by the
participants with most enjoying the interactive format, the discussion
opportunities and taking away a degree of reassurance about the potential merits
of the Consultant Appraisal process. Every participant was asked to complete a
short evaluation form at the end of the course and to date the return rate has
been 90%. The form rates each section of the course for delivery and content on
a 1-5 scale where 1=Poor and 5=Excellent. More importantly the evaluation form
asked for open-ended comments on the issues raised in each section of the
course.
This
simple evaluation showed that the average scores for each component of the
course were consistently above the middle of the five-point scale. For the Introduction
and Background the average score was 3.7 (range 2.7-4.3), for Terminology
and Concerns it was 3.7 (2.7-4.1), for Documentation
it was 3.4 (2.9-4.1), for Role Play it
was 3.7 (3.1-4.4) and for the Concluding
Discussion it was 3.6 (2.8-4.1). The qualitative feedback through the
open-ended comments was extensive but has been clustered into commonly occurring
themes – see Table 2.
Comments were made on the course and on the issues arising during the courses,
some positive, some negative and yet others aimed at the overall strategy and
political drive behind the introduction of Consultant Appraisal. Many of the
themes that arose were similar to those identified by the Core Group prior to
their two-day Training Programmes.
Further
SCOTS training courses are being provided between January and March 2003 by
which time approximately two-thirds of Scotland’s Consultants will have
attended. Currently, the number one priority is to get Consultant Appraisal up
and running by April 2003. This would be within one year of the target set in
the SE Circular. The system cannot be all-singing and all-dancing this year but
if it succeeds and grows over the years it will gain in complexity and become
more meaningful. Remember it will be at least 2005 before the first round of GMC
Revalidation and presumably R-DAY could be as much as a further five years hence
for some practitioners. So there is time for Consultant Appraisal to evolve into
the main vehicle for five-yearly Revalidation. To facilitate this, Trusts will
need to give the process high priority and this will include giving their
Consultants the necessary time, support and advice. It will need to include the
means by which the appropriate information is gathered for each Consultant
through engagement with Medical Records, IM&T Departments etc. Clearly
Trusts will also need to make available to their staff a Consultant Appraisal
Folder in hard copy and/or in electronic format.
The
GMC has not at the time of writing made a definitive statement on its
requirements for Revalidation. However, it has embraced Consultant Appraisal as
the preferred route for Consultants and has piloted the system. The areas
highlighted in the medical press as being deficient for Revalidation are Working Relationships with Colleagues and Relations with Patients. There seems to be a pressing need to
develop and pilot 360º Appraisal Tools10 for these purposes and to
make them freely available if this is the solution.
With
regard to Clinical Academics, the Follett Report11 included direction
that University Staff should undergo joint University and NHS Appraisal and a
separate SE Letter12 with Guidelines was issued in October 2002. Some
Clinical Academics have attended for training with NHS Consultants while for
others there are plans for separate SCOTS Courses. The precise detail is not of
great importance since the process for Clinical Academics is virtually identical
to that for NHS Consultants, albeit with a greater emphasis on teaching and
research. The one area of concern to many with experience of true appraisal
interviews is the recommendation that Clinical Academics will have Appraisal
conducted jointly by a University and an NHS appointee except where, by mutual
consent of all three parties, a sole appraiser covers all the ground. This
detracts from the one-on-one, two-way and confidential nature of the best
appraisal systems. It would surely be possible for both appraisers to input to
the system up to and including the exchange of information prior to the
appraisal meeting that itself is conducted on a one-to-one basis. This will need
further thought and discussion.
Appraisal
is also being introduced for Non-Consultant Career Grade (NCCG) Doctors13 and
again national agreement has been reached with the BMA. It will become a
mandatory contractual requirement and all appraisals will be recorded on the
national documentation that is contained within the SE Circular13.
The grades to which this applies include staff grades, associate specialists,
clinical assistants, hospital practitioners, CMOs and SCMOs and all other
non-consultant doctors employed on local contracts. Notably it also applies to
locum doctors who have been in post for more than two months. All those involved
in the process, appraisers and appraisees, should receive appropriate training,
but there is no plan to repeat the national training initiative that was
undertaken for Consultant Appraisal. There should now be local experts and
Consultant Appraisal Steering Groups within each Trust that have the necessary
experience to undertake training for NCCG Doctors at a local level since the
guidance and the documentation are virtually identical to that for Consultant
Appraisal.
Over
the past year there has been a huge training effort in preparation for
Consultant Appraisal. During this time there have been many issues debated but
there have been some recurring and important themes that have emerged and that
are worthy of comment by way of conclusion. Clearly the bare bones of Consultant
Appraisal, as written in the SE Circulars, has caused anxiety for some. However,
it is heartening to note that HR staff from the Scottish Executive have made it
clear that the process must be developmental and educational, and that the
documentation is open to interpretation so that it gains the confidence of the
Profession. This is the framework in which all of the training has been
delivered and from which the following comments have been synthesised.
·
Consultant Appraisal should be focussed on the appraisee with the outcome
of the process, the Personal Development Plan, being used by the individual as a
platform to pursue and seek support for meeting the agreed objectives over the
following year.
·
If the process is used by managers for management as an in-depth
performance review and nothing else it is doomed to failure and at best will
become an exercise to be completed in a minimalist fashion and as quickly and as
perfunctorily as possible.
·
It is crucial to select the best individuals for the role of appraiser.
This means those with good interpersonal, listening and communication skills who
have had specific training in Consultant Appraisal. They will not always be
those in Trust line-management positions and senior managers should have enough
confidence in their Consultant colleagues to delegate without abdicating
responsibility.
·
The process will require facilitation by management, not least so that
the time and effort involved is recognised and given due priority within already
over-stretched working schedules, and not as another extra. Appraisers should
not be over-burdened, with most authorities suggesting a maximum of about six
appraisals in one year.
·
There must be an outcome from Consultant Appraisal beyond that for
individuals. The Form 4 returns should be analysed and anonymous summary
information presented to Trust Boards so that recurring or serious issues that
arise can be dealt with through prioritisation since there is no direct
allocation of extra resource for Consultant Appraisal. Similarly, central
monitoring of the outcomes from all Trusts should be undertaken.
·
Consultants should now be engaged in Consultant Appraisal even though it
is not yet developed comprehensively. An evolutionary approach is required and
with this the confidence of the Consultant body should be won and the system
held in high regard.
Many
appraisal systems used in other sectors have failed rapidly, so let us embrace
the above messages, get the show on the road and give it the chance to succeed.
Acknowledgements
In
addition to the authors, all of the following have contributed to Consultant
Appraisal Training in Scotland:
John
Anderson, Graham Buckley, Donald Cameron, Shona Cowan, Kerry Chalmers, Stephen
Gallacher, Steven Haddow, George Harvey, John Hiscox, Bob Ironside, Peter
Johnston, Liz Kelly, Sean Kelly, Barry Klassen, David Large, Matty Lough, Bill
McKerrow, Patrick McKinley, Rose Martin, Richard Metcalfe, Rodney Mountain, Ray
Newton, Margaret Noble, Jess Panesar, Bill Reid, Claire Robb, Elizabeth
Robertson, Jane Ross, Neil Stevenson, Kim Walker, Karen Watson, Roger White,
Alan Wood
REFERENCES
1
Jolly B and Grant J. The Good Assessment Guide. Joint Centre for Education in
Medicine, London 1997.
2
Ilgen DR, Fisher CD, Taylor MS. Consequences of individual feedback on behaviour
in organizations. J Appl Psychol 1979; 64: 349-71.
3
West MA, Borrill CS, Dawson JF, Scull J, Carter M, Anelay S, et al. The link
between the management of employees and patient mortality in acute hospitals.
Int J Human Resource Management 2002; 13: 1299-1310.
4
Murphy KR, Cleveland JN. Understanding performance appraisal. London: Sage,
1995.
5
West MA. How can good performance among doctors be maintained? Department of
Health’s proposals are wise but need to be implemented with care. BMJ 2002;
325: 669-70.
6
Scottish Executive Health Department. Consultants’ Contract: Annual Appraisal
for Consultants. NHS Circulars PCS (DD)2001/2 and PCS(DD)2001/7.
7
Scottish Executive Health Department. Annual Appraisal for Consultants:
Consultants in Public Health Medicine. NHS Circular PCS(DD)2002/1.
8
General Medical Council. Good Medical Practice. GMC, London 2001. www.gmc-uk.org
9
Scottish Executive Health Department. Consultant Appraisal: A Brief Guide. www.scotland.gov.uk
10
King J. 3600 Appraisal. BMJ 2002; 324:S195-6.
11
Department for Education and Skills. A Review of Appraisal, Disciplinary and
Reporting Arrangements for Senior NHS and University Staff with Academic and
Clinical Duties. A report to the Secretary of State for Education and Skills, by
Professor Sir Brian Follett and Michael Paulson-Ellis, September 2001.
12
Scottish Executive Health Department. Annual Appraisal for Clinical Academic
Consultants: The Follett Review. Letter, 28 October 2002.
13 Scottish Executive Health Department. Annual Appraisal for Non-Consultant Career Grade (NCCG) Doctors. NHS Circular PCS(DD)2002/7.