Do psychological tests have any real value?

Think about your most troublesome colleague or junior doctor. What is the root of the problem? Is it their clinical skills or knowledge - or the fact that they constantly upset the nursing staff, or are hopelessly disorganised, or seem unable to make decisions?

Perhaps you are a senior hospital doctor who has worked in the NHS for 20 years. The recent relentless changes and government directives have taken their toll and you are becoming increasingly frustrated with medicine. At this late stage in your career, what else could you do? What are you best suited to and what further skills, qualities or experience might you need to move into something new?

Perhaps you are a senior hospital doctor who has worked in the NHS for 20 years. The recent relentless changes and government directives have taken their toll and you are becoming increasingly frustrated with medicine. At this late stage in your career, what else could you do? What are you best suited to and what further skills, qualities or experience might you need to move into something new?

What could account for these difficulties? One way of diagnosing these underlying causes is through a skilled and in-depth interview which explores in detail the way in which the doctor works, their motivation, style of communication and leadership, mechanism for handling stress and pressure, and so on.

However, some attributes, for example, anxiety, vulnerability, lack of trust, may not easily be detected through an interview. This is where psychometric measures have an important rule - not to 'catch people out' but to help the doctor, and the employer, to understand what is causing the problem and to then consider the most appropriate ways to support and address the problem.

What is psychometric testing?

Psychometric testing was used originally in the armed services to select new recruits. Psychological tests of ability and personality went on to become popular in industry in the early 1980s.

In their quest for 'best practice' and 'level playing fields', human resource directors and personnel managers seized upon the psychological tests as a way to make selection more fair and transparent, and to help minimise the risks of recruiting or promoting the wrong individuals.

Psychometric tests include measures of personality, ability and intelligence that allow you to assess and predict competence, motivation and behaviour. However, it is a misnomer to refer to all such measures as 'tests' - especially as the word itself provokes anxiety. A test is a measure of ability (such as verbal reasoning). It has right or wrong answers and is completed within a time limit.

Measures of personality, on the other hand, are questionnaires rather than tests - they have no right or wrong answers, no 'good' or 'bad' profile and are generally untimed. They measure personal values, attitudes, beliefs, motivation, preferences and needs. For example, they can be used to assess behavioural factors in poorly performing doctors.

Well-researched measures allow results to be compared with a norm group. This is a typical comparison group - for example, other managers, graduates, doctors or the general male/female population.

Psychometric testing for recruitment and selection

How often have you heard a colleague say: 'If only we had spotted this at his interview we would never have taken him on', or: 'There was no sign of this in her CV or when we interviewed her. How could we have known she would cause such problems?'

Evidence suggests that problems experienced with junior doctors, for example, are less to do with their clinical skills and more often to do with their personal attributes.

Yet recruitment and selection these days is a legal minefield. Anxious not to fall foul of employment law, personnel departments and interview panels feel increasingly constrained about what they are allowed to ask a candidate. The result is a safe interview but one which is largely ineffective in terms of providing any real understanding of an individual's characteristics, motivation and ability.

For example, a young, bright SpR applies for her first consultant post. She is clinically competent, ambitious, enthusiastic and assertive in her manner.

At interview, however, she alludes in passing to one or two differences of opinion with former colleagues. She also mentions that she enjoys being in control and prefers to make decisions herself. She is keen to do some research - but has little track record to date.

How bright is she? How stable is she under pressure and stress? How does she resolve conflict with colleagues? What influences the way she makes decisions and judgements? A well-chosen battery of psychometric measures can shed further light on these questions.

A general personality questionnaire, for example, can help to identify an individual's emotional stability, need for variety and stimulation, openness to new ideas and opinions, style of influencing others, willingness to lead and style of teamworking.

Psychometric testing for career development and guidance

Or look at this scenario. You are a consultant with an increasingly 'difficult' SHO whom you are currently supervising. She has completed all her hospital jobs and is struggling to decide which specialty she wants to pursue. She is losing confidence in her own abilities and does not know where she fits in. She has twice failed her MRCP Part 1.

This is a useful example of where psychometric testing can help a consultant supervisor gain deeper insight into an SHO's difficulties.

First, some reasoning ability tests would help to confirm her intellectual capacity. Has she failed her exam because she is not bright enough or because of other factors?

Secondly, a 'career beliefs' questionnaire would help to identify what motivates her in a career. For example, is she motivated by an environment where she can be independent, or where she can be part of a team? How ambitious is she? Is she driven by status and recognition, by money or by wanting to care for patients?

Thirdly, a general personality questionnaire would identify all the factors mentioned in the recruitment scenario - specifically, how open, flexible, conscientious and resilient she is.

Finally, a further targeted measure could provide an insight into the way the trainee makes decisions and judgements, what kinds of working relationships she would best respond to or initiate, and even her preferred style of learning.

Taken together, these measures will give a clear indication of potential strengths, limitations and development needs.

The same applies to the earlier example of a senior consultant looking for a possible change in his career. If he wanted to move to a pharmaceutical company in a medical advisory role, what attributes, experience and skills would he need? The psychometric measures would help him assess how closely his profile matches the requirements for that kind of job or career.

Can anyone use these tests?

Would you give a tray of surgical instruments to an untrained individual and ask him or her to choose the right ones for an operation? Or allow an untrained individual to interpret an X-ray or scan, or diagnose and treat a health problem in a colleague?

How much knowledge, skill, experience or training would that individual need? What might be the consequences of allowing an untrained person to carry out these tasks?

By the same token, effective and responsible psychological testing requires training and skill in selecting appropriate measures, administering them in a standardised way (to reduce bias) and, most importantly, scoring and interpreting them appropriately and accurately.

Of these four activities, those that require professional skill are selecting the right measures and interpreting the results. Does this mean you need to call in a chartered psychologist every time you want to use psychological tests?

The British Psychological Society has a strict code of conduct that specifies different levels of competence required for test administration and interpretation. In addition, the test publishers and test agencies that supply most commonly used measures have conditions under which you can purchase their materials, such as attending their qualifying workshops or having specific professional qualifications or previous experience.

Can I make a decision based only on psychological test information?

Again, consider the medical analogy. Would you treat a patient based only on a health questionnaire completed by that patient or others? If not, what else would you need to know? And how would you find out?

Similarly, would you make a major career decision based only on the results of your psychological tests and questionnaires? Of course not.

There is substantial evidence that a good interview combined with appropriate psychometric testing is the most predictive of future behaviour and certainly more predictive than either tests or interview alone.

Finally, it is important to emphasise that psychological tests can aid and inform judgement about an individual, but do not in themselves provide the decision, solution or conclusion to any issues.

The implications of the results need to be considered by a skilful professional or team. Wherever possible, this feedback should be discussed with the candidate.

Feedback must be handled with care and sensitivity to encourage the individual to accept the results, reduce defensiveness and increase self-awareness.

Making the most of psychometric feedback

There is no good or bad profile. Every characteristic has a positive and a negative side. A headstrong and aggressive individual will get things done and will be undeterred by difficulties. But she may alienate people and find teamworking difficult.

A warm, shy individual may be well liked but may make little contribution in meetings. A wild extrovert may be a great team player and a riot on the wards, but may be easily distractible and demand attention.

How you make sense of this information largely depends on what you want it for in the first place. If it is to help you appoint a new consultant, you need to be very clear about the kind of person you are seeking and then see how closely the candidate matches these characteristics.

Do you want someone who is technically and clinically brilliant, operates flawlessly but who has no respect for his colleagues or empathy for his patients? Do you want someone who is a great team player but who has trouble making tough decisions?

No candidate is perfect, and sometimes they fall short in some areas but their other qualities more than compensate. The psychometric data will help to reveal potential risks. You will also need to identify areas that can be developed or trained, such as communication skills, and those that are relatively fixed, such as interest in people.

For poorly performing doctors, psychometric data, when combined with other information, should be used to help the doctor recognise the problem, understand its cause(s) and - most importantly - agree an acceptable and practical way forward.

Our experience is that, after initial anxiety and scepticism, most doctors find this kind of information about themselves more revealing than they anticipated and they are often surprised by the level of understanding that can emerge from looking at the patterns across all the data sources. "So it's because of this that I keep doing that . . . I hadn't ever made the connection."

If you want to use the psychometric data for career development purposes, the feed- back will require more of a counselling approach. Many measures lend themselves more to counselling than selection because they are more likely to elicit honest answers.

Can you fake your own results?

Most robust and well validated measures cannot be easily falsified. Ultimately it depends upon the motivation of the doctor to present an accurate picture of themselves. There is obviously scope for bias if using the measures for recruitment or for handling performance problems.

For this reason, however robust the measure, they should never be used in isolation. It is essential to garner other sources of information about an individual, through direct observation of their clinical practice for example.

A professional assessor will try to establish a good relationship with the candidate before testing begins and will have the skills to confront any discrepancies that might emerge between the different sources of data.

So what's the verdict about psychometric testing?

In skilled hands these measures are potentially powerful, helpful, and can add more to our understanding of an individual or team than interviews alone. In unskilled hands, however, they can at best be misleading or at worst harmful.

Yet, to dismiss all psychological testing out of hand, is to miss the valuable contribution that it can make to our understanding of ourselves and others. Most of all, they should not be used to make any final decisions about an individual - but rather to help establish some consistent themes, patterns or trends in an individual's performance that can inform a decision or a development plan rather than dictate it.