In a second article on the contentious friends and family test, Rachel Davis and colleagues look at why most doctors and nurses disagree with its scoring method

Family

Is the friends and family test benchmark worthy?

Since April last year it has been a contractual requirement for all NHS trusts providing adult inpatient and accident and emergency care to collect data on the friends and family test.

‘Trust staff have significant concerns about the scoring method’

While the government has hailed the test as having a “transformative effect on patient care”, it has attracted strong criticism from others.

Following a review that found the test was “widely misunderstood” by staff and the public, and concerns about the data being “gamed”, NHS England yesterday announced changes to how the test is implemented and presented from April 2015.

Research commissioned by the Care Quality Commission reported that the original test – which asks patients how likely they would be to recommend a hospital or A&E department to friends and family – lacks validity, is open to misinterpretation and is not appropriate for use in the NHS. 

Concerns surrounding the transparency with which results can be compared have also been raised due to the differing methods of data collection and huge variations in response rates between hospitals.

Misleading method

Aside from these criticisms, one of the most pressing concerns is the net promoter scoring method used to calculate the headline metric.

‘Opponents have questioned the value of the test score and say it paints an inaccurate picture’

A score of between -100 and +100 is produced by categorising patients’ responses as “promoters” - those “extremely likely” to recommend a service; neutrals - “likely” to recommend; and detractors – “neither likely nor unlikely”, “unlikely” or “extremely unlikely”.

Research has found that the general public have rejected this method for being over-complicated and misleading. The same research revealed that trust staff including chief directors of nursing, patient experience leads and communications professionals also have significant concerns about it. Despite this, the scoring methodology was employed, with monthly results released online on NHS England and NHS Choices.

Opponents of the approach have already questioned the value of the test score and say it paints an inaccurate picture of hospitals’ results, which could be grossly misleading to those who view it.

Clinicians’ view

Importantly, one notable gap in the evidence base is how doctors and nurses view the scoring method, and whether they echo the concerns raised by others. Understanding this is important for several reasons.

First, healthcare professionals will often be the ones initially approaching patients and explaining the purpose of the test; if they question its credibility, their motivation to do this may be challenged.

‘How the test is fed back will affect how doctors and nurses interpret it and, in turn, their inclination to take action’

Second, if results are displayed on hospital wards, patients may ask staff about the meaning of the score. Currently, however, NHS guidance for staff on friends and family fails to mention the scoring method, so the extent to which healthcare professionals are even aware of how the test score is derived is unclear.

Third - and perhaps most importantly - the way in which ward level data on the test is fed back will affect how doctors and nurses interpret it and, in turn, their inclination to take action based on the findings.

Pouring over scoring

Here, we present our initial findings on a much larger programme of work on the perspectives of frontline staff on the test. The aim is to build on Ipsos MORI’s research by revisiting the main scoring options originally proposed for friends and family and examining doctors’ and nurses’ attitudes towards them.

We collected data from 183 doctors and nurses from surgical and medical wards, teaching sessions and theatres across six hospitals in London and the Midlands. It was found that presenting the test headline metric as the percentage of patients that would recommend a hospital - “extremely likely” or “likely” - was the most preferred option, whereas the net promoter score methodologies were least favoured.

Of the 183 respondents, 115 provided feedback on their reasons for choosing their preferred scoring method and 60 also took the opportunity to provide their rationale against the other scoring options.

Approaches employing the score were frequently branded as “confusing” and “illogical”, whereas methods using percentages were considered “simple” and “self-explanatory”.

In addition, while some healthcare professionals viewed the net score favourably, stating it was the “fairest way of calculating the test result”, some disliked the fact it could produce a score with a minus sign.

‘Approaches employing the net promoter score were frequently branded as “confusing” and “illogical”’

General feedback from respondents further strengthened this view, with 45 (25 per cent) stating that scoring options that could produce a positive or negative score - from -100 to +100 - could be difficult to understand, particularly among patients and the general public.

Our results show that scoring options employing the current net promoter method were viewed least favourably by 81 per cent of individuals.

Only 2 per cent felt the method - adopted by the Department of Health - was the best option for scoring the test.

Conversely, 69 per cent felt calculating the percentage of patients that recommend a hospital/ward - “extremely likely” or “likely” - would be the best way of presenting the test score, with many commenting that it was self-explanatory and widely understood by all.  

Benchmark worthy or not?

A number of conclusions can be drawn from these findings.

First, doctors and nurses appear to mirror the views of the general public and non-frontline NHS staff that the test’s scoring methodology is unintuitive, unnecessarily complicated and misleading.

Second, they echo previous arguments against using calculation methods that could produce a negative score for friends and family - in this case, a range between -100 to +100 - as this could lead to confusion around what the test score actually means.

‘The public may misinterpret the results, leading to misinformed decisions about healthcare providers’

Third, the opinion that communicating the test score as a percentage would enable greater transparency on the quality of healthcare services - particularly for patients and the general public - is further confirmed.

A lack of support could lead to healthcare professionals becoming dismissive of the data, as well as any suggested improvements resulting from the findings.

Equally, given how “complicated” the general public find the test score, they may misinterpret the results, leading to misinformed decisions about healthcare providers.

From a commissioning perspective, this also brings into question whether the data should even be used to benchmark providers and whether financial rewards should be aligned to hospitals results.

Rachel Davis is a senior research associate, Merrillee Briggs is a research nurse and Sonal Arora is a clinical lecturer of Imperial College London. Eshaa Sharma and Mahmoud Asgheddi are junior doctors of Russells Hall Hospital and Watford General Hospital respectively

Topics