The friends and family test is a foe to the NHS

In claiming that the “friends and family” test will “shine a light on standards of care” the health secretary may hope to draw parallels with the lady with the lamp. However, the test shows none of Florence Nightingale’s compassion, dedication to duty or commitment to statistical rigour. 

‘Any failings in NHS care will not be uncovered by these unreliable data. On the contrary, the test itself is at fault’

When the first quarter’s results of the test were published at the end of last month, news reports, including those by the BBC, named 36 “failed” wards. The casual brutality with which these wards have been selected as examples of all that is wrong with the NHS is a disgrace. 

The friends and family test provides no reliable evidence of poor care, but these reports will have had a devastating effect on the ward managers and nurses singled out for vilification. They may also have caused unnecessary distress to patients treated on those wards.

Reckless approach

This reckless approach to measuring and reporting patients’ experience undermines years of research dedicated to the responsible conducting and reporting of survey data. Any failings in NHS care will not be uncovered by these unreliable data. On the contrary, the test itself is at fault, as the following points illustrate.

For eight of those 36 wards, the score was based on just one response and 19 wards had fewer than five responses. Publishing these results breaks the Department of Health’s own publication guidance: “To avoid a risk of compromising confidentiality of individual responses, where there are less than five responses for a given ward in a single month the results should not be published.”

The scoring method is somewhat bizarre as only “extremely likely [to recommend this ward]” responses are scored positively. Patients may well respond differently if they knew that “likely” condemns the ward with a negative score.

There were 565 wards with no responses but these have escaped the spotlight so, for many wards, one patient’s response was the difference between vilification and obscurity. Ward staff are likely to conclude that no patient feedback is the safest option.

The test does not conform to basic scientific standards. Asking patients whether they would recommend a ward makes little sense to patients and Picker Institute research confirms it confuses them. Hospital care is not like choosing a hotel or car hire firm.

Littered with flaws

Inexplicably, the “friends and family” test’s 25 page implementation guidance is peppered with lofty declarations on the importance of scientific rigour. It even notes that “the use of an alternative method could result in a variance between local and national results and this is likely to cause confusion or lead to issues of confidence in the results”. That’s a good point, but its authors seem to have ignored it.

‘Nurses were often reluctant to accept responsibility for their patients’ feedback, some were hurt, some were defensive’

Differences in the way NHS trusts implement the test mean that comparisons are unreliable; there are no independent checks on how it is administered or how data is collated.

Furthermore, the incentives and opportunities for gaming are numerous: low response rates increase the risk of bias and the test is usually administered to patients by the NHS staff who have cared for them, wasting nurses’ time and raising patients’ concerns about giving honest feedback.

Perhaps the implementation of the test was inspired by perceived shortcomings in the national inpatient surveys led by the Care Quality Commission.  There are concerns the annual survey results take too long to get back to staff and are not ward specific, so staff claim, “That doesn’t happen on my ward.”

A recent study by the Greenwich University sought to address these criticisms while retaining the scientific rigour of national surveys. We conducted ward specific inpatient surveys and returned the results to ward nursing staff as soon as they were available.

Meaningless data

We also compared two ways of giving nurses their survey results: one group received written results only, while others had regular ward meetings to discuss them. The ward meetings were hard work; nurses were often reluctant to accept responsibility for their patients’ feedback, some were hurt, some were defensive, and the meetings themselves were often uncomfortable. But they were effective.

‘Both the friends and family test and the national surveys fail to make proper use of the data they generate’

We found written feedback alone had no effect, even with ward specific results, but supplementing that feedback with ward meetings had a highly significant positive effect on patients’ experiences. 

Simply informing staff about their patients’ feedback is not enough. NHS staff need time, training and management support to listen actively to patient feedback and respond to it in a productive way. 

To date, both the friends and family test and the national surveys fail to make proper use of the data they generate. If the national patient surveys are perceived to be too long, too slow or not specific to wards they could be adapted without sacrificing their scientific rigour.

Resources will be used more effectively if they are focused on the “active listening” part of patient feedback, not on collecting meaningless data.

Rachel Reeves is principal research fellow in the school of health and social care at Greenwich University