The friends and family test can make the grade

After 14 months of fanfare, controversy, preparation and speculation, the national friends and family test went live. Every inpatient ward and accident and emergency department in England now has a friends and family test score: a number from -100 to 100, calculated as the proportion of patients “extremely likely” to recommend it less the proportion who are “unlikely” or “neither likely nor unlikely” to recommend.

‘It’s the verbatim qualitative information, not the single number score, which will be of most use in unpicking problems’

Much of the media coverage of the results focused on the 36 hospital wards that had “failed” the inpatient test for June: these wards had a score of less than zero. But between them, they had a total of 360 responses and a response rate of 15.9 per cent. Twenty five of them had fewer than 10 responses each; eight had only a single response.

Now that the sound and the fury of the initial coverage has passed, what’s to be done with the data? In some ways, the friends and family test is a different tool to different trades; its strengths, weaknesses and utility differ at local and national levels. Locally, it’s a quality improvement intervention − the systematic attempt to collect patient feedback and embed it into the culture of services. Nationally, it’s a data collection, albeit a controversial one.

The right response

As NHS England and Picker Institute Europe’s preliminary analyses have noted, there are likely to be “mode effects” when comparing organisations that used different methodologies for the test. This, along with variation in response rates and the representativeness of participants, threatens comparability between organisations. If trusts have used a consistent approach locally, though, then their boards and management teams should be able to track ward, site or trust level trends.

Most importantly, NHS staff and teams should look to use the results for quality improvement. It’s the verbatim qualitative information, not the single number score, which will be of most use in unpicking the problems that need addressing. Indeed, recent research by Rachel Reeves et al has shown that use of patients’ written comments from ward specific surveys can help to improve patient experiences if used as part of facilitated, improvement focused ward meetings. 

‘It benefits no one to report results from groups so small that they could have shared a taxi home: the data is meaningless’

At a national level, the sheer achievement of taking the friends and family test from announcement to reality in just over a year shouldn’t be underestimated. But making this grand a gesture at such pace inevitably means a degree of compromise and a rollout with room for improvement. 

With this in mind, it’s laudable that NHS England has both committed to evaluating the way the test is conducted and has walked the walk on transparency by releasing full experimental data at ward, site and trust level. This allows researchers across the country to interrogate the data and examine its strengths and weaknesses. 

We have already started looking at how the results relate to other patient experience data; others have compared the data with mortality indices and explored results for trusts featured in Sir Bruce Keogh’s report on excess mortality. Even at first glance, there are some key areas where the test could be significantly improved as a data collection.

Improve the data

First, set a minimum number of respondents for reporting of data. It benefits no one to report results from groups so small that they could have shared a taxi home: the data is effectively meaningless, the judgment spurious and patient confidentiality is seriously compromised.

It’s easy to remedy this: supress all data based on 10 or fewer responses, and report rolling quarterly results for wards too small to get 10 responses a month.

This would prevent unhelpful and counterproductive “naming and shaming” based on derisory numbers of responses. This is important.  It’s hard to imagine staff morale not being undermined by being labelled as part of a “failing” ward, and research shows the experiences of staff and patients are related. Allowing wards to become scapegoats does not seem likely to lead to improvement.

‘The frontline of the NHS should be supported to act on patient feedback and encouraged to challenge poor practice without fear’

Second, standardise the approach to data collection. If this isn’t practical − for example because trusts have already committed themselves to long term contracts with providers − then improve the information collected about how surveys are administered. 

At the moment, individual level data on mode of response is not available and only aggregate data is collected nationally. This makes it impossible to fully unpick possible biases and seriously limits the comparability of data. Similarly, the current methodology provides no data on the demographics of respondents: this should be changed to enable both analysis of differences between groups and case-mix adjustment. 

Ultimately, the friends and family test has potential as a catalyst for change, but it must be developed and its inherent risks and tensions managed. Nationally, the centre should work to provide the best tools and intelligence. Locally, the frontline of the NHS should be supported to act on patient feedback and encouraged to challenge poor practice without fear. 

As Don Berwick’s review of patient safety neatly summaries: “Quantitative targets… should never displace the primary goal of better care.” For the friends and family test to be a success, this must never be forgotten. 

Chris Graham is director of research and policy at Picker Institute Europe