Measuring outcomes of communications teams

What success looks like for communications and engagement teams used to boil down to a board’s perception of the media coverage it generated. Fuelled by the traditional make-up of boards and the tangibility of press cuttings, it has been a tricky area to steer into new territory.

So it’s no surprise that the success measurements of communications and engagement teams have been disproportionately skewed by a need to prove how well the organisation was doing in the pages of its local newspapers.

According to Helen Stevens, associate director of communications at Barnsley Hospital, there are lots of reasons why this stopped working some time ago. First, she says, local and regional newspapers are not the influential powerhouses they once were.

“With their rapid decline in advertising and sales came a reduction in numbers of reporters,” she says. “The upshot was fewer specialist health reporters, slimmer newspapers and thereby dumbed down copy and less opportunity to get stories in. And with fewer people actually buying them, there’s also a big question mark over their influence.”

‘Until we can show the board, the taxpayer and our colleagues that our discipline can directly attribute its work to improving patient care, we will forever be the team that “deals with the media”’

Second, she’s clear that society cannot ignore the rise of social media and its impact on our lives. Even if you’re someone who isn’t using the internet to shop or to connect with other people, you will know someone who is.

“Online shopping last Christmas was 20 per cent higher than in 2011; 150,000 people sign up to Twitter daily and there are one billion active Facebook users, of which 50 per cent log in on any given day. Slowly but surely, businesses are realising that Twitter, LinkedIn and Facebook are savvy ways to connect to their customer base.”

A further factor is that media coverage does not tell her or her board anything about the intermediary effect of her team’s work. That is, the impact of what patients, staff and stakeholders are saying about them.

Helen adds: “There is far greater resource and effort going into patient, staff and stakeholder engagement – yet we don’t play back the reputational outcomes consistently. Instead they tend to surface as one-off reports (patient surveys, staff surveys) that aren’t connected.”

Also, the correlation between more robust reporting mechanisms across most parts of service delivery led to better outcomes for patients.

“It struck me that if we wanted to prove our worth as a support service that impacted on patient care, we needed to do the same. We needed not just better collection of data but to change our thinking from outputs to outcomes too.”

Out of date

At the same time, the professional bodies behind the discipline were getting to grips with how to measure communications and engagement.

In 2010, these bodies, which include the Institute for Public Relations and International Association for the Measurement and Evaluation of Communications, created seven principles to good measurement. All are now recognised by the profession as standard. These include: goal setting; measuring outcomes rather than outputs; the effect on business results; media measurement needing quantity and quality; measuring social media; the “advertising value equivalent measurement” being out of date and the importance of transparency and replicability.

Indeed, it isn’t that long ago that advertising value equivalent, or AVE, was the industry standard: a simple measurement, which took the volume of column inches proactive of work and translated it into its AVE. It only ever really worked with positive coverage and even then was pretty blunt. According to Helen, having just this is a bit like only having the four-hour emergency department measurement – it just doesn’t come close to giving the whole picture. And it is woefully out of date.

And so it was with these seven principles in mind that the communications team at Barnsley Hospital sat down to develop a set of measurements that would demonstrate to the public and the board the value of their work. The result was an online communications dashboard that splits activity, outputs and outcomes into four areas of work: print and broadcast media, social media, internal communications and external communications. The team makes it available online for two reasons – because they believe in transparency and because they want to inspire other communications teams to do the same.

Where they are still developing, says Helen, is properly measuring outcomes. She found that where measurements already existed, it was straightforward to connect the impact of their work but setting up new routine outcome measurements was difficult.

She says: “Take internal communications, where we already collect monthly data on turnover and performance against both Monitor and Commissioning for Quality and Innovation targets. We could legitimately connect the work of engaging with staff on day to day and strategic organisational issues with both performance and staff turnover.

“Though even then, it’s not straightforward to assume a causal link. We are getting better at taking baseline data so we can factor in isolation but it’s not always a simple process.”

Return on investment

The team also benefits from free analytical tools that enable measurement of impact across social media. Where they struggled was with the outcomes of their intermediary work.

“We don’t have market share or sales figures to go by, which is what the private sector would use,” says Helen. “That said, the dashboard has been a great starting point. It helps us measure our ongoing work and campaigns, it helps us plan and prioritise and it gives our work more credibility in the boardroom. Most importantly, it helps us to work out the return on investment on our work, so we can discover if the campaign was worth doing in the first place.”

The Barnsley team created the online dashboard knowing it wasn’t a perfect measurement system but their aim was to start somewhere and continually improve. They are hopeful that the new NHS planning guidance, Everyone Counts, which has the marketing principle of meeting customer needs at its core, will be part of its next phase.

Helen says: “In particular, the offer of listening to patients and increasing their participation will translate into outcomes that will help us to demonstrate the impact of intermediary work – whether that is through the tangible experiences of our patients and their families or the conversations taking place across social media.”

The team is already looking at how the “friends and family test” can be included in the dashboard, as well as local commissioner metrics for evaluating the social and economic return on investment of patient and public involvement activities.

There’s evidence too that communications directors across the country are starting to develop more sophisticated outcome-based measurements. A scan of NHS board papers shows a smattering of organisations have started to discuss it, but it isn’t at scale and there’s no consistency.

Helen’s ambition is for all NHS organisations to have the same outcomes measurements, transparently available.

“Until we can show the board, the taxpayer and our colleagues that our discipline can directly attribute its work to improving patient care, we will forever be the team that ‘deals with the media’. Communications is about brand. It’s about managing reputation, issues and crises. It’s about staff engagement, social and community engagement and advocacy. Measuring all this is difficult but critical. We need to overcome the difficulties and work together on a set of NHS specific measurements.”