Tap into the power and potential of big data

There is a lot of excitement about how “big data” is creating changes in the commercial and technological worlds.

‘A huge quantity of open data has been released and most of it is neither personally identifiable nor politically influenced’

The devices and systems that we use every day − such as computer systems, websites, mobile phones − produce increasingly large quantities of raw data. 

In the commercial world, those who can work out how to interpret and use this data intelligently are doing great things with it: they can use data to model variation and performance; predict change; tailor services to individual people; and to make better decisions for their organisations.

Sensitivity surrounding data

In healthcare, data driven thinking has huge potential to help make health services more efficient and effective. But with it comes huge ethical and practical challenges.

Health data is uniquely sensitive: witness the current controversy over care.data’s extraction of patient records, where the potential benefits of large scale data mining across care pathways are being debated against the risks of de-anonymisation and leakage.

Politically sensitive organisational data has its own weaknesses too; last week we heard that the National Audit Office found NHS waiting time figures “need to be treated with a degree of caution” and that the quality and outcomes framework appeared to be “gamed” by some doctors.

• Alex Kafetz: The information revolution hasn’t stopped care failings

Raw power

There is an alternative that allows us to access “data driven” benefits without getting into these problem areas: open data. This is raw data that’s freely available for anyone to download and use.

The Health and Social Care Information Centre has released a huge quantity of this open data and most of it is neither personally identifiable nor politically influenced. For example, the monthly GP prescribing data files taken from pharmacy returns run to hundreds of millions of rows, providing a rich source for asking and answering questions about patterns of prescribing, disease and other behaviours within the health service.

‘Raw prescribing and practice data is freely available as open data and we can slice it many ways to get under the bonnet of what’s happening’

Nesta has today issued the Which Doctors Take Up Promising Ideas? report examining how and why GPs take up new innovations and new drugs.

A few years ago this research might have been done purely with surveys, interviews and other qualitative research. Now, raw prescribing and practice data is freely available as open data, and we can slice it many different ways to really get under the bonnet of what’s happening.

Revealing results

Some findings were surprising. For example, it is sometimes assumed that smaller practices or older GPs are inherently more conservative, but we found the prescribing behaviour of neighbouring practices was much more important than any inherent characteristics of GPs that we measured.

‘There is extra power in using open data because it is a source of shared understanding and transparency’

Some findings were less surprising: bigger practices were the earliest adopters of IT tools; presumably because they have more admin support to deal with the implementation and stand to gain more from adoption.

Ultimately the power of this approach and of the availability of the raw data itself is that we can use it to figure out what is going on and how to run things better. For example, we can discover how to effectively encourage take-up of new practices in a way that minimises the time taken, money spent or the frustration caused by a more “blanket” approach.

Dealing with millions or billions of items of data is a specialised job in itself and we’re seeing other organisations trying to make that part easier. Swindon GP Gavin Jamie, for example, maintains a quality outcomes framework data website and the Connected Digital Economy Catapult − which I have worked on − is about to launch a hub that aims to show different ways of using and analysing open health data to encourage technologists as well as health practitioners to build the data applications that the health service needs.

Powerful potential

The potential for open (and closed) data to have an impact on healthcare in the UK is huge. Using data more effectively could help with prescribing, understanding patient needs, targeting services and generally empower health practitioners to focus on the things that are most important and effective for their patients.

There is extra power in using open data because it is a source of shared understanding and transparency; if you want to question, explore or improve on Nesta’s findings, for example, the data is right there to do it.

It is clear that there is a big opportunity for the health profession to learn from the best and highest impact uses of data in the commercial world and to use them for the best ends that we can: it’s going to be exciting to see what happens next.

Francine Bennett is chief executive and co-founder of big data specialist Mastodon C