Doctor level data needs to more transparency

It’s a sad day for the NHS’s transparency programme when the cardiothoracic surgeons want out.

The Guardian reports that the Society for Cardiothoracic Surgery – long held up nationally and internationally as bastions of transparency – has written to Simon Stevens, chief executive of NHS England, to ask for a rethink of the policy of publishing patient death rates. 

This is the organisation that under the stewardship of Sir Bruce Keogh, was lauded for its bravery as being first to publish its data some 10 years ago. 

The latest move is a continuation of the push back against very specific information being published about hospital doctors and GPs. 

The debate about hospital/trust level information seems to be won. 

‘Doctor level data is dividing the sector’

The Keogh review used mortality rates as a trigger to pick hospitals to visit and the fact that most of those selected were put in special measures indicates the reliability of this process. 

Despite this, hospitals dispute their usefulness: medical directors know the cheat codes, they blame case mix, cite one of the plethora of other measures that portray their organisation more favourably, or tell the scrutinisers to back off by committing to a casenote review. But doctor level data is dividing the sector.

Disputed data

Things have not been helped by problems of accuracy – the Care Quality Commission publishing the wrong GP data last year (admittedly at GP practice level but the difference is moot) opened the door for the Royal College of General Practitioners to call for the whole idea to be scrapped.

NHS England hasn’t helped by allowing the publication of surgeon level outcomes last year to much fanfare – albeit using a methodology that, in my opinion, hasn’t had enough external scrutiny and is not fit for purpose.

‘Either we’re not measuring the right thing, or we are measuring the right thing in the wrong way’

The MyNHS website shows comparisons of surgeon outcomes for more than 25 operations across 13 clinical specialties. But only three of these have any outliers (performance “higher than expected”) and in two of these there is one surgeon only. 

This has happened for one of two reasons. Either we’re not measuring the right thing, or we are measuring the right thing in the wrong way. 

In elective orthopaedic surgery, for example, death is particularly unlikely – so much so, that it’s probable every death would be thoroughly investigated by an NHS trust regardless of risk factor and without the need of statistical monitoring to trigger or alert this.

Pertinent measures

Much more pertinent measures would be:

• re-do rates;
• patient reported outcomes; or
• a more qualitative assessment of whether the surgeon had discussed the surgery with the patient in the context of the improvement they were seeking – for example, being able to walk without pain – and whether this had been achieved.

Where mortality is the “right” measure, more scrutiny of the methodology is needed. Surgeons under the auspices of the Healthcare Quality Improvement Partnership are designing the way mortality is measured themselves without much challenge from NHS England and beyond.

‘The sector would benefit from some quasi-independent examination of methodologies’

In the same way that the Office of Budgetary Responsibility provides independent and authoritative analysis of the UK’s public finances, the health sector would benefit from some quasi-independent examination of methodologies before information is published on NHS Choices, heralded by the health secretary and proudly publicised as a “major breakthrough in NHS transparency” by NHS England.

This could be fulfilled by a sub-committee of the National information Board.

Right of reply

Furthermore, I would give every consultant who has information about them published a public right of reply. This would be by adding functionally on NHS Choices to allow them to upload commentary next to their data. 

Of course, there is a good chance anyone with negative information will be quick to explain how their patients are special and different, or how they tend to take on more risky cases. 

However, this is an assertion that could easily be confirmed by analysts at the General Medical Council or the Care Quality Commission.

‘I would give every consultant who has information about them published a public right of reply’

If factual, the surgeon has every right for that narrative to appear next to their data and this might allay the fears raised in the SCS’s letter that publication promotes “risk averse behaviour, with some surgeons less willing to operate on patients with a perceived high risk of mortality.” 

If this statement is queried and it seems that the surgeon doesn’t properly understand their outcomes, well what better place to start a revalidation interview.

Alex Kafetz is a director at ZPB Associates and a lay member of the National Information Board