The hospital standardised mortality ratio is no indicator of a provider’s quality of care, argues Kailash Chand

Confusing data

In February health secretary Jeremy Hunt announced an annual review of roughly 2,000 patient case notes to establish “a national rate of avoidable deaths every year”.

However, Nick Black, professor of health services research at the London School of Hygiene and Tropical Medicine, on whose research this announcement is based says an estimated number of avoidable deaths generated from such a process “wouldn’t have any meaning”.

In the same month the Academy of Medical Royal Colleges reported that there are significant limitations on relying on hospital standardised mortality ratio (HSMR) as a marker of quality of care in hospital.

Special treatment

HSMR was one of the key pieces of data used by Sir Bruce Keogh to decide which hospitals should be reviewed and put under special measures in 2013.

Is this the right approach? Just because a hospital has a higher than normal mortality rate does not necessarily mean the quality of care it provides is poor.

‘There could be factors resulting in changes in the dying pattern of people’

There is a wealth of indicators used to measure safety, including patient incident reports, falls and methicillin resistant staphylococcus aureus infections, so a different estimate of avoidable deaths can be produced depending on which one was used to adjust the national rate.

So there are a range of reasons why HSMR is not a reliable methodology, among them:

  • HSMR is a statistical score based on complex calculations. A score of 100 simply means that the expected amount of patients died in the period covered as the statistical model predicts.
  • The statistical processes behind the rate have been challenged ever since the process was introduced. Experts in the field have been arguing about the value of HSMR for more than five years and the opponents’ arguments have not been fully dismissed or resolved.
  • Poor data entry in a hospital that delivers excellent care can result in that hospital having a high HSMR figure, even though no patient has been harmed or died unexpectedly.
  • Factors such as the availability of hospice and social care in a local area can also cause a hospital to have a higher than expected HSMR score. If a hospital is more easily able to discharge terminally ill patients, it might have a better HSMR score – even though the care it gives is no better than that of a hospital that finds it harder to move patients to a more appropriate place to die.
  • Equally, if a patient chooses to die in hospital, rather than at home, this might affect the HSMR score. Arguments about how well HSMR takes factors such as these into account play a significant role in the challenges to its validity.
  • HSMR is regularly “rebased”, so the value of a score of 100 changes too. If the majority of hospitals improve their care – and as a result fewer people with a given diagnosis die than previously – this causes the baseline against which hospitals are compared to change. In simple terms, if hospital A and hospital B both have a score of 98 one year and both improve their care, but B improves more than A, the following year hospital A could have a score of 105, while hospital B keeps a score of 98. In this scenario it might look as though hospital A had got worse, but in reality it has got better but at a slower pace.
  • The inventor of the HSMR is clear that it is not a measure of anything – it is just an indicator that should be looked at alongside other data to get an idea of how well a hospital is performing.

Tameside Hospital Foundation Trust, in my patch, is the only hospital in special measures whose HSMR score has worsened.

In Tameside we know the hospital is in the process of delivering significant improvement plans that are subject to external scrutiny. The effects of these plans may not show in the HSMR figures.

I have not seen the latest HSMR data for the FT. It is based on data over a period of time, so recent changes to hospital practices may not be reflected in the score.

Deaths in question

Every time a patient dies unexpectedly, it is important that his or her death – and the care received before death – is properly investigated and lessons are learnt.

The coroner has a key role in investigating unexpected and suspicious deaths, as well as instances where the cause of death is not clear.

The coroner has legal powers to require health and care services to make changes if he or she feels that the way that care was given resulted in a premature death. We are aware, through our contact with bereaved families, that this process is used in our area.

‘It is important that every time a patient dies unexpectedly, his or her death is properly investigated’

Tameside also undertakes regular reviews of patient deaths and a local GP is involved in this process to provide external assurance.

There could be factors resulting in changes in the dying pattern of people and we are not clear whether these are reflected in the HSMR calculations.

For example, as people live longer, more are admitted to hospital with a wider range of long term health conditions, not all of which may be known at the time they are admitted.

This means that what initially looks like a simple illness could be significantly more serious and their chances of survival much lower than initially thought.

Furthermore, some public health experts suggest that reductions in local authority budgets over the past five years have resulted in people having their social care withdrawn leaving them to wait longer before they become eligible for support. They believe there is a correlation between this and a drop in overall life expectancy in some areas.

‘The coroner has legal powers to require services to make changes if they feel the care resulted in premature death’

In Tameside we have areas where people will have at least one long term illness that limits their quality of life well before they reach retirement age.

We would welcome media activity that helps people to understand the impact of their lifestyles on their long term health sufficiently early that they can change their lifestyles and enjoy more years of healthy life.

As an “unreliable” and flawed way of comparing the quality of hospital care in England, HMSR is not necessarily a definitive indicator of failings in the quality of care.

So perhaps hospitals should not be maligned with this unreliable data.

Kailash Chand is deputy chair of the British Medical Association

Topics