Developing an index of conscientiousness

Medical professionalism is a valued attribute, but our inability to measure it accurately or predict its appearance in doctors and medical students makes it hard to foster. One of the problems is that professionalism, rather like happiness, is an elusive and multi-faceted concept that means different things to different people.

Another problem is that methods of assessing it, such as staff expert judgements and peer assessment, have tended to be either highly subjective or overly dependent on a limited number of student observations, or both.

John McLachlan, professor of medical education at Durham University, says that “people often define professionalism in complex ways and then try to measure what they have just defined. This not only ignores the fact that professionalism is a cultural construct that varies from time to time and place to place but also leads to the use of unpopular, expensive, low-validity assessment methods”.

Inspired by earlier studies, particularly research by Professor Maxine Papadakis and colleagues on the links between problematic behaviour and subsequent disciplinary action by a medical board, Professor McLachlan and his team at Durham University Medical School developed a tool called the Conscientiousness Index (CI). 

The index measures the extent to which a student diligently performs a selection of administrative tasks associated with their medical course or otherwise behaves diligently when given the opportunity to do so. The team hypothesised that conscientiousness, whilst not the sole trait or construct involved in professionalism, was nevertheless an important one.

Clinical Conscientiousness Index

When tested with first and second year undergraduate medical students at Durham University in 2006-2007, the results suggested a good correlation between conscientiousness as they had defined it and staff views of individual students’ professionalism. There also seemed to be a relationship between conscientiousness and the frequency of critical incident reports, which record lapses in professional judgement, although this relationship was not quantifiable at the time.

In May 2009 Professor McLachlan and others concluded that the CI provided a scalar, objective proxy measure of the construct of professionalism as perceived by staff members in an undergraduate medical school. Subsequent work by Dr Gabrielle Finn, Dr Marina Sawdon and others from the Durham team demonstrated a positive correlation between CI scores and peer estimations of professionalism.

‘For a small minority of students and clinicians, this could make for uncomfortable reading, but test results so far have shown that most are likely to score extremely highly on conscientiousness’

Research published in the September 2012 edition of Academic Medicine now suggests that the CI can be adapted for use in the clinical phase of an undergraduate medical course, in a different medical school and country. Working with the authors of the original CI studies, a team led by Dr Martina Kelly from the School of Medicine, University College Cork, replicated the work on the CI with third year undergraduate medical students during the academic year 2009-2010. They called the adapted tool the Clinical Conscientiousness Index (CCI). 

The team found a significant correlation between CCI scores and staff views on students’ professionalism, and also between CCI scores and student performance in objective structured clinical examinations (OSCEs). No correlation was found between CCI scores and students’ portfolio performance, but portfolios comprise a number of activities that do not necessarily relate to professionalism.

The results of these studies suggest that the CI is a relatively objective way of assessing professionalism that correlates with individuals’ subjective views about professional behaviour and with a number of other professionalism assessment methods. That bodes well for the production of robust and defensible results. The reliance of the CI on data already routinely collected at an administrative level also makes it relatively cheap and simple to use.

Uncomfortable reading

As one might expect, the CI is being tested for use and adopted in an increasing number of areas. Dr Sawdon says that “while it was originally used for research purposes, the CI now forms part of the summative assessment process in at least one medical school, and could therefore determine the progression of some students from the current academic year onwards”.  

For Professor McLachlan, “the next phase in the development of the CI could include testing it for use as a measure in clinical practice. It would then be a fairly small step to integrating it into the revalidation process”. He also predicts that, if these developments continue, “employers in all health fields will be taking a much greater interest in the degree of conscientiousness displayed by applicants for healthcare posts”.

For a small minority of students and clinicians, this could make for uncomfortable reading, but test results so far have shown that most are likely to score extremely highly on conscientiousness. More importantly, use of the CI is likely to raise levels of professionalism and therefore also patient safety and trust in the medical profession. After all, Professor McLachlan asks, “If you had to choose between a doctor who was conscientious, and one who was not, which one would you prefer”?

Boaz Nathanson