The release of the surgical outcomes data last Friday was like a firework. There was a loud bang and a brief flash when people looked and then it disappeared into the darkness. In fact the story was more about the data that wasn't released than the data that was. Surgeons who defied the call for their data to be published were looked on with suspicion and outed at the behest of Jeremy Hunt, the Health Secretary.
Once upon a time the only thing people talked about when it came to the assessment of quality in angioplasty was the number of procedures performed with high volume operators and centres regarded as the best. Part of this was because procedural volume was easy to measure whilst other quality markers were not. Today sees the publication of the coronary angioplasty data which has been collated by the British Cardiovascular Intervention Society (BCIS). The society has been collecting data on angioplasty since 1992 and presenting it as an annual audit on their website. In 2012 it became possible to link an individual angioplasty procedure to the GMC number of the consultant responsible and hence to produce the operator level data which has been published today.
BCIS has tried to not turn this a beauty contest between interventional cardiologists and they have made it difficult, but not impossible, to compare the data. They rightly state that it is not possible to compare different cardiologist and this is true since the case mix of one will be different from another. There are some cardiologists who only perform only elective angioplasty on stable angina patients or who work in hospitals that don't perform primary angioplasty for acute heart attacks. These cardiologists should have very low death rates.
BCIS have used a measure of self reported in-hospital death rate, stroke rate and requirement for emergency bypass surgery. From this they calculate a MACCE (major adverse cardiac and cerebrovascular event) rate for each interventional cardiologist. Then they calculate an expected MACCE rate and its 95% confidence interval based on patient factors including age, sex, history of stroke, urgency of procedure using a model developed by the North West Quality Improvement study (NWQIP). In a nutshell provided the cardiologist's MACCE rate is less than the 95% confidence interval then they are not regarded as an "outlier" and their practice is regarded as safe.
So what determines MACCE? It is dependent on patient, interventional cardiologist and post-procedure factors. In most cases death, stroke and need for emergency bypass surgery are more dependent on the type of presentation, co-morbidity of the patient and severity of the coronary artery disease. The skill of the cardiologist might be better judged in terms of procedural complications such as vascular access site bleeding, the need for urgent repeat coronary intervention, the number of acute stent thrombosis or stent restenosis. These measures whilst more difficult to collect would provide a better estimate of quality.
Measuring survival to hospital discharge as an outcome also leads to anomalies. Take this scenario, Cardiologist A treats an elective, stable patient with a stent and the patient is discharged only to be readmitted 48h later with an acute stent thrombosis causing an acute heart attack. This rare complication can sometimes be caused because of stent under-deployment. Cardiologist B is on call and treats the patient in the catheter lab to salvage the acutely blocked artery but sadly the patient dies. In the current analysis this death is recorded as belonging to Cardiologist B not Cardiologist A. A move to the measurement of 30 day mortality is needed but there will be difficulties in apportioning deaths when two cardiologists have treated the same patients.
Another important skill, which is very difficult to measure, is that of case selection. Some cardiologists will attempt emergency angioplasty on almost all patients presenting with an acute myocardial infarction even if the chance of survival is very low. With the publication of this data it is likely that cardiologists will start to think twice before taking such patients to the catheter lab where the procedure may be futile, the outcome inevitable and the death counted on their angioplasty statistics. Interventional cardiologists have often been put in a position where an emergency patient has been turned down by a cardiothoracic surgeon because the risk of operation is deemed to high risk and they have then gone on to offer angioplasty. This behaviour may well change as interventional cardiologists are subject to increasing scrutiny.
The other issue is training the next generation of interventional cardiologists. If the consultant is responsible for the procedure and the data is going to be published in a public forum how comfortable will they be allowing a fellow or registrar to be the first operator in a difficult case. If a mistake is made by the trainee cardiologist during the procedure this could affect the outcome. On the other hand if we don't train the interventional cardiologists of the future what hope will there be for us when we are old and need intervention ourselves.
So what of the data released today. First we have to accept that some of it is missing as not all hospitals contributed so any conclusion drawn here are based on the data that is available. In 2012, 559 consultant cardiologists performed 83,121 angioplasty procedures. There were 22,426 primary angioplasties performed to treat acute ST elevation myocardial infarctions, 30,399 to treat acute coronary syndromes and 27,860 to treat stable angina. the rest were coded as other or unknown. The majority of angioplasty in the UK is used to treat emergency patients with only 34% of procedures for stable angina. This is very different from the data from other reports such as the New York State where 83% of PCI procedures were done for stable angina.
Some hospitals perform a very large number of procedures. For example Liverpool Heart and Chest performed 2929 with the Freeman in Newcastle coming a close second with 2832 procedures. At the other end of the spectrum are the private hospitals which tend to perform mainly elective PCI usually, though not exclusively, by cardiologists who also work in NHS hospitals as well. Hospitals are supposed to perform 250 cases per year to remain viable, 17 hospitals reported completing less procedures than this.
What can we learn from the individual data? BCIS have been careful to present the data in a way in which makes the comparison of different cardiologists very difficult. Each operator's data is presented separately on an individual webpage and so to make any direct comparison laborious. I suspect this was a deliberate measure but it does not make it impossible to abstract the data and to look at differences.
Most surprisingly they also do not separate the outcome data into elective and non-elective cases. It is obvious that the MACCE rate for elective angioplasty will be low. Currently patients are quoted a 1% chance of heart attack, stroke, death or need for emergency bypass surgery when they sign the consent form. The actual MACCE rate for this procedure is probably around 0.2%. Compare this to the MACCE rate of more than 50% for a patient presenting with an acute myocardial infarction, cardiogenic shock and following an out of hospital cardiac arrest.
No cardiologist had MACCE rate greater 95% of predicted so it appears that everyone is operating within the expect levels. I suspect this reflects the crude nature of the MACCE measurement and the NWQIP model which has been used. Looking at the high volume operators the predicted MACCE rates is running about twice as high as the actual MACCE rate suggesting that it does not predict particularly accurately a contemporary patient population. Statistically out of 559 operators one might expect at least 25 to lie above the 95% confidence interval simply by chance alone - none do, again questioning the accuracy of the risk prediction model and any conclusions regarding safety are dubious.
BCIS recommend that interventional cardiologists should perform at least 75 PCI procedures per year in order to maintain competence. Nearly 20% of operators are apparently performing less than this with many of the interventional cardiologists working solely in the private sector doing very small numbers of cases each year. The median number of procedures per cardiologist is 138 although there are some cardiologists performing significantly larger numbers of procedures.
Some cardiologists MACCE rates fall very close to the 95% confidence limit and some very far away. For example the largest volume operator in the UK who did 591 cases has a MACCE rate of only 0.17% (95% CI: 3.12%) although 62% of his patients were elective and only 5% primary PCI cases. At the other end of the procedural volume spectrum there is great difficulty in analysing data because of the small numbers. When a cardiologist has done only 25 procedures the confidence interval is so wide as to preclude any meaningful analysis.
The publication of the angioplasty data by BCIS is a step in the right direction. Hopefully it will lead to an improvement in the quality of data collection and accuracy. What is sorely needed is the development of more accurate risk and quality control models as well as moving to measurement of 30 day mortality. We are at the beginning of a long and interesting journey and at the end we will see that quality in angioplasty is more than procedural volume.
Grayson AD, Moore RK, Jackson M, et al. Multivariate prediction of major adverse cardiac events after 9914 percutaneous coronary interventions in the north west of England. Heart 2006 May;92(5):658-63.
BCIS Individual Operator Level Data Website
PCI in New York State Report 2008-2010
Dr Richard Bogle
The opinions expressed in this blog are strictly those of the author and should not be construed as the opinion or policy of my employers nor recommendations for your care or anyone else's. Always seek professional guidance instead.