The accurate appraisal of operator and programmatic quality of percutaneous coronary interventions (PCI) is a fundamental principle of interventional practice. Identifying areas of potential improvement is vital to delivering optimal outcomes. Moreover, patients have the right to know the competency of the operators and programmes available to them in order to make better-informed consumer choices.1-3 Therefore, every interventionalist should be familiar with how PCI quality is measured and how the metrics are analysed.
Public reporting programmes routinely emphasise unadjusted 30-day survival as the standard metric of quality.3,4 This outcome has been chosen because it is easy to obtain and is a simple measure for laypeople to grasp. Culpability is allocated if a procedure was performed within 30 days prior to a death, regardless of its contribution to the result and despite limited control over the variables predictive of outcomes. For example, the operator and the programme can not alter the size of an acute infarct, the presence of shock, the delay in arrival to the emergency department after the myocardial infarction onset, the severity and extent of coronary stenoses, or the pre-morbid conditions of the patients; factors that critically determine procedural-related mortality.
Data from registries and clinical trials demonstrate that the occurrence of PCI complications is much more dependent on patient-specific factors than on procedural error. Furthermore, the selection of very low-risk patients, together with subconscious and/or intentional risk aversion, makes performance measures such as survival subject to gaming. In clinical circumstances where very high-risk procedures are indicated, performing such cases in large volumes will often lead to false censure of excellent operators, who are willing take on the toughest cases; however, most deaths within 30 days relate to the underlying illness and not the conduct of the PCI procedure. The event rate is inherently higher in this group, and no matter how competent the team, deaths are inevitable. Consequently, 30-day mortality does not accurately reflect the cognitive or technical skills of the interventionist.
To partially correct for the confounding effects of elevated intrinsic risk of emergency and high-risk PCI, risk adjustment algorithms have been developed that calculate an expected mortality based on a weighted formula of comorbidities associated with worse outcomes.4,5 The 30-day risk-adjusted mortality rate (RAMR) is essentially the ratio of observed mortality to expected mortality (O:E ratio) multiplied by the average 30-day mortality rate (approximately 1.3%). However, RAMR is an insufficient metric in isolation as the adjustment algorithms fail to completely compensate for the higher risk; deaths occurring even in patients with features that make survival unlikely leave a small O:E fraction, because observed mortality (the numerator) can never be zero once there is a death. This inaccuracy accumulates with each additional death, and the more high-risk cases that are performed, even if accomplished with better-than-expected mortality results, the more inaccurate the O:E ratio becomes as a quality indicator.6
Recognition of this problem has led many to advocate another approach: to exclude cases from analysis that are anticipated to have a high intrinsic risk, such as acute myocardial infarction. There are statistical arguments against this proposed method, including that risk is a continuous variable, and there is no clear cut boundary to objectively ‘draw the line’; any reporting that involves exclusion of deaths gives the appearance of a lack of transparency; and if deaths from high-risk cases are not counted, the mortality rate of PCI with contemporary pharmacotherapy and techniques is very low, regardless of an interventionalist’s technical and cognitive abilities. Hence, finding meaningful differences to identify opportunities for improvement will be mathematically impossible. Accordingly, a single death in these cases might expose a systematic quality issue, but may also reflect high-risk not completely accounted in the algorithm. When usual risk patients die, it is typically indirectly related to bleeding, heart failure, arrhythmias, or renal failure. The occurrence of these procedural complications is most often related to pre-procedural clinical status rather than operator competence.
The absence of resolution to this complex statistical and clinical dispute has resulted in the public, who are the presumed audience for this information, not knowing how to correctly interpret the data. Further, the apparent simplicity of the presented numbers conceals the fallacy that unversed individuals (including medical staff, quality review committees, hospital administrators, and third-party payers, who are highly interested in this information) may think that every death ought to be preventable; that is, that the ideal mortality rate is zero. But this credulous view leads to risk avoidance, both conscious and unintentional, not better quality.7 Thus, some very high-quality physicians and laboratories that have a high-risk case mix are incorrectly identified as being of subpar quality because their O:E ratio can never be zero. Further, a consumer is unable to distinguish quality among operators with non-zero O:E ratios, without knowing the risk of the patients treated: observed mortality is routinely shared, but expected mortality is often not.
Recently, the New York State Registry decided to try a hybrid approach.8 They will be reporting their individual RAMR but excluding acute myocardial infarction cases; however, they will report the overall programme RAMR, including all deaths to the hospital administration. The idea is to reflect individual physician routine outcomes, which are usually reported to the public, and assuage the sensitivities of the operator, while not discarding any data reflective of the programme. Most believe this is a step in the right direction, and give this registry substantial credit for exploring a new methodology, which other national and regional registries have eschewed for years. However, it is possible that this compromise in reporting could create misunderstandings between the hospital administration and catheterisation laboratory leadership, who will be working with different numbers. Moreover, this is not an authentic solution to the various methodological concerns, and may even create a new problem. When confronted with the new reports, insurance carriers or hospital administrations might insist on more conservative case selection to improve the reported outcomes, which is not what is best for patients, who despite being the highest risk patients, are the ones most likely to benefit from such procedures.
To develop a better approach, interventional cardiologists must strongly advocate ending quality assessment based primarily on procedural mortality, and instead encourage registries and in-hospital quality programmes to incorporate metrics that genuinely reflect the excellence of care delivered. To accurately evaluate programme quality, they must identify, collect, and quantify measurable and potentially modifiable metrics that are actionable and truly representative of quality of service. An effective ongoing evaluation process relevant to contemporary practice, with appropriate benchmarks for comparison, must be adopted.
Recently, a comprehensive framework that more accurately appraises PCI quality was proposed.9,10 Four broad aspects of practice form the basis of appraisal: case selection, technical expertise, case complexity, and clinical results. Measurable parameters in all these quality categories were identified. Quality of life, persistence of angina, re-hospitalisation, repeat revascularisation, and follow-up myocardial infarction are all critical endpoints that are currently go unreported as primary quality indicators. These outcomes should be essential components in a revised quality framework.
Additionally, reduction of specific non-fatal complications (e.g., haematomas, bleeding, new dialysis, stroke, periprocedural myocardial infarction, and stent thrombosis) should be part of the evaluation process. Case selection based on the correlation of coronary stenoses with regional ischaemia, function, and viability should be included in the assessment. The use of physiologic testing and intracoronary imaging are necessary to fully optimise strategy and results. It is extremely important that, in scenarios where the relative merits of PCI versus medical therapy or bypass surgery are debatable, substantial latitude for patient preference must be integrated. Current guidelines that emphasise survival as the sole important outcome metric should be modified; improved mortality might be an anticipated benefit in some clinical situations, but not all. Random case review and comparison of outcomes to a disease-specific registry are also strongly suggested.
The central measure of a high-performance healthcare system is the delivery of high-value care. Value is a parameter that combines the attributes of high effectiveness and low cost. Comparing the outcomes of those treated in one programme versus similar patients treated in comparable institutions provides benchmarks of efficacy, and provides insight into risk aversity. Cost is a dominant covariate of length of hospital stay and secondary services, and hence a powerful and objective correlate of complications. If the risks, and especially the benefits compared to medical therapy seem to be equivalent, the less invasive option will be preferred. For this reason, to establish the value of our work, its effectiveness and cost benefits must be fully depicted.
Once these proper measures of excellence are accepted, the profession should embrace a more complete and balanced account of the merits of PCI, which better demonstrate the skills and proficiency of its practitioners. The time has come for interventional cardiologists to insist that such programmes be implemented, as they are essential to a full comprehension of the role of PCI in the modern treatment armamentarium. Allowing our results to be judged by inaccurate measures because they are simple and cheap to collect will continue to be detrimental to progress.
