Observational studies are being used more frequently to compare drug effectiveness. Authors typically report the proportion of patients reaching a defined clinical threshold or response rates (e.g., American College of Rheumatology [ACR] response or clinical disease activity index [CDAI] low disease activity rates) after a set time. Comparing response rates in that setting is hampered by two major threats. Firstly, patient, disease, and treatment characteristics often differ for each treatment group. Secondly, assessing drug maintenance after a certain period of time excludes the analysis from all of the patients who discontinued their treatment for ineffectiveness or intolerance, thus resulting in an attrition bias, which may overestimate drug effectiveness. While several methods have been proposed, none account for both confounding and attrition.
The aim of this study was to propose two new methods, propensity-score matched LUNDEX (PSM-LUNDEX) and CARRAC, to adequately compare response rates in patients with different baseline characteristics, while accounting for attrition, and compare them to established methods (complete case [CC] analysis and LUNDEX).
The different methods are illustrated using CDAI low disease activity (≤10) rates in data from a collaboration of registries, using 3,448 patients treated by a biologic, either intravenously (n=2,414) or subcutaneously (n=1,034).1
The first method is CC, where the response rate is computed as the percentage of responders in total number of patients still on the treatment at the given time point.
The second method is the LUNDEX,2 in which CC is corrected for attrition by multiplying it by the Kaplan-Meier estimates of the survival, thus considering all patients not on treatment as non-responders.
The third method is PSM-LUNDEX, in which the patients are firstly selected in both exposure groups using PSM, and then the LUNDEX is used to compute the response rate.
The fourth method is called confounder-adjusted response rate with attrition correction (CARRAC) by reason for drug discontinuation. This method firstly computes estimates of drug survival for the main reasons of drug discontinuation, such as ineffectiveness, adverse events, and remission. Then, it estimates the response rate using random effect individual patient data meta-analysis with estimates for each reason of drug discontinuation, combined using weights of the first step.
Estimated response rates differed by >20%, depending on the method used (Figure 1). Compared to CC, both PSM-LUNDEX and LUNDEX methods yielded much lower response rates, while the CARRAC method estimated response rate was between these estimates. Compared to CC analysis, differences in response between the intravenous and subcutaneous groups were smaller for the LUNDEX methods, larger for PSM-LUNDEX, and close to CC for the CARRAC method.
Both LUNDEX methods underestimate the true response rates by considering all patients stopping treatment as non-responders, while CC overestimates it by considering patients stopping as having a similar response rate to patients continuing treatment. The CARRAC method, which accounts for attrition by reason for drug discontinuation, obtained response rate estimates in between CC and LUNDEX corrections.
There are limitations for each method. CC does not correct for confounding or attrition bias; LUNDEX does not correct for confounding; for PSM-LUNDEX, overlapping propensity score only allowed selection of 561 patients per group; and CARRAC requires information on reason for stopping. Several other analyses, such as inverse probability weighting or instrumental variable, should be examined to obtain confounder and attrition-adjusted estimates of response rate. Furthermore, simulation studies are needed to assess the most accurate estimation method.