Chapter 9 198 RESULTS Scenario 1: Drift In scenario 1 ’Drift’, patients from the trial are similar to patients from the external data (i.e., their covariates come from the same underlying distribution), but the outcomes differ due to a random drift term δ. The scenario of drift is the standard situation where methods such as the MPP are usually evaluated. The results for type I error rate and RMSE are shown in Figure 3 B. The RMSE of the analysis without external data (Ignore) is approximately 0.034. In case there is no drift, pooling the two data sources gives the lowest RMSE (approximately 0.023), 32% lower than ignoring the external data. The RMSE of pooling increases considerably when there is a nonzero drift, e.g., with a drift of δ = 0.375, the RMSE of pooling is 0.05 - a 47% increase compared with ignoring external data, and the type I error rate becomes severely inflated. For all cases except sub-setting 2, the RMSE and type I error rates of the ProPP and the MPP almost overlap and show the same characteristics (see Figure 3). In this scenario, where sample sizes are equal and patients are similar, all patients have approximately a probability of 1 2 to be in the trial or the external data (and hence odds wi of 0.5/(1 − 0.5) = 1). When wi = 1 for all patients, the ProPP specification in Equation 6 simplifies to the MPP specification in Equation 4. Sub-setting 2 (N0 = 400, Ne = 2000) shows that a relatively larger sample size in the external data causes the ’Pooling’ and the ’MPP’ methods to exhibit an increased RMSE and an inflated type I error rate (up to 25 percent in the MPP). The weights wi in the ProPP naturally account for such a difference in sample size and prevent this (unwanted) behavior. Compared with the hybrid methods of Wang, our method has a lower RMSE, at the cost of an inflated type I error rate. Due to the pre-specified amount of borrowing, Wang’s methods show a stricter control of the type I error rate in the simulations, but unlike the MPP and ProPP, this inflation continues to increase for higher values of drift, because the amount of borrowing is preset in these methods (see, for example, Wang 20 % in Figure 3A). The results of this scenario show that the MPP and the ProPP have similar performance in terms of mitigating prior-data conflict due to unmeasured confounding. Note that the ProPP provides additional safeguards against measured confounding, which by design did not occur in this scenario. The fact that the amount of borrowed external data in the ProPP does not automatically increase with the sample size of the external data, in which this method differs from the MPP, seems an advantage. the hybrid methods proposed by Wang, the ProPP has lower RMSE, but this comes at the cost of a type I error rate inflation.
RkJQdWJsaXNoZXIy MTk4NDMw