Tree models for covariate-dependent method agreement with repeated measurements in clinical research

Abstract

Background: In clinical research, the Bland-Altman analysis is commonly used to assess agreement of metric measurements made by two or more techniques, devices or methods. The approach can also deal with repeated measurements per subject or observational unit. However, a strong and implicit assumption is that agreement of methods is homogeneous across subjects. Objective: To extend the previously introduced multivariable modeling of conditional method agreement with single measurements per subject to the frequent case of repeated measurements. Methods: Appropriate regression trees, called conditional method agreement trees (COAT), are generalized to capture the dependence of the parameters of the Bland-Altman analysis on covariates. These parameters, the expectation and variance of the differences between the methods, are decomposed into subject-specific components to account for repeated measurements. Whilst the theoretical, asymptotic properties of tree models are known, a simulation study was carried out to assess the performance of COAT in finite samples. A comparison of devices measuring cardiac output serves as an application example. Results: COAT is applicable to the two relevant cases of paired and unpaired repeated measurements. In the simulation study, it controlled the type-I error at the nominal level and could detect covariate-dependent method agreement with increasing sample size. The Adjusted Rand Index, a measure of concordance between the estimated and true subgroups, reached very high values close to the maximum of 1. The analysis of cardiac output showed that patients' characteristics may influence the agreement between measuring devices, with implications for use in patient care. Conclusion: COAT can explicitly define subgroups of heterogeneous method agreement in dependence of covariates with appropriate statistical testing in case of repeated measurements.

Figures (9)

• Figure 1: CO measurements for a sample patient from the dataset. At five measurement time points, five measurements were taken for each method, and mean values were calculated. Grey symbols indicate measurements that were not used further in the application. The black circles and triangles at the first measurement time point represent measurements used in the unpaired setting of the application, as they cannot be directly matched to one another. The mean values for all measurement time points, represented by horizontal lines, were used in the paired setting of the application.
• Figure 2: COAT for conditional agreement of cardiac output (CO) measurements of two different technologies. CO-PAC and CO-FC are compared for the unpaired measurements of CO. The minimal subgroup size (minsize$= 6$) and the significance level ($\alpha = 0.05$) are set as model parameters. The mean measurements are included along with other patient characteristics as covariates.
• Figure 3: COAT for conditional agreement of cardiac output (CO) measurements of two different technologies. CO-PAC and CO-FC are compared for the unpaired measurements of CO. The minimal subgroup size (minsize$= 6$) and the significance level ($\alpha = 1$) are set as model parameters. The tree growth was restricted to a maximal depth of two (maxdepth$= 2$). The mean measurements are not included as covariate.
• Figure 4: COAT for conditional agreement of cardiac output (CO) measurements of two different technologies. CO-PAC and CO-FC are compared for the paired measurements of CO. The minimal subgroup size (minsize$= 6$) and the significance level ($\alpha = 0.05$) are set as model parameters. The mean measurements are included along with other patient characteristics as covariates.
• Figure 6: Two-sample BA test of differences in method agreement of cardiac output (CO) measurements between patients with and without vascular disease. CO-PAC and CO-FC are compared. Setting $\alpha = 1$ ensures that the test result is provided independent of the size of the respective p-value.