Construction and Verification of a Predictive Model for the Progression of Aortic Valve Calcification

Queues and baseline checks

The Multi-Ethnic Study of Atherosclerosis (MESA) is a community-based, multicenter prospective cohort study sponsored by the National Heart, Lung, and Blood Institute. Baseline examinations were conducted between July 2000 and September 2002. Visit 2 took place from July 2002 to January 2004 and visit 3 took place from January 2004 to July 2005. Institutional review boards at each center approved the study, and all participants provided written informed consent (19). A complete description of the design of MESA has been published elsewhere (20, 21). This manuscript was prepared using MESA Research Materials obtained from the National Heart, Lung, and Blood Institute (NHLBI) Biologic Specimen and Data Repository Information Coordinating Center and does not necessarily reflect the opinions or views of the MESA or the NHLBI.

Data for this study were obtained from the initial examination of the MESA cohort. Demographic details such as age, gender, race, education, income, smoking, drinking habits, and drug use were collected through questionnaires (22). Resting blood pressure was measured three times in a sitting position and the average of the second and third readings was recorded as systolic and diastolic blood pressure, along with heart rate. Hypertension was defined as systolic blood pressure ≥140 mmHg, diastolic blood pressure ≥90 mmHg, or the use of medications, combined with a self-reported diagnosis of hypertension. Diabetes classification included categories of no diabetes, impaired fasting glucose, and diabetes. Ankle-brachial index (ABI), calculated as the ratio of systolic pressure, was measured in the dorsal foot artery or posterior tibial artery to that in the brachial artery (23). Blood samples collected from participants fasting for 12 hours were used to measure and calculate fasting glucose (FG), total cholesterol (TC), triglycerides (TG), low-density lipoprotein cholesterol (LDL-C), high-density lipoprotein cholesterol (HDL-C), lipoprotein[a] (Lp[a]), interleukin-6 (IL-6), C-reactive protein (CRP), N-terminal pro-B-type natriuretic peptide (NT-proBNP), creatinine, and homocysteine (Hcy) (23). The estimated glomerular filtration rate (eGFR) for each sample was computed using the Chronic Kidney Disease Epidemiology Collaboration (CKD-EPI Creatinine Equation) (2021).

Measurement of valve calcification

The Agatston score was employed to assess AVC through continuous CT scans in a semi-automatic manner. The reliability of this measurement was ensured through duplicate assessments, demonstrating a high average agreement both within and between observers (kappa >0.9). Aortic valve calcification measurements were conducted at baseline (n = 6814, 2000–2002) and subsequently at visit 2 (2002–2004) or visit 3 (2004–2006).

Aortic valve calcification was defined as any calcification observed in the aortic lobes (24). Progression of AVC was identified as a positive difference between two scans (25, 26). At baseline, the distinction for coronary artery calcification, AVC, and mitral valve calcification focused solely on the presence or absence of calcification. A value of Agatston score = 0 indicated the absence of calcification, while Agatston score >0 signified the presence of calcification.

Statistical method

We described baseline features by AVC progression. R software (version 4.3.0) was used for analysis in this study. The ‘corrgram’ package is to be used to analyze the correlation of all variables in the baseline. The MESA data set was randomly divided into two groups according to 7:3, constituting the train set and the validation set. The train set was used to construct the model, and the validation set population was used for the internal verification of the model. LASSO-Cox regression model combines LASSO regularization with Cox proportional risk model to reduce the dimensionality of high dimensional data sets. Using 10-fold cross-validation with an optimal penalty coefficient λ to achieve both variable identification and variable contraction, preventing overfitting, is valuable when dealing with multiple predictors that may be unrelated or redundant data sets. The ‘Glmnet’ package is to be used to screen progression-related variables. Risk scores for each sample were estimated by weighting Cox regression coefficients. At the same time, the C-index was calculated to determine the accuracy of the model constructed by the selected variables. Using the ‘survival’ package, all samples were classified as high and low risk based on the optimal threshold for the risk score, and progressive curves were described for different risk populations. The ‘ROC’ package was used to describe the ROC curve, assess the sensitivity and specificity of the model, and calculate the AUC value from the ROC curve. The ability of the model to predict disease progression was further verified in the validation set. The ‘rms’ package was used to develop a nomogram and calibrate the probability of the train and validation sets predicting disease progression for 2, 3, and 4 years. The ‘ggDCA’ package was used to plot DCA and evaluate the clinical utility of the model. The ‘Shiny’ package was used to build a visual website platform.

Baseline characteristics

Baseline data from the initial MESA examination, along with the number of participants undergoing CT scans during visits 2 and 3, are presented in Figure 1. The study comprised a total of 2,533 patients, of whom 244 exhibited progression at visits 2 and 3. Progression of AVC was determined by a difference of ≥0 between the AVC Agatston score at the second or third follow-up and the baseline AVC Agatston score.

Figure 1

The baseline characteristics of the population in this study include: age (61.372±10.284 years), gender (male 55.27%), ethnicity (Caucasian 45.08%, Chinese 7.78%, African American 25.54%, Hispanic 21.59%), median follow-up time (30.28 months), progression rate (9.63%) and so on. Stratified by progression status, Table 1 illustrates the baseline clinical features of all patients. The mean AVC at baseline was 119.152 ± 290.947 Agatston units (AU) in those who progressed, and 13.195 ± 115.153 AU in those who did not. Among the 2,227 patients without AVC at baseline, only 86 exhibited progression, with a mean longitudinal change of 18.032 ± 28.895 AU/year. Of the 306 patients with positive AVC at baseline, 158 developed AVC, experiencing a mean longitudinal change of 58.980 ± 130.309 AU/year, while the remaining 148 patients showed no progression, with a mean longitudinal change of –41.473 ± 120.752 AU/year. Pairwise correlations for these baseline measures are depicted in Figure 2.

Figure 2

Progression data

The median follow-up duration for the study was 30.28 months. By the conclusion of the follow-up period, 244 individuals exhibited progression, while 2,289 individuals did not. Cumulative progression rates at years 1, 2, 3, 4, and 5 years were observed as 0% (0/1,121), 8.30% (93/1,121), 9.03% (153/1,694), 9.64% (241/2,500), and 9.63% (244/2,533), respectively.

Prediction model based on LASSO-Cox regression

Patients were randomly assigned to a train set and a validation set at a ratio of 7:3. The comparison of most included variables between the two groups revealed no statistically significant differences (p > 0.05) (Table S1). LASSO regression was applied for variable screening, and the variation characteristics of these variable coefficients are depicted in Figures 3A and 3B. A 10-fold cross-validation was implemented to analyze variables, indicating an optimal lambda value logarithmically around -5, where 12 variables (age, gender, drinking, waist-to-hip ratio (WHR), ankle-brachial index, hypertension, fasting glucose, HDL-C, lipoprotein[a], heart rate, CAC degree, AVC degree) yielded the lowest prediction error.

Figure 3

To address excessive variables and prevent overfitting, Akaike Information Criterion (AIC) was employed, resulting in the selection of seven variables with excellent predictive performance. These variables included age, sex, waist-to-hip ratio, fasting glucose, Lp[a], resting heart rate, and the presence or absence of AVC at baseline. A Cox regression model was constructed based on the parameters derived from LASSO regression screening (Figure 3C and Table 2). The C-index of the model was determined to be 0.869 (standard error = 0.014).

The LASSO-Cox regression analysis was utilized to compute the risk score for each sample in the training set. The survminer package facilitated the determination of the optimal threshold to classify patients in the training set into high-risk and low-risk groups (Figure 3D). Kaplan-Meier analysis demonstrated a higher progression rate in the high-risk group. ROC curves were employed to assess the predictive performance of variables in the training set, revealing high accuracy and sensitivity in predicting 2-year, 3-year, and 4-year progression rates (AUC2 = 0.869; AUC3 = 0.879; AUC4 = 0.895) (Figure 3E).

The following formula illustrates how to calculate the 2-year, 3-year, and 4-year risk of AVC progression:

The validation set was utilized to substantiate the predictive efficacy of the model, applying the same formula for risk calculation. Using the identical approach, patients were categorized into high-risk and low-risk groups (Figure 3F). Notably, in the validation set, the main results are consistent with the training set (Figure 3G).

Calibration and clinical application of LASSO-Cox regression model

A nomogram for predicting the progression of AVC was developed based on the seven selected variables (Figure 4A). Calibration curves illustrating the performance of the models in predicting progression at 2, 3, and 4 years demonstrated commendable concordance between predictions and observations across both the training and validation sets (Figures 4B and 4C). Additionally, the DCA curve further exhibited a strong agreement between the predicted probability of disease progression from the nomogram and the actual probability (Figure 5).

Figure 4

Figure 5

To enhance the clinical utility of the nomogram, we have provided an online platform (https://avc1.shinyapps.io/DynNomapp/). This tool allows for the quick estimation of the probability of progression in patients with mild to moderate AVC by adjusting variable parameters (example was shown in Figure S1).