Research Group
The Epidemiological Health Survey on Early Menopause in Chinese Women is a national multicenter study. From September 1, 2023 to February 5, 2024, the researchers conducted an epidemiological survey of menopause status in Chinese women in 13 cities in 12 states. The selection is based on geographic location and the proportion of population in each state, using a multi-stage stratified cluster sampling method. The inclusion criteria were as follows: (1) Local household registration or women who are older than 6 months, older than 36-60 years, and are not pregnant. (2) informed consent and voluntary cooperation to complete all research content. Physical measurements (including height, weight, waist perimeter, blood pressure, etc.) and questionnaire surveys (including demographics, lifestyle, menopause symptoms, depression, anxiety, medical history, family history, and female-specific factors) were collected. Ethical approval for this study was obtained from the Ethics Review Committee of the National Centers for Women and Children's Health at the Centers for Disease Control and Prevention in China (FY2022-14). We comply with all relevant ethical regulations, including the Declaration of Helsinki. Informed consent was obtained from all individual participants included in the study.
A total of 52,803 women were surveyed, including 33,819 premenopausal women and 18,984 postmenopausal women. In this study, we constructed an early menopause prediction model using cross-section design. Therefore, only postmenopausal women were included. Women who were taking menopause hormone therapy and women who experienced surgical menopause were excluded. Variables with missing data >20% were excluded based on the rule of thumb that validity was compromised when the missing data reaches >20%39. There were only a few variables removed, and risk factors for menopause were not known. For the remaining variables, missing values correlate with observed variables, and therefore showed missing random patterns. Therefore, multiple assignments were employed by chain formula (mouse) that handled the uncertainty of missing data through generation and pooling of multiple data sets. This method maintained a variable relationship with minimal information loss. The variable distribution is similar before and after attribution, indicating no bias. For categorical variables with a missing rate of less than 1%, the most frequent categories were used for assignment. Finally, 18,015 postmenopausal women were included in model training and internal testing, including 2,193 women who experienced early menopause (menopause before age 45). The complete process of participant selection is shown in Figure 1a.
Early evaluation of natural menopause
Information on spontaneous menopause was obtained by participants' self-reported menopause status and dates they experienced their last menstrual period. Women under the age of 45 were defined as experiencing early spontaneous menopause. To reduce recall bias, surveys were designed using clear definitions and time anchors, and interviewers were trained to help participants recall the event as accurately as possible.
Candidate features
A total of 123 variables derived from anthropometric measurements and questionnaire surveys were probably related to early spontaneous menopause and were selected as candidate features. This feature includes seven sociodemographic variables, 18 environmentally related variables, 12 lifestyle factors, 11 dietary variables, 11 sleep related variables, 2 mental health variables, 19 reproductive factors, 15 history variables, 15 menopause symptoms variables, 6 family history variables, and six attitudinal indicators. Information on lifestyle factors (e.g., physical activity), sleep, and diet were collected specifically to reflect premenopausal status. Specific explanations of all candidate features were listed in Supplementary Table 3.
Predictor identification
First, to avoid collinearity or multicolinearity, Spearman correlation analysis was used to detect the severity of multicolinearity. For feature pairs with Spearman rank order correlation coefficients (ρ) If it exceeds 0.7, one of the features was removed. Second, we adopted the Boruta algorithm for functional dimension reduction (Fig. 1B). The Boruta algorithm is based on the Random Forest Classification Algorithm. It shuffles the real attributes and creates shadow attributes. Both REAL and shadow attributes form a new functional matrix for training and determine the importance of each attribute via Z-score. The Z-score of the actual attribute is compared to the maximum Z-score of the Shadow Attribute (MZSA). If the actual attribute's Z-score is significantly higher than MZSA (“above the score of the 95th percentile of the shadow feature), it is considered an important variable (confirmed). Otherwise it is considered unimportant (deny). The process is repeated until a pre-determined number of random forests is reached, and all shadow attributes are removed40.
Model training and evaluation
First, 18,015 postmenopausal women were split into training sets (80% of data) and test sets (20% of data), and then the training sets were split into five uniformly sized partitions to perform a 5x cross-validation.
We chose the 10 ml algorithm. From a methodological perspective, these algorithms cover the major categories of ML. This includes multi-layer perceptron neural networks (MLP) for deep learning, decision tree and random forest for tree-based models, category boost (cat boost), light gradient boost machine (light GBM), gradient boost tree (GBDT), adabing boosting (adabosting), adabing adabing boosting (gradient boosting tree) (xgboost) for ensemble learning as well as logistic regression and linear support vector classification (LinearSVC). Given that this study is a questionnaire-based prediction task, all of these models have the ability to process categorical data. Hyperparameters in each model were optimized through randomized search using cross-validation, utilizing 5x cross-validation before fitting across the training set. In the model structure process, categorical variables were transformed using one-hot encoding and continuous variables were normalized.
In class mitigation data, many ML methods have been “biased” towards the majority class.41. In prediction of early menopause, the dataset was very unbalanced, with the ratio of women with early menopause to women without menopause at ~1:8. To address this issue, we applied the EasyEnsemble algorithm42.
Model evaluation and explanation
The accuracy and area under the receiver operating characteristics (ROC) curve were used for comparisons across multiple models. Based on the area under the curve (AUC) values obtained, predictive models with the best performance in the independent test set were selected. Data reported that due to class imbalance, accuracy, Recall, F1 score, MCC, and balanced accuracy also better assessed minority class model performance alongside AUC. Accuracy and recall are important for identifying high-risk women without high false positives, and F1 scores and MCC provide an overview metric for this trade-off. To improve the interpretability of the model, we used the Shapley Additive Actresistivative Explenation (SHAP) approach to quantify the importance of functionality.43. To visualize the results, we created a bar plot of the input features that contribute to the optimal predictive model. This feature is ranked by the absolute value of the average SHAP value, indicating the relative importance of the features of the model prediction. We also added a SHAP-dependent contribution plot for the most influential features to show how each single feature affects prediction.
Simplifying the model
After finding the optimal predictive model (i.e., xgboost model), we reduced the number of 70 predictors identified by the Boruta algorithm to simplify the model. Predictors were initially ranked by importance. We then used a forward stepwise approach to add one factor in each of this ranking. Each addition calculated the AUC for the model. The curves plotted AUC values against the sequentially added factors and visualized how the AUC values changed by adding predictors.
External verification
External validation was performed using data from the Chinese Health and Resignation Longitudinal Study (Charls) conducted in China in 2011, 2013, 2015, and 2018 in China over 45 years of age.44. To retain as many predictors from the original model as possible, external validation was conducted in a population of postmenopausal women under the age of 55 using the 2011 Charls data. A total of 16 predictors were used for external validation. External validation was performed on the original model reconstructed with these 16 predictors.
Statistical analysis
Baseline summary statistics were presented as the average containing standard deviation (SD) of continuous variables as a percentage of categorical data. Statistical analyses were performed using R 4.2.3 and Python 3.6.
