Evaluation metrics
In this study, we use Root Mean Square Error (RMSE), Mean Absolute Error (MAE), Mean Absolute Percentage Error (MAPE) , and Correlation Coefficient (CORR) as metrics to evaluate the performance of the models.
RMSE is a commonly used metric for evaluating the accuracy of a predictive model. It measures the differences between observed values and predicted values, with lower values indicating better accuracy. The formula for RMSE is defined as
$$\begin{aligned} \text {RMSE} = \sqrt{\frac{1}{n} \sum _{i=1}^{n} (y_i – \hat{y}_i)^2}, \end{aligned}$$
(3)
where \(y_i\) is the observed value, \(\hat{y}_i\) is the predicted value, and \(n\) is the number of observations.
MAE is another metric used to assess the accuracy of a model’s predictions. Like RMSE, lower MAE values indicate better accuracy. The formula for MAE is
$$\begin{aligned} \text {MAE} = \frac{1}{n} \sum _{i=1}^{n} |y_i – \hat{y}_i|. \end{aligned}$$
(4)
MAPE measures the average percentage difference between observed values and predicted values, relative to observed values. MAPE is expressed as a percentage, with lower values indicating better accuracy. The formula for MAPE is
$$\begin{aligned} \text {MAPE} = \frac{100\%}{n} \sum _{i=1}^{n} \left| \frac{y_i – \hat{y}_i}{y_i} \right| . \end{aligned}$$
(5)
CORR is a statistical measure that quantifies the degree of association between two variables. The value of CORR usually ranges between -1 and 1. A value close to 1 indicates a strong positive correlation between the two variables, while a value close to -1 indicates a strong negative correlation. The formula for CORR is:
$$\begin{aligned} \text {CORR} = \frac{\sum _{i=1}^{n} (x_i – \bar{x})(y_i – \bar{y})}{\sqrt{\sum _{i=1}^{n} (x_i – \bar{x})^2 \sum _{i=1}^{n} (y_i – \bar{y})^2}}, \end{aligned}$$
(6)
where \( x_i \) and \( y_i \) are the individual data points, and \( \bar{x} \) and \( \bar{y} \) are the means of \( x \) and \( y \) respectively.
These metrics are essential to gauge the effectiveness and accuracy of the models in predicting visibility and help in understanding how closely the predictions align with the actual observations.
Comparative models
To better evaluate the performance of GCN-GRU, we introduce four comparative models as follows:
GRU, a deep learning model, simplifies the design of LSTM11 by combining the forget gate and input gate into a single update gate. Additionally, it merges the cell state and the hidden state of LSTM, leading to a more architectural model with fewer parameters.
Random Forest23, a robust ensemble learning model, classifies and regresses by combining multiple decision trees. Its primary advantage is its ability to handle high-dimensional data with numerous features and its robustness to outliers and missing data. The core idea of the random forest is to construct multiple decision trees by randomly selecting training samples and feature subsets. Then, by averaging or majority voting on the predictions of these trees, it produces the final prediction. This method can reduce the overfitting problem and enhance the model’s generalization ability.
XGBoost24, a gradient boosting tree model, performs ensemble learning by progressively adding tree models within the gradient boosting framework. XGBoost has gained widespread attention for its outstanding predictive performance and computational efficiency. It optimizes the model by minimizing a loss function, using gradient information to update the tree model, thereby gradually reducing prediction errors. Furthermore, XGBoost introduces regularization to prevent overfitting.
LightGBM25, a lightweight ensemble learning model based on gradient boosting trees, adopts a histogram-based algorithm to improve computational efficiency. LightGBM proposes the Gradient-based One-Side Sampling (GOSS) to remove small-gradient instances and introduces Exclusive Feature Bundling (EFB) to reduce feature dimensions. These techniques substantially reduce memory consumption and computation time while maintaining good predictive performance.
As the time series models, both GRU and GCN-GRU can employ multi-step and single-step forecasting. Multi-step forecasting continuously generates predictions for multiple future time steps, suitable for long-term trend prediction but potentially accumulates initial prediction errors over time. In contrast, single-step forecasting predicts the value for only the next time step, quickly adapting to recent data changes and suitable for scenarios requiring rapid response. Therefore, a total of seven models were involved in the comparison: GCN-GRU (Single), GCN-GRU (Multi), GRU (Single), GRU (Multi), LightGBM, Random Forest, and XGBoost. Moreover, parameters for Random Forest, XGBoost, and LightGBM were optimized using Bayesian search to enhance the model’s generalizability. As for the time series models, we empirically set the optimal parameters.
Analysis of visibility influencing factors
To accurately identify and analyze the key factors influencing visibility in Jiangsu Province, we employed the maximum Relevance Minimum Redundancy (mRMR) algorithm for feature selection. We identified several critical meteorological and environmental factors that significantly impact visibility.
From Fig. 6a, it can be inferred that Relative Humidity (RHU) and \(PM_{2.5}\) concentration are the most significant factors affecting visibility. Besides, there is a certain redundancy between some variables, such as Temperature (TEM), hourly Maximum Temperature (TEMmax), and hourly Minimum Temperature (TEMmin). Only one among them needs to be selected for model training. Based on the ranking results of mRMR and previous research26,27,28, we choose Relative Humidity (RHU), 2-meter Wind Speed (WS), 2-meter Wind Direction (WD), Precipitation (PREC), hourly Highest Temperature (TEMmax), \(PM_{2.5}\), \(PM_{10}\), CO and \(O_{3}\) concentration as the input variables of the model.
Figure 6b displays the variation of visibility and other elements for Station 58027 (Coordinates: \(117.15^\circ \)E, \(34.28^\circ \)N) in January 2019. The red dashed lines indicate threshold values for some elements. For instance, low visibility events occurring below 80% relative humidity are identified as haze, between 80% and 95% as fog-haze, and above 95% as fog. \(PM_{2.5}\) concentration greater than 100 indicates air pollution and more than 300 indicates severe pollution. \(PM_{10}\) concentration greater than 100 is also deemed as air pollution. Visibility below 10 kilometers is classified as a low-visibility event.
Ax represents periods of low visibility, while Bx represents periods of high visibility. For instance, during the A1 period, the low visibility was caused by fog-haze, with air humidity ranging between 70% and 80%, low wind speed, and high concentrations of several pollutants. By the B2 period, the haze had dissipated, primarily due to the gradual increase in wind speed. The A2 period witnessed the occurrence of thick fog, caused by short-term rainfall that increased the air humidity and reduced the air pollutants due to the washing effect of the rain. The A3 period represents a typical fog-haze scenario, with humidity levels between 80% and 95% and high concentrations of pollutants. During the B2 period, due to reduced humidity and increased wind speed, the fog-haze dispersed, improving visibility. The low visibility during A4 was caused by precipitation. Despite significant wind speeds, short-duration heavy rainfall can suddenly reduce visibility. Once the rainfall ceased, the visibility returned to normal.
Following the analysis of visibility variations during periods marked as Ax (low visibility) and Bx (high visibility), we further employed the Spearman’s Rank Correlation Coefficient29 to assess the relationships between visibility and various meteorological and environmental factors. This method is particularly effective for non-normally distributed data and is less sensitive to outliers, revealing the rank-order relationships between variables. The analysis results are presented in Table 1. Our analysis indicates that factors such as relative humidity, \(PM_{2.5}\) concentration, and CO concentration exhibit negative correlations with visibility, suggesting that an increase in these factors typically leads to a decrease in visibility. Conversely, the highest temperature, wind speed, and ozone concentration show positive correlations with visibility, implying that visibility may improve under these conditions.
Model forecasting
We train the model on data from year 2017 to 2018. The dataset was divided into 60% for training, 20% for validation, and 20% for testing to ensure comprehensive training and effective validation under various data conditions. Since historical information beyond 72 hours minimizes future visibility trends, a 72-hour time window was selected as a critical parameter for model training.
To validate the model’s applicability and robustness across different geographic locations, 11 meteorological stations within Jiangsu Province were selected for testing, considering the representativeness of geographical distribution and data completeness. Table 2 and Fig. 2 show the specific station information.
Comparison of Test Set Results for GCN-GRU Single Step Prediction, GCN-GRU Multi step Prediction, GRU Single Step Prediction, GRU Multi step Prediction, LightGBM, Random Forest and XGBoost.
Figure 7 illustrates the performance of seven models for forecast horizons of 1–6 h. As the forecast horizon increased, the correlation coefficient (CORR) decreased, while the root mean square error (RMSE), mean absolute percentage error (MAPE), and mean absolute error (MAE) increased, indicating a decline in forecasting accuracy over time. Compared to traditional machine learning models, time series prediction models significantly improved the accuracy of visibility forecasts, especially for more extended forecast periods.
Our analysis also found that single-step prediction results were almost identical to multi-step predictions but required substantially less time. Therefore, single-step prediction was chosen as the primary method for visibility forecasting. Additionally, when comparing the GRU-GCN model to the standalone GRU model, the GRU-GCN model exhibited improvements. In 6-hour forecasts, the average CORR increased by 3.32%, RMSE decreased by 17.52%, MAPE reduced by 26.62%, and MAE decreased by 16.53%.
Case analysis: forecasting application
In our study, particular attention was given to the performance of the GCN-GRU single-step prediction model in practical application scenarios. Taking Station 58027 as an example, we conducted rolling forecasts for data from January 2019, and the results are shown in Fig. 8. In this case, the initial 72 hours of data served as historical input and thus did not contain forecast values. The results showed that the trends predicted by the GCN-GRU model generally aligned with the actual values, but there were discrepancies in the prediction of extreme values. Specifically, the model tended to slightly overestimate at lower values and underestimate at extremely high values.
This consistency in trend prediction and deviation in extreme values could be attributed to several factors. Firstly, the distribution of training data might be sparse in the extreme value regions, making it challenging for the model to learn behaviors in these situations accurately. Secondly, the model’s structure might be more suited to capturing general trends rather than extreme events. Moreover, predicting extreme values in time series data is inherently more challenging, especially in dynamic and changing environmental conditions.
January 2019 GCN-GRU rolling forecast results.
However, it is essential to note that such inaccuracies in extreme value prediction do not significantly impact the model’s effectiveness in practical applications. In most cases, particularly in visibility forecasting scenarios, it is more crucial to capture changes in future trends rather than to predict each specific value accurately. In this regard, the GCN-GRU model demonstrated good performance, providing valuable insights for timely and effective decision-making.
In conclusion, although the GCN-GRU model requires improvements in predicting extreme values, it still exhibits strong potential and practicality in real-world applications. Future work could focus on optimizing the model structure, improving data preprocessing strategies, and incorporating new algorithms and techniques to enhance the model’s predictive accuracy in various situations, especially for extreme values.
