Smoothing and denoising the original time series data effectively improves forecast accuracy.twenty three. Therefore, in this study, the SG filter was used to remove the noise in the original water quality time series. The SG filter is a filtering method based on local polynomial least-squares fitting in the time domain. The best feature of this filter is that it can guarantee that it does not change the shape and width of the signal while removing the noise.
subsequence \ (X \) the size of the window \(w=2m+1\) can be expressed as:
$$\left\{{x}_{sm},. . , {x}_{s}, . . , {x}_{s+m}\right\}, s\in\left[m+1, T-m\right].$$
(1)
polynomial \(p\left(i\right)\) orderly \(R\) The is used to fit the data points within the window is defined as:
$$p\left(i\right)=\sum_{v=0}^{R}{a}_{v}{i}^{v}, i\in \left[-m,m\right]$$
(2)
where \({a}_{v}\) denotes the vth coefficient of the SG filter.
Then use the least squares method to minimize the error. \(\epsilon\).
$$\epsilon =\sum_{i=-m}^{m}{(p\left(i\right)-{x}_{s+i})}^{2}.$$
(3)
Then you can find the best fit \(p\left(0\right)\) of the center point of the window \({x}_{s}\) by calculation \({a}_{0}\). Sliding the window reveals each point in the series. \ (X \) It will be the center point in the window until all values in the series are smoothed.Finally we get the smoothed sequence \({x}{\prime}\).
STL time series decomposition method
In order to better extract the trend and nonlinear characteristics of the river water quality time series and preserve the seasonal trend characteristics of the series, we used the STL decomposition technique to decompose the original water quality time series. Seasonal and trend decomposition (STL) using Loess is a very popular and robust decomposition method for time series, and Loess is a method for estimating nonlinear relationships. STL is aimed at decomposing time series data \({Y}_{v}\) At some point it became a trend (\({tv set}\)), season (\({S}_{v}\)) and the residual (\({R}_{v}\)), represented as \({Y}_{v}={T}_{v}+{S}_{v}+{R}_{v}\)twenty four. The algorithm consists of an outer loop and an inner loop. The outer loop is primarily used to assign robust weights to each data point through residuals to mitigate the effects of outliers. The inner loop is nested within the outer loop and primarily performs trend fitting and periodic component calculations. Here’s what the inner loop does for the kth epoch:twenty five:
-
1.
Detrending. remove the trend component from the original series, \({Y}_{v}-{T}_{v}^{(k)}\).
-
2.
Smoothing of cycle subseries. Each periodic subseries obtained in step 1 is smoothed by Loess to produce a preliminary seasonal series shown below. \({C}_{v}^{(k+1)}\).
-
3.
low pass filtering.sequence \({C}_{v}^{(k+1)}\) The data obtained in step 2 are processed by moving average and Loess regression to obtain the resulting sequence. \({L}_{v}^{(k+1)}\)This is equivalent to extracting the lowpass of a periodic subsequence.
-
Four.
Detrending the Smoothed Cycle subseries.Calculate seasonal trends \({S}_{v}^{(k+1)}={C}_{v}^{(k+1)}-{L}_{v}^{(k+1)}\ ).
-
Five.
Out of season. Subtracting the periodic component gives \({Y}_{v}-{S}_{v}^{(k+1)}\).
-
6.
Trend smoothing. Run a loess regression on the cyclic series obtained in step 5 to get the trend component. \({T}_{v}^{(k+1)}\).
Once the inner loop reaches the accuracy requirement, the outer loop is started and the residual component is \({R}_{v}^{(k+1)}\) It is computed by the estimated trend and seasonal component of the outer loop. The calculation method is as follows.
$${R}_{v}^{(k+1)}={Y}_{v}-{T}_{v}^{\left(k+1\right)}-{S}_ {v}^{(k+1)}.$$
(Four)
Improved TCN model
Temporal Convolutional Networks (TCNs) are based on traditional one-dimensional fully convolutional neural network models, combining causal convolution, dilated convolution, and residual block structure, so the model extracts features from time series and makes predictions. It has the ability to achieveIt can effectively solve the problem of deep network performance degradation in the process of network training26. Figure 2a shows the standard TCN residual block structure including extended causal convolution, weight norm, ReLU activation function and dropout regularization. Extended causal convolution is used for feature extraction, weight norms can adjust hidden layer inputs to combat gradient explosion problems in networks, ReLU activation functions introduce nonlinearity into the model, and dropout regularization operations are Neurons can be randomly dropped. Prevents overfitting and accelerates model training speed according to certain probabilities.If \ (X \) Residual Block Input, Residual Block Output \(o\) can be expressed as:
$$o=activate\left(x+F\left(x\right)\right),$$
(Five)
where \(Activation\) is the activation function and \(F(x)\) Residual. I remained, \(F(x)\) Although is not actually 0, the deep learning network’s training performance does not degrade because the stacked layers of a deep learning network can always learn new features.
(be) standard TCN residual block structure (b) improved TCN residual block structure (c) TCN network structure adopted by us.
Casual convolution was originally proposed in WaveNets networks. Since traditional CNN models cannot handle the sequence problem directly, causal convolution can abstract the sequence as \({x}_{1}, {x}_{2},. . .,{x}_{t}\) and \({y}_{1}, {y}_{2},. . .,{y}_{t-1}\) Predict \({y}_{t}\) and make \({y}_{t}\) A value that is close to the actual value. Compared to Recurrent Neural Networks (RNNS), models using causal convolution do not use recurrent connections, so time series data can be input in parallel, resulting in faster training of the network and large There are advantages.27. However, expanding the receptive field of neural network neurons with standard causal convolution requires either stacking many network layers or using large convolution kernels. To solve the problem of the limited receptive field of standard causal convolution, TCN combines dilated and causal convolution, using dilated casual convolution (DCC), to reduce the number of neurons without incurring a significant increase in computational cost. Increases receptive fields.28. The one-dimensional extended causal convolution operation is represented as
$$F\left(s\right)= \sum_{i=0}^{k-1}f(i){x}_{s-di},$$
(6)
where \ (X \) is the input sequence, \(f(i)\) Filter, also called convolution kernel. \(d\) is the expansion coefficient, \(k\) is the size of the convolution kernel. \(s-di\) Allows convolution of past inputs only. Figure 3 shows the extended causal convolution structure with extended coefficients. \(d=\mathrm{1,2},4\) Similar to the convolution kernel \(k=2\).
Dilated causal convolution with dilation coefficients d = 1, 2, 4 and kernel k = 2.
The attention mechanism is primarily a simulation of the human brain’s attentional allocation form, the essence of which is to assign weights to different functions within the hidden layer in order to emphasize the impact of important functions.29. The self-attention mechanism is a refinement of the attention mechanism, aimed at capturing internal correlations in the data to further improve the predictive power of the model.30. Figure 4 shows the structure of the self-attention mechanism. The self-attention mechanism consists of Q(Query), K(Key), and V(Value) vectors obtained by multiplying the input data by three matrices. \({W}_{q}\), \({W}_{k}\)and \({W}_{v}\). In our proposed model, \({W}_{q}\), \({W}_{k}\)and \({W}_{v}\) is the transformation of the water quality data and the attention matrix used to determine feature attention can be calculated by equation (1). (7).
$$attention\left(Q,K,V\right)=softmax\left(\frac{Q{K}^{T}}{\sqrt{{d}_{K}}}\right),$$
(7)
where T. refers to the transpose of the matrix, \({d}_{K}\) refers to the dimension of K..
Structural diagram of self-attention.
Add self-attention before each convolutional unit in the TCN residual block structure. The improved TCN residual block structure is shown in Fig. 2b. Based on the above improved residual block structure, the basic structure of the deep TCN model adopted in this paper is shown in Fig. 2c. The model consists of an input layer, refined residual blocks, and an output layer. The input layer mainly receives decomposed water quality time series data. Two stacked residual block structures were used to increase model depth and make model training more appropriate. The output layer is a fully connected layer that takes the output vectors of the TCN model and computes the predicted values.
