Comparative analysis of machine learning models for detecting water quality anomalies in treatment plants

Machine Learning


A revised Quality Index (QI) serves as an effective metric for assessing water quality, enhancing the management of water purification processes. The suggested model can be trained using historical water quality data33, which may assist in forecasting potential water quality concerns and recommending suitable purification techniques. This model is capable of learning and adapting over time, thereby enhancing the accuracy and efficiency of water purification management. Figure 1 illustrates the development of the proposed model.

Fig. 1
figure 1

Construction of proposed model.

The first step entails gathering a dataset related to water quality, which includes a range of attributes. Relevant features from the dataset are selected for use in the preprocessing and model development stages. The dataset requires preprocessing before it can be used for model training. This includes tasks such as data cleansing, encoding categorical variables, and normalizing numerical features.The dataset is then partitioned into two subsets: the training set and the testing set. The training set is used to develop the model, while the testing set is used to evaluate its effectiveness. Typically, this separation is carried out using a ratio of 70% for training and 30% for testing. This ensures that there is sufficient data for the model to learn effectively, while also providing a substantial portion for assessing the model’s performance.A stacked ensemble model represents a sophisticated approach in machine learning, integrating various base models to enhance overall performance and reliability. H2O Auto DL is an automated Machine learning algorithm designed to manage extensive datasets and is well-suited for developing a stacked ensemble model.Following model training, a feature analysis is performed to identify the key elements that influence water quality predictions. This aids in understanding which features exert the greatest influence and can be leveraged to enhance the model.The final trained model is produced through the training process using the stacked ensemble H2O Auto DL method. It is equipped to generate forecasts on new data based on its analysis of the training set. The evaluation of the trained model is then conducted using the testing data. To determine the model’s effectiveness, its performance can be compared with alternative models or industry standards.

Water quality detection

The suggested algorithms for detecting water quality incorporate a Machine learning-based encoder and decoder. The encoder processes raw data obtained from water quality sensors and transforms it into a feature representation. This feature representation is then transmitted to the decoder, which reconstructs the data and detects any anomalies or issues related to water quality.This process leverages the capabilities of artificial neural networks, enabling them to learn and identify patterns within the data to generate precise predictions. Training the algorithm on an extensive dataset of water quality measurements allows it to accurately detect and classify various types of water quality issues. Figure 2 illustrates the encoder and decoder components of the proposed model.

Fig. 2
figure 2

Encoder and decoder of the proposed model.

The encoder and decoder serve as essential components within a sophisticated Machine learning architecture known as the Transformer. These components are responsible for processing and understanding input data and subsequently converting it into meaningful output.The encoder begins by receiving the input data, which is first tokenized—divided into smaller, manageable units called tokens. These tokens are then transformed into numerical representations using an embedding layer. To retain sequence information, positional encoding is applied, assigning a unique value to each token based on its position in the input sequence. This step helps the model recognize the order and relative position of tokens.The encoded input then passes through a multi-head attention mechanism, which allows the model to focus on different parts of the input data simultaneously. This enhances the model’s comprehension and learning capability. The output from this step is combined and normalized to reduce complexity and improve the interpretability of the information.The decoder processes the tokenized target data, representing the desired output sequence. Like the encoder, it performs tokenization, embedding, and positional encoding. Additionally, it employs a masked multi-head attention mechanism on the target input. This prevents the model from accessing future tokens during training, ensuring predictions are made only from already generated parts of the output sequence.The decoder then applies another multi-head attention layer, which allows it to attend to relevant parts of both the encoder’s output and the target sequence simultaneously. This output is again added and normalized before being passed into a fully connected feedforward network for further processing.Finally, the result from the feedforward network undergoes another addition and normalization process, and a final fully connected layer generates the predicted output sequence.

Classification

The input layer serves as the initial interface that acquires data from external sources. The Input Sample Layer specifically denotes a collection of input data samples. The Convolutional layer executes a mathematical operation on the input data by utilising a collection of filters (kernels) to identify features within the input image. The kernel size is 7 × 7 with a stride of 2, indicating that the filter advances 2 pixels at each step to encompass the full input image. Batch Normalisation serves as a method to enhance the stability and performance of a neural network through the standardisation of inputs to each layer. This approach minimises internal covariate shift, facilitating quicker learning and enhancing generalisation capabilities. Figure 3 illustrates the classification module.

Fig. 3
figure 3

Activation functions serve to incorporate non-linearity into the neural network. This is implemented subsequent to the Batch Normalisation layer to introduce non-linear characteristics to the output. Following the initial convolutional layer, a max-pooling layer is implemented with a kernel size of 3 × 3 and a stride of 2. This layer reduces the dimensionality of the data by down-sampling the feature maps, enhancing the efficiency of the network. Subsequently, a Conv block is implemented, comprising a sequence of Conv layers featuring various filter sizes, succeeded by Batch Normalisation and Activation layers. The layers collaborate to enhance the extraction of features from the input data. An Identify block is subsequently implemented, resembling the Conv block, yet it incorporates an addition operation that merges the output of the Conv layers with the original input. This aids in maintaining crucial characteristics of the input data while also enhancing the network’s performance. The min-max normalisation approach is used to scale the feature in the [0, 1] range.

$$t’=\frac{{t – {{\hbox{min} }_A}}}{{{{\hbox{max} }_A} – {{\hbox{min} }_A}}}$$

(1)

Here, \({\hbox{min} _J}\) and \({\hbox{max} _J}\) are the min and max values.

$$WQI=\frac{{\sum\nolimits_{{b=1}}^{M} {{s_i}} \times {w_i}}}{{\sum\nolimits_{{b=1}}^{M} {{z_b}} }},$$

(2)

$${s_b}=100 \times \left( {\frac{{{T_b} – {T_{ideal}}}}{{{S_b} – {S_{Ideal}}}}} \right),$$

(3)

Where \({s_b}\)the original parameter’s testing value and N is the full attribute.

$${z_b}=\frac{Y}{{{Q_b}}},$$

(4)

Where, Y is the proportionality consistent.

$$Y=\frac{1}{{\sum\nolimits_{{b=1}}^{M} {{q_i}} }},$$

(5)

$$K=P\left( {Z1.H1+Z2.H2+i} \right)$$

(6)

$$p\left( h \right)=\sum\limits_{{{x_a}\varepsilon q}} {{\alpha _a}{k_a}Y\left( {{h_a},h} \right)} +b$$

(7)

$$k={i_0}+{i_1}{h_1}+{i_2}{h_2}+…{i_b}{h_b}$$

(8)

The RMSE has computed by the following Eq. 9

$$RMSE=\sqrt {\frac{1}{N}\sum\nolimits_{{b=1}}^{M} {{{\left( {ZSB_{J}^{b} – ZSB_{J}^{b}} \right)}^2}} }$$

(9)

The MAE measures prediction mistakes without considering sign. It estimates the absolute differences between predicted and actual values over the test sample.

$$MAE=\frac{1}{M}\sum\nolimits_{{b=1}}^{M} { – ZSB_{J}^{b}}$$

(10)

$$H_{{CEF}}^{J}\sum\limits_{{l,m}} {{\Theta _j}A_{l}^{m}\left\langle {{r^l}} \right\rangle \widehat {O}_{l}^{m}}$$

(11)

Where, J = Complete angular momentum quantity. The constant ground state has expressed as the following,

$${K_1}= – 3J\left( {J – \frac{1}{2}} \right){\Theta _2}A_{2}^{0}\left\langle {{r^2}} \right\rangle .$$

(12)

Where \(V_{l}^{m}\)is complete harmonic factor.

$$V\left( r \right)=\sum\limits_{{l,m}} {V_{l}^{m}\left( r \right){Y_{l,m}}\left( {\theta ,\phi } \right)}$$

(13)

Let, expressed the diffusion model has demonstrate as the following,

$${S_v}={Y_f}{v^{0.5}}+D$$

(14)

The permeability of water has measured as the following,

$$A=\frac{T}{{J \times v \times F}}$$

(15)

A minimal water value of 0.0 is optimal for model simulation. Negative numbers indicate a bias of overestimation in the model, whereas positive values signify a bias of underestimation.

$$PREI=\left( {\frac{{{k_b} – \widehat {{{k_b}}}}}{{{k_b}}}} \right) \times 100$$

(16)

Where \({k_b}\)original quality index for \({b_{th}}\) opinion and \({\widehat {K}_b}\) is the mean performance.

$$Q{O_f}=B\left( {Z{S_{fb}}} \right)+{\alpha _b}$$

(17)

The distinction between predicted and observed variances is articulated as follows.

$$T\left( h \right)=T\left( {{\alpha _1}} \right)+T\left( {{\alpha _2}} \right)+…+T\left( {{\alpha _m}} \right)$$

(18)

$$QO={q_{\overline {h} }}=\sqrt {T\left( {\overline {h} } \right)} =\frac{{q{c_{\overline {h} }}}}{{\sqrt M }}$$

(19)

RSS is an averages standard uncertainty for each variable.

$$DO={\left[ {\sum\limits_{{b=1}}^{m} {{{\left[ {{d_b}QO\left( {{h_b}} \right)} \right]}^2}} } \right]^{\frac{1}{2}}}$$

(20)

$${D_b}=\frac{{\partial p\left( {{h_b}} \right)}}{{\partial {h_b}}}=\frac{{\partial {k_b}}}{{\partial {h_b}}}$$

(21)

In this case, y is an input variable that can be randomly chosen, k is a coverage aspect, and Cu is the complete ambiguity in the random data.

$$y={v_{t,1 – \alpha /2}}$$

(22)

The scientifically defined variables root mean square error (RMSE), mean absolute error (MAE), mean square error (MSE), and coefficient of determination (R2) are

$$RMSE=\sqrt {\frac{1}{m}\sum\limits_{{b=1}}^{m} {{{\left( {{k_b} – {{\widehat {k}}_b}} \right)}^2}} }$$

(23)

$$MAE=\frac{1}{m}\sum\limits_{{b=1}}^{m} {\left| {{k_b} – {{\widehat {k}}_b}} \right|}$$

(24)

An Average pooling layer is implemented, which diminishes the spatial dimension of the output data by calculating the average of adjacent pixel values. The fully connected layer serves as the concluding stage in the convolutional neural network, where the outputs from the preceding layers are flattened and input into a conventional fully connected neural network. This layer conducts classification utilising the features that have been extracted from the input data. The Softmax layer is utilised to transform the output from the Full conv layer into a probability distribution across the various classes. This enables the network to generate a prediction based on the input data and categorise it into one of the established classifications.

figure afigure a

Proposed Algorithm: Encoder-Decoder-Based Water Quality Assessment and Anomaly Detection

The proposed algorithm is designed to enhance water purification management by leveraging advanced Machine learning techniques and a modified Quality Index (QI). The algorithm begins by preprocessing the water quality data, including normalization and feature selection, to ensure that only the most relevant parameters are utilized for model training. The adaptive QI is calculated by weighting each feature’s importance, providing a dynamic evaluation of water quality that adjusts in real-time according to detected anomalies. This continuous adjustment enhances decision-making within water treatment plants, ensuring that the purification processes are responsive to changing water conditions. The algorithm’s Machine learning architecture, using an encoder-decoder approach, effectively captures complex patterns in water quality data, enabling accurate anomaly detection and trend forecasting.

To further improve water purification management, the model is deployed on IoT devices for real-time monitoring. Anomalies in water quality are detected by comparing real-time data with reconstructed values from the Machine learning model using a threshold-based method. The algorithm then visualizes detected anomalies and QI trends, supporting predictive maintenance and strategic decision-making.A continuous feedback loop enables the model to learn from real-time data, ensuring adaptability to evolving water quality patterns. This approach not only forecasts water quality but also recommends appropriate purification methods, ultimately enhancing the operational efficiency of water treatment plants.By integrating Machine learning with a dynamically adjusted Quality Index (QI), the algorithm provides a transformative solution for sustainable water resource management.



Source link

Leave a Reply

Your email address will not be published. Required fields are marked *