A comparative analysis of classical machine learning models with quantum-inspired models for predicting world surface temperature

In this section, we will discuss about the methods used in the proposed work. We have trained the dataset on three Classical Machine Learning models five Neural Network model and three Quantum Machine algorithms for comparative analysis.

Classical machine learning algorithms

The following section provides a succinct overview of the classical machine learning algorithms utilized in our study. Specifically, our analysis incorporates three prominent methodologies: Autoregressive Moving Average (ARMA), Autoregressive Integrated Moving Average (ARIMA), and Seasonal Autoregressive Integrated Moving Average (SARIMA). These methodologies have been extensively employed in time series analysis and forecasting tasks due to their robustness and proven efficacy.

ARMA (autoregressive moving average)

ARMA stands for Autoregressive Moving Average. It’s a statistical method used for time series analysis and forecasting. ARMA models are a combination of autoregressive (AR) and moving average (MA) models.

The autoregressive (AR) model:

• AR models predict the next value of a time series using prior data.

• It is assumed that the current value of the time series is linearly dependent on its previous values.

Mathematically it can be expressed as Eq. (1).

Where,

X_t = Time series value at time t.

c = Constant term.

ϕ_i = Autoregressive (AR) coefficients (lags 1 to p).

θ_j = Moving average (MA) coefficients (lags 1 to q).

\(\epsilon _{t}\) = White noise error term (i.i.d., mean 0, variance σ²).

ARIMA (autoregressive integrated moving average)

ARIMA stands for Autoregressive Integrated Moving Average. It is a popular statistical approach for time series forecasting and analysis, especially when the data exhibits non-stationary behavior. The ARIMA model, like the ARMA model, has autoregressive (AR) components. These phrases indicate the relationship between the current observation and prior time steps. The integrated (I) component indicates the difference in the time series to make it stationary. This differencing includes subtracting successive observations to eliminate patterns and seasonality. Moving Average (MA) Component: The ARIMA model, like the ARMA model, has a moving average component that represents the connection between the current observation and residual errors from prior observations.

Mathematically it can be expressed as Eq. (2).

where,

L = Lag operator (L^k X_t = X_t−k).

Δ^d = d-th order differencing (ΔX_t = X_t – X_t−1).

Other terms same as ARMA.

SARIMA (seasonal autoregressive integrated moving average)

SARIMA, or Seasonal Autoregressive Integrated Moving Average, is an extension of the ARIMA model that takes seasonality into account for analyzing and predicting time series data. SARIMA models are very effective for dealing with data that has periodic patterns or seasonality.

SARIMA models have the same components as ARIMA models, but with an extra seasonal component. There are several types of components: autoregressive (AR), integrated (I), moving average (MA), seasonal autoregressive (SAR), seasonal integrated (SI), and seasonal moving average (SMA).

We employed the SARIMAX method in Python with an order of (1, 0, 1) to train the AutoRegressive Moving Average (ARMA) model, ARIMA method with an order of (2, 2, 2) to train the AutoRegressive Integrated Moving Average (ARIMA) model and SARIMAX method with a seasonal order of (15, 2, 5, 5) to train the Seasonal AutoRegressive Integrated Moving Average (SARIMA) model.

Neural networks

The ensuing section delineates the neural network models integrated into our study, encompassing a range of sophisticated architectures tailored for time series analysis. Specifically, our investigation harnesses the power of Long Short-Term Memory (LSTM) networks, Stacked LSTM models, Convolutional Neural Network-Long Short-Term Memory (CNN-LSTM) hybrids, and Convolutional LSTM (ConvLSTM) architectures. These state-of-the-art methodologies have garnered widespread recognition for their ability to capture intricate temporal dependencies within sequential data, making them well-suited for our research objectives.

Neural networks model with swish activation

To effectively train our neural network model, we began by transforming our data into trainable features and an output feature. We structured the data such that the preceding five years served as the trainable features, while the data from the sixth year was designated as the output. This setup allowed us to establish an input size of 5 features and an output size of 1.

From Table 2 you can see, the initial layer of our neural network model was configured with 5 neurons to accommodate the input features, while the final layer comprised a single neuron to generate the output. Nestled between these layers were hidden layers designed to enhance the model’s capacity for learning intricate patterns within the time-series data. We opted for a Dense layer with 20 neurons followed by another Dense layer with 16 neurons. The activation function employed in both these layers was Swish, which is characterized by the formula x * sigmoid(x)³⁹. Swish is renowned for its smoothness, non-monotonic behavior, and its efficacy in modeling time-series data¹⁸. Its unbounded nature above and boundedness below make it a fitting choice for our purposes. During the training phase, we utilized a learning rate of 0.01 and conducted 200 epochs of training. The default batch size of 32 was employed, ensuring efficient processing of the data during each iteration. Table 2 explains neural network model with swish activation architecture.

Vanilla LSTM

Table 3 presents a comprehensive overview of the architectural composition of the Vanilla long Short-Term memory (LSTM) model employed in our study⁴⁰. The model begins with an input layer comprising 5 neurons, facilitating the ingestion of the input data. Subsequently, an LSTM layer is introduced, featuring rectified linear unit (ReLU)¹⁶ activation functions and housing 50 neurons, enabling the network to capture Temporal dependencies within the data effectively. Following the LSTM layer, a dense layer is incorporated with an output size of 1, signifying the final output of the model. Notably, the model’s compilation process was carried out utilizing the Adam³⁵ optimizer.

Stacked LSTM

Table 4 provides a detailed depiction of the architecture employed in our investigation, specifically focusing on the stacked LSTM model³⁸. The initial layer of this model comprises 5 neurons, serving as the input layer responsible for processing the incoming data. Subsequently, the architecture incorporates two consecutive LSTM layers, each comprising 50 neurons and employing ReLU⁴¹ activation functions. This dual-layer LSTM configuration enhances the model’s capacity to capture intricate Temporal patterns and dependencies within the dataset. Following the stacked LSTM layers, a dense layer with an output size of 1 is integrated to present the final output of the model. Notably, the compilation process of this model utilized the Adam optimizer⁴².

CNN LSTM

Table 5 unveils the intricate architecture of the convolutional neural network long Short-Term memory (CNN LSTM) model³⁶a departure from the preceding models that operated on one-dimensional data. Distinguishing itself by its ability to process two-dimensional data, this model necessitates an initial reshaping of the input data to conform to a 2D array structure. Given the inherent limitations of converting an input size of 5 to a 2D array, we opted to utilize data spanning the past four years, presenting it as a 2D array with dimensions of 2 × 2. The model commences with an input layer comprising four neurons, representing the 2 × 2 shape of the reshaped input data. Subsequently, a Conv1D layer, nested within a timedistributed layer, is introduced, boasting 64 filters and employing ReLU²⁷ activation functions. This convolutional layer serves to extract intricate features from the input data, facilitating enhanced pattern recognition capabilities. Following this, a MaxPooling1D layer with a pool size of 2 is incorporated within the timedistributed layer, followed by a flatten layer, aimed at Preparing the data for subsequent processing. The architecture further integrates an LSTM layer with 50 neurons, utilizing ReLU¹⁴ activation functions to capture Temporal dependencies within the dataset. Lastly, a dense layer with an output size of 1 is appended to yield the final output of the model. Notably, the model’s compilation process was executed utilizing the Adam optimizer³⁰.

ConvLSTM

Table 6 unveils the intricate architecture of the convolutional long Short-Term memory (ConvLSTM) model, uniquely designed to operate on three-dimensional data. Given its distinctive capability to process data in three dimensions, an initial reshaping of the input data was imperative to conform to a 3D array structure. This reshaping was executed such that the third dimension possessed a size of 1, resulting in data with a dimensionality of 1 × 2 × 2, effectively capturing the essence of the dataset. The model’s architectural journey commences with an input layer comprising four neurons, reflecting the 1 × 2 × 2 shape of the reshaped input data. Subsequently, a ConvLSTM2D layer is introduced, featuring 64 filters and employing ReLU¹² activation functions. This convolutional LSTM layer plays a pivotal role in extracting intricate Spatiotemporal features from the input data, thereby facilitating enhanced pattern recognition capabilities. Following this, a flatten layer is incorporated to prepare the data for subsequent processing stages. Lastly, the architecture integrates a dense layer with an output size of 1, serving as the final output layer of the model. The compilation of this model was meticulously carried out using the Adam optimizer³⁰.

Quantum machine learning algorithms

The forthcoming section offers a concise overview of the quantum machine learning (QML) algorithms incorporated into our study, representing a pioneering foray into harnessing quantum computing techniques for predictive modeling tasks. Our investigation embraces a trio of quantum methodologies: Quantum Neural Networks (QNN), Variational Quantum Regression (VQR), and Quantum Support Vector Regression (QSVR). These cutting-edge algorithms leverage the unique principles of quantum mechanics to enable unprecedented computational capabilities, promising transformative advancements in machine learning and predictive analytics.

QNN (quantum neural networks)

Quantum Neural Networks (QNNs) are a new frontier in the field of quantum computing, expressing a break from traditional neural network structures while keeping its fundamental principles⁹. Unlike their classical counterparts, QNNs do not have discrete quantum neurons. They do, however, have structural similarities, with layers of qubits processing and transmitting data. In principle, QNNs work through a sequence of qubit layers, similar to those seen in traditional neural networks. Each layer of qubits takes input from the previous layer, evaluates it, and then propagates the results to the next layer. Notably, these layers do not require to have uniform widths, allowing for greater freedom in the distribution of qubits between levels.

IBM made pioneering achievements in the creation of QNNs with their revolutionary module known as ‘EstimatorQNN’. This module provides as a foundation for building Quantum Neural Networks, allowing for the integration of parametrized quantum circuits with defined input data and weights, as well as optional observables. The EstimatorQNN framework relies on mixed quantum circuits, which are made up of two separate components: the feature map and the ansatz. The feature map is critical in providing input parameters to the network, whereas the ansatz includes the weight parameters required for the network’s activities⁵. To speed up the building of these circuits, IBM proposes the QNNCircuit idea, which streamlines the generation of both feature maps and ansatzes.

When employing a QNNCircuit within the EstimatorQNN module, the necessity for explicit input and weight parameters is obviated, as these elements are automatically derived from the circuit itself. Using this QNN, feature map, ansatz and an optimizer we can train NeuralNetworkRegressor model. Then we used NeuralNetworkRegressor method in Python with a feature map of ZZFeatureMap()⁵and ansatz of RealAmplitudes to train the model. This model was trained using the COBYLA optimizer with a maximum of 100 iterations and base quantum neural network of EstimatorQNN.

VQR (variational quantum regressor)

Variational Quantum Regressor (VQR) is a quantum machine learning algorithm designed for regression tasks²⁹. Here’s a quick description of how VQR normally operates:

• Quantum circuit design: VQR uses a parameterized quantum circuit (PQC) as its quantum component⁴. The PQC is commonly made up of layers of quantum gates that may be changed using parameters.

• Cost function: VQR seeks to minimize a cost function that quantifies the difference between the quantum circuit’s anticipated outputs and the actual outputs of the training dataset.

• Training procedure: The training method often includes an optimization loop in which the quantum circuit’s parameters are iteratively changed to minimize the cost function. This optimization can be carried out using traditional optimization approaches such as gradient descent or alternative algorithms.

• Data encoding: Input data must be encoded into quantum states, generally using techniques like amplitude encoding or other quantum encoding methods.

• Measurement: After encoding the data and passing it through the quantum circuit, measurements are made to determine the predictions.

• Classical post-processing: The predictions acquired from the quantum circuit are frequently post-processed classically to increase accuracy or to turn them into the required output format.

VQR’s particular parameters change based on the technology and quantum circuit employed. However, typical criteria may include:

• Number of qubits: Determines the size and complexity of the quantum circuit.

• Depth of the quantum circuit: Determines the number of layers in the parameterized quantum circuit.

• Learning rate: A parameter used in the optimization algorithm to control the size of parameter updates.

• Number of training iterations: Determines how many iterations of the optimization loop are performed during training.

• Regularization parameters: Additional parameters used to prevent overfitting and improve generalization performance.

VQR training entails experimenting with and tweaking these parameters to obtain the ideal values. We employed the VQR() method in Python with a feature map of ZZFeatureMap()⁵and ansatz of RealAmplitudes() to train the Variational Quantum Regressor (VQR) model. This model was trained using the COBYLA optimizer with a maximum of 100 iterations.

QSVR (quantum support vector regressor)

Quantum support vector regression (QSVR) is similar to classical support vector machines (SVM)¹⁴ especially in its application to high-dimensional classification and regression applications. However, QSVR distinguishes itself by using quantum kernels to detect patterns inside datasets. In many algorithms, data points have a clearer structure when projected onto a higher-dimensional feature space, which is helped by a kernel function. The kernel function, abbreviated as K, acts on two inputs, x and y, each of n dimensions⁴³. The function f converts these n-dimensional inputs to an m-dimensional space, where m usually surpasses n by a large margin. This processing typically reveals underlying links within the data. It takes a feature map as parameter.

The quantum kernel algorithm’s key feature is its capacity to calculate a kernel matrix⁴⁴. This n-dimensional matrix, composed of the data points x and y and the feature map f, captures the dataset’s complicated linkages and affinities. Surprisingly, this kernel matrix acts as a key component, coordinating traditional machine learning approaches such as support vector classification, spectral clustering, and ridge regression. We employed the QSVR method in Python with a feature map of ZZFeatureMap and Quantum Kernal of FidelityQuantumKernel to train the Quantum Support Vector Regressor (QSVR) model.

Quantum advantage in high-dimensional data

Quantum machine learning (QML) algorithms, particularly those utilizing quantum kernels, have a distinct advantage when dealing with high-dimensional data. This issue often arises in climate science and other complex areas. Traditional methods struggle with the heavy computational load needed to process and analyze these datasets. In contrast, quantum algorithms utilize the principles of superposition and entanglement to navigate high-dimensional feature spaces efficiently⁴⁵ .This advantage is particularly clear in Quantum Support Vector Regression (QSVR), where quantum kernels help the model capture complex non-linear relationships that classical kernels may overlook⁴³.

The natural parallelism of quantum computing allows QML models to assess multiple data points at the same time. This greatly cuts down the time needed for high-dimensional data analysis. For example, QSVR’s use of fidelity-based quantum kernels improves prediction accuracy and shows strong performance against noise, which makes it especially fitting for real-world tasks like climate modeling⁴³. Additionally, Schuld points out that quantum kernel methods can map data into much larger Hilbert spaces, revealing patterns that classical systems cannot find⁴⁶. This quantum advantage highlights the potential of QML in addressing the challenges of scalability and complexity in high-dimensional datasets.

Model architecture justification

We put a lot of thought into our model architectures to effectively tackle the unique features of climate time-series data while also considering the limitations of current quantum hardware. For the neural network side of things, we opted for an LSTM configuration with 50 neurons per layer. This choice is backed by research from Sherstinsky which shows it’s the sweet spot for capturing decade-long dependencies in climate patterns³⁷. We also chose the Swish activation function (f(x) = x·σ(βx), where σ is the sigmoid function) because it really shines in temporal modeling tasks. As highlighted by Swish’s smooth, non-saturating gradients do a better job than traditional ReLU activations when it comes to learning complex, non-stationary climate trends³⁶.

When it comes to our quantum implementations, we’re taking a variational approach, using qubit counts of 5, 8, and 10. These were carefully selected through Pareto optimization, balancing circuit expressibility with the limitations of NISQ devices⁹.This selection gives us enough Hilbert space dimensionality to represent climate features while still keeping coherence times manageable. We specifically chose the COBYLA optimizer for a few reasons:

1. (i)

   it’s resilient to noise in the variational quantum eigensolver context, it operates without derivatives, which helps us avoid the barren plateaus problem⁴⁵.

2. (ii)

   it has a solid track record of converging in shallow quantum circuits with 12 qubits or fewer. Our ansatz designs intentionally incorporate temporal inductive biases through Cyclic parameterized gates to capture periodic climate oscillations, entangling layers that are scaled according to the autocorrelation window of the input features and measurement bases that align with the main climate oscillation modes.