Using machine learning methods to predict electric vehicles penetration in the automotive market

Since the time series data in this study are monthly, eleven binary columns have been added to the dataset to reflect the effect of each month (in the first month of every year, the column corresponding to the first month is set to 1, and the column for the other months is set to 0). An example of this binary data is shown in Table 3.

For most vehicles, data includes 79 months (January 2014 through July 2020). According to Fig. 5, the last 14 months are selected for the testing set as rolling cross-validation. Using cross-validation on a rolling basis is one way to validate the time-series model. Starting with a subset of data for training, forecasting for later data points and then checking the accuracy of the forecasts. The same forecasted data points are included in the next training dataset, and further forecasts are made.

The model is cross-validated using 12 forecasting stages, with each stage predicting sales in the next three months. During each prediction stage, the preceding months are divided into training and validation (70% for training and 30% for validation. Then these data are transferred to the model, the model predicts sales in the next three months, then the forecast date is moved forward by one month, and this process has been repeated 12 times. Vehicle sales in the next three months are predicted each time the model runs, assuming most of the vehicle’s characteristics remain the same. Due to fluctuation and changes in economic conditions, a three-month time horizon is used for predicting the future.

Overfitting is one of the principal problems in ANN training. The Dropout layers between the neural network layers are one of the best solutions in the ANN to avoid overfitting. During the dropout layer, the number of neurons trained in each layer and those discarded is determined randomly (rather than activating all neurons at once, only a fraction are activated)³⁸. TensorFlow’s early stopping tool is another basic solution to avoid overfitting. Early stopping works in the following way: during the repetition of training, the validation data is used to calculate the error value, and whenever the validation error value increases throughout several epochs, the model is ready to be stopped, and overfitting is prevented. For all three models, both solutions are used to prevent overfitting. Dimensionality reduction is another way to prevent model overfitting. In this study, Principal Component Analysis was used in several modes to reduce dimensions, but this technique was not used due to the significant decrease in model performance.

In order to improve the modeling process, all three models’ hyperparameter values and network architectures were determined by Automated Machine Learning (AutoML). AutoML is the process of automating ML applications. The number of hidden layers, the number of neurons in these layers, and the dropout rate was determined by the Tuners. Several values are introduced to the Tuner for each hyperparameter. The Tuner trains different model versions and selects the best one based on the best result (lowest error or loss) on the validation data. This method sets the hyperparameters to the optimal value, and the model is then applied to a test dataset.

The model’s error or loss is calculated using the Mean Absolute Error (MAE) loss function in all three models. Selecting a suitable optimization algorithm for the DL model is essential to reduce the run time and reach the desired result. Adam’s optimization algorithm is used for these models, which is a generalized version of stochastic gradient descent. It reduces memory usage, converges faster, and corrects high variance and learning rates³⁹.

Comparison of models

With the validation data, hyperparameters are adjusted, and the model is built to predict vehicle sales over the next three months (three months following the last validation date). The model run-time for all vehicles was very long due to the many vehicle types (357). In a random sample of 15 vehicles, different models’ states were compared using fixed data, and the results were compared between the three models.

The sale of each vehicle is predicted in 12 stages; each prediction stage includes the prediction for the next three months, respectively, the first month of the prediction, the second month of the prediction, and the third month of the prediction. In total, the first predictions include 12 months, the second predictions include 12 months, and the third predictions include 12 months. Model performance was evaluated using the Mean Absolute Percentage Error (MAPE), the Root Mean Square Error normalized by the change range (\(NRSME_{range}\)), and the Root Mean Square Error normalized by the mean value (\(NRSME_{mean}\)) according to Eqs. 15–18.

According to the above equations, \(y_{t}\) denotes the actual value at time t, \(\hat{y}_{t}\) denotes the predicted value at time t, \(y_{max}\) denotes the maximum actual value, \(y_{min}\) denotes the minimum actual value, \(y_{mean}\) denotes the average actual value, and T is equal to the total number of predicted samples. The average error values of all vehicles were calculated to compare the results of various models. A weighted average was calculated using the total number of sales of each car per month as a weight for the vehicle according to Eq. (19) since the numbers of vehicle sales are not on the same scale, and the error rate is more important in vehicles with high sales. A further method of checking the models’ performance is to compare the R-square, slope, and intercept of the linear regressions fitted on predicted and observed data for all three models. Table 4 summarizes the evaluation results of the models.

In the proposed hybrid model, the error values are lower, the R-square accuracy is higher, the slope value is closer to 1, and the intercept is closer to 0. At this stage, the proposed hybrid model was recognized as preferable to both the LSTM and ConvLSTM models.

Implementation of the proposed hybrid model to predict the share of EVs

For all vehicles, the proposed hybrid model has been implemented, and 12 points of prediction have been used to determine the sale of all vehicles. Linear regression was fitted on the predicted sales and actual values to evaluate the model’s performance, as shown in Table 5.

Primary data segments vehicles by specifications according to segments like CAR-SMALL_COMPACT, CAR-MID_FULL SIZE, MINIVAN LARGE, and PICKUP LARGE. Each segment consists of similar vehicles in appearance and specifications that compete with one another. Segments that include EVs have been separated to determine the share of EVs. Based on actual and predicted sales, the shares of electric and gasoline vehicles have been compared and evaluated for each month of the test data. For example, the CAR-MID/FULL-SIZE segment includes 28 vehicles (23 gasoline vehicles and five EVs). Figure 6 shows the share of EVs in this segment based on twelve prediction stages (three months per stage), separately for the first, second, and third months of each prediction.

All segments’ MAEs for EVs’ share forecasting in the forecast’s first, second, and third months are shown in Table 6. The average MAE value of all segments was calculated as 3.2% for the first months, 3.8% for the second months, and 3.5% for the third months. The average value for all segments and all forecast months was calculated at about 3.5%, which shows that the proposed hybrid model performed well.

As part of the model analysis, the segments that included EVs were separated again and ranked by sales within each segment. The rankings were based on actual sales (actual rank) and predicted sales (predicted rank); the actual rank and predicted rank were used for evaluation. Kendall-Tau correlation (Kendall’s correlation) is commonly used to check the concordance of two ranked lists; this technique was used to examine the actual and predicted rankings in this study. Kendall’s correlation rate for two rating lists \(r_{a}\) and \(r_{b}\) (\(\tau_{{r_{a} , r_{b} }}\)) is represented by Eq. (20) ⁴⁰.

In Eq. (20), \(n_{c}\) represents the number of concordant pairs, \(n_{d}\) represents the number of discordant pairs, and n represents the total number of ranks in each of the rating lists⁴⁰. The maximum number of discordant pairs between two ranking lists equals \(\frac{1}{2} n\left( {n – 1} \right)\), and Kendall’s correlation equals + 1 if all pairs of ranks are concordant and -1 if none are concordant ⁴⁰. For all segments, Kendall’s correlation values were calculated separately for the first, second, and third prediction months, and the average values are shown in Table 7. The average Kendall’s correlation value of all segments was calculated as 0.76 for the first months, 0.742 for the second months, and 0.75 for the third months. The average Kendall’s correlation value for all segments and all forecast months was calculated at about 0.75, which indicates the great performance of the proposed hybrid model in predicting the ranking.

Sensitivity analysis

Sensitivity analysis was performed to determine which features significantly impacted the trained model. Thus, for each vehicle, the pre-trained model that was evaluated in previous stages has once again predicted the number of vehicle sales with new input data, and its outputs have been assessed. All features, except the investigated feature, are valued at their average. For the investigated feature, the five values from the training data (the min value, the first quartile, the second quartile, the third quartile, and the max value) are taken into consideration. Five predictions were made based on these five values, and a range of changes in predicted sales was calculated. The change ranges for all features have been measured, and the four features with the most extensive range have been identified. As an example, during the sensitivity analysis of the BMW I3 for 2020, the following four features had the broadest range of changes: the Consumer Price Index (CPI), the equivalent MPG for EVs, the Google search score for car prices (Google Trends), and the car price. This EV’s sensitivity analysis plots are shown in Fig. 7.

Based on Eq. (21), slope values for the four characteristics with the most extensive range of changes are calculated in different parts of the graph, and the results are summarized in Table 8. For example, the number of sales of this EV has decreased by 8 for every thousand-dollar increase in price when the price is in the range of the minimum value to the first quarter. As the slope is zero percent in the second and third parts of the graph, the price in the first, second, and third quartiles is equal, and when the price is in the third quartile to the maximum price, the number of sales for this EV decrease by 6 for every thousand-dollar increase in price.

There has been a decrease in car sales due to the increase in the CPI. It is also true that with the increase in the CPI, the final price of the car and the price of auto parts have increased, which has led to a decrease in the desire to buy this car. The second feature is equivalent MPG for EVs, a higher equivalent MPG indicating better performance and less fuel consumption in a fixed distance has led to an increase in sales of this car. The third feature identified is the increase in the car price search score on Google (Google Trend), an indicator that buyers are more curious about this car, contributing to its sales. The fourth specified feature of the car is its price, and its sales have decreased with the increase in its price. As a result of the sensitivity analysis, the manufacturers of this car could use policies such as lowering the price of the car and its parts (CPI and car price), improving the performance of the vehicle’s engine (the equivalent MPG), and developing advertisements and introducing the car to the public (Google trend score) to increase sales.

Sensitivity analysis has been conducted for each EV, and the results show different sensitivity for each vehicle. From each segment that includes EVs, one vehicle was selected as a sample, and the results of its sensitivity analysis are shown in Table 9.

Each EV’s sensitivity analysis identifies features that differ from the others, as shown in Table 9. According to the results of the sensitivity analysis, ten features that were most frequently found in the sensitivity analysis of all the EVs were identified as the most influential features: Shoppers, Min price, CPI, Sales, Google Trends score 3 (Price), Make & model news score, Personal income per capita, Make news score, Interest Rates on 60-month, and Mean options score, respectively.