Evaluation of machine learning and deep learning algorithms for fire prediction in Southeast Asia

The fire prediction results for sample data from Laos and Thailand are illustrated in Figs. 5a–f and 6a–f, respectively. The results for other Southeast Asia countries are given in Appendix (Figures S1–S3). For all countries, including Thailand and Laos, six models were utilized for fire prediction: (a) Persistence, (b) MLP, (c) CNN, (d) LSTM, (e) CNN-LSTM, and (f) ConvLSTM. The fire data collected from the VIIRS satellite is represented in blue for 2012 to 2023, with the fitted model curve shown in red for 2012 to 2020. Forecasts for the years 2021 to 2029 are displayed in green, while a dotted purple line indicates the trend.

Before the model evaluations, we also assessed the temporal fire trends from 2012 to 2014 using the Mann–Kendall seasonal trend statistic for the various countries. Results suggested Brunei, Cambodia, and Indonesia all show negative Mann–Kendall Statistics, indicating a slight decline in their respective datasets. However, the Z-statistics for these countries are relatively close to zero, with values of − 0.1773 for Brunei, − 0.3949 for Cambodia, and − 0.5037 for Indonesia, which indicates that despite the observed decrease in these countries, the trends are not strong. In contrast, Laos, Myanmar, the Philippines, and Timor Leste all exhibit weak positive trends, with total Mann–Kendall Statistics of 125, 59, 32, and − 131, respectively. Laos, which has a positive statistic, also shows a Z-statistic of 0.2031. Myanmar and the Philippines display Z-statistics close to zero (0.0959 and 0.0520, respectively), implying that their slight increases are insignificant. Despite showing a negative statistic, Timor Leste also failed to show statistical significance with a Z-statistic of − 0.2129. Malaysia and Thailand are more comparable to the first group of countries with negative Mann–Kendall Statistics of − 315 and − 118, respectively, suggesting a declining trend and a Z-statistic of − 511 and − 0.1917, respectively. Similarly, Vietnam, with a Mann–Kendall Statistic of − 122 and a Z-statistic of − 0.1982, also reflects a weak, statistically insignificant decrease. In comparing and contrasting these countries, the overall picture is that most countries showed an increasing or decreasing trend, with less statistically significant seasonal trends. It is important to note that the model parameterization and fitting were performed based on the original data without any modifications.

The ConvLSTM model, which integrates convolutional and LSTM layers, demonstrated the lowest RMSE for fire prediction in Laos, highlighting the necessity of incorporating both spatial and temporal patterns in this context. In contrast, the CNN model achieved the lowest RMSE in Thailand, indicating that a convolutional approach is more effective at capturing relevant data patterns compared to other models.

Tables 1, 2, and 3 present the evaluation metrics used for model comparison across different Southeast Asian countries. Table 1 provides Root Mean Square Error (RMSE) values, Table 2 presents Mean Absolute Error (MAE), and Table 3 lists R-squared (R²) values. These tables highlight the best-performing model for each region, with the lowest error values marked in bold for RMSE and MAE tables, and highest R-squared values.

Specific to RMSE, the metric showed higher values which can be attributed to highly varying spatial and temporal nature of fires in the region and inherent fire data characteristics used from 2012 to 2024. The Convolutional Neural Network (CNN) model demonstrated consistently strong performance across Brunei, Indonesia, Malaysia, the Philippines, Timor-Leste, and Thailand. In these countries, CNN achieves the lowest RMSE values, with standout performances of 9.32 in Brunei, 7278.05 in Indonesia, 416.75 in Malaysia, 1666.81 in the Philippines, 400.40 in Timor-Leste, and 11,387.51 in Thailand. This suggests that CNN effectively captures spatial patterns relevant to the metrics for these regions. Meanwhile, the ConvLSTM model shows the best results for Laos, Myanmar, and Vietnam, suggesting it is more effective in areas with complex spatiotemporal dynamics. ConvLSTM achieves an RMSE of 13,241.27 in Laos, 13,965.56 in Myanmar, and 2207.06 in Vietnam. For Cambodia, the CNN-LSTM hybrid model stands out, performing best with an RMSE of 9784.36, indicating that the combined spatial and temporal features captured by CNN-LSTM suit the data characteristics in this region. Overall, CNN is the most frequently optimal model for regions with predominantly spatial dependencies, while ConvLSTM and CNN-LSTM excel in areas requiring a blend of spatial and temporal feature extraction. Model specific results and important discussion points are provided below.

The Persistence model, which assumes future values mirror past observations, serves as a simple and computationally efficient baseline, occasionally capturing patterns in stable or slow-changing phenomena. However, its limitations are evident in its high RMSE values across most countries, suggesting its ineffectiveness in capturing dynamic or rapidly changing systems, such as those influenced by urbanization or climate variability. In countries like Indonesia, Myanmar, and Laos, the Persistence model performs poorly, with RMSE values significantly higher than those of more advanced models, highlighting its inability to account for complex trends and temporal patterns.

The MLP model, a basic neural network architecture, generally outperformed the Persistence model, achieving relatively low RMSE values in countries with simpler or less dynamic patterns, such as Malaysia and Timor-Leste. However, its lack of mechanisms for handling temporal dependencies limited its effectiveness for sequential data that requires memory of previous time steps. Thus, in countries with complex temporal patterns, such as Laos and Myanmar, the MLP model’s RMSE values are considerably higher than those of CNN, LSTM, or ConvLSTM, underscoring its limitations in capturing intricate temporal dynamics and spatial dependencies within the data.

Compared to the persistence and MLP models, the CNN models are known for its ability to capture spatial dependencies. Thus, it demonstrated a strong performance across several countries, achieving the lowest RMSE values in Brunei (9.41), Indonesia (7278.05), Malaysia (416.75), the Philippines (1666.81), Timor-Leste (400.40), and Thailand (11,387.51). This indicates that CNNs are well-suited for capturing spatial patterns, making them effective in regions where spatial features, such as landscape characteristics, are significant predictors of fires. However, CNNs lack an inherent temporal memory, limiting their effectiveness for capturing time-dependent processes like long-term climatic trends or seasonal changes. This limitation is evident in countries like Myanmar and Laos, where higher RMSE values suggest that CNNs may struggle with complex temporal dynamics.

Relatively, LSTM models are well-suited for sequential data, capturing long-term dependencies and temporal trends, making them beneficial for datasets with strong temporal dynamics. Thus, we noted that in countries like Brunei and Malaysia, the LSTM model showed reasonable performance, though it did not reach the lowest RMSE values. However, LSTMs lack the ability to capture spatial dependencies, which may contribute to their relatively high RMSE values in regions like Cambodia, Laos, and Thailand, where both spatial and temporal factors are crucial. Additionally, LSTMs are computationally more demanding than simpler models, posing a potential drawback for large-scale or real-time applications.

In contrast to either CNN or the LSTM models alone, the CNN-LSTM hybrid model leverages both the spatial feature extraction of CNNs and the temporal memory of LSTMs, making it highly effective for spatiotemporal data. Thus, it performs particularly well in Cambodia, achieving the lowest RMSE (9784.36), indicating its strength in capturing both spatial and temporal patterns, especially useful for complex phenomena like agriculture or climate-driven processes. However, the CNN-LSTM model did not consistently outperform simpler models across all countries such as in Brunei, Indonesia, and the Philippines, CNN alone achieves better results, suggesting that the hybrid model’s complexity may be unnecessary in cases dominated by spatial patterns. Additionally, the CNN-LSTM model has higher computational requirements, which can limit efficiency compared to single-architecture models like CNN or LSTM.

ConvLSTM is an advanced model that integrates convolutional and recurrent layers to effectively capture spatial dependencies and maintain temporal memory in spatiotemporal data. Thus, it effectively captured spatial dependencies and temporal memory, achieving the lowest RMSE values in Laos (13,241.27), Myanmar (13,965.56), and Vietnam (2207.06). This makes it suitable for scenarios with complex spatiotemporal dependencies. However, its higher computational cost can be a limitation, particularly in regions like Brunei and Malaysia, where simpler spatial dependencies prevail. In these cases, CNNs or other basic models can outperform ConvLSTM, suggesting that its complexity may be unnecessary when temporal variability is lower, allowing simpler models to achieve comparable or better results with reduced resource demands.

The RMSE differences between models stem from their ability to capture temporal fire patterns. Simpler models like Persistence and MLP lack trend-learning capabilities, leading to higher RMSE values. Persistence assumes fire occurrences repeat past patterns, missing trends, and sudden changes. MLP offers slight improvements but struggles with sequential dependencies. CNN models, though designed for spatial data, effectively captured short-term temporal patterns, making them well-suited for fire prediction. ConvLSTM builds on this by combining spatial and temporal tracking, allowing it to model both short-term fluctuations and long-term trends. In contrast, LSTM, which specializes in long-term dependencies, performed worse.

In addition to RMSE, we evaluated model performance using MAE and R² to ensure a more comprehensive assessment. R² provides insight into how well the model explains variance in fire occurrence, while MAE offers an intuitive measure of prediction error by quantifying absolute deviations. The comparison of RMSE, MAE, and R² across models further validates CNN and ConvLSTM as the best-performing models.

The results indicate that ConvLSTM achieved the highest R² values and the lowest MAE, reinforcing its effectiveness in capturing fire trends. CNN also exhibited strong performance across all metrics, making it a competitive alternative. In contrast, simpler models such as Persistence and MLP showed lower R² values and higher MAE, confirming their limitations in modeling fire occurrences.

ConvLSTM had the highest computational cost among the six models evaluated due to its combination of convolutional layers and LSTM units, which require more parameters and complex operations. This resulted in longer training times and higher memory usage, making real-time applications more challenging. In general, computational speed across models followed the pattern: Persistence < MLP < CNN < LSTM < CNN-LSTM < ConvLSTM. Hyperparameter tuning involved 5–7 parameters per model, each with multiple possible values, leading to a large configuration space particularly for deep architectures like CNN-LSTM and ConvLSTM. To manage this, we employed parallel processing and CPU acceleration using all available cores on the local system. Models designed for handling sequential dependencies, such as CNN-LSTM and ConvLSTM, require significantly more computational resources than simpler architectures like CNN and MLP. While ConvLSTM achieved better accuracy in some regions, its deployment in real-world settings would require hardware acceleration (e.g., GPUs/TPUs) and optimization techniques such as model pruning and quantization to improve efficiency. In contrast, simpler models like CNN and MLP are computationally lighter and can be more practical for operational use where rapid temporal predictions are required. Future research should explore the trade-off between model complexity and computational efficiency, balancing accuracy with real-time feasibility.

In summary, the CNN model is most effective in countries with fire data that had primarily spatial dependencies, such as Brunei, Indonesia, Malaysia, the Philippines, and Timor-Leste. Meanwhile, ConvLSTM performed well in countries with significant spatiotemporal dependencies, such as Laos, Myanmar, Thailand, and Vietnam. Cambodia is best served by the CNN-LSTM hybrid model, suggesting a need for balanced spatial and temporal processing capabilities. This pattern highlights the diverse data characteristics across Southeast Asian countries and the importance of selecting models suited to each region’s needs.

The insights gained from our analysis of satellite-derived fire data emphasize the critical need for model evaluation before integrating them into any decision support systems (DSS) for fire management and mitigation. Given the unique temporal and spatial patterns across Southeast Asian countries, a one-size-fits-all approach is insufficient; instead, tailoring models to each region’s specific characteristics is essential for effective decision-making³⁹. Evaluating multiple machine learning and deep learning models upfront optimizes resource allocation by identifying the most efficient option and enhances the accuracy and reliability of predictions that inform critical management decisions. Incorporating both spatial and temporal patterns alongside historical trends has been shown to improve prediction accuracy and using robust statistical metrics^40,41. Integrating changes in weather and recurring seasonal weather patterns can improve fire forecasting accuracy. Integrating socioeconomic factors into fire forecasting can improve accuracy by accounting for human behavior, land use changes, and infrastructure influencing fire risk, such as population density, urbanization⁴¹, and economic activities. Combining these factors with ensemble methods, which combine multiple models or algorithms, can improve prediction robustness^42,43, particularly when addressing complex fire dynamics across varied regions. Ultimately, such a rigorous evaluation process can help improve and adapt models, ensuring that decision support systems remain responsive to fire-related risks and environmental challenges.

It should also be noted that the above deep learning models can be computationally intensive and require resources. For example, models such as ConvLSTM are computationally intensive due to their combination of convolutional operations for spatial feature extraction and LSTMs for temporal dependencies. Their high memory, storage, and processing demands make real-time applications challenging, often requiring Graphics Processing Units (GPUs), Tensor Processing Units (TPUs), or cloud-based solutions. Running typical deep learning models requires hardware, typically GPUs (e.g., NVIDIA A100, RTX 4090) for efficient training and inference or TPUs (e.g., Google TPUv4) for TensorFlow workloads. CPUs (e.g., Intel Xeon) are suitable for lightweight tasks. Cloud solutions like AWS, Google Cloud, Google Colab and Azure provide scalable GPU/TPU instances. A minimum of 16GB RAM is needed, with 32GB + for larger models, and SSDs are preferred for fast data access. Large input data and sequential dependencies increase inference time, leading to latency issues in time-sensitive applications. Additionally, high energy consumption can limit deployment on devices. Strategies like model pruning, quantization, parallel processing, and hybrid approaches can be employed to optimize real-time performance. Leveraging cloud computing can enhance efficiency, making these models more practical for real-time fire prediction and monitoring.

By customizing models to match each country’s fire patterns, authorities can better assess risks, use resources wisely, and take early action to reduce fire damage. The findings also show that these models must improve as fire patterns change. Future research should look at how factors such as land use, human activity, FRP, and location affect fires to make predictions more useful for prevention and response. Combining different modeling methods like deep learning with traditional approaches could make forecasts more accurate and reliable. Improving model efficiency would also help fire agencies get real-time updates, making fire prediction systems more practical for quick action. Using satellite data and new technology will be key to building more innovative and more adaptable fire management strategies.