Machine learning based optimization of fly ash content for improving geopolymer concrete compressive strength

The sensitivity of input features influencing the CS of FA-GC was evaluated using SHAP values, providing a quantitative and interpretable measure of each feature’s impact on model predictions. Figure 5 illustrates this analysis: Panel (a) shows individual SHAP values for each feature, while Panel (b) presents the ranking of feature importance based on cumulative SHAP contributions.

The analysis indicates that Al₂O₃ (%), SiO₂ (%), and Coarse aggregate (kg/m³) have the strongest positive influence on CS, confirming their dominant role in geopolymer chemistry and mix design. Fine aggregate (kg/m³) and Duration (hr) have relatively minor effects, while Temperature (°C) and Water (kg/m³) exhibit moderate impacts, with temperature positively and water negatively affecting CS.

These results highlight the key features to prioritize in FA-GC mix optimization. Increasing alumina and silica content, optimizing coarse aggregate proportion, carefully controlling curing temperature, and limiting excess water can enhance CS, yielding more robust and durable geopolymeric concrete.

Figure 6 presents a scatter plot matrix showing the relationships between various features and the CS of FA-GC. In this plot, each point represents a sample, comparing different features with CS. By examining this plot, clear patterns of correlation or lack thereof between features and CS can be identified.

Based on the results observed in Fig. 6, properties such as FA (kg/m³), SiO₂ (%), and Coarse aggregate (kg/m³) show a good positive correlation with CS. This implies that as these properties increase, the predicted values for CS also increase. These properties definitely follow a trend in the scatter plot with densely packed points, which show strong correlation with CS. On the other hand, features like Fine aggregate (kg/m³) and Duration (hr) appear to have weaker correlations with CS. The points for these features are more dispersed, indicating a less consistent relationship with CS. These observations align with the SHAP analysis, where Fine aggregate (kg/m³) and Duration (hr) were found to have a smaller impact on CS prediction. This scatter plot matrix, in conjunction with SHAP analysis, provides valuable insights into the key features influencing CS prediction and offers guidance for optimizing the mix design of FA-GC for improved performance.

Based on the PDP results shown in Fig. 7, each feature exhibits a distinct influence on the compressive strength (CS) of FA-GC. The main observations are as follows:

• FA (kg/m³): CS increases with FA content up to an optimal level and then plateaus, indicating the importance of maximizing FA proportion carefully.

• SiO₂ (%) and Al₂O₃ (%): Both have consistently positive effects on CS, highlighting their critical role in strengthening geopolymeric bonds.

• Coarse aggregate (kg/m³): Increases CS up to a certain limit, suggesting the need for optimized content.

• Fine aggregate (kg/m³): Exhibits a minor influence compared to coarse aggregate.

• NaOH (kg/m³) and Na₂SiO₃ (kg/m³): Both positively affect CS, with NaOH having a stronger impact due to its activator role.

• Water (kg/m³): Negatively affects CS, as higher water content reduces material strength.

• Temperature (°C): Higher curing temperatures enhance CS through improved geopolymerization.

• Duration (hr): Minimal effect on CS, indicating curing time is less critical in this context.

Overall, SiO₂, Al₂O₃, and coarse aggregate have the greatest influence and should be prioritized in mix optimization, while water and NaOH also play significant roles. Fine aggregate and curing duration have weaker effects. Optimizing these key factors can substantially improve FA-GC performance.

To gain deeper insights into the nonlinear influence of each variable on compressive strength and to identify potential threshold effects, one-dimensional Accumulated Local Effects (ALE) analyses were performed for all features used in the machine learning models. Figure 8 illustrate the isolated contribution of individual input parameters while accounting for feature interdependencies.

The ALE curves demonstrate clear nonlinear behaviors and highlight several threshold regions. The AA/FA ratio exhibits two critical zones: an increase in compressive strength begins around 0.47–0.52, while a noticeable negative effect appears beyond 0.58. The content of Al₂O₃ shows a transition from a negative to a positive influence above approximately 16%, indicating that sufficient alumina availability enhances geopolymerization. Similarly, SiO₂ content displays a sharp threshold near 65%, beyond which the effect on strength becomes strongly negative, suggesting silica oversaturation. Aggregates play distinct roles: fine aggregate exhibits an optimal range of 400–505 kg/m³, with strength reduction above 520 kg/m³, while coarse aggregate negatively affects strength when exceeding 1185–1210 kg/m³, implying that excessive aggregate reduces binder continuity. FA shows a significant positive effect above 410 kg/m³, indicating a critical minimum required for optimal geopolymerization. Activator components show pronounced threshold behavior. The Na₂SiO₃ dosage enhances strength above 118–122 kg/m³, while the Na₂SiO₃/NaOH ratio declines sharply beyond ≈ 2.0, suggesting that excessive silicate relative to hydroxide can destabilize the reaction. Both NaOH concentration and alkali molarity (NaOH) present strong nonlinear effects: compressive strength rises significantly when NaOH exceeds ≈ 60 kg/m³ and molarity surpasses ≈ 13–14 M, then plateaus. The PCE content shows a distinct negative shift beyond ≈ 4, indicating overdosing of superplasticizer may hinder geopolymer formation. The water content demonstrates a clear drop in strength when exceeding ≈ 125 kg/m³, underscoring the sensitivity of geopolymerization to water-to-binder ratios. In contrast, curing temperature and curing duration show negligible ALE effects, implying that within the tested range they exert minimal influence compared to chemical composition and mix proportions. These findings confirm that the machine learning models capture complex, nonlinear dependencies and critical thresholds across multiple input variables. Identifying these ranges provides valuable guidance for optimizing mixture design to maximize compressive strength while maintaining material and chemical efficiency.

The hyperparameters of the models, tuned by Optuna, are presented in Table 2. These values were essential in achieving the best performance for each model in predicting the CS of FA-GC during the evaluation.

Next, the performance of the models AutoInt, M5Prime, HistGBoost, and TabPFN in predicting the CS of FA-GC is examined, and their results are compared. The results obtained from the evaluations help identify the best surrogate model for use in optimizing the mix design of FA-GC. During the training phase, the performance of the models was evaluated based on various metrics. Table 3 presents the evaluation results of the models using different metrics such as R², RMSE, sMAPE, WMAPE, and PBIAS. These results are provided to compare the models’ performance in predicting the CS of FA-GC.

From the findings outlined in Table 3, a detailed analysis of the performance of the models in forecasting the CS of FA-GC during the training period reveals striking differences in most of the evaluation measures employed, including R², RMSE, sMAPE, WMAPE, and PBIAS. R² measures indicate that TabPFN possesses the largest explanatory power with a value of 0.981 and accounts for 98.1% of the variation in the CS of FA-GC. This shows that TabPFN has the best fit among the models examined. HistGBoost then has an R² of 0.948, demonstrating high predictive validity, although lower than TabPFN’s. AutoInt and M5Prime have lower R² values of 0.913 and 0.901, respectively, which means these models account for less of the variance in the target variable. In RMSE terms, the mean degree of error, TabPFN leads the rest with the least of 1.495, which indicates that its values are closest to true values. HistGBoost has a greater RMSE of 2.439, which indicates greater deviation from true values. AutoInt and M5Prime have greater RMSE of 3.159 and 3.377, respectively, indicating greater discrepancy between their values and observed values. The sMAPE metric, reflecting the accuracy of percentage errors, supports the findings of TabPFN’s superior performance, with the lowest sMAPE of 4.614%. This indicates minimal percentage error in its predictions, making it the most reliable model. HistGBoost follows with a sMAPE of 7.786%, while AutoInt and M5Prime show even higher errors of 9.815% and 10.176%, respectively, pointing to a lower degree of precision. Similarly, in terms of WMAPE, which accounts for the weighted accuracy of predictions, TabPFN leads with a value of 4.493%, further corroborating its overall effectiveness. HistGBoost has a WMAPE of 6.714%, while AutoInt and M5Prime present higher values of 9.161% and 9.982%, respectively, indicating greater weighted errors. Finally, the PBIAS values reveal subtle biases in the predictions. TabPFN exhibits a small positive bias of 0.455%, suggesting a tendency to slightly overestimate the CS. HistGBoost shows no bias with a PBIAS of 0, while AutoInt and M5Prime display negative PBIAS values of −0.269 and 0.043%, respectively, indicating a slight underestimation in their predictions.

Figure 9 provides a comprehensive visual assessment of the predictive performance of the four ML models—TabPFN, HistGBoost, AutoInt, and M5Prime—during the training phase, using three diagnostic plots: (1) the scatter plot of measured versus predicted CS values, (2) the residual plot showing prediction errors across samples, and (3) the KDE plot for comparing the distributions of predicted and actual values.

Based on the diagnostic plots in Fig. 9, the TabPFN model demonstrates the most accurate and stable predictive performance among the evaluated models. In the scatter plot, its predicted values are very close to the regression line, with nearly all observations falling well within the ± 10% error band. Such accuracy supports the earlier statistical outcomes—namely the model’s greater R² and RMSE values—by confirming strong correlation between predicted and actual CS values.

The residual plot for TabPFN further highlights its robustness, displaying a tight and uniform distribution of errors centered around zero. This pattern reflects minimal bias and indicates homoscedastic behavior, confirming the model’s stability across the entire prediction range. Additionally, the KDE plot reveals an almost perfect overlap between the measured and predicted distributions, suggesting that TabPFN not only achieves pointwise accuracy but also faithfully captures the statistical structure of the data. The HistGBoost model also shows competitive performance, albeit slightly inferior to TabPFN. Its scatter plot reflects a strong linear correlation; however, a broader spread of data points and a few deviations beyond the ± 10% margin suggest a marginal decline in precision. While the residual plot indicates low overall bias, it exhibits larger fluctuations, pointing to less consistent predictive behavior. In the KDE plot, the predicted and measured distributions are closely aligned, though slight discrepancies in the upper tail imply limited generalization under extreme conditions.

In contrast, the AutoInt and M5Prime models exhibit relatively weaker performance. Both models show greater dispersion in their scatter plots, with numerous data points falling outside the ± 10% threshold. Their residual plots present higher variability and non-uniform error patterns, indicative of unstable and less reliable predictions. Also, their KDE plots indicate apparent divergence between the predicted and actual distributions, pointing to their poor capacity to capture the actual data distribution. Generally, the visual diagnostics in Fig. 9 reflect the statistical measures in Table 3. The TabPFN model clearly outperforms its competitors, showing higher accuracy, reliability, and distributional accuracy at training. In testing, the models’ performance was again measured using various metrics.

Table 4 presents the evaluation results of the models based on criteria such as R², RMSE, sMAPE, WMAPE, and PBIAS during the testing phase. These results are provided to further compare the models’ performance in predicting the CS of FA-GC on new data.

The performance of the models in the testing stage, as shown by the results in Table 4, reflects considerable variations in some of the most critical indicators like R², RMSE, sMAPE, WMAPE, and PBIAS. All these indicators form a holistic evaluation of the ability of the models to estimate the CS of FA-GC with accuracy and reliability. In R², or the model’s proportion of variance in CS of FA-GC explained, the optimum result is achieved by TabPFN with an R² of 0.916. This means TabPFN explains 91.6% of the variance in the data, indicating a good fit and high prediction precision. HistGBoost, at 0.894 R², follows closely behind, with good predictive power but less explanatory power than TabPFN. AutoInt, at 0.892 R², and M5Prime, with the lowest R² of 0.867, indicate that these models are weaker at understanding underlying variance in the CS of FA-GC, with the poorest fit overall from M5Prime. The RMSE values also support the improved performance of TabPFN, which achieves the lowest RMSE value of 4.02. This implies the model’s predictions are closest to actual values with minimum deviation. HistGBoost (RMSE = 4.522) and AutoInt (RMSE = 4.567) achieve slightly higher errors, with the highest deviation between their predicted and actual values. M5Prime has the highest RMSE of 5.062, which means a higher error in its predictions compared to the other models. On an sMAPE scale, recording percentage prediction error, TabPFN once more has the best performance with an sMAPE of 8.201%. This translates to the lowest percentage error for the model, which is an affirmation of the model’s high accuracy level. M5Prime has a later sMAPE of 11.222%, while AutoInt and HistGBoost lead to higher sMAPE measures of 12.994% and 13.622%, respectively, and illustrate that the models are worse with respect to percentage error. The WMAPE metric, which accounts for the weighted accuracy of predictions, also favors TabPFN with the lowest WMAPE of 9.331%. This demonstrates the model’s strong performance in terms of weighted error. HistGBoost (WMAPE = 12.73%) and AutoInt (WMAPE = 12.909%) show higher weighted errors, while M5Prime (WMAPE = 12.611%) exhibits similar levels of weighted inaccuracy. Finally, the PBIAS values, which indicate the degree of bias in predictions, reveal that TabPFN exhibits the least negative bias with a value of −2.33%, suggesting a slight tendency to underestimate the CS of FA-GC. HistGBoost follows with a PBIAS of −3.195%, while AutoInt and M5Prime show more significant negative biases of −3.986% and − 3.071%, respectively, indicating a greater tendency to underestimate the values.

Figure 10 illustrates the testing-phase prediction performance of four ML models—AutoInt, HistGBoost, M5Prime, and TabPFN—through scatter plots, residual error plots, and KDE distributions comparing predicted and actual CS values.

Based on Fig. 10, the TabPFN model once again exhibits the most reliable generalization performance among all evaluated models. On the scatter plot, the predicted values fall very near along the regression line with nearly all points in the ± 10% error range. This visual reliability corroborates the model’s strong statistical performance in the above—most importantly, its high R² (0.916) and low RMSE (4.02)—which indicates high predictive accuracy on novel data. The residual plot also verifies the finding, with a tight and symmetrical scatter of errors around the zero line, which verifies low bias and high stability across the range of predictions. Moreover, the KDE plot provides a virtually perfect overlap between predicted and measured distributions, ensuring that TabPFN is able to accurately capture the pointwise accuracy and also the underlying statistical pattern of the dataset. The HistGBoost model also delivers solid performance, albeit with slightly diminished precision relative to TabPFN. In its scatter plot, most predicted values fall close to the regression line and within the error band, but the overall dispersion is more pronounced. The residual plot indicates modest fluctuations with occasional spikes, suggesting areas of localized error. Although the KDE curves for measured and computed values remain largely similar, minor discrepancies—particularly in the tails—indicate slight limitations in distributional fidelity.

In contrast, the AutoInt and M5Prime models show weaker predictive capabilities in the testing phase. Both models exhibit greater spread in their scatter plots, with a larger number of predictions falling outside the acceptable error margin, suggesting reduced alignment with actual measurements. Their residual plots show more erratic and wider deviations from zero, reflecting less consistent prediction behavior and higher variance. The KDE plots further highlight these limitations, as visible divergences between the predicted and actual distributions—especially in extreme ranges—indicate insufficient ability to replicate the true data characteristics. In conclusion, the visual analyses in Fig. 10 align closely with the quantitative outcomes reported in Table 4, affirming that TabPFN is the most effective model in terms of both predictive accuracy and distributional robustness during the testing phase.

To evaluate and rank the performance of the applied artificial intelligence models in predicting the CS of FA-GC, the Taylor diagram was employed as a robust graphical analysis tool. This diagram simultaneously illustrates three essential metrics—correlation coefficient, standard deviation, and cRMSE—allowing for a comprehensive and visual comparison of model accuracy and consistency. The results for both the training and testing stages are presented in Fig. 11.

As indicated in Fig. 11, the TabPFN model also exhibited the highest accuracy with a cRMSE value of 1.50, revealing a high correspondence between its forecast and the actual observed values in the training interval. On the Taylor diagram, this model is positioned closest to the reference point of the measured data, indicating its optimal predictive performance. Following TabPFN, the HistGBoost model achieved a cRMSE of 2.45, performing reasonably well, although with a slightly greater deviation. The AutoInt and M5Prime models showed larger errors, with cRMSE values of 3.18 and 3.40, respectively, placing them behind in terms of training accuracy. During the testing phase, which evaluates the generalization capacity of the models for unseen data, the same tendency is observed. TabPFN was once more the top-performing model with cRMSE = 4.042, placed in a favorable position in the Taylor diagram and proving its stability for datasets. AutoInt and HistGBoost models showed nearly similar results (cRMSE = 4.516 and 4.518, respectively), with moderate accuracy. In contrast, M5Prime produced the highest prediction error with cRMSE = 5.083, which indicates the worst and most unstable performance in testing. In conclusion, based on the Taylor diagram analysis and cRMSE values, the TabPFN model consistently exhibits the best performance across both training and testing phases.

In order to further examine the performance of the models and support the findings obtained from the Taylor diagram, the NDA metric was employed. This statistical approach provides a more detailed assessment of how closely the models’ predictions align with actual values. The NDA results for both the training and testing stages are presented in Fig. 12, offering additional validation of model accuracy and consistency.

In the training phase, the TabPFN model clearly demonstrates the narrowest and tallest QDDR curve, which is symmetrically centered around zero. This distribution indicates minimal variance and high concentration of residuals near zero, affirming the model’s exceptional accuracy and consistency during training. Compared to other models:

• HistGBoost and AutoInt exhibit moderately wide distributions, suggesting a higher spread of errors and lower reliability.

• M5Prime shows the widest and flattest distribution, indicating the largest deviations from actual values and lower predictive confidence.

This confirms that among all models, TabPFN provides the most stable and precise predictions during training, further supporting the results from the Taylor diagram.

In the testing stage, although the QDDR curves for all models are generally wider due to exposure to unseen data, TabPFN continues to maintain the most concentrated distribution around zero. This sustained narrowness underlines the model’s superior generalization capability and robustness in predicting new data points. Meanwhile:

• AutoInt and HistGBoost show comparable but wider distributions, reflecting greater variance and reduced reliability in generalization.

• M5Prime, once again, presents the broadest and flattest curve, confirming its comparatively weak performance under uncertainty.

The NDA results in Fig. 12 validate the findings from the Taylor diagrams by quantitatively demonstrating that the TabPFN model consistently outperforms the other models in both training and testing stages, offering the lowest discrepancy, highest stability, and most reliable predictions.

It is crucial that artificial intelligence models not only have high accuracy but also high reliability. In this regard, to assess the uncertainty of the evaluated models, two metrics, CI and R-Factor, have been used. The results of this evaluation are shown in Fig. 13.

Figure 13 shows that TabPFN is the most excellent model from the uncertainty point of view because it has the lowest CI value of 0.697 and the lowest R-Factor value of 2.730 among all the models under consideration. These results indicate that TabPFN is the most accurate and stable prediction in comparison to the other models. Following TabPFN, the HistGBoost model (CI = 0.704, R-Factor = 2.759) and M5Prime (CI = 0.708, R-Factor = 2.774) have very similar performance, both reducing uncertainty and enhancing the stability of prediction. On the other hand, the AutoInt model with CI = 0.716 and R-Factor = 2.807 shows the highest level of uncertainty and variation in predictions and is therefore less desirable in comparison to the other models. In general, TabPFN, with the highest performance—evinced through the lowest figures for CI and R-Factor—is the most desirable model with greater accuracy and stability of predictions compared to the other models.

To investigate the weaker performance of M5Prime and AutoInt, the following reasons can be noted:

AutoInt

Embedding layers and multi-head interaction blocks introduce many degrees of freedom relative to the sample size, leading to overparameterization and a persistent train–test gap. Performance is sensitive to optimization choices (learning rate, warm-up, regularization, initialization); even with early stopping and multiple seeds, variance remains high and rare regimes are underfit. Under collinearity, attention weights become unstable across near-equivalent inputs, amplifying spurious interactions and reducing identifiability. Predictive uncertainty is less well calibrated in sparsely covered regions of the design space, yielding wider dispersion and less reliable confidence assignments.

M5Prime

As a model tree with piecewise-linear leaves, capacity is limited for non-additive, high-order interactions, producing structured residuals the linear leaves cannot absorb. Greedy splitting under collinearity oscillates among near-equivalent predictors, increasing variance; enlarging the minimum leaf size improves stability but increases bias, sharpening the bias–variance trade-off. Local linearity yields weak extrapolation outside split ranges and creates discontinuities at split boundaries, degrading accuracy and stability when inputs shift slightly or measurement noise moves samples across thresholds.

To further assess the predictive capability and generalizability of the TabPFN model, an external validation was conducted using independent data. External validation serves as an effective approach to evaluate whether a model trained on a specific dataset can reliably predict unseen data from different sources, thereby reducing the risk of overfitting and confirming its practical applicability. For this purpose, additional data were extracted from Nath and Sarker⁵⁸, which are completely independent of the dataset used in the present study. These external data were used to test the TabPFN model’s performance beyond the original dataset, as shown in Table 5. The results of this evaluation are summarized in Fig. 14, where the model demonstrates strong predictive accuracy. On average, the TabPFN model predicted the independent data with an error of 10% across 9 samples, confirming its ability to generalize well to new, unseen data. This external validation complements the internal evaluation and further supports the robustness of the developed model.

As the TabPFN model is demonstrated to possess both high predictive accuracy and high reliability, it can be trusted for use as a surrogate model in optimization algorithms for concrete mix design. Its superior performance in both the areas of lesser uncertainty and stable prediction results renders it extremely well-suited for guiding the optimization process with high accuracy, resulting in more efficient and reliable mix design solutions.

Table 6 displays the experimental and predicted values of CS for FA-GC.

As previously mentioned, this study aimed to develop an optimal mix design for maximizing the CS of FA-GC using four optimization algorithms: HHO, GWO, LOA, and PBA. In this context, the TabPFN model was employed as a surrogate model to predict CS accurately, and the mix design optimization was conducted under two separate scenarios. In the first scenario, the objective was to determine the optimal proportions of two key chemical components—SiO₂ and Al₂O₃—as the primary constituents of FA, to maximize the CS. For this purpose, three existing experimental mix designs were selected. In these designs, 12 out of 14 input variables were fixed based on experimental data, while only SiO₂ and Al₂O₃ were treated as decision variables. The optimization algorithms were then applied to simulate the optimal combination of these two chemical parameters for achieving the highest possible CS. In the second scenario, the goal was to identify a fully optimized mix design by treating all input variables as decision variables.

For the implementation of the optimization algorithms, two main constraints were considered. The first constraint ensured that the decision variables were within an acceptable range. The second constraint required that the sum of the weights of the input parameters divided by their density equals 1. The constraints mentioned can be expressed in the form of Eqs. (7 and 8).

where ρ_i​ is the density of material i. Table 7 displays the optimized values of SiO₂ and Al₂O₃ for each optimization algorithm. Additionally, the experimental values are included in this table for comparison. The variable notation in the tables of the following text is based on Table 1.

Table 8 presents the optimization results for FA-GC mix designs in Scenario 2, displaying the values of key parameters for each algorithm (HHA, LOA, GWO, and PBA). Additionally, Fig. 15 compares the CS of FA-GC mixtures optimized by different algorithms across both Scenarios 1 and 2.

The results show that optimizing the FA-GC mix designs using different algorithms leads to better performance in increasing CS. In Scenario 2, the HHA algorithm achieves the highest CS (61.56 MPa), indicating its superiority over the LOA, GWO, and PBA algorithms. Compared to the experimental values, the Optimized Mix reaches a CS of 61.6 MPa, showing a significant improvement over the initial mixes. These results emphasize that the correct selection of optimization algorithms can have a significant impact on improving the performance of FA-GC mixes, with HHA demonstrating the greatest effectiveness among them.

In addition to predictive accuracy and data efficiency, interpretability is a key criterion in selecting surrogate models for engineering applications. Interpretability refers to the model’s ability to explain the relationships between input variables and its decision-making logic in a way that is understandable to humans. This property is particularly important in industrial contexts, where decision-makers often require not only accurate predictions but also a clear understanding of the factors influencing those predictions.

Following the optimized mix designs, this section critically compares the four optimizers in terms of computational footprint and real-world scalability, and offers practitioner-oriented guidance.

All four methods are population-based. Per-iteration cost scales as:

where C_eval​ is the surrogate call (TabPFN inference in our pipeline). Memory is O(N_pop×D) with D features. Fitness evaluations are embarrassingly parallel across individuals; communication overhead is small.

HHO

Adaptive exploration–exploitation with multiple update regimes. Strong performance on rugged, nonlinear landscapes; good best-found quality under equal budgets. Overhead is moderate due to regime switching; tuning sensitivity is modest. Suitable when solution quality is the priority.

GWO

Parameter-lean and easy to tune. Low setup cost and stable behavior as dimension grows. Convergence can be slightly slower, but wall-clock efficiency and scalability are favorable. Suitable under tight time or resource constraints and for industrial deployment.

LOA

Rapid information sharing yields fast early progress and a balanced exploration–exploitation profile. Similar per-iteration cost to other population methods. Risk of premature convergence under homogeneous imitation; diversity mechanisms (noise, restarts) mitigate this. Useful when quick propagation of good patterns is desired.

PBA

Dynamic population management with aggressive moves can improve evaluation efficiency late in the search and adapt to changing constraints. Slight managerial overhead. Appropriate for scenarios with nonstationary requirements or short evaluation budgets.

Under equal computational budgets in our experiments, HHO achieved the best objective values, GWO was close and stable, and LOA/PBA exhibited advantages in specific regimes consistent with their design trade-offs.

Practitioner guidance

• Quality-first (ample budget, complex landscape): prefer HHO.

• Simplicity and scalability (limited time/resources, easy deployment): prefer GWO.

• Fast early progress and strong information diffusion: consider LOA.

• Dynamic or evolving constraints, short budgets: consider PBA.

In this study, the TabPFN model was selected as the surrogate model due to its superior predictive performance and its ability to learn effectively from limited data. However, because TabPFN is based on a transformer architecture with millions of nonlinear parameters, it is inherently opaque and often regarded as a “black box.” To enhance interpretability, we employed the SHAP method as a post-hoc analysis tool to quantify the contribution of each feature to the model’s predictions.

The SHAP analysis results, reveal the relative importance of different input features in predicting the compressive strength of concrete. According to these results, Al₂O₃, SiO₂, and Coarse aggregate have the most significant impact on the model output. Furthermore, the direction of these effects is consistent with engineering knowledge: for example, increasing cement content strongly enhances compressive strength, whereas increasing the water-to-cement ratio reduces it. These findings demonstrate that, despite its complex structure, the model’s internal logic aligns with known physical and material science principles.

In contrast, simpler models such as M5Prime are inherently interpretable. M5Prime is based on decision tree structures, and its output is expressed as a set of human-readable rules and linear equations. The decision-making path is explicitly defined for each prediction, allowing users to easily trace and understand the reasoning behind the model’s output. Moreover, the simpler structure of M5Prime typically results in higher stability under small data perturbations, and the resulting decision rules can be directly analyzed without the need for additional interpretation tools. A comparison of key interpretability characteristics of TabPFN and M5Prime is presented in Table 9.

This comparison highlights that the choice of surrogate model ultimately depends on the objectives of the application. If predictive accuracy and data efficiency are the primary goals, TabPFN is a highly powerful candidate. However, if interpretability and explainability are of greater importance, simpler models like M5Prime may be more appropriate. The optimization framework proposed in this study is flexible and can accommodate both types of models—highly accurate or highly interpretable—without requiring modifications to the overall optimization process.

To summarize model performance and uncertainty, Table 10 reports R², RMSE, cRMSE, 95% CI width, and R‑Factor for the train and test phases.

Meta-learned prior and few-shot generalization

Pretraining supplies a data-driven prior that improves test R² and reduces error on small datasets.

Automatic high-order interactions

Attention captures nonlinear feature interactions without manual feature engineering.

Smooth response surface

The learned mapping yields stable gradients and lower cRMSE, which benefits the optimizer’s inner loop.

Better calibration and tighter uncertainty

Narrower 95% CI and lower R-Factor indicate more reliable predictions.

Robustness to collinearity and noise

Learned inductive biases preserve rank performance across metrics.

Fast inference

After pretraining, per-sample prediction is lightweight, which suits iterative optimization.