ML/GA-based performance optimization of PBG-enhanced THz microstrip patch antennas on PTFE–SWCNT

Polytetrafluoroethylene (PTFE)

The proposed antenna is printed on a (PTFE) substrate of permittivity\(\:{({\upepsilon\:}}_{\text{r}}=2.1)\). The potential of using PTFE material resides in its very low tangent loss \(\:\text{t}\text{a}\text{n}{\updelta\:}=0.0002\:\)that minimizes energy dissipation and improves the radiation efficiency. Further, PTFE exhibits the lowest and stable relative permittivity over other polymers. It presents a chemical and thermal stability having an outstanding resistance to corrosive substances and a high melting temperature (over 327°c). Moreover, it proves its high flexibility having a low tensile modulus that is between 50 and 90 Kpsi³⁵. The effectiveness of this material as antenna substrate is well investigated in a previous work⁷.

Single wall carbon nanotubes (SWCNT)

The utilized conductive material is SWCNT for both radiating element and ground plane layers having a conductivity of \(\:3{e}^{6}\) S/m. SWCNT materials have become the suitable choice for their flexibility, biocompatibility, cost effectiveness and performance stability thanks to their good thermal conductivity (up to 3500 W/K·m), high tensile strength (around 400 GPa) and a density of 1.3 g/cm³^36,37.

Antenna design

The antenna design consists of a conventional rectangular antenna operating at 0.3 THz. The basic dimensions are determined by Eqs. (1)-(5).

Where c is the speed of light, f is the resonant frequency and \(\:{{\upepsilon\:}}_{\text{e}\text{f}\text{f}}\) is the effective relative permittivity.

A microstrip line of width 56.94 μm and a length of 100 μm feed the antenna. The antenna’s basic return loss in the range 0.1–0.5 THz is depicted in Fig. 1 based on numerical simulations under CST MWS, which uses the Finite Integration Technique (FIT) to solve Maxwell’s equations for electromagnetic fields. However, the impedance matching is not optimal at the desired frequency. To improve matching, notches have been introduced in the antenna design as a matching technique, where the feed line is shifted inward from the edge of the antenna by a distance LE, as illustrated in Fig. 2 that presents the final antenna geometry. Changing the notches’ length (LE) results in the control of the antenna input resistance, leading to an enhanced impedance matching³⁸. Following a parametric study on this parameter, the final antenna dimensions are summarized in Table 1.

Return loss and bandwidth analysis

The return loss of the final antenna is plotted in Fig. 3 as a function of frequency. The antenna exhibits a resonance at 0.3 THz, where the return loss drops well below − 10 dB, reaching a value of −45.61 dB at the desired frequency. This value confirms efficient power transfer and minimal reflected power, with a bandwidth of 15.8 GHz.

Realized gain and Far field pattern of the proposed RLA antenna

Figure 4 presents the simulated far-field radiation patterns (E-plane and H-plane) and the 3D realized gain of the antenna at 0.3 THz. The radiation pattern is directional, perpendicular to the antenna plane, with a realized gain of approximately 5.14 dBi and an efficiency of 64%.

The frequency-dependent behavior of the antenna’s gain is illustrated in Fig. 5. This figure highlights the antenna’s peak gain at the resonant frequency and shows how the gain varies across the operating bandwidth, providing insight into the antenna’s efficiency and radiation performance over the frequency range of interest.

Integration of the photonic band gap (PBG) structure

The PBG structure is created by introducing periodic cubic air gaps arranged in a square lattice within the substrate, as illustrated in Fig. 6. The dimensions of the PBG cuboid gaps are determined using the equations provided in¹⁷.

A parametric study was conducted to examine the impact of air gap side length Y, lattice constant D, and substrate thickness h on antenna bandwidth, gain, and efficiency. The results serve as a dataset for training AI models to learn the underlying physical relationships and predict or optimize future antenna designs. These studies are summarized in Tables 1, 2, 3, 4, 5, 6, 7, 8, 9, 10 and 11 in Annex A.

Increasing the gap side length improves both bandwidth and gain; for Y = 60 μm, the antenna achieves a bandwidth of 37.6 GHz and a gain of 7 dBi. This enhancement is attributed to the reduced effective permittivity associated with larger air gaps. Figure 7 illustrates the effect of Y on antenna performance, with D = 80 μm and h = 85 μm.

The effect of the lattice constant is evident for values of 70 and 80 μm, yielding peak bandwidth, gain, and efficiency of 37.6 GHz, 7.2 dBi, and 85%, respectively. These values are optimal compared to 50–60 μm or 90–100 μm, which show improved but limited performance. Figure 8 illustrates the effect of varying the PBG lattice constant D on antenna performance, with y and h fixed at 60 μm and 85 μm, respectively. As D increases from 80 μm to 100 μm, the resonant frequency shifts slightly, and the return loss magnitude changes, indicating that D has a noticeable impact on antenna performance.

Regarding substrate thickness, the study highlights its direct impact on antenna performance. Generally, increasing the substrate thickness improves performance up to a certain point, after which bandwidth and gain begin to diminish, with peak values typically observed between 75 and 115 μm. However, it inversely affects return loss, as the minimum impedance matching value increases with thicker substrates, reaching a maximum at 125 μm. Figure 9 illustrates the effect of varying h on antenna performance.

From the tables presented in Annex A, the optimal configuration is Y = 60 μm, D = 80 μm, and h = 85 μm, yielding a bandwidth of 37.6 GHz, a gain of 7.04 dBi, and an efficiency of 84%.

AI-Based modeling and optimization

This section presents the artificial intelligence (AI) techniques employed for the modeling and optimization of the proposed terahertz (THz) antenna. The objective was to predict antenna performance based on its design parameters and to identify optimal configurations maximizing performance metrics. The methodology comprises two principal stages: (1) supervised machine learning for performance prediction and (2) metaheuristic optimization using Genetic Algorithm (GA) guided by learned models.

Dataset preparation

A parametric study was conducted using CST Microwave Studio, where three geometrical design variables of the photonic bandgap (PBG) antenna were varied systematically:

Each combination of these parameters generated a unique antenna configuration. For each configuration, the following performance metrics were extracted from the CST simulation results:

• Bandwidth (GHz) : Calculated using the − 10 dB return-loss bandwidth around the primary resonance frequency.

• Gain (dBi) : Realized gain measured at the main resonance frequency of 0.3 THz.

• Efficiency (%) : Percentage of input power successfully radiated, measured at 0.3 THz.

• Return Loss (dB) : Minimum return loss at resonance, indicative of impedance matching.

A total of N = 256 simulations were performed, covering the design space via a full factorial or Latin Hypercube Sampling method (specify which, if applicable). The input parameter bounds were:

• Y ∈ [50, 70] µm.

• D ∈ [70, 90] µm.

• H ∈ [75, 95] µm.

Each simulation entry in the dataset is a vector:

Where:

\(\:{x}_{i}:\:\)are the input parameters.

\(\:{y}_{i}:\:\)are the CST derived outputs.

All data were normalized between 0 and 1 during training for numerical stability, but predictions were de-normalized for performance evaluation.

Supervised learning models

The goal of the supervised learning models was to accurately learn the nonlinear mapping:

Feature and label construction.

• Inputs (X): 3 continuous variables [Y, D, h].

• Outputs (Y): 4 continuous targets [BW, G, Eff, RL].

Model types and configuration.

In this work, the input parameters of the dataset are the PBG geometrical variables: air-gap side (Y), lattice constant (D), and substrate thickness (h). The output performance metrics predicted by the models are bandwidth (GHz), realized gain (dBi), radiation efficiency (%), and return loss (dB). All models were trained to learn the mapping from {Y, D, h} → {bandwidth, gain, efficiency, return loss} using data generated from CST simulations. Model performance was evaluated using the coefficient of determination (R²) between predicted and CST values, which provided a reliable measure of prediction accuracy for this study.

Four supervised machine learning algorithms were applied to learn the mapping between the antenna’s design variables and its performance metrics:

• Linear Regression (LR): A basic model to capture linear relationships between inputs and outputs.

• Assumes linear relationship: \(\:y={w}^{T}x+b\)

• No regularization; used as a baseline model.

• K-Nearest Neighbors (KNN): A non-parametric model relying on the proximity of data points for prediction.

• k = 5 (tuned via cross-validation).

• Distance metric: Euclidean.

• Weights: Inverse distance weighting.

• Decision Tree (DT): A rule-based model capable of capturing nonlinear dependencies through hierarchical data splitting.

• Max depth: huit (8) (optimized to prevent overfitting).

• Split criterion: Mean Squared Error (MSE).

• Neural Network (NN): A feed forward multi-layer perception was implemented to model complex nonlinear interactions in the dataset.

• Architecture:

Input Layer: 3 neurons (for Y, D, h).

Hidden Layers: [64, 32] neurons.

Activation: ReLU.

Output Layer: 4 neurons (for BW, G, Eff, RL), linear activation.

• Optimizer: Adam (learning rate = 0.001).

• Batch size: 16, Epochs: 300.

• Framework: Keras with TensorFlow backend.

Each model was trained and validated using the complete dataset, and performance was evaluated using the coefficient of determination (R²) between predicted and actual values in order to measure models accuracy. This coefficient is determined by Eq. (8)³⁹. Among the models, the neural network showed superior generalization ability, consistently achieving R² values above 0.94 for all output variables.

Where \(\:{\text{X}}_{\text{i}}\:\)is the actuali^th output (from CST), \(\:{\widehat{\text{X}}}_{\text{i}}\)is the i^th predicted output and \(\:\stackrel{-}{\text{X}}\)presents the mean of actual or true values.

Genetic algorithm for design optimization

Following the training of the supervised learning models, a Genetic Algorithm (GA) was implemented to optimize the antenna design variables with respect to performance metrics. The goal was to find the optimal combination of design inputs that simultaneously maximize gain and bandwidth while minimizing return loss.

Inputs and outputs.

• Design variables (search space): air gap Y ∈ [50–70] µm, lattice constant D ∈ [70–90] µm, and substrate thickness h ∈ [75–95] µm.

• Performance metrics (outputs): gain (G), bandwidth (BW), and return loss (RL).

Cost function (Fitness function).

The fitness function f combines the output metrics predicted by the Neural Network model to guide the optimization. It is designed to maximize gain and bandwidth while minimizing return loss, commonly expressed as:

Normalization example for a metric x:

where w₁, w₂, w₃ are weighting coefficients reflecting the relative importance of each metric, and the normalized terms scale each output between 0 and 1 based on observed minimum and maximum values.

Mathematical framework of the GA.

Initialization.

Generate an initial population P₀={x₁,x₂,…,x_N} where each individual x_i is a vector representing (Y, D,h) within their respective bounds. Each parameter is initialized randomly within predefined bounds:

Fitness evaluation.

For each individual x_i, use the trained Neural Network to predict performance metrics:

• Gain G(xi).

• Bandwidth BW(xi).

• Return Loss RL(xi).

Compute function:

Where w₁, w₂, and w₃ are weights (w₁ = 0.4, w₂ = 0.4, w₃ = 0.2) reflecting metric importance.

Selection.

Select parent individuals probabilistically based on their fitness scores. Roulette Wheel Selection where the selection probability Pi of individual i is:

The selection leads to a mating pool of size N with fitter individuals more likely to be chosen.

Elitism

Top 5% of individuals preserved without modification each generation.

Crossover.

Generate offspring by recombining pairs of parents from the mating pool. Use multi-point crossover: randomly choose crossover points and exchange segments of parameter vectors.

Example

with two parents: \(\:Parent1=(Ya,\:Da,\:ha)\);\(\:Parent2=(Yb,\:Db,\:hb)\)

After crossover, the value is 0.75:

Mutation.

Introduce random perturbations to offspring parameters to preserve diversity. Probability of mutation might be around 0.18 (18%).Example mutation for parameter Y:

Where is\(\:\:\aleph\:\left(0,{\sigma\:}^{2}\right)\) Gaussian noise with zero mean and variance \(\:{\sigma\:}^{2}\).

Replacement.

Form the new population P_t+1from offspring and optionally elite individuals from current population. Elitism keeps the best-performing individuals unchanged to retain quality.

The population size remains constant\(\:\left|{P}_{t+1}\right|=N\).

Termination.

Repeat fitness evaluation, selection, crossover, mutation, and replacement until:

• Maximum generations T reached (T = 1800).

• Fitness improvement between generations is below a threshold (less than 0.001), or a satisfactory fitness level is attained.

• OR fitness exceeds predefined target (f ≥ 0.95).

This mathematical framework enables the GA to explore and exploit the search space defined by (Y, D, h) efficiently, balancing exploration via mutation/crossover and exploitation via selection based on fitness.

Outcome.

The GA converged to an optimal design configuration at Y = 60 μm, D = 80 μm, and h = 85 μm, which produced performance predictions consistent with the CST simulation results, validating the effectiveness of the optimization process.

Implementation tools.

• Genetic Algorithm implemented using DEAP (Distributed Evolutionary Algorithms).

• Neural Network model used as surrogate within GA evaluation loop.

• Execution environment: Intel i7 CPU, 32 GB RAM.

Optimization result

The Genetic Algorithm (GA), implemented in MATLAB R2023a using the built-in Global Optimization Toolbox, successfully converged after approximately 1500 generations. The convergence was evaluated based on the stabilization of the fitness function and the marginal improvement threshold (Δf < 0.001) over successive generations.

The optimal antenna design parameters obtained were:

• Air gap side (Y) = 60 μm.

• Lattice constant (D) = 80 μm.

• Substrate thickness (h) = 85 μm.

The trained Neural Network (developed using the Deep Learning Toolbox) was used to predict the performance metrics corresponding to these parameters. The predicted values were:

To confirm these predictions, the optimal configuration was re-simulated in CST Microwave Studio. The CST results showed excellent agreement with the Neural Network predictions, with deviations below 5% across all metrics. This validates the efficiency and reliability of the AI-guided design optimization process using MATLAB.

Reproducibility considerations

• To ensure the transparency, repeatability, and reproducibility of this work within the MATLAB environment, the following procedure and resource was adopted:

• The dataset was prepared and processed using MATLAB tables, containing:

  • Input features: [Y, D, h]

  • Output metrics: \(\:\left[\text{B}\text{a}\text{n}\text{d}\text{w}\text{i}\text{d}\text{t}\text{h},\:\text{G}\text{a}\text{i}\text{n},\:\text{E}\text{f}\text{f}\text{i}\text{c}\text{i}\text{e}\text{n}\text{c}\text{y},\:\text{R}\text{e}\text{t}\text{u}\text{r}\text{n}\:\text{L}\text{o}\text{s}\text{s}\right]\)

• Normalization was performed using the normalize() function with the ‘range’ method, scaling features to [0,1].

• Implemented using fitnet()-i from the Deep Learning Toolbox.

• Network architecture:

  • Hidden layers: 2 layers with 64 and 32 neurons.

  • Activation function: ‘tansig’ for hidden layers, ‘purelin’ for output.

• Training function: ‘trainlm’ (Levenberg–Marquardt).

• Training parameters:

• Model evaluation: Coefficient of determination (R²) computed using custom MATLAB scripts.

Genetic algorithm configuration

• Implemented using MATLAB’s ga() function.

• Fitness function implemented as a separate function file (fitnessFunction.m), incorporating the trained neural network to evaluate:

lb = [50, 70, 75]; % lower bounds for [Y, D, h].

ub = [70, 90, 95]; % upper bounds for [Y, D, h].

To ensure reproducibility of stochastic processes (e.g., GA, NN weight initialization):

All essential files have been organized in the project directory:

1. 1.

   dataset.mat : Original and normalized datasets.

2. 2.

   trainNN.m : Script to train the Neural Network.

3. 3.

   fitnessFunction.m : GA fitness function linked to trained NN.

4. 4.

   runGA.m : Main script to execute the Genetic Algorithm.

5. 5.

   evaluateResults.m : Script to compare predictions vs. CST.

Sensitivity analysis (using MATLAB’s designSensitivity functions or manual perturbation).