Comparative analysis of reinforcement learning and artificial neural networks for inverter control in improving the performance of grid-connected photovoltaic systems

Comparison results under ramp changes in solar radiation

The specifications for the full microgrid are presented in Table 2, along with the features of the PV solar panel system, the component values of the boost converter, and essential output metrics, including grid voltage regulation, current, and power. The controllers can successfully regulate and optimize PV system performance in the face of shifting environmental elements and system demands by assessing them under these diverse settings.

In the system described in Fig. 1, the output PV solar panel power for both ANN and RL controllers is shown in Fig. 15. With its effective operation and minimal ripples, the RL controller consistently generates satisfactory results in terms of PV power generation. In contrast, the ANN controller performs poorly between 1.5 and 2.25 s, with a lower output and greater power ripples, despite overall performance being satisfactory. In Fig. 16, the PV current for both controllers are shown. The ANN controller exhibits noticeable ripples and an uneven current supply within the 1.5–2.25-second interval, demonstrating poor performance once again. The RL controller, however, provides reliable and respectable current performance. The PV voltage output for both controllers is displayed in its effective operation and minimal ripples, the RL controller consistently generates satisfactory results in terms of PV power generation. In contrast, the ANN controller performs poorly between 1.5 and 2.25 s, with lower output and greater power ripples, even if it works well overall. In Fig. 17, the PV voltage for both controllers are shown. The ANN controller exhibits noticeable ripples and an uneven current supply within the 1.5–2.25-second interval, demonstrating poor performance once again.

The DC-DC output current of the converter is illustrated in Fig. 18. The ANN controller performs worse than the RL controller when the time interval between 1.8 and 2.2 s is zoomed in on. However, the ANN controller exhibits undershooting, reduced current, and noticeable ripples. In contrast, the RL technique maintains a more constant current output. The voltage output of the boost converter of the DC-DC type with both controllers is illustrated in Fig. 19. All things considered, the ANN controller performs better than the RL approach, even if it produces outcomes that are generally acceptable. The ANN controller exhibits fewer ripples throughout the 1.5–2 s period, while the RL method routinely produces better outcomes. The injected current control for both controllers is displayed in Fig. 20. When compared to the ANN Controller, the RL controller exhibits superior alignment with the intended performance,

typically following the reference current line closely.

The active power output of the PV solar panel system to the grid is illustrated in Fig. 21. In comparison to the ANN controller the RL approach yields a greater active power value of 100 kW while the ANN controller only manages to produce 82 kW. The temperature and irradiation variations occur between 0.5 and 2 s, during which the RL controller operates more effectively and sustains a more consistent power output. On the other hand, the ANN controller performs worse from 1.5 to 2 s and exhibits a drop in power output between 1 and 1.5 s. The output voltage of the inverter to the three-phase power grid is illustrated in Fig. 22. When the grid maintains regulation, both controllers provide an output voltage of 20 kV.

Figures 23 and 24 show the grid-injected current for the ANN and RL controllers, respectively. Compared to the ANN, the RL controller performs better and exhibits fewer overshoots. While the ANN controller only produces four amps of output current, the RL controller produces six amps. More current is sent to the grid by the RL controller as a result. When focusing on the period between 2 and 2.08 s, the RL controller performs noticeably better than the ANN controller, exhibiting greater grid current management capabilities. In comparison to the ANN controller, the RL controller provides more consistent and adequate power and current values for the microgrid, which makes it a better option for inverter control in a PV system.

Table 3 presents a detailed comparison between the ANN and RL control methods in a grid-connected PV system subjected to ramp changes in solar irradiance. The table includes key metrics such as grid power output (P_Grid), PV power output (P_PV), PV voltage (V_PV), the corresponding irradiance level, and the control technique used.

The results clearly show that, in every operational scenario, the RL-based controller performs noticeably better than the ANN-based controller. The ANN controller achieves only 33.15 kW and 31.75 kW at high irradiance (1000 W/m²), whereas the RL controller obtains 109.66 kW of PV power and 102.25 kW of grid power, respectively, representing gains of more than 230% in PV power and 220% in grid power. Furthermore, at the same irradiance, RL delivers a PV voltage of 282.12 V as opposed to 239.22 V with ANN, maintaining superior voltage stability. Even at low irradiance levels (200 W/m²), RL exhibits more flexibility, generating outputs that are more stable and powerful. The learning capacity of RL, which permits dynamic policy updates and improved adaptability to environmental unpredictability, is responsible for this performance disparity. The ANN controller only reaches 33.15 kW and 31.75 kW at high irradiance (1000 W/m²), but the RL controller obtains 109.66 kW of PV power and 102.25 kW of grid power, respectively, which represents gains of more than 230% in PV power and 220% in grid power. Furthermore, at the same irradiance, RL delivers a PV voltage of 282.12 V as opposed to 239.22 V with ANN, maintaining superior voltage stability. Even at low irradiance levels (200 W/m²), RL exhibits more flexibility, generating outputs that are more stable and powerful. The learning capacity of RL, which permits dynamic policy updates and improved adaptability to environmental unpredictability, is responsible for this performance disparity.

Figures 25 and 26 illustrate the THD of the current under a ramp input condition using ANN and RL, respectively. In Fig. 25, the ANN-based controller exhibits a noticeable spike in THD at approximately 1.55 s, where the distortion briefly exceeds 5%. This peak indicates a moment of instability or high-frequency distortion, suggesting that while the ANN controller is adaptive, it may introduce transient nonlinearities or struggle with smooth dynamic transitions under certain conditions. Conversely, Fig. 26 demonstrates the performance of the RL-based controller, which maintains a significantly more stable response. The THD remains consistently below 1% throughout the 3-second interval, reflecting superior harmonic control and reduced current distortion.

Figure 27 shows the THD of the voltage under ramp solar radiation using the ANN controller. Its capacity to adjust to shifting environmental circumstances is confirmed by the RL controller’s THD for voltage under ramp conditions, which shows better voltage regulation and fewer distortions as shown in Fig. 28.

Comparison results with random variations in solar radiation

A model incorporating random fluctuations in solar radiation and a mean value of 500 W/m² is displayed in Fig. 29. In this model, the outside temperature is kept constant. Rainy weather and cloud cover are two examples of the variables that might contribute to these variations in solar energy. Maintaining the efficacy of the suggested control system and guaranteeing dependable operation in a range of environmental circumstances depends on effectively managing these erratic variations in radiation.

In Fig. 30, two control systems—RL and ANN—are used to compare the maximum production of PV electricity under fast random changes in solar radiation. In terms of controlling current output during these variations, the RL control system performs noticeably better than the ANN method. Figure 30 A illustrates that the RL control system displays a greater current output than ANN between 0 and 0.5 s at a solar radiation intensity of 460 W/m². The ANN system delivers 155 amperes of current, while the RL system maintains 180 amperes while radiation drops below 250 W/m² over 0.5 to 1 s. The current of the RL system climbs to 245 amps while that of the ANN system stays at 210 amps when radiation levels reach 600 W/m² for 1.5 to 2 s.

Significant current instability is seen in ANN during this period. ANN obtains a maximum current of 200 amperes during the stabilization phase, whereas RL achieves a maximum current of 240 amperes, during the 2 to 2.5 s period. The RL current lowers to 160 amperes and the ANN dips to 118 amperes as radiation declines from 2.5 to 3 s to 310 W/m². Comparing the ANN system with the RL control system, Figure 30B shows that the RL control system offers more stability and maximum power output under random radiation variations. This is most noticeable when radiation maximizes from 250 W/m² to 600 W/m², which occurs between 1 and 2 s. When it comes to controlling PV power output amid these abrupt fluctuations in solar radiation, the RL system often beats the ANN system.

The voltage variations in PV cells under ANN and RL system management are shown in Fig. 30C. In contrast to the ANN system, the RL control system attains a more steady voltage despite some early oscillations. Both systems settle at 285 volts after 0.5 s, but the RL system keeps the voltage higher and more constant throughout that time. This suggests that, particularly in situations with rapidly fluctuating weather, the RL system consistently outperforms the ANN control system in raising the output voltage of the PV solar panel cells.

Figure.31. depicts the grid power output for both RL and ANN control systems under random solar radiation. The RL system not only produces higher power but also maintains greater stability, demonstrating superior efficiency in utilizing the PV power system compared to the ANN system.

Figure 32 illustrates the association between the three-phase current and the random fluctuations in solar irradiance with the RL controller, tracking the top power level for grid feeding. The present variation is depicted as it moves through the ANN control system in Fig. 33 and as it moves through the RL system in Fig. 32. The relationship between the three-phase currents in the PV solar panel system and fluctuations in solar radiation is seen in Figs. 32 and 33. Time is plotted on the x-axis, while the y-axis illustrates the values at present. Phase B currents are represented by the blue line, phase C currents by the red line, and phase A currents by the yellow and green lines.

The conversion of variations in solar energy into variations in current is seen in Fig. 32. It records the highest power point that may be delivered to the grid by showing the link between the three-phase current and the random variations in solar irradiation. Figures 32 and 33 show the current fluctuation as it passes through the ANN control system and the RL system, respectively. It is shown that the RL system responds faster and more accurately than the ANN control system.

The voltage stability for the RL and ANN control systems, respectively, is shown in Figs. 34(a) and 34(b). Stable voltage outputs are efficiently maintained by both systems. In contrast to the ANN system, the RL control system shows marginally smoother voltage regulation. This increased stability is important since stable voltage has a direct impact on the solar system’s power production and overall performance. These results demonstrate that although both control techniques work well, voltage stability is somewhat greater with the RL approach.

Table 4 shows the efficiency of the ANN and RL control techniques for a grid-connected PV system under various solar irradiance situations is contrasted in this table. According to the results, grid power (P_Grid) varies between 23.86 kW and 33.99 kW, while PV power (P_PV) for the ANN controller spans from − 6.15 kW to 41.78 kW. The RL controller, on the other hand, displays higher and more constant figures, with grid power falling between 40.45 kW and 56.43 kW and PV power between 49.29 kW and 57.93 kW. Furthermore, with RL, the PV voltage (V_PV) continuously displays larger values, up to 286.72 V, as opposed to the maximum of 274.03 V under ANN. These results show that RL performs more consistently and effectively, particularly when exposed to different amounts of irradiance.

Using the ANN controller, Fig. 35 shows the current THD under random solar radiation. The greater performance of RL in handling dynamic situations is demonstrated by the decreased harmonic distortions in the THD of the current under ramp conditions when utilizing the RL controller as opposed to the ANN controller as shown in Fig. 36.

Using the ANN controller, Fig. 37 displays the THD of the voltage under random solar radiation. The RL controller’s THD for voltage under ramp settings, which exhibits improved voltage regulation and less distortions as seen in Fig. 38, validates its ability to adapt to changing environmental conditions.