BCX says trust and accountability are critical to the success of AI applications

Applications of AI


Garith Peck, BCX Managing Executive: Cloud and Cyber ​​Security

Garith Peck, BCX Managing Executive: Cloud and Cyber ​​Security

In the digital age, organizations are keen to adopt and use AI to take advantage of the immediacy and pervasiveness of the technology. However, trusting AI with access to your systems and data without a comprehensive understanding of how it works is risky.

This is according to Garris Peck, Managing Executive of Cyber ​​Security and Cloud at BCX. He presented at the Chartered CIO Conference 2026, recently hosted by the Chartered CIO Council, in partnership with ITWeb and sponsored by BCX.

tackle the theme Digital trust and sustainable innovationPeck said organizations continue to deploy AI at scale, intentionally or unintentionally, often without considering the level of autonomy with which the technology operates.

There is no doubt about the power of AI, he said. Given AI’s ability to influence and change biases, beliefs and mindsets, the real question is how quickly organizations can build trust, he noted.

Peck likened the deployment of AI and the autonomy associated with it to the role of a commercial pilot.

“In every flight, a pilot uses an autopilot at some point. Automation is a huge part of modern aviation…The autopilot manages all the functions, avionic sensors, and a central server within the autopilot automation system. But the (human) pilot never leaves the cockpit. He or she is there to ensure trust and accountability. When it comes to AI, there are no “pilots in the cockpit” in the enterprise. ” he said.

Evolution of Cyber ​​Security

Discussing the role of AI in cybersecurity, Peck said the discipline has evolved beyond the days of relying solely on antivirus software, firewalls, and patches.

“Cybersecurity remains one of the most complex areas in IT,” says Peck. “Confidentiality and integrity are the cornerstones of cyber security, its ‘DNA.’ But AI is changing the way we take things from a cyber perspective. AI interprets, generates, and reasons, and we act autonomously.”

Peck said that with the advent of AI, the threat landscape is no longer just technical, but also cognitive and psychological.

“Machines influence how we perceive reality…AI’s biggest risk is manipulating human trust at scale. The cyber battlefield is not about machines attacking machines; after all, it is propaganda used to attack individuals,” he added.

Peck also warned that the speed with which AI generates information makes it difficult for humans to verify reality quickly enough.

“80% of people who use technology are not tech savvy.”

He added that advances in deepfake audio and video technology are making it increasingly difficult to verify authenticity.

“The future crisis will not be a data breach, but a breakdown in trust in the truth.”

Peck said organizations are increasing their reliance on AI, even though AI and cybersecurity are not yet fully mature.

“The most dangerous thing about AI is that AI failures can appear normal, so it can appear that the workflow is still complete and the answer is the actual result,” he added. “What happens if AI makes a mistake? The public will not blame the algorithm, but the organization and its leaders.”

According to BCX, although South Africa has made progress in its digital transformation efforts, including the digitization of some social services and advances in healthcare, education and telecommunications, it continues to face several challenges. These include digital literacy, infrastructure constraints, and a limited supply of skills.

Peck also highlighted the issue of shadow AI, explaining that it is a growing concern for cybersecurity professionals, especially with the rise of agent AI.

“This means trust is more important than ever in emerging markets,” Peck said. “The technology that defines our time is trust, not intelligence. Digital mistrust is economic inequality…Humans can and will adapt, but AI influences judgment and shapes perceptions, confidence, and beliefs.”

Peck said the role of the CIO is changing, and in the AI ​​era, the CIO will serve as the trust architect and steward of machine responsibilities.

“Technology leaders must ensure that progress does not outweigh trust…They must have a ‘pilot in the cockpit’ with appropriate processes and procedures. Technology must be safe by design.”



Source link