AI can save lives, but it requires guardrails to manage risk, protect patient safety and ensure privacy.
In publishing a report on AI this week, AMA President Daniel McMullen said that the use of AI in healthcare must be clinically led, safe and central with the sole purpose of advancing the health and well-being of patients and the broader community.
AI has the potential to transform and study Australia's healthcare, but when implemented without robust protection measures, it poses new risks to patients and health professionals.
Dr. McMullen told the Medical Republic:
“However, AI can be much broader than that, and it's important that doctors and the public are aware of where AI can be added and what types of guardrails are needed to ensure there is safe and proper use of AI.”
New AI Guidance
Meanwhile, the Australian Committee on Safety and Health Care Quality has released clinician guidance and provides a complementary framework for AMA's policy advocacy on safe and responsible AI integration in Australian health care.
This guidance is consistent with our own position for the safe and responsible use of AI, particularly with regard to the principle that AI must replace clinical judgment, and the final decision must always be with a practitioner. This guidance strengthens this by highlighting the responsibility of clinicians to critically assess the alerts against automation bias and the output of AI.
Regarding data governance and privacy, he opposed the sale of patient data and emphasized the risk of re-identification. The committee's guidance instructs clinicians to comply with privacy laws and to confirm how data is stored and used, particularly for AI training and third-party sharing.
The committee's publications provide practical clinician-level guidance for actually assessing, using and monitoring AI tools. However, given the high-risk nature of Healthcare, we continue to advocate for structural reforms to ensure that AI tools meet the highest standards of safety, reliability and data integrity, such as a dedicated AI health advisory body integrated with AIPRA and TGA.
We also welcome the committee's recognition of “scope creep” in evolving AI tools. The AMA argues that it needs to determine whether the AI tools will be adjusted, not the origin. General purpose AI (GPAI) such as scribes in large language models must undergo the same scrutiny as dedicated clinical AI.
