• Artificial intelligence (AI) seems to be everywhere these days, and healthcare is no exception.
• There is no doubt that in terms of patient health, workflows and system efficiency, AI will benefit the health system.
  
• But there are legitimate concerns about the accuracy of such tools
• For more stories, visit the Tech and Trends homepage. 

Artificial intelligence (AI) seems to be everywhere these days, and healthcare is no exception.

There are computer vision tools that can detect suspicious skin lesions as well as a specialist dermatologist can.

Other tools can predict coronary artery disease from scans. There are also data-driven robots that guide minimally-invasive surgery.

To precisely diagnose diseases and guide treatment choices, AI is used to analyse patients’ genomic and molecular data.

For instance, machine learning has been applied to detect Alzheimer’s disease and to help choose the best antidepressant medication for patients with major depression.

Deep learning methods have been used to model electronic health record data to predict health outcomes for patients and provide early estimates of treatment cost.

With new language-based generative AI technologies like ChatGPT, the clinical world is abuzz with talk of chatbots for answering patient questions, helping doctors take better notes, and even explaining a diagnosis to a concerned grandchild.

There is no doubt that in terms of patient health, workflows and system efficiency, AI will benefit the health system.

But there are legitimate concerns about the accuracy of such tools, including how well they work in new settings (such as a different country or even a different hospital from where they were created), and whether they “hallucinate” – or make things up.

Developing ‘medical grade’ tools

In our recent article in the Medical Journal of Australia, we argue using AI effectively in healthcare will require retraining of the workforce, retooling health services, and transforming workflows.

Critically, we also need to collect evidence AI tools are “medical grade” before we use them on patients.

Many claims made by the developers of medical AI may lack appropriate scientific rigour and evaluations of AI tools may suffer from a high risk of bias. This means the tests run to ensure their accuracy are too narrow.

READ MORE | Why does it seem that SA is lagging behind in global dialogue on AI governance?

AI tools can make errors, or stop working when the application context changes. Conversational agents such as chatbots may produce misleading medical information that may delay patients seeking care. They may also make inappropriate recommendations.

All this means we need standards for the AI tools that impact diagnosis and treatment of patients. Clinicians should be given training on how to critically assess AI applications to understand their readiness for routine care.

We should expect to be able to replicate the results from one context to another, under real-world conditions. For example, a tool developed using historical data from a hospital in New York should be carefully trialled with live patient data in Broome before we trust it.

Randomised controlled trials of AI tools, where these differences are controlled for, would represent a gold standard of evidence for their use.

We can’t just copy what other countries do

It is important to carefully examine how AI tools are embedded into workflows to support clinical decisions. The benefits and risks of a tool will depend on precisely how the human clinician and the tool work together.

We don’t want to just export our clinical datasets and import back the models built with them without adapting to our contexts and workflows. We need to monitor the clinical deployments of AI tools into our settings.

Without some degree of algorithmic sovereignty, a nation is exposed to new risks and the benefits of the technology will be limited.

AI has the potential to transform medicine. It can do so by harnessing computational power to discern subtle patterns in complex data spanning biology, images, sensory and experiential data, and more.

With care and strategic investment, innovations in AI will surely benefit clinicians and patients alike.

• Karin Verspoor, Dean, School of Computing Technologies, RMIT University, RMIT University; David Hansen, CEO, Australian e-Health Research Centre, CSIRO, and Enrico Coiera, Professor of Medical Informatics, Macquarie University
• This article is republished from The Conversation under a Creative Commons license. Read the original article.