How AI is improving diagnosis of Australia's deadliest skin cancer

What's the problem?

Early detection of melanoma can mean the difference between a small skin excision and a fatal diagnosis — but many people in regional and rural areas of Australia do not have easy access to screening services.

Even for trained specialists, identifying a single cancerous mole on a body that may have hundreds of visually similar benign ones can be difficult, and results can vary significantly between practitioners.

It can also be challenging to pick up on the tiny changes in a mole from one screening visit to the next that may indicate cancerous growth.

“Different levels of resources are available in urban, regional and rural healthcare. The strength of these AI tools is that they will improve health equity by equipping the workforce with fast, accurate ways to diagnose melanoma early with the aim to reduce the burden on patients, families and the healthcare system.”

— Professor Peter Soyer

What's UQ's solution?

Drawing on expertise across dermatology, public health, telehealth, genomics, bioinformatics and medical imaging, the UQ team has spent the past decade building a growing suite of AI tools to address different problems in melanoma surveillance.

These tools count moles and freckles, grade sun damage, flag lesions that have changed, and search for new patterns of risk — all in a way that is automated, consistent between patients and visits, and proposed to be more accurate than traditional examination.

The team's longer-term ambition is to bring these tools together into a single, personalised picture of a patient's skin — their skin imaging phenotype — to stratify melanoma risk more precisely than can be achieved through a single tool alone.

The team is part of the Australian Centre of Excellence in Melanoma Imaging and Diagnosis (ACEMID), where it is contributing expertise to a roadmap for a future national melanoma screening program where telehealth and AI tools may play a role.

How does the technology work?

The work is enabled by a key piece of hardware: a whole-body scanner holding 92 cameras that uses cross-polarised light to cut through surface glare and reflection to show the pigmentation and lesions within the skin itself.

Automated software then stitches the photos into a 3D avatar — a high-resolution map of the body showing every mole — that the clinician can look at closely. When the person returns for a follow-up, the new scan is aligned with the old, and any lesion that has changed in size, colour or shape can be picked out for review.

As part of their contribution to ACEMID, the UQ, USYD and Monash teams are scanning 10,000 patients across Queensland, New South Wales and Victoria to provide a rich dataset of images.

Convolutional neural networks for mole counting and mapping sun damage

The UQ team trained a convolutional neural network — an AI that learns by spotting patterns in millions of images — to count moles directly on the 3D avatar.

Number of moles is the strongest predictor of melanoma risk but can differ greatly between different clinicians counting the same patient. The resulting algorithm is the first of its kind in the world and produces a consistent count every time.

In a separate project, the team created a 3D heatmap with skin colour-coded by sun damage: green for mild, yellow for moderate, red for severe — allowing clinicians to see UV damage rather than rely on patient recall of past sunburns.

The team designed a visual rating scale that non-specialists could use to rate a patch of skin, then had lay annotators and dermatology students use it to label nearly 25,000 patches from 107 volunteers.

The team used this massive dataset to train a convolutional neural network to grade sun damage automatically, which involved teaching the network to perform two tasks simultaneously: assessing skin damage and pigmentation.

This ensured the tool could tell the difference between skin damage and skin pigmentation, so it didn't confuse a darker complexion for more sun damage — and was the first AI developed using images captured by total-body photography.

Unsupervised clustering to predict melanoma risk

The research team has also used unsupervised clustering — AI that finds patterns without being told what to look for — to predict invasive melanoma risk.

They fed automated mole counts and clinician-rated sun damage scores into a clustering algorithm, and identified four whole-body phenotypes that predict the risk of invasive melanoma across different body sites — new categories of melanoma risk that hadn't been formally characterised previously.

Building a foundation model for dermatology

The UQ team also contributed clinical expertise and imaging data to PanDerm, an ACEMID project led by Monash University to build a foundation model for dermatology.

Foundation models are pre-trained on large, varied datasets that can be adapted to many tasks at once. PanDerm was trained on more than two million skin images contributed by 11 institutions worldwide, including UQ.

This model can process the four kinds of images a dermatologist routinely uses and reason across them simultaneously in the same way a clinician does in forming a diagnosis. The tool allowed dermatologists to improve their skin cancer diagnostic accuracy by 11 per cent, and non-specialists by 16.5 per cent.

What is the impact?

The most immediate gain will be for clinicians without specialist training. A GP examining a worrying mole can run it through a validated AI tool — using medical images they can already take — and get back a recommendation on whether to refer, and how urgently, a model that is being tested in the UK's National Health System. In Australia, such a model could most benefit regional and rural Australia, where the distance to a dermatologist can be the difference between catching a melanoma early and not catching it at all.

Further down the line, the same tools could change who gets watched closely in the first place. By identifying the small group of high-risk people — earlier, and more precisely than current methods allow — the system can focus specialist time where it's needed, reduce unnecessary referrals, and catch changing moles before they spread. The team's stated aim is melanoma care that's faster, more accurate, and shared more fairly across the country than it is today.