A retrospective multi-centre clinical study of a deep learning system in identifying breast cancer through the assessment of mammograms

ISRCTN	ISRCTN15504688
DOI	https://doi.org/10.1186/ISRCTN15504688
ClinicalTrials.gov (NCT)	Nil known
Clinical Trials Information System (CTIS)	Nil known
Integrated Research Application System (IRAS)	304086
Protocol serial number	KMT004, IRAS 304086, CPMS 50959
Sponsor	Kheiron Medical Technologies (United Kingdom)
Funders	Accelerated Access Collaborative (AAC) in partnership with NHSx and the National Institute for Health Research (NIHR), Kheiron Medical Technologies

Submission date: 10/01/2022
Registration date: 03/02/2022
Last edited: 11/02/2025

Recruitment status: No longer recruiting
Overall study status: Completed
Condition category: Cancer

Prospectively registered

Protocol

Statistical analysis plan

Results

Individual participant data

Record updated in last year

Plain English summary of protocol

Background and study aims
Breast cancer is a leading cause of cancer-related mortality among women worldwide, accounting for approximately 600,000 deaths annually.

There is a need for rigorous large-scale studies to assess the performance of artificial intelligence (AI) for the diagnosis of breast cancer from breast scans (mammography). This should be done on diverse cohorts of women across multiple screening sites and on unenriched data representative of a true screening population.

The aim of this study is to evaluate the performance of a novel AI system in detecting breast cancer on diverse cohorts and unenriched data representative of a true screening population.

Who can participate?
Being a retrospective study, no participants are directly involved in the study, and there will be no effect or change to any participant’s care. The study will evaluate the AI system based on its analysis of historical, de-identified cases from study sites where outcomes data (e.g. biopsy, histopathology results, follow-up information) is also collected.

What does the study involve?
Eligible cases will be presented to the AI system for analysis.

What are the possible benefits and risks of participating?
No benefits or risks of participating are anticipated.

Where is the study run from?
Kheiron Medical Technologies (UK)

When is the study starting and how long is it expected to run for?
March 2021 to December 2023

Who is funding the study?
The study is funded by an AI Award, awarded to Kheiron Medical Technologies, by the Accelerated Access Collaborative (AAC) in partnership with NHSX and the National Institute for Health Research (NIHR) (UK)

Who is the main contact?
Dr. Annie Ng
annie@kheironmed.com

Contact information

Dr Annie Ng
Scientific

Kheiron Medical Technologies
2nd Floor, Stylus Building
112-116 Old Street
London
EC1V 9BG
United Kingdom

Phone	+44 (0)7379467701
Email	annie@kheironmed.com

Dr Nisha Sharma
Scientific

St James's University Hospital
Beckett Street
Leeds
LS9 7TF
United Kingdom

Phone	+44 (0)1132433144
Email	nisha.sharma2@nhs.net

Study information

Primary study design	Other
Study design	Retrospective multi-centre clinical study of a CE marked medical device
Secondary study design
Study type	Participant information sheet
Scientific title	A retrospective multi-centre clinical study of a novel medical technology solution in the assessment of mammography images
Study acronym	ARIES
Study objectives	The primary aim of this study is to evaluate the performance of Kheiron’s software, Mia, in detecting malignancy to determine its effectiveness to serve as decision support in breast screening in a multi-centre setting. Assessing the standalone behaviour of Mia characterises the contribution it could have as an independent reader in the overall double reading workflow. Assessing simulated double reading performance with Mia in various workflows enables the evaluation of Mia as an independent reader within various double reading configurations and workflows in breast screening.
Ethics approval(s)	Ethics approval not required
Ethics approval additional information	Ethics approval not required as the research is limited to the use of previously collected, pseudonymised data.
Health condition(s) or problem(s) studied	Decision support in breast cancer screening
Intervention	The intervention is the sponsor's deep learning software (Mia), assessed on de-identified randomised retrospective breast screening cases and outcomes. Comparison is made against the control arm of existing reference outcomes within the retrospective dataset where the deep learning software was not in use.
Intervention type	Device
Phase	Not Applicable
Drug / device / biological / vaccine name(s)	Mia
Primary outcome measure(s)	Sensitivity of the standalone case-wise malignancy detection performance of Mia, measured as the number of positive cases recalled divided by the total number of positive cases, over the full study dataset time period. Specificity of the standalone case-wise malignancy detection performance of Mia, measured as the number of confirmed negative cases not recalled divided by the total number of confirmed negative cases, over the full dataset time period.
Key secondary outcome measure(s)	Current secondary outcome measures as if 17/07/2023: 1. Measurement of relevant clinical metrics for Mia standalone and descriptives of cancer subtypes that Mia picks up to understand Mia’s contribution to double reading. 2. Comparison of Mia standalone and the historical first reader to understand differences in how their performance contributes to double reading. 3. Measurement of relevant clinical metrics, including resource/workload savings, and descriptives of cancer subtypes picked up in double reading workflows that incorporate Mia as an independent reader, to understand the performance of each workflow and to inform future health economic assessments. 4. Comparison of double reading workflows with and without Mia on clinically relevant metrics to understand differences in performance. 5. Comparison of double reading workflows with Mia against UK national guidelines to assess if guidelines thresholds would be met if Mia was used in double reading. 6. Secondary outcomes will also include performance stratified by region/site and ethnicity to confirm generalisability. _____ Previous secondary outcome measures: 1. Recall rate, negative flag rate, cancer detection rate, sensitivity, specificity, interval cancer rate, positive predictive value, percentage of interval cancers recalled, area under the receiver operating characteristic curve will be measured for Mia’s standalone performance over the study dataset time period and a selected one year period with the most complete data 2. Recall rate, cancer detection rate, sensitivity, specificity, interval cancer rate, positive predictive value, arbitration rate will be measured for various simulated double reading workflows with Mia over the study dataset time period and a selected one year period with the most complete data 3. Non-inferiority and superiority and associated absolute and relative differences between Mia standalone against the historical first reader will be measured for cancer detection rate, sensitivity, specificity, for either the study dataset time period or a selected one year period with the most complete data 4. Non-inferiority and superiority and associated absolute and relative differences between simulated double reading with Mia against historical double reading will be measured for cancer detection rate, sensitivity, specificity, recall rate, for either the study dataset time period or a selected one year period with the most complete data
Completion date	31/12/2023

Eligibility

Participant type(s)	Patient
Age group	Adult
Sex	Female
Target sample size at registration	1000000
Key inclusion criteria	1. Female participants 2. Participants attending for breast screening purposes (normal and opportunistic screening)* 3. Participants for whom a 2D FFDM standard four-view mammography examination was acquired (MLO-R, CC-R, MLO-L, CC-L) * Includes: 1) early recall cases (e.g. participants brought back for screening earlier than the established screening interval); and 2) participants of UK age extension trial (AgeX).
Key exclusion criteria	Does not meet inclusion criteria
Date of first enrolment	31/01/2022
Date of final enrolment	30/06/2023

Locations

Countries of recruitment

United Kingdom
England

Study participating centres

North London Breast Screening Programme (Royal Free London NHS Foundation Trust)

Pond Street
Hampstead
London
NW3 2QG
United Kingdom

Gateshead Breast Screening Unit

Queen Elizabeth Hospital
Shrieff Hill
Gateshead
NE9 6SX
United Kingdom

North East Devon Breast Screening Programme

Royal Devon and Exeter NHS Hospital Foundation Trust
Barrack Road
Exeter
EX2 5DW
United Kingdom

Results and Publications

Individual participant data (IPD) Intention to share	Yes
IPD sharing plan summary	Available on request
IPD sharing plan	The datasets generated and analysed during the current study will be available upon request from science@kheironmed.com. Data will be shared according to a data-sharing plan.

Study outputs

Output type	Details	Date created	Date added	Peer reviewed?	Patient-facing?
Participant information sheet	Participant information sheet	11/11/2025	11/11/2025	No	Yes

Editorial Notes

11/02/2025: The following changes were made:
1. The study acronym was added.
2. The Individual participant data (IPD) sharing plan and summary were added.
3. The intention to publish date was changed from 31/12/2024 to 01/06/2025.
17/07/2023: The following changes were made to the trial record:
1. The secondary outcome measures were changed.
2. The sponsor email was changed.
3. The study participating centres Breast Test Wales, Tresliske Hospital, Queen Elizabeth Hospital, HSC Public Health Agency Breast Screening were removed and Gateshead Breast Screening Unit, North East Devon Breast Screening Programme were added.
4. The countries of recruitment Northern Ireland and Wales were removed.
02/03/2022: Internal review.
14/01/2022: Trial's existence confirmed by Public Health England.