A prospective study to assess the impact and benefits of an artificial intelligence system in double reading for breast cancer screening
| ISRCTN | ISRCTN95571932 |
|---|---|
| DOI | https://doi.org/10.1186/ISRCTN95571932 |
| IRAS number | 316873 |
| Secondary identifying numbers | KMT014, IRAS 316873, CPMS 53675 |
- Submission date
- 30/08/2023
- Registration date
- 29/09/2023
- Last edited
- 04/10/2023
- Recruitment status
- No longer recruiting
- Overall study status
- Completed
- Condition category
- Cancer
Plain English summary of protocol
Background and study aims
Normally, mammograms (breast X-rays) are looked at by two qualified healthcare professionals (often called ‘readers’), to decide if more tests are needed. In this study, a piece of new technology using artificial intelligence (AI) software (known as ‘Mia’) will also look at some mammograms in the background, in addition to the two readers. The study aims to see how the AI performs when compared to readers, and the impact of introducing AI into breast screening. This will help to understand if AI might help us find more cancers and how humans may interact with AI in the real world.
Who can participate?
Breast screening participants at Leeds Teaching Hospitals aged between 49 and 70 years old who have a standard 4-view mammogram taken of sufficient quality
What does the study involve?
Half of eligible women (1 in 2) will be chosen at random to have their mammogram looked at by the AI, as well as two readers. The other half will have their mammogram looked at by two readers as normal. Breast screening appointments will go ahead as normal. Women will not need to have any extra X-rays, and women will find out their results in the usual way.
What are the possible benefits and risks of participating?
The long history of breast screening shows human readers miss a very small number of breast cancers on mammograms. It is possible a small proportion of women whose cancers may have been missed by human readers will benefit from earlier cancer detection by having AI involved. This is because AI and humans do not analyse mammograms in the same way.
Using AI to look at mammograms could mean that more women are called back for further tests. This may make participants feel anxious about having further tests, but most people who need further tests will not be diagnosed with breast cancer. In the breast screening programme, about 4 in every 100 women are asked to come back for more tests after screening. Out of these 4 women, 1 will be found to have cancer. The rest will not have cancer and will go back to having screening invitations every three years.
Where is the study run from?
Kheiron Medical Technologies (UK)
When is the study starting and how long is it expected to run for?
April 2022 to August 2024
Who is funding the study?
NHS Accelerated Access Collaborative (AAC) National Institute for Health and Care Research (NIHR) Artificial Intelligence in Health and Care Award (UK)
Who is the main contact?
Dr Annie Ng, annie@kheironmed.com (UK)
Contact information
Principal Investigator
Director of Breast Screening and Clinical Lead for breast imaging
Leeds Teaching Hospital NHS Trust
St James's University Hospital
Beckett Street
Leeds
LS9 7TF
United Kingdom
| Phone | +44 (0)771 892 4193 |
|---|---|
| Nisha.Sharma2@nhs.net |
Scientific
2nd Floor, Stylus Building
116 Old Street
London
EC1V 9BG
United Kingdom
 ORCID ID |
0000-0002-0016-2275 |
|---|---|
| Phone | +44 (0)7379467701 |
| annie@kheironmed.com |
Public
2nd Floor, Stylus Building
116 Old Street
London
EC1V 9BG
United Kingdom
| Phone | +44 (0)7379467701 |
|---|---|
| annie@kheironmed.com |
Study information
| Study design | Prospective feasibility study |
|---|---|
| Primary study design | Interventional |
| Secondary study design | Randomised controlled trial |
| Study setting(s) | Medical and other records |
| Study type | Diagnostic |
| Participant information sheet | Not available in web format, please use contact details to request a participant information sheet. |
| Scientific title | A prospective study to assess the impact and benefits of an AI system in double reading for breast cancer screening |
| Study acronym | LIBRA: Leeds Investigation of BReast screening AI |
| Study objectives | This study aims to evaluate mammography intelligent assessment (Mia)’s influence on arbitration and the potential clinical benefit of Mia by involving Mia’s opinion as a third independent read in arbitrated breast screening cases. |
| Ethics approval(s) |
Approved 01/11/2022, South East Scotland Research Ethics Committee 02 (2nd Floor, Waverley Gate, 2-4 Waterloo Place, Edinburgh, EH1 3EG, United Kingdom; +44 (0)131 536 9000; ruth.fraser4@nhslothian.scot.nhs.uk), ref: 22/SS/0077 |
| Health condition(s) or problem(s) studied | Breast cancer |
| Intervention | This is a single-site prospective feasibility study comparing double reading with Mia (AI software as a medical device) with standard double reading in a local breast screening programme. The study will evaluate Mia’s influence on arbitration and the potential clinical benefit of Mia by involving Mia’s opinion as a third independent read in arbitrated breast screening cases. 50% of all eligible cases will be randomly assigned to Arm A, and the other 50% to Arm B to encourage the diligent reading of all cases according to the standard of care. Approximately 6876 total screens are expected to be included in the total sample size. An active dissent approach will be used where the sample obtained is more likely to be representative of a real-world screening population and a better evaluation of the behavioural impact and benefits that Mia may have in real-world deployments in double reading. Potential participants will have multiple opportunities to learn about the study, including thorough materials that have been designed in collaboration with PPI Advisors, and will have the option to opt out of participation. In Arm A, Mia’s outputs will be shared with the human readers as part of the arbitration process. In Arm B, Mia will not analyse cases until after the positive evidence collection period is complete and these cases will be handled in a retrospective analysis. As such, Mia’s outputs for control arm cases will not be available to human readers as part of the arbitration process during the study. The following procedures will be applied for Arm A: 1. All cases will continue to be read according to the site’s routine double-reading workflow. Therefore, human Readers R1 and R2 will read all cases as normal 2. Human Readers R1 and R2 will always be blinded to Mia’s opinion in the study 3. Opinions for R1, R2 and Mia on eligible cases will be compared 4. When R1, R2 and Mia all agree to not recall a case, this is taken as the final recall decision and follows the routine screening pathway. Where R1, R2 and Mia all agree to recall a case, consensus arbitration takes place. When there is a disagreement between any of the readers, where any but not all of the readers (R1, R2, or Mia) decide to call back a case, discordant arbitration takes place. 5. All cases to be arbitrated will continue to be arbitrated according to the site’s local standard operating practice and will include R1, R2 and Mia opinions 6. After arbitration, the recall or no recall decision will follow the site’s standard pathway. As illustrated by the sequence of events in the intervention arm, there is no change to the participant’s visit in terms of how they receive their mammogram and their general appointment. No further visits or interventions/procedures are required. |
| Intervention type | Device |
| Pharmaceutical study type(s) | Not Applicable |
| Phase | Not Applicable |
| Drug / device / biological / vaccine name(s) | Mammography intelligent assessment (Mia) |
| Primary outcome measure | The absolute and relative differences in consensus between arbitration and Mia will be compared between Arm A and Arm B using readers' opinions at the end of recruitment |
| Secondary outcome measures | All secondary outcome measures will be measured at the end of recruitment. Measurements including consensus, discordance rate, recall rate, arbitration rate, and proportion of cases/cancers in different scenarios will use reader opinions to calculate. Measurements including cancer detection rate, sensitivity, specificity, and positive predictive value will use reader opinions and cancer follow-up information to calculate. 1. Clinical metrics including recall rate, cancer detection rate, sensitivity, specificity, and positive predictive value will be measured for each reader 2. Discordance rate between Mia and Reader 1 to quantify potential workload saving 3. Proportion of cases and cancers in different scenarios in Arm A 4. Consensus between arbitration and each reader per scenario and across all scenarios in Arm A 5. Recall rate and cancer detection rate of arbitration per scenario in Arm A 6. Potential to reduce recall rate and increase cancer detection rate in specific scenarios in Arm A 7. Measurement comparisons between Arm A and Arm B, measuring the proportion of cases and cancers in different scenarios and consensus between arbitration and each reader in different scenarios 8. Measurements and comparisons between simulated double reading in Arm A to Arm B, measuring recall rate, arbitration rate, cancer detection rate, sensitivity, specificity, and positive predictive value 9. Simulations of the 'extra reader' workflow, measuring recall rate, cancer detection rate, sensitivity, specificity, and positive predictive value, to understand potential clinical benefits |
| Overall study start date | 08/04/2022 |
| Completion date | 31/08/2024 |
Eligibility
| Participant type(s) | Service user |
|---|---|
| Age group | Mixed |
| Lower age limit | 49 Years |
| Upper age limit | 70 Years |
| Sex | Female |
| Target number of participants | 1320 arbitrated cases (estimated to result in about 6876 participants) |
| Key inclusion criteria | 1. Participants routinely invited or self-referred for breast screening 2. Participants identified as female 3. Participants for whom a 2D FFDM standard four-view mammography examination was acquired |
| Key exclusion criteria | 1. Patients with cosmetic breast implants 2. Very high risk patients, including genetic abnormalities etc. 3. Patients who have actively dissented from participation in the study 4. Cases marked as technical recall |
| Date of first enrolment | 31/10/2023 |
| Date of final enrolment | 29/02/2024 |
Locations
Countries of recruitment
- England
- United Kingdom
Study participating centre
Beckett Street
Leeds
LS9 7TF
United Kingdom
Sponsor information
Industry
Stylus Building
116 Old Street
London
EC1V 9BG
England
United Kingdom
| Phone | None available |
|---|---|
| science@kheironmed.com | |
| Website | https://www.kheironmed.com/ |
"ROR" |
https://ror.org/01r3ct535 |
Funders
Funder type
Government
No information available
Results and Publications
| Intention to publish date | 31/08/2025 |
|---|---|
| Individual participant data (IPD) Intention to share | No |
| IPD sharing plan summary | Data sharing statement to be made available at a later date |
| Publication and dissemination plan | Planned publication in a high-impact peer-reviewed journal |
| IPD sharing plan | The data-sharing plans for the current study are unknown and will be made available at a later date. |
Editorial Notes
04/10/2023: Internal review.
12/09/2023: Study's existence confirmed by Lothian NHS Board, South East Scotland Research Ethics Committee 02 (UK).
