Condition category
Eye Diseases
Date applied
10/02/2020
Date assigned
18/03/2020
Last edited
18/03/2020
Prospective/Retrospective
Prospectively registered
Overall trial status
Ongoing
Recruitment status
Recruiting

Plain English Summary

Background and study aims
One in ten of all patients referred to hospitals in the UK are for problems involving the eyes. Many of the most urgent referrals are for problems in the retina, a light-sensitive layer at the back of the eye which allows us to see. Doctors can now detect and diagnose these diseases earlier than ever before thanks to a technology called Optical Coherence Tomography (OCT). This light-based technology is safe, comfortable and quick for the patient and provides the results within seconds. Excitingly, OCT is increasingly being installed in high street optometrist practices, and is no longer limited to hospital eye clinics. This could transform the way patients with retinal diseases are cared for, but only if these scans are accurately interpreted enabling early diagnosis and correct decisions over who should be referred to the hospital eye service, and how urgently. Currently, however, the introduction of this new technology on the high street is not always matched by the availability of those skilled at interpreting these scans. This is leading to a huge number of patients being referred inappropriately, which is increasing the pressure on hospital eye services and delaying access to care for patients that do need treatment. Additionally, unnecessary referral causes distress and inconvenience. This study aims to use new technologies to improve this referral process, moving hospital-level expertise to the high street without the specialist needing to leave the hospital, and helping the hospital eye service and high street optometrists work together to refer the ‘right patient at the right time’.

Who can participate?
People attending the involved community optometry practices, who underwent an OCT, and who in the opinion of the community optometrist have any suspicion of a retinal condition

What does the study involve?
The first technology to be evaluated is ‘teleophthalmology’ in which OCT scans taken by high street optometrists are automatically reviewed by hospital specialists remotely. In this part of the study high street optometrists with OCT will be divided into two groups: half of the practices, selected by chance, will continue to refer patients using the existing paper-based system, with the other half installing a leading teleophthalmology platform (‘Big Picture’) to allow instant transfer of scans to the eye hospitals for review and advice within 24 hours. The researchers will check whether the new referral system can safely lead to fewer unnecessary visits to the eye hospitals and whether it improves the time it takes for referred patients to be seen or treated. They will also assess the cost of the new system to the NHS and ask what patients and healthcare practitioners think about it – their confidence in its safety and data privacy, and the effect on patient experience. The second technology to be evaluated is the interpretation of retinal scans by ‘artificial intelligence’ (AI). In a recent study from the UK, members of the study team and a leading technology company worked together to develop an AI algorithm that can diagnose retinal diseases and provide referral advice to the same standard as leading UK experts. This exciting development could enable expert-level care to be digitally embedded into every optometry practice as standard but first they need further evidence of how this AI technology would perform in the real world. In this part of our study, they will use this technology (‘the DeepMind algorithm’) on all the OCT scans collected from the participating high street optometrists and the researchers will assess how accurate it is in providing the correct advice for referral. A key part of this study is to collect the opinions of patients and practitioners on the potential role of Artificial Intelligence for eye referrals. What the researchers learn from this part of the study will also help determine what further evaluation might be needed before AI retinal diagnosis could become mainstream, and identify any concerns about this.

What are the possible benefits and risks of participating?
People involved in the study will continue to receive care as needed and will be referred to a hospital-based eye unit if required. There are no direct benefits from participation but the study can help establish a care pathway that is more friendly and convenient to patients. The study involves minimal risks for participants. A safety net arrangement is in place for patients who are not referred to hospital eye services.

Where is the study run from?
1. Moorfields Eye Hospital NHS Foundation Trust (UK)
2. Manchester Royal Eye Hospital (UK)
3. Birmingham University Hospitals (UK)

When is the study starting and how long is it expected to run for?
March 2020 to February 2023

Who is funding the study?
National Institute for Health Research (NIHR) (UK)

Who is the main contact?
Dr Konstantinos Balaskas
konstantinos.balaskas@moorfields.nhs.uk

Trial website

http://readingcentre.org

Contact information

Type

Scientific

Primary contact

Dr Konstantinos Balaskas

ORCID ID

Contact details

Moorfields Eye Hospital NHS Foundation Trust
London
EC1V 9PD
United Kingdom
+44 (0)207 566 2815
k.balaskas@nhs.net

Additional identifiers

EudraCT number

Nil known

ClinicalTrials.gov number

Nil known

Protocol/serial number

v1.0, NIHR127773

Study information

Scientific title

Tele-opHthalmology-enablEd and aRtificial Intelligence-ready referral pathway for coMmunity optomEtry referralS of retinal disease: the HERMES study - a cluster randomised superiority trial with a linked observational diagnostic accuracy study

Acronym

HERMES

Study hypothesis

Can we improve the care of patients with sight-threatening retinal disease by using new technologies – teleophthalmology and artificial intelligence (AI) - to disseminate expert care from the hospital to the high-street? Specifically, the researchers will ask:
1. Is timely and appropriate review for patients with retinal disease improved by embedding cloud-based teleophthalmology technologies in high-street optometrist practices?
2. Can this remote review be safely performed by an AI Decision Support System (AI DSS) previously validated in a hospital setting (Moorfields-DeepMind algorithm)?
3. What are the barriers and enablers to the adoption of these digital technologies?

Primary objectives: To compare the proportion of referrals classified as unnecessary (cases that can be safely managed without a HES consultation) between standard care and teleophthalmology referral pathway (objective 1) and to assess the diagnostic (referral) accuracy of AI DSS in the context of this referral pathway (objective 2). Enablers and barriers to adoption of these digital technologies will be assessed through a Human-Computer Interaction (HCI) analysis. A full economic analysis of the digital referral pathways will be performed.

Addressing objective 1, an interventional superiority cluster randomised trial (RCT) will be performed comparing standard practice for referral of suspicious retinal disease with teleophthalmology (‘Big Picture’ platform) between community optometry and HES.

Addressing objective 2 a prospective observational study will be conducted integrating the data of the RCT to assess the diagnostic (referral) accuracy of an advanced AI DSS (the Moorfields-DeepMind algorithm) for the automated diagnosis and referral recommendation for retinal disease. A within-trial and model-based economic evaluation will estimate the efficiency of alternative referral models for retinal disease. An HCI analysis using qualitative methods will assess the feasibility of implementation of both digital technologies.

Ethics approval

Not provided at time of registration

Study design

Interventional superiority cluster randomised trial

Primary study design

Interventional

Secondary study design

Cluster randomised trial

Trial setting

Community

Trial type

Diagnostic

Patient information sheet

Not available in web format, please use contact details to request a participant information sheet

Condition

Retiro-choroidal disease

Intervention

The intervention pathway is the tele-ophthalmology model for referral of patients with suspicion of retinal disease from community optometry to HES using the Big Picture platform. Patients who attend participating community optometry practices will undergo a clinical assessment and OCT scan. Patients with a suspicion of any retinal disease at the opinion of the community optometrist will be included in the study and their OCT and ‘Smart History’ will be transferred via the Big Picture platform to corresponding HES. The ‘Smart History’ is obtained on iPads via the Big Picture platform and will contain the same standardised clinical information as in standard practice community optometry consultations. In each case, human experts based in HES will make a referral decision remotely (‘tele-HES’) after review of OCT and clinical information on the Big Picture platform. The referring community optometrist will also make their own referral recommendation independent of HES. In each case both the decision made by the community optometrist and the one made by remote review in ‘tele-HES’ will be recorded but the decision made by ‘tele-HES’ will be the one implemented. The following scenarios can occur in the intervention arm:
1. Community optometrist decision: Refer urgently to HES —> OCT scan and ‘Smart History’ are transferred to ‘tele-HES’ and reviewed within 24 h remotely by human expert —> Referral Decision is made in ‘tele-HES’ (refer urgently, refer routinely, don’t refer) and fed-back to the community optometry practice to be implemented.
2. Community optometrist decision: Refer routinely to HES —> OCT scan and ‘Smart History’ are transferred to ‘tele-HES’ and reviewed within 24 h remotely by human expert —> Referral Decision is made in ‘tele-HES’ (refer urgently, refer routinely, don’t refer) and fed-back to the community optometry practice to be implemented.
3. Community optometrist decision: Don’t refer to HES —> OCT scan and ‘Smart History’ are transferred to ‘tele-HES’ and reviewed within 24 h remotely by human expert —> Referral Decision is made in ‘tele-HES’ (refer urgently, refer routinely, don’t refer) and fed-back to the community optometry practice to be implemented.
The decision made in ‘tele-HES’ will be the one implemented in every case in the intervention pathway. The remote review of OCTs and ‘Smart History’ at ‘tele-HES’ will be performed by expert clinicians (medics or specialist optometrists) experienced in retinal clinics (minimum of two years’ experience of independent practice in the context of retinal clinics in HES) based at Moorfields Eye Hospital, Manchester Royal Eye Hospital or Birmingham Eye Hospital with access to senior advice by Consultant Ophthalmologists specialising in retinal disease.

In the first part of the study, high street optometrists with OCT will be divided into two groups: half of the practices, selected by chance, will continue to refer patients using the existing paper-based system, with the other half installing a leading teleophthalmology platform (‘Big Picture’) to allow instant transfer of scans to the eye hospitals for review and advice within 24 hours. The researchers will then check whether the new referral system can safely lead to fewer unnecessary visits to the eye hospitals and whether it improves the time it takes for referred patients to be seen or treated. They will also assess the cost of the new system to the NHS and ask what patients and healthcare practitioners think about it – their confidence in its safety and data privacy, and the effect on patient experience.

In the second part of the study, the researchers will use the DeepMind algorithm on all the OCT scans collected from the participating high street optometrists and assess how accurate it is in providing the correct advice for referral. A key part of this study is to collect opinions of patients and practitioners on the potential role of Artificial Intelligence for eye referrals. What the researchers learn from this part of the study will also help determine what further evaluation might be needed before AI retinal diagnosis could become mainstream, and identify any concerns about this.

Intervention type

Device

Phase

Not Applicable

Drug names

Primary outcome measure

Cluster RCT:
Proportion of false-positive referrals (unnecessary HES visits) in the current referral pathway and the teleophthalmology referral pathway (against the Reference Standard) at the end of the recruitment period. The primary endpoint selected is patient-centric as unnecessary visits to HES are associated with significant anxiety and inconvenience for patients as demonstrated by pre-application PPI work, while at the same time having significant implications for NHS services in terms of costs and relative efficiency.

AI study:
Diagnostic accuracy (sensitivity and specificity) of the referral decision made by the Moorfields-DeepMind AI (dichotomous analysis: refer to HES, do not refer to HES)

Secondary outcome measures

All measured at the end of recruitment:

Cluster RCT:
1. Proportion of wrong diagnosis and wrong referral urgency (as a percentage %) in standard and teleophthalmology pathways against the reference standard
2. Proportion of false-negative referrals (as a percentage %) patients that would have benefited from a HES review) as well as sensitivity and specificity in standard and teleophthalmology pathways against the reference standard
3. Time from referral to consultation (in days) for urgent and routine referrals in standard and teleophthalmology pathways
4. Time from referral to treatment (in days) for urgent maculopathies (wet AMD and Retinal Vein Occlusions) in standard and teleophthalmology pathways
5. Number of uncommon referrals (rare disease) that can be safely triaged in the teleophthalmology pathway

AI study:
1. Diagnostic accuracy (sensitivity and specificity) of Moorfields-DeepMind AI for the diagnosis of retinal disease
2. Diagnostic accuracy (sensitivity and specificity) of Moorfields-DeepMind AI for referral urgency (routine or urgent referral)
3. Proportion of false-positive referrals (as a percentage %) in the standard and teleophthalmology pathways when human assessors are replaced by the AI DSS
4. Proportion of wrong diagnosis and wrong referral urgency (as a percentage %) in the standard and teleophthalmology pathways when human assessors are replaced by AI DSS
5. Uptime and end-to-end inference speed (in seconds) of technical infrastructure supporting the AI DSS
6. Average time of end-to-end output (referral recommendation) by the AI DSS (in hours)

Overall trial start date

01/03/2020

Overall trial end date

28/02/2023

Reason abandoned (if study stopped)

Eligibility

Participant inclusion criteria

1. People attending the involved community optometry practices who underwent an OCT
2. People who at the opinion of the community optometrist have any suspicion of a retinal condition (including dry AMD, wet AMD, diabetic retinopathy, macular oedema, macular holes, epiretinal membranes, central serous chorio-retinopathy, genetic eye disease)
3. Macular OCT scan performed using either Topcon 3D OCT-2000 or Heidelberg OCT1 device (performed with Heidelberg ‘dense’ acquisition settings)

Participant type

Patient

Age group

Adult

Gender

Both

Target number of participants

794

Participant exclusion criteria

1. People with any non-retinal ocular co-morbidities in either eye other than cataract
2. People with media opacities, inability to position or fixate or any other reason that prevents acquisition of good quality OCT scans (at the discretion of the community optometrist)

Recruitment start date

01/09/2020

Recruitment end date

30/08/2022

Locations

Countries of recruitment

United Kingdom

Trial participating centre

Moorfields Eye Hospital NHS Foundation Trust
City Road
London
EC1V 2PD
United Kingdom

Trial participating centre

Manchester Royal Eye Hospital
Oxford Road
Manchester
M13 9WL
United Kingdom

Trial participating centre

Birmingham University Hospitals
Mindelsohn Way
Birmingham
B15 2TH
United Kingdom

Sponsor information

Organisation

Moorfields Eye Hospital NHS Foundation Trust

Sponsor details

162 City Road
London
EV1V 9PD
United Kingdom
+44 (0)207 566 2815
k.balaskas@nhs.net

Sponsor type

Hospital/treatment centre

Website

http://www.moorfields.nhs.uk/

Funders

Funder type

Government

Funder name

National Institute for Health Research

Alternative name(s)

NIHR

Funding Body Type

government organisation

Funding Body Subtype

National government

Location

United Kingdom

Results and Publications

Publication and dissemination plan

Planned outputs:
1. High-impact peer-reviewed publications in Ophthalmology and Health Service Research journals
2. Presentations in conferences, including the Royal College of Ophthalmologists, the Association for Research in Vision and Ophthalmology, the American Academy of Ophthalmology conference, The College of Optometrists
3. The output of this program has the potential to influence the healthcare market by validating digital care pathways for patients with retinal disease. The outcomes of this research will be communicated to NHS England and the Department of Health to inform policy on the role of digital technologies, including tele-medicine and AI DSS.
The research team and the sponsor organisation will actively approach and engage key parties such as the College of Optometrists, stakeholders in the community optometry market and Clinical Commissioning Groups. A detailed engagement plan will be formulated to disseminate the results of this research in order to inform policy decisions for optimising patient care. The proposed route to market will involve commissioning arrangements for the adoption of tele-ophthalmology referral pathways between community optometry and HES.

IPD sharing statement
The data sharing plans for the current study are unknown and will be made available at a later date.

Intention to publish date

01/06/2023

Participant level data

To be made available at a later date

Basic results (scientific)

Publication list

Publication citations

Additional files

Editorial Notes

17/03/2020: Trial's existence confirmed by the NIHR.