Plain English Summary
Background and study aims
One in ten of all patients referred to hospitals in the UK are for problems involving the eyes. Many of the most urgent referrals are for problems in the retina, a light-sensitive layer at the back of the eye which allows us to see. Doctors can now detect and diagnose these diseases earlier than ever before thanks to a technology called Optical Coherence Tomography (OCT). This light-based technology is safe, comfortable and quick for the patient and provides the results within seconds. Excitingly, OCT is increasingly being installed in high street optometrist practices, and is no longer limited to hospital eye clinics. This could transform the way patients with retinal diseases are cared for, but only if these scans are accurately interpreted enabling early diagnosis and correct decisions over who should be referred to the hospital eye service, and how urgently. Currently, however, the introduction of this new technology on the high street is not always matched by the availability of those skilled at interpreting these scans. This is leading to a huge number of patients being referred inappropriately, which is increasing the pressure on hospital eye services and delaying access to care for patients that do need treatment. Additionally, unnecessary referral causes distress and inconvenience. This study aims to use new technologies to improve this referral process, moving hospital-level expertise to the high street without the specialist needing to leave the hospital, and helping the hospital eye service and high street optometrists work together to refer the ‘right patient at the right time’.
Who can participate?
People attending the involved community optometry practices, who underwent an OCT, and who in the opinion of the community optometrist have any suspicion of a retinal condition
What does the study involve?
The first technology to be evaluated is ‘teleophthalmology’ in which OCT scans taken by high street optometrists are automatically reviewed by hospital specialists remotely. In this part of the study high street optometrists with OCT will be divided into two groups: half of the practices, selected by chance, will continue to refer patients using the existing paper-based system, with the other half installing a leading teleophthalmology platform (‘Big Picture’) to allow instant transfer of scans to the eye hospitals for review and advice within 24 hours. The researchers will check whether the new referral system can safely lead to fewer unnecessary visits to the eye hospitals and whether it improves the time it takes for referred patients to be seen or treated. They will also assess the cost of the new system to the NHS and ask what patients and healthcare practitioners think about it – their confidence in its safety and data privacy, and the effect on patient experience. The second technology to be evaluated is the interpretation of retinal scans by ‘artificial intelligence’ (AI). In a recent study from the UK, members of the study team and a leading technology company worked together to develop an AI algorithm that can diagnose retinal diseases and provide referral advice to the same standard as leading UK experts. This exciting development could enable expert-level care to be digitally embedded into every optometry practice as standard but first they need further evidence of how this AI technology would perform in the real world. In this part of our study, they will use this technology (‘the DeepMind algorithm’) on all the OCT scans collected from the participating high street optometrists and the researchers will assess how accurate it is in providing the correct advice for referral. A key part of this study is to collect the opinions of patients and practitioners on the potential role of Artificial Intelligence for eye referrals. What the researchers learn from this part of the study will also help determine what further evaluation might be needed before AI retinal diagnosis could become mainstream, and identify any concerns about this.
What are the possible benefits and risks of participating?
People involved in the study will continue to receive care as needed and will be referred to a hospital-based eye unit if required. There are no direct benefits from participation but the study can help establish a care pathway that is more friendly and convenient to patients. The study involves minimal risks for participants. A safety net arrangement is in place for patients who are not referred to hospital eye services.
Where is the study run from?
1. Moorfields Eye Hospital NHS Foundation Trust (UK)
2. Manchester Royal Eye Hospital (UK)
3. Birmingham University Hospitals (UK)
When is the study starting and how long is it expected to run for?
March 2020 to February 2023
Who is funding the study?
National Institute for Health Research (NIHR) (UK)
Who is the main contact?
Dr Konstantinos Balaskas
Dr Konstantinos Balaskas
Moorfields Eye Hospital NHS Foundation Trust
+44 (0)207 566 2815
Tele-opHthalmology-enablEd and aRtificial Intelligence-ready referral pathway for coMmunity optomEtry referralS of retinal disease: the HERMES study - a cluster randomised superiority trial with a linked observational diagnostic accuracy study
Can we improve the care of patients with sight-threatening retinal disease by using new technologies – teleophthalmology and artificial intelligence (AI) - to disseminate expert care from the hospital to the high-street? Specifically, the researchers will ask:
1. Is timely and appropriate review for patients with retinal disease improved by embedding cloud-based teleophthalmology technologies in high-street optometrist practices?
2. Can this remote review be safely performed by an AI Decision Support System (AI DSS) previously validated in a hospital setting (Moorfields-DeepMind algorithm)?
3. What are the barriers and enablers to the adoption of these digital technologies?
Primary objectives: To compare the proportion of referrals classified as unnecessary (cases that can be safely managed without a HES consultation) between standard care and teleophthalmology referral pathway (objective 1) and to assess the diagnostic (referral) accuracy of AI DSS in the context of this referral pathway (objective 2). Enablers and barriers to adoption of these digital technologies will be assessed through a Human-Computer Interaction (HCI) analysis. A full economic analysis of the digital referral pathways will be performed.
Addressing objective 1, an interventional superiority cluster randomised trial (RCT) will be performed comparing standard practice for referral of suspicious retinal disease with teleophthalmology (‘Big Picture’ platform) between community optometry and HES.
Addressing objective 2 a prospective observational study will be conducted integrating the data of the RCT to assess the diagnostic (referral) accuracy of an advanced AI DSS (the Moorfields-DeepMind algorithm) for the automated diagnosis and referral recommendation for retinal disease. A within-trial and model-based economic evaluation will estimate the efficiency of alternative referral models for retinal disease. An HCI analysis using qualitative methods will assess the feasibility of implementation of both digital technologies.
Not provided at time of registration
Interventional superiority cluster randomised trial
Primary study design
Secondary study design
Cluster randomised trial
Patient information sheet
Not available in web format, please use contact details to request a participant information sheet
The intervention pathway is the tele-ophthalmology model for referral of patients with suspicion of retinal disease from community optometry to HES using the Big Picture platform. Patients who attend participating community optometry practices will undergo a clinical assessment and OCT scan. Patients with a suspicion of any retinal disease at the opinion of the community optometrist will be included in the study and their OCT and ‘Smart History’ will be transferred via the Big Picture platform to corresponding HES. The ‘Smart History’ is obtained on iPads via the Big Picture platform and will contain the same standardised clinical information as in standard practice community optometry consultations. In each case, human experts based in HES will make a referral decision remotely (‘tele-HES’) after review of OCT and clinical information on the Big Picture platform. The referring community optometrist will also make their own referral recommendation independent of HES. In each case both the decision made by the community optometrist and the one made by remote review in ‘tele-HES’ will be recorded but the decision made by ‘tele-HES’ will be the one implemented. The following scenarios can occur in the intervention arm:
1. Community optometrist decision: Refer urgently to HES —> OCT scan and ‘Smart History’ are transferred to ‘tele-HES’ and reviewed within 24 h remotely by human expert —> Referral Decision is made in ‘tele-HES’ (refer urgently, refer routinely, don’t refer) and fed-back to the community optometry practice to be implemented.
2. Community optometrist decision: Refer routinely to HES —> OCT scan and ‘Smart History’ are transferred to ‘tele-HES’ and reviewed within 24 h remotely by human expert —> Referral Decision is made in ‘tele-HES’ (refer urgently, refer routinely, don’t refer) and fed-back to the community optometry practice to be implemented.
3. Community optometrist decision: Don’t refer to HES —> OCT scan and ‘Smart History’ are transferred to ‘tele-HES’ and reviewed within 24 h remotely by human expert —> Referral Decision is made in ‘tele-HES’ (refer urgently, refer routinely, don’t refer) and fed-back to the community optometry practice to be implemented.
The decision made in ‘tele-HES’ will be the one implemented in every case in the intervention pathway. The remote review of OCTs and ‘Smart History’ at ‘tele-HES’ will be performed by expert clinicians (medics or specialist optometrists) experienced in retinal clinics (minimum of two years’ experience of independent practice in the context of retinal clinics in HES) based at Moorfields Eye Hospital, Manchester Royal Eye Hospital or Birmingham Eye Hospital with access to senior advice by Consultant Ophthalmologists specialising in retinal disease.
In the first part of the study, high street optometrists with OCT will be divided into two groups: half of the practices, selected by chance, will continue to refer patients using the existing paper-based system, with the other half installing a leading teleophthalmology platform (‘Big Picture’) to allow instant transfer of scans to the eye hospitals for review and advice within 24 hours. The researchers will then check whether the new referral system can safely lead to fewer unnecessary visits to the eye hospitals and whether it improves the time it takes for referred patients to be seen or treated. They will also assess the cost of the new system to the NHS and ask what patients and healthcare practitioners think about it – their confidence in its safety and data privacy, and the effect on patient experience.
In the second part of the study, the researchers will use the DeepMind algorithm on all the OCT scans collected from the participating high street optometrists and assess how accurate it is in providing the correct advice for referral. A key part of this study is to collect opinions of patients and practitioners on the potential role of Artificial Intelligence for eye referrals. What the researchers learn from this part of the study will also help determine what further evaluation might be needed before AI retinal diagnosis could become mainstream, and identify any concerns about this.
Primary outcome measure
Proportion of false-positive referrals (unnecessary HES visits) in the current referral pathway and the teleophthalmology referral pathway (against the Reference Standard) at the end of the recruitment period. The primary endpoint selected is patient-centric as unnecessary visits to HES are associated with significant anxiety and inconvenience for patients as demonstrated by pre-application PPI work, while at the same time having significant implications for NHS services in terms of costs and relative efficiency.
Diagnostic accuracy (sensitivity and specificity) of the referral decision made by the Moorfields-DeepMind AI (dichotomous analysis: refer to HES, do not refer to HES)
Secondary outcome measures
All measured at the end of recruitment:
1. Proportion of wrong diagnosis and wrong referral urgency (as a percentage %) in standard and teleophthalmology pathways against the reference standard
2. Proportion of false-negative referrals (as a percentage %) patients that would have benefited from a HES review) as well as sensitivity and specificity in standard and teleophthalmology pathways against the reference standard
3. Time from referral to consultation (in days) for urgent and routine referrals in standard and teleophthalmology pathways
4. Time from referral to treatment (in days) for urgent maculopathies (wet AMD and Retinal Vein Occlusions) in standard and teleophthalmology pathways
5. Number of uncommon referrals (rare disease) that can be safely triaged in the teleophthalmology pathway
1. Diagnostic accuracy (sensitivity and specificity) of Moorfields-DeepMind AI for the diagnosis of retinal disease
2. Diagnostic accuracy (sensitivity and specificity) of Moorfields-DeepMind AI for referral urgency (routine or urgent referral)
3. Proportion of false-positive referrals (as a percentage %) in the standard and teleophthalmology pathways when human assessors are replaced by the AI DSS
4. Proportion of wrong diagnosis and wrong referral urgency (as a percentage %) in the standard and teleophthalmology pathways when human assessors are replaced by AI DSS
5. Uptime and end-to-end inference speed (in seconds) of technical infrastructure supporting the AI DSS
6. Average time of end-to-end output (referral recommendation) by the AI DSS (in hours)
Overall trial start date
Overall trial end date
Reason abandoned (if study stopped)
Participant inclusion criteria
1. People attending the involved community optometry practices who underwent an OCT
2. People who at the opinion of the community optometrist have any suspicion of a retinal condition (including dry AMD, wet AMD, diabetic retinopathy, macular oedema, macular holes, epiretinal membranes, central serous chorio-retinopathy, genetic eye disease)
3. Macular OCT scan performed using either Topcon 3D OCT-2000 or Heidelberg OCT1 device (performed with Heidelberg ‘dense’ acquisition settings)
Target number of participants
Participant exclusion criteria
1. People with any non-retinal ocular co-morbidities in either eye other than cataract
2. People with media opacities, inability to position or fixate or any other reason that prevents acquisition of good quality OCT scans (at the discretion of the community optometrist)
Recruitment start date
Recruitment end date
Countries of recruitment
Trial participating centre
Moorfields Eye Hospital NHS Foundation Trust
Trial participating centre
Manchester Royal Eye Hospital
Trial participating centre
Birmingham University Hospitals
National Institute for Health Research
Funding Body Type
Funding Body Subtype
Results and Publications
Publication and dissemination plan
1. High-impact peer-reviewed publications in Ophthalmology and Health Service Research journals
2. Presentations in conferences, including the Royal College of Ophthalmologists, the Association for Research in Vision and Ophthalmology, the American Academy of Ophthalmology conference, The College of Optometrists
3. The output of this program has the potential to influence the healthcare market by validating digital care pathways for patients with retinal disease. The outcomes of this research will be communicated to NHS England and the Department of Health to inform policy on the role of digital technologies, including tele-medicine and AI DSS.
The research team and the sponsor organisation will actively approach and engage key parties such as the College of Optometrists, stakeholders in the community optometry market and Clinical Commissioning Groups. A detailed engagement plan will be formulated to disseminate the results of this research in order to inform policy decisions for optimising patient care. The proposed route to market will involve commissioning arrangements for the adoption of tele-ophthalmology referral pathways between community optometry and HES.
IPD sharing statement
The data sharing plans for the current study are unknown and will be made available at a later date.
Intention to publish date
Participant level data
To be made available at a later date
Basic results (scientific)