DemDx AI-driven Ophthalmology Triage System clinical evaluation in accident and emergency
| ISRCTN | ISRCTN25862260 |
|---|---|
| DOI | https://doi.org/10.1186/ISRCTN25862260 |
| IRAS number | 310792 |
| Secondary identifying numbers | DAYA1006, IRAS 310792, CPMS 55499 |
- Submission date
- 16/02/2023
- Registration date
- 07/08/2023
- Last edited
- 20/09/2023
- Recruitment status
- No longer recruiting
- Overall study status
- Completed
- Condition category
- Eye Diseases
Plain English summary of protocol
Background and study aims
To meet the rising demand for emergency eye care, accurate triage in adult eye casualty is needed to appropriately provide emergency medical resources to those who most need them. This study will implement a previously developed Artificial Intelligence (AI) driven Clinical Decision Support System in adult eye casualty and evaluate the impact on nurses' triage performance.
Who can participate?
Registered nurses who have completed competency training for triage in adult eye casualty
What does the study involve?
The study will implement the DemDx AI-driven Triage System (DOTS) within adult eye casualty triage led by nurses. During the study, all participating nurses will be required to conduct triage using the device. The study will then evaluate the impact of the tool on triage performance by comparing it to triage performance in current practice without the device. Metrics for comparison include but are not limited to the proportion of correctly triaged urgent and non-urgent cases, the proportion of cases triaged as non-urgent and subsequently re-attended A&E, triage time, and cost.
What are the possible benefits and risks of participating?
Participating in the study gives participants the opportunity to gain first-hand experience using the DemDx AI-powered triage platform and use it to support their clinical decision-making and workflow. The researchers are not aware of the potential risks of participation as the safety and workflow feasibility of the AI support system has been previously demonstrated.
Where is the study run from?
Moorfields Eye Hospital (UK)
When is the study starting and how long is it expected to run for?
February 2022 to October 2023
Who is funding the study?
The study is funded by NIHR AI Award (AI_AWARD0167) (UK)
Who is the main contact?
Dr Mariane Melo
mariane.melo@nhs.net
Contact information
Principal Investigator
Moorfields Eye Hospital
162 City Road London
EC1V 6PD
London
EC1V 6PD
United Kingdom
| Phone | 07367205215 |
|---|---|
| mariane.melo@nhs.net |
Study information
| Study design | Single-centred pre-post quasi-experimental observational study |
|---|---|
| Primary study design | Observational |
| Secondary study design | Cohort study |
| Study setting(s) | Hospital |
| Study type | Screening |
| Participant information sheet | Not available in web format, please use contact details to request a participant information sheet |
| Scientific title | Clinical evaluation and implementation of DemDx AI-driven Ophthalmology Triage System (DOTS) in an adult eye casualty accident and emergency (A&E) |
| Study acronym | DOTS |
| Study objectives | The primary hypothesis is that the use of the DemDX AI-driven Ophthalmology Triage System (DOTS) increases the accuracy of urgent referrals to adult eye casualty and reduces unnecessary appointments in accident and emergency (A&E). The secondary hypotheses include: 1. DOTS differential diagnoses suggestions have acceptably high sensitivity and specificity to support referral decision-making. 2. The use of DOTS decreases patient waiting time from referral to consultation for urgent referrals and does not increase resource usage. 3. The design of DOTS is user-centred and the workflow within the AI-driven pathway is non-inferior to its current counterpart. 4. Users trust the results given by the triage platform and follow the recommendations most of the time. |
| Ethics approval(s) | Approved 22/05/2023, Yorkshire & The Humber - Sheffield Research Ethics Committee (3rd Floor, Barlow House, 4 Minshull Street, Manchester, M1 3DZ, UK; +44 (0)207 104 8282, +44 (0)207 104 8290; sheffield.rec@hra.nhs.uk), ref: 23/YH/0068 |
| Health condition(s) or problem(s) studied | Triaging of patients who attend adult eye casualty |
| Intervention | The clinical investigation will be a pre-post quasi-experimental study, comparing the triage performance after implementing the AI-driven triage support tool A&E triage pathway to prospective cases using the existing triage pathway. The A&E triage department at the enrolled site will have access to and be required to use DOTS throughout the trial period to support their triage decisions. Users will input the structured clinical representation of the case in the input form and state their own triage decision and submit it to the DOTS. DOTS will show the recommendations given by the AI algorithm on the results page and refine the results with suggested additional clinical features. Participants will then record if they wish to follow the triage recommendation or not. All cases classified as elective/low risk will be evaluated individually by an Ophthalmologist for cross-checking. Re-attendance of patients triaged as elective/low risk within 10 days will be audited by Ophthalmologist and classified as false negatives as appropriate. For quality control checks and to evaluate the reliance of the nurses using the DemDX DSS, the platform will not give a result in 10% of the cases but the data will still be recorded. Data collected using DOTS will be linked with the outcomes triage referral outcome, discharge diagnoses and treatment recorded on the EMR system. Performance data of the service and resource utilization will be analyzed and compared with the same period in the previous year. Structured surveys will be sent to users to evaluate their trust in the users and their reliance on DOTS. |
| Intervention type | Device |
| Pharmaceutical study type(s) | Not Applicable |
| Phase | Phase I |
| Drug / device / biological / vaccine name(s) | DemDx AI-driven Ophthalmology Triage System (DOTS) |
| Primary outcome measure | The proportion of false positive referrals to same-day Eye Casualty A&E appointment or UCC (urgent care clinic, same-week appointments) consultations compared with retrospective data from matched cases in the study site. A false positive is a patient who has been inappropriately referred to same-day Eye Casualty A&E appointment or UCC consultation but who could have been given appropriate advice at triage and safely discharged without a face-to-face consultation. Adjusted relative risk (RR) will be reported. This is a before-and-after study where the triage data will be collected with the intervention for about 3 months and compared with the same period in the previous year. Patient data will be collected at one point during the trial but it will be monitored for 14 days for re-attendance. |
| Secondary outcome measures | 1. In the accuracy sub-study, agreement levels of DOTS for suggesting urgent and non-urgent differential diagnoses will be compared with the diagnoses at discharge 2. Rate of Eye Casualty A&E re-attendance within 14 days (potential false negatives) 3. Patient waiting time compared to pre-intervention as previously reported by the Performance and Information team 4. Cost-effectiveness analysis of intervention using false positive referrals and resource utilisation as compared with pre-intervention data This is a before-and-after study where the triage data will be collected with the intervention for about 3 months and compared with the same period in the previous year. Patient data will be collected at one point during the trial but it will be monitored for 14 days for re-attendance. |
| Overall study start date | 02/02/2022 |
| Completion date | 01/10/2023 |
Eligibility
| Participant type(s) | Health professional |
|---|---|
| Age group | Adult |
| Sex | Both |
| Target number of participants | 20 triage nurses with target 1198 triage visits in A&E |
| Total final enrolment | 15 |
| Key inclusion criteria | Registered nurses who have completed competency training in triage in adult eye casualty |
| Key exclusion criteria | 1. Nurses who have not completed ophthalmic triage competency training 2. Triage nurses in paediatric eye casualty |
| Date of first enrolment | 17/06/2023 |
| Date of final enrolment | 07/07/2023 |
Locations
Countries of recruitment
- England
- United Kingdom
Study participating centre
162 City Road
London
EC1V 2PD
United Kingdom
Sponsor information
Hospital/treatment centre
162 City Road
Moorfields Eye Hospital
London
EC1V 6PD
England
United Kingdom
| Phone | +44 (0)207 566 2059 |
|---|---|
| rowena.vohora@nhs.net | |
| Website | http://www.moorfields.nhs.uk/ |
"ROR" |
https://ror.org/03zaddr67 |
Funders
Funder type
Government
Government organisation / National government
- Alternative name(s)
- National Institute for Health Research, NIHR Research, NIHRresearch, NIHR - National Institute for Health Research, NIHR (The National Institute for Health and Care Research), NIHR
- Location
- United Kingdom
Results and Publications
| Intention to publish date | 01/11/2023 |
|---|---|
| Individual participant data (IPD) Intention to share | Yes |
| IPD sharing plan summary | Available on request |
| Publication and dissemination plan | 1. Planned publication in a high-impact peer-reviewed journal 2. Clinical Evaluation Report for MHRA submission 3. International conference presentation |
| IPD sharing plan | Anonymised data may be accessed upon request from the Chief Investigator, Alex Day (alex.day1@nhs.net) after the completion of the study (September 2023). Data will include information about triage cases, discharge diagnoses and the risk stratification given by the too. Consent will be obtained from participants (triage staff) but not patients, however, all patient-identifiable data will be anonymised by members of the direct care team. |
Study outputs
| Output type | Details | Date created | Date added | Peer reviewed? | Patient-facing? |
|---|---|---|---|---|---|
| HRA research summary | 20/09/2023 | No | No |
Editorial Notes
20/09/2023: A link to the HRA research summary was added.
05/09/2023: Internal review.
27/02/2023: Trial's existence confirmed by the NIHR.
