An evaluation of Ufonia’s conversational agent Dora for cataract surgery follow-up
| ISRCTN | ISRCTN16038063 |
|---|---|
| DOI | https://doi.org/10.1186/ISRCTN16038063 |
| IRAS number | 297548 |
| ClinicalTrials.gov number | NCT05213390 |
| Secondary identifying numbers | IRAS 297548, CPMS 49658 |
- Submission date
- 08/11/2021
- Registration date
- 11/01/2022
- Last edited
- 29/07/2024
- Recruitment status
- No longer recruiting
- Overall study status
- Completed
- Condition category
- Surgery
Plain English Summary
Background and study aims
This project will apply AI technology to meet the gap between increasing demand and the limited capacity of high-volume healthcare services. Ufonia proposes to replace routine clinical follow-up with Dora - a natural-language AI assistant delivered via a regular telephone call. The project will develop evidence that will support the safe deployment of Ufonia’s automated telemedicine platform to deliver calls to cataract surgery patients at two large NHS hospital trusts.
Who can participate?
Patients aged 18 years and over who are listed for routine cataract surgery at either Imperial or Oxford NHS Trusts
What does the study involve?
The Dora call doesn't require patients to download any application, be provided with any device or receive any training. They simply receive a call and have a conversation just as they would have done with a human clinician, on top of the care they would receive if not participating in the study. All participants will also be asked to complete a short survey about their experience, and a small group of participants will be randomly selected for an interview to discuss their experience in more depth.
What are the possible benefits and risks of participating?
This safety check is additional to the usual standard of care at the two sites where the study is being conducted. The process will be overseen by a clinician and the system is designed to provide a thorough assessment every time. This study will also benefit future patients through a more convenient follow-up service. Potential disadvantages are that this is a new system so has not been demonstrated to be effective - this is why the system is currently overseen by a clinician.
Where is the study run from?
Oxford University Hospitals and Imperial College Healthcare NHS Trusts (UK)
When is the study starting and how long is it expected to run for?
March 2020 to March 2022
Who is funding the study?
National Institute for Health Research (NIHR) (UK)
Who is the main contact?
Dr Eduardo Normando, Chief Investigator (e.normando@imperial.ac.uk)
Dr Kanmin Xue, Principal Investigator (kanmin.xue@ouh.nhs.uk);
Dr Edward Meinert, Co-Investigator (edward.meinert@plymouth.ac.uk)
Contact information
Scientific
Imperial College Healthcare NHS Trust
153-173 Marylebone Rd
London
NW1 5QH
United Kingdom
 ORCID ID |
0000-0002-5774-8082 |
|---|---|
| Phone | +44 (0)20 3312 3206 |
| e.normando@imperial.ac.uk |
Public
Centre for Health Technology
6 Kirkby Place, Room 2
Plymouth
PL4 6DN
United Kingdom
| ORCID ID |
0000-0003-2484-3347 |
|---|---|
| Phone | +44 (0)1752600600 |
| edward.meinert@plymouth.ac.uk |
Study information
| Study design | Mixed methods non-randomized study |
|---|---|
| Primary study design | Interventional |
| Secondary study design | Non randomised study |
| Study setting(s) | Home |
| Study type | Screening |
| Participant information sheet | 40639_PIS_V3.0.pdf |
| Scientific title | Autonomous telemedicine - cataract surgery follow-up at two NHS trusts |
| Study hypothesis | An autonomous conversational agent can identify which patients need clinical follow up after a cataract operation. |
| Ethics approval(s) | Approved 01/07/2021, Research Ethics Committee (REC) London Centre (2 Redman Place, HRA, E20 1JQ; +44 (0)2071048029, chelsea.rec@hra.nhs.uk), REC ref: 21/PR/0767 |
| Condition | Clinical follow-up of postoperative symptoms in patients who have undergone routine cataract surgery |
| Intervention | The autonomous conversational agent (Dora) is configured to deliver calls through a telephone connection as a real-time, stand-alone system: the operator inputs individual patient details to initiate the call and completes a summary in the electronic health record (EHR) afterwards. The entire conversation will be supervised by a clinician who can interrupt if Dora does not collect sufficient information from the patient. All patients who participate in the study will receive the Dora call (one arm). One of the two clinical trial sites (Imperial) sees all of their postoperative cataract patients at a face-to-face out-patient appointment with an ophthalmologist or specialist nurse. The Dora call will be conducted shortly before this appointment, so the results of the Dora call can be compared with the face-to-face appointment to assess whether any complications were correctly identified and if the management plan remains the same. The Oxford University Hospitals NHS Trust does not conduct routine face-to-face follow-up appointments, so there is no direct comparator for that site. For patients at both clinical sites, however, health records will be examined 90 days after the call to determine whether the patient presented to an eye clinic with any complications from surgery, to enable an assessment of DORA’s safety. |
| Intervention type | Other |
| Primary outcome measure | Agreement is measured by inter-rater reliability (the degree of agreement between DORA and the clinician on their assessments of the individual symptoms and the management plan) and whether or not the clinician had to interrupt the call to ask clarifying questions. Measured at 0 days. |
| Secondary outcome measures | 1. Clinical safety measured using complications identified from patients’ electronic health records up to 90 days following cataract surgery, congruence between complications identified and management planned in DORA call and face-to-face follow up within a couple of days of the Dora call (Imperial), and comparison to data from patients attending eye casualty (Oxford) at 0 days 2. Feasibility measured by calculating the proportion of autonomous calls that were completed without needing any intervention from the supervising clinician and reviewing the clinician-reported reasons for asking clarifying questions during the call at 7 days 3. Usability measured by the System Usability Scale and the Telehealth Usability Scale, delivered to patients via online or telephone survey within 2 days of the Dora call 4. Usability, acceptability, satisfaction, and appropriateness measured by semi-structured interviews with a small, stratified random sample of participants, conducted through the intervention period 5. Cost-effectiveness measured by a comparison of the costs of implementing Dora and the costs of the usual standard of care over the duration of the intervention delivery period |
| Overall study start date | 01/03/2020 |
| Overall study end date | 24/03/2022 |
Eligibility
| Participant type(s) | Patient |
|---|---|
| Age group | Adult |
| Lower age limit | 18 Years |
| Sex | Both |
| Target number of participants | 591 |
| Total final enrolment | 225 |
| Participant inclusion criteria | 1. Willing and able to provide informed consent 2. Aged 18 years or older 3. On the waiting list for routine cataract surgery. Cataract surgery as part of a combined procedure with other ocular surgery will not be included 4. No history or presence of significant ocular comorbidities that would be expected to alter the risks of cataract surgery or normal post-operative follow-up schedule. Note that significant ocular comorbidities do not include stable, chronic, or inactive ocular conditions such as amblyopia, drop-controlled stable glaucoma or ocular hypertension, previous squint surgery, inactive macular pathology, previous refractive surgery, or previous vitreoretinal surgery with stable retina |
| Participant exclusion criteria | 1. Individuals with any condition that could preclude the ability to comply with the study or follow-up procedures 2. Presence of ocular or systemic uncontrolled disease (unless deemed not clinically significant by the Investigator and Sponsor) 3. Involved in current research related to this technology or been involved in related research to this technology prior to recruitment 4. Cognitive difficulties, hearing impairment or non-English speakers 5. History of current or severe, unstable or uncontrolled systemic disease (unless deemed not clinically significant by the Investigator and Sponsor) |
| Recruitment start date | 17/09/2021 |
| Recruitment end date | 31/01/2022 |
Locations
Countries of recruitment
- England
- United Kingdom
Study participating centres
153-173 Marylebone Rd
London
NW1 5QH
United Kingdom
John Radcliffe Hospital
Headley Way
Headington
Oxford
OX3 9DU
United Kingdom
Sponsor information
University/education
Exhibition Rd
South Kensington
London
SW7 2BX
England
United Kingdom
| Phone | +44 (0)20 7589 5111 |
|---|---|
| e.normando@imperial.ac.uk | |
| Website | http://www.imperial.edu/ |
"ROR" |
https://ror.org/041kmwe10 |
Funders
Funder type
Government
Government organisation / National government
- Alternative name(s)
- National Institute for Health Research, NIHR Research, NIHRresearch, NIHR - National Institute for Health Research, NIHR (The National Institute for Health and Care Research), NIHR
- Location
- United Kingdom
Results and Publications
| Intention to publish date | 01/08/2023 |
|---|---|
| Individual participant data (IPD) Intention to share | Yes |
| IPD sharing plan summary | Stored in non-publicly available repository, Not expected to be made available |
| Publication and dissemination plan | The project's two clinical sites are leading medical centres with strong reputations for high quality patient care. They have large teams of clinicians, with extensive professional networks and training programmes that involve junior staff from across the globe. These factors mean that their participation in this project will ensure its outputs are communicated to a wide audience. The project’s academic partners are world renowned institutions with a track record of delivery and publication of high-quality research. Senior clinical academics will ensure the project is conducted robustly and even at this early stage the outputs are presented and published in high impact journals. |
| IPD sharing plan | Audio recordings, transcriptions, and meta-data about the calls will be securely stored in UK data centres with strict role-based access control. The transcription service will only have reference to the unique IDs and the audio recording will be reviewed by the PI to remove any identifying information before being shared. Patient identifiable data will not be sold to any other party and will not be shared with any organisation unless they are a partner in the study and have an appropriate information sharing agreement in place. Records of consent will be kept for 3 years after the publication of final study results, but no other personally identifiable information will be stored beyond the end of the study. |
Study outputs
| Output type | Details | Date created | Date added | Peer reviewed? | Patient-facing? |
|---|---|---|---|---|---|
| Participant information sheet | version 3.0 | 08/11/2021 | No | Yes | |
| Protocol file | version 1.0 | 17/05/2021 | 08/11/2021 | No | No |
| HRA research summary | 28/06/2023 | No | No | ||
| Basic results | 22/11/2023 | No | No | ||
| Results article | 03/07/2024 | 29/07/2024 | Yes | No |
Additional files
Editorial Notes
29/07/2024: Publication reference added.
22/11/2023: Basic results uploaded.
03/05/2023: The intention to publish date was changed from 01/05/2023 to 01/08/2023.
14/02/2023: The intention to publish date has been changed from 01/02/2023 to 01/05/2023.
10/10/2022: The overall trial end date was changed from 01/10/2022 to 24/03/2022.
15/06/2022: The following changes have been made:
1. The recruitment end date has been changed from 06/03/2022 to 31/01/2022.
2. The overall trial end date has been changed from 01/05/2022 to 01/10/2022 and the plain English summary updated accordingly.
3. The intention to publish date has been changed from 01/06/2022 to 01/02/2023.
07/03/2022: The following changes were made to the trial record:
1. The total final enrolment was added.
2. The ClinicalTrials.gov number was added.
08/11/2021: Trial's existence confirmed by the Health Research Authority.
