Real-world investigation of the use of a mental health chatbot app while waiting for mental health services
| ISRCTN | ISRCTN14644939 |
|---|---|
| DOI | https://doi.org/10.1186/ISRCTN14644939 |
| IRAS number | 310377 |
| ClinicalTrials.gov number | NCT05533190 |
| Secondary identifying numbers | CPMS 52808, IRAS 310377 |
- Submission date
- 07/12/2022
- Registration date
- 17/01/2023
- Last edited
- 22/11/2023
- Recruitment status
- No longer recruiting
- Overall study status
- Completed
- Condition category
- Mental and Behavioural Disorders
Plain English summary of protocol
Background and study aims
Mental health concerns can have a large impact on individuals, healthcare systems, and the economy. Over a million people are referred to UK mental health services each year, but many have to wait a long time for assessment and treatment and receive limited sessions. This study will examine the impact of a digital health app - called Wysa - on symptoms of depression and anxiety for people waiting for UK mental health services.
Who can participate?
Adults who can speak and understand English and have been referred or have referred themselves to the Improving Access to Psychological Therapies (IAPT) pathway
What does the study involve?
After agreeing to participate, patients complete a set of surveys assessing baseline symptoms of depression, anxiety, and health-related quality of life. All participants will then be randomly assigned to either receive Wysa immediately for 3 months or to wait 3 months to receive Wysa. At the end of the 3 months, all participants will complete the same questionnaires. A small group of participants will also be invited to take part in an interview to discuss their experiences with Wysa.
What are the possible benefits and risks of participating?
Using Wysa has the potential to reduce symptoms of depression and anxiety. Participants assigned to the group that has access to Wysa will receive immediate mental health and well-being support from the app, in addition to the standard treatment they will receive in the IAPT service. This research will also benefit patients in the future.
Potential risks are minimized by providing Wysa as an addition to appointments with IAPT, so all participants will still be assessed and treated in line with current NHS and IAPT guidance. There may be occasions where Wysa doesn’t understand what a user is saying or might just keep giving the same answer. This could be frustrating, but we will use this information to help improve Wysa in the future.
All research projects carry some risk that might not be known about before the research project starts, but the team has done everything it can to help make sure these do not happen. The main risk of this research project is that it could increase participants' level of distress - such as if using Wysa is causing frustration and anxiety. Participants can withdraw from the project at any time if they feel it is making their symptoms worse. There is also a risk of the physical impact of increased reliance on mobile phone screen time, which could be mitigated through mobile phone screen access time controls. There is also a risk of storage of potentially identifiable conversations with Wysa, for example, if participants' were to reference their full name or the name of others when interacting with Wysa. There is a procedure which is run on the system to prevent the storage of such information, but should this occur and information cannot be deleted, a notification will be made on the participants' clinical record providing them with the ability to review this information and ask for its removal from Wysa. If Wysa detects language or phrases which have been flagged as indicative of risk of self-harm, suicidality, harm from others or harm to others, Wysa will clarify if it has understood the participant's meaning correctly. If it has, Wysa will trigger the SOS pathway. This pathway can also be manually triggered at any time by clicking on the SOS button on the top right side of the screen and will provide links to local crisis helpline numbers and processes. It will remind participants that Wysa is an AI chatbot and is unable to help during emergencies, but that participants can call these numbers for immediate support.
Where is the study run from?
The study is based out of the Central North West London NHS Trust (UK) and the data analysis is being led by the University of Plymouth (UK)
When is the study starting and how long is it expected to run for?
July 2021 to July 2023
Who is funding the study?
National Institute for Health and Care Research (NIHR) Artificial Intelligence in Health and Care Award (UK)
Who is the main contact?
Prof Edward Meinert (University of Plymouth), edward.meinert@plymouth.ac.uk (UK)
Emma Selby (Wysa), emma@wysa.ai (UK)
Contact information
Principal Investigator
Centre for Health Technology
University of Plymouth
Plymouth
PL4 6DN
United Kingdom
 ORCID ID |
0000-0003-2484-3347 |
|---|---|
| Phone | +44 (0)1752 600600 |
| edward.meinert@plymouth.ac.uk |
Scientific
Centre for Health Technology
University of Plymouth
Plymouth
PL4 6DN
United Kingdom
| Phone | +44 (0)1752 600600 |
|---|---|
| edward.meinert@plymouth.ac.uk |
Public
Wysa
22 Wenlock Road
London
N1 7GU
United Kingdom
| Phone | None provided |
|---|---|
| emma@wysa.ai |
Study information
| Study design | Randomized study |
|---|---|
| Primary study design | Interventional |
| Secondary study design | Randomised controlled trial |
| Study setting(s) | Home |
| Study type | Treatment |
| Participant information sheet | 42867_PIS_v1.1_.pdf |
| Scientific title | Real-world randomised controlled trial of an artificial intelligence-enabled app as an early intervention and support tool in the mental health referral care pathway |
| Study objectives | Patients awaiting mental health services who can use an AI-enabled self-help and triage app (Wysa) are more likely to have reduced symptoms of depression and anxiety compared to patients without access. |
| Ethics approval(s) | Approved 25/05/2022, London - Stanmore Research Ethics Committee (2nd Floor, 2 Redman Place, Stratford, London, E20 1JQ, UK; +44 (0)207 104 8387, (0)207 104 8263; stanmore.rec@hra.nhs.uk), ref: 22/PR/0467 |
| Health condition(s) or problem(s) studied | Mental health |
| Intervention | The study will be a non-blinded randomised controlled trial and mixed-methods evaluation. We are conducting a randomised controlled trial to compare whether Wysa can reduce symptoms of depression and anxiety better than the current standard of care. The mixed-methods evaluation will enable us to gain more in-depth feedback about users' experiences with the Wysa app. The study will last 17 months: 5 months of preparation, 9 months of implementation and follow-up, and 3 months of post-evaluation analysis and preparation for sharing results. We plan to recruit 480 individuals over 18 years old who are referred, or self-refer, to the Improving Access to Psychological Therapies (IAPT) service for mental health support. People will be screened for eligibility by a psychology assistant, emailed information about the study, and then called by the assistant to gather informed consent. After consenting, participants will be randomly assigned to the intervention group or a waitlist control group by a computer algorithm. Participants in the intervention group will be given immediate access to Wysa and participants in the control group will receive access after 3 months. Outcomes will be measured at 3 months post-randomisation in both groups. The researchers evaluating clinical outcomes will be blinded to the participants' groups. A random subset of participants (stratified based on demographics) will also be invited to participate in an interview about their experience with Wysa. These interviews will last no more than 40-60 minutes and will take place over the phone or via video-calling software. The study will not alter the current standard of care, as the intervention will be delivered in addition to existing clinical pathways. |
| Intervention type | Behavioural |
| Primary outcome measure | 1. Depression severity measured using the Patient Health Questionnaire-9 (PHQ-9) at baseline and 3 months |
| Secondary outcome measures | 1. Anxiety severity measured using the Generalised Anxiety Disorder Assessment (GAD-7) at baseline and 3 months 2. Crisis identification measured using the number of users identified by the app for escalation of care compared to the number of patients in the control group who access accident and emergency or out-of-hours services while waiting for treatment over the 9-month intervention period 3. Uptake rates of the app by participants randomised into the intervention group measured using app download and log in data over the 9-month intervention period 4. Dropout rates of participants who are randomised into the intervention group and start using the app measured using app download and log in data over the 9-month intervention period 5. Frequency and duration of app use measured using app metrics over the 9-month intervention period 6. Qualitative feedback about engagement with the app and its acceptability will be assessed in a subset of participants measured using semi-structured interviews; randomly selected participants will be interviewed once for 40-60 minutes after their use of the Wysa app during the 9-month intervention period 7. Self-reported acceptability measured through the app using automated review questions periodically requested during the general use of the tool over the 9-month intervention period 8. General health state measured using the EuroQol EQ-5D-5L questionnaire at baseline and 3 months 9. Impact of health on everyday life measured using the Short Form 12 (SF-12) health survey at baseline and 3 months 10. Cost-analysis measured using compiled cost data relating to health and social care service use, medication and treatment use, Wysa's implementation, and patients' personal payments or related productivity losses over the 9-month intervention period |
| Overall study start date | 01/07/2021 |
| Completion date | 20/07/2023 |
Eligibility
| Participant type(s) | Patient |
|---|---|
| Age group | Adult |
| Lower age limit | 18 Years |
| Sex | Both |
| Target number of participants | Planned Sample Size: 480; UK Sample Size: 480 |
| Total final enrolment | 76 |
| Key inclusion criteria | 1. Willing and able to provide informed consent 2. Aged 18 years or older 3. User is confident in their ability to speak and understand English at a proficient level 4. Own a mobile device capable of supporting Wysa 5. A valid email address 6. Referred or self-referred to proceed through the standard IAPT care pathway |
| Key exclusion criteria | 1. Patients ineligible for the standard IAPT care pathway 2. Patients with previous and current known major mental illness such as Schizophrenia, severe depression, any co-morbid neurological or neuro-psychiatric condition such as epilepsy 3. Patients with current psychosis or a history of psychotic symptoms within the last 6 months 4. Patients with suicidal ideation 5. Patients scoring > 15 points on PHQ 9 6. Patients scoring > GAD-7 7. Patients with significant cognitive disorders 8. Patients with noted neurodevelopmental conditions such as autism or ADHD 9. Patients previously diagnosed with a personality disorder 10. Patients who been under the care of CMHT or a specialised mental health services in the last 2 years 11. Patients who failed IAPT previously 12. Patients with referrals for specialist presentations of pre-existing, diagnosed conditions requiring a specialised assessment beyond the standard clinical pathway 13. Incapable of self-consent 14. In a dependent/unequal relationship with the research or care teams or any PPI representatives |
| Date of first enrolment | 16/12/2022 |
| Date of final enrolment | 24/04/2023 |
Locations
Countries of recruitment
- England
- United Kingdom
Study participating centre
350 Euston Road
Regents PLACE
London
NW1 3AX
United Kingdom
Sponsor information
Hospital/treatment centre
Level 2
Marine Building
Drake Circus
Plymouth
PL4 8AA
England
United Kingdom
| Phone | +44 (0)1752588959 |
|---|---|
| plymouth.sponsor@plymouth.ac.uk | |
| Website | https://www.plymouth.ac.uk/ |
"ROR" |
https://ror.org/008n7pv89 |
Funders
Funder type
Government
Government organisation / National government
- Alternative name(s)
- National Institute for Health Research, NIHR Research, NIHRresearch, NIHR - National Institute for Health Research, NIHR (The National Institute for Health and Care Research), NIHR
- Location
- United Kingdom
Results and Publications
| Intention to publish date | 31/08/2024 |
|---|---|
| Individual participant data (IPD) Intention to share | Yes |
| IPD sharing plan summary | Stored in non-publicly available repository |
| Publication and dissemination plan | Planned publication in a high-impact and peer reviewed journal. The protocol is planned to be published in BMJ Open by Spring 2023. |
| IPD sharing plan | The datasets generated during and/or analysed during the current study will be stored in a non-publicly available repository (the University of Plymouth archive servers). Anonymised data will be made available on request from Professor Edward Meinert (edward.meinert@plymouth.ac.uk) subject to a data use agreement between the Contractor and the third party requesting the data. If any participant does not consent to the sharing of their anonymised data for other research purposes, that data will be removed from the dataset before it is made available to other researchers, and the file will be noted accordingly to indicate that it is a subset of the initial dataset. |
Study outputs
| Output type | Details | Date created | Date added | Peer reviewed? | Patient-facing? |
|---|---|---|---|---|---|
| Participant information sheet | version 1.1 | 17/01/2023 | No | Yes | |
| Protocol (preprint) | version 1.1 | 29/09/2022 | 17/01/2023 | No | No |
| HRA research summary | 28/06/2023 | No | No | ||
| Basic results | 22/11/2023 | No | No |
Additional files
Editorial Notes
22/11/2023: Basic results uploaded.
17/10/2023: The overall end date was changed from 31/10/2023 to 20/07/2023. Total final enrolment added.
23/03/2023: The following changes were made to the trial record:
1. The recruitment end date was changed from 31/03/2023 to 24/04/2023.
2. The overall end date was changed from 31/08/2023 to 31/10/2023.
3. The plain English summary was updated to reflect these changes.
07/12/2022: Trial's existence confirmed by the NIHR.
