Supporting the mental wellbeing of children through interactions with a chatbot
| ISRCTN | ISRCTN17178640 |
|---|---|
| DOI | https://doi.org/10.1186/ISRCTN17178640 |
| Secondary identifying numbers | TUD-EUR-Erasmus MC |
- Submission date
- 08/03/2022
- Registration date
- 15/03/2022
- Last edited
- 06/04/2023
- Recruitment status
- No longer recruiting
- Overall study status
- Completed
- Condition category
- Mental and Behavioural Disorders
Plain English summary of protocol
Background and study aims
Anxiety disorders are one of the most frequently identified mental health issues in children and adolescents. Symptoms of anxiety and mood problems often emerge in adolescence, but they frequently go unrecognized or untreated. Persistently increased anxiety levels in adolescents can impede academic performance and social and emotional functioning.
One assessment method that has been offered using mHealth technology is the experience sampling method (ESM). ESM enables the measurement of subjective experiences, such as thoughts, feelings, and behavior, in individuals’ daily lives.
The purpose of this study is to compare the beneficial effects of a conversational agent ('chatbot') to that of traditional questionnaires. We want to see if participants (both adults and adolescents) in the conversational agent group fill out more questionnaires (compliance) and have a better user experience than those in the traditional questionnaires group. Both adults and adolescents will be randomly assigned to one of these groups.
Who can participate?
English-speaking volunteers aged 18 years or older can participate in the initial study.
Dutch-speaking volunteers aged 10 - 18 years can participate in the main study.
What does the study involve?
This study consists of two stages. In the first stage, an evaluation of the two applications, conversational agent-based ESM (C-ESM) and typical app-based ESM (T-ESM), will be undertaken online with adults utilizing the Prolific crowdsourcing platform. We will use the adult study's feedback to improve the prototype applications, ensuring that any problems or poor user experiences are addressed before conducting research with the adolescent. A total of 400 participants will be randomly assigned to the C-ESM or the T-ESM. Adult and adolescent studies will follow the same study protocol.
In the second stage, we will evaluate the effectiveness of the C-ESM with 128 adolescents (10-18 yrs) from the general population. Adolescents will be randomly allocated to a C- ESMor T- ESM. ESM is delivered for two weeks at three random times a day. ESM questions will be adopted from the validated questionnaires, including engagement, self-reflection, compatibility with personal values, acceptability, usefulness, and user experience. We will calculate the compliance and retention rate post-hoc by analyzing the data.
Participants in both groups need to accept General Data Protection Regulation (GDPR) consent form before signing into ESM apps. To ensure that participants comprehend the study's objectives and how to utilize our applications, they will receive interactive instructions. We obtained separate ethical approval for the adolescent and adult study from an Ethical Review Board of Delft university of technology
What are the possible benefits and risks of participating?
Participants will gain more insight into their feelings and behavior through our proposed chatbot-based intervention, which has been shown to be more effective and engaging for participants than traditional web or mobile-based interventions. We do not anticipate any risks associated with using the applications.
Where is the study run from?
Delft University of Technology (the Netherlands)
When is the study starting and how long is it expected to run for?
October 2021 to December 2023
Who is funding the study?
OPEN MIND (Netherlands)
Who is the main contact?
Tahir Abbas, t.abbas-1@tudelft.nl
Contact information
Principal Investigator
Web Information Systems
Building 28
Delft University of Technology
Delft
2628 XE Delft
Netherlands
 ORCID ID |
0000-0002-0558-6106 |
|---|---|
| Phone | +31 657175589 |
| t.abbas-1@tudelft.nl |
Principal Investigator
Faculty of Technology, Policy and Management
Building 31
Delft University of Technology
Delft
2628 BX
Netherlands
| ORCID ID |
0000-0001-5374-1687 |
|---|---|
| Phone | +31 152783956 |
| o.kudina@tudelft.nl |
Scientific
Web Information Systems
Building 28
Delft University of Technology
Delft
2628 XE Delft
Netherlands
| ORCID ID |
0000-0002-6189-6539 |
|---|---|
| Phone | +49 17672309578 |
| u.k.gadiraju@tudelft.nl |
Scientific
dr. Molewaterplein 40
Rotterdam
3000 CA
Netherlands
| Phone | +31 10-7040209 |
|---|---|
| j.s.legerstee@erasmusmc.nl |
Study information
| Study design | Parallel multi-arm pilot randomized controlled trial |
|---|---|
| Primary study design | Interventional |
| Secondary study design | Randomised parallel trial |
| Study setting(s) | Other |
| Study type | Screening |
| Participant information sheet | Not available in web format, please use the contact details to request a patient information sheet. |
| Scientific title | Value-sensitive conversational agents to support mental wellbeing in children: an experience sampling approach |
| Study objectives | H1: C-ESM will increase the compliance compared to the T-ESM app H2: C-ESM application will have a greater retention rate than T-ESM application. H3: C-ESM app will lead to significantly higher worker engagement than T-ESM app. H4: C-ESM will be more compatible with personal values than T-ESM, resulting in a positive effect on thoughts regarding its usefulness and trustworthiness. H5: Users who interact with the C-ESM app have a better user experience than users who interact with the T-ESM app. H6: Those that interact with the C-ESM will engage in a greater amount of self-reflection than participants in the control group. |
| Ethics approval(s) | 1. Approved 04/10/2021, Human Research Ethics Committee, TU Delft, P.O. Box 5015, 2600 GA, Delft, The Netherlands; +31 152783260; j.b.j.grootkormelink@tudelft.nl), ref: Ir. J.B.J. Groot Kormelink, secretary HREC 2. Approved 15/10/2021, Human Research Ethics Committee, TU Delft, P.O. Box 5015, 2600 GA, Delft, The Netherlands; +31 152783260; j.b.j.grootkormelink@tudelft.nl), ref: Ir. J.B.J. Groot Kormelink, secretary HREC |
| Health condition(s) or problem(s) studied | Identifying and preventing the development of emotional problems in adolescents |
| Intervention | Randomisation: This study aims to examine two digital interventions for adolescents who may be experiencing emotional or depressive symptoms (and adults in the initial formative study). We will use randomization to assign participants to one of two arms: intervention or control. The randomization process will be carried out using the Qualtrics randomizer, which employs block randomization to ensure balanced groups. Intervention group: The participants in the intervention group will be asked to engage with a mobile application that uses a “conversational interface” to deliver ESM to tap into the daily life symptoms experiences of adolescents. This intervention is referred to as C-ESM. The program will be offered through an application that participants may download to their smartphone or tablet and access at any time through access codes. All questions on emotions, sleep quality, and mood in the C-ESM will be presented in a conversational way, with participants interacting through quick replies or by answering open-ended questions to elaborate on their feelings. A particular emphasis has been made on making this interaction delightful and exciting for the participants. Control Group: This application, which we will utilize in the control group, will be comparable to the C-ESM in terms of content, information flow, and notifications, but will transmit the material through a normal mobile app format (radio buttons, checkboxes, text fields). The program will be delivered through a mobile application that users may download to their smartphone or tablet. For the adolescent study, we plan to use schools and teachers as contact points (building on a similar previous studies) to recruit the pupils and disseminate information sheets about the study, embedded as the first pages of the Informed Consent form. Unless the students are 16 y.o., a parent or guardian must sign the Informed Consent form next to the pupil to be able to take part in the study. For two weeks, participants will be asked to complete surveys that will be sent to them via the ESM applications at various times throughout the day. The first notification will be sent randomly between 9 and 10 a.m. via a beep. The participant will be prompted to complete the ESM questionnaire after receiving this notification. The morning questionnaire will assess both the quality and amount of sleep by asking questions such as "What time did you go to bed?" and "What time did you wake up?" and a few broad questions concerning the participant's social connectedness and emotions, such as "Who have you been in contact with online in the last 5 minutes?" and "how are you feeling now?" Between 11 a.m. and 7 p.m., the second notification will be delivered at random. The questions about social connectedness and emotions will be repeated in the midday questionnaire. Between 8 and 9 p.m., the final notification will be sent. Participants will be asked to record their unpleasant/bad experiences in the evening questionnaire, as well as to briefly describe the negative incident and its overall severity (e.g., “Everyone is going through unpleasant things every day, small or large. Now think of what you experienced today for negatives”). Following each notification, a reminder will be delivered 10 minutes and 40 minutes later. The ESM questionnaire will remain open for 45 minutes after each notification. Weekend alerts will be sent with a one-hour delay. After two weeks, we will ask them to complete questionnaires about self-reflection, engagement, compatibility with personal values, acceptability, usefulness, and user experience, as well as compliance and retention rates, which will be measured post-hoc. We will employ statistical significance tests to assess our hypotheses and get a better understanding of the effect of our proposed treatments on self-reporting measures. Additionally, we will solicit participants' general impressions of the application and its usefulness. As a follow-up study, we will conduct two focus groups, each 5-6 pupils from the study, to qualitatively understand their experiences of the study, the usability of the chatbot and the website, any challenges or discoveries that went in this regard. This will help to provide an encompassing quantitative and qualitative assessment of the technological interfaces delivering ESM questionnaires, assess their effectiveness and compliance rates better. |
| Intervention type | Behavioural |
| Primary outcome measure | Compliance rate: the total number of completed responses divided by the total number of times the questionnaires are presented. The total number of questionnaires also includes instances when users opt not to respond to the questionnaires. Measured at the end of the study |
| Secondary outcome measures | 1. Self-reflection measured through a multiple choice question measured at follow up 2. The retention rate is the proportion (or percentage) of participants who are included in the final analysis. Subjects who leave from a study, for example, because the data collection technique is perceived to be excessively difficult, are excluded. Measured at the end of the study. 3. User engagement measured using the User Engagement Scale (UES) at follow up 4. Compatibility of app use with personal values: The compatibility of personal values and technology use will be assessed by the measure proposed by Elena Karahann at follow up 5. Perceived usefulness measured using a four-item scale at follow up |
| Overall study start date | 04/10/2021 |
| Completion date | 01/12/2023 |
Eligibility
| Participant type(s) | Healthy volunteer |
|---|---|
| Age group | Mixed |
| Lower age limit | 18 Years |
| Sex | Both |
| Target number of participants | Formative Study: total 400 (Intervention: 200, control: 200) Adolescent study: total 128 (Intervention: 64, control: 64) |
| Key inclusion criteria | Formative study: 1. Adult males/females (18 years or older) 2. Must have access to the internet 3. Must have a smartphone that allows for the downloading of mobile applications 4. Able to consent to participate 5. Able to understand and comprehend English Main study with adolescents: 1. Adolescents (from 10-18 years old) 2. Must live in the Netherlands 3. Must be able to understand Dutch 4. Must be able to consent to participate or their parents need to provide consent. 5. Must have access to the internet 6. Must have a smartphone that allows for the downloading of mobile applications |
| Key exclusion criteria | 1. In adolescent research, participants must NOT be younger than ten years old and older than 18 years old; in the formative study, participants must NOT be younger than eighteen years old and greater than 60 years old. 2. Incapable of providing permission to participate 3. Incapable of comprehending English (for purpose of formative study) and Dutch (for adolescent study) |
| Date of first enrolment | 01/01/2023 |
| Date of final enrolment | 01/08/2023 |
Locations
Countries of recruitment
- Netherlands
- United Kingdom
- United States of America
Study participating centres
3015 CN
Netherlands
2600 AA Delft
The Netherlands
Delft
2600 AA
Netherlands
Sponsor information
University/education
PO Box 5
2600 AA Delft
The Netherlands
Delft
2600 AA
Netherlands
| Phone | +31 152789111 |
|---|---|
| info@tudelft.nl | |
| Website | http://www.tudelft.nl/en/ |
"ROR" |
https://ror.org/02e2c7k09 |
Hospital/treatment centre
dr. Molewaterplein 40
Rotterdam
3000 CA
Netherlands
| Phone | +31 10 704 0 704 |
|---|---|
| info@erasmusmc.nl | |
| Website | http://www.erasmusmc.nl/ |
Funders
Funder type
Charity
No information available
Results and Publications
| Intention to publish date | 01/07/2024 |
|---|---|
| Individual participant data (IPD) Intention to share | No |
| IPD sharing plan summary | Not expected to be made available |
| Publication and dissemination plan | Planned publication in a high-impact peer-reviewed journal. |
| IPD sharing plan | The datasets generated during and/or analysed during the current study are not expected to be made available |
Editorial Notes
06/04/2023: The following changes have been made:
1. The recruitment end date has been changed from 01/04/2023 to 01/08/2023.
2. The overall trial end date has been changed from 01/07/2023 to 01/12/2023 and the plain English summary updated accordingly.
23/12/2022: The following changes were made to the trial record:
1. The recruitment start date was changed from 01/07/2022 to 01/01/2023.
2. The recruitment end date was changed from 01/09/2022 to 01/04/2023.
3. The overall end date was changed from 01/12/2022 to 01/07/2023.
4. The intention to publish date was changed from 01/12/2022 to 01/07/2024.
5. The plain English summary was updated to reflect these changes.
08/06/2022: The recruitment start date has been changed from 01/06/2022 to 01/07/2022.
11/04/2022: The recruitment start date has been changed from 01/04/2022 to 01/06/2022.
14/03/2022: Trial's existence confirmed by TU Delft.
