Why do chatbots work? A randomized controlled trial to test the effect of behaviorally-informed chatbot functionalities on vaccine uptake
| ISRCTN | ISRCTN14701620 |
|---|---|
| DOI | https://doi.org/10.1186/ISRCTN14701620 |
| Secondary identifying numbers | 2023022 |
- Submission date
- 10/06/2024
- Registration date
- 13/06/2024
- Last edited
- 08/08/2024
- Recruitment status
- No longer recruiting
- Overall study status
- Completed
- Condition category
- Infections and Infestations
Plain English Summary
Background and study aims
Despite overwhelming medical evidence on the protection offered by second, third and fourth COVID-19 vaccine doses, uptake of these doses remain low despite available supply. Similarly, despite evidence of the protection offered by the influenza (flu) vaccine, vaccine uptake remains low despite available supply. There is particularly high risk for people living with risk factors and people older than 65 years.
This study will design and implement an intervention to increase immunization behaviors amongst the general adult population eligible for influenza vaccines and COVID-19 booster doses in Tucuman, Argentina. It will create a WhatsApp chatbot with personalized messages informing participants they are eligible for the next dose, sharing practical information on where and when to get vaccinated, providing planning prompts and enabling them to set their own reminders to get their next dose of the COVID-19 or flu vaccine on a convenient date.
This is a large-scale randomized controlled trial to assess the impact of behaviourally-informed chatbot features on COVID-19 and influenza vaccination as measured through Ministry of Health data.
Who can participate?
Residents of Tucuman aged 18 years or older who are eligible for either their next COVID-19 dose or a government-subsidized flu vaccine, and have an active WhatsApp account. COVID-19 vaccine eligibility includes individuals aged 18-49 years who have not received a dose in the last 12 months and those aged 50+ years or with health risk factors who have not received a dose in the last 6 months. Flu vaccine eligibility includes individuals aged 65+ years, those with health risk factors, health and security personnel, and those with a prior prescription for the flu vaccine. Additionally, participants have a unique, validly formatted mobile phone in the Tucumán Ministry of Health databases.
What does the study involve?
Participants are randomly assigned at the individual level to one of four groups in equal proportions. Three of these groups receive a WhatsApp chatbot with different added functionalities, while a control group receives no messages. Participants in treatment 1 (T1) are sent a Whatsapp chatbot with a personalized Whatsapp message reminding them of their eligibility to get their next COVID-19 and/or flu vaccine, a prompt to plan a day and time to receive their vaccine(s), and the option to set a reminder message for a day before their selected vaccination day. Participants treatment 2 (T2) are sent a Whatsapp chatbot including all T1 functionalities plus the option to locate a health center to receive their vaccine and set a reminder that includes a Google Maps link from their current location to the selected center, in addition to T1 functionalities. Participants in treatment 3 are sent a Whatsapp chatbot including all T2 functionalities plus a video from a popular local artist encouraging vaccination. Across all treatments, participants can choose to reply to the chatbot’s messages by interacting with a set menu of options. Those who choose not to interact with the chatbot do not receive any future messages. Participants in the control group do not receive any messages.
What are the possible benefits and risks of participating?
Participants may benefit from receiving personalized information about Influenza and COVID-19 vaccinations, aimed at promoting vaccination uptake. Major risks are not anticipated and the risk of lower vaccination outcomes was minimized by developing messages based on participatory research in Argentina.
Where is the study run from?
Tucuman Ministry of Public Health (Argentina)
When is the study starting and how long is it expected to run for?
January 2023 to September 2024
Who is funding the study?
Vaccine Confidence Fund (USA)
Who is the main contact?
1. Adelaida Barrera, adelaida.barrera@bi.team
2. Sebastian Salomon-Ballada, sebastian.salomon.b@gmail.com
Contact information
Scientific, Principal Investigator
58 Victoria Embankment
London
EC4Y 0DS
United Kingdom
| sebastian.salomon@bi.team |
Public, Principal Investigator
58 Victoria Embankment
London
EC4Y 0DS
United Kingdom
| adelaida.barrera@bi.team |
Principal Investigator
58 Victoria Embankment
London
EC4Y 0DS
United Kingdom
| rosario.ramos@bi.team |
Principal Investigator
58 Victoria Embankment
London
EC4Y 0DS
United Kingdom
| ivanbudassi@gmail.com |
Study information
| Study design | Four-arm randomized controlled trial |
|---|---|
| Primary study design | Interventional |
| Secondary study design | Randomised controlled trial |
| Study setting(s) | Community |
| Study type | Efficacy |
| Participant information sheet | Not available in web format, please use the contact details to request a participant information sheet |
| Scientific title | Why do chatbots work? A randomized controlled trial to test the effect of behaviorally-informed chatbot functionalities on vaccine uptake |
| Study hypothesis | The researchers are aiming to reject the following hypotheses that the difference between COVID-19 or flu vaccination rate over a 4-week period is statistically indistinguishable among: Primary analysis: H1: Participants who are sent a WhatsApp chatbot (T1-T2-T3) compared to those who are assigned to not receive any message Secondary analysis: H2: Participants who are sent a basic Whatsapp chatbot (T1) compared to those who are sent the same chatbot with an additional health center locator functionality (T2) H3: Participants who are sent the T2 Whatsapp chatbot compared to those who are sent the same chatbot with an additional motivational video message featuring a popular figure (T3) Robustness check: H4: Participants who receive a basic Whatsapp chatbot (T1) compared to those who receive the same chatbot with an additional health center locator functionality (T2) H5: Participants who receive the T2 Whatsapp chatbot compared to those who receive the same chatbot with an additional motivational video message featuring a popular figure (T3) |
| Ethics approval(s) |
Approved 21/06/2023, Comité de Bioética, Hospital Universitario Fundación Favaloro (Bioethics committee, Favaloro Foundation University Hospital) (Av. Belgrano 1746, Buenos Aires, C1095, Argentina; +54 (0)11 4378 1229; comitedebioetica@ffavaloro.org), ref: CBE 962-22 Acta 172 |
| Condition | Promotion of COVID-19 and influenza vaccination in the general population |
| Intervention | The unit of randomisation is the individual as identified by an anonymized ID called 'patient ID'. Using a random number generator, the researchers assigned participants to one of four groups. Participants in the control group won’t be sent any messages. However, following the trial period, if any messages are found to be effective at encouraging vaccination, these individuals would be sent the most effective message. Participants in treatment 1 will be sent a WhatsApp chatbot that: 1. Sends them a personalized WhatsApp message reminding them to get their next COVID-19 and/or flu vaccine 2. Prompts them to plan a day and time to receive their vaccine(s) 3. Reminds them a day before their selected vaccination day Participants in treatment 2 will be sent a WhatsApp chatbot that includes all T2 functionalities and: 1. Helps them locate a health center where they can receive their vaccine(s) 2. Reminds them a day before their selected vaccination day, including a Google Maps link to get from their current location to the selected health center Participants in treatment 3 will be sent a WhatsApp chatbot that includes all T3 functionalities and: 1. Sends a video by the popular band La Mosca from Argentina to complement the WhatsApp messages |
| Intervention type | Behavioural |
| Primary outcome measure | 'Next dose’ COVID-19 or Flu Vaccination Rate, a binary indicator of whether the individual receives a vaccination within 4 weeks of the assigned data for the sending of the initial message (COVID-19 or flu) (1 if they do, 0 if not) |
| Secondary outcome measures | Effect of chatbot functionalities on ‘Next dose’ COVID-19 or flu vaccination rate, a binary indicator of whether the individual receives a vaccination within 4 weeks of the assigned data for the sending of the initial message (COVID-19 or flu) (1 if they do, 0 if not) |
| Overall study start date | 23/01/2023 |
| Overall study end date | 30/09/2024 |
Eligibility
| Participant type(s) | Population |
|---|---|
| Age group | Adult |
| Lower age limit | 18 Years |
| Sex | Both |
| Target number of participants | 295,155 |
| Participant inclusion criteria | Current participant inclusion criteria as of 05/08/2024: 1. Is 18 years of age or older 2. Is a resident of the province of Tucuman 3. Is eligible to receive their next COVID-19 dose or a government-subsidized flu vaccine: 3.1. COVID-19 vaccine eligibility criteria: 3.1.1. Individual is between 18 and 49 years old and is eligible for their booster shot based on time since last dose, i.e. has not had a COVID-19 shot in the last 12 months 3.1.2. Individual is 50 years old or older or has health risk factors and has not had a COVID-19 shot in the last 6 months 3.2. Government-subsidized flu vaccine eligibility criteria: 3.2.1. Citizens who are 65 years old or above 3.2.2. Citizens with a health risk factor 3.2.3. Health/security personnel 3.2.4. Citizens who had a medical prescription to receive the flu vaccine before 4. Mobile number is unique to one individual within the database 5. Mobile number has a valid format in Argentina 6. The user has an active WhatsApp account Previous participant inclusion criteria: 1. Is 18 years of age or older 2. Is a resident of the province of Tucuman 3. Is eligible to receive their next COVID-19 dose or a government-subsidized flu vaccine: 3.1. COVID-19 vaccine eligibility criteria: 3.1.1. Individual is between 18 and 49 years old and is eligible for their booster shot based on time since last dose, i.e. has not had a COVID-19 shot in the last 12 months 3.1.2. Individual is 50 years old or older or has health risk factors and has not had a COVID-19 shot in the last 6 months 3.2. Government-subsidized flu vaccine eligibility criteria: 3.2.1. Citizens who are 65 years old or above 3.2.2. Citizens with a health risk factor 3.2.3. Health/security personnel 3.2.4. Citizens who had a medical prescription to receive the flu vaccine before 4. Mobile number is unique to one individual within the database 5. Mobile number has a valid format in Argentina |
| Participant exclusion criteria | Does not meet the inclusion criteria |
| Recruitment start date | 07/08/2024 |
| Recruitment end date | 30/08/2024 |
Locations
Countries of recruitment
- Argentina
Study participating centre
San Miguel de Tucumán
T4109ADB
Argentina
Sponsor information
Research organisation
58 Victoria Embankment
London
EC4Y 0DS
England
United Kingdom
| dpo@bi.team | |
| Website | https://www.bi.team/ |
"ROR" |
https://ror.org/03mk5b468 |
Funders
Funder type
Charity
Private sector organisation / Other non-profit organizations
- Alternative name(s)
- VCF
- Location
- United States of America
Results and Publications
| Intention to publish date | 31/08/2025 |
|---|---|
| Individual participant data (IPD) Intention to share | Yes |
| IPD sharing plan summary | Stored in publicly available repository, Data sharing statement to be made available at a later date |
| Publication and dissemination plan | Planned publication in a peer-reviewed journal. |
| IPD sharing plan | The researchers intend for anonymity-preserving data used in the analysis to be made publicly available in a publicly available repository. Further details regarding data sharing, including repository name and link, will be made available at a later date. |
Study outputs
| Output type | Details | Date created | Date added | Peer reviewed? | Patient-facing? |
|---|---|---|---|---|---|
| Protocol file | 11/06/2024 | 13/06/2024 | No | No | |
| Statistical Analysis Plan | SAP included in protocol | 11/06/2024 | 08/08/2024 | No | No |
Additional files
- 45593_PROTOCOL_11Jun24.pdf
- 45593_PROTOCOL_11Jun24.pdf
- SAP included in protocol
Editorial Notes
08/08/2024: Statistical analysis plan uploaded.
08/08/2024: The recruitment start date was changed from 10/06/2024 to 07/08/2024.
05/08/2024: The following changes were made:
1. The overall study end date was changed from 31/08/2024 to 30/09/2024.
2. The participant inclusion criteria were changed.
3. The target number of participants was changed from 419,251 to 295,155.
4. The recruitment end date was changed from 30/06/2024 to 30/08/2024.
10/06/2024: Study's existence confirmed by the Bioethics committee, Favaloro Foundation University Hospital.
