Assessing a suite of digital health interventions to improve immunisation services in Pakistan
| ISRCTN | ISRCTN23078223 |
|---|---|
| DOI | https://doi.org/10.1186/ISRCTN23078223 |
- Submission date
- 10/11/2021
- Registration date
- 15/11/2021
- Last edited
- 21/01/2025
- Recruitment status
- No longer recruiting
- Overall study status
- Completed
- Condition category
- Other
Prospectively registered
Protocol
Statistical analysis plan
Results
Individual participant data
Plain English Summary
Background and study aims
Zindagi Mehfooz (ZM) is a suite of tools used to capture data when people arrive at primary health care facilities for immunisation services. Children are registered in the system and each vaccination given is recorded within the application at the first visit and each subsequent visit. To remind caregivers of when they need to next visit the clinic, text message reminders are sent up to 3 automatic personalized SMS reminders a day before, on the day of, and six days after the scheduled immunisation date (if the child failed to show up for his appointment). A child is considered to have completed the full course of immunisations after the second dose of measles. All data inputted into the ZM app are collated on a central web-based dashboard which supervisors can use to monitor progress, identify defaulters, and take corrective action to achieve targets.
The aim of the study is to see if the digital health intervention improves how many children complete their immunisation within the recommended time.
Who can participate?
Vaccinators, caregivers, supervisors, managers, and program staff who are involved in the implementation and or use ZM will be invited to participate.
What does the study involve?
The study involves the analysis of data from the ZM system, analysis of data from immunisation surveys, and in-depth interviews and observations with users of the ZM platform.
What are the possible benefits and risks of participating?
There are no risks to participating. This is the first study of this kind and the benefits of the study will be significant for countries that are looking for potential digital tools to improve their immunisation programme performance.
Where is the study run from?
Essential Programme for Immunisation, Sindh Province (Pakistan)
When is the study starting and how long is it expected to run for?
January 2020 to December 2022
Who is funding the study?
The study is funded by Gavi, The Vaccine Alliance (Switzerland) and coordinated by HealthEnabled (South Africa).
Who is the main contact?
Dr Patricia Mechael, patty@healthenabled.org
Contact information
Dr Patricia Mechael
Public
Public
4101 Albemarle St NW
Apt 521
Washington
20016
United States of America
 ORCID ID |
0000-0002-1034-0549 |
|---|---|
| Phone | +1 9732228252 |
| patty@healthenabled.org |
Study information
| Study design | Large-scale cost-effectiveness intervention study with pre-post design |
|---|---|
| Primary study design | Interventional |
| Secondary study design | Non randomised study |
| Study setting(s) | GP practice |
| Study type | Prevention |
| Participant information sheet | Not available in web format, please use contact details to request a participant information sheet. |
| Scientific title | Evaluating the impact and cost-effectiveness of Zindagi Mehfooz, a suite of digital health interventions, to improve the coverage, timeliness, and completeness of immunisation services in Pakistan |
| Study hypothesis | In places where Zindagi Mehfooz is used the way it was designed to be used, immunisation coverage and equity are higher. |
| Ethics approval(s) | Approved 11/04/2021, Interactive Research & Development IRB (15 Beach Road #02-01, Singapore, 189677; +65 6372 8778; irb@ird.global), ref: IRD_IRB_2021_11_001 |
| Condition | Coverage and equity of routine immunisation in children under the age of 2 years old |
| Intervention | Study Goal: The overall goal of the study will be to evaluate the impact and cost-effectiveness of Zindagi Mehfooz, a suite of digital health immunisation interventions, on increasing the proportion of children 12-23 months fully immunised in Sindh province through the increased availability and use of data. Study Aim 1 (Reach). Determine barriers and facilitators to programme initiation Objective 1.1. (Programme reach) Draw from ZM system data and programme records on vaccinator registration and training and Government records on numbers of vaccinators to determine the proportion of eligible vaccinators trained by the programme and registered within ZM from 2017 to 2019 in alignment with MICS 6 from 2018-2019 and MICS 5 in 2014 Objective 1.2. (Provider use) Conduct secondary analysis of system-generated data to determine what proportion of vaccinators trained to use ZM upload details for at least 1 client in Sindh Objective 1.3. (Stakeholder perceptions) Conduct in-depth interviews with vaccinators, clients, and supervisors and other key stakeholders) to understand perceptions about the benefits of ZM and effects on immunisation service delivery (decision-support algorithm, registry, SMS alerts and reminders) Study Aim 2 (Registry coverage). Determine proportion of children amongst those eligible at a population level contained within the ZM registry Objective 2.1. Model estimated number of children eligible for vaccination in study area over time using census data by district based on the ZM implementation period per district from 2017 to 2019 in alignment with MICS 6 from 2018-2019 and MICS 5 in 2014 Objective 2.2. Draw from ZM system-generated data to determine the number of unique children contained within ZM vaccination registry Objective 2.3. Conduct in-depth interviews and observations with vaccinators, clients, and supervisors (and other key stakeholders) to understand their views on differences in registry coverage observed and effects on uptake of immunisation services Study Aim 3 (Data quality, use). Determine the availability, use, and quality of ZM data across key stakeholder groups Objective 3.1. Conduct secondary analyses of system-generated data, reports and dashboards to determine frequency of ZM data use by supervisors, managers, and Government staff at all levels. Objective 3.2. Conduct secondary analyses of system-generated data to determine data accuracy, completeness, and timeliness of entry by clinic, and provider characteristics. Objective 3.3. Conduct in-depth interviews with key stakeholders at the district and provincial levels as well as ZM staff to understand differences in newly available data (disaggregated gender data, geographical data, GIS analysis, vaccinator monitoring data) and data use patterns, perceptions on link between data use and immunisation coverage, completion, and timeliness. Objective 3.4. Conduct direct observations of immunisation services to understand the broader context within which implementation is occurring and to identify barriers to data entry and use. Study aims 1 (programme reach), 2 (registry coverage), and 3 (Data quality and use) involve study participants. Study participants will be invited to participate in an in-depth interview that lasts about 30-45 minutes. Direct observation in health facilities and district health offices will also be conducted to document the state of the environment in which ZM is used, especially aspects that might impact how effectively it is used. Study Aim 4 (Impact): To determine whether ZM usage (dose-response high use, moderate use, and low use) in 27 districts of Sindh province is associated with significant differences in the proportion and timeliness of children 12-23 months fully immunised (BCG, DTP3, IPV/OPV3, Measles1) from 2014 to 2019. Objective 4.1. Compare the proportion of children 12-23 months whose caregivers report receiving (both reported and from card) BCG, DTP3, IPV/OPV3, and Measles1 immunisation in MICS-6 2018-19 to those who reportedly received BCG, DTP3, IPV-1/OPV3, Measles1 in MICS-5 2014 (full immunization). Objective 4.2. Compare the proportion of children 12-23 months who did not receive DPT 1 between MICS-6 2018-19 and MICS-5 2014 (zero dose vaccination). Objective 4.3. Compare the proportion of children 12-23 months whose caregivers report receiving on time (from vaccination card) BCG, DTP3, IPV/OPV3, and Measles1 immunisation in MICS-6 2018-2019 to those who reportedly received BCG, DTP3, IPV-1/OPV3, Measles1 in MICS-5 2014 (timeliness). Study Aim 5 (Impact). To determine whether ZM implementation in 27 districts of Sindh province is associated with significant differences in the proportion of children 12-23 months in the poorest and poorer socioeconomic strata (and other dimensions of equity) fully immunised (BCG, DTP3, IPV/OPV3, Measles1) from 2014 to 2019. Objective 5.1. Compare the estimates of zero dose immunisation, full immunization, and timeliness of immunisations MICS- 6 2018-2019 and MICS 5 2014 across the wealth quintiles, by the child’s gender, levels of mother’s / father’s education, and other sociodemographic characteristics. Study Aim 6 (Impact). To determine the incremental cost-effectiveness of ZM as compared to the status quo in 27 districts of Sindh province from a program perspective from 2014 to 2019. Objective 6.1. Using an ingredients approach and drawing from budget and expense summary by category for the calendar year 2019, estimate the economic costs of ZM implementation, including program development, start-up and ongoing implementation. Objective 6.2. Draw from MICS analyses to determine incremental changes in coverage for individual vaccines (BCG, DTP3, IPV/OPV3, and Measles1) for the calendar year 2019. Objective 6.3. Use the lives saved tool to model the incremental lives saved associated with changes in immunisation outcomes for the calendar year 2019 as compared to 2014 Objective 6.4. Estimate the incremental cost per life saved and cost per Disability Adjusted Life Year Averted of ZM implementation in calendar years 2019-20 versus status quo of 2014 (pre-implementation) Objective 6.5. Conduct probabilistic sensitivity analyses to identify key cost drivers and assess uncertainty. |
| Intervention type | Biological/Vaccine |
| Pharmaceutical study type(s) | Not Applicable |
| Phase | Not Applicable |
| Drug / device / biological / vaccine name(s) | Zindagi Mehfooz suite of digital health immunisation interventions |
| Primary outcome measure | All outcome measures will be assessed from 2017 to 2019 in alignment with Multiple Indicator Cluster Surveys (MICS) 6 from 2018-2019 and MICS 5 in 2014: 1. Programme reach is measured as the percentage of eligible vaccinators registered in the system - number of vaccinators registered in ZM/ total number of vaccinators. 2. Provider use is measured as the proportion of registered vaccinators who register one client per vaccination day 3. Registry coverage is measured as the percentage of children registered in ZM / total number of eligible children based on census data 4. Data use is measured as the percentage of supervisors, managers, and government stakeholders who log into ZM to use ZM data/ total number of registered supervisors, managers, and government stakeholders 5. Impact of the use of ZM on immunisation coverage - the proportion of children 12-23 months whose caregivers report receiving (both reported and from card) BCG, DTP3, IPV/OPV3, and Measles1 immunisation in MICS-6 2018-19 to those who reportedly received BCG, DTP3, IPV-1/OPV3, Measles1 in MICS-5 2014 (full immunization) & proportion of children 12-23 months who did not receive DPT 1 between MICS-6 2018-19 and MICS-5 2014 (zero dose vaccination). 6. Equity - the estimates of zero dose immunisation, full immunization, and timeliness of immunisations MICS- 6 2018-2019 and MICS 5 2014 compared across the wealth quintiles, by the child’s gender, levels of mother’s/father’s education, and other sociodemographic characteristics. 7. Cost-effectiveness - the lives saved tool is used to model the incremental lives saved associated with changes in immunisation outcomes for the calendar year 2019 as compared to 2014 & Estimate the incremental cost per life saved and cost per Disability Adjusted Life Year Averted of ZM implementation in calendar years 2019-20 versus status quo of 2014 (pre-implementation). Probabilistic sensitivity analyses conducted to identify key cost drivers and assess uncertainty. |
| Secondary outcome measures | User perceptions of benefits and challenges with ZM as measured through in-depth interviews with vaccinators, supervisors, managers, and caregivers and ZM implementation team conducted in January-March 2022, reflecting on experiences with ZM from 2017-2019 and during COVID-19 2020-2022 |
| Overall study start date | 01/01/2020 |
| Overall study end date | 31/12/2022 |
Eligibility
| Participant type(s) | Mixed |
|---|---|
| Age group | Adult |
| Sex | Both |
| Target number of participants | 90 |
| Participant inclusion criteria | Participants are employed by the Government of Sindh and/or IRD or are caregivers of children brought in for immunisation and enrolled within the ZM system. They will be associated with districts and facilities prioritized through the quantitative analysis process segmented as follows: 1. High ZM use and high Immunisation coverage 2. High ZM use and low Immunisation coverage 3. Low ZM use and high Immunisation coverage 4. Low ZM use and low Immunisation coverage The target will be 1 district per category with 3 facilities per district for an estimate of 12 facilities and 3 vaccinators per facility and their associated supervisors and district health managers. |
| Participant exclusion criteria | Does not consent |
| Recruitment start date | 01/01/2022 |
| Recruitment end date | 30/06/2022 |
Locations
Countries of recruitment
- Pakistan
Study participating centre
Essential Programme for Immunisation
Sindh Province Pakistan
Karachi
-
Pakistan
Karachi
-
Pakistan
Sponsor information
Gavi
Charity
Charity
Global Health Campus
Chemin du Pommier 40
Geneva
1218 Grand-Saconex
Switzerland
| Phone | +41 22 909 65 00 |
|---|---|
| info@gavi.org | |
| Website | http://www.gavi.org/ |
"ROR" |
https://ror.org/0141yg674 |
Funders
Funder type
Charity
GAVI Alliance
Government organisation / Trusts, charities, foundations (both public and private)
Government organisation / Trusts, charities, foundations (both public and private)
- Alternative name(s)
- Gavi, Gavi The Vaccine Alliance
- Location
- Switzerland
Results and Publications
| Intention to publish date | 31/12/2022 |
|---|---|
| Individual participant data (IPD) Intention to share | Yes |
| IPD sharing plan summary | Available on request |
| Publication and dissemination plan | Report of full research results and summary paper in a high-impact peer-reviewed journal. The protocol will be submitted for publication. |
| IPD sharing plan | The datasets generated during and/or analysed during the current study are/will be available upon request from Patricia Mechael (patty@healthenabled.org) |
Study outputs
| Output type | Details | Date created | Date added | Peer reviewed? | Patient-facing? |
|---|---|---|---|---|---|
| Protocol file | 01/09/2021 | 15/11/2021 | No | No | |
| Funder report results | Preliminary findings by evaluation aim and objectives | 31/01/2023 | 05/06/2023 | No | No |
| Results article | 11/10/2024 | 21/01/2025 | Yes | No |
Additional files
- 40654_PROTOCOL_01Sep21.pdf
- ISRCTN23078223_Evaluation Report.pdf
- Preliminary findings by evaluation aim and objectives
Editorial Notes
21/01/2025: Publication reference added.
05/06/2023: An unpublished funder report document has been uploaded as an additional file.
15/11/2021: Trial's existence confirmed by the Interactive Research & Development IRB.
