Evaluation of the effectiveness of Can't Wait to Learn, an EdTech numeracy and reading programme, in government schools in Isingiro district, Uganda
| ISRCTN | ISRCTN11460734 |
|---|---|
| DOI | https://doi.org/10.1186/ISRCTN11460734 |
| Secondary identifying numbers | 109296-001 |
- Submission date
- 08/03/2022
- Registration date
- 10/03/2022
- Last edited
- 16/01/2025
- Recruitment status
- No longer recruiting
- Overall study status
- Completed
- Condition category
- Other
Plain English summary of protocol
Background and study aims
In the current climate of increasingly protracted crises and forced displacement, there is a growing demand for evidence-based and scalable education programmes. War Child Holland has been awarded a Global Partnership for Education and Knowledge and Innovation Exchange (GPE KIX) global grant to generate knowledge and evidence on how education technology innovations can be adapted and scaled to improve education access and quality for refugee and displaced children in conflict-affected countries. KIX is a joint initiative from GPE and the International Development Research Centre (IDRC), aiming to connect expertise, innovation, and knowledge to strengthen education systems and accelerate progress toward Sustainable Development Goal 4 in GPE partner countries. War Child Holland’s KIX research focuses on its flagship EdTech programme, Can’t Wait to Learn, in Uganda, Chad, and Sudan.
Can't Wait to Learn is an English reading and numeracy game-based learning programme delivered on a tablet. It is aligned with the national Primary 3 (P3) curriculum, with all content approved by the National Curriculum Development Centre (NCDC). The game includes three main components: 1) the game world, which is an experiential setting with characters and stories, 2) instructional videos of local children and adults explaining academic concepts and tasks, and 3) mini-games, in which children engage with academic content and progress to more difficult concepts and skills as and when they pass each mini-game, i.e. the game is individually paced. An animated game guide begins each mini-game by instructing on the task at hand, and children receive in-game rewards to foster motivation. Each child has their own account which generates log data on usage and progression through the game content. The game world, characters and storyline were co-created with children, resulting in an interface that reflects children’s reality and aspirations. The game is offline and the tablets are secured in custom-made and locally manufactured cages, and charged using solar power generated from procured solar panels. Before implementation, teachers are trained in the use of the game, tablet management and basic maintenance, how to foster independent learning, and how to blend Can't Wait to Learn with their class timetable.
While positive evidence for Can't Wait to Learn exists, a rigorous evaluation of its effectiveness is still needed. This study will compare the effectiveness of Can’t Wait to Learn integrated into normal schooling in public schools on learning outcomes versus normal schooling alone.
The specific aims of the study are:
1. To evaluate the effectiveness of Can't Wait to Learn integrated into regular education in public schools on numeracy and reading competencies, compared to regular education alone
2. To evaluate the effect of Can't Wait to Learn on children and education staff’s psychosocial wellbeing, compared to education as usual
3. To understand the perceived experience and effectiveness of Can't Wait to Learn from the perspective of multiple stakeholders.
Who can participate?
Children in Primary 3 (P3) at 30 public schools in Isingiro district in southwest Uganda
What does the study involve?
Half of the schools will be assigned to Can't Wait to Learn (‘intervention’ schools) and half of the schools will continue as normal (‘control’, or comparison schools). The assignment of intervention or control will be done randomly, which ensures that the two groups are theoretically comparable in terms of the individual differences between schools and the inevitable external factors that affect schools and children's learning. In the intervention schools, Can't Wait to Learn will replace two English and two Maths lessons per week. One additional session of both Can't Wait to Learn games will be done per week after lunch. In the control schools, there will be no changes to normal lessons and teaching. From all schools, 50 children will be selected at random from P3 to participate in the study. Children who have hearing, visual, speech or cognitive impediments that significantly impair their ability to listen to or watch a video on a tablet, see images on a screen, participate in assessments, or understand instructions will not be invited to participate. Children whose behaviour poses a risk to the safety of other children or teaching materials, or those who live in a child-headed household, will also not be invited to participate. In March 2022, each child’s parent or guardian will be asked to consent to their child’s participation and complete a survey. The children will be asked to assent to their own participation and complete numeracy, reading and psychological wellbeing assessments. Six months later, the same children will do the assessments again and the results will be compared between those in the intervention and control schools.
What are the possible benefits and risks of participating?
War Child Holland believes that children in the intervention schools will improve more in reading and numeracy than children in the control schools. As Can't Wait to Learn is an educational programme that has been shown positive results on children’s learning in other settings, the researchers feel that there are no severe risks to the participants. However they do acknowledge the following risks: an increased possibility of COVID-19 transmission, teachers may feel overwhelmed, the Can't Wait to Learn hardware could increase security risks to the school, and participants may have concerns regarding the sharing and use of their personal data.
Where is the study run from?
War Child Holland (Netherlands)
When is the study starting and how long is it expected to run for?
July 2021 to November 2022
Who is funding the study?
International Development Research Centre (Canada)
Who is the main contact?
Jasmine Turner
jasmine.turner@warchild.nl
Contact information
Principal Investigator
War Child Holland
Helmholtzstraat 61-G
Amsterdam
1098 LE
Netherlands
 ORCID ID |
0000-0001-5925-8039 |
|---|---|
| Phone | +31 (0)619041949 |
| mark.jordans@warchild.nl |
Principal Investigator
University of Notre Dame
200 Visitation Hall
Notre Dame
Notre Dame
IN 46556
United States of America
| Phone | +1 (0)574 631 4449 |
|---|---|
| ndsa@nd.edu |
Public
War Child Holland
Helmholtzstraat 61-G
Amsterdam
1098 LE
Netherlands
| Phone | +31 (0)627596190 |
|---|---|
| jasmine.turner@warchild.nl |
Scientific
War Child Holland
Helmholtzstraat 61-G
Amsterdam
1098LE
Netherlands
| ORCID ID |
0000-0001-5925-8039 |
|---|---|
| Phone | +31 (0)619041949 |
| mark.jordans@warchild.nl |
Study information
| Study design | Multicentre interventional cluster randomized controlled trial |
|---|---|
| Primary study design | Interventional |
| Secondary study design | Cluster randomised trial |
| Study setting(s) | School |
| Study type | Other |
| Participant information sheet | 41313_PIS_caregiver.pdf |
| Scientific title | Evaluation the effect of Can't Wait to Learn integrated into formal education compared to education-as-usual in government schools in Southwest Uganda on numeracy and reading outcomes: a cluster randomized controlled trial |
| Study objectives | Children doing Can't Wait to Learn integrated in formal education will demonstrate larger gains in numeracy and reading competency compared to children doing education-as-usual. |
| Ethics approval(s) | 1. Approved 27/08/2021, The AIDS Support Organisation (TASO) Research Ethics Committee (REC) (Mulago Hospital Complex, PO Box 10443, Kampala, Uganda; +256 (0)414 532 580/1; mail@tasouganda.org), ref: TASOREC/056/2021-UG-REC-009 2. Approved 22/10/2021, Uganda National Council of Science and Technology (UNCST) (Plot 6 Kimera Road, Ntinda, PO Box 6884, Kampala, Uganda; +256 (0)414 705500; info@uncst.go.ug), ref: SS1024ES |
| Health condition(s) or problem(s) studied | Numeracy and reading competency levels amongst learners in Primary 3 (P3) in government schools |
| Intervention | Randomisation: Random selection of schools (n = 30) is done following the application of the eligibility criteria to all schools in the Isingiro district. This is effectuated by a statistician who subsequently randomly allocated the schools into two groups (A and B). At the end of the RCT inception meeting, a coin flip is conducted by the Isingiro District Education Officer (DEO) to assign the treatment or control condition to Group A and B. In this way, the statistician is blinded to the condition allocation of the schools. A sample of 50 children is randomly selected from the P3 class of each selected school. An additional 5 children per class are randomly selected to form a waitlist. The statistician is masked to the assigned condition. Treatment condition: Can't Wait to Learn (CWTL). CWTL is an English reading and numeracy game-based learning programme delivered on a tablet. It is aligned with the national P3 curriculum, with all content approved by the National Curriculum Development Centre (NCDC). The game includes three main components: 1) the game world, which is an experiential setting with characters and stories, 2) instructional videos of local children and adults explaining academic concepts and tasks, and 3) mini-games, in which children engage with academic content and progress to more difficult concepts and skills when they get above a pre-determined proportion of questions correct, i.e. progression through the content and curriculum is based on individual performance. An animated game guide begins each mini-game by instructing on the task at hand, and children receive in-game rewards to foster motivation. Each child has their own account which generates log data on usage and progression through the game content. The game world, characters and storyline were co-created with children, resulting in an interface that reflects children’s reality and aspirations. The game is offline and the tablets are secured in custom-made and locally manufactured cages, and charged using solar power generated from procured solar panels. Before implementation, teachers are trained in the use of the game, tablet management and basic maintenance, how to foster independent learning, and how to blend CWTL with their scholastic timetable. Dosage: Three 45-minute sessions of CWTL numeracy and three 45-minute sessions reading per week are implemented per week. Two CWTL numeracy and two CWTL reading sessions replace normal Maths and English lessons per week. Duration: 6 months Control condition: Education-as-usual (EAU). EAU consists of 1 hour per day of numeracy and 1 hour per day of literacy, both in English, taught by the class teacher. In addition, classes like sports and art are usually replaced with catch-up classes for numeracy and literacy, led and designed by the class teacher. In some schools, phonics lessons replace sports and art. As EAU will therefore vary somewhat by school, the dosage of literacy and numeracy classes (including catch up) will be tracked. |
| Intervention type | Behavioural |
| Primary outcome measure | Numeracy and reading competency levels measured using custom-made academic assessments at baseline and 6 months later. The assessments are aligned with internationally-used measures, including Early Grade Reading Assessment (EGRA) and Early Grade Math Assessment (EGMA), but adapted to be better suited to detect granular change. The assessments have been piloted and their psychometrics analysed. These will be reported. |
| Secondary outcome measures | Children's psychological wellbeing measured using the Stirling Children's Wellbeing Scale at baseline and 6 months |
| Overall study start date | 13/07/2021 |
| Completion date | 25/11/2022 |
Eligibility
| Participant type(s) | Learner/student, Other |
|---|---|
| Age group | Mixed |
| Sex | Both |
| Target number of participants | 30 clusters; 102 participants per cluster (50 children, 50 caregivers, 2 education personnel) |
| Total final enrolment | 1507 |
| Key inclusion criteria | Participants include: children, caregivers (i.e. biological parents/the individual(s) responsible for the daily care and wellbeing of the child) and education personnel (e.g. teachers, headteachers). Child inclusion criteria: 1. Assents to participating in the research 2. Caregiver consent is obtained 3. Is enrolled in grade P3 Caregiver inclusion criteria: 1. Consents to participate in the research 2. Has a child participating in the research Education personnel: 1. Willing to participate in the research 2. Works at, or represents, a school selected for the research 3. Teachers only: teaches P3 |
| Key exclusion criteria | 1. Hearing, visual, and speech impediments that significantly impair the child’s ability to listen to or watch a video on a tablet, see images on a screen, and/or participate in assessments 2. Children unable to understand pedagogical explanations for reading or numeracy 3. Behaviour that poses risks to the safety of other children or pedagogical materials 4. Resides in a child-headed household, i.e. head of household is under 18 years old No exclusion criteria for caregivers or education personnel |
| Date of first enrolment | 21/03/2022 |
| Date of final enrolment | 14/04/2022 |
Locations
Countries of recruitment
- Uganda
Study participating centres
Uganda
Uganda
Uganda
Uganda
Uganda
Uganda
Uganda
Uganda
Uganda
Uganda
Uganda
Uganda
Uganda
Uganda
Uganda
Uganda
Uganda
Uganda
Uganda
Uganda
Uganda
Uganda
Uganda
Uganda
Uganda
Uganda
Uganda
Uganda
Uganda
Uganda
Sponsor information
Charity
Helmholtzstraat 61-G
Amsterdam
1098 LE
Netherlands
| Phone | +31 (0)204227777 |
|---|---|
| info@warchild.nl | |
| Website | https://www.warchildholland.org/research-and-development/ |
"ROR" |
https://ror.org/01tq9ra93 |
Funders
Funder type
Government
Government organisation / Local government
- Alternative name(s)
- Centre de recherches pour le développement international, IDRC, CRDI
- Location
- Canada
Results and Publications
| Intention to publish date | 01/04/2023 |
|---|---|
| Individual participant data (IPD) Intention to share | Yes |
| IPD sharing plan summary | Stored in non-publicly available repository |
| Publication and dissemination plan | Outputs will include: 1. Academic journal articles: the trial results will be published in a peer-reviewed, preferably open-access journal; publications and authorship arrangements will adhere to the War Child Holland (WCH) R&D publication policy 2. Conferences: results will be presented in at least two international conferences, which will be selected based on the relevance of the research findings 3. Workshops: workshops will be held with key stakeholders at relevant timepoints over the course of the research study Dissemination of results: 1. Local community: inception and synthesis meetings will be held before and after the trial 2. Global education in emergencies community: research reports will be shared widely, and presentations given at key conferences, meetings, and sector working groups 3. International development community: research reports will be shared widely, and presentations given at key conferences, meetings, and sector working groups 4. Academic community: results will be disseminated widely through the academic community through publication in a peer-reviewed journal, and presentation of the results at international conference(s) |
| IPD sharing plan | The datasets generated during and/or analysed during the current study will be stored in a non-publicly available repository. The repository is a secured EU-based server of War Child’s R&D department (the R&D Data Storage). As soon as the data are collected and data quality is checked, they will be uploaded on the R&D Data Storage by Jasmine Turner (Researcher) or Jamal Anan (Research Coordinator). All data will be removed at the latest 10 years after the end of the project, personal data will be removed at the latest 2 years after the end of the project. The datasets will be shared according to the R&D Data Management Plan which is GDPR compliant and based on the University of Edinburgh’s Mantra Research Data Management Training. The data will be shared for research purposes only and anyone with whom the data is shared needs to sign a data-sharing agreement for data processors, including the GDPRs standard contractual clauses. R&Ds Data Managers grant secured access to the relevant files on the Data Storage and monitor the access rights structurally. |
Study outputs
| Output type | Details | Date created | Date added | Peer reviewed? | Patient-facing? |
|---|---|---|---|---|---|
| Participant information sheet | Caregiver consent for children | 10/03/2022 | No | Yes | |
| Participant information sheet | Child assent | 10/03/2022 | No | Yes | |
| Other files | Registry of Efficacy and Effectiveness Studies | 12/12/2022 | No | No | |
| Protocol file | 01/12/2022 | 19/12/2022 | No | No | |
| Statistical Analysis Plan | 19/12/2022 | No | No | ||
| Basic results | 16/01/2025 | 16/01/2025 | No | No |
Additional files
- 41313_PIS_child.pdf
- Child assent
- 41313_PIS_caregiver.pdf
- Caregiver consent for children
- ISRCTN11460734_SAP_V4.pdf
- Registry of Efficacy and Effectiveness Studies
- 41313 CWTL RCT statistical analysis plan_clean.pdf
- 41313 Uganda cRCT_protocol_v.4 01Dec2022.pdf
- ISRCTN11460734_BasicResults_16Jan25.pdf
Editorial Notes
16/01/2025: Basic results added.
19/12/2022: The following changes were made to the trial record:
1. Uploaded protocol (not peer-reviewed) as an additional file.
2. The statistical analysis plan was uploaded as an additional file.
3. Another additional file was recategorised from SAP to Other files.
12/12/2022: The following changes were made to the trial record:
1. Statistical analysis plan uploaded.
2. The overall trial end date was changed from 23/12/2022 to 25/11/2022.
3. Total final enrolment added.
10/03/2022: Trial's existence confirmed by the Uganda National Council of Science and Technology (UNCST).
