Effect of high-dosage tutoring on mathematical skills and growth mindset in prevocational secondary education
| ISRCTN | ISRCTN39944387 |
|---|---|
| DOI | https://doi.org/10.1186/ISRCTN39944387 |
- Submission date
- 30/09/2022
- Registration date
- 30/09/2022
- Last edited
- 30/09/2022
- Recruitment status
- No longer recruiting
- Overall study status
- Ongoing
- Condition category
- Other
Plain English summary of protocol
Background and study aims
In Dutch education, students from low-income backgrounds differ meaningfully in educational achievement from economically more privileged students (Gelijke Kansen Alliantie, 2021). Students from low-income backgrounds are overrepresented in the lower tracks of prevocational secondary education (e.g., in praktijkonderwijs and vmbo-basis/kader). This difference is seen clearly in numerical ability and mathematical skills (OECD, 2018). At the end of primary school, a relatively large portion of students from low-income backgrounds have not achieved a level of numerical ability and math skills that is assumed needed to start secondary education effectively (indicated in the Dutch education system as reference level 1F). Low math skills are associated with lower completed educational level, loss of quality of life, social economic exclusion, and poverty (Crawford & Cribb, 2013; Gross et al., 2009) and contribute to a cycle of intergenerational poverty. Educational policy should therefore strive toward equal educational opportunities for all.
Evidence-based educational interventions contribute to this objective. From both international (Guryan et al., 2021; Nickow et al., 2020) and Dutch, national research (De Ree et al., 2021a,b), an educational intervention consisting of High-Dosage Tutoring (HDT) seems to be a promising way to reduce achievement gaps in math. In the Netherlands, HDT is provided by, among others, The Bridge Learning Interventions (TBLI). HDT specifically focuses on strengthening numerical ability and mathematical skills, using principles of mastery learning and growth mindset. The aim of the current project is to (further) study the effectiveness of HDT by performing a cluster-randomized controlled trial at four secondary education schools. Participating in the intervention are pupils from first- and second-year classes of prevocational secondary education.
Who can participate?
Students in their first or second year of prevocational secondary education
What does the study involve?
Classes at the prevocational secondary education level will be randomly allocated, stratified by level of education (pro, vmbo-b, vmbo-k, vmbo-g, vmbo-t), to either the intervention or control condition. Ten students with the poorest math skills per class in the intervention condition will receive high-dosage math tutoring (HDT) for the duration of one school year. The tutoring is high dosage because it is provided three times a week during that school year for 45 or 50 minutes per tutoring session. Tutoring will be provided during regular school hours, so the length of the tutoring session is dependent on regular school lesson length. During these hours, the participants of the HDT-program would miss some of their regular instructional hours. The math tutoring will be provided in a fixed 1-to-2 setting (1 tutor to 2 students). This means that the same pairs of students will receive math tutoring from the same tutor for the complete duration of the intervention.
Main effects of HDT will be evaluated by comparing the math skills of the ten students with the poorest math skills per class in the intervention condition to the ten students with the poorest math skills per class in the control intervention (who have received education as usual). The development of math skills will be measured by administering a standardized math test before HDT (pretest: time point 0), after half a year of HDT (midtest: time point 1), after one year of HDT (posttest: time point 2), and one full year after HDT has been completed (one-year follow-up: time point 3). This test will be administered to all students from all participating classes to also evaluate whether HDT affects the mean math skills of a class of students in which only a subset of students has received HDT (spillover effects). To evaluate the effect of HDT on the academic domain of comprehensive reading, a standardized comprehensive reading test will be administered to all students before and after HDT (at time points 0 and 2). To evaluate the effects of HDT on self-beliefs and motivation, questionnaires on mindset, effort beliefs, and math self-efficacy will be administered to all students before and after HDT (at time points 0 and 2). In addition, the Math Effort Task will be administered to all students before and after HDT (at time points 0 and 2) to evaluate the effect of HDT on willingness of students to expend effort during a math task.
What are the possible benefits and risks of participating?
Participants in the intervention condition with poor math skills and numerical ability will potentially improve their math performance at school by profiting from High-Dosage math Tutoring. There are no risks of physical injury or harm, or of unacceptable stress or anxiety by taking part in this study.
Where is the study run from?
University of Amsterdam (Netherlands)
When is the study starting and how long is it expected to run for?
September 2022 to January 2026
Who is funding the study?
Dutch Research Council (Nederlandse Organisatie voor Wetenschappelijk Onderzoek, NWO)
Who is the main contact?
Dr. Jurgen Tijms
j.tijms@uva.nl
Contact information
Principal Investigator
Nieuwe Achtergracht 166
Amsterdam
1018 WV
Netherlands
 ORCID ID |
0000-0002-9080-8569 |
|---|---|
| Phone | +31 (0) 20 525 34 88 |
| j.tijms@uva.nl |
Scientific
Nieuwe Achtergracht 166
Amsterdam
1018 WV
Netherlands
| Phone | +31 (0) 20 525 34 88 |
|---|---|
| j.tijms@uva.nl |
Public
Nieuwe Achtergracht 166
Amsterdam
1018 WV
Netherlands
| Phone | +31 (0) 20 525 34 88 |
|---|---|
| r.e.dejong@uva.nl |
Study information
| Study design | Single-center interventional single-blinded cluster-randomized controlled trial |
|---|---|
| Primary study design | Interventional |
| Secondary study design | Cluster randomised trial |
| Study setting(s) | School |
| Study type | Treatment |
| Participant information sheet | 42524 PIS.pdf |
| Scientific title | Effectiveness of high-dosage tutoring in prevocational secondary education students from low-income backgrounds: A cluster-randomized controlled trial |
| Study objectives | 1. High-Dosage Tutoring significantly improves the mathematical skills of students in prevocational secondary education, both on the short term (directly after the program has ended) and medium term (at one-year follow-up). 2. High-Dosage Tutoring results in a stronger growth mindset, less negative effort beliefs, higher mathematics self-efficacy, and higher willingness to expend effort during a mathematical task in students in prevocational secondary education directly after the program has ended. 3. High-Dosage Tutoring does not have a significantly negative impact on the reading comprehension skills of students in prevocational secondary education, both on the short term (directly after the program has ended) and medium term (at one-year follow-up) (i.e., gains in mathematical skills will not come at the expense of losses in reading comprehension skills, as compared to the control group). 4. Underachievers benefit more from the High-Dosage Tutoring than others. 5. Tutors vary in performance, as measured by variation in learning gains of students. 6. There is spillover effect of the High-Dosage Tutoring intervention on nonparticipants in the same classes. |
| Ethics approval(s) | Approved 30/05/2022, The Economics and Business Ethics Committee (EBEC), University of Amsterdam (Amsterdam Business School, Plantage Muidergracht 12, 1012 TV Amsterdam, The Netherlands; +31 20 525 5311; secbs-abs@uva.nl), ref: 20220526100557 |
| Health condition(s) or problem(s) studied | Mathematical skills and growth mindset in young adolescents from low-income backgrounds |
| Intervention | The intervention will consist of High-Dosage Tutoring (HDT) provided by the organization The Bridge Learning Interventions. Students who will receive HDT will receive math tutoring three times a week during one school year of prevocational secondary education for 45 or 50 minutes per session (length of the tutoring sessions is dependent on regular school lesson length). The math tutoring will start at the beginning or at the midway point of the first or second school year of prevocational secondary education, meaning that the math tutoring will end after the first or second school year or at the midway point of the second or third school year of secondary education. One tutor will provide math tutoring to two students in a fixed setting. The tutoring is embedded in the regular school day and will take place during school hours in a separate room within the school building according to a fixed schedule. A cluster-randomized controlled trial will be performed, in which school classes will be randomized, using a 1:1 allocation ratio (approximately), to either the intervention or control (no treatment) condition. Randomization will be stratified by level of secondary education as much as possible (pro, vmbo-b, vmbo-k, vmbo-g, vmbo-t). Ten students with the poorest math skills per class in the intervention condition will receive HDT. The scores on a Dutch standardized math test administered preintervention will determine who those students are. To evaluate the possible difference in effectiveness between tutors, we will try and randomize tutors to student pairs. Our decisions regarding the randomization of tutors will depend on whether The Bridge Learning Intervention does not have a specific preference for a match between two students and a specific tutor. This way, the randomization of tutors would not influence the ecological validity of the intervention. |
| Intervention type | Behavioural |
| Primary outcome measure | Mathematical skills are measured using a Dutch standardized math test at pretest, midtest (halfway the intervention), posttest and at one-year follow-up. |
| Secondary outcome measures | 1. General mindset and mindset specific to math will be measured using a Dutch translation of the Implicit Theories of Intelligence Scale - Self-Theory version by De Castella & Byrne (2015) at baseline and endline. 2. Effort beliefs will be measured using the effort beliefs scale by Blackwell, Trzesniewski, and Dweck (2007) at baseline and endline. 3. Mathematics self-efficacy will be measured using the self-efficacy subscale of the Patterns of Adaptive Learning Scales (PALS) by Midgley et al. (2000) at baseline and endline. 4. Willingness to expend effort during a mathematical task will be measured by the computerized Math Effort Task (Janssen et al., 2022), developed by the research team based on Engle-Friedman et al. (2014), at baseline and endline. 5. Reading comprehension skills will be measured using a Dutch standardized comprehensive reading test at baseline and endline. |
| Overall study start date | 17/03/2022 |
| Completion date | 31/01/2026 |
Eligibility
| Participant type(s) | Healthy volunteer |
|---|---|
| Age group | Child |
| Lower age limit | 12 Years |
| Upper age limit | 15 Years |
| Sex | Both |
| Target number of participants | Four schools will participate for two consecutive school years (i.e., four schools will provide two year cohorts). Schools who serve predominantly low-income populations are prioritized in the preselection of schools. Schools are expected to have approximately 10 eligible classes at the prevocational secondary education level per year cohort. Therefore, we expect 4 schools * 2 year cohorts per school * 10 classes per year cohort = 80 classes (clusters) to be randomized. In each class, ten students with the poorest math skills will be recruited to participate. To measure the direct effect of the HDT intervention, we will compare the ten lowest-scoring students between classes assigned to the treatment and the control condition. To measure spillover effect on non-participants, we will compare the rest (those students with higher math scores) between classes assigned to the treatment and the control condition. This means that we aim to include 400 students in our main analysis (40 classes * 10 students per class) in the intervention condition and 400 students (40 classes * 10 students per class) in the control condition. To measure spillover effects, we aim to include another, approximately 400 students in the intervention condition and another, approximately 400 students in the control condition. For our main hypotheses 1. and 2., we have worked out the following power calculations. For this we have used comparable data from other research projects. Based on the calculations, we think that standard errors of our main standardized effects of around 0.05 (for math scores and the Math Effort Task) are reasonable targets. Based on this, the Minimum Detectable Effects are 0.17 (based on a beta = 0.20 and an alpha = 0.05). We expect, based on the literature and previous research to find effects that are larger than the MDE (e.g., De Ree et. al. (2021a)). |
| Key inclusion criteria | Students who are in their first or second year of prevocational secondary education at the start of the intervention period (approx. 12-15 years of age). |
| Key exclusion criteria | No participant exclusion criteria are applicable. |
| Date of first enrolment | 03/10/2022 |
| Date of final enrolment | 31/01/2024 |
Locations
Countries of recruitment
- Netherlands
Study participating centre
Amsterdam
1018 WV
Netherlands
Sponsor information
University/education
Faculty of Social and Behavioural Sciences
Amsterdam Institute for Social Science Research (AISSR)
Nieuwe Achtergracht 166
Amsterdam
1018 WV
Netherlands
| Phone | +31 (0)20 525 22 62 |
|---|---|
| aissr@uva.nl | |
| Website | https://aissr.uva.nl/ |
"ROR" |
https://ror.org/04dkp9463 |
Funders
Funder type
Government
Government organisation / National government
- Alternative name(s)
- Netherlands Organisation for Scientific Research, Dutch National Scientific Foundation, Dutch National Science Foundation, Dutch Research Council (Nederlandse Organisatie voor Wetenschappelijk Onderzoek), NWO:Nederlandse Organisatie voor Wetenschappelijk Onderzoek, Nederlandse Organisatie voor Wetenschappelijk Onderzoek (NWO), Dutch Research Council, Dutch Research Council, Netherlands, NWO
- Location
- Netherlands
Results and Publications
| Intention to publish date | 31/01/2027 |
|---|---|
| Individual participant data (IPD) Intention to share | No |
| IPD sharing plan summary | Data sharing statement to be made available at a later date |
| Publication and dissemination plan | 1. During the trial, three interim reports on the progress of the research project will be provided to the Dutch Research Council (Nederlandse Organisatie voor Wetenschappelijk Onderzoek, NWO), the client and funder of the trial. A final report including evaluation of the effects of HDT will be provided on 30/09/2025. 2. Planned publication on the main results of the trial in a peer-reviewed journal. For this publication, a working paper will be openly accessible during the trial. This working paper will include some of the details and approaches of the intervention. The results are only presented at the end of the project. The final publication will be open access. 3. One or two planned publication(s) on specific elements of the research design in a peer-reviewed journal. The final publication(s) will be open access. 4. A minimum of two planned publications in journals for educational professionals. One publication will focus on the experience of students, educational professionals, and parents with the implementation of HDT. The other publication will focus on the evaluation of the effects of HDT. |
| IPD sharing plan | The data-sharing plans for the current study are unknown and will be made available at a later date. |
Study outputs
| Output type | Details | Date created | Date added | Peer reviewed? | Patient-facing? |
|---|---|---|---|---|---|
| Participant information sheet | in Dutch | 30/09/2022 | No | Yes |
Additional files
- 42524 PIS.pdf
- in Dutch
Editorial Notes
30/09/2022: Trial's existence confirmed by University of Amsterdam
