Investigating the impact of the Maths-Whizz intelligent tutoring programme on maths attainment for Year 2 - 5 pupils
| ISRCTN | ISRCTN13444898 |
|---|---|
| DOI | https://doi.org/10.1186/ISRCTN13444898 |
- Submission date
- 27/06/2024
- Registration date
- 02/07/2024
- Last edited
- 01/07/2024
- Recruitment status
- No longer recruiting
- Overall study status
- Ongoing
- Condition category
- Other
Plain English summary of protocol
Background and study aims
Maths-Whizz, is an EdTech programme designed to support primary school pupils’ maths attainment. A previous study indicates that Maths-Whizz may have a positive effect on maths attainment, though this study only examined the effect on one year group. This trial is larger in scope, aiming to investigate the impact of participation in Maths-Whizz on maths attainment for Year 2, 3, 4 and 5 pupils in schools in England.
Who can participate?
Junior, infant or primary schools with Year 2, 3, 4 and 5 pupils can participate, as long as they have the appropriate IT infrastructure, don’t have mixed year group maths teaching in the 2024/25 academic year, have not used Maths Whizz in the 2023/24 academic year and are not participating in another EEF-funded trial in the 2024/25 academic year. All pupils in participating classes in the intervention year groups can access the Maths-Whizz programme.
What does the study involve?
66 schools have been recruited to the trial. However, within these 66 schools there are 3 infant/junior school pairs. These will be treated as one school within the trial, effectively reducing the number of schools to 63. At each participating school/school-pair, pupils in year groups 2 to 5 will participate. Schools will be randomised to one of two study arms. Those in arm 1 will deliver Maths-Whizz to pupils in years 2 and 4, whilst those in arm 2 will deliver Maths-Whizz to pupils in years 3 and 5. In each arm, pupils in the year groups not receiving Maths-Whizz will act as the control group.
Each school will be provided an implementation plan to ensure that Maths- Whizz implementation is embedded within the school’s core maths provision.
Pupils in intervention classes will engage with the Maths-Whizz Tutor via their online platform for about an hour each week in addition to regular maths lessons. Maths-Whizz can also be used for maths homework. Pupils in control classes will continue with their maths learning as normal.
Pupils will complete both Renaissance Star Maths assessments and the Maths & Me survey prior to randomisation, and at the end of the 2024/25 academic year. The effectiveness of Maths-Whizz will be measured by how much intervention pupils have improved their maths score, compared to control pupils.
What are the possible benefits and risks of participating?
The potential benefit to pupils taking part in Maths-Whizz is improved understanding of maths and consequently improved test attainment.
No risks of participating are anticipated.
Where is the study run from?
The study is being run by the National Foundation for Educational Research (NFER) in England. Delivery of the Maths-Whizz platform occurs at schools taking part in the trial. Data on pupil participation in Maths-Whizz will be collected on the Maths-Whizz platform and shared with NFER.
When is the study starting and how long is it expected to run for?
October 2023 to August 2026.
Who is funding the study?
This trial is being funded by the Education Endowment Foundation (UK)
Who is the main contact?
Aarti Sahasranaman, a.sahasranaman@nfer.ac.uk
Contact information
Public, Principal Investigator
National Foundation for Educational Research
The Mere
Upton Park
Slough
SL1 2DQ
United Kingdom
 ORCID ID |
0000-0001-6827-6132 |
|---|---|
| Phone | +44 (0)1753 574123 |
| s.welbourne@nfer.ac.uk |
Scientific
National Foundation for Educational Research
The Mere
Upton Park
Slough
SL1 2DQ
United Kingdom
| ORCID ID |
0009-0002-6119-6927 |
|---|---|
| Phone | +44 (0)1753 574123 |
| g.schwendel@nfer.ac.uk |
Study information
| Study design | Two-arm interleaved cluster randomized controlled trial with random allocation at the school level |
|---|---|
| Primary study design | Interventional |
| Secondary study design | Cluster randomised trial |
| Study setting(s) | School |
| Study type | Other |
| Participant information sheet | https://www.nfer.ac.uk/for-schools/participate-in-research/participate-in-research-projects/evaluation-of-whizz-education-maths-whizz-intelligent-tutoring-programme/ |
| Scientific title | Randomised controlled trial investigating the effect of participation in the Maths-Whizz intelligent tutoring programme on maths attainment amongst Year 2 - 5 pupils |
| Study objectives | What is the overall impact of Maths-Whizz on the maths outcomes of children in years two to five? |
| Ethics approval(s) | Ethics approval not required |
| Ethics approval additional information | There was no formal ethical approval for this trial. However, the evaluation will be conducted in accordance with NFER’s Code of Practice, available at NFER Code of Practice. Each participating school’s headteacher will provide their agreement to participate in the trial by signing the Memorandum of Understanding (MoU) that outlines the responsibilities of all parties involved in the trial. NFER will share a parent letter and withdrawal form with schools to be sent to parents/carers of all pupils that schools intend to nominate for screening. Through the withdrawal form, parents/carers will have the opportunity to withdraw their child from the evaluation and associated data processing at any stage of the trial. |
| Health condition(s) or problem(s) studied | To improve maths attainment amongst all Year 2 to Year 5 pupils. |
| Intervention | After signing up to the study and completing the MoU, each school will provide initial information on pupils participating in the study. Once this has been collected from each school, and baseline assessments for the primary and secondary outcomes have been completed, randomisation will take place. This will be at the school-level; schools will be randomised in a 1:1 ratio to two arms. Schools in arm 1 will deliver Maths-Whizz to pupils in Years 2 and 4 and schools in arm 2 will deliver Maths-Whizz to pupils in Years 3 and 5. Pupils in year groups not receiving the programme in each arm will serve as controls for intervention pupils in the other arm. Randomisation will not be stratified. The syntax used to perform the randomisation will be stored for reproducibility and transparency and will be included as an appendix in the statistical analysis plan and final report. The statistician will not be blinded to group allocation. Randomisation allocation data will then be passed to NFER’s Research and Product Operations team, who will liaise with schools. Each school will be provided with an implementation plan for Maths-Whizz alongside training and onboarding from a Whizz Education Success Partner (ESP) to enable school leadership and teaching staff to facilitate pupil engagement. For each class, teachers will either allow the Maths-Whizz Tutor to determine the topics to be covered or they can choose Topic Focus to direct Maths-Whizz to directly support the topic they are teaching in class. When teachers choose Topic Focus, Maths-Whizz will set up differentiated lessons for each pupil in the class. Teachers can access assessments and progress reports to change course for each pupil depending on their changing needs and learning progress. Pupils will engage in individualised learning through the Maths-Whizz online intelligent tutor. Lessons are automated and differentiated for every individual pupil depending on their unique needs and pace of learning. The programme consists of engaging pupils with the Maths-Whizz Tutor for about an hour each week. This can be divided into several sessions, such as three times a week (e.g., 3 x 20 minute sessions per week), and is provided in addition to regular maths lessons. Maths-Whizz may also be deployed within the school’s homework policy. Details of how each school will implement Maths-Whizz will be determined during the implementation planning stage. Teachers will be able to access the initial assessment and learning data of all pupils to determine each child’s needs and progress. Parents will also be able to engage with their child’s learning by monitoring their needs and progress. |
| Intervention type | Other |
| Primary outcome measure | Measured at Baseline: prior to randomisation (June & July 2024) and Endline: summer 2025 (June & July): Renaissance Star Maths assessment. The assessment produces a ‘Unified Scaled Score’ ranging from 600 – 1400 across all year groups from Year 1 to Year 13, thus allowing data from the four different year groups in the trial to be combined in the analysis. |
| Secondary outcome measures | Measured at Baseline: prior to randomisation (June & July 2024) and Endline: summer 2025 (June & July): Pupils’ maths enjoyment and self-efficacy and will be assessed using the Maths and Me survey. This consists of two scales: mathematical self-perceptions and enjoyment of mathematics, which are measured by 18 items scored on a five-point Likert scale. |
| Overall study start date | 01/10/2023 |
| Completion date | 31/08/2026 |
Eligibility
| Participant type(s) | Learner/student |
|---|---|
| Age group | Child |
| Lower age limit | 6 Years |
| Upper age limit | 10 Years |
| Sex | Both |
| Target number of participants | 8,672 |
| Total final enrolment | 11883 |
| Key inclusion criteria | School inclusion criteria: 1. Be an Infant or Primary school with children in Years 2, 3, 4 and 5 as of 1 September 2024 2. Have a suitable level of IT provision including devices such as tablets/laptops/desktops and sufficient wi-fi capacity (0.5 Mbps internet connection per concurrent pupil) 3. Agree to contribute £500 towards the cost of the programme (usual cost £1875 per school plus £25 per pupil per year) Pupil inclusion criteria: 4. Attendance at a school that meets the above criteria 5. Completing Year 2, 3, 4 or 5 during the 2024/2025 academic year |
| Key exclusion criteria | School exclusion criteria: 1. Not have mixed year group maths teaching (i.e. children who are from more than one-year group within the same maths class) in any of Years 2, 3, 4 and 5 in the 2024-25 academic year even if the year groups are mixed. 2. Not have implemented Maths-Whizz or any Whizz associated services in the 2023-24 academic year. 3. Not be participating in another EEF-funded trial in the 2024-25 academic year. Pupil exclusion criteria: 4. In Reception Year, Year 1 or Year 6 5. Pupils in Years 2-5 with severe visual impairments will not be able to access the Maths-Whizz platform |
| Date of first enrolment | 02/02/2024 |
| Date of final enrolment | 10/05/2024 |
Locations
Countries of recruitment
- England
- United Kingdom
Study participating centre
Slough
SL1 2DQ
United Kingdom
Sponsor information
Charity
9th Floor
Millbank Tower
21-24 Millbank
London
SW1P 4QP
England
United Kingdom
| Phone | +44 (0)20 7802 1676 |
|---|---|
| info@eefoundation.org.uk | |
| Website | https://educationendowmentfoundation.org.uk/ |
"ROR" |
https://ror.org/03bhd6288 |
Funders
Funder type
Charity
Private sector organisation / Trusts, charities, foundations (both public and private)
- Alternative name(s)
- EducEndowFoundn, Education Endowment Foundation | London, EEF
- Location
- United Kingdom
Results and Publications
| Intention to publish date | 31/05/2026 |
|---|---|
| Individual participant data (IPD) Intention to share | Yes |
| IPD sharing plan summary | Stored in non-publicly available repository |
| Publication and dissemination plan | We intend to publish the final report on the Education Endowment Foundation's website |
| IPD sharing plan | The datasets analysed during this study will be stored in a non-publicly available repository, the EEF data archive. Within three months of the end of project, NFER will send school and pupil data to the Education Endowment Foundation's (EEF) data archive partner. This will include enough pupil-level data for an analyst to replicate the impact analysis. At this point, EEF’s data archive partner will keep a copy of the data and EEF will become the Data Controller. This data may be shared in an anonymised form with other research teams. Further matching to NPD and other administrative data may take place during subsequent research. |
Study outputs
| Output type | Details | Date created | Date added | Peer reviewed? | Patient-facing? |
|---|---|---|---|---|---|
| Protocol (other) | 27/06/2024 | No | No |
Editorial Notes
01/07/2024: Trial's existence confirmed by Education Endowment Foundation.
