Evaluating the impact of adapting to counterintuitive concepts on pupils’ maths and science attainment
| ISRCTN | ISRCTN12838371 |
|---|---|
| DOI | https://doi.org/10.1186/ISRCTN12838371 |
| Secondary identifying numbers | NATCEN ref: P15462 |
- Submission date
- 05/05/2021
- Registration date
- 07/05/2021
- Last edited
- 06/05/2025
- Recruitment status
- No longer recruiting
- Overall study status
- Completed
- Condition category
- Other
Plain English summary of protocol
Background and study aims
International surveys reveal that the UK faces challenges in improving children and young people’s Maths and Science skills. Children’s ability to learn Maths and Science concepts is limited by their ability to inhibit perceptual evidence (what they see, feel or hear) or pre-existing beliefs. Many mistakes in Maths and Science are made because children tend to answer with an intuitive response. For example, when children are taught that the world is round, there is no direct visual evidence to support this idea, as the horizon looks flat. Even after learning that the world is round, children may respond incorrectly when asked about the shape of the world, because of their limited ability to inhibit their initial response. Counterintuitive concepts are therefore often the basis for common misconceptions in Maths and Science. Stop and Think is a computer programme that aims to raise Key Stage 2 Maths and Science attainment by improving pupils’ ability to respond to and learn from counterintuitive concepts. The aim of this study is to assess the effect of Stop and Think on the Maths and Science attainment of Year 3 and 5 pupils.
Who can participate?
All Year 3 and 5 pupils in eligible schools can participate. All non-academically selective, state primary schools in England will be eligible for the trial unless pupils from Year 3 and Year 5 are taught in the same class; the school has previously partnered with the Stop and Think programme; or the school has taken part in piloting work for the misconceptions tests which are being used as a secondary outcome measure.
What does the study involve?
Every participating school will have one intervention and one control year group. The control year group will receive teaching as usual. For the intervention year group schools will be expected to deliver a total of 30 Stop and Think sessions, three times per week over a 10-week period. Each session lasts around 12 minutes. The intervention is delivered to pupils at the start of Maths and Science lessons by their teacher or teaching assistant. The intervention is designed to be a whole-class activity, with children working through the problems together as a group. Teachers may decide how the pupils interact with the software to input their answers, as long as the process of selecting is not based on the first child who responds (as this would undermine the ‘stop and think’ process). Maths and Science attainment are measured among pupils from disadvantaged backgrounds and all Year 3 and 5 pupils at the end of the study.
Where is the study run from?
The National Centre for Social Research (NatCen) and the Behavioural Insights Team (UK)
When is the study starting and how long is it expected to run for?
August 2020 to April 2024
Who is funding the study?
Education Endowment Foundation (EEF) (UK)
Who is the main contact?
Helena Takala, Helena.takala@natcen.ac.uk
Dr Enes Duysak, enes.duysak@natcen.ac.uk
Contact information
Public, Scientific
35 Northampton Square
London
EC1V 0AX
United Kingdom
| Phone | +44 (0)20 75497175 |
|---|---|
| Helena.takala@natcen.ac.uk |
Scientific
35 Northampton Square
London
EC1V 0AX
United Kingdom
| Phone | +44 (0)207 549 8505 |
|---|---|
| enes.duysak@natcen.ac.uk |
Study information
| Study design | Two-arm cluster randomized controlled effectiveness trial |
|---|---|
| Primary study design | Interventional |
| Secondary study design | Cluster randomised trial |
| Study setting(s) | School |
| Study type | Other |
| Participant information sheet | Not available in web format, please use contact details to request a participant information sheet. |
| Scientific title | Stop and think: learning counterintuitive concepts |
| Study acronym | Stop & Think |
| Study objectives | 1. What is the impact of Stop and Think on Maths attainment of Year 3 and 5 pupils from disadvantaged backgrounds? 2. What is the impact of Stop and Think on Maths attainment of all Year 3 and 5 pupils? 3. What is the impact of Stop and Think on Science attainment of all Year 3 and 5 pupils? 4. What is the impact of Stop and Think on Science attainment of all Year 3 and 5 pupils from disadvantaged backgrounds? 5. What is the impact of Stop and Think on all Year 3 and 5 pupils' misconception in Maths? 6. What is the impact of Stop and Think on all Year 3 and 5 pupils' misconception in Science? |
| Ethics approval(s) | Approved 09/11/2020, NatCen Research Ethics Committee (REC) (35 Northampton Square, London, EC1V 0AX, UK; +44 (0)20 7250 1866; joanne.mcbride@natcen.ac.uk), ref: P15462 |
| Health condition(s) or problem(s) studied | Maths/Science attainment of Year 3 and 5 pupils from disadvantaged backgrounds |
| Intervention | The intervention is called Stop and Think : Learning Counterintuitive Concepts. During the intervention delivery period, schools will be expected to deliver a total of 30 Stop and Think sessions, three times per week over a 10-week period. Each session lasts around 12 minutes. The intervention is delivered to pupils at the start of Maths and Science lessons by their teacher or teaching assistant. The intervention is designed to be a whole-class activity, with children working through the problems together as a group. Teachers may decide how the pupils interact with the software to input their answers, as long as the process of selecting is not based on the first child who responds (as this would undermine the ‘stop and think’ process). Every school involved in the trial will have one intervention and one control year group, ensuring equal allocation to the treatment and control groups. School-level randomisation will assign Year 3 in each school to treatment or control, with Year 5 in the same school assigned to the opposite condition. So, for example, in a school in which Year 3 is randomly assigned to treatment, Year 5 will be allocated to control. This means that 50% of schools will have Year 3 allocated to treatment and Year 5 allocated to control, while the remaining 50% of schools will have Year 5 allocated to treatment and Year 3 allocated to control. Before randomisation, schools will be stratified by class-form entry size (whether there are 1, 2 or 3+ classes per year group per year) and the school-level proportion of free school meal (FSM)-eligible pupils prior to randomisation to ensure balance across treatment and control groups. The control group will receive teaching as usual. |
| Intervention type | Behavioural |
| Primary outcome measure | Mathematical skills and knowledge measured using the GL Progress Test in Maths (GL PTM). Age-appropriate versions of the test will be delivered to the two year groups (PTM8 for Year 3 and PTM10 for Year 5). The primary outcome will be measured at endline after the intervention is completed. Please note that 50% of pupils in the trial will be randomly allocated to take Maths tests (the primary outcome measure) and 50% to take Science tests (secondary outcome measure). |
| Secondary outcome measures | 1. Science skills and knowledge measured using the GL Progress Test in Science (GL PTS). Age-appropriate versions of this test will be delivered to the two year groups (PTS8 for Year 3 and PTS10 for Year 5). 2. Common misconceptions in Maths and Science measured using tests developed by NatCen to estimate the effect of Stop and Think on intermediate outcomes. Four different tests will be developed for this analysis (one test per subject per year group). During endline testing, pupils will take age-appropriate tests for common misconceptions in Maths or Science, depending on their randomised allocation to either Maths or Science attainment tests as outlined above. The 50% of pupils randomised to take the GL Progress Test in Maths will also take the common misconceptions test in Maths. The 50% randomised to take the GL Progress Test in Science will take the common misconceptions test in Science. The secondary outcomes will be measured at endline after the intervention is completed. |
| Overall study start date | 19/08/2020 |
| Completion date | 30/04/2024 |
Eligibility
| Participant type(s) | Other |
|---|---|
| Age group | Child |
| Sex | Both |
| Target number of participants | The researchers aim to recruit 180 primary schools assuming that 9% will be lost from the trial after recruitment, meaning 164 schools will be included in the post-intervention analysis. The researchers expect to have around 6840 pupils across two trial arms from 180 primary schools. |
| Key inclusion criteria | All non-academically selective, state primary schools in England will be eligible for the trial unless: 1. Pupils from Year 3 and Year 5 are taught in the same class 2. The school has previously partnered with the Stop and Think programme 3. The school has taken part in piloting work for the misconceptions tests which are being used as a secondary outcome measure All Year 3 and Year 5 pupils in recruited schools will then be eligible for the trial. |
| Key exclusion criteria | Primary schools and their pupils will not be eligible to participate if: 1. Pupils from Year 3 and Year 5 are taught in the same class 2. The school has previously partnered with the Stop and Think programme 3. The school has taken part in piloting work for the misconceptions tests |
| Date of first enrolment | 01/11/2021 |
| Date of final enrolment | 31/07/2022 |
Locations
Countries of recruitment
- England
- United Kingdom
Study participating centre
London
SW1H 9NP
United Kingdom
Sponsor information
Charity
35 Northampton Square
London
EC1V 0AX
United Kingdom
| Phone | +44 (0)2072501866 |
|---|---|
| info@natcen.ac.uk | |
| Website | https://www.natcen.ac.uk/ |
"ROR" |
https://ror.org/057z98j75 |
Charity
Millbank Tower
21-24 Millbank
London
SW1P 4QP
United Kingdom
| Phone | +44 (0)207 802 1676 |
|---|---|
| info@eefoundation.org.uk | |
| Website | https://educationendowmentfoundation.org.uk/ |
|
"ROR" |
https://ror.org/03bhd6288 |
Funders
Funder type
Charity
Private sector organisation / Trusts, charities, foundations (both public and private)
- Alternative name(s)
- EducEndowFoundn, Education Endowment Foundation | London, EEF
- Location
- United Kingdom
Results and Publications
| Intention to publish date | 30/04/2025 |
|---|---|
| Individual participant data (IPD) Intention to share | No |
| IPD sharing plan summary | Stored in repository |
| Publication and dissemination plan | 1. Submit the final version of the Education Endowment Foundation (EEF) report 2. Submit the data to EEF data archive 3. Update the ISRCTN registry with results 4. The study protocol will be available in May 2021, and the Statistical Analysis Plan (SAP) for this study will be published in January 2022 |
| IPD sharing plan | The Education Endowment Foundation (EEF) requires the researchers to store pupil level data in a repository. They will be archiving only anonymised impact evaluation data, including primary outcomes, secondary outcomes and any other data required for the statistical analysis, linked to pupil-level National Pupil Database (NPD). Upon project completion, relevant datasets will be submitted to the EEF data archive, managed by FFT Education. The EEF data archive will then be transferred to the Office for National Statistics (ONS) through the Secure Research Service (SRS). The dataset will be stored in the FFT space within the SRS. Once the dataset is archived in the FFT space within the SRS, the archive will be open to the research community for analysis within the SRS. The EEF is committed to supporting open science and methodological developments by facilitating lawful access to their data archive for secondary analysis. It is likely this will follow the typical process for accessing NPD data by applying to the DfE, with additional EEF involvement at the approval stage. To ensure the data submitted to the archive is processed in accountable ways, the researchers will apply strict data protection safeguards from the very start of the evaluation. This includes ensuring that evaluation participants (data subjects) and/ or their parents, carers or legal guardians are sent information letters and/ or privacy notices that are fully compliant with the General Data Protection Regulation and Data Protection Act 2018. At the end of the project, these documents must be included in the evaluation report. Purposes mentioned above and the data sharing, linking, and matching required for them, will be clear to participants at recruitment stage. Clear opportunities to withdraw from such data processing in case of objections will also be provided. |
Study outputs
| Output type | Details | Date created | Date added | Peer reviewed? | Patient-facing? |
|---|---|---|---|---|---|
| Funder report results | 12/03/2025 | 02/05/2025 | No | No | |
| Protocol file | version 1 | 28/03/2022 | 02/05/2025 | No | No |
| Statistical Analysis Plan | version 1 | 05/10/2023 | 02/05/2025 | No | No |
Additional files
Editorial Notes
06/05/2025: Contact details updated.
02/05/2025: The following changes were made to the trial record:
1. Uploaded protocol (not peer-reviewed) as an additional file.
2. The statistical analysis plan was uploaded as an additional file.
3. A funder report was uploaded as an additional file.
17/08/2021: The study contact has been updated and the plain English summary has been updated accordingly.
06/05/2021: Trial's existence confirmed by the NatCen Ethics Committee.
