Plain English Summary
Background and study aims
This study is linked to the Birkbeck UNLOCKE project (https://www.isrctn.com/ISRCTN54726482). This phase of the study assesses whether the counterintuitive concepts intervention 'Stop and Think' produces improvements in children's scientific and mathematical understanding.
Who can participate?
Children in Years 3 and 5 (ages 8 and 10) from about 80 primary schools across England
What does the study involve?
Classes are randomly allocated to either an intervention, active control or passive control group. Both the intervention and active control groups participate in teacher-led computer-based programmes. The intervention is 'Stop and Think', which uses content based on the maths and science curriculum and is delivered as a short session before the children's maths or science lesson. The active control is See+, with content based on the Personal, Social and Health Education (PSHE) curriculum. Both are delivered three times a week for a period of 10 weeks. The passive control group continue with "business as usual". The children's maths and science test scores are compared before and after the study to assess whether the intervention has affected their mathematical and scientific understanding.
What are the possible benefits and risks of participating?
This study may identify whether the Stop and Think computer programme is a useful tool in boosting children's mathematical and scientific understanding and whether it could and should be rolled out more widely. The way the study has been designed means that all schools get to use the computer-based intervention in either year 3 or year 5. Since the intervention and control is split along the year groups children are unlikely to feel like they are missing out. Some children may find the process of testing stressful but it will be completed in the familiar classroom environment.
Where is the study run from?
National Foundation of Educational Research (UK)
When is study starting and how long is it expected to run for?
January 2017 to November 2017
Who is funding the study?
Education Endowment Foundation (UK)
Who is the main contact?
Simon Rutt
Trial website
Additional identifiers
EudraCT number
ClinicalTrials.gov number
Protocol/serial number
EECC
Study information
Scientific title
UNLOCKE: Understanding the learning of counterintuitive concepts in science and mathematics education through a behavioural and neuroimaging study of year 3 and year 5 primary-school inhibitory control training (in comparison to social skills training or lessons as usual)
Acronym
Study hypothesis
This research trial is linked to the Birkbeck registered trial ISRCTN54726482. It is the External Statistical Evaluation of the intervention.
The primary research question is: does the use of the counterintuitive learning intervention impact on learners' mathematics and science achievement?
The secondary research is: what is the impact of the counterintuitive learning intervention on learners' inhibition control? An additional secondary research question will explore if there is an impact of the social skills computer programme, used by the control plus group, on learners' mathematics and science achievement in comparison to the counterintuitive concepts programme. This will determine if any identifiable effect is due to using a computer programme rather than any specific content.
Ethics approval
Birkbeck received approval from University College London Research Ethics Committee, 01/07/2017, ref: 1602/006. The final trial design will be reviewed by NFER's ethics and code of conduct committee. The trial does not include the solely Birkbeck-based additional neurocognitive testing which required opt-in parental consent.
Study design
Multicentre interventional cluster randomised controlled trial
Primary study design
Interventional
Secondary study design
Cluster randomised trial
Trial setting
Schools
Trial type
Other
Patient information sheet
Not available in web format, please use the contact details to request a patient information sheet
Condition
Inhibitory control and maths and science performance in primary school education
Intervention
Multicentre interventional 10-week trial with random allocation of groups to intervention, active and passive control conditions assessed by blind evaluators. A cluster design is planned, where year 3 and year 5 groups will be randomly allocated to either the intervention or control/control plus with 1:1 ratio. This will be unbalanced design with a ratio of 2:1:1 for the intervention versus the control and control plus. This will mean that no school is solely a control school which will help with recruitment and reduce attrition.
Classes assigned to intervention will receive the teacher-led computer-based activity called 'Stop and Think', which will introduce children to counterintuitive concepts in mathematics and science. This will occur as a 12-minute intervention prior to mathematics or science lessons three times a week.
Classes assigned to control will carry on 'business-as-usual'.
Classes assigned to control plus will receive a teacher-led computer-based learning activity called See+, which will occur during PSHE lessons. During this activity, children observe and reflect upon social interactions and engage in social-emotional learning through a series of computerised animated stories with virtual characters engaging in social scenarios.
All statistical analysis of the intervention impact will be conducted at the pupil level, comparing the average pupil maths and science scores in the intervention group with average scores in the control groups. The average difference will be measured in a multilevel linear regression model that takes account of pre-test score as covariate, along with the stratification variable used at randomisation (whether the school is a one form entry or two form entry). a three level model will be created to account for pupils being clustered within classes and within schools. The main analysis will be 'intention to treat', reflecting the reality of how interventions are delivered in practice and avoiding attrition bias.
Intervention type
Other
Phase
Drug names
Primary outcome measure
Mathematics and science year 3 and year 5 outcomes measured using the GL Assessments Progress Test in Maths and Progress Test in Science at 2 weeks following intervention completion. PTM8 and PTS8 will be used for year 3 and PTS10 and PTM10 will be used for year 5. Key Stage 1 assessment data is used as a pre-test measure (and therefore covariate in any analysis). A combined KS1 assessment measure will be used that would include task/test assessments in reading, writing and maths as the correlation with science outcomes at KS2 is still strong.
Secondary outcome measures
Pupil scores on the chimeric animal Stroop task at 2 weeks following intervention completion
Overall trial start date
01/01/2017
Overall trial end date
30/11/2017
Reason abandoned (if study stopped)
Eligibility
Participant inclusion criteria
1. In Year 3 and 5 (approximate ages 8 and 10) of one of our participating schools
2. Male or female
3. Consent given with not opting-out
Participant type
Other
Age group
Child
Gender
Both
Target number of participants
All Year 3 and Year 5 pupils present in the selected schools, apart from those who have opted-out. This equates to ~6700 children across ~280 classes in ~80 schools.
Participant exclusion criteria
Participants will be excluded from the statistical analysis if the parental opt-out form is received by Birkbeck
Recruitment start date
01/01/2017
Recruitment end date
31/10/2017
Locations
Countries of recruitment
United Kingdom
Trial participating centre
National Foundation for Educational Research
The Mere
Upton Park
Slough
SL1 2DQ
United Kingdom
Funders
Funder type
Charity
Funder name
Education Endowment Foundation (EEF) (UK)
Alternative name(s)
Funding Body Type
Funding Body Subtype
Location
Funder name
Wellcome Trust
Alternative name(s)
Funding Body Type
private sector organisation
Funding Body Subtype
International organizations
Location
United Kingdom
Results and Publications
Publication and dissemination plan
The protocol will published online shortly after the trial is registered. The statistical analysis plan will be published early next year. Study results will be published as a part of a joint EEF publication with other RCT research studies in November 2018. The publication will be freely available to download. The research itself will be peer reviewed prior to publication.
IPD sharing plan
The evaluator (NFER) will submit the trial data to EEF’s data contractor Fisher Family Trust (FFT). FFT will collate and anonymise the data for upload to the UK Data Archive. The archived data will be available in an anonymised form with restricted access for research purposes only.
Intention to publish date
01/11/2018
Participant level data
Stored in repository
Basic results (scientific)
Publication list
2019 results in https://educationendowmentfoundation.org.uk/public/files/Projects/Evaluation_Reports/Stop_and_Think.pdf (added 11/09/2020)