The external evaluation of Birkbeck University of London: Learning of Counterintuitive Concepts
| ISRCTN | ISRCTN20284041 |
|---|---|
| DOI | https://doi.org/10.1186/ISRCTN20284041 |
| Secondary identifying numbers | EECC |
- Submission date
- 28/11/2017
- Registration date
- 02/01/2018
- Last edited
- 11/09/2020
- Recruitment status
- No longer recruiting
- Overall study status
- Completed
- Condition category
- Other
Plain English summary of protocol
Background and study aims
This study is linked to the Birkbeck UNLOCKE project (https://www.isrctn.com/ISRCTN54726482). This phase of the study assesses whether the counterintuitive concepts intervention 'Stop and Think' produces improvements in children's scientific and mathematical understanding.
Who can participate?
Children in Years 3 and 5 (ages 8 and 10) from about 80 primary schools across England
What does the study involve?
Classes are randomly allocated to either an intervention, active control or passive control group. Both the intervention and active control groups participate in teacher-led computer-based programmes. The intervention is 'Stop and Think', which uses content based on the maths and science curriculum and is delivered as a short session before the children's maths or science lesson. The active control is See+, with content based on the Personal, Social and Health Education (PSHE) curriculum. Both are delivered three times a week for a period of 10 weeks. The passive control group continue with "business as usual". The children's maths and science test scores are compared before and after the study to assess whether the intervention has affected their mathematical and scientific understanding.
What are the possible benefits and risks of participating?
This study may identify whether the Stop and Think computer programme is a useful tool in boosting children's mathematical and scientific understanding and whether it could and should be rolled out more widely. The way the study has been designed means that all schools get to use the computer-based intervention in either year 3 or year 5. Since the intervention and control is split along the year groups children are unlikely to feel like they are missing out. Some children may find the process of testing stressful but it will be completed in the familiar classroom environment.
Where is the study run from?
National Foundation of Educational Research (UK)
When is study starting and how long is it expected to run for?
January 2017 to November 2017
Who is funding the study?
Education Endowment Foundation (UK)
Who is the main contact?
Simon Rutt
Contact information
Scientific
National Foundation of Educational Research
The Mere
Upton Park
Slough
SL1 2DQ
United Kingdom
Study information
| Study design | Multicentre interventional cluster randomised controlled trial |
|---|---|
| Primary study design | Interventional |
| Secondary study design | Cluster randomised trial |
| Study setting(s) | School |
| Study type | Other |
| Participant information sheet | Not available in web format, please use the contact details to request a patient information sheet |
| Scientific title | UNLOCKE: Understanding the learning of counterintuitive concepts in science and mathematics education through a behavioural and neuroimaging study of year 3 and year 5 primary-school inhibitory control training (in comparison to social skills training or lessons as usual) |
| Study objectives | This research trial is linked to the Birkbeck registered trial ISRCTN54726482. It is the External Statistical Evaluation of the intervention. The primary research question is: does the use of the counterintuitive learning intervention impact on learners' mathematics and science achievement? The secondary research is: what is the impact of the counterintuitive learning intervention on learners' inhibition control? An additional secondary research question will explore if there is an impact of the social skills computer programme, used by the control plus group, on learners' mathematics and science achievement in comparison to the counterintuitive concepts programme. This will determine if any identifiable effect is due to using a computer programme rather than any specific content. |
| Ethics approval(s) | Birkbeck received approval from University College London Research Ethics Committee, 01/07/2017, ref: 1602/006. The final trial design will be reviewed by NFER's ethics and code of conduct committee. The trial does not include the solely Birkbeck-based additional neurocognitive testing which required opt-in parental consent. |
| Health condition(s) or problem(s) studied | Inhibitory control and maths and science performance in primary school education |
| Intervention | Multicentre interventional 10-week trial with random allocation of groups to intervention, active and passive control conditions assessed by blind evaluators. A cluster design is planned, where year 3 and year 5 groups will be randomly allocated to either the intervention or control/control plus with 1:1 ratio. This will be unbalanced design with a ratio of 2:1:1 for the intervention versus the control and control plus. This will mean that no school is solely a control school which will help with recruitment and reduce attrition. Classes assigned to intervention will receive the teacher-led computer-based activity called 'Stop and Think', which will introduce children to counterintuitive concepts in mathematics and science. This will occur as a 12-minute intervention prior to mathematics or science lessons three times a week. Classes assigned to control will carry on 'business-as-usual'. Classes assigned to control plus will receive a teacher-led computer-based learning activity called See+, which will occur during PSHE lessons. During this activity, children observe and reflect upon social interactions and engage in social-emotional learning through a series of computerised animated stories with virtual characters engaging in social scenarios. All statistical analysis of the intervention impact will be conducted at the pupil level, comparing the average pupil maths and science scores in the intervention group with average scores in the control groups. The average difference will be measured in a multilevel linear regression model that takes account of pre-test score as covariate, along with the stratification variable used at randomisation (whether the school is a one form entry or two form entry). a three level model will be created to account for pupils being clustered within classes and within schools. The main analysis will be 'intention to treat', reflecting the reality of how interventions are delivered in practice and avoiding attrition bias. |
| Intervention type | Other |
| Primary outcome measure | Mathematics and science year 3 and year 5 outcomes measured using the GL Assessments Progress Test in Maths and Progress Test in Science at 2 weeks following intervention completion. PTM8 and PTS8 will be used for year 3 and PTS10 and PTM10 will be used for year 5. Key Stage 1 assessment data is used as a pre-test measure (and therefore covariate in any analysis). A combined KS1 assessment measure will be used that would include task/test assessments in reading, writing and maths as the correlation with science outcomes at KS2 is still strong. |
| Secondary outcome measures | Pupil scores on the chimeric animal Stroop task at 2 weeks following intervention completion |
| Overall study start date | 01/01/2017 |
| Completion date | 30/11/2017 |
Eligibility
| Participant type(s) | Other |
|---|---|
| Age group | Child |
| Lower age limit | 8 Years |
| Upper age limit | 10 Years |
| Sex | Both |
| Target number of participants | All Year 3 and Year 5 pupils present in the selected schools, apart from those who have opted-out. This equates to ~6700 children across ~280 classes in ~80 schools. |
| Key inclusion criteria | 1. In Year 3 and 5 (approximate ages 8 and 10) of one of our participating schools 2. Male or female 3. Consent given with not opting-out |
| Key exclusion criteria | Participants will be excluded from the statistical analysis if the parental opt-out form is received by Birkbeck |
| Date of first enrolment | 01/01/2017 |
| Date of final enrolment | 31/10/2017 |
Locations
Countries of recruitment
- England
- United Kingdom
Study participating centre
Upton Park
Slough
SL1 2DQ
United Kingdom
Sponsor information
Charity
9th Floor
Millbank Tower
21-24 Millbank
London
SW1P 4QP
United Kingdom
"ROR" |
https://ror.org/03bhd6288 |
|---|
Funders
Funder type
Charity
No information available
Private sector organisation / International organizations
- Location
- United Kingdom
Results and Publications
| Intention to publish date | 01/11/2018 |
|---|---|
| Individual participant data (IPD) Intention to share | No |
| IPD sharing plan summary | Stored in repository |
| Publication and dissemination plan | The protocol will published online shortly after the trial is registered. The statistical analysis plan will be published early next year. Study results will be published as a part of a joint EEF publication with other RCT research studies in November 2018. The publication will be freely available to download. The research itself will be peer reviewed prior to publication. |
| IPD sharing plan | The evaluator (NFER) will submit the trial data to EEF’s data contractor Fisher Family Trust (FFT). FFT will collate and anonymise the data for upload to the UK Data Archive. The archived data will be available in an anonymised form with restricted access for research purposes only. |
Study outputs
| Output type | Details | Date created | Date added | Peer reviewed? | Patient-facing? |
|---|---|---|---|---|---|
| Funder report results | results | 11/09/2020 | No | No |
Editorial Notes
11/09/2020: Publication reference added.
