Digital feedback in primary maths
| ISRCTN | ISRCTN50447665 |
|---|---|
| DOI | https://doi.org/10.1186/ISRCTN50447665 |
| Secondary identifying numbers | RR-1974-EEF |
- Submission date
- 02/06/2017
- Registration date
- 20/06/2017
- Last edited
- 03/04/2020
- Recruitment status
- No longer recruiting
- Overall study status
- Completed
- Condition category
- Other
Plain English summary of protocol
Background and study aims:
Of all possible teacher influences on pupil achievement, teacher feedback is among the most powerful. Some studies have suggested that computer-assisted feedback or feedback provided in the form of video or audio are among the most effective. The use of digital feedback will allow teachers to additionally improve their feedback in two ways. Firstly, it allows teachers to provide specific feedback, tailored to the needs of individual pupils. Secondly, it allows teachers to provide two different types of feedback: direct feedback, which is corrective in nature, and focuses on the work that is being conducted in the class at that time, and delayed feedback which aims at building a broader understanding of what is being taught in the curriculum. The aim of this study is to find out whether digital feedback has a positive effect of pupils attainment in mathematics.
Who can participate?
All Year 4 and 5 mathematics teachers and Year 4 and 5 pupils at participating state schools.
What does the study involve?
All year 4 or 5 classes in a given school are randomly allocated to one of two groups. Those in the first group continue with business as usual for the duration of the study. Those in the econd group take part in the digital feedback program. Teachers are provided with resources to quickly assess their pupils’ understanding of a topic, for example, through multiple choice questions (MCQ). Teachers are then supported to provide effective feedback to help those pupils struggling with a topic, by using tablets to record video summaries of their feedback, rather than writing down comments. The pupils then have class time to review the feedback and develop from it.
What are the possible benefits and risks of participating?
There are no financial costs, nor financial benefits, for participating in the evaluation. By participating in the evaluation, teachers will be providing valuable information that may help other schools decide whether this form of feedback is effective and worth pursuing. There is minimal risk in this evaluation which largely consists of the completion of surveys. There is a small risk that confidential information is released to the public. Only RAND will have access to any of the information that is collected. RAND will monitor any concerns teachers may have through surveys and will report anonymous and aggregated results to the implementation team and schools. In schools where only small number of teachers participate the anonymity of teachers will be guaranteed through combined reporting for schools with small numbers of participating teachers.
Where is the study run from?
The study is run from RAND Europe and Bishop Grosseteste University and takes place in 40 schools in the Lincolnshire area in England (UK)
When is study starting and how long is it expected to run for?
December 2016 to March 2019
Who is funding the study?
Education Endowment Foundation (UK)
Who is the main contact?
1. Dr Alex Sutherland (public),
alex_sutherland@rand.org
2. Ms Miriam Broeks (scientific),
mbroeks@rand.org
Contact information
Public
RAND Europe
Westbrook Centre
Milton Road
Cambridge
CB4 1YG
United Kingdom
| Phone | +44 1223 353 329 ext. 2589 |
|---|---|
| alex_sutherland@rand.org |
Scientific
RAND Europe
Westbrook Centre
Milton Road
Cambridge
CB4 1YG
United Kingdom
| Phone | +44 1223 353 329 ext. 2534 |
|---|---|
| mbroeks@rand.org |
Study information
| Study design | Two-arm randomised controlled trial |
|---|---|
| Primary study design | Interventional |
| Secondary study design | Randomised controlled trial |
| Study setting(s) | School |
| Study type | Other |
| Participant information sheet | Not available in web format, please use the contact details below to request a patient information sheet |
| Scientific title | The effectiveness of digital feedback on maths performance in children in year 4 and 5 of primary school |
| Study acronym | Kyra |
| Study objectives | Hypotheses: 1. Digital feedback (both immediate and delayed) will have a positive effect on pupil outcomes (in mathematics) who are in randomly assigned intervention classes 2. Digital feedback will have a positive effect on improving outcomes for FSM pupils |
| Ethics approval(s) | 1. RAND's Human Subject's Protection Committee (HSPC), 28/04/2017, ref: 2017-0100-AM01 2. RAND Internal Review Board (IRB), 28/04/2017, ref: IRB00000051 |
| Health condition(s) or problem(s) studied | Maths performance in typically developing children in year 4 and year 5 |
| Intervention | The study will consist of a two-arm randomised control trial (RCT) in 40 primary schools in England, using year 4 and year 5 classes as the unit of assignment. The intervention will start September 2017 and last for the duration of the school year 2017-2018. All year 4 or 5 classes in a given school will be randomly assigned to treatment or control. We will aim to stratify the randomisation by teachers’ and students’ performance, such as years of teaching experience, average baseline scores and the proportion of Free School Meals (FSM) students, depending on the availability of data. All year groups which are part of the trial will have a 50% chance to be assigned to the treatment condition. Control group: Schools in the control group will teach “business as usual”, apart from that pupils will be asked to make an external mathematics test that will be independently administered and invigilated in June 2018. Intervention group: Year 4 and Year 5 mathematics teachers in the intervention group will receive initial training in the use of the app and in giving effective feedback, and take part in monthly “professional learning communities”, led by the intervention team and Bishop Grosseteste University. A Research Lead in each school will provide ongoing support and coaching to the teachers. Research Leads will also receive training in the use of the app and in giving effective feedback. The training will seek to familiarise teachers and Research Leads with the app to ensure they know how to work with it. In addition, teachers and Research Leads will be trained in giving high-quality feedback, which covers the content of feedback. Teachers will be expected to embed the approach into their normal class teaching and will receive additional support from the Research Lead in doing so. Two main procedures form the core of the intervention: 1. Diagnostic tests and video feedback 2. Coaching and professional learning communities (PLCs) Pupils in the intervention group classes will receive digital feedback throughout the school year 2017-2018. The intervention will be evaluated throughout the year with the last data collection point directly after the intervention has ended (June-July 2018). |
| Intervention type | Behavioural |
| Primary outcome measure | Attainment in mathematics is assessed using digital tests administered and collected by the Australian Council for Educational Research (ACER) at the end of the academic year 2017-2018. |
| Secondary outcome measures | Pupil engagement is measured using a pupil engagement test administered to all pupils at the end of the academic year 2017-2018. |
| Overall study start date | 01/12/2016 |
| Completion date | 01/03/2019 |
Eligibility
| Participant type(s) | Healthy volunteer |
|---|---|
| Age group | Adult |
| Sex | Both |
| Target number of participants | 120 |
| Key inclusion criteria | Schools: 1. State schools 2. Have enough tablets for one year group/class (roughly 1 tablet for every 4 or 5 pupils) 3. Have not had the Digital Feedback intervention before Teachers: All Year 4 and Year 5 mathematics teachers in schools taking part in the trial will be included in the trial. Pupils: All Year 4 and Year 5 pupils attending participating schools. |
| Key exclusion criteria | Schools: Have had the Digital Feedback intervention before. Teachers: Do not teach year 4 or 5. Pupils: Not in year 4 or 5. |
| Date of first enrolment | 01/03/2017 |
| Date of final enrolment | 12/05/2017 |
Locations
Countries of recruitment
- England
- United Kingdom
Study participating centres
Milton Road
Cambridge
CB4 1YG
United Kingdom
Lincoln
LN1 3DY
United Kingdom
Sponsor information
Charity
9th Floor
Millbank Tower
21-24 Millbank
london
SW1P 4QP
United Kingdom
| Phone | +44 207 802 1676 |
|---|---|
| info@eefoundation.org.uk | |
| Website | https://educationendowmentfoundation.org.uk/ |
"ROR" |
https://ror.org/03bhd6288 |
Funders
Funder type
Charity
No information available
Results and Publications
| Intention to publish date | 01/10/2019 |
|---|---|
| Individual participant data (IPD) Intention to share | Yes |
| IPD sharing plan summary | Available on request |
| Publication and dissemination plan | The findings from this study will be reported in the main study report containing details of the study design, implementation, process data and results for all included pupils. This will be reported in the Spring of 2019 using the Education Endowment Foundation (EEF- the funder of this trial) reporting template. The research team will also seek to publish the results in an educational journal but this will not be published until after the EEF report. |
| IPD sharing plan | The datasets generated during and/or analysed during the current study are/will be available upon request from the Education Endowment Foundation (EEF) from Peter Henderson (Peter.Henderson@eefoundation.org.uk) |
Study outputs
| Output type | Details | Date created | Date added | Peer reviewed? | Patient-facing? |
|---|---|---|---|---|---|
| Funder report results | results | 03/04/2020 | No | No |
Editorial Notes
03/04/2020: Publication reference added.
09/09/2019: The intention to publish date was changed from 01/03/2019 to 01/10/2019.
03/05/2019: The following changes were made to the trial record:
1. Drs Julie Belanger and Janna van Belle were removed as study contacts.
2. Dr Alex Sutherland was added as public contact.
3. Miriam Broeks was added as scientific contact.
4. The plain English summary was updated to reflect these changes.
