Plain English Summary
Background and study aims:
Of all possible teacher influences on pupil achievement, teacher feedback is among the most powerful. Some studies have suggested that computer-assisted feedback or feedback provided in the form of video or audio are among the most effective. The use of digital feedback will allow teachers to additionally improve their feedback in two ways. Firstly, it allows teachers to provide specific feedback, tailored to the needs of individual pupils. Secondly, it allows teachers to provide two different types of feedback: direct feedback, which is corrective in nature, and focuses on the work that is being conducted in the class at that time, and delayed feedback which aims at building a broader understanding of what is being taught in the curriculum. The aim of this study is to find out whether digital feedback has a positive effect of pupils attainment in mathematics.
Who can participate?
All Year 4 and 5 mathematics teachers and Year 4 and 5 pupils at participating state schools.
What does the study involve?
All year 4 or 5 classes in a given school are randomly allocated to one of two groups. Those in the first group continue with business as usual for the duration of the study. Those in the econd group take part in the digital feedback program. Teachers are provided with resources to quickly assess their pupils’ understanding of a topic, for example, through multiple choice questions (MCQ). Teachers are then supported to provide effective feedback to help those pupils struggling with a topic, by using tablets to record video summaries of their feedback, rather than writing down comments. The pupils then have class time to review the feedback and develop from it.
What are the possible benefits and risks of participating?
There are no financial costs, nor financial benefits, for participating in the evaluation. By participating in the evaluation, teachers will be providing valuable information that may help other schools decide whether this form of feedback is effective and worth pursuing. There is minimal risk in this evaluation which largely consists of the completion of surveys. There is a small risk that confidential information is released to the public. Only RAND will have access to any of the information that is collected. RAND will monitor any concerns teachers may have through surveys and will report anonymous and aggregated results to the implementation team and schools. In schools where only small number of teachers participate the anonymity of teachers will be guaranteed through combined reporting for schools with small numbers of participating teachers.
Where is the study run from?
The study is run from RAND Europe and Bishop Grosseteste University and takes place in 40 schools in the Lincolnshire area in England (UK)
When is study starting and how long is it expected to run for?
December 2016 to March 2019
Who is funding the study?
Education Endowment Foundation (UK)
Who is the main contact?
1. Dr Julie Belanger (public)
2. Dr Janna van Belle (scientific)
Trial website
Contact information
Additional identifiers
EudraCT number
ClinicalTrials.gov number
Protocol/serial number
RR-1974-EEF
Study information
Scientific title
The effectiveness of digital feedback on maths performance in children in year 4 and 5 of primary school
Acronym
Kyra
Study hypothesis
Hypotheses:
1. Digital feedback (both immediate and delayed) will have a positive effect on pupil outcomes (in mathematics) who are in randomly assigned intervention classes
2. Digital feedback will have a positive effect on improving outcomes for FSM pupils
Ethics approval
1. RAND's Human Subject's Protection Committee (HSPC), 28/04/2017, ref: 2017-0100-AM01
2. RAND Internal Review Board (IRB), 28/04/2017, ref: IRB00000051
Study design
Two-arm randomised controlled trial
Primary study design
Interventional
Secondary study design
Randomised controlled trial
Trial setting
Schools
Trial type
Other
Patient information sheet
Not available in web format, please use the contact details below to request a patient information sheet
Condition
Maths performance in typically developing children in year 4 and year 5
Intervention
The study will consist of a two-arm randomised control trial (RCT) in 40 primary schools in England, using year 4 and year 5 classes as the unit of assignment. The intervention will start September 2017 and last for the duration of the school year 2017-2018. All year 4 or 5 classes in a given school will be randomly assigned to treatment or control. We will aim to stratify the randomisation by teachers’ and students’ performance, such as years of teaching experience, average baseline scores and the proportion of Free School Meals (FSM) students, depending on the availability of data. All year groups which are part of the trial will have a 50% chance to be assigned to the treatment condition.
Control group: Schools in the control group will teach “business as usual”, apart from that pupils will be asked to make an external mathematics test that will be independently administered and invigilated in June 2018.
Intervention group: Year 4 and Year 5 mathematics teachers in the intervention group will receive initial training in the use of the app and in giving effective feedback, and take part in monthly “professional learning communities”, led by the intervention team and Bishop Grosseteste University. A Research Lead in each school will provide ongoing support and coaching to the teachers. Research Leads will also receive training in the use of the app and in giving effective feedback. The training will seek to familiarise teachers and Research Leads with the app to ensure they know how to work with it. In addition, teachers and Research Leads will be trained in giving high-quality feedback, which covers the content of feedback. Teachers will be expected to embed the approach into their normal class teaching and will receive additional support from the Research Lead in doing so. Two main procedures form the core of the intervention:
1. Diagnostic tests and video feedback
2. Coaching and professional learning communities (PLCs)
Pupils in the intervention group classes will receive digital feedback throughout the school year 2017-2018. The intervention will be evaluated throughout the year with the last data collection point directly after the intervention has ended (June-July 2018).
Intervention type
Behavioural
Phase
Drug names
Primary outcome measures
Attainment in mathematics is assessed using digital tests administered and collected by the Australian Council for Educational Research (ACER) at the end of the academic year 2017-2018.
Secondary outcome measures
Pupil engagement is measured using a pupil engagement test administered to all pupils at the end of the academic year 2017-2018.
Overall trial start date
01/12/2016
Overall trial end date
01/03/2019
Reason abandoned
Eligibility
Participant inclusion criteria
Schools:
1. State schools
2. Have enough tablets for one year group/class (roughly 1 tablet for every 4 or 5 pupils)
3. Have not had the Digital Feedback intervention before
Teachers:
All Year 4 and Year 5 mathematics teachers in schools taking part in the trial will be included in the trial.
Pupils:
All Year 4 and Year 5 pupils attending participating schools.
Participant type
Healthy volunteer
Age group
Adult
Gender
Both
Target number of participants
120
Participant exclusion criteria
Schools:
Have had the Digital Feedback intervention before.
Teachers:
Do not teach year 4 or 5.
Pupils:
Not in year 4 or 5.
Recruitment start date
01/03/2017
Recruitment end date
12/05/2017
Locations
Countries of recruitment
United Kingdom
Trial participating centre
RAND Europe
Westbrook Centre
Milton Road
Cambridge
CB4 1YG
United Kingdom
Trial participating centre
Bishop Grosseteste University
Longdales Road
Lincoln
LN1 3DY
United Kingdom
Sponsor information
Organisation
Education Endowment Foundation
Sponsor details
9th Floor
Millbank Tower
21-24 Millbank
london
SW1P 4QP
United Kingdom
+44 207 802 1676
info@eefoundation.org.uk
Sponsor type
Charity
Website
Funders
Funder type
Charity
Funder name
Education Endowment Foundation
Alternative name(s)
Funding Body Type
Funding Body Subtype
Location
Results and Publications
Publication and dissemination plan
The findings from this study will be reported in the main study report containing details of the study design, implementation, process data and results for all included pupils. This will be reported in the Spring of 2019 using the Education Endowment Foundation(EEF- the funder of this trial) reporting template. The research team will also seek to publish the results in an educational journal but this will not be published until after the EEF report.
IPD sharing statement:
The datasets generated during and/or analysed during the current study are/will be available upon request from the EducationEndowment Foundation (EEF) from Peter Henderson (Peter.Henderson@eefoundation.org.uk)
Intention to publish date
01/03/2019
Participant level data
Available on request
Results - basic reporting
Publication summary