The effect of incentive payments for teachers on pupil attainment
| ISRCTN | ISRCTN12967252 |
|---|---|
| DOI | https://doi.org/10.1186/ISRCTN12967252 |
| Secondary identifying numbers | N/A |
- Submission date
- 21/02/2017
- Registration date
- 24/02/2017
- Last edited
- 03/09/2019
- Recruitment status
- Stopped
- Overall study status
- Stopped
- Condition category
- Other
Plain English summary of protocol
Background and study aims
The aim of this study is to find out whether monetary incentives and coaching for teachers motivates them to improve their pupils’ attainment. Although some recent research has demonstrated that performance-based compensation holds promise for improving pupil achievement, in three studies researchers found small, non-significant impacts on pupil achievement. However, research from developing nations indicates that teacher performance pay can be effective. One study found that providing teachers with bonuses upfront, and then asking for money back if pupil achievement does not improve sufficiently, was an effective way to improve pupil achievement. Coaching is also included as part of this study because some teachers may be motivated by pay incentives, but may not know what to do to improve their performance. Coaching is intended to fill this gap and may increase teacher performance compared to the use of incentive pay alone. There is evidence that coaching can improve both teacher performance and pupil outcomes.
Who can participate?
Year 2-6 primary school academy teachers and Year 7-8 secondary academy mathematics teachers
What does the study involve?
Participating teachers are randomly allocated to either the intervention or the control group for financial incentives. Teachers allocated to the intervention group receive a “loss-framed” financial incentive of £4000 at the beginning of the school year. This is defined as “loss-framed” because a percentage of this amount may be recouped at the end of the year depending on the teacher’s pupils’ progress relative to similar pupils. Teachers allocated to the control group continue with 'business as usual'. A group of teachers are also randomly selected to receive coaching. This excludes teachers who act as coaches and teachers who definitely need to receive coaching as indicated by the head teacher. Teachers allocated to the coaching condition, as well as teachers who definitely need coaching, receive personalised coaching throughout the year during which lessons are observed weekly and small changes are recommended by the coach during short 30-minute sessions. At the end of the 2017-2018 school year the mathematics test results of the two groups are compared.
What are the possible benefits and risks of participating?
For pupils in participating classes the study may improve learning outcomes, and no real risks are anticipated. Teachers allocated to receive financial incentives may experience stress. The loss aversion setup, whereby teachers receive a sum of money at the start of the year that may have to be repaid, can carry the following risks of stress: financial stress following a fear of spending the money over the course of the year without being able to repay; lack of understanding of how the incentive repayment works, leading to false expectations about financial gains (or losses); and stress about teaching practices to ensure that pupils achieve higher grades in order to retain the financial incentive.
Where is the study run from?
RAND Europe (UK)
When is the study starting and how long is it expected to run for?
September 2016 to December 2019
Who is funding the study?
Education Endowment Foundation (UK)
Who is the main contact?
Dr Alex Sutherland
Contact information
Scientific
Westbrook Centre
Milton Road
Cambridge
CB4 1YG
United Kingdom
 ORCID ID |
0000-0003-0276-9705 |
|---|
Study information
| Study design | Two-stage randomised controlled trial |
|---|---|
| Primary study design | Interventional |
| Secondary study design | Randomised controlled trial |
| Study setting(s) | School |
| Study type | Other |
| Participant information sheet | Not available in web format, please use the contact details to request a participant information sheet |
| Scientific title | A randomised controlled trial of the 'Incentives and Coaching Research' intervention for pupils in Primary Years 2-6 and Secondary Maths pupils in Year 7 and 8 in schools which are part of the United Learning Academies in the UK |
| Study acronym | Payment by Results in Education (PbREd) |
| Study objectives | 1. Hypothesis 1: financial incentives (loss-framed) will have a positive effect on pupil outcomes (in this trial mathematics) who are in randomly assigned intervention classes. 2. Hypothesis 2: coaching of teachers will have a positive effect on pupil outcomes (in this trial mathematics) who are in randomly assigned intervention classes. 3. Hypothesis 3: financial incentives (loss-framed) combined with coaching will have a greater positive effect on pupil outcomes (in this trial mathematics) than those who are randomly assigned to control classes or to incentives alone. |
| Ethics approval(s) | RAND Human Subjects Protection Committee, 26/01/2017, ref: RAND HSPC: 2016-0475-AM01 |
| Health condition(s) or problem(s) studied | Mathematics comprehension in Year 2-6 Primary school pupils and Year 7-8 Secondary school pupils |
| Intervention | Participating teachers will be allocated to either the intervention or the control group for financial incentives. Teachers randomised to the financial incentive condition will receive a “loss-framed” financial incentive of £4000 at the beginning of the school year. This is defined as “loss-framed” because a percentage of this amount may be recouped at the end of the year depending on the teacher’s pupils’ progress relative to similar pupils. Control groups will proceed with 'business as usual'. In a second round of randomisation, a group of teachers will be randomised to receive coaching. Two groups of teachers will not be randomised to coaching, namely teachers who act as a coach, and teachers who definitely need to receive coaching as indicated by the head teacher. Teachers randomised to the coaching condition, as well as teachers who will definitely receive coaching, will receive personalised coaching throughout the year during which lessons are observed weekly and small changes recommended by the coach during short 30-minute sessions. Duration of the intervention: 1 year (school year 2017-2018) Follow-up: outcome testing is at the end of the 2017-2018 school year, with no further follow-up |
| Intervention type | Behavioural |
| Primary outcome measure | The trial will be measuring primary outcomes for different year groups: Year 3, 4, 5, 7 and 8: the primary outcome measure at the end of Years 3, 4, 5, 7 and 8 will be a stand-alone measure of mathematics administered in the summer term of 2018, one year after the intervention begins. The measure of mathematics will be administered and collected by the Australian Council for Educational Research (ACER). The study will use ACERs Essential Learning Metric (ELM) for Mathematics for Years 3, 4, 5, 7 and 8. The tests will be digital and will take 45-50 minutes to complete. The tests consist of two sets of measures: mathematical content (number, algebra, measurement, geometry, statistics and probability); and mathematical processes (understanding, fluency, problem solving and reasoning). Year 2 and 6: the primary outcome measure at the end of Years 2 and 6 will be the Key Stage 1 and Key Stage 2 test results for mathematics, administered in the summer term of 2018. |
| Secondary outcome measures | 1. Cost effectiveness of the intervention through collecting data on intervention costs. Cost data will be collected through surveys of head teachers and teachers. Questions on costs will be included in surveys administered at midline (3 months after the start of the intervention, around December 2017), and at endline (in the summer term of 2018). Cost data will focus on prerequisite costs required for the implementation of the intervention, additional paid staff time required, paid staff cover required and unpaid overtime required 2. Compliance with allocation, through monitoring of the payout of financial incentives and monitoring of the coaching |
| Overall study start date | 01/09/2016 |
| Completion date | 31/12/2019 |
| Reason abandoned (if study stopped) | Participant recruitment issue |
Eligibility
| Participant type(s) | Other |
|---|---|
| Age group | Child |
| Sex | Both |
| Target number of participants | The target number of schools is 42; the target number of teachers is 400 (200 financial incentives, 200 control); the expected number of pupils is 13,000, depending on school size |
| Total final enrolment | 46 |
| Key inclusion criteria | All Year 2-6 Primary school pupils in participating schools in classes in which the teacher has agreed to participate and all Year 7-8 Secondary school pupils in participating schools in mathematics classes in which the teacher has agreed to participate. |
| Key exclusion criteria | 1. Special education facilities for pupils with physical and/or mental and/or behavioural disorders 2. Pupil Referral Units |
| Date of first enrolment | 01/01/2017 |
| Date of final enrolment | 01/05/2017 |
Locations
Countries of recruitment
- England
- United Kingdom
Study participating centre
Milton Road
Cambridge
CB4 1YG
United Kingdom
Sponsor information
Research organisation
Westbrook Centre
Milton Road
Cambridge
CB4 1YG
United Kingdom
| Website | http://www.rand.org/randeurope.html |
|---|---|
"ROR" |
https://ror.org/037pk1914 |
Funders
Funder type
Charity
No information available
Results and Publications
| Intention to publish date | 01/03/2019 |
|---|---|
| Individual participant data (IPD) Intention to share | No |
| IPD sharing plan summary | Not expected to be made available |
| Publication and dissemination plan | The main report will be for the Education Endowment Foundation, which will be published in the Spring of 2019. In addition, the University of California at San Diego and RAND Europe will separately develop peer-reviewed papers. |
| IPD sharing plan | The full dataset will not be made publicly available as it contains pupil names and teacher identifiers that could compromise confidentiality. The data will be held with the EEF’s data contractor FFT Education and in an anonymised form to the UK Data Archive. The Education Endowment Foundation, the funder of the evaluation, uses data gathered from the research it funds to look at the longer-term benefits of the approaches being tried out in schools. This information is passed – in anonymous form – to the EEF’s partner FFT Education, who conduct follow ups via the National Pupil Database. Pupil names will only be used for to match pupils to the National Pupil Database. |
Editorial Notes
03/09/2019: The trial's status was changed to 'Stopped'. The trial was terminated in April 2017 because of poor recruitment (https://educationendowmentfoundation.org.uk/public/files/Motivating_teachers_with_incentivised_pay_and_coaching.pdf).
