Condition category
Not Applicable
Date applied
05/11/2016
Date assigned
30/11/2016
Last edited
30/11/2016
Prospective/Retrospective
Retrospectively registered
Overall trial status
Ongoing
Recruitment status
No longer recruiting

Plain English Summary

Background and study aims
There is a significant amount of evidence to suggest that timely provision of feedback improves learning. ‘Visible Classroom’ (VC) is an app and training programme that aims to encourage teachers to reflect on their teaching and develop their classroom practice. The app aims to improve student achievement by supporting teachers’ professional practice development by using personalised feedback on teaching. This feedback is designed to encourage teachers to reflect critically on their teaching and develop their classroom practice, which would be expected to have flow on effects for their pupils’ learning and attainment. The aim of the study is to test whether the VC application improves student attainment in English and Maths.

Who can participate?
Up to 140 primary schools in the UK that have at least one year 5 and one year 6 class, as long as they have not previously used the VC application.

What does the study involve?
Schools participating in the trial are randomly allocated to one of two groups. In schools in the first group, every term, primary school teachers record their normal lesson practice, and upload audio files to a website. After every upload, Ai Media, a captioning company, provides a verbatim transcript of the teacher’s talk. Alongside the word-for-word transcript will be simple analytics (e.g. speed of instruction, quantity of teacher vs student talk) a few hours after the lesson. After several hours of lessons have been uploaded, teachers receive a more detailed report, written by experts at the University of Melbourne. This will identify areas for improvement in their teaching, based on their self-selected teaching goals and the evidence from their transcripts. Teachers are expected to discuss the feedback with a mentor – another teacher at their school. In schools in the second group are expected to continue with business as usual. At the end of the school year, pupils results in KS2 exams are examined to compare attainment between the groups.

What are the possible benefits and risks of participating?
The main potential benefit of participating in the trial is the opportunity to improve pupil’s academic attainment. Even for those schools allocated to the control group, these schools still have the opportunity to benefit from the programme once the study is complete. There are no notable risks involved in participating.

Where is the study run from?
The study is being run by The Schools, Students and Teachers network (SSAT) and takes place in 140 schools from across England (UK)

When is the study starting and how long is it expected to run for?
January 2016 to November 2018

Who is funding the study?
Education Endowment Foundation (UK)

Who is the main contact?
Mr Daniel Carr

Trial website

https://educationendowmentfoundation.org.uk/our-work/projects/the-visible-classroom-2015/

Contact information

Type

Public

Primary contact

Mr Daniel Carr

ORCID ID

Contact details

4 Matthew Parker Street
Behavioural Insights Team
London
SW1H 9NP
United Kingdom

Additional identifiers

EudraCT number

ClinicalTrials.gov number

Protocol/serial number

N/A

Study information

Scientific title

Does providing feedback to teachers on their audio recorded lessons has an impact on student outcomes for KS2 results in maths and English?

Acronym

VC

Study hypothesis

Use of the Visible Classroom app and associated mentory increases the educational attainment of Y5 and Y6 students (age 9-11) in combined KS2 maths and English scores.

Ethics approval

University of Melbourne Humanities and Applied Sciences Human Ethics Sub-Committee, 15/09/2016, ref: 1647332

Study design

Single-blind cluster randomised controlled trial

Primary study design

Interventional

Secondary study design

Cluster randomised trial

Trial setting

Schools

Trial type

Other

Patient information sheet

https://drive.google.com/file/d/0B6asyPNMbWTySWZ3NmU1MmRtMlE/view

Condition

Academic attainment

Intervention

The trial is a two-arm, school-level cluster randomised trial. The two arms are (1) ‘business as usual’ control arm, and (2) the Visible Classroom intervention. Approximately 140 schools will be recruited to this trial.
Following recruitment of schools, schools will be randomised to either the control or treatment arm. This will be conducted using data analysis and statistical software Stata. The randomisation will follow a two-stage process:
1. The schools will be stratified on the basis of the proportion of FSM students (split across the median sample proportion), 2010-11 Key Stage 1 (KS1) Average Point Score (split across the median sample score), and whether schools were offered entry into the trial on a capped or uncapped basis. This last stratification covariate is included due to a decision after school recruitment began to begin capping the number of teachers per school who could be involved in the trial. This decision was made to avoid a cost overrun in the delivery team’s budget. As capped schools may select which teachers can be involved in this trial on the basis of merit or some other element correlated with student attainment, we made the decision to include this covariate in our stratification.
2. A random number will be generated within each block to ensure that the proportion of FSM students, KS1 results, and number of and capped and uncapped schools are balanced across trial arms. We will use data from DfE’s Performance Tables to determine the blocking characteristics, and SSAT will note which schools are offered capped entry during recruitment.
The randomisation will occur following recruitment and after baseline data collection. The Stata code used for the randomisation will be recorded in the final report.

Control group: The control or ‘business as usual’ arm will not have any intervention provided.

Intervention group: The intervention, or Visible Classroom (VC) arm, involves teachers audio recording lessons and receiving detailed feedback on their teaching practices. Every term, primary school teachers will record their normal lesson practice, and upload audio files to a website. After every upload, Ai Media, a captioning company, will provide a verbatim transcript of the teacher’s talk. Alongside the word-for-word transcript will be simple analytics (e.g. speed of instruction, quantity of teacher vs student talk) a few hours after the lesson. After several hours of lessons have been uploaded, teachers will receive a more detailed report, written by experts at the University of Melbourne. This will identify areas for improvement in their teaching, based on their self-selected teaching goals and the evidence from their transcripts. Teachers will be expected to discuss the feedback with a mentor – another teacher at their school.

The intervention programme will be delivered over the school academic year (approximately 7 months), with follow up for outcome measure collection occurring at approximately 8 months post randomisation.

Intervention type

Behavioural

Phase

Drug names

Primary outcome measures

Combined maths and English KS 2 results for Year 5 and 6 is measured using results from national Key Stage 2 examinations. If results are consistent, the analysis will be done on the combined Y5 & Y6 data. Taking into account potential issues of implementation for Y6, as it is a ‘high-stakes’ year, we will alternatively report two headline findings recorded one year apart for Y5 and Y6 individually.

For the year 6 cohort the outcome measure collection will come 7 months after randomisation and the intervention begins. For the year 5 cohort, who sit their KS2 exams one year later, outcome measures will be collected 19 months after randomisation. Analysis of both outcomes will incorporate a pre-randomisation record of student results utilising KS1 teacher assessments, which are held on record for each participant.

Secondary outcome measures

These secondary measures assess student results in KS2 exams on a year level and subject specific basis (for a total of 4 outcomes). The time of measurement is the same as for the primary outcome measures.
1. KS2 English results for Y5
2. KS2 English results for Y6
3. KS2 maths results for Y5
4. KS2 maths results for Y6
2. FSM and teacher subgroup analysis
For FSM students and for the teacher subgroup analysis, the empirical strategy (and outcome measures) will remain the same with the addition of interaction analysis considering the interaction between the intervention and the FSM students and teachers who are deemed high on adherence to the rubric at baseline
3. Dosage analysis (IV approach) of the combined time the teacher spends recording lessons and using the online feedback which is available for teachers on their online dashboard is measured using survey responses collected in May 2017
4. Way that self-reported changes in teaching practice moderate any effect on attainment are assessed using the survey of teacher practice in May 2017

Overall trial start date

06/01/2016

Overall trial end date

30/11/2018

Reason abandoned

Eligibility

Participant inclusion criteria

School inclusion criteria:
1. Upfront transfer of data to the evaluators including eligible student UPNs
2. A completed MOU
3. Agreement that teachers in both the VC intervention and control arm complete a survey at the end of the trial period
4. Inclusion of both Y5 & Y6 in the intervention, with a minimum of two teachers providing baseline data and attending the training session
5. Access to an iPad/smartphone and sufficient internet connection to upload recording (internet connection to be verified at baseline data gathering)
6. The schools must not be using Visible Learning plus, a similar intervention designed by the University of Melbourne Project Team

Participant type

Other

Age group

Child

Gender

Both

Target number of participants

Target recruitment number for this study is 140 schools, with a predicted average student cohort of 42 students per year level. As this trial will involve both 2016-17 year 5 and 6 students, this would mean an average of 84 students per school, for a total of 11,760 participants.

Participant exclusion criteria

1. School has less than two teachers across years 5 and 6
2. School is not able to send staff to the proposed training dates should they be assigned to the treatment group
3. School staff have access to a tablet or mobile device that is capable of downloading the ‘Visible Classroom’ application
4. The school must not have previous experience using the ‘Visible Classroom’ application

Recruitment start date

01/06/2016

Recruitment end date

28/10/2016

Locations

Countries of recruitment

United Kingdom

Trial participating centre

The Schools, Students and Teachers network (SSAT)
5th Floor, Central House 142 Central Street
London
EC1V 8AR
United Kingdom

Sponsor information

Organisation

Behavioural Insights Team

Sponsor details

4 Matthew Parker Street
London
SW1H 9NP
United Kingdom

Sponsor type

Research organisation

Website

behaviouralinsights.co.uk

Funders

Funder type

Charity

Funder name

Education Endowment Foundation

Alternative name(s)

Funding Body Type

Funding Body Subtype

Location

Results and Publications

Publication and dissemination plan

Planned publication of an evaluation report.

IPD Sharing plan:
Participant level data is quite sensitive, as it contains personal and academic information that could be used to identify trial participants. As such, this data will not be made publicly available. However, participant level data will be transferred and stored with the Education Endowment Foundation (the project funder) and (in an anonymised form) the UK Data Archive.

Intention to publish date

30/11/2018

Participant level data

Not expected to be available

Results - basic reporting

Publication summary

Publication citations

Additional files

Editorial Notes