The effectiveness of the Nuffield Early Language Intervention programme

ISRCTN	ISRCTN12991126
DOI	https://doi.org/10.1186/ISRCTN12991126
Protocol serial number	NELI-2018
Sponsor	The Education Endowment Foundation
Funder	Education Endowment Foundation

Submission date: 01/06/2018
Registration date: 05/06/2018
Last edited: 16/02/2023

Recruitment status: No longer recruiting
Overall study status: Completed
Condition category: Mental and Behavioural Disorders

Prospectively registered

Protocol

Statistical analysis plan

Results

Individual participant data

Plain English summary of protocol

Background and study aims
This study will evaluate the effectiveness of the Nuffield Early Language Intervention programme, which is a 20-week educational programme suitable for children in reception year (aged 4 – 5) in UK primary schools. The programme is designed to improve the language skills of children with language learning weaknesses. It is delivered by specially trained teaching assistants working with children individually and in small groups. The version of the programme to be evaluated in the current study has been published by Oxford University Press (see OUP website for details of the programme https://global.oup.com/education/content/primary/series/nuffield-intervention/?region=uk ). Evidence from previous studies has shown the programme has a positive effect on children’s language skills. The current study will seek more robust evidence, from a larger sample of children, for the effectiveness of the published version of this programme, which includes training for teaching assistants by Elklan, a specialist in educational training.

Who can participate?
Children in reception classes (aged 4 to 5) at about 200 primary schools from a number of geographical areas across the UK, including Manchester, Bristol, Cornwall, Essex, Hertfordshire, Surrey, Warwickshire and the North West of England

What does the study involve?
Before the intervention programme begins all children in all participating reception classrooms undergo a short app-based assessment of four core language skills: expressive vocabulary, receptive vocabulary, sentence repetition, and listening comprehension. From this assessment the five children in each participating reception classroom with the poorest language scores are selected for detailed language assessment. After pretesting is completed, the schools are randomly allocated to one of two groups to either receive the Nuffield Early Language Intervention programme or to not receive it. The five children in each participating class in intervention group schools then receive the programme for 20 weeks (January to June 2019). Once the programme has been completed, all children in every school are assessed again using the same assessments (June/July 2019). The language of children in intervention group schools is compared to those in control group schools to see how much the children who took part have improved as a result of the Nuffield Early Language Intervention programme.

What are the possible benefits and risks of participating?
Children taking part in the intervention will benefit from additional small group and individual teaching sessions. Schools in the control group will receive monetary compensation and can choose to purchase the Nuffield Early Language Intervention programme once the study is completed. There are no anticipated risks of participation.

Where is the study run from?
1. University of Oxford (UK): intervention delivery team
2. RAND Europe (UK): independent evaluator

Who is funding the study?
Education Endowment Foundation (EEF) (UK)

Who is the main contact?
Dr Alex Sutherland, alex@researchdesign.uk

Contact information

Dr Gillian West
Scientific

Department of Education
The University of Oxford
15 Norham Gardens
Oxford
OX2 6PY
United Kingdom

ORCID ID	0000-0002-4413-3601

Dr Alex Sutherland
Principal investigator

c/o Department of Social Policy and Intervention
Barnett House
32 -37 Wellington Square
Oxford
OX1 2ER
United Kingdom

ORCID ID	0000-0003-0276-9705
Phone	+44 (0)7881 556801
Email	alex@researchdesign.uk

Study information

Primary study design	Interventional
Study design	2-year cluster randomised controlled trial
Secondary study design	Cluster randomised trial
Study type	Participant information sheet
Scientific title	The effectiveness of the Nuffield Early Language Intervention programme: a two-arm cluster randomised controlled trial
Study objectives	This is a cluster-randomised controlled to evaluate the effectiveness of the Nuffield Early Language Intervention programme. This is a 20-week evidence-based programme suitable for children in reception year in UK primary schools. The programme is designed to improve the oral language skills of children with language learning weaknesses. The programme is delivered by specially trained teaching assistants working with children individually and in small groups. The version of the programme to be evaluated in the current trial has been published by Oxford University Press (Fricke, Bowyer-Crane, Snowling, & Hulme, 2018), with training for the teaching assistants provided by Elklan. The OUP website provides details of the programme https://global.oup.com/education/content/primary/series/nuffield-intervention/?region=uk . Evidence from existing trials has shown the programme to have positive effects on children’s language skills (Fricke, Bowyer-Crane, Haley, Hulme, & Snowling, 2013; Fricke et al, 2017). The current trial will seek more robust evidence, from a larger sample, and an independent evaluation, for the effectiveness of the published version of this programme, with training to those using the programme delivered “at scale”.
Ethics approval(s)	The Departmental Research Ethics Committee (DREC) in the Department of Education, The University of Oxford, 27/04/2018, ref: ED-CIA-18-192
Health condition(s) or problem(s) studied	Language disorder
Intervention	Current interventions as of 26/07/2018: The Education Endowment Foundation (EEF) has granted funding to RAND Europe and the University of Oxford to conduct a cluster-randomized trial in approximately 200 primary schools in the UK assessing the effectiveness of The Nuffield Early Language Intervention (NELI) programme. RAND are the EEF’s independent evaluators and are responsible for the evaluation study. Oxford are the intervention team and are responsible for school recruitment and intervention delivery. Schools will be recruited from a number of geographical areas including Manchester, Bristol, Cornwall, Essex, Hertfordshire, Surrey, Warwickshire and the North West of England. All children in all participating classrooms will undergo language assessments at pretest in two phases and all pretest data will be collected prior to randomisation. Phase 1 – Screening Every child in the participating reception classrooms in every school will be assessed using the Automated Test of Language Abilities (ATLAS) app, developed by the Oxford team. This test assesses four core language skills: expressive vocabulary, receptive vocabulary, sentence repetition, and listening comprehension. This testing will be conducted by school staff. Scoring of responses is done online using the app. Based on a composite score (composite z-scores from the 4 subtests) from the ATLAS the 5 children in each participating reception classroom with the poorest language scores will be selected for further testing in Phase 2. Phase 2 – In-depth testing Detailed individual tests will be given to the 5 children identified in screening, as follows: CELF recalling sentences subtest, CELF expressive vocabulary subtest, Renfrew Action Picture test (information and grammar) and the YARC Early Word Reading subtest. These baseline tests will be administered by testers working for the Oxford team, but will be undertaken pre-randomisation to ensure baseline is unbiased by allocation knowledge. After pretesting is completed, schools will be randomised (within each area) by RAND Europe to one of two arms: intervention vs control. Intervention schools will receive the 20-week Nuffield Early Language Intervention programme and training in its use by Elklan tutors. The control schools will be required to agree to testing of children and will undertake not to use the Nuffield Early Language Intervention programme in that year. Control schools will be provided with monetary compensation (£1000 per school) which they may use to purchase the Nuffield Early Language Intervention programme the following year. At posttest the same measures will be used as at pretest. Individually administered language tests will be given by new independent testers working for a third party who will be sub-contracted by the RAND Europe team. The posttest assessors will be blind to condition, but will be trained to administer the tests by the Oxford team. ATLAS testing at posttest will be conducted by school staff, but school staff will not be blind to allocation. Previous interventions: Funding has been provided by the Education Endowment Foundation (EEF) to conduct a cluster-randomized trial in approximately 200 primary schools in the UK assessing the effectiveness of The Nuffield Early Language Intervention programme. Schools will be recruited from a number of geographical areas including Manchester, Bristol, Cornwall, Essex, Hertfordshire, Surrey, Warwickshire and the North West of England. All children in all participating classrooms will undergo language assessments at pretest in two phases and all pretest data will be collected prior to randomisation. Phase 1 – Screening Every child in the participating reception classrooms in every school will be assessed using the Automated Test of Language Abilities (ATLAS). This test assesses four core language skills: expressive vocabulary, receptive vocabulary, sentence repetition, and listening comprehension. This testing will be conducted by school staff. Scoring of responses is done online using the app. Based on a composite score (composite z-scores from the 4 subtests) from the ATLAS the 5 children in each participating reception classroom with the poorest language scores will be selected for further testing in Phase 2. Phase 2 – In-depth testing Detailed individual tests will be given to the 5 children identified in screening, as follows: CELF recalling sentences subtest, CELF expressive vocabulary subtest, Renfrew Action Picture test (information and grammar) and the YARC Early Word Reading subtest. These tests will be administered by testers trained by the research team. After pretesting is completed, schools will be randomised (within each area) by an independent evaluator (RAND corporation) to one of two arms: intervention vs control. Intervention schools will receive the 20-week Nuffield Early Language Intervention programme and training in its use by Elklan tutors. The control schools will be required to agree to testing of children and will undertake not to use the Nuffield Early Language Intervention programme in that year. Control schools will be provided with monetary compensation (£1000 per school) which they may use to purchase the Nuffield Early Language Intervention programme the following year. At posttest the same measures will be used as at pretest. ATLAS testing at posttest will be conducted by school staff. Individually administered language tests will be given by new testers, blind to condition, who will be trained to administer the tests by the research team.
Intervention type	Behavioural
Primary outcome measure(s)	Current primary outcome measure: The main trial analysis will be conducted by RAND using multilevel structural equation modelling (SEM) to account for the hierarchical nature of the study data and cluster-randomisation. The primary outcome will be a latent language variable created from the four individually administered language tests ((i) CELF recalling sentences subtest, (ii) CELF expressive vocabulary subtest, Renfrew Action Picture test ((iii) information and (iv) grammar). Analyses will be based on this latent outcome variable. The same language latent variable will be created for pretest and posttest scores by the RAND team. In the main analysis model, the pretest latent variable will be the covariate, and the posttest latent variable the outcome measure. The analyses will also be undertaken on each subscale used to create the latent variable. The effects of the intervention will be measured by an appropriate effect size such as standardized differences in means for a group dummy variable comparing pupils in treatment and control schools. Both primary and secondary outcomes will be measured at pretest (baseline) in September/October 2018 and at posttest (on completion of the 20-week intervention in June/July 2019). The funder is open to conducting a longer-term (30-week) follow-up but that has not yet been confirmed. A secondary analysis will be undertaken that follows the EEF’s stated preference for using multilevel modelling for clustered designs that uses a predicted outcome based on the SEM (EEF, 2018). Previous primary outcome measure: Analyses will be based on latent variable ANCOVA models implemented in an SEM framework. The primary outcome will be a language latent variable defined by loadings from the individually administered language tests (CELF recalling sentences subtest, CELF expressive vocabulary subtest, Renfrew Action Picture test (information and grammar). The same language latent variable will be created for pretest and posttest scores. The pretest latent variable will be the covariate, and the posttest latent variable the outcome measure. The effects of the intervention will be measured by the y-standardized regression coefficient for a group dummy variable. The effects of clustering within schools will be accounted for by using robust (Huber-White) cluster standard errors. Both primary and secondary outcomes will be measured at pretest (baseline) in September 2018 and at posttest (on completion of the 20-week intervention in June/ July 2019).
Key secondary outcome measure(s)	Current secondary outcome measures: The secondary outcome will be a language latent variable defined by loadings from the ATLAS language app subtests (expressive vocabulary, receptive vocabulary, sentence repetition, listening comprehension). The analysis plan for this secondary analysis is identical to that for the primary outcome. The study will also evaluate any changes in word reading scores from the EWR test by using multilevel modelling on a latent variable from this test. Both primary and secondary outcomes will be measured at pretest (baseline) in September 2018 and at posttest (on completion of the 20-week intervention in June/July 2019). Previous secondary outcome measures: The secondary outcome will be a language latent variable defined by loadings from the ATLAS language app subtests (expressive vocabulary, receptive vocabulary, sentence repetition, listening comprehension). The analysis plan for this secondary analysis is identical to that for the primary outcome. We will also evaluate any changes in word reading scores from the EWR test by using ANCOVA on a latent variable from this test. The effects of clustering within schools will be accounted for in all analyses by using robust (Huber-White) cluster standard errors. Both primary and secondary outcomes will be measured at pretest (baseline) in September 2018 and at posttest (on completion of the 20-week intervention in June/July 2019).
Completion date	30/09/2021

Eligibility

Participant type(s)	Other
Age group	Child
Lower age limit	4 Years
Upper age limit	5 Years
Sex	All
Target sample size at registration	1000
Key inclusion criteria	1. Children must be in Reception class 2. Children must be aged 4 to 5 years at pretest 3. Children will be selected for having language weaknesses in comparison to their peers at school entry
Key exclusion criteria	1. Children will be excluded if they have a significant visual impairment 2. Children will be excluded if they have a significant auditory impairment 3. Children will be excluded if they have significant behavioural difficulties, such that they would not be able to take part small group activities
Date of first enrolment	11/06/2018
Date of final enrolment	30/07/2018

Locations

Countries of recruitment

United Kingdom

Study participating centre

Primary schools across the UK

-
United Kingdom

Results and Publications

Individual participant data (IPD) Intention to share	Yes
IPD sharing plan summary	Stored in repository
IPD sharing plan	Anonymised participant level data will be entered into the Education Endowment Foundation’s (EEF) archive by the Fisher Family Trust (FFT: the organisation appointed to manage the EEF’s data archive). Data will include the pupil outcome data from pre- and post-test language assessments, which will be linked with National Pupil Database (NPD) data. A privacy notice detailing this information will circulated to all participants along with information on the trial and an option to opt out of the study.

Study outputs

Output type	Details	Date created	Date added	Peer reviewed?	Patient-facing?
Results article		30/03/2021	01/12/2022	Yes	No
Protocol article	v1.0	15/05/2019	15/02/2023	Yes	No
Funder report results		31/05/2020	15/02/2023	Yes	No
Participant information sheet	Participant information sheet	11/11/2025	11/11/2025	No	Yes
Statistical Analysis Plan	v1.0	01/04/2019	15/02/2023	Yes	No

Editorial Notes

16/02/2023: Internal review.
15/02/2023: The following updates have been made to the study record and the plain English summary updated accordingly:
1. Principle investigator contact added.
2. Protocol article uploaded.
3. Uploaded statistical analysis plan.
4. Funder report results uploaded.
06/12/2022: A contact was removed.
01/12/2022: Publication reference added.
11/01/2021: The following changes were made to the trial record:
1. The overall end date was changed from 30/09/2019 to 30/09/2021.
2. The intention to publish date was changed from 01/07/2020 to 01/07/2022.
26/07/2018: The following changes were made to the trial record:
1. A new contact was added.
2. The interventions and the primary and secondary outcome measures were updated.