Evaluating the effectiveness of the Nuffield Early Language Intervention

ISRCTN	ISRCTN14881793
DOI	https://doi.org/10.1186/ISRCTN14881793
ClinicalTrials.gov (NCT)	Nil known
Clinical Trials Information System (CTIS)	Nil known
Protocol serial number	NELI-R 2020
Sponsor	Education Endowment Foundation
Funder	Education Endowment Foundation

Submission date: 16/09/2020
Registration date: 17/09/2020
Last edited: 17/09/2020

Recruitment status: No longer recruiting
Overall study status: Completed
Condition category: Mental and Behavioural Disorders

Prospectively registered

Protocol

Statistical analysis plan

Results

Individual participant data

Record updated in last year

Plain English summary of protocol

Background and study aims
The aim of this study is to evaluate the effectiveness of the Nuffield Early Language Intervention – Reception (NELI-R) programme, when coupled with a new online training model for teachers and teaching assistants (TAs) delivering the programme in schools. NELI-R is a 20-week evidence-based programme suitable for children in reception year in UK primary schools. The programme is designed to improve the oral language skills of children with language learning weaknesses and is delivered by trained teaching assistants. Evidence from previous studies has shown the programme has a positive effect on children’s language skills.

Who can participate?
Children in reception classes (aged 4 to 5) at about 60 primary schools in the UK

What does the study involve?
Before the intervention programme begins all children in all participating reception classrooms will be given a short app-based language assessment of four core language skills: expressive vocabulary, receptive vocabulary, sentence repetition, and listening comprehension. Class teachers will also complete a short teacher rating of behavioural adjustment for each child (October/November 2020).
From the language assessment, the five children in each participating reception classroom with the poorest language scores are selected to receive the NELI-R programme (January/July 2021). Once the NELI-R programme has been completed, all children in every school are assessed again using the same assessments (June/July 2021). The improvement in language and behavioural adjustment found in children taking part in the NELI-R intervention is compared to the improvement of the remaining children in each class who did not take part in NELI-R.

What are the possible benefits and risks of participating?
All schools taking part in the study will receive the Nuffield Early Language Intervention and training to deliver it. Children taking part in the intervention will benefit from additional small group and individual teaching sessions. There are no anticipated risks of participation.

Where is the study run from?
University of Oxford (UK)

When is the study starting and how long is it expected to run for?
August 2020 to September 2021

Who is funding the study?
Education Endowment Foundation (EEF) (UK)

Who is the main contact?
Dr Gillian West
gillian.west@education.ox.ac.uk

Contact information

Dr Gillian West
Scientific

Department of Education
University of Oxford
15 Norham Gardens
London
OX2 6PY
United Kingdom

ORCID ID	0000-0002-4413-3601
Phone	+44 (0)1865 274021
Email	gillian.west@education.ox.ac.uk

Study information

Primary study design	Interventional
Study design	Regression discontinuity design
Secondary study design	Regression discontinuity design
Study type	Participant information sheet
Scientific title	Evaluating the effectiveness of the Nuffield Early Language Intervention using a regression discontinuity design
Study objectives	The researchers will conduct a study to evaluate the effectiveness of the Nuffield Early Language Intervention – Reception (NELI-R) programme, when coupled with a new online training model for teachers and TAs delivering the programme in schools. NELI-R is a 20-week evidence-based programme suitable for children in reception year in UK primary schools. The version of the programme to be evaluated in the current trial has been published by Oxford University Press (Fricke, Bowyer-Crane, Snowling, & Hulme, 2018). The programme is designed to improve the oral language skills of children with language learning weaknesses and is delivered by trained teaching assistants in schools. A recent randomised controlled trial of NELI-R (West et al., 2020) found that children receiving the programme made significantly larger gains than a business-as-usual control group on a latent variable reflecting standardised measures of language ability (d = .26). Teaching assistants (TAs) in the trial received 2.5 days of in person training to deliver the intervention programme, with classroom teachers attending the first half day of training. This new trial will investigate the effectiveness of the same intervention programme when coupled with online teacher and TA training.
Ethics approval(s)	Approved 03/09/2020, The Departmental Research Ethics Committee (DREC) in the Department of Education, The University of Oxford (15 Norham Gardens, Oxford, OX2 6PY, UK; +44 (0)1865 274024; staff.curec@education.ox.ac.uk), ref: ED-CIA-21-005
Health condition(s) or problem(s) studied	Language disorder
Intervention	Funding has been provided by the Educational Endowment Foundation (EEF) to conduct a study of the Nuffield Early Language Intervention - Reception, coupled with online training for teachers and TAs delivering the intervention in 50 – 60 UK primary schools. Schools taking part in the trial are from a number of geographical areas: Hertfordshire, Buckinghamshire, Oxfordshire, Greater London, Kent and Liverpool. All schools will receive the NELI programme and online training. The study will use a regression discontinuity design (RDD), comparing an estimate of the treatment effect (the NELI-R language intervention) on language skills for those children with poor language skills selected for the NELI-R intervention group, compared to similar children who “just miss” assignment to the groups. The RDD is useful in educational settings because it allows treatment to be given to all children who need it, as determined by a specified cut-off point (Cappelleri & Trochim, 2003). The critical data for this study will be collected by schools using the LanguageScreen app (LanguageScreen.com). LanguageScreen is an online language test, which allows school staff (teachers, teaching assistants, trained volunteers) to assess children’s oral language skills. All assessments will be conducted by school staff. There are four subtests in LanguageScreen (Receptive vocabulary, Expressive vocabulary, Sentence repetition, Listening comprehension) all delivered via an App on a tablet. Scoring is automated within the app and assessments upload to a secure website. This method of assessment has already been validated in the previous EEF funded RCT evaluating the effectiveness of NELI (West et al., 2020). In that trial LanguageScreen was highly reliable (alpha .97), showed massive longitudinal stability over a 6-month period (r =.87; an under-estimate of test-retest reliability) and correlated very highly with standardized language tests administered at the same time by speech and language therapists (r=.95). LanguageScreen also yielded an estimate of the intervention effect size (d = .32) which was similar to the estimated effect size for the intervention derived from the individually administered standardised language tests (d= .26). In the current trial, at pretest all children in participating classrooms will be assessed with LanguageScreen. Only children with severe social and behavioural difficulties, or severe sensory impairments, will be excluded. The five children in each classroom with the lowest LanguageScreen scores will be assigned to receive NELI-R. Class teachers will also complete the 12-item behavioural adjustment teacher rating of the Brief Early Skills and Support Index (BESSI; https://www.cfr.cam.ac.uk/system/files/documents/revised-bessi-manual-sept-2015.pdf ) for all children in participating Reception classes. The same measures will be used at post-test as at pretest.
Intervention type	Behavioural
Primary outcome measure(s)	Language is measured using the LanguageScreen assessment (https://www.LanguageScreen.com) at pretest in October/November 2020 and at posttest in June/July 2021. Analyses will be based on an RDD quasi-experimental design. A language total score will be created from the 4 LanguageScreen app subtests (expressive vocabulary, receptive vocabulary, sentence repetition, listening comprehension). A multi-level model (children clustered within schools) will relate posttest LanguageScreen scores to pretest LanguageScreen scores and a group dummy variable (intervention = 0; control = 1). The pre-test LanguageScreen scores and pre-test LanguageScreenscores squared will be added as predictors to the model to assess whether the function relating pre-test to post-test scores is linear or involves a quadratic component. Interactions between group and the linear and quadratic components will be tested for by adding grouppre-test and grouppre-test squared interaction terms to the model, in accordance with recommendations made by Trochim (2006). Non-significant terms will be dropped iteratively, in order to arrive at the model that provides the most parsimonious fit to the data. A discontinuity between a regression function for the intervention group and the function for the control group; a higher intercept for the intervention group would provide evidence for a causal effect of the intervention in increasing post-test scores.
Key secondary outcome measure(s)	Behavioural adjustment is measured using 12-item behavioural adjustment subtest from the Brief Early Skills and Support Index teacher rating (BESSI) at pretest in October/November 2020 and at posttest in June/July 2021. The researchers will also evaluate any changes in behavioural adjustment by using the same comparison of group regression functions as specified above with a composite behavioural adjustment measure defined by 12 items from the rating scale grouped into three subsets (Behavioural regulation = 7 items; Attention/hyperactivity = 3 items; Sociability = 2 items).
Completion date	01/09/2021

Eligibility

Participant type(s)	Other
Age group	Child
Lower age limit	4 Years
Upper age limit	5 Years
Sex	All
Target sample size at registration	350
Key inclusion criteria	1. Children must be in Reception class 2. Children must be aged 4 to 5 years at pretest 3. Children receiving the NELI-R programme will be selected for having language weaknesses in comparison to their peers at school entry
Key exclusion criteria	1. Children with such severe sensory impairments that they are unable to access the language screening assessment 2. Children with such severe emotional or behavioural issues that they are unable to access the language screening assessment
Date of first enrolment	28/09/2020
Date of final enrolment	23/10/2020

Locations

Countries of recruitment

United Kingdom

Study participating centre

Primary schools across the UK

-
United Kingdom

Results and Publications

Individual participant data (IPD) Intention to share	Yes
IPD sharing plan summary	Stored in repository
IPD sharing plan	At the end of the trial data on children’s scores on the pre- and posttest language screening will be entered into the EEF’s data archive (NB: FFT is the organisation appointed to manage EEF’s data archive). FFT will be responsible for all data archiving and for the management and security of the anonymised trial data. Anonymised data with all personal identifiers removed will also be stored on the UK Data Archive https://www.data-archive.ac.uk/.

Study outputs

Output type	Details	Date created	Date added	Peer reviewed?	Patient-facing?
Participant information sheet	Participant information sheet	11/11/2025	11/11/2025	No	Yes

Editorial Notes

17/09/2020: Trial's existence confirmed by the Departmental Research Ethics Committee (DREC) in the Department of Education, The University of Oxford.