Plain English Summary
Background and study aims
The aim of this study is to evaluate the effectiveness of the Nuffield Early Language Intervention – Reception (NELI-R) programme, when coupled with a new online training model for teachers and teaching assistants (TAs) delivering the programme in schools. NELI-R is a 20-week evidence-based programme suitable for children in reception year in UK primary schools. The programme is designed to improve the oral language skills of children with language learning weaknesses and is delivered by trained teaching assistants. Evidence from previous studies has shown the programme has a positive effect on children’s language skills.
Who can participate?
Children in reception classes (aged 4 to 5) at about 60 primary schools in the UK
What does the study involve?
Before the intervention programme begins all children in all participating reception classrooms will be given a short app-based language assessment of four core language skills: expressive vocabulary, receptive vocabulary, sentence repetition, and listening comprehension. Class teachers will also complete a short teacher rating of behavioural adjustment for each child (October/November 2020).
From the language assessment, the five children in each participating reception classroom with the poorest language scores are selected to receive the NELI-R programme (January/July 2021). Once the NELI-R programme has been completed, all children in every school are assessed again using the same assessments (June/July 2021). The improvement in language and behavioural adjustment found in children taking part in the NELI-R intervention is compared to the improvement of the remaining children in each class who did not take part in NELI-R.
What are the possible benefits and risks of participating?
All schools taking part in the study will receive the Nuffield Early Language Intervention and training to deliver it. Children taking part in the intervention will benefit from additional small group and individual teaching sessions. There are no anticipated risks of participation.
Where is the study run from?
University of Oxford (UK)
When is the study starting and how long is it expected to run for?
August 2020 to September 2021
Who is funding the study?
Education Endowment Foundation (EEF) (UK)
Who is the main contact?
Dr Gillian West
gillian.west@education.ox.ac.uk
Trial website
Contact information
Type
Scientific
Primary contact
Dr Gillian West
ORCID ID
http://orcid.org/0000-0002-4413-3601
Contact details
Department of Education
University of Oxford
15 Norham Gardens
London
OX2 6PY
United Kingdom
+44 (0)1865 274021
gillian.west@education.ox.ac.uk
Additional identifiers
EudraCT number
Nil known
ClinicalTrials.gov number
Nil known
Protocol/serial number
NELI-R 2020
Study information
Scientific title
Evaluating the effectiveness of the Nuffield Early Language Intervention using a regression discontinuity design
Acronym
Study hypothesis
The researchers will conduct a study to evaluate the effectiveness of the Nuffield Early Language Intervention – Reception (NELI-R) programme, when coupled with a new online training model for teachers and TAs delivering the programme in schools. NELI-R is a 20-week evidence-based programme suitable for children in reception year in UK primary schools. The version of the programme to be evaluated in the current trial has been published by Oxford University Press (Fricke, Bowyer-Crane, Snowling, & Hulme, 2018). The programme is designed to improve the oral language skills of children with language learning weaknesses and is delivered by trained teaching assistants in schools. A recent randomised controlled trial of NELI-R (West et al., 2020) found that children receiving the programme made significantly larger gains than a business-as-usual control group on a latent variable reflecting standardised measures of language ability (d = .26). Teaching assistants (TAs) in the trial received 2.5 days of in person training to deliver the intervention programme, with classroom teachers attending the first half day of training. This new trial will investigate the effectiveness of the same intervention programme when coupled with online teacher and TA training.
Ethics approval
Approved 03/09/2020, The Departmental Research Ethics Committee (DREC) in the Department of Education, The University of Oxford (15 Norham Gardens, Oxford, OX2 6PY, UK; +44 (0)1865 274024; staff.curec@education.ox.ac.uk), ref: ED-CIA-21-005
Study design
Regression discontinuity design
Primary study design
Interventional
Secondary study design
Regression discontinuity design
Trial setting
Schools
Trial type
Treatment
Patient information sheet
Not available in web format, please use the contact details to request a participant information sheet
Condition
Language disorder
Intervention
Funding has been provided by the Educational Endowment Foundation (EEF) to conduct a study of the Nuffield Early Language Intervention - Reception, coupled with online training for teachers and TAs delivering the intervention in 50 – 60 UK primary schools. Schools taking part in the trial are from a number of geographical areas: Hertfordshire, Buckinghamshire, Oxfordshire, Greater London, Kent and Liverpool.
All schools will receive the NELI programme and online training. The study will use a regression discontinuity design (RDD), comparing an estimate of the treatment effect (the NELI-R language intervention) on language skills for those children with poor language skills selected for the NELI-R intervention group, compared to similar children who “just miss” assignment to the groups. The RDD is useful in educational settings because it allows treatment to be given to all children who need it, as determined by a specified cut-off point (Cappelleri & Trochim, 2003).
The critical data for this study will be collected by schools using the LanguageScreen app (LanguageScreen.com). LanguageScreen is an online language test, which allows school staff (teachers, teaching assistants, trained volunteers) to assess children’s oral language skills. All assessments will be conducted by school staff. There are four subtests in LanguageScreen (Receptive vocabulary, Expressive vocabulary, Sentence repetition, Listening comprehension) all delivered via an App on a tablet. Scoring is automated within the app and assessments upload to a secure website.
This method of assessment has already been validated in the previous EEF funded RCT evaluating the effectiveness of NELI (West et al., 2020). In that trial LanguageScreen was highly reliable (alpha .97), showed massive longitudinal stability over a 6-month period (r =.87; an under-estimate of test-retest reliability) and correlated very highly with standardized language tests administered at the same time by speech and language therapists (r=.95). LanguageScreen also yielded an estimate of the intervention effect size (d = .32) which was similar to the estimated effect size for the intervention derived from the individually administered standardised language tests (d= .26).
In the current trial, at pretest all children in participating classrooms will be assessed with LanguageScreen. Only children with severe social and behavioural difficulties, or severe sensory impairments, will be excluded. The five children in each classroom with the lowest LanguageScreen scores will be assigned to receive NELI-R. Class teachers will also complete the 12-item behavioural adjustment teacher rating of the Brief Early Skills and Support Index (BESSI; https://www.cfr.cam.ac.uk/system/files/documents/revised-bessi-manual-sept-2015.pdf ) for all children in participating Reception classes. The same measures will be used at post-test as at pretest.
Intervention type
Behavioural
Phase
Drug names
Primary outcome measure
Language is measured using the LanguageScreen assessment (https://www.LanguageScreen.com) at pretest in October/November 2020 and at posttest in June/July 2021.
Analyses will be based on an RDD quasi-experimental design. A language total score will be created from the 4 LanguageScreen app subtests (expressive vocabulary, receptive vocabulary, sentence repetition, listening comprehension). A multi-level model (children clustered within schools) will relate posttest LanguageScreen scores to pretest LanguageScreen scores and a group dummy variable (intervention = 0; control = 1).
The pre-test LanguageScreen scores and pre-test LanguageScreenscores squared will be added as predictors to the model to assess whether the function relating pre-test to post-test scores is linear or involves a quadratic component. Interactions between group and the linear and quadratic components will be tested for by adding group*pre-test and group*pre-test squared interaction terms to the model, in accordance with recommendations made by Trochim (2006). Non-significant terms will be dropped iteratively, in order to arrive at the model that provides the most parsimonious fit to the data. A discontinuity between a regression function for the intervention group and the function for the control group; a higher intercept for the intervention group would provide evidence for a causal effect of the intervention in increasing post-test scores.
Secondary outcome measures
Behavioural adjustment is measured using 12-item behavioural adjustment subtest from the Brief Early Skills and Support Index teacher rating (BESSI) at pretest in October/November 2020 and at posttest in June/July 2021.
The researchers will also evaluate any changes in behavioural adjustment by using the same comparison of group regression functions as specified above with a composite behavioural adjustment measure defined by 12 items from the rating scale grouped into three subsets (Behavioural regulation = 7 items; Attention/hyperactivity = 3 items; Sociability = 2 items).
Overall trial start date
01/08/2020
Overall trial end date
01/09/2021
Reason abandoned (if study stopped)
Eligibility
Participant inclusion criteria
1. Children must be in Reception class
2. Children must be aged 4 to 5 years at pretest
3. Children receiving the NELI-R programme will be selected for having language weaknesses in comparison to their peers at school entry
Participant type
Other
Age group
Child
Gender
Both
Target number of participants
350 - 450 children (depending on the number of classes each school enrolls on the trial)
Participant exclusion criteria
1. Children with such severe sensory impairments that they are unable to access the language screening assessment
2. Children with such severe emotional or behavioural issues that they are unable to access the language screening assessment
Recruitment start date
28/09/2020
Recruitment end date
23/10/2020
Locations
Countries of recruitment
United Kingdom
Trial participating centre
Primary schools across the UK
-
United Kingdom
Sponsor information
Organisation
Education Endowment Foundation
Sponsor details
9th Floor
Millbank Tower
21 - 24 Millbank
London
SW1P 4QP
United Kingdom
+44 (0)207 802 1676
info@eefoundation.org.uk
Sponsor type
Charity
Website
Funders
Funder type
Charity
Funder name
Education Endowment Foundation
Alternative name(s)
EEF
Funding Body Type
private sector organisation
Funding Body Subtype
Trusts, charities, foundations (both public and private)
Location
United Kingdom
Results and Publications
Publication and dissemination plan
Results of the study will be published in a peer-reviewed scientific journal at the end of the trial. No additional documents are or will be available to be included with the pre-registration and the protocol is not published/online.
IPD sharing statement
At the end of the trial data on children’s scores on the pre- and posttest language screening will be entered into the EEF’s data archive (NB: FFT is the organisation appointed to manage EEF’s data archive). FFT will be responsible for all data archiving and for the management and security of the anonymised trial data. Anonymised data with all personal identifiers removed will also be stored on the UK Data Archive https://www.data-archive.ac.uk/.
Intention to publish date
01/11/2021
Participant level data
Stored in repository
Basic results (scientific)
Publication list