Evaluating the effectiveness of the Nuffield Early Language Intervention
| ISRCTN | ISRCTN14881793 |
|---|---|
| DOI | https://doi.org/10.1186/ISRCTN14881793 |
| Secondary identifying numbers | NELI-R 2020 |
- Submission date
- 16/09/2020
- Registration date
- 17/09/2020
- Last edited
- 17/09/2020
- Recruitment status
- No longer recruiting
- Overall study status
- Completed
- Condition category
- Mental and Behavioural Disorders
Plain English summary of protocol
Background and study aims
The aim of this study is to evaluate the effectiveness of the Nuffield Early Language Intervention – Reception (NELI-R) programme, when coupled with a new online training model for teachers and teaching assistants (TAs) delivering the programme in schools. NELI-R is a 20-week evidence-based programme suitable for children in reception year in UK primary schools. The programme is designed to improve the oral language skills of children with language learning weaknesses and is delivered by trained teaching assistants. Evidence from previous studies has shown the programme has a positive effect on children’s language skills.
Who can participate?
Children in reception classes (aged 4 to 5) at about 60 primary schools in the UK
What does the study involve?
Before the intervention programme begins all children in all participating reception classrooms will be given a short app-based language assessment of four core language skills: expressive vocabulary, receptive vocabulary, sentence repetition, and listening comprehension. Class teachers will also complete a short teacher rating of behavioural adjustment for each child (October/November 2020).
From the language assessment, the five children in each participating reception classroom with the poorest language scores are selected to receive the NELI-R programme (January/July 2021). Once the NELI-R programme has been completed, all children in every school are assessed again using the same assessments (June/July 2021). The improvement in language and behavioural adjustment found in children taking part in the NELI-R intervention is compared to the improvement of the remaining children in each class who did not take part in NELI-R.
What are the possible benefits and risks of participating?
All schools taking part in the study will receive the Nuffield Early Language Intervention and training to deliver it. Children taking part in the intervention will benefit from additional small group and individual teaching sessions. There are no anticipated risks of participation.
Where is the study run from?
University of Oxford (UK)
When is the study starting and how long is it expected to run for?
August 2020 to September 2021
Who is funding the study?
Education Endowment Foundation (EEF) (UK)
Who is the main contact?
Dr Gillian West
gillian.west@education.ox.ac.uk
Contact information
Scientific
Department of Education
University of Oxford
15 Norham Gardens
London
OX2 6PY
United Kingdom
 ORCID ID |
0000-0002-4413-3601 |
|---|---|
| Phone | +44 (0)1865 274021 |
| gillian.west@education.ox.ac.uk |
Study information
| Study design | Regression discontinuity design |
|---|---|
| Primary study design | Interventional |
| Secondary study design | Regression discontinuity design |
| Study setting(s) | School |
| Study type | Treatment |
| Participant information sheet | Not available in web format, please use the contact details to request a participant information sheet |
| Scientific title | Evaluating the effectiveness of the Nuffield Early Language Intervention using a regression discontinuity design |
| Study objectives | The researchers will conduct a study to evaluate the effectiveness of the Nuffield Early Language Intervention – Reception (NELI-R) programme, when coupled with a new online training model for teachers and TAs delivering the programme in schools. NELI-R is a 20-week evidence-based programme suitable for children in reception year in UK primary schools. The version of the programme to be evaluated in the current trial has been published by Oxford University Press (Fricke, Bowyer-Crane, Snowling, & Hulme, 2018). The programme is designed to improve the oral language skills of children with language learning weaknesses and is delivered by trained teaching assistants in schools. A recent randomised controlled trial of NELI-R (West et al., 2020) found that children receiving the programme made significantly larger gains than a business-as-usual control group on a latent variable reflecting standardised measures of language ability (d = .26). Teaching assistants (TAs) in the trial received 2.5 days of in person training to deliver the intervention programme, with classroom teachers attending the first half day of training. This new trial will investigate the effectiveness of the same intervention programme when coupled with online teacher and TA training. |
| Ethics approval(s) | Approved 03/09/2020, The Departmental Research Ethics Committee (DREC) in the Department of Education, The University of Oxford (15 Norham Gardens, Oxford, OX2 6PY, UK; +44 (0)1865 274024; staff.curec@education.ox.ac.uk), ref: ED-CIA-21-005 |
| Health condition(s) or problem(s) studied | Language disorder |
| Intervention | Funding has been provided by the Educational Endowment Foundation (EEF) to conduct a study of the Nuffield Early Language Intervention - Reception, coupled with online training for teachers and TAs delivering the intervention in 50 – 60 UK primary schools. Schools taking part in the trial are from a number of geographical areas: Hertfordshire, Buckinghamshire, Oxfordshire, Greater London, Kent and Liverpool. All schools will receive the NELI programme and online training. The study will use a regression discontinuity design (RDD), comparing an estimate of the treatment effect (the NELI-R language intervention) on language skills for those children with poor language skills selected for the NELI-R intervention group, compared to similar children who “just miss” assignment to the groups. The RDD is useful in educational settings because it allows treatment to be given to all children who need it, as determined by a specified cut-off point (Cappelleri & Trochim, 2003). The critical data for this study will be collected by schools using the LanguageScreen app (LanguageScreen.com). LanguageScreen is an online language test, which allows school staff (teachers, teaching assistants, trained volunteers) to assess children’s oral language skills. All assessments will be conducted by school staff. There are four subtests in LanguageScreen (Receptive vocabulary, Expressive vocabulary, Sentence repetition, Listening comprehension) all delivered via an App on a tablet. Scoring is automated within the app and assessments upload to a secure website. This method of assessment has already been validated in the previous EEF funded RCT evaluating the effectiveness of NELI (West et al., 2020). In that trial LanguageScreen was highly reliable (alpha .97), showed massive longitudinal stability over a 6-month period (r =.87; an under-estimate of test-retest reliability) and correlated very highly with standardized language tests administered at the same time by speech and language therapists (r=.95). LanguageScreen also yielded an estimate of the intervention effect size (d = .32) which was similar to the estimated effect size for the intervention derived from the individually administered standardised language tests (d= .26). In the current trial, at pretest all children in participating classrooms will be assessed with LanguageScreen. Only children with severe social and behavioural difficulties, or severe sensory impairments, will be excluded. The five children in each classroom with the lowest LanguageScreen scores will be assigned to receive NELI-R. Class teachers will also complete the 12-item behavioural adjustment teacher rating of the Brief Early Skills and Support Index (BESSI; https://www.cfr.cam.ac.uk/system/files/documents/revised-bessi-manual-sept-2015.pdf ) for all children in participating Reception classes. The same measures will be used at post-test as at pretest. |
| Intervention type | Behavioural |
| Primary outcome measure | Language is measured using the LanguageScreen assessment (https://www.LanguageScreen.com) at pretest in October/November 2020 and at posttest in June/July 2021. Analyses will be based on an RDD quasi-experimental design. A language total score will be created from the 4 LanguageScreen app subtests (expressive vocabulary, receptive vocabulary, sentence repetition, listening comprehension). A multi-level model (children clustered within schools) will relate posttest LanguageScreen scores to pretest LanguageScreen scores and a group dummy variable (intervention = 0; control = 1). The pre-test LanguageScreen scores and pre-test LanguageScreenscores squared will be added as predictors to the model to assess whether the function relating pre-test to post-test scores is linear or involves a quadratic component. Interactions between group and the linear and quadratic components will be tested for by adding group*pre-test and group*pre-test squared interaction terms to the model, in accordance with recommendations made by Trochim (2006). Non-significant terms will be dropped iteratively, in order to arrive at the model that provides the most parsimonious fit to the data. A discontinuity between a regression function for the intervention group and the function for the control group; a higher intercept for the intervention group would provide evidence for a causal effect of the intervention in increasing post-test scores. |
| Secondary outcome measures | Behavioural adjustment is measured using 12-item behavioural adjustment subtest from the Brief Early Skills and Support Index teacher rating (BESSI) at pretest in October/November 2020 and at posttest in June/July 2021. The researchers will also evaluate any changes in behavioural adjustment by using the same comparison of group regression functions as specified above with a composite behavioural adjustment measure defined by 12 items from the rating scale grouped into three subsets (Behavioural regulation = 7 items; Attention/hyperactivity = 3 items; Sociability = 2 items). |
| Overall study start date | 01/08/2020 |
| Completion date | 01/09/2021 |
Eligibility
| Participant type(s) | Other |
|---|---|
| Age group | Child |
| Lower age limit | 4 Years |
| Upper age limit | 5 Years |
| Sex | Both |
| Target number of participants | 350 - 450 children (depending on the number of classes each school enrolls on the trial) |
| Key inclusion criteria | 1. Children must be in Reception class 2. Children must be aged 4 to 5 years at pretest 3. Children receiving the NELI-R programme will be selected for having language weaknesses in comparison to their peers at school entry |
| Key exclusion criteria | 1. Children with such severe sensory impairments that they are unable to access the language screening assessment 2. Children with such severe emotional or behavioural issues that they are unable to access the language screening assessment |
| Date of first enrolment | 28/09/2020 |
| Date of final enrolment | 23/10/2020 |
Locations
Countries of recruitment
- United Kingdom
Study participating centre
United Kingdom
Sponsor information
Charity
9th Floor
Millbank Tower
21 - 24 Millbank
London
SW1P 4QP
United Kingdom
| Phone | +44 (0)207 802 1676 |
|---|---|
| info@eefoundation.org.uk | |
| Website | https://educationendowmentfoundation.org.uk/ |
"ROR" |
https://ror.org/03bhd6288 |
Funders
Funder type
Charity
Private sector organisation / Trusts, charities, foundations (both public and private)
- Alternative name(s)
- EducEndowFoundn, Education Endowment Foundation | London, EEF
- Location
- United Kingdom
Results and Publications
| Intention to publish date | 01/11/2021 |
|---|---|
| Individual participant data (IPD) Intention to share | No |
| IPD sharing plan summary | Stored in repository |
| Publication and dissemination plan | Results of the study will be published in a peer-reviewed scientific journal at the end of the trial. No additional documents are or will be available to be included with the pre-registration and the protocol is not published/online. |
| IPD sharing plan | At the end of the trial data on children’s scores on the pre- and posttest language screening will be entered into the EEF’s data archive (NB: FFT is the organisation appointed to manage EEF’s data archive). FFT will be responsible for all data archiving and for the management and security of the anonymised trial data. Anonymised data with all personal identifiers removed will also be stored on the UK Data Archive https://www.data-archive.ac.uk/. |
Editorial Notes
17/09/2020: Trial's existence confirmed by the Departmental Research Ethics Committee (DREC) in the Department of Education, The University of Oxford.
