The effectiveness of the Nuffield Early Language Intervention programme
| ISRCTN | ISRCTN12991126 |
|---|---|
| DOI | https://doi.org/10.1186/ISRCTN12991126 |
| Secondary identifying numbers | NELI-2018 |
- Submission date
- 01/06/2018
- Registration date
- 05/06/2018
- Last edited
- 16/02/2023
- Recruitment status
- No longer recruiting
- Overall study status
- Completed
- Condition category
- Mental and Behavioural Disorders
Plain English Summary
Background and study aims
This study will evaluate the effectiveness of the Nuffield Early Language Intervention programme, which is a 20-week educational programme suitable for children in reception year (aged 4 – 5) in UK primary schools. The programme is designed to improve the language skills of children with language learning weaknesses. It is delivered by specially trained teaching assistants working with children individually and in small groups. The version of the programme to be evaluated in the current study has been published by Oxford University Press (see OUP website for details of the programme https://global.oup.com/education/content/primary/series/nuffield-intervention/?region=uk ). Evidence from previous studies has shown the programme has a positive effect on children’s language skills. The current study will seek more robust evidence, from a larger sample of children, for the effectiveness of the published version of this programme, which includes training for teaching assistants by Elklan, a specialist in educational training.
Who can participate?
Children in reception classes (aged 4 to 5) at about 200 primary schools from a number of geographical areas across the UK, including Manchester, Bristol, Cornwall, Essex, Hertfordshire, Surrey, Warwickshire and the North West of England
What does the study involve?
Before the intervention programme begins all children in all participating reception classrooms undergo a short app-based assessment of four core language skills: expressive vocabulary, receptive vocabulary, sentence repetition, and listening comprehension. From this assessment the five children in each participating reception classroom with the poorest language scores are selected for detailed language assessment. After pretesting is completed, the schools are randomly allocated to one of two groups to either receive the Nuffield Early Language Intervention programme or to not receive it. The five children in each participating class in intervention group schools then receive the programme for 20 weeks (January to June 2019). Once the programme has been completed, all children in every school are assessed again using the same assessments (June/July 2019). The language of children in intervention group schools is compared to those in control group schools to see how much the children who took part have improved as a result of the Nuffield Early Language Intervention programme.
What are the possible benefits and risks of participating?
Children taking part in the intervention will benefit from additional small group and individual teaching sessions. Schools in the control group will receive monetary compensation and can choose to purchase the Nuffield Early Language Intervention programme once the study is completed. There are no anticipated risks of participation.
Where is the study run from?
1. University of Oxford (UK): intervention delivery team
2. RAND Europe (UK): independent evaluator
Who is funding the study?
Education Endowment Foundation (EEF) (UK)
Who is the main contact?
Dr Alex Sutherland, alex@researchdesign.uk
Contact information
Scientific
Department of Education
The University of Oxford
15 Norham Gardens
Oxford
OX2 6PY
United Kingdom
 ORCID ID |
0000-0002-4413-3601 |
|---|
Principal Investigator
c/o Department of Social Policy and Intervention
Barnett House
32 -37 Wellington Square
Oxford
OX1 2ER
United Kingdom
| ORCID ID |
0000-0003-0276-9705 |
|---|---|
| Phone | +44 (0)7881 556801 |
| alex@researchdesign.uk |
Study information
| Study design | 2-year cluster randomised controlled trial |
|---|---|
| Primary study design | Interventional |
| Secondary study design | Cluster randomised trial |
| Study setting(s) | School |
| Study type | Treatment |
| Participant information sheet | Not available in web format, please use the contact details to request a patient information sheet |
| Scientific title | The effectiveness of the Nuffield Early Language Intervention programme: a two-arm cluster randomised controlled trial |
| Study hypothesis | This is a cluster-randomised controlled to evaluate the effectiveness of the Nuffield Early Language Intervention programme. This is a 20-week evidence-based programme suitable for children in reception year in UK primary schools. The programme is designed to improve the oral language skills of children with language learning weaknesses. The programme is delivered by specially trained teaching assistants working with children individually and in small groups. The version of the programme to be evaluated in the current trial has been published by Oxford University Press (Fricke, Bowyer-Crane, Snowling, & Hulme, 2018), with training for the teaching assistants provided by Elklan. The OUP website provides details of the programme https://global.oup.com/education/content/primary/series/nuffield-intervention/?region=uk . Evidence from existing trials has shown the programme to have positive effects on children’s language skills (Fricke, Bowyer-Crane, Haley, Hulme, & Snowling, 2013; Fricke et al, 2017). The current trial will seek more robust evidence, from a larger sample, and an independent evaluation, for the effectiveness of the published version of this programme, with training to those using the programme delivered “at scale”. |
| Ethics approval(s) | The Departmental Research Ethics Committee (DREC) in the Department of Education, The University of Oxford, 27/04/2018, ref: ED-CIA-18-192 |
| Condition | Language disorder |
| Intervention | Current interventions as of 26/07/2018: The Education Endowment Foundation (EEF) has granted funding to RAND Europe and the University of Oxford to conduct a cluster-randomized trial in approximately 200 primary schools in the UK assessing the effectiveness of The Nuffield Early Language Intervention (NELI) programme. RAND are the EEF’s independent evaluators and are responsible for the evaluation study. Oxford are the intervention team and are responsible for school recruitment and intervention delivery. Schools will be recruited from a number of geographical areas including Manchester, Bristol, Cornwall, Essex, Hertfordshire, Surrey, Warwickshire and the North West of England. All children in all participating classrooms will undergo language assessments at pretest in two phases and all pretest data will be collected prior to randomisation. Phase 1 – Screening Every child in the participating reception classrooms in every school will be assessed using the Automated Test of Language Abilities (ATLAS) app, developed by the Oxford team. This test assesses four core language skills: expressive vocabulary, receptive vocabulary, sentence repetition, and listening comprehension. This testing will be conducted by school staff. Scoring of responses is done online using the app. Based on a composite score (composite z-scores from the 4 subtests) from the ATLAS the 5 children in each participating reception classroom with the poorest language scores will be selected for further testing in Phase 2. Phase 2 – In-depth testing Detailed individual tests will be given to the 5 children identified in screening, as follows: CELF recalling sentences subtest, CELF expressive vocabulary subtest, Renfrew Action Picture test (information and grammar) and the YARC Early Word Reading subtest. These baseline tests will be administered by testers working for the Oxford team, but will be undertaken pre-randomisation to ensure baseline is unbiased by allocation knowledge. After pretesting is completed, schools will be randomised (within each area) by RAND Europe to one of two arms: intervention vs control. Intervention schools will receive the 20-week Nuffield Early Language Intervention programme and training in its use by Elklan tutors. The control schools will be required to agree to testing of children and will undertake not to use the Nuffield Early Language Intervention programme in that year. Control schools will be provided with monetary compensation (£1000 per school) which they may use to purchase the Nuffield Early Language Intervention programme the following year. At posttest the same measures will be used as at pretest. Individually administered language tests will be given by new independent testers working for a third party who will be sub-contracted by the RAND Europe team. The posttest assessors will be blind to condition, but will be trained to administer the tests by the Oxford team. ATLAS testing at posttest will be conducted by school staff, but school staff will not be blind to allocation. Previous interventions: Funding has been provided by the Education Endowment Foundation (EEF) to conduct a cluster-randomized trial in approximately 200 primary schools in the UK assessing the effectiveness of The Nuffield Early Language Intervention programme. Schools will be recruited from a number of geographical areas including Manchester, Bristol, Cornwall, Essex, Hertfordshire, Surrey, Warwickshire and the North West of England. All children in all participating classrooms will undergo language assessments at pretest in two phases and all pretest data will be collected prior to randomisation. Phase 1 – Screening Every child in the participating reception classrooms in every school will be assessed using the Automated Test of Language Abilities (ATLAS). This test assesses four core language skills: expressive vocabulary, receptive vocabulary, sentence repetition, and listening comprehension. This testing will be conducted by school staff. Scoring of responses is done online using the app. Based on a composite score (composite z-scores from the 4 subtests) from the ATLAS the 5 children in each participating reception classroom with the poorest language scores will be selected for further testing in Phase 2. Phase 2 – In-depth testing Detailed individual tests will be given to the 5 children identified in screening, as follows: CELF recalling sentences subtest, CELF expressive vocabulary subtest, Renfrew Action Picture test (information and grammar) and the YARC Early Word Reading subtest. These tests will be administered by testers trained by the research team. After pretesting is completed, schools will be randomised (within each area) by an independent evaluator (RAND corporation) to one of two arms: intervention vs control. Intervention schools will receive the 20-week Nuffield Early Language Intervention programme and training in its use by Elklan tutors. The control schools will be required to agree to testing of children and will undertake not to use the Nuffield Early Language Intervention programme in that year. Control schools will be provided with monetary compensation (£1000 per school) which they may use to purchase the Nuffield Early Language Intervention programme the following year. At posttest the same measures will be used as at pretest. ATLAS testing at posttest will be conducted by school staff. Individually administered language tests will be given by new testers, blind to condition, who will be trained to administer the tests by the research team. |
| Intervention type | Behavioural |
| Primary outcome measure | Current primary outcome measure: The main trial analysis will be conducted by RAND using multilevel structural equation modelling (SEM) to account for the hierarchical nature of the study data and cluster-randomisation. The primary outcome will be a latent language variable created from the four individually administered language tests ((i) CELF recalling sentences subtest, (ii) CELF expressive vocabulary subtest, Renfrew Action Picture test ((iii) information and (iv) grammar). Analyses will be based on this latent outcome variable. The same language latent variable will be created for pretest and posttest scores by the RAND team. In the main analysis model, the pretest latent variable will be the covariate, and the posttest latent variable the outcome measure. The analyses will also be undertaken on each subscale used to create the latent variable. The effects of the intervention will be measured by an appropriate effect size such as standardized differences in means for a group dummy variable comparing pupils in treatment and control schools. Both primary and secondary outcomes will be measured at pretest (baseline) in September/October 2018 and at posttest (on completion of the 20-week intervention in June/July 2019). The funder is open to conducting a longer-term (30-week) follow-up but that has not yet been confirmed. A secondary analysis will be undertaken that follows the EEF’s stated preference for using multilevel modelling for clustered designs that uses a predicted outcome based on the SEM (EEF, 2018). Previous primary outcome measure: Analyses will be based on latent variable ANCOVA models implemented in an SEM framework. The primary outcome will be a language latent variable defined by loadings from the individually administered language tests (CELF recalling sentences subtest, CELF expressive vocabulary subtest, Renfrew Action Picture test (information and grammar). The same language latent variable will be created for pretest and posttest scores. The pretest latent variable will be the covariate, and the posttest latent variable the outcome measure. The effects of the intervention will be measured by the y-standardized regression coefficient for a group dummy variable. The effects of clustering within schools will be accounted for by using robust (Huber-White) cluster standard errors. Both primary and secondary outcomes will be measured at pretest (baseline) in September 2018 and at posttest (on completion of the 20-week intervention in June/ July 2019). |
| Secondary outcome measures | Current secondary outcome measures: The secondary outcome will be a language latent variable defined by loadings from the ATLAS language app subtests (expressive vocabulary, receptive vocabulary, sentence repetition, listening comprehension). The analysis plan for this secondary analysis is identical to that for the primary outcome. The study will also evaluate any changes in word reading scores from the EWR test by using multilevel modelling on a latent variable from this test. Both primary and secondary outcomes will be measured at pretest (baseline) in September 2018 and at posttest (on completion of the 20-week intervention in June/July 2019). Previous secondary outcome measures: The secondary outcome will be a language latent variable defined by loadings from the ATLAS language app subtests (expressive vocabulary, receptive vocabulary, sentence repetition, listening comprehension). The analysis plan for this secondary analysis is identical to that for the primary outcome. We will also evaluate any changes in word reading scores from the EWR test by using ANCOVA on a latent variable from this test. The effects of clustering within schools will be accounted for in all analyses by using robust (Huber-White) cluster standard errors. Both primary and secondary outcomes will be measured at pretest (baseline) in September 2018 and at posttest (on completion of the 20-week intervention in June/July 2019). |
| Overall study start date | 01/03/2018 |
| Overall study end date | 30/09/2021 |
Eligibility
| Participant type(s) | Other |
|---|---|
| Age group | Child |
| Lower age limit | 4 Years |
| Upper age limit | 5 Years |
| Sex | Both |
| Target number of participants | The exact numbers of children participating in the trial is not known until recruitment is completed. Oxford will seek to recruit at least 200 schools, and would expect each school to have at least 25 children in the Reception class. Based on ATLAS screening, Oxford will select the 5 children in each classroom with the poorest language skills giving a total sample of 1000 children (500 intervention and 500 control). This will be a cluster-randomised design with approximately, 100 clusters (schools) in each arm. The schools will be spread across approximately 10 areas. |
| Participant inclusion criteria | 1. Children must be in Reception class 2. Children must be aged 4 to 5 years at pretest 3. Children will be selected for having language weaknesses in comparison to their peers at school entry |
| Participant exclusion criteria | 1. Children will be excluded if they have a significant visual impairment 2. Children will be excluded if they have a significant auditory impairment 3. Children will be excluded if they have significant behavioural difficulties, such that they would not be able to take part small group activities |
| Recruitment start date | 11/06/2018 |
| Recruitment end date | 30/07/2018 |
Locations
Countries of recruitment
- United Kingdom
Study participating centre
United Kingdom
Sponsor information
Charity
9th Floor, Millbank Tower
21 - 24 Millbank
London
SW1P 4QP
United Kingdom
| Website | https://educationendowmentfoundation.org.uk/ |
|---|---|
"ROR" |
https://ror.org/03bhd6288 |
Funders
Funder type
Charity
Private sector organisation / Trusts, charities, foundations (both public and private)
- Alternative name(s)
- EducEndowFoundn, Education Endowment Foundation | London, EEF
- Location
- United Kingdom
Results and Publications
| Intention to publish date | 01/07/2022 |
|---|---|
| Individual participant data (IPD) Intention to share | No |
| IPD sharing plan summary | Stored in repository |
| Publication and dissemination plan | Results of the study will first be published by RAND Europe in a report for the EEF. The study will subsequently be published in a peer-reviewed scientific journal at the end of the trial. |
| IPD sharing plan | Anonymised participant level data will be entered into the Education Endowment Foundation’s (EEF) archive by the Fisher Family Trust (FFT: the organisation appointed to manage the EEF’s data archive). Data will include the pupil outcome data from pre- and post-test language assessments, which will be linked with National Pupil Database (NPD) data. A privacy notice detailing this information will circulated to all participants along with information on the trial and an option to opt out of the study. |
Study outputs
| Output type | Details | Date created | Date added | Peer reviewed? | Patient-facing? |
|---|---|---|---|---|---|
| Results article | 30/03/2021 | 01/12/2022 | Yes | No | |
| Funder report results | 31/05/2020 | 15/02/2023 | Yes | No | |
| Protocol article | v1.0 | 15/05/2019 | 15/02/2023 | Yes | No |
| Statistical Analysis Plan | v1.0 | 01/04/2019 | 15/02/2023 | Yes | No |
Editorial Notes
16/02/2023: Internal review.
15/02/2023: The following updates have been made to the study record and the plain English summary updated accordingly:
1. Principle investigator contact added.
2. Protocol article uploaded.
3. Uploaded statistical analysis plan.
4. Funder report results uploaded.
06/12/2022: A contact was removed.
01/12/2022: Publication reference added.
11/01/2021: The following changes were made to the trial record:
1. The overall end date was changed from 30/09/2019 to 30/09/2021.
2. The intention to publish date was changed from 01/07/2020 to 01/07/2022.
26/07/2018: The following changes were made to the trial record:
1. A new contact was added.
2. The interventions and the primary and secondary outcome measures were updated.
