Effectiveness trial of Nuffield Early Language Intervention - Preschool
| ISRCTN | ISRCTN17801846 |
|---|---|
| DOI | https://doi.org/10.1186/ISRCTN17801846 |
- Submission date
- 09/08/2024
- Registration date
- 15/08/2024
- Last edited
- 12/08/2024
- Recruitment status
- No longer recruiting
- Overall study status
- Ongoing
- Condition category
- Other
Plain English Summary
Background and study aims
The Nuffield Early Language Intervention (NELI) Preschool is an oral language enrichment programme, developed by Charles Hulme and his research group at the University of Oxford, and is now delivered by OxEd and Assessment (OxEd), a University spinout company founded to translate the research into practical application. It is designed for children in the year before they begin school (aged 3-4). Previous studies indicate that NELI has a positive impact on oral language skills of small groups of pre-school children, and on older children in their reception year. This trial is wider in scope, expanded to including PVI nursery settings alongside maintained nursery settings, and aims to investigate the impact of NELI Preschool on oral language skills of 3–4-year-olds in nursery settings in England.
Who can participate?
• State maintained or PVRI settings with a minimum of fourteen 3 to 4-year-old children attending 15+ hours per week in the 2023-24 academic year that are located within nine Stronger Practice Hubs (SPHs): East of England Early Years Stronger Practice Hub, Early Years South West Stronger Practice Hub, Northern Lights Early Years Stronger Practice Hub, Yorkshire and Humber Together Early Years Stronger Practice Hub, Kent Early Years Stronger Practice Hub, Derbyshire and Nottinghamshire Early Years Stronger Practice Hub, Thames Valley Early Years Stronger Practice Hub, London South Early Years Stronger Practice Hub, Great North Early Years Stronger Practice Hub.
• Settings are not eligible if they are participating in another SPH funded programme delivered in the 2024-2025 academic year including allocation to the control group for one of the other SPH trials (Early Talk Boost, One Programme, EYCP, Concept Cat, Communication Friendly Settings).
• Settings cannot take part if participating in another EEF funded evaluation in the early years or if they are participating in the DfE Early Years Professional Development Programme during the year of evaluation delivery (2024/25).
• Settings that took part in the previous evaluation of NELI Preschool (known as the NELI – Nursery or NELI-N efficacy trial) by the University of Oxford are not eligible to take part in this evaluation.
• All 3 to 4-year-old children (as on 31st August 2024) who are in the pre-school classroom are eligible to take part in the evaluation. Children who ‘move up’ or join the pre-school classroom after settings have completed LanguageScreen are not eligible for the evaluation; however, in settings allocated to the intervention group, these children can still take part in the Whole Group Sessions.
What does the study involve?
Once a setting has signed up to take part in the evaluation, they will share data for the 3–4-year-old children enrolled in the 2024-2025 academic year, complete online surveys and administer LanguageScreen assessments with all 3 -4 year-olds (approximately 10 minutes per child) as part of the baseline activities. After this is complete, settings will be randomised to either the intervention or control group. Nominated staff from the intervention settings will complete the online training, deliver the NELI Preschool programme to all 3-4-year-old children in the pre-school classroom between January and June 2025, and complete weekly attendance registers for NELI Preschool sessions delivered. Control settings will continue with their usual practice during this time. After the intervention period all settings will complete online surveys and facilitate administration of LanguageScreen and RAPT assessments by the NFER Test Administrators. NELI Preschool leads from intervention settings will be invited to complete a follow-up online survey one year after the trial ends. Additionally, a small number of intervention settings will be invited to act as (anonymous) case studies for the evaluation. Participation in this element is voluntary.
What are the possible benefits and risks of participating?
Settings will contribute to the evidence relating to the impact of NELI Preschool on language skills for 3 to 4-year-old children, and their participation will also help to understand how best to implement the programme in the future and at a larger scale.
Where is the study run from?
The study is being run by the National Foundation for Educational Research (NFER) in England. Delivery of the NELI programme occurs at settings taking part in the trial. Data on children’s participation (including outcome measures) will be collected at the settings and shared with NFER.
When is the study starting and how long is it expected to run for?
August 2023 to July 2026
Who is funding the study?
Education Endowment Foundation (UK)
Who is the main contact?
Palak Roy, p.roy@nfer.ac.uk
Contact information
Public, Scientific
NFER
The Mere
Upton Park
Slough
SL1 2DQ
United Kingdom
 ORCID ID |
0009-0009-3951-5937 |
|---|---|
| Phone | +44 1753 574123 |
| p.roy@nfer.ac.uk |
Principal Investigator
NFER
The Mere
Upton Park
Slough
SL1 2DQ
United Kingdom
| ORCID ID |
0000-0001-6827-6132 |
|---|---|
| Phone | +44 1753 574123 |
| s.welbourne@nfer.ac.uk |
Study information
| Study design | Two-arm cluster randomized controlled trial with random allocation at setting level |
|---|---|
| Primary study design | Interventional |
| Secondary study design | Cluster randomised trial |
| Study setting(s) | Childcare/pre-school |
| Study type | Other |
| Participant information sheet | https://www.nfer.ac.uk/for-schools/participate-in-research/participate-in-research-projects/effectiveness-trial-of-nuffield-early-language-intervention-preschool/ |
| Scientific title | A cluster randomised controlled trial of the effectiveness of the Nuffield Early Language Intervention (NELI) – Preschool programme on the oral language skills of pre-school children aged 3-4. |
| Study hypothesis | The primary research question for this trial is: How effective is NELI Preschool at improving oral language skills of 3-4 year-old children in intervention settings compared to children in control group settings? The secondary research questions for this trial are: 1. How effective is NELI Preschool at improving different aspects of children’s oral language skills as measured by the subtests of LanguageScreen and Renfrew Action Picture Test (RAPT)? 2. How effective is NELI Preschool at improving the language skills of the subgroup of six children selected to receive the targeted component of the intervention? 3. How effective is NELI Preschool at improving the language skills of the subgroup of six children who should have been selected to receive the targeted component of the intervention? 4. How effective is NELI Preschool at improving the language skills of children who only receive the whole-class (enrichment) component of the intervention? 5. How effective is NELI Preschool at improving the language skills of disadvantaged children (as identified by Early Years Pupil Premium or EYPP)? 6. Is NELI Preschool effective at improving language skills of 3-4-year-old children in Private, Voluntary and Independent (PVI) intervention settings compared to children in PVI control settings? |
| Ethics approval(s) |
Approved 12/10/2023, NFER Code of Practice Group (The Mere, Upton Park, Slough, SL1 2DQ, United Kingdom; +44 (0)1753574123; Enquiries@nfer.ac.uk), ref: NELIN |
| Condition | The primary analysis is of NELI Preschool’s targeted intervention, for children identified as requiring additional targeted support (those with the lowest language scores at baseline) for oral language skills. |
| Intervention | Summary: Prior to randomisation, all settings will administer the LanguageScreen assessment. They will then be randomised into two arms, intervention and control, on a 1:1 basis. Randomisation will be stratified by two variables: (i) nursery setting type – maintained nursery settings and PVI settings and (ii) setting size (settings with one preschool class vs those with more than one preschool class). Randomisation will be carried out by an NFER statistician using R Code, which will be stored for reproducibility and transparency. The statistician will be provided with a minimum set of variables required to carry out the randomisation: each setting’s unique ID and the two stratification variables. After randomisation, practitioners from intervention settings will receive NELI Preschool training and will deliver the intervention to children for 20 weeks. NFER will share baseline LanguageScreen assessment results with intervention settings who will use this data along with OxEd’s Targeted Group Selection Guidance to select children for the targeted intervention. By contrast, control settings will continue their usual practice and will not have access to LanguageScreen assessment results or NELI Preschool training or materials during the trial. After the completion of the intervention, LanguageScreen and RAPT assessments will be administered to settings within both study arms. Intervention detail: The intervention takes place over twenty weeks. Five whole-class sessions are delivered each week, along with three small-class sessions and one one-to-one session (per targeted child). Whole-class sessions are intended to last 15-20 minutes, small group sessions 10-15 minutes and one-to-one sessions 5-10 minutes. At three points (weeks 1, 10 and 20), children within the targeted group are assessed to determine how they are progressing in relation to appropriate language development targets. Settings are provided with material for 20 storybooks, one for each week of the intervention. Targeted sessions use weekly session cards, story sequencing cards and press-out characters to facilitate small-world play. Settings are provided with a parent ‘home learning link’ template that can be sent out to parents for each week of the intervention, to inform them of what their child is doing and to promote parental engagement in the language development process. Settings are given access to an online training platform, as well as an online Delivery Support Hub, along with access to the online LanguageScreen App that is used for completing the pupil screening assessments required for selecting the group for the targeted component of the programme (Note that the LanguageScreen is also the baseline assessment for the trial). The online training course was developed by the research team and the University of Oxford (now at OxEd). The training course forums and Delivery Support Hub are moderated by the OxEd team and speech and language specialists. Supporting webinars are hosted by the OxEd team alongside occasional experts and special guests to provide additional guidance and support. Before outlining the research questions, it is helpful to define the NELI Preschool implementation models and a number of different sample populations that used for analysis. All settings will be asked to choose their preferred NELI Preschool implementation model prior to randomisation. These models, described below, are in line with the Targeted Group Selection Guidance that the intervention settings will receive. A. One targeted intervention group with six children – This is the most common format where settings are single form entry and have only one preschool class. The setting will run one whole class session and will select six children for targeted intervention. This model will also be selected by settings with more than one preschool class, and where the setting will run a separate whole class session for each class but each class does not have at least 12 children who attend the setting for 15 hours or more (see model B for more information on this additional criterion for running more than one targeted intervention group). In this case, settings will run a separate whole-class session for each class, but select six children for targeted intervention across the setting, rather than per class. B. Multiple targeted intervention groups with six children per group – This model works best in larger, multiple form entry settings where NELI Preschool can be run separately for each class. This means the setting runs the whole programme more than once, including separate targeted intervention groups, one for each class. Six children per class will be selected for targeted intervention. The additional criterion for running more than one targeted intervention group is to have at least 12 children who attend the setting for 15 hours or more in each class, as per the eligibility criteria for settings to participate in the trial (see Participant selection section below). In cases where there are not at least 12 such children, the setting will select six children across the setting for targeted intervention group (Model A above). C. One targeted intervention group with six or more children but split across multiple classes – This model is ideal for settings that run more than one whole group sessions (one for each class), but they find the targeted intervention in Model B too intensive and unrealistic. This means settings would ideally select three children from each class to be included in targeted intervention irrespective to the number of children in each class who attend 15 or more hours. This means that the evaluation team will have prior knowledge of each setting's intention regarding the selection of children for targeted intervention. Specifically, they will know whether the setting plans to select three or six children per class or if they intend to select six children per setting for the targeted intervention. This information will be available for all settings prior to randomisation. Analysis samples • S1 Ideally Targeted Children: NFER’s sample of children in a setting who should have been selected for the targeted component of NELI Preschool if the settings followed OxEd’s Targeted Group Selection Guidance correctly (as per the setting’s preferred implementation model A, B or C, as described above). Usually, these will be the six children within a class who have the lowest baseline language score, attend at least 15 hours per week and are not flagged as being unable to participate in small group work (i.e., due to behaviour, or complex special educational needs). The selection of children in this sample will be based on the preference of implementation model that settings provide prior to randomisation. • S2 Practitioner Targeted Children (intervention settings only): Practitioner sample of children (actually selected by the setting to receive the targeted intervention). Settings will identify these children and share this information with NFER. In the majority of settings, it is expected that the S2 sample will be the same as S1 if settings followed OxEd’s Targeted Group Selection Guidance correctly. It will only be different from S1 if settings/practitioners applied their own discretion to select children for the targeted component. • S3 Enrichment-only Sample: NFER’s sample of six randomly chosen children (per class in a setting), who attend the setting at least 15 hours per week and are not part of S1 or S2. • S4 EYPP: All children in the setting between 3 and 4 who are entitled to Early Years Pupil Premium (EYPP). This may include children who attend the settings for less than 15 hours a week and are part of the other samples. The primary research question is: RQ1: How effective is NELI Preschool at improving oral language skills of 3-4 year-old children in intervention settings compared to children in control group settings? To answer the primary research question, we will use all children in S1 and S3 in intervention and control settings. Secondary research questions for this trial will ascertain effectiveness of NELI Preschool on the six components of the primary outcome measure. These research questions will utilise the six secondary outcomes mentioned earlier. RQ2: How effective is NELI Preschool at improving different aspects of children’s oral language skills as measured by the subtests of LanguageScreen and RAPT? These RQs will in turn be answered for: LanguageScreen subtests of Expressive Vocabulary, Listening Comprehension, Receptive Vocabulary and Sentence Repetition, and RAPT subtests of Information and Grammar. The sample for RQ2 will be identical to the sample used for RQ1. Additional research questions for this trial will ascertain effectiveness of NELI Preschool for subgroups of children. As explained in the previous section on samples, S1 and S2 samples may differ in intervention settings if settings do not follow OxEd’s Targeted Group Selection Guidance fully. Therefore, we propose two separate research questions to determine the programme’s effectiveness on children who should have been selected to receive the targeted component of the intervention and those who were actually selected to receive the targeted component of the intervention. RQ3a: How effective is NELI Preschool at improving the language skills of the subgroup of six children selected to receive the targeted component of the intervention (S2)? To answer this research question, we will use all children in S2 in intervention settings and S1 in control settings. S1 in control settings is necessary as a comparison here because control settings will not have access to the intervention including LanguageScreen data which is an integral part of the intervention. Consequently, they will not be able to identify children who would have received the targeted component, hence won’t have an S2. RQ3b: How effective is NELI Preschool at improving the language skills of the subgroup of six children who should have been selected to receive the targeted component of the intervention (S1)? RQ4: How effective is NELI Preschool at improving the language skills of children who only receive the whole-class (enrichment) component of the intervention (S3)? RQ5: How effective is NELI Preschool at improving the language skills of disadvantaged children (as identified by EYPP; S4)? RQ6: Is NELI Preschool effective at improving language skills of 3-4-year-old children in PVI intervention settings compared to children in PVI control settings? This research question will utilise the same group of children as RQ1 (S1 and S3) for PVI settings only. |
| Intervention type | Other |
| Primary outcome measure | The primary outcome measure is a latent oral language variable that will be formed from expressive and receptive subtests in LanguageScreen and the Renfrew Action Picture Tests (RAPT) 1. LanguageScreen is administered both at baseline and post intervention 2. Renfrew Action Picture Tests (RAPT) are only administered post intervention Baseline LanguageScreen assessments will be administered by setting practitioners prior to randomisation. Endline assessments (both LanguageScreen and RAPT) will be independently administered by NFER Test Administrators who will visit each participating setting after the completion of the intervention. |
| Secondary outcome measures | These are based on the data collected for the primary outcome measure, and are the individual expressive and receptive subtests of LanguageScreen and the RAPT.LanguageScreen is administered both at baseline and post intervention. Renfrew Action Picture Tests (RAPT) are only administered post intervention. 1. LanguageScreen Expressive Vocabulary 2. LanguageScreen Listening Comprehension 3. LanguageScreen Receptive Vocabulary 4. LanguageScreen Sentence Repetition 5. RAPT information 6. RAPT grammar |
| Overall study start date | 09/08/2023 |
| Overall study end date | 31/07/2026 |
Eligibility
| Participant type(s) | Learner/student, Other |
|---|---|
| Age group | Child |
| Lower age limit | 3 Years |
| Upper age limit | 4 Years |
| Sex | Both |
| Target number of participants | 320 settings; 3840 children |
| Participant inclusion criteria | Setting inclusion criteria: 1. Settings must either be state-maintained or PVI nurseries. Settings registered as ‘childcare on domestic premises’ will also be eligible to take part in the trial, as these settings have more than four people who provide care and the settings are subject to the same requirements as nurseries for staffing ratios and staff qualification levels. 2. Settings should have a minimum of fourteen 3 to 4-year-old children attending 15 or more hours per week during the 2023-2024 academic year (starting September 2023). This criterion is a proxy for the minimum number of children required for the trial in 2024-25, i.e., minimum of 12 children per setting . 3. Settings must be willing to allocate time for a minimum of two nursery staff members to complete online NELI Preschool training, encouraging as many practitioners who work with 3 to 4-year-olds as possible to complete the online training. 4. Settings must be located within the following Local Authorities from nine SPHs: Bexley, Bournemouth, Christchurch and Poole, Bracknell Forest, Brighton and Hove, Bromley, Buckinghamshire, Cambridgeshire, Cornwall, Croydon, Darlington, Derby, Derbyshire, Devon, Dorset, Durham, Ealing, East Riding of Yorkshire, East Sussex, Gateshead, Greenwich, Hammersmith and Fulham, Hampshire, Hartlepool, Hillingdon, Hounslow, Isle of Wight, Kent, Kingston upon Hull, Kingston upon Thames, Lambeth, Lewisham, Medway, Merton, Middlesbrough, Milton Keynes, Newcastle upon Tyne, Norfolk, North East Lincolnshire, North Lincolnshire, North Tyneside, North Yorkshire, Northumberland, Nottingham, Nottinghamshire, Oxfordshire, Plymouth, Portsmouth, Reading, Redcar and Cleveland, Richmond upon Thames, Slough, South Tyneside, Southampton, Southwark, Stockton-on-Tees, Suffolk, Sunderland, Surrey, Sutton, Torbay, West Berkshire, West Sussex, Wandsworth, Windsor and Maidenhead, Wokingham, and York. 5. Settings are not eligible if they are participating in another SPH funded programme delivered in the 2024-2025 academic year including allocation to the control group for one of the other SPH trials (e.g., Early Talk Boost, The ONE Programme, EYCP, Concept Cat, Communication Friendly Settings). 6. Settings cannot take part if they are participating in another EEF-funded programme or early years evaluation in the in 2024-25 academic year. Although this is identified as an eligibility criterion, there is some flexibility for the OxEd team (in agreement with NFER and the EEF) to recruit settings to the trial even if they are part of an EEF evaluation. This is especially true where the programme in question is not very similar to NELI Preschool in terms of its recipients and/or aims, or it is viewed as a business as usual (BAU). 7. Settings from the efficacy trial or those taking part in the formative evaluation cannot take part in this trial. Pupil inclusion criteria: 1. All 3–4-year-old children on 31st August 2024 and those who can engage with LanguageScreen assessment (e.g., do not have complex special educational needs) will be assessed with LanguageScreen at baseline and will form part of the trial. 2. This will also include children who attend the setting for less than 15 hours a week, as some of these children may subsequently be discovered to be entitled to EYPP. Only the children with baseline LanguageScreen will be included in the trial. Settings will be asked not to assess children who were not between 3 and 4-years old on 31st August 2024 or have complex special educational needs even though they may be present in the preschool classroom. These children can join whole class sessions (if their setting is randomised to intervention group) but will not form part of the trial. Inclusion within each of the analysis samples will depend on the criteria for each of those samples as set out above. |
| Participant exclusion criteria | Settings exclusion criteria: 1. Participating in another SPH funded programme delivered in the 2024-2025 academic year. 2. Participating in another EEF-funded programme or early years evaluation in the in 2024-25 academic year. 3. Took part in the previous efficacy trial or the formative evaluation. Pupil exclusion criteria: 1. No special educational needs that precludes using the LanguageScreen assessment. |
| Recruitment start date | 31/01/2024 |
| Recruitment end date | 24/07/2024 |
Locations
Countries of recruitment
- England
- United Kingdom
Study participating centre
Upton Park
Slough
SL1 2DQ
United Kingdom
Sponsor information
Charity
9th Floor
Millbank Tower
21-24 Millbank
London
SW1P 4QP
England
United Kingdom
| Phone | +44 (0)20 7802 1676 |
|---|---|
| info@eefoundation.org.uk | |
| Website | https://educationendowmentfoundation.org.uk/ |
"ROR" |
https://ror.org/03bhd6288 |
Funders
Funder type
Charity
Private sector organisation / Trusts, charities, foundations (both public and private)
- Alternative name(s)
- EducEndowFoundn, Education Endowment Foundation | London, EEF
- Location
- United Kingdom
Results and Publications
| Intention to publish date | 31/03/2026 |
|---|---|
| Individual participant data (IPD) Intention to share | Yes |
| IPD sharing plan summary | Stored in non-publicly available repository |
| Publication and dissemination plan | We intend to publish the final report on the Education Endowment Foundation's website in March 2026, with an addendum report that additionally includes analyses from the longitudinal impact and IPE expected to be published in June 2027. |
| IPD sharing plan | After the main evaluation report is published on the EEF website (likely to be in March 2026), NFER will share children’s personal data (except the data collected from the NPD and audio recordings from RAPT assessments) with OxEd via NFER’s secure online portal, which will enable OxEd to link it with the data they hold and undertake their own research to publish findings from the research trial in scientific journals. At this point, OxEd will become an independent data controller. Three months after the publication of the final evaluation report (currently planned for June 2027), pseudonymised matched child-level data will be added to the EEF archive, which is managed by FFT Education on behalf of the EEF and hosted by the Office of National Statistics (ONS). This will enable the EEF and other research teams to use the pseudonymised data as part of subsequent research through the ONS Approved Researcher Scheme. At this point, the EEF becomes the data controller and is responsible for taking decisions about the means and purposes of processing. Data will be deposited within the Secure Research Service (SRS) within three months of publication of the addendum report (i.e. by September 2027). Once finalised, the archived data will be open to the research community for analysis within the SRS, and can be accessed by applying to the DfE. |
Study outputs
| Output type | Details | Date created | Date added | Peer reviewed? | Patient-facing? |
|---|---|---|---|---|---|
| Protocol file | version 1.0 | 01/08/2024 | 09/08/2024 | No | No |
Additional files
Editorial Notes
09/08/2024: Trial's existence confirmed by Education Endowment Foundation.
