Steps to Evaluation: How to set up an evaluation with external organisations

ISRCTN	ISRCTN26468622
DOI	https://doi.org/10.1186/ISRCTN26468622
Secondary identifying numbers	UoB_FS_EvProto_4Projects

Submission date: 17/11/2017
Registration date: 16/01/2018
Last edited: 07/06/2018

Recruitment status: No longer recruiting
Overall study status: Completed
Condition category: Other

Prospectively registered

Protocol

Statistical analysis plan

Results

Individual participant data

Record updated in last year

Plain English Summary

Background and study aims
A lot has been written about how to evaluate programmes to establish if they are effective in achieving their aims. However there is little guidance on how to actually work with organisations to identify which programmes they offer are suitable for evaluation and the steps involved in setting up those evaluations. Research evaluating programmes or interventions rarely discuss the steps involved prior to evaluation to identify suitable projects for evaluation and describe the steps required, often with academic institutions working in partnership with external organisations, to set up an evaluation. The aim of this study is to help determine the steps used to evaluate projects.

Who can participate?
Children under five as well as stakeholders working on the identified interventions.

What does the study involve?
Four interventions have been identified as a consequence of following the steps outlined in this protocol. Sign 4 Little Talkers/Big Feelings teaches sign language and stories for pre-school children during a two month intervention period. Data is collected at baseline and after two months. Follow-up visits are conducted up to three months post intervention delivery. The HENRY programme is an 8 week course for parents with children with concerns regarding weight and eating habits. Surveys containing a number of behavioural measures are completed at the end of the 8 week course and again after 3 months. Incredible Years is a 14 week programme for parents with children with behavioural problems. Parents fill in behavioural measures at the beginning and end of the course, and are followed up after 3 months. Parents as Partners is a 16 week course offering group counselling to parents to improve couple relationships to in turn improve child wellbeing and developmental outcomes. Relationship measures are taken at the beginning and end of the course and 3 months afterwards.

What are the possible benefits and risks of participating?
As this protocol details the stages required prior to setting up evaluations, participants are not involved and consequently there are no direct benefits or risks. Any impact on participants would be anticipated when each programme identified by the steps described in this protocol is described in detail in separate, forthcoming protocols.

Where is the study run from?
1. University of Bedfordshire (UK)
2. Flying Start Luton (UK)

When is the study starting and how long is it expected to run for?
May 2016 to November 2018

Who is funding the study?
1. University of Bedfordshire (UK)
2. Flying Start Luton (UK)

Who is the main contact?
Dr Rosemary Davidson
rosemary.davidson@beds.ac.uk

Contact information

Dr Rosemary Davidson
Public

Institute for Health Research
University of Bedfordshire
University Square
Luton
LU1 3JU
United Kingdom

ORCID ID	0000-0002-0010-4809
Phone	+44 796 667 6974
Email	rosemary.davidson@beds.ac.uk

Prof Gurch Randhawa
Scientific

Institute for Health Research
University of Bedfordshire
Putteridge Bury Campus
Hitchin Road
Luton
LU2 8LE
United Kingdom

ORCID ID	0000-0002-2289-5859
Email	gurch.randhawa@beds.ac.uk

Study information

Study design	Observational study
Primary study design	Observational
Secondary study design	Cohort study
Study setting(s)	Community
Study type	Quality of life
Participant information sheet	No participant information sheet available, however forthcoming participant information sheets will be made available for each intervention identified as a consequence of this evaluation approach.
Scientific title	Steps to Process and Outcomes Evaluation: The Flying Start Luton approach
Study acronym	STEPS
Study hypothesis	Much has been written about how to evaluate projects, yet there is little on the steps required prior to this to identify suitable projects for evaluation. This is particularly salient when working in the community with partner organisations who wish their provision to be evaluated in order to build an evidence base and rationale behind their work.
Ethics approval(s)	This is a protocol to allow evaluators to get to the stage where they can identify projects and thereafter seek ethical approval to evaluate a specific project, therefore no ethical approval is needed at this stage. Ethical approval will be sought for the next stage of the evaluation, where the projects have been identified, allowing us to give specific details to ethics committees.
Condition	Early years health and well-being.
Intervention	Four interventions have been identified as a consequence of following the steps outlined in this protocol. Sign 4 Little Talkers/Big Feelings teaches sign language and stories for pre-school children during a two month intervention period. Data is collected at baseline and after two months. Follow-up visits are conducted up to three months post intervention delivery. The HENRY programme is an 8 week course for parents with children with concerns regarding weight and eating habits. Surveys containing a number of behavioural measures are completed at the end of the 8 week course and again after 3 months. Incredible Years is a 14 week programme for parents with children with behavioural problems. Parents fill in behavioural measures at the beginning and end of the course, and are followed up after 3 months. Parents as Partners is a 16 week course offering group counselling to parents to improve couple relationships to in turn improve child wellbeing and developmental outcomes. Relationship measures are taken at the beginning and end of the course and 3 months afterwards.
Intervention type	Behavioural
Primary outcome measure	Sign 4 1. Listening and Attention, Speaking, Managing Feelings & Behaviour, and Understanding are measured using Early Years Outcomes at baseline and 6 month follow-up HENRY 1. Self reported measures relating to parenting skills and healthy lifestyles are taken at baseline and 3 month follow-up. Incredible Years 1. Children’s behaviour is measured using the Strengths & Difficulties questionnaire at baseline and 14 weeks Parents as Partners 1. Parenting style is measured using the Revised Parenting Index at baseline and 16 weeks
Secondary outcome measures	Sign 4 1. Children’s well-being is measured using Leuven Well-being scales at baseline and 6 month follow-up. HENRY 1. Group facilitator and parental views of HENRY sessions are measured using surveys and interviews at 8 weeks and 3 month follow-up IY 1. Group facilitator and parental views of Incredible Years sessions are measured using surveys and interviews at 14 weeks Parents as Partners 1. Group facilitator and parental views of Parents as Partners sessions are measured using surveys and interviews at 14 weeks
Overall study start date	05/05/2016
Overall study end date	01/11/2018

Eligibility

Participant type(s)	Other
Age group	Mixed
Sex	Both
Target number of participants	200
Participant inclusion criteria	Participants are children under five as well as stakeholders working on the identified interventions.
Participant exclusion criteria	Any other human participants who do not fit the inclusion criteria detailed above.
Recruitment start date	05/05/2017
Recruitment end date	01/06/2018

Locations

Countries of recruitment

England
United Kingdom

Study participating centres

University of Bedfordshire

Institute for Health Research
University Square
Luton
LU1 3JU
United Kingdom

Flying Start Luton

The TOKKO Building
7 Gordon Street
Luton
LU1 2QP
United Kingdom

Sponsor information

University of Bedfordshire
University/education

University Square
Luton
LU1 3JU
England
United Kingdom

Website	https://www.flyingstartluton.com

Flying Start Luton
Government

The TOKKO Building
7 Gordon Street
Luton
LU1 2QP
United Kingdom

Phone	+44 1582 548356
Email	flyingstart@luton.gov.uk
Website	https://www.flyingstartluton.com

University of Bedfordshire
Not defined

Website	http://www.beds.ac.uk/
"ROR"	https://ror.org/0400avk24

Funders

Funder type

University/education

University of Bedfordshire
Private sector organisation / Universities (academic only)

Location: United Kingdom

Flying Start Luton

No information available

Results and Publications

Intention to publish date	16/01/2018
Individual participant data (IPD) Intention to share	Yes
IPD sharing plan summary	Available on request
Publication and dissemination plan	Steps to Evaluation article submitted to BMC Medical Research Methodology awaiting response. Two outcome and four process evaluation articles are planned from the subsequent evaluation of the projects selected in the above protocol.
IPD sharing plan	The datasets generated during and/or analysed during the current study are/will be available upon request from Dr Rosemary Davidson (rosemary.davidson@beds.ac.uk).

Editorial Notes

07/06/2018: Prof Gurch Randhawa has been added as a scientific contact