Improving the implementation of a bullying prevention program in primary schools with tailored mentorship

ISRCTN	ISRCTN15558617
DOI	https://doi.org/10.1186/ISRCTN15558617
ClinicalTrials.gov (NCT)	Nil known
Clinical Trials Information System (CTIS)	Nil known
Protocol serial number	02-2022
Sponsor	University of Turku
Funder	INVEST Research Flagship, funded under the flagship scheme of the Academy of Finland (decision number: 320162)

Submission date: 22/02/2022
Registration date: 25/02/2022
Last edited: 18/12/2023

Recruitment status: No longer recruiting
Overall study status: Completed
Condition category: Other

Prospectively registered

Protocol

Statistical analysis plan

Results

Individual participant data

Record updated in last year

Plain English summary of protocol

Background and study aims
Bullying is common worldwide among primary school students, with negative short and long-term consequences for those involved. To overcome the problem, several bullying prevention programs have been developed. To ensure a safe learning environment for all students, in 2006, the Finnish Ministry of Education funded the University of Turku research team to develop a bullying prevention program for basic education. The KiVa antibullying program includes several program components and was evaluated in a randomized controlled trial during 2007–2009. The roll-out of the program took place in basic education (grades 1–9, children aged 7–15) nationwide in Finland from 2009 onward. Prior evaluation studies have proven the program to be effective in reducing bullying (Kärnä et al., 2011) and also cost-effective (Persson et al., 2018). However, program efficacy alone does not ensure program fidelity and schools typically do not deliver all of the program components over time. Thus, low program fidelity of bullying prevention programs is a serious concern given the effort and investment placed in the development, initial evaluation, and scaling up of such programs. It is quite evident that even the most effective interventions will fail if they are not implemented properly. The IMPRES implementation support model has been developed to improve the program fidelity of bullying prevention programs. The primary goal of this study is to assess to which extent such implementation support can improve program fidelity and consequently also decrease the prevalence of bullying victimization and perpetration. We also aim 1) to assess how the mentorship may have influenced the impact of the KiVa program on student outcomes and 2) to assess variables that may help us understand the magnitude as well as the mechanisms supporting the impact of the mentorship program on implementation success. Finally, we will conduct a cost-effectiveness evaluation of the mentorship program.

Who can participate?
Finnish basic education schools including grade levels 1–6 (children aged from 7 to 12 years of age) that have at least 100 students and are registered as KiVa antibullying program users during the school year 2021-2022 and 2022-2023

What does the study involve?
Participating schools are allocated to one of two groups. Those in the intervention group are offered implementation support that consists of three components which are 1) Assessing the current state of the KiVa program implementation fidelity and identification of individual and organizational level facilitators and barriers to implementation, 2) Training (3 hours) on implementation fidelity for all staff members, and 3) Four mentoring sessions for 4-6 selected staff members (topics: identifying gaps in implementation fidelity as well as in individual and organizational level facilitators and barriers, creating an action plan, executing the plan, and planning implementation ahead). Those in the control group will continue delivering the KiVa antibullying program as usual. At the end of the trial, they will receive a feedback report based on the data collected and consultation of 1-2 hours.
At pre-, post-intervention and 1-year follow-up assessments, teachers and students will answer a comprehensive questionnaire aiming to better understand the impact of the KiVa IMPRES mentorship and the potential mechanisms of change involved. Our primary objective is to document the impact of the KiVa IMPRES mentorship on the fidelity of the universal and indicated actions. Our second objective is to evaluate how students’ perceptions of the quality of KiVa lesson delivery and of their teachers’ antibullying attitudes and behaviors may have changed following the KiVa IMPRES mentorship. We also aim to assess whether children in the KiVa IMPRES condition report lower levels of bullying victimization and perpetration. Also, we aim to identify the potential mechanisms of change involved in the delivering of the KiVa IMPRES mentorship. Therefore, we will measure a set of distal outcomes that may be affected by the intervention. More specifically, teachers’ self-efficacy, moral disengagement and attitudes towards bullying among students will be measured. An Implementation Capacity Scale evaluating teacher and school level facilitators and barriers of bullying prevention programs' implementation fidelity will be validated during this trial. In addition, we will investigate the cost-effectiveness and the administrative sustainability of the KiVa IMPRES implementation support offered to schools compared to no support. Lastly, focus group interview data will be collected to explore staff members’ experiences on the implementation support process.

What are the possible benefits and risks of participating?
Participating schools may benefit from the additional training and mentoring potentially improving their bullying prevention practices and the implementation of them. Individual teachers’ skills, knowledge and motivation to prevent bullying and intervene may be strengthened. In addition, children may also benefit from improved bullying prevention efforts. There are no notable risks with participating in this trial. Nonetheless, teachers who follow the IMPRES mentorship may feel moderately overwhelmed or stressed as they will be asked to adapt their daily practices. Also, children may encounter discomfort when they will be asked to answer questions about their bullying experiences and their teachers’ attitude towards bullying. However, careful attention has been taken to formulate questions that may not trigger overwhelmingly negative emotions or distress for young children.

Where is the study run from?
This study is run from the University of Turku (Finland) and takes place in 24 public schools in different parts of Finland

When is the study starting and how long is it expected to run for?
January 2021 to May 2024

Who is funding the study?
This study is funded by INVEST Research Flagship, funded under the flagship scheme of the Academy of Finland (decision number: 320162)

Who is the main contact?
Senior Researcher Fellow Sanna Herkama (sanna.herkama@utu.fi)

Contact information

Dr Sanna Herkama
Principal investigator

Assistentinkatu 5
Turku
20014
Finland

ORCID ID	0000-0002-0807-6961
Phone	+358 (0)405621586
Email	sanna.herkama@utu.fi

Study information

Primary study design	Interventional
Study design	Interventional cluster randomized controlled trial
Secondary study design	Cluster randomised trial
Study type	Participant information sheet
Scientific title	Improving the implementation fidelity of a school-based KiVa antibullying program in Finnish primary schools with tailored mentorship: a cluster randomized controlled trial
Study acronym	IMPRES
Study objectives	Current hypothesis as of 30/05/2023: The main objective of the IMPRES trial is to improve the program fidelity of the school-based KiVa antibullying program in Finnish primary schools. Program fidelity refers to the extent to which intervention follows the given program guidelines. As such, it is a multi-dimensional outcome which requires several indicators. Also, KiVa antibullying program is a multi-component program which involves universal and indicated actions. Therefore, there are four main study hypotheses to fully evaluate program fidelity. Universal component: H1. Schools that receive the IMPRES mentorship will deliver a higher number of KiVa lessons than schools that do not receive the mentorship. H2. Schools that receive the IMPRES mentorship will deliver KiVa lessons of higher quality than schools that do not receive the mentorship. Indicated component: H3. Schools that receive the IMPRES mentorship will follow the KiVa recommended methods in addressing bullying cases more often than schools that do not receive the mentorship. H4. Schools that receive the IMPRES mentorship will follow more consistently the recommended procedure in handling bullying cases than schools that do not receive the mentorship. _____ Previous hypothesis: The main objective of the IMPRES trial is to improve the program fidelity of the school-based KiVa antibullying program in Finnish primary schools. Program fidelity refers to the extent to which intervention follows the given program guidelines. As such, it is a multi-dimensional outcome which requires several indicators. Also, KiVa antibullying program is a multi-component program which involves universal and indicated actions. Therefore, there are four main study hypotheses to fully evaluate program fidelity. Universal component: H1. Schools that receive the IMPRES mentorship will deliver a higher number of KiVa lessons than schools that do not receive the mentorship. H2. Schools that receive the IMPRES mentorship will deliver KiVa lessons of higher quality than schools that do not receive the mentorship. Indicated component: H3. Schools that receive the IMPRES mentorship will follow the KiVa recommended guidelines in addressing bullying cases more often than schools that do not receive the mentorship. H4. Schools that receive the IMPRES mentorship will organize follow-up discussions more consistently after addressing bullying cases than schools that do not receive the mentorship.
Ethics approval(s)	1. Approved 01/10/2021, Ethics Committee for Human Sciences at the University of Turku, Humanities and Social Sciences Division (FI-20014 Turun yliopisto, Finland; +358 (0)29 450 5000, +358 (0)50 303 0346; eettinen@utu.fi), ref: not applicable 2. Focus group interviews conducted as part of the process evaluation approved 15/02/2023, Ethics Committee for Human Sciences at the University of Turku, Humanities and Social Sciences Division (FI-20014 Turun yliopisto, Finland; +358 (0)29 450 5000, +358 (0)50 303 0346; eettinen@utu.fi), ref: not applicable
Health condition(s) or problem(s) studied	Bullying prevention
Intervention	The schools will be randomized in a 1:1 ratio to either the intervention group (KiVa IMPRES) or the control group (KiVa regular). The KiVa antibullying program is based on a whole-school approach and includes two main components which are universal actions to prevent bullying and indicated actions to stop ongoing bullying (for more about the program see Salmivalli, Kärnä & Poskiparta, 2010). The universal actions for primary schools consist of lessons delivered to students, including KiVa lessons targeted to Grades 1 and 4. Also, awareness of antibullying attitudes and practices in school is raised with KiVa symbols (posters and recess supervisors’ vests). To support program implementation, the school staff is provided with materials for a kick-off event for students and for a staff meeting. Also, KiVa provides infographics and newsletters for introducing KiVa to parents. To address acute bullying cases, a KiVa team is formed from school staff and guidelines to handle bullying cases are provided. Finally, KiVa schools monitor their progress in bullying prevention and bullying prevalence with feedback based on KiVa annual student and staff online surveys. Finally, to ensure high-quality program delivery, schools are provided with quality recommendations (e.g., planning, informing). KiVa regular condition (control group): The schools in the control group will implement the KiVa antibullying program as usual. At the end of the trial, these schools will receive a feedback report based on the data collected and consultation of 1-2 hours with an experienced KiVa trainer. KiVa IMPRES condition (intervention group): KiVa IMPRES schools are provided with implementation support over one academic school year (2022-2023). These schools will receive implementation support via mentorship sessions. The mentorship support unfolds in three components: 1. Assessment: The KiVa trainer team responsible for providing the mentoring will assess the current state of the KiVa program implementation fidelity and identify possible individual and organizational level facilitators and barriers to implementation. 2. Training: All school staff members will receive three hours of training on effective bullying prevention, the importance of implementation fidelity, and information on the current state of implementation based on the assessment provided by a pair of licenced KiVa trainers (Aug/Sep). 3. Four mentoring sessions (two hours/session; Sep, Oct, Jan, Apr) for 4-6 selected staff members including the headmaster. The topics targeted are: identifying gaps in implementation fidelity as well as in individual and organizational level facilitators and barriers, creating an action plan, executing the plan, and planning implementation ahead.
Intervention type	Behavioural
Primary outcome measure(s)	Current primary outcome measures as of 30/05/2023: Universal component: 1. Number of KiVa lessons: at pre-, post-intervention and at 1-year follow-up, teachers will report whether they delivered or not each of the 10 universal KiVa lessons. 2. Quality of KiVa lessons: at pre-, post-intervention and at 1-year follow-up, teachers will answer five items measuring the quality of the KiVa lesson implementation (e.g., student engagement, lesson suitability, investment in the lessons). Indicated component: 3. Following recommended intervention guidelines: at pre-, post-intervention and at 1-year follow-up, KiVa team members will report which method they used to try to make the bullying situation stop (Johander et al., 2020) 4. At pre-intervention, organisation of follow-up discussion: KiVa team members will answer whether they organized follow-up discussions, as recommended by the KiVa program, after the first discussion with the students involved in bullying cases (Johander et al., 2020). In addition, at post-intervention and 1-year follow-up: KiVa team members will answer nine questions on how consistently (1 = never, 2 = almost never, 3 = often, 4 = almost always, 5 = always) they have followed the recommended procedure for handling bullying cases. _____ Previous primary outcome measures: Universal component: 1. Number of KiVa lessons: at pre-, post-intervention and at 1-year follow-up, teachers will report whether they delivered or not each of the 10 universal KiVa lessons. 2. Quality of KiVa lessons: at pre-, post-intervention and at 1-year follow-up, teachers will answer five items measuring the quality of the KiVa lesson implementation (e.g., student engagement, lesson suitability, investment in the lessons). Indicated component: 3. Following recommended intervention guidelines: at pre-, post-intervention and at 1-year follow-up, KiVa team members will report which method they used to try to make the bullying situation stop (Johander et al., 2020) 4. Organisation of follow-up discussion: at pre-, post-intervention and at 1-year follow-up, KiVa team members will answer whether they organized follow-up discussions, as recommended by the KiVa program, after the first discussion with the students involved in bullying cases (Johander et al., 2020)
Key secondary outcome measure(s)	Current secondary outcome measures as of 30/05/2023: Students: 1. Perception of teacher’s antibullying attitudes and behaviors will be measured at pre-, post-intervention and at 1-year follow-up by using 2 items (students aged 7–9) or 9 items (students aged 10–12) developed for the purposes of this study 2. Quality of the KiVa lesson delivered by the teacher will be measured at pre-, post-intervention and at 1-year follow-up by 2 items (students aged 7–9) or 5 items (students aged 10–12) developed for the purposes of this study 3. Bullying victimization will be measured at pre-, post-intervention and at 1-year follow-up by using 1 item (students aged 7–9) or 6 items (students aged 10–12) from the revised Olweus Bully/Victim Questionnaire (OBVQ) (Salmivalli et al. 2011) 4. Bullying perpetration will be measured at pre-, post-intervention and at 1-year follow-up by using 1 item (students aged 7– 9) or 6 items (students aged 10–12) from the revised Olweus Bully/Victim Questionnaire (OBVQ) (Salmivalli et al. 2011) Tertiary outcome measures: Teachers: 1. Self-efficacy to intervene in bullying will be assessed at pre-, post-intervention and at 1-year follow-up with a 9-item questionnaire developed for another bullying prevention project (Tolmatcheff et al., in preparation) 2. Moral disengagement will be assessed at pre-, post-intervention and at 1-year follow-up with a 9-item questionnaire; the questionnaire has been developed for the purpose of the study. 3. Perception of bullying as a malleable problem will be assessed at pre-, post-intervention and at 1-year follow-up with an 8-item questionnaire which aims to evaluate teacher’s perception of agency on bullying behaviors among students (Ahtola et al. 2012) 4. Focus group interviews on the mentorship process: These interviews will be conducted in April as part of the last mentoring session. Organisational features: Implementation capacity measure: In parallel of this trial, we aim to develop and test a questionnaire assessing the implementation capacity for a school-based bullying prevention program. Administrative sustainability: 1. Monitoring: The school personnel will assess at pre-, post-intervention and at 1-year follow-up how much their school use and share the results from the KiVa annual survey with their own staff, the parents, and the students from the school using a 3-item questionnaire answered by the entire staff and two additional questions are targeted to the KiVa team members. 2. Planning: The school personnel will answer at pre-, post-intervention and at 1-year follow-up a questionnaire assessing the level of planification for efficient program delivery using a 4-item questionnaire answered by the entire staff and five additional questions are targeted to the KiVa team members. 3. Informing: The school personnel will assess at pre-, post-intervention and at 1-year follow-up the extent to which information about bullying prevention in their schools is shared with the various school actors involved in the KiVA program using a 2-item questionnaire answered by the entire staff and one additional question is targeted to the KiVa team members. Economic sustainability: Cost-effectiveness evaluation: We will conduct two different analyses based on two different outcome measures, student-reported victimization experiences and bullying perpetuation experiences from the revised Olweus Bully/Victim Questionnaire (OBVQ) at pre-, post-intervention and at 1-year follow-up _____ Previous secondary outcome measures: Students: 1. Perception of teacher’s antibullying attitudes and behaviors will be measured at pre-, post-intervention and at 1-year follow-up by using 2 items (students aged 7–9) or 9 items (students aged 10–12) developed for the purposes of this study 2. Quality of the KiVa lesson delivered by the teacher will be measured at pre-, post-intervention and at 1-year follow-up by 2 items (students aged 7–9) or 5 items (students aged 10–12) developed for the purposes of this study 3. Bullying victimization will be measured at pre-, post-intervention and at 1-year follow-up by using 1 item (students aged 7–9) or 6 items (students aged 10–12) from the revised Olweus Bully/Victim Questionnaire (OBVQ) (Salmivalli et al. 2011) 4. Bullying perpetration will be measured at pre-, post-intervention and at 1-year follow-up by using 1 item (students aged 7– 9) or 6 items (students aged 10–12) from the revised Olweus Bully/Victim Questionnaire (OBVQ) (Salmivalli et al. 2011) Tertiary outcome measures: Teachers: 1. Self-efficacy to intervene in bullying will be assessed at pre-, post-intervention and at 1-year follow-up with a 9-item questionnaire developed for another bullying prevention project (Tolmatcheff et al., in preparation) 2. Moral disengagement will be assessed at pre-, post-intervention and at 1-year follow-up with a 6-item questionnaire; the questionnaire is an adapted version of the students’ moral disengagement in bullying situation scale from Thornberg and Jungert (2013) where the questions have been modified to evaluate teachers’ moral disengagement regarding bullying cases among students. 3. Perception of bullying as a malleable problem will be assessed at pre-, post-intervention and at 1-year follow-up with an 8-item questionnaire which aims to evaluate teacher’s perception of agency on bullying behaviors among students (Ahtola et al. 2012) 4. Focus group interviews on the mentorship process: These interviews will be conducted in April as part of the last mentoring session. Organisational features: Implementation capacity measure: In parallel of this trial, we aim to develop and test a questionnaire assessing the implementation capacity for a school-based bullying prevention program. Administrative sustainability: 1. Monitoring: The school personnel will assess at pre-, post-intervention and at 1-year follow-up how much their school use and share the results from the KiVa annual survey with their own staff, the parents, and the students from the school using a 3-item questionnaire answered by the entire staff and two additional questions are targeted to the KiVa team members. 2. Planning: The school personnel will answer at pre-, post-intervention and at 1-year follow-up a questionnaire assessing the level of planification for efficient program delivery using a 4-item questionnaire answered by the entire staff and five additional questions are targeted to the KiVa team members. 3. Informing: The school personnel will assess at pre-, post-intervention and at 1-year follow-up the extent to which information about bullying prevention in their schools is shared with the various school actors involved in the KiVA program using a 2-item questionnaire answered by the entire staff and one additional question is targeted to the KiVa team members. Economic sustainability: Cost-effectiveness evaluation: We will conduct two different analyses based on two different outcome measures, student-reported victimization experiences and bullying perpetuation experiences from the revised Olweus Bully/Victim Questionnaire (OBVQ) at pre-, post-intervention and at 1-year follow-up
Completion date	31/05/2024

Eligibility

Participant type(s)	Employee, Learner/student, Other
Age group	Mixed
Lower age limit	7 Years
Upper age limit	12 Years
Sex	All
Target sample size at registration	5280
Key inclusion criteria	Finnish schools offering basic education for grades 1–6 (aged 7–12 years), having at least 100 students and being registered as KiVa antibullying program users in the school year 2021–2022 and 2022–2023
Key exclusion criteria	1. Swedish-speaking primary schools 2. Schools which are taking part in a municipal initiative to retrained their school staff to the KiVa program for the 2022-2024 period.
Date of first enrolment	01/03/2022
Date of final enrolment	31/05/2024

Locations

Countries of recruitment

Finland

Study participating centre

University of Turku

Assistentinkatu 5
Turku
20014
Finland

Results and Publications

Individual participant data (IPD) Intention to share	No
IPD sharing plan summary	Data sharing statement to be made available at a later date
IPD sharing plan	The data-sharing plans for the current study are unknown and will be made available at a later date

Study outputs

Output type	Details	Date created	Date added	Peer reviewed?	Patient-facing?
Protocol article		15/12/2023	18/12/2023	Yes	No
Participant information sheet	Participant information sheet	11/11/2025	11/11/2025	No	Yes
Study website	Study website	11/11/2025	11/11/2025	No	Yes

Editorial Notes

18/12/2023: Publication reference added.
30/05/2023: The following changes have been made:
1. The study hypothesis has been changed.
2. The ethics approval has been updated to include the approval for the focus group interviews.
3. The primary outcome measures have been changed.
4. The secondary outcome measures have been changed.
5. The recruitment end date has been changed from 30/04/2022 to 31/05/2024.
23/02/2022: Trial's existence confirmed by the Ethics Committee for Human Sciences at the University of Turku, Humanities and Social Sciences Division.