Improving feedback: how expected treatment response information helps therapists
| ISRCTN | ISRCTN11638237 |
|---|---|
| DOI | https://doi.org/10.1186/ISRCTN11638237 |
- Submission date
- 14/12/2023
- Registration date
- 24/04/2024
- Last edited
- 18/06/2024
- Recruitment status
- No longer recruiting
- Overall study status
- Completed
- Condition category
- Mental and Behavioural Disorders
Plain English Summary
Background and study aims
Many mental health care organizations measure their patients' progress through routine outcome monitoring. The way that progress is measured and the way therapists are provided with feedback on their patients' progress strongly differs between organizations. The question is whether all methods are equally effective. Research in the United States shows that providing feedback to therapists based on a prediction model can improve patient outcomes, especially for those patients who are not progressing well in treatment. In the Netherlands, a prediction model for Dutch patients was built, based on almost 2000 patients in four mental health care organizations.
Who can participate?
Patients (aged 18-65 years old) who are receiving outpatient treatment at GGZ Noord-Holland-Noord and GGZ Dimence in the four participating locations
What does the study involve?
In this study, two types of feedback and one control condition are compared:
* Control group: the therapist gets no feedback.
* Outcome monitoring feedback: the therapist gets feedback on the patient's progress in a progress chart.
* Complex feedback based on the Dutch prediction model: the therapist gets a progress chart that compares the patient's actual progress with the expected treatment response.
Intervention:
The intervention consists of providing feedback to therapists on their patients' progress. In the outcome monitoring feedback condition, the therapist gets feedback on the patient's progress in progress charts and tables. The progress of the patient can be viewed by the therapist at all times, by logging on to the feedback system (RequestXL), but is also actively provided by e-mail at sessions 1, 3, 5, 10 and 15.
In the complex feedback condition with a prediction model, the actual treatment course (based on the OQ scores) of the patient is compared with the predicted treatment course. The expected treatment course is calculated by a formula. The progress of the patient can be viewed by the therapist at all times, by logging on to the feedback system (RequestXL), but is also actively provided by e-mail when the patient is not progressing well. The therapist then receives an e-mail with high urgency.
In the complex feedback condition, feedback is also provided on the ASC. The ASC is administered when the patient goes off track (through the 75% negative bound of the confidence interval around the predicted treatment course) and measures the therapeutic alliance, motivation, social support and life events. The ASC is combined with so-called Clinical Support Tools, a set of Microsoft Word documents that provide practical tips on improving therapeutic alliance, motivation and social support. The practical tips are based on a literature review on these topics.
Main study parameters/endpoints:
The primary outcome measure of the study is patients' dysfunctioning on the Outcome Questionnaire. More specifically, the progress that the patient makes during treatment. The results will be analysed using multilevel analysis, which has the advantage of being able to handle missing data well. Therefore, data from all patients, including those that dropped out of treatment or dropped out of the study can be used in the analysis.
What are the possible benefits and risks of participating?
Benefits: Participants in the active conditions may benefit from participating by having their therapists respond to the feedback about treatment progress and being able to identify more quickly when there is a lack of progress. In addition, completing the questionnaires in all conditions may help patients summarize their thoughts before sessions, which has been reported as helpful by previous patients.
Burden: The burden for the patient consists of completing a 5-minute questionnaire before each treatment session, for a maximum of 15 times. After the research period (end of treatment or after 15 sessions) they will get two follow-up measures after 3 and 6 months. The average treatment duration for outpatient individual treatment was around 9 sessions in a previous study at one of the treatment settings. If patients go off track, they are asked to complete an additional questionnaire, which also takes about 5 minutes to complete. In addition to the self-report questionnaires, patients are getting a diagnostic interview that takes approximately 1 hour. The burden for the therapist consists of completing brief questionnaires during the treatment of a participating patient (total commitment around 10 minutes) and completing a longer questionnaire once every 6 months (about 20 minutes per administration, total of 5 administrations).
Risks: The risks for both the patient and the therapist are minimal.
Where is the study run from?
Leiden University (The Netherlands)
When is the study starting and how long is it expected to run for?
April 2009 to September 2014
Who is funding the study?
Investigator initiated and funded (The Netherlands)
Who is the main contact?
Dr Kim de Jong, kjong@fsw.leidenuniv.nl (The Netherlands)
Contact information
Public, Scientific, Principal Investigator
Wassenaarseweg 52
Leiden
2333 AK
Netherlands
| Phone | +31 71 525 7995 |
|---|---|
| kjong@fsw.leidenuniv.nl |
Study information
| Study design | Two-year randomized controlled single-blinded study |
|---|---|
| Primary study design | Interventional |
| Secondary study design | Randomised controlled trial |
| Study setting(s) | Hospital, Other therapist office |
| Study type | Treatment |
| Participant information sheet | Not available in web format, please use contact details to request a participant information sheet. |
| Scientific title | Taking feedback to the next level: Efficacy of expected treatment response feedback to therapists |
| Study hypothesis | Main questions: 1. Does providing feedback to therapists improve treatment outcome? 2. Does providing feedback based on the prediction model lead to better outcomes than progress feedback alone? Secondary questions: 1. How well can therapists predict their patient's progress? 2. Are there differences between therapists in treatment outcomes and is this related to the way they use the feedback? |
| Ethics approval(s) |
Approved 02/11/2010, Medical Ethical Committee of the Leiden University Medical Centre (Postbus 9600, Leiden, 2333 ZC, Netherlands; +31 (0)71 526 91 11; metc_ldd@lumc.nl), ref: NL30987.058.09 |
| Condition | Psychiatric patients |
| Intervention | The psychiatric patients who applied for outpatient individual psychiatric treatment were randomly assigned, with a simple randomization method using a software program to assign patients to one of the three conditions: The control condition, the outcome monitoring (ROM) condition, or the clinical support tool (CST) condition. The intervention consists of providing feedback to therapists on their patients' progress. In the control condition, therapists receive no feedback about their patient’s progress. In the ROM feedback condition, feedback was given in the form that most routine outcome monitoring feedback is being provided. The therapist was presented with a graph of the patient’s progress after sessions 1, 3, 5, 10 and 15. In the graph, the patient’s progress on the OQ-45 is monitored. The therapist receives feedback on the OQ-45 total score, the subscale scores and the critical items. In the CST feedback condition, the therapists receive feedback on the patient’s progress, based on an expected treatment response model. The ETR model is based on data collected from 1540 outpatients in three different mental health care organizations in the Netherlands. Therapists receive an alert when the patient deviates from the expected track. An error bound around the expected treatment response for the patient indicates when the therapist is signalled. If a 75% failure boundary was crossed by the patient, the therapist received an orange warning signal, indicating that the patient had an increased chance of a negative treatment outcome. If the 95% failure boundary was crossed, the therapist received a red warning signal, indicating that the patient was highly likely to have a poor treatment outcome if he or she continued on the current track. When the patient signals orange or red, the clinical support tool, the Assessment for Signal Clients (ASC) questionnaire was administered to the patient, and the therapist received feedback about the scores on the ASC, in addition to the scores on the Outcome Questionnaire (OQ-45). |
| Intervention type | Behavioural |
| Primary outcome measure | Patient’s progress measured using the Outcome Questionnaire-45 item version (OQ-45), Dutch version, measured weekly for up to 15 sessions or a maximum of 1 year |
| Secondary outcome measures | 1. Therapist expectations of treatment outcome measured using a self-constructed instrument at sessions 1, 5, 10 and 15 2. Therapists' use of the progress feedback tool measured using the Assessment for Signal Clients (ASC) questionnaire at sessions 5, 10, 15 3. Patients' treatment dropout, defined as the patient ending treatment before the therapist's recommendation, measured using data registered in the electronic patient file as "treatment ended on the initiative of the patient" by the end of the study 4. Patients' study dropout, defined as patients discontinuing participation in the study, measured using data registered in the trial's participants follow system by the research team before ending treatment or reaching 15 sessions of treatment |
| Overall study start date | 01/04/2009 |
| Overall study end date | 01/09/2014 |
Eligibility
| Participant type(s) | Patient |
|---|---|
| Age group | Adult |
| Lower age limit | 18 Years |
| Upper age limit | 65 Years |
| Sex | Both |
| Target number of participants | 600 |
| Total final enrolment | 511 |
| Participant inclusion criteria | 1. Age between 18 to 65 years old 2. Sufficient mastery of the Dutch language in reading and speaking |
| Participant exclusion criteria | 1. Psychotic disorder 2. Current crisis 3. High risk of decompensation 4. Severe manic episode |
| Recruitment start date | 01/12/2010 |
| Recruitment end date | 01/09/2013 |
Locations
Countries of recruitment
- Netherlands
Study participating centres
Heiloo
1850 BA
Netherlands
Deventer
7411 GT
Netherlands
Sponsor information
Hospital/treatment centre
Postbus 18
Heiloo
1850 BA
Netherlands
| Phone | +31 0886565010 |
|---|---|
| onderzoek@ggz-nhn.nl | |
| Website | https://www.ggz-nhn.nl/ |
"ROR" |
https://ror.org/00b3xjw51 |
Funders
Funder type
Other
No information available
Results and Publications
| Intention to publish date | 01/06/2024 |
|---|---|
| Individual participant data (IPD) Intention to share | No |
| IPD sharing plan summary | Not expected to be made available |
| Publication and dissemination plan | There is a plan for 3 papers out of this study. The manuscripts will be submitted in high impact peer-reviewed journals. Paper 1 Taking feedback to the next level: Efficacy of expected treatment response feedback therapists Research questions: • Does providing therapists with feedback about their patient's progress improve patient outcomes • Exploratory: Do patients’ complexity moderate the feedback-outcome relationship? Paper 2 Therapists’ expectations of treatment outcomes and how they change over time Research questions: • Do therapists’ expectations change over time and are there differences in trajectories based on the feedback condition? • Is the effect of feedback on changes in therapist expectations moderated by within-therapist variability in expectations? • Is the effect of feedback on changes in therapists’ expectations predicted by a discrepancy in expectations and patient progress? Paper 3 Predictors of therapists’ accuracy in predicting treatment outcomes Research questions: • What therapist and client characteristics predict therapists’ accuracy in predicting patients’ treatment outcomes? o Therapist characteristics: Sex, age, years of experience, profession, self-efficacy, self-efficacy in a treatment context, internal and external feedback propensity, attitude towards feedback, locus of control, big five personality domains o Client characteristics: complexity o Treatment characteristics: Tx length, NOT cases • Is this moderated by treatment condition? Additional papers might be written based on secundary analyses. |
| IPD sharing plan | The datasets generated during and/or analysed during the current study are not expected to be made available due to the participating mental health care organizations owning this data, and we just have permission for its use with the specific research questions that we have. Data collected is highly sensitive data (on patient functioning and symptoms) that cannot be shared as individual data. |
Study outputs
| Output type | Details | Date created | Date added | Peer reviewed? | Patient-facing? |
|---|---|---|---|---|---|
| Protocol file | version 3 | 29/09/2010 | 23/04/2024 | No | No |
Additional files
Editorial Notes
18/06/2024: Sponsor information was updated.
23/04/2024: Study's existence confirmed by the Medical Ethical Committee of the Leiden University Medical Centre.
