Plain English Summary
Background and study aims
Late recognition of deteriorating patients (i.e their condition getting worse) in hospitals causes delays in treatment for these patients, which then results in an increase in mortality (ill health) and morbidity (death) despite the widespread introduction of vital sign-based “early warning scores”.
Therefore, developing systems that recognise earlier when a patient is at risk of a severe but reversible deterioration in health is a key goal for the NHS. The HAVEN Project aims to produce a hospital-wide IT system that enables a continuous risk assessment in all hospital patients, and predicts those at risk of deterioration.
Who can participate?
Data are collated from the records of all adult patients over the age of 18 who are admitted to
all participating hospitals. Data without identifying information are sent to the central office in
Oxford. Staff members from Oxford and Portsmouth trusts are also recruited to help design the
user interface.
What does the study involve?
An IT system is developed that routinely stores electronic data (including demographics, laboratory results and vital signs recordings) to create a continuous risk assessment. Data is gathered from different local databases; at present they are not integrated or displayed in a way that supports decision making or calculation of patient risk. Risk prediction algorithms are then developed (programs that look at the information and decide whether a patient is likely to be at risk of deteriorating) and tested. These use the records of patients who are admitted to hospital and then admitted to an intensive care unit (ICU) after two or more days in hospital. The information about these patients illustrates the pathway from the first signs of deterioration on the ward to ICU admission. There is no direct involvement for patients who are included in the project. Information from their hospital stay is collected by the NHS IT groups in a way that will allow the university-based research team to review the information without knowing who the people are. During the project, the automatic system is checked to see whether it has correctly recognised if patients needed intensive care treatment during their hospital stay. Doctors and nurses who are part of the research team need access to individual patient information to check this. The algorithms are used to create an interface allowing clinical staff to identify, rank, review and treat patients who, without acute medical intervention (treatment), will deteriorate and require ICU admission. Members of staff are asked questions about the information they would look for to recognise that a patient was becoming unwell. They are then observed as they carry out their usual daily tasks.
What are the possible benefits and risks of participating?
There are no risks to patients whose data is used in the project and the potential benefit is the development of a hospital-wide IT system that is capable of recognising patients at risk of deterioration. Clinical participants (healthcare staff) will need to find time to meet with the research team to discuss the project plans in some detail as well as participate in the cognitive task analysis and user interface testing.
Where is the study run from?
The project is run from the University of Oxford. It takes place at the University of
Portsmouth, Portsmouth Hospitals NHS Trust, and Oxford University Hospitals NHS Foundation
Trust. Other hospitals taking part provide data only.
When is the study starting and how long is it expected to run for?
August 2015 to March 2023
Who is funding the study?
Department of Health and the Wellcome Trust through the Health Innovation Challenge Fund
provided the initial project funding. The ongoing work is funded by the NIHR Oxford Biomedical
Research Centre.
Who is the main contact?
Rachel Henning
ccrg@ndcn.ox.ac.uk
Trial website
Contact information
Type
Public
Primary contact
Ms Rachel Henning
ORCID ID
Contact details
Kadoorie Centre for Critical Care Research and Education
Level 3
John Radcliffe Hospital
Headley Way
Headington
Oxford
OX3 9DU
United Kingdom
+44 (0)1865 231448
ccrg@ndcn.ox.ac.uk
Type
Scientific
Additional contact
Dr Peter Watkinson
ORCID ID
http://orcid.org/0000-0003-1023-3927
Contact details
Department of Clinical Neurosciences
University of Oxford
Oxford
OX3 9DU
United Kingdom
Additional identifiers
EudraCT number
ClinicalTrials.gov number
Protocol/serial number
160405-HAVEN-v1
Study information
Scientific title
Hospital Alerting Via Electronic Noticeboard (HAVEN)
Acronym
HAVEN
Study hypothesis
Production of a hospital-wide IT system that enables a continuous risk assessment in all hospital patients, optimised for end-user functionality.
Ethics approval
1. South Central Oxford C Research Ethics Committee, 20/06/2016, ref: 16/SC/0264
2. Confidentiality Advisory Group, 04/07/2016, ref: 16/CAG/0066
Study design
Observational mixed quantitative/qualitative study
Primary study design
Observational
Secondary study design
Mixed quantitative/qualitative methodology
Trial setting
Hospitals
Trial type
Other
Patient information sheet
No participant information sheet available
Condition
Patients at risk of deterioration (and requiring intensive care unit admission) in hospital
Intervention
Patients will be identified by the system as likely (or not) to require intensive care unit admission. The system generated results will be compared against reality and discrepancies reviewed.
Creation of the risk prediction score will be a multi-stage process involving:
1. Candidate Variable Selection
Candidate variables will be selected by a systematic review of variables associated with in-hospital deterioration; analysis of data of 5000 admissions to ICUs at the Royal Berkshire Hospital, the Churchill Hospital, Oxford and the John Radcliffe Hospital, Oxford to identify common factors to patients admitted from general wards; analysis of 127,000 patient episodes in Portsmouth Hospital to obtain information on patents who did not require ICU admission, and compare the frequency of candidate variables between those who deteriorated and those who remained stable; and a modified two-round
Delphi Process to review possible candidate variables identified in the above processes and to obtain further suggestions for variables of interest.
2. Score Creation
To create the risk prediction score, the project will utilise standard statistical approaches(including linear and non-linear regression), machine learning techniques and techniques such as Gaussian processes for trend analysis using trajectories in multi-dimensional input space. Algorithms will be developed in which correlations between the various time-series are learned automatically, allowing early detection of physiological deterioration by identifying unexpected changes in correlation or other changes in dynamics. The algorithms will include probabilistic, Bayesian methods that can cope with missing or artefactual data from one or more input time-series in a principled, robust
manner, as is required for a realistic real-time risk estimation system.
3. Score Validation
The risk prediction score will be validated by comparing observed ICU admissions with predicted ICU admissions. The goodness-of-fit of observed to expected ICU admissions will be described using the Hosmer-Lemeshow (HL) test.
4. Clinical Expert Review
The first risk prediction model will be run on new inpatient episodes. A list of false negatives (patients in whom the algorithm reported a low risk, but the patient was admitted to ICU) and false positives (patients in whom the algorithm reported a high risk, but the patient was not admitted to ICU) will be generated. Clinicials from the project team to examine these patients’ standard clinical records (EPR, paper notes, etc) to determine new factors (or combinations of factors) used by the clinical team to
determine the patients’ need for ICU.
5. Score Re-validation
The score will be re-validated following the modifications made to the risk algorithm after step iv.
Intervention type
Other
Phase
Drug names
Primary outcome measure
1. Validated risk prediction score identifying at least 50% of deteriorating patients admitted to the ICU at least 12 hrs ahead of their admission, with a false positive rate of under 60%. This will be validated by comparing observed ICU admissions with predicted ICU admissions. The goodness-of-fit of observed to expected ICU admissions will be described using the Hosmer-Lemeshow (HL) test. This outcome will be measured during the second half of 2017
2. Data aggregation software in place capable of supplying all data fields to the algorithm
3. Working interface complete with functional underlying software and prototype graphics. A minimum success criterion would be usability scale results above average (>68%). This will be designed using Human Factors methods. This will involve process mapping and task analysis of existing workflows used to recognise deterioration of patients on the ward. Knowledge gained will be used to produce a design specification for the interface. This will then be tested formally assessing efficiency, effectiveness and user satisfaction as outlined in ISO 9421
The end of the study will be a key timepoint for overall evaluation of the outcomes from the
project.
Secondary outcome measures
No secondary outcome measures
Overall trial start date
01/08/2015
Overall trial end date
31/03/2023
Reason abandoned (if study stopped)
Eligibility
Participant inclusion criteria
1. Aged 16 years and over
1. Admitted to any hospital from Portsmouth Hospitals NHS Trust since 1st January 2010 or Oxford University Hospitals NHS Foundation Trust since 1st January 2014
Participant type
Patient
Age group
Adult
Gender
Both
Target number of participants
450-500,000 patients
Participant exclusion criteria
1. Patients <16 years of age
2. Patients whose data are not entered into local electronic patient records
Recruitment start date
01/08/2015
Recruitment end date
31/07/2018
Locations
Countries of recruitment
United Kingdom
Trial participating centre
Oxford University Hospitals NHS Foundation Trust
Oxford
OX3 9DU
United Kingdom
Trial participating centre
Portsmouth Hospitals NHS Trust
Portsmouth
PO6 3LY
United Kingdom
Trial participating centre
Royal Preston Hospital
Sharoe Green Lane
Fulwood
Preston
PR2 9HT
United Kingdom
Trial participating centre
Warwick Hospital
Lakin Road
Warwick
CV34 5BW
United Kingdom
Funders
Funder type
Government
Funder name
Department of Health/Wellcome Trust Health Innovation Challenge Fund
Alternative name(s)
Funding Body Type
Funding Body Subtype
Location
Funder name
NIHR Oxford Biomedical Research Centre
Alternative name(s)
Funding Body Type
Funding Body Subtype
Location
Results and Publications
Publication and dissemination plan
Current publication and dissemination plan as of 26/06/2020:
Publications are expected throughout the project with a final report within 12 months of the end of the project.
A full list of project-related publications is available from the project website: https://www.ndcn.ox.ac.uk/research/critical-care-research-group-kadoorie-centre/research-studies/
hospital-alerting-via-electronic-noticeboard-haven/academic-publications-from-the-haven-project
Previous publication and dissemination plan:
1. A protocol for a systematic review to determine the variables with a statistical association with unplanned ICU admission will be published in late 2016. This will be followed by the publication of the systematic review itself
2. A protocol for a systematic review examining usability evaluation methods used to assess information visualisations of patient data will be published in late 2016. This will be followed by the publication of the systematic review itself
3. There will be a final report within 12 months of the end of the project
4. Further interim papers are likely and will be confirmed at a later date
IPD sharing statement
The datasets generated during and/or analysed during the current study are/will be available upon request. Currently researchers need to apply to Peter Watkinson (HAVEN Chief Investigator) to access the de-identified data. Applicants will need to outline their hypothesis and analysis plan. Valid submissions will be reviewed by a ‘Data Access Committee’ who will grant access (or not). At the end of the HAVEN project the trialists will provide a full access statement.
Intention to publish date
31/03/2024
Participant level data
Available on request
Basic results (scientific)
Publication list
2019 protocol in https://pubmed.ncbi.nlm.nih.gov/31511294/ (added 31/07/2020)
2020 development of mathemetical model in https://pubmed.ncbi.nlm.nih.gov/32677936/ (added 31/07/2020)