Diagnosis of axillary lymph node status in primary breast cancer without a surgical procedure - a human factors validation test
| ISRCTN | ISRCTN99301435 |
|---|---|
| DOI | https://doi.org/10.1186/ISRCTN99301435 |
- Submission date
- 10/11/2024
- Registration date
- 15/11/2024
- Last edited
- 16/12/2024
- Recruitment status
- No longer recruiting
- Overall study status
- Completed
- Condition category
- Cancer
Plain English summary of protocol
Background and study aims
NILS (Non-invasive Lymph node Staging) is a web-based software that uses artificial intelligence to predict sentinel lymph node status in women who are planned for primary surgery due to breast cancer and are clinically axillary lymph node-negative. NILS is not yet CE-marked and is not available on the market. Previously, formative usability engineering evaluation has been performed with NILS. This study is a summative evaluation study in preparation of market access. The main aim is to present evidence that the NILS interface is considered safe. The study will be conducted according to the standard IEC 62366-1:2015. For the purpose of this pre-market study, a study under simulated conditions will be performed.
Who can participate
This study is for licensed medical physicians specialized in general surgery or oncology (including medical and radiation oncologists).
What does the study involve?
During the simulated-use testing, the test participant will be given the opportunity to use the web-calculator independently and as naturally as possible in their own work environment, without unnecessary interference from the test leader or observer. The test participants will be asked to perform five scenarios, with five different simulated patient cases. After calculating each case using NILS, the test participants will fill in the appropriate clinical pathway based on the NILS result and other available information. The participants’ interaction with the interface will be controlled by a test leader and an observer who will follow a predefined protocol. All tasks have been identified (risk management procedure according to ISO 14971) and critical tasks specified.
The study will take place in three different health care regions and 4 different hospitals in Sweden. The conditions under which the simulated-use testing will be conducted will be sufficiently realistic so that the results of the testing are generalizable to actual intended use. Therefore, fabricated, but realistic, patient cases will be used for the testing. In addition, cases are created to ensure that all critical tasks are performed during the test and that the test conditions are sufficiently realistic to represent actual conditions of use.
After each case the test participant will be asked to fill in the clinical pathway that they find most appropriate based on the information they have received and the NILS calculation. Several pathways can be appropriate: “Consider omitting SLNB”, “Consider performing SLNB”, “Definitely perform SLNB”, and “NILS cannot assist in making the clinical decision”. The test participants will following be asked to fill in a validated questionnaire about their experience of the device. Interview questions will be asked after the use of the interface if any tasks were incorrectly or close to incorrectly executed.
What are the possible benefits and risks of participating?
The participants will receive a small symbolic compensation for the participation. No risks of participating have been identified.
Where is the study run from?
Lunds University Cancer Center in cooperation with the South Swedish Health Care Region.
When is the study starting and how long is it expected to run for?
November 2024 to June 2025
Who is funding the study?
1. The Governmental Funding of Clinical Research within the National Health Services (Sweden).
2. The Swedish Cancer Society (Sweden).
3. Lunds University (Sweden).
4. Region Skåne/South Swedish Health Care Region (Sweden)
5. The Swedish Research Council (Sweden)
6. The Erling Persson Foundation (Sweden).
Funding resources had no role in the study design, data collection, analyses, data interpretation, writing of the manuscript, or the decision to submit the manuscript for publication.
Who is the main contact?
MD Anna Allfelt, anna.allfelt@med.lu.se
MD, PhD Ida Skarping, ida.skarping@med.lu.se
Contact information
Principal Investigator
Bröst och melanommottagning, SUS
Tora Kjellgrens gata 27
Malmö
205 02
Sweden
 ORCID ID |
0000-0001-7515-3130 |
|---|---|
| Phone | +46 (0)46176241 |
| lisa.ryden@med.lu.se |
Scientific
Klinisk fysiologi och nuklearmedicin, SUS
Entrégatan 7
Lund
221 85
Sweden
| ORCID ID |
0000-0002-1265-8649 |
|---|---|
| Phone | +46 (0)46177309 |
| ida.skarping@med.lu.se |
Public
Bröst och melanommottagning, SUS
Tora Kjellgrens gata 27
Malmö
205 02
Sweden
| ORCID ID |
0009-0002-7889-9821 |
|---|---|
| Phone | +46 (0)40335357 |
| anna.allfelt@med.lu.se |
Study information
| Study design | Pre-market human factors validation test under simulated conditions. |
|---|---|
| Primary study design | Observational |
| Secondary study design | Human factor vaildation test |
| Study setting(s) | Hospital |
| Study type | Other |
| Participant information sheet | See outputs table |
| Scientific title | Human factors validation study of an artificial neural network‑based preoperative decision‑support tool for noninvasive lymph node staging (NILS) in women with primary breast cancer |
| Study objectives | From a usability point-of-view evaluating the final product with intended-use testing, the hypothesis is to present evidence that the interface for NILS is considered safe. |
| Ethics approval(s) | Ethics approval not required |
| Ethics approval additional information | This usability study is not a clinical study, and it does not require an ethical approval from the Swedish Ethical Review Authority. The usability study is approved by the Head of Research in Region Skåne regarding the intended conduction of the study and Region Skåne is thus the legal authority for the study. |
| Health condition(s) or problem(s) studied | Pre-market human factor validation test with intended (actual) users. |
| Intervention | The test participants will be greeted by the test leader and be provided a written instruction for use manual. The test participants will be presented with five simulated cases and are expected to fill in the patient information correctly and press calculate. The test participants are then expected to interpret the result of the calculation correctly and consider the calculated results of probability of a benign axillary nodal status together with other sources of information when subsequently determining the most appropriate clinical pathway for the patient. The hypothetical treatment decision per se will not be evaluated. This will be done five times covering all the cases. The test participants will following be asked to fill in a validated questionnaire about their experience of the device. Lastly a short oral interview will take place if indicated. The total time required per test-participant is approximately 45-60 min. No follow-up is planned. |
| Intervention type | Other |
| Primary outcome measure | 1. Frequency of observed outcome and result during the test, analysis per task. Observed outcome is defined as correct use, close call, use difficulty, and use error. Result is defined as passed or failed. 2. Frequency of passed critical tasks per scenario. 3. Frequency of observed outcome and result during the test, analysis per task description in association with potential harm (false estimated indication of benign sentinel lymph node). Observed outcome is defined as correct use, close call, use difficulty, and use error. Result is defined as passed or failed. 4. Average point (median) on Likert scale 1-5 and 1-7: 1 “strongly disagree” to 5 respectively 7 “strongly agree” in validated questionnaire (Lewis J.R 2018 and 1995). 5. Review of all comments in validated questionnaire (Lewis J.R 2018 and 1995) and interview data with the purpose to detect potential safety and performance concerns. Since NILS has not yet been released to the market and has had limited contact with the intended users, it is likely that some usability issues may be detected. However, a large amount of user errors is not acceptable. Therefore, the team has set the acceptance criteria to 90%. Each test protocol interview data will be reviewed and all comments with potential safety and performance concerns will be analyzed to assess any required changes in design. In case any significant change of the NILS web page will be needed, the change will be re-assessed using a new scenario with the purpose to re-evaluate the potential concern. The qualitative evaluation will be acceptable when no potential safety and performance concerns remain to be analyzed or addressed. |
| Secondary outcome measures | There are no secondary outcome measures |
| Overall study start date | 01/01/2024 |
| Completion date | 30/06/2025 |
Eligibility
| Participant type(s) | Health professional |
|---|---|
| Age group | Adult |
| Lower age limit | 18 Years |
| Upper age limit | 65 Years |
| Sex | Both |
| Target number of participants | 20 |
| Key inclusion criteria | 1. Intended users - licensed medical physicians specialized in general surgery or oncology (including medical and radiation oncologists) 2. >18 years 3. Both sexes |
| Key exclusion criteria | 1. Any former contact with the device |
| Date of first enrolment | 12/09/2024 |
| Date of final enrolment | 31/12/2024 |
Locations
Countries of recruitment
- Sweden
Study participating centres
Malmö
214 28
Sweden
Lund
222 42
Sweden
Kristianstad
291 33
Sweden
Karlskrona
371 41
Sweden
Växjö
352 34
Sweden
Sponsor information
Government
Västra Storgatan 12
Kristianstad
291 89
Sweden
| Phone | +46 (0)44-3093000 |
|---|---|
| ida.skarping@med.lu.se | |
| Website | https://www.skane.se/ |
Funders
Funder type
Government
Government organisation / Universities (academic only)
- Alternative name(s)
- Lund University, Universitas Lundensis, Universitas Gothorum Carolina, Royal Caroline Academy, Regia Academia Carolina, Lund University | Lund, Sweden | LU, Lunds universitet, LU
- Location
- Sweden
No information available
No information available
No information available
No information available
No information available
Results and Publications
| Intention to publish date | 01/12/2025 |
|---|---|
| Individual participant data (IPD) Intention to share | Yes |
| IPD sharing plan summary | Available on request |
| Publication and dissemination plan | Planned publication in an international peer reviewed journal. |
| IPD sharing plan | The dataset generated and analyzed during the current study will be available upon reasonable request from Anna Allfelt, anna.allfelt@med.lu.se |
Study outputs
| Output type | Details | Date created | Date added | Peer reviewed? | Patient-facing? |
|---|---|---|---|---|---|
| Other files | Instructions for participants | 15/11/2024 | No | No | |
| Other files | Instructions for use | 15/11/2024 | No | No |
Additional files
- 46376 Instructions for use.pdf
- Instructions for use
- 46376 Instructions for participants.pdf
- Instructions for participants
Editorial Notes
16/12/2024: The sponsor email was changed.
15/11/2024: Trial's existence confirmed by Region Skane.
