Outcomes of Integrating Smart Phrase Interface Technology to Improve Cancer Symptom Management

Communication errors are the most frequent cause of sentinel events in United States healthcare with oncology tele-health particularly at risk [1]. Assessing cancer treatment-related symptoms remotely is significantly error-prone for frontline telephone-triage nurses toggling multiple electronic health record (EHR) screens for toxicity-risk predictors while simultaneously providing over-the-phone emotional support. Evidence-based approaches to improve and standardize telephone-triage symptom assessment, well-embedded into nursing practice, documented in the electronic health record (EHR), highly reliable, sustainable, and widely disseminated are needed to reduce cancer treatment toxicity risks and enhance patient/family engagement and comfort. 

Clinician-driven telephone-triaging and clinical pathways have been used to safely, effectively and efficiently manage symptoms using evidence-based algorithms as early as 1999 [2, 3]. Limited recent attempts to interface these within the electronic health record (EHR) show promise to improve care transitions, handoff communications, access, nurse-patient relationships and patient/ family education; decrease emergency room visits, delays in care, and readmission-related hospital-acquired infections [4,5,6]. Despite these advances, significant challenges to implementing high quality, cost effective remote triaging exist [7,8].Major barriers include clinician lack of knowledge, computer skills, and confidence to remotely assess, triage, and guide patients in self-care for symptom treatment, time management, discomfort in not recommending use of the emergency room, documentation issues, and lack of user-friendly access to software [9-12,4, 6].

The purpose of this translational research project was to: (1) develop and integrate an evidence-based tele-triage nursing assessment decision-support interface tool into a multi-site cancer center’s EHR and (2) to evaluate its usability and impact on nurse /patient satisfaction and nurse-sensitive quality outcomes. It was believed that an interface tool, called a smart phrase held much promise for improving patient satisfaction with symptom management. By their involvement in all aspects of the project, tele-triage nursing job performance, usability, and utilization for documentation and quality outcome measurement might also be enhanced.

The overall project’s framework centered on research translation theory using Default’s 6-step Collaborative Research Utilization (CRU) model. This process uses sequentially designed activities in which practice-based clinicians are paired with academic researchers and students to promote evidence-based practice by addressing the long lag time from when research discoveries are made to when they become integrated into clinician’s day-to-day care. Used to change practice in over 200 targeted areas, the process was successfully tested in eight previous studies to improve pain management, handoff communications, and other clinical issues [13-16].

The project site, a multi-site cancer center included three hospitals and one full-service satellite outpatient clinic with the goal of providing state-of-the-art cancer care and support for over 14,000 patients/families in the northeastern United States. Hospital members included a large urban academic tertiary medical center, a medium-sized Magnet-designated urban teaching hospital, and a small urban community hospital, also Magnet-designated. Care teams include board-certified hematologists/oncologists, nutritionists, pharmacists, social workers, and 61 nurses, the majority oncology certified. As members of a large system with highly integrated computerized patient information and nursing documentation system in place for three years, the center maintains a 24-hour hotline call center for oncology patients. Currently an average of 1,875 symptom management calls per month is answered with 25% of patients surveyed for satisfaction outcomes and a 30% response-rate. Nurse-sensitive quality outcomes are analyzed using the system’s Outpatient Oncology Patient Satisfaction Survey [17] and the system’s quality database [18].

In the first CRU step, clinical, theoretical, and validated empirical evidence on telephone triaging and symptom assessment and management was accessed (Table 1). 

Table 1: Evidence summaries of 3 Roundtables

Clinical evidence

Informal brain-storming focus groups were led by a consulting academic nurse researcher at the lead hospital site with 23 tele-triage nurses who identified that remotely assessing risks of cancer treatment-related symptoms was significantly burdensome for them. As frontline contacts for these vulnerable patients, guiding patients to the most appropriate level of care (from self-management to the emergency room), nurses described issues related to multi-tasking, risks of missed communications, and lack of standardized assessment. Free texting documentation without the ability to track quality measures posed additional challenges in evaluating their care. Patient satisfaction and quality data revealed an opportunity to improve patient satisfaction related to symptom assessment and management [19]. In the focus groups it was suggested that using smart phrase technology might be useful to explore as an approach to assist the nurses.

Empirical evidence

Next, hospital librarians and a doctoral nursing student conducted comprehensive CINAHL and Med-line searches [6]. They found that earlier studies validated nurse experiences related to these barriers and pointed to the need for developing a decision-support technology tool and evaluating it by the users who should be involved in its design from the beginning.In their search, Goodhue and Thompson’s Task Technology Fit Theory (TTF), was found to be useful to guide evaluating the usability of new technology because of its ability to describe, explain, and predict user performance and provide understanding of the match between task characteristics, individual user characteristics, and functionality of new technology [20]. No studies were found that described smart phrase designs, their impact on improving remote cancer symptom management, nor examined usability and associated nursing job performance outcomes of using this unique approach to remote symptom assessment. Eight studies were selected for roundtable discussions in step 2 of the CRU process.

Three research roundtables were held in the hospitals to review the eight studies. As the second step in the CRU model, research roundtables have been successfully used to teach clinical nursing staff and students about how to evaluate the strength of evidence and readiness for integration into practice [21,22,13,16,17]. With the nurse researcher leading the roundtables, telephone triage nurses, inpatient oncology nurses, members of the center’s quality team, affiliating undergraduate and graduate nursing students and their clinical instructors determined methodological strength of each study by applying Polit and Beck’s 2012 criteria for determining level of evidence strength [23]. They also evaluated them for fit of setting, potential implementation risks, readiness for change among telephone-triage nurses, resources required, and current practice according to Stetler’s2001clinical criteria [24]. It was determined that according to demographics in this setting, embedding assessment and management parameters in a smart phrase might cue nurses towards improved symptom assessment and help predict hospitalization indication in nearly 70% of patients treated [18]. Evidence summary tables of the eight studies including 31 recommendations for a smart phrase design and its implementation were constructed (Table 2) and forwarded to a smart phrase design team. 

Table 2: Evidence summary of roundtable 

Led by the Nursing Safety and Quality Manager in collaboration with 6 tele-triage nurses, a design team developed the smart phrase based on the 31 recommendations. After analyzing hotline call types received in the center and patient satisfaction data, they chose the five most commonly occurring symptom management calls on which to focus. These included: managing chemotherapy side effects of fatigue, appetite loss and nausea, fever, pain, and diarrhea and constipation [18]. They determined that rapid uptake and sustainability might be enhanced by embedding forcing functions, easily-accessed hyperlinks and pop-ups designed by the triage nurses themselves. For example, one attractive smart phrase feature they incorporated was that when a call was triaged, as soon as the name and date of birth were received, the EHR screen would automatically display patient-related demographics, related diagnostics, and active problem list. Triage nurses could then select the symptom(s) from the displayed list or add others, and standardized assessment cues would appear for documentation and interventions, with direction to the most appropriate levels of care. Three samples of smart phrase designs were developed and the 23 tele-triage nurses voted on their choice. It was embedded into the EMR in December, 2017 with the belief that it held much promise for improving patient satisfaction with symptom management tailored to patient’s individualized needs. It was also hypothesized that usability, documentation, quality measurement ability, and tele-triage nurse job performance would be enhanced (Table 3). 

Table 3: Evidence summary table and recommendations on roundtable

Mentored by the consulting translational nurse researcher, and the clinical manager of one hospital site as the project director, an evaluation study was designated exempt by both hospital and university institutional review boards. A repeated-measures, phased-in design was used to compare implementation outcomes at each site and overall. The research team was especially interested in evaluating changes in patient satisfaction related to symptom management before, during, and after imbedding the smart phrase into the EHR. Usability and tele-triage nurse outcomes were studied in the context of an affiliated doctoral student’s dissertation [6].

Measurement of Patient Outcomes

Patient outcomes were assessed along a 5-point trajectory of 6-month and 3-month pre-implementation, at baseline implementation, and three and six month’s post-implementation. Measured and summarized de-identified data of 1875 responses to the 2017 Lifespan Health System Outpatient Oncology Patient Satisfaction Survey data [18] and data from the center’s 2017 Lifespan EPIC Quality Database [19] were used for analysis of nurse-sensitive patient satisfaction outcomes. Seven variables related to symptom management were chosen to evaluate for changes including: patient satisfaction with managing chemotherapy side effects, fatigue management education, managing appetite loss, emotional needs addressed, home-based instructions, pain well controlled, and perceived safety and security.

Measurement of Tele-triage Nurse Outcomes

TTF theory was selected for its ability to guide evaluating smart phrase usability and potential impact on tele-triage nurse utilization and performance. (Figure 1) displays the evaluation measures used and their relationship to TTF theory as proposed [6]. 

Figure 1: Evaluation Measures and Relationship to Task Technology Fit Theory 

Six months prior to implementation, demographic data were collected on variables found to affect computer performance including age, gender, years in nursing practice, years in oncology nursing practice, years working as a tele- triage nurse, years using the EHR, and years using computer technology [25-30]. Three instruments were used to measure usability, self-perceived job performance, and utilization of the smart phrase. Previous studies supported a learning curve of six months post computer information systems training to measure these [31]. Overall smart phrase usability was measured six months following go-live implementation, using Brooks [32] 10-item, five-point Likert “System Usability Survey (SUS)” with high Cronbach’s alpha reported at .92 [33]. Differences in nurse-user self-perceptions of job performance six months before and six months afterwards was measured using Default’s (2017) 15-item “Telephone Triage Nurse Survey” (TTNS) [34].Successful content validity testing conducted by an expert panel of eight advanced practice oncology nurses rated survey items sampled from studies on best telephone triage practices [4,12] as consistent with conceptual and operational definitions of six dimensions of competency in oncology tele-triage job performance. They included: technical or psychomotor skills, knowledge of cancer symptoms and safety risk, interdisciplinary communication, confidence in one’s determination of the level of care needed, and values and attitudes on telephone triaging for symptom assessment. Inter-item internal consistency yielded a high Cronbach's alpha of 0.79. Utilization, or the extent to which nurses integrated the SmartPhrase into their work was measured using 5-point Likert Scale as a posttest only [34,35]. Based on original questionnaire, respondents reported on impact the new technology had on effectiveness, productivity, and job performance.

Data collection

After collecting all pretest patient outcome data, demographic and pretest nurse outcome data, the project manager and one advanced practice nurse from each hospital site, all highly experienced in tele-triaging and oncology nursing, trained all 23 tele-triage nurses on the smart phrase. They provided “just-in-time” coaching to provide real-time feedback during the first three months month post-implementation from January to March, 2019.

Analysis

All analyses were carried out using Statistical Analysis System (SAS) software, version 9.4. Descriptive statistics were used to describe, summarize, and synthesize collected data using frequencies and percentages, means and standard deviations. Bivariate analyses were conducted to describe differences in the pre to post-test means of TTNS scores and explore the relationships among the variables including the demographic variables of years of experience and the usability score. Generalized Linear Mixed Models were used to construct hierarchical piecewise regressions to model the full variety of outcome measures at p

Findings

Patient Outcomes: Three patient satisfaction outcomes improved significantly (p>.05) from 6 months pre-implementation to 6 months after embedding the smart phrase into the EHR at all center sites. These included fatigue management, managing overall chemotherapy side effects, and safety and security felt during hospitalization. For patient satisfaction with fatigue management, a spike occurred in all sites from the moderately low 80’s percent satisfaction scores to a high of 89 three months posttest, and then leveled off to 87 percent. Satisfaction with overall chemotherapy side effect management gradually increased from 92.6 percent to a high of 94.2, leveling at 94. Patients scored satisfaction with safety and security felt during hospitalization at a moderately low 90’s percent mean score in pre implementation months and baseline at delivery, followed by a dramatic spike upward from the go-live date to a high of 97 percent 3 months post-implementation and leveled off to 96% post-implementation (Table 4). 

 Table 4: Five-point pre to post implementation trajectory of significant patient satisfaction outcomes at p

Teletriage-nurse outcomes: Descriptive statistics for the demographic variables including tele-triage nurse years of nursing experience, years in oncology nursing, years as a tele-triage nurse, experience with the EHR system, and with the system’s documentation system are summarized in (Table 5) [6]. While they tended to be well experienced in nursing and oncology, they tended to be less experienced as a tele-triage nurse and familiar with EHR systems and the hospital’s documentation system. 

Table 5: Descriptive Statistics of Demographic Variables for all Participants (n=23)

Usability 

Total mean usability across all tele-triage nurse subjects correlated to marginal acceptability at 58.06, a value lower than the SUS benchmark score of 68 [36],but still deemed usable and could be improved. The only demographic variable contributing to the direction (positive or negative aspect of usability) was nurses’ years of oncology experience. The greater the years of oncology nursing experience, the less the nurse perceived the usefulness of the tool and satisfaction with its effect on their job performance.Interestingly however, the opposite effect occurred with less experienced tele-triage nurses.They perceived greater usability and improved job performance than their more experienced counterparts (Table 6).

 Table 6: Relationship of usability to years of oncology experience 

For actual utilization, nurses thought their use of the smart phrase significantly streamlined documentation time, enabling them to complete documentation accurately within the shortest time possible (P< .0001) and they depended on it in their work routines (P< .003).

The finding that three of the nurse-sensitive measures related to patient satisfaction improved significantly (P< .05) at the three and 6-month data collection points is significant. Prior to and at the go-live implementation date, patient satisfaction with fatigue management were in the low to mid-80 percent level and did not change significantly over this six-month trajectory. Of all of the selected 7 variables of interest, fatigue management had the lowest of the pretest scores. The significant (P<.05) spike to the high of 89 and leveling off to 87 is optimistic and an important finding. Fatigue ranks as the most debilitating cancer treatment-related symptom with approximately 82-96% of patients reporting cancer-related fatigue during treatment and adversely affecting their daily activities, social relationships, mood, and quality of life [37]. However the slight decline leveling off to 87% bears caution. Monitoring at the one-year point was suggested. With patient satisfaction with chemotherapy side effect management remaining in the low 90’s throughout the 6-month pre-implementation points and gradually increasing from the go-live day to 94 percent is encouraging, though the change was not as dramatic as with fatigue management. The most dramatic spike occurred with patient’s satisfaction with safety and security felt during hospitalization. The dramatic spike upward from the go-live date to a high of 97 percent and remaining significantly higher may indicate greater confidence in the tele-triage nurses response to their hotline call for help.

The way EHR's structure and present patient information, process data and generate clinical reminders (e.g. alerts and popup messages) too often impedes healthcare provider’s time with a patient and has a direct effect on their decision-making and nurses are rarely involved in their design [38].For several years, the Agency for Healthcare Research and Quality had considered usability as one of four current Health Information Technology priorities in the United States [39], and this continues to be an issue. In sync with the literature, testing the usability of the EHR tool was recognized as critical for identifying design features of EHR systems that result in errors, poor utilization and ineffective use [40, 41].Usability of the smart phrase tool was marginally acceptable to the tele-triage nurse users and needs to be improved with the only variable contributing to the direction of usability was years of oncology experience. The finding that less experienced tele-triage nurses found smart phrase usability greater and enhanced their job performance underscores the importance of using actual end-users evaluation of technology to support their effective and efficient clinical work as demonstrated in the literature [42,43].Use of the smart phrase for less experienced oncology nurses may have functioned as a training tool for their new position in tele-triaging oncology symptom management. More experienced oncology nurses may already know the assessment process needed for each symptom patients present, and usability of the smart phrase was not equal to their level of expertise. Perhaps they did not need cues to their assessment process, and it only distracted them. However it is important to be able to easily track the success of interventions and quality metrics through documentation, other than free texting. The finding that the smart phrase tool shortened the time of nursing documentation and enabled nurses to complete tasks accurately within the shortest time possible is significant. Since tele-triage nurses generally do more documentation and process more symptom assessment information than other health care providers, using the new smart phrase may potentially directly affect their workflow and job performance on a daily basis.

Following the finding of positive outcomes related to patient satisfaction and satisfaction in less experienced tele-triage nurses, along with the decrease in documentation time, the decision was made to continue using the smart phrase while continuing to monitor patient satisfaction results.

Further translational projects in other settings may provide compelling evidence that such smart phrases may significantly increase clinician uptake, sustainability, and impact nurse-sensitive outcomes. Other smart phrases have since been developed as decision-support tools for clinicians at the cancer center, and testing is in progress.

This translational research project addressed the goal of applying the CRU model to enhance staff nurse uptake, sustainability, and improve patient outcomes related to remote telephone-triaging of symptom assessment as it has with other clinical issues. Evidence-based approaches to improve standardized telephone-triage symptom assessment, well-embedded into nursing practice, documented in the EHR, highly reliable, sustainable, and widely disseminated are needed to reduce treatment toxicity risks and enhance patient/family engagement and comfort. Significant advantages may include developing present and future clinicians competent in symptom assessment and management for cancer patients, as well as in using the evidence of research to change practice. Integrating new technology can be perceived as an onerous and frustrating experience, and so it was critical to test usability of the smart phrase tool. Dalky’s examination of changes over time will provide a deeper understanding of the evaluation of human factors related to usability. Learning more about such decision-support tools from tele-triage nurses’ viewpoint may help to determine linkages between usability and how it may impact their nursing care performance and patient outcomes. Results of this project will inform as to the usefulness of the smart phrase tool for remote symptom management. Establishing a continual evaluation and feedback loop may provide valuable insight into the design of the decision-support smart phrase tool and ultimately fit the workflow of tele-triage clinicians.

Nurses face usability challenges when new technology is introduced. This translational study is a step forward in designing and developing information processing tools that enhance clinician performance and patient outcomes. Meaningful clinician participation in the development of the standardized, evidence-based, patient-centered approach to nurses’ remote symptom management was achieved. Using the CRU Model can facilitate the integration of new knowledge both in the clinical and academic community.