Journal of Practical & Professional Nursing Category: Clinical Type: Research Article

Outcomes of Integrating Smart Phrase Interface Technology to Improve Cancer Symptom Management

Dufault M1*, Dalky A2, Ritz J3, Begnoche M4, Chauvin M5, Cherenzia K6, Corrente J7, Laquale P8 and Machan J9
1 Professor, College Of Nursing, University Of Rhode Island, United States
2 Assistant Professor, Health Management And Policy Department, Jordan University Of Science And Technology, Jordan
3 Clinical Nurse Manager, Lifespan Cancer Center, Newport Hospital, United States
4 Nursing Safety And Quality Manager, Rhode Island Hospital, Lifespan Cancer Center, United States
5 Staff Nurse, Lifespan Cancer Center, Newport Hospital, United States
6 Program Manager, Lifespan Cancer Center Address: Rhode Island Hospital, United States
7 Clinical Nurse Specialist, The Miriam Hospital, Lifespan, United States
8 Clinical Manager, Radiation Oncology Clinic, Lifespan Cancer Center Address: Rhode Island Hospital, United States
9 Adjunct Associate Professor Psychology, University Of Rhode Island, United States

*Corresponding Author(s):
Dufault M
Professor, College Of Nursing, University Of Rhode Island, United States

Received Date: Jan 26, 2021
Accepted Date: Feb 12, 2021
Published Date: Feb 19, 2021


Assessing cancer treatment-related symptoms is challenging for frontline telephone-triage nurses toggling multiple electronic health record (EHR) screens for information on toxicity-risk predictors while simultaneously providing over-the-phone emotional support. Decision-support tools embedded into computer-telephone interface software, called smart phrases, are not widely used nor empirically evaluated for usability and remote symptom management impact. The process and outcomes of designing and empirically evaluating a telephone-triage smart phrase embedded into the EMR of a northeast multi-site cancer center serving 14,000 patients is described. Using a 6-step collaborative translating-research-to-practice model as process, Task-Technology-Fit Theory guided patient/nurse outcome evaluation. Analysis of repeated-measures using Generalized Linear Mixed Models, hierarchical regressions constructed the full variety of outcomes (p<.05). Analysis of patient outcomes of 375 telephone-triaged calls at each of five data-collection points found patient satisfaction on managing chemotherapy side effects, fatigue, and perceived safety/security improved significantly 6 months following smart phrase integration. Smart phrase usability and utilization increased, nursing documentation time decreased, and self-perceived effects on job performance increased for less-experienced oncology nurses from pre-to-6-month-post implementation. Evidence based approaches to standardize telephone-triage symptom assessment, well-embedded into practice, documented in the EHR, highly reliable, sustainable, and widely disseminated show promise for enhancing patient/family engagement, and improving job performance in novice tele-triage oncology nurses.


Nursing symptom management; Smart phrase; Telephone triaging; Translational research


Communication errors are the most frequent cause of sentinel events in United States healthcare with oncology tele-health particularly at risk [1]. Assessing cancer treatment-related symptoms remotely is significantly error-prone for frontline telephone-triage nurses toggling multiple electronic health record (EHR) screens for toxicity-risk predictors while simultaneously providing over-the-phone emotional support. Evidence-based approaches to improve and standardize telephone-triage symptom assessment, well-embedded into nursing practice, documented in the electronic health record (EHR), highly reliable, sustainable, and widely disseminated are needed to reduce cancer treatment toxicity risks and enhance patient/family engagement and comfort. 

Clinician-driven telephone-triaging and clinical pathways have been used to safely, effectively and efficiently manage symptoms using evidence-based algorithms as early as 1999 [2, 3]. Limited recent attempts to interface these within the electronic health record (EHR) show promise to improve care transitions, handoff communications, access, nurse-patient relationships and patient/ family education; decrease emergency room visits, delays in care, and readmission-related hospital-acquired infections [4,5,6]. Despite these advances, significant challenges to implementing high quality, cost effective remote triaging exist [7,8].Major barriers include clinician lack of knowledge, computer skills, and confidence to remotely assess, triage, and guide patients in self-care for symptom treatment, time management, discomfort in not recommending use of the emergency room, documentation issues, and lack of user-friendly access to software [9-12,4, 6].


The purpose of this translational research project was to: (1) develop and integrate an evidence-based tele-triage nursing assessment decision-support interface tool into a multi-site cancer center’s EHR and (2) to evaluate its usability and impact on nurse /patient satisfaction and nurse-sensitive quality outcomes. It was believed that an interface tool, called a smart phrase held much promise for improving patient satisfaction with symptom management. By their involvement in all aspects of the project, tele-triage nursing job performance, usability, and utilization for documentation and quality outcome measurement might also be enhanced.


The overall project’s framework centered on research translation theory using Default’s 6-step Collaborative Research Utilization (CRU) model. This process uses sequentially designed activities in which practice-based clinicians are paired with academic researchers and students to promote evidence-based practice by addressing the long lag time from when research discoveries are made to when they become integrated into clinician’s day-to-day care. Used to change practice in over 200 targeted areas, the process was successfully tested in eight previous studies to improve pain management, handoff communications, and other clinical issues [13-16].


The project site, a multi-site cancer center included three hospitals and one full-service satellite outpatient clinic with the goal of providing state-of-the-art cancer care and support for over 14,000 patients/families in the northeastern United States. Hospital members included a large urban academic tertiary medical center, a medium-sized Magnet-designated urban teaching hospital, and a small urban community hospital, also Magnet-designated. Care teams include board-certified hematologists/oncologists, nutritionists, pharmacists, social workers, and 61 nurses, the majority oncology certified. As members of a large system with highly integrated computerized patient information and nursing documentation system in place for three years, the center maintains a 24-hour hotline call center for oncology patients. Currently an average of 1,875 symptom management calls per month is answered with 25% of patients surveyed for satisfaction outcomes and a 30% response-rate. Nurse-sensitive quality outcomes are analyzed using the system’s Outpatient Oncology Patient Satisfaction Survey [17] and the system’s quality database [18].


In the first CRU step, clinical, theoretical, and validated empirical evidence on telephone triaging and symptom assessment and management was accessed (Table 1). 


Study Objective

Sample and Type

Design / Method


Strength of Evidence (Polit & Beck, 2012 criteria)/Recommendations







Gleason, K., O'neill, E. B., Goldschmitt, J., Horigan, J., & Moriarty, L. (2013).

To standardize triage practice and ultimately improve the effectiveness of telephone triage management.

13 Phone Triage RN’s at North Shore LIJ Health Systems Monter Cancer Center, Center for Advanced Medicine

Seven questions using the online tool, SurveyMonkey®online tool was used to assess to evaluate the nurse’s comfort with telephone triage and its management, identification of obstacles to telephone triaging, knowledge of dis- ease process, and recognition of oncologic emergencies.

After a three-month trial of educational presentations, a 25% increase in knowledge about diseases, a 20% increase in management of side effects, and a 20% increase in the comfort level of the nurses not referring patients to the emergency room for evaluation were noted

Level VI/






1. Ty Type of communication tool for documenting useful






2. Nurse satisfaction questions too broad—need to be more specific related to outcome measures.







Kuntz, G, Tozer, J. M., Snegosky, J, Fox, J. MD, MHA, Neumann, K, MD. (2014).

Innovative multipartite oncology medical home model supported by payment reform.

Four oncology practices (29 physicians) participated and enrolled 85 patients

Expert Opinion

The total estimated cost savings for year 1 was $46,228.

Level VII/






1. drilled down symptoms are useful for what we are typing to build






2. need to examine # of calls






3. Practices that embedded changes into their daily workflow (e.g. advance care planning and triage algorithm) were most successful






4. Having comprehensive written program documentation and consistently applied training facilitates practice understanding.






5. The importance of patient education in making the medical oncologist the acknowledged primary care physician during active treatment.






6. Planning of timeline for implementation must come from the triage nurses so they can effectively absorb new approaches.

Tariq, Amina, Westbrook, Johanna, Byrnes, Mary, Robinson, Maureen, Baysari, Melissa, T. (2017)

To evaluate usability of a decision support system for telephone triage nurses.

Stakeholders: members of the research team and the clinical lead telephony services, Users: 9 nurses (n=9)

Mix method design, qualitative/quantitative

Forty one unique heuristic violations were identified in the interface design of the decision support software with median severity of 2.25 (range 0—4, with 0=no problem to 4=catastrophic problem).

Level V/






1.Useful to identifying types of riagetriage calls. Note that current practices free texting and this is not useful for standardizing reports, and ability to examine quality issues.







Table 1: Evidence summaries of 3 Roundtables

Clinical evidence

Informal brain-storming focus groups were led by a consulting academic nurse researcher at the lead hospital site with 23 tele-triage nurses who identified that remotely assessing risks of cancer treatment-related symptoms was significantly burdensome for them. As frontline contacts for these vulnerable patients, guiding patients to the most appropriate level of care (from self-management to the emergency room), nurses described issues related to multi-tasking, risks of missed communications, and lack of standardized assessment. Free texting documentation without the ability to track quality measures posed additional challenges in evaluating their care. Patient satisfaction and quality data revealed an opportunity to improve patient satisfaction related to symptom assessment and management [19]. In the focus groups it was suggested that using smart phrase technology might be useful to explore as an approach to assist the nurses.

Empirical evidence

Next, hospital librarians and a doctoral nursing student conducted comprehensive CINAHL and Med-line searches [6]. They found that earlier studies validated nurse experiences related to these barriers and pointed to the need for developing a decision-support technology tool and evaluating it by the users who should be involved in its design from the beginning.In their search, Goodhue and Thompson’s Task Technology Fit Theory (TTF), was found to be useful to guide evaluating the usability of new technology because of its ability to describe, explain, and predict user performance and provide understanding of the match between task characteristics, individual user characteristics, and functionality of new technology [20]. No studies were found that described smart phrase designs, their impact on improving remote cancer symptom management, nor examined usability and associated nursing job performance outcomes of using this unique approach to remote symptom assessment. Eight studies were selected for roundtable discussions in step 2 of the CRU process.


Three research roundtables were held in the hospitals to review the eight studies. As the second step in the CRU model, research roundtables have been successfully used to teach clinical nursing staff and students about how to evaluate the strength of evidence and readiness for integration into practice [21,22,13,16,17]. With the nurse researcher leading the roundtables, telephone triage nurses, inpatient oncology nurses, members of the center’s quality team, affiliating undergraduate and graduate nursing students and their clinical instructors determined methodological strength of each study by applying Polit and Beck’s 2012 criteria for determining level of evidence strength [23]. They also evaluated them for fit of setting, potential implementation risks, readiness for change among telephone-triage nurses, resources required, and current practice according to Stetler’s2001clinical criteria [24]. It was determined that according to demographics in this setting, embedding assessment and management parameters in a smart phrase might cue nurses towards improved symptom assessment and help predict hospitalization indication in nearly 70% of patients treated [18]. Evidence summary tables of the eight studies including 31 recommendations for a smart phrase design and its implementation were constructed (Table 2) and forwarded to a smart phrase design team. 


Study Objective

Sample and Type

Design / Method


Strength of Evidence/recommendations

Maloney, K, Denno, M, Kider, T et al (2013).

To evaluate the oncology phone service as a consultation support for chemotherapy administration, symptom management, and care coordination.

1,353 phone calls managed by oncology nursing team. Originating from nurses on medical, surgical, women’s health, and critical care units, physicians, bed management center staff, radiology areas, and nursing coordinators.

Oncology phone data are collected with each call and tabulated online. Data is reviewed categorically. Time, date of request, who receives the request, name of requestor, role, contact number and department or inpatient care unit needed support, chemotherapy verification, education, bed placement, line care, lab draws, and drain management. Monitoring of compliance with data documentation completed through team meetings and quarterly review of online data.

Data collection allows identification of gaps in practice and need for process improvement projects. Action plans include expansion of Sunrise Clinical Manager, computerized ordering system, order sets for specific indications and dosing parameters, and creation of regime grids and keys for practice. The oncology phone provides an effective means of delivering specialized care to patients who are not housed in locations where oncology nurses are readily available.

Level VI/






1. having 1 go-to person very positive






2. recommended need for development at Newport where there is no specialty unit






3. study identified fields for data collection






4. need to evaluate patient portal, and provide training to pts.

Stacey, D, Carley, M, Kohli, M, et al. (2014).

To explore current remote symptom support training programs provided to nurses in ambulatory oncology programs in Canada.

Of 36 delivered invitations, 28 programs responded (77.8%) representing 10 provinces.

Survey methods environmental scale

17 programs offer telephone symptom support training, 7 shared training materials, and elements of training included: symptom management guidelines (n=6), telephone triage process/principles (n=5), competent telephone practices (n=4), documentation (n=4), professional standards (n=3), paper based resources (n=3), e-learning modules (n=1). No programs were rigorously evaluated. Training is variable, Opportunities exist to identify core competencies and evaluate if training enhances delivery of remote symptom support.

Level VI/






1. Take away message: how to teach communication skills






2. See table 3 for types of skills for training triage nurses






3. Use of repeat-back technique a critical aspect of patient education: need to add to smart phrase development






4. Consider dual screen and functionality of it







Stacey, K. Skrutkowski, M, Carley, M et al. (2015)

To evaluate the impact of training on nurses satisfaction and perceived confidence using symptom protocols for remotely supporting patients undergoing cancer treatment.

107 nurses who provide remote support to patients with cancer

Retrospective pre-post-study

22 workshops, 30-60 minutes were conducted with nurses. 90 completed the survey. Post-workshop nurses had improved self-confidence to assess, triage, and guide patients in self-care for cancer treatment-related symptoms, and use protocols to facilitate symptom assessment, triage, and care. Workshops rated easy to understand, comprehensive, and provided information on remote symptom management. Some specified that they workshop did not provide enough time for role play, but Workshop increased nurses perceived confidence with providing remote symptom support and was well received.

Level IV/






1. use of algorithms important






2. top 5 reasons identified for calls categorized. They may be embedded from care plans






3. use of scripting and consistent questions as well as drop downs for appropriate questions






4. Recommend meeting of all triage nurses as a group. Ask them to write down questions they generally ask as part of their routine prior to meeting.

Table 2: Evidence summary of roundtable 


Led by the Nursing Safety and Quality Manager in collaboration with 6 tele-triage nurses, a design team developed the smart phrase based on the 31 recommendations. After analyzing hotline call types received in the center and patient satisfaction data, they chose the five most commonly occurring symptom management calls on which to focus. These included: managing chemotherapy side effects of fatigue, appetite loss and nausea, fever, pain, and diarrhea and constipation [18]. They determined that rapid uptake and sustainability might be enhanced by embedding forcing functions, easily-accessed hyperlinks and pop-ups designed by the triage nurses themselves. For example, one attractive smart phrase feature they incorporated was that when a call was triaged, as soon as the name and date of birth were received, the EHR screen would automatically display patient-related demographics, related diagnostics, and active problem list. Triage nurses could then select the symptom(s) from the displayed list or add others, and standardized assessment cues would appear for documentation and interventions, with direction to the most appropriate levels of care. Three samples of smart phrase designs were developed and the 23 tele-triage nurses voted on their choice. It was embedded into the EMR in December, 2017 with the belief that it held much promise for improving patient satisfaction with symptom management tailored to patient’s individualized needs. It was also hypothesized that usability, documentation, quality measurement ability, and tele-triage nurse job performance would be enhanced (Table 3). 


Study Objective

Sample and Type

Design / Method


Strength of Evidence/Recommendations

Hawley, E. Loney, M, et al. (2011)

To reduce throughput through the ED for patients with febrile neutropenia through implementation of a best practice model including throughput processes and educational tools.

42 retrospective chart reviews for patients with cancer

Retrospective chart reviews of ED-door to-antibiotic times

ED door to antibiotic times improved by 22% from a mean wait time of 138 minutes to 91.6 minutes. Cancer center door to antibiotic times improved by 80% from a mean wait time of 70 minutes to 52.6 minutes.

Level VI/






1. Parallels use of CCC sepsis bundle






2. Simple assessment smart phrase needed






3. Use of scripting may be helpful in design and embedding of smart phrase.







Beaver, K, Tysver-Robinson, D, et al (2009)

To compare traditional hospital follow-up with telephone follow-up by specialist nurses after treatment for breast cancer.

374 women treated for breast cancer who were at low to moderate risk of recurrence in outpatient clinics in two hospitals in NW England

Randomized equivalence trial

1. Need for information on genetic risk remained the highest at the end of the trial.

Level II /





2. Pt satisfaction: No difference between groups initially but at the middle and end of the trial, responses was significantly more positive in the telephone group.

1. Identified useful variables to measure: general health, participants need for information, participant satisfaction, time to detection of recurrent disease





3. Difference between randomized groups was not significant for time to detection of recurrence.

2. Proactive upstream thinking.






3. Needs to have a structured approach to training, well defined.






4. To monitor the integrity of the integration, all telephone consults were recorded with consent of the women.






5. Implications for asking for more information (survivor’s group follow-ups.






6. Need to integrate patient preferences.






7. More work need to be done to identify the specific needs of patients at high risk of recurrence with subsequent adaptation of the intervention instrument.

Table 3: Evidence summary table and recommendations on roundtable


Mentored by the consulting translational nurse researcher, and the clinical manager of one hospital site as the project director, an evaluation study was designated exempt by both hospital and university institutional review boards. A repeated-measures, phased-in design was used to compare implementation outcomes at each site and overall. The research team was especially interested in evaluating changes in patient satisfaction related to symptom management before, during, and after imbedding the smart phrase into the EHR. Usability and tele-triage nurse outcomes were studied in the context of an affiliated doctoral student’s dissertation [6].


Measurement of Patient Outcomes

Patient outcomes were assessed along a 5-point trajectory of 6-month and 3-month pre-implementation, at baseline implementation, and three and six month’s post-implementation. Measured and summarized de-identified data of 1875 responses to the 2017 Lifespan Health System Outpatient Oncology Patient Satisfaction Survey data [18] and data from the center’s 2017 Lifespan EPIC Quality Database [19] were used for analysis of nurse-sensitive patient satisfaction outcomes. Seven variables related to symptom management were chosen to evaluate for changes including: patient satisfaction with managing chemotherapy side effects, fatigue management education, managing appetite loss, emotional needs addressed, home-based instructions, pain well controlled, and perceived safety and security.

Measurement of Tele-triage Nurse Outcomes

TTF theory was selected for its ability to guide evaluating smart phrase usability and potential impact on tele-triage nurse utilization and performance. (Figure 1) displays the evaluation measures used and their relationship to TTF theory as proposed [6]. 

Figure 1: Evaluation Measures and Relationship to Task Technology Fit Theory 

Six months prior to implementation, demographic data were collected on variables found to affect computer performance including age, gender, years in nursing practice, years in oncology nursing practice, years working as a tele- triage nurse, years using the EHR, and years using computer technology [25-30]. Three instruments were used to measure usability, self-perceived job performance, and utilization of the smart phrase. Previous studies supported a learning curve of six months post computer information systems training to measure these [31]. Overall smart phrase usability was measured six months following go-live implementation, using Brooks [32] 10-item, five-point Likert “System Usability Survey (SUS)” with high Cronbach’s alpha reported at .92 [33]. Differences in nurse-user self-perceptions of job performance six months before and six months afterwards was measured using Default’s (2017) 15-item “Telephone Triage Nurse Survey” (TTNS) [34].Successful content validity testing conducted by an expert panel of eight advanced practice oncology nurses rated survey items sampled from studies on best telephone triage practices [4,12] as consistent with conceptual and operational definitions of six dimensions of competency in oncology tele-triage job performance. They included: technical or psychomotor skills, knowledge of cancer symptoms and safety risk, interdisciplinary communication, confidence in one’s determination of the level of care needed, and values and attitudes on telephone triaging for symptom assessment. Inter-item internal consistency yielded a high Cronbach's alpha of 0.79. Utilization, or the extent to which nurses integrated the SmartPhrase into their work was measured using 5-point Likert Scale as a posttest only [34,35]. Based on original questionnaire, respondents reported on impact the new technology had on effectiveness, productivity, and job performance.

Data collection

After collecting all pretest patient outcome data, demographic and pretest nurse outcome data, the project manager and one advanced practice nurse from each hospital site, all highly experienced in tele-triaging and oncology nursing, trained all 23 tele-triage nurses on the smart phrase. They provided “just-in-time” coaching to provide real-time feedback during the first three months month post-implementation from January to March, 2019.


All analyses were carried out using Statistical Analysis System (SAS) software, version 9.4. Descriptive statistics were used to describe, summarize, and synthesize collected data using frequencies and percentages, means and standard deviations. Bivariate analyses were conducted to describe differences in the pre to post-test means of TTNS scores and explore the relationships among the variables including the demographic variables of years of experience and the usability score. Generalized Linear Mixed Models were used to construct hierarchical piecewise regressions to model the full variety of outcome measures at p


Patient Outcomes: Three patient satisfaction outcomes improved significantly (p>.05) from 6 months pre-implementation to 6 months after embedding the smart phrase into the EHR at all center sites. These included fatigue management, managing overall chemotherapy side effects, and safety and security felt during hospitalization. For patient satisfaction with fatigue management, a spike occurred in all sites from the moderately low 80’s percent satisfaction scores to a high of 89 three months posttest, and then leveled off to 87 percent. Satisfaction with overall chemotherapy side effect management gradually increased from 92.6 percent to a high of 94.2, leveling at 94. Patients scored satisfaction with safety and security felt during hospitalization at a moderately low 90’s percent mean score in pre implementation months and baseline at delivery, followed by a dramatic spike upward from the go-live date to a high of 97 percent 3 months post-implementation and leveled off to 96% post-implementation (Table 4). 

 Table 4: Five-point pre to post implementation trajectory of significant patient satisfaction outcomes at p

Teletriage-nurse outcomes: Descriptive statistics for the demographic variables including tele-triage nurse years of nursing experience, years in oncology nursing, years as a tele-triage nurse, experience with the EHR system, and with the system’s documentation system are summarized in (Table 5) [6]. While they tended to be well experienced in nursing and oncology, they tended to be less experienced as a tele-triage nurse and familiar with EHR systems and the hospital’s documentation system. 

Variable of Experience (years)






















Telephone triage







EHR system







Life Chart Epic







Table 5: Descriptive Statistics of Demographic Variables for all Participants (n=23)


Total mean usability across all tele-triage nurse subjects correlated to marginal acceptability at 58.06, a value lower than the SUS benchmark score of 68 [36],but still deemed usable and could be improved. The only demographic variable contributing to the direction (positive or negative aspect of usability) was nurses’ years of oncology experience. The greater the years of oncology nursing experience, the less the nurse perceived the usefulness of the tool and satisfaction with its effect on their job performance.Interestingly however, the opposite effect occurred with less experienced tele-triage nurses.They perceived greater usability and improved job performance than their more experienced counterparts (Table 6).

 Table 6: Relationship of usability to years of oncology experience 

For actual utilization, nurses thought their use of the smart phrase significantly streamlined documentation time, enabling them to complete documentation accurately within the shortest time possible (P< .0001) and they depended on it in their work routines (P< .003).


The finding that three of the nurse-sensitive measures related to patient satisfaction improved significantly (P< .05) at the three and 6-month data collection points is significant. Prior to and at the go-live implementation date, patient satisfaction with fatigue management were in the low to mid-80 percent level and did not change significantly over this six-month trajectory. Of all of the selected 7 variables of interest, fatigue management had the lowest of the pretest scores. The significant (P<.05) spike to the high of 89 and leveling off to 87 is optimistic and an important finding. Fatigue ranks as the most debilitating cancer treatment-related symptom with approximately 82-96% of patients reporting cancer-related fatigue during treatment and adversely affecting their daily activities, social relationships, mood, and quality of life [37]. However the slight decline leveling off to 87% bears caution. Monitoring at the one-year point was suggested. With patient satisfaction with chemotherapy side effect management remaining in the low 90’s throughout the 6-month pre-implementation points and gradually increasing from the go-live day to 94 percent is encouraging, though the change was not as dramatic as with fatigue management. The most dramatic spike occurred with patient’s satisfaction with safety and security felt during hospitalization. The dramatic spike upward from the go-live date to a high of 97 percent and remaining significantly higher may indicate greater confidence in the tele-triage nurses response to their hotline call for help.


The way EHR's structure and present patient information, process data and generate clinical reminders (e.g. alerts and popup messages) too often impedes healthcare provider’s time with a patient and has a direct effect on their decision-making and nurses are rarely involved in their design [38].For several years, the Agency for Healthcare Research and Quality had considered usability as one of four current Health Information Technology priorities in the United States [39], and this continues to be an issue. In sync with the literature, testing the usability of the EHR tool was recognized as critical for identifying design features of EHR systems that result in errors, poor utilization and ineffective use [40, 41].Usability of the smart phrase tool was marginally acceptable to the tele-triage nurse users and needs to be improved with the only variable contributing to the direction of usability was years of oncology experience. The finding that less experienced tele-triage nurses found smart phrase usability greater and enhanced their job performance underscores the importance of using actual end-users evaluation of technology to support their effective and efficient clinical work as demonstrated in the literature [42,43].Use of the smart phrase for less experienced oncology nurses may have functioned as a training tool for their new position in tele-triaging oncology symptom management. More experienced oncology nurses may already know the assessment process needed for each symptom patients present, and usability of the smart phrase was not equal to their level of expertise. Perhaps they did not need cues to their assessment process, and it only distracted them. However it is important to be able to easily track the success of interventions and quality metrics through documentation, other than free texting. The finding that the smart phrase tool shortened the time of nursing documentation and enabled nurses to complete tasks accurately within the shortest time possible is significant. Since tele-triage nurses generally do more documentation and process more symptom assessment information than other health care providers, using the new smart phrase may potentially directly affect their workflow and job performance on a daily basis.


Following the finding of positive outcomes related to patient satisfaction and satisfaction in less experienced tele-triage nurses, along with the decrease in documentation time, the decision was made to continue using the smart phrase while continuing to monitor patient satisfaction results.


Further translational projects in other settings may provide compelling evidence that such smart phrases may significantly increase clinician uptake, sustainability, and impact nurse-sensitive outcomes. Other smart phrases have since been developed as decision-support tools for clinicians at the cancer center, and testing is in progress.


This translational research project addressed the goal of applying the CRU model to enhance staff nurse uptake, sustainability, and improve patient outcomes related to remote telephone-triaging of symptom assessment as it has with other clinical issues. Evidence-based approaches to improve standardized telephone-triage symptom assessment, well-embedded into nursing practice, documented in the EHR, highly reliable, sustainable, and widely disseminated are needed to reduce treatment toxicity risks and enhance patient/family engagement and comfort. Significant advantages may include developing present and future clinicians competent in symptom assessment and management for cancer patients, as well as in using the evidence of research to change practice. Integrating new technology can be perceived as an onerous and frustrating experience, and so it was critical to test usability of the smart phrase tool. Dalky’s examination of changes over time will provide a deeper understanding of the evaluation of human factors related to usability. Learning more about such decision-support tools from tele-triage nurses’ viewpoint may help to determine linkages between usability and how it may impact their nursing care performance and patient outcomes. Results of this project will inform as to the usefulness of the smart phrase tool for remote symptom management. Establishing a continual evaluation and feedback loop may provide valuable insight into the design of the decision-support smart phrase tool and ultimately fit the workflow of tele-triage clinicians.


Nurses face usability challenges when new technology is introduced. This translational study is a step forward in designing and developing information processing tools that enhance clinician performance and patient outcomes. Meaningful clinician participation in the development of the standardized, evidence-based, patient-centered approach to nurses’ remote symptom management was achieved. Using the CRU Model can facilitate the integration of new knowledge both in the clinical and academic community.


  1. Tariq A, Westbrook J, Byrne M, Robinson M, Baysari M, et al. (2017) Applying a human factors approach to improve usability of a decision support system in tele-nursing. Collegian24: 27-236.
  2. Dufault M, Lessne CW (1999) Using a collaborative research utilization model to develop and test the effects of clinical pathways for pain management.Journal of Nursing Care Quality 13: 19-33.
  3. Flannery M, Phillips S, Lyons C (2009) Examining telephone calls in ambulatory oncology. Journal of Oncology Practice 5: 57-60.
  4. Gleason K, O’Neill E, Goldschmitt J, Horigan J, Moriarty L, et al. (2013) Ambulatory oncology nurses making the right call: assessment and education in telephone triage practices.Clinical Journal of Oncology Nursing 17: 335-336.
  5. Waters T, Webster J, Stevens L, Li T, Kaplan C, et al. (2015) Community oncology medical homes: physician-driven change to improve patient care and reduce costs. Journal of Oncology Practice11: 462-477.
  6. Dalky A F (2018) Evaluating Usability of an Electronic Health Record Smart phrase: Triage Nuses use and Perceptions, University of Rhode Island, Kingston, Rhode Island, USA.
  7. Murdoch J, Barnes R, Pooler J, Lattime V, Fletcher E, et al. (2015) The impact of using computer decision-support software in primary care nurse-led telephone triage: interactional dilemmas and conversational consequences. Soc Sci Med 126: 36-47.
  8. Varley A, Warren FC, Richards S, Calitri R, Chaplin K, et al. (2016) The effect of nurses’ preparedness and nurse practitioner status on triage call management in primary care: A secondary analysis of cross-sectional data from the ESTEEM trial. Int J Nur Stud 58: 12-20.
  9. Stacey D, Bakker D, Green E, Zanchetta M, Conlon M, et al. (2007) Ambulatory oncology nursing telephone services: A provincial survey. Canadian Oncology Nursing Journal17: 1-5.
  10. Hawley E, Loney M, Wiece M (2011) Development of tools and processes to improve treatment times in patients with febrile neutropenia. Clinical Journal of Oncology Nursing 15: E53-57.
  11. Maloney K, Denno M, Kide, T, McClintock K, Moore A, et al. (2013) The Oncology phone: an innovative program for the management of the oncology population in an academic medical center. Clinical Journal of Oncology Nursing 17: 387-391.
  12. Stacey D, Skrutkowski M, Carley M, Kolari E, Shaw T, et al. (2015) Training oncology nurses to use remote symptom support protocols: A retrospective pre-/post study. Oncology Nursing Forum42: 174-182.
  13. Dufault M, Bielecki C, Collins E, Willey C (1995) Changing nurses' pain assessment practice: a collaborative research utilization approach. Journal of Advanced Nursing 21: 634-645.
  14. Tracy S, Dufault M, Kogut S, Martin V, Rossi S, et al. (2006) Translating best practices in nondrug postoperative pain management. Nursing Research 55: S57-S67.
  15. Dufault M, Garman D (2007) The research roundtable: A strategy to build ongoing nursing staff competency in translating research into practice. In Relationship Based Care Field Guide. Creative Healthcare Management Inc. Minneapolis, MN.
  16. Dufault M, Duquette C, Temkin C, Lavin M, Ehmann J, et al. (2010) Translating an evidence-based protocol for nurses shift handoffs. Worldviews on Evidence-Based Nursing 7: 59-75.
  17. Lifespan Health System Outpatient Oncology Report.
  18. Lifespan EPIC Quality Database (2017).
  19. Goodhue D L, Thompson R L (1995) Task-technology fit and individual performance. MIS quarterly 19: 213-236.
  20. Janken J, Dufault M, Yeaw E (1988) Research roundtables: Increasing student/staff nurse awareness of the relevancy of research to practice. Journal of Professional Nursing 4: 186-191.
  21. Dufault M (2004) Testing a collaborative research utilization model to translate best practices in pain management. Worldviews Evid Based Nurs.1: S26-32.
  22. Polit D, Beck C (2012) Nursing research: Generating and assessing evidence for nursing practice9th Nursing research Pg 1-37.
  23. Stetler C (2001) Updating the Stetler Model of research utilization to facilitate evidence-based practice. Nursing Outlook 49: 272-279.
  24. Kuntz G, Tozer J, Snegosky J, Fox J, Neumann K (2014) Michigan oncology medical home demonstration project: first-year results. Journal of Oncology Practice 10: 294-297.
  25. Stacey D, Carley M, Kohli J, Skrutkowski M, Avery J, et al. (2014) Remote symptom support training programs for oncology nurses in Canada: an environmental scan. Can Oncol Nurs J 24: 78-88.
  26. Beaver K, Robinson DT, Campbell M, Twomey M, Williamson S, et al. (2009) Comparing hospital and telephone follow-up after treatment for breast cancer: randomised equivalence trial. BMJ 338: a3147.
  27. Goodhue D (1998) Development and measurement validity of a Task-Technology Fit instrument for user evaluation of information systems. Decision Sciences 29: 105-138.
  28. Reid C, Hurst C, Anderson D (2013) Examination of socio-demographics and job satisfaction in Australian registered nurses. Collegian 20: 161-167.
  29. Eo YS, Kim YH, Lee NY (2014) Path analysis of empowerment and work effectiveness among staff nurses. Asian Nursing Research 8: 42-48.
  30. Xiang Z, Tussyadiah I (2014) Information and communication technologies in tourism 2014. In Proceedings of the enter 2014 PhD Workshop, Springer International Publishing: ChamPg No: 1-146.
  31. Brooke J (1996) SUS-A quick and dirty usability scale. Usability evaluation in industry 189: 4-7.
  32. Bango A, Kortum P, Miller J (2008) An empirical evaluation of the system usability scale. International Journal of Human Computer Interaction 24: 574-594.
  33. Dufault M (2017) Telephone Triage Nurse Survey. Unpublished raw data.
  34. Attefalk L, Langervik G (2001) Socio-technical soft systems methodology, Master's thesis, University of Gothenburg Pg No: 1-133.
  35. Brooke J (2013) SUS-a retrospective. Journal of usability studies 8: 29-40.
  36. Gresham G, Sydney MDy, Zipunnikov V (2018) Fatigability and endurance performance in cancer survivors: Analyses from the Baltimore Longitudinal Study of Aging. Cancer 124: 1279-1287.
  37. Edwards PJ, Moloney KP, Jacko JA, Sainfort F (2008) Evaluating usability of a commercial electronic health record: A case study.International Journal of Human-Computer Studies, 66: 718-728.
  38. AHRQ (2013) Health IT for Improved Chronic Disease Management. 24: 227-236.
  39. Purc Stephenson RJ, Thrasher C (2010) Nurses’ experiences with telephone triage and advice: A meta-ethnography. Journal of Advanced nursing 66: 482-494.
  40. Page CAK, Schadler A (2014) A nursing focus on EMR usability enhancing documentation of patient outcomes. Nursing Clinics 49: 81-90.
  41. Darmon D, Sauvant R, Staccini P, Letrilliart L (2014) which functionalities are available in the electronic health record systems used by French general practitioners? An assessment study of 15 systems. Int J Med Inform 83: 37-46.
  42. Raglan GB, Margolis B, Paulus RA, Schulkin J (2015) Electronic Health Record Adoption among Obstetrician/Gynecologists in the United States: Physician Practices and Satisfaction. J Healthc Qual 39: 144-152.
  43. Topaz M, Ronquillo C, Peltonen LM, Pruinelli L, Sarmiento RF, et al. (2016) Advancing Nursing Informatics in the Next Decade: Recommendations from an International Survey. Stud Health Technol Inform 225:123–127.

Citation: Dufault M, Dalky A, Ritz J, Begnoche M, Chauvin M, Cherenzia K, Corrente J, Laquale P, Machan J (2021) Outcomes of Integrating Smart Phrase Interface Technology to Improve Cancer Symptom Management. J Pract Prof Nurs 4: 024.

Copyright: © 2021  Dufault M, et al. This is an open-access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited.

© 2021, Copyrights Herald Scholarly Open Access. All Rights Reserved!