Patient-Reported Outcome Measures: The Accuracy of Patient-Reported Revision Spinal Surgery

Repeat lumbar spine operations are generally undesirable, implying progression of degenerative changes, persistent symptoms or treatment of complications [1]. Revision rates of surgical procedure in lumbar spine surgery varies in the literature between 0.5% and 24%, with recurrent disc herniation being the most common cause [2].

Relieving pain, restoring function, and improvement of the patient’s quality of life are the main objectives of intervention for different spinal pathologies [3]. Those important health aspects have traditionally been assessed by clinician-based assessments; however, these assessments are inherently biased and may not reflect patients’ perception of their health status [3]. Therefore, to assess the efficacy of various surgical treatments for lumbar spine disorders, Patient-reported Outcome Instruments/Measures (PROIs/PROMs) have to be the gold standard tool to measure outcomes of lumbar spine surgery [4].

Application of PROMs questionnaire in spine surgery have not been appraised comprehensively [5]. The purpose of this study is to evaluatethe accuracy of patient reported revision rate in spinal surgery, and to establish whether patients have undergone revision spinal surgery, and additionally whether this was performed at the same or a different spine level. To the authors’ best knowledge, this type of question in PROMs has not been investigated before. The assessment of such a question will determine its utility in future quaternaries for repeated lumbar surgeries, particularly in light of increased scrutiny of quality of care.

The data of 4,237 patients who completed The Spine Tango COMI (Combined Outcome Measure Index) questionnaire at 3 monthsafter spinal surgeryin a single tertiary referral spinal centre between August 2011 and August 2015 were included in the study. The COMI questionnaire data for these patients was accessed retrospectively to note the replies obtained from answering the question “Since the operation in our hospital, have you had any further operation (s) in your lumbar spine (back) in our or in other hospitals? (No; Yes, but at a different level of the spine; Yes, at the same level of the spine (same segment). The surgical history of the patients was accessed to compare it with the answers obtained from the patients’ questionnaire. Any intervention including revision surgery, management of complications, and spinal injection from the time of the last lumbar surgery to date of the questionnaire were noted and compared with the responses obtained from the patients in the questionnaire.

Patients’ responses were considered to be positive (Answered yes to the above-mentioned question) if they were correct regarding both the revision rate and the level of intervention (segment). Similarly, Patients’ response was considered to be negative (Answered no to the above-mentioned question) it they have not had any further lumbar spine surgery.

Patients with positive answers (Answered yes to the above-mentioned question) were further sub classified to true positive (Answered the revision rate and the level or segment correctly) and false positive (Answered either the revision rate and/or the level or segment incorrectly). Patients with negative answers (Answered no to the above-mentioned question) were further sub classified to true negative (Have no further surgery since their last lumbar spine operation) and false negative (Have had further lumbar spine surgery).

The validity of the obtained results was evaluated by calculating the sensitivity, specificity, and accuracy of data of the revision rate question mentioned above. The sensitivity, the specificity, and the accuracy of question asked were calculated using the following equations [6]:

Sensitivity = True positive/ (True positive + false negative) ^*100%.

Specificity = True negative/ (False positive + true negative) ^*100%

Accuracy of data = True positive + true negative/ (total number) ^*100%

In August 2015, the data of 4,237 patients who completed PROMs at 3 months after spinal surgery between August 2011 and August 2015 were accessed through the EPR system (Electronic Patient Records). A number of 4,133 patients reported no further surgeries. 4,128 out of 4,133 patients were truly negative (Had no further lumbar spine surgery), and 5 were false negative (As they had further surgical interventions on their lumbar spine).

104 patients reported revision surgery. 72 patients out of 104 were truly positive and described the correct segment, whereas, the remaining 32 patients were falsely positive. This is because they had no surgeries, had spinal injections or surgery at a different level.

This sensitivity of this question was found to be 93.5%, specificity 99.2%, and accuracy of the question 99.1% (Table 1).
 

Table 1

Sensitivity = [72/(72+5)]*100% = 93.5%
Specificity = [4128/(32+4128)]*100% = 99.2%
Accuracy of the question = [(72+4128)/4137]*100% = 99.1%

In the last few years, there has been a tremendous change in the way of evaluating and reporting patients’ outcomes after surgery [7]. Different PROMs that have been developed in recent years have shown to be both reliable and valid [8].

One of the main aims for the use of PROMs is to ensure that patient perspectives will be taken into account, in addition to other clinical measures of outcomes. Furthermore, PROMs will assist in determining the relative cost utility of different interventions such as rate of spinal surgery revision. This will inform commissioning decisions in health-care policies [8].

In the current study, all the 4,237 patients have filled the Spine Tango COMI questionnaire. The COMI questionnaire is a concise, multidirectional outcome instrument that was designed to evaluate patients’ outcome after a spine surgery [9,10]. This questionnaire was validated extensively in published literature for the assessment of spine surgeries [11], and it was approved to be used in the Spine Tango Framework [12]. However, the previous efforts emphasized mainly on the evaluation of the validity of COMI score (as a whole) in predicting and assessing patients’ outcome after spine surgery. Nonetheless, up to our awareness, the validity of (at least) one component of the questionnaire (the revision rate question) has yet to be assessed.

More than 96% of patients reported no further surgeries required after their initial index procedure and their answers match findings obtained from tracking their history at the EPR system from the time when the quaternaries were handed to patients and going backwards. This implies the high level of awareness among patients and their understanding of what further surgery means. For instance, a considerable percentage of this, 96%, had further spinal injections following their surgery and they did not count this as a surgical procedure. Only a small percentage, less than 1%, of patients answered the question incorrectly and it has been found to have had further lumbar spine surgeries following their index procedure. This indicates their misunderstanding and inappropriateness in replying to the asked question. However, it is still a very small percent and forms a negligible part of the overall percentage.

Approximately 69% of patients who reported to have had further surgeries were both correct with regard to the rate of revision and the level of intervention. This signify a considerable level of understanding among patients to what further surgeries means as well as their relatively high level of understanding the level of the operating segment. This encourages the use of such type of questions to be used in assessment of subjective postoperative outcomes for patients in the UK. Nevertheless, around 11% reported to have had further surgeries where found to be incorrect in one or both parts of the question asked in the questionnaire. This could indicate misunderstanding of revision surgery for example spinal injections or they did not get the segment or the level correct. To improve the specificity of such a question, explanation or rewarding of this question particularly the second part (which level) could improve its specificity.

The results of this study have shown that the accuracy of this revision surgery question in detecting patients who have no further surgical interventions after their initial operation (the question specificity) was more than 99%. This implies that this question can be applied to assess the rate of re-operation on lumbar spine following an index procedure. In other words, subjective response from patients can be considered as one of validated PROMs. Furthermore, the sensitivity of this question (its ability to accurately detect patients who had a revision surgery after their first operation) was as high as its specificity, and it was equal to 93.5%. This can be improved by raising the awareness among patients with the importance of such a question and add further explanation to the question particularly to the second part.

In this study we challenged a specific part of COMI questionnaire that is not subjective, and it needed to be answered correctly by patients to be accepted as a valid assessment tool. While most of other questions in the COMI questionnaire are subjective and could not be challenged easily (as each patient will give his own opinion based on his own impression/satisfaction with the outcomes of his surgery). This is of crucial clinical significance, as the results of the current study did show high sensitivity, specificity, and accuracy of the question assessed, implying the accuracy of the COMI questionnaire as a PROMs tool. Furthermore, the results are in consensus with the previously published literature regarding the usefulness of COMI questionnaire. Consequently, this study proves the validity of such a question to be used as part of the COMI questionnaire and one of the PROMs that should be used routinely while following patients after lumbar spine surgery.

However, this study has some limitations. It is a retrospective study. The retrospective study is exposed to misleading associations arising from the circumstances under which have been obtained [13]. Future forward studies are required to control these circumstances and improve results of such type of study. Repeated assessment of this question with a later date with some rewarding might improve further the sensitivity, specificity, and accuracy of the question.

This study demonstrates the utility of this question to assess the revision surgery rates in spinal surgery. However, with the lower than expected sensitivity, revision rates may be reported by patients as higher than they actually are.

1. Martin BI, Mirza SK, Comstock BA, Gray DT, Kreuter W, et al. (2007) Reoperation rates following lumbar spine surgery and the influence of spinal fusion procedures. Spine 32: 382-387.
2. Fritzell P, Knutsson B, Sanden B, Strömqvist B, Hägg O (2015) Recurrent versus primary lumbar disc herniation surgery: patient-reported outcomes in the Swedish Spine Register Swespine. Clin Orthop Relat Res 473: 1978-1984.
3. McCormick JD, Werner BC, Shimer AL (2013) Patient-reported outcome measures in spine surgery. J Am Acad Orthop Surg 21: 99-107.
4. Guzman JZ, Cutler HS, Connolly J, Skovrlj B, Mroz TE, et al. (2016) Patient-reported outcome instruments in spine surgery. Spine 41: 429-437.
5. Hunt KJ, Hurwit D (2013) Use of patient-reported outcome measures in foot and ankle research. J Bone Joint Surg Am 95: 118.
6. Parikh R, Mathai A, Parikh S, Sekhar GC, Thomas R (2008) Understanding and using sensitivity, specificity and predictive values. Indian J Ophthalmol 56: 45-50.
7. DeVine J, Norvell DC, Ecker E, Fourney DR, Vaccaro, et al. (2011) Evaluating the correlation and responsiveness of patient-reported pain with function and quality-of-life outcomes after spine surgery. Spine 36: 69-74.
8. Browne J, Jamieson L, Lewsey J, van der Meulen J, Black N, et al. (2007) Patient reported outcome measures (PROMs) in elective surgery. Report to the Department of Health 12.
9. Mannion AF, Elfering A, Staerkle R, Junge A, Grob D, et al. (2005) Outcome assessment in low back pain: How low can you go? Eur Spine J 14: 1014-1026.
10. Mannion A, Porchet F, Kleinstück F, Lattig F, Jeszenszky D, et al. (2009) The quality of spine surgery from the patient’s perspective: Part 2. Minimal clinically important difference for improvement and deterioration as measured with the Core Outcome Measures Index. Euro Spine J 18: 374-379.
11. Kleinstück FS, Grob D, Lattig F, Bartanusz V, Porchet F, et al. (2009) The influence of preoperative back pain on the outcome of lumbar decompression surgery. Spine 34: 1198-1203.
12. Röder C, Chavanne A, Mannion A, Grob D, Aebi M (2005) SSE Spine Tango-content, workflow, set-up. Euro spine J 14: 920-924.
13. Mantel N, Haenszel W (1959) Statistical aspects of the analysis of data from retrospective studies of disease. J Natl Cancer Inst 22: 719-748.