Journal of Ophthalmology & Clinical Research Category: Clinical Type: Research Article

Readability Assessment of Oculoplastic Surgery Patient Education Materials

Haaris M Khan1*, Ammar M Khan2 and Michael E Ashenhurst2
1 Faculty of medicine, University of British Columbia 317 - 2194 Health Sciences Mall, Vancouver, BC V6T 1Z3, Canada
2 Department of ophthalmology, University of Calgary, Calgary, Canada

*Corresponding Author(s):
Haaris M Khan
Faculty Of Medicine, University Of British Columbia 317 - 2194 Health Sciences Mall, Vancouver, BC V6T 1Z3, Canada
Tel:+1 6043767175,
Email:haarism@student.ubc.ca

Received Date: Apr 29, 2021
Accepted Date: May 06, 2021
Published Date: May 13, 2021

Abstract

Purpose: The internet has become a primary resource for many patients to inform themselves on numerous medical conditions and associated treatments. Furthermore, an individual’s health literacy remains an important indicator of their health status. Studies have found the readability of adult patient education material in various medical specialties to be too high for the average adult. As such, we sought to investigate the readability of online patient education material related to oculoplastic surgery. 

Study Design: Descriptive study 

Methods: The terms blepharoplasty, ptosis, ectropion, entropion, dacryocystorhinostomy, nasolacrimal duct obstruction, dry eye, chalazion, lacrimal gland tumor, orbital tumor were inputted into Google’s search engine and the first 25 eligible websites for each term were analyzed. A total of 250 websites were analyzed using five different readability formulas to evaluate the grade level and difficulty of the websites. These formulas included the The Flesch-Kincaid formula (FKG), Flesch Reading Ease (FRE) score, Gunning-Fog Score (GFS) and Coleman-Liau Index (CLI), and the Simple Measure of Gobbledygook (SMOG). 

Results: The number of websites that met the recommended reading level ranged from 0% to 8.8% depending on the formula used. The mean and standard deviation for the average grade level included an FKG score of 9.03±1.99, GF score of 9.11±2.66, SMOG score of 10.30±1.32, CLI score of 11.01±1.97, and an FRE score of 43.44±14.83. SMOG found no websites to be of the recommended reading level while CLI found only 1. Websites found to be of a reading level of grade 11 to college higher (college level) ranged from 28.8% for FKGL, 31.2% for GF, 40.0% for SMOG and 56.0% for CLI. 

Conclusion: Our study found that the majority of websites analyzed contained patient education material that was at a reading level much higher than recommended for the general public. Difficult patient education material can lead to misinterpretation of health information which can negatively affect patient care. We suggest a multifaceted approach in order to ensure patients are able to comprehend complex medical information and feel comfortable making decisions about treatment options.

Keywords

Health literacy; Oculoplastics; Patient education; Readability

Introduction

An individual’s health literacy can have a direct impact on their quality of life, with lower healthy literacy being associated with poor health outcomes [1].The Internet has revolutionized the way information is disseminated and consumed. A significant number of patients are now turning to the Internet before their physician for medical advice and treatment options [2]. However, the content found online may not only be false, but may also lead to misinformation due to the difficulty of the reading material. Many health websites are often riddled with medical jargon that can cause unnecessary patient anxiety and confusion. The American Medical Association (AMA) recommends that the reading level of patient education material should be no higher than the sixth grade in order for the general public to understand and comprehend the information [3]. Unfortunately, studies have been done to investigate the readability of health information online and the results are often unsatisfactory [4-8]. Readability of material can be assessed using a variety of formulas that assign a score based on sentence length, complexity of words, syllables and various other factors. 

In this study, we sought to investigate the readability of online patient information on terms related to oculoplastics. The field of oculoplastics encompasses a number of surgical procedures and as far we know, there have not been any studies investigating the readability of a variety of oculoplastics terms. Many of these procedures are quite common such as blepharoplasty which is a common surgical procedure done to modify the contour and configuration of the eyelids and often involves removing redundant fat, muscle, and skin [9]. There are a number of potential complications and risks involved with any of these procedures, and thus it is important that patients are accurately informed [9]. Misinformation can lead to increased stress before surgery which has been associated with poorer surgical outcomes and low postoperative wellbeing [10]. Therefore, the purpose of our study was to investigate whether the readability of online patient information on terms related to oculoplastics is of an appropriate reading level for the general public.

Methods

The terms blepharoplasty, ptosis, ectropion, entropion, dacryocystorhinostomy, nasolacrimal duct obstruction, dry eye, chalazion, lacrimal gland tumor and orbital tumorwere inputted into Google’s search engine and the first 25 eligible websites were analyzed while any websites that were non-operational were excluded. A total of 250 websites were analyzed and the search was conducted on January 25, 2021. Websites that were video links, non-functional, duplicates, irrelevant to oculoplastics or review pages for clinics were excluded from the search. 

The readability of websites was assessed using five formulas that are displayed in table 1. The Flesch-Kincaid formula (FKGL) is a formula that estimates grade level based on an equation that includes total words, total sentences and syllables [11]. Similarly, the Flesch Reading Ease (FRE) score also uses similar variables but with different weighting [12]. The FRE score is a 100-point scale with higher scores being associated with text that is more understandable. The Gunning-Fog Score (GFS) is an equation based on similar variables, but also takes into account the complexity of the words [13]. Complexity is based on whether or not the word contains three or more syllables. The Coleman-Liau Index (CLI) considers the average number of letters and sentences per 100 words and does not account for syllables [14]. Finally, the Simple Measure of Gobbledygook (SMOG) index is calculated based on the number of polysyllabic words in three ten-sentence samples [15]. Theses formulas have been commonly used in previous studies that have investigated readability, and are considered to be good measures of readability [4,7,8,16]. Aside from FRE which uses a 100-point scale, the remaining equations all produce a value that indicates the US academic grade level necessary to comprehend the material. In order to improve accuracy, an online readability calculator recommended by the National Institutes of Health (NIH) was used to evaluate the websites using the five formulas [8,17,18]. The program was configured to omit navigation titles, headers and footers from each website in order to increase accuracy. Descriptive statistics and further analysis were conducted using IBM SPSS Statistics version 23 and Microsoft Excel.

Table 1: Readability formulas used for evaluation of websites.

L = Avg. number of letters/100 words, S = Avg. number of sentences/100words.

Results

The mean and standard deviation for the average grade level included an FKG score of 9.03±1.99, GF score of 9.11±2.66, SMOG score of 10.30±1.32 and a CLI score of 11.01±1.97 (Figure 1). The mean FRE score was 43.44±14.83 with scores less than 60 being classified as “difficult”(Kher et al., 2017) 91% of websites analyzed by FRE were found to have a score less than 60. “Nasolacrimal duct obstruction” was found to have the highest average grave level of 10.73 while “dry eye” was found to have the lowest average grade level at 8.57 (Table 2). The number of websites that met the recommended reading level of below the sixth grade ranged from 0% for SMOG, 0.4% for CLI, 5.2% for FKGL and 8.8% for GF (Table 3). SMOG found no websites to be of the recommended reading level while CLI found only 1. Websites found to be of a reading level of grade 11 or higher (ranged from 28.8% for FKGL, 31.2% for GF, 40.0% for SMOG and 56.0% for CLI (Table 3). 

 Figure 1: Mean & SD of websites evaluated. Graphical representation of the grade level scored by the various readability formulas. 

Search Term

Flesh Reading Ease

Flesch Kincaid Grade Level

Gunning Fog Score

Coleman Liau Index

SMOG

Index

Blepharoplasty

47.45 ± 8.72

8.45±1.20

9.46±2.68

10.62±1.45

10.28±1.11

Ptosis

44.20± 9.09

8.93±1.23

9.11±2.55

10.87±1.47

10.00±1.11

Ectropion

43.90±9.46

8.86±1.25

8.55±2.68

10.58±1.48

9.92±1.11

Entropion

40.62±9.58

9.06±1.25

8.54±2.68

11.09±1.48

10.18±1.11

Dacryocystorhinostomy

43.44±9.82

9.21±1.28

8.78±2.60

11.32±1.63

10.74±1.21

Nasolacrimal Duct Obstruction

39.30±9.81

9.90±1.28

9.83±2.60

12.37±1.62

10.83±1.21

Dry Eye

7.68±10.04

7.06±1.29

7.58±2.57

9.78±1.63

9.87±1.22

Chalazion

44.27±9.31

8.78±1.25

9.74±2.57

9.77±1.61

9.80±1.22

Lacrimal Gland Tumor

38.33±9.35

9.64±1.33

9.60±2.47

11.39±1.67

10.77±1.27

Orbital tumor

35.09±9.35

10.37±1.34

9.54±2.52

12.32±1.64

10.61±1.29

Table 2: Mean and standard deviation of readability scores.

Grade Level

Percentage of Websites Per Readability Formula

 

FKGL

GF

 

CLI

SMOG

Grade 6 or below

5.2%

8.8%

 

0.4%

0%

Grades 7-10

66.0%

60.0%

 

43.6%

60%

Grade 11 or higher

28.8%

31.2%

 

56.0%

40%

Table 3: Readability scores based on grade level.

FKGL = Flesch-Kincaid Grade Level; GF = Gunning Fog Score; CLI = Coleman Liau Index; SMOG = Simple Measure of Gobbledygook

Discussion

With the advent of the Internet, information access and dissemination has been completely redefined. However, this information can only be widely beneficial if the material is understandable for majority of the population. Our study found very few of the websites evaluated to be of the sixth grade or lower as recommended by the AMA. Although that is a lower bar, there was still a large percentage of websites found to be of a reading level of grade 11 or even college which is quite alarming. Overall, these findings depict a clear barrier when it comes to educating patients on health-related material. Information that is presented in a difficult writing style with a plethora of medical terminology has limited utility and may in-fact be counterproductive. Instead of informing patients, complex material may cause undue stress and confusion. As such, patients may misunderstand information and consequently avoid treatment or professional advice. In a study of over 1,000 physicians, 85% of them noted that patients came to their visit with information they found online and a considerable proportion of these physicians found this to not be helpful [19]. There is likely little consideration given to the readability of websites due to lack of resources, money, absence of time, awareness and practicality. Moving forward, an effort needs to be made to inform all health-related websites about the potential ramifications of complex reading material. Although there are many formulas, research has indicated that the SMOG index may be the most suitable formula for health related material based on its consistency and use of recent validation criteria [16]. Therefore, a simple evaluation using SMOG or any other preferred readability formula can provide great insight into potential readability pitfalls. A common criticism against readability assessments in that reducing the difficulty of text can over-simplify the information and thus hinder the accuracy of the information [3,4]. However, based on evidence, it is clear that health literacy positively impacts health outcomes, and thus improving reading ease will likely be more beneficial to patients [20,21]. 

In our case of oculoplastics, appropriate health information can help mitigate preoperative stress and confusion. Although readability has been assessed in other specialities including ophthalmology, to our knowledge this is the only study to evaluate the readability of terms related to oculoplastics and oculoplastic surgery using five different assessment tools [6-8,22]. Since many of these oculoplastic procedures are fairly common, it would be well-advised for oculoplastic surgeons to screen the information they give to patients to match their level of reading comprehension. Some limitations to our study include the use of a single search engine and the number of websites evaluated. 

Given the large amount of health information found online, it is exceedingly difficult to assess all material for readability. However, it would be greatly beneficial to increase awareness among healthcare providers about the potential challenges that are associated with complex material that patients are not able to comprehend. By matching the difficulty of information to a patient’s level of health literacy, one can expect considerable benefits in communication and patient satisfaction. Overall, with a multifaceted approach to patient care that considers health literacy, physicians can make great strides in improving clinical outcomes.

Funding

This research did not receive any specific grant from funding agencies in the public, commercial, or not-for-profit sectors.

Conflict of Interest

No conflicting relationship exists for any authors.

Ethical Considerations

This study adhered to the Declaration of Helsinki. The UBC Clinical Research Ethics Board was contacted, and this study did not require IRB approval.

References

  1. Wang C, Li H, Li L, Xu D, Kane RL, et al. (2013) Health literacy and ethnic disparities in health-related quality of life among rural women: Results from a Chinese poor minority area. Health Qual Life Outcomes 11: 153.
  2. Gualtieri LN (2009) The doctor as the second opinion and the internet as the first. Proceedings of the 27th International Conference Extended Abstracts on Human Factors in Computing Systems - CHI EA ’09 Pg no’s: 2489-2498.
  3. Weiss BD, Schwartzberg JG, Davis TC, Parker RM, Williams MV, et al. (2003) Health Literacy: A Manual for Clinicians. Chicago: American Medical Foundation and Americal Medical Assocaition.
  4. Badarudeen S, Sabharwal S (2010) Assessing readability of patient education materials: Current role in orthopaedics. Clin Orthop Relat Res 468: 2572-2580.
  5. Betschart P, Abt D, Schmid HP, Viktorin P, Langenauer J, et al. (2018) Readability assessment of commonly used urological questionnaires. Investig Clin Urol 59: 297-304.
  6. Fitzsimmons PR, Michael BD, Hulley JL, Scott GO (2010) A readability assessment of online Parkinson’s disease information. J R Coll Physicians Edinb 40: 292-296.
  7. Freda MC (2005) The readability of American Academy of Pediatrics patient education brochures. J Pediatr Health Care 19: 151-156.
  8. Kher A, Johnson S, Griffith R (2017) Readability Assessment of Online Patient Education Material on Congestive Heart Failure. Adv Prev Med 2017: 9780317.
  9. Oestreicher J, Mehta S (2012) Complications of Blepharoplasty: Prevention and Management. Plast Surg Int 2012: 252368.
  10. Wilson CJ, Mitchelson AJ, Tzeng TH, El-Othmani MM, Saleh J, et al. (2016) Caring for the surgically anxious patient: a review of the interventions and a guide to optimizing surgical outcomes. Am J Surg 212: 151-159.
  11. Flesch R (1948) A new readability yardstick. J Appl Psychol 32: 221-233.
  12. Kincaid J, Fishburne R, Rogers R, Chissom B (1975) Derivation of New Readability Formulas (Automated Readability Index, Fog Count And Flesch Reading Ease Formula) For Navy Enlisted Personnel. Institute for Simulation and Training.
  13. Gunning R (1968) Technique of Clear Writing (Revised Ed). New York: McGraw-Hill, USA.
  14. Coleman M, Liau TL (1975) A computer readability formula designed for machine scoring. Journal of Applied Psychology 60: 283-284.
  15. McLaughlin GH (1969) SMOG Grading-a New Readability Formula. Journal of Reading 12: 639-646.
  16. Wang LW, Miller MJ, Schmitt MR, Wen FK (2013) Assessing readability formula differences with written health information materials: Application, results, and recommendations. Res Social Adm Pharm 9: 503-516.
  17. Get Your Readability Score | Flesch-Kincaid + unique readability tools – Readable.
  18. How to Write Easy-to-Read Health Materials: MedlinePlus, USA.
  19. Murray E, Lo B, Pollack L, Donelan K, Catania J, et al. (2003) The impact of health information on the Internet on health care and the physician-patient relationship: National U.S. survey among 1.050 U.S. Physicians. J Med Internet Res 5: 17.
  20. Bill-Harvey D, Rippey R, Abeles M, Donald MJ, Downing D, et al. (1989) Outcome of an osteoarthritis education program for low-literacy patients taught by indigenous instructors. Patient Educ Couns 13: 133-142.
  21. Fouad MN, Kiefe CI, Bartolucci AA, Burst NM, Ulene V, et al. (1997) A hypertension control program tailored to unskilled and minority workers. Ethn Dis 7: 191-199.
  22. Edmunds MR, Barry RJ, Denniston AK (2013) Readability assessment of online ophthalmic patient information. JAMA Ophthalmol 131: 1610-1616.

Citation: Khan HM, Khan AM, Ashenhurst ME (2021) Readability Assessment of Oculoplastic Surgery Patient Education Materials. J Ophthalmic Clin Res 8: 080.

Copyright: © 2021  Haaris M Khan, et al. This is an open-access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited.


Herald Scholarly Open Access is a leading, internationally publishing house in the fields of Sciences. Our mission is to provide an access to knowledge globally.



© 2024, Copyrights Herald Scholarly Open Access. All Rights Reserved!