Information Quality Analysis of YouTube Videos on HIFU for Prostate Cancer: How much is too much?

Prostate adenocarcinoma is the most diagnosed cancer among men [1]. This highly prevalent disease in the localized stage is often tackled with either whole gland treatment (radical prostatectomy or radiation) vs. active surveillance [2-4]. However, in recent times there has been a push towards focal therapy with goals of eradicating significant cancer within the prostate while preserving genitourinary function [5,6]. Trans-rectal High-Intensity Focused Ultrasound (HIFU) ablation of prostate tissue) HIFU for prostate cancer, recently approved by the FDA offers patients and providers another treatment option to consider in management of this disease [7]. Thus, it is imperative that information that patients peruse be reliable and of good quality. 

One source that is often cited by patients is YouTube. YouTube, a free global video platform easily accessible to anyone with an internet connection provides a medium through which to communicate information about relevant medical procedures [8,9]. Be it patient testimonials, steps of the procedure, outcomes, etc. it has been shown that patients and even many physicians often turn to YouTube to learn more about procedures especially in the field of newer technologies [9]. Thus, it is reasonable to assume that one medium for prostate cancer patients to learn about HIFU would be through YouTube service. Because of the ability of anyone regardless of training to be able to upload videos unto YouTube without quality control or content review, this often calls into question the validity and authenticity of it with regards to patient education. That is, what kind of information is available to patients who want to learn about HIFU for prostate cancer and is this information any good? 

This study then aims to quantify and evaluate the information quality, actionability and understandability of YouTube videos that discuss various aspects of HIFU for prostate cancer treatment with goal of determining the quality of these videos as a resource for patients.

Search strategy and eligibility screening 

A YouTube search was conducted through January of 2022 using the keywords “HIFU & Prostate Cancer”. The top 39 videos were then assessed and 17 videos with greater than 1,000 views from the search result were included for evaluation and scoring. 

Variable extraction and coding process 

One author viewed each video to assess for inclusion eligibility. Inclusion criteria included: videos describing personal experiences with HIFU, HIFU Surgical approach, or general information on HIFU as treatment option for prostate cancer. Only videos in the English language were considered. Exclusion criteria included: duplicate videos, videos not relevant to HIFU such videos that discussed extensively radiotherapy or prostatectomy and videos describing HIFU for other purposes besides prostate cancer. 

Two validated scoring tools, the DISCERN instrument for assessing quality of health information and the Patient Education Materials Assessment Tool for Audiovisual materials (PEMAT-A/V) were utilized for assessing information quality, understandability, and actionability. 

DISCERN 

The DISCERN instrument was utilized to evaluate YouTube videos on the health information quality of content [10,11]. The DISCERN instrument utilizes fifteen independent questions and a summary question in order to provide a valid and reliable way of assessing the quality of written information on treatment choices for a health problem. The first 8 questions focus on the reliability and sourcing of the video information, questions 9-15 focusing on specific details on information regarding treatment choices and question 16 evaluating an overall quality rating of the video. A breakdown of DISCERN scoring is provided (Appendix 1). 

Each question from the DISCERN tool was graded on a scale of 1 to 5 (unless otherwise specified), with 1 being “no”, and 5 being “yes”, with numbers 2-4 indicating partial level of agreement. The DISCERN publication was utilized to maintain consistent grading throughout [11]. See appendix 1 for the complete list of questions asked by the DISCERN tool. 

Appendix 1: DISCERN Questions 

The following are the specific questions that the authors used as provided by the DISCERN tool. 

1. Are the aims clear? (if answer = 1, input 0 for question 2)
2. Does it achieve its aims? (if question #1 = 1, input 0)
3. Is it relevant?
4. Is it clear what sources of information were used to compile the publication (other than the author or producer)?
5. Is it clear when the information used or reported in the publication was produced?
6. Is it balanced and unbiased?
7. Does it provide details of additional sources of support and information?
8. Does it refer to areas of uncertainty?
9. Does it describe how each treatment works?
10. Does it describe the benefits of each treatment?
11. Does it describe the risks of each treatment?
12. Does it describe what would happen if no treatment is used?
13. Does it describe how the treatment choices affect overall quality of life?
14. Is it clear that there may be more than one possible treatment choice?
15. Does it provide support for shared decision-making?
16. Based on the answers to all of the above questions, rate the overall quality of the publication as a source of information about treatment choices" 

PEMAT-A/V 

PEMAT-A/V [12,13] was utilized to systematically assess videos based on their “understandability” and “actionability”. This tool provides 13 items for evaluation of understandability and 4 items for actionability. The higher the score, the more understandable or actionable the material. Videos are determined to be “understandable” if at least 70% of the understandability items are met and similarly, are deemed “actionable” if at least 70% of the actionability items are met. A breakdown of PEMAT-A/V scoring is provide (Appendix 2). 

Each question from the PEMAT-A/V tool was graded on a scale of 0 or 1 (unless otherwise specified), with 0 being “no”, and 1 being “yes”. The PEMAT-A/V publication was utilized to maintain consistent grading throughout [12]. See appendix 2 for the complete list of PEMAT-A/V questions. 

Appendix 2: PEMAT-A/V Questions 

The following are the specific questions that the authors used as provided by the PEMAT-A/V tool to evaluate understandability. 

1. The material makes its purpose completely evident.
2. The material uses common, everyday language.
3. Medical terms are used only to familiarize audience with the terms. When used, medical terms are defined.
4. The material uses the active voice.
5. The material breaks or "chunks" information into short sections.
6. The material’s sections have informative headers.
7. The material presents information in a logical sequence.
8. The material provides a summary.
9. The material uses visual cues (g., arrows, boxes, bullets, bold, larger font, highlighting) to draw attention to key points.
10. Text on the screen is easy to read.
11. The material allows the user to hear the words clearly (g., not too fast, not garbled).
12. The material uses illustrations and photographs that are clear and uncluttered.
13. The material uses simple tables with short and clear row and column headings. 

The following are the specific questions that the authors used as provided by the PEMAT-A/V tool to evaluate actionability. 

1. The material clearly identifies at least one action the user can take.
2. The material addresses the user directly when describing actions.
3. The material breaks down any action into manageable, explicit steps.
4. The material explains how to use the charts, graphs, tables, or diagrams to take actions. 

Scoring and Interrater Reliability 

Utilizing the two above evaluation tools, one author (CL) with no specific urological training viewed every video and scored each video utilizing the two validated scoring tools. A second author (KNA), with urological training scored and evaluated the videos using the aforementioned tools.

Video characteristics 

The search term “HIFU and prostate cancer” yielded mostly videos with less than 1000 views. Therefore, the addition of the search term “Prostate cancer treatment” was added to the inclusion criteria to better identify the videos that inquisitive potential patients may watch. This additional search term yielded 17 total videos that met the inclusion criteria of having over 1,000 views. 

A total of 17 videos were found to meet the inclusion criteria for scoring. Of the 17 total videos, 82% featured a medical doctor, with average views of 14,076 and runtime of 8 minutes 56 seconds. 

Clinical Characteristics 

Of the 17 videos, 70.5% reported false statements. 3 videos were of patient’s personal experience, 1 video was featured on TV Station and 13 were educational centered on lectures on HIFU for prostate cancer.

 Figure 1: Breakdown of DISCERN score between two raters. 

Understandability and Actionability - PEMAT-A/V 

Interrater reliability (IRR) between the two video reviewers was calculated in Google Sheets using percent of absolute agreement (Table 1). Total IRR for all rated videos using DISCERN was 68.75%. Understandability on PEMAT between raters was 76.5% with actionability of 33.3%. Patient education materials are actionable when consumers of diverse backgrounds and varying levels of health literacy can identify what they can do based on the information presented. 

Table 1: Scores of Understandability and Actionability of Patient Education Materials. 

Quality and Understandability - DISCERN 

The overall mean DISCERN scoring for all videos is summarized in Figure 1. Most videos did well in addressing aims, being relevant and being balanced and unbiased. However, videos scored poorly in describing alternative treatments, risks, and shared decision making.

There have been studies commenting on misinformation regarding urological conditions on social media [14]. But, to the best of our knowledge, this paper is the first of its kind analyzing the quality of information on HIFU for prostate cancer on any social media platform. 

The overall quality of videos regarding this topic were found to average. The biggest criticism is that a lot of videos were largely outdated. One video with over 5,000 views spoke of reservations with adopting HIFU largely because of lack of data regarding the treatment. However, HIFU has been widely adopted in Europe and South America for almost two decades now and there is widespread, reliable data on the efficacy of this treatment modality [15,16]. Another criticism of the videos was that they were of poor “actionability” with a score of 33.3%. It is important that medical information on social media platforms reflects the population it serves and is easily processed. Currently, the National Institute of Health (NIH) recommends that patient education materials be at a sixth grade reading level and this is important for clinicians to keep in mind when disseminating information [17]. 

This paper highlights the importance of presenting patients with the most updated information to allow them to make a well-informed decision regarding treatment choice tIt also makes the case for presenting information in a manner that is of good “actionability” and able to be understood by patients of all diverse backgrounds. It is important that any information discusses risks, benefits, alternatives as well as brief overview of the procedure in a succinct manner. However, there are several limitations that can be addressed in future studies. Although YouTube is the most widely used social media site, it would be interesting to assess the quality of information across other popular platforms such as Reddit, Twitter, etc. Another variable that could be studied would be to have patients with no medical background and various educational backgrounds representative of the population watch said videos and gauge their assessment. This will give clinicians a better view of how to better parse information regarding HIFU for prostate cancer treatment.

As of 2019, at least 72% of the population in the US was on at least one social media platform [18]. This medium is a space through which a lot of patients are getting information. The overall poor performance of YouTube videos on HIFU for prostate cancer treatment with respect to educational quality, understandability, and actionability provides clinicians a launching pad to bridge the divide and provide content that allows patients to make well-informed decisions.

No external source of funding

The authors of this manuscript have no conflicts of interest to disclose

Data cited are readily and available at: https://docs.google.com/spreadsheets/d/12PQsHuSsMlDgQkm7-IspAlH3Miw30pdFZK_YFtD-Bg8/edit#gid=782650154

1. Siegel RL, Miller KD, Fuchs HE, Jemal A (2021) Cancer Statistics, 2021. CA Cancer J Clin. 71:7-33.
2. NCCN Guideline (2019) Urothelial Carcinoma of the Prostate. Retrieved Online.
3. Bill-Axelson A, Holmberg L, Ruutu M, Garmo H, Stark JR, et al. (2005) Radical prostatectomy versus watchful waiting in early prostate cancer. The New England journal of medicine; 352: 1977-1984.
4. Thompson I, Thrasher JB, Aus G, Burnett A, Candy-Hagino ED, et al. (2007) Guideline for the management of clinically localized prostate cancer: 2007 update. J Urol 2007; 177: 2106-31.
5. Cordeiro ER, Cathelineau X, Thüroff S, Marberger M, Crouzet S, et al. (2012) High-intensity focused ultrasound (HIFU) for definitive treatment of prostate cancer. BJU International; 110: 1228-1242.
6. Ripert T, Azémar MD, Ménard J, Bayoud Y, Messaoudi R, et al. (2010) Transrectal high-intensity focused ultrasound (HIFU) treatment of localized prostate cancer: review of technical incidents and morbidity after 5 years of use. Prostate Cancer Prostatic Dis; 13: 132-137.
7. FDA Clears Focused Ultrasound System for Prostate Cancer Treatment (2015) Oncology Times; 37: 37-37.
8. Smailhodzic E, Hooijsma W, Boonstra A, Langley DJ (2016) Social media use in healthcare: A systematic review of effects on patients and on their relationship with healthcare professionals. BMC Health Serv Res; 16: 442.
9. Ventola CL (2014) Social media and health care professionals: benefits, risks, and best practices. P T.; 39: 491-520.
10. Charnock D, Shepperd S (2004) Learning to DISCERN online: applying an appraisal tool to health websites in a workshop setting. Health Educ Res; 19: 440-446.
11. Charnock D, Shepperd S, Needham G, Gann R (1999) DISCERN: an instrument for judging the quality of written consumer health information on treatment choices. J Epidemiol Community Health; 53: 105-111.
12. PEMAT Tool for Audiovisual Materials (PEMAT-A/V) (2013) Agency for Healthcare Research and Quality. Retrieved Online.
13. Shoemaker SJ, Wolf MS, Brach C (2014) Development of the Patient Education Materials Assessment Tool (PEMAT): a new measure of understandability and actionability for print and audiovisual patient information. Patient Educ Couns; 96: 395-403.
14. Loeb S, Taylor J, Borin J, Mihalcea R, Perez-Rosas V, et al. (2020) Fake News: Spread of Misinformation about Urological Conditions on Social Media. European Urology Focus; 6: 437-439.
15. Ziglioli F, Baciarello M, Maspero G, Bellini V, Bocchialini T, et al. (2020) Oncologic outcome, side effects and comorbidity of high-intensity focused ultrasound (HIFU) for localized prostate cancer. A review. Annals of Medicine and Surgery; 56: 110-115.
16. Zhou Z, Zhou Y, Yan W (2019) Re: Stephanie Guillaumier, Max Peters, Manit Arya, et al. A Multicentre Study of 5-year Outcomes Following Focal Therapy in Treating Clinically Significant Nonmetastatic Prostate Cancer. Eur Urol 2018;74:422-9. European Urology; 75: e114.
17. Weiss BD (2003) Health literacy: A manual for clinicians. Chicago, IL: American Medical Association Foundation and American Medical Association; Pages 1-51.
18. Pew Research Center Social Media Fact Sheet. Accessed: April 26, 2022.