YouTube as a Potential Training Method for Laparoscopic Splenectomy
Online video-hosting platforms are important sources of information for various fields. Surgical trainees are actively using YouTube as an adjunct to their training. In this study, we assessed the accuracy of the information available on laparoscopic splenectomy in regard to quality, medical accuracy, and feasibility as a surgical training tool. On March 23, 2022, YouTube was searched using the keyword “laparoscopic splenectomy.” The video quality was scored using an author-developed arbitrary scoring system. Videos were classified as private practice (PP), secondary hospital (SH), and academic institution (AI). Of 100 videos selected, 34 were excluded, and 66 were included in the study. Of the 66 videos, 4 (6.1%) were classified as good, 51 (77.3%) as moderate, and 11 (16.7%) as poor. The mean views, likes, and dislikes did not show any significant differences among the 3 groups. Videos in the good and moderate groups were significantly longer than those in the poor group. More videos were uploaded from PP than SH or AI [45 (68.2%) versus 5 (7.6%) from SH versus 16 (24.2%) from AI]. The mean scores of AI videos were significantly higher than that of PP or SH videos (AI 11.7 ± 0.5 versus PP 10.2 ± 0.4, P = 0.034, AI 11.7 ± 0.5 versus SH 8.2 ± 0.9, P = 0.004). AI videos had higher educational value than PP or SH videos, but this difference was not recognized by viewers. AI videos can be referred to by surgical trainees as adjuncts to the training program.Objective
Methods
Results
Conclusion
YouTube, an online video-hosting platform on which users can freely upload multimedia and interact with each other, has become one of the most frequently used websites. Although YouTube is mainly used for recreational purposes, it is an important source of information in various fields. This holds true for the field of medicine as well, in which videos on various medical and surgical conditions are actively uploaded and accessed on YouTube.1 Extensive research has been performed on the quality of patient information available on YouTube on topics ranging from gallstone disease and abdominal aortic aneurysm to the COVID-19 pandemic.2–4
The principles of contemporary surgical education have evolved and are very different from those of traditional methods. Though participating in surgery as an assistant is an integral part of surgical training that can never be truly replaced, many surgical trainees are actively seeking various multimedia platforms, such as YouTube, as an adjunct to their training.5–7 Various studies have been conducted to verify the accuracy of medical videos and videos of basic surgical procedures, such as appendectomy and cholecystectomy, uploaded to YouTube.8–12
Although splenectomy is performed less frequently than appendectomy or cholecystectomy, it is an important procedure that is performed due to hematologic disease or trauma. Like many other operations, laparoscopy is increasingly applied to splenectomy.13–15 Therefore, laparoscopic splenectomy is an essential skill for surgeons. However, most residency training programs do not focus on splenectomies. In addition, as it is less frequently performed, there are fewer opportunities for trainees to observe or assist in the operation, and there are fewer training materials. Therefore, the trainees have no choice but to rely on multimedia materials, including YouTube videos. An extensive literature search did not produce any studies so far that have verified the accuracy of YouTube videos about laparoscopic splenectomy.
Therefore, in this study, we analyzed laparoscopic splenectomy videos uploaded to YouTube to assess the video quality, medical accuracy, and its feasibility as a surgical training tool.
Methods
Study design
On March 23, 2021, YouTube was searched using the keyword “laparoscopic splenectomy.” In previous studies on people’s behavior in internet search engines, it was confirmed that more than 90% of people did not proceed beyond the third results page. Considering the possibility of exclusion, 100 videos corresponding to the first 5 pages were selected and analyzed. Videos showing the surgical procedure of laparoscopic splenectomy were included. Videos other than those that showed the entire procedure, including irrelevant content, such as patient education, animation, and commercials, and videos showing other procedures, such as partial splenectomy or hand-assisted laparoscopic splenectomy, were excluded. Finally, 66 of the 100 videos were included in the study.
For each video, the length of the video, days online, sources of the videos (primary practice, hospital, or academic institution), and the number of views, dislikes, and comments were collected as data.
The institutional review board of our institution reviewed this study, and this study was exempt from approval because only public access data were used.
Outcome measurement
Because there was no previously established standard for evaluating video quality, the authors created an arbitrary scoring system based on 15 categories considered important for surgical education. Previous guidelines and literature data for laparoscopic splenectomy were used as references.16–24 The scoring system is shown in Table 1.
The scoring system has 15 categories, and the overall score ranges from 0 to 20. The following were analyzed: patient position, trocar insertion, accessory spleen examination, dissection method, demonstration of dissection of ligaments (splenocolic, gastrosplenic, and splenorenal ligaments), pancreas tail visualization, splenic pedicle dissection, splenic pedicle division, bag insertion, morcellation and removal, subtitles, narration, and video resolution. The videos were classified as follows: 0–8 points were poor, 9–14 points were moderate, and 15–20 points were good.
To maintain objectivity, 2 individual researchers scored each video once without consulting each other, and in case of a disagreement, the 2 individuals discussed their views to come to a mutually agreed upon score.
Apart from scoring the video for quality, we additionally collected data on the source of each video by checking the uploader’s account details and analyzed it as a factor. Each video was classified into private practice (PP) if it was uploaded by a single individual; secondary hospital (SH) if it was uploaded to the account of a general hospital not equipped with academic research facilities; and academic institution (AI) if the video was uploaded to the account of an institution for academic research or education, such as a university hospital or academic society.
Statistical analysis
Data analysis was performed using SPSS Statistics (Version 20.0, IBM, Armonk, New York). Continuous variables were analyzed using the Kruskal-Wallis test and post hoc analysis using Boferroni’s method. Fisher’s exact test was used to analyze the correlation between uploaded sources and video quality. Pearson’s correlation analysis was used for the correlation analysis between various variables. A weighted kappa score was used to evaluate interobserver variability.
Results
Of the 100 videos analyzed, 34 were excluded, and 66 were finally included in the study. Of the 34 excluded, 21 did not show the surgical process; 12 showed procedures other than laparoscopic splenectomy, such as partial splenectomy and hand-assisted splenectomy; and 1 was a duplicate video. A flowchart of the study design is shown in Fig. 1.
Citation: International Surgery 109, 2; 10.9738/INTSURG-D-23-00010.1
Each video was classified according to its quality and upload source. Table 2 shows the comparison and analysis of each video factor that has been classified as good, moderate, and poor. Of the 66 videos, 4 (6.1%) were classified as good, 51 (77.3%) as moderate, and 11 (16.7%) as poor. The mean score was 15.5 ± 1.0 in the good group, 10.7 ± 1.8 in the moderate group, and 7.3 ± 1.5 in the poor group; the overall mean score was 10.4 ± 2.5. The mean score showed a significant difference in each of the 3 groups, indicating that the classification system was valid. There was no significant difference between the groups in the days online, mean views, likes, dislikes, comments, and uploaded sources. The length of the video was the only factor that showed a significant difference among the groups. The videos of the good and moderate groups were significantly longer than those of the poor group.
Table 3 displays the analysis of each video factor, classified by the upload source as PP, SH, and AI. PP uploaded 45 (68.2%) videos, SH uploaded 5 (7.6%), and AI uploaded 16 (24.2%). The mean score of AI videos was significantly higher than that of SH videos (11.7 ± 0.5 versus 8.2 ± 0.9, P = 0.004) or PP videos (11.7 ± 0.5 versus 10.2 ± 0.4, P = 0.034). There were more comments on PP videos than on AI videos (2.8 ± 5.4 versus 0.0, P < 0.001). In addition, AI videos were online significantly longer than the PP videos (2878 ± 1159 versus 1542 ± 1031, P < 0.001). The mean views, likes, and dislikes did not show any significant differences among the three groups.
Fig. 2 shows a chart of the correlations between each factor. The video score did not show a significant relationship with most of the factors; the only significant correlation was the number of likes, and there was a weak positive correlation between the score and the number of likes. On the other hand, the number of views, likes, dislikes, and comments showed a strong positive correlation with each other.
Citation: International Surgery 109, 2; 10.9738/INTSURG-D-23-00010.1
To evaluate the objectivity of scoring, interobserver variance was analyzed using the weighted kappa score and was found to be 0.745, which indicated sufficient objectivity.
Discussion
Of the 66 included videos, 6.1% of all videos were classified as good, 77.3% as moderate, and 16.7% as poor quality. The number of views, likes/dislikes, comments, and upload source did not show any significant differences between the groups classified by quality. The good- or moderate-quality videos were significantly longer than the poor-quality videos. Of the included videos, 68.2% were uploaded from PP, 7.6% from SH, and 24.2% from AI. The video quality score was significantly higher in the AI group than the SH group. Although the AI videos were online longer than the PP videos, there were significantly more comments on the PP videos than on the AI videos. In addition, the correlation analysis showed that the score for video quality had a weak positive correlation with the number of likes but did not show a significant correlation with the number of views, dislikes, and comments.
Due to changes in the surgical training environment, such as duty hour restrictions, surgical education tools available on the internet are gaining popularity. WebSurg (http://Websurg.com), TVASurg (https://pie.med.utoronto.ca/TVASurg/), and MedTube (http://medtube.net) are some examples. These tools are a valuable adjunct to formal surgical training in that they are readily accessible. Also, because these platforms are dedicated to surgical training, the quality of videos are acceptable. But the biggest limitation for these platforms is that they are not well known and seldom accessed by surgical trainees. Also, the quality of videos uploaded on these sites are variable. For example, sites such as WebSurg have a rigorous peer-review system, much like the publication process of an academic paper. However, other sites have only a perfunctory process, essentially allowing anyone to upload their videos.
In the case of YouTube, there is no peer-review process, and it is commonly used by surgical trainees. YouTube is teeming with information from uploads about various topics, and the public’s access to it is increasing. Available videos on topics from various branches of the medical field are increasing, and research on their utility value has been conducted in some of these fields. In the field of general surgery, studies on the educational value of YouTube videos showing procedures such as laparoscopic appendectomy and cholecystectomy, bariatric surgery, and totally extraperitoneal and transabdominal preperitoneal hernial repairs to medical personnel have been conducted. Furthermore, studies on the educational value of YouTube videos for diseases such as gallstones and abdominal aortic aneurysm to the public and not medical staff have been conducted.2,3,8–12 However, to date, there have been no studies on the educational value of YouTube surgical videos describing laparoscopic splenectomy.
There have been many attempts to analyze the quality of YouTube videos, and 1 systematic review concluded that YouTube has a vast amount of data related to health care, but some of them are incorrect.1 Because a standard method for the analysis of YouTube videos has not yet been established, the authors have chosen an arbitrary scoring system for each study. To evaluate the educational value of a video showing a surgical procedure, it is not sufficient to simply evaluate whether the procedure was performed safely without error. A strict evaluation considering the educational value, that is, the video’s influence on the learning of the surgical technique and the reproducibility in the real world is imperative. As shown in Table 1, the videos were scored under meticulous and demanding conditions. For future research, standard guidelines for analyzing YouTube videos are necessary.
Whether it is a study evaluating the educational value of surgical videos or the accuracy of patient information videos, we found that only a few studies conclude that YouTube videos are of a high enough quality to have educational value. The results of this study on laparoscopic splenectomy videos were not significantly different from those of previous studies. The views and number of mean likes/dislikes/comments of the videos did not show a significant difference according to quality. The number of likes alone showed a weak but positive correlation with video quality, which is a meaningful result when compared with previous studies. It can be inferred that there is no or insignificant difference in the degree to which viewers watch the low- or high-quality videos, and the viewers’ preference does not significantly depend on the quality of the video.
In the analysis according to the upload source, the quality of the AI videos was significantly higher than those of the SH and PP videos. Nevertheless, the views and number of likes/dislikes did not differ depending on the upload source. This means that, although a video has more educational value when an AI uploads it, the degree of viewing and viewer preference do not reflect this. In addition, AI videos were online significantly longer than PP videos. Considering that the videos were selected in the search order after searching by keyword, it can be inferred that the turnover rate of the AI videos is not higher than that of PP videos; PP uploads videos more actively than AI.
This study had several limitations. Because the authors analyzed the uploaded videos at a specific “snapshot” point, it is thought that this does not reflect the fact that the videos on YouTube may change over time. In addition, the order of the presented videos was listed in relation to the arbitrarily chosen keyword “laparoscopic splenectomy,” which is the basic setting of YouTube; thus, the study may not reflect that the setting may be different for each user. Finally, the standard method for the analysis of a YouTube video as an educational tool has not yet been established. Hence, the use of an arbitrary scoring system for video quality analysis can be considered as one of the limitations.
Conclusion
The quality of laparoscopic splenectomy surgery videos accessible on YouTube is diverse, most being moderate in terms of educational quality. In addition, videos uploaded from AIs had higher educational value than those uploaded from SH or PP groups. However, the differences in the quality of these videos were not sufficiently recognized by users. Nevertheless, for the purpose of being an adjunct to surgical training programs, trainees should refer to videos produced by AIs. Furthermore, the establishment of a universally accepted, simple peer-review system that can filter poor-quality videos is essential.
Flowchart of video analysis.
Correlation analysis of video score and views, likes, dislikes, and comments.
Contributor Notes