YouTube Videos Demonstrating the Nasopharyngeal Swab Technique for SARS-CoV-2 Specimen Collection: Content Analysis

Introduction

Qualitative real-time polymerase chain reaction (RT-PCR) of nasopharyngeal secretions is the gold standard for testing respiratory viruses, including SARS-CoV-2 [1]. However, major concerns have been raised regarding false-negative rates of RT-PCR tests in community testing locations [2]. An early retrospective review of community hospital testing performed in China reported a test sensitivity of only 71% [3]. Although the false-negative results could be attributed to various reasons, including laboratory errors, patient misidentification, and inadequate collection of specimens, improper technique resulting in the swabs not reaching the target site of the nasopharynx is a potentially pervasive but modifiable error.

The trajectory from the nostril to the nasopharynx is often presumed to be along the dorsum of the nose, likely because of the visual appearance of the external nose. In reality, the correct trajectory is along the floor of the nose in the direction back toward the ear. Deviating from this trajectory can lead to pain from contacting a deviated septum or nasal turbinate or failure to reach the nasopharynx. Although the Center for Disease Control and Prevention has provided guidance regarding the proper nasopharyngeal swab (NPS) technique, vivid descriptions of painful patient experiences are currently commonplace in the media [4].

Although many health centers around the world are likely providing proper training to frontline health care workers, there is concern that improper NPS techniques for specimen collection may lead to false-negative results in RT-PCR tests [5]. This is a significant concern, as false-negative test results underestimate the prevalence of COVID-19 and give a false sense of security to patients as well as the health care workers caring for them [6]. Moreover, the use of improper NPS techniques limits public health efforts in identifying and contact tracing the spread of the virus. Thus, with the widely established use of NPS as a large-scale screening tool for COVID-19 and other respiratory viral diseases, ensuring a proper collection technique is used is essential in yielding sensitive test results [7],

Accordingly, the purpose of this study was to determine how the NPS technique for SARS-CoV-2 is instructed or demonstrated and how the NPS test is administered in real life by reviewing videos hosted on the web-based video-sharing platform YouTube (Google, LLC) [8].

Methods

Sample Size Determination

The sample size calculation was performed using the Kelsey methodology for cross-sectional study with a power of 80% and 2-sided confidence level of 0.05%; the exposure ratio was estimated to be 1:1 and odds ratio was estimated to be 3. The total estimated sample size was 64.

YouTube Database Search

YouTube is a widely used social media database of videos uploaded by the general public. Due to the broad accessibility of this database, there was no requirement for research approval by the Institutional Review Board–Human Subjects. The terms “nasopharyngeal swab,” “nasopharyngeal test,” “nasal swab,” “coronavirus swab,” “coronavirus test,”“covid swab,” and “covid test” were used to query the YouTube video database [8]. The query was filtered by setting the “sort by” filter to “by upload date” to compile all videos published from January 1, 2020, to May 15, 2020. Inclusion and exclusion criteria were defined to screen all search results. The inclusion criterion was that the NPS test is performed on screen with visualization of swab insertion into either naris. The exclusion criteria were duplicate videos, non–COVID-19 swab indication, and swab testing intended for anatomical regions other than the nasopharynx (eg, anterior nasal swabbing).

Video Evaluation and Data Collection

Three faculty rhinologists individually reviewed the selected NPS demonstration videos for swab angle and swab depth. Swab angle was assessed as either “along the nasal floor” or “not along the nasal floor.” Swab depth was assessed as either “appropriate depth” or “inappropriate depth.” All reviewers were blinded to each other’s assessments. The following ancillary video data were collected: video author type (“medical,” including physician, registered nurse, physician’s assistant, or nurse practitioner, vs “nonmedical”), operator type (“medical” vs “nonmedical”), type of video (“instructional vs “noninstructional”), specialty (“otolaryngology” vs “other”), country of origin (“United States [USA]” vs “other”), number of likes, number of author subscribers, time in nasal cavity, time at nasopharynx, and video post date relative to May 15, 2020.

Statistical Analysis

Interrater reliability among the 3 reviewers was assessed using Fleiss’ Kappa. The overall score for swab angle and swab depth for each NPS demonstration video was determined based on the majority score with agreement between at least 2 of the 3 reviewers. Video data were also compared between YouTube videos demonstrating the correct NPS technique and those demonstrating an incorrect NPS technique. Multiple linear regression analysis was performed to determine predictive variables among video data for videos demonstrating the correct NPS technique. Statistical significance was set at P=.05. All statistical analyses were performed on Microsoft Excel (Microsoft Corp.).

Results

The final qualitative analysis included 126 independent, unique videos. The video selection process, including screening for inclusion and exclusion criteria, is summarized in Figure 1.

The κ value indicating interrater reliability for the 3 reviewers was 0.66 for swab angle (P<.001; 95% CI=0.56-0.76) and 0.68 for swab depth (P<.001; 95% CI=0.58-0.78). For the assessment of swab angle, there was complete agreement among all reviewers, with all 3 scores consistent for 74.6% (94/126) of all videos. For the assessment of swab depth, there was complete agreement among all 3 reviewers for 76.1% (96/126) of all videos.

Moreover, we found that 52.3% (66/126) of all NPS demonstration videos had the correct angle, and 46% (58/126) showed appropriate depth (Figure 2). In addition, 45.2% (57/126) of all videos had both correct NPS angle and appropriate depth, whereas 46.8% (59/126) of the videos had both incorrect NPS angle and inappropriate depth. We observed concordance between the swab angle and depth (ie, correct swab angle with appropriate swab depth or incorrect swab angle with inappropriate swab depth) in 92% (116/126) of the videos. The agreement between these measures was nearly equivalent with regard to both measures being correct compared with both measures being incorrect. In the remaining approximately 8% (10/126) of the videos, 8 videos demonstrated correct swab angle but inappropriate swab depth, and the remaining 2 videos demonstrated incorrect swab angle but appropriate swab depth.

‎View this figure

‎View this figure

Table 1 compares the video data between videos demonstrating NPS with correct technique (both correct swab angle and appropriate swab depth) and those demonstrating NPS with incorrect technique (incorrect swab angle and/or inappropriate swab depth). We found that approximately half of the videos with the correct NPS technique were from a medical video author, instructional in nature, and/or of US origin. Videos demonstrating the correct NPS technique also were posted onto YouTube closer to May 15, 2020, compared to those demonstrating an incorrect technique. ENT-related providers were only found in videos demonstrating the correct NPS technique. Video viewership metrics, including number of views, likes, and author subscriber count, varied widely between videos demonstrating correct and incorrect NPS techniques. For correctly performed NPS techniques, the median time at the nasopharynx was 4 seconds. By definition, no time was spent at the nasopharynx in incorrectly performed NPS techniques, with regard to both incorrect depth and/or angle.

Multiple linear regression analysis was performed on video data for videos demonstrating the correct NPS technique as the reference dependent variable (Table 2). The correct NPS technique was associated with the NPS operator identifying as a medical professional and, additionally, as a provider within the ENT specialty. There was also a significant association between correct NPS technique and recency of video post date relative to May 15, 2020.

Discussion

Principal Findings

This study demonstrates that almost half (46.8%) of all NPS demonstration videos reviewed showed incorrect swab angle and inappropriate depth, as judged by 3 rhinologists. The noninstructional videos provide a broad vantage point of how testing is actually performed in the real world, so it is alarming to find that the proper technique may be so infrequently used. Moreover, 51% of all instructional videos demonstrated an improper technique. Although it is unknown how many of these instructional videos are actually being used by viewers for the purposes of learning the NPS technique, it still highlights the fact that those claiming to be experts, whether local, national or otherwise, may not have a complete understanding of the NPS technique. Furthermore, there were no statistically significant differences between the viewership of videos demonstrating the correct technique and those demonstrating incorrect NPS techniques. This finding is consistent with previously published work, especially in the early months of the COVID-19 pandemic showing the pervasiveness of uncredentialed, low-quality media publicly available on the internet [9]. These results are not totally unexpected given the lack of sinonasal anatomic knowledge most NPS operators have and the inherent difficulty of navigating the septum and turbinates to reach the nasopharynx. It is noteworthy that all videos of otolaryngologists performing or instructing the NPS were done correctly. These findings emphasize the onus of the otolaryngology field to educate our colleagues on sinonasal anatomy and proper NPS technique during the ongoing COVID-19 pandemic.

Although lower respiratory samples such as bronchoalveolar lavage and sputum samples have higher viral loads in patients with SARS-CoV-2, NPS testing is considered the best alternative over other minimally invasive specimen collection options such as oropharyngeal swabs, blood samples, or stool samples [1]. NPS is widely used to test for other respiratory viral infections and has supplanted nasopharyngeal aspirates for its accuracy and convenience in this setting [10]. However, poor techniques used for the NPS method may convert this test into a simple nasal swab. The NPS test is inherently uncomfortable for the patient even with good technique, and patients or the NPS operator may retract prematurely before the swab reaches the correct location and saturates with mucus. There has been limited attention paid towards the impact proper technique has on the accuracy of results in NPS testing even with regard to influenza or other respiratory viral testing.

Although a comparison of viral loads between the nasal cavity and the nasopharynx has not been reported for SARS-CoV-2, significant differences in viral loads have been demonstrated for other viruses; nevertheless, this is not direct evidence that NPS technique may affect testing accuracy [11,12]. However, NPS sampling technique has been shown to potentially affect false-negative rates in SARS-CoV-2 in a single study [13]. Ma et al [14] demonstrated significantly improved test accuracy and patient comfort when patients were tested in a supine position compared to in a sitting position. Of note, an increasing number of tests are performed via drive-through testing to provide convenience, increase throughput, and adhere to social distancing recommendations [14]. Despite the rapid adoption of this modality, there has not been substantial review of its effect on testing accuracy, and patient and operator positioning may not be optimized for proper NPS technique.

Because of these multiple unknowns in the current SARS-CoV-2 testing climate, we feel that specimen collection technique is at least one aspect that could be easily remedied. Three points need to be emphasized to frontline health care workers performing the NPS technique: trajectory angle, depth, and patient expectations. The swab should be angled to follow the floor of the nose, and the depth required to reach the nasopharynx, approximately 9-10 cm in adults, is often surprising to non-otolaryngologists [7]. In many cases, this means that almost the entire length of the swab is inserted into the nasal cavity with only a small portion left to hold outside the nose. Finally, both the patient and the operator should set proper expectations for the procedure: the NPS is uncomfortable but should not cause sharp or severe pain. Such discomfort should indicate to the operator that anatomic obstruction such as a deviated septum is occluding the pathway, and a modified trajectory or contralateral approach should then be attempted.

Study Limitations

This study has some limitations, including the lack of consistent video angle and quality. The types of videos ranged from professionally produced instructional videos to “selfies” posted by patients themselves. Although this inconsistency could potentially introduce difficulty in the judging NPS technique being demonstrated, we excluded videos that were clearly difficult to be analyzed. Nevertheless, although a moderate level of interrater agreement was demonstrated by Fleiss’ Kappa analysis, the remaining discordance may be attributable to the large variety in video quality. We believe that the inclusion of a wide array of video types would allow for a more complete view of real-life NPS testing across the globe. Additionally, there was no way to correlate proper or improper NPS technique with false-positive or false-negative testing rates. Finally, reproducibility of the results may be limited due to the involvement of only 3 reviewers, although all of them were board-certified otolaryngologists with fellowship training in rhinology.

Conclusions

The majority of NPS demonstration videos evaluated in this study used an improper technique. This technical deficiency may affect false-negative rates for SARS-CoV-2 testing. Therefore, based on these findings, we suggest that otolaryngologists should work to educate their medical colleagues and frontline health care workers who perform NPS techniques about relevant anatomy and technical considerations.