This is an open-access article distributed under the terms of the Creative Commons Attribution License (https://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work, first published in JMIR Public Health and Surveillance, is properly cited. The complete bibliographic information, a link to the original publication on http://publichealth.jmir.org, as well as this copyright and license information must be included.
Between 2016 and 2017, the national mortality rate involving opioids continued its escalation; opioid deaths rose from 42,249 to 47,600, bringing the public health crisis to a new height. Considering that 69% of adults in the United States use online social media sites, a resource that builds a more complete understanding of prescription drug misuse and abuse could supplement traditional surveillance instruments. The Food and Drug Administration has identified 5 key risks and consequences of opioid drugs—misuse, abuse, addiction, overdose, and death. Identifying posts that discuss these key risks could lead to novel information that is not typically captured by traditional surveillance systems.
The goal of this study was to describe the trends of online posts (frequency over time) involving abuse, misuse, addiction, overdose, and death in the United States and to describe the types of websites that host these discussions. Internet posts that mentioned fentanyl, hydrocodone, oxycodone, or oxymorphone were examined.
Posts that did not refer to personal experiences were removed, after which 3.1 million posts remained. A stratified sample of 61,000 was selected. Unstructured data were classified into 5 key risks by manually coding for key outcomes of misuse, abuse, addiction, overdose, and death. Sampling probabilities of the coded posts were used to estimate the total post volume for each key risk.
Addiction and misuse were the two most commonly discussed key risks for hydrocodone, oxycodone, and oxymorphone. For fentanyl, overdose and death were the most discussed key risks. Fentanyl had the highest estimated number of misuse-, overdose-, and death-related mentions (41,808, 42,659, and 94,169, respectively). Oxycodone had the highest estimated number of abuse- and addiction-related mentions (3548 and 12,679, respectively). The estimated volume of online posts for fentanyl increased by more than 10-fold in late 2017 and 2018. The odds of discussing fentanyl overdose (odds ratios [OR] 4.32, 95% CI 2.43-7.66) and death (OR 5.05, 95% CI 3.10-8.21) were higher for social media, while the odds of discussing fentanyl abuse (OR 0.10, 95% CI 0.04-0.22) and addiction (OR 0.24, 95% CI 0.15-0.38) were higher for blogs and forums.
Of the 5 FDA-defined key risks, fentanyl overdose and death has dominated discussion in recent years, while discussion of oxycodone, hydrocodone, and oxymorphone has decreased. As drug-related deaths continue to increase, an understanding of the motivations, circumstances, and consequences of drug abuse would assist in developing policy responses. Furthermore, content was notably different based on media origin, and studies that exclusively use either social media sites (such as Twitter) or blogs and forums could miss important content. This study sets out sustainable, ongoing methodology for surveilling internet postings regarding these drugs.
Curbing the opioid misuse and abuse epidemic has proven to be a challenging public health problem [
The United States Food and Drug Administration (FDA) has responded by identifying 5 key risks and consequences that are part of boxed warnings for opioids: misuse, abuse, addiction, overdose, and death [
One of the more recent additions to public health surveillance of opioids is the monitoring of internet discussions on public blogs, forums, and social media [
The Researched Abuse, Diversion and Addiction-Related Surveillance (RADARS) System is a compilation of individual data collection programs that collect product-specific and geography-specific data to form a mosaic understanding of the abuse, misuse, and diversion of prescription drugs [
The purpose of this study was to characterize the trends of abuse, misuse, addiction, overdose, and death in the United States using internet posts that mention fentanyl, hydrocodone, oxycodone, or oxymorphone and to establish a sustainable ongoing methodology for surveilling internet postings regarding these drugs.
Data were collected by scraping internet posts that mentioned 1 of 4 drugs of interest: fentanyl, hydrocodone, oxycodone, or oxymorphone. These drugs were selected because they have been involved in a variety of behaviors, have been subject to differential regulation, or were frequently prescribed in the United States. Branded oxymorphone and oxycodone products have undergone increasingly restrictive regulation due to findings related to their abuse (such as Opana ER [
Data for this project were collected as part of RADARS System ongoing surveillance of the abuse, misuse, and diversion of prescription opioids. All data were collected using a web-crawling platform (Salesforce.com Inc) that scrapes data from public websites that permit content viewing by a third party. Examples of sites that permit this type of crawling include Twitter, Reddit, public blogs and forums, and comment sections on many news sites, while private sites such as personal Facebook pages, Bluelight, and other password-protected sites do not permit this type of crawling. Posts mentioning fentanyl, hydrocodone, oxycodone, or oxymorphone were identified based on specified search-string criteria (such as opioid name, associated misspellings, product names, and unique slang terms) for the 4 opioids (
The study protocol was reviewed and approved by the Colorado Multiple Institutional Review Board prior to the initiation of the RADARS System Web Monitoring Program. Since the publicly available posts were obtained through the Web Monitoring Program and are reported in an aggregated, anonymous manner, it was determined that it was not necessary to consider the Web Monitoring Program as research involving human subjects.
As part of routine web monitoring, posts were screened for predetermined exclusion criteria using a 2-step process. The scraped posts were programmatically screened for predetermined keywords; phrases associated with uninformative posts were excluded. Programmatic exclusions did not remove all uninformative posts; therefore, manual screening for exclusion was also used. The exclusion criteria (for both steps) were defined as posts occurring outside of the surveillance period; in a language other than English; originating outside of the United States; from originating sources other than social media, blogs, or forums; containing the name of a drug of interest that was used in a context unrelated to that drug; that were considered spam (unsolicited online messages); that referenced online pharmacies, news, or pop-culture with no further commentary concerning the drug of interest; or for which the coder was unable to determine a theme. For posts that met one or more exclusion criteria, only the originating posts were removed. If related posts contained informative content (such as a comment mentioning overdose appearing below a news article), then they were not excluded.
Each post was classified by origin as either
Due to the very large volume of posts collected, sampling was necessary to identify a subset of posts for manual coding. Posts were required to have occurred between January 1, 2015 and December 31, 2018. A total of 5,048,517 posts were collected for sampling. A stratified random sample without replacement and with proportional allocation was taken from the population of identified posts. Strata included both time (by week) and origin (social media or blogs and forums) of the online posts. If there were less than 2 posts that fell into a given week, that week was folded into a biweekly stratum and weights were adjusted accordingly [
A team of 3 trained coders manually reviewed the sample of posts in order to identify reasons for opioid-use outcomes (abuse and misuse), and key medical outcome measures (addiction, overdose, and death).
Definitions of terms used in the RADARS System Web Monitoring Program.
Term | Definition |
Abuse | “A mention that indicates the use of a drug to gain a high, euphoric effect or some other psychotropic effect.” |
Addiction | “A mention that indicates one or more of the following: 1) psychological or physical dependence on a drug; 2) tolerance to the psychotropic effects of a drug; 3) withdrawal effects when discontinuing use of a drug.” |
Death | “A mention that indicates a death has occurred due to a drug of interest.” |
Mention | “Any occasion of a reference to a drug or drug class that appears in a post. One post may contain multiple mentions.” |
Misuse | “A mention that indicates the improper or incorrect use of a drug for reason other than the pursuit of a psychotropic effect.” |
Overdose | “A mention that indicates the accidental or intentional overdose of a drug, using a dangerous amount of a drug (i.e. a quantity greater than recommended or generally prescribed), or use which may result in a medical intervention.” |
Post | “A single point of communication entered by one individual at one specific time point.” |
After manual coding, sampling weights were applied to calculate the estimated number and percentage of posts for each substance in the original population. Odds ratios (OR) and corresponding 95% confidence intervals were calculated from weighted logistic regression for the origin of the posts (social media versus blogs and forums) using all posts in the study period; odds ratios greater than 1.0 indicated higher odds that the post originated from social media. Interrater reliability was calculated between the 3 coders. Predefined acceptability criteria were set, and results were deemed acceptable if 3-way agreement was greater than 90% and if the average coefficient (Gwet AC1) was greater than 0.60 [
Flowchart of data cleaning process for data collected from the 1st quarter of 2015 through to the 4th quarter of 2018 where percentages represent the proportion of exclusions at each step.
Interrater reliability was acceptable according to both criteria for all the coding variables. Three-way percent agreement was high for misuse and abuse (97.8%), addiction (99.4%), overdose (99.9%), and death (99.6%). Gwet AC1 coefficient was also high for misuse and abuse (0.98), addiction (0.99), overdose (1.0), and death (1.0)
Number of posts analyzed in samples and corresponding population estimates.
Key risks | Drug Mentions | ||||||||||
Fentanyl | Hydrocodone | Oxycodone | Oxymorphone | ||||||||
Sample analyzeda (n=4649), n | Population |
Sample analyzed, n (n=8974) | Population |
Sample analyzed, n (n=8100) | Population |
Sample analyzed, n (n=3114) | Population |
||||
Abuse | 24 | 627 |
42 | 2181 |
43 | 3548 |
57 | 189 |
|||
Misuse | 130 | 41808 |
199 | 10379 |
107 | 7997 |
49 | 165 |
|||
Addiction | 73 | 4435 |
183 | 8766 |
176 | 12679 |
152 | 526 |
|||
Overdose | 271 | 42659 |
36 | 1911 |
48 | 3633 |
24 | 84 |
|||
Death | 427 | 94169 |
23 | 913 |
47 | 3291 |
37 | 125 |
aSample analyzed refers to the number of posts manually reviewed by the team of coders.
bPopulation estimate refers to the extrapolated number of posts.
Estimated number of quarterly fentanyl posts.
Estimated number of quarterly hydrocodone posts.
Estimated number of quarterly oxycodone posts.
Estimated number of quarterly oxymorphone posts.
Substantial proportions of posts discussing key risks originated outside of social media. An estimated 48.0% (95% CI 46.2%-49.8%) of oxymorphone posts, 16.8% (95% CI 15.9%-17.6%) of oxycodone posts, 18.3% (95% CI 17.5%-19.1%) of hydrocodone posts, and 14.6% (95% CI 13.6%-15.6%) of fentanyl posts originated from blogs and forums. Out of all social media posts discussing key risks, Twitter was a substantial origin. An estimated 19.9% (95% CI 18.9%-20.8%) of oxymorphone social media posts, 49.8% (95% CI 48.9%-50.7%) of oxycodone social media posts, 62.8% (95% CI 61.9%-63.7%) of hydrocodone social media posts, and 30.6% (95% CI 29.8%-31.4%) of fentanyl social media posts originated from Twitter. Odds ratios for the origin of discussion of key risks are shown in
Post origin by key risk across drugs. Odds ratios greater than 1.0 indicate higher likelihood of discussion on social media.
The purpose of this study was to demonstrate a sustainable, ongoing methodology of evaluating internet posts that mention drugs by presenting the trends of online discussion regarding abuse, misuse, addiction, overdose, and death and documenting the post origin of these key risks. The estimated numbers of posts that discussed misuse, abuse, and addiction decreased for hydrocodone, oxycodone, and oxymorphone over the surveillance period, which is not surprising since prescription guidelines have tightened the availability of these drugs [
Anonymity has a profound influence on the likelihood of revealing sensitive behaviors [
Due to the unsolicited nature of internet postings, qualitative analysis of web content can be used to identify unknown knowledge gaps in substance-abuse research. For future research efforts using the method reported here, data can be examined for polysubstance use, methods of tampering with abuse-deterrent formulations, or low-frequency side effects. Furthermore, negative outcomes are not the only topic to study. Events can stimulate online discussion that supports the proper therapeutic use of a drug or discussions can compare the efficacy of similar drug products.
One limitation of this study is that only publicly available websites were studied. Unique information likely exists on websites with policies that prevent public scraping (such as most of Facebook or Bluelight). The second limitation was the unstructured nature of the raw data and the potential ambiguity associated with manually coding these key risks, which was addressed by team meetings to ascertain group consensus. Furthermore, interrater reliability was assessed and found to be satisfactory. Finally, separating illicit from licit fentanyl was challenging, and likely much of the discussion referred to illicit instead of licit fentanyl.
Use of internet posts reveals a unique perspective to the opioid epidemic that is not found using traditional surveillance systems and can be a gateway to understanding qualitative aspects of drug use. Anonymity and the unsolicited nature of these data offer advantages to understanding emerging trends. Surveillance of diverse content providers should be used to understand how policy or other interventions are received by the broader community.
Search keyword examples.
Food and Drug Administration
Researched Abuse, Diversion and Addiction‐Related Surveillance
This work was supported by the Researched Abuse, Diversion and Addiction-Related Surveillance (RADARS) System. The RADARS System is supported by subscriptions from pharmaceutical manufacturers, government, and nongovernment agencies for surveillance, research, and reporting services. RADARS System is the property of Denver Health and Hospital Authority, a political subdivision of the State of Colorado. Denver Health retains exclusive ownership of all data, databases, and systems. No subscriber participated in the conception, analysis, drafting, or review of this manuscript.
All authors contributed equally to the work.
Authors were employed by Denver Health and Hospital Authority during this work.