The internet has evolved into an increasingly popular platform for searching for health information and purchasing medications, with more people opting to turn to online marketplaces due to convenience and cost considerations. The online pharmacy market has experienced exponential growth during the past decade in parallel with the rapid proliferation of global e-commerce. The global online pharmacy market was valued at an estimated US $68 billion in 2021, with a compound annual growth rate of 16.8%, with research indicating that the internet (including social media) is now frequently used to purchase medicines online [1,2]. Properly regulated online pharmacies, often accessible via search engine results, dispense prescription and nonprescription medicines directly to patients especially benefiting individuals in remote areas and patients who are disabled or housebound. The COVID-19 pandemic further amplified behaviors associated with purchasing medicines via the internet; thus, most countries now have regulations in place to govern the delivery of medicinal products remotely.

However, the increasing global demand for online medication purchases has also attracted malicious actors, leading to the proliferation of illegal online pharmacies—websites that fail to meet national or international regulations and have not undergone regulatory review and verification. Illegal online pharmacies use extensive rogue digital marketing strategies and search engine optimization to boost their ranking and visibility on search engine results pages (SERPs) [3]. Due to the uncontrolled nature of the internet, patients often encounter both legitimate and illegitimate vendors while conducting searches for medicines online. While several national and international verification or accreditation systems exist, such as the National Association of Boards of Pharmacy (NABP)’s Digital Pharmacy Accreditation program and the “.pharmacy” domain registry (USA) [4] and the European Commission’s EU logo for online sale of medicines (EU) [5], patients and health professionals continue to have issues verifying the credibility of online pharmacy websites appearing in search engine results [6].

Illegal online vendors endanger health by selling medicines without requiring a valid prescription and supplying substandard and falsified medicines [7] that could lead to dangerous patient outcomes [3,8]. This illegal practice has broad public health consequences, including eroding trust in health care delivery, compromising pharmacy supply chain safety, and potentially contributing to antimicrobial resistance due to the presence of substandard and adulterated products [9]. Despite persistent warnings from researchers and regulators who have called for reform and enhanced monitoring, the continued online presence of illegal pharmacies remains largely unchecked. Law enforcement efforts have had limited effectiveness in keeping up with the growing number and diversity of illicit marketplaces, public awareness campaigns show limited efficacy in changing consumer behavior, and search engine providers have yet to enforce more stringent controls on their organic search results [10,11].

This lack of accountability, awareness, and inaction has facilitated the rampant growth of illicit online drug sales for a variety of therapeutic classes (eg, antibiotics, controlled substances, and weight loss drugs) [9,12,13]. A recent study revealed that compromised results redirecting to active illicit online pharmacies were present in search query results of several European countries, with the most affected regions having up to one-third of the SERP links associated with illegal online pharmacies [14]. Other recent public health threats include fake COVID-19 products offered via the internet during the pandemic [15,16]. Although no “magic bullet” exists, effective regulation of these websites likely lies in the hands of search engine providers, as these companies have effective methodologies to screen advertisements and prevent vendors of illegal products from using paid promotion for their services. However, unpaid organic results (ie, that are not sponsored ads) are seemingly uncontrolled.

Interest and commercial adoption of generative artificial intelligence (AI)–based conversational chat features and applications are rapidly expanding throughout society. Yet, improper integration of generative AI into search engine results could further complicate and exacerbate the illegal online pharmacy issue. As of June 2023, Google continued to dominate the global search market with 84.6 billion monthly visits, while Microsoft Bing was a distant second with 1.2 billion monthly visits according to web analytics data by Similarweb Ltd [17]. With the emergence of generative AI, especially after witnessing the surging popularity of OpenAI’s ChatGPT, search engine giants have rushed to integrate generative AI into their search interfaces, giving rise to Microsoft Bing’s Chat feature, also known as Microsoft Copilot, and Google’s Search Generative Experience (SGE). After Microsoft launched Bing Chat in February 2023, Bing search crossed 100 million daily active users for the first time in its history [18].

These recent developments will transform the way global users search for and interact with health information online. Large language models (LLMs) and generative AI, when implemented and used responsibly, have the potential to revolutionize search by delivering accurate, safe, and personalized results through a user-friendly experience. However, they also carry potential risks and ethical considerations, particularly when it comes to public health, as recently highlighted by the World Health Organization that has called for caution in using these technologies [19-21]. LLMs, lacking the ability to reason, may produce results with critical mistakes and have demonstrated significant drawbacks, such as generating misinformation and falsifying data, potentially leading to patient injury that in turn raises liability concerns [22] while concomitantly highlighting the need for a comprehensive framework to address present compliance and reliability issues, especially in regulated settings like health care [23]. Other published studies have examined the use, impact, and potential threat of LLMs in pharmacy education and practice (eg, answering clinical pharmacy questions), their use in medical consultations regarding drug-to-drug interactions and drug-related questions related to risk, and evaluated LLM-generated responses to prompts containing vaccine conspiracies and misconceptions [24-33]. However, no study to our knowledge has specifically evaluated LLMs in the context of popular search engine integration, and how they may generate content that could direct consumers to illegal websites selling medication online.

Hence, several questions arise that warrant further inquiry in the context of patient safety, information quality, and potential consumer exposure to harmful medication access associated with LLMs. The focus of this study is to conduct an exploratory study to identify whether these novel search tools will influence consumer interaction with the online pharmacy market and whether they will assist or potentially harm consumers by exposing them to illegal websites through SERPs. In response, this study conducts a structured comparative analysis using different prompts to conduct a comparative assessment of AI-generated recommendations of Google and Bing for hypothetical users looking to buy prescription drugs online.

A total of 262 links were provided by the generative search engine replies to our queries, with 136 generated from Google SGE and 126 from Microsoft Bing Chat. Of the links provided, 47.33% (124/262) suggested an active online pharmacy website that dispensed medications. It is important to note that a larger proportion of the results provided by both search engines did recommend legitimate pharmacies (82/262, 31.29%), with Google SGE at 25.74% (35/136) and Bing Chat at 37.3% (47/126). However, we also observed a notable presence of recommended links to illegal or unlicensed online pharmacies on both platforms. Specifically, 13.23% (18/136) of Google SGE’s responses and 19.04% (24/126) of links provided in Bing Chat’s generative replies were found to direct users to known illegal online pharmacies. (Table 1 and Figure 1 for example of Google SGE recommendation for illegal online seller of antidiabetic drug semaglutide that has been reported as counterfeited and sold online, including a recommendation to the semaspace website, which has been issued a warning letter from the US Food and Drug Administration for introducing misbranded and unapproved semaglutide and has subsequently been shut down.) The remaining 61.02% (83/136) of Google’s and 43.65% (55/126) of Bing Chat’s recommendations were for informational sites, articles, or other online sources, that is, telemedicine consultation websites, not directly selling medications to consumers.

A closer examination of the results for prescription medications queried reveals distinct differences between the 2 search engines’ generative feature recommendations. This suggests that both have likely implemented some form of additional controls to filter illegal sellers from results or that these recommendations are filtered or reviewed by other training or referenced data, although correct classification is not consistent (Multimedia Appendix 1 for additional examples of illegal sellers in recommendations). Although the overall occurrence of legitimate pharmacy websites was higher (P=.08) in Bing Chat (38/86, 44%) compared to Google SGE (29/92, 31%), the number of recommendations leading to illegal online sellers was significantly higher (P=.001) for Bing Chat (21/86, 24%) compared to Google SGE (6/92, 6%). The proportion of links to rogue websites was notably higher for the antibiotic amoxicillin (9/24, 37%) and the proton pump inhibitor omeprazole (7/19, 37%) in Bing Chat. However, Google’s generative AI search results showed an absence (0%) of illegal seller recommendations for these medications. Instead, Google SGE’s recommendations included several illegal websites (3/23, 13%) offering the sale of sildenafil or Viagra, a commonly counterfeited erectile dysfunction medication [36]. In contrast, Bing Chat appeared to exclude illegal sellers of this drug (Table 1).

Specific to controlled substance recommendations, these narcotic medications hold a high potential for abuse and dependence and are subject to special regulatory and legal requirements at the national (eg, national controlled substance acts) and international (eg, United Nation conventions and treaties) levels and are generally not available for purchase and dispensing online. Despite these prohibitions, suggestions for where to purchase controlled drugs were returned using the simple prompt used in this study, which led to a significantly higher (P=.02) number of rogue sellers in Google SGE’s suggestions (12/44, 27%) compared to 7% (3/40) from Bing Chat. Notably, for the popular anxiolytic alprazolam or Xanax, a substantially higher number of illegal pharmacy suggestions (10/20, 50%) was observed compared to legitimate pharmacies (2/20, 10%) in Google SGE results. Xanax is also a controlled substance subject to abuse and counterfeiting [37]. The results of recommendations for controlled substances carry heightened consumer risk due to the high potential for abuse and known counterfeiting of versions of these drugs laced with fentanyl, which has led to overdose deaths due to poisoning [38].

Bing Chat provides a generative response to every query and also provides sources by default for key parts of the generated response. However, these links do not always directly relate to the topic of the AI-generated text, and in some instances, these may even be contradictory. For instance, when we asked Bing Chat, “Where can I buy fentanyl online?” the generated response began with, “I'm sorry, but I cannot help you with that.” This was followed by a well-reasoned explanation that “fentanyl is highly addictive and dangerous and can cause serious harm or even death.” Subsequently, it explained that “it is illegal to buy or sell fentanyl without a prescription,” and added, “I strongly advise you to avoid buying fentanyl online or anywhere else and seek professional help if you are struggling with addiction.” Finally, Bing AI offered help in finding resources for addiction treatment (Figure 2, screenshot on the left).

This response is perfectly appropriate and demonstrates that the AI recognized the inherent danger of the situation from the user’s query and generated a sound and constructive response. This indicates that chatbots can be programmed to produce highly aligned responses reflective of public health concerns about sourcing medications online. However, it is disconcerting to note that the links provided in the “Learn more” section are not effectively monitored. The first link given to the user for this prompt led to an illegal online pharmacy (Figure 2, screenshot on the right). This is a notable weakness of Bing Chat. The majority of concerns we observed were in the hyperlinks within the generated response or recommended links below the response in the “Learn more” section. This issue could be attributed to the lack of stringent oversight over reviewing whether organic search results generated by the search engine provider include illegal sellers, which consequently surface in generative AI-related responses. This laxity allows illegal pharmacies to rank high within the organic SERPs and, in turn, find their way into the recommendations offered to users.

At the time of the study evaluation, the Google SGE was still in early experimental access in the United States and was not available in other locations. Contrary to Bing Chat, Google SGE did not generate extensive detailed generative responses to all user queries, and at times, the generative response was simply limited to “Here are some results,” followed by recommended links. As Google SGE also provides links along with its responses, and since it relies on the organic results ranking high on the SERPs to recommend links to users, it also returned questionable recommendations as observed in Bing Chat’s responses. Specifically, we encountered instances where illegal pharmacy websites were directly recommended to the user both within the generative text and in the recommended links for both platforms.