The Institutional Review Board of the University of Pennsylvania reviewed this study and deemed it exempt human subjects research under Category (4) of Paragraph (b) of the US Code of Federal Regulations Title 45 Section 46.101 for publicly available data sources (45 CFR §46.101(b)(4)).

Between September 2020 and January 2021, we used the Twitter Streaming Application Programming Interface (API) to collect more than 3 million tweets containing keywords that men may include in posts reporting that they are gay, bisexual, or MSM. As a preliminary approach, we deployed handwritten, high-precision regular expressions—search patterns designed to automatically match text strings—on the 3 million tweets to filter out noise and identify actual self-reports (Multimedia Appendix 1). After automatically removing retweets and “reported speech” (eg, quotations, news headlines) [11], the regular expressions matched 8603 tweets that were posted by 6358 users geolocated in the United States [12].

In addition to tweet-based regular expressions, we also deployed handwritten regular expressions on the user profile metadata of the 3 million tweets collected from the Twitter Streaming API (Multimedia Appendix 1). The regular expressions matched the profile metadata of 4127 users geolocated in the United States [12]. After removing duplicate users from our tweet- and profile-based searches, we identified a total of 10,043 unique users. Figure 1 illustrates our automated pipeline for identifying men in the United States who have reported on Twitter that they are gay, bisexual, or MSM. To assess the extent to which they demographically represent MSM in the United States with new HIV diagnoses, we analyzed the state- and county-level geolocations [12] of these 10,043 users and drew upon a validated NLP tool [13] to automatically identify their ages.

True positives and false positives were manually distinguished by 2 annotators in a random sample of 1000 (10%) of the 10,043 users that were identified by our automated pipeline, consisting of 500 matching tweets and 500 matching profiles. True positives were defined as tweets or profiles in which the users reported that they are gay, bisexual, or MSM. Overall interannotator agreement (Cohen κ) based on independent, dual annotations for all 1000 users was 0.81, which is deemed to be “almost perfect agreement” [14]. More specifically, interannotator agreement was 0.83 for the 500 tweets and 0.79 for the 500 profiles. Upon resolving the disagreements, 417 (83.4%) tweets and 430 (86%) profiles were annotated as true positives and 83 (16.6%) tweets and 70 (14%) profiles were annotated as false positives. Based on this evaluation, our automated pipeline has an overall precision of 0.85, where precision = true positives / (true positives + false positives). Table 1 provides examples of tweets and profiles that were manually annotated as true or false positives. The majority of the profiles that were annotated as false positives were users that mentioned being transgender or nonbinary—populations that are beyond the scope of this study.

To assess the utility of our automated pipeline for identifying MSM in the United States who may be particularly at risk of acquiring HIV, we analyzed their state- and county-level geolocations and ages. We detected a US state–level geolocation for 8756 (87.6%) of the 10,043 users identified by our automated pipeline, including users from all 50 states and the District of Columbia. As Figure 2 illustrates, the largest numbers of users were detected in California, New York, Texas, Florida, Illinois, Pennsylvania, Ohio, and Georgia. We detected a county-level geolocation for 6240 (71.2%) of these 8756 users. Table 2 presents the 15 counties for which we detected at least 100 users. We detected an age of ≥13 years [10] for 4782 (47.6%) of the 10,043 users, with a mean age of 31.9 (SD 13.1) years and a median age of 29 years. Table 3 presents the age distribution, based on each user’s most recent tweet containing a self-report of age.

Our study demonstrates that gay men, bisexual men, or MSM in the United States publicly report their sexual orientation on Twitter and that these users can be accurately identified on a large scale. Moreover, among the 8756 users for which our automated pipeline detected a US state–level geolocation, 5096 (58.2%) were in the 10 states with the highest numbers of new HIV diagnoses [10]. Among the 6240 users for which a county-level geolocation was detected, 4252 (68.1%) were in counties or states considered priority jurisdictions by the Ending the HIV Epidemic initiative [15]. Furthermore, the age distribution of the users reflected the ranking of the most frequent age groups with new HIV diagnoses among MSM in the United States [10], with the 25-34 years age group first and the 13-24 years age group second. More specifically, these 2 age groups represent both the majority of the users in this study and the majority of MSM with new HIV diagnoses [10]. The mean (31.9 years) and median (29 years) ages of the users are within the age group (25-34 years) with the largest number of new HIV diagnoses, which is also the only age group in which HIV infections have increased since 2014 [10]. Therefore, our automated pipeline can be used as the basis for PrEP-related interventions targeted directly at MSM who are largely in the regions and age groups most affected by HIV in the United States, including younger men who may face challenges when discussing their same-sex sexual behaviors with their PCPs [5].

This paper presented an automated NLP pipeline that can be used to identify MSM in the United States who may be at risk of acquiring HIV, laying the groundwork for using Twitter on a large scale to directly target PrEP-related interventions at this population.