This is an open-access article distributed under the terms of the Creative Commons Attribution License (https://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work, first published in JMIR Public Health and Surveillance, is properly cited. The complete bibliographic information, a link to the original publication on http://publichealth.jmir.org, as well as this copyright and license information must be included.
Social media is gaining recognition as a platform for delivering public health messages. One area attracting attention from public health researchers and professionals is Facebook’s advertising channel. This channel is reported to have a broad reach and generate high user engagement with the disseminated campaign materials. However, to date, no study has examined the communication process via this channel which this study aimed to address.
The specific objectives of the study were to (1) examine user engagement for a public health campaign based on the metadata provided by Facebook, (2) analyze comments generated by the campaign materials using text mining, and (3) investigate the relationship between the themes identified in the comments and the message and the sentiments prevalent in the themes that exhibited significant relationships.
This study examined a New Zealand public health pilot campaign called “Don’t Know? Don’t Drink,” which warned against drinking alcohol during pregnancy. The campaign conveyed the warning through a video and three banner ads that were delivered as news feeds to women aged 18-30 years. Thematic analysis using text mining performed on the comments (n=819) identified four themes. Logistic regression was used to identify meaning-making themes that exhibited association with the message.
The users’ engagement was impressive with the video receiving 203,754 views. The combined likes and shares for the promotional materials (video and banner ads) amounted to 6125 and 300, respectively. The logistic regression analysis showed two meaning-making themes, namely, risk of pregnancy (
The user engagement observed in this study was consistent with previous research. The numbers reported for views, likes, and shares may be seen as unique interactions over the fixed period of the campaign; however, survey research would be required to find out the true evaluative worth of these metadata. A close examination of the comments, employing text mining, revealed that the message was not accepted by a majority of the target segment. Self-identity and conformity theories may help to explain these observed reactions, albeit warrant further investigations. Although the comments were predominantly negative, they provide opportunities to engage back with the women. The one-way communication format followed in this campaign did not support any two-way engagement. Further investigation is warranted to establish whether using a two-way communication format would have improved the acceptability of such public health messages delivered via social media. The findings of this study caution using a one-way communication format to convey public health messages via Facebook’s advertising channel.
Social media websites accumulate a vast amount of user-generated content [
The use of social media as an information source for health topics has opened channels of communication between health care providers and their patients. Both individual health care providers [
Although support for using social media to communicate public health messages is gaining momentum, there are still areas that warrant further investigation. One such area is social media’s advertising channel. Social media sites are set up with the right to communicate with their users. Owners of these websites lease out this right to contact users within their system. Individuals and organizations lease out the right to disseminate messages, as news feed, to the accounts of their target audience. Researchers have successfully tested Facebook’s advertising channel for recruiting specific subjects to participate in health studies [
This paper reports the findings of a study that examined the use of Facebook’s advertising channel as a vehicle for health promotion. The study was on a New Zealand public health campaign called “Don’t Know? Don’t Drink” [
The campaign targeted women aged 18-30 years, based on evidence that these women are at risk for drinking during pregnancy [
The aim of this study was to better understand the communication process of Facebook’s advertising channel for conveying public health messages. The specific objectives of the study were to (1) examine user engagement for the video and banner ads based on metadata provided by Facebook, (2) analyze comments generated by the campaign materials using text mining, and (3) investigate the relationship between the themes identified in the comments and the message and the sentiments prevalent in the themes that exhibited significant relationships.
The metadata provided by Facebook are designated as counts of likes, shares, and views. Likes and shares are provided for all postings, whereas views are provided for videos only. By clicking the like button, Facebook users convey their approval of a posting. Facebook displays the total number of likes received for a posting. The share button allows users to share a posting with others within their network. The total number of shares for a posting is also displayed. Views indicate the number of times a video was run. Although it is hard to assign specific evaluative worth, metadata does help more generally to assess the audience’s level of engagement with a campaign.
The comments made on postings contain textual information that, when classified and analyzed, indicate how and to what extent the target audience made meaning of and reflected the campaign’s embedded message. This study used text mining to reveal the meaning-making and message reflecting themes within the comments. As all the promotional materials were designed to convey a single message, the comments received for all the postings (both the video and banner ads) were combined and analyzed. The number of comments (n=819) was sufficient for conducting a thematic analysis. The methodology employed for the thematic analysis comprised first defining the corpus of words and then carrying out text parsing, stemming, filtering, and dimensional reduction via unsupervised and supervised classification of terms into topics or themes for meaning-making and message reflection, respectively.
The extraction of comments from Facebook required expanding the conversation threads to make all the comments visible. Following this, an exact copy of the comments was captured by maintaining the format of the Facebook pages. The raw data was scrubbed down to remove all meta-information attached to the comments (eg, time stamps and likes) and redundant formats (eg, extra carriage returns, horizontal tabs, and empty rows). At the end of this scrubbing, the Facebook usernames and their comments were left in separate rows. Each row was converted into a text file containing only the name and the comment. There were as many text files as there were comments, forming the corpus of terms analyzed. The text files were imported into SAS Enterprise Miner 12.3 (SAS Institute Inc) for thematic analysis. The identifiable names attached to the comments were replaced by codes to ensure that the confidentiality of the data was maintained throughout the analyzing and reporting process so as to minimize any unintentional harm to the participants.
The aim of the text mining was to discover underlying themes in the corpus of words. The text mining steps included text parsing, text filtering, and theme generation [
The presence of the reflective message in the themes was identified using a supervised classification via a user-supplied topic-term-role based on the terms and grammar roles derived from the sponsoring stakeholder’s message. Each comment document was assigned a “message reflection theme information weight” using this method. The interval scale message measurement variable for the message reflection theme was then modified to produce a binary value based on the message reflection topic weight threshold, where “one” represented a strong evidence of the theme being present in the comments and “zero” represented otherwise. The dependent variable for this analysis was thus derived using the keywords of the message (see
The information weight threshold and binary coding modifications explained above were applied to each of the comments to produce the dependent message reflection theme variable and the independent or predictor meaning-making theme variables. The critical alpha for inferential statistics for the predictive model used was .05, for the evidence of an association to be deemed to be strongly apparent.
Message
“A woman who thinks there is a chance she may be pregnant should stop drinking alcohol until she knows she is not pregnant.”
Keywords
woman, think, chance, pregnant, stop, drink, know, not
On social media, users express their ideas voluntarily. and the bias of social desirability has a surprisingly minimal influence. Such free-flowing comments convey what is on the users’ minds more honestly [
Sentiment analysis was used to explain the outcome of the logistic regression. The themes meeting the logistic regression threshold for strong evidence for the presence of the reflective message retained in the stepwise selection process were subjected to sentiment analysis using Mathematica 10.3 (Wolfram Research, Inc., Mathematica, version 10.3) [
The metadata reported are for the four promotional materials used in the campaign, which included a video and three banner advertisements. The total number of likes, shares, and views were obtained from the promotional materials’ respective Facebook pages.
After several iterations over the number of unsupervised classification themes extracted, the best theme clarity was obtained for the singular value decomposition that extracted four themes. By using the cut-off weights, the keywords and archetypes for the themes were established. The comments above the cut-off weight formed the archetypes and the numbers for each of the theme are shown in
Metadata for the campaign posting.
Posting | Number of comments | Likes | Shares | Views |
Don’t Know? Don’t Drink (video) | 146 | 1281 | 92 | 203,754 |
Missed a pill? Maybe? (ad #1) | 330 | 2127 | 101 | N/Aa |
You used a condom, right? (ad #2) | 250 | 1714 | 69 | N/A |
Definitely, definitely not pregnant? (ad #3) | 86 | 1003 | 38 | N/A |
Don't Know? Don't Drink profile thumbnail | 7 | 0 | 0 | N/A |
Total | 819 | 6125 | 300 | 203,754 |
aN/A: not applicable.
Themes and their corresponding keywords.
Labels | Term cut-off | Keywords | Comment cut-off | Number of archetypesa | Exemplarb |
Risk of pregnancy | 0.317 | sex, pregnancy, woman, mean, contraception | 0.318 | 67 | “...No contraception is 100%. What a dumb ad” |
Alcohol and culture | 0.320 | wine, eat, raw, fish, pregnancy, | 0.331 | 69 | “Explain all the European pregnant women who drink wine on a daily basis because it’s a cultural habit...?” |
Credibility of the campaign | 0.288 | stop, stupid, risk, drink, problem | 0.278 | 49 | “What a stupid campaign, everything has ‘risks’ but this just seems like a big fat waste of time.” |
Contraceptionfailure | 0.319 | pill, miss, condom, time, period | 0.352 | 35 | “...The ad should be ‘missed a pill? wear a condom’(or probably get him to wear one hahaha)” |
aThe number of comments with meaning-making theme information weight above a critical level.
bExemplars are snippets of the archetypes with the highest information weight for that meaning-making theme.
Using logistic regression, we investigated whether the contents of the message were related to, or predicted by, the comments. The stepwise regression retained coefficient estimates for two of the themes, namely, “risk of pregnancy” and “alcohol and culture” using the critical alpha value of .05 (
The baseline odds for strong evidence in the comments that the message was received and reflected in the themes was one in three (0.33). The odds of this message reflection approximately doubled when the first meaning-making theme (risk of pregnancy) was included (see
Sentiment analysis was conducted to further illuminate the interpretation of the regression model results. Using Mathematica [
Association between the meaning-making themes and message.
Parameter | Degrees of freedom | Estimate | Standard error | Wald χ2 | Exp (Est) | |
Intercept | 1 | −1.1058 | 0.2693 | 16.86 | <.001 | 0.331 |
Risk of pregnancy | 1 | 0.7782 | 0.2629 | 8.76 | .003 | 2.177 |
Alcohol and culture | 1 | 1.1161 | 0.2588 | 18.60 | <.001 | 3.053 |
Probability of the message provoking a message-reflective response.
Parameter | Probability |
Without any meaning-making | 0.25 (0.331/1+0.331) |
Risk of pregnancy cognition | 0.42 (0.331×2.177/1+0.331×2.177) |
Alcohol and culture cognition | 0.50 (0.331×3.053)/1+(0.331×3.053) |
All related cognition | 0.69 (0.332×2.177×3.053)/1+(0.332×2.177×3.053) |
Sentiment analysis of comments in the two themes.
Meaning making themes | Positive comments |
Neutral comments |
Negative comments |
Risk of pregnancy | 15 (23) | 18 (27) | 33 (50) |
Alcohol and culture | 23 (34) | 15 (22) | 30 (44) |
Total | 38 (28) | 33 (25) | 63 (47) |
Despite the lack of empirical evidence, observed in the literature, the use of Facebook for communicating health messages is gaining popularity [
The video used in this campaign produced an impressive number of views (203,754). However, it is hard to reach a conclusion on how the message was received based on this metadata alone. Going by the definition of likes, the number of people who approved the video was less than one percent (0.63%), that is, 1281 likes given by 203,754 individuals who viewed the video (see
This study goes further than the previously reported study [
The findings from the logistic regression indicated the base level likelihood of the message being featured in the comments was comparatively small (1 in 3; see in the column Exp [Est] for the intercept in
Negative comments can influence subsequent viewers’ evaluations, as Walther et al [
Sentiment analysis was carried out for the two themes included in the logistic regression equation. The proportion of positive comments (28%— total for risk of pregnancy and alcohol and culture in
The Don’t Know? Don’t Drink campaign adopted a one-way communication format. That is, the message, encoded in promotional materials, was disseminated in one direction via Facebook’s advertising channel. The line of communication was unidirectional with no further interaction between the sender and the receivers. Evaluation of such social campaigns is carried out separately, usually by an independent agency. In the current instance, the comments provided are a first-hand assessment of the acceptability of the message. The findings from the logistic regression and the sentiment analysis revealed that the message was mostly “not” received “as intended” by those most vulnerable to an alcohol-exposed pregnancy. This study cautions the use of a one-way communication model for conveying such warning messages via Facebook’s advertising channel. At most, it could be said that the communication process was initiated. The communication effectiveness could be enhanced using a two-way communication format, which enables the promoter to respond to the negative comments. Thus, the negative comments could be seen as opportunities, both to provide scientific evidence and reinforce the message. To understand the communication process further, the testing of Facebook’s advertising channel using a two-way communication format is warranted (H5).
Facebook offers the option to moderate the comments by hiding or deleting them to prevent derogatory comments being viewed by others. The temptation to moderate the negative comments by managers of such public campaign needs to be recognized as it can impact negatively on the findings. However, in this study, as the number of negative comments was greater than the positive ones for the campaign, it is very unlikely that this was the case.
In a two-way communication, such negative comments open the channel of communication to engage back with the commenters with additional explanation and information. In doing so, concrete evidence is placed in the domain of the target audience. Hence, negative comments must be viewed as a means to an end, which is to make the current information available to the target population, thereby alleviating the knowledge gaps. Therefore, we suggest that public campaigns using social media include a protocol against moderating the comments solely based on sentiment valence so as to maximize the intervening opportunities and maintain the completeness and quality of the raw data.
The user engagement observed in this study was consistent with previous research. The examination of the comments revealed that the message was not received favorably by a majority of women. The observations made in this study provided a direction for future research, to both understand the target audience (traits of women who drink alcohol) and enhance the effective use of Facebook’s advertising channel for health promotion activities. Future investigations are required to find out whether the self-identity and conformity theories can explain why the abstinence in a pregnancy message was largely unaccepted. Although the positive comments were fewer in number compared with the negative, they also need to be studied to find out whether that attitude resulted from the campaign or they preexisted. The negative comments should be seen as an opportunity for engagement with the target population.
The one-way communication format used in the current campaign could only observe the commencement of the communication process. Future investigation is needed to find out whether a two-way communication format would help both promoters and researchers to engage back, which might improve the acceptability of public health messages delivered via social media. Finally, the observations made in this study caution against using a one-way communication format in conjunction with Facebook advertising to convey warning messages.
The authors would like to acknowledge with thanks the Health Promotion Agency of New Zealand for funding the thematic analysis of the “Don’t Know Don’t Drink” campaign.
None declared.