This paper is in the following e-collection/theme issue:

Infoveillance, Infodemiology, Digital Disease Surveillance, Infodemic Management (1041) Digital Mental Health Interventions, e-Mental Health and Cyberpsychology (1514) Infodemiology and Infoveillance (1349) Depression and Mood Disorders; Suicide Prevention (1363)

Published on in Vol 8, No 5 (2022): May

Preprints (earlier versions) of this paper are available at https://preprints.jmir.org/preprint/31800, first published .
Exploring the Risk of Suicide in Real Time on Spanish Twitter: Observational Study

Exploring the Risk of Suicide in Real Time on Spanish Twitter: Observational Study

Exploring the Risk of Suicide in Real Time on Spanish Twitter: Observational Study

Authors of this article:

Claudia García-Martínez1 Author Orcid Image ;   Bárbara Oliván-Blázquez2 Author Orcid Image ;   Javier Fabra3 Author Orcid Image ;   Ana Belén Martínez-Martínez4 Author Orcid Image ;   María Cruz Pérez-Yus2 Author Orcid Image ;   Yolanda López-Del-Hoyo2 Author Orcid Image
Article Authors Cited by (3) Tweetations (11) Metrics

    Original Paper

    1Department of Psychology and Sociology, University of Zaragoza, Zaragoza, Spain

    2Department of Psychology and Sociology, University of Zaragoza, Institute for Health Research Aragón, Zaragoza, Spain

    3Department of Computer Science and Systems Engineering, Aragón Institute of Engineering Research, University of Zaragoza, Zaragoza, Spain

    4Department of Nursing and Physiatry, Institute for Health Research Aragón, University of Zaragoza, Zaragoza, Spain

    Corresponding Author:

    Yolanda López-Del-Hoyo, PhD

    Department of Psychology and Sociology

    University of Zaragoza

    Institute for Health Research Aragón

    Pedro Cerbuna, 12

    Zaragoza, 50009

    Spain

    Phone: 34 630030125

    Email: yolandal@unizar.es


    Background: Social media is now a common context wherein people express their feelings in real time. These platforms are increasingly showing their potential to detect the mental health status of the population. Suicide prevention is a global health priority and efforts toward early detection are starting to develop, although there is a need for more robust research.

    Objective: We aimed to explore the emotional content of Twitter posts in Spanish and their relationships with severity of the risk of suicide at the time of writing the tweet.

    Methods: Tweets containing a specific lexicon relating to suicide were filtered through Twitter's public application programming interface. Expert psychologists were trained to independently evaluate these tweets. Each tweet was evaluated by 3 experts. Tweets were filtered by experts according to their relevance to the risk of suicide. In the tweets, the experts evaluated: (1) the severity of the general risk of suicide and the risk of suicide at the time of writing the tweet (2) the emotional valence and intensity of 5 basic emotions; (3) relevant personality traits; and (4) other relevant risk variables such as helplessness, desire to escape, perceived social support, and intensity of suicidal ideation. Correlation and multivariate analyses were performed.

    Results: Of 2509 tweets, 8.61% (n=216) were considered to indicate suicidality by most experts. Severity of the risk of suicide at the time was correlated with sadness (ρ=0.266; P<.001), joy (ρ=–0.234; P=.001), general risk (ρ=0.908; P<.001), and intensity of suicidal ideation (ρ=0.766; P<.001). The severity of risk at the time of the tweet was significantly higher in people who expressed feelings of defeat and rejection (P=.003), a desire to escape (P<.001), a lack of social support (P=.03), helplessness (P=.001), and daily recurrent thoughts (P=.007). In the multivariate analysis, the intensity of suicide ideation was a predictor for the severity of suicidal risk at the time (β=0.311; P=.001), as well as being a predictor for fear (β=–0.009; P=.01) and emotional valence (β=0.007; P=.009). The model explained 75% of the variance.

    Conclusions: These findings suggest that it is possible to identify emotional content and other risk factors in suicidal tweets with a Spanish sample. Emotional analysis and, in particular, the detection of emotional variations may be key for real-time suicide prevention through social media.

    JMIR Public Health Surveill 2022;8(5):e31800

    doi:10.2196/31800

    Keywords

    suicide; prevention; social media; Twitter; emotional analysis; eHealth; big data; content analysis; emotional content; risk factors; mental health; public health; suicide prevention 


    As the cause of more than 800,000 deaths every year, suicide has become a global public health priority. It is the second leading cause of death in young people aged 15 to 29 years, and for every death, it is estimated that there are 20 other suicide attempts [1]. In Spain, suicide has been the main cause of unnatural death since 2012 [2].

    According to the World Health Organization [3], suicidal behavior refers to a range of behaviors that includes thinking about suicide (or ideation), planning to commit suicide, attempting suicide, and suicide itself.

    Until a few decades ago, research efforts have been focused on curbing suicide deaths by trying to predict their occurrence. This predictive approach consisted of semistructured risk assessment using lists of risk factors and sometimes included suicide risk questionnaires or scales to express risk as low, moderate, or high [4,5]. Because suicide deaths are statistically a rare event, it has been difficult to develop sensitive tools with sufficient predictive value [6]. Recent reviews [5,7-10] of these models advocate a shift from models based on suicide prediction to those that emphasize assessment and management of the risk of suicide by identifying variables related to suicide behavior and stratifying risk in terms of severity and temporality.

    Suicidal behavior has been consistently found to be associated with emotional states such as depression and hopelessness [11,12]. Bryan and Rudd [9] collected different variables that have been empirically demonstrated to be essential for risk assessment: predisposition to suicidal behavior (ie, psychiatric diagnoses, previous suicidal behavior), identifiable precipitants or stressors (ie, significant loss, relationship instability), a patient’s symptomatic presentation (eg, anhedonia, low self-esteem, sadness, dyssomnia, fatigue), presence of hopelessness, nature of suicidal thinking (eg, ideation, suicidal plan, lethality of means, explicit suicidal intent), impulsivity and self-control, and protective factors (eg, social support, life satisfaction). Emotional dysregulation seems to be also an important predictor of suicidal outcomes [13,14].

    To improve accuracy in risk evaluations, ecological momentary assessment has been used to study suicidal behavior [15], which involves repeated sampling of people's behavior in real time in their natural environments, now typically collected via smartphones. This approach attempts to minimize recall bias and maximize ecological validity. Recent research using mobile phone–based momentary ecological assessments showed that suicidal ideation varied over short periods of time [16], indicating that real-time assessments and ecological validity could be a crucial approach for suicide prevention.

    With more than 3.8 billion users around the globe [17], social media has transformed the world. People express their thoughts and emotions through social media [18]. These new forms of social interaction have been linked to suicidal behavior, nevertheless, recent studies [19] have highlighted the potential for social media to offer assistance in suicide prevention.

    Twitter currently has 340 million users [20] who, in microblogging format, communicate what they are thinking or doing at a particular moment publicly with a limited number of characters. People express suicidal tendencies on Twitter [21], and although there is a support mechanism among users, this system is not automatic or in real time.

    According to Christensen [22], web-based interventions for suicide prevention have focused mainly on three directions: (1) web-based screening for suicidality, (2) web-based therapeutic interventions for suicide prevention, and (3) real-time identification of individuals at risk, either by people or by computer language processing systems.

    The use of social media in the real-time detection of mental health has already been proven. Specifically, Twitter has been proven useful in predicting depression [23-25], postpartum depression [24], and even posttraumatic stress disorder [26,27]. Machine learning algorithms have been used to assess the risk of suicide and identify suicidal individuals. Automatic machine learning classification systems that are able to effectively differentiate people who are at risk of suicide from those who are not [28-30] and identify temporal patterns in posts before suicide [31] have been developed. Reviews on the subject [22,32] yield similar results: social media is an empirically tested tool for suicide detection, but further validation is needed.

    Recent studies have incorporated human coders in order to create language classification systems or validate machine learning results from natural language processing systems [33]. Nevertheless, the need to incorporate mental health experts in suicide assessments for the improvement of accuracy has been noted [34]. Furthermore, the detection of the risk of suicide for social media users located in Spain has yet to be explored.

    Our objective was to analyze the risk of suicide among Twitter users who post in Spanish, by assessing the emotional content of their posts and other variables that have been identified as being related to suicidal behavior, such as perception of defeat, helplessness, and social support.


    Design

    We conducted a cross-sectional exploratory study. To collect and analyze tweets, we used a framework based on computer technologies that we had developed previously [35]. This is a full framework (Figure 1) that has been engineered and implemented using various technologies and has been structured around a multidisciplinary team of professionals from health sciences and professionals with specialization in information technology. We focused on the Expert’s evaluation stage. The first step was to obtain the Twitter entries from keywords related to suicide. To identify potentially emotional tweets, a large vocabulary of emotional terms was compiled from different sources, including the Spanish adaptation of Affective Norms for English Words [36], which provides a set of emotional normative scales for a set of words, and the Spanish dictionary of the Linguistic Inquiry and Word Count [37], which is an analysis software that calculates the degree to which people use different categories of words across a wide spectrum of texts. The use of Linguistic Inquiry and Word Count software to assess positive and negative emotions has been validated [38,39].

    ‎
    Figure 1. Methodology for the early detection and prevention of the risk of suicide on Twitter.
    View this figure

    Adding properties to the text contained in the tweet facilitates and improves the identification and classification of groups at risk of suicide. Therefore, a series of properties associated with the text were obtained and added. The properties were based both on external natural language processing systems and platforms, and on internal algorithms that obtained the information through a text evaluation platform, which is completed by selected reviewers. The emotional vocabulary was organized by combining the hierarchy of emotions [40] and the tree of emotions [41]. Each emotional word was classified into 6 categories of primary emotions (love, joy, surprise, anger, sadness, and fear) and 25 subgroups of secondary emotions using affective and emotional text processing software (Indico, version 2020) that provides a toolkit of application programming interfaces (APIs). We used the following APIs for text-based analysis: Sentiment Analysis, Text Tags, Language Detection, Emotions, Personality and Personas. These APIs do not support the use of emoticons, and they ignore the appearance of these elements during text processing. A low or moderate use of emojis was observed in the content captured from Twitter that were relevant to the study. However, given that the use of emojis is ubiquitous, this fact must be taken into account when extending this work. The study of the criteria for matching between emojis and suicide or suicidal tendency is an aspect that we propose in future work in the short term.

    To cluster data groups with similar characteristics—in our case, we wanted to identify groups at risk of suicide—we selected k-means clustering [42]. This method is based on partitioning data into k well-defined groups. To select the k value, a cross-validation method was used [43].

    The clusters were analyzed in order to understand the characteristics of each one. This step needed to be carried out by a group of reviewers who were experts in the subject in order to understand the quality of the groups. The objective of this analysis was to validate that the clusters corresponded to groups at risk of suicide. At this stage, human coding was used to determine the degree of relationships of the classified tweets, based on the judgment of researchers from the fields of mental health and medicine who specialized in suicide prevention and had training in detecting the risk of suicide.

    An automatic classifier with clusters and tweets as input was created. This classifier was capable of receiving new tweets and classifying them into one of the groups in order to determine whether there was a risk of suicide or not. We used a long short-term memory neural network.

    Tweets were selected using Twitter’s API. Because most Twitter accounts were not geolocated, we selected posts written in Spanish. We checked this using both the Twitter information obtained through the API and by using a Language Detection API (Indico). We excluded posts in Latin-Spanish language because the cultural context of the tweet could be unknown. These tweets were collected between November 2019 and February 2020. The study included 25 psychologist evaluators with clinical or research experience who were trained to assess the tweets for the variables of the study. Each tweet was randomly assigned to 3 different evaluators. Each expert evaluated a tweet independently, without knowing to which other evaluators the tweet had been assigned. Assessments were carried out using a smartphone-based software. The tweets were evaluated between the months of March and April 2020.

    Study Variables

    Primary

    Only posts from the data set that were considered to be relevant by most experts were subsequently analyzed. Tweets were considered relevant if the content of the text was related to a potential risk of suicide of the author of the tweet. Tweets in which—(1) the text did not correspond to content related to a possible risk of suicide or associated emotional states; (2) the text was written in a language other than Spanish or the text was written in Latin Spanish; (3) the text of the tweet was ambiguous (ie, the context was unknown); or (4) the text of the tweet was not suicide-relevant for any other reason were and discarded.

    Secondary
    Outcome Variable

    The outcome variable was the severity of the risk of suicide at the moment of writing the tweet. We used a scale based on a suicide risk continuum with 5 levels [9]: (0) nonexistent risk, (1) mildly suicidal, (2) moderately frequent, (3) severely frequent, and (4) extremely frequent with intense and enduring suicidal ideation, specific plans, clear subjective and objective intent, impaired self-control, severe dysphoria, many risk factors, and no protective factors (ie, extreme risk).

    Valence

    Valence represented if the emotional valence of the tweet was positive or negative. If the value was greater than 50, it was considered that the text expressed a positive or pleasant feeling. If it was less than 50, it was considered to have a negative valence, that is, it expressed a negative, aversive, or unpleasant feeling. This was assessed on a scale from 0 and 100, using 2 decimal places.

    Emotional Content

    The level of anger, joy, fear, sadness, and surprise expressed in each tweet was evaluated on a scale from 0 and 100 (with 2 decimal places), with values closest to 100 being the highest level of the emotion.

    Relevant Personality Traits

    Traits—extroversion, if the author of the text showed extroversion(ie, is a person who is focused and interested in the outside world); sensory, if the author has a sensory tendency when processing information (ie, is a person who pays attention to details and prefers to work with concrete facts than with speculation or possibilities); rational, if the person has a tendency to make decisions based on logic, using an analytical and objective approach(ie, is a person who supports their decisions with impersonal analysis rather than with personal values); and judgment, if the author of the post has a preference for a planned (stable and organized, rather than spontaneous and flexible) life—were evaluated on a scale between 0 and 100 (with 2 decimal places), with values closest to 100 being the closest to that trait. These personality traits were collected from Myers-Briggs Type Indicators [44].

    Other Variables

    Other relevant variables on suicide risk assessments were also collected in the event that the information available was sufficient to assess them (or left blank if the parameter was not identifiable in the content of the tweet): (1) feelings of defeat, rejection, or both, if it was possible to identify a stressful event that generated feelings of defeat, rejection, or both feelings in the text of the author; (2) desire to escape from the situation, or if desire or will to run from a situation can be identified, evaluated using a dichotomous scale (yes or no); (3) social support or possibility of perceived help was evaluated dichotomously (yes or no); (4) feelings of helplessness or lack of coping resources were evaluated dichotomously (yes or no); (5) the general risk of suicide for the author of the tweet was evaluated on a scale between 0 and 4, with 0 being no risk and 4 being an extreme risk; (6) daily recurring thoughts of suicide was evaluated dichotomously (yes or no), and the intensity of autolytic thoughts was assessed on a scale from 0 to 10, with 0 being not intense and 10 being very intense; (7) content related to the tweet author’s sleep, insomnia, or hypersomnia was evaluated dichotomously (yes or no).

    Statistical Analysis

    Statistical analyses were conducted using SPSS software (version 22; IBM Corp). First, the sample distribution was analyzed. Kolmogorov-Smirnov values <0.05 were obtained for all variables; thus, nonparametric statistics were used. For quantitative variables, median and interquartile range were calculated, and for qualitative variables, frequency and percentages were calculated. The dependent variable (the severity of the risk of suicide at the present time) was analyzed as a continuous scale with a minimum of 0 and a maximum of 4. Spearman correlations between the severity of the risk of suicide at the time and the variables were calculated. Severity of the risk of suicide at the time was compared between qualitative variables using the Mann-Whitney U test (when there were 2 different groups) or the Kruskal-Wallis test (when there were more than 2 groups). A multivariate model was developed for severity of the suicidal risk at the time of tweeting. The independent variable was added into the regression model [45], and a final model was obtained. Linear regression was used since the residuals of the model had a finite mean, constant variance, and normal distribution (above all, because the sample size was very high; with the central limit theorem, any distribution with constant mean and variance, if it has a large enough sample size, has a normally distributed mean). However, bootstrapping analysis with 2000 samples was also conducted. The mean value of the 2 or 3 evaluators was used for continuous and qualitative variables, and the coinciding value between 3 evaluators, or 2 of 3 evaluators, was used. The interrater reliability was calculated using Fleiss κ. P values <.05 were considered to be significant.

    Ethical Issues

    All procedures contributing to this work complied with the ethical standards of the Clinical Research Ethics Committee of Aragón (Department of Health, Government of Aragón, Spain) and with the Helsinki Declaration of 1975, as revised in 2013 [46]. The study protocol was approved by the Clinical Research Ethics Committee of Aragón, Spain (17/0127, with the number PI21/164).


    A total of 2509 tweets were obtained, of which 2018 were deemed not relevant by 3 evaluators, and 275 were deemed not relevant by 2 of 3 evaluators. There were 216 tweets that were found to be relevant by most evaluators, with 68 tweets considered to be relevant by all evaluators, exhibiting moderate reliability (Fleiss κ=0.41).

    Tweets mainly conveyed sadness and defeat, with there being no desire to escape, no support, and no feelings of helplessness (Table 1). The median overall risk of suicide was 1.50 (IQR 1.00) on a scale from 0 to 4. The median severity of risk was 1.00 (IQR 1.16); 96.9% (186/192) of tweets did not indicate the presence of daily recurring thoughts of suicide, and the median intensity of suicidal thoughts was 4.50 (IQR 3.00) on a scale from 0 to 10.

    Table 1. Description of the tweets deemed relevant.
    VariablesValue
    Valencea, median (IQR)21.58 (24.25)
    Emotional contenta (n=216), median (IQR)

    		Anger24.00 (34.00)

    		Joy0.00 (1.50)

    		Fear17.25 (32.37)

    		Sadness51.41 (39.12)

    		Surprise0.50 (5.50)
    Relevant personality traitsa (n=216), median (IQR)

    		Extroversion28.00 (34.29)

    		Sensory25.16 (29.25)

    		Rational19.50 (27.37)

    		Judgement19.00 (32.00)
    Feelings of defeat or rejection (n=98), n (%)

    		Defeat61 (62.2)

    		Rejection16 (16.3)

    		Both21 (21.4)
    Desire to escape (n=161), n (%)

    		Yes25 (15.5)

    		No136 (84.5)
    Social support or possibility of perceived help (n=196), n (%)

    		Yes4 (2.0)

    		No192 (98.0)
    Feelings of helplessness (n=152), n (%)

    		Yes57 (37.5)

    		No95 (62.5)
    Suicide risk variablesb (n=216), median (IQR)

    		General risk1.50 (1.00)

    		Severity suicidal risk at present moment (real-time risk)1.00 (1.16)
    Daily recurrent thoughts of suicide (n=192), n (%)

    		Yes6 (3.1)

    		No186 (96.9)
    Intensity of autolytic thoughtsc, median (IQR)4.50 (3.00)
    Content related to insomnia or hypersomnia (n=210), n (%)

    		Yes4 (1.9)

    		No206 (98.1)

    aThese variables were evaluated on a scale from 0 to 100.

    bThese variables were evaluated on a scale from 0 to 4.

    cThis variable was evaluated on a scale from 0 to 10.

    There were direct correlations between severity of the risk of suicide at the time of generating the tweet and sadness, general risk, and intensity of suicide thoughts, as well as inverse correlations with extroversion, rational trait, and joy (Table 2).

    The severity of risk of suicide at the time of generating the tweet was higher in people who expressed feelings of defeat, rejection, desire to escape, feelings of helplessness, lack of social support, and daily recurrent thoughts (Table 3).

    The linear regression model (R2=0.750; adjusted R2=0.710) showed that the intensity of autolytic thoughts, fear, and valence were predictors of the severity of the risk of suicide at the time (Table 4). Both the intensity of autolytic thoughts and valence had positive coefficients, and fear had a negative coefficient. This indicated that when intensity was higher, valence was more positive, and when fear was lower, the severity of suicidal risk was higher. The model explained 75% of the variance.

    Table 2. Spearman correlations between variables and severity of the risk of suicide at the time of writing the tweet.
    VariablesρP value
    Valence–0.069.31
    Emotional content

    		Anger–0.013.85

    		Joy–0.234.001

    		Fear–0.097.16

    		Sadness0.266<.001

    		Surprise–0.075.27
    Relevant personality traits

    		Extroversion–0.22.001

    		Sensory–0.115.09

    		Rational–0.244<.001

    		Judgement–0.128.06
    Suicide risk variables

    		General risk0.908<.001

    		Intensity of autolytic thoughts0.766<.001
    Table 3. Comparison of severity of the risk of suicide at the time of writing the tweet between qualitative variables.
    VariablesSeverity of the risk of suicide at the moment (real-time risk), median (IQR)P value
    Feelings of defeat or rejection

    		Defeat1.33 (1.42).003

    		Rejection0.33 (1.25)

    		Both1.66 (1)
    Desire to escape

    		Yes1 (0)<.001

    		No1 (1.17)
    Social support or possibility of perceived help

    		Yes0.33 (0.66).03

    		No1 (1.16)
    Feelings of helplessness

    		Yes1.5 (1).001

    		No1 (1.17)
    Daily recurrent thoughts of suicide

    		Yes2.16 (1.30).007

    		No1 (1)
    Content related to insomnia or hypersomnia

    		Yes0.75 (0.63).22

    		No1 (1.16)
    Table 4. Linear regression model coefficients indicating the relationship to severity of the risk of suicide at the time of writing the tweet.
    VariablesCoefficient (95% CI)P value
    Constant0.110 (–0.169, 0.412).45
    Intensity of autolytic thoughts0.311 (0.250, 0.370).001
    Fear–0.009 (–0.015, –0.005).01
    Valence0.007 (0.002, 0.013).009

    Suicide prevention is a crucial field that needs to be developed to stop preventable deaths worldwide. The findings of our study reveal that social media can be used to help to identify individuals at risk. These findings suggest that it is possible to identify suicidal behavior through Spanish tweets, and it is possible to identify these posts, not only by using a suicide-related lexicon but also, by filtering tweets based on their emotional content. Tweets that show sadness, defeat, and perceived lack of social support suggest that there is a risk of suicide. These variables have been commonly associated with symptoms of depression and hopelessness [12,47].

    One of the main challenges in suicide prevention is identifying not only the people at risk of experiencing suicidal behavior at some point in their lives but those who are at risk at a particular moment, in our case, while they are writing the tweet.

    Suicidal ideation and its risk factors can fluctuate over short periods of time [16], which demonstrates the importance of differentiating general suicide risk (or suicide status, such as in individuals with long-term risk factors) from real-time suicide risk. In this exploratory study, we emphasized assessment of suicide risk at the moment of writing the post. Our results provide some clues about the phenomenon of suicidal behavior.

    In our study, although the variable desire to escape from the situation was not identifiable in most posts showing potential risk, it was related to an increase in the severity of the risk at the time of writing the tweet in posts that expressed a desire to escape.This outcome could suggest that the variable of the desire to run from a suffering situation may be only relevant in situations with an increased risk of suicide at that time. We found the same pattern for feelings of helplessness at the time, which was only identifiable in high-risk tweets. These results are consistent with recent conceptualizations of acute suicidal behavior that suggest that this feeling of entrapment—“in which the escape from an unbearable life situation is perceived as both urgent and impossible” [48]—is linked with imminent suicidal behavior. Nevertheless, further research is needed.

    Individuals exhibiting high levels of negative urgency and emotion reactivity might be more likely to develop suicidal ideation and resort to self-harm while experiencing negative affective states [14,49]. In our sample, tweets with higher risk (at the time of writing the tweets) were identified by higher sadness, higher general risk, and higher intensity of suicidal thoughts on a daily basis. In addition, they similarly showed feelings of defeat and rejection, as well as the perception of a lack of social support. These results are consistent with those in literature, with lack of social support or isolation being an especially well-established risk factor [3].

    Although the role of impulsivity in suicidal behavior has not been clearly defined yet [50,51], it appears that a considerable proportion of suicide attempts are related to impulsive behavior [52,53]. In our study, we also included personality trait variables. Our results suggest that people at risk at the time of writing their tweet might show less extroversion and less rational personality traits in their posts, which could be associated with greater impulsiveness. Further research would shed more light on this subject.

    Although insomnia or sleep problems variables have been identified as risk factors in suicide assessments, our findings suggest that this variable is not identifiable or relevant to social media posts. One study [54] notes that only nightmares are associated with suicidality.

    We obtained preliminary data that might help us to predict increased real-time suicide risk. The interpersonal theory of suicide [55] posits that the simultaneous occurrence of 2 psychological states, a perceived burden to others, and a frustrated belonging or social isolation, as well as hopelessness regarding the potential of these states to change, results in the desire for suicide. Our findings are consistent with this theory, showing that sadness, feeling of defeat, or perceived lack of support are related to high-risk tweets. However, according to this theory, the simultaneous presence of suicidal desire and a high tolerance for pain and fear of death would be necessary to produce lethal or near-lethal behavior. The interpersonal theory of suicide also posits that high risk occurs when tolerance for pain and fear increases [49,56]. Our results seem to also suggest that real risk appears when ideation intensity increases and the fear of suicide decreases, making the resulting emotional valence less unpleasant. In other words, the detection of a decrease in fear and, therefore, a less aversive emotional state could predict an increased risk of suicidal behavior at the time on social media. If these variables effectively and consistently prove their ability to predict an increase in risk at the time, we could generate real-time machine learning systems that would detect predictor emotional states, such as the decrease in fear, to prevent potential deaths.

    Our study has strengths but also limitations. The cross-sectional design of this study provided limited data about suicidal phenomenon on social networks. In the future, it would be interesting to be able to design a study that screens variation in emotional states and suicide risk over time as this may provide more relevant information on how the risk of suicide varies.

    Twitter is considered to be one of the most popular social media platforms, with the greatest immediacy in posting, but because there is a character limit, some variables are barely detectable or measurable. More complex feelings or constructs are difficult to evaluate. Future research could include the evaluation of conversation threads or tweets in their context (for example, by taking into account tweets within the previous 24 hours), which would allow more accurate evaluations of relevant variables in suicide risk assessments.

    The interrater reliability was moderate, differences between research and the clinical professional profile of the experts might have had an impact on reliability. In future investigations, reliability will be assessed based on expert profiles, and intensive specific training will be conducted.

    One of the strengths of our study is that it incorporates experts in suicide risk assessment. In future research, it would be interesting to introduce validated scales to measure suicidality so that expert risk detection could be effectively validated. Moreover, it would be interesting for future research to use social media users with past suicide attempts to screen and validate our findings.

    Our findings—identifying emotional content that might be relevant for real-time suicide prevention—can contribute to the development of new technology-based screening systems; however, more robust research is needed to establish whether social media screening can effectively reduce suicide outcomes and whether there is a way to ethically reach those individuals at risk.

    Acknowledgments

    We would like to thank Afda Aragón, the B21_20R, B03_20R and DisCo-T21-20R Research Groups of the Department of Research, Innovation and Universities of the Aragon Regional Authority (Spain); and the European Regional Development Fund Otra forma de hacer Europa.

    Conflicts of Interest

    None declared.

    1. Suicide rate estimates, age-standardized estimates by WHO region. World Health Organization. 2016.   URL: https://apps.who.int/gho/data/view.main.MHSUICIDEASDRREGv?lang=en [accessed 2022-03-01]
    2. Defunciones según causa de muerte Año 2020. Instituto Nacional de Estadística. 2020.   URL: https://www.ine.es/prensa/edcm_2020.pdf [accessed 2021-10-01]
    3. Preventing suicide: a global imperative. World Health Organization. 2014 Aug 17.   URL: https://www.who.int/publications/i/item/9789241564779 [accessed 2022-05-12]
    4. Large MM. The role of prediction in suicide prevention. Dialogues Clin Neurosci 2018 Sep;20(3):197-205 [FREE Full text] [Medline]
    5. Pisani AR, Murrie DC, Silverman MM. Reformulating suicide risk formulation: from prediction to prevention. Acad Psychiatry 2016 Aug 14;40(4):623-629 [FREE Full text] [CrossRef] [Medline]
    6. Gibbons RD. The statistics of suicide. Shanghai Arch Psychiatry 2013 Apr;25(2):124-130 [FREE Full text] [CrossRef] [Medline]
    7. Berman AL, Silverman MM. Suicide risk assessment and risk formulation part II: suicide risk formulation and the determination of levels of risk. Suicide Life Threat Behav 2014 Aug 29;44(4):432-443. [CrossRef] [Medline]
    8. Wortzel H, Homaifar B, Matarazzo B, Brenner LA. Therapeutic risk management of the suicidal patient: stratifying risk in terms of severity and temporality. J Psychiatr Pract 2014 Jan;20(1):63-67. [CrossRef] [Medline]
    9. Bryan CJ, Rudd MD. Advances in the assessment of suicide risk. J Clin Psychol 2006 Feb;62(2):185-200. [CrossRef] [Medline]
    10. Kessler RC, Bossarte RM, Luedtke A, Zaslavsky AM, Zubizarreta JR. Suicide prediction models: a critical review of recent research with recommendations for the way forward. Mol Psychiatry 2020 Jan 30;25(1):168-179 [FREE Full text] [CrossRef] [Medline]
    11. Beck AT, Brown G, Berchick RJ, Stewart BL, Steer RA. Relationship between hopelessness and ultimate suicide: a replication with psychiatric outpatients. Am J Psychiatry 1990 Feb;147(2):190-195. [CrossRef] [Medline]
    12. Ribeiro JD, Huang X, Fox KR, Franklin JC. Depression and hopelessness as risk factors for suicide ideation, attempts and death: meta-analysis of longitudinal studies. Br J Psychiatry 2018 May 28;212(5):279-286. [CrossRef] [Medline]
    13. Anestis MD, Bagge CL, Tull MT, Joiner TE. Clarifying the role of emotion dysregulation in the interpersonal-psychological theory of suicidal behavior in an undergraduate sample. J Psychiatr Res 2011 May;45(5):603-611. [CrossRef] [Medline]
    14. Dour HJ, Cha CB, Nock MK. Evidence for an emotion-cognition interaction in the statistical prediction of suicide attempts. Behav Res Ther 2011 Apr;49(4):294-298. [CrossRef] [Medline]
    15. Sedano-Capdevila A, Porras-Segovia A, Bello HJ, Baca-García E, Barrigon ML. Use of ecological momentary assessment to study suicidal thoughts and behavior: a systematic review. Curr Psychiatry Rep 2021 May 18;23(7):41. [CrossRef] [Medline]
    16. Kleiman EM, Turner BJ, Fedor S, Beale EE, Huffman JC, Nock MK. Examination of real-time fluctuations in suicidal ideation and its risk factors: results from two ecological momentary assessment studies. J Abnorm Psychol 2017 Aug;126(6):726-738. [CrossRef] [Medline]
    17. Clement J. Social media - statistics and facts. Statista.   URL: https://www.statista.com/topics/1164/social-networks/ [accessed 2021-05-01]
    18. Paris C, Christensen H, Batterham P, O'Dea B. Exploring emotions in social media. 2015 Oct 27 Presented at: IEEE Conference on Collaboration and Internet Computing; October 27-30, 2015; Hangzhou, China. [CrossRef]
    19. Luxton DD, June JD, Fairall JM. Social media and suicide: a public health perspective. Am J Public Health 2012 May;102 Suppl 2:S195-S200. [CrossRef] [Medline]
    20. Fernandez R. Twitter en España - Datos estadísticos. Statista.   URL: https://es.statista.com/temas/3595/twitter-en-espana/ [accessed 2021-05-01]
    21. Help center. Twitter Inc.   URL: https://help.twitter.com/en/safety-and-security/self-harm-and-suicide [accessed 2021-05-01]
    22. Christensen H, Batterham P, O'Dea B. E-health interventions for suicide prevention. Int J Environ Res Public Health 2014 Aug 12;11(8):8193-8212 [FREE Full text] [CrossRef] [Medline]
    23. Choudhury M, Kiciman E, Dredze M, Coppersmith G, Kumar M. Discovering shifts to suicidal ideation from mental health content in social media. 2016 May 07 Presented at: CHI Conference on Human Factors in Computing Systems. Association for Computing Machinery; May 7-12, 2016; San Jose, California p. 2098-2110. [CrossRef]
    24. Choudhury M, Gamon M, Counts S, Horvitz E. Predicting depression via social media. 2021 Aug 03 Presented at: Proceedings of the International AAAI Conference on Web and Social Media; July 8-11, 2013; Cambridge, Massachusetts p. 128-137.
    25. Nadeem M. Identifying depression on Twitter. arXiv Preprint posted online on Jan 25, 2016 [FREE Full text]
    26. Reece AG, Reagan AJ, Lix KLM, Dodds PS, Danforth CM, Langer EJ. Forecasting the onset and course of mental illness with Twitter data. Sci Rep 2017 Oct 11;7(1):13006 [FREE Full text] [CrossRef] [Medline]
    27. Coppersmith G, Harman C, Dredze M. Measuring post traumatic stress disorder in twitter. 2014 Presented at: Proceedings of the 8th International Conference on Weblogs and Social Media, ICWSM; June 1-4, 2014; Michigan p. 579-582.
    28. Desmet B, Hoste V. Emotion detection in suicide notes. Expert Syst Appl 2013 Nov;40(16):6351-6358. [CrossRef]
    29. Braithwaite SR, Giraud-Carrier C, West J, Barnes MD, Hanson CL. Validating machine learning algorithms for Twitter data against established measures of suicidality. JMIR Ment Health 2016 May 16;3(2):e21 [FREE Full text] [CrossRef] [Medline]
    30. Lv M, Li A, Liu T, Zhu T. Creating a Chinese suicide dictionary for identifying suicide risk on social media. PeerJ 2015;3:e1455 [FREE Full text] [CrossRef] [Medline]
    31. Huang X, Xing L, Brubaker JR, Paul MJ. Exploring timelines of confirmed suicide incidents through social media. 2017 Aug 23 Presented at: IEEE International Conference on Healthcare Informatics; August 23-26, 2017; Park City, Utah. [CrossRef]
    32. Robinson J, Cox G, Bailey E, Hetrick S, Rodrigues M, Fisher S, et al. Social media and suicide prevention: a systematic review. Early Interv Psychiatry 2016 Apr 19;10(2):103-121. [CrossRef] [Medline]
    33. Franco-Martín MA, Muñoz-Sánchez JL, Sainz-de-Abajo B, Castillo-Sánchez G, Hamrioui S, de la Torre-Díez I. A systematic literature review of technologies for suicidal behavior prevention. J Med Syst 2018 Mar 05;42(4):71. [CrossRef] [Medline]
    34. Burnap P, Colombo G, Amery R, Hodorog A, Scourfield J. Multi-class machine classification of suicide-related communication on Twitter. Online Soc Netw Media 2017 Aug;2:32-44 [FREE Full text] [CrossRef] [Medline]
    35. Fabra J, Martínez-Martínez A, Lopez-Del-Hoyo Y, Perez-Yus MC, Oliván-Blázquez B. Detecting Suicide Risk Through Twitter. 2020 Oct 25 Presented at: The Fourteenth International Conference on Advanced Engineering Computing and Applications in Sciences; Octobre 25-29, 2020; Nice, France p. 6-11.
    36. Redondo J, Fraga I, Padrón I, Comesaña M. The Spanish adaptation of ANEW (affective norms for English words). Behav Res Methods 2007 Aug;39(3):600-605. [CrossRef] [Medline]
    37. Pennebaker M, Booth R, Francis. Linguistic inquiry and word count. LIWC. 1999.   URL: https://www.liwc.app/ [accessed 2022-05-12]
    38. Cheng Q, Li TM, Kwok C, Zhu T, Yip PS. Assessing suicide risk and emotional distress in Chinese social media: a text mining and machine learning study. J Med Internet Res 2017 Jul 10;19(7):e243 [FREE Full text] [CrossRef] [Medline]
    39. Kahn J, Tobin RM, Massey AE, Anderson JA. Measuring emotional expression with the Linguistic Inquiry and Word Count. Am J Psychol 2007;120(2):263-286. [Medline]
    40. Parrott WG. Emotions in Social Psychology: Essential Readings. Philadelphia, USA: Psychology Press; 2001.
    41. Shaver P, Schwartz J, Kirson D, O'Connor C. Emotion knowledge: further exploration of a prototype approach. J Pers Soc Psychol 1987 Jun;52(6):1061-1086. [CrossRef] [Medline]
    42. Faber V. Clustering and the continuous k-means algorithm. Los Alamos Science 1994;22:138-144 [FREE Full text]
    43. Stone M. Cross-validatory choice and assessment of statistical predictions. J R Stat Soc Series B Stat Methodol 2018 Dec 05;36(2):111-133. [CrossRef]
    44. Briggs-Myers L, Briggs KC. Myers-Briggs Type Indicator. Palo Alto, California: Consulting Pyschologist Press; 1985.
    45. Hamilton JD. Time Series Analysis. New Jersey: Princeton University Press; 1994.
    46. World Medical Association. World Medical Association Declaration of Helsinki: ethical principles for medical research involving human subjects. JAMA 2013 Nov 27;310(20):2191-2194. [CrossRef] [Medline]
    47. Minkoff K, Bergman E, Beck AT, Beck R. Hopelessness, depression, and attempted suicide. Am J Psychiatry 1973 Apr;130(4):455-459. [CrossRef] [Medline]
    48. Bloch-Elkouby S, Gorman B, Schuck A, Barzilay S, Calati R, Cohen LJ, et al. The suicide crisis syndrome: a network analysis. J Couns Psychol 2020 Oct;67(5):595-607. [CrossRef] [Medline]
    49. Anestis MD, Joiner TE. Examining the role of emotion in suicidality: negative urgency as an amplifier of the relationship between components of the interpersonal-psychological theory of suicidal behavior and lifetime number of suicide attempts. J Affect Disord 2011 Mar;129(1-3):261-269. [CrossRef] [Medline]
    50. Baca-Garcia E, Diaz-Sastre C, García Resa E, Blasco H, Braquehais Conesa D, Oquendo MA, et al. Suicide attempts and impulsivity. Eur Arch Psychiatry Clin Neurosci 2005 Apr 19;255(2):152-156. [CrossRef] [Medline]
    51. Anestis MD, Soberay KA, Gutierrez PM, Hernández TD, Joiner TE. Reconsidering the link between impulsivity and suicidal behavior. Pers Soc Psychol Rev 2014 Nov 26;18(4):366-386. [CrossRef] [Medline]
    52. Wyder M, De Leo D. Behind impulsive suicide attempts: indications from a community study. J Affect Disord 2007 Dec;104(1-3):167-173. [CrossRef] [Medline]
    53. Lim M, Lee S, Park J. Differences between impulsive and non-impulsive suicide attempts among individuals treated in emergency rooms of South Korea. Psychiatry Investig 2016 Jul;13(4):389-396 [FREE Full text] [CrossRef] [Medline]
    54. Bernert R, Joiner TE, Cukrowicz KC, Schmidt NB, Krakow B. Suicidality and sleep disturbances. Sleep 2005 Sep;28(9):1135-1141. [CrossRef] [Medline]
    55. Van Orden KA, Witte TK, Cukrowicz KC, Braithwaite SR, Selby EA, Joiner TE. The interpersonal theory of suicide. Psychol Rev 2010 Apr;117(2):575-600 [FREE Full text] [CrossRef] [Medline]
    56. Davidson CL, Wingate LR, Rasmussen KA, Slish ML. Hope as a predictor of interpersonal suicide risk. Suicide Life Threat Behav 2009 Oct;39(5):499-507. [CrossRef] [Medline]

    ‎
    API: application programming interface

    Edited by H Bradley; submitted 06.07.21; peer-reviewed by X Li, K Reuter; comments to author 08.11.21; revised version received 22.12.21; accepted 24.02.22; published 17.05.22

    Copyright

    ©Claudia García-Martínez, Bárbara Oliván-Blázquez, Javier Fabra, Ana Belén Martínez-Martínez, María Cruz Pérez-Yus, Yolanda López-Del-Hoyo. Originally published in JMIR Public Health and Surveillance (https://publichealth.jmir.org), 17.05.2022.

    This is an open-access article distributed under the terms of the Creative Commons Attribution License (https://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work, first published in JMIR Public Health and Surveillance, is properly cited. The complete bibliographic information, a link to the original publication on https://publichealth.jmir.org, as well as this copyright and license information must be included.