Impact of Firearm Surveillance on Gun Control Policy: Regression Discontinuity Analysis

Background: Public mass shootings are a significant public health problem that require ongoing systematic surveillance to test and inform policies that combat gun injuries. Although there is widespread agreement that something needs to be done to stop public mass shootings, opinions on exactly which policies that entails vary, such as the prohibition of assault weapons and large-capacity magazines. Objective: The aim of this study was to determine if the Federal Assault Weapons Ban (FAWB) (1994-2004) reduced the number of public mass shootings while it was in place. Methods: We extracted public mass shooting surveillance data from the Violence Project that matched our inclusion criteria of 4 or more fatalities in a public space during a single event. We performed regression discontinuity analysis, taking advantage of the imposition of the FAWB, which included a prohibition on large-capacity magazines in addition to assault weapons. We estimated a regression model of the 5-year moving average number of public mass shootings per year for the period of 1966 to 2019 controlling for population growth and homicides in general, introduced regression discontinuities in the intercept and a time trend for years coincident with the federal legislation


Introduction
Background Approximately 44,000 people are killed and an additional 100,000 people are injured by a gun each year in the United States [1,2]. Mass shooting fatalities, as a particular type of gun injury event, account for <1% of all gun deaths [3] and have largely been ignored until recently [4,5]; yet, mass shooting events occur multiple times per year [6]. This information is based on insights from firearm surveillance performed by a variety of researchers, and state and federal agencies on incidence, prevalence, risk factors, injuries, deaths, and precipitating events, similar to the surveillance of infectious diseases such as COVID-19 [7][8][9][10][11][12][13][14][15][16][17][18][19][20][21]. Teutch and Thacker [22] defined public health surveillance as the ongoing systematic collection, analysis, and interpretation of health data, essential to the planning, implementation, and evaluation of public health practice, closely integrated to the dissemination of these data to those who need to know and linked to prevention and control.
Not only do surveillance systems generate hypotheses to test but they also provide the data to test them.
The Federal Assault Weapons Ban (FAWB, also known as the Public Safety and Recreational Firearms Use Protection Act) included a ban on the manufacture for civilian use or sale of certain semiautomatic firearms defined as assault weapons as well as certain large-capacity magazines (LCMs). The Act was in effect for 10 years from 1994 until it sunsetted in 2004. Semiautomatic weapons (rapid fire) and assault weapons (second grip plus other features) are distinct; however, the two are often incorrectly conflated as similar [23][24][25][26]. Semiautomatic weapons are defined as weapons that automatically load another cartridge into a chamber, preparing the weapon for firing, but requiring the shooter to manually release and press the trigger for each round [23][24][25][26]. By contrast, automatic weapons are similarly self-loading, but allow for a shooter to hold the trigger for continuous fire [27]. Furthermore, the FAWB also prohibited certain ammunition magazines that were defined as "large-capacity" cartridges [28] containing more than 10 bullets [29]. These LCMs can feed ammunition to semiautomatic weapons that do not meet the criteria of being considered assault weapons. Furthermore, LCMs are considered one of the most important features of the FAWB as research has found a relationship between bans on LCMs and casualty counts at the state level [30][31][32][33][34]. The 10-year federal ban was signed into law by President Clinton on September 13, 1994 [28].
Firearm surveillance data have been used to test potential policy responses to prevent mass shootings, including the FAWB [32,[34][35][36][37][38][39], Extreme Risk Protection Orders (also known as red flag laws) [40][41][42][43][44][45], and federal and state LCM bans [31,32,46]. In particular, it seems likely that the FAWB and LCM bans have potential to affect mass shootings because they regulate weapons and ammunition formats that are designed to enable rapid discharge, which is a key feature in mass shooting incidents [24,47]. Other types of gun deaths may not be responsive to the FAWB or LCM bans. As an example, Extreme Risk Protection Orders or "Red Flag" orders [43,48], which temporarily prohibit at-risk individuals from owning or purchasing firearms, may be effective for preventing firearm suicides or domestic violence homicides [49] but less effective for public mass shooters [50,51]. The prohibition of LCMs may have no impact on firearm suicide because suicide decedents only require one bullet to kill themselves [52].
Several studies during and after the FAWB attempted to determine if gun policy that restricts the production and sale of assault weapons and LCMs decreased gun deaths [53,54]. These initial studies make meaningful contributions to the literature because they describe what constitutes assault weapons, magazine capacity, ballistics, and loopholes in the FAWB legislation [3,[53][54][55][56][57]. However, these studies have found little to no evidence that these policies have had any overall effect on firearm homicides, gun lethality, or overall crime [58][59][60][61]. Since deaths from public mass shootings comprise less than 1% of all homicides based on our definition, testing whether or not the FAWB/LCM ban has an impact on homicide would wash out the effect. Since the FAWB/LCM ban may be effective at specific types of gun deaths, sampling must be limited to specific types of shooters over overall gun deaths or tests for lethality [62,63]. Finally, the variation in research findings is related to differences in research design, sampling frame, and case definition of a public mass shooting [3,[53][54][55][56]64,65].
Our study differs from other studies that evaluated the efficacy of the FAWB because we used economic methods and a different outcome variable. Specifically, we focused on whether the FAWB resulted in fewer public mass shooting "events," whereas other studies evaluated the number of gun injuries and deaths that occurred during the course of a mass shooting.

Objective
The aim of this study was to test whether curbing access to certain types of guns and magazines will decrease mass shooting events. We sought to empirically answer if there was a relationship between the FAWB and a reduction in mass shooting events.

Data Source
We created a firearm surveillance system based on the National Institute of Justice-funded Violence Project dataset, which culled mass shooting events from 1966 to 2019 [6]. Consistent with earlier studies, we rely on the original Federal Bureau of Investigation (FBI) definition of a massacre, specifically where 4 or more people are killed within a single timeframe. We differentiate our mass shootings from others in that our inclusion criteria require the shootings to have occurred in a public setting.
We adapted this definition to only include massacres that involved gun deaths of 4 or more victims to isolate a particular type of mass shooter [66]. Many firearm surveillance systems that include mass shootings use a lower threshold of persons shot and many do not include deaths. An FBI report on active shooters in mass shooting events identified planning and preparation behaviors that are central to prevention [67]. This more narrow definition isolates premeditation, whereas broader definitions may include shooters that are more reactive [68]. Our case definition does not include family annihilators or felony killers because familicides are defined by the victim-offender relationship, public massacres are defined by location, and felony killings are distinguished by motive [69]. This differentiation is consistent with other mass shooting studies [70][71][72].
We examined the annual number of public mass shootings occurring between 1966 and 2019 that resulted in 4 or more fatalities. The hypothesis was that the FAWB reduced the number of public mass shootings per year during the period of the ban. We used regression discontinuity analysis to test the hypothesis. Regression discontinuity analysis is a standard economist tool used in policy analysis taking advantage of quasi-experimental designs [65,73].

Analyses
Regression discontinuity analysis allows for discontinuities or shifts in both the intercept and the slope of the trend line at both the onset and sunset of the FAWB. That is, we introduced intercept shift parameters in 1995 and 2005, and trend shift parameters for the periods 1995-2004 and 2005-2019. A statistically significant shift in a parameter indicates a discontinuity (ie, a finding that the FAWB had a statistically significant effect on the number of public mass shootings). We tested for statistical significance of the intercept and trend shift parameters both independently and jointly. All statistical inference was based on a significance level set at .05. We used the Huber-White robust residuals, which attenuate problems of autocorrelation, heteroscedasticity, and some types of model misspecification [74].
We then used the estimated model for two types of counterfactual analysis. First, we used the model to predict the number of public mass shootings that would have occurred had the FAWB not been in place. The difference between this counterfactual prediction and the modeled number of incidents with the FAWB in place provided an estimate of the number of public mass shootings that the FAWB prevented.
Second, we projected forward the number of public mass shootings that would have occurred had the FAWB been permanent (ie, continued from 2004 through to the end of the sample period). We note that in some sense, this is an "out of sample" exercise because even though the sample extends to 2019, the FAWB ended in 2004; thus, this exercise would not pick up events in the past 15 years that would have augmented or compromised the effects of the FAWB. The difference between the modeled number of public mass shootings and the projected counterfactual number of public mass shootings could provide an estimate of the number of public mass shootings that the FAWB prevented.
We performed a regression of the 5-year moving average of public mass shootings on the US population in millions, the homicide rate, and discontinuity variables to capture both the effects of the FAWB and its discontinuation. We did not introduce a trend line for the entire sample period because it is highly collinear with the population variable. For the period of the FAWB's implementation, we originally introduced an intercept shift, time trend, and shift in the homicide rate; for the post-FAWB period, we introduced an intercept shift and a time trend. Due to collinearity, we retained only the trend shift in the final model for the FAWB period; for the post-FAWB period, we retained both the intercept and the trend shift.

Results
We identified a total of 170 public mass shooting events, the primary outcome variable, with 4 or more fatalities between 1966 and 2019. The 5-year cumulative number of public mass shootings is shown in Figure 1

Principal Findings
In total, 1225 people were killed in a mass shooting over the past 53 years with more than half occurring in the last decade, a function of increases in mass shootings and weapon lethality [62,63,75]. Public mass shooting fatalities and injuries far outpace population growth [75]. Between 1966 and 2019, the US population increased by 67% [76], whereas public mass shooting deaths increased by over 5-fold. The rise in public mass shootings throughout the sample period is in fact partially a function of population growth and homicide rate, along with the effects of the FAWB and its removal. An increase in the US population of 1 million people was associated with an increase of .040 (P<.005) public mass shootings per year. During the post-FAWB period, the increase in population from approximately 300 million in 2005 to 330 million in 2019 should be associated with an increase of 1.2 public mass shootings per year, compared to the actual increase of 4 public mass shootings per year in the data (5-year moving average). After controlling for population growth and homicide rate, a positive and statistically significant coefficient (.081, P=.001) on the 2005-2018 trend was seen. This further indicates a separate, nonpopulation trend of increasing violence operating during the post-FAWB period. The negative coefficient on the homicide rate invalidates the hypothesis that decreases in the numbers of public mass shootings are simply reflections of an overall decreasing homicide rate. The negative intercept discontinuity is consistent with an effect of the FAWB that persists somewhat beyond the immediate end of the ban. The positive trend coefficient is consistent with the hypothesis that the FAWB was associated with a decrease in the number of public mass shootings, as the expiration of the FAWB was associated with a shift from a downward trend to an upward trend in the number of public mass shootings per year.
The most striking finding from this study is that there was a reduction in the number of public mass shooting events while the FAWB was in place. Using prediction models in combination with regression slopes, we estimate that 11 public mass shootings were avoided due to the FAWB. By projecting what would have happened if the FAWB remained in place, we found that there would have been significantly fewer public mass shootings if the FAWB had remained in place to 2019. Remarkably, although it is intuitive that the removal of assault weapons and magazine clips will reduce the lethality of a mass shooting, we observed an inverse relationship between weapons/ammunition and mass shooting events, meaning that mass shooters may be less likely to perpetrate a mass shooting without rapid fire military-style weapons. This is an independent effect, which indirectly leads to fewer injuries and deaths. DiMaggio et al [64] also found evidence of a decrease in public mass shootings during the ban; however, their study period was shorter and was restricted to 51 public mass shootings. Unlike our study, they implicitly modeled public mass shootings as a random instance of general gun homicides that had a high death count [64]. In contrast, our findings suggest that public mass shootings are a unique type of premeditated gun violence. We found that prior to enactment of the FAWB, the rate of public mass shootings was increasing. During enactment of the FAWB, there was a downward trend of mass shooting events. After the FAWB was lifted, public mass shootings increased dramatically. Firearm homicides in general follow no such patterns.
This effect was not found in the work of Koper, Roth, and colleagues [53][54][55]; however, their inclusion of all gun homicides masks the ban's effect on mass shootings. Even though Peterson and Densley's [77] work focused on perpetrator histories and not the FAWB, their findings that ease of gun access is characteristic of public mass shooters further supports our study. We restricted the inclusion criteria to public mass shootings to specifically test the effectiveness of the FAWB on public mass shooting events.
Regardless of the FAWB, bringing a semiautomatic rifle with high magazine capacity to a massacre significantly increases the number of fatalities and injuries. The increase in deaths is a function of rapid fire and increased ballistic energy. The increase in injuries is also a function of rapid fire and high-capacity magazines, enabling the shooter to shoot more people in crowded venues quickly before the crowd can disperse or hide. When controlling for the FAWB, the use of assault rifles decreased by half during implementation of the ban and tripled after the ban was lifted. This is a particularly important finding given that the FAWB had loopholes and that overall violent crime is decreasing [78]. First, all people with an assault weapon prior to the FAWB were allowed to retain their semiautomatic weapons [54,64]. Second, without a buyback program, semiautomatic weapons remained in the community [54,64]. Third, the ban did not target some military assault-like weapons [54,64]. Finally, a major loophole found in gun control legislation is that buyers can bypass background checks by purchasing their weapons and ammunition from gun shows, through illegal purchasing, or legally purchasing their guns and ammunition from another gun owner [57,63,[79][80][81][82][83][84][85][86][87]. Even with these loopholes and issues, there was still a significant reduction in public mass shootings during the FAWB. These loopholes indicate that most people who purchase assault weapons do not become mass shooters; however, mass shooters require assault weapons and LCMs to carry out a mass shooting. Ban effectiveness might have improved if all assault weapons were included in the FAWB.
Some recent studies have specifically analyzed the effects of LCM bans on the incidence of public mass shootings. In a review of state legislation, Webster et al [88] found that bans of LCMs were associated with a significant reduction in the incidence of fatal public mass shootings. This study shows that the FAWB, which included a ban on LCMs, was associated with fewer fatalities and injuries during mass shootings in addition to fewer public mass shooting events. Koper et al [27] previously reported that 19% of public mass shootings resulting in 4 or more fatalities included the use of LCMs, while only 10% involved an assault weapon. Klarevas et al [29] found a similar pattern in shootings of 6 or more people, in which 67% of shooters utilized LCMs, whereas only 26% utilized an assault weapon. Because our study only looked at effects of the FAWB, which included an LCM ban, we were only able to determine the combined effects of limiting assault weapons and LCMs. To be clear, the reduction in the number of public mass shootings, and resulting fatalities and injuries, may be a function of the ban on assault weapons, assault weapons plus LCMs, or only LCMs. We cannot separate out their independent effects at the national level.
Unlike our study, Webster et al [88] did not evaluate the incidence of assault weapons used in public mass shootings. Rather, they focused on fatalities from public mass shootings vs public mass shooting events. Although Webster et al [88] utilized the FBI Supplemental Homicide Report as their dataset, which is a voluntary reporting measurement system prone to errors in reporting, their findings are applicable to our analysis.

Limitations
Although we found statistically significant decreases during the FAWB, we cannot isolate aspects of the policy that are attributed to the decline. Most notably, the FAWB also included LCMs during the ban. It may be that the type of gun and/or the type of magazine resulted in a decline. Indeed, assault weapons and LCMs provide the means to carry out a mass shooting; however, there are likely other factors beyond this study that partially explain the radical increase in public mass shootings in the post-FAWB period. For example, the FAWB was in place from 1994 to 2004, which is the same time period that the US population largely adopted the internet, along with associated social communication software and websites. This may have resulted in better tracking of public mass shootings or increased media coverage. Because our study specifically targeted the federal legislation, we omitted state-level gun policies such as state-level prohibitions on certain types of guns, LCMs, or more lethal types of bullets. It is likely that the internet serves as a contagion and as a guide to potential mass shooters, allowing them to access weapons and multiple stories about other mass shooters [62,67,89,90].

Conclusions
In summary, public mass shootings are a unique and specific type of homicide by a gun. We found evidence that public mass shootings are qualitatively different from general homicides because after the FAWB expired, mass shooting events increased while general homicides decreased. The increase in public mass shootings was more dramatic in the final 10 years of the study period following the end of the FAWB. We suspect that these outcomes may be improved by removing existing semiautomatic weapons with large bullet capacity by creating a buyback program for all rapid-firing weapons. Moreover, the legislation would be strengthened if it closed loopholes that allow gun buyers to get around the background check legislation and other purchase prohibitions by exempting gun shows and internet or person-to-person purchases, which were exempted from the FAWB and LCM ban [87].