Dr. Caillin Langmann’s Criticism of:
Firearm Legislation and Firearm-Related Fatalities in the United States
By Eric W. Fleegler, MD, MPH; Lois K. Lee, MD, MPH; Michael C. Monuteaux, ScD; David Hemenway, PhD; Rebekah Mannix, MD, MPH
JAMA Intern Med. 2013;():1-9. doi:10.1001/jamainternmed.2013.1286.
As published online March 6, 2013 https://archinte.jamanetwork.com/article.aspx?articleid=1661390
– The study by Fleegler, et al.,:
- Does not actually conclude that States with higher gun laws have less homicide or suicide, in fact shows no association
- Demonstrates Assault Weapon bans have no effect on homicide
- Demonstrates Laws that prohibit guns in public places have no effect on homicide
– The study is simplistic with only limited analysis or conclusions
– It only examines a single time period which is an inferior method of demonstrating association
– It suffers from potential data selection bias by leaving out relevant information to achieve a biased result, ie: District of Columbia with high gun laws and high homicide rates is left out
– The scoring of laws is highly suspect and is an unvalidated scoring system
This study in JAMA is a very simple study with very limited and basic analysis. Even so, it actually demonstrates several interesting conclusions, two of which are that Assault Weapons have no effect on homicide, nor do laws that allow guns in public places, such as Conceal Carry Licenses. Moreover it does not actually conclude that States with higher gun laws have less homicide or suicide.
There are some problems with the study. This study only looks at a single shot in time, these are the worst epidemiological studies in terms of determining an association. For example I might take two States, one with more TVs and one with less and compare cancer rates. I might find an association between more TVs and higher cancer rates. However the real question is, is cancer really related to TV’s or some other factor? Has cancer or the number of TV’s been increasing or decreasing over time? What were cancer rates in these States before TVs, could it have been higher in the high cancer State before TVs?
Better studies look at what happens over time. Do changes in laws actually change the rates of homicide or suicide? Do changes in the number of TVs increase cancer or decrease? Those are the more valid studies, and many have been published showing no effects from these laws. For instance John R Lott and several other independent authors have published studies in this area looking at the effects of guns over time in a large sample of counties and find, as his book is called, “More Guns, Less Crime”. The other question is: are there more guns in some States because there is more crime and people respond by aquiring guns to defend themselves. Gary Kleck looks at this issue in a much more rigorous statistical analysis and finds that once one takes into account that factor, one finds that higher gun ownership decreases homicide rates. I looked at a closed system, Canada, for changes in homicide and spousal homicide using a time based analysis to take into account prior trends unrelated to legislative effects and also concluded that there was no association between Canada’s gun laws and homicide or spousal homicide as well as crimes such as discharge of firearm with intent.
Let’s actually look at the study. The first issue is the categorization of gun laws into a scoring system. This is what we call an unvalidated scoring system. In other words why do certain laws get more points in the system? Does it mean a certain law is more effective? How do they know that? What basis are laws assigned more points? An honest peer reviewer would ask the question: did you give more points to certain laws to increase your chances of finding a statistical result at the end? For example I want to determine the effects of several drugs on cancer rates. I assign the drug I like 5 points and the drug I don’t like 1 point. Then I run my regression and of course weighting the drug I like higher causes it to provide a stronger effect. In this paper what one could do is take a state with a low rate of homicide and arbitrarily give it a very high score compared to other states until one found a difference in your statistical analysis. This is what could be happening in Figure 2 of their paper.
Moreover, in Figure 2, when one includes both independent and dependent variables in one’s regression one will end up with a relationship. In this case they include suicide by firearm/total suicide in the regression as an independent or causal factor and then determine that it is related to gun deaths. Of course it would be, just like if I included automobile accidents/any accident as a causal factor and then related it to the result of death.
Second let’s take look at the conclusions from the results. When writing a paper one has to be conscious of bias of interpretation of results. When you write a paper you have to ask yourself could the results be interpreted to make an entirely opposite conclusion? Could one conclude that firearms laws are having no effect?
Looking at Table 3 in the paper, one looks at Model 3 because it has the most other socioeconomic factors in it like poverty, etc. to account for other causes of homicide rates as well as firearms ownership. In the model 3 column under suicide: no significant association is found.
Under homicide it appears states with the highest number of laws have no statistical association, contradicting the conclusion that higher laws is associated with less homicide. The only significant result is that a State with some laws has some effect. However if you look at the confidence intervals, the range approaches 0.93 which means that its very close to not having an association at all. (1 would be no association).
Looking at Table 4 raises some questions. They conclude that Brady checks decrease suicide and homicide but the Confidence Intervals near 1, which means that the association is very weak or possibly not associated at all. One would never prescribe a drug if it had these results. As part of analysis, we should also apply a logic test to science whenever we look at correlations. For example it seems unclear logically why an assault weapon would increase your risk of suicide? I mean you only need one bullet for that. It should increase the risk of homicide but it doesn’t. Same with the guns in public places, one would question how it affects suicide? However from their results it doesn’t cause homicide rates to increase.
The next thing to examine is what is known as selection bias; that is when one leaves out data it could cause a different result. In this study they leave out North Dakota, which has an extremely low homicide rate like Hawaii and a low number of gun laws. Moreover, they also leave out the District of Columbia which has the highest number of gun laws as well as an extremely high homicide rate of 12.5/100,000. If one excludes Louisiana from the analysis, as an outlier with a very high homicide rate and a low gun law value, one finds that homicide rates are lower in the States with lower numbers of gun laws. Conversely if one adds the District of Columbia and North Dakota to the data one finds again that the States with lower gun laws have lower homicide rates. Excluding data is a dangerous step to take in any scientific analysis.
A rigorous scientist would look at these results and conclude that there is possibly no association between laws and homicide and suicide. Moreover one would also conclude that Assault weapons are not associated with homicide nor is guns in public places, ie: guns on campus or Conceal Carry laws. That final conclusion has been documented in multiple studies, including the National Academy of Sciences analysis of Firearms.
Dr. Caillin Langmann, M.D., Ph.D.
Div. Emergency Medicine