Terms like “Effect Size Homogeneity” sure to impress
By Dorothy Cotton
By Dorothy Cotton
Periodically, I get adoring fan mail from readers of this column, extolling my many fine virtues and Pulitzer-prize deserving writing. And periodically I get snippy email or letters from people who think I live in some ivory tower somewhere and am unacquainted with the Real World. The latter type of mail tends to come when I talk about research, particularly of the more academic type. Generally my response to these readers is to try to enlighten them to the error of their uninformed ways, and to encourage them to even consider the possibility that their own personal experience does not cover the entire gamut of possible knowledge. I am a firm believer that the best judgments and conclusions come from more rather than less knowledge, and that one source of knowledge is published peer reviewed research of the sort often done in universities.
That was my general line of thinking as I sat down to read a paper called “Just following orders: a meta-analysis of the correlates of American police officer use of force decisions,” by a guy named Colin Bolger who is at Miami University in Ohio. I should have known this was not going to end well when I saw that Miami University was in Ohio. Surely it should be in Florida?? I should have known there was going to be trouble when I noted, as I waded through the article, that it took 6 full pages to describe HOW he had done the study, what to include and what not to include. A meta-analysis, for those of you who have not run into this form of research before, is a study in which the authors do not actually do any more original research, but rather collect everything that has been done so far and try to figure out what it says it you take it all as one big lump. There are complicated statistics involved, and you get to use terms like “effect size homogeneity” and “fail safe N” and “trim and fill analysis.”
Bolger begins his study by noting that in the past, research on use of force has tended to examine four areas of interest:
• Characteristics of the encounter itself: Is there evidence of criminal behaviour? Does the suspect resist? Is there a weapon? Are there other people around?
• Characteristics of the suspect: Are alcohol or drugs involved? Does the suspect appear to be of lower class? Is the suspect male or female? Is race an issue? Is the suspect hostile? Is there evidence of mental illness?
• Characteristics of the officer: What is the officer’s level of education? How long have they been at this policing thing (years of experience)? What is the officer’s race? Gender?
• Characteristics of the community: What is the general socio-economic status of the community? How dangerous is the area? What is the racial composition of the community?
So Bolger dug up all the studies he could find, eliminated the ones that were badly done or otherwise weird, and ended up with 19 studies. And what did he find? (Roll of the drums, please.)
• Within the category of “encounter” variables, things are most likely to go south if the offense is serious, the suspect resists, there is an arrest, there is conflict, there are a whole bunch of officers involved and the contact was initiated by the police.
• Within the category of “suspect” characteristics, race, sex, demeanor, class and intoxication were all significant predictors.
• For officer characteristics, the only thing that was predictive was gender (males being more likely to use force)
• Nothing related to community characteristics panned out.
Hmm.. and from all this we have learned….I gotta say there is not a lot of news here. But sometimes, the interesting stuff is in what’s not there. Near the end of the article, the author comments that it appears that individual officer characteristics have little to do with anything, and it seems like mostly the use of force is determined by the characteristics of the situation itself. That makes good sense and I would think that that is how it should be.
I felt a little frustrated after reading this article, and only partly because I have no idea what things like “aggregate mean effect size by predictor domains” means. But I find frustrating that the research really does not address the issue of whether force should have been used and whether the appropriate level of force was used. I’d love to see a study that looks at incidents when the use of force was NOT appropriate to the situation—and see what precipitates the inappropriate use of force. I suspect there are different predictors for that…but we really do not know.
Yeah, I know that is pretty well impossible to do this kind of research. I can see a letter going out to all police officers from some university academic, saying “We are recruiting officers for our study and if you use force inappropriately and are more aggressive than you ought to be in citizen encounters, give us a call!”
So instead of reading this article, I suggest you read the report “Police Encounters with People in Crisis: Independent Review of the Use of Lethal Force by the Toronto Police Service” carried out by the Honorable Frank Iacobucci (online at http://www.tpsreview.ca). While the purpose of this report is in theory much narrower than an overall assessment of factors related to use of force and of course in places is very specific to the TPS, the fact is that the report is extremely comprehensive and does address many of the intricacies of the decisions that lead up to use of force—and in particular use of force that turns deadly. This report talks in detail about the categories of variables identified in Bolger study—and more. The fact that there are 84 recommendations speaks to the complexity of the problem—and the extensive bibliography, list of contributors, and variety of research strategies used by the author speaks to the need for a comprehensive approach to a complex problem. To quote the report itseld (p. 7):
Now THAT is research.
Have a look.