Part 2: An Evaluation on the Impact of Active Bystandership Training
This post presents findings from an ongoing evaluation of Active Bystandership (AB) training conducted by the Baltimore Police Department (BPD) through the Ethical Policing is Courageous (EPIC) program. EPIC has several components including officer training, increased support for officer health and wellness, and policy and procedure changes to facilitate more active bystandership behavior.
We previously reported findings from a survey of officers who took the training. The survey found that officers felt more confident and willing to intervene because of the training and that they recognized the benefits of AB for the health and safety of their coworkers.
In Part 2, we explore the program’s potential impacts on several key organizational outcomes.
Data and Methodology
The research team analyzed data related to the following outcomes:
- Community complaints against officers, which can include a range of alleged misbehaviors.
- Officer use of force. BPD officers must document any time force is used, and all officially recorded use of force events were included in our analysis.
- Referral of officers for (a) early Intervention or (b) support and guidance. BPD uses both early intervention and support and guidance to reduce the likelihood that minor problems develop into serious problems.
For all outcomes, we explored changes in the volume of events over time, aggregated to counts per month. ARIMA time series analysis adjusted for serial correlation in the error structure while exploring the effect associated with the intervention. Officers received Active Bystander training between November 2020 and April 2021. To understand the impact of the training, we analyzed data from January 2017 through December 2021. Although the implementation of EPIC had other components, we focus our analysis on the implementation of the officer training.
Using monthly data, we generated an intervention variable that was set to 0 prior to November 2020, and 1 afterwards. We created a linear trend variable to understand the overall trend in the outcome, and an interaction variable between trend and intervention to investigate changes in trend in the outcome following the onset of the training.
The frequency of contact between the police and community can affect outcomes. Complaints or use of force, for example, may increase when the police have more frequent contact with the community. To control for the contact effect, we obtained calls for service data, arrest, and part I incidents data from 2017 through 2021 from Baltimore City’s Open Data Portal. These data were aggregated at the monthly level and merged with the main dataset.
We also controlled for the number of COVID cases per 1,000 population in Baltimore and the time-period before and after the murder of George Floyd. Seasonal components were examined and adjusted for when appropriate. Analyses were conducted using the forecast package in R.
Results
Community Complaints – Complaints against officers were on a downtrend from 2017 (at the start of available data) to April 2020. After April 2020, complaints began rising. Quantifying the exact effect of the intervention is challenging given the macro factors (e.g., changes in routine activities and police-community contacts due to Covid and the social justice protest in the summer of 2020) that occurred around the same time as the intervention (November 2020 to April 2021).
The data show that arrest activities and the number of sworn officers were significantly and positively associated with the number of monthly complaints. Additional officers on the street potentially means more contact with the community, and greater arrest activities could increase the number of negative encounters that would result in a complaint.
Nevertheless, we found no evidence that the EPIC training reduced or reversed the upward trend in monthly complaint counts.
Use of Force – Officer use of force in Baltimore has been decreasing significantly over time. We found that the number of sworn officers was inversely related with use of force–the more officers, the less use of force. This may suggest that understaffing of officers or increased workload could contribute to use of force. The exact mechanism of this relationship is unknown, but it seems likely that increasing work demands or work stress can reduce an officer’s ability to effectively deescalate situations.
Unlike complaints, we did not see any reversal in the trends for use of force post-2020. Use of force events had been steadily dropping and continued to do so following the training. The increases in citizen complaints beginning in early 2020 may have reflected changes in public attitudes or the procedures related to citizen reporting more than officer behaviors.
Even though use of force events declined over time, the implementation of EPIC training did not appear to create additional reductions over what would have been expected.
Early Intervention and Support and Guidance Referrals – There were no consistent trends in early intervention referrals over the study period, although the slight downward trend began increasing after the training. The rise in referrals post-intervention was not statistically significant, but it could indicate an organizational culture that was becoming more supportive of peer intervention. Arrest activities were positively associated with early intervention activities; part I crime was negatively associated with early interventions. Holding everything else constant, more arrests and fewer part I crime were associated with more early intervention activities.
Support and guidance referrals increased over time; the greatest increase occurred before the training.
What did we learn?
The pandemic (and its associated changes in routine activities) and the national demonstrations demanding policing reform brought about significant changes to the environment in which law enforcement operates.
Agencies across the nation reported reducing their contact with community members due to COVID. States passed legislation to reform various aspects of policing practices–use of force, training, certification, oversight, union restrictions, duty to intervene. At the same time, major cities have experienced persistent increases in violence and gun crime. Further, the staffing shortages in law enforcement have been exacerbated due to police resignations following the protests and reduced recruitment ability due to perceived public hostility against police officers.
Against this backdrop, it is perhaps unsurprising to see an increase in citizen complaints during and after 2020. Similar changes in the pattern for use of force, however, were not observed. Together, these patterns seem to indicate that the underlying cause may be related to changes in citizen reporting and the broader social environment rather than changes in policing behaviors.
Results suggests that EPIC training did not have a significant impact on the key outcomes the study examined, but results should be interpreted with caution. Macro factors, such as the COVID public health crisis and the nationwide protests, may have overpowered the effect of the training especially since they occurred closely in time. Whether these increases would have been even stronger without the training cannot be answered in the current context due to a lack of appropriate comparisons.
While the legitimacy crisis in law enforcement may not fully account for the increases in complaints, it appears to have been an important factor that shaped the social environment in which these changes occurred. Specifically, the start of increases in citizen complaints was closely aligned with the timing of the murder of George Floyd.
On the positive side, police use of force continued to decline following COVID-19, demonstrations for police reform, and the implementation of EPIC. We also saw increases over time in the number of support and guidance referrals for officers. Over time, this approach could help improve officer health and wellness. Finally, despite the increases in complaints against officers, it stayed substantially below pre-COVID levels.
Although the results from our analysis do not demonstrate that EPIC training had a consistently positive impact, there is reason for optimism and continued study. Data from the survey of officers (reported previously), found high levels of support for the training and belief that the training would result in positive organizational changes.
Our ability to explore important outcome metrics was hampered by larger social and environmental factors that made evaluation difficult. Longer-term follow-up will be essential for understanding if this trend continues as the public health crisis associated with COVID eases and community activity levels return to more typical levels.
