In the summer of 2012, the city of Seattle entered into a Settlement Agreement with the U.S. Department of Justice (DOJ) arising out of DOJ’s investigation of the Seattle Police Department’s (SPD’s) use of force and concerns around biased policing. Although the DOJ investigation did not find that SPD officers engaged in biased policing, it did note concerns around outcome disparities. As a result, SPD implemented new policies and training, improved data warehousing and processing infrastructure, explored patterns of disparity, and evolved its understanding of the collateral harms associated with policing.
Police Performance Management via an Equity, Accountability & Quality (EAQ) Program
In 2020, Seattle completed a 2-year evaluation period that confirmed the stability of improvements made under the Settlement Agreement. This coincided with an unprecedented call for police reform nationwide. In response to this need, and to build on SPD’s improved accountability, SPD developed a new performance management program that focuses on measuring the Equity, Accountability, and Quality (EAQ) components of policing. This project will review disparate outcomes of policing, over- and under-policing of communities, and service quality for awareness, mitigation, and continuous improvement.
SPD is undertaking this effort in partnership with the Office of Inspector General and is collaborating with RTI International as an external quality assurance and evaluation partner.
Challenges of Measuring EAQ in Policing
Police departments across the United States use a police performance management system called Compstat to track and measure crime data. Compstat facilitates timely information sharing with police leadership to identify and address crime concerns and areas of police performance improvement.
SPD’s EAQ program will supplement Compstat’s goals by providing a more holistic look at police performance through the following evaluation methods:
Equity: SPD will use advanced statistical techniques to monitor racial disparities in police stop outcomes. By controlling for other, non-race factors present during the stop, evaluators can identify and compare equity issues.
Accountability: SPD will use Automatic Vehicle Locator (AVL) data, which track an officer’s location every few seconds, to assess over- or under-policing across the community. Evaluators will look for disparities in where police spend their time and where police are needed throughout the city, as determined by calls for service.
Quality: SPD uses body-worn cameras (BWC), and automated transcription of BWC footage can assess the quality of interactions between officers and residents. Machine learning methods will interpret words and phrases in the interaction to assess professionalism and the overall sentiment of the conversation. At a system level, this will highlight areas for additional training or intervention.
Ongoing Monitoring for Equity, Accountability, and Quality in Police Performance
SPD will monitor EAQ measures through an accountability forum that takes place regularly. The data collected from the EAQ measures will be aggregated and will not be available at an individual officer level. The data are intended to provide system-level analysis and not individual performance metrics.
Importantly, EAQ will not replace Compstat, but it will provide a greater, more holistic understanding of the implications of policing on communities. Put simply: It will offer a look at how police are interacting with the community beyond crime statistics.
Evidence-Based Analysis for Greater Police Accountability
RTI serves as a third-party quality assurance and evaluation partner of SPD’s EAQ program. Innovative programs like this necessitate third-party evaluation to ensure that evidence-based, scientific rigor is applied to the methodology. Given the country’s call for greater transparency and accountability of police, validation from a third party like RTI also provides important credibility and legitimacy—departments and communities interested in implementing the EAQ program can feel confident in the methodology. This is especially important as other departments may follow SPD’s lead and set a new standard for the field.
Although Compstat will remain a ubiquitous system to assess crime and police performance, this equity, accountability and quality project paves the way for a more comprehensive performance management system to complement Compstat and will improve SPD’s service to Seattle residents. SPD’s EAQ project is a novel response to a critical and timely national issue: evaluating and improving policing in the United States.
