A behaviorally-anchored rating system to monitor treatment integrity for community clinicians using the adolescent community reinforcement approach

This study evaluated a process for training raters to reliably rate clinicians delivering the Adolescent Community Reinforcement Approach (A-CRA) in a national dissemination project. The unique A-CRA coding system uses specific behavioral anchors throughout its 73 procedure components. Five randomly-selected raters each rated "passing" and "not passing" examples of the 19 A-CRA procedures. Ninety-four percent of the final ICCs were at least 'good' (≥.60) and 66.7% were 'excellent' (≥.75), and 95% of the ratings exceeded the 60% or better agreement threshold between raters and the gold standard. Raters can be trained to provide reliable A-CRA feedback for large-scale dissemination projects.