By Scott Burris
The most important topic we did NOT address in our PHLR methods book was valid methods for rating laws for characteristics like “stringency.” I am not aware of any general work on this. Nonetheless, it is not uncommon for researchers to create scales purporting to measure the distribution of some characteristic(s) over a group of laws. It seems often to be done by some facially plausible means (e.g., penalties) or through a Delphi or similar expert process. For example, Woodruff and colleagues1 developed a stringency scale for tanning laws that distributes characteristics of laws (age covered, standards for eye protection) on 2-5 point scales based on a priori judgments. Chriqui and colleagues used an expert advisory committee to rate the strength of clean indoor air laws.2 For alcohol control policies, Nelson et al used a Delphi method.3 As a field, however, I can’t see that we have much consensus on how to create and validate such scales. So to start a discussion, some thoughts on a basic typology of scales, with some possible measures and examples off the top of my head:
Approach | Possible Measures | Examples |
Assessment based on apparent features of the legal text | Magnitude of penalty | Fine; imprisonment |
Comprehensiveness of coverage of categories of actors engaged in the regulated activity | Distracted driving: all drivers, novice drivers, bus drivers | |
Comprehensiveness of coverage of specific behaviors constituting or relating to the regulated activity | Distracted driving: all device use; manual use; texting | |
Procedural efficiency | Number of distinct steps required to enforce or comply with law | |
Legal assessments | Consistency with other requirements (constitutionality or preemption, for example) | |
Qualitative assessments | “Clarity” or “specificity” of rule | |
Assessment based on evidence, expert knowledge or prediction of the implementation of the law by legal agents | Incentives for enforcement | Resources, required reporting of enforcement actions/outcomes |
Social marketing investment | MADD social marketing against drunk driving | |
“Technical” feasibility of enforcement | Consistency with mechanisms/methods already in use, cost, procedural complexity | |
“Social” feasibility of enforcement | Normative consistency with current practices or values,* political constraints | |
“Legal” feasibility of enforcement | Probability of legal challenge, procedural complexity | |
Qualitative assessments | “Clarity” or “specificity” of rule as perceived by enforcers | |
Assessment based on evidence, expert knowledge or prediction of the reaction to the law of regulated parties | Likelihood that regulated parties will learn of the law | Social marketing, publicity, high enforcement levels |
Consistency with general theories of compliance | Deterrence, legitimacy, procedural fairness, expected utility of compliance | |
Social support for compliance | Consistency of required behavior with current norms* | |
Feasibility of compliance | Availability of technology, | |
Qualitative assessments | “Clarity” or “specificity” of rule as perceived by regulated parties | |
Hybrid methods |
* The normativity of the required behavior or enforcement mechanism would not be a stable measure, since we would expect passage and enforcement of the law to change norms over time (e.g., drunk driving)
Has my admittedly quick search for methods guidance on this missed some good sources? How does this rough typology and examples strike you? I’d be very happy to get some comments and suggestions.
1. Woodruff SI, Pichon LC, Hoerster KD, Forster JL, Gilmer T, Mayer JA. Measuring the stringency of states’ indoor tanning regulations: Instrument development and outcomes. Journal of the American Academy of Dermatology. 2007;56(5):774-780.
2. Chriqui JF, Frosh M, Brownson RC, et al. Application of a rating system to state clean indoor air laws (USA). Tob Control. Mar 2002;11(1):26-34.
3. Nelson TF, Xuan Z, Babor TF, et al. Efficacy and the Strength of Evidence of U.S. Alcohol Control Policies. American Journal of Preventive Medicine. 2013;45(1):19-28.