NARA's approach to Key Indicators and Differential Monitoring is grounded in regulatory science, focusing squarely on Fiene's theory of regulatory compliance which clearly demonstrates the efficacy of the risk assessment and key indicator methodologies as being the most cost effective and efficient ways to implement differential monitoring. Click HERE to learn more.

• Differential monitoring is a method used to determine the frequency of inspections needed and the scope that will be required based on the setting’s licensing history. This method focuses measuring tools to determine the number and scope of inspections that should be conducted at a licensed setting.

• Integrated Monitoring is another approach to program monitoring which integrates regulatory compliance and quality into a continuous approach. Quality standards are the focal point of integrated monitoring systems.

Focused measuring tools are licensing inspection instruments that increase the effectiveness and efficiency of a regulatory oversight agency without producing recurring operational costs. In other words, focused measurement tools maximize performance while minimizing costs.
Click HERE to learn more.

1. Weighted Risk Assessment assigns weighted scores to licensing regulations based on the level of risk to persons in care in the event of regulatory noncompliance. Click HERE to learn more.

2. Key Indicator Systems (KIS) identify a subset of licensing regulations from an existing set of regulations that statistically predict regulatory compliance with the entire set of regulations. Click HERE to learn more.

3. Quality Indicator Systems identify a subset of licensing regulations and other non-regulatory factors to predict the overall quality of care provided in a licensed setting. Click HERE to learn more.

The purpose of a licensing indicator system is to increase the efficiency and effectiveness of an existing licensing system by refocusing the emphasis of the licensing process. It is intended to complement, and not replace, an existing licensing measurement system.

Additional information, including the Schedule of Services for Measurement Tools, can be viewed on NARA – The Right Choice. 

Resources

• A Treatise on the Theory of Regulatory Compliance, by Richard J. Fiene, Ph.D.
• Regulatory Compliance Key Indicator Metric and Matrix Update/Revision Technical Research Note, by Richard Fiene, Ph.D.
• Risk Assessment and Licensing Decision Making Matrices: Taking into Consideration Rule Severity andRegulatory Compliance Prevalence Data, by Sonya Stevens, Ed.D. & Richard Fiene, Ph.D.
• Technical Research Notes Regulatory Science, Differential Monitoring, Licensing Measurement
• Regulatory Compliance and Monitoring Systems (4th Edition)
• NARA Selected Readings LMS Fiene
• Licensing Measurement and Monitoring Systems Webinar Slides & Notes
• Journal of Regulatory Science: Regulatory Compliance Monitoring Paradigms and the Relationship of Regulatory Compliance/Licensing with Program Quality: A Policy Commentary
• Regulatory Compliance and Quality Programs: Constraints and Opportunities for Integration
• The Uncertainty-Certainty Matrix for Licensing Decision Making, Validation, Reliability, and Differential Monitoring Studies
  
  

  ---

Studies and Projects

Foundational Studies

• Colorado, 2014 - This study analyzed Colorado's quality rating system, the Qualistar Rating, for generating key indicators. Read the full study.
• Georgia, 2014 - Validation of Georgia's process for determining if state-regulated child care facilities are compliant with basic state health and safety requirements. Read the full study.
• Oregon, 2013 - This study provided Oregon's OCC with a basic risk factor analysis comparing its child care center rules to Stepping Stones (SS) standards. Read the full study.
• Head Start, 2013  - This project provided the Office of Head Start (OHS) Key Indicators for their Head Start Performance Standards. Read the full study.

Differential Monitoring Projects (NARA) 

• Saskatchewan 2022 - 2023
• Focus: The Saskatchewan Ministry of Education has completed the design, development, and validation of a new Program Quality Indicator (PQI) Tool/Scale. Saskatchewan Program Quality Indicators Tool This is the last component to their design of a comprehensive differential monitoring system.
  Project Activities: Key indicators methodology, Quality indicator development, regulatory analysis assessed with program quality, demonstration of the regulatory compliance ceiling effect, use of the regulatory compliance scale, data collection and data analysis for the development of the quality key indicators, center based infant, toddler and preschool classrooms, use of the ERS scales for validation purpose, validation and reliability testing of scale, independent observers/data collectors utilizing PQI and ERS scales, and development of the PQI scale. Read the Full Report here. 
• Florida, 2021
  Focus: Development of a Differential Monitoring Key Indicator System (KIS) for the Florida Department of Children of Families, Office of Child Care (OCC).
  Project Activities: stakeholder feedback sessions, data collection and analysis, development of an inspection checklist, policy and procedure development, and staff and stakeholder training. Read the Full Report here.
• Alberta, Canada, 2020 - 2021
  Focus: Development of Alberta Child Care’s licensing Key Indicator System (KIS) as well as additional analyses and recommendations based on the findings for Alberta Child Care’s Child Care Centers, Preschool Programs, and Out of School Care (OSC).
  Project Activities: data collection and analysis to determine key indicators. Read the Licensing Key Indicator Predictor Rule Results Report here - coming soon.
• Washington, 2019 - 2020
  Focus: Validation of Washington State’s Child Care Risk Assessment and Licensing Decision Making Tiered System
  Project Activities: validation study of Washington State’s Department of Children, Youth and Families child care Risk Assessment Licensing Decision Making Tiers System (RALDMTS). The validation involves two key components: 1) Validation of the measurement strategy used to determine the licensing decision making for child care centers and family child care homes; 2) Validation of the licensing system in juxtaposition to the program quality measures (ERS & CLASS) as part of their QRIS – Quality Rating and Improvement System utilized in Washington. Read the full report here.
• Washington, 2019 - 2020
  Focus: Licensing Measures and Outputs Validation Study for the Department of Children, Youth, and Families Child Care
  Project Activities: Validation of the measurement strategy used to determine the licensing decision making for child care centers and family child care homes; Validation of the licensing system in juxtaposition to the program quality measures (ERS & CLASS) as part of their QRIS – Quality Rating and Improvement System. Read the full report.
• Saskatchewan, Canada, 2019 - 2021
  Focus: Key Indicator study and Weighted Risk Assessment study for the Ministry of Education in the Province of Saskatchewan
  Project Activities: Key Indicator study as well as trends in regulatory compliance; Home-based Weighted Risk Assessment.  Read the full Key Indicator report. Read the full Weighted Risk Assessment report. Read the full CCC/FCC Validation Study Report here.Read the Full Report here.
• Minnesota, 2019
  Focus: Family Child Care Key Indicator Study for Minnesota Department of Human Services Licensing Division
  Project Activities: Key Indicator Data Analysis. Read the full report. Follow-up: Minnesota Family Child Care Task Force - Interim Legislative Report, March 2020
• Indiana, 2018 - 2019
  Focus: Key Indicators for the Family and Social Services Administration, Office of Early Childhood and Out of School Learning
  Project Activities: Focus Groups; Survey & Data Collection; Statistical Analysis; Indicator Work Tools & Manual; Training. Read the full report.
• Montana, 2016 - 2017
  Focus: Development of Key Indicators for Department of Public Health and Human Services
  Project Activities: Statistical Analysis; Policy Development; Staff Training; Stakeholder Meetings. Read the full report.
• Michigan, 2016 - 2017
  Focus: Development of Key Indicators for licensed child care and group homes and registered family homes
  Project Activities: Data Collection & Analysis; Policy Development; Work Tools and Instruction Manual; Staff Training; Stakeholder Meetings; Pilot Program; Statewide Implementation. Read the full report.
• Ontario, 2015 - This project outlined recommended methodology for developing an abbreviated inspection tool; a tiered licensing approach to child care licensing. Read the full report.
• Illinois, 2013 - 2014
  Focus: Development of a Weighted Licensing and Key Indicator System
  Project Activities: Determination of Eligibility Criteria; Data Collection & Analysis; Identification of Additional Factors; Development of Indicator Work Tools & Instructional Manual; Staff Training; Stakeholder Meetings. Read the full report. Follow-up: Illinois DCFS - Day Care Licensing Annual Report to the General Assembly, FY 2019
• Kansas, 2012 - 2013
  Focus: Develop a Key Indicator System
  Project Activities: Data Capacity Building and Analysis; Policy Development; Stakeholder Meetings. Read the full report.
• Washington, 2011 - This project provided Washington's Department of Early Learning a differential monitoring process to ensure efficiency while keeping children in licensed safe care. Read the full study.