Best Practices for Implementing Safety Performance Metrics and KPIs Across Ontario Industries: Effectiveness of Compliance Frameworks in Supporting Comprehensive Safety Scorecards

Safety performance metrics and key performance indicators (KPIs)

I. INTRODUCTION

Overview

Safety performance metrics and key performance indicators (KPIs) are essential tools for organizations to monitor, manage, and improve occupational health and safety (OHS) outcomes. In Ontario, industries are governed by a range of compliance frameworks, including the Occupational Health and Safety Act (OHSA), Workplace Safety and Insurance Board (WSIB) reporting requirements, ISO 45001, and Certificate of Recognition (COR) audits. These frameworks aim to standardize safety practices and reporting, thereby supporting the development of comprehensive safety scorecards for safety professionals and business leaders (Doronina & Safonov, 2016)(Tappura et al., 2022).

Context

The implementation of safety metrics involves both leading indicators (proactive measures such as safety training, hazard identification, and safety climate) and lagging indicators (reactive measures such as injury rates and lost time incidents). The effectiveness of compliance frameworks in Ontario is measured by their ability to facilitate the collection, analysis, and application of these indicators across diverse industries, enabling organizations to proactively manage risks and improve safety outcomes (Manjourides & Dennerlein, 2019)(Tappura et al., 2022). This report synthesizes data-driven findings from recent research to identify best practices for implementing safety metrics and evaluates the effectiveness of current compliance frameworks in supporting comprehensive safety scorecards.

II. DATA SYNTHESIS

Data Trends and Key Findings

1. Importance of Leading and Lagging Indicators

• Leading Indicators: Safety Management Systems (SMS), safety climate, psychosocial safety climate, and proactive OHS management practices are consistently linked to improved safety performance (Manjourides & Dennerlein, 2019)(Omidi et al., 2021)(Tappura et al., 2022).

• Lagging Indicators: Recordable injury case rates (RC), days away, restricted, or transferred (DART) rates, and adverse event reporting are standard lagging metrics (Manjourides & Dennerlein, 2019)(Greenberg et al., 2009).

2. Effectiveness of Safety Management Systems and Audits

• Each one-point increase in SMS scores is associated with a 34% reduction in the odds of a recordable case rate greater than zero (OR: 0.66, 95% CI: 0.57–0.79) and a 9% reduction in recordable case rate if one occurs (RR: 0.91, 95% CI: 0.88–0.94) (Manjourides & Dennerlein, 2019).

• OHS management audits show limited evidence of reliability and validity, with only 5 out of 17 audit methods demonstrating content validity and only 3 showing acceptable inter-rater reliability (Robson & Bigelow, 2010).

3. Key Factors Influencing Safety Performance

• "Rule breaking" and "organizational communication" are the most influential safety climate and psychosocial safety climate factors, respectively, affecting safety performance (Omidi et al., 2021).

• The safety performance map identifies 42 factors across seven perspectives: OHS management, leadership, structure, processes, culture, individual behavior, and performance (Tappura et al., 2022).

4. Compliance Frameworks and Standards

• ISO 45001 emphasizes risk-oriented thinking and integrated management systems, supporting proactive safety management (Doronina & Safonov, 2016).

• COR audits and WSIB reporting requirements provide structured approaches but face challenges in measurement reliability and comprehensive coverage (Robson & Bigelow, 2010).

5. Digital Tools and Data Integration

• Digital Twin (DT) technology is increasingly used to support safety management, enabling real-time monitoring and dynamic updating of safety metrics (Agnusdei et al., 2021).

  
  

Dynamic Table 1: Summary of Key Data Points on Safety Metrics and Frameworks

Metric/Framework	Leading Indicators	Lagging Indicators	Effectiveness (Key Data)	Limitations/Challenges	Reference
Safety Management System (SMS)	SMS score, safety programs, special elements	RC, DART rates	34% reduction in RC odds per 1-point SMS increase	Inconsistent association for some programs	(Manjourides & Dennerlein, 2019)
OHS Management Audits	Audit scores, compliance	Audit-based performance	Only 5/17 methods with content validity; low inter-rater reliability	Sparse evidence, reliability issues	(Robson & Bigelow, 2010)
ISO 45001	Risk-oriented thinking, integrated management	Not specified	Supports proactive, preventive safety management	Implementation complexity	(Doronina & Safonov, 2016)
Safety Climate Factors	Rule breaking, communication	Safety performance scores	Rule breaking and communication most influential	Focused on process industry	(Omidi et al., 2021)
Digital Twin Technology	Real-time data, simulation	Not specified	Enhances operator control and safety monitoring	Emerging, not widely adopted	(Agnusdei et al., 2021)
Safety Performance Map	42 factors across 7 perspectives	OHS outcomes	Recurring factors across industries; supports proactive measurement	Qualitative, needs quantitative validation	(Tappura et al., 2022)

III. ANALYSIS

Detailed Analysis

A. Best Practices for Implementing Safety Metrics and KPIs

1. Balanced Use of Leading and Lagging Indicators

2. Leading indicators such as SMS scores, safety climate, and proactive OHS management are strongly associated with reduced injury rates and improved safety outcomes (Manjourides & Dennerlein, 2019)(Omidi et al., 2021)(Tappura et al., 2022).

3. Lagging indicators remain essential for benchmarking and regulatory compliance but should be complemented by leading indicators for a comprehensive safety scorecard (Greenberg et al., 2009).

4. Prioritization of Influential Factors

5. "Rule breaking" and "organizational communication" should be prioritized in safety climate assessments, as they have the greatest influence on safety performance (Omidi et al., 2021).

6. The safety performance map's 42 factors provide a holistic framework for measuring and managing safety, applicable across industries (Tappura et al., 2022).

7. Integration of Digital Tools

8. Adoption of Digital Twin technology enables dynamic, real-time monitoring of safety metrics, supporting both leading and lagging indicators (Agnusdei et al., 2021).

B. Effectiveness of Compliance Frameworks

1. Ontario’s OHSA and WSIB Reporting

2. These frameworks mandate the collection and reporting of lagging indicators (e.g., injury rates), providing a baseline for safety performance measurement (Manjourides & Dennerlein, 2019)(Greenberg et al., 2009).

3. However, they are less prescriptive regarding leading indicators, which limits their effectiveness in supporting proactive safety management (Tappura et al., 2022).

4. ISO 45001

5. ISO 45001’s risk-oriented and integrated management approach aligns with best practices for proactive safety measurement, supporting the development of comprehensive scorecards (Doronina & Safonov, 2016).

6. Implementation requires organizational commitment and may face challenges in integration with existing systems (Doronina & Safonov, 2016).

7. COR Audits

8. COR audits provide structured assessments but suffer from limited evidence of measurement reliability and validity, raising concerns about their use as performance metrics (Robson & Bigelow, 2010).

   
   

Dynamic Table 2: Effectiveness of Compliance Frameworks in Supporting Safety Scorecards

Framework/Standard	Support for Leading Indicators	Support for Lagging Indicators	Measurement Reliability	Proactive/Reactive Focus	Key Limitations	Reference
OHSA	Limited	Strong	High (for lagging)	Reactive	Lacks leading indicator focus	(Manjourides & Dennerlein, 2019)(Greenberg et al., 2009)
WSIB Reporting	Limited	Strong	High (for lagging)	Reactive	Focus on post-incident data	(Manjourides & Dennerlein, 2019)
ISO 45001	Strong	Not specified	Not assessed	Proactive	Implementation complexity	(Doronina & Safonov, 2016)
COR Audits	Moderate	Moderate	Low	Mixed	Reliability/validity concerns	(Robson & Bigelow, 2010)

C. Patterns and Discrepancies

• There is a consistent pattern that leading indicators, when measured and managed effectively, result in improved lagging outcomes (e.g., reduced injury rates) (Manjourides & Dennerlein, 2019)(Omidi et al., 2021).

• Compliance frameworks are more effective at supporting lagging indicators than leading indicators, with ISO 45001 being the exception due to its proactive, risk-based approach (Doronina & Safonov, 2016).

• Measurement reliability is a significant challenge, particularly for audit-based frameworks like COR (Robson & Bigelow, 2010).

  
  

IV. DISCUSSION

Contextualizing Data

The data indicate that best practices for implementing safety performance metrics involve a balanced approach that integrates both leading and lagging indicators. Leading indicators such as SMS scores, safety climate, and proactive OHS management practices are critical for predicting and preventing adverse safety outcomes (Manjourides & Dennerlein, 2019)(Omidi et al., 2021)(Tappura et al., 2022). Lagging indicators, while essential for compliance and benchmarking, are insufficient on their own for comprehensive safety management (Greenberg et al., 2009).

Compliance frameworks in Ontario provide a strong foundation for lagging indicator measurement but are less effective in supporting the systematic collection and use of leading indicators. ISO 45001 stands out as a framework that encourages proactive, risk-based safety management, aligning with best practices identified in the literature (Doronina & Safonov, 2016). However, the effectiveness of audit-based frameworks like COR is limited by concerns about measurement reliability and validity (Robson & Bigelow, 2010).

Digital tools such as Digital Twin technology offer promising avenues for real-time, dynamic safety monitoring, but their adoption is still emerging (Agnusdei et al., 2021).

Gaps and Areas for Further Research

• There is a need for improved measurement reliability and validity in audit-based frameworks (Robson & Bigelow, 2010).

• More research is required to validate the effectiveness of digital tools and integrated management systems in diverse industrial contexts (Agnusdei et al., 2021).

• Frameworks should evolve to mandate or incentivize the use of leading indicators for a more proactive approach to safety management (Tappura et al., 2022).

  
  

V. CONCLUSION

Summary

The synthesis of data from recent research demonstrates that the best practices for implementing safety performance metrics and KPIs in Ontario industries involve a balanced integration of leading and lagging indicators. Leading indicators, particularly those related to safety management systems, safety climate, and organizational communication, are strongly associated with improved safety outcomes. Compliance frameworks such as OHSA and WSIB reporting are effective in supporting lagging indicator measurement but are less robust in facilitating proactive, leading indicator-based safety management. ISO 45001 provides a comprehensive, risk-based approach that aligns with best practices, while audit-based frameworks like COR face challenges in measurement reliability.

Answer to the Research Question

The best practices for implementing safety performance metrics and KPIs across all industries in Ontario include:

• Integrating both leading (e.g., SMS scores, safety climate, proactive management) and lagging (e.g., injury rates, DART) indicators into safety scorecards (Manjourides & Dennerlein, 2019)(Omidi et al., 2021)(Tappura et al., 2022).

• Prioritizing influential factors such as rule breaking and organizational communication in safety climate assessments (Omidi et al., 2021).

• Leveraging digital tools for real-time safety monitoring where feasible (Agnusdei et al., 2021).

• Utilizing ISO 45001 as a framework for proactive, risk-based safety management (Doronina & Safonov, 2016).

  
  

Current compliance frameworks are effective in supporting lagging indicator measurement but require enhancements to fully support comprehensive, proactive safety scorecards. ISO 45001 offers the most robust support for leading indicators, while audit-based frameworks need improved measurement reliability to be effective performance tools (Robson & Bigelow, 2010)(Doronina & Safonov, 2016).

Recommendations

• Mandate or incentivize the use of leading indicators in compliance frameworks to promote proactive safety management.

• Improve the reliability and validity of audit-based frameworks such as COR to ensure accurate performance measurement.

• Encourage the adoption of digital tools for dynamic, real-time safety monitoring and data integration.

• Adopt a holistic approach, as outlined in the safety performance map, to ensure all relevant factors are considered in safety scorecards (Tappura et al., 2022).

Dynamic tables throughout this report summarize the critical data points and framework comparisons, providing a clear, evidence-based foundation for best practices in safety performance measurement and management in Ontario.

References

Doronina, O., & Safonov, A. (2016). [New International Standard ISO 45001:2016 <>. Meditsina Truda i Promyshlennaia Ekologiia, 10, 40–44.

Tappura, S., Haapavirta, R., & Jääskeläinen, A. (2022). Designing a map for measuring and managing safety performance. In International Journal of Occupational Safety and Ergonomics (Vol. 29, Issue 2, pp. 613–626). Informa UK Limited. https://doi.org/10.1080/10803548.2022.2061759

Manjourides, J., & Dennerlein, J. T. (2019). Testing the associations between leading and lagging indicators in a contractor safety pre‐qualification database. In American Journal of Industrial Medicine (Vol. 62, Issue 4, pp. 317–324). Wiley. https://doi.org/10.1002/ajim.22951

Omidi, L., Salehi, V., Zakerian, S. A., & Nasl Saraji, J. (2021). Assessing the influence of safety climate-related factors on safety performance using an Integrated Entropy-TOPSIS Approach. In Journal of Industrial and Production Engineering (Vol. 39, Issue 1, pp. 73–82). Informa UK Limited. https://doi.org/10.1080/21681015.2021.1958937

Greenberg, M. D., Haviland, A. M., Yu, H., & Farley, D. O. (2009). Safety Outcomes in the United States: Trends and Challenges in Measurement. In Health Services Research (Vol. 44, Issue 2p2, pp. 739–755). Wiley. https://doi.org/10.1111/j.1475-6773.2008.00926.x

Robson, L., & Bigelow, P. (2010). Measurement properties of occupational health and safety management audits: a systematic literature search and traditional literature synthesis. Canadian Journal of Public Health, 101 Suppl 1, S34-40.

Agnusdei, G. P., Elia, V., & Gnoni, M. G. (2021). Is Digital Twin Technology Supporting Safety Management? A Bibliometric and Systematic Review. In Applied Sciences (Vol. 11, Issue 6, p. 2767). MDPI AG. https://doi.org/10.3390/app11062767