Key Performance Indicators for HSE Audit Programs
There continues to be intense discussion in the health, safety, and environment (HSE) audit community over how to evaluate the performance of audit programs. Senior executives in both private and public sector organizations want to know whether audit programs are meeting expectations and resulting in compliance improvements. Further, they are looking for clear measures that help them make that determination. Unfortunately, all too often, the focus is on reporting the total number of findings on audits as a simple and easily understood metric—and it is both of these. But is this metric meaningful?
The short answer to the question is no, principally because all findings are not created equal. For example, as discussed in a previous article by this author, “Let’s say that an audit team finds that a regulatory program does not exist at a site because the site erroneously believes that the program does not apply to them. One finding. A second team visits the site 2 years later and finds that much work has been undertaken and the program has been largely implemented. However, there are still four administrative requirements that are not being met completely. Four findings. It’s pretty clear that the one finding on the first audit far outstrips the importance of the four findings on the second audit. Hence, if one were using total number of findings as a measure, the results would be quite misleading.”
As also discussed in the previous article, similar fallacious logic is sometimes applied to other outcomes that might be anticipated by the implementation of an audit program, including a reduction in environmental releases and workplace injuries or improved compliance as defined by a reduction in fines and enforcement actions. And, in fact, even expecting a reduction in the number of overall findings on audits year-on-year can be misleading as well. This is principally due to change, which could include, among other factors, changes in management at the site, changes in the products manufactured and chemicals used, changes in physical facilities and equipment (e.g., decommissioning and startup of new process equipment); or changes in the volume and severity of regulations affecting the site. It’s interesting to note that federal US HSE regulations alone (i.e., Titles 29 and 40 of the Code of Federal Regulations) have increased by more than 3,200 pages in the past 5 years. This has consequences in meeting the goal of full compliance.
- If all of the above factors aren’t the best measures of success, how then should one approach this issue of assessing the performance of HSE audit programs? First, it might be valuable to identify the five fundamental questions that one would ask of any audit program in order to assess its effectiveness. They are:
- Has the universe of auditable facilities been identified?
- Are the facility audits being conducted at appropriate frequencies based on the risks posed?
- Are the audits conducted professionally using independent, qualified auditors supported by appropriate audit tools (e.g., protocols, regulatory databases)?
- Are audit reports issued promptly and are the identified audit findings corrected in a timely fashion?
- Is there a sense from the outcomes of the audits that the sites are actually improving?
Defining key performance indicators (KPIs) that would help to answer these questions might go a long way toward understanding the value that an audit program provides. This article attempts to do that by posing a set of eight KPIs, which are summarized along with candidate annual performance metrics. Five of the KPIs focus on how well the program is managed, and three focus on the actual outcomes of the audits. The remainder of this article explains the nature of each of the KPIs and their metrics in much more detail. The entire set of KPIs can help to answer the question: “Is the audit program meeting its objectives and is it making a difference?”