Internal Audit Performance: Thoughts on What to Measure and How to Measure It.

Internal Audit Performance: Thoughts on What to Measure and How to Measure It.

Recently I have been doing some research on the state of internal auditing. I’m planning some presentations and webcasts designed to take a fresh approach to defining what internal auditors do, or should do, and how to measure the value added by internal audit departments. (If some of this spills over to external audit, so much the better).

I should point out that I began my professional career as an internal auditor, fresh from earning my professional designation iby spending a couple of years with a firm that is now part of Ernst & Young. Over an internal audit career period of 15 years I rose to become a Chief Audit Executive with a staff of about 90.

The questions I have been asking myself recently are the same questions I asked myself at the time I was a CAE. The difference is that the environment has completely changed, partly due to regulatory changes, partly due to changes in professional standards, and largely due to technology. Answers now exist that were not apparent then.

I have recently read two new independent studies that discuss the measurement of internal audit performance. The first is an August 2009 publication by the Aberdeen Group titled “Beyond Demonstrating Compliance: The Reinvention of Internal Audit”. The second study was published by the IIA’s Global Audit Information Network (GAIN) in September 2009. It is titled “Knowledge Report: Measuring Internal Audit Performance”.
Both discuss a range of measures for evaluating the performance of internal auditors and internal audit departments. The measures range from the time and cost time to conduct an “audit” to the completion of the audit work plan. They are the same measures I and every CAE struggled with. I now believe they are partially meaningful at best and dangerous at worst. They don’t measure the value added, they measure the activity performed. You can score well and still do bad work.

Here is the crux of my problem. I simply do not believe the goal of internal auditing is to perform audits.

Looking at the most recent (and quite excellent) INTERNATIONAL STANDARDS FOR THE PROFESSIONAL PRACTICE OF INTERNAL AUDITING (STANDARDS), I could not even find a definition of what an audit was. In fact the STANDARDS never ever use the word “audit” as a noun. Virtually every instance of the word “audit” was as an adjective, as in audit plan, audit report etc. The STANDARDS use the phrase “audit activity”, not “audit”. Auditors are supposed to perform “audit activities”, they are not limited to doing “audits”, a small but very significant distinction.

Measuring the speed and efficiency and proficiency of performing audits is useless if the “audits” do not produce value. Think of how other professions handle the dilemma of measuring the activity vs. the value the activity produces.

A physician spends a lot of time writing prescriptions. Should we measure the effectiveness of physicians by the number of pills they prescribe or their speed in doing? When we measure the number of audits and audit findings, that is what we are doing for auditors. It misses the big picture. I believe we measure the value of medical professionals not by the activities they perform, e.g. prescribing medication, but by the value they add in terms of improved health, early problem diagnosis, and life expectancy, pain avoided or lessened etc. We don’t measure the “activities” of dentists and lawyers in terms of cavities filled or lawsuits filed, we measure dental health achieved, legal problems avoided and justice served.

So the big question remains, what is the value added by internal auditors? What is the purpose of “audit activities”? My answer is that the value of audit activities is only indirectly linked to the number, results and duration of audits performed, and may even be inversely linked.

Surely the value of audit activities must lie in the information created or produced and made available on the status of risk and control. What kind of information is that? It is simply the number and types of risks and controls facing the organization, the processes or other contexts where these risks and controls exist, the impact on the performance of the organization and the threats posed by risks, and related information, including key risk indicators, key performance indicators, deficiencies, issues, loss events and incidents to name just a few.

Internal audit reports contain opinions on control effectiveness. The information on the status of individual risks and controls on which the opinions are based is locked in working papers. Think of how useless an Auditors Report on managements financial statements from a public accounting firm would be if the financial statements were not visible and available.

What is true now that was not true when I was a CAE is that technology exists to capture and report on this information. In my days as CAE the information was locked in paper files. Today, technology exists to portray all of an organizations risk and control status information in great detail. It is no longer necessary to merely report on “control effectiveness” without clearly and completely depicting what the “effective” controls are, what risks they are effective in mitigating and what risks are managed in other ways.

My belief is that the role of internal audit is to drive up the quantity and quantity of reliable information on the status of risk and control.

How can we measure the quantity of information? To begin with lets just count the risks and controls and related information gathered in “audit activities”

How do we measure the quality? Let’s decide that good information on risk and control, in a business process or elsewhere, will mean that there will be no unintended surprises. If unforeseen risks impact the business, then previous risk and control knowledge was less than reliable. Can we measure these things? Absolutely! Track performance problems in business processes where controls were reported as “effective” and risks were accepted by management. Track new and repeat Issues. Track incidents and loss events.

What’s stopping internal auditors from changing? My suspicion is that the biggest roadblock is the reluctance of internal audit practitioners to embrace the necessary technology. The GAIN report referred to earlier mentions technology primarily in the context of a tool for measuring or reporting performance. The intelligent employment of enterprise technology for gathering and reporting on risk and control status information is not considered as a performance criteria..

The Aberdeen report suggests that “auditors looked almost sheepish when describing the depth of their dependence on spreadsheets, as if they know better but stay on the same path anyway”. Spreadsheets are useful for automating audits; they are not useful for adding value through “audit activities” as I see them defined.