Case studies in performance auditing
Muhammad Akram Khan
A CASE STUDY in performance auditing * describes various decisions made by auditors as they plan, execute, report, and follow up stages of an audit. Presentation styles of the case study vary according to their purpose and audience. The most common application of a case study is in the classroom, where students are asked to analyze a “real-life” problem and suggest a solution. Case studies may also be used in internal audit training, or in seminars and forums where internal auditors share professional experiences with others.
Developing the Case Study
Case studies may describe any of the decision-making processes that occur during the planning, executing, reporting, and follow-up stages of an audit. The best person to write the case study is probably someone who is experienced in performance auditing but who has not been associated with the particular project that will be the subject of the case study. The case study writer should have access to all working papers and audit files and should interview the auditors and audit managers to collect information on various questions not documented in the audit files.
A clear conception of the project objectives and the expected audience will be an important aspect in selecting an appropriate performance audit report, which is the first step in developing a case study. For example, if the purpose is to highlight a performance audit of a foreign funded project, then compiling a short list of audit reports of projects with a significant element of foreign assistance will be the first step. Similarly, if the objective is to demonstrate audit approach in the use of consultants with the audit team, then obviously those audit reports where the auditors hired consultancy services would form part of the short list.
In making a final choice from the short list, preference should probably be given to assignments with interesting features, such as wide geographical coverage, application of some novel analytical techniques, or impact evaluation. The case study writer should be prepared to explain reasons for selecting a particular audit project.
Although guidelines can help auditors to adopt a uniform pattern for presenting a case study, each case study may be amended in the light of actual performance audit exercise. In practice, some of the following sections may not appear at all, while some others may have to be added.
Preliminary Survey Report
Planning for a performance audit usually involves preparation of a preliminary survey report (PSR). The key components of a PSR include a brief introduction of the project, audit objectives, audit criteria, key issues, audit approach, evaluation of internal controls, and audit sample. The case study should briefly discuss these components, highlighting the reasons for various decisions taken at each stage.
The case study should list the audit objectives determined by the auditors in planning the audit, and any delimitation or expansion in the scope of audit should be explained. For example, if, in a certain case, the auditors decided not to cover effectiveness aspects of a project, and/or the auditors decided to extend their audit to certain fraud investigations, the case writer should state reasons for these changes.
The objectives of the project; estimated costs and benefits; location; beneficiaries; and sponsoring, executing and operating agencies should be explained in an introduction that is brief and to the point. Case study writers may need to remind themselves that they are not writing project audit reports where complete information is usually of essence.
Key Issues and Audit Criteria
In this section the case study writer should list the key issues raised in the project audit report. For each issue the case study should specify the sub-issues and performance indicators and then mention the criteria adopted by the auditors. Statements describing various options and constraints and relate the procedure for finalizing the criteria should be included.
For example, in a road construction project, the road design could be a significant issue. The study should explain how the auditors decided the parameters of the design. (They might have obtained the help of consultants, or they may have in-house capability to help decide the criteria for the most economical and effective design, for example.) Whatever procedure was adopted should be stated in detail for each issue, so that the methodologies relevant to audit criteria are clearly illustrated.
Audit approach in performance auditing relates to auditor’s decisions regarding the strategy used to carry out the audit. In making their decisions, auditors consider the size of the project, data collection problems, state of auditee controls, condition of the auditee information upkeep, competence of the auditors, and the time frame to complete the project. The case study should state which of these or other aspects were relevant in a particular audit and how the auditors finalized their approach.
Key Internal Controls
In large audit projects the auditors identify key internal controls and then evaluate their soundness before they decide to rely on them. The evaluation of internal controls gives the auditors required assurance about the economy, efficiency, and effectiveness of the project. It also helps them decide on the sample size for compliance and substantive testing. The case study should explain:
* How the auditors identified the key internal controls; whether or not they prepared flowcharts or used a descriptive method.
* What tests were conducted to evaluate the controls.
* What conclusions were reached regarding the strength or weakness of the controls.
Selecting a sample of the data is also a part of most large audit projects. For example, a primary education project involving several thousand schools spread all over the country would make it obligatory to rely on sample of the data. The case study should provide answers to the following questions:
* Why did the auditors decide to resort to judgmental or statistical sampling, whichever was the case?
* How was the size of the sample determined?
* What was the level of materiality?
* What was the planned level of audit risk?
* Was the sample selected on the basis of random numbers, by monetary unit sampling basis, or by some other method?
Although the term “substantive testing” is generally used in the context of financial auditing, it has an application in performance auditing as well. The performance auditors need to satisfy themselves that the data supplied by the auditee is reliable. Moreover, they like to ascertain whether or not the auditee operations contain any material incidence of waste or inefficiency. Therefore, they have to plan for certain substantive tests and review procedures to get this assurance. The case writer should highlight any substantive tests and review procedures planned by the performance auditors.
The qualifications of the auditors should be addressed, with comments on the collective ability and competence of the auditors to carry out the performance audit of the selected project. In case the auditors collectively lacked competence for some aspect of the audit, the case study should state how this deficiency was overcome, e.g., by hiring consultants.
The audit program, a written plan for the execution of audit, outlines the logic, strategy, and procedures to be followed in the data and its analysis. The case study should highlight salient features of the audit program, illustrating reasons for various decisions taken by the auditors. For example, if in a certain audit assignment the auditors decided to use computers for selecting the sample, the case study should explain the rationale and the procedure employed. Similarly, if the auditors decided to carry out physical verification of certain assets, the reasons and the procedures adopted should be explained.
Another important area in an audit program is the time provided for audit. The case study should explain how time was budgeted, monitored, and controlled. In case of time overruns in audit execution, the reasons for lost time should be explained, along with the strategy adopted by audit management to make up time losses. The audit program also incorporates the plan for human resources, and the case study shoul describe the workload of each person deployed on the performance.
Collection and Analysis of Data
The data collection methods adopted by the auditors should be explained, including rationales for any unusual techniques or procedures. For example, in a given situation the auditors may have hired consultants to carry out surveys of a particular area. The case study should review the conditions which led to such a decision and state how the consultants’ output was reviewed and used in the final audit report. Questions regarding the reliability of the data collected through consultants would likely be of special interest.
Auditors may apply a number of techniques to analyze data: simple comparisons and reconciliations; statistical methods, such as mean, mode, median, standard deviation, correlation, extrapolation, regression analysis, and hypothesis testing; cost-benefit analysis; efficiency pricing; output budgeting; sensitivity testing; or cost-effectiveness analysis. The case study should mention the techniques used by the auditors along with reasons for reliance on a particular technique.
Most case studies will usually offer a brief summary of the audit report, so that readers can be informed about the results of the auditors’ efforts during audit execution. The findings need not be in detail since the purpose here is not to persuade the reader about the authenticity of the findings (as it is in the actual audit report), but to highlight conclusions of the audit assignment.
The various stages through which the audit report passed within and outside the audit department should be described in the case study. Information should be provided about the time required to finalize the audit report and any significant delays that occurred. The case study should also indicate whether or not the report was checked within the department for fact-validation, internal consistency, quality of analysis, and other criteria. Disclosure should be made as to whether or not the report was issued to the auditee and if its final version included the auditee response and further comments of the auditors.
Presentation of the Report
The last phase of performance audit is the presentation of its report. In governments where parliamentary democratic traditions are followed, the presentation would be to the Public Accounts Committee (PAC), a select standing committee of the legislature thta examines the reports of the Auditor General. In this example, the case study should mention the reaction of the PAC and the action taken by the auditee on the recommendations of the PAC. It should also mention any follow-up review by the auditors after a certain period.
The internal and external problems faced by the auditors should be described. Internal problems might include such factors as a shortage of trained auditors, nonavailability of financial resources for audit, lack of coordination among different branches, and nonavailability of technical advice.
External constraints might include noncooperation of the auditee in presenting data or providing an appropriate schedule for audit, inappropriate data, absence of data, scatteredness of the project, remoteness of the audit project site, and highly technical and/or sophisticated operations, as examples. The case study should focus on the significance of these issues and discuss the approach adopted by the auditors to make their results credible.
This section of the study is an objective assessment of the audit approach, procedures, and outputs. Comments on various decisions taken by the auditors should be offered, including criticisms of the audit, where necessary. The case study may point our areas where:
* The audit approach was deficient.
* The auditors relied on invalid, insufficient or irrelevant evidence.
* The auditors adopted uneconomical or inefficient procedures.
* The auditors arrived at incorrect conclusions from otherwise valid evidence.
* The audit procedures would not reveal shortcomings in economy, efficiency, and effectiveness.
This section summarizes the pitfalls in the audit decisions and show need for appropriate action.
Case studies provide a valid technique for sharing experiences and disseminating results of performance audits and may function as effective tools for teaching, training, and sharing professional knowledge and insights. The guidelines presented may be helpful in structuring the case study, but a creative approach is most effective in writing the case study. The techniques, processes, and lessons learned during an audit assignment can be clearly presented in case studies, and benefits can be applicable to internal auditors everywhere.
Muhammad Akran Khan is Director General, Research and Development, in the Department of the Auditor General of Pakistan.
* Performance auditing is defined here as an assessment of an organization’s activities to see if the resources are being managed with economy, efficiency, and effectiveness, and to see if accountability requirements are being met reasonably. The author, who is in he office of the Department of the Auditor General of Pakistan, notes that this definition is used in Pakistan and other Third World Countries, as well as elsewhere. In some areas, this activity is referred to as “operations auditing.”
COPYRIGHT 1991 Institute of Internal Auditors, Inc.
COPYRIGHT 2008 Gale, Cengage Learning