Testing application controls – use of computer-assisted auditing tools and techniques in reviewing application controls

Testing application controls – use of computer-assisted auditing tools and techniques in reviewing application controls – Computers & Auditing

Dave Coderre

It’s almost criminal to use audit software only to produce standard reports. In the hands of experienced, creative auditors, there is no limit to the use of computer-assisted auditing tools and techniques (CAATTs).

Reviews of application controls, for example, can definitely be enhanced by CAATTs. By using audit software to access an application’s data directly, auditors can use simple commands to verify edit checks, to test calculations performed by the application, and to examine the data for reasonableness. In addition, searching for gaps and duplicates can help to determine the completeness and timeliness of the data.

* Application Controls

Edit checks for key fields can be verified by stratifying or classifying the transactions on the values for the field. To illustrate, if an auditor totalled the number of transactions for each value in the “Sex” field, the results might be: M = 2,341; F = 2,295; D = 2; and N = 1. Such totals indicate that the edit check, which limits possible values to “F” and “M”, is not working properly and is allowing the typos “D” and “N” to be keyed instead. In another example where all insurance policies must start with a numeric character (0-9), a test of the first character of the policy number field would quickly highlight all transactions that did not meet this criterion.

Using audit software, it might be easy to recalculate and verify calculations made by the system. For example, if the system uses the quantity and unit price fields to calculate the total cost, the auditor could use audit software to perform the same calculation and identify any transactions where his or her calculated values differ from those of the application.

Reasonableness checks also can be used to examine possible value data ranges for key fields. For example, by calculating the current age, based on the date of birth field, the auditor could easily identify ages, such as negative values and values over 100, that fall outside of the expected range. In addition, combinations of fields can be checked for reasonableness. A single record should not contain the values Sex = M and Pregnant = Yes.

In addition, audit software can be used to test the validity of the contents of a field. For example, date fields can be checked for invalid dates and numeric fields checked for any non-numeric values.

* Testing Controls

In the following situation, an audit of the accounts payable system was designed to assess various controls over the integrity and appropriateness of the data. The manual review of the application control over the payment of duplicate invoices determined that duplicates were identified based upon two fields: the vendor code and the invoice number. The application would reject and flag any transaction where the combination of vendor number and invoice number was not unique.

The invoice numbers were dependent on the vendor’s invoicing system and, thus, beyond the company’s control. However, the assignment of vendor codes was performed by the invoice processing sections of the company. Therefore, the auditor pursued the review of the control over duplicate payments one step further by examining the controls over the assignment of the vendor code.

The auditor was concerned when she found that any invoice processing clerk at any of the invoice processing offices could add, modify, and delete entries in the vendor table. The vendor table contained all qualified vendors and their codes, and the auditor felt that unrestricted access to this table was a control weakness.

Using audit software, the auditor summarized the vendor table by vendor name and found numerous vendors with more than one vendor number. Vendors located at the same address but with slightly different names, such as ABC Limited, ABC Ltd, and ABC Ltd., had a different vendor code assigned for each spelling of the vendor name:

Vendor Name Vendor Code Vendor Address

ABC Limited N3450D12 101 Grey Rock

ABC Ltd. N5478X23 101 Grey Rock

ABC Ltd. N5471C10 101 Grey Rock

The auditor informed management that the poor control over the vendor table, which allowed not only different vendor numbers to be assigned to the same vendor, but also permitted any invoice processing clerk to make changes to the table, compromised the application control over the payment of duplicate invoices.

Management wasn’t convinced that the exposure was significant. They maintained that other, compensating controls, including a manual review of payments by the budget managers, would catch any duplicate invoices.

The auditor realized that, without further audit evidence, management would not take steps to address the control weakness in the assignment of the vendor code. She then performed two tests for duplicate payments. In addition to using the combination of vendor code and invoice number to identify duplicates, a second test looked for transactions involving the same invoice number and the same payment amount.

The first test, as she expected, did not identify any duplicate invoices; the application controls were preventing transactions with the same vendor code and invoice number from being processed. However, the second test identified several thousand potential duplicate transactions. Upon reviewing the detailed transactions in the second file, the auditor realized that the criteria were not sufficiently restrictive and that many transactions had been incorrectly identified as potential duplicates. Too many firms used similar invoice number sequences – for example, 96-1 for the first invoice in 1996 – and too many invoices were for even dollar amounts.

Since the object was to determine if any duplicate invoices were being paid, not to identify all duplicate payments, the auditor refined the criteria by requiring that the invoice number be at least four characters in length and that the invoice amount be greater than $1,000. The second extraction produced 214 possible duplicate transactions totalling just over $1.5 million. A detailed review of these eliminated 36 transactions for a variety of reasons, including different vendor addresses, different items purchased, and different delivery locations.

The final file contained 178 transactions totalling more than $1 million. The auditor selected the ten largest payments and requested copies of the actual invoices.

Nine of the ten payments turned out to be duplicates; however, in two cases the vendors had returned the duplicate payments and a credit was entered into the system. The total overpayment for the remaining seven duplicate invoices amounted to almost half a million dollars. When management was presented with the results of the auditor’s test for duplicates, they readily agreed to implement tighter controls over the assignment of vendor codes. They proceeded with a review of the other 168 potential duplicate transactions identified by the auditor and ordered recovery action for all identified duplicate payments.

* Effective Auditing

Both manual and computer-assisted auditing techniques can be used to evaluate the effectiveness of an application’s controls. In this case, the use of audit software allowed the auditor to use the actual data to verify the application’s controls over the processing of duplicate transactions. It enabled the auditor to search through millions of records for transactions that met the audit-specified criteria – in hours rather than months. CAATTs also permitted the auditor to adjust the criteria for duplicate transactions after reviewing the initial results – providing a more accurate sample for further review.

While the test for duplicates did not attempt to identify all duplicates, and not all of the selected transactions were duplicates, the test did validate the auditor’s initial concerns and highlighted a weakness in the application’s controls. The auditor’s effectiveness and success in meeting her audit objectives were unquestionably complemented by the use of CAATTs.

Dave Coderre, MBA, is Manager, Information Support and Analysis, Director General Audit in the Canadian Department of National Defence in Ottawa.

Internet address dcoderre@dgs.dnd.ca

COPYRIGHT 1996 Institute of Internal Auditors, Inc.

COPYRIGHT 2004 Gale Group