In several prior studies, ERISA audit qualify has been found deficient. This latest study confirms earlier results, and In this report, DOL attempts to fathom the reasons for the poor performance.

Assessment of the Quality of Employee Benefit Plan Audits

U.S. Department of Labor

Pension and Welfare Benefits Administration

Office of the Chief Accountant

March 1997

Abstract of Findings

Why Did Some Audits Comply, and Others Not Comply, With Established Professional Standards?

Of the 262 audit engagements selected for on-site workpaper reviews, 212 (81%) were considered to comply with GAAS. For these audits, we concluded that the audit was conducted within the framework of GAAS, meaning that:

• the audit work was adequately planned and supervised;
• the audit field work was properly performed; and
• the results of the audit were properly communicated in compliance with existing professional standards.

For 50 of the audits in our sample, we concluded they were not conducted within the framework created by GAAS. In most cases, the GAAS deficiency related to a failure to perform any audit work whatsoever in a relevant audit area.

We analyzed these 262 audits to determine which attributes and factors contributed to the engagements that were performed in accordance with professional auditing standards, and those that were not. The common attributes and factors affecting audit quality included:

• the size of the IQPA firm performing the audit engagement;
• adequate technical training and knowledge to perform the engagement;
• awareness of the unique nature of auditing an employee benefit plan;
• implementation of an internal structure of process controls which helped ensure a quality work product;
• plan audits viewed as being ancillary;
• performance of a reasonable number of audits as part of the firms' professional practice;
• performance of no audit work;
• failure to understand the limited scope audit exception; and
• the period of time available to adapt to new technical guidance.

Size of the IQPA Firm Contributed to the Quality of Audit Engagement

Our review provided some valuable insights into the correlation between the size of an IQPA and the likelihood of one of their audits containing audit deficiencies. Most notably, in our sample, firms with 20 or less employees:

• were responsible for 64% of the deficient audits;
• accounted for more deficient audits than larger firms when taken as a percentage of the total number of audits. Specifically, firms with 1-5 employees represented 9% of our sample but comprised approximately 27% of the audits with deficiencies. Likewise, firms with 6-20 employees represented 18% of our sample but comprised approximately 38% of the audits with deficiencies; and
• performed deficient audits at a far greater rate than other size firms: 54% of the audits performed by firms with 1-5 employees were deficient. Similarly, 41% of the audits performed by firms with 6-20 employees contained audit deficiencies.

Adequate Technical Training and Knowledge

One of the basic tenets of GAAS requires that IQPAs possess adequate technical competence to complete the engagement. We found that auditors performing adequate work were able to apply their accounting and auditing expertise to employee benefit plans. Such expertise included availing themselves of the technical guidance and training available for employee benefit plans.

As was mentioned in the previous section of this report, the AICPA has taken an aggressive role in providing technical guidance for employee benefit plan auditors. In March 1991, the AICPA issued its first update to the audit guide for employee benefit plans since 1983, with yearly updates being issued thereafter. Since that time, the guide has been updated five times.

Additionally, the AICPA also published Audit Risk Alerts and Financial Accounting and Reporting Checklists for employee benefit plans and other guidance dealing with pension and welfare benefit plans. Starting in December 1990, the AICPA (in conjunction with DOL) has held an annual Employee Benefit Plans Update Conference. The conference has consistently been one of the AICPA's best attended conferences, attracting 450 to 650 participants each year. The AICPA also developed a self-study course for employee benefit plan audits. Additionally, other professional organizations have developed conferences and self-study materials dealing with employee benefit plans.

We believe that where practitioners availed themselves of professional guidance and training, the resultant audit work conformed with professional standards. It was common for these IQPAs to have a copy of the AICPA's Audit and Accounting Guide, Audits of Employee Benefit Plans (with conforming changes as of May 1, 1992). Additionally, we noted that these practitioners had available to them current guidance on professional accounting and auditing standards. Based on discussions with these practitioners, we found that they had either taken advantage of the AICPA guidance and educational programs (or other related conferences) or had completed other relevant self-study courses.

For audits which did not comply with professional standards, we believe that most practitioners did not possess an adequate level of technical knowledge for the audit engagement. The reasons for the lack of compliance included not being aware of the applicable professional guidance (e.g., the industry audit guide), having the guidance but not using it, and/or applying the guidance in an inappropriate manner.

Indicative of the failure to possess adequate technical training and knowledge were four audit engagements where we concluded that the auditor had failed almost every element of the general and fieldwork standards. Additionally, where IQPAs did not possess the level of technical knowledge required to conduct these types of audits, we believe these practitioners failed to exercise due professional care with respect to the conduct of the audit engagement.

In some cases, the IQPAs we visited did not have any copies of the industry audit guide or other guidance published by professional organizations. In these cases, the IQPA had not developed an audit program or erroneously used an audit program not designed for an employee benefit plan (e.g., audit programs for non-profit entities). Unfortunately, in other cases the IQPA thought they did possess adequate technical knowledge but a review of their work demonstrated otherwise. The resulting audits generally exhibited gross failures of multiple professional standards.

In one instance, the IQPA knew he had little or no technical knowledge and performed a substandard audit. Subsequent to our review of his work product, he obtained the necessary guidance materials, educated himself, and re-performed the audit work. Review of this new work indicated that it conformed with professional standards. This demonstrates that, where an IQPA commits to education on standards related to employee benefit plan audits, an acceptable work product emerges.

Overall, while we believe that sufficient guidance and training is available to practitioners conducting audits of employee benefit plans, feedback from practitioners indicated that present guidance could be improved in certain areas where more specific guidance may be lacking (e.g., audits of multiemployer plans) .

Awareness of Uniqueness of Employee Benefit Plan Audits

Employee benefit plan audits present different and unique challenges for auditors. This is because the scope of the audit transcends the money held in trust and the financial activity of the plan. Employee benefit plans exist and operate in a regulated environment and, accordingly, involve the examination of the plan's operation within that legal framework. For example, functional areas such as benefit payments, participant data, plan obligations and prohibited transactions are areas which auditors would typically not encounter during a corporate financial statement audit.

Benefit payments, participant data and plan obligations are all related areas in that the auditing of one of these areas often overlaps that of the other two. For example, eligibility testing affects all three areas: a benefit can be paid only to eligible participants and beneficiaries, a benefit payment must be properly applied to individual participant accounts, and the total plan benefit affects the plan's overall financial obligation. Audit procedures often include the review of the same underlying documentation in each of these three areas, such as personnel records, payroll records, and participant account records. In addition, the plan instrument and related documents set forth requirements and parameters that must be adhered to for each of these areas.

Accordingly, it is easy to see that audits of employee benefit plans, in certain respects, are quite different from audits of other entities. Auditing of the above three distinct areas can be of great importance in that these areas can have a significant impact on the plan's financial statements and tax qualification status and, consequently, can significantly impact the opinion rendered by an IQPA on such financial statements.

We found that acceptable plan audits recognized and took into account the regulatory framework and its relationship with, and possible impact on, the financial activity of the plan. In the 212 audits that were considered to be acceptable, the IQPA performed the necessary level of audit field work in these unique areas to support the opinion expressed on the plan's financial statements.

Most of the deficient audits identified in our sample failed two or more of these unique audit areas. Discussions with these practitioners indicated that they did not recognize or understand the impact these audit areas had on the overall audit engagement. In most of these cases, the auditor focused merely on auditing the financial activity of the plan.

Quality Review and Internal Process Controls

GAAS relates to the conduct of individual audit engagements; quality control standards relate to the conduct of a firm's audit practice as a whole. Thus, GAAS and quality control standards are related, and the quality control policies and procedures that a firm adopts may affect both the conduct of individual audit engagements and the conduct of a firm's audit practice as a whole. These quality control review processes are established to ensure that a minimum standard of audit quality is achieved in all audit engagements.

Our review disclosed that firms that performed acceptable audits had implemented a system of quality control review processes. The nature and extent of these processes varied depending on many factors, such as the firm's size, its personnel, the nature of its practice, and appropriate cost-benefit considerations. Conversely, the firms that performed grossly deficient work had not implemented an adequate system of internal quality controls.

In our sample, large firms tended to have very formal processes for internally reviewing audit work. These processes included:

• standardized checklists and audit programs;
• multiple levels of engagement oversight;
• independent quality reviewers; and
• staff with specialized skills in unique areas, such as employee benefit plan audits.

Smaller firms generally employed less formal processes but, nonetheless, were able to reach the same level of quality.

The AICPA has instituted a practice monitoring program (peer review) in order for firms to maintain their AICPA membership. Among other things, this program is designed to ensure quality in the performance of accounting and auditing engagements. This program is based on the principle that a systematic monitoring and educational process is the most effective way to attain high-quality performance throughout the profession. Those conducting the reviews are held to the same applicable professional standards as the firms they are reviewing. In one instance, however, we reviewed a set of audit workpapers which we found to be grossly deficient. Upon discussion with the practitioner, this same set of audit workpapers was reviewed during the firm's most recent peer review and found to be acceptable.

Plan Audits are Perceived as Necessary Only to Fulfill Governmental Requirements

In discussions with IQPAs, we learned that many of their employee benefit plan clients believe that the only benefit to having an audit is to satisfy a government regulatory requirement. They do not view the audit as providing any value or protection to plan participants and beneficiaries, nor do they view it as a means of fulfilling their fiduciary obligations to the plan.

One practitioner we visited had performed little or no audit work. When questioned about the lack of work, the practitioner stated that his audit fee was $750 and, therefore, he performed up to $750 worth of audit work. Clearly, the IQPA's opinion was not supported by the audit work performed. As a result, plan participants and beneficiaries are being misled by the level of assurance implied in the audit report. Unfortunately, this audit does not serve the purpose for which it was intended; to protect the benefits of plan participants and beneficiaries. We believe that the attitude of this auditor of this auditor is indicative of the fact that this plan's audit was viewed as unimportant.

IQPAs told us they have a difficult time justifying to their clients the level of work necessary to perform an audit in accordance with professional standards. This has resulted in significant cost pressures in the performance of the audit. We were told that practitioners were losing their clients to other IQPAs who would perform the audits for lower cost because they would not adhere to established professional standards. This situation occurs because some firms view this work as low risk with limited exposure to lawsuits.

Reasonable Portion of Practice Includes Audit Work

For the 212 acceptable audits, we generally found that they were performed by IQPAs for which audit engagements comprised a reasonable amount of the firm's overall professional practice. Most of these firms devote the resources necessary to train and educate their staff with respect to professional audit standards. In many cases, they have also received specific training relating to employee benefit plan audits. Accordingly, these firms and the staff they assign to audit employee benefit plans are generally more aware of the intricacies of these engagements.

During our review, we found that certain firms, recognizing the importance of employee benefit plan audits, have begun to dedicate specific staff to perform all of the audit work for these engagements for the firm. These firms have adopted a philosophy that specialization in specific areas will improve the overall quality of audits in those areas.

Alternatively, for the audits we reviewed which contained significant deficiencies, the audit of the employee benefit plan frequently represented the only ERISA audit the firm performed. In some cases it also represented the firm's only audit engagement.

Failure to Perform Necessary Audit Work

Of the substandard audits in our sample, the most common reason for an IQPA to have failed a given audit area was that IQPA did no work or substantially no work in that area. Such lack of audit work, in this group of audits, was not limited to those audit areas particular to employee benefit plans, such as prohibited transactions and participant data. Many of these IQPAs also did no audit work in areas such as management representations, internal controls, commitments and contingencies, and subsequent events. These areas must be included in the audit of any entity, not just employee benefit plans. In short, these IQPAs concentrated only on tying down account balances instead of auditing non-account balance areas too. Therefore, a sufficient audit was not performed.

Audit areas related to participant data, plan obligations, benefit payments, and party-in- interest/prohibited transactions are unique to employee benefit plans. We found that these four areas resulted in the most number of audit failures and, therefore, an auditor with limited employee benefit plan experience might not be aware of the significance involved with these areas.

Failure to Understand Limited-Scope Audit Exception

As previously discussed, under current law a plan administrator may elect to exclude from an audit the plan assets that are held by and certified to by certain regulated financial institutions. When performed in accordance with professional standards, these audits typically result in a "disclaimer of opinion."

A disclaimer of opinion does not provide any assurance by the IQPA on the financial information presented on the plan's financial statements taken as a whole.

Our review identified instances where auditors performed almost no audit work, believing that the limited-scope audit exemption only required that the auditor obtain a certification from the financial institution. Where an IQPA performs a limited-scope audit and fails to perform sufficient audit work in other areas not covered by the certification, the user of the plan's financial statements has virtually no assurance with respect to the financial operation of the plan.

Education, Technical Training and Guidance

• PWBA should continue outreach efforts with state CPA societies, the AICPA and other professional organizations to provide technical training to IQPAs and other plan professionals;
• PWBA should consider developing materials designed to educate plan administrators about hiring a competent plan auditor. Such materials could be in the form of a question and answer pamphlet. Educating plan administrators is important because they need to understand the importance of the audit requirement and that they have the ultimate responsibility for the accuracy and completeness of the plan's annual report filing;
• The AICPA should assess the effectiveness of peer review programs in highlighting weaknesses in the quality of employee benefit plan audits. The fact that some practitioners who received an "unqualified" peer review but were deemed to have performed deficient audit work is indicative that the peer review process may not be highlighting all audit deficiencies. The AICPA should explore changes to its peer review program so that practitioners receiving "unqualified" opinions are in fact meeting the minimum professional standards regarding employee benefit plan audits. PWBA should encourage and assist the AICPA with this effort;
• PWBA should work with the AICPA to develop educational materials for peer reviewers to use when they are conducting peer and quality reviews of employee benefit plan audits. Ultimately, the peer review is only as good as the individual(s) performing the review. Equipped with relevant knowledge regarding the uniqueness of employee benefit plan audits, peer reviewers will better be able to identify deficient audit work; and
• The AICPA should encourage the development of a series of smaller, regional employee benefit plan accounting and auditing training conferences designed to attract the small to medium size IQPA firms. While the AICPA has developed a very successful annual employee benefit plans update conference, it may not be convenient or cost effective for many smaller practitioners to attend. A regionally based conference may attract more of the necessary target audience - small to medium size IQPA firms performing a limited number of employee benefit plan audits.

Among DOL's suggestions for improving quality:

• OCA should revise the criteria for targeting cases to identify a greater number of plan filings with audits performed by small to medium size IQPA firms. Currently, targeting is based on factors which do not relate to the size of the IQPA firm;
• PWBA should consider including the name and employer identification number (EIN) of the plan's IQPA on the Form 5500, regardless of whether or not the IQPA is compensated by the plan. This will enhance PWBA's ability to identify and target IQPAs who are performing deficient professional work by targeting specific IQPAs in instances where PWBA believes deficient work has been performed;
• To improve compliance with ERISA's reporting and disclosure requirements, OCA should continue to identify and reject those plan filings which do not meet ERISA's reporting and disclosure requirements with respect to supplemental schedules.