CIS558 Assignment 1 ERM Roadmap

Anonymous
timer Asked: Apr 24th, 2017

Question Description

The following material may be useful for the completion of this assignment. You may refer to the documents titled “Embracing Enterprise Risk Management: Practical Approaches for Getting Started” and “Developing Key Risk Indicators to Strengthen Enterprise Risk Management”, located at http://www.coso.org/-ERM.htm.

Imagine you are an Information Technology Manager employed by a business that needs you to develop a plan for an effective Enterprise Risk Management (ERM) program. In the past, ERM has not been a priority for the organization. Failed corporate security audits, data breaches, and recent news stories have convinced the Board of Directors that they must address these weaknesses. As a result, the CEO has tasked you to create a brief overview of ERM and provide recommendations for establishing an effective ERM program that will be used as a basis to address this area moving forward.

Write a three to four (3-4) page paper in which you:

  1. Summarize the COSO Risk Management Framework and COSO’s ERM process.
  2. Recommend to management the approach that they need to take to implement an effective ERM program. Include the issues and organizational impact they might encounter if they do not implement an effective ERM program.
  3. Analyze the methods for establishing key risk indicators (KRIs).
  4. Suggest the approach that the organization needs to take in order to link the KRIs with the organization’s strategic initiatives.
  5. Use at least three (3) quality resources in this assignment (in addition to and that support the documents from the COSO Website referenced in this assignment). Note: Wikipedia and similar Websites do not qualify as quality resources.

Your assignment must follow these formatting requirements:

  • Be typed, double spaced, using Times New Roman font (size 12), with one-inch margins on all sides; citations and references must follow APA or school-specific format. Check with your professor for any additional instructions.
  • Include a cover page containing the title of the assignment, the student’s name, the professor’s name, the course title, and the date. The cover page and the reference page are not included in the required assignment page length.

Use this template APA_Template_With_Advice_(6th_Ed) .doc

Moeller IT Audit, CONTROL, SECURITY and IT Audit, CONTROL, and SECURITY Robert R. Moeller E1FFIRS 08/30/2010 19:24:8 Page 2 E1FFIRS 08/30/2010 19:24:8 Page 1 IT Audit, Control, and Security E1FFIRS 08/30/2010 19:24:8 Page 2 E1FFIRS 08/30/2010 19:24:8 Page 3 IT Audit, Control, and Security ROBERT MOELLER John Wiley & Sons, Inc. E1FFIRS 08/30/2010 19:24:8 Page 4 Copyright # 2010 by John Wiley & Sons, Inc. All rights reserved. Published by John Wiley & Sons, Inc., Hoboken, New Jersey. Published simultaneously in Canada. No part of this publication may be reproduced, stored in a retrieval system, or transmitted in any form or by any means, electronic, mechanical, photocopying, recording, scanning, or otherwise, except as permitted under Section 107 or 108 of the 1976 United States Copyright Act, without either the prior written permission of the Publisher, or authorization through payment of the appropriate per-copy fee to the Copyright Clearance Center, Inc., 222 Rosewood Drive, Danvers, MA 01923, (978) 750-8400, fax (978) 646-8600, or on the web at www.copyright.com. Requests to the Publisher for permission should be addressed to the Permissions Department, John Wiley & Sons, Inc., 111 River Street, Hoboken, NJ 07030, (201) 748-6011, fax (201) 748-6008, or online at www.wiley.com/go/permissions. Limit of Liability/Disclaimer of Warranty: While the publisher and author have used their best efforts in preparing this book, they make no representations or warranties with respect to the accuracy or completeness of the contents of this book and specifically disclaim any implied warranties of merchantability or fitness for a particular purpose. No warranty may be created or extended by sales representatives or written sales materials. The advice and strategies contained herein may not be suitable for your situation. You should consult with a professional where appropriate. Neither the publisher nor author shall be liable for any loss of profit or any other commercial damages, including but not limited to special, incidental, consequential, or other damages. For general information on our other products and services or for technical support, please contact our Customer Care Department within the United States at (800) 762-2974, outside the United States at (317) 572-3993 or fax (317) 572-4002. Wiley also publishes its books in a variety of electronic formats. Some content that appears in print may not be available in electronic books. For more information about Wiley products, visit our web site at www.wiley.com. Library of Congress Cataloging-in-Publication Data: Moeller, Robert R. IT audit, control, and security / Robert Moeller. p. cm. Includes bibliographical references and index. ISBN: 978-0-471-40676-1 (cloth); 978-0-470-87741-8 (ebk); 978-0-470-87767-8 (ebk); 978-0-470-87768-5 (ebk) 1. Information technology—Auditing. 2. Electronic data processing departments— Auditing. 3. Computer security. 4. Computer networks—Security measures. I. Title. T58.5.M645 2010 658.4’78–dc22 2010013505 Printed in the United States of America 10 9 8 7 6 5 4 3 2 1 E1FFIRS 08/30/2010 19:24:8 Page 5 Dedicated to my best friend and wife, Lois Moeller. Lois has been my companion and partner for over 40 years, whether we are on our Lake Michigan sailboat, skiing in Utah or elsewhere, traveling to interesting places in the world, gardening in the backyard, or cooking its produce. E1FFIRS 08/30/2010 19:24:8 Page 6 E1FTOC 09/17/2010 15:36:11 Page 7 Contents Introduction xiii PART ONE: AUDITING INTERNAL CONTROLS IN AN IT ENVIRONMENT Chapter 1: SOx and the COSO Internal Controls Framework Roles and Responsibilities of IT Auditors Importance of Effective Internal Controls and COSO COSO Internal Control Systems Monitoring Guidance Sarbanes-Oxley Act Wrapping It Up: COSO Internal Controls and SOx Notes Chapter 2: Using CobiT to Perform IT Audits Introduction to CobiT CobiT Framework Using CobiT to Assess Internal Controls Using CobiT in a SOx Environment CobiT Assurance Framework Guidance CobiT in Perspective Notes Chapter 3: IIA and ISACA Standards for the Professional Practice of Internal Auditing Internal Auditing’s International Professional Practice Standards Content of the IPPF and the IIA International Standards Strongly Recommended IIA Standards Guidance ISACA IT Auditing Standards Overview Codes of Ethics: The IIA and ISACA Notes Chapter 4: Understanding Risk Management Through COSO ERM Risk Management Fundamentals Quantitative Risk Analysis Techniques IIA and ISACA Risk Management Internal Audit Guidance COSO ERM: Enterprise Risk Management 1 3 4 6 21 22 31 31 32 33 35 39 51 54 55 55 57 58 61 75 76 79 81 82 83 92 94 97 vii E1FTOC 09/17/2010 viii 15:36:11 & Page 8 Contents IT Audit Risk and COSO ERM Notes Chapter 5: Performing Effective IT Audits IT Audit and the Enterprise Internal Audit Function Organizing and Planning IT Audits Developing and Preparing Audit Programs Gathering Audit Evidence and Testing Results Workpapers and Reporting IT Audit Results Preparing Effective IT Audits Notes PART TWO: AUDITING IT GENERAL CONTROLS Chapter 6: General Controls in Today’s IT Environments Importance of IT General Controls IT Governance General Controls IT Management General Controls IT Technical Environment General Controls Note Chapter 7: Infrastructure Controls and ITIL Service Management Best Practices ITIL Service Management Best Practices ITIL’s Service Strategies Component ITIL Service Design ITIL Service Transition Management Processes ITIL Service Operation Processes Service Delivery Best Practices Auditing IT Infrastructure Management Note Chapter 8: Systems Software and IT Operations General Controls IT Operating System Fundamentals Features of a Computer Operating System Other Systems Software Tools Chapter 9: Evolving Control Issues: Wireless Networks, Cloud Computing, and Virtualization Understanding and Auditing IT Wireless Networks Understanding Cloud Computing Storage Management Virtualization PART THREE: AUDITING AND TESTING IT APPLICATION CONTROLS Chapter 10: Selecting, Testing, and Auditing IT Applications IT Application Control Elements Selecting Applications for IT Audit Reviews 113 115 117 118 122 127 132 142 148 149 151 153 154 157 158 174 174 175 176 179 181 189 194 198 199 200 201 202 206 209 214 215 220 225 227 229 230 239 E1FTOC 09/17/2010 15:36:11 Page 9 Contents Performing an Applications Controls Review: Preliminary Steps Completing the IT Applications Controls Audit Application Review Case Study: Client-Server Budgeting System Auditing Applications under Development Importance of Reviewing IT Application Controls Notes & ix 242 249 255 258 266 266 Chapter 11: Software Engineering and CMMi 267 Software Engineering Concepts CMMi: Capability Maturity Model for Integration CMMi Benefits IT Audit, Internal Control, and CMMi Note 267 269 280 281 282 Chapter 12: Auditing Service-Oriented Architectures and Record Management Processes 283 Service-Oriented Computing and Service-Driven Applications IT Auditing in SOA Environments Electronic Records Management Internal Control Issues and Risks IT Audits of Electronic Records Management Processes Notes 284 294 300 301 303 Chapter 13: Computer-Assisted Audit Tools and Techniques 304 Understanding Computer-Assisted Audit Tools and Techniques Determining the Need for CAATTs CAATT Software Tools Steps to Building Effective CAATTs Importance of CAATTs for Audit Evidence Gathering 305 308 311 326 327 Chapter 14: Continuous Assurance Auditing, OLAP, and XBRL 329 Implementing Continuous Assurance Auditing Benefits of Continuous Assurance Auditing Tools Data Warehouses, Data Mining, and OLAP XBRL: The Internet-Based Extensible Markup Language Newer Technologies, the Continuous Close, and IT Audit Notes PART FOUR: IMPORTANCE OF IT GOVERNANCE Chapter 15: IT Controls and the Audit Committee Role of the Audit Committee for IT Auditors Audit Committee Approval of Internal Audit Plans and Budgets Audit Committee Briefings on IT Audit Issues Audit Committee Review and Action on Significant IT Audit Findings IT Audit and the Audit Committee Chapter 16: Val IT, Portfolio Management, and Project Management Val IT: Enhancing the Value of IT Investments IT Systems Portfolio and Program Management 330 338 339 346 351 351 353 355 356 357 359 360 362 363 364 371 E1FTOC 09/17/2010 x 15:36:11 & Page 10 Contents Project Management for IT Auditors Notes Chapter 17: Compliance with IT-Related Laws and Regulations Computer Fraud and Abuse Act Computer Security Act of 1987 Gramm-Leach-Bliley Act HIPAA: Healthcare and Much More Other Personal Privacy and Security Legislative Requirements IT-Related Laws, Regulations, and Audit Standards Chapter 18: Understanding and Reviewing Compliance with ISO Standards Background and Importance of ISO Standards in a World of Global Commerce ISO Standards Overview ISO 19011 Quality Management Systems Auditing ISO Standards and IT Auditors Notes Chapter 19: Controls to Establish an Effective IT Security Environment Generally Accepted Security Standards Effective IT Perimeter Security Establishing an Effective, Enterprise-Wide Security Strategy Best Practices for IT Audit and Security Notes Chapter 20: Cybersecurity and Privacy Controls IT Network Security Fundamentals IT Systems Privacy Concerns PCI-DSS Fundamentals Auditing IT Security and Privacy Security and Privacy in the Internal Audit Department Notes Chapter 21: IT Fraud Detection and Prevention Understanding and Recognizing Fraud in an IT Environment Red Flags: Fraud Detection Signs for IT and Other Internal Auditors Public Accounting’s Role in Fraud Detection IIA Standards and ISACA Materials for Detecting and Investigating Fraud IT Audit Fraud Risk Assessments IT Audit Fraud Investigations 374 383 384 386 387 390 395 403 404 407 408 410 419 421 421 422 423 429 430 432 433 434 435 443 446 447 448 453 454 455 456 461 462 464 467 E1FTOC 09/17/2010 15:36:11 Page 11 Contents IT Fraud Prevention Processes Fraud Detection and the IT Auditor Notes Chapter 22: Identity and Access Management Importance of Identity and Access Management Identity Management Processes Separation of Duties Identify Management Controls Access Management Provisioning Authentication and Authorization Auditing Identity and Access Management Processes Note Chapter 23: Establishing Effective IT Disaster Recovery Processes IT Disaster and Business Continuity Planning Today Building and Auditing an IT Disaster Recovery Plan Building the IT Disaster Recovery Plan Disaster Recovery Planning and Service Level Agreements Newer Disaster Recovery Plan Technologies: Data Mirroring Techniques Auditing Business Continuity Plans Disaster Recovery and Business Continuity Planning Going Forward Notes Chapter 24: Electronic Archiving and Data Retention Elements of a Successful Electronic Records Management Process Electronic Documentation Standards Implementing Electronic IT Data Archiving Auditing Electronic Document Retention and Archival Processes Chapter 25: Business Continuity Management, BS 25999, and ISO 27001 IT Business Continuity Management Planning Needs Today BS 25999 Good Practice Guidelines Auditing BCM Processes Linking the BCM with Other Standards and Processes Notes Chapter 26: Auditing Telecommunications and IT Communications Networks Network Security Concepts Effective IT Network Security Controls Auditing a VPN Installation Note & xi 468 471 471 472 473 474 477 478 479 481 485 486 487 489 497 503 505 506 508 508 509 510 516 517 519 521 522 524 540 543 543 544 545 549 555 557 E1FTOC 09/17/2010 xii 15:36:11 & Page 12 Contents Chapter 27: Change and Patch Management Controls 558 IT Change Management Processes Auditing IT Change and Patch Management Controls Notes 559 573 576 Chapter 28: Six Sigma and Lean Technologies Six Sigma Background and Concepts Implementing Six Sigma Lean Six Sigma Notes 577 578 580 587 590 Chapter 29: Building an Effective IT Internal Audit Function 591 Establishing an IT Internal Audit Function Internal Audit Charter: An Important IT Audit Authorization Role of the Chief Audit Executive IT Audit Specialists IT Audit Managers and Supervisors Internal and IT Audit Policies and Procedures Organizing an Effective IT Audit Function Importance of a Strong IT Audit Function Note 592 593 595 596 598 599 601 604 605 Chapter 30: Professional Certifications: CISA, CIA, and More 606 Certified Information Systems Auditor Credentials Certified Information Security Manager Credentials Certificate in the Governance of Enterprise IT Certified Internal Auditor Responsibilities and Requirements Beyond the CIA: Other IIA Certifications CISSP Information Systems Security Professional Certification Certified Fraud Examiner Certification ASQ Internal Audit Certifications Other Internal Auditor Certifications Note 607 609 611 612 623 628 628 629 630 631 Chapter 31: Quality Assurance Auditing and ASQ Standards 632 Duties and Responsibilities of Quality Auditors Role of the Quality Auditor Performing ASQ Quality Audits Quality Assurance Reviews of IT Audit Functions Future Directions for Quality Assurance Auditing Notes Index 633 635 638 641 647 648 649 E1CINTRO 09/18/2010 11:6:36 Page 13 Introduction: Importance of IT Auditing W E L C O M E T O T H E W O R L D of IT Audit, Control, and Security. Much has changed in information technology (IT) auditing since we published our first edition of this book when we were then called Computer Auditors. Back in those days, traditional mainframe or legacy computer systems were still common, we had difficulty envisioning laptop systems as serious business information systems tools, and the Internet was little more than an e-mail and text document communications tool for many. Computer security then was largely based on locked, secured mainframe facilities, and we were just seeing the very first computer viruses. Many auditors, both internal and external, typically had only limited knowledge about IT systems controls, and there were wide knowledge gaps among auditors, systems security specialists, and developers. It is hard to focus on just one development or event that has turned our view of IT audit controls into a separate discipline. However, the overall influence of the Web along with audit, security, and internal controls concerns has made IT controls more important to many today. This book focuses on both the technical and professional issues facing today’s audit, security, and internal control specialists in an information systems environment, with the goal of providing an understanding of key IT audit security and internal controls issues. We have expanded our audience beyond just auditors to include IT security and internal control specialists as well. Although some may not have not have specific job titles covering these audit, security, and internal controls disciplines, many professionals in today’s enterprises have a responsibility to ensure that good IT controls have been installed and are operating. IT auditors are key persons responsible for assessing these controls. Although the individual chapters of this book, outlined next, cover a broad range of technical and audit-related topics, each of the chapters focuses on three broad IT audit topic areas: 1. Technology-driven audit and internal controls. The effective IT auditor today needs to have a good understanding of a wide range of IT technologies as well as appropriate related audit, security, and control issues and techniques. As our first broad concentration area, the text addresses some of the more significant technology changes today along with their audit, security, and internal control implications. This book is not a detailed technology tutorial, but we describe important IT issues and introduce their IT control procedures in the following broad areas: & Electronic commerce systems, including the use of the XBRL protocol as well as wireless and cloud computing. This area generally goes under the name of ‘‘e-business’’ with evolving standards and good practices. xiii E1CINTRO 09/18/2010 xiv & & & & & & 11:6:37 & Page 14 Introduction Modern application implementation processes, including the use of comprehensive enterprise resource planning (ERP) software packages, software as a service (SAAS) implementation approaches and object oriented-application development processes. Effective IT continuity planning processes. Because virtually all enterprise operations today are tied to often interlocking IT processes, facilities must be in place to restore them to normal operations if some unexpected event arises. Systems infrastructure controls for managing existing applications and operations. Configuration management, service-level agreements, and effective customer service functions are all important in today’s modern IT environment. Effective IT governance procedures. Whether Sarbanes-Oxley Act (SOx) rules or international standards guidelines, all IT organizations today must understand and comply with the many new rules covering all aspects of IT governance and operations. The importance of storage management. Effective processing rules and IT governance requirements require that we keep backup copies of much of the data we use in IT operations as well as database operations to allow an enterprise to search for and retrieve that data easily. IT storage audit, security and internal control issues, and newer concepts such as virtualization are important IT concerns. Modern computer security procedures, including trusted networks and firewallprotected systems. Enterprises need to protect their data in light of ever-increasing threats in today’s Web and wireless environments. This list is not all-encompassing but highlights the overall topics in these chapters. Although some of these expressions may seem like buzzwords or techno-jargon to some readers, the chapters to come introduce many technical concepts with an emphasis on their related audit, security, and internal control concepts and procedures. 2. Security, privacy, and continuity issues. As the second broad topic area in this book, we discuss disaster recovery planning as well as effective continuity and information systems security processes in a modern IT environment. The emphasis is more on getting the business back in operation rather than just getting the IT resources working again. Closely related to security matters, privacy is another issue facing IT auditors. We are increasingly seeing legislation mandating privacy protections over multiple types of information systems data, such as medical records, financial data, and other areas. These new rules have encouraged many enterprises to install strengthened internal controls. 3. Auditing legislative and governance changes. Professionals constantly face legal and other changes that impact their work. Understanding and developing appropriate procedures is this book’s third broad objective. Although it occurred years ago now, the catastrophic failure of the then-prominent corporation, Enron, introduced a raft of new issues. Based on its stock market capitalization, Enron was then a rapidly growing large company engaged in trading oil, gas, and other commodities. Enron’s financial reports, in retrospect, contained many red-flag warnings of possible troubles. Despite these warning signs, Enron’s external auditors seemingly looked the other way. Enron subsequently collapsed, hurting many and leaving a trail of recriminations and questions about the overall independence and objectivity of its external auditors. As a result, the U.S. Congress passed the Sarbanes-Oxley Act (SOx), which changed the E1CINTRO 09/18/2010 11:6:37 Page 15 Introduction & xv process of auditing internal accounting controls for financial management as well as for external and internal auditors. The text discusses how SOx rules continue to impact auditors and financial management with a focus on internal controls, security, and IT auditing. The worldwide market meltdown starting in 2008 has caused a series of other concerns when what were then major financial and other enterprises worldwide either totally failed or lost value to investors. Audit, security, and internal control issues and concerns do not just arise because of a single event, such as the 9/11 terrorism attack or the Enron bankruptcy. Our third focus area, auditing legal and governance changes, evolves over time, and we will discuss newer issues and what we feel are evolving concerns. Our emphasis over these next chapters is on newer rules, evolving technologies, and their combined impact on today’s IT audit professional. Our text covers a blend of audit, internal control, and security issues that are key knowledge areas for IT auditors. These are also topics where financial (often external) auditors, internal audit management, and other professionals who are not IT audit specialists should have at least some general understanding. IT security and controls issues have become so pervasive today that enterprise professionals at many levels should have a general knowledge of them. An overall objective of this book is to highlight areas that are the most important, from an internal controls risk perspective, for today’s IT auditors. These chapters present a high-level overview of each of the three broad objective areas just discussed. No matter what the technical topic, there are always opportunities to present even more detailed technical information. However, we focus on areas that we feel are important to today’s professional, whether an enterprise IT audit staff member, a manager, or a student learning more. In summary, the chapter-by-chapter IT audit, control, and security topics discussed in this book include: Chapter 1, SOx and the COSO Internal Controls Framework. The SarbanesOxley Act and its internal controls assessment rules have been the biggest regulatory changes in decades, and they have impacted both auditors and enterprise management in the United States and worldwide. This chapter summarizes the SOx Section 404 requirements for internal control reviews tosupport an enterprise’s financialreports.Our emphasis, however, is on internal accounting controls reviews of primarily IT applications. The chapter also discusses the newer financial audit AS5 rules released in late 2008. In addition, we discuss the Committee of Sponsoring Organizations (COSO) framework on internal controls as well as some the newly released COSO guidance materials for monitoring internal control systems. This chapter emphasizes an IT auditor’s responsibility for understanding and using the COSO internal controls framework. Chapter 2, Using CobiT to Perform IT Audits. A more IT-oriented internal controls framework, called Control Objectives for Information and related Technology (CobiT), was in place even before SOx, and many enterprises began to use CobiT when SOx became the law as a preferred tool for complying with its Section 404 internal controls procedures. The CobiT framework provides guidance on evaluating and understanding internal controls, with an emphasis on enterprise IT resources. CobiT is not a replacement for the COSO internal controls framework but is a different way to look at internal controls in today’s IT-centric world. E1CINTRO 09/18/2010 xvi 11:6:37 & Page 16 Introduction Although originally launched as a tool to help specialist internal and external auditors who reviewed IT-related internal controls, CobiT today is a helpful tool for evaluating all internal controls across an enterprise. It emphasizes the linkage of IT with other business resources to deliver overall value to an enterprise. This chapter provides an overview of the CobiT framework and its key components. More important, the chapter describes the relationship between CobiT objectives and the COSO internal control framework for use in internal audit reviews. Even if an internal auditor does not use the CobiT framework in reviews of internal controls, all internal and IT auditors should have a high-level knowledge of the basic CobiT framework. Knowledge of CobiT and the COSO internal controls framework will help an IT auditor to better understand the role of IT controls and risks in many enterprise environments. Chapter 3, IIA and ISACA Standards for the Professional Practice of Internal Auditing. Every profession requires a set of standards to govern their practices, general procedures, and ethics. The key standards for all internal auditors are the Institute of Internal Auditors’ (IIA’s) Professional Standards for the Practice of Internal Auditing, a set of guidance materials that were most recently revised in 2009. This chapter summarizes the current IIA standards and provides guidance on how to apply them. The chapter also introduces the IIA’s Global Technology Audit Guide (GTAG) series IT guidance materials. An understanding of the IIA’s International Standards is an absolute must for all internal and IT auditors. These standards provide the support for many if not nearly all internal audit professional activities. This chapter also revisits the IIA Code of Ethics, an important supporting foundation for internal and IT auditors, as well as the Code of Ethics for the Information Systems Audit and Control Association (ISACA). ISACA members are often IIA members or Certified Public Accountants, but the ISACA Code of Ethics places a special emphasis on IT-related activities. Although ISACA does not have the same type of standards as the IIA, its CobiT IT internal control framework provides standards guidance for IT auditors. The chapter also highlights ISACA’s standards and guidelines. These are particularly important for IT auditors. In addition, Chapter 31 introduces another very important set of internal audit standards, the quality audit standards from the American Society for Quality (ASQ). ASQ’s internal audit standards and its quality auditors represent a different dimension and discipline when contrasted with the IIA’s and ISACA’s approaches and standards. They also represent an area that must be better represented and understood in the overall world of internal auditing. Chapter 4, Understanding Risk Management through COSO ERM. Although the term enterprise risk management (ERM) is frequently used, many IT auditors do not have a consistent understanding of and how to use it as a tool for effective internal audit reviews. This chapter introduces the COSO Enterprise Risk Management (COSO ERM) framework and its elements. Although their basic framework models look similar, COSO ERM is different from the COSO internal controls framework discussed in Chapter 1. COSO ERM is an important reference to better understand and evaluate the risks surrounding internal controls at all levels. This chapter describes the major elements of the COSO ERM framework and looks at how IT auditors can better build E1CINTRO 09/18/2010 11:6:37 Page 17 Introduction & xvii COSO ERM into their review processes as well as steps for auditing the effectiveness of an enterprise’s risk management processes. Every IT auditor needs to have an understanding of risk assessment approaches and the overall risk management process, with an emphasis on COSO ERM. This chapter presents IT audit techniques for understanding and assessing risks in many areas, from selecting items to review to evaluating risks as part of IT audit reviews. Chapter 5, Performing Effective IT Audits. This chapter describes basic processes for performing an effective IT audit. Basic reviews can be performed by many specialists in an enterprise, but this chapter focuses on basic audit steps for performing an IT internal audit review, ranging from risk-based audit planning to preparing effective internal audit reports. This chapter does not describe the overall internal audit review process but emphasizes some of the key elements necessary to perform effective IT-related reviews. Chapter 6, General Controls in Today’s IT Environments. IT processes and systems today range from an application to control an enterprise’s accounting general ledger to the all-pervasive Internet. Although the lines of separation are sometimes difficult, we can generally think of IT controls on two broad levels: application controls that cover a specific process—such as an accounts payable application to pay invoices from purchases—and what are called general IT controls. This latter category covers internal controls that do not relate just to specific IT applications but are important for all aspects of an enterprise’s IT operations. The concept of IT general controls goes back to the early days of centralized, mainframe computers. At that time, internal auditors sometimes looked for such things as an access control lock on a computer center door as a general control that covered all processes and applications operating from within the centralized IT operations center. Today, we often think of the processes that cover all enterprise IT operations as the IT infrastructure, a concept further discussed in Chapter 7. This chapter discusses reviews of IT general controls from an IIA standards and CobiT perspective. Although general controls were once considered in terms of centralized mainframe computer center operations in an earlier era, today we should think of them as those controls impacting any set of similar IT machine resources. For example, an internal audit function may equip its entire staff with laptop computers, and good general controls are necessary in this environment to encourage all internal and IT auditors to use common software control procedures on those assigned laptops. General controls weaknesses can impact all IT processes. Chapter 7, Infrastructure Controls and ITIL Service Management Best Practices. This chapter looks at IT general controls based on the worldwide recognized set of best or good practices called the Information Technology Infrastructure Library (ITIL). These ITIL recommended best practices outline the type of framework an IT auditor should consider when reviewing IT internal control risks and recommending effective IT general controls improvements. ITIL processes cover what we frequently call the IT infrastructure—the supporting processes that allow IT applications to function and deliver their results to systems users. For example, an ITIL process outlines best practices for installing an effective IT help desk operation for all systems users. All too often, auditors have focused their attention E1CINTRO 09/18/2010 xviii 11:6:37 & Page 18 Introduction on the application development side of IT and ignored important service delivery and support IT processes. An enterprise can put massive efforts into building and implementing a new budget forecasting system, but that application will be of little value unless there are good problem and incident management processes in place to allow the users of this budget forecasting system to resolve systems difficulties. Also needed are good capacity and availability processes to allow the new application to run as expected. ITIL processes are part of what is called the IT infrastructure. IT auditors should have a good understanding of these enterprise processes, and they should be covered in IT audit general controls reviews. Chapter 8, Systems Software and IT Operations General Controls. Whether it is a Microsoft Windows operating system on a laptop computer or Linux controlling an office server, the operating system (OS) and its supporting software are key components to any computer system operation. This chapter discusses some of the various OS types and the supporting systems software that are essential in an IT operation. IT auditors should have a very general understanding of the purposes and importance of these types of IT software and should look for effective general controls when performing reviews in this important general controls area. Chapter 9, Evolving Control Issues: Wireless Networks, Cloud Computing, and Virtualization. New IT technologies make many processes easier to use or more efficient, but they often introduce new internal control concerns. This chapter has selected three of these newer areas and considers internal control risks and potential audit procedures for each. Wireless networks is the first topic here. Although it is certainly convenient to not have to connect IT terminals and other devices through a formal cable network, the very open environment of using wireless technology approaches introduces some security and control risks. The chapter looks at wireless networks from an audit, security, and control perspective. As a second topic area, the chapter introduces the rapidly evolving IT configuration called cloud computing. Although the term sounds almost exotic to many, it has become a significant concept today with our growing dependence on using Web-based applications for many business processes rather than applications downloaded to home office servers. Cloud computing processes are also called web services, software as a service (SaaS), or service-oriented architecture (SOA). In cloud computing, many different Internet applications—supported by multiple vendors and operating on multiple servers—operate together out of what looks like a large fuzzy Internet cloud. This chapter introduces cloud computing concepts and discusses some security and controls concerns that may impact IT auditors in their assessments of IT general controls. This chapter’s third topic is virtualization. With the massive amount of data and information that most enterprises retain, storage management is very important for almost all IT operations. Whether it is the very high-capacity miniature USB devices so common today or new database tools, there is a need to install appropriate controls in these storage management environments. In virtualization, any device can be defined to look like another. This concept can create a challenging environment for an IT auditor, and the chapter provides some introductory internal controls guidance to these evolving areas. E1CINTRO 09/18/2010 11:6:37 Page 19 Introduction & xix Chapter 10, Selecting, Testing, and Auditing IT Applications. In order to perform internal controls reviews in specific areas of enterprise operations, such as accounting, distribution, or engineering, IT auditors must have the skills to understand, evaluate, and test the controls over their supporting IT applications. Reviews of specific application controls often are more critical to achieving overall audit objectives than reviews of general IT controls. This chapter discusses approaches to review internal accounting controls in IT applications, using several different types of applications as examples. The chapter also discusses audit approaches for evaluating and testing those application controls as well as techniques for reviewing new applications under development. We focus on the internal control characteristics of different types of applications and then discuss how to select appropriate applications in internal controls reviews. There are many differences from one application to another; this chapter focuses on how an IT auditor should select higher-risk applications as candidates for IT audit reviews; the tools and skills needed to understand and document application internal controls; and, finally, processes to test and evaluate those applications. Chapter 11, Software Engineering and CMMi. The Carnegie-Mellon University Software Engineering Institute’s IT-based Capability Maturity Model for integration (CMMi1) is an effective approach for an enterprise and its IT software development functions to assess how well they are organized. It is a measure on whether processes are well managed, repeatable, or even unpredictable. IT auditors can use CMMi as a tool to measure how well they are doing, and it can serve as a guide for process improvements. The chapter provides an overview of how IT audit specialists can use CMMi in their assessments of internal controls and in organizing their own IT projects. Chapter 12, Auditing Service-Oriented Architecture and Record Management Processes. SOA is an IT systems approach where an application’s business logic or individual functions are modularized and presented as services for consumer/client applications. This chapter introduces SOA concepts for the IT auditor and discusses internal control and IT auditor issues surrounding the development and operations of IT applications using this technology. The chapter also reviews the importance of effective records management systems in today’s enterprises from an internal controls and IT audit perspective. Today, almost all business records are created and most live their entire lives electronically. Failure to manage electronic records and physical records in accordance with established records management principles ignores potential risk. Chapter 13, Computer-Assisted Audit Tools and Techniques. IT auditors test and review the internal controls surrounding their IT systems, and they often need tools to better understand and evaluate the completeness and accuracy of the data stored in the IT applications’ files and databases. This chapter reviews approaches to retrieving data through computer-assisted audit tools and techniques (CAATTs), the use of independent auditor–controlled software to assess organization IT files and documents. Whether purchased software administered by an IT auditor or an operational procedure to better analyze IT data, many tools and techniques can help make audit reviews of IT-supported systems more efficient and effective. The chapter provides IT auditors with a basic understanding on the general use of CAATTs to access and review automated data to support IT audits. E1CINTRO 09/18/2010 xx 11:6:37 & Page 20 Introduction Chapter 14, Continuous Assurance Auditing, OLAP, and XBRL. Continuous assurance auditing (CAA) is the process of installing control-related monitors in automated systems such that these monitors will send messages to internal auditors if the system’s processing signals a deviation from an established audit limit or parameter. This chapter discusses CAA as an improved alternative approach for reviewing automated systems as well as an overview of continuous monitoring (CM), business controlled procedures that can be subject to periodic internal IT audits. The chapter also introduces XBRL, the extensible business reporting language developed by the American Institute of Certified Public Accountants. XBRL is a standards-based way to communicate business and financial information across multiple enterprises. IT auditors should have a basic understanding of XBRL, a methodology that is growing in recognition and use, and its necessary supporting internal controls. Chapter 15, IT Controls and the Audit Committee. The management or supervisory authority of enterprise’s internal audit function is the board of directors’ audit committee. This committee is responsible for approving internal audit plans, reviewing audit reports, hiring internal audit management, and taking other actions as appropriate. Although historically much of an audit committee’s interests have been based on audited financial statements and an enterprise’s financial audit, IT audit has an important role here as well. This chapter examines planning and reporting key IT audit activities to the enterprise’s audit committee. Chapter 16, Val IT, Portfolio Management, and Project Management. This chapter looks at three knowledge areas that are important to IT auditors: (1) ISACA’s enterprise value initiative, called Val IT, to better manage and understand IT investments; (2) portfolio management approaches to better deal with the large number of diverse applications and IT resources in the typical enterprise; and (3) project management good practices to better control and manage many IT activities. Internal audit functions normally should develop an audit universe list, a compilation of all potential auditable entities for that enterprise. The chapter also looks at audit universe portfolio management from an IT audit perspective. The chapter also discusses the importance of effective project management procedures. Audits and internal control activities should be planned and performed in a well-organized manner. The program and project management procedures described in the chapter will aid in this process. Chapter 17, Compliance with IT-Related Laws and Regulations. Both in the United States and worldwide, a wide range of laws and regulations impact enterprise IT operations. Some of these have direct IT connections, such as the U.S. Computer Fraud and Abuse Act (CFAA), while others, such as SOx, are not really IT-related laws but nevertheless have multiple IT relationships. This chapter reviews a series of primarily U. S. IT-related laws and regulations, highlighting areas that should be considered in IT audit reviews. Chapter 18, Understanding and Reviewing Compliance with ISO Standards. The International Standards Organization (ISO) has guidance that covers a wide range of areas, such as defining fastener screw threads in an automobile engine, the thickness of a personal credit card, and IT quality standards. This chapter provides an overview and introduction to several of the many ISO standards that are particularly E1CINTRO 09/18/2010 11:6:37 Page 21 Introduction & xxi important for IT auditors, with a focus on ISO 27001 and 27002 computer security standards. The chapter also provides an introduction to several other ISO standards, including international standards for IT management systems and for quality management. Enterprise compliance with appropriate ISO standards is important worldwide today, as they establish benchmarks for worldwide compliance. Chapter 19, Controls to Establish an Effective IT Security Environment. Effective IT security is very important in all enterprises. Beyond password controls and backup processes discussed in other chapters, an enterprise needs a high-level enterprise commitment to IT security as well as strong procedures to build effective IT security processes. These procedures can range from such matters as a strong enterprise code of conduct setting the rules for enterprise associates to overall management policies promoting the need for an effective security environments. This chapter outlines some techniques as well IT audit procedures to review enterprise-wide IT security effectiveness. Chapter 20, Cybersecurity and Privacy Controls. In our Web-dominated world today, IT security or cybersecurity and privacy controls over data and information are very important. This chapter discusses these controls from two focus areas: (1) some of the many cybersecurity and privacy concerns that IT auditors should consider in their reviews of systems and processes and (2) IT privacy issues. We have limited our focus to only some cybersecurity areas because the field of IT security controls is vast and sometimes very technical, beyond the skills of many auditors. Regarding IT privacy, there is a growing set of issues about how much personal data and information individuals should allow to be given to interested enterprises, government authorities, and even other individuals. Chapter 21, IT Fraud Detection and Prevention. An effective auditor needs to recognize potential fraudulent business practices as part of any IT audit and then should recommend controls and procedures to limit exposure to the fraudulent activity. This chapter outlines some of the red flags—common conditions that an IT auditor might encounter when faced with a potential fraud as well as steps to identify, test, and properly process fraudulent activities. Fraud investigation can be a very detailed and specialized activity, but IT auditors should have a high level understanding of how to audit for potentially fraudulent activities as well as of processes for investigating and reporting fraud. Fraudulent activities represent a breakdown in a wide range of good practices and procedures, but IT auditors must recognize that fraudulent activities always may exist. Chapter 22, Identity and Access Management. With all of the personal data stored in so many IT databases and systems, there is always a major concern that some perpetrator will hijack and steal someone’s personal information to use for improper purposes. The concern that someone will steal this personal information is called identity management. Although certain laws are in place to prohibit such actions, an enterprise needs to establish strong procedures to discourage such improper activities. Starting with effective IT password access controls and the monitoring of potential password violations, an enterprise needs to have strong identity and access management processing in place. This chapter outlines procedures that are effective for IT security management and provides guidance for performing effective IT audits. E1CINTRO 09/18/2010 xxii 11:6:37 & Page 22 Introduction Chapter 23, Establishing Effective IT Disaster Recovery Processes. This chapter introduces what is called IT disaster recovery planning processes. (Chapter 25 discusses business continuity controls, which are similar but very different.) IT disaster recovery includes effective backup processes for restoring all aspects of IT operations, whether a classic server-based data center or a network of laptop or desktop system operations. The chapter briefly introduces some of the technical tools, such as data mirroring, that improve IT disaster recovery procedures today. The chapter concludes with guidance on testing disaster recovery plans as well as steps for auditing such plans. Because of our massive dependence on IT operations, effective disaster recovery plans are key components in IT operations. Chapter 24, Electronic Archiving and Data Retention. Because so much data and supporting documentation is recorded on databases or other IT media formats, it is important to preserve backup copies in separate, independent locations. This is as true for an enterprise with massive business transactions as it is for an IT auditor. The chapter discusses some best practice approaches for saving and archiving data as well as control procedures for its access. In addition, the chapter outlines steps for an audit of data retention processes. Chapter 25, Business Continuity Management, BS 25999, and ISO 27001. This chapter introduces best practices for effective enterprise IT continuity and disaster recovery planning processes including establishing enterprise internal controls in enterprise-critical areas. Going beyond Chapter 24’s disaster recovery backup procedures, continuity planning is based on the concept that an enterprise needs to have processes in place to resume normal business operations in the event of a major disruption in IT services. This major task involves restoring business process operations beyond just the IT applications. This chapter also discusses BS 25999, a U.K.-based standard for business continuity management as well as the ISO 27001 international standards. The chapter uses BS 25999 to describe the processes necessary for effective continuity planning for a sample enterprise. Chapter 26, Auditing Telecommunications and IT Communications Networks. Moving beyond the wireless networks discussed in Chapter 9, enterprise IT systems typically are tied to vast telecommunications networks, internally or through the Web. This chapter provides an introduction to the wide variety of network topologies in place today and outlines processes surrounding IT network controls and tools such as scanners and sniffers. The chapter outlines approaches for an IT auditor’s internal controls and security-related review of an enterprise’s telecommunications including its wireless operations. Chapter 27, Change and Patch Management Controls. Enterprises can be exposed to major security vulnerabilities in the event of unauthorized or inappropriate changes to its IT systems and programs. Strong IT change management processes are always needed. This chapter discusses effective IT change and patch management controls and introduces procedures for auditing internal controls in these areas. Chapter 28, Six Sigma and Lean Technologies. Enterprise operations managers, at all levels, are regularly looking for ways to improve their operations, whether in shop-floor production processes, office administrative procedures, or elsewhere. Many E1CINTRO 09/18/2010 11:6:37 Page 23 Introduction & xxiii have found six sigma processes to be effective here. This chapter provides a high-level introduction to six sigma concepts and how they can be applied to enterprise IT operations. We provide an overview of six sigma and the lean approaches to implementing it. Even though an internal or IT audit function may not be using six sigma concepts as part of their overall operations, IT auditors should have a basic understanding of this important quality improvement concept. Chapter 29, Building an Effective IT Internal Audit Function. Other chapters have discussed many aspects of IT audits. This chapter steps back and looks at some the requirements for an enterprise to build an effective IT audit function as part of their overall internal audit group. The chapter reviews IT audit positions descriptions, audit planning, workpapers, audit reports, and other factors needed to build an effective IT internal audit function. Chapter 30, Professional Certifications: CISA, CIA, and More. IT auditors have a need for strong and well-recognized professional certifications. Many have joined the profession with no specific certification requirements beyond their undergraduate college degrees; others attained accounting degrees and prepared for the Certified Public Accountant (CPA) examination. Today, an IT auditor can take a qualifying examination and complete other requirements to become a Certified Information Systems Auditor (CISA), a Certified Internal Auditor (CIA), a Certified Fraud Examiner (CFE), or any of a series of other certifications. This chapter discusses the professional designations that are important to the IT auditor, with an emphasis on the CIA and CISA certifications, including their qualification and examination requirements. In addition, the chapter considers some other certification options available to IT auditors, including the Certified Information Systems Security Professional (CISSP) requirements. Chapter 31, Quality Assurance Auditing and ASQ Standards. This chapter reviews the role of quality auditors in an enterprise, their practices and standards. There are many similarities between the activities of quality auditors and the IT auditors that are the main focus of this book. With a growing convergence of enterprise activities to improve governance, IT processes, and internal controls, we can expect to see these internal audit groups become more closely aligned. Although we focus more on IIA and ISACA types of IT internal auditors, there also is a need for a general understanding of the roles, responsibilities, and activities of quality auditors. In addition, we also consider internal audit Quality Assurance (QA) reviews of an IT audit function performed by members of the internal audit team themselves or by contracted outside reviewers. HOW TO USE THIS BOOK The role of an IT auditor is changing now and will change even more in the future. The internal control SOx, COSO, and CobiT models presented in Chapters 1 and 2 suggest a much broader role for the IT audit and internal controls specialist of the future. In addition, technology changes and new concepts, such as many of the newer issues introduced throughout this book, will require that IT auditors develop expanded knowledge needs. An overall objective of this book is to introduce some of these newer E1CINTRO 09/18/2010 xxiv 11:6:37 & Page 24 Introduction concepts. Our objective is not to provide an exhaustive tutorial in each subject area but to discuss concepts and related IT audit, security, and internal controls issues. The chapters throughout this book contain many suggested audit programs—the steps necessary to perform actual IT audits. There is an increasing need for persons with a broad knowledge of IT audit, security, and internal controls. Although there is also a need for some with very specialized skills, such as a computer crime investigator or a financial auditor with detailed knowledge of a specialized area, such as bond indentures, today’s IT auditor should have good skills in the overall areas of audit, control, and security. Providing an overview and some background guidance is a goal of this edition. The author hopes that this edition and potentially more frequent updates will help to provide both new and experienced audit and internal control specialists with information to help them become more effective professionals. E1CINTRO 09/18/2010 11:6:37 Page 25 IT Audit, Control, and Security E1CINTRO 09/18/2010 11:6:37 Page 26 E1C01 08/06/2010 14:5:3 Page 1 I PART ONE Auditing Internal Controls in an IT Environment E1C01 08/06/2010 14:5:3 Page 2 E1C01 08/06/2010 14:5:3 Page 3 1 CHAPTER ONE SOx and the COSO Internal Controls Framework T H E C O N C E P T O F I N TE R N A L controls assessments has been around since the inception of auditing and has been an important concept going back to the early days of information technology (IT) auditing. Although there have been many definitions of internal controls, a good one for IT auditors is that internal control is a process, affected by an entity’s board of directors, management, and other personnel, that is designed to provide reasonable assurance regarding the achievement of objectives in the categories of effectiveness and efficiency of operations, reliability of an enterprise’s financial reporting, and an enterprise’s systems and process compliance with laws and regulations. This well-recognized definition was established by the U.S. Committee of Sponsoring Organizations (COSO), an internal controls standards-setting authority that we will be discussing further in this chapter. Audit professionals are responsible for reviewing and assessing enterprise management controls. Internal auditors do not construct and administer these controls—that is the responsibility of management. Auditors, acting as independent parties, both review and perform tests of enterprise internal controls to report to management and other parties whether they are adequate. These reviewers consist of both internal and external auditors, with external auditors in the United States following the rules and standards of the American Institute of Certified Public Accountants (AICPA). Internal auditors follow a similar but different set of standards and generally subscribe to the guidelines of the Institute of Internal Auditors (IIA), their international professional organization. Both of these audit organizations have heritages going back to paper-and-pencil days, before today’s pervasive use and reliance on IT systems and processes. Over the years, the Information Systems Audit and Control Association (ISACA) and its IT audit professionals have provided guidance for IT-related internal controls. IT auditors serve 3 E1C01 08/06/2010 14:5:3 4 Page 4 & SOx and the COSO Internal Controls Framework in both external and internal audit roles, although most professionals may serve as internal auditors for their enterprises. This chapter outlines the role of an IT auditor, particularly an IT internal auditor, in today’s business enterprise. In addition, the chapter discusses two important IT audit concepts: the COSO internal control standards and the Sarbanes-Oxley Act (SOx) internal control review rules. Both COSO internal controls and SOx started as U.S. internal controls guidance rules but have become worldwide standards. They both had their origins as general financial and operations review standards and are now very applicable to IT audit environments as well. Today’s IT auditor must understand and use the COSO internal controls framework and SOx internal controls review procedures. Although these rules and procedures have origins in financial reporting and auditing, in today’s IT-centric world, COSO internal controls and SOx are equally important to IT auditors. Enterprises need to follow these rules in order to assert or attest to regulators that their organizations have effective internal controls in place and that they are operating in compliance with those newer rules. The chapters in this volume rely on the internal control rules and procedures as we discuss a wide range of other IT audit, control, and security topics. ROLES AND RESPONSIBILITIES OF IT AUDITORS Much of this chapter and others focuses on the roles and responsibilities of an internal audit specialist, whom we call an IT or information systems auditor. Although sometimes serving as a member of a public accounting firm or outside consulting organization, IT auditors are generally members of an enterprise internal audit organization. An internal audit group is led by a manager with the title of chief audit executive (CAE) and is staffed by internal auditors with skills in reviewing and understanding operational and financial controls as well as compliance and regulatory issues impacting the enterprise. With IT processes and tools so pervasive in today’s enterprise, all internal auditors should have a good understanding of IT controls and processes, but many internal audit functions require the skills of what we are calling an IT auditor.1 Traditional internal auditors always have had skills in understanding, testing, and evaluating what were once traditional paper-based controls and procedures. Starting in the 1970s, as enterprises started to build and implement more and more computerbased applications, they needed internal audit specialists who understood the new systems. Thus the role of the IT auditor was born. The field once was called electronic data processing (EDP). Auditors are now sometimes known as information systems (IS) auditors or computer audit specialists; however, we are using the expression IT auditor throughout this book. An IT auditor is a specialist who follows the standards and principles of the IIA and often is a member of ISACA as well. There are many recognized specialist skills here, including the IT security procedures discussed in Chapter 19 and IT auditors skilled in computer-assisted audit tools and techniques (CAATTs), but most IT auditors are expected to have a strong E1C01 08/06/2010 14:5:3 Page 5 Roles and Responsibilities of IT Auditors & 5 Auditor, Information Systems JOB DESCRIPTION Job Summary: Under direction of the Chief Audit Executive (CAE) and internal audit management, audits, reviews, tests, and evaluates IT-based applications and control procedures and reviews electronic security over the enterprise IT services network. CHARACTERISTIC JOB TASKS AND RESPONSIBILITIES May include any and/or all of the following: 1. Designs a technology-based audit approaches; analyzes and evaluates enterprise IT processes to assess internal controls and minimize risks; performs risk analysis of the enterprise’s information technology infrastructure and services network; evaluates the possible risks of various computer systems; prepares reports documenting findings and risk assessment; evaluates management responses to findings and risk assessment. 2. Works independently or with other members of internal audit to review enterprise internal controls, following the COSO internal controls framework. 3. Examines the effectiveness of the information security policies and procedures; identifies inadequacies within the existing security program and possible action to be taken. 4. Develops and implements computer-assisted audit tools and techniques (CAATTs) to assist overall internal audit efforts and performs other IT-related tests of controls, as appropriate. 5. Develops and presents training workshops for audit staff on security controls and risk concepts. 6. Conducts and oversees investigation of inappropriate computer use. 7. Performs special projects and other duties as assigned; provides input on departmental administrative activities. KNOWLEDGE, SKILLS, ABILITIES, AND PERSONAL CHARACTERISTICS & Knowledge of auditing, information systems, and network security & Investigation and process flow analysis skills & Interpersonal/human relations skills & Verbal and written communication skills & Ability to exercise good judgment & Ability to maintain confidentiality & Ability to use IT desktop office tools, vulnerability analysis tools, and other IT tools MINIMUM QUALIFICATIONS Education and experience equivalent to: & Bachelor’s degree in computer science, computer programming, or accounting & Certified Information Systems Auditor (CISA) credentials or candidate & Certified Internal Auditor credential preferred EXHIBIT 1.1 IT Auditor Job Description general set of skills in evaluating IT-based internal controls. Exhibit 1.1 is a position description for a typical senior IT auditor. This chapter emphasizes the importance of IT audit processes in performing internal controls reviews in today’s heavily IT powered enterprises. IT audit specialists also have important key roles in the corporate governance of today’s enterprise. They have skills that are unique but certainly should be adopted by all members of an internal audit team in expanding their IT knowledge and internal controls review procedures. E1C01 08/06/2010 14:5:3 6 Page 6 & SOx and the COSO Internal Controls Framework IMPORTANCE OF EFFECTIVE INTERNAL CONTROLS AND COSO Internal control is one of the most important and fundamental concepts that external and internal auditors and business professionals at all levels must understand. The business professional builds and uses internal controls; auditors review and test the operational, IT, and financial systems and processes with a goal of evaluating their internal controls. Although internal and external auditors have different objectives, most of our references in this chapter apply to IT auditors, who have a major responsibility to understand and assess IT-related internal controls. Although there have been many slightly different definitions of internal controls in the past, of COSO standards provides an appropriate definition. It recognizes that internal control extends beyond just accounting and financial matters and includes all enterprise processes. Also, because IT is so embedded into almost all business processes, IT-related internal controls are a major portion of our overall understandings of internal controls. An enterprise unit or process has good internal controls if it: 1. 2. 3. 4. 5. Accomplishes its stated mission in an ethical manner Produces accurate and reliable data Complies with applicable laws and enterprise policies Provides for economical and efficient uses of its resources Provides for appropriate safeguarding of assets All members of an enterprise are responsible for the internal controls in their area of operation and for operating them effectively. Despite or perhaps because of this broad and wide-reaching internal controls definition, many business professionals have had problems in fully understanding and applying internal control concepts. Looking at our definition a bit differently, the concept of an internal control and supporting control processes goes back to the basic mechanical and paperwork procedures that once existed throughout everyday life. Control processes are necessary for activities inside and outside today’s enterprise, and many basic concepts and principles are the same no matter where the control is implemented. An automobile provides some basic controls examples. When the accelerator—a speed control—is pressed, the automobile goes faster. When the brake— another control—is depressed, the automobile slows or stops. When the steering wheel is turned, the vehicle turns. The driver controls the automobile, and all three of these represent the car’s basic internal control system. If the driver does not use or improperly uses the accelerator, brake, or steering wheel, the automobile will operate out of control. Expanding this concept just a bit, a stop sign, traffic direction sign, or gate crossing barriers all represent external controls to the auto and its driver. The driver is the operator of the automobile-based internal control process or system but has little decision authority over the message delivered from a traffic light external control. From an internal control perspective, an enterprise can be compared to our automobile example. There are many enterprise systems and processes at work, such E1C01 08/06/2010 14:5:3 Page 7 Importance of Effective Internal Controls and COSO & 7 as accounting operations, sales processes, and IT systems. If management does not operate or direct these processes properly, the enterprise may operate out of control. All members of an enterprise should develop an understanding of the appropriate control systems and then determine if they are properly connected to manage the enterprise. These systems are referred to as the enterprise’s internal control systems. Internal Controls Standards Background Although the concept and definition of internal controls is fairly well understood today, this was not true until the late 1980s. The general concept may have been understood, but there was no consistent agreement among many interested about what was meant by ‘‘good internal controls.’’ Early definitions that first came from the AICPA and were used by the U.S. Securities and Exchange Commission (SEC) for the Securities Exchange Act of 1934 provide a good starting point. Although there have been changes over the years, the AICPA’s first codified standards, called the Statement on Auditing Standards2 (SAS No. 1) defined the practice of financial statement external auditing in the United States for many years. This AICPA definition of internal control has been subject to changes and reinterpretations over the years. Throughout the 1970s, the SEC and AICPA released many internal control definitions, and the major external auditing firms developed voluminous interpretations and guidelines. Things changed in the late 1970s and early 1980s, a period when there were many major U.S. enterprise failures due to factors such as high inflation and the resultant high interest rates. Many times enterprises reported adequate earnings in their audited financial reports, only to suffer a financial collapse shortly after the release of favorable audited financial reports. A few of these failures were caused by fraudulent financial reporting, although many others were due to high inflation or other enterprise instability issues. Nevertheless, several members of Congress proposed legislation to ‘‘correct’’ these potential business and audit failures. Bills were drafted and congressional hearings held, but no legislation was passed. In response to these concerns as well as the lack of legislative action, the National Commission on Fraudulent Financial Reporting was formed. It consisted of five professional organizations: the IIA and AICPA, mentioned previously; the Financial Executive International (FEI), an association of senior financial managers; the American Accounting Association (AAA); and the Institute of Management Accountants (IMA). The AAA is a professional organization for the academic accountants, and the IMA is the professional organization for managerial or cost accountants. The National Commission on Fraudulent Financial Reporting came to be called the Treadway Commission after the name of its chairperson. Its major objectives were to identify causal factors that allowed fraudulent financial reporting and to make recommendations to reduce their incidence. The Treadway Commission’s final report, issued in 1987, included recommendations to management, boards of directors, the public accounting profession, and others.3 It also called for management reports on the effectiveness of their internal control systems and emphasized key elements in what it felt should be a system of internal control, including a strong control environment, codes of conduct, a competent and involved audit committee, and a strong internal audit function. E1C01 08/06/2010 14:5:3 8 Page 8 & SOx and the COSO Internal Controls Framework The Treadway Commission report again pointed out the lack of a consistent definition of internal control, suggesting further work was needed. The same Committee of Sponsoring Organizations that managed the Treadway report subsequently contracted with outside specialists and launched a project to define internal control. Although it issued no standards, the Treadway Commission released the COSO internal control framework, discussed in the next sections and referenced throughout this book. COSO Internal Control Framework As mentioned, COSO refers to the five professional auditing and accounting organizations that formed a committee to develop this internal control report; its official title is Integrated Control–Integrated Framework.4 Throughout this book, we refer to it as the COSO internal controls report or framework. This is in contrast to COSO enterprise risk management (COSO ERM) enterprise resource management framework introduced in Chapter 4. First released in September 1992, the COSO internal controls report proposed a common framework for the definition of internal control as well as procedures to evaluate those controls. In a very short number of years, the COSO internal controls framework has become the recognized worldwide standard for understanding and establishing effective internal controls in virtually all business systems. The next paragraphs provide a fairly detailed description of the COSO internal controls framework and its use by internal auditors and business professionals for internal controls assessments and evaluations. Virtually every public corporation has a complex control procedures structure. Following the format of a classic organization chart, there may be levels of senior and middle management in multiple operating units or within different activities. In addition, control procedures may be somewhat different at each of these levels and components. For example, one unit may operate in a regulated business environment where its control processes are very structured, while another unit may operate almost like an entrepreneurial start-up with a far less formal structure. Different levels of management in these enterprises will have different control concern perspectives. The question ‘‘How do you describe your system of internal controls?’’ might receive different answers from persons in different levels or units in each of these enterprise components. COSO provides an excellent description of this multidimensional concept of internal controls, defining internal control in this way: Internal control is a process, affected by an entity’s board of directors, management, and other personnel, designed to provide reasonable assurance regarding the achievement of objectives in the following categories: & & & Effectiveness and efficiency of operations Reliability of financial reporting Compliance with applicable laws and regulations5 Using this very general definition of internal control, COSO uses a three-dimensional framework to describe an internal control system in an enterprise. Exhibit 1.2 14:5:3 Page 9 & 9 s on tro l om pl ia nc e C ep or tin g al R Information and Communication Internal Controls Internal Control Activities Business Entity-Level Controls Monitoring Internal Controls Business Unit Activities Fi na nc i In te rn al C In te rn al C on tro l s on tro l s Importance of Effective Internal Controls and COSO In te rn al C 08/06/2010 O pe ra tio ns E1C01 Internal Control Risk Assessment Internal Controls Environment EXHIBIT 1.2 COSO Internal Controls Framework describes this COSO internal control framework as a three-dimensional model with five levels on the front-facing side and the three major components of internal control— effectiveness and efficiency of operations, reliability of financial reporting, and compliance with applicable laws and regulations—taking somewhat equal segments of the model with slices across its top. The right-hand side of the exhibit shows three segments, but there could be more, depending on the structure of the enterprise. Each of the COSO internal control framework’s levels, from Monitoring on top down to the Internal Controls Environment, is discussed in greater detail in the sections to come. The idea here is that when we look at the middle internal control activity layer— such as the period-end financial close—we should consider that control in terms of the business unit or entity or the multiple divisions on the side of the framework where that control has been installed. However, in this three-dimensional model, each control is related to all others in the same row, stack, or column. The point of the COSO internal controls framework is that we should always consider each identified internal control in terms of how its components relate to E1C01 08/06/2010 14:5:3 10 Page 10 & SOx and the COSO Internal Controls Framework other associated internal control elements in the framework. In an example of end-ofperiod financial close internal controls, the enterprise should have information and communication links attached to the financial close processes, and the control should be monitored. Dropping down a level, there should be risk assessment activities associated with that financial controls process, and it should operate in an appropriate internal controls environment. Compliance and operations issues also contain factors for specific internal controls that may function at any level in the enterprise organization. All IT auditors should have a strong understanding of the COSO internal controls framework. No matter what area is under review, IT auditors always need to review and consider internal controls in this type of a multilevel and three-dimensional manner. Starting with the first or bottom front-facing level, our text describes the COSO internal controls framework in greater detail. Control Environment The foundation of the COSO internal controls framework is what COSO calls the internal control environment, the foundation for all other components of internal control. It has an influence on each of the three objectives and all unit and entity activities. The control environment reflects the overall attitude of, awareness of, and actions by the board of directors, management, and others concerning the importance of internal controls in the enterprise. There are many fundamental concepts here, and each enterprise will have its own unique internal control foundation. Enterprise history and culture often play a major role in forming this internal control environment. When an enterprise historically has had a strong management emphasis on producing error-free products and when senior management communicates the importance of high-quality products to all levels of the organization, the COSO control environment becomes a major enterprise internal control factor. The content and format of messages from the chief executive officer (CEO) or other senior managers are known as the tone at the top—management’s messages to all stakeholders. However, if senior management has a reputation of looking the other way at policy violations, this same negative message will be communicated to other levels in the enterprise. A positive tone at the top by senior management is a key element of a strong enterprise control environment. IT auditors should always try to understand and evaluate the overall control environment when performing virtually all reviews. When the internal control environment is weak, auditors almost certainly will find additional control concern areas. The control environment consists of the following components. Integrity and Ethical Values. If the enterprise has developed a strong code of conduct that emphasizes integrity and ethical values, and if stakeholders appear to follow that code, all stakeholders will have assurances that the enterprise has a good set of values. A code of ethics or conduct is an important component of organizational governance. Internal audit codes of conduct are discussed in Chapter 3. However, even if an enterprise has a strong code of conduct, its principles can be violated through ignorance E1C01 08/06/2010 14:5:3 Page 11 Importance of Effective Internal Controls and COSO & 11 rather than by deliberate employee malfeasance. In many instances, employees may not know that they are doing something wrong or may erroneously believe that their actions are in the enterprise’s best interests. This ignorance often is caused by poor senior management moral guidance rather than by any individual employee intent to deceive. The enterprise’s policies and values must be communicated to all organization levels. Although there can always be bad apples in any enterprise, strong moral messages will encourage everyone to act correctly. The objective should always be to transmit appropriate messages or signals throughout the enterprise. All stakeholders, and certainly all internal auditors, should have a good understanding of their enterprise’s code of conduct and how it is applied. If the existing code is out of date, if it does not appear to address important ethical issues facing an enterprise, or if management does not appear to be communicating the code to all stakeholders on a recurring basis, management needs to wake up and correct this deficiency. The code of conduct describes the rules for ethical behavior, and senior management should transmit a proper ethical message throughout the enterprise. Other incentives and temptations, however, can erode this overall control environment. Individuals may be tempted to engage in dishonest, illegal, or unethical acts if their enterprise gives them strong incentives or temptations to do so. For example, an enterprise may establish very high, unrealistic performance targets for sales or production quotas. If there are strong rewards for the achievement of these performance goals—or, worse, strong threats for missed targets—employees may be encouraged to engage in fraudulent or questionable practices to achieve those goals. A strong internal audit function is a major component of the COSO control environment. If internal audit finds that management is placing constraints on the audit function, the CAE should remind senior management of internal audit’s importance as part of the enterprise’s overall internal control structure and, more important, should communicate these concerns to the board of director’s audit committee. Commitment to Competence. An enterprise’s control environment can be seriously eroded if a significant number of positions are filled with persons lacking required job skills. An enterprise needs to specify the required competence levels for its various job tasks and to translate those requirements into necessary levels of knowledge and skill. By placing the proper people in appropriate jobs and giving adequate training when required, an enterprise is satisfying this important COSO control environment component. Board of Directors and Audit Committee. The control environment is very much influenced by the actions of an enterprise’s board of directors and its audit committee. An active and independent board is an essential component of the COSO control environment. By setting high-level policies and reviewing overall enterprise conduct, the board and its audit committee have the ultimate responsibility for setting this tone at the top. Management’s Philosophy and Operating Style. The philosophy and operating style of senior management has a considerable influence over an enterprise’s control environment. Some top-level managers frequently take significant enterprise-level risks E1C01 08/06/2010 14:5:3 12 Page 12 & SOx and the COSO Internal Controls Framework in their new business or product ventures while others are very cautious or conservative. Some managers seem to operate by the seat of their pants while others insist that everything must be properly approved and documented. Some may take very aggressive approaches in their interpretations of tax and financial reporting rules while others go by the book. These comments do not necessarily mean that one approach is always good and the other bad. These management philosophy and operational style considerations are all part of an enterprise’s control environment. Although no one set of styles and philosophies is best for all enterprises, these factors such as a strong organization structure and effective human resource policies are important when considering the other components of internal control in an enterprise. Organizational Structure. The organizational structure internal control component provides a framework for planning, executing, controlling, and monitoring activities to help achieve overall objectives. This control environment factor relates to how functions are managed and organized. Organizational structure is an important aspect of the enterprise’s control environment, but no one structure provides any preferred internal controls environment. An organizational structure is the manner or approach for individual work efforts to be both assigned and integrated for the achievement of overall goals. Every enterprise needs an effective plan of organization, and a weakness in organizational controls can have a pervasive effect throughout the total control environment. Despite clear lines of authority, however, enterprises sometimes have built-in inefficiencies that can become greater over time as they expand, causing control procedures to break down. Assignment of Authority and Responsibility. The assignment of authority and responsibility in the control environment is similar to the organizational structure component just discussed. An enterprise’s organizational structure defines the assignment and integration of the total work effort. The assignment of authority is essentially the way responsibilities are defined in terms of formal job descriptions and are structured in terms of enterprise organization charts. Although job assignments can never fully escape some overlapping or joint responsibilities, the more precisely these responsibilities can be stated, the better. The failure to clearly define authority and workplace responsibility often causes confusion and conflict between individual and group work efforts. Human Resources Policies and Practices. Human resources (HR) practices cover personnel hiring, orientation, training, evaluating, and counseling, promoting, compensating, and taking appropriate remedial actions. Although the enterprise HR function should have adequate published policies and guidance materials, its actual practices should send strong messages to employees regarding expected levels of internal controls compliance, ethical behavior, and competence. The higher-level employee who openly abuses or ignores an HR policy quickly sends a message to other levels in the enterprise. The message grows even louder when a lower-level employee is disciplined for violating that same policy while everyone looks the other way at the higher-level violator. E1C01 08/06/2010 14:5:3 Page 13 Importance of Effective Internal Controls and COSO & 13 Effective HR policies and procedures are a critical component in the overall control environment. Messages from the top of strong enterprise structures will accomplish little if the enterprise does not have strong HR policies and procedures in place. IT audit should always consider the HR element of the control environment when reviewing other parts of the internal control framework. Summary. Just as a strong foundation is necessary for a multistory building, the control environment provides the foundation for the other components of internal control. An enterprise that is building a strong internal control structure should give special attention to placing solid foundation bricks in this control environment foundation. Of course, IT auditors should keep these concepts, such as effective HR policies, in mind when assessing internal controls. The COSO internal control environment does not require just a series of ‘‘do the debits equal the credits?’’ types of accounting rules but strong overall enterprise-wide policies that are effective. Risk Assessment The next level above the control foundation on the COSO internal control framework is risk assessment. An enterprise’s ability to achieve its objectives can be at risk due to a variety of internal and external factors. Understanding and managing the risk environment are basic elements of the internal control foundation, and an enterprise should have a process in place to evaluate the potential risks that may impact attainment of its various objectives. This risk assessment component has its focus on internal controls within an enterprise and has a much narrower focus than the COSO ERM framework discussed in Chapter 4. COSO internal controls risk assessment should be a forward-looking process that is performed at all levels and for virtually all activities within the enterprise. COSO describes risk assessment as a three-step process: 1. Estimate the significance of the risk. 2. Assess the likelihood or frequency of the risk occurring. 3. Consider how the risk should be managed, and assess what actions must be taken. This COSO risk assessment process places the responsibility on management to assess whether a risk is significant and, if so, to take appropriate actions. COSO internal controls also emphasizes that risk analysis is not a theoretical process but often is critical to an entity’s economic and operational success. As part of its assessment of internal control, management should take steps to assess the risks that may impact the overall enterprise as well as the risks over various enterprise activities or entities. A variety of risks, caused by either internal or external sources, may affect the enterprise. The risk assessment element of the COSO internal controls Framework is an area where there has been much misunderstanding and confusion because of the similarly named COSO ERM framework discussed in Chapter 4. The risk assessment component of the COSO internal controls framework includes risk assessments for within an individual enterprise. The COSO ERM framework covers the entire entity and beyond. These are really two separate but related issues, and one is not a replacement for the other. E1C01 08/06/2010 14:5:3 14 Page 14 & SOx and the COSO Internal Controls Framework Control Activities The next layer up in the COSO internal control framework is called control activities. These are the processes and procedures that help ensure that actions identified to address risks are carried out. Control activities exist at all levels and, in many cases, may overlap one another. They are essential elements to building and then establishing effective enterprise internal controls. The COSO internal controls framework identifies a series of these activities that are generally classified as manual, IT, or management controls; they are also described in terms of whether they are preventive, corrective, or detective control activities. Although no one set of internal control definitions is correct for all situations, COSO internal controls recommends these control activities for an enterprise: & & & & & & Top-level reviews. Management and internal auditors, at various levels, should review the results of their performance, contrasting those results with budgets, competitive statistics, and other benchmark measurements. Management actions to follow up on the results of these top-level reviews and to take corrective action represent a key control activity. Direct functional or activity management. Managers at various levels should review operational reports from their control systems and take corrective action as appropriate. Many management systems have exception reports covering these control activities. For example, an IT security system should have a mechanism to report unauthorized access attempts, with a control activity to follow up on reported events and take appropriate corrective action. Some of these activities link closely with the information technology infrastructure library (ITIL) best practices discussed in Chapter 7. Information processing. IT systems often contain controls to check for compliance in certain areas and then report any internal control exceptions. Those exception items should receive corrective action by automated systems procedures, by operational personnel, or by management. Other control activities include controls over the development of new systems or over access to data and program files. Physical controls. An enterprise should have appropriate control over its physical assets, including fixtures, inventories, and negotiable securities. An active program of periodic physical inventories represents a often significant control activity here, and IT auditors can play a major role in monitoring compliance here. Performance indicators. Management should relate sets of data, both operational and financial, to one another and take appropriate analytical, investigative, or corrective actions. This process represents an important enterprise control activity that can also satisfy financial and operational reporting requirements. Segregation of duties. Duties should be divided or segregated among different people to reduce the risk of error or inappropriate actions. This basic internal control procedure should be on almost every IT auditor’s radar screen. The control activities highlighted here represent only a small number of the many control activities performed in the normal course of business but involve policies E1C01 08/06/2010 14:5:3 Page 15 Importance of Effective Internal Controls and COSO & 15 establishing what should be done and procedures to affect them. Even though control activities sometimes may be communicated only orally, they should be implemented thoughtfully, conscientiously, and consistently. This recognition and communication of control activities is a strong message for internal auditors reviewing such internal control activities. Even though an enterprise may have a published policy covering a given area, there should be established internal control procedures to support that policy. Procedures are of little use unless there is a sharp focus on the condition to which the policy is directed. All too often, an enterprise may establish an exception report as part of an automated system while that exception report receives little more than a cursory management review by its recipients. However, depending on the types of conditions reported, those exceptions should receive appropriate follow-up actions, which may vary depending on the size of the enterprise and the activity reported in the exception report. These control activities should be closely related to one another to identify risks from the COSO internal controls risk assessment component. Internal control is a process, and appropriate control activities should be installed to address identified risks. Control activities should not be installed just because they seem to be the right thing to do, even if there are no significant risks in the area where the control activity would be installed. Sometimes control activities may be in place that perhaps once served some control risk concern, although the concerns have largely gone away. A control activity should not be discarded because there has been no recent history of control violations, but management needs to reevaluate these relative risks periodically. All internal control activities should contribute to the overall control structure, and IT auditors should keep this concept in mind as they review internal controls and make recommendations. The COSO internal controls framework emphasizes that control procedures are needed over all significant IT systems: operational, financial, and compliance related. COSO internal controls breaks down information systems controls into the wellrecognized general and application controls. General controls apply to much of the information systems function to help ensure adequate control procedures over all applications. A physical security lock on the door to the IT server center is such a general control for all applications running within that facility. IT general controls are discussed in Chapter 6. Application controls, discussed in Chapter 10, are also important IT control areas for evaluating the overall adequacy of internal controls. The COSO internal controls framework document concludes with a discussion of the need to consider the impact of evolving technologies; it should always be considered when evaluating IT control activities. Due to the rapid introduction of new technologies, what is new today will soon be replaced by something else. Communications and Information The COSO internal controls framework in Exhibit 1.2 describes most internal control components as layers, one on top of another, starting with the control environment as the foundation. As another way to look at the framework, Exhibit 1.3 describes the COSO framework as a pyramid-shaped model with the information and communication component as a side element that spans across other components. As important portions E1C01 08/06/2010 14:5:3 16 Page 16 & SOx and the COSO Internal Controls Framework Internal Control Monitoring Control Activities Information and Communication Risk Assessment Compliance Internal Controls Internal Control Environment EXHIBIT 1.3 Operations Internal Controls Financial Reporting Internal Controls COSO Internal Controls Foundation Components of the internal control framework, information and communications are related but distinct components. Appropriate information, supported by IT systems, must be communicated up and down the enterprise in a manner and time frame that allows people to carry out their responsibilities. In addition to formal and informal communication systems, enterprises must have effective procedures in place to communicate with internal and external parties. As part of any evaluation of internal controls, there is a need to understand these information and communication flows in the enterprise. An enterprise needs information at all levels to achieve its operational, financial, and compliance objectives. For example, the enterprise needs information to prepare financial reports that are communicated to outside investors as well as internal cost and external market preference information to make correct marketing decisions. This information must flow both from the top levels of the enterprise on down to lower levels as well from lower levels back to upper levels. COSO internal control takes a broad approach to the concept of information systems, recognizing that they can be manual, automated, or even conceptual. Any of these information systems can be either formal or informal. Regular conversations with customers or suppliers can be highly important sources of information and are an informal type of an information system. The effective enterprise should have information systems in place to listen to customer requests and/ or complaints and to forward that customer-initiated information to appropriate personnel. COSO internal controls also emphasize the importance of keeping information and supporting systems consistent with overall enterprise needs. Information systems adapt to support changes on many levels. IT auditors, for example, often encounter cases where an IT application was implemented years earlier to support different needs. Although its controls may have been good, the system may not support the enterprise’s current needs. COSO internal controls take a broad view of these types of systems and point to the need to understand both manual processes and automated technologies. E1C01 08/06/2010 14:5:3 Page 17 Importance of Effective Internal Controls and COSO & 17 Monitoring The pyramid view of COSO internal controls shows the monitoring component as the capstone, upper level of the COSO internal control components. Although internal control systems will work effectively with proper support from management, control procedures and both information and communication linkages must be in place to monitor all of these other activities. Monitoring has long been the role of IT auditors, who perform reviews to assess compliance with established procedures; however, COSO now takes a broader view of this control procedures monitoring. COSO internal control recognizes that control procedures and other systems change over time. What appeared to be effective when it was first installed may not be that effective in the future due to changing conditions, new procedures, or other factors. A monitoring process should be in place to assess the effectiveness of established internal control components and to take corrective action when appropriate. This internal control component cannot be relegated just to the internal audit while management seems to remain oblivious to other potential control problems. An enterprise needs to establish a variety of monitoring activities to measure the effectiveness of their established internal controls as well as through separate evaluations of ongoing internal control activities to monitor performance and take corrective action when required. Many routine business functions can be characterized as monitoring activities, and COSO internal control gives examples of this important component of internal control: & & & & Operating management normal functions. Normal management reviews over operations and financial reports are an important ongoing monitoring activity, but special attention should be given to reported exceptions and internal control deviations. Internal control is enhanced if reports are reviewed on a regular basis and corrective action initiated for any reported exceptions. Communications from external parties. External communication monitors, such as a customer complaint telephone number, are important, and the enterprise needs to monitor closely the messages from these calls and initiate corrective actions based on the calls when appropriate. Enterprise structure and supervisory activities. Senior management should always review summary reports and take corrective actions, but the first level of supervision often plays an even more significant role in monitoring. Direct supervision of clerical activities, for example, should routinely review and correct lower-level errors and assure improved clerical employee performance. This is also an area in which the importance of an adequate separation of duties is important, and dividing duties between employees allows them to serve as a monitoring check on one another. Physical inventories and asset reconciliation. Periodic physical inventories, whether of storeroom stock, negotiable securities, or IT assets, are an important monitoring activity. An annual inventory in a retail store, for example, may indicate a significant merchandise loss. A possible reason for this loss could be theft, pointing to the need for better security controls. E1C01 08/06/2010 14:5:4 18 Page 18 & SOx and the COSO Internal Controls Framework These are just a few examples of COSO internal controls monitoring activities. These types of procedures are often in place in many enterprises but are not thought of as ongoing monitoring activities. Any function or process that reviews enterprise activities on a regular basis and then suggests potential corrective actions can be thought of as a monitoring activity. The COSO internal control framework points out the importance of ongoing monitoring activities and also suggests that ’’it may be useful to take a fresh look from time to time’’ at the effectiveness of internal controls through separate evaluations. The frequency and nature of these separate reviews greatly depend on the nature of the enterprise and the significance of the risks it must control. Management may want to initiate periodic evaluations of its entire internal controls, but most evaluations should be initiated to assess specific control areas. Often these reviews are initiated when there has been an acquisition, a change in business, or some other significant activity. COSO also emphasizes that these evaluations may be performed by direct line management through self-assessment reviews. IT audit does not have to perform these reviews unless requested, and considerable time may pass before internal audit may schedule a self-assessment type of review in areas of operations. However, responsible management should consider scheduling and performing self-assessments on a regular basis. This type of internally generated review can point out potential control problems and cause operating management to take corrective action. Because these self-assessment reviews typically are not as comprehensive as normal internal audits, follow-up reviews should be launched if potentially significant problems are encountered through limited self-assessment reviews. Internal Control Evaluation Process. The COSO internal controls guidance materials outline an evaluation process for reviewing internal controls. The evaluator should first develop an understanding of the system design, test key controls, and then develop conclusions based on the test results. This is really the IT audit process. COSO internal control also mentions benchmarking as an alternative approach. Benchmarking is the process of comparing an enterprise’s processes and control procedures with those of peer enterprises. Comparisons are made with similar enterprises or against published industry statistics. This approach is convenient for some measures but filled with dangers for others. For example, it is fairly easy to benchmark the size, staffing levels, and average compensations of a sales function against comparable enterprises in the same general industry; however, the evaluator may encounter difficulties in trying to compare other factors due to the many small differences that make all enterprises unique. Evaluation Action Plans. COSO internal control recognizes that many highly effective procedures are informal and undocumented. Many of these undocumented controls, however, can be tested and evaluated in the same manner as documented ones. An appropriate level of documentation makes any evaluation of internal control more efficient and facilitates employees’ understanding of how the process works, but such documentation is not always essential. IT auditors reviewing an enterprise’s internal financial controls systems always request to see systems documentation as part of their review work. If an existing process is informal, undocumented, but recognized as E1C01 08/06/2010 14:5:4 Page 19 Importance of Effective Internal Controls and COSO & 19 effective, the review team will need to prepare its own action documentation to explain how the process works and the nature of its internal controls. Reporting Internal Control Deficiencies. When internal control deficiencies are identified—whether through processes in the internal control system itself, monitoring activities, or other external events—they should be reported to appropriate levels of enterprise management. The key question for the IT audit evaluator is to determine what should be reported, given the many details that may be encountered, and to whom the reports should be directed. COSO internal control states that ‘‘all internal control deficiencies that can affect the entity’s attaining its objectives should be reported to those who can take necessary action.’’ This COSO internal control statement makes sense but often is difficult to implement. The modern enterprise, no matter how well organized, is often guilty of a variety of internal control errors or omissions. COSO internal control suggests that all of these should be identified and reported and that even seemingly minor of errors should be investigated in order to understand if they were caused by overall control deficiencies. The COSO internal controls report uses the example of an employee’s taking a few dollars from the petty cash fund. Although this could be viewed as a minor matter due to the small size of the theft, it still should be viewed as an overall control breakdown on several levels. The monetary amount may not be significant, but COSO internal control urges that the matter be investigated rather than ignored, since ‘‘such apparent condoning of the personal use of the entity’s money might send an unintended message to employees.’’ Prior to SOx, external auditors regularly applied the concept of materiality when performing reviews and decided that some errors and irregularities were so small that they were not material to the external auditor’s overall conclusion. In the first years of SOx compliance reviews with the original Auditing Standard No. 2 (AS 2) guidelines, the message from many external auditors was that materiality issues should not be considered—an error is an error. This approach caused many managers to wonder why their external auditors were raising issues on what they felt were minor matters. With the AS 5 rules discussed later in the chapter, materiality and relative risk now must be considered when evaluating the efficiency and effectiveness of internal controls. The COSO internal controls guidance concludes by discussing to whom to report internal control deficiencies in the enterprise. In one paragraph, COSO internal control provides guidance that is useful for evaluations: Findings on internal control deficiencies usually should be reported not only to the individual responsible for the function or activity involved, who is in the position to take corrective action, but also to at least one level of management above the directly responsible person. This process enables that individual to provide needed support or oversight for taking corrective action, and to communicate with others in the enterprise whose activities may be affected. Where findings cut across organizational boundaries, the reporting should cross over as well and be directed to a sufficiently high level to ensure appropriate action. The enterprise should also develop reporting procedures such that all internal control deficiencies encountered through IT audit reviews of ongoing operations are reported to appropriate levels of the enterprise. Management reporting and monitoring E1C01 08/06/2010 14:5:4 20 Page 20 & SOx and the COSO Internal Controls Framework is a highly important aspect of internal control. Internal audit has a lead role in that process through IT audit reviews and should be aware of the need for other monitoring processes when reviewing and evaluating internal controls. Other Dimensions of the COSO Internal Controls Framework We sometimes forget that the COSO internal controls framework should be reviewed and evaluated as the three-dimensional model, shown in Exhibit 1.2. In addition to the front-facing dimension of that model covering control activities, the right side covers entities or activities, and the top side or dimension of the framework cube covers the three dimensions of all internal controls: 1. Effectiveness and efficiency of operations 2. Compliance with applicable laws and regulations 3. Reliability of financial reporting Each of the control areas just discussed—from the control environment to monitoring—should also be considered with respect to those other two dimensions. Regarding the right side dimension, internal controls should be installed and evaluated across all units in the enterprise. This does not mean that a control activity, such as an expense approval process, must be identical in all units, such as at corporate headquarters or a sales office in a remote geographic location. However, there should be a consistent set of control processes throughout the enterprise with consideration given for the relative risks and scopes of operations. Internal controls should be consistent, but they should be applied appropriately in individual operating units. The top dimension of the COSO internal controls framework is even more significant. It says that internal control activities should be installed in all enterprise operating units with respect to the three factors of internal controls: reliability of operations, regulatory compliance, and financial reporting effectiveness. Looking at internal controls from this three-dimensional viewpoint, there may always be some variations but the framework should be under a basic and consistent internal controls. Consider the example of a subsidiary facility in a central Asian nation, far away from its U.S. headquarters. Country expense approval procedures may be subject to local laws, and other processes may be somewhat different due to communication distances or differences in local IT systems. However, those internal controls still should be implemented in a manner that ensures reliability in financial reporting as results are reported to corporate headquarters. All internal control considerations must be considered in terms of the COSO threedimensional cube. That is, the control must be considered in terms of where it fits in the overall enterprise and its relationship to the three control objective areas just discussed. This concept provides IT auditors with a powerful way of looking at internal controls from a total perspective. The COSO internal controls framework continues to be an important standard and set of guidance materials for measuring and evaluating internal controls. The COSO internal control framework is becoming the worldwide standard for building and developing effective internal controls. It is a continuous process in each of its three dimensions. On the front-facing side of the model, the monitoring component E1C01 08/06/2010 14:5:4 Page 21 COSO Internal Control Systems Monitoring Guidance & 21 on top is of little value unless internal control processes are in place all the way down to the internal control environment foundation. Similarly, effective internal controls must be installed in all levels of organizational units, and each of those controls must be sensitive to the three top-facing internal control elements. COSO INTERNAL CONTROL SYSTEMS MONITORING GUIDANCE Extensive guidance materials on the COSO internal controls framework have been available with sources ranging from AICPA auditing standards, various ISACA materials, and our own additional guidance materials.6 However, many professionals had been seeking more specific guidance on how to implement COSO internal controls in business operations. A three-volume set of internal controls guidance materials was published by COSO in 2009.7 These volumes emphasize the importance of establishing processes to monitor the effectiveness and efficiency of established internal controls. Our previous description of the COSO internal controls framework, as shown in Exhibit 1.1, suggests that enterprises implement internal control monitoring processes in a manner similar to the way in which a manufacturing organization monitors the continued effectiveness and efficiency of its manufacturing procedures. The materials suggest that enterprises establish a four-phase or -stage monitoring process, as shown in Exhibit 1.4, where 4. Develop and implement cost-effective procedures to evaluate that persuasive information. 4. Implement Monitoring 3. Identify Information 3. Identify information that will persuasively indicate whether the internal control system is operating effectively. EXHIBIT 1.4 1. Understand and prioritize risks to enterprise objectives. 1. Prioritize Risks 2. Identify Controls 2. Identify key controls across the internal control system that address those prioritized risks. Monitoring Design and Implementation Process E1C01 08/06/2010 14:5:4 22 Page 22 & SOx and the COSO Internal Controls Framework an enterprise should first prioritize and understand the risks to its organizational objectives, then identify the controls that address those prioritized risks. IT auditors play a key role in the third step: identifying information that will persuasively indicate that the internal control system is operating effectively. The model calls for implementing cost-effective procedures to evaluate the information gathered through monitoring processes. The internal controls evaluation process is also very similar to the continuous assurance auditing procedures discussed in Chapter 14. SARBANES-OXLEY ACT The Sarbanes-Oxley Act (We will refer to the Act as SOx) is a U.S. law enacted in 2002 as a response to a series of accounting misdeeds and financial failures with an objective to improve public company financial reporting, audit, and enterprise governance processes. It first had a major impact on businesses in the United States and now is recognized worldwide. Although SOx’s auditing and internal control rules have directly changed many external auditor practices, the act has also had a major impact on IT auditors. A general understanding of SOx, with an emphasis on its Section 404 internal accounting control rules, is a key knowledge requirement for all IT auditors. Here we provide a high-level overview of SOx today with an emphasis on its Section 404, the rules that are most important to IT auditors. We summarize SOx requirements for reviews of internal accounting controls—a process important for IT auditors. In addition, we summarize the relatively new external auditing standards called Auditing Standard No. 5 (AS 5), a set of risk-based auditing approaches that also emphasize the importance of internal audit’s work in performing financial reporting internal control reviews. IT auditors should have general knowledge and understanding of SOx internal control rules.8 Key Elements The official name of SOx is the Public Accounting Reform and Investor Protection Act. It became law in August 2002 with most of the final detailed rules and regulations released by the end of 2003. Business professionals refer to it as the Sarbanes-Oxley Act, from the names of its principal congressional sponsors, which is shortened to SOx, SOX, or Sarbox, among many other variations. SOx introduced a series of totally changed processes for external auditing and gave new governance responsibilities to senior executives and board members. SOx also established the Public Company Accounting Oversight Board (PCAOB), a rule-setting authority under the SEC that issues financial auditing standards and monitors external auditor governance. As happens with all financial and securities-related federal laws, an extensive set of specific regulations and administrative rules has been developed by the SEC based on the SOx legislation. U.S. federal laws are organized and issued as separate sections of legislation called titles, with numbered sections and subsections under each. Much of the SOx legislation contains rules that are not that significant for most internal auditors and business E1C01 08/06/2010 14:5:4 Page 23 Sarbanes-Oxley Act Title Subject & 23 Rule or Requirement 101 Establishment of PCAOB Overall rules for the establishment of the PCAOB, including its membership requirements. 104 Accounting Firm Inspections Schedule for PCAOB inspections of registered public accounting firms. 108 Auditing Standards The PCAOB will accept current but will issue its own new auditing standards. 201 Out-of-Scope Practices Outlines prohibited accounting firm practices, such as internal audit outsourcing, bookkeeping, and financial systems design. 203 Audit Partner Rotations The audit partner and the reviewing partner must rotate off an assignment every five years. 301 Audit Committee Independence All audit committee members must be independent directors. 302 Corporate Responsibility for Financial Reports The CEO and CFO must personally certify their periodic financial reports. 305 Officer and Director Bars If compensation is received as part of fraudulent/illegal accounting, the benefiting officers or director is required to personally reimburse funds received. 404 Internal Control Reports Management is responsible for an annual assessment of internal controls. 407 Financial Expert One audit committee director must be a designated financial expert. 408 Enhanced Review of Financial Disclosures The SEC may schedule extended reviews of reported information based on certain specified factors. 409 Real-Time Disclosure Financial reports must be distributed in a rapid and current manner. 1105 Officer or Director Prohibitions EXHIBIT 1.5 The SEC may prohibit an officer or director from serving in another public company if guilty of a violation. Sarbanes-Oxley Act Key Provisions Summary professionals. For example, Section 602(d) of Title I states that the SEC ‘‘shall establish’’ minimum professional conduct standards or rules for SEC practicing attorneys. Although this rule perhaps is good to know, it does not have any IT audit impact. Exhibit 1.5 summarizes the major titles of SOx, although our focus is on Titles I and IV. Our intent is not to describe all of these sections or to reproduce the full text of this legislation—it can be found on the Web9—but to highlight portions of the law that are more significant to internal audit and business professionals. Of interest, even though internal control processes very much rely on both external and internal auditors, the original SOx legislation makes almost no direct references to the important roles and responsibilities of internal auditors. The importance of internal audit’s role in SOx internal control reviews was highlighted subsequently in the AS 5 rules, released in mid2007 and discussed later in this chapter. Our emphasis throughout is on the role of internal audit in today’s SOx environment. E1C01 08/06/2010 14:5:4 24 Page 24 & SOx and the COSO Internal Controls Framework Title I: Public Company Accounting Oversight Board SOx introduced significant new rules for external auditors. Prior to SOx, the AICPA had guidance-setting responsibility for all external auditors and their public accounting firms through its overall responsibility for the Certified Public Accountant (CPA) certification. Although state Boards of Accountancy actually license CPAs, the AICPA previously had overall responsibility for the profession. External audit standards were set by the AICPA’s Auditing Standards Board (ASB). Although basic standards—called generally accepted auditing standards (GAAS)—have been in place over the years, newer auditing standards were released as numbered Statements of Auditing Standards (SASs). Much of GAAS was just good auditing practices—for example, accounting transactions must be backed by appropriate documentation—while the SASs covered specific areas requiring better definition. SAS No. 99, for example, covered the consideration of fraud in a financial statement audit. The AICPA’s code of professional conduct required CPAs to follow and comply with all applicable auditing standards. The AICPA’s GAAS and its numbered SAS standards were accepted by the SEC, and these auditing rules defined external auditing standards and the tests necessary for an audited financial statement. However, the accounting scandals that led to the passage of SOx signaled that the AICPA-led process of establishing auditing standards was ‘‘broken’’; SOx took this audit standards-setting process away from the AICPA, which was dominated by major public accounting firms, and created the PCAOB, a nonfederal, nonprofit corporation with the responsibility to oversee all audits of corporations subject to the SEC. The PCAOB does not replace the AICPA but assumes responsibility for the external auditing practices for AICPA members. The AICPA continues to administer the CPA examination, with its certificates awarded on a state-by-state basis, and sets auditing standards for U.S. private, non-SEC organizations. SOx Title I defines PCAOB auditing practices for external auditors; other audit process and corporate governance rules have changed how internal auditors coordinate their work with external auditors. Although SOx Title I contains many new rules, perhaps the three most important to IT auditors are that the PCAOB now has both major responsibility for public accounting firms, sets their external auditing standards, and sets audit standards rules such as workpaper retention. The next paragraphs briefly describe these SOx Title I external audit process rules. & & PCAOB administration and public accounting firm registration. The PCAOB is administered through an SEC-appointed board with required membership that is not dominated by CPA and public accounting firm interests. The PCAOB is responsible for overseeing and regulating all public accounting firms that practice before the SEC and for establishing auditing standards. Auditing, quality control, and independence standards. The PCAOB has the authority to establish auditing and related attestation standards, quality control standards, and ethics standards for registered public accounting firms. SOx recognizes previously issued AICPA auditing standards and has issued a limited number of new standards to date, such as AS 5 for the review and evaluation of internal controls. SOx rules further specify that an external auditor’s evaluation must E1C01 08/06/2010 14:5:4 Page 25 Sarbanes-Oxley Act & & & 25 contain a description of material weaknesses as well as any material noncompliance matters found. External auditors are required to update the effectiveness of internal controls, and an absence of this documentation should be considered a weakness of internal controls. Audit workpapers retention. PCAOB standard AS 3, Audit Documentation, mandates that audit workpapers and other supporting materials should be maintained for a period of not less than seven years. This requirement is in response to an infamous event just prior to the fall of the corporation that prompted SOx, Enron, and its auditor, Arthur Andersen. Enron was still in operation but was under some financial pressures when the SEC announced that it was going to conduct an onsite investigation. Enron’s external auditors, Arthur Andersen, used an internal firm policy to justify destruction of all but the most current of their Enron audit documentation. This was a factor that led to the establishment of this SOx rule. Scope of internal control testing. PCAOB rules require external auditors to describe the scope of both their testing processes and their test findings. Prior to SOx, external auditors sometimes used internal firm policies to justify the smallest test sizes, and they frequently tested only a very small number of items despite being faced with very large test populations. If no problems were found, they expressed an opinion for the entire population based on the results of a very limited sample. External auditors now must pay greater attention to the scope and reasonableness of their testing procedures, and the supporting documentation must clearly describe the scope and extent of testing activities. Title IV: Enhanced Financial Disclosures and Section 404 SOx Title IV is designed to correct some financial reporting disclosure problems, to tighten up conflict of interest rules for corporate officers and directors, to mandate a management assessment of internal controls, to require senior officer codes of conduct, and other matters. There is a lot of material here, but the most significant nugget for internal auditors is Section 404, Management’s Assessment of Internal Controls. SOx requires that all annual 10K reports must contain an internal controls report stating management’s responsibility for establishing and maintaining an adequate system of internal controls as well as management’s assessment, as of the fiscal year ending date, of the effectiveness of those installed internal control procedures. These are what have popularly become known as the Section 404 rules. Internal and IT audit, outside consultants, and even the management team—but not the external auditors—have the responsibility to review and assess the effectiveness of their internal controls, and external auditors are then to attest to the sufficiency of the internal control reviews built and controlled by management. Section 404 reviews are supported by the AS 5 standards discussed later in this section and are particularly important to internal auditors because the rules specify that external auditors may elect to use the work of internal auditors in their internal controls reviews. All IT auditors should have a basic understanding of SOx Section 404, whether they are acting as consultants in helping to build these internal accounting controls or acting in support of enterprise external auditors by auditing those internal accounting controls. E1C01 08/06/2010 14:5:4 26 Page 26 & SOx and the COSO Internal Controls Framework SOx Section 404 rules state that an enterprise is responsible for reviewing, documenting, and testing its own internal accounting controls, with those review results then passed on to the enterprise’s external auditors, who are charged with reviewing and attesting to that work as part of their review of the reported financial statements. When SOx first became the law, Section 404 reviews were a major pain point for many enterprises because external auditors were required to follow a very detailed set of AS 2 financial accounting audit procedures that did not give any allowance for small errors or omissions. Section 404 auditing rules have changed with the release of AS 5 in 2007; a more risk-based audit approach also allows external auditors to better use the work of internal auditors in their assessments. Section 404 Internal Controls Assessments Management always has had the overall responsibility for designing and implementing internal controls over an enterprise’s operations. Although the standards for what constituted good internal controls were not always that well defined in the past, they have remained a fundamental management concept. SOx Section 404 requires an annual internal control report, with these information elements, as part of an enterprise’s SEC-mandated Form 10K annual report: & & A formal management statement acknowledging the enterprise’s responsibility for establishing and maintaining an adequate internal control structure and procedures for financial reporting An assessment, as of the end of the most recent fiscal year, of the effectiveness of the enterprise’s internal control structure and procedures for financial reporting In addition, the external audit firm that issued the supporting audit report is required to review and report on management’s assessment of its internal financial controls. Simply put, management is required to report on the quality of its internal controls, and its public accounting firm must audit or attest to that management developed an internal controls report in addition to the normal financial statement audit. Management has always been responsible for preparing periodic financial reports, and external auditors then audited those financial numbers and certified that they were fairly stated. With SOx Section 404, management is now responsible for documenting, testing and reporting on their internal financial controls effectiveness. External auditors then review the supporting materials leading up to that internal financial controls report to assert that the report is an accurate description of the internal control environment. To the non–financial statement auditor and for some IT auditors, this might appear to be an obscure or almost trivial requirement. Even some IT auditors who perform operational audits primarily may wonder about the nuances in this process. However, this process follows a basic internal control on the importance of maintaining a separation of duties where the person who develops transactions should not be the person who approves them. Under Section 404 procedures, the enterprise builds and documents its own internal control processes, then an independent party such, as internal audit, reviews and tests those internal controls. Finally, the external auditors E1C01 08/06/2010 14:5:4 Page 27 Sarbanes-Oxley Act & 27 review and attest to the adequacy of this process. Their financial audit procedures will be based on these internal controls. This Section 404 process improves things from preSOx days when external auditors frequently built, documented, and then audited their own internal controls—a separation-of-duties shortcoming. Identifying Key Processes to Launch a Section 404 Compliance Review Whether based on IT systems or even manual procedures, the basic processes for every enterprise should normally be considered in terms of some basic accounting cycles: & & & & & & & Revenue cycle. Processes dealing with sales or other enterprise revenue. Direct expenditures cycle. Expenditures for material or direct production costs. Indirect expenditures cycle. Operating costs that cannot be tied directly to production activities but are necessary for overall business operations. Payroll cycle. Covers all personnel compensation. Inventory cycle. Although inventory eventually will be applied as direct production expenditures, time-based processes are needed for holding inventory until applied to production. Fixed assets cycle. Property and equipment require separate accounting processes, such as periodic depreciation accounting over time. General controls IT cycle. This set of processes covers IT controls that are general or applicable to all IT operations and are discussed in Chapter 6. We will discuss some of these processes in Chapter 5 in the context of planning and performing effective IT audits. The identification of these key enterprise processes is an initial Section 404 compliance step, and an enterprise should document, understand, and test all of these ‘‘key processes.’’ IT audit often can be a major help here, as it may have already reviewed the prime systems and supporting IT processes through its annual audit reviews and documentation. Internal Audit’s Role Even though SOx does not give specific responsibilities to internal audit, IT auditors are an important enterprise resource for the completion of Section 404 internal controls assessments. Under SOx, a separate and independent function within the enterprise— often internal or IT audit—reviews and documents the internal controls covering key processes, identifies key control points, and then tests those identified controls. External audit then reviews that work and attests to its adequacy. For many enterprises, IT audit can be a key resource for performing these internal controls reviews for technologybased processes. When SOx first became the law, internal audit functions often distanced themselves from Section 404 reviews because of potential internal auditor independence standards. The IIA Standards, as discussed in Chapter 3, now allow internal auditors to act as consultants to help document and establish effective internal control processes. The CAE, financial management, and the audit committee should work with the enterprise’s external auditors to define responsibilities for their Section 404 internal E1C01 08/06/2010 14:5:4 28 Page 28 & 1. 2. 3. 4. 5. 6. 7. 8. 9. 10. 11. 12. 13. 14. 15. SOx and the COSO Internal Controls Framework Determine status of review: Is this the first round of Section 404 reviews for the entity or a subsequent-year follow-up? If a new review, follow the work steps to understand, document, and test key processes. Otherwise, plan for a subsequent-period review. Review the detailed documentation covering prior 404 reviews, including process flow charts, internal control gaps identified and remediated, as well as overall project planning documentation for prior review. Review any recently published PCAOB rules covering Section 404 reviews and related auditing changes, and adjust review procedures to reflect those changes. Meet with the external audit firm responsible for the current Section 404 attestations and determine if there are any changes in documentation and testing philosophy, with an emphasis on AS5 rules, from that prior review. Consider any organization changes since the past review, including acquisitions or major reorganizations, and modify review coverage, if necessary. Through meetings with senior and IT management, identify if new systems or processes have been installed over the past period and if those new changes have been reflected in updated documentation. Review any internal control weaknesses identified in the past review, and assess whether internal control corrections reported as installed appear to be working. Assess the status of existing Section 404 documentation, and determine the extent of new documentation preparation necessary. Assuming the prior Section 404 review was done by internal audit, determine that appropriate knowledgeable, trained resources are available to perform the upcoming review. Interview all parties involved in the prior Section 404 review exercise to assess any lessons learned and develop plans for corrective actions in the upcoming review. Based on discussions with external auditors and senior management, determine scope materiality parameters for the upcoming review. Determine that the software, if any, used to document prior review is still current, and make any changes necessary to have adequate tools in place to perform the upcoming review. Prepare a detailed project plan for the upcoming Section 404 review, with consideration given to coordination of review activities at business entity units and external auditors. Submit plan for approval by senior management. EXHIBIT 1.6 Planning Considerations for a Section 404 Internal Controls Review control reviews. These reviews are performed on an annual process, with documentation prepared and tested in the first year updated and retested in future periods. All parties should develop a cost-effective approach to achieve these SOx requirements and assess their IT applications and controls. IT audit–led SOx Section 404 reviews should be planned and conducted like any new IT audit project as discussed in Chapter 5 on planning and developing effective IT audits. Exhibit 1.6 outlines some planning considerations for an IT audit–led Section 404 internal controls review. Internal audit can play a major role in helping senior management establish Section 404 compliance. Based on the internal audit standards discussed in Chapter 3, internal audit should recommend internal control improvements as the new processes are being developed, or internal audit can act as consultants for installing those new internal control processes. E1C01 08/06/2010 14:5:4 Page 29 Sarbanes-Oxley Act & 29 AS 5 Rules and Internal Audit Shortly after SOx became law in the United States, the PCAOB released its AS 2 guidance, which called for external auditors to take very conservative and detailed approaches on their audits of financial statements. AS 2 mandated a ‘‘look at everything’’ detailed audit approach, and enterprise external audit bills became much more expensive in those first SOx years. However, there were frequent complaints by industry leaders and others with a general consensus that AS 2 needed some revisions. The SEC and the PCAOB agreed to revise AS 2, and AS 5 was issued in late May 2007. AS 5 is a set of standards for external auditors who review and certify published financial statements. The new rules are also important for internal auditors. AS 5 introduces risk-based rules with an emphasis on the effectiveness of internal controls, oriented to enterprise facts and circumstances. In addition, AS 5 calls for external auditors to consider including reviews of appropriate internal audit reports in their financial statement audit reviews. It allows external auditors to place more emphasis on management’s ability to establish and document key internal controls. AS 5 rules are particularly important for IT auditors because external auditors can rely on the work of internal auditors in their Section 404 assessments. AS 5 has three broad objectives: 1. Focus internal control audits on the most important matters. AS 5 calls on external auditors to focus their reviews on areas that present the greatest risk that an internal control will fail to prevent or detect a material misstatement in financial statements. This approach calls for external auditors to focus on identifying material weaknesses in internal control in their audits, before material misstatements of financial statements arise. AS 5 also emphasizes the importance of auditing higherrisk areas, such as the financial statement period-end close process and controls designed to prevent fraud by management. At the same time, the new standard provides external auditors a range of alternatives for addressing lower-risk areas, such as by more clearly demonstrating how to calibrate the nature, timing, and extent of testing based on risk, as well as how to incorporate knowledge accumulated in previous years’ audits into the auditors’ assessment of risk. Also very important to internal auditors, AS 5 allows external auditors to use the work performed by an enterprise’s internal auditors, when appropriate. 2. Eliminate audit procedures that are unnecessary to achieve their intended benefits. AS 5 does not include the previous AS 2 standard’s detailed requirements to evaluate management’s own evaluation process and clarifies that an internal control audit does not require an opinion on the adequacy of management’s processes. For example, AS 5 focuses on the multilocation dimensions of risk in an enterprise and reduces requirements that external auditors should test a ‘‘large portion’’ of an enterprise’s operations or financial positions. This should allow a reduction in financial audit work. 3. Make the financial audit clearly scalable to fit the size and the complexity of any enterprise. In order to provide guidance for audits of smaller, less complex companies, AS 5 calls for tailoring internal control audits to fit the size and E1C01 08/06/2010 14:5:4 30 Page 30 & SOx and the COSO Internal Controls Framework complexity of the enterprise being audited. The standard has guidance on how to apply AS 5 to smaller, less complex enterprises as well as the units of larger enterprises. Following AS 5, external auditors may consider using the work of others to help perform their SOx financial statement internal control audits. Although it was not as well defined under previous AS 2 rules, AS 5 now explicitly states that an external auditor may use the work performed by, or receive direct assistance from, internal auditors, other company personnel, or third parties working under the direction of management or the audit committee, to provide evidence about the effectiveness of financial reporting internal controls. This is a major change for internal auditors. Of course, external auditors are signing off on or attesting to the audit results, and they must assess the competence and objectivity of the persons whose work they plan to use. The higher the degree of competence and objectivity of others, the greater use an auditor may use their work. In particular, AS 5 calls for an assessment of the competence and objectivity of internal auditors. Competence means the attainment and maintenance of a level of understanding and knowledge that enables persons to perform the tasks assigned to them, and objectivity means the ability to perform those tasks impartially and with intellectual honesty. To assess competence, an external auditor should evaluate the qualifications and ability of the internal auditors or others to perform the work the external auditor plans to use. To assess objectivity, AS5 calls for an external auditor evaluation of whether factors are present that either inhibit or promote a person’s ability to perform with the necessary degree of objectivity the work the auditor plans to use. AS 5 goes on to state that external auditors should not use the work of persons who have ‘‘a low degree of objectivity, regardless of their level of competence,’’ and also should not use the work of persons who have a low level of competence regardless of their degree of objectivity. Personnel whose core function is to serve as a testing or compliance authority at an enterprise, such as internal and IT auditors, normally are expected to have greater competence and objectivity in performing the type of work that will be useful to the external auditor. This may be an area where the CAE, as well as the audit committee and senior management, may want to challenge external auditors if they see no role for internal audit in this financial statement audit planning process. Although AS 5 talks about internal auditors in an almost generic fashion, the role of the professional IIA member IT auditor is important here. Based on the IIA’s International Professional Practice of Internal Auditing standards, as summarized in Chapter 3, an IT auditor can be expected to have the competence and objectivity necessary for help in supporting an external auditor’s review of Section 404 internal controls. Although other persons, such as outside consultants, can be used to assist external auditors in their financial statement internal control reviews, IT auditors should have a major role here in assisting with Section 404 and AS 5 audit compliance. Internal audit’s ongoing role here should be viewed with a level of caution. We have discussed how IT auditors often are excellent resources to identify, document, and test key Section 404 internal control processes. They could do this in a support role for the external auditor’s attestation reviews. However, pure separation-of-duties independence rules say E1C01 08/06/2010 14:5:4 Page 31 Notes & 31 that they cannot perform these reviews, as internal auditors, within the enterprise and then act as a third-party helpmate for the external auditors to help attest to that same work. This conflict of duties should be clearly understood by all parties, and internal auditors and management should exercise care to prevent it. WRAPPING IT UP: COSO INTERNAL CONTROLS AND SOx This chapter has introduced two important concepts for IT auditors: the COSO internal controls framework and SOx internal controls standards. IT auditors work in a variety of enterprise environments, but today they will almost always encounter COSO internal control framework rules and SOx internal control review requirements under AS 5. Although this chapter provides just a summary description of each of these standards, and many auditors will require a greater understanding, all IT auditors should have a general knowledge and understanding of the COSO internal controls framework and SOx rules for understanding internal controls. These are worldwide standards requirements for today’s effective IT auditor. NOTES 1. More information on the total roles and responsibilities of internal audit in today’s enterprise can be found in Robert Moeller, Brink’s Modern Internal Auditing, 7th ed. (Hoboken, NJ: John Wiley & Sons, 2009). 2. Statement on Auditing Standards No. 1, Codification of Auditing Standards and Procedures, AICPA, Professional Standards. 3. National Commission on Fraudulent Financial Reporting, Report of the National Commission on Fraudulent Financial Reporting (1987). 4. Internal Control—Integrated Framework, www.coso.org/publications.htm Note: This reference is for the COSO internal controls report, which can be ordered through the AICPA at www.cpa2biz.com. 5. AICPA-published COSO internal control standards are described in the Statement on Auditing Standards (SAS) numbers 103, 105, 106, 107, 109, 110, and 112. 6. See Robert Moeller, Sarbanes-Oxley Internal Controls: Effective Auditing with AS5, CobiT, and ITIL (Hoboken, NJ: John Wiley & Sons, 2008). 7. COSO, Guidance on Monitoring Internal Control Systems (2009). www.coso.org/documents/COSO_Guidance_On_Monitoring_Intro_online1.pdf. 8. Here we are presenting only a high-level summary of SOx requirements. See Moeller, Sarbanes-Oxley Internal Controls, for much more information. 9. As a public document, the text of the law can be found in many Web locations. One source is http://fl1.findlaw.com/news.findlaw.com/hdocs/docs/gwbush/sarbanesoxley 072302.pdf. E1C02 09/16/2010 13:15:37 Page 32 2 CHAPTER TWO Using CobiT to Perform IT Audits T H E C O M M I T T E E O F SP O N S O R I N G Organizations (COSO) internal controls framework, as introduced and discussed in Chapter 1, has become the standard tool for measuring and evaluating internal accounting controls for a wide span of systems and processes since the 1990s and under the Sarbanes-Oxley Act (SOx). However, some professionals, and information technology (IT) auditors in particular, have expressed concerns about using the COSO internal controls framework in today’s IT-oriented world. The concern is that the published COSO internal controls guidance just does not give enough emphasis to IT tools and processes. For example, the published COSO internal controls guidance materials (see Chapter 3) look at IT application internal controls only at a very high level, when there appears to be a need for more IT-specific internal control guidance. A more IT-oriented internal controls framework, called CobiT (Control Objectives for Information and related Technology), was in place long before SOx; it was first released in 1996. Although earlier CobiT versions were more oriented to providing IT auditor and controls guidance, CoboT has became a preferred tool for complying with SOx Section 404 internal controls procedures for many enterprises. CobiT provides guidance on evaluating and understanding internal controls, with an emphasis on enterprise IT resources. CobiT is not a replacement for the COSO internal controls framework but is a different way to look at internal controls in today’s IT-centric world. Although originally launched as guidance to help those professionals once called computer auditors—specialist internal and external auditors who reviewed IT-related internal controls—CobiT has evolved into a helpful tool for evaluating all internal controls across an enterprise. CobiT emphasizes the linkage of IT with other business resources to deliver overall values to an enterprise today. This chapter provides an overview of the CobiT framework and its key components. More important, the chapter 32 E1C02 09/16/2010 13:15:37 Page 33 Introduction to CobiT & 33 describes the relationship between CobiT objectives and the COSO internal control framework for use in all internal audit reviews. CobiT and its related principles can be used or mapped to other IT initiatives as well. For example, Chapter 11 on software engineering and Capability Maturity Model for integration (CMMi) highlights CMMi linkages to CobiT, as does Chapter 25’s discussion on the ISO 17799 standards. In addition, Chapter 16 introduces the related Val IT, an approach to better recognize the value of all IT assets in the enterprise. Val IT addresses assumptions, costs, risks, and outcomes related to a balanced portfolio of IT-enabled business investments. Even if an internal auditor does not use the CobiT framework in reviews of internal controls, all IT and internal auditors should have a high-level knowledge of the basic CobiT framework. As mentioned, CobiT had its origins as an IT audit guidance tool, but it is much broader today. In addition to the COSO internal controls framework, knowledge of CobiT will help an internal auditor to better understand the role of IT controls and risks in many enterprise environments. INTRODUCTION TO CobiT An unusual word for many, CobiT is an acronym that is becoming increasingly recognized by auditors, IT professionals, and many enterprise managers. CobiT is an important internal control framework that can stand by itself but is an important support tool for documenting and understanding COSO and SOx internal controls and recognizing the value of IT assets in an enterprise. A general or working knowledge of CobiT should be an IT auditor requirement. The CobiT standards and framework are issued and regularly updated by the IT Governance Institute (ITGI) and their closely affiliated professional organization, the Information Systems Audit and Control Association (ISACA). ISACA is more focused on IT auditing while ITGI’s emphasis is on research and governance processes. ISACA also manages the Certified Information Systems Auditor (CISA) examination and professional designation as well as its newer Certified Information Systems Manager (CISM) and the Certified in the Governance of Enterprise IT (CGEIT) certification and examination. The Certified Information Security Manager (CISM) certification targets IT security managers and promotes the advancement of professionals who wish to be recognized for their IT governance-related experience and knowledge. These auditrelated professional certifications are discussed in Chapter 30. ISACA was originally known as the EDP Auditor’s Association (EDPAA), a professional group begun in 1967 by internal auditors who felt that their then current professional organization, the Institute of Internal Auditors (IIA), was not giving sufficient attention to the importance of IT systems and technology controls as part of internal audit activities. We have almost forgotten that EDP once stood for electronic data processing, today an almost archaic term for IT. Over time, this professional enterprise broadened its focus and became ISACA, while the IIA has also long since embraced strong technology issues. The EDPAA, originally an upstart IT audit professional organization, began to develop IT audit professional guidance materials shortly after its formation. Just as the 09/16/2010 13:15:37 & Using CobiT to Perform IT Audits e l u ry V a live e D St lig r a t e nm g i en c t 34 Page 34 A Ri nag sk em ent e anc rm ent rfo Pe agem n Ma IT Governance Ma E1C02 Resource Management EXHIBIT 2.1 CobiT IT Governance Focus Areas EDPAA evolved into the well-respected ISACA and now the ITGI, its original IT audit standards became an excellent set of internal control objectives that evolved to CobiT, now in its 2007 version 4.1 edition.1 With virtually all enterprise processes today tied to IT-related facilities, an understanding of the overall area of IT governance is critical. The CobiT framework is often described as a pentagon covering five broad and interconnected areas of internal controls, as illustrated in Exhibit 2.1. The exhibit shows CobiT’s major areas of emphasis arranged around the important core concept of IT governance: 1. Strategic alignment. Efforts should be in place to align IT operations and activities with all other enterprise operations. These efforts include establishing linkages between enterprise business operations and IT plans as well as processes for defining, maintaining, and validating quality and value relationships. 2. Value delivery. Processes should be in place to ensure that IT and other operating units deliver promised benefits throughout a delivery cycle and with a strategy that optimizes costs while emphasizing the intrinsic values of IT and related activities. 3. Risk management. Management, at all levels, should have a clear understanding of an enterprise’s appetite for risk, compliance requirements, and the impact of significant risks. Both IT and other operations have their own and joint risk management responsibilities that may individually or jointly impact the entire enterprise. 4. Resource management. With an emphasis on IT, there should be an optimal investment in, and the proper management of, critical IT resources, applications, E1C02 09/16/2010 13:15:37 Page 35 CobiT Framework & 35 information, infrastructure, and people. Effective IT governance depends on the optimization of knowledge and infrastructure. 5. Performance measurement. Processes should be in place to track and monitor strategy implementation, project completions, resource usage, process performance, and service delivery. IT governance mechanisms should translate implementation strategies into actions and measurements to achieve these goals. These five CobiT internal control concerns are the elements of the CobiT framework and define IT governance. The CobiT framework is an effective tool for documenting IT and all other internal controls. This chapter looks this framework in the broader perspective of using CobiT to assist in the IT governance processes of management, enterprise, and internal auditing. The following sections provide an overall description of the CobiT framework and its key elements to link business with IT goals through key controls and effective measurement metrics. In addition, the chapter describes mapping CobiT standards with the COSO internal control framework, discussed in Chapter 1, with the information technology infrastructure library (ITIL) best practices introduced in Chapter 7, and for overall IT and corporate governance. Elements and key components of IT governance are discussed as well. The CobiT framework is an effective mechanism for documenting and understanding internal controls at all levels. Although CobiT first started primarily as a set of ‘‘IT audit’’ guidance materials, it is a much more powerful tool today. CobiT FRAMEWORK IT processes and their supporting software applications and hardware devices are key components in any enterprise today. Whether a small retail business with a need to keep track of its inventory and pay employees, or a large Fortune 50 corporation, all need a wide set of interconnected and often complex IT processes that are closely tied to their business operations. That is, enterprise business processes and their supporting IT resources should work in a close information-sharing relationship. IT cannot and certainly should not tell business operations what types of IT processes and systems to implement, but IT provides information to help influence business decisions. In the very early days of computer systems, often IT managers felt they had lots of answers and promoted systems solutions to their businesses, sometimes with very counterproductive results. However, this relationship has changed today; IT and business operations generally should have a close mutual relationship of shared requirements and information. Internal auditors must understand the needs and information-sharing requirements on both sides. As discussed in Chapter 1 in the COSO internal controls framework, IT has responsibilities over a series of other related process areas that are audited by or through established audit guidelines, are measured by a series of performance indicator measures and activities, and are made effective through activity goals. All of these can also easily become part of CobiT, a control framework including both IT and business processes. Chapter 1 described the COSO internal control framework and its importance in defining SOx internal controls. An IT auditor might ask, ‘‘I understand and use COSO E1C02 09/16/2010 13:15:37 36 & Page 36 Using CobiT to Perform IT Audits which responds to Enterprise Information Business Requirements COBIT drive the investments in IT Reources that are used by to deliver IT Processess EXHIBIT 2.2 Basic CobiT Principles Source: COBIT 4.1 ß 1996–2001 IT Governance Institute. All rights reserved. Used by permission. internal controls. Why another framework?’’ The answer here is that CobiT provides an alternative approach to define and describe internal controls that has more of an IT emphasis than the pure COSO internal controls framework. Information and supporting IT processes often are the most valuable assets for all enterprises today, and management has a major responsibility to safeguard its supporting IT assets, including their automated systems. Management, users, and IT auditors all need to understand these information-related processes and the controls that support them. This combination of IT processes amd internal controls focuses on the effectiveness and efficiency of IT resources, processes, and overall business requirements. Exhibit 2.2 describes these basic CobiT principles, with business requirements driving the demand for IT resources and those resources initiating IT processes and enterprise information in a continuous, circular manner. Management should be interested in the quality, cost, and appropriate delivery of its IT-related resources whose control components are the same as the COSO internal control elements discussed in Chapter 1. Internal controls over IT resources are very much based on the effectiveness and efficiency interdependencies of these IT components. IT governance is a key concept that was not strongly emphasized as a CobiT element prior to SOx. It is an important internal control concept today with the ITGI playing a strong leadership role. As described in the IT governance pentagon in Exhibit 2.1, CobiT defines IT governance as a series of key areas ranging from keeping focus on strategic alignments to the importance of both risk and performance measurement when managing IT resources. We refer to this IT governance pentagon again as we navigate through the CobiT framework. CobiT looks at internal controls from three IT dimensions: resources, processes, and information criteria, described in the CobiT Cube illustrated in Exhibit 2.3. Similar to the COSO internal controls framework cube discussed in Chapter 1 and Chapter 4 on the COSO ERM framework, this CobiT model looks at IT controls from a three-dimensional perspective. That is, each component on one surface relates to the two other connecting dimensions. However, CobiT’s front-facing dimension with its pictorial descriptions of 09/16/2010 13:15:37 Page 37 CobiT Framework & 37 Business Requirements s lity y ce ity ity es cy ian iabil en cien entia grit ilabil l p e d Int fi a m Rel Ef onfi Av Co C People Information PROCESSES Applications DOMAINS Infrastructure v cti fe Ef IT Processes E1C02 ACTIVITIES es urc IT EXHIBIT 2.3 o es R CobiT Cube Source: COBIT 4.1 ß1996–2001 IT Governance Institute. All rights reserved. Used by permission. process flow diagrams sometimes scared off non-IT people from considering CobiT. The non–IT savvy professional—and there are many—may look at the process diagrams on the face of the CobiT cube and decide this approach must be too technical. This is not at all correct. We describe and explain the CobiT framework and why it can be valuable for understanding SOx internal controls and improving both IT audit and governance practices in the next sections. CobiT Cube Components IT Resources The IT resources side of the three-dimensional CobiT cube framework represents all of an enterprise’s IT assets, including its people, the application systems, installed technology, IT facilities, and the value of data. The right-hand side of the framework cube represents the necessary concerns and considerations for all of the resources necessary for the control and administration of enterprise IT resources. Either individually or as groups, these resources should be considered when evaluating controls in an IT environment: & & & & Applications consisting of both automated user systems and manual or automated procedures to process information Information, including input, output, and processed data, for use by business processes Technology and facility infrastructure components including hardware, operating systems, databases, networks, and the environments that house and support them Key and specialized personnel to plan, organize, acquire, implement, support, monitor, and evaluate IT services E1C02 09/16/2010 13:15:37 38 & Page 38 Using CobiT to Perform IT Audits We have started our CobiT description from the right-hand side of the CobiT cube, but internal control considerations always must be considered in terms of how they relate to other components on that side of the CobiT cube as well as with others in this three-dimensional perspective. The point here is that IT resources should always be considered as a key component to IT governance and internal controls. IT Processes The second and front-facing dimension of the CobiT cube refers to IT processes and consists of three segments: domains, processes, and activities. Domains are groupings of IT activities that match to organizational areas of responsibility, with four specific domain areas defined in CobiT: 1. Planning and enterprise. This domain area covers the strategy and tactics that allow IT to best contribute to and support enterprise business objectives. This type of IT strategic vision message should be communicated throughout the enterprise— the message of IT’s mission and what it is trying to accomplish for the overall enterprise. 2. Acquisition and implementation. IT solutions need to be identified, developed, or acquired and both implemented and integrated with business processes. This domain area covers changes and maintenance of existing systems. 3. Delivery and support. This domain area covers the actual delivery of required services, both application and infrastructure tools. The actual process of application data and controls is covered within this domain. 4. Monitoring and evaluation. This area includes control processes, including quality and compliance monitoring, as well as external and internal audit evaluation procedures. Within an IT enterprise, the process to identify and build new applications— traditionally called systems development life cycle (SDLC) procedures—could be viewed as part of the CobiT implementation domain, and quality assurance could be viewed as part of the monitoring domain. For the planning and enterprise domain, CobiT suggests these specific processes: & & & & & & & & & & & Define a strategic IT plan. Define the information architecture. Determine technological direction. Define the IT enterprise and relationships. Manage the IT investment. Communicate management aims and direction. Manage human resources. Ensure compliance with external requirements. Assess risks. Manage projects. Manage quality. E1C02 09/16/2010 13:15:37 Page 39 Using CobiT to Assess Internal Controls & 39 Individual processes are the next level down. They are a series of joined activities with natural control breaks. Finally, activities are the actions needed to achieve measurable results. Activities have a life cycle whereas tasks are discrete. We can think of the systems development life cycle (SDLC) process as a cycle where a new application is designed, implemented, operated over time, and then replaced with an improved process. Business Requirements The third dimension of the CobiT model consists of business requirements. These seven components should be considered when evaluating all business requirements and with consideration given to the following necessary IT resources and process criteria elements: 1. 2. 3. 4. 5. 6. 7. Effectiveness Efficiency Confidentiality Integrity Availability Compliance Reliability All IT overall systems or processes should be evaluated with consideration given to one or more of these seven criteria areas. Emphasis will vary, but all IT processes should have elements of one or more of these criteria. For example, for a given IT application, an IT auditor may be concerned about its confidentiality and integrity controls. Business functions typically establish such requirements for their general business needs. For IT applications, each of these attributes is discussed in more detail in the next section as well as in Chapter 8 on planning and performing IT general controls audits. Similar to the COSO cube internal control model from Chapter 1 and the COSO enterprise risk framework discussed in Chapter 4, the CobiT cube presents an effective way to understand the relationships among business requirements, IT processes, and IT resources. The three-dimensional nature of the model emphasizes the cross relationships and interdependencies between business and IT processes. In our IT-dependent world, this is a useful way for an IT auditor to look at and understand internal controls. CobiT is a rich—sometimes almost too rich—set of processes for focusing on business and IT goals and key controls, and for identifying key measurement metrics. The next sections discuss CobiT in some detail, but the reader is encouraged to consult the ITGI’s CobiT reference materials at www.isaca.org for further information. USING CobiT TO ASSESS INTERNAL CONTROLS Besides the CobiT cube, with its forward face showing process flow diagrams to emphasize relationships, the published CobiT guidance materials2 can look formidable to many internal auditors and other business professionals as well as even some IT professionals. The basic CobiT reference material is published in a nearly 200-page manual filled with an array of charts and tables. It is a useful set of materials, but some study may be E1C02 09/16/2010 13:15:37 40 & Page 40 Using CobiT to Perform IT Audits required to fully understand the concepts behind the CobiT framework. The following sections should help an IT auditor to navigate through the published CobiT framework, and more importantly, use it to develop and assess enterprise internal controls. Although any dimension of the CobiT cube can be used to understand control environments, the four previously discussed domains, starting with Planning and Enterprise, is an effective first step in the CobiT cube diagram. Based on these initial three CobiT control cube dimensions, each IT process should be evaluated through these five navigation steps: I. The control of [Process Name] II. Which satisfy [List of Business Requirements] III. By focusing on [List of important IT goals] IV. Is achieved by [List of Control Statements] V. And is measured by [List of key metrics] This five-step process dialogue can go from number I on down or can start at the base level and navigate up. In either case, the CobiT framework says that the control of any process should be satisfied by a list of supporting business requirements and that those business goals should focus on important IT goals. This only makes sense. A designated process would just be an idle name unless supported by specific business and IT requirements to drive and govern that process. Each of those requirements should be defined by one or more control statements with specific control practices. Finally, we must assess whether matters are operating effectively, and key measurement metrics are necessary. Although CobiT’s emphasis historically has been on IT, this type of analysis should be used for a wide range of internal control–related audits, IT related or not. Each major control objective described in the published CobiT guidance material is based on the ITGI’s navigation framework shown in Exhibit 2.4. The upper left corner of that exhibit shows business requirements. While this is a blank sample, in the published CobiT guidance, each of these requirements are marked with a P to indicate a primary requirement, S for a secondary, or left blank for a not applicable control objective. The lower right-hand corner lists the IT resource areas. If any are applicable, they are noted with a check mark. The lower left-hand corner shows the same pentagon diagram we saw in Exhibit 2.1. Here sections are shaded or marked if they are of primary or secondary importance. The center of each of the CobiT guidance pages has the ‘‘Control over the IT Process of . . . ’’ series of statements completed for each control objective. We will show examples of completed statements as we review the CobiT domains. Even though the CobiT navigation and supporting documentation is thorough and somewhat elegant, it can scare away first-time some auditors. The next sections look at CobiT navigation across various selected domains to give a feel for its organization. Even if auditors are accustomed to using the COSO internal controls framework, they should at least experiment with using CobiT in selected reviews. This chapter provides an introduction and overview to CobiT, its supporting ITGI professional organization has a wide variety of published and educational offerings on the use of CobiT that can be found on previously referenced Web site www.isaca.org. 13:15:37 Page 41 Using CobiT to Assess Internal Controls & 41 ct i Ef ven fic es i s on enc fid y e In ntia te lit g Av rity y ai l C abil om ity p R lian el ia ce bi lit y Plan and Organize Acquire and Implement C Deliver and Support Control over the IT process of Monitor and Evaluate process name that satisfies the business requirement for IT of summary of most important business goals by focusing on summary of most important IT goals is achieved by key controls and is measured by Primary EXHIBIT 2.4 e ur Pe op le ru ct n st io at fra In rm fo Ap pl ic at io ns key metrics In 09/16/2010 Ef fe E1C02 Secondary Navigating the CobiT Framework Source: COBIT 4.1 ß1996–2001 IT Governance Institute. All rights reserved. Used by permission. Planning and Enterprise CobiT calls for a high level group of processes that set the direction for the enterprise and its IT resources. For this domain, CobiT has 10 high-level Planning and Organizing (PO) control objectives, defined and numbered as follows: PO1 PO2 PO3 PO4 PO5 PO6 PO7 PO8 PO9 PO10 Define a Strategic Plan Define the Information Architecture Determine Technological Direction Define IT Process, Enterprise, and Relationships Manage the IT Investment Communicate Management Aims and Direction Manage IT Human Resources Manage Quality Assess and Manage IT Risks Manage Projects These are all very high-level concepts. Many IT professionals and other managers might say ‘‘Of course!’’ when an internal auditor asks whether they have such a strategic plan, have defined their information architecture, or are in compliance with any of the other high-level PO objectives. However, CobiT drills down into its defined control objective areas in greater detail. While many of the issues are similar, this is in strong contrast to the monitoring element of COSO internal controls as discussed in E1C02 09/16/2010 13:15:38 42 & Page 42 Using CobiT to Perform IT Audits Chapter 1. There, the concepts are only at a fairly high level; CobiT is much more specific. For example, using the PO1 control objective on defining a strategic plan, CobiT then expands this to six more detailed objectives: PO1.1 PO1.2 PO1.3 PO1.4 PO1.5 PO1.6 IT Value Management Business-IT Alignment Assessment of Current Performance IT Strategic Plans IT Tactical Plans IT Portfolio Management The numbering here is important as the published CobiT guidance materials references each of these and other objectives in terms of references to their inputs and outputs. CobiT’s published materials provides a high-level description for each of these objectives. For example, the PO1.4 objective on strategic plans states: Create a strategic plan that defines, in co-operation with the relevant stakeholders, how IT will contribute to the enterprise’s strategic objectives (goals) and related costs and risks. It includes how IT will support IT-enabled investment programs and operational service delivery. It defines how the objectives will be met and measures and will receive formal sign-off from the stakeholders. The IT strategic plan should cover investment/operational budget, funding sources, sourcing strategy, acquisition strategy, and legal and regulatory requirements. The strategic plan should be sufficiently detailed to allow the definition of tactical IT plans. This paragraph is an example of one of the many control objectives outlined throughout the CobiT guidance. All are covered elsewhere. The objectives here do not tell the professional how to write an IT strategic plan but do provide excellent guidance to build such a plan, no matter the size or status of the enterprise. These general objectives are also good tools for internal auditors in their needs to build review criteria in any of these areas. IT auditors can develop those audit objectives by taking each sentence of the objectives and developing audit review areas. For each of the CobiT objectives, the guidance material also contains what is called a supporting RACI chart. A tool that evolved from quality initiatives in the 1960s, RACI charts are good tools to identify roles and responsibilities. Using a spreadsheet format, activities are identified in a side column and functions or position descriptions are located in cells across the top. Responsibilities for those activities are identified in intersecting cells through one or several of the RACI initials: R ¼ Responsible (who owns the problem or process) A ¼ Accountable (who must sign off on the activity before it is effective) C ¼ Consulted (who has the information and/or capability to complete the work) I ¼ Informed (who must be informed of the results but need not be consulted) This type of chart can be useful in many areas to help identify responsibilities over multiple areas. Exhibit 2.5 is a RACI chart, adapted from CobiT materials, on the PO1 objective to define a strategic IT plan. Going down the column of responsibilities, in this E1C02 09/16/2010 13:15:38 Page 43 Using CobiT to Assess Internal Controls CobiT PO1 Activities 43 & Business Process Head of Head IT Internal CEO CFO Executive CIO Owner Operations Admin. Audit Link Business Goals to IT Goals C I A/R R C Identify Critical Dependencies and Current Performance C C R A/R C C C C Build an IT Strategic Plan A C C R I C C C Build IT Tactical Plans C I A C C C I Analyze Program Portfolios and Manage Service Portfolios C I A R C C I EXHIBIT 2.5 I RACI Chart Example example, the Business Process Owner acts as a Consultant on linking business goals, building tactical plans, and identifying critical dependencies; is Informed on processes for building the strategic plan; and is Responsible for analyzing program portfolios. The chart outlines responsibilities for such people as the enterprise’s head of IT or chief information officer (CIO), the process owner, and internal audit. This type of RACI chart appears in the published guidance for each of the CobiT control objectives. The CobiT material concludes with a summary analysis of the control objective. This is a metrics-based set of considerations that outlines the activity goals for a given control objective that are measured by key performance indicators (KPIs) that drive process goals and are measured by process-related key goal indicators. The latter drive IT goals that are measured by IT key goal indicators. This CobiT documentation is explained in more detail as we review the other CobiT control objectives. For the major control objective, the guidance material discusses each PO control objective in the same manner and following the same approach with suggested highlevel controls review approaches. However, many of these CobiT items may be found only in larger IT enterprises, although the CobiT guidance material has a range of approaches for each objective. For example, the CobiT objective PO3.5 calls for the need for an enterprise IT architecture board or function. This is valuable guidance, but many smaller IT functions do not have the resources to establish such a formal IT architecture board function. Managers who use CobiT and auditors who evaluate compliance should always remember that CobiT is a set of best practices guidance materials, not mandatory requirements. Internal auditors should always use the CobiT guidance with some level of caution, recognizing that CobiT often only specifies some very high-level ideal environments. An auditor reviewing IT general controls in a smaller enterprise who follows CobiT guidance to the letter and recommends such a formal ‘‘IT Architecture’’ board could get laughed out of someone’s office. Acquisition and Implementation Each of the CobiT high-level control objectives discusses the control procedures in the same general format. Whether it is in-house software development efforts or purchased E1C02 09/16/2010 13:15:38 44 & Page 44 Using CobiT to Perform IT Audits IT components, the recommended high-level CobiT acquisition and implementation (AI) objectives are: AI1 AI2 AI3 AI4 AI5 AI6 AI7 Identify Automated Solutions Acquire and Maintain Application Software Acquire and Maintain Technology Infrastructure Enable Operation and Use Procure IT Resources Manage Changes Install and Accredit Solutions and Changes Each of the detailed objectives in this domain covers control procedures over the implementation of new tools. Although the emphasis is on IT software, the internal control concepts also can be applied to the acquisition and implementation of many new enterprise tools. Space does not allow complete coverage of each control objective, but we will examine AI6 on managing change as an example of how CobiT uses its basic framework to outline the importance of this control area. For example, earlier we outlined CobiT’s five-step process for evaluating control objectives. The outline for these steps appears in the center of the Exhibit 2.4 navigation page. The AI6 objectives on managing changes follow the same five-step process and are outlined in this way: I. Control over the IT Process of managing changes II. That satisfies the business requirement for IT of responding to business requirements in alignment with business strategy, while reducing solution and delivery defects and rework III. By focusing on controlling impact assessment, authorization, and implementation of all changes to the IT infrastructure, applications, and technical solutions, minimizing errors due to incomplete request specifications and halting implementation of unauthorized changes IV. Is achieved by & Defining and communicating change procedures, including emergency changes & Assessing, prioritizing, and authorizing changes & Tracking status and reporting on changes V. And is measured by & Number of disruptions or data errors caused by inaccurate specifications or incomplete impact assessment & Application or infrastructure rework caused by inadequate change specifications & Percent of changes that follow internal change controls processes. This series of statements, taken from the CobiT guidance materials, describes the control requirements and measure for this AI6 control objective. The CobiT guidance materials have a similar set of statements for each control objective. This summary of E1C02 09/16/2010 13:15:38 Page 45 Using CobiT to Assess Internal Controls & 45 control considerations is useful when attempting to better understand the characteristics of each control. That same guidance material looks at how each control objective relates to the other two sides of the CobiT cube. For the AI6 managing changes control objective, it indicates that on the IT Resources side, all CobiT defined internal controls are important or have a check mark to help in in understanding this control objective. That is, the control object impacts applications, information, infrastructure, and people. Turning to the upper left side of the navigation sheet business requirements dimension, the guidance material indicates objectives of whether they are of primary, secondary, or of no significant importance. For this AI6 example control objective, the business requirements of effectiveness, efficiency, integrity, and availability are of primary importance while reliability is of secondary importance. The remaining two business requirements, confidentiality and compliance, however are not considered significant to this control objective. For each control objective, the CobiT guidance material is based on an image of the Exhibit 2.1 focus areas pentagon for the AI6 control objective, and value delivery is of prime importance with resource management secondary. The CobiT guidance material does not provide detailed discussion of the reasons for that designation. Professionals working with the CobiT control objectives usually can deduce why a given IT governance area has been designated as of primary or secondary significance. Delivery and Support Following the same general format, the third high-level CobiT control objective is called Delivery and Support (DS). This control objective largely covers service management issues related to the ITIL business process objectives, as discussed in Chapter 7, and highlights some of the changes to our understandings of internal controls that have evolved since the enactment of SOx in 2002. Both CobiT and ITIL were in existence at that time, but the SOx Section 404 emphasis on effective internal controls has brought things together. The CobiT DS control objectives are also similar to the ITIL internal controls to enhance business processes. Both cover the important area of what is known as IT service management, the processes required to ensure efficient IT operations and to deliver these services. In earlier days, concerns about IT internal controls focused on individual application-by-application controls. Much attention was paid to the higher-level general controls, such as perimeter security or disaster recovery planning, but financial auditors often focused on computational and balancing controls in specific applications. However, no matter how well designed, all such IT applications must operate in an efficient and almost automated process. There will always be smaller problems, such as a legitimate systems user becoming locked out by entering incorrect passwords, and there is a need for efficient service and problem management processes to report and resolve such matters. The CobiT DS control objectives cover many of these important areas: DS1 DS2 Define and Manage Service Levels Manage Third-Party Services E1C02 09/16/2010 13:15:38 46 & DS3 DS4 DS5 DS6 DS7 DS8 DS9 DS10 DS11 DS12 DS13 Page 46 Using CobiT to Perform IT Audits Manage Performance and Capacity Ensure Continuous Service Ensure Systems Security Identify and Allocate Costs Educate and Train Users Manage Service Desk and Incidents Manage the Configuration Manage Problems Manage Data Manage the Physical Environment Manage Operations These control objectives represent important areas of IT operations that historically have not received sufficient attention from IT auditors. The CobiT material looks at each of these objectives in the same general format, summarizing how each control objective is achieved and measured as well as the relationships and interdependencies across all three sides of the CobiT cube. Many of these control objective areas did not receive sufficient attention in internal controls reviews prior to SOx Section 404 and its Auditing Standard No. 5 (AS 5) rules. COSO objectives address internal controls at a high level but sometimes does not address more detailed service management–related internal control issues. The CobiT DS10 control objective for problem management is an example: Effective problem management requires the identification and classification of problems, root cause analysis and resolution of problems. The problem management process also includes identification of recommendations for improvement, maintenance of problem records and review of the status of corrective actions. An effective problem management process improves service levels, reduces costs, and improves customer convenience and satisfaction. IT users have had problems over the years in reporting problems and seeking resolutions with various systems and applications. Insensitive IT operations staffs frequently did not do an appropriate job in resolving reported problems efficiently. All too often, if an application totally failed, there would be a strong effort to get it back in operation; smaller, less critical problems would be brushed off in a cavalier manner as something to be ‘‘considered in the next update.’’ The published CobiT guidance material links this control objective to others that provide for its inputs as well as outputs. For example, the objectives of AI6 on change authorization, DS8 for incident reporting, DS9 for IT configuration management, and DS13 on error logs all provide inputs to the DS10 control objective. Our purpose is to not reproduce the full contents of all of the published CobiT control objectives but to give the IT auditor a feel for CobiT’s approach. Here and for all of the domains and objectives, CobiT provides a powerful way to look at the breadth and depth of these IT-related internal controls and their relationships. We have discussed how each CobiT control has a series of detailed objectives, has other control inputs and outputs, and has a RACI chart balancing functions and E1C02 09/16/2010 13:15:38 Page 47 Using CobiT to Assess Internal Controls & 47 Activity Goals and Metrics Assigning sufficient authority to problem manager Performing root cause analysis of reported problems Analyzing terms Taking ownership of problems and problem resolution are measured by Key Performance Indicators Average duration between the logging of a problem and identification of root cause. % of problems for which root cause analysis was undertaken The frequency of reports or updates to an ongoing problem based on its severity. drive Process Goals Record and track operational problems through resolutions. Investigate the root cause of all significant problems. Define solutions for identified operations problems. are measured by Process Key Goal Indicators % of problems recorded and tracked % of problems that recur by time and severity % of problems resolved with required time period # of open, new, and closed problems by severity Average and standard deviation of time lag between problem identification and resolution Average and standard deviation of time lag between problem resolution and closure drive IT Goals Ensure satisfaction of end users with service offerings and service levels. Reduce solution and service delivery defects and rework. Protect the achievement of IT objectives. are measured by IT Key Goal Indicators # of recurring problems with impact on business # of business disruptions caused by operational problems EXHIBIT 2.6 CobiT Example: DS10 Manage Problems Goals and Metrics responsibilities for each control. In addition, the published CobiT guidance materials have a goals and metrics section for each control objective. Exhibit 2.6 shows this goals and metrics chart for the DS10 Manage Problems control objective. Each published CobiT control objective has a similar set of these very useful analyses. With problem management, for example, three suggested measurement metrics should be E1C02 09/16/2010 13:15:38 48 & Page 48 Using CobiT to Perform IT Audits considered. One is performing root cause analyses of reported problems, which is an important goal that is sometimes missed. The related RACI chart highlights that the problem manager is responsible for this activity with others given a consulting role. Under each activity goal, a table of KPIs follows that drive a set of process goals. With different specific content for each CobiT control, this type of analysis provides all parties with a good set of standards for measuring the performance of control areas and establishing metrics to assess achievement of these goals. This analysis for our selected control objective of problem management is a good example of the power of the published CobiT materials. Many IT operations have some types of help desk function to report and resolve problems. Here, CobiT provides some good suggestions for the types of measures and metrics that can be used to evaluate the achievement of this control objective. Similar tables of goals and objectives as well as detailed control objectives exist for each CobiT control objective. These are requirements similar to the SOx Section 404 auditing procedures discussed in Chapter 1 or the internal audit professional practice standards referenced in Chapter 3. However, this set of CobiT materials provides excellent guidance materials for establishing and then measuring effective internal controls. IT auditors should become familiar with CobiT, Monitoring and Evaluation The fourth CobiT domain is called Monitoring and Evaluation (ME). This set of control objectives emphasizes CobiT as a closed loop process that effectively never ends. CobiT calls for establishing baseline measures to allow an enterprise to measure how it is performing and to provide the enterprise with future opportunities. This domain area covers quality assurance areas that are traditionally more common to manufacturing and other operations areas than they have been to IT. Although not discussed in the CobiT guidance materials, the pioneering quality assurance work of Edward Deming provides a way of considering this CobiT domain area. A consultant helping to rebuild Japan in the aftermath of World War II, Deming developed quality standards and approaches that helped Japan rebuild and establish the quality practices that are used worldwide today. Among his approaches, Deming developed a quality system that called for business processes to be analyzed and measured to identify the sources of variations that cause products to deviate from customer requirements. He proposed that business processes be viewed or defined in a continuous feedback loop so that managers could identify and change any parts of the process that needed improvement. This concept defines a continuous, never-ending cycle where we should always monitor current process performance and take actions to implement improvements to that process. Deming called this the Plan, Do, Check Act cycle (PDCA), as shown in Exhibit 2.7. The steps here are: Step Step Step Step 1. 2. 3. 4. Plan. Business processes should be designed or revised to improve results. Do. Implement to plan and measure its performance. Check. Assess the measurements and report the results. Act. Decide on needed changes to improve results. 13:15:38 Page 49 Using CobiT to Assess Internal Controls INVESTIGATE 49 N CORRECT AND STANDARDIZE & A PL T CLARIFY OBJECTIVES IDENTIFY POSSIBLE CAUSES REVIEW FEEDBACK AND MAKE CORRECTIONS STANDARDIZE DO, CHECK, ACT EVALUATE AND VALIDATE PILOT STUDY SOLUTION TO VERIFY DATA COUNTERMEASURE TRAINING BENCHMARK BEST PRACTICE IDENTIFY TEAM ROLES IMPLEMENT QUICK FIX ENLIGHTEN AND IMPLEMENT CARRY OUT TRIALS TO PROVE CAUSES ANALYZE DATA TO UNDERSTAND HOW PROBLEM OCCURS K EC H C IDENTIFY POSSIBLE SOLUTIONS EXHIBIT 2.7 O COMMUNICATION D 09/16/2010 AC E1C02 Deming’s PDCA Cycle Source: Robert R. Moeller, Sarbanes-Oxley Internal Controls: Effective Auditing with AS5, CobiT, and ITIL (Hoboken,NJ:JohnWiley&Sons,2008).Copyrightß 2008JohnWiley&Sons.Reprintedwithpermissionof John Wiley & Sons, Inc. Although Deming’s focus was on postwar reconstruction and on industrial production, his concepts have been carried forward and are very appropriate for today’s business environments, including IT operations and SOx internal control monitoring. Chapter 1 discussed the status of SOx today and its continuing monitoring requirements. This CobiT monitoring and evaluation component calls for such continuous monitoring processes. Following the same format as the other CobiT domains, the ME domain component has four principal control objectives: ME1 ME2 ME3 ME4 Monitor and Evaluate IT Performance Monitor and Evaluate Internal Controls Ensure Regulatory Compliance Provide IT Governance This area is of particular interest to internal auditors as well as other members of an enterprise. The control material for ME2 on monitoring and evaluating internal E1C02 09/16/2010 13:15:38 50 & Page 50 Using CobiT to Perform IT Audits controls is a good example of CobiT’s strength. It states that the process of monitoring and evaluating internal control is achieved by defining the system of IT controls embedded in the IT process framework, by monitoring and reporting on the effectiveness of these internal controls, and by reporting exceptions to management for corrective action. This is really the Deming PDCA process just discussed, and it should be measured by: & & & Number of internal control breaches Number of control improvement initiatives Number and coverage of control self-assessments As with most of the CobiT framework, the material here focuses on IT controls, but many of these concepts can be generalized to an overall internal controls review process. The term control self-assessments refers to the process of ongoing internal reviews on the completeness and effectiveness of internal controls. This CobiT controls objective has seven detailed supporting objectives. These detailed controls have been somewhat abbreviated from the CobiT guidance materials to describe their essence. Although CobiT is oriented to internal reviews of these primarily IT resource areas, this guidance is particularly important for internal auditors in their reviews of IT and all other internal controls: ME2.1. Monitoring of the Internal Control Framework. IT auditors should continuously monitor the control environment and framework using industry best practices and benchmarking to improve the control environment and framework. ME2.2. Supervisory Review. In addition to auditor reviews, CobiT calls for management to monitor and report on the effectiveness of IT internal controls through supervisory reviews, including compliance with policies and standards, information security, change controls, and controls established in service-level agreements. ME2.3. Control Exceptions. Record information regarding all control exceptions and ensure that it leads to analysis of the underlying cause and to corrective action. Management should decide which exceptions should be communicated to the individual responsible for the function and which exceptions should be escalated. ME2.4. Control Self-Assessments. IT management should evaluate the completeness and effectiveness of the internal controls over IT processes, policies, and contracts through a continuing program of self-assessment. ME2.5. Assurance of Internal Control. IT operations should obtain, as needed, further assurance of the completeness and effectiveness of internal controls through third-party reviews by the corporate compliance function, internal audit, outside consultants, or certification bodies. ME2.6. Internal Control at Third Parties. Assess the status of each internal external provider’s internal controls and confirm they comply with legal and regulatory requirements and contractual obligations. ME2.7. Remedial Actions. Identify and initiate remedial actions based on the controls assessments and reporting. This includes follow-up of all assessments E1C02 09/16/2010 13:15:38 Page 51 Using CobiT in a SOx Environment & 51 including: (1) review, negotiation, and establishment of management responses; (2) assignment of responsibility for remediation or risk acceptance; and (3) tracking the results of the actions taken. These CobiT control objectives are described as ‘‘detailed’’ but provide openings for a wide range of even more detailed control procedures. For example, ME2.1 on Monitoring the Internal Control Framework requires IT auditors or other internal controls specialists to develop detailed control procedures that typically may result in a program of many more tests or steps. This CobiT control objective, as well as all of the others in the supporting documentation, has a section on assessing the maturity of each internal control. Maturity here refers to the Capability Maturity Model for Integration (CMMI), introduced in Chapter 11, and a five-level assessment measure designed and developed by Carnegie Mellon University.3 The model has defined levels for when controls can be assessed from a CMMI level 1 of nonexistent, level 2 as initial or ad hoc controls all the way to level 5, called optimized controls. CobiT rates each of its controls against this CMMI measure. For example, CobiT defines that an enterprise will be at level 3, defined process controls for ME2, Monitor and Evaluate Internal Controls, when management supports and has institutionalized internal control monitoring. The guidance goes on to say that policies and procedures should have been developed for processing and reporting internal control monitoring activities. To achieve this CMMI level, an educational and monitoring program for internal control evaluations should have been defined. CMMI is discussed in greater detail in Chapter 11. We have shown this limited extract for ME2, but the published CobiT materials also have a similar limited set of CMMI maturity level guidance materials for each of their internal controls. Although summarized at a very high level, these maturity model guidelines allow an enterprise to assess how it is doing with regard to each of CobiT’s internal controls. USING CobiT IN A SOx ENVIRONMENT When SOx first became effective in the United States, there was little guidance on how to implement and manage its Section 404 internal controls reviews. The Public Company Accounting Oversight Board initially indicated that they were going to establish some specific standards but initially left enterprises and their external auditors on their own. With its heavy emphasis on high-level IT-oriented internal controls, many enterprises adopted CobiT as the internal control framework of choice. This section reviews using the CobiT framework to help achieve SOx compliance. Chapter 1 discussed SOx Section 404 internal controls assessment requirements and highlighted risk-based approaches for evaluating internal controls with an emphasis on the COSO internal controls framework. CobiT is a powerful alternative internal controls assessment framework, particularly in environments with a heavy concentration of IT processes and resources. As discussed, both COSO internal controls and CobiT use threedimensional frameworks to describe their internal control environments. Each is similar, 09/16/2010 13:15:38 52 & Page 52 Using CobiT to Perform IT Audits EXHIBIT 2.8 Risk Assessments Section 404 Control Environment Section 302 R IT Pr o ce ss es B eq u u si n ir e em s s en ts S Co ta n te tr m ol en ts P C ro o n ce t du r o re l s CobiT Objectives COSO Internal Control Components E1C02 Control Activities Information and Communication Monitoring Relationship between COSO Components and CobiT Objectives but with slight differences in classifications and terminology. Exhibit 2.8 shows how the CobiT framework maps to the COSO internal controls model. CobiT’s prime objectives, from Planning and Enterprise to Monitoring and Evaluation, can be used to understand and evaluate internal controls through COSO’s five internal control components. Whether considering COSO internal controls in general or when using CobiT, an internal auditor should move through a series processes from planning to performing risk assessments and on to identifying, documenting, and evaluating key internal controls. With SOx, the industry-wide increased emphasis on IT governance and the recognition of the criticality of IT in most internal control decisions, CobiT has gone through multiple revisions up through its current 4.1 edition. CobiT’s sponsoring IT Governance Institute has been doing an excellent job releasing publications that map the CobiT framework to these other standards. For example, a very detailed study maps the CobiT framework to SOx audit requirements.4 Exhibit 2.9 is an extract of this published CobiT guidance showing how major CobiT control objective areas links with the major COSO components of internal control. This link-up ties together even better by going a level lower. For example, CobiT objective AI6, Managing Changes, under the Acquire and E1C02 09/16/2010 13:15:39 Page 53 Using CobiT in a SOx Environment & 53 COSCO Components Control Risk Control Information and Environment Assessment Activities Communication Monitoring Plan & Organize COSO Control Objective Define a strategic IT plan X Define the information architecture X X X X Determine technological direction Define the IT organization and relationships X X Communicate management aims and directions X X Manage human resources X X Manage the IT investment Ensure compliance with external relationships Assess risks X X X X X X X X Manage projects Manage quality X Acquire and Implement COSO Control Objective Identify automated solutions Acquire and maintain application software X Acquire and maintain technology infrastructure X Develop and maintain procedures X Install and accredit X Manage changes X X X Deliver and Support COSO Control Objective Define and manage service levels X Manage third-party services X Manage performance and capacity X X Ensure continuous service X X Ensure systems security X X X X Identify and allocate costs X X X X Educate and train users X X X X X X X X Assist and advise customers Manage the configuration X X Manage problems and incidents X X X Manage data X X Manage facilities X Manage operations X X X Monitor and Evaluate COSO Control Objective Monitor the processes X Assess internal control adequacy Obtain independent assurance X X Provide for independent audit EXHIBIT 2.9 X COSO and CobiT Relationships X E1C02 09/16/2010 13:15:39 54 & Page 54 Using CobiT to Perform IT Audits Implement control domain impacts the COSO components of Control Activities and Monitoring. The current published CobiT detailed control objectives tie to each of these COSO components and the current version of CobiT is in draft form at this time of publication. There is a close relationship between these CobiT and COSO control objectives and components. The full set of CobiT control objectives materials will provide strong support for an internal auditor performing a SOx Section 404 internal controls assessment review. Although the concepts can be used in any internal control area, the emphasis is on IT applications and processes. For many enterprises, an understanding and assessment of those IT-associated internal controls is key to achieving SOx compliance. CobiT has been around for some years now, but too many have viewed it as just a specialized IT audit tool, and not a more general help for other internal audit work. Although CobiT’s emphasis continues to be on IT, all internal auditors should explore the CobiT framework as a tool for helping with current and evolving SOx compliance requirements. CobiT ASSURANCE FRAMEWORK GUIDANCE The CobiT framework provides guidance for establishing effective internal controls with an emphasis on IT resources. In 2008 the ITGI released its Information Technology Assurance Framework (ITAFTM)5 guidance, a good-practice-setting model to provide guidance on the design, conduct, and reporting of IT audit and assurance assignments. The objective of this guidance is to establish standards that address IT audit and assurance professional roles and responsibilities; knowledge and skills; and diligence, conduct, and reporting requirements. This new CobiT guidance focuses on another perhaps soon to be common audit-related word, assurance. This term is also found in IIA basic references that state: ‘‘Internal auditing is an independent, objective assurance and consulting activity designed to add value and improve an organization’s operations. It helps an organization accomplish its objectives by bringing a systematic, disciplined approach to evaluate and improve the effectiveness of risk management, control, and governance processes.’’ Assurance services cover all forms of internal auditing, risk management, and compliance services. This CobiT guidance covers a wide range of reviews performed by an internal auditor. The overall objective of ITAF is to define a set of standards to help ensure the quality, consistency, and reliability of IT assessments, based on a set of good-practice-setting guidelines and procedures. Although the ITAF document refers to its guidance as ‘‘standards,’’ CobiT’s ISACA professional organization is not recognized as widely as the IIA’s International Standards for the Professional Practice of Internal Auditing. Chapter 3 outlines the IIA Standards and summarizes the CobiT-related ITAF Standards. We mention the new ITAF Standards to highlight the fact that internal auditing standards are being developed to help with reviews in a CobiT environment. However, internal auditors should understand that ITAF is new; and it may achieve more recognition as it becomes better accepted and perhaps fine-tuned. E1C02 09/16/2010 13:15:39 Page 55 Notes & 55 CobiT IN PERSPECTIVE Whether operational, financial, or IT specialists, all internal auditors should have at least a high-level understanding of the CobiT framework. It is a particularly useful tool for assessing internal controls in a more IT-oriented environment—the type of environment that we almost always encounter today. The decision to use CobiT in internal audits should not be a one-time or individual audit–level decision. Rather, internal audit should train key members of the audit team in the use of CobiT and then try using it to assess internal controls on some other audit currently being developed and documented using the IT audit techniques discussed in Chapters 8 and 10. Unfortunately, the IIA hasn’t really given proper reference to CobiT and its IIA members. Although the IIA now has some good internal audit standards for its members, as discussed in Chapter 3, the IIA Standards do not come close to CobiT as a tool for helping to define and understand IT controls. All IT auditors should understand and use CobiT. If internal audit feels that CobiT will offer some improvements to ongoing audit processes, the concept should first be discussed with the audit committee to explain reasons for changing internal audit approaches. If the enterprise places a heavy reliance on IT systems and processes, using CobiT seems to be a good logistical move. However, an overall internal audit function should avoid having IT internal audit specialists use CobiT assessment processes while the rest of internal audit uses established operational/ financial internal audit standards. CobiT is an elegant—sometimes even too elegant—internal control framework and evaluation tool for assessing internal controls. Perhaps the largest impediment to its overall use is that CobiT was originally constructed as primarily an IT audit tool. Although the move from ISACA to the ITGI sponsorship has broadened its appeal and focus, the published CobiT guidance materials have a very heavy IT focus. This focus sometimes scares away some potential users. The real strength of CobiT is its IT governance focus, as described in Exhibit 2.1. That exhibit illustrates the importance of the strategic alliance of business and IT resources with value delivery, resource management, risk management, and performance measurement processes. All five of these areas allow an enterprise to establish effective IT governance, and CobiT should help in managing and understanding these concepts. We can expect CobiT published standards and practices to continue to broaden and go beyond just ‘‘IT audit’’ special concepts. All internal auditors should learn to use and understand the CobiT internal controls assessment framework. NOTES 1. IT Governance Institute, CobiT—Governance, Control and Audit for Information and Related Technology, 4th ed. (Rolling Meadows, IL: Author, 2000). 2. IT Governance Institute, CobiT 4.1 (Rolling Meadows, IL: Author, 2007). 3. Capability Maturity Model1 Integration (CMMI) is a Carnegie Mellon University– developed process improvement approach that provides organizations with the essential E1C02 09/16/2010 13:15:39 56 & Page 56 Using CobiT to Perform IT Audits elements of effective processes. It can be used to guide process improvement across a project, a division, or an entire organization. 4. IT Governance Institute, IT Control Objectives for Sarbanes-Oxley, 2nd ed. (Rolling Meadows, IL: Author, September 2006). 5. IT Governance Institute, ITAFTM: A Professional Practices Framework for IT Assurance (Rolling Meadows, IL: Author, 2008). E1C03 09/16/2010 13:14:38 Page 57 3 CHAPTER THREE IIA and ISACA Standards for the Professional Practice of Internal Auditing E V E R Y P R O F E S S I O N R E Q U I R E S A set of standards to govern its practices, general procedures, and ethics. These standards allow specialists performing similar work to call themselves professionals in their area of expertise. The key standards for all internal auditors are the Institute of Internal Auditors’ (IIA’s) International Professional Standards for the Practice of Internal Auditing, a set of guidance materials that in the pre-Web paper-document days were known as the Red Book by many internal auditors. These standards have gone through multiple revisions over the years and more recently have been issued in a set of internal audit mandatory and guidance materials known as the International Professional Practices Framework (IPPF). Although the IIA Standards have been in place for many years, 2008 brought some significant new changes to internal audit standards. Standards that in the past were more guidance directed, stating that an internal auditor ‘‘should’’ do something, now are far more definitive and specify practice areas where an internal auditor ‘‘must.’’ This chapter summarizes the current IIA Standards and provides guidance on how they apply to today’s information technology (IT) auditor. An understanding of the IIA’s International Standards for the Professional Practice of Internal Auditing is an absolute must requirement for IT and all other internal auditors. These standards and the IPPF provide the support for many if not nearly all internal audit professional activities. We also summarize the IT auditing standards of the Information Systems and Control Association (ISACA). As discussed in Chapter 2 on the Control Objectives for Information and related Technology (CobiT) framework, ISACA is the leading professional organization for IT auditors. ISACA’s IT audit standards complement the IIA Standards but are much more technically specific. These standards are relatively new, compared to the IIA Standards; the first ISACA standards were released in the late 1990s. A full set of these IT audit–specific standards now exists, which is essential for IT auditing activities. 57 E1C03 09/16/2010 13:14:38 58 & Page 58 IIA and ISACA Standards for the Professional Practice of Internal Auditing This chapter focuses on these important IIA and ISACA standards for IT auditing. Some other, but often less significant, IT audit–related standards in place that are not included in this chapter. For example, Chapter 31 outlines the American Society for Quality (ASQ) internal audit standards; the guidance is important but is effectively subservient to the IIA international standards. ASQ’s internal audit standards and its quality auditors represent a different dimension and discipline from the IIA’s approaches and standards. It also represents an area that must be better represented and understood in the overall world of internal auditing. Of course, a different set of standards also impacts auditors involved in external auditing—public accountants—whether financial or IT audit related. Outside of the Sarbanes-Oxley Act (SOx) rules highlighted in Chapter 1, public accounting IT audit standards are beyond this book’s scope. We also visit the IIA code of ethics for internal auditors, an important supporting foundation for internal auditors in today’s world of frequent questions regarding professional ethics. And we also consider the ISACA’s code of ethics. Most ISACA members are IT audit specialists who are often IIA or Certified Public Accountants (CPAs). The ISACA code of ethics places a special emphasis on their IT-related activities. The IIA’s International Standards for the Professional Practice of Internal Auditing represent a must-know set of information for all internal auditors today, whether they are IT specialists or not. Similarly, IT auditors should have a good understanding of the very complementary ISACA standards. Although any standards are evolving sets of rules that may not exactly reflect all industry practices at a point in time, the IIA and ISACA standards define a set of guidelines for internal auditors worldwide to follow in their service to management. The new IIA International Standards for the Professional Practice of Internal Auditing, summarized here, are available from the IIA.1 They represent important guidance for today’s internal auditor and should be in every internal auditor’s professional library. INTERNAL AUDITING’S INTERNATIONAL PROFESSIONAL PRACTICE STANDARDS Internal auditors work in a large variety of enterprises and are asked to perform internal audit reviews in a number of operational, financial, and IT-related areas. Despite this diversity, an enterprise audit committee and senior management expects internal auditors to perform reviews in a competent and consistent manner. Internal audits performed using a set of recognized standards are a key approach to meet those management expectations. As the premier and leading worldwide internal audit professional enterprise, the IIA develops and issues standards that define the basic practice of internal auditing. Its International Standards for the Professional Practice of Internal Auditing is designed to: & & Delineate basic principles for the practice of internal auditing. Provide a framework for performing and promoting a broad range of value-added internal audit activities. E1C03 09/16/2010 13:14:38 Page 59 Internal Auditing’s International Professional Practice Standards & & & 59 Establish the basis for the measurement of internal audit performance. Foster improved organizational processes and operations The IIA Standards aid in this process; they provide a guideline both for the audit committee and management to measure their internal auditors as well as for internal auditors to measure themselves. Although the overall emphasis of this book is on IT audit, security, and control issues, because the IIA Standards and the ISACA standards set some constraints on all internal audit activities, whether IT related or not, they both are discussed here. Background of the IIA Standards Internal auditing is a profession that developed its own standards and processes over the years. Its professional organization, the Institute of Internal Auditors, first issued standards for what was called the Professional Practice of Internal Auditing in 1978 with an objective ‘‘to serve the entire profession in all types of business, in various levels of government, and in all other enterprises where internal auditors are found . . . to represent the practice of internal auditing.’’ Prior to these 1978 Standards, the most authoritative document was called the Statement of Responsibilities of Internal Audit, a statement originally issued by the IIA in 1947 and revised many times over the years. The IIA Standards are developed by the IIA’s Professional Standards Committee based on members’ own professional expertise as well as comments from IIA members and other interested parties. Because of the diverse group of participants who developed the earlier Standards, their final language often had some overlap, compromise, and incompleteness. The early IIA Standards were often weak in IT-related audit issues; that area has been strengthened through supporting practice advisories and guides. All internal auditors today, as IIA members, are expected to follow these standards. Internal auditors may also come from other professional areas, such as banking or external audit firms, and many disciplines have their own professional standards that, generally, are not in conflict with the IIA Standards. They may use slightly different terminology, as will be discussed with the ISACA code of ethics, but must follow audit practices that generally fit under the IIA Standards. As a matter of practice, however, the IIA’s International Standards for the Professional Practice of Internal Auditing governs the work of internal auditors worldwide, and it is important to understand them. When there appears to be a conflict and when the individual questioning that conflict is working as an internal auditor, the IIA’s Standards take precedence over any other professional standards. The IIA has historically published its standards in a small publication known as the Red Book. With a changing world and impressions of the role of internal auditors, these Standards have changed. The older IIA Standards, up through the year 2004 releases, contained an almost impossible level of detail. For example, an older general auditing standard on audit follow-up procedures guidance for completed audits had 23 individual sub- or sub-subclarifying standards. The IIA Standards have always contained good general guidance, but those older Standards were often far too specific. For example, a substandard of that older section 440.01.12.a stated: the ‘‘Director of Internal Audit E1C03 09/16/2010 13:14:38 60 & Page 60 IIA and ISACA Standards for the Professional Practice of Internal Auditing must establish the procedures for the time frame in which audit report responses are required.’’ Although certainly a valid guideline, it does little good to tell an auditee that IIA Standards ‘‘require’’ that audit report responses must be delivered in 7 days if the response is that it will take 10 days. This is just one small example. Many of the older IIA Standards were almost too detailed for effective internal audit management. In addition, those older standards had little to say about IT-related internal controls issues. The IIA Standards of past years were complex, sometimes difficult to implement, and often followed only in a broad, haphazard manner. The current IIA Standards provide a much more realistic set of guidance materials to allow an internal audit function to perform effectively and efficiently. There was a major change to the IIA Standards in 2004 with revisions that permitted internal auditors to act as in-house consultants as well as in their traditional attest role as auditors. These changes codified the supporting role of many IT auditors who, because of their specialized IT security and control knowledge, often acted as internal IT consultants to their enterprises. Earlier Standards, going back to the first days of internal auditing, prohibited internal auditors from acting as consultants. This prohibition was often ignored, but the 2004 revisions clarified this issue and divided IIA Standards into those covering internal audit assurance activities and others for internal audit consulting work. IIA’s Current Standards: What Has Changed IIA Standards are updated regularly to reflect current needs by both business and practicing internal audit professionals. For example, because of expressed professional concerns about the quality of some internal audit functions, in 2007 the standards were revised to add Attribute Standard 1312 on External Assessments: External assessments must be conducted at least once every five years by a qualified, independent reviewer or review team from outside the organization. The potential need for more frequent external assessments as well as the qualifications and independence of the external reviewer or review team, including any potential conflict of interest, must be discussed by the CAE [chief audit executive] with the Board. Such discussions must also consider the size, complexity and industry of the organization in relation to the experience of the reviewer or review team. This was a significant change to internal auditing standards. This new standard calls for a qualified external reviewing to visit an internal audit function to review its quality procedures and standards. The IIA publishes practice advisories on all of its standards, including this change. For external reviews, two practice advisories outline detailed requirements for reviewers; one requirement is that an external review team should consist of individuals who are competent in the professional practice of internal auditing and the external assessment process. The advisories cover all aspects of this significant change to internal audit practices. This 2007 IIA Standards requirement for independent quality assurance reviews was a major change to internal audit practices. Nevertheless, at the time of this E1C03 09/16/2010 13:14:38 Page 61 Content of the IPPF and the IIA International Standards & 61 publication, it has not been actively embraced by all internal audit functions. The onceevery-five-year requirement does not have strong beginning and end date requirements; nor does it make any allowances for the size of the enterprise. For example, when the SOx Section 404 requirements were first released, compliance dates were staggered by the size of the enterprise and whether it was domestic or foreign. However, the IIA has established peer review processes. Other internal auditors have volunteered to conduct independent quality assurance reviews, and an increasing number of consulting firms advertising through the Web are offering these services. All internal auditors should be aware of the quality review requirements. In January 2009, the IIA released a major revision to its standards. These changes are significant in that they revise the standards requirements from stating practice areas that an internal auditor should do to stating that an internal auditor must do something. For example and with the change emphasized in bold italics, the older Standard 1100 on internal auditor Independence and Objectivity stated: ‘‘The internal audit activity should be independent, and internal auditors should be objective in performing their work.’’ The new revisions state: ‘‘The internal audit activity must be independent, and internal auditors must be objective in performing their work.’’ The required change is significant whenever IT internal auditors state that their work is in compliance with internal auditing standards. The next sections provide a summary of the IIA international standards, with an emphasis on those have the most impact on IT auditors. These standards set the rules for all internal audit activity, whether the auditor is performing internal audits, serving as an internal consultant, or managing the overall internal audit function. CONTENT OF THE IPPF AND THE IIA INTERNATIONAL STANDARDS The IIA’s IPPF, as shown in Exhibit 3.1, is divided into sets of mandatory and strongly recommended guidance materials. The mandatory standards are the upper three sectors of this diagram starting with audit definitions, the International Standards, the IIA code of ethics all on top. These top segments define the he basic definition of internal auditing: Internal auditing is an independent, objective assurance and consulting activity designed to add value and improve an organization’s operations. It helps an organization accomplish its objectives by bringing a systematic, disciplined approach to evaluate and improve the effectiveness of risk management, control, and governance processes. This definition defines what internal auditors do on a very high level. Although this basis standard has been in place for many years, some managers historically looked at their internal auditors as little more than accounting clerks or assistants to their external auditors. With its emphasis on being an objective assurance and consulting activity, this definition defines internal auditor tasks and expectations. E1C03 09/16/2010 13:14:38 62 & Page 62 IIA and ISACA Standards for the Professional Practice of Internal Auditing Mandatory Guidance International Standards Audit Definitions Code of Ethics IPPF Audit Guides Position Papers Practice Advisories Strongly Recommended EXHIBIT 3.1 IIA International Professional Practices Framework The lower three segments of the exhibit show three strongly recommended internal audit standards: position papers, practice advisories, and practice guides. These IIA published materials describe approaches to assist internal auditors in employing IIA Standards. The internal audit International Standards consist of Attribute and Performance Standards. The Attribute Standards address the characteristics of enterprises and parties performing internal audit activities, while the Performance Standards describe the nature of internal audit activities and provide quality criteria against which the performance of these services can be evaluated. These Attribute and Performance Standards apply to all internal audit services; other assurance and consulting Implementation Standards apply to specific types of engagements. This split in types of internal audit standards reflects the fact that internal auditors sometimes do strictly audit assurance projects, such as reviewing internal control effectiveness in some area, and sometimes do work related to internal audit consulting. The Attribute Standards are numbered in sections as part of the number 1000 Series of Standards, while Performance Standards are classified in the 2000 Series. Implementation Standards, further designated as (A) for assurance and coded with an ‘‘A’’ following the standard number (e.g., 1130.A1) or (C) for consulting and noted by a ‘‘C’’ following the standard number (e.g., nnnn.C1), are organized under each of these E1C03 09/16/2010 13:14:38 Page 63 Content of the IPPF and the IIA International Standards & 63 Attribute and Performance standards. The next sections describe the Attribute and Performance standards in some detail as well as some of the descriptive Implementation Standards. Recognizing that internal auditors may be asked to review internal controls or to act more as internal consultants, multiple sets of Implementation Standards may apply: a set for each of the major types of internal audit activity. Exhibits 3.2 and 3.3 summarize the IIA Attribute and Performance standards by their numbers along with brief descriptions. Our objective here is not to just reproduce all of these IIA published standards but to describe some that are more significant to IT auditors. All internal auditors, however, should obtain a copy of the standards from the IIA and develop a good understanding of their contents. Knowledge of these standards is a requirement for all internal auditors, and the IIA Web site (theiia.org) is an official source for these IIA internal audit standards. Internal Audit Attribute Standards Attribute Standards address the characteristics of enterprises and individuals performing internal audit activities. Numbered from paragraph 1000 to 13000, they cover broad areas that define the attributes or characteristics of today’s internal auditor. Here, and for the Performance Standards later in the chapter, we list and describe selected standards (listed by their standards paragraph numbers), and offer comments when appropriate. 1000—Purpose, Authority, and Responsibility. The purpose, authority, and responsibility of the internal audit activity should be formally defined in an internal audit charter, consistent with these standards, and approved by the board of directors. Separate Implementation Standards state that internal auditing assurance and consulting services must be defined in the internal audit charter. This standard is supported by substandards defining an internal auditor’s purpose, authority, and responsibility roles when serving in assurance and in consulting roles. 1010—Recognition of the Definition of Internal Auditing. This standard was released in 2009 and emphasizes the importance of the IIA Standards, the code of ethics, and an internal audit charter in outlining internal audit’s roles and responsibilities. This is the type of standard that an internal auditor can use when about defining internal audit’s role in an enterprise. Audit charters are a key component that defines an internal audit group.2 1100—Independence and Objectivity. The internal audit activity must be independent, and internal auditors must be objective in performing their work. Subsections discuss the importance of both individual and organizational objectivity as well as the need to disclose any impairment to internal audit independence or objectivity. 1110—Organizational Independence. Although the IIA International Standards do not specify that internal audit should report to the audit committee, that reporting relationship must be free from any interference in determining the scope of internal auditing, performing work, and communicating results. We often think of internal audit as a key component in today’s SOx-defined E1C03 09/16/2010 13:14:39 64 & Page 64 IIA and ISACA Standards for the Professional Practice of Internal Auditing Standard # Description 1000 Purpose, Authority and Responsibility 1000.A1 Purpose, Authority and Responsibility for Assurance 1000.C1 Purpose, Authority and Responsibility for Consulting 1010 Recognition of the Definition of Internal Auditing, the Code of Ethics, and the Standards in the Internal Audit Charter 1100 Independence and Objectivity 1110 Organizational Independence 1110.A1 Direct Interaction with the Board 1120 Individual Objectivity 1130 Impairment to Independence or Objectivity 1130.A1 Impairment due to former responsibilities 1130.A2 Audit of functions for which CAE is responsible 1130.C1 Scope of impairment for consulting 1130.C2 Disclosure of impairment when consulting 1200 Proficiency and Due Professional Care 1210 Proficiency 1210.A1 CAE acquiring necessary assurance engagement competencies 1210.A2 Identification of fraud indicators 1210.A3 Information technology risk controls and tools 1210.C1 CAE acquiring necessary consulting engagement competencies 1220 1230 1300 EXHIBIT 3.2 Interference 1111 Due Professional Care 1220.A1 Scoping for assurance engagements 1220.A2 Use of technology-based audit techniques 1220.A3 Risk identification 1220.C1 Scoping for consulting engagements Continuing Professional Development Quality Assurance and Improvement Program 1310 Requirements for Quality Assurance and Improvement Program 1311 Internal Assessment 1312 External Assessment 1320 Reporting on the Quality Assurance and Improvement Program 1321 Use of conformance with IIA Standards 1322 Disclosure of Nonconformance IIA Attribute Standards Summary E1C03 09/16/2010 13:14:39 Page 65 Content of the IPPF and the IIA International Standards Standard # 2000 & 65 Description Managing the Internal Audit Activity 2010 Planning 2010.A1 2010.C1 Annual risk assessment Acceptance of consulting engagements 2020 Communication and Approval 2030 Resource Management 2040 Policies and Procedures 2050 Coordination 2060 Reporting to Sr. Management and the Board 2100 Nature of Work 2110 Governance 2110.A1 Evaluation of ethics programs 2110.A2 Assessing information technology governance 2110.C1 2120 Consistency of ethics and values when consulting Risk Management 2120.A1 Evaluating organization’s risk exposure 2120.A2 Evaluating fraud risks 2120.C1 Reviewing risks during consulting 2120.C2 Risk knowledge obtained during consulting 2120.C3 Limitations of involvement in risk management 2130 Control 2130.A1 Evaluating the adequacy and effectiveness of controls 2130.A2 Assessing achievement of goals and objectives 2130.A3 Assessing consistency of results with goals and objectives 2130.C1 Reviewing controls when consulting 2130.C2 Knowledge of controls gained from consulting engagements 2200 Engagement Planning 2201 Planning Considerations 2201.A1 2201.C1 2210 Agreement with clients on engagement scope and objectives Engagement Objectives 2210.A1 Preliminary assessment of risks 2210.A2 Probability of significant errors and other exposures 2210.A3 Setting criteria to evaluate controls 2210.C1 Focusing consulting on governance, risk management and control 2220 Engagement Scope 2220.A1 Scope of assurance engagement 2220.A2 Consulting opportunities during assurance engagement 2220.C1 2230 Planning engagements with external parties Scope of consulting engagement Engagement Resource Allocation (Continued ) EXHIBIT 3.3 IIA Performance Standards Summary E1C03 09/16/2010 13:14:40 66 & Page 66 IIA and ISACA Standards for the Professional Practice of Internal Auditing Standard # Description 2240 Engagement Work Programs 2240.A1 Procedure for managing information 2240.C1 Work program for consulting engagements 2300 Performing the Engagement 2310 Identifying Information 2320 Analysis and Evaluation 2330 Documenting Information 2330.A1 Controlling access to engagement records 2330.A2 Retention requirements 2330.C1 2340 Information retention and release policies for consulting Engagement Supervision 2400 Communicating Results 2410 Criteria for Communicating 2410.A1 Final communications of engagement results 2410.A2 Acknowledgement of satisfactory performance 2410.A3 Releasing results to parties outside the organization 2410.C1 Communicating results of consulting engagements 2420 Quality of Communications 2421 Errors and Omissions 2430 Use of conformance with IIA Standards 2431 Engagement Disclosure of Nonconformance 2440 Disseminating Results 2440.A1 CAE responsibility for communication of results 2440.A2 Assessment of conditions for releasing results to outsiders 2440.C1 CAE responsibility for communication of consulting results 2440.C2 Communication of significant issues identified when consulting 2500 Monitoring Progress 2500.A1 Establishing a follow-up process 2500.C1 Monitoring exposition of results for consulting engagements 2600 EXHIBIT 3.3 Resolution of Senior Management’s Acceptance of Risk (Continued ) corporate world with board audit committees, but internal audit can operate in many different international locations or for many different types of enterprises. Whether serving a not-for-profit organization in the United States or a governmental agency in a developing country, internal audit always must exhibit organizational independence. 1120—Individual Objectivity. This standard repeats a basic principle of internal auditing: Internal auditors must have an impartial, unbiased attitude and avoid conflicts of interest. 1130—Impairments to Independence or Objectivity. If internal audit’s independence or objectivity is impaired in fact or appearance, the details of E1C03 09/16/2010 13:14:40 Page 67 Content of the IPPF and the IIA International Standards & 67 the impairment must be disclosed as part of the audit work. The impairment could be a management-imposed impairment or one due to the background of or other circumstances regarding an individual internal auditor. There are several Assurance and Consulting attribute standards here, but one summarizes this standard: 1130.A1 and 1130.C1—Internal auditors should refrain from assessing specific operations for which they were previously responsible. Objectivity is presumed to be impaired if an internal auditor provides either assurance or consulting services for an activity for which the internal auditor had responsibility within the previous year. These first standards are important. Because of their specialized knowledge, internal auditors sometimes are asked to go back to the group where they once worked to audit it or act as a consultant. No matter how impartial they may try to act, others will not view them as objective. 1200—Proficiency and Due Professional Care. Engagements must be performed with proficiency and due professional care. There are two important proposed new Implementation Standards here: 1210.A1—The chief audit executive should obtain competent advice and assistance if the internal audit staff lacks the knowledge, skills, or other competencies needed to perform all or part of the engagement. 1210.A2—An internal auditor must have sufficient knowledge to identify the indicators of fraud and the manner in which it is managed by the organization, but the auditor is not expected to have the expertise of a person whose primary responsibility is detecting and investigating fraud. Even though many IT auditors are not regularly involved in fraud-related issues, this fraud-related guidance is weak. The American Institute of Certified Public Accountants (AICPA) Statement of Accounting Standards No. 99 external audit standard requires external auditors to aggressively think about ‘‘red flags,’’ that are indicators that might include the possibility of fraud, as well as to look for potential fraud in the course of their audits. Although this AICPA guidance is certainly not an IIA professional standards decision, internal auditors should maintain a greater awareness about the possibility of fraud in the course of their internal audits. IT internal auditors are often the best investigators to find these circumstances. Chapter 21 discusses such fraud detection and prevention activities from an IT audit perspective. 1210.A3—Internal auditors must have sufficient knowledge of key IT risks and controls and available technology-based audit techniques to perform their assigned work. However, not all internal auditors are expected to have the expertise of an IT or technical auditor whose primary responsibility is IT auditing. This standard on the importance of IT auditing provides support to many other issues discussed throughout this book. 1210.C1—The CAE must decline consulting engagements or obtain competent advice and assistance if the internal auditors lack the knowledge, skills, or other competencies needed to perform all or part of a consulting engagement. Recognizing there is a need for IT audit specialists, the A1 and A3 Assurance E1C03 09/16/2010 13:14:40 68 & Page 68 IIA and ISACA Standards for the Professional Practice of Internal Auditing Standards as well as C1 covering consulting are all important in IT-related environments. This standard says that all internal auditors must have sufficient knowledge of key IT risks and controls and available technology-based audit techniques to perform their assigned work (emphasis added). All internal auditors should have some level of IT-controls related knowledge and should reserve their more technical issues for IT audit specialists. If an internal audit function lacks sufficient technical knowledge, the CAE should ensure that all members have that knowledge, through training, or bring in consultants to provide support. 1220—Due Professional Care. Internal auditors must use due professional care and act in a reasonably prudent and competent manner when performing all internal audit activities. Due professional care does not imply infallibility. Another section of this standard states that in exercising due professional care, an internal audit must consider: & Extent of work needed to achieve the engagement’s objectives. & Relative complexity, materiality, or significance of matters to which assurance procedures are applied. & Adequacy and effectiveness of risk management, control, and governance processes. & Probability of significant errors, irregularities, or noncompliance. & Cost of assurance in relation to potential benefits. This IIA standard really says that an internal auditor must be cautious in beginning and performing an internal audit. The first of the previous bullet points regarding the extent of work says that an internal auditor, for example, must perform an adequate level of investigation and testing before making a final audit recommendation. Computer-assisted audit tools and techniques are discussed as part of the new standards released in 2004: 1220.A2—In exercising due professional care, the internal auditor must consider the use of technology based audit tools and other data analysis techniques. This guidance very much relates to the importance of computer-assisted audit tools and techniques discussed in Chapter 13. 1220.A3—The internal auditor must be alert to the significant risks that might affect objectives, operations, or resources. However, assurance procedures alone, even when performed with due professional care, do not guarantee that all significant risks will be identified. As discussed in Chapter 4 on risk management, an understanding of risk assessment techniques is an important knowledge area for internal auditors. This guidance has been part of the IIA Standards going back to the early versions and should be part of an internal auditor’s procedures. The standards continue in this section with 1230—Continuing Professional Development, a standard on the requirement for continuing professional education and development. 1300—Quality Assurance and Improvement Program. An internal audit function must develop and maintain a quality assurance and improvement program that covers all aspects of activities and continuously monitors its effectiveness. The quality assurance and improvement program must be designed to enable an evaluation of the internal audit activity’s conformance with Internal Auditing Standards as well as an evaluation of whether internal auditors apply the code of E1C03 09/16/2010 13:14:40 Page 69 Content of the IPPF and the IIA International Standards & 69 ethics. The program also assesses the efficiency and effectiveness of the internal audit activity and identifies opportunities for improvement. The standards here call for both internal and external quality reviews and emphasize the importance of good-quality assurance processes within internal audit. Quality assurance activities as well as quality audits are discussed in Chapter 31. Six supporting standards follow 1300, but the two more important of these standards for IT auditors here are: 1311—Internal Assessments—Internal audit management must have internal assessment processes in place that include both the ongoing monitoring of the performance of the internal audit activity and periodic reviews performed through self-assessment or by other persons within the enterprise with sufficient knowledge of internal audit practices. 1312—External Assessments—As discussed previously, external assessments must be conducted at least once every five years by a qualified, independent reviewer or review team from outside the organization. The CAE must discuss this need for more frequent external assessments with the board audit committee and the qualifications and independence of the external reviewer or review team, including any potential conflicts of interest. There are four additional substandards under 1300 defining other areas on the quality of internal audit work. For example, 1321 provides guidance on what an internal auditor can report regarding conformance these IIA Standards. Internal Audit Performance Standards Performance standards describe the nature of internal audit activities and provide quality criteria against which these services can be measured. There are six Performance Standards, outlined in Exhibit 3.3, along with substandards and Implementation Standards that apply to compliance audits, fraud investigations, and control selfassessment projects. We are only summarizing the standard here for the purpose of describing internal audit processes. Interested IT audit professionals should contact the IIA Web site to obtain these full standards. 2000—Managing the Internal Audit Activity. The CAE must effectively manage the internal audit activity to ensure it adds value to the enterprise. This standard covers six sub-Standards covering Planning, Communication and Approval, Resource Management, Policies and Procedures, Coordination, and Reporting to the Board and Senior Management. These sub-Standards generally describe such good internal audit management practices as 2040 on Policies and Procedures stating that the CAE must establish such guides. 2060—Reporting to the Board and Senior Management. This substandard contains guidance applicable to today’s SOx rules: ‘‘The chief audit executive should report periodically to the board and senior management on the internal audit activity’s purpose, authority, responsibility, and performance relative to its plan. Reporting must also include significant risk exposures and control issues, corporate governance issues, and other matters needed or requested by E1C03 09/16/2010 13:14:40 70 & Page 70 IIA and ISACA Standards for the Professional Practice of Internal Auditing the board and senior management. Reporting must also include significant risk exposures and control issues, including fraud risks, governance issues, and other matters needed or requested by senior management and the board.’’ 2100—Nature of Work. Internal audit activity includes evaluations and contributions to the improvement of risk management, control, and governance systems using ‘‘a systematic and disciplined approach.’’ Earlier IIA Standards did not really address the important area of risk management, further discussed in Chapter 4 and outlined next. 2110—Governance. Internal audit must assess and make appropriate recommendations for improving the governance process in its accomplishment of these objectives: & Promoting appropriate ethics and values within the enterprise & Ensuring effective organizational performance management and accountability & Communicating risk and control information to appropriate areas of the enterprise & Coordinating the activities of and communicating information among the board, external and internal auditors, and management 2120—Risk Management. Internal audit must assist the enterprise by identifying and evaluating significant exposures to risk and contributing to the improvement of risk management and control systems. Determining whether risk management processes are effective is a judgment resulting from an internal auditor’s assessment that: & Organizational objectives support and align with an enterprise’s mission. & Significant risks are identified and assessed. & Appropriate risk responses are selected that align risks with the enterprise’s risk appetite. & Relevant risk information, enabling staff, management, and the board to carry out their responsibilities, is captured and timely communicated across the enterprise. Risk management processes should be monitored through ongoing management activities, separate evaluations, or both. 2120.A1—Internal audit activity must monitor and evaluate the effectiveness of the enterprise’s risk management system. 2120.A2—The internal audit activity must evaluate risk exposures relating to the enterprise’s governance, operations, and IT regarding the Committee of Sponsoring Organizations (COSO) standards of internal control. 2120.C1—During consulting engagements, internal auditors must address risk consistent with the engagement’s objectives and must be alert to the existence of other significant risks. 2120.C2—Internal auditors must incorporate knowledge of risks gained from consulting engagements into the process of identifying and evaluating significant risk exposures of the enterprise. 2120.C3—When assisting management in establishing or improving risk management processes, internal auditors must refrain from assuming any management responsibility by actually managing risks. E1C03 09/16/2010 13:14:40 Page 71 Content of the IPPF and the IIA International Standards & 71 Substandard 2130 on Control follows. Although not summarized here, this IIA control standard is very consistent with the SOx internal controls review requirements discussed in Chapter 1. A major function of internal audit activities includes evaluating the effectiveness and adequacy of internal controls. 2200—Engagement Planning. Internal auditors must develop a plan for each audit engagement, including the scope, objectives, timing, and resource allocations. An important aspect of all internal audits, planning is discussed in Chapter 5 on the key elements for performing effective IT audits. 2201—Planning Considerations. In planning an audit engagement, internal auditors should consider: & The objectives of the activity being reviewed and the means by which the activity controls its performance. & The significant risks to the activity, its objectives, resources, and operations and the means by which the potential impact of risk is kept to an acceptable level. & The adequacy and effectiveness of the activity’s risk management and internal control processes compared to a relevant control framework or model. & The opportunities for making significant improvements to the activity’s risk management and control processes. 2201.A1—When planning an engagement for parties outside the enterprise, internal auditors must establish a written understanding with those outside parties about objectives, scope, respective responsibilities and other expectations, including restrictions on distribution of the results of the engagement and access to engagement records. 2201.C1—Internal auditors must establish, and generally document, an understanding with consulting engagement clients about objectives, scope, respective responsibilities, and other client expectations. For significant engagements, this understanding must be documented. 2210 Engagement Objectives. This almost naturally assumed requirement states that the objectives must be established for each audit engagement. Internal auditors should never go on fishing expeditions, looking around for problems with no clear objectives. 2210.A1—Internal auditors must conduct a preliminary assessment of the risks relevant to the activity under review, and engagement objectives must reflect the results of this assessment. 2210.A2—The internal auditor must consider the probability of significant errors, irregularities, noncompliance, and other exposures when developing the engagement objectives. The standard relates to the risk assessment considerations discussed previously. 2210.A3—Adequate criteria are needed to evaluate controls. Internal auditors must ascertain the extent to which management has established adequate criteria to determine whether objectives and goals have been accomplished. If adequate, internal auditors must use such criteria in their evaluation. If inadequate, internal auditors must work with management to develop appropriate evaluation criteria. E1C03 09/16/2010 13:14:40 72 & Page 72 IIA and ISACA Standards for the Professional Practice of Internal Auditing 2210.C1—Consulting engagement objectives must address risks, controls, and governance processes to the extent agreed on with the client. 2220—Engagement Scope. The established audit scope must be sufficient to satisfy the objectives of the engagement. 2220.A1—The scope of the engagement must include consideration of relevant systems, records, personnel, and physical properties, including those under the control of third parties. 2220.A2—If significant consulting opportunities arise during an assurance engagement, a specific written understanding as to the objectives, scope, respective responsibilities, and other expectations must be reached and the results of the consulting engagement communicated in accordance with these consulting standards. This substandard says that an internal auditor can begin an audit as a strictly assurance level of review but may expand it to a consulting-level audit if there is a need or management request. 2220.C1—In performing consulting engagements, internal auditors must ensure that the scope of the engagement is sufficient to address the agreed-on objectives. If internal auditors develop reservations about the scope during the engagement, they must discuss these reservations with the auditee to determine whether to continue with the engagement. 2230—Engagement Resource Allocation. Internal auditors must determine the appropriate resources necessary to achieve the audit engagement objectives. Staffing must be based on an evaluation of the nature and complexity of each engagement, time constraints, and available resources. 2240—Engagement Work Program. Internal auditors must develop and document work programs that achieve the engagement objectives. These work programs must establish procedures for identifying, analyzing, evaluating, and recording information during the engagement. They must be approved prior to their implementation, and any adjustments must be approved promptly. Work programs for consulting engagements may vary in form and content depending on the nature of the engagement. 2300—Performing the Engagement. Internal auditors must identify, analyze, evaluate, and record sufficient information to achieve an audit engagement’s objectives and must base conclusions and engagement results on appropriate analyses and evaluations. 2310—Identifying Information. Internal auditors must identify sufficient, reliable, relevant, and useful information to achieve the engagement’s objectives. Sufficient information is factual, adequate, and convincing so that a prudent, informed person would reach the same conclusions as the auditor. Reliable information is the best attainable information through the use of appropriate engagement techniques. Relevant information supports engagement observations and recommendations and is consistent with the objectives for the engagement. Useful information helps an enterprise meet its goals. 2320—Analysis and Evaluation. Internal auditors must base conclusions and engagement results on appropriate analyses and evaluations. 2330—Recording Information. Internal auditors must record relevant information to support the conclusions and engagement results. E1C03 09/16/2010 13:14:40 Page 73 Content of the IPPF and the IIA International Standards & 73 2330.A1—The CAE must control access to engagement records and must obtain the approval of senior management and/or legal counsel prior to releasing such records to external parties, as appropriate. 2330.A2—The CAE must develop retention requirements for engagement records, regardless of the medium in which each record is stored, that are consistent with the enterprise’s guidelines and any pertinent regulatory or other requirements. 2330.C1—The CAE must develop policies governing the custody and retention of engagement records as well as their release to internal and external parties. These policies must be consistent with the enterprise’s guidelines and any pertinent regulatory or other requirements. 2340—Engagement Supervision. Engagements must be properly supervised to ensure that objectives are achieved, quality is assured, and staff is developed. The extent of supervision required will depend on the proficiency and experience of internal auditors and the complexity of the engagement. The CAE has overall responsibility for supervising the engagement, whether performed by or for the internal audit function, but may designate appropriately experienced members of the internal audit function to perform the review. Appropriate evidence of this supervision is documented and retained. 2400 and 2410 Communicating Results. Internal auditors must communicate their engagement results including the audit’s objectives and scope as well as applicable conclusions, recommendations, and action plans as well as the internal auditor’s overall opinion and or conclusions. 2410.A1—Final communication of engagement results must, where appropriate, contain the internal auditor’s overall opinion and/or conclusions. 2410.A2—Internal auditors are encouraged to acknowledge satisfactory performance in engagement communications. 2410.A3—When releasing engagement results to parties outside the enterprise, the communication must include limitations on distribution and use of the results. 2410.C1—Communication of the progress and results of consulting engagements will vary in form and content depending on the nature of the engagement and the needs of the client. 2420—Quality of Communications. Communications must be accurate, objective, clear, concise, constructive, complete, and timely. 2421—Errors and Omissions—If a final communication contains a significant error or omission, the CAE must communicate corrected information to all parties who received the original communication. 2430—Use of the expression ‘‘Conducted in conformance with the International Standards for the Professional Practice of Internal Auditing.’’ Internal auditors are encouraged to report that their engagements are ‘‘conducted in conformance with the International Standards for the Professional Practice of Internal Auditing.’’ However, internal auditors may use that statement only if the results of the quality assurance and improvement program demonstrate that the internal audit activity conforms to the standards. E1C03 09/16/2010 13:14:40 74 & Page 74 IIA and ISACA Standards for the Professional Practice of Internal Auditing 2431—Engagement Disclosure of Noncompliance with IIA Standards—When noncompliance with the Standards impacts a specific engagement, communication of the results must disclose the: & Principle or rule of conduct of the code of ethics or standard(s) with which full conformance was not achieved. & Reason(s) for noncompliance. & Impact of noncompliance on the engagement. 2440—Disseminating Results. The CAE is responsible for communicating the final results of audit work to appropriate parties who can ensure that the results are given due consideration. 2440,A1—The CAE is responsible for communicating the final results to parties who can ensure that the results are given due consideration. 2440.A2—If not otherwise mandated by legal, statutory, or regulatory requirements, prior to releasing results to parties outside the enterprise, the CAE must: & Assess the potential risk to the enterprise. & Consult with senior management and/or legal counsel as appropriate. & Control dissemination by restricting the use of the results. 2440.C1 and C2—The CAE is responsible for communicating the final results of consulting engagements to clients. During consulting engagements, risk management, control, and governance issues may be identified. Whenever these issues are significant to the enterprise, they must be communicated to senior management and the board. 2500—Monitoring Progress. The CAE must establish and maintain a system to monitor the disposition of results communicated to management as well as a followup process to monitor and ensure that management actions have been effectively implemented or that senior management has accepted the risk of not taking action. 2600—Resolution of Management’s Acceptance of Risks. When the CAE believes that senior management has accepted a level of residual risk that may be unacceptable to the enterprise, the CAE must discuss the matter with senior management. If the decision regarding residual risk is not resolved, the CAE and senior management must report the matter to the board for resolution. The current IIA Standards represent a significant improvement over the older and very lengthy standards that were in place through the 1990s. The Standards conclude with a glossary of terms to better define the roles and responsibilities of internal auditors. Various glossary terms are introduced in other chapters but one that is important for internal auditors is the definition of independence. The word frequently appears in internal auditing literature, but the IIA’s official definition of internal auditor independence is: Independence is the freedom from significant conflicts of interest that threaten objectivity. Such threats to objectivity must be managed at the individual auditor level, the engagement level, and the organizational level. This is an important concept for today. We again emphasize that these past paragraphs are not the verbatim IIA Standards but an edited and annotated version. E1C03 09/16/2010 13:14:40 Page 75 Strongly Recommended IIA Standards Guidance & 75 Some of the more minor standards statements have been eliminated in this chapter, a few words in some cases have been slightly changed, and descriptive comments have been added. As previously stated, internal auditors are advised to obtain the official version of these standards through the Institute of Internal Auditors. STRONGLY RECOMMENDED IIA STANDARDS GUIDANCE IIA Standards as well as the code of ethics are not optional or best practices recommendations, such as the information technology infrastrucuture library IT service life cycle best practices discussed in Chapter 7. Professionals who are members of the IIA are expected to follow these standards as a condition of their professional IIA membership. As discussed, in the early days of the IIA Standards, they were often just sort-of followed. Newly revised and streamlined, they represent mandatory standards for the completion of internal audits today. As shown in Exhibit 3.1, the IPPF framework has both a mandatory section—the upper three segments of the chart—and also a section called ‘‘Strongly Recommended,’’ consisting of position papers, practice advisories, and practice guides. These materials provide guidance and interpretations for IIA members. IIA Standards Practice Advisories IIA practice advisories are designed to assist internal auditors in applying the definition of internal auditing, the code of ethics, and the standards and promoting good practices. Practice advisories address internal auditing approaches, methodologies, and other considerations, but not detailed processes or procedures. They include practices relating to international, country, or industry-specific issues; specific types of engagements; and legal or regulatory issues. The practice advisories take the often very general words found in the standards and add some interpretations and clarifications. Exhibit 3.4 shows an example practice advisory for substandard 2130.A1-1 on Information Reliability and Integrity. These practice advisories are updated and released from time to time when the IIA membership raises questions about some matter. IIA Standards Position Papers The objective of the standards position papers is to assist a wide range of interested parties, including those not in the internal audit profession, in understanding significant governance, risk, or control issues and delineating related roles and responsibilities of internal auditing. This is a new standards initiative; to date, only two papers have been issued. The objective of these position papers is to describe the role of internal auditing in some specialized area. For example, a position paper on the role of internal auditing in enterprise in risk management (ERM) outlines areas that should be core internal audit roles with regard to ERM, areas that can be legitimate areas for internal audit concern, and other roles in this area that internal audit should not undertake. This position paper is referenced in Chapter 4 on understanding risk management, and it contains some good guidance materials. E1C03 09/16/2010 13:14:40 76 & Page 76 IIA and ISACA Standards for the Professional Practice of Internal Auditing Practice Advisory 2130.A1-1: Information Reliability and Integrity Primary Related Standard 2130.A1—The internal audit activity must evaluate the adequacy and effectiveness of controls in responding to the risks within the organization’s governance, operations, and information systems regarding the: Reliability and integrity of financial and operational information; Effectiveness and efficiency of operations; Safeguarding of assets; and Compliance with laws, regulations, and contracts. 1. 2. 3. 4. 5. Internal auditors determine whether senior management and the board have a clear understanding that information reliability and integrity is a management responsibility. This responsibility includes all critical information of the organization regardless of how the information is stored. Information reliability and integrity includes accuracy, completeness, and security. The chief audit executive (CAE) determines whether the internal audit activity possesses, or has access to, competent audit resources to evaluate information reliability and integrity and associated risk exposures. This includes both internal and external risk exposures, and exposures relating to the organization’s relationships with outside entities. The CAE determines whether information reliability and integrity breaches and conditions that might represent a threat to the organization will promptly be made known to senior management, the board, and the internal audit activity. Internal auditors assess the effectiveness of preventive, detective, and mitigation measures against past attacks, as appropriate, and future attempts or incidents deemed likely to occur. Internal auditors determine whether the board has been appropriately informed of threats, incidents, vulnerabilities exploited, and corrective measures. Internal auditors periodically assess the organization’s information reliability and integrity practices and recommend, as appropriate, enhancements to, or implementation of, new controls and safeguards. Such assessments can either be conducted as separate stand-alone engagements or integrated into other audits or engagements conducted as part of the internal audit plan. The nature of the engagement will determine the most appropriate reporting process to senior management and the board. EXHIBIT 3.4 IIA Practice Advisory Example: Information Reliability and Integrity Source: Issued January 2009, PA 2130.A1-1 Revised. ß 2009 The Institute of Internal Auditors. IIA Standards Practice Guides Practice guides provide detailed guidance for conducting internal audit activities. They include detailed processes and procedures, such as tools and techniques, programs, and step-by-step approaches, as well as examples of deliverables. At the time of this publication, these guides are published as a series of Global Technology Audit Guides, with multiple titles relating to IT audit, such as ‘‘Developing the IT Audit Plan.’’ ISACA IT AUDITING STANDARDS OVERVIEW It has been mentioned several times how the IIA has released standards-related guidance since its founding in the mid-twentieth century but that guidance and even the emphasis of that professional organization was perceived to be weak in IT E1C03 09/16/2010 13:14:40 Page 77 ISACA IT Auditing Standards Overview & 77 audit and control related areas, at least in its early years. The launch of what is now ISACA corrected matters for IT audit professionals. Almost from its start, this organization started releasing IT audit guidance first as audit checklists, established the Certified Information Systems Auditor certification discussed in Chapter 30, and then the CobiT framework discussed in Chapter 2. ISACA has published extensive IT audit guidance materials in addition to ongoing updated versions of the CobiT framework, but until 2008, it never published any auditing standards. Beginning in that year, ISACA began to issue preliminary standards, which have now evolved into a full suite of IT audit standards. In many respects, the specialized nature of IT auditing and the skills necessary to perform appropriate IT audits require standards that go beyond the IIA Standards just discussed and apply specifically to IT auditing. As a result, ISACA has since issued standards, guidelines, and procedures that define mandatory requirements for information systems (IS) auditing and reporting. These standards provide IT auditors with a minimum level of acceptable performance required to meet the professional responsibilities set out in the ISACA code of professional ethics, discussed in the next section, and provide management and other interested parties with the profession’s expectations concerning the work of IT audit practitioners. Just as compliance with the IIA Standards is mandatory for IIA members, CISA holders are expected to comply with these ISACA standards. Failure to comply may result in an investigation into the CISA holder’s conduct by the ISACA board of directors or appropriate ISACA committee and, ultimately, in disciplinary action. Exhibit 3.5 is a list of the ISACA audit standards at the time of publication. The standards cover the overall internal audit process with an emphasis on IT-specific areas. Copies of these standards are available through the ISACA Web site (www.isaca.org), and the standards will be referred to in many specific audit areas throughout this book. To provide an example of the ISACA audit standards, Exhibit 3.6 is the standard on IT S1 Audit Charter 1 S2 Independence S3 Professional Ethics and Standards S4 Competence S5 Planning S6 Performance of Audit Work S7 Reporting S8 Follow-Up Activities S9 Irregularities and Illegal Acts S10 IT Governance S11 Use of Risk Assessment in Audit Planning S12 Audit Materiality S13 Using the Work of Other Experts S14 Audit Evidence S15 IT Controls S16 E-Commerce EXHIBIT 3.5 ISACA IT Audit Standards Summary E1C03 09/16/2010 13:14:41 78 & Page 78 IIA and ISACA Standards for the Professional Practice of Internal Auditing Introduction 01 ISACA standards contain the basic, mandatory principles and essential procedures together with related guidance. 02 The purpose of this ISACA standard is to establish standards and provide guidance regarding IT controls. 03 The IT auditor should evaluate and monitor IT controls that are an integral part of the internal control environment of the organization. 04 The IT auditor should assist management by providing advice regarding the design, implementation, operation, and improvement of IT controls. 05 Management is accountable for the internal control environment of an organization including IT controls. An internal control environment provides the discipline, framework, and structure for the achievement of the primary objective of the system of internal control. 06 CobiT defines control as ‘‘the policies, procedures, practices and organizational structures, designed to provide reasonable assurance that business objectives will be achieved and that undesired events will be prevented or detected and corrected.’’ Also, CobiT defines a control objective as ‘‘a statement of the desired result or purpose to be achieved by implementing control procedures in a particular process.’’ 07 IT controls are comprised of general IT controls, which include pervasive IT controls, detailed IT controls, and application controls, and refer to controls over the acquisition, implementation, delivery, and support of IT systems and services. 08 General IT controls are controls that minimize risk to the overall functioning of the organization’s IT systems and infrastructure and to a broad set of automated solutions (applications). 09 Application controls are a set of controls embedded within applications. 10 Pervasive IT controls are general IT controls that are designed to manage and monitor the IT environment and, therefore, affect all IT-related activities. They are a subset of general controls, being those general IT controls that focus on the management and monitoring of IT. 11 Detailed IT controls are made up of application controls plus those general IT controls not included in pervasive IT controls. 12 The IT auditor should use an appropriate risk assessment technique or approach in developing the overall IT audit plan and in determining priorities for the effective allocation of IT audit resources to provide assurance regarding the state of IT control processes. Control processes are the policies, procedures, and activities that are part of a control environment, designed to ensure that risks are contained within the risk tolerances established by the risk management process. 13 The IS auditor should consider the use of data analysis techniques including the use of continuous assurance, which allows IT auditors to monitor system reliability on a continuous basis and to gather selective audit evidence through the computer when reviewing IT controls. 14 When organizations use third parties, they can become a key component in an organization’s controls and its achievement of related control objectives. The IT auditor should evaluate the role that the third party performs in relation to the IT environment, related controls, and IT control objectives. Standard Commentary EXHIBIT 3.6 ISACA IT Audit Standards Example: S15 IT Controls Source: IT Standards, Guidelines, and Tools and Techniques for Audit Assurance and Control Professionals. ß 2008 ISACA. All rights reserved. Used by permission. E1C03 09/16/2010 13:14:41 Page 79 Codes of Ethics: The IIA and ISACA 15 & 79 The following ISACA and IT Governance Institute (ITGI) guidance should be referred to for further information regarding IT controls: & Guideline G3 Use of Computer-Assisted Audit Techniques (CAATs) & Guideline G11 Effect of Pervasive IS Controls & Guideline G13 Using Risk Assessment in Audit Planning & Guideline G15 Planning & Guideline G16 Effect of Third Parties on an Organization’s IT Controls & Guideline G20 Reporting & Guideline G36 Biometric Controls & Guideline G38 Access Controls & COBIT Framework and Control Objectives Operative Date 16 This ISACA standard is effective for IS audits beginning 1 February 2008. EXHIBIT 3.6 (Continued ) Controls. Although a general internal audit standard, it has a heavy emphasis on IT and relies frequently relies on CobiT framework. ISACA guidelines are similar to the IIA Standards practice advisories discussed earlier. They provide very specific guidance for IT audits in special areas, are released fairly regularly, and provide specific guidance in various more technical areas. In order to strengthen the ISACA IT audit standards process, these guidelines are referenced in specific ISACA standards. For example, Exhibit 3.6 on IT Controls concludes with a list of applicable guidelines, such as a reference to Guideline G16, ‘‘Effect of Third Parties on an Organization’s IT Controls.’’ The ISACA Auditing Procedures are similar to the IIA’s IPPF Position Papers, except that the ISACA documents tend to be much more technical and IT related, covering specific areas. For example, the ISACA Guideline P9 is titled ‘‘Evaluation of Management Controls over Encryption Methodologies’’ and covers a very specific and specialized IT controls area. Materials in these ISACA will be referred to in other chapters. For example, topics in this audit procedure will be referenced in Chapter 20 on cybersecurity. CODES OF ETHICS: THE IIA AND ISACA Codes of ethics are key elements in both the IIA and ISACA Standards. The purpose of the IIA’s code of ethics is to promote an ethical culture in the profession of internal auditing. This code of ethics is displayed in Exhibit 3.7. It is necessary and appropriate for a profession that depends on the trust placed on users of internal audit services on objective assurances about risk management, control, and governance. The IIA’s current code of ethics was released in 2009 and is based on the principles of internal auditor integrity, objectivity, confidentiality, and competency.3 These are the behavioral norms expected of internal auditors and are intended to guide the ethical conduct of all internal auditors. E1C03 09/16/2010 13:14:41 80 & Page 80 IIA and ISACA Standards for the Professional Practice of Internal Auditing Introduction to the Code of Ethics The purpose of The Institute’s Code of Ethics is to promote an ethical culture in the profession of internal auditing. Internal auditing is an independent, objective assurance and consulting activity designed to add value and improve an organization’s operations. It helps an organization accomplish its objectives by bringing a systematic, disciplined approach to evaluate and improve the effectiveness of risk management, control, and governance processes. A code of ethics is necessary and appropriate for the profession of internal auditing, founded as it is on the trust placed in its objective assurance about governance, risk management, and control. The Institute’s Code of Ethics extends beyond the Definition of Internal Auditing to include two essential components: 1. 2. Principles that are relevant to the profession and practice of internal auditing. Rules of Conduct that describe behavior norms expected of internal auditors. These rules are an aid to interpreting the Principles into practical applications and are intended to guide the ethical conduct of internal auditors. ‘‘Internal auditors’’ refers to Institute members, recipients of or candidates for IIA professional certifications, and those who perform internal audit services within the Definition of Internal Auditing. Applicability and Enforcement of the Code of Ethics This Code of Ethics applies to both entities and individuals that perform internal audit services. For IIA members and recipients of or candidates for IIA professional certifications, breaches of the Code of Ethics will be evaluated and administered according to The Institute’s Bylaws and Administrative Directives. The fact that a particular conduct is not mentioned in the Rules of Conduct does not prevent it from being unacceptable or discreditable, and therefore, the member, certification holder, or candidate can be liable for disciplinary action. EXHIBIT 3.7 IIA Code of Ethics Source: Issued January 2009 Code of Ethics Revised. ß 2009 The Institute of Internal Auditors. The IIA code of ethics replaces an earlier and also a rather lengthy 1988 version that had 11 specific articles defining preferred practices; that version, in turn, replaced a 1968 version with eight articles. The current 2000 version, with its highlighted emphasis on integrity, objectivity, confidentiality, and competency becomes much easier to understand and recognize than the rather detailed articles in the prior version. Any person performing internal audit services, whether or not a member of the IIA, should follow this code of ethics. Professional certificates, including the Certified Internal Auditor (CIA) designation, are discussed in Chapter 30. The IIA code of ethics applies to both individuals and entities that provide internal auditing services. For IIA members and recipients of or candidates for IIA professional certifications, breaches of the code of ethics will be evaluated and administered according to IIA bylaws and administrative guidelines. The IIA goes on to state that even if a particular conduct is not mentioned in this code, this does not prevent the conduct or practice from being unacceptable or discreditable. Violators of this code, whether an IIA member, a CIA, or a candidate, can be become liable for professional disciplinary action. As discussed, ISACA, as well as its affiliated research arm, the IT Governance Institute, is the professional audit enterprise that represents or speaks primarily for IT auditors. ISACA currently leads the IIA on technology-related issues. With its IT audit E1C03 09/16/2010 13:14:41 Page 81 Notes & 81 The Information Systems Audit and Control Association, Inc. (ISACA) sets forth this Code of Professional Ethics to guide the professional and personal conduct of members of the Association and/or its certification holders. Members and ISACA Certification holders shall: 1. 2. 3. 4. 5. 6. 7. Support the implementation of, and encourage compliance with, appropriate standards, procedures and controls for information systems. Perform their duties with due diligence and professional care, in accordance with professional standards and best practices. Serve in the interest of stakeholders in a lawful and honest manner, while maintaining high standards of conduct and character, and not engage in acts discreditable to the profession. Maintain the privacy and confidentiality of information obtained in the course of their duties unless disclosure is required by legal authority. Such information shall not be used for personal benefit or released to inappropriate parties. Maintain competency in their respective fields and agree to undertake only those activities, which they can reasonably expect to complete with professional competence. Inform appropriate parties of the results of work performed; revealing all significant facts known to them. Support the professional education of stakeholders in enhancing their understanding of information systems security and control. Failure to comply with this Code of Professional Ethics can result in an investigation into a member’s or certification holder’s conduct and, ultimately, in disciplinary measures. EXHIBIT 3.8 ISACA Code of Professional Ethics Source: IT Standards, Guidelines, and Tools and Techniques for Audit Assurance and Control Professionals. ß 2009 ISACA. All rights reserved. Used by permission. and IT governance orientation, ISACA represents a somewhat different group of auditors. Historically, ISACA drew a large number of members from IT audit specialists and public accounting external audit firms, and it has had a very strong international membership in some areas of the world. Many IIA members are also ISACA members, and while the two groups do not have many joint meetings or other endeavors, each represents an important segment of the audit community. Just as the IIA and ISACA each has its own audit standards, ISACA also has a code of ethics, as shown in Exhibit 3.8. Because of its IT heritage, the ISACA code is more oriented to technology-related issues. It is a set of professional standards that applies to and should be of particular value to IT audit professionals. Although the wording is different, there is nothing in the ISACA code that is really contrary to the IIA code. Internal auditors, whether working primarily in IT areas or with a more general internal controls orientation, should exercise strong ethical practice in their work. NOTES 1. Institute of Internal Auditors, Standards for the Professional Practice of Internal Auditing (Altamonte Springs, FL: Author, 2004). 2. Robert Moeller, Brink’s Modern Internal Auditing, 7th ed. (Hoboken, NJ: John Wiley & Sons, 2009). 3. www.theiia.org. E1C04 08/06/2010 13:58:13 Page 82 4 CHAPTER FOUR Understanding Risk Management through COSO ERM I N F O R M A T I O N T E C H N O L O G Y ( I T ) A N D other internal auditors need to identify all of the business risks they face in their review activities—IT, financial, and operational as well as social, ethical, and environmental risks—and to assess that these risks are managed at an acceptable level. Understanding risks is a major component of achieving Sarbanes-Oxley (SOx) compliance, as discussed in Chapter 1, and is part of the international audit standards of the Institute of Internal Auditors (IIA) and the Information Systems Audit and Control Association (ISACA) discussed in Chapter 3. Risk has too often been one of those terms where IT auditors and internal control specialists often say ‘‘Yes, we must consider risks!’’ even though their understandings and assessments of risk are not well understood or defined. One professional’s concept and understanding of risk may be very different from another’s, even though they are both working for the same enterprise and in similar areas. IT auditors need to have an understanding of risk management and how it impacts their approaches for assessing or developing effective internal controls. This chapter begins by discussing some of the fundamental concepts behind risk management for IT auditors, considers the various types of risk facing an enterprise, and then looks at quantitative approaches for assessing risks. We then introduce internal audit risk management standards, tools, and approaches as defined in ISACA and IIA guidance materials and standards. An ongoing problem in an IT auditor’s use and understanding of the concept of risk had been the lack of a consistent definition of what is really meant by risk. The word has some origins in the insurance industry, but the concept of risk has not been consistently used even in insurance areas, let alone by IT auditors or other business professionals. Many have 82 E1C04 08/06/2010 13:58:13 Page 83 Risk Management Fundamentals & 83 talked about how they had ‘‘considered risk’’ when implementing a control or process, but they often had no consistent definitions here. The question of what steps were followed in such a risk consideration might produce a wide range of answers. This all changed when the Committee of Sponsoring Organizations (COSO) released its enterprise risk methodology, COSO Enterprise Risk Management Integrated Format standards (COSO ERM).1 This is a framework to allow an enterprise to consider and assess its risks at all levels, whether it is in an individual area, such as for an IT development project, or for global risks regarding an international expansion. Because it was released by the same COSO guidance-setting function that is responsible for the COSO internal controls framework, COSO ERM sometimes looks like its internal controls ‘‘brother.’’ However, COSO ERM has a much different feel and approach. This chapter introduces the COSO ERM framework and its elements, with an emphasis on why COSO ERM can be an important tool to better understand and evaluate the risks surrounding internal controls at all levels, but with an emphasis on IT resources. We describe major elements of the COSO ERM framework and also look at how IT auditors can better build COSO ERM concepts into their internal controls audits and reviews of IT resources. Although the basic framework models look similar to the COSO internal controls framework, COSO ERM, with its emphasis on enterprise-wide risks, is different. RISK MANAGEMENT FUNDAMENTALS Every enterprise makes business decisions and invests in resources to provide value for its stakeholders, but these investments and other activities have uncertain outcomes and are always subject to uncertainties or risks—whether the failure of a key IT business process, the challenge caused by a new and aggressive competitor, or the damage and even loss of life caused by a major weather disturbance. Risk management is a concept where an enterprise should use insurance mechanisms to provide a shield or protection from those risks. We make these decisions based on assessments of relative risks and the costs to cover them through the purchase of insurance. These risks and insurance costs also change over time. Fire insurance to cover an individual’s home is an example of this evolving change. Back in the days of oil lanterns as a source for light and straw stored in a nearby stable, there was always a high risk of fires. We need only to think of the great Chicago fire of 1871 where, as legend suggests, a cow kicked over a lantern and caused a fire that devastated the city. The risk of fires in the typical building is not that great today, and fire insurance is not too expensive, in a relative sense. However, there is always the possibility of a lightning strike or electrical malfunction causing a fire in a structure, and mortgage finance companies require fire insurance coverage. Even if there is no mortgage, all prudent persons today purchase such fire insurance even if not required. A destructive fire to one’s home presents a low-level but consistent risk. An individual homeowner might assess other potential risks, such as for earthquakes, and not purchase insurance for that type of risk. In a given geographic area, the possibility of an earthquake may appear so low that an owner would not consider purchasing any insurance, despite its low cost. In another situation, an individual may live by a body of E1C04 08/06/2010 13:58:14 84 & Page 84 Understanding Risk Management through COSO ERM water where there are damaging floods every several years. Even if one can purchase flood insurance in such an environment—and most insurance companies will not even offer it—the cost of flood insurance here will be very expensive. Some people may decide to accept the risk of a flood in future years and go without insurance coverage. In all of these cases, there has been an insurance purchaser risk management decision. Starting with these insurance-buying foundations, enterprise risk management, as it is practiced today, is essentially a post-1960s phenomenon. Moving beyond concerns about natural weather-related events, risk management began to emphasize protecting enterprises against a major catastrophe, such as the risks surrounding a computer system back in the mainframe days, when most information systems assets were stored in centralized facilities. The concern about managing risks surrounding that one centralized computer system moved to a general concern about managing a wide range of other business risks. (IT disaster recovery planning is discussed in Chapter 23.) Risk assessment, in combination with other audit techniques, should be considered in making such planning decisions, with consideration given to the nature, extent, and timing of any planned audit procedures; the areas or business functions to be audited; and the amount of time and resources to be allocated to an audit. A key risk assessment and audit planning concept, IT auditors should understand and make their assessments in terms of the levels of what are known as inherent, control, and detection risks. & & & Inherent risk. As defined by the U.S. government’s Office of Management and Budget (OMB), inherent risk is the ‘‘potential for waste, loss, unauthorized use, or misappropriation due to the nature of an activity itself.’’ Inherent risk is the susceptibility of an audit error to occur in a way that could be material, individually or in combination with other errors, assuming that there were no related internal controls. Major factors that affect enterprise inherent risk are the size of its budget, the strength and sophistication of management, and the very nature of its activities. Inherent risk is outside the control of management and usually stems from external factors. For example, the major retailer Wal-Mart is so large and dominant in its markets that it faces various inherent risks due to its sheer size. Inherent risk for most IT audit areas is usually high, since the potential effects of errors ordinarily span several business systems and many users. Control or residual risk. This is the risk that remains after management responses to risk threats and countermeasures have been applied. There is virtually always some level of residual risk. These are also the risks of an error occurring in an audit that could be materially significant, individually or in combination with other errors, but that will not be prevented or detected and corrected on a timely basis by the internal control system. For example, the control risk associated with manual reviews of computer system activity logs can be high because activities requiring investigation can be easily missed, owing to the volume of logged information. The control risk associated with IT data validation procedures is ordinarily low because the processes are consistently applied. Detection risk. This is the risk that the IT auditor’s substantive procedures will not detect an error that could be material, individually or in combination with other errors. For example, the detection risk associated with identifying breaches of E1C04 08/06/2010 13:58:14 Page 85 Risk Management Fundamentals & 85 security in an application system is ordinarily high because logs for the whole period of the audit may not be available at the time of the audit. The detection risk associated with identifying a lack of disaster recovery plans is ordinarily low, since existence is verified easily. Inherent, control, and detection risk are fundamental risk assessment concepts. IT auditors should have a general understanding of these concepts and how they will help an IT auditor to review and assess controls. Enterprises today face a wide variety of IT-related risks. Management needs to assess these risks in order to make rational decisions related to cost and risk avoidance. This is the process of risk management. Although an enterprise today can just make seat-of-the-pants decisions to assess a potential threat as high, medium, or low risk and then make quick insurance or risk protection decisions based on those options, more sophisticated qualitative or quantitative tools are available to help them understand and evaluate these risks. IT auditors should have a general understanding of these processes as well. The next sections briefly survey some fundamental modern risk management approaches with the aim of helping to establish more effective IT risk management procedures in an enterprise. An effective risk management process typically requires four steps: 1. 2. 3. 4. Identify the relevant potential risks facing an activity. Quantitatively or qualitatively assess those identified risks. Prioritize risk and response planning. Perform ongoing risk status monitoring. There is always a need to identify and understand the various risks facing an enterprise, to assess those risks in terms of their cost or impact and probability, to develop responses in the event of a risk occurrence, and to develop documentation procedures to describe what happened as well as corrective actions going forward. This is true for specific IT-related risks as well as for other enterprise-wide non-IT risks. The four-step risk management process should be implemented at all levels of an enterprise and is particularly useful for IT systems and related resources. Whether the company is a smaller one operating in a limited geographic area or a larger, worldwide enterprise, these risk management approaches should be developed for the entire enterprise. Doing so is particularly important for the worldwide enterprise with multiple operating units engaged in different business operations and with facilities in different countries. Some risks in one unit may directly impact or be related to risks in another, while other risk considerations may be independent from the whole. Although the overall focus of this book is on the activities of IT auditors, risk management should be an enterprise-wide responsibility, managed by risk management specialists in the enterprise. Many have established a chief risk officer (CRO) to take responsibility for these activities. Risk Identification As the first step in the previously outlined four-part risk management process, the enterprise’s CRO or a supporting team should try to identify all possible risks that may E1C04 08/06/2010 13:58:14 86 & Page 86 Understanding Risk Management through COSO ERM impact the success of their enterprise, ranging from the larger significant risks to the overall business down to lesser risks associated with individual projects or smaller business units. This type of risk identification process requires a studied, deliberate approach to looking at potential risks in each area of operations and then identifying the more significant risk areas that may impact each operation in a reasonable time period. The idea here is not to list every possible risk but for an enterprise to identify those that might have a major impact on operations within a reasonable time period. This exercise can be difficult; it requires estimating the probabilities of various identified risks occurring or the nature of the consequences if the enterprise has to face the risk. This risk identification process should occur at multiple levels, with an understanding that a risk that impacts an individual business unit or project may not have that great of an impact on the entire enterprise or beyond. Conversely, a major risk that impacts the entire economy will flow down to the individual enterprise and its separate business units. Some major risks are so infrequent but can be so cataclysmic that it is difficult to identify them as possible future events. For a larger enterprise, a good way to start a risk identification process is to begin with a high-level organization chart that lists corporate-level as well as operating units. Each of those units may have facilities in multiple locations and may also consist of multiple and different types of operations. Each separate facility will then have its own departments or functions. Some of these separate business units may be closely connected to one another while others represent little more than corporate investments. An enterprise-wide initiative, which is a difficult and sometimes complicated task, should be launched to identify all significant risks in these individual areas. This type of exercise can have interesting and sometimes troubling results. For example, a corporatelevel senior manager may be aware of some product liability risks, but a front-line supervisor in an operating unit may look at the same risks from an entirely different perspective. A marketing manager, for example, may be concerned about a competitor’s pricing strategies or the risk of pricing activities that would put the enterprise in violation of restraint of trade laws. An IT manager may be concerned about the risk of a computer virus attack on application systems but will have little knowledge of pricing issue risks. More senior management typically will be aware of a different level and set of risks than would be on the minds of the operations-oriented staff. Still, all of these risks should be identified and considered on an operating unit–by–operating unit basis and over the entire enterprise. To be effective, this risk identification process requires much more than just sending out an e-mail to all operating units with a request for recipients to list the key risks in their units. Such a request typically will result in a wide range of inconsistent answers with no common approach. A better method is to ask people at all levels of the enterprise to serve as risk assessors. Within each significant operating unit, key people should be identified from operations, finance/accounting, IT, and unit management. Their goal should be to identify and then help assess risks in their units built around a risk identification model framework. This type of initiative can be led by the CRO or a designated risk assessment team. IT auditors often play a key role here because of their ongoing understandings of IT disaster recovery risks. E1C04 08/06/2010 13:58:14 Page 87 Risk Management Fundamentals & 87 Enterprise-Wide Strategic Risks External Factors Risks & & & & & Internal Factors Risks Industry Risk Economy Risk Competitor Risk Legal and Regulatory Change Risk Customer Needs and Wants Risk & & & & Reputation Risk Strategic Focus Risk Parent Company Support Risk Patent/Trademark Protection Risk Operations Risks Process Risks & & & & Supply Chain Risk Customer Satisfaction Risk Cycle Time Risk Process Execution Risk Compliance Risks & & & & Environmental Risk Government Rules and Regulatory Risk Policy and Procedures Risk Litigation Risk People Risks & & & & Human Resources Risk Employee Turnover Risk Performance Incentive Risk Training Risk Finance Risks Treasury Risks & & & Credit Risks Interest Rate Risk Foreign Exchange Risk Capital Availability Risk & & & & & Trading Risks Capacity Risk Collateral Risk Concentration Risk Default Risk Settlement Risk & & & Commodity Price Risk Duration Risk Measurement Risk Information Risks Financial Risks & & & & & Accounting Standards Risk Budgeting Risk Financial Reporting Risk Taxation Risk Regulatory Reporting Risk EXHIBIT 4.1 Operational Risks & & & & Terrorist Attack Pricing Risk Performance Measurement Risk Employee Safety Risk Technological and IT Risks & & & & Information Access Risk Business Continuity Risk Availability Risk Infrastructure Risk Types of Enterprise Risks An effective idea here is to outline some high-level ‘‘straw man’’2 risks that may impact various operating units. Knowledgeable people can then look at these lists and modify them as appropriate. Exhibit 4.1 shows some types of major risks that may impact an enterprise, including various strategic, operations, and finance risks. A chief executive officer (CEO) might use such a high-level list to respond to a shareholder annual meeting question ‘‘What worries you at the end of the day?’’ This is the type of first-pass list that an enterprise can use to get started on a detailed risk identification. Other senior managers in the enterprise—often the CEO and supporting staff—can meet with senior management and ask some of these types of questions to identify such high-level risks. This very general, high-level risk model can serve as a basis to better define the broad range of specific risks facing various units of an enterprise. An IT auditor should be able to expand an entry, such as Business Continuity Risk listed under Technological Risks, into a long list of detailed technology-related risks associated with business continuity. An operations manager who is the user of IT resources might look at business continuity risks from a different perspective and may introduce other risks E1C04 08/06/2010 13:58:14 88 & Page 88 Understanding Risk Management through COSO ERM associated with what happens if IT services are not available. In order to have a better understanding of the risks facing an enterprise, it is often best to expand these lists to establish a more complete set of risks. An enterprise management team should then use this more complete list of potential enterprise risks and ask themselves such questions as: & & & Is the risk common across the overall enterprise or unique to one business group? Will the enterprise face this risk because of internal or external events? Are the risks related, such that one risk may cause another to occur? The idea is to gain a strong understanding of the nature of enterprise-level risks and then to highlight major risks, including, for example, the risks of: a significant fall in customer satisfaction ratings, a new and very large competitor entering the market, and an identified significant internal control weakness as part of the financial statement close. Any of these major risks could present significant challenges to an enterprise. Enterprise management should review their risks and highlight those that appear to be most critical in order to prepare a final set of risks by the overall enterprise and by specific operating units. Because viewpoints and perspectives will vary across the enterprise, these identified risks should be shared with responsible operating, IT, and financial management, giving them opportunities to provide feedback. The idea here is to identify the population of risks that threaten an enterprise, both at an individual unit level and on a total enterprise basis. These risks will not necessarily be the enterprise’s core risks but they often are a starting point for enterprise risk assessments. Key Risk Assessment Principles After the significant enterprise risks are identified, a second important step is to assess their likelihood and relative significance. A variety of approaches can be used here, ranging from best-guess qualitative conjectures to detailed mathematical quantitative analyses. The idea is to help decide which of a series of potentially risky events management should worry the most about. A simple but often effective approach is to take the list of identified risks and circulate them to key managers with a questionnaire asking for each risk: & & What is the likelihood of this risk occurring over the next one-year period? Using a range of 1 to 9, assign a best-guess score as follows: & Score 1 if you see almost no chance of that risk happening during the period. & Score 9 if you feel the event will almost certainly happen during the period. & Score 2 through 8 depending on how you feel the likelihood the event will fall between these two ranges. What is the significance of the risk in terms of cost to the enterprise? Again using the same 1 to 9 scale, scoring ranges should be set depending on the financial significance of the risk. A risk whose costs could lower earnings per share by perhaps 1 cent might qualify for the maximum score of 9. 08/06/2010 13:58:14 Page 89 Risk Management Fundamentals & 89 9 R-6 8 Significance E1C04 R-3 7 R-1 6 I R-2 II 5 R-4 4 R-5 3 2 III IV 1 1 2 3 4 5 6 7 8 9 Likelihood EXHIBIT 4.2 Risk Assessment Analysis Map Source: Robert R. Moeller, COSO Enterprise Risk Management: Understanding the New Integrated ERM Framework (Hoboken, NJ: John Wiley & Sons). Copyright ß 2007 John Wiley & Sons. Reprinted with the permission of John Wiley & Sons, Inc. Questionnaires here should be circulated independently to knowledgeable people to score each of the identified risks per these two measures. As an example, assume that an enterprise has identified six risks, R-1 through R-6, and four managers are asked to separately evaluate each risk in terms of its likelihood and significance. These scores can be averaged by both factors and plotted on a risk assessment analysis chart as shown in Exhibit 4.2. R-1 had an average Likelihood score of about 3.75 and a Significance score of 7.00, and this score is plotted in quadrant I of the chart. For example, R-1 is relatively significant but not that likely to occur. After all identified risks are plotted in this manner, the high-likelihood and more significant risks in quadrant II should receive immediate management attention. This type of chart provides a good qualitative measure to understand some of the significant risks surrounding an enterprise. This high-risk assessment process works well when an enterprise has identified a relatively small number of risks. It is fairly easy to look at the analysis chart and focus on remediation planning for the high-likelihood and significant risks in the upper left-hand quadrant. Often, however, an enterprise may identify a much larger set of risks, with risk ranges of only 1 to 9 as well as plots on the example chart will not provide sufficient detail. It is often a better approach to express risk significance and impact estimates in terms of two-digit percentage estimates (e.g., 72%) of achieving some risk or as a probability. However, just increasing the number of digits, from a 7% to a full 72%, does not increase the accuracy of the assessment. More attention should be given to better understanding the relationship between probabilities covering independent and related risk events. Combined estimates of likelihood and significance are key factors in any risk analysis assessment. The combination of two probabilities requires one to go back to some basic mathematical concepts. A basic rule of probability is that we cannot add up independent probability estimates to develop a joint estimate. If the probability of risk A occurring is 60% and the probability of a separate but related risk B is also 60%, we E1C04 08/06/2010 13:58:14 90 & Page 90 Understanding Risk Management through COSO ERM cannot say that the probability of both occurring is 0.60 þ 0.60 ¼ 1.20. This 120% does not make sense! Rather, the joint probability of two independent events is the product of the two separate probabilities: PrðEvent 1Þ  PrðEvent 2Þ ¼ PrðBoth EventsÞ That is, if Event 1 has a probability of 0.60 and Event 2 is also 0.60, the combined probability of both events occurring is (0.60)  (0.60) ¼ 0.36. In terms of the assessments, if a risk has a 60% significance estimate or we are 60% certain that the risk will occur, and if the impact has been rated at 60%, there is a 36% probability that we will achieve both of those risks. We can also call this the risk score for the individual risk. An accurate risk assessment process, however, requires more than just top-of-thehead estimates stated in percentage estimates. Enterprise management should take a hard look at identified risks and gather more information, if required. For example, during the risk identification process, one manager may have identified the consequences of a new tariff law as a serious risk. However, responsible managers may want to better understand its actual consequences. Perhaps the law is not applicable to the unit in question, or it often does not go in to effect until some years in to the future. The point here is that all identified risks may need some additional information before they can be assessed accurately. Risk independencies must always be considered and evaluated throughout the organizational structure. Although any entity should be concerned about risks at all levels of the organization, it really has control over only the risks within its own sphere. The 2002 example of the fall of the major public accounting firm Arthur Andersen in the wake of the Enron collapse is an example. Each city-by-city and country-by-country unit of that once esteemed public accounting firm had its own risk assessment procedures, following firm-wide standards. However, a risk event at one operating office, in Houston, Texas, caused the firm to collapse worldwide. An operating office in another area, such as Toronto, might not have even have fully anticipated such risks in a faraway Houston. The point here is that risks within an enterprise are often very interdependent. Each operating unit is responsible for managing its own risks but may be subject to the consequences of risk events on units above or below it in the organization structure. The examples used in this chapter show a short list of identified risks, but a typical enterprise will end up with a very long list of potential risks. A next step is to take the established significance and likelihood estimates, calculate risk rankings, and identify the most significant risks across the entity reviewed. Exhibit 4.3 is an example of this type of analysis. Based on the likelihood and significance scores from Exhibit 4.2, the product of these two values gives relative risk rankings. Risks C and G have the highest risk rank scores and would be plotted in the upper right-hand quadrant as the most significant risks in this sample. These risks would be called the risk drivers for this set identified risk. That is, these are the key risk areas requiring attention. An organization should focus its attention going forward on these primary risks. These risk-ranked schedules should be organized on a unit-by-unit basis and adjusted to accommodate all related risks in parallel with as well as above or below the entity being evaluated. E1C04 08/06/2010 13:58:15 Page 91 Risk Management Fundamentals 91 & Identified Risk Significance Probability Likelihood Probability Risk Score (P  I) Rank A 0.55 0.30 0.17 8 B 0.88 0.24 0.21 7 C 0.79 0.66 0.52 1 D 0.77 0.45 0.35 4 E 0.35 0.88 0.31 5 F 0.54 0.49 0.26 6 G 0.62 0.72 0.45 2 H 0.66 0.20 0.13 9 I 0.90 0.45 0.41 3 J 0.12 0.88 0.11 10 EXHIBIT 4.3 Risk Scoring Schedule This analysis requires that a risk management team identify its unit-by-unit assessed risks to make certain that risk likelihood and significance estimates are appropriate throughout. All too often, risk events that occur far away from corporate headquarters can cause major problems. An example from nearly 30 years ago can be drawn from a risk event at Union Carbide, a U.S. corporation. On the night of December 2, 1984, over 40 tons of poisonous gases leaked from a pesticide factory owned by Union Carbide in Bhopal, India, killing more than 20,000 residents.3 After much legal wrangling, Union Carbide, which had built the plant in 1969, settled a civil suit brought by the Indian government by agreeing to pay US$470 million for damages suffered by the half-million people who were exposed to the gas. The company maintained that this payment was made out of a sense of ‘‘moral’’ rather than any ‘‘legal’’ responsibility since the plant was operated by a separate Indian subsidiary, Union Carbide India Limited (UCIL), but court proceedings revealed that senior management’s cost-cutting measures had effectively disabled safety procedures essential to prevent such disasters or alert employees to problems. Dow Chemical has since taken over Union Carbide and also denies responsibility for this disaster. However, because of the tremendous loss of life in Bhopal and because Dow Chemical is much larger than what Union Carbide and its UCIL subsidiary had been, ongoing actions haunted Dow Chemical over many years. The Bhopal gas leak is an example of how a risk event at a distant and relatively small unit can have disastrous consequences for a major corporation. The risk identification and assessment rules outlined in this chapter would not have accounted for a catastrophe of this magnitude, and each unit in an enterprise needs to recognize the likelihood and consequences of risks at individual unit levels. As noted, a risk event at a small foreign subsidiary can bring down the entire enterprise. Risk management at all levels should recognize that catastrophes can happen, although we never can predict risks of this major consequence; an enterprise should always be aware of the worst disaster that can happen. This Bhopal incident was much more than an IT-related matter; we have cited it as the type of a major risk that can impact an enterprise. Risk assessment at all levels E1C04 08/06/2010 13:58:15 92 & Page 92 Understanding Risk Management through COSO ERM should be based on two estimates: the relative significance probability and the likelihood of the risk event occurring. QUANTITATIVE RISK ANALYSIS TECHNIQUES There is little value in identifying significant risks unless an enterprise has at least some preliminary plans in place for the action steps necessary if one of the risks is incurred. The idea is to estimate the cost impact of incurring some identified risk and then to apply that cost to a risk factor probability to derive an expected value or cost of the risk. Often this exercise that does not require detailed cost studies with lots of supporting historical trends and estimates. Rather, expected cost estimates should be performed by front-line people at various levels of the enterprise who have a good level of knowledge of the area or risk implications. The idea here is to go through each of the identified risks—or, if time is limited, only the key risks—and estimate the costs of incurring the risk. Because the kinds of risks discussed involve such matters as the failure of an IT hardware component, the drop in market share, or the impact of a new government regulation, typically they are not the types of costs that one can just look up in a current vendor catalog. Some typical risks, arbitrarily labeled A, B, and C, illustrate this type of thinking: Risk A: Loss of up to x% market share due to changing consumer tastes & & Estimate the reduction in sales and loss of profits due to the x% drop. Estimate how much will it cost to begin to restore the lost market position. Risk B: Temporary loss of major manufacturing facility for zz days due to major database failure & & Estimate the best- and worst-case costs to get the database temporarily repaired and back in operation within zz days. Estimate the extra labor and production costs incurred during the interim. Risk C: Loss of information systems for two days due to pernicious IT network virus & & Estimate the business and profitability loss during the down period. Estimate the cost to transfer operations to the business continuity site. The above factor examples illustrate the type of thinking needed to estimate the costs of recovering from some risk event. It is often difficult to determine what it would cost to recover from these risks. There is no need to perform detailed, time-consuming analyses. Knowledgeable people who understand the risk area often can provide good estimates on the basis of: 1. What is the best-case cost estimate if it is necessary to incur the risk? This is an assumption that there will be only limited impact if the risk occurs. E1C04 08/06/2010 13:58:15 Page 93 Quantitative Risk Analysis Techniques 93 & 2. What would a sample of knowledgeable people estimate for the cost? For Risk A mentioned earlier, the director of marketing might be asked to supply an estimate. 3. What is the expected value or cost of incurring the risk? This is the type of risk that might include some base costs as well as other factors, such as additional labor requirements. 4. What is the worst-case cost of incurring the risk? This is a what-if-everything-goeswrong type of estimate. We have suggested using four estimates as an idea of the ranges of costs. However, one best-guess estimate should be selected from the four estimates—usually something between estimates 2 and 3. These estimates and supporting work should be documented, with the selected cost estimate entered as the cost impact on the risk response planning schedule in Exhibit 4.4. These are the same risks that were identified in Exhibit 4.3, but here they are ordered by risk rank. This reordering is important when an enterprise has a long list of identified risks. The expected value or cost values in Exhibit 4.4 are just the products of the cost impacts and their risk scores. Exhibit 4.4 shows an estimate of what it would cost an enterprise to incur some risk. Although the numbers selected for these samples are very arbitrary, they show how a risk management specialist should interpret this type of analysis. Risk C, for example, has a high likelihood and significance as well as fairly high expected cost to correct. This is the type of risk that management should identify as a candidate for corrective actions. However, the next risk on the schedule, Risk G, also belongs in the upper left-hand quadrant but with a relatively low cost to implement. Management may decide to accept this risk or to develop some other form of remediation plan. Risk H is an example of another risk with a high cost to accept that risk, with its fairly high significance and a low likelihood of occurrence. The numbers shown for Risk H are the kinds where management frequently decides to hope for the best and live with the risk. It will be expensive if management incurs the risk, but it also would be expensive to install corrective action facilities. Identified Risk Significance Probability Likelihood Probability Risk Score (P  I) Rankings Cost Impact Expect Cost (Cost  Score) C 0.79 0.66 0.52 1 $ 120,600 $ 62,881 G 0.62 0.72 0.45 2 $ 785,000 $350,424 I 0.90 0.45 0.41 3 $ 15,000 $ D 0.77 0.45 0.35 4 $ 27,250 $ 9,442 E 0.35 0.88 0.31 5 $ 52,350 F 0.54 0.49 0.26 6 $ 1,200 6,075 $ 16,124 $ 318 B 0.88 0.24 0.21 7 $ 12,650 $ 2,672 A 0.55 0.30 0.17 8 $ 98,660 $ 16,279 H 0.66 0.20 0.13 9 $ 1,200,980 $158,529 J 0.12 0.88 0.11 10 $ $ EXHIBIT 4.4 Risk Ranking Expected Cost Estimate 88,600 9,356 E1C04 08/06/2010 13:58:15 94 & Page 94 Understanding Risk Management through COSO ERM IIA AND ISACA RISK MANAGEMENT INTERNAL AUDIT GUIDANCE Risk management and understanding risks are important elements in both the standards and supporting guidance materials released by the IIA for all internal auditors and ISACA, with its more IT audit focus. Chapter 3 reviewed many of the internal audit standards. This section examines the levels of guidance available to assist IT auditors in their risk-related reviews. Each of these important IT audit professional organizations provides both standards and supporting guidance to help IT auditors to consider risks in their IT audit review activities. Which standards should the IT audit professional follow? This question was answered somewhat in Chapter 3. The IIA International and ISACA IT Audit Standards are not in conflict with each other and are very similar with regard to risk management. Some chief audit executives (CAEs) and audit committee members are more familiar with the IIA offerings, and that will dictate the standards selection approach. However, if the enterprise has selected Control Objectives for Information and related Technology (CobiT) as its guidance framework for SOx compliance work, the ISACA standards are the better approach. IIA Risk-Related International Standards and Management Guidance The need to understand and assess risks has been part of the IIA International Standards going back to their earliest versions. Risk concepts are part of both the IIA’s Attribute and Performance Standards. For example, the supporting Attribute assurance standard 1220.A3, titled Risk Identification, states: Internal auditors must be alert to the significant risks that might affect objectives, operations, or resources. However, assurance procedures alone, even when performed with due professional care, do not guarantee that all significant risks will be identified. The problem with this standard, however, is that it really does not give much guidance on what is meant by a ‘‘significant risk.’’ This lack of definition is one of those issues that has led to the COSO ERM framework and its definition of risk management. Without a clear and consistent understanding, internal auditors may miss key areas of audit risk in their review work. As discussed in Chapter 3, the IIA International Standards for the Professional Practice of Internal Auditing consist of Attribute Standards that define areas of internal audit activity at a high level and Performance Standards that define minimum requirements for performing all levels of internal audits. There is a full Performance Standard, 2120, along with several assurance and consulting substandards on risk management. International Professional Practice Standard 2120 on risk management says: ‘‘The internal audit activity must evaluate the effectiveness and contribute to the improvement of risk management processes.’’ With a bit more guidance than Assurance substandard 1220.A3, this risk-related IIA International Standard is supported by two Assurance and three Consulting Standards related to risk management: E1C04 08/06/2010 13:58:15 Page 95 IIA and ISACA Risk Management Internal Audit Guidance & 95 2120.A1. The internal audit activity must evaluate risk exposures relating to the organization’s governance, operations, and information systems regarding the: & Reliability and integrity of financial and operational information; & Effectiveness and efficiency of operations; & Safeguarding of assets; and & Compliance with laws, regulations, and contracts. 2120.A2. The internal audit activity must evaluate the potential for the occurrence of fraud and how the organization manages fraud risk. 2120.C1. During consulting engagements, internal auditors must address risk consistent with the engagement’s objectives and be alert to the existence of other significant risks. 2120.C2. Internal auditors must incorporate their knowledge of risks gained from consulting engagements into their evaluation of the organization’s risk management processes. 2120.C3. When assisting management in establishing or improving risk management processes, internal auditors must refrain from assuming any management responsibility by actually managing risks. The Assurance substandard A1 gives a broader picture of the areas an internal auditor should review and consider with regard to risks. Whether the review is IT related or in other areas, an internal auditor should take a wide view regarding risk exposures. Substandard A2 on fraud risks is the only place where the topic of fraud appears in the IIA international standards. IT audit fraud detection and prevention issues are discussed in Chapter 21. Chapter 3 discussed the mix of IIA mandatory and strongly recommended guidance materials. One of the recommended guidance materials is an IIA publication in their Guides to the Assessment of IT (GAIT) series titled GAIT for Business and IT Risk.4 This useful guide is available to IIA members. Many of its concepts appear later in this chapter on considering risk when auditing IT-related controls. ISACA Audit Risk-Related Standards and Management Guidance With its focus more on IT audit-related issues, ISACA has taken a stronger position on considering risk issues through its CobiT framework, discussed in Chapter 2, and the ISACA IT Audit Standards. Exhibit 4.5 is the ISACA Audit Standard S11 on the use of risk assessment in audit planning. The exhibit shows the full standard with only a few minor changes (American versus English spelling). It also follows the ISACA format where standards items in bold type are considered to be mandatory for ISACA member IT auditors. Similar to the IIA International Standards, this standard says that IT auditors should follow a consistent risk evaluation process to plan areas for their audit reviews. ISACA also has a companion audit guideline, G13, with the same title. A fairly detailed guide, it advises that there are many risk assessment methodologies available from which the IT auditor may choose. These range from simple classifications of high, medium, and low, based on the IS auditor’s judgment, to complex calculations to E1C04 08/06/2010 13:58:15 96 & Page 96 Understanding Risk Management through COSO ERM Introduction 01 ISACA IT Auditing Standards contain the basic principles and essential procedures, identified in bold type, that are mandatory, together with related guidance. 02 The purpose of this standard is to establish standards and provide guidance regarding the use of risk assessment in audit planning. Standard 03 The IT auditor should use an appropriate risk assessment technique or approach in developing the overall IT audit plan and in determining priorities for the effective allocation of IT audit resources. 04 When planning individual reviews, the IT auditor should identify and assess risks relevant to the area under review. Commentary 05 Risk assessment is a technique used to examine auditable units in the IS audit universe and select areas for review to include in the IT annual plan that have the greatest risk exposure. 06 An auditable unit is defined as a discrete segment of every organization and its systems. 07 Determination of the IT audit universe should be based on knowledge of the organization’s IT strategic plan, its operations, and discussions with responsible management. 08 Risk assessment exercises to facilitate the development of the IT audit plan should be carried out and documented at least on an annual basis. Organizational strategic plans, objectives, and the enterprise risk management framework should be considered as part of the risk assessment exercise. 09 The use of risk assessment in the selection of audit projects allows the IT auditor to quantify and justify the amount of IT audit resources needed to complete the IT audit plan or a particular review. Also, the IT auditor can prioritize scheduled reviews based on perceptions of risk and contribute toward the documentation of risk management frameworks. 10 An IT auditor should carry out a preliminary assessment of the risks relevant to the area under review. IT audit engagement objectives for each specific review should reflect the results of such a risk assessment. 11 Following the completion of the review, the IT auditor should ensure that the organization’s enterprise risk management framework or risk register is updated, if one has been developed, to reflect findings and recommendations of the review and subsequent activity. 12 The IT auditor should refer to IT auditing guideline G13, Use of Risk Assessment in Audit Planning, and the IS auditing procedure P1, IS Risk Assessment Measurement. EXHIBIT 4.5 ISACA Standard S11 Use of Risk Assessment in Audit Planning Source: IT Standards, Guidelines, and Tools and Techniques for Audit Assurance and Control Professionals. ß 2005 ISACA. All rights reserved. Used by permission. provide numeric risk ratings. IT auditors should consider the level of complexity and detail appropriate for the activity being audited. IT auditors should include, at a minimum, an analysis, within their methodology, of the risks to the enterprise resulting from the loss of and controls supporting system availability, data integrity, and business information confidentiality. In developing an appropriate risk assessment methodology, IT auditors should consider such things as: & & The type of information required to be collected and the extent to which the information required is already available The cost of software licenses required to use the methodology E1C04 08/06/2010 13:58:15 Page 97 COSO ERM: Enterprise Risk Management & & & & 97 The amount of additional information required to be collected before reliable output can be obtained, and the cost of collecting this information (including the time required to be invested in the collection exercise) The opinions of other users of the methodology, and their views of how well it has assisted them in improving the efficiency and/or effectiveness of their audits The willingness of management to accept the methodology as the means of determining the type and level of audit work carried out No single risk assessment methodology can be expected to be appropriate in all situations, but IT auditors should reevaluate the appropriateness of their chosen risk assessment methodologies and make changes the can justify as appropriate. COSO ERM: ENTERPRISE RISK MANAGEMENT COSO ERM is a framework to help enterprises have a consistent definition of their risks. It is also an important tool for understanding and improving SOx internal controls. COSO ERM was launched in a manner similar to the development of the COSO internal control framework, as discussed in Chapter 1. Just as there had been no consistent definition of internal controls, industry accounting professionals felt there had been no consistent enterprise-level definition of risk. For example, assessing risks by their likelihood and probability of occurrence is important for IT auditors and other professions, but this method was not universally accepted elsewhere. Similarly, our discussion of IIA and ISACA standards for considering risk really do not fully define what is meant by this concept of enterprise-wide risk for IT auditors and others. Until recently, there was no commonly accepted definition of risk management and no comprehensive framework outlining how the process should work, which made risk communication among board members and management difficult and sometimes frustrating.5 Similar to the manner in which the COSO internal controls framework was developed, COSO developed an enterprise risk management framework. It was first published just after the enactment of SOx in September 2004. Just as the COSO internal controls framework started by proposing a consistent definition of its subject, the COSO ERM framework document starts by defining enterprise risk management: Enterprise risk management is a process, effected by an entity’s board of directors, management and other personnel, applied in a strategy setting and across the enterprise, designed to identify potential events that may affect the entity, and manage risk to be within its risk appetite, to provide reasonable assurance regarding the achievement of entity objectives. This definition is rather academic sounding. Professionals should consider the key points supporting this COSO ERM framework definition, including: & ERM is a process. An often misused word, the dictionary definition of a process is a ‘‘set of actions designed to achieve a result.’’ Although this definition does not provide much help for IT audit professionals, the idea here is that a process is not a E1C04 08/06/2010 13:58:15 98 & & & & & Page 98 Understanding Risk Management through COSO ERM static procedure, such as the use of an employee badge to allow only certain authorized persons to enter a locked IT server facility. Such a badge procedure—like a key to a lock—only either allows or does not allow someone entry to the facility. A process tends to be a more flexible arrangement. In a credit approval process, for example, acceptance rules are established with options to alter them, given other considerations. An enterprise might bend the credit rules for an otherwise good credit customer that is experiencing a short-term problem. ERM is that type of process. An enterprise often cannot define its risk management rules through a small, tightly organized rule book. Rather, there should be a series of documented steps to review and evaluate potential risks and to take action based on a wide range of factors across the entire enterprise. ERM process is implemented by people in the enterprise. An ERM will not be effective if it is implemented only through a set of rules sent in to an operating unit from a distant corporate headquarters, where the people who drafted the rules may have little understanding of the various decision factors that arise. The risk management process must be managed by people who are close enough to the risk situation to understand the various factors surrounding and implications of that risk. ERM is applied through the setting of strategies across the overall enterprise. Every enterprise is constantly faced with alternative strategies regarding a vast range of potential future actions. Should the entity acquire another complementary business to expand growth or just build internally? Should it adopt a new technology in its manufacturing processes or stick with the tried and true? An effective ERM set of processes should play a major role in helping to establish those alternative strategies. Since many enterprises are large with varied operating units, ERM should be applied across that entire enterprise using a portfolio type of approach that blends a mix of high- and low-risk activities. Concept of risk appetite must be considered. A newer concept for many, risk appetite is the amount of risk, on a broad level, that an enterprise and its individual managers are willing to accept in their pursuit of value. Risk appetite can be measured in a qualitative sense by looking at risks in such categories as high, medium, or low; alternatively, it can be defined in a qualitative manner. The basic idea behind risk appetite is that every manager and, collectively, every enterprise has some appetite for risk. Some will accept risky ventures that promise potentially high returns while others prefer a more guaranteed-return, low-risk approach. One can think of this appetite for risk concept in terms of an example of two investors. One may prefer to invest in very-low-risk but typically low-return money market or index funds; another may invest in low-cap start-up technology stocks with expectations of very high returns. That latter investor can be described as having a high appetite for risk. As another example, on a street intersection with a Walk or Don’t Walk crossing lights, the person who keeps crossing the intersection when the light begins to flash ‘‘Walk,’’ meaning it will soon change to ‘‘Don’t Walk,’’ has a higher appetite for risk. ERM provides only reasonable, not positive, assurance on objective achievements. The idea here is that an ERM, no matter how well thought out or implemented, cannot provide management or others with any assured E1C04 08/06/2010 13:58:15 Page 99 COSO ERM: Enterprise Risk Management & & 99 guarantee of outcomes. A well-controlled enterprise, with people at all levels consistently working toward understood and achievable goals, may achieve those objectives period after period, even over multiple years. However, an unintentional human error, an unexpected action by another, or even a natural disaster can occur. Despite an effective ERM process, an enterprise can experience a major and totally unexpected catastrophic event. Reasonable assurance does not provide absolute assurance. ERM is designed to help attain the achievement of objectives. An enterprise, through its management, should work to establish high-level common objectives that can be shared by all stakeholders. Examples here include such matters as achieving and maintaining a positive reputation within an enterprise’s business and consumer communities, providing reliable financial reporting to all stakeholders, and operating in compliance with laws and regulations. The overall ERM program for an enterprise should help it to achieve those objectives. ERM-related goals and objectives are of little value unless they can be organized and modeled together in a way that management can look at the various aspects of the task and understand—at least sort of—how they interact and relate in a multidimensional manner. This is a real strength of the COSO internal controls framework. It describes, for example, how an enterprise’s compliance with regulations impacts all levels of internal controls, from monitoring processes to the control environment, and how that compliance is important for all enterprise entities or units. The COSO ERM framework provides some common definitions of risk management and can help to achieve SOx internal control objectives as well as better risk management processes throughout the enterprise. Although COSO ERM is designed to provide guidance to the total enterprise, auditors should consider its concepts when performing reviews and assessments at multiple levels. The next sections describe the COSO ERM framework in greater detail. The ERM framework looks very similar to the COSO internal controls framework discussed in Chapter 1, but it has different objectives. Key Elements The COSO internal control framework, has become a worldwide model for describing and defining internal controls and has been the basis for establishing SOx Section 404 compliance. Perhaps because some of the same team members were involved with both COSO internal controls and ERM, the COSO ERM framework—at first glance—looks very similar to the COSO internal controls framework. The COSO ERM framework is also shown in Exhibit 4.6 as a three-dimensional cube with the components of: & & & Four vertical columns representing the strategic objectives of enterprise risk. Eight horizontal rows or risk components. Multiple levels to describe any enterprise, from a ‘‘headquarters’’ entity level to individual subsidiaries. Depending on organization size, there can be many ‘‘slices’’ of the model here. E1C04 08/06/2010 13:58:15 100 & Page 100 Understanding Risk Management through COSO ERM Risk Management Objectives C GI TE RA ST NS IO AT ER CO Objective Setting Event Identification Risk Components IAN L MP Risk Assessment Risk Response Control Activities SUBSIDIARY BUSINESS UNIT DIVISION ENTITY LEVEL RE OP Internal Environment CE G IN RT PO Information and Communication Monitoring Entity and UnitLevel Component EXHIBIT 4.6 COSO ERM Framework Source: Robert R. Moeller, COSO Enterprise Risk Management: Understanding the New Integrated ERM Framework (Hoboken, NJ: John Wiley & Sons). Copyright ß 2007, John Wiley & Sons. Used with permission of John Wiley & Sons. This section focuses on the front-facing horizontal components of COSO ERM, with brief discussions on COSO ERM’s other two dimensions and how they all relate to one another. The concept behind the ERM framework is to provide a model for enterprises to consider and understand their risk-related activities at all levels as well as how these risk components impact one another. An objective of this chapter is to help internal auditors at all levels—from from the CAE to staff IT auditors—to better understand COSO ERM and learn how it can help manage a wide range of risks facing enterprises. Because the COSO ERM framework diagram looks very similar to the COSO internal controls framework, some have incorrectly viewed COSO ERM as just a new update to their familiar COSO internal controls framework. However, COSO ERM has different objectives and uses. COSO ERM should not be considered just a new and improved or revised version of the COSO internal control framework. It is much more. The next sections outline this framework from a risk components perspective. & Internal environment component. Looking at the front of the COSO ERM cube, there are eight levels, with the internal environment component located at the top of ERM framework. The internal environment may be thought of as the capstone to COSO ERM. It holds the framework together. This capstone component is similar to the box at the top of an organization chart that lists the CEO as the designated head of a function. This level defines the basis for all other components in an enterprise’s ERM model, influencing how strategies and objectives should be E1C04 08/06/2010 13:58:15 Page 101 COSO ERM: Enterprise Risk Management & & & & & & & & 101 established, how risk-related business activities are structured, and how risks are identified and acted on. The COSO ERM internal environment component consists of these elements: Risk management philosophy. These are the shared attitudes and beliefs that characterize how the enterprise should consider risk in everything it does. More than a message in a code of conduct, a risk management philosophy is the attitude that should allow stakeholders at all levels to respond to high-risk proposals with an answer along the lines of ‘‘No, that’s not the kind of venture our company will be interested in.’’ Risk appetite. Appetite is the amount of risk an enterprise is willing to accept in the pursuit of its objectives. An appetite for risk can be measured in either quantitative or qualitative terms, but all levels of management should have a general understanding of their enterprise’s overall risk appetite. The term appetite was not often used by internal auditors prior to COSO ERM, but it is a useful expression that describes an overall risk philosophy. Board of director attitudes. The board and its committees have a very important role in overseeing and guiding an enterprise’s risk environment. The independent, outside directors in particular should closely review management actions, ask appropriate questions, and serve as a check and balance control for the enterprise. Integrity and ethical values. This ERM element requires more than just a published code of conduct and includes a well-thought-out mission statement and integrity standards. There should be a strong corporate culture to guide the enterprise, at all levels, in helping to make risk-based decisions. This area should be an essential component in every ERM framework today. Commitment to competence. Competence refers to the knowledge and skills necessary to perform assigned tasks. Management decides how these critical tasks will be accomplished by developing strategies and assigning the proper people to perform them. With a strong commitment to competence, managers at all levels should take steps to achieve their promised goals. Organization structure. An enterprise should develop an organization structure with clear lines of authority, responsibility, and appropriate reporting. Every professional has seen situations where an organization does not allow for appropriate lines of communication. There can be many situations in which the organization structure needs improvement to achieve effective ERM. Assignments of authority and responsibility. This ERM component refers to the degree to which authority and responsibility is assigned or delegated. The trend in many enterprises today is to flatten organizations by eliminating middle management levels. These structures usually encourage employee creativity, faster response times, and greater customer satisfaction. However, this type of customerfacing organization requires strong procedures and rules for the staff as well as ongoing management processes so that lower-level staff decisions can be overruled if necessary. All individuals should know how their actions interrelate and contribute to the overall objectives of the enterprise. E1C04 08/06/2010 13:58:15 102 & & Page 102 Understanding Risk Management through COSO ERM Human resource (HR) standards. Practices regarding employee hiring, training, compensating, promoting, disciplining, and all other actions send messages to staff regarding what is favored, tolerated, or forbidden. When management winks at or ignores some gray-area activities rather than taking a strong stand, that message is usually informally and quickly communicated to others. Strong standards are needed to ensure that HR rules are both communicated to all stakeholders and are enforced. Other COSO ERM guidance materials include examples of the components necessary to build an effective internal environment. Many refer to the standards and approaches an enterprise should implement to accept and manage various levels of risk; others refer to just good business practices. No matter whether an enterprise has a high or low appetite for risk, it needs to establish control environment practices to manage those risks. For example, the enterprise can give its sales force rather free rein to do deals without much management supervision and approval. Yet everyone should know the legal, ethical, and management policy limits of those free-rein practices. Processes should be in place such that if anyone steps over the line regarding these limits, swift remedial actions are taken and communicated. The COSO ERM internal environment components of the enterprise’s risk management philosophy and its relative appetite for risk feed other elements of the COSO ERM framework. A risk management philosophy is often defined by the board of directors’ attitudes and their policies, and the concept risk appetite is often a softer measure, where an enterprise has determined that it will accept some risks but reject others in terms of their likelihood and impact. Exhibit 4.7 shows a risk appetite map illustrating where an enterprise should recognize the range in which it is willing to accept risks in terms of their likelihood and impact. This exhibit says an enterprise may be willing to get involved in a high-negative-impact project if there is a low likelihood of an occurrence. There is a third dimension to this chart as well. An enterprise sometimes will have a greater appetite for a more risky endeavor if there is a higher potential return. Objective-Setting Components Ranked below the internal environment in the COSO ERM framework, objective setting outlines important conditions to help management create an effective ERM process. This emphasizes that an enterprise mission statement is a crucial element for setting objectives; it is a general, formal statement of purpose and a building block for the development of specific functional strategies. COSO ERM calls for an enterprise to formally define its goals with a direct linkage to its mission statement, along with measurement criteria to assess whether it is achieving these risk management objectives. The COSO ERM’s objective-setting component should formally define the enterprise’s risk appetite in terms of risk tolerance and guidelines whether to accept these risks or not. Establishing and enforcing risk tolerances can be very difficult, if these rules are not clearly defined, well understood, and strictly enforced. An enterprise should 13:58:15 Page 103 High COSO ERM: Enterprise Risk Management & 103 Medium Exceeding Risk Appetite Within Risk Appetite Low 08/06/2010 Impact E1C04 Low Medium High Likelihood EXHIBIT 4.7 Risk Appetite Map Source: Robert R. Moeller, COSO Enterprise Risk Management: Understanding the New Integrated ERM Framework (Hoboken, NJ: John Wiley & Sons, 2007). Copyright ß 2007, John Wiley & Sons. Used with permission of John Wiley & Sons. establish tolerable ranges of acceptable risks in many areas. For example, products coming off production lines might have acceptable preestablished error rates of no greater than 0.005%. That is an acceptably low error rate in many areas, and production management here would accept the risk of any product warranty claims or damage to the enterprise’s reputation if there were errors within that relatively narrow limit. Of course, today’s quality assurance emphasis on six sigma programs, as discussed in Chapter 28, brings those tolerance limits much tighter.6 The point here is that an enterprise should define its risk-related strategies and objectives and should decide on its appetite and tolerances for these risks. That is, it should determine the level of risks it is willing to accept and, given those risk tolerance rules, how far it is willing to deviate from these preestablished measures. In order to manage and control risks at all levels, an enterprise needs set its objectives and define its tolerances for engaging in risky practices and for adherence to these rules. Things will not work if the enterprise establishes some risk-related objectives but then proceeds to ignore them. Event Identification Events are enterprise incidents or occurrences—external or external—that affect the implementation of an ERM strategy and the achievement of its objectives. Although our tendency is to think of events in a negative sense—determining what went wrong—they can be positive as well. Many enterprises today have strong performance monitoring tools in place, monitoring costs, budgets, quality assurance, compliance, and the like. However, going beyond activities on a production assembly line, monitoring processes should include external economic, natural environmental, and political events. E1C04 08/06/2010 13:58:15 104 & Page 104 Understanding Risk Management through COSO ERM An enterprise needs to clearly define its significant risk events and then have processes in place to monitor them in order to take any necessary appropriate actions. This is a forward-thinking type of process that is often difficult to recognize in many enterprises. Looking at these internal and external potential risk events and deciding which ones require further attention can be a difficult process. Some are immediate needs, and others very future directed. An enterprise should establish processes to review potentially significant risks and then take action. Risk Assessments While the internal environment component is COSO ERM’s cornerstone, risk assessment processes allow an enterprise to consider the extent to which these potential risk-related events may be considered as part of an enterprise’s achievement of its objectives. These risks should be assessed from the likelihood of the risk occurring and its potential impact. A key part of this risk assessment process, however, is the need to consider the inherent and residual risks, as discussed previously. These two concepts imply that an enterprise will always face some risks. After management has addressed the risks that came out of its risk identification process, some residual risks to remedy still will exist. In addition, there are always some inherent risks where management can do little. As mentioned previously in our discussion of inherent risk, Wal-Mart can take some steps to reduce its inherent risks related to market dominance but can do essentially nothing regarding the inherent risk of a major earthquake. Risk likelihood and impact are two other key components necessary for performing risk assessments. Likelihood is the probability or possibility that a risk will occur. In many instances, this can be a key management assessment stated in the terms of a high, medium, or low likelihood of the risk occurring. There are also some good quantitative tools to develop likelihood estimates, but it does little good to estimate the likelihood of a risk occurring unless there is strong supporting data. Estimating the impact if a risk event occurs is a bit easier. Examples include, for ITrelated risks, the impact of a data server and network center catastrophic loss or failure. An enterprise can develop some relatively accurate estimates such as the cost of replacing facilities and equipment, the cost of restoring systems, and the cost of lost business due to the failure. However, the whole concept behind ERM is not to develop precise, actuarial-level calculations regarding these risks but to gain some measure to provide for an effective risk management framework. Those detailed calculations can be delegated to insurance estimators and others. Risk likelihoods and potential impacts can be analyzed through a series of quantitative and qualitative measures. The idea, however, is to assess all of the identified risks and to rank them in terms of likelihood and impact in a consistent manner. That is, each identified risk can be ranked on an overall relative scale of perhaps 1 to 10, with consideration given to the impact and likelihood of each. Then one can identify the risks that should receive the most thorough management attention. Overall approaches to reviewing these various likelihood and impact risks need to be considered. Risk assessment is a key component of the COSO ERM framework. E1C04 08/06/2010 13:58:15 Page 105 COSO ERM: Enterprise Risk Management & 105 This is where an enterprise evaluates all of the identified risks that might impact its various objectives, considers their potential likelihoods and impacts, considers their interrelationship on a unit-by-unit or total enterprise basis, and then develops strategies for appropriate responses. In some respects, this COSO ERM risk assessment process is not too different from the classic risk assessment techniques that have been used over the years. What is unique is that COSO ERM suggests that an enterprise should take a total approach, across all units and covering all major strategic concerns, to identify its risks in a consistent and thorough manner. Risk Response Strategies Having assessed and identified its more significant risks, COSO ERM risk response calls for measured actions to these various identified risks. There should be a careful review of estimated risk likelihoods and potential impacts, with consideration given to associated costs and benefits, to develop appropriate risk response strategies. These risk responses can be handled in any of four basic approaches: 1. Risk Avoidance. This is a strategy of walking away from a risk—such as selling a business unit that gives rise to a risk, exiting from a risky geographic area, or dropping a product line. The problem is that enterprises often do not drop a product line or walk away until after the risk event has occurred with its associated costs. Unless an enterprise has a very low appetite for risk, it is difficult to walk away from an otherwise successful business area or product line on the basis of a potential future risk. Avoidance can be a costly strategy if investments were made to get into an area with a subsequent pull-out to avoid the risk. A collective lessons-learned understanding of past activities often can help with this strategy. If the enterprise had been involved in some area in the past with unfavorable consequences, this may be a good way to avoid the risk once again. With the tendency of constant changes and short employment tenures, this collective history is too often lost and forgotten. An enterprise’s well-understood and communicated appetite for risk is perhaps the most important consideration when deciding if a risk avoidance strategy is appropriate. 2. Risk Reduction. A wide range of business decisions may be able to reduce certain risks. Product line diversification may reduce the risk of too strong of a reliance on one key product line, or splitting IT operations into two geographically separate locations will reduce the risk of some catastrophic failure. There exists a wide range of often-effective strategies to reduce risks at all levels, going down to the obvious and mundane, such as cross-training employees to reduce the risk of someone departing unexpectedly. 3. Risk Sharing. Virtually all enterprises regularly share some of their risks through the purchase of insurance, but other risk-sharing techniques are available as well. For financial transactions, an enterprise can engage in hedging operations to protect from possible price fluctuations, or an enterprise can share potential business risks and rewards through corporate joint venture share-the-risk agreements. The idea is to have another party acceptsome of a potential risk as well asto share in any resultant rewards. E1C04 08/06/2010 13:58:15 106 & Page 106 Understanding Risk Management through COSO ERM 4. Risk Acceptance. This is a strategy of taking no action, such as when an enterprise self-insures by taking no action to reduce a potential risk. Essentially, an enterprise should look at a risk’s likelihood and impact in light of its established risk tolerance and then decide whether to accept that risk or not. Acceptance is often an appropriate strategy for some of the risks that an enterprise faces. Management must develop a general response strategy for each of its risks using an approach built around one or a mixture of the risk avoidance strategies. In doing so, it should consider the costs versus benefits of each potential risk response as well as strategies that best align with the enterprise’s overall risk appetite. For example, an enterprise’s recognition that the impact of a given risk is relatively low would be balanced against a low risk tolerance that suggests that insurance should be purchased to provide a potential risk response. For many risks, appropriate responses are obvious and almost universally understood. An IT operation, for example, spends the time and resources to back up its key data files and implements a business continuity plan. There are typically no questions regarding the need for these basic approaches, but various levels of management may question the frequency of backup processes or how often the continuity plan needs to be tested. That is, they may question the extent and cost of planned risk prevention measures. An enterprise should go back to its established risk objectives as well as the tolerance ranges for those objectives. Then it should readdress both the likelihoods and impacts associated with each to develop an overall set of the planned risk responses. This is perhaps the most difficult step in building an effective COSO ERM program. It is comparatively easy to identify a 5% likelihood risk that there will be a fire in the scrap materials bin and then to establish a risk response remedy to install a nearby fire extinguisher. However, responses to most risks are much more complex and require fairly detailed planning and analysis. If there is a risk that an enterprise could lose an entire manufacturing operation due to a key but old equipment plant production failure, potential risk responses might include: & & & & Acquire backup production equipment to serve as spare parts for cannibalization. Shut down the manufacturing production line with plans to move it elsewhere. Arrange for a specialized shop to rebuild/reconstruct the old equipment. Reengineer the manufactured product along with plans for new product introduction. Developing risk responses requires a significant amount of planning and strategic thinking. For example, one of the response strategies just mentioned is to acquire a set of backup equipment. If that is to be the approved strategy, action must be taken to acquire the backup equipment before this activity can even be identified as an actual risk response strategy. The idea is that all risks listed on such an analysis should be measured against the same impact factors, based on an accept, avoid, share, or reduce risks strategy. COSO ERM calls for risks to be considered and evaluated on an entity- or portfoliowide basis. This can be a difficult process in a large, multi-unit, multiproduct enterprise, but it provides a starting point in getting the various risks organized for identification of E1C04 08/06/2010 13:58:15 Page 107 COSO ERM: Enterprise Risk Management & 107 more significant risks that may impact the enterprise. The idea here is to look at these various potential risks, their probability of occurrence, and the impacts of each. A good analysis here should highlight areas for more detailed attention. Control Activities COSO ERM’s control activities are the policies and procedures necessary to ensure action on identified risk responses. Although some of these activities may relate to an identified and approved risk response in only one area of the enterprise, they often overlap across multiple functions and units. The control activities component of COSO ERM should be tightly linked with the risk response strategies and actions previously discussed. Having selected appropriate risk responses, an enterprise should select control activities necessary to ensure that these control activities are executed in a timely and efficient manner. The process of determining if control activities are performing properly is very similar to completing SOx Section 404 internal control assessments, as discussed in Chapter 1. COSO ERM calls for approaches of identifying, documenting, testing, and then validating these risk protection controls. Many control activities under the COSO internal controls framework are fairly easy to identify and test, including these internal control areas: & & & & Separation of duties. Essentially, the person who initiates a transaction should not be the same person who authorizes that transaction. This area is discussed in Chapter 6 on IT general controls. Audit trails. Processes should be organized such that final results can be easily traced back to the transactions that created them. This area is particularly important for reviews of IT applications, as discussed in Chapter 10. Security and integrity. Control processes should have appropriate control procedures such that only authorized persons can review or modify them. IT security controls are discussed in Chapter 19. Documentation. Processes should be appropriately documented. Chapter 12 discusses the importance of IT documentation and records management systems. These well-recognized control procedures are applicable to all IT internal control processes and apply to many risk-related events. Many IT professionals—whether they have an accounting and auditing background or not—often can easily define some of these key controls, which are necessary in most business processes. For example, if asked to identify the types of internal controls that should be built into an accounts payable system, many would identify the significant control points that checks issued from the system must be authorized by independent persons, that accounting records must be in place to keep track of the checks issued, and that the check-issuing process should be such that only authorized persons can initiate such transactions. These are generally well widely understood internal control procedures. An enterprise often faces a more difficult task in identifying control activities to support its ERM framework. Although there is no accepted or standard set of ERM control activities at this time, the COSO ERM documentation suggests several areas: E1C04 08/06/2010 13:58:15 108 & & & & & & & Page 108 Understanding Risk Management through COSO ERM Top-level reviews. Senior management should be very aware of the identified risk events within their organizational units and perform regular top-level reviews on the status of identified risks. Direct functional or activity management. In addition to top-level reviews, supporting unit managers should have a key role in risk control activity monitoring. Information processing. Whether it is IT systems and processes or softer forms such as paper-based or messages, information processing represents a key component in an enterprise’s risk-related control activities. Appropriate control procedures should be established with an emphasis on enterprise IT processes and risks. Physical controls. Many risk-related concerns involve physical assets, such as IT equipment, inventories, securities, and physical plants. Whether it is dealing with physical inventories, inspections, or plant security procedures, an enterprise should install appropriate risk-based physical control activity procedures. Performance indicators. The typical enterprise today employs a wide range of financial and operational reporting tools that can also support risk event–related performance reporting. Where necessary, performance tools should be modified to support this important COSO ERM control activity component. Segregation of duties. Segregation of duties is a classic control activity: The person who initiates certain actions should not be the same person who approves them. These control activities can be expanded to cover other key areas. Some will be specific to individual units within the enterprise, but each control activity, singly and collectively, should be important components of supporting the enterprise’s ERM framework. The next sections describe control activities from many perspectives of IT operations. Information and Communication This COSO ERM component links together each of the other components. For example, the risk response component receives residual and inherent risk inputs from risk assessment as well as risk tolerance support from the objective-setting component. COSO ERM risk response then provides risk response and risk portfolio data to control activities as well as feedback to the risk assessment. Although it is relatively easy to describe how information should be communicated from one COSO ERM component to another in a simple flow diagram, in practice the process is far more complex. Many enterprises have a complex web of operational and financial information systems for their basic processes that often are not very well linked. These linkages become even more complex for many ERM processes, given that many basic enterprise applications do not lend themselves directly to processes for risk identification, assessment, and risk response. Going beyond a comprehensive ERM information application for an enterprise, there is a need to develop risk monitoring and communications systems that link with customers, suppliers, and other stakeholders. E1C04 08/06/2010 13:58:15 Page 109 COSO ERM: Enterprise Risk Management & 109 The information half of the ERM information and communication component is normally thought of in terms of IT strategic and operational information systems. ERM communication is the second aspect of this component. It talks about communication beyond just IT applications, such as the need for mechanisms to ensure that all stakeholders receive messages regarding the enterprise’s interest in managing its risks abs using a common risk language throughout the enterprise. COSO ERM will be of little value to an enterprise unless the overall message of its importance is communicated to all stakeholders in a common and consistent manner. Monitoring Component Placed at the base of the stack of ERM framework model components shown in Exhibit 4.6, monitoring processes are necessary to determine that all installed ERM components work effectively. People in the enterprise change, as do supporting processes and internal and external conditions, but the monitoring component helps ensure that ERM is working effectively on a continuous basis. The monitoring component should include processes to flag exceptions or violations in other components of the ERM process. For example, an IT accounts receivable billing system should identify the overall financial and operational risks if customer bills are not paid on a timely basis. An ongoing—almost real-time—credit collections monitoring tool could provide senior management with day-to-day and trending data on the status of collections. Dashboard monitoring tools, discussed in Chapter 14, are types of ERM monitors that can work on a continuous basis. Going beyond IT dashboard tools, enterprise management should take an overall responsibility for ERM monitoring. In order to establish an effective ERM framework, monitoring should include ongoing reviews of the overall ERM process, ranging from identified objectives to the progress of ongoing ERM control activities. Separate or individual evaluation monitoring processes refers to detailed reviews of individual risk processes by a qualified reviewer, such as internal or IT audit. Here the review can be limited to specific areas or cover the entire ERM process for an enterprise unit. Internal audit is often the best internal source to perform such specific ERM reviews. The role of internal audit in the ERM process and its role in monitoring, in particular, is discussed in the next sections. Other Dimensions of COSO ERM: Risk Management Objectives Although much of our COSO ERM discussion here is on the front-facing side of the three-dimensional framework, the two other dimensions—the operational and organizational levels—should always be considered as well. Each component of COSO ERM operates in this three-dimensional space, where each must be considered in terms of the other related categories. The top-facing components of strategic, operations, reporting, and compliance risk objectives are important for understanding and implementing COSO ERM. In addition, although Exhibit 4.6 shows each of these topfacing risk objectives as having the same relative size, the category of operations-level risk objectives is often viewed as a much broader and higher-exposure risk category than the others. E1C04 08/06/2010 13:58:15 110 & Page 110 Understanding Risk Management through COSO ERM As an example, the top-facing component of the three-dimensional ERM framework, the operations-level risk objective, calls for the identification of risks for each enterprise unit or component. Identification of these risk objectives often requires detailed information gathering and analysis, particularly for a larger enterprise covering multiple geographic areas, product lines, or business processes. Of course, IT operations are usually a major component here. In order to gather more detailed background information on potential operations risks, an internal audit survey of direct on-the-floor members of the enterprise, along with follow-up questions, should survey potential operations risks across all levels of the enterprise. This type of survey is similar to the types of questions often used in other IT audit internal control assessments. With COSO ERM’s portfolio view of risks, an enterprise should avoid rolling things up to too much of a summary level, missing or rounding off important lower-level risks. Managers at all levels should be aware that they are responsible for accepting and managing the risks within their own operational units. Too often, unit managers believe that risk management is of concern only to some senior-level, headquarters type. The importance of COSO ERM and operations risk management should be communicated to all levels of an enterprise. Internal and IT auditors should act as eyes and ears here and report all observed operations risks. Reporting Risk Management Objectives This risk objective covers the reliability of an enterprise’s reporting, including the internal and external reporting of financial and nonfinancial data. Accurate reporting is critical to an enterprise’s success in many dimensions. News reports often relate the discovery of inaccurate corporate financial reporting and the resultant stock market repercussions for the offending entity; that same inaccurate reporting can cause problems in many areas. No matter what the industry, an enterprise faces major risks from inaccurate reporting. Operating units must make certain that reported results are correct before they are passed up to the next level in the organization, and consolidated numbers must be accurate, whether on financial reports, tax returns, or any of a myriad of other areas. Although good internal controls are necessary to ensure accurate reporting, COSO ERM is concerned about the risk of authorizing and releasing inaccurate reports. Strong internal controls should minimize the risk of errors, and an enterprise should always consider the risks associated with inaccurate reporting. Small errors and discrepancies can be ignored over time until a major error needs to be disclosed. The risk of such inaccurate reporting should be a concern at all levels of the enterprise. Legal and Regulatory Compliance Risk Objectives Every enterprise has requirements to comply with a wide range of laws, governmentimposed regulations, or industry standards. Although compliance risks can be monitored and recognized, legal risks are sometimes totally unanticipated. In the United States, for example, an aggressive plaintiff legal system can pose a major risk to otherwise well-intentioned enterprises. Asbestos litigation during the 1990s and beyond E1C04 08/06/2010 13:58:15 Page 111 COSO ERM: Enterprise Risk Management & 111 is an example. A fibrous mineral, asbestos has three extraordinary characteristics: It works as an insulator for heat and electricity; it resists other dangerous chemicals; and, when inhaled, it has been found to cause illnesses that may take decades to develop. Asbestos is a natural insulation material that previously was used extensively in building materials and was considered totally benign. Too much direct contact with asbestos fibers over time, however, can cause severe lung problems and even death. Underground miners extracting asbestos have met that fate. However, in the past, asbestos was used in many products, such as wrappers to insulate heating pipes or as fire protection wall barriers. The risks to persons working or living in a structure with these asbestos-sealed pipes are minimal, but aggressive litigators have brought actions against corporations, claiming that anyone who could have had any contact, no matter how minimal, with a product that used asbestos could be at risk sometime in the future. The result was litigation directed against companies that had manufactured products containing some asbestos, calling for damages based on potential human risks in future years. Because of huge damage awards, virtually all major corporations that once used asbestos have gone bankrupt or out of business, or have had to pay huge court-imposed damage losses. This is the type of legal risk that is very difficult to anticipate but that can be disastrous to an enterprise. COSO ERM recommends that compliance-related risks be considered for each of the risk framework components, whether in the context of the internal environment, objective setting, or risk monitoring, as well as across the enterprise. The COSO ERM guidance material does not offer much additional material on this compliance objective other than to state that it refers to conformance with applicable laws and regulations. These COSO ERM elements are important components of the risk management framework that need to be communicated and understood. All enterprises face a wide range of legal and regulatory compliance requirements; some impact virtually all enterprises, and others are related to only single business units in a specialized industry sector. The nature of those compliance risks needs to be communicated and understood through all levels of an enterprise. This is an area where an enterprise may accept a certain level of risk in terms of its concerns regarding legal compliance. Although an enterprise should not deliberately ignore a major law because of a feeling it never will be caught, it should always take a reasoned approach to risks in conjunction with its overall philosophy and risk appetites. For example, many regulatory rules specify that all expenditures over US$25 must be supported by a receipt. Although there usually are no reasonableness guidelines here, another enterprise could decide that ‘‘all expenditures’’ goes down to an employee travel expenses of less than $1.00, while another will require receipts for anything above $25.00. The latter enterprise has made a decision that the costs of documenting small expenditures is greater than any fine it might receive if caught in a regulatory compliance issue. This type of a risk-related decision is similar to the SOx Auditing Standard No. 5 on financial internal controls rules discussed in Chapter 1. In order to manage and establish legal and regulatory risk objectives, the board of directors, the CEO, and members of management need to have an understanding of the nature and extent of all of the regulatory risks the enterprise faces. The legal department, key managers, internal audit, and others can help in assembling this information. There are many regulatory E1C04 08/06/2010 13:58:15 112 & Page 112 Understanding Risk Management through COSO ERM enterprise-level risks ranging from major to minor, but regulatory risks are never ‘‘minor’’ when an enterprise is found to be in violation of one or another of them. Another Dimension of COSO ERM: Entity-Level Risks The third dimension of the COSO ERM framework calls risks to be considered on an organization or entity level. Exhibit 4.6 shows four divisions in this framework diagram: entity level, division, business unit, and subsidiary risks. This is not a prescribed company-type division, and COSO ERM suggests that risks should closely follow the official organization chart. COSO ERM risks should be identified and managed within each significant organizational unit, including risks on an entity-wide basis through individual business units. Consider an enterprise with four major operating divisions and with multiple business units or subsidiary units under which each would have an ERM framework that reflects all of these units. These risks are important on an overall organizational level, but there should be some consideration of these risk crossing unit boundaries on a unit-by-unit basis to as low a level as necessary to allow the enterprise to understand and manage its risks. COSO ERM does not specify how thinly these unit-level risks should be sliced, and the criticality and materiality of individual business units should be given consideration. For a major fast-food restaurant chain with thousands of units, it almost certainly would not be reasonable to include each individual unit as a separate component in the risk model. Rather, management should define its organizational-level risks at a level of detail that will cover all significant, manageable risks. Entity Risks Encompassing the Entire Enterprise Multiple business unit–level risks should roll up to their entity-level risks. It is easy for an enterprise to consider some unit-level risks using pre-SOx public accounting terminology as being ‘‘not material,’’ an enterprise has to think of all risks as potentially significant. For example, consider a relatively small subsidiary in a developing country that is manufacturing casual clothing. Often, such a subsidiary unit would be so small in terms of total corporate revenue contributions or its relative size that it can easily slip under the radar screen on a senior corporate level. However, due to political or social issues at the host country, the enterprise may soon find itself at the center of attention regarding this small subsidiary operation. In such a situation, journalists may ask the CEO to publicly comment on policies and procedures at that operation, even though the CEO may only vaguely know of its existence. The point here is that both major as well as seemingly small risks can impact an entire enterprise. The delivery of tainted food produced at one small unit of a large fastfood chain can impact the prospects and reputation of the total enterprise. Although it is relatively easy to identify high-level, entity-wide risks, such as compliance with SOx Section 404, and to identify and monitor these local unit are part of the COSO ERM process as well, and care must be taken that smaller potential risks are not ignored. As risks are identified through organization-wide objective setting, they should be considered on an entity-wide basis as well as by individual operating units. Those E1C04 08/06/2010 13:58:15 Page 113 IT Audit Risk and COSO ERM & 113 individual unit risks should be first reviewed and consolidated to identify any key risks that may impact the overall organization. Business Unit Entity-Level Risks Entity risks must be considered for each significant organizational unit, whether a major production division with multiple plants or a small IT service unit. Even the risks identified in the minority ownership position in a foreign country sales company—risks unique to that unit—should roll up to the operating division and then to the entity. An example here could be entity-level risks that might result from failures in manufacturing or human rights standards from a small subsidiary in a developing country. Risk events here can cause an embarrassment to the overall enterprise, but they should have been controlled all the way down to that small company unit. The previously discussed Bhopal, India, plant disaster brought down the parent corporation. Depending on the complexity and number of operating units, risk responsibility often can best start as a push-down process where corporate-level management formally outlines its major risk-related concerns and asks responsible management at each major division to survey risk objectives through the operating units within that division. In this manner, significant risks can be identified at all levels and then managed at levels where they can receive the most direct, local support. A major concept surrounding COSO ERM is that an enterprise faces a wide range of significant risks at all levels. Some may be significant while others may be just troubling annoyances and viewed as minor. The COSO ERM framework provides a mechanism to consider these risks; it is an important tool to help an IT auditor identify and assess effective internal controls. IT AUDIT RISK AND COSO ERM The COSO ERM framework addresses a risk management approach that is very useful for IT auditors. It is applicable to all industries and encompasses all types of risk. With its focus on recognizing an enterprise’s appetite for risk and the need to apply risk management within the context of overall strategy setting, COSO ERM presents IT auditors with a risk management methodology that is specified by but unfortunately undefined in the IIA and ISACA auditing standards. COSO ERM is an important tool for understanding the multiple risks an enterprise faces today and is particularly useful in an IT internal controls environment. IT auditors will encounter risk and risk management issues in many areas where they are performing reviews, and the effective IT auditor should understand risk management processes. All too often, an IT auditor performing an internal controls review in some area is told that the area was or was not selected because of ‘‘risk considerations.’’ IT Auditors should understand basic risk management processes in order to ask the right questions and to review the adequacy of those risk management processes. COSO ERM is an area where an IT auditor can improve the overall internal controls processes in the areas reviewed. With a focus on the COSO ERM framework E1C04 08/06/2010 13:58:16 114 & Page 114 Understanding Risk Management through COSO ERM as well as general good risk management practices, IT auditors can provide a service to their enterprise by planning and performing reviews of enterprise risk management processes. Of course, to review COSO ERM practices and implementation procedures, IT auditors need to develop a strong understanding of ERM controls and processes. The idea for IT auditors is to establish some high-level review objectives for the effectiveness of COSO ERM, gather detailed implementation data, and then assess the effectiveness of COSO ERM in an enterprise and as a tool to support and enhance SOx compliance. Exhibit 4.8 provides guidance for auditing COSO ERM IT audit procedures. Step Audit Procedure 1 Meet with appropriate managers to gain an understanding of the enterprise’s ERM implementation strategy, its planned scope, and current implementation status. 2 Develop a strategy for reviewing ERM processes, perhaps with a focus emphasizing all internal environment processes on an entity level as well as the status of all components for a selected subsidiary or business unit. 3 Develop detailed internal audit plans for the components selected for review, and publish engagement letters announcing the planned audits. 4 Review enterprise-wide ERM guidance materials in place to assess whether ERM objectives are being adequately communicated and assess areas where communication may be lacking. 5 Risk management philosophy and appetite 5.1 Meet with appropriate members of management to assess whether a risk management philosophy has been defined and communicated. 5.2 Through surveys or interviews, meet with selected members of the enterprise to determine if the risk appetite has been communicated. 6 Risk management integrity and ethical values 6.1 Review published codes of conduct and other materials to determine if risk-related ethical values are being communicated. 6.2 Review a sample of enterprise communications and assess whether attention is given to ERM philosophies. 7 Risk management organization structure 7.1 Meet with human resources management to assess whether processes are in place to communicate the ERM philosophy to enterprise. 7.2 Review the code of conduct records to determine that it has been periodically updated, that all stakeholders have acknowledged it, and that code compliance records are in place. 7.3 Based on a review of organization charts and other documentation, assess whether the ERM philosophy appears to be in place throughout selected units in the enterprise. 8 Select one subsidiary or enterprise unit to determine if enterprise-wide ERM objectives and risk components are in place for the selected unit 8.1 Assess compliance with ERM internal objectives for the selected business unit. 8.2 Assess compliance with ERM objectives-setting processes for the selected business unit. 8.3 Assess compliance with ERM event notification processes for the selected business unit. EXHIBIT 4.8 Auditing COSO ERM IT Audit Procedures E1C04 08/06/2010 13:58:16 Page 115 Notes & 115 8.4 Assess compliance with ERM risk assessment for the selected business unit. 8.5 Assess compliance with ERM risk response processes for the selected business unit. 8.6 Assess compliance with ERM control activity processes for the selected business unit. 8.7 Assess compliance with ERM information and communication processes for the selected business unit. 8.8 Assess compliance with ERM risk monitoring processes for the selected business unit. EXHIBIT 4.8 (Continued ) Because the two framework models look quite similar, it is very easy to miss thinking about the unique characteristics of COSO ERM as compared to COSO internal controls. As discussed in Chapter 1, it took many years for the COSO internal controls framework to be recognized as more than an interesting technical study. COSO internal controls has now been codified as an auditing standard by the American Institute of Certified Public Accountants’ Auditing Standards Board, and the Public Company Accounting Oversight Board mandated that COSO internal controls should be the internal controls standard. Arriving after SOx, COSO ERM does not yet have that same level of recognition. The IIA was an important early proponent. Elements of ERM can be seen in the new version of the ISACA control objectives for IT (CobiT) framework, as discussed in Chapter 2, but it still does not have the same importance for an enterprise as do COSO internal controls. This recognition may take some time. The facts that the two frameworks look alike and both have COSO in their names have caused some confusion. However, the riskrelated emphasis on the new auditing standards as well as an increasing recognition of risk issues in professional literature has somewhat increased professional interest in enterprise risk management. The three-dimensional ERM framework helps to place risk and internal control issues in a better perspective when evaluating IT internal controls. Risk management and COSO ERM, in particular, are standards that should be part of every IT and internal auditor’s tools. IT auditors should use risk management principles when deciding which areas to select for their reviews and then use risk principles when assessing audit evidence. Perhaps even more important, COSO ERM will grow in importance and recognition as more enterprises understand and adopt the ERM framework. IT auditors should understand COSO ERM to both audit compliance to these identified processes and should act as consultants to management to help with more effective implementations. NOTES 1. COSO Enterprise Risk Management—Integrated Framework, COSO. 2. A straw man is an old military term meaning that a man or an idea that is filled only with straw is easy to attack or refute. 3. ‘‘Bhopal Faces Risk of Poisoning,’’ November 14, 2004, http://news.bbc.co.uk/2/hi/ south_asia/4010511.stm Note: This one of many references on this issue. A search for ‘‘Bhopal, India’’ and ‘‘Dow Chemical’’ will yield a large amount of information. E1C04 08/06/2010 13:58:16 116 & Page 116 Understanding Risk Management through COSO ERM 4. Institute of Internal Auditors, GAIT for Business and IT Risk (Altamonte Springs, FL: www.theiia.org. 5. ‘‘COSO Releases a New Risk Management Framework,’’ Accounting Today, October 25, 2004. 6. Six sigma is a disciplined methodology for eliminating defects (driving toward six standard deviations between the mean and the nearest specification limit) in any process, from manufacturing to transactional and from product to service. E1C05 09/18/2010 11:1:19 Page 117 5 CHAPTER FIVE Performing Effective IT Audits I N F O R M A T I O N T E C H N O L O G Y ( I T ) A U D I T O R S have many roles and responsi- bilities in today’s IT-centric enterprises, but the most important of these is to perform effective IT audits, under the overall supervision of their audit committee. To be effective, it is important that an IT auditor has a good understanding of the Committee of Sponsoring Organizations’ (COSO) internal controls framework and the Sarbanes-Oxley (SOx) internal controls assessment standards discussed in Chapter 1. Perhaps even more important for any auditor reviewing IT controls is an understanding of the IT Governance Association’s control objectives for IT (CobiT) framework introduced in Chapter 2. An understanding of the risk management concepts discussed in Chapter 4 is another essential key IT audit knowledge requirement. In addition, every IT internal auditor is expected to follow the appropriate audit standards as issued by the Institute of Internal Auditors (IIA) as well as the Information Systems Audit and Control Association (ISACA). The preceding four chapters have outlined some of these essential tools necessary for performing effective IT audits and reviewing enterprise IT security and internal controls. This chapter brings these IT audit building blocks together to discuss what it takes to perform effective IT audits. With IT-related internal control and security issues so pervasive in all enterprises today, skilled resources should be in place to assess enterprise IT security and control issues and then to independently report the results of those reviews to appropriate levels of management and the board of directors’ audit committee. For most enterprises today, IT audit is a specialized skill that is an essential element of the overall internal audit function. Sometimes IT audit processes are performed by a separate group of specialists, and in other cases it is an additional skill performed by the regular operational and financial internal auditors. The chapter discusses this internal audit organization structure, including position descriptions for IT audit specialists. In addition, we discuss approaches for organizing and planning IT audits, including 117 E1C05 09/18/2010 11:1:19 118 & Page 118 Performing Effective IT Audits reviews of general IT controls, specific applications, and specific IT areas, such as security and privacy controls. In addition, this chapter highlights techniques for developing audit programs for performing consistent IT audits and approaches for developing IT audit workpapers. The chapter then goes through the activities necessary to perform an effective IT audit, including steps to gather and assess audit evidence. Although it is a major subject itself, we also look at approaches for pulling samples of audit evidence and then evaluating those sample results to extract audit conclusions. We then discuss the full circle of IT audit activities, including reporting results through effective internal audit reports. An effective IT audit group is usually a key component of an overall enterprise internal audit function. The chapter concludes with a discussion of coordinating IT audits with other financial and operational internal audit activities. From an overall enterprise perspective and certainly from a board of directors’ audit committee view, IT audit should be a key resource in assessing and helping to improve the overall security and control environment in an enterprise. IT AUDIT AND THE ENTERPRISE INTERNAL AUDIT FUNCTION IT auditors generally operate on several different levels. Sometimes they are a special audit group that provides IT-related support to a public accounting firm’s financial attest audit function, or they may serve in an independent consulting firm providing IT audit services. Most commonly, however, they act either as IT audit specialists within an enterprise’s internal audit function or serve as internal auditors with strong IT functions. Our objective here is not to discuss all of the requirements to build a successful internal audit function in today’s enterprise; other reference sources for building an overall internal audit function are available.1 However, as a key element, every internal audit function needs to have an internal audit charter, approved by its audit committee and senior management. An internal audit charter is a formal document to describe the mission, independence and objectivity, scope and responsibilities, authority, accountability, and standards of the enterprise’s internal audit function. Because internal audit is a function that has free rein to look at a wide range of records and to ask questions at all levels, some type of enterprise-level authorizing authority is needed. Since the internal function reports to the audit committee of the board in a corporate structure, that audit committee should authorize internal audit’s rights and responsibilities through a formal authorizing document or resolution—usually called an internal audit charter. There are no fixed requirements for such an authorizing document, but an internal audit charter should affirm internal audit’s: & & & Independence and objectivity Scope of responsibility Authority and accountability This charter, then, is the authorizing document that an internal auditor can use when a manager in a separate and sometimes remote organizational unit questions why E1C05 09/18/2010 11:1:19 Page 119 IT Audit and the Enterprise Internal Audit Function & 119 an internal auditor is asking to see or review certain documents or to gain access to some enterprise facility. The charter says that senior management—the board of directors’ audit committee—should have access to enterprise records. More important, the charter provides a high level of authorization for the enterprise’s internal audit function. There is no fixed format for the contents of an audit charter. The IIA’s international internal audit standards, as discussed in Chapter 3, refer to the need for an internal audit charter, but the IIA does not provide that much specific format guidance. Exhibit 5.1 is an example internal audit charter from Global Computer Products, an example company that we will be using in other chapters. This example charter clearly outlines internal audit’s authority as well as such responsibilities as developing a risk-based audit plan and issuing timely audit reports. It also makes appropriate high-level references to the role of IT audit as part of the company’s internal audit functions. Internal Audit Department Authorizing Charter Internal Audit’s Mission The mission of Global Computer Products Internal Audit is to ensure that company operations follow high standards both by providing an independent, objective assurance function and by advising on best practice. By using a systematic and disciplined approach, Internal Audit helps Global Computer Products accomplish its objectives by evaluating and improving the effectiveness of risk management, internal control, information technology systems, and governance processes. Independence and Objectivity To ensure independence, Internal Audit reports directly to the Board of Directors’ Audit Committee, and to maintain objectivity, Internal Audit is not involved in day-to-day company operations or internal control procedures. Scope and Responsibilities The scope of Internal Audit’s work includes the review of risk management procedures, internal control, information systems, and governance processes. This work also involves periodic testing of transactions, best practice reviews, special investigations, appraisals of legal and regulatory requirements, and measures to help prevent and detect fraud. To fulfill its responsibilities, Internal Audit shall: & Identify and assess potential risks to Global Computer Product’s operations. & Review the adequacy of controls established to ensure compliance with policies, plans, procedures, and business objectives. & Assess the reliability and security of financial and management information and supporting systems and operations that produce this information. & Assess the reliability and integrity of information systems controls, including installed applications, server-based computer hardware, and connecting networks. & Assess the means of safeguarding assets. & Review established processes and propose improvements. EXHIBIT 5.1 Internal Audit Charter Example (Continued ) E1C05 09/18/2010 11:1:19 120 & & & & Page 120 Performing Effective IT Audits Appraise the use of resources with regard to economy, efficiency, and effectiveness. Follow up recommendations to make sure that effective remedial action is taken. Carry out ad hoc appraisals, investigations, or reviews requested by the Audit Committee and Management. Internal Audit’s Authority In order to promote effective controls at reasonable cost, Internal Audit is authorized, in the course of its activities, to: & Enter all areas of Global Computer Products operations and have access to any documents and records considered necessary for the performance of its functions. & Require all members of staff and Management to supply requested information and explanations within a reasonable period of time. Accountability Internal Audit shall prepare, in liaison with management and the Audit Committee, an annual audit plan that is based on business risks, the results of other internal audits, and input from management. The plan shall be presented to senior management, including the General Counsel, for approval by the Audit Committee. Any needed adjustments to the plan should be communicated to and approved by the Audit Committee. Internal Audit is responsible for planning, conducting, reporting, and following up on audit projects included in the audit plan, and deciding on the scope and timing of these audits. The results of each internal audit will be reported through a detailed audit report that summarizes the objectives and scope of the audit as well as observations and recommendations. In all cases, follow-up work will be undertaken to ensure adequate response to internal audit recommendations. Internal Audit also will submit an annual report to senior Management and to the Audit Committee on the results of the audit work including significant risk exposures and control issues. Standards Internal Audit adheres to the standards and professional practices published by the Institute of Internal Auditors as well as the appropriate standards issued by the Information Technology Governance Institute for information technology-related reviews. EXHIBIT 5.1 (Continued ) An internal audit charter would be little more than a nice-sounding document unless there is a strong internal audit function in place to launch and perform these key internal audit activities. These include understanding the areas in any enterprise that should be candidates for internal audit reviews, building an effective internal audit organization and team, and establishing supporting procedures to allow those internal audits. Although an internal audit charter is an essential authorization to launch a new internal audit function, many if not most internal audit functions today have a charter that may have been developed and approved in the past. If one is already in place, it is often a good idea for the chief audit executive (CAE) to revisit that charter and present it to the audit committee to reaffirm their understanding of the role and responsibilities of internal audit. In particular, the charter should include some clear authorizing language for IT audit activities. An approved charter gives internal audit, including its IT audit function, the authority to begin its audit assurance activities throughout the enterprise. There are many ways to organize an internal audit function, and its IT audit resources are often established as a separate group of internal audit specialists, performing their own E1C05 09/18/2010 11:1:19 Page 121 IT Audit and the Enterprise Internal Audit Function EXHIBIT 5.2 & 121 Internal Audit IT Audit Organization Chart reviews but closely coordinated with the normal financial and operational internal audit activities. In this arrangement, IT auditors frequently report to an IT audit manager under the CAE with an organization structure where IT audit operates separately from the operational and financial internal auditors but closely coordinates their work under the CAE. There are many variations possible here, but Exhibit 5.2 shows an internal audit organization chart with IT audit as a separate unit under the CAE. No matter where this internal audit resource fits in the internal audit department, IT auditors should have a minimum level of special IT-related skills. These skills go beyond such basic skills as knowing how to construct a complex database retrieval request program or understanding IT network security risks, reaching into such areas as how to build and run an effective enterprise IT operation. If he or she does not aspire to become a senior internal audit manager or even the CAE, the effective IT auditor should have many of the overall skills and knowledge to be a potential chief information officer (CIO), the head of an enterprise IT function. Many IT auditors have a broad, general level of IT knowledge and skills; others specialize in such IT knowledge areas as having strong skills in developing computerassisted audit tools and techniques (CAATTs), as discussed in Chapter 13, or as an IT audit security specialist, discussed in Chapter 19. However, Exhibit 5.3 outlines the overall skill requirements for most IT auditors. This exhibit summarizes key IT audit skill requirements, and some internal auditors performing reviews in operational or financial areas may have some of the same skills. An IT auditor should be able to understand and assess a wide range of IT controls. E1C05 09/18/2010 11:1:20 122 & Page 122 Performing Effective IT Audits IT systems and knowledge areas are pervasive in all aspects of business and span a wide range of options and technologies. However, an IT internal auditor should be expected to have at least a high-level working knowledge of these areas: & Business application systems—whether for accounting, business or other purposes—and the basic balancing and integrity controls surrounding all automated systems & Data management processes—whether a formal database or spreadsheet data—and the importance of validating and maintaining that data & Systems development life cycle (SLDC) processes to implement and build business application systems, including both in-house development projects and purchased software packages & Storage management and the importance of backup and recovery processes, including such concepts as storage virtualization an storage networks & Computer operating systems basic functions—whether on a hand-held wireless device, a laptop system or larger system—and the potential risks and vulnerabilities if such IT systems are not updated or maintained & Computer systems architectures, with an emphasis on Web, client-server configurations, and telecommunications & IT service operations processes, with an emphasis on problem management, access controls, and general application management & IT service design processes, including continuity, capacity, and information security management processes & Governance and service strategy processes as well as essential IT financial management processes & Programming or coding techniques sufficient to construct and implement computer-assisted audit procedures appropriate to the enterprise environment & Ongoing interest and curiosity to understand and explore newer and evolving technology concepts, such as storage management virtualization EXHIBIT 5.3 IT Internal Auditor Basic Knowledge Requirements ORGANIZING AND PLANNING IT AUDITS Using IT applications controls over a system installed at a server processing facility, this section walks us through the steps necessary to conduct an IT audit of application controls at such a facility. Of course, the overall IT audit process requires a wellorganized and managed internal audit function, led by an involved and effective CAE and a committed audit committee. IT auditing requires a wide range of interrelated skills and knowledge domains that cannot be just described as one sequential set action steps but includes many interrelated activities. This section outlines general steps for performing any IT audit, but it focuses on an IT internal controls review of the purchasing and accounts payable server-based IT application installed at our example company, Global Computer Products. Our hypothetical company has installed a vendor-supplied software package to purchase goods, through formal purchase orders in its production cycle and to recognize their receipt and pay for them through supporting accounting processes. Establishing IT Audit Objectives An effective IT audit function cannot just say ‘‘We haven’t been to our Muddville plant in years. Let’s visit them and schedule an audit.’’ Rather, internal audit candidate E1C05 09/18/2010 11:1:20 Page 123 Organizing and Planning IT Audits & 123 selections should require risk-based assessments, a formal management request, or an audit committee directive to perform a review. Potential audit candidates would be assembled in an IT audit plan and approved by the CAE and audit committee. IT audit must clearly establish some objectives for a planned review. A high-level objective statement should be established for each individual planned IT audit. The statement does not have to consist of detailed lists of requirements but should be an objective statement with sufficient information to tell the auditee (the function to be audited), management, and others what IT internal audit is trying to accomplish when launching an audit in some area. Some examples of internal audit objective statements are: & & & . . . to assess the adequacy of purchasing system internal accounting controls at the Global Computer Products Minneapolis facility as well as the purchasing processes at multiple branch facilities, interfaces to the accounts payable system at corporate headquarters, and automated systems to support these processes. . . . to update documented IT processes and test internal controls, as necessary, for fixed asset management processes to satisfy their Sarbanes-Oxley Act (SOx) Section 404 requirements. . . . to review the internal controls in place over maintenance for the IT configuration management database and supporting procedures. Each of these fairly brief statements describes what IT audit is planning to accomplish in an upcoming review. The project can be expanded as the reviews get started, but these objective statements get an internal audit started or launched. Closely tied to the objective statement, a scope statement is sometimes a valuable addition. For example, an objective statement can identify a planned review of quality management production processes in international operations, and a scope statement would limit the review to only Australia/New Zealand operations. The scope statement better defines what the new audit is trying to accomplish. These IT audit preliminary objective and scope statements should be reviewed with management or others requesting the audit. An effective way to describe these internal audit plans is through an audit planning memo. This document is not presented directly to the auditee yet, but is an internal planning document that describes what IT audit is planning to accomplish, who will be doing the review, and its approximate timing. Such a memo is an essential starting document for initiating an internal audit. Exhibit 5.4 shows a sample audit planning memo where an internal audit supervisor outlines the objectives of a planned internal audit, who will be assigned to do the work, and its estimated timing. Of course, even though our exhibit shows this as a ‘‘memo’’ from the old hard-copy paper days, today’s planning memo would almost certainly be an electronic document. In this example, assume that internal audit has decided to launch a review of the Muddville Division facility operations and that IT audit has decided to coordinate its work at that same facility with others in the internal audit team. We will assume that IT audit has established general guidance audit programs for reviews of these areas and has enough general knowledge about the operations to get started in launching the IT audit. E1C05 09/18/2010 11:1:20 124 & Page 124 Performing Effective IT Audits February 2, 20xx To: Workpaper Files From: L. C. Tuttle, Audit Supervisor Subject: Accounts Payable Systems IT Audit Planning Memo This memo is to document the planned review of key IT purchasing and accounts payable systems processes at Global Computer Products manufacturing facility at, Muddville, MN. The review will be performed by two members of our IT audit staff with L. C. Tuttle as project leader and Herman Hollerith providing support for our review of network and IT systems controls. The objective of this review will be to assess the adequacy of purchasing system internal accounting automated controls at the Global Computer Products Muddville facility as well as the purchasing processes at multiple branch facilities, interfaces to the accounts payable system at corporate headquarters, and automated systems to support these processes. The audit is scheduled to begin on about March 15, 20xx, and has been budgeted for a total of XX hours of time from the on-site audit team. A detailed plan, including expected hours by each auditor, will be prepared prior to the start of this review. The review will emphasize controls over linkages from the purchasing system to other enterprise manufacturing database systems. In addition, the review will update documentation and perform tests, as necessary, to support SOx Section 404 requirements covering this process. All recommendations and audit findings will be reported in a normal internal audit department report. L. C. Tuttle, IT Audit Supervisor W. J. Rawdon, IT Audit Manager EXHIBIT 5.4 Audit Planning Memo Example Sometimes, when IT audit has only limited knowledge of some area, auditors will schedule a preliminary review or request documents for review to gain some general knowledge about the facility. Depending on the role of IT audit in the enterprise, these reviews may be coordinated with regular internal audit activities or may be separate, ITonly reviews. For many audits, it is often best to coordinate IT audit reviews with any other organization internal audit activities. Establishing Preliminary IT Audit Plans A good early IT audit step is to look at other IT and general internal audits either in process or planned for the short term, to consider the availability of internal audit resources, and then to prepare a preliminary internal audit plan. Let us assume that our example Muddville facility is out of town but will not require any long-distance, international travel to visit the location. We also assume that although the IT audit team has an understanding of the general IT operations there, it has little information about the planned application to be reviewed. Based on the information available, IT E1C05 09/18/2010 11:1:20 Page 125 Organizing and Planning IT Audits Date: March 15, 20XX To: Workpaper Files From: L.C. Tuttle, IT Audit Manager Subj: IT Accounts Payable Systems Application Audit Planning Memo & 125 This memo is to document the planned review of the IT accounts payable application at the Global Computer Products Muddville distribution facility. The review is planned to begin on about April 15, 20XX and will be staffed with two members of the IT audit organization, Herman Hollerith and Susan Smyth. The review will include financial transaction testing procedures with key corporate data center financial systems as well as linkages with the Muddville materials purchasing and receiving systems. In addition, we will review IT software change management controls over system components at the Muddville facility as well as attached corporate systems. We are anticipating using our computerassisted audit software as part of our review here. This software accesses some of your Accounts Payable database files for analysis and recalculations. We will advise you of these software requirements when we arrive for the review. Our review will emphasize controls over your recently implemented auto-pay process and its linkages to your distribution database systems. All audit concerns from this review will be discussed with you prior to any release of our audit reports. Please contact us with any questions. L. C. Tuttle, IT Audit Supervisor EXHIBIT 5.5 Preliminary Applications Review IT Audit Plan audit would prepare a preliminary plan for the upcoming Muddville IT applications review audit. Exhibit 5.5 is an example of the type of preliminary IT audit plan that might be developed for such an applications controls review. Because the information here is only very preliminary, the plan does not use specific dates but assumes that two IT audit staff members will be assigned to do the work. The preliminary plan shown only uses approximate estimated hours at this time. Care should always be taken not to seal such preliminary plans in stone, as more information may force all planned estimates up or down. Annual IT Audit Plans Having developed preliminary plans for this audit, IT audit needs to assess other IT audit needs or audit requests for the period, match its plans with regular internal audit review work, assess its budgets and resources, and develop an IT audit plan for the upcoming, often annual period. This would be a high-level plan indicating relative risks and showing audit start and end dates. Exhibit 5.6 is an example of such an annual IT audit plan. Although this plan document shows only IT audit-related projects, normally it would be merged with plans for other operational and financial internal audit activities to show total planned internal audit activities for an upcoming period. IT audit plans, following a format similar to Exhibit 5.6, should be approved by both the audit committee and the CAE. These types of plans should be updated as E1C05 09/18/2010 11:1:20 126 & Page 126 Performing Effective IT Audits Global Computer Products IT Audit Department April to June, 20XX IT Audit Project Schedule Project # Audit Auditor Activity A23-C06 Muddville AP IT Application Review H. Hollerith Document System Controls A23-C06 Muddville AP IT Application Review H. Hollerith Develop and Implement CAATT A23-C06 Muddville AP IT Application Review H. Hollerith A23-C06 Muddville AP IT Application Review A23-C06 April May June 80 20 20 0 30 20 Tests of Transactions 16 16 12 S. Smyth Tests of Transactions 12 8 8 Muddville AP IT Application Review S. Smyth Other Location Coordination 0 10 20 A23-C06 Muddville AP IT Application Review L. Tuttle Manage IT Audit 12 12 12 A28-378 Storage Mgmt. Virtualization Review E. Zimbalist Document Key Audit Points 60 36 12 A28-378 Storage Mgmt. Virtualization Review E. Zimbalist Develop Testing Approach 0 10 10 A28-378 Storage Mgmt. Virtualization Review H. Hollerith Perform Storage Tests 0 8 36 A28-378 Storage Mgmt. Virtualization Review L. Tuttle Manage IT Audit 12 8 8 E04-000 Firewall Security Review S. Smyth Test and Document Controls 0 36 40 E04-137 Firewall Security Review E. Zimbalist Review Security Procedures 60 20 18 EXHIBIT 5.6 IT Audit Annual Plan Example required, but they would provide a basis for launching and monitoring IT audit activities. Going beyond the Exhibit 5.5 preliminary individual audit plan and the approved Exhibit 5.6 IT audit function plan, there are many other documents and procedures necessary to launch a program of IT audits. These include engagement letters that formally announce to an affected management, IT audit’s plans to begin a review in some area.2 Starting with steps for planning an internal audit and then continuing through a variety of audit processes, the next sections outline the steps necessary for an IT audit applications review at a unit of our sample company, a representative IT internal audit. Our objective here, and in other supporting chapters, is to suggest a series of internal audit procedures for performing reviews. Whether as an individual professional or as the enterprise’s IT audit department or function, internal audit will be more effective if all members of the IT audit staff follow consistent, professional procedures in performing their reviews. They will become a strong enterprise resource in the eyes of management, which should expect consistent, quality approaches from the internal audit resources. E1C05 09/18/2010 11:1:20 Page 127 Developing and Preparing Audit Programs & 127 DEVELOPING AND PREPARING AUDIT PROGRAMS IT audits should be organized and performed in a consistent manner with an objective of minimizing arbitrary or unnecessary auditor procedures. To help achieve this goal of audit consistency, IT and all internal auditors use what are called audit programs to perform their audit procedures in a consistent and effective manner for similar types of audits. The term program refers to a set of auditor procedures similar to the steps in a computer program, instructions that go through the same program instructions every time the process is run. For example, a computer program to calculate pay includes instructions to read the time card file of hours worked, look up the employee’s rate stored in another file, and then calculate the gross pay. The same steps apply for every employee unless there are exceptions, such as overtime rates, coded into the payroll program. Similarly, an audit program is a set of preestablished steps an internal auditor performs. An audit program is a tool for planning, directing, and controlling audit work and a blueprint for action, specifying the steps to be performed to meet audit objectives. It represents the auditor’s selection of the best methods for getting the job done and serves as a basis for recording the work steps performed. Effective IT auditors should have a series of generalized audit programs prepared for most of their recurring IT audit activities. Many of these programs, such as one covering an observation of logon practices for visiting a server processing facility, often are used from year to year and from one computer hardware center to another with little change. In other situations, an IT auditor may have to modify a standard program to the unique aspects of a particular audit. In some situations, a standard audit program will not be applicable. For example, an IT auditor may want to review IT controls for a new business entity with some unique control characteristics, or audit management may want to take a different approach because of problems encountered with similar previous reviews. Based on planned audit objectives and data gathered in the preliminary and field surveys, the incharge IT auditor may want to prepare a customized audit program to guide the review. This may be little more than a standardized program with minimal local changes, or it may be a unique set of audit procedures based on the preliminary planning and the results from the field survey. In order to prepare this program, auditors first should have an understanding of the characteristics of what constitutes an adequate audit program. IT Audit Program Formats and Their Preparation An audit program is a procedure describing the steps and tests to be performed by an IT auditor when actually visiting an IT operations facility, reviewing an application, or performing some other audit review process. The program should be finalized after the completion of the preliminary and field surveys and before starting the actual IT audit. It should be constructed with several criteria in mind, the most important of which is that the program should identify the aspects of the area to be further examined and the sensitive areas that require audit emphasis. A second important purpose of an internal audit program is that it should guide both less and more experienced IT internal auditors. For example, management may request that IT audit observe perimeter physical security controls at a data center. This E1C05 09/18/2010 11:1:21 128 & Page 128 Performing Effective IT Audits type of review consists of fairly standard procedures to ensure, among other matters, that all visitors present valid badges to gain entry and that any entry monitor does not just wave people through with no checking. A less experienced IT auditor may not be aware of these procedure steps, and even experienced internal auditors may forget one or another. An audit program outlines the required audit steps for this type of procedure. An established IT audit department should build a library of audit programs, established over time, for tasks such as an audit of small system general controls or a review of program change management. When planning a review where such established audit programs exist, IT audit management needs only to use these established programs with consideration given to any changed conditions that have been discovered through the preliminary or field surveys. The audit program is revised as necessary, with the changes approved by audit management prior to the start of the review. For many IT audit departments, appropriate established audit programs may not be available for many areas. This is because IT auditors typically face a wide and diverse set of areas for review, but they will not have the time or resources to review every area on a frequent basis. Established programs prepared for prior audits, such as specialized application reviews, often become out of date due to new systems or changed technologies. The IT auditor responsible for the field survey or another member of audit management should update any existing audit program or prepare a revised set of audit program steps for the planned review. Depending on the type of planned IT audit, programs usually follow one of three general formats: (1) a set of general audit procedures, (2) audit procedures with detailed instructions for the auditor, or (3) a checklist for compliance reviews. The next examples illustrate these audit program types. Exhibit 5.7 contains a few general audit program steps for an IT applications controls review. The program steps Note: These IT audit program steps represent a limited set of IT audit procedures from what would be a much larger set of audit steps for a review of extended application controls. Step Review Area Control Audit Tests Application Audit Trails Application should contain automated tracking of changes made to data, indicating the time of the change and specific user. Select a sample of changes from change log files and trace to log files and determine if there is evidence of change reviews. Application Audit Trails Application should provide automated tracking and highlighting of overrides to normal processing parameters. Document the nature of override procedures and determine that there is evidence of management reviews and corrective actions when needed. Interface Application Balancing Application should have automated checking processes in place to balance key control balances between systems. Review and document balancing processes, and select a sample of processes to determine that balancing is working correctly. Automated File Functionality Calculations Master files or rating tables should be in place to provide consistent values to all applications. Compare key input values for all impacted applications through walk-through review. EXHIBIT 5.7 Application Processing Controls Sample Audit Program Extract E1C05 09/18/2010 11:1:21 Page 129 Developing and Preparing Audit Programs Audit:____________ Location:____________ & 129 Date:____________ # Audit Steps: Data Center Petty Cash Review 1 Assess the need for a petty cash facility within the IT operations area. Determine if alternative procedures might be available through the corporate finance office or other area. 2 Prior to review, determine who in the IT facility is responsible for the petty cash fund balances, receipt requirements, replenishment procedures, and guidelines for authorized disbursements. 3 Perform this petty cash review on a surprise basis. Identify yourself to cashier, request that the cashier function be closed but observe audit during your initial review, and make a detailed count of cash in the account as well as any included personal checks. 4 Having performed the cash count in presence of cashier, ask cashier to acknowledge results of the auditor’s cash count. 5 If any personal checks found in the cash count were over one day old, ask why they were not deposited or if they are being held as collateral for an employee short-term loan fund. If such a fund, assess propriety of this practice. 6 Reconcile the audited cash count with the fund’s disbursement register, noting any differences. 7 Determine that all cash disbursements recorded have been made to valid employees for authorized IT-specific purposes. 8 Observe office security procedures covering the fund and determine that funds are locked or otherwise secured. 9 Review procedures for fund replenishments. Select a prior period, review its supporting documentation, and reconcile activity to purchase journal. 10 Assess the overall control procedures, propriety, and efficiency of this petty cash process. Comment as appropriate. 11 Determine that the petty cash function is used only for authorized IT facility small cash disbursements rather than a general change or short-term loan fund. 12 Document the results of the review, and take steps to initiate immediate corrective actions if any problems were encountered during this review. Initial & Date W/P Ref. ____________________ Signature ________________________Date ____________ EXHIBIT 5.8 Audit Program: Data Center Petty Cash Review define high-level audit objectives as well as internal audit procedures to test compliance with those objectives. An actual audit program would be much more detailed and extensive, but the exhibit shows the theme of this form of audit program. The program outlines the high level or general audit steps that IT auditors will need to follow, and IT audit should tailor its library of general audit programs covering reviews of major IT applications and data centers to the specific unit or facility that it is reviewing. Exhibit 5.8 is an example of a more detailed audit program covering audit steps for a review of petty cash controls at a data center. It consists of general audit procedures to E1C05 09/18/2010 11:1:21 130 & Page 130 Performing Effective IT Audits review cash at any unit of a multifacility organization. Petty cash controls are one of the smaller, less critical internal control concerns for many operational and data center operations, and an IT auditor often ignores this area when reviewing data center general controls. IT audit often elects to make detailed audit programs even more specific or detailed. The program shows the steps that should be included in any such audit and illustrates an example audit program. Exhibit 5.8 represents a typical IT audit program format where audit tasks are broken into numbered steps with space on the form allowed for the initials and date of the auditor completing the audit step as well as a column for a reference to the workpaper that describes the audit step. For example, for step 2 the start of this process, the IT auditor performing the procedure would document data center cashier responsibilities. Typically, an established internal audit function would have developed these types of audit programs for many of its regular or periodic audits. The IT audit team visiting an organizational unit could then use standard programs to review general and application internal controls in a consistent manner from one unit to the next. This is particularly important in a larger multi-unit enterprise where IT audit management wants to have assurance that controls over the area were reviewed and evaluated in a consistent manner, no matter who is the assigned auditor or the location. This sample audit program is shown as a printed document that could be developed and controlled by internal audit. Of course, such a document would typically be located in the auditor’s laptop system. In some instances, the in-charge auditor might prepare a custom program to evaluate certain special procedures encountered during the field survey. At one time, the checklist-format audit program was internal audit’s most common format. Often a more junior internal auditor was given an audit program composed of a long list of questions requiring ‘‘yes,’’ ‘‘no,’’ or ‘‘not applicable’’ responses and would complete these program steps either through examinations of documents or through interviews. Exhibit 5.9 is an example of a checklist-format audit program for reviewing ethics and code of conduct policies. The program steps here cover more than just the IT function and can be applied to the total enterprise. Yes-and-no responses, when asked in an information-gathering context, are often appropriate. A checklist-format audit program has two weaknesses, however. First, while a series of yes-or-no interview responses can lead an experienced IT auditor to look at problem areas or to ask other more probing questions, these same points may be missed when a less experienced auditor just completes the questionnaire and does not go beyond the yeses and nos, nor dig a bit deeper as to where those responses might lead. A procedures-oriented audit program better encourages follow-up inquiries in other areas where information gathered may raise questions. Checklist-format IT audit programs were also very common in the early days of IT auditing general controls reviews, when many operational internal auditors did not have extensive IT controls backgrounds and needed detailed checklists as support. They were often called ‘‘Control Objectives’’ with survey statements requiring a yes-or-no auditor response. ISACA’s predecessor organization, the EDP Auditors Organization, was very successful at publishing those checklist documents in the early days of IT auditing. Rapidly changing technology has reduced the importance of IT audit checklists today. The questionnaire-format IT audit program also tends to cause auditors to miss examining necessary evidential matter while asking only the checklist questions. An E1C05 09/18/2010 11:1:21 Page 131 Developing and Preparing Audit Programs Audit Controls Step 1 Does the enterprise have a written code of business conduct? 2 Does the code clearly apply to the IT organization? 3 Is the code distributed to all stakeholders? 4 Are new employees given an orientation to this code of conduct? 5 Are all employees required to acknowledge that have read, understand and agree to abide by this code? 6 Are training programs periodically delivered to all employees regarding compliance with the code? 7 Does the code address standards that govern employee conduct in their dealings with suppliers and customers? 8 Is there an effective mechanism in place to follow up on reports of suspected violations of the code? 9 Is there an effective mechanism in place to allow employees to confidentially report suspected violations of the code? YES & NO 131 N/A 10 Is there a mechanism in place to allow employees to be informed of the results of their reported ethics concerns? 11 Is compliance with the code’s provisions a standard used for measuring employee performance at all levels? 12 Is there a procedure in place to update the code on a periodic basis? EXHIBIT 5.9 ___________________ Checklist Format Audit Program for Business Ethics Procedures inexperienced IT auditor can too easily check ‘‘yes’’ on the questionnaire without determining, for example, whether that response is properly supported by audit evidence. An example would be a question regarding whether some critical document is regularly approved. It is easy to ask the question, receive an answer of ‘‘yes,’’ and never follow up to see if those documents actually were approved. Each of these audit program formats works for different types of reviews, provided the IT auditor gives some thought to the program questions. The key concern is that all audits should be supported by some type of audit program that documents the review steps performed. This approach allows audit management to recognize what procedures the auditors did or did not perform in a given review. Strong and consistent audit programs are an important step to improving the overall quality of the IT audits performed. The reliability of the materials and processes to be reviewed and internal audit’s other understandings about an operation should also be considered when developing audit program for a specific facility or resource. There is little value in developing an IT audit program at a facility that calls for a review of systems and procedures that are no longer in use. In developing an audit program, an IT auditor should try to select audit steps that are meaningful and that will produce reliable forms of audit evidence. For example, the audit program often calls for detailed tests in a given critical, high-risk area rather than suggesting that the information can be gathered through interviews. Advanced audit techniques should also be incorporated into audit programs wherever practicable. For example, CAATTs, discussed in Chapter 13, can perform selected audit steps, and the use of procedures such as statistical sampling allow an IT E1C05 09/18/2010 11:1:21 132 & Page 132 Performing Effective IT Audits internal auditor to extract data easily from larger populations. Members of the audit staff who have IT audit or other technical skills should be consulted when preparing these audit program steps. There is no single best or set format for an IT audit program; however, the program should be a document that IT auditors can use to guide their efforts as well as to record activities. GATHERING AUDIT EVIDENCE AND TESTING RESULTS Identifying higher-risk areas, developing audit programs to support consistent results, and planning IT audits are just preliminary steps to performing IT audits. An IT auditor must review a variety and sometimes a vast amount of materials to develop audit conclusions. Known as audit evidence, these materials can be database file records, paper documents, the auditor’s observation of staff member actions, or any information that will support an audit objective. An IT auditor should assemble this audit evidence to develop an audit conclusion. In many cases, there is a vast amount of audit evidence to help support developing an audit conclusion. For example, an IT auditor may be interested in how an automated quality assurance process is recording incoming materials receipts, as recorded on a large enterprise resource planning (ERP) systems database. For a larger enterprise, there could be several million database records of interest. A very important way to review these large volumes of data is to pull a representative sample of them and draw the audit conclusion from that sample data. In addition to gathering supporting audit information, audit sampling is an important IT auditor skill. Types of Audit Evidence As discussed in Chapter 3, both IIA and ISACA audit standards state that an IT auditor should examine and evaluate information on all matters related to the planned audit objectives. This information is called audit evidence and covers everything an IT auditor reviews or observes. An auditor should gather evidence in support of his or her evaluation—what is known as sufficient, competent, relevant, or useful audit evidence. A well-constructed audit program should guide an IT auditor in this evidence-gathering process. Multiple types of evidence—stronger or weaker—can be useful in developing audit conclusions. For example, actually observing some activity or obtaining an independent confirmation of that activity is one of the strongest forms of evidence. However, a casual response to an IT auditor’s question covering that same activity will be weaker. It is not that an auditor thinks the IT staff member responding to the question is not telling the truth, but an IT auditor’s actually observing some event is far superior to just hearing about it. Internal auditors will encounter different levels of audit evidence and should attempt to design their audit procedures to look for and rely on the best available audit evidence. Exhibit 5.10 provides some ranges of best evidence for different classifications of materials. The idea that a written, signed document is better evidence than a casual response should be no surprise to an IT auditor, but it is always good to keep these concepts in mind. E1C05 09/18/2010 11:1:22 Page 133 Gathering Audit Evidence and Testing Results & 133 Evidence Classification Strongest Weakest Audit Technique Observation/Confirmation Casual Inquiry IT File or Database Materials Auditor-Controlled CAATT Results Printed Extract from File Data Origin of the Evidence Corroborative Materials Underlying Statistics Relationship of the Auditee External Document Auditee Internal Document IT Data Repository Enterprise-Controlled Data Center Outside Service Provider Form of Audit Evidence Written with Signatures Oral Comments Sophistication of Evidence Formal Documentation Informal such as Notes Location of Evidence Connected to Area Reviewed Derived/Supporting Materials Source of Audit Evidence Product of Internal Audit Work Other Supporting Materials EXHIBIT 5.10 IT Auditor ‘‘Best Evidence’’ Classification The audit field survey and the subsequent development of an audit program are just preliminary activities to performing the actual IT internal audit. It is often more efficient to have supervisory personnel complete these preliminary steps before assigning staff IT auditors for the actual review. These supervisory auditors, either audit management or experienced in-charge auditors, usually have the experience to make quick assessments of field situations and to fine-tune the overall audit approach. However, once the survey and the completed audit programs have been reviewed and approved by IT audit management, the next challenge is performing the actual IT audit to meet its desired audit objectives. The preparatory work from the survey will play an important role in ensuring the audit’s success; however, the IT auditor now faces the day-to-day challenges of performing the actual IT audit. The actual audit steps performed will depend on the characteristics of the entity audited. A financially oriented audit of a credit and collection function will be quite different from an operational review of a design engineering department. The financial audit might include independent confirmations of account balances; the operational audit typically includes extensive interviews with management and supporting documentation to assess key internal controls. Despite these differences, all internal audits should be performed and supervised following a general set of principles or standards. This will ensure that internal audits are properly directed and controlled. Audit Sampling Approaches The IT audit process begins with first establishing audit objectives, then planning and preparing for the audit, and finally evaluating the audited results to determine if the audit objectives have been satisfied, if supporting internal controls are adequate, if the materials reviewed are sufficient to develop an audit conclusion, and if there is a need for corrective action–based audit recommendations. This process of testing, assessing, and then evaluating audit evidence is often a challenge for many IT auditors. For example, an auditor can review a sample of 100 items and find no problems with 99 of them. Should that one internal control problem exception cause the IT auditor to highlight this matter as an exception, or should the auditor give that single exception a pass and go E1C05 09/18/2010 11:1:22 134 & Page 134 Performing Effective IT Audits forward? There often are no easy answers, but an IT auditor should evaluate this audit evidence and try to make the appropriate decision. Audit sampling is a key approach for IT auditors, when faced with a large volume of audit evidence, to select a sample from that data and to make an overall audit assessment based on the results of the sampled data. That is, rather than looking at every item in an area of audit concern to develop evidence to support an audit, an IT auditor can examine a limited set of sample items to develop audit conclusions over the entire set or population of data. An IT auditor has a challenge here to extract a sample of items that will be representative of the entire population. If there are 100,000 file record transactions on a data file and if an IT auditor looks at only 50 of them, finding 10 exceptions (20% of the sample), can the auditor conclude that 20% of the entire population of transactions, or 20,000, are exceptions? This audit conclusion is true only if the sample of 50 drawn is representative of the entire population. Audit sampling techniques can help an IT auditor determine an appropriate sample size and develop an opinion for this type of audit task. Audit sampling has two major branches: statistical and nonstatistical. Statistical sampling is a mathematical-based method of selecting representative items that reflect the characteristics of the entire population. Using the results of audit tests on the statistically sampled items, an audit opinion can be developed for the entire group. For example, an IT auditor could develop a statistical sample of items in an inventory system, test those sample items for their physical quantity or value, and then express an opinion on the value or accuracy of the entire inventory. Nonstatistical sampling, also called judgmental sampling, is not supported by mathematical theory and does not allow an internal auditor to express statistically precise opinions on the entire population. Nevertheless, nonstatistical or judgmental sampling is often a useful audit tool. When planning any audit that includes the examination of a large number of transactions or other evidence, an IT auditor should always ask ‘‘Should I use audit sampling?’’ The correct answer here is often not just a simple yes or no but may be complicated by such factors as the number or nature of items to be sampled, a lack of technical expertise or IT software availability to do the sampling, a sometimes fear of the mathematical focus of sampling, and the potential nonacceptance of the sampling results by management. Sampling is also a word that is frequently misused by many internal auditors. All too often, an operational internal auditor, faced with a file cabinet filled with hundreds of documents to review, will pull out one or two items from the front and perform audit procedures based on this limited selection. Although this examination of two items may be appropriate for an audit observation, an internal auditor should not try to draw conclusions for the entire population based on that limited a potentially nonrepresentative sample. To develop this type of conclusion effectively, IT auditors need to: & & & & Understand the total population of items of concern and develop a formal sampling plan regarding those items. Draw a sample from the population based on that sample selection plan. Evaluate the sampled items against audit objectives. Develop conclusions for the entire population based on audit sample results. E1C05 09/18/2010 11:1:22 Page 135 Gathering Audit Evidence and Testing Results & 135 These steps represent the process of audit sampling, an IT audit procedure to examine less than 100% of the items within a class of transactions for the purpose of drawing a conclusion for the entire population based on the sample audit results. Why use audit sampling? We often hear reports on the results of statistical sampling techniques in consumer research, government studies, or quality-control testing on a production assembly line, and audit sampling can be a very effective tool for IT auditors as well. Although 100% examinations work for limited amounts of audit evidence, an IT auditor using CAATTs almost always looks at a sample—either very large or small—of the audit evidence. The IT auditor then draws an audit conclusion based on the results of that sample. With formal audit sampling, an auditor can draw a conclusion along the lines of ‘‘Based on the results of our audit sample, we are 98% certain the true inventory balance is between X and Y.’’ This type of statement and process is discussed in greater detail in the next paragraphs. Audit sampling is more frequently used by operational or financial internal auditors rather than by IT auditors because of the challenges presented by reviews of large populations of data, such as in physical inventories or financial transactions. However, these techniques are also useful for IT auditors who often examine large amounts of IT file or database data. Whenever an IT auditor needs to draw conclusions based on a population of multiple items but does not want to examine the entire population, audit sampling can often introduce better and more efficient audits. Reasons encourage the use of audit sampling include: & & & Conclusions may be drawn regarding an entire population of data. When statistical sampling is used, the sample results can be accurately projected over the entire population without performing a 100% review of that population, no matter how large. For example, an IT auditor interested in the occurrence of some error condition in a large volume of log file error records could select a statistical sample of these documents, test the sample for the error condition, and then make a perhaps 98% certain type of estimate about the occurrence of that error condition in the entire population of the log file records. This will provide a strong audit position with significant audit savings. Sample results are objective and defensible. Internal control errors often occur on a random basis over the total items subject to error, and each error condition should have an equal opportunity of selection in an audit sample. An audit test based on random selection is objective and even defensible in a court of law. Conversely, a sample based on auditor judgment could be distorted due to intentional or unintentional bias in the selection process. An IT auditor looking for potential problems might examine only the larger or sensitive items, ignoring others. Less sampling may be required through the use of audit sampling. Using statistics, IT auditors need not increase the size of a sample directly in proportion to increases in the size of the population to be sampled. Although a sample of 60 items may be sufficient to express an audit opinion over a population of 500 items, that same sample of 60 still can be sufficient for a population of 5,000. An auditor who E1C05 09/18/2010 11:1:22 136 & & & & Page 136 Performing Effective IT Audits does not use statistical approaches will often oversample large populations because of the incorrect belief that larger populations require proportionately larger samples. By using statistics-based sampling procedures, less testing may be required. Statistical sampling may provide for greater accuracy than a 100% test. When voluminous amounts of data items are counted in their entirety, the risk of significant clerical or audit errors increases. However, a small sample typically receives very close scrutiny and analysis. The more limited sample would be subject primarily only to sampling errors resulting from the statistical projection. Audit coverage of multiple locations is often more convenient. Audits can be performed at multiple locations with small samples taken at individual sites to complete an overall sampling plan. In addition, an audit using comprehensive statistical sampling may be started by one IT auditor and subsequently continued by another. Each of their sample results can be combined to yield one set of audit results. Sampling procedures can be simple to apply. In years past, an internal auditor often was required to use either tables published in sampling manuals or complex computer systems to develop a sampling plan and sample selection. With the availability of software packages for laptop computers, audit sampling today has become much more simplified. Despite the advantages of audit sampling, an IT auditor must keep in mind that exact information cannot be obtained about a population of items based on just a sample, whether it is judgmental or statistical. Only by making a 100% test and following good audit procedures can an auditor obtain exact information. With statistical sampling, regardless of the number of items examined, positive information can be obtained about all of the items in the population within a level of statistical confidence. There are three common audit sampling approaches depending on the audit’s objectives: attributes, variables, and discovery sampling. Attributes sampling is an approach used to measure the extent or level of occurrence of various conditions or attributes—in other words, to assess internal controls. For example, an IT auditor might want to test for the attribute of whether invoice documents have received proper approval signatures. An invoice will either be correctly approved or not—a ‘‘yes’’ or ‘‘no’’ qualitative condition. Normally, the attribute measured is the frequency of an error or other type of deficiency. The extent of the existence of the particular deficiency, such as improperly approved documents, determines the seriousness of the situation and how internal audit will report its findings and recommendations. Attributes or characteristics can be applied to any physical item, financial record, internal procedure, or operational activity. Attributes sampling often measures compliance with a designated policy, procedure, or established standard and is an effective test for internal controls. A control is determined to be working or not working. Sort-of working is not an appropriate conclusion. An IT auditor tests conditions in the selected items and then assesses whether the overall population is in compliance with the control attribute. Variables sampling deals with the size of a specified population, such as account balances or tests of individual sample items. Here the auditor’s focus is on how much as E1C05 09/18/2010 11:1:22 Page 137 Gathering Audit Evidence and Testing Results & 137 opposed to the yes-or-no focus of attributes sampling. The objective of variables sampling is to project total estimated quantities for some account or adjustments to the account on the basis of the auditor’s statistical sample. Illustrative would be a sample to estimate the total value of an inventory based on sample results. Variables sampling is concerned with absolute amounts as opposed to the number or extent of a particular type of error. This approach is more common for financial internal auditors. The third type of statistical sampling, discovery sampling, is used when an auditor wants to pull a sample from a large volume of data without the statistical processes associated with variables and attributes sampling. The next sections discuss attributes sampling methods in some detail but because of space limitations do not provide an IT auditor with enough information to become an expert in these statistical sampling techniques. All IT auditors should have a general understanding of audit sampling techniques, with an emphasis on attributes sampling. IT auditors will find this useful in their own work as well as when they work with operation and financial internal auditors. Attributes Sampling Procedures Attributes sampling is the process of pulling a sample to estimate the proportion of some characteristic or an attribute of interest in a population. For example, an IT auditor may be interested in the rate of occurrence of some monetary error or compliance exception that might exist in a population of accounts payable disbursement vouchers. The auditor here would test for the number of items that have some type of error condition, not the total monetary value of all of the errors. This type of test is very appropriate for assessing the level of internal control in some specific account, and can be a very important approach for SOx Section 404 internal controls tests. The starting point in attributes sampling is to estimate an expected rate of errors (i.e., how many errors can the IT auditor and management tolerate?). Depending on the items sampled and the culture of the enterprise, this expected error rate could be as little as 0.01% or as large as 5% or even more. Even if senior management states that no errors will be allowed in some highly critical operation, all parties often recognize that there may be a small or very small possibility of an error, and depending on the criticality of the operation, such a very small error rate will be accepted. An expected error rate is the recognition that certain types of operations contain errors no matter how good the other controls and procedures are. If IT audit were to perform a 100% examination of an account but find only a small number of errors—say, 0.5%—it might be difficult to convince management that its controls are weak. Management might expect and tolerate a 1% error rate here and not express much concern at audit’s findings. In an attributes sampling test, the IT auditor must estimate the expected rate of errors in the population sampled, based on management’s stated expectations, other audit tests, or just internal audit assumptions. Along with estimating the expected error rate, an IT auditor must decide on the acceptable precision limits and the degree of desired confidence for the sample. In other words, an IT auditor would like to be able to say ‘‘I am 99% confident that the error rate of this account is less than 1%.’’ These estimates will allow an auditor to determine the size of a sample that will provide a reliable conclusion regarding the condition tested. E1C05 09/18/2010 11:1:22 138 & Page 138 Performing Effective IT Audits This determination is made through statistical methods and can be obtained from common statistical software packages, Web resources, or even from manual tables found in the old statistical sampling books. These factors provide an initial basis for the size of the sample to be reviewed. The auditor now selects this sample and examines the items sampled to determine the number of errors that exist in them. As can usually be expected, the error rate in a sample is normally higher or lower than the previously estimated acceptable error rate. If lower, the auditor has established that the condition tested is safely within the limits selected. If the sample shows a higher error rate, the auditor must determine whether the results are satisfactory and what further action, if any, is needed. Conceivably, the sample can be expanded, but IT audit will often feel that there is an adequate basis for arriving at a conclusion. The key to meaningful attributes sampling is to take an appropriate sample and properly develop an audit conclusion based on the sample results. Attributes sampling, once commonly used by both internal and external auditors, is now used less frequently by auditors because of its sometimes difficult computational requirements and the auditor statistical knowledge required. However, there are good IT audit sampling tools available, and attributes sampling is an effective tool to assess the status of some control procedures. Performing an Attributes Sampling Test Attributes sampling is useful when an auditor is faced with a rather large number of items to be examined and wants to test whether certain controls are working. An IT auditor must first define the specific nature of the compliance tests to be performed, the nature of the sampling units, and the population characteristics. Attributes sampling is a yes-or-no type of audit test where the item or attribute sampled must either be correct or incorrect; there can be no measures of almost correct or close enough. In a test of the completeness of travel report approvals, for example, enterprise procedures may state that the responsible manager must approve all travel reports greater than $100. Thus any voucher not approved by the responsible manager would be considered a compliance error. Internal audit should carefully define the types of tests to be performed as well as the acceptance and rejection rules. Although it is possible to separately sample for two or more different attributes, each statistical test should concentrate on compliance with one such test criteria. If multiple criteria are used in a single test, the failure of any one would mean that the entire item sampled is out of compliance. The size of the population as well as the auditor’s tolerance for errors will impact the number of items to be sampled. If an auditor is testing for travel policy compliance and if there is a requirement for manager approval for vouchers over a $25.00 limit, should internal audit treat a nonapproved $25.01 item as an error, or should it allow for a perhaps 5% or 10% exception rate? As much as possible, these items should be defined in advance. In addition, IT audit should have a clear understanding of the number and location of the items to be sampled. If initial plans are to sample all travel accounting reports, those reports must be available or readily accessible. If some items are filed at a remote, international location, internal audit may not be able to sample all such reports unless it E1C05 09/18/2010 11:1:22 Page 139 Gathering Audit Evidence and Testing Results & 139 gains access to the remote, international reports as well as the national items filed centrally. Otherwise, internal audit should reduce the scope of the population sampled and look only at domestic travel accounting reports. An auditor must first make some preliminary estimates, based on observations and other audits, of what is expected from the sample results and then pull an actual audit sample based on those expectations. For example, if an auditor expects a fairly high level of errors in the population, the auditor’s sample should be sufficient to confirm or refute those initial expectations. Internal auditors need to estimate such things as the maximum tolerable error rate, and then the initial sample size. Other key attributes for sampling are: & & & & & & Maximum tolerable error rate Desired confidence level Estimated population error rate Initial sample size Selecting the sample to perform audit procedures Evaluating the results of the attributes sampling test Maximum Tolerable Error Rate Statisticians also call the maximum tolerable error rate estimate the desired upper precision limit. This is the error rate an internal auditor will allow while still accepting the overall internal controls. The idea is that a typical population may have some errors. In the previously discussed audits of travel expense reports that were reviewed for departmental management approvals, a realistic internal auditor recognizes that there may be some errors, such as the $25.01 vouchers that are above the $25.00 requirement. This is an error an internal auditor might accept but still feel that internal controls are generally adequate. The maximum tolerable error rate is normally expressed as a percentage that can vary based on the nature of the items reviewed. In the previous example, an auditor might accept a 5% tolerable error rate or upper precision limit. In other instances, a smaller or larger estimate can be used, but this estimate should never be more than 10%. Such an estimate indicates major internal control problems, and the resultant attribute sample may provide little further information. If an internal auditor knows that internal controls are very bad, it is of little value to take an attribute sample to verify what he or she has already determined through other audit procedures. Similarly, an internal auditor should normally expect some errors and establish some reasonable value for this rate, perhaps 1% or 2%. Desired Confidence Level The desired confidence level is a measure of an auditor’s confidence in the results of a sample. That is, internal auditors generally would like 95% or 98% certainty that the results of their sample are representative of the actual population. An internal auditor will never be 100% certain that a condition exists unless he or she reviews essentially 100% of the items in the population. If a population of 100 items contains one error, an E1C05 09/18/2010 11:1:22 140 & Page 140 Performing Effective IT Audits auditor might look at a sample of 10 items and find no errors. The auditor may look at 20, 30, 50, or even 90 items and still not find more than that one error. The only way to be 100% certain that the population contains a 1% error rate is to look at 100% of the items. However, an internal auditor typically should look at a much smaller sample and still be able to state that he or she is 95% or 98% certain that the error rate is no more than 1%. The assumed confidence level value, usually 95% or 98%, along with the estimated population size, will determine the size of the sample needed to test the estimated population. Too large of a confidence level may require too large of a sample. Too low of a confidence level may reduce the size of the sample, but the results may be questionable. Management typically would not accept an internal audit finding that states the auditor is ‘‘75% confident’’ that some condition is true. Estimated Population Error Rate In attributes sampling, an internal auditor estimates the level of errors in a population and then takes a statistical sample to confirm or refute those assumptions. In order to calculate the sample size, an auditor also needs to estimate the expected rate of occurrence of errors in that same population. This estimate, together with the confidence level and the maximum tolerable error rate, determines the size of the sample. For example, if the confidence level is 95% and the maximum tolerable error rate is 5%, the auditor should select a sample of 1,000 items in a very large population if the estimated population error rate is 4%. A smaller estimated population error rate will reduce the sample size. Given the same parameters, an estimated population error rate of 1% will drive the sample size down from 1,000 to 100 items. If the expected population error rate is very large—greater than 50%—the required sample size will become very large. Generally, the larger the difference between the maximum tolerable error rate and the estimated population error rate, the smaller the necessary sample size. Initial Sample Size The preceding three factors determine the necessary sample size. Although calculation formulas can be found in statistics textbooks, IT auditors normally use audit software to develop attributes sampling plans. A Web search for attributes sampling software will provide a wide range of options where an IT auditor only needs to provide (1) the maximum tolerable error rate, (2) the confidence level, (3) the estimated population error rate, and (4) the approximate sample size. The software then provides the required sample size for the attributes test. For example, if the confidence level is 99%, the maximum tolerable error rate is not over 5%, and the estimated error rate is 4%, an auditor should examine 197 items for an attributes test over a population of about 1,000 items. Attributes sample sizes tend to be large—a problem for many auditors—and it often is difficult to justify the larger sample sizes needed to perform a statistically correct attributes test. In some instances an internal auditor can modify the sample size by modifying sampling assumptions. For example, the previously mentioned sample of 197 for a 1,000 items population will become much smaller if the confidence level is lowered from 99% to 95%. E1C05 09/18/2010 11:1:22 Page 141 Gathering Audit Evidence and Testing Results & 141 Selecting the Sample to Perform Audit Procedures Having made some audit sample assumptions and determined the sample size, the next step is to pull the actual items for review. Random sampling procedures can be used to select items to review, and multiple attributes also can be tested using the same set of sample items. The concept to remember is that the internal auditor will be performing a separate yes-or-no type test for each of the individual attributes on each of the items in the sample. Workpaper documentation should describe all items selected as part of the attributes test. Spreadsheet software is useful here for recording the results of the audit tests, but the internal audit procedures should be performed with great care. If an audit fails to recognize an error condition in the selected sample items, that fact will throw off the conclusions reached as part of the overall sample. With a large population, each sample item may speak for hundreds or even thousands of actual items. Each sample item should be evaluated carefully and consistently against the established attributes. An assessment of ‘‘close enough’’ should not be used. If some attribute measurement is too stringent for certain items, internal audit should consider reevaluating the entire sample set. An IT auditor may be looking for several error conditions but then find another error not included in the original test design. If significant, internal audit may then want to redefine the overall attribute test. Evaluating the Results of the Attributes Sampling Test As discussed, prior to actually selecting and evaluating the sample items, an IT auditor will have made initial assumptions regarding the maximum tolerable error rate, the reliability, and the level of confidence as well as about how many compliance errors would be tolerated to assess whether the controls are adequate. The next key step is to evaluate the sample results against those assumptions to determine if an internal control problem exists. Recall that an upper precision limit or maximum tolerable error rate and a confidence level formed the standards used to determine the sample size and perform the sampling test. An internal auditor should now assess the actual error rate of the sampled items and calculate an upper precision limit based on those sample errors. That precision limit, computed on the basis of the actual sample, should be less than or equal to the desired precision limits established at the beginning of the sample exercise in order for the auditor to report favorable results from the sample. There can be a major audit finding if the results of an audit sample do not meet the preliminary criteria. Although these audit criteria should have been well thought out and approved before beginning the test, sometimes internal audit or management may decide that the original assumptions were too conservative. A new upper precision limit or confidence level could be used and the sample results measured against it. This approach should be used only with the greatest caution. In effect, the auditor here is attempting to justify some bad results. Were the matter ever to reach a court of law, internal audit would have a tough time justifying why it had altered its assumptions to make the sample results look good. A better approach when the results are unfavorable is to expand the sample size. When attributes sampling results turn out unfavorably, management sometimes claims that internal audit only looked at some very unusual items and that the remainder E1C05 09/18/2010 11:1:22 142 & Page 142 Performing Effective IT Audits of the population is not that bad. An increase in the sample size will have the effect of decreasing the computed upper precision limit, assuming that the auditor does not find a substantial number of additional errors. However, internal audit should weigh the relative costs and benefits of this approach. A better approach is to report the internal control problem based on the current results and to expand the sample size in a subsequent audit review. Management should, it is hoped, take steps during the interim to improve internal controls in the area of interest. Attributes sampling is a very useful technique for assessing one or several internal controls in an area of audit interest. Because estimates of such things as the maximum tolerable error rate are made in advance, it is difficult to dispute the audit test assumptions when compared to sample results. Similarly, because random number or similar techniques typically are used to select the sample items, it would be difficult to claim auditor bias in the selections. This discussion presents attributes sampling only on a very high level. An IT auditor needs more study and experience to use it successfully, but attributes sampling is a technique that many IT auditors will find useful. Attributes Sampling Advantages and Limitations Whenever there is an audit need to review a large number of items, attributes sampling procedures can provide a statistically accurate assessment of selected control features. Although statistical theory requires a relatively large sample size, an IT auditor can review some control or condition within a sample of that data and then can state, within a preestablished confidence value or percentage, that the number of errors in a total population will not exceed a designated value or that the control is working. Attributes sampling is not useful for determining the estimated correct value on an account such as an inventory book value but is an extremely useful tool for reviewing control procedures in a variety of operational areas. Attributes sampling equips an IT auditor with a very powerful tool to assess internal controls in a large population of data through the evaluation of a limited sample. Although the technique often is too time consuming or complex for many audit tests over paper-based records, an IT auditor should develop a basic understanding of attributes sampling to use when appropriate. The technique is particularly appropriate when an auditor is sampling from IT files or records and using automated tools. It also provides support if the initial, judgmental results of an internal controls review indicate problems in an area and when management disputes the preliminary results from audit’s limited, judgmental sample as being ‘‘unrepresentative.’’ A follow-up attributes sample will allow IT audit to take another look at the data and come back with a stronger statement about the status of internal controls surrounding the area in dispute. WORKPAPERS AND REPORTING IT AUDIT RESULTS This chapter has presented some fundamental techniques that are essential for all IT and other internal auditors, including audit planning preliminary surveys, audit programs, and attributes statistical sampling. The next sections outline two other important skills for performing effective IT audits: the development of audit workpapers and preparing E1C05 09/18/2010 11:1:22 Page 143 Workpapers and Reporting IT Audit Results & 143 effective audit reports. There is much more content in each of these areas for an IT auditor, but the next sections discuss the preparation of audit workpapers, to document ongoing audit endeavors and the preparation of audit reports. Each of these is not just an IT audit technique but essential components of the overall internal audit process. Preparing IT Audit Workpapers Workpapers are the written records kept by an IT auditor to document review materials, notes, and other sample material—the evidential matter—gathered or accumulated during an audit. The term workpaper is a rather archaic auditor expression that describes a physical or computer file that includes the various schedules, analyses, memoranda prepared, and, in many cases, copies of documents secured as part of an audit. The common characteristic of all workpapers, however, is that they describe the results of the internal audit work performed and should be formally retained for subsequent reference and substantiation of reported audit conclusions and recommendations. Workpapers are the bridge between actual internal audit procedures and the audit reports issued. Not an end in themselves but a means to an end, workpapers are created to fit particular audit tasks and are subject to a great deal of flexibility. They must support and document the purposes and activities of an IT auditor, regardless of their specific form. Thus, workpaper principles and concepts are more important than any specific formats. Internal audit workpapers also can have considerable legal significance. In certain investigations, they have been handed over, through court orders, to government, legal, or regulatory authorities. When scrutinized by outsiders in this context, inappropriate workpaper notes or schedules can easily be taken in the wrong context. They form the documented record of both who performed the audit and who reviewed that work. IT audit workpapers are the only record of that audit work performed, and they may provide future evidence of what did or did not happen in the area of audit interest at some point in time. This section provides general guidance for preparing, organizing, reviewing, and retaining workpapers. Once organized in bulky legal-size paper folders, audit workpapers today are usually stored as computer-based folders or a combination of paper and computer format documents. As a side note, we use the term workpaper, although many have used working paper or work paper. All mean the same thing. The objective of audit workpapers is to document that an adequate audit was conducted following professional standards. The IT auditor can perhaps better understand the overall role of workpapers in the audit process by considering the major functions these documents serve: & & Basis for planning an audit. Workpapers from a prior audit provide an IT auditor with background information for conducting a current review in the same overall area. They may contain descriptions of the entity, evaluations of internal control, time budgets, audit programs used, and other results of past audit work. Record of audit work performed. Workpapers describe the current audit work performed and reference it to an established audit program, discussed previously. Even if the audit is of a special nature, such as an IT network fraud investigation E1C05 09/18/2010 11:1:22 144 & & & & & & Page 144 Performing Effective IT Audits where there may not be a formal audit program, a record should be established of the actual audit. This workpaper record should include a description of activities reviewed, copies or Web links to representative files, the extent of the audit coverage, and the results obtained. Use during the audit. In many instances, the workpapers prepared play a direct role in carrying out the specific audit effort. For example, the workpapers can contain control logs for such areas as the responses received as part of an accounts receivable customer balance independent confirmation audit. Similarly, a flowchart might be prepared and then used to provide guidance for a further review of the actual activities in some process. Each of these would have been included in the workpapers in a previous audit step. Support for specific audit conclusions. The final product of most internal audits is a formal audit report containing findings and recommendations. The findings may be actual evidence, such as a copy of a purchase order lacking a required signature, or derived evidence, such as the output report from a computerassisted procedure against a data file or notes from an interview. The workpapers should provide sufficient evidential matter to support the specific audit findings that would be included in an audit report. Reference source. Workpapers can answer additional questions raised by management, the operational or financial internal audit group, or external auditors. Such questions may be in connection with a particular audit report finding or its recommendation, or they may relate to other inquiries. For example, management may ask IT audit if a reported systems weakness problem also exists at another location that is not part of the current audit. The workpapers from that review may provide the answer. Workpapers also provide basic background materials that may be applicable to future audits of the particular entity or activity. Staff appraisal. The performance of an IT audit staff member during a review— including that auditor’s ability to gather and organize data, evaluate it, and arrive at conclusions—is reflected in the workpapers. Audit coordination. Internal auditors on occasion exchange workpapers with external auditors, each relying on the other’s work. In addition, government auditors, in their regulatory reviews of internal controls, may request to examine the internal auditor’s workpapers. In some respects, audit workpapers are no different from the formal files of correspondence, e-mails, and notes that are part of any well-managed organization. A manager would keep files of incoming and outgoing correspondence, notes based on telephone conversations, and the like. However, these files are based on just good practices and may vary from one manager to another in an organization. The manager may never be called on to retrieve these personal files to support some organization decision or other action. Internal audit workpapers are different in that they may also be used to support or defend the conclusions reached from the audit. They may be reviewed by others for various reasons. Members of an internal audit organization may work on common projects and need to share workpapers to support their individual components of a larger E1C05 09/18/2010 11:1:22 Page 145 Workpapers and Reporting IT Audit Results AR-2-50 1 Global Computer Products IT Audit General IT Controls Review of MUDDVILLE Server Operations & 145 RRM yy/mm/dd The IT audit team, led by L.C. Tuttle, began an IT general controls review at the Muddville distribution facility in Muddville, MN. After informing the IT manager, W.J. Rawdon, of the planned review, the assigned audit team met and visited with Rawdon on March 15, 20xx. He informed us of recent supervisor changes in server maintenance operations but otherwise provided assurances that there had been no significant changes in controls since our most recent review at the facility, yy/mm/dd. IT Audit also informed Rawdon of our plans to review controls over the storage virtualization project in place at the facility. The manager in charge of the project met with IT audit and gave an overview of this project. We requested a current project plan for that effort and were advised it was currently being updated, with a completion date of the following week. We requested a copy of the most recent plan and were told it was not available. IT audit then requested a walk-through tour of the server physical facilities. Pending more detailed tests, the audit team observed generally poor, sloppy physical housekeeping in many of the areas visited. Rawdon observed our concerns and claimed the problem was due to the inexperience of the new supervisor. We informed Rawdon that our detailed review would begin at 9:00 the following morning. EXHIBIT 5.11 IT Audit General Controls Review Workpaper Example audit project or to take over an audit performed previously by another member of the audit staff. It is essential that an internal audit department have a set of standards to ensure consistent workpaper preparation. Workpaper Documentation Organization Most IT auditors prepare workpapers on their laptop computers, where they maintain many commentaries and schedules on secure files and folders. Regardless of page size or media, the purpose of a workpaper sheet is to provide a standard framework for documenting IT audit activities. Workpaper pages should be titled, dated, initialed by the preparer, and prepared in a neat and orderly manner. Exhibit 5.11 is an example workpaper page from an IT audit general controls review illustrating substance and format standards. The next paragraphs expand on that same basic workpaper format. A typical IT audit gathers a large amount of materials to document the audit process. With the wide range of IT activities reviewed and the equally wide range of audit procedures, the form and content of those individual workpapers may vary greatly. The major categories depend on the nature of the audit materials and the work performed. For IT internal audits, workpapers can be separated into these broad audit areas: & & & Permanent files Administrative files Audit procedures files Permanent Files Many audits are performed on a periodic basis and follow repetitive procedures. Rather than capture all of the data necessary every time each audit is performed, certain data can E1C05 09/18/2010 11:1:22 & 146 Page 146 Performing Effective IT Audits be gathered from what is called a permanent workpaper file, which contains data of a historical or continuing nature pertinent to current audits. Some of this data may include: & & & & & & & Overall organization charts of the audit unit Charts of financial accounts (if appropriate for audit) and copies of major policies and procedures Copies of the last audit report, the audit program used, and any follow-up comments Financial statements about the entity, if appropriate, and any other potentially useful analytical data Information about any unique aspects of the IT hardware and software environment Information about the audit unit (descriptions of major products, production processes, and other newsworthy matters) Logistical information to help the next auditors, including notes regarding travel arrangements A permanent file is not meant to be permanent in that it will never change; rather, it provides an IT auditor starting a new assignment a source of background material to help plan the new audit. Over the course of a new audit, an IT auditor may come across other materials to update or include in the permanent file. The permanent file is a source of continuity to tie audits together over time. Exhibit 5.12 shows an index from an audit permanent file binder. In the past, internal auditors sometimes loaded up their audit permanent files with materials that do not deserve permanent file status—for example, copies of various procedures that will have changed by the time of the next scheduled audit. Materials readily available at the time of the next audit need not be retained in permanent files unless certain ongoing procedures were based on those earlier materials. This administrative planning audit workpaper material should be kept to a minimum. Administrative Files Although separate workpaper administrative files may not be necessary for a smaller audit, the same general administrative workpaper materials should be incorporated Global Computer Products IT Audit Permanent File Index INDEX PERMANENT FILE DATE A-13 Corporate Data Center IT General Controls A-24 Brussels Distribution Operations IT General Controls 05/11/2012 A-25 Muddville Operations IT General Controls 03/15/2011 B-17 Corporate ERP CAATTs 06/30/2005 B-38 Financial Systems CAATTs 09/30/2011 C-92 Corporate IT Service Delivery Controls 03/15/2013 C-04 Corporate Service Delivery Controls 06/05/2012 EXHIBIT 5.12 IT Workpaper Permanent File Index Example 02/01/2012 E1C05 09/18/2010 11:1:22 Page 147 Workpapers and Reporting IT Audit Results & 147 somewhere in all audit workpaper sets. If only a single auditor or limited review, this material may be incorporated into the single workpaper. Audit Procedures Files Audit procedures files and folders record the actual audit work performed and vary with the type and nature of the audit assignment. That is, this is the documentation covering the actual audit. For example, the workpapers for an IT applications audit may contain interview notes and auditor observations and these workpaper documentation elements: & & & & & Listings of completed audit procedures. Workpapers are a central repository documenting the IT audit procedures and include copies of the audit programs along with the signed initials of the auditors and the dates of the audit steps. Commentary notes may be on the programs or attached as cross-referenced supplementary notes. Descriptions of operational procedures. Workpapers should briefly describe the nature and scope of the specific IT activity reviewed. This description can be in flowchart or narrative form. The auditor should always note on the workpaper the source of information to develop this description. For example, a member of auditee management may have described the process or the auditor may have gathered this information through observation. Review activities. Many IT audit workpapers cover specific investigations that appraise selected activities. These can include testing of data, observations of performance, inquiries to designated individuals, and the like. This is perhaps the most common type of workpaper prepared by the internal auditor. It follows no one form but serves only to describe the audit activities performed and the results. Organization documents. Basic documents here include organization charts, minutes of meetings, policy statements, contracts, and the like. Although some of these might be more appropriate for the permanent file, others are unique to a particular audit. Sometimes it may be valuable to retain a limited set of hard-copy documents, but file records should retain active Web links to audit-related reference materials. The purpose of these documents is to provide references to audit decisions or processes. Findings point sheets or drafts of reports. Point sheets are review documents, normally prepared by in-charge audit supervisors or responsible audit management, that reflect an audit’s supervisory review and any questions or review points that may have been raised as a result of that audit. The idea is that these point sheets allow the audit supervisor to raise questions about the audit along with appropriate IT auditor responses. The format of these documents can vary. Exhibit 5.13 is an example of a workpaper review point sheet. In addition, for smaller audits that do not have an administrative file, several draft versions of the written report should be included either as copies or references. These drafts can be annotated to show major changes, the persons responsible for authorizing those changes, and in some cases the reasons for the changes. The review functions of a software product, such as Microsoft Word, can be very useful here. E1C05 09/18/2010 11:1:22 148 Page 148 & Performing Effective IT Audits Global Computer Products Workpaper Review Notes—Muddville Storage Management General Controls yy/mm/dd Prepared by: JAS W/P Ref. Audit Procedure A-13 Manager approval message missing for SystemMatic consulting reference. A-14 Workpapers should note date and time of server center walk-through. B-02 Documentation for yymmdd continuity test inadequate— describe Day 2 wrap-up. C-03 Workpaper cross-reference missing. C-15 ERP Firewall tests missing. G-02 Audit program step 4.7 does not appear complete— review assessments. EXHIBIT 5.13 & Disposition corrected—DSA mm/dd corrected—DSA mm/dd Workpaper Point Sheets Example Audit bulk files. Internal audits often produce large amounts of evidential materials, which should be retained but not included in the primary workpapers. For example, an IT audit may perform a survey that results in a large number of returned questionnaires. These materials should be classified as workpapers but should be retrieved from the bulk file as necessary. Workpapers are the method of documentation of communications within an internal audit department, from one IT audit or auditor to the next. They are also a means of communication with the enterprise’s external auditors. An internal audit department should establish overall workpaper standards covering their style, format, and content. Some specific details do not need to be frozen, given the various types of audits performed and evolving audit automation procedures. However, workpaper contents should be prepared consistently for all audits. The audit procedures workpaper file, for example, should contain materials covering each of the listed areas. PREPARING EFFECTIVE IT AUDITS This chapter has summarized some the key elements that are necessary to perform any internal audit, with an emphasis on IT audits. Although IT audit is often a unique and very special function within an enterprise’s internal audit function, IT auditors should follow many of the same procedures and processes as the other operational and financial internal auditors within an internal audit organization. IT auditors have a unique role in an internal audit organization, but they must work with other members of the internal audit team to perform effective audits. This chapter has talked about many of the tasks necessary to perform effective IT audits, but there are many other details necessary. Performing effective IT audits is a comprehensive and important process. Exhibit 5.14 shows some of the steps necessary E1C05 09/18/2010 11:1:22 Page 149 Notes & 149 Processes for Performing an Effective IT Internal Audit Schedule Audit Assign Auditors Preliminary Assessment Document Processes Evaluate Controls Prepare Findings Supporting Internal Audit Subprocesses Develop Test Plan Collect Evidence Perform Audit Test Process Activities Document Results Evaluate Test Results Tasks Follow up on Results EXHIBIT 5.14 Corrective Action Plan Performing Effective IT Audits Process Steps to perform any IT audit. This exhibit provides just one path in what is normally a much larger process requiring many steps and going much beyond this chart. The overall process described here should be considered while reading the next chapters, which discuss the many and varied tasks and opportunities for IT auditors. This chapter has described key processes for performing effective IT audits. The overall message here is that IT audits are in many respects not that different from many other internal audits. However, they always should be integrated with other enterprise internal audit processes, be consistent with other enterprise internal audit standards, and emphasize the unique risks and concerns associated with enterprise IT controls. Even though an IT audit group may perform many very specialized IT security and control reviews, this work always should be coordinated with other enterprise internal audit activities. Whether IT audit or the operational and financial internal audit team members, all are reporting to the enterprise’s audit committee. Satisfying audit committee key goals is the key element to performing effective IT internal audits. NOTES 1. For an overall description of a successful internal audit function, see Robert Moeller, Brink’s Modern Internal Auditing: A Common Body of Knowledge, 7th ed. (Hoboken, NJ: John Wiley & Sons, 2009). 2. Descriptions of these supporting documents and discussion on their use can be found in ibid. E1C05 09/18/2010 11:1:22 Page 150 E1C06 09/16/2010 13:20:56 Page 151 II PART TWO Auditing IT General Controls E1C06 09/16/2010 13:20:56 Page 152 E1C06 09/16/2010 13:20:56 Page 153 6 CHAPTER SIX General Controls in Today’s IT Environments E N TE R P R I S E IN F O R M A T I O N T E C H N O L O G Y ( I T ) processes and systems cover many areas, ranging from an IT application to control an enterprise’s accounting general ledger, file server devices to manage enterprise data, and connections to the all-pervasive Internet. An IT auditor should have a strong understanding of IT internal control techniques covering many of these technologies and processes. Although the lines of separation are sometimes difficult, we generally think of IT controls on two broad levels: application controls that cover a specific process, such as an accounts payable system to pay invoices from purchases, and what are called IT general controls. IT general controls cover all aspects of IT operations and are necessary in order for specific application controls to be effective. Although many types of application-specific controls are discussed here and in later chapters, an example application control is a self-balancing routine in an accountingrelated system that flags an error or stops processing if the accounts are out of balance for that specific application. An important control for that specific accounting application, it assumes that no one can inappropriately alter this application, by changing a parameter file or otherwise bypassing things, to override these self-balancing controls. A process to prevent unauthorized changes to all such applications is called a general control. IT general controls cover broad areas of the overall IT environment, such as computer operations, access to programs and data, program development and program changes, network controls, and many others. In order for specific application controls to be effective, there must also be strong overriding effective IT general controls in place. The concept of IT general controls goes back to the early days of centralized, mainframe computers. Before there were many specialized IT auditors, internal auditors sometimes looked for such things as an access control lock on a computer center door 153 E1C06 09/16/2010 13:20:56 154 & Page 154 General Controls in Today’s IT Environments as a general control that covered all processes and applications operating within that centralized IT operations center. Today, we often think of the set of processes that cover all IT enterprise operations as the IT infrastructure. Because of the many possible variations in techniques here, no one technique is really right or wrong here. An enterprise should establish and implement a set of best practices that will serve as guidance for establishing IT general controls best practices. This chapter looks at IT general controls from an IT auditor’s perspective. Using the control classification hierarchy from Exhibit 6.1, this chapter reviews some key governance, management, and technical IT general controls. Many IT general controls also cover specific specialized areas, discussed in other chapters (such as Chapter 20 on cybersecurity and privacy controls), and this chapter discusses many IT general control procedures as they apply to enterprises of all sizes. Our overall focus, however, is on effective IT audits of these general controls. IMPORTANCE OF IT GENERAL CONTROLS IT auditors became involved with early IT procedures—then called data processing controls—when accounting applications were first installed on punched-card accounting systems on very early computer systems. Those systems of long ago were often installed in glass-walled rooms within corporate lobbies to impress visitors with the enterprise’s ‘‘sophistication.’’ However, those same early computer systems were not particularly sophisticated and internal auditors, who were often unfamiliar with data processing technology, would audit around the computer. That is, they might look at input controls procedures and the application’s outputs to check whether the inputs balanced to the output reports. At this time, there was little question about accuracy and controls when a report was produced by an IT system. An auditor would just go around the actual computer program processing procedures, assuming them to be accurate. Things changed in the early 1970s when a California-based insurance company, Equity Funding, seemed to be almost growing too fast. Its external auditors decided to run their own audit software programs against Equity Funding’s files, an almost unheard-of audit procedure at the time. The result was the discovery of a massive fraud with invalid data recorded on system files. Under management direction, fictitious insurance policy data had been entered on these computer files. Equity Funding’s external auditors had previously audited around that computer system, relying on printed output reports, with no supporting procedures to verify the correctness of supporting computer programs and files. A massive fraud was discovered. In the aftermath of the Equity Funding affair, organizations such as the American Institute of Certified Public Accountants (AICPA) and the Institute of Internal Auditors (IIA) began to emphasize the importance of reviewing data processing operations and application controls. A new professional specialty, computer or IT auditing, was launched. In those early days of business data processing, most IT systems were considered to be ‘‘large,’’ and standard sets of auditor control objectives and procedures were developed for reviewing controls. Many are still applicable today, but IT auditors must look at these control objectives from a somewhat different perspective in today’s E1C06 09/16/2010 13:20:56 Page 155 Importance of IT General Controls & 155 IT environments. The profession began to think of IT controls in terms of the controls within a specific application and what are called general controls, pervasive controls surrounding all information systems operations. IT general controls cover all information systems operations and include: & & & & & Reliability of information systems processing. Good controls need to be in place over all IT systems operations. Discussed throughout this chapter, these controls often depend on the nature and management of the specific size and type of systems used. Integrity of data. Processes should be in place to ensure a level of integrity over all data used in various application programs. This is a combination of the general operations controls discussed in this chapter as well as specific application controls discussed in Chapter 10. Integrity of programs. New or revised programs should be developed in a wellcontrolled manner to provide accurate processing results. These control issues are part of the overall process of application program development and discussed in Chapter 7 on information technology infrastructure library (ITIL) best practices. Controls over the proper development and implementation of systems. Controls should be in place to ensure the orderly development of new and revised information systems. These control issues are discussed in Chapter 11. Continuity of processing. Processes should be in place to back up key systems and to recover operations in the event of an unexpected outage—what was once called disaster recovery planning and today is often known as business continuity planning. These control issues are discussed in Chapters 23 and 25. This chapter discusses general controls over in-house IT operations ranging from client-server systems to desktop operations as well as older, larger mainframe computer systems. Although there are differences in size and management of these different systems, all should be subject to the same general control needs. IT auditors should follow the general audit steps outlined in Chapter 5 for their reviews of IT general controls as well as for many types of IT audits: & & & & & Develop the objectives of the area or function to be reviewed or audited. Assess the area being audited to determine if the audit objectives are being met. If the controls assessed do not appear to meet audit objectives, perform a formal test of these controls to better understand compliance. Evaluate the results of audit tests and the resulting risks of internal control failures. Develop recommendations for corrective actions based on these test results, and report them as part of the internal IT audit reporting process. This basic audit process is described in more detail in Chapter 5 and is used in IT audit exercises in other chapters. IT auditors always should develop control objectives for the area reviewed, review the supporting materials, test the control objectives, and finally develop control improvement recommendations based on the nature of the materials reviewed. 13:20:56 156 & Page 156 General Controls in Today’s IT Environments Governance Policies IT Standards Management and Organization Management 09/16/2010 Physical and Environmental Controls Systems Software Controls Systems Development Controls Technical E1C06 Application-Based Controls EXHIBIT 6.1 IT General and Application Controls Hierarchy As another and perhaps more appropriate way of looking at IT general and application controls, Exhibit 6.1 describes an IT controls hierarchy. At the top of this triangle configuration are IT policies defining the overall enterprise IT organization. Some of these general control requirements are discussed in the sections that follow; others involving the enterprise audit committee are found in Chapter 15. Moving down the Exhibit 6.1 hierarchy are general controls for IT standards, organization of the IT function, and physical and environmental controls. The next level down in the hierarchy brings two categories of technical-level general controls: systems software and systems development general controls, followed by application-based controls at the base of the controls hierarchy. Understanding and reviewing IT general controls as well as making appropriate recommendations for improvements or corrective actions are key IT audit activities. The next sections outline many of the key IT general controls areas of interest to IT auditors. E1C06 09/16/2010 13:20:57 Page 157 IT Governance General Controls & 157 IT general controls are also discussed in other chapters. For example, Chapter 17 discusses the general controls area of compliance with IT laws and regulations, and Chapter 26 reviews auditing telecommunications networks. Strong IT general controls are important; effective applications controls are of little value sunless they are supported by strong IT general controls. IT GOVERNANCE GENERAL CONTROLS The concept of IT governance is relatively new. An IT auditor generally will not find any reference to IT governance in any IT management book published before the turn of this century. The concept evolved through the Committee of Sponsoring Organizations (COSO) internal control framework, as discussed in Chapter 1, where enterprise governance issues appear as the foundation of that internal controls framework as the internal controls environments component. COSO talks about a wide range of enterprise governance controls, such as the importance of an active audit committee and a senior management tone at the top, and these same high-level internal control messages apply to the IT function as well. IT internal controls at a governance level involves ensuring that effective IT management and security principles, policies, and processes with appropriate compliance measurement tools are in place to assess and measure those controls. Effective IT governance controls require an active audit committee that goes beyond just financial reporting issues and reviews a wide range of enterprise issues, including IT matters. As discussed in Chapter 5, internal control findings and recommendations that involve IT-related issues should be reported to the audit committee in a matter that they can understand the issues. Although Sarbanes-Oxley (SOx) rules require that every audit committee must have at least one ‘‘financial expert,’’ there are no such requirements for a board-level IT expert. Rather, the IT audit function and the overall IT audit group should present their recommendations and issues such that board members can easily understand IT audit’s concerns and issues. This is often a challenge for IT auditors not to get too technical and to present things in an easy-to-understand manner. IT governance controls can also present concerns because sometimes top-level enterprise governance issues are not actively communicated and promoted to an enterprise’s IT function. This was particularly true years ago, when many of the IT staff worked behind the locked doors of the computer room and did not have much dayto-day contact with other enterprise activities. This same isolation, however, still exists today; the IT function frequently is separated from many other enterprise activities. Chapter 8 discusses processes for planning and performing all levels of IT general controls audits. Review steps for assessing IT infrastructure controls are included in that chapter’s IT audit procedures. For example, while reviewing IT operations, an IT auditor should determine that members of the IT staff understand such matters as an enterprise’s overall employee code of conduct or the existence of any enterprise-level whistleblower programs to independently report any potential problems. E1C06 09/16/2010 13:20:57 158 & Page 158 General Controls in Today’s IT Environments IT MANAGEMENT GENERAL CONTROLS Exhibit 6.1 shows three levels of IT controls: standards, organization and management controls, and physical and environmental general controls. Following high-level policies, these should be considered in a top-down manner. That is, IT standards should follow top-level policies, with good IT organization and management controls at the next level down. The third level of IT management general controls is the overall category of IT physical and environmental controls. Although we are describing these as levels of internal controls, IT auditors should recognize that they are not separate internal control domains; all are related to each other on many levels. In addition, an enterprise often does not have just one IT function; it may have multiple units differing by geographic location, line of business, or IT technology. IT auditors always should look for certain minimum control requirements in each of these areas but realize some IT general controls may differ across different environments. IT Standards Standards exist to support the requirements of an enterprise’s generally stated policies. Larger enterprises are in a position to develop their own IT standards, but many will adopt best practices or recognized professional practices as the basis for their IT standards. The ITIL set of best practices (discussed in Chapter 7) is an example of IT documents that can be established as an enterprise’s IT standards. IT auditors should look for established standards within an IT organization, covering such areas as new systems development, software acquisition, documentation, and applications control procedures, among many other issues. Evidence should be in place to demonstrate that procedures have been communicated and followed by all members of the IT organization. Examples of these IT general controls standards include: & & & E-mail communications protocols. An enterprise can experience many e-mail abuses, such as massive and unnecessary attachments connected to messages, excessive use of ‘‘cute’’ graphics, or the sharing of confidential data. Similar to the auditor codes of conduct introduced in Chapter 3, an enterprise and its IT function should develop standards for e-mail use and messaging. Systems development standards. Although IT functions develop far fewer from-scratch programs and applications today and typically purchase necessary software packages, systems development standards are needed. Enterprises need to establish systems development life cycle (SDLC) process standards to develop and implement new IT applications. The SDLC process is discussed in Chapter 10. IT documentation. Standards should specify the minimum level of documentation required for each application or IT installation. These standards should include different classes of applications, processes, and physical IT facilities. Chapter 12 discusses IT documentation and records management processes. These are just a few examples of the types of standards an IT auditor should expect to find when reviewing general controls in an IT facility. For a larger enterprise, these E1C06 09/16/2010 13:20:57 Page 159 IT Management General Controls & 159 standards may exist at multiple levels, with general standards issued through the chief information officer (CIO) office and more detailed standards in place for development and processing centers. An IT auditor should document and understand these standards and, if appropriate, perform audit compliance procedures against these established standards. IT Organization and Management Effective IT organization and management processes are an important area of general controls. Whether a larger enterprise with multiple, large facility data centers or a smaller business with a server system and a limited number of attached terminals, an IT auditor should have a good understanding of overall management controls, including provisions for adequate separations of duties, financial management, and overall change management controls. If a new IT facility that has not been reviewed in the past, an IT auditor should perform a preliminary review of the IT function’s organization and management controls, as described in Exhibit 6.2. An IT audit–led preliminary survey often is not structured as a formal IT audit; it can be an informal review to better understand the internal controls processes in place. A first-time survey of the IT organizations should be documented in audit workpapers, as discussed in Chapter 5; workpapers should be updated for subsequent follow-up reviews. This is not a formal IT audit but a way to gain some background information to serve as a basis for scheduling other IT audit reviews or for determining the status of controls issues that surfaced in prior reviews. Although a preliminary survey can cover a wide range of areas regarding IT organization and management issues, the need for adequate separations of duties is vital in many controls and is often the most major control issue. The functions of initiating, authorizing, inputting, processing, and checking data should be separated to ensure that no individual can create an error, omission, or other irregularity and then authorize it as evidence. Traditional separations of duties within an IT environment are divided between systems development and operations, where operations personnel responsible for running production systems should have little or no contact with the development process. IT Physical and Environmental Larger System General Controls In older, traditional IT environments, computer operations were often IT audit’s prime area of internal control concerns. Computer operators had considerable power to make changes or to bypass systems controls, such as overriding data file label protections, making changes to program processing sequences, or inserting unauthorized program instructions into production applications. Although these overrides still are possible today, the complexity of large computer operating systems, the often-complex connections between systems servers, and the sheer volume of work passing through a modern IT operations center make unauthorized operator override actions much more difficult. IT audit often has greater risks to consider, and many once-common IT operations control improvement recommendations are no longer feasible. For example, older business data center computers had a terminal monitor or even a console printer attached to record operator commands; IT auditors traditionally recommended that these console logs be E1C06 09/16/2010 13:20:57 160 & Page 160 General Controls in Today’s IT Environments 1. Obtain basic information about the environment through initial exploratory discussions with information technology (IT) management. 2. Review the organizational chart and position titles to determine that appropriate separation of functions exists. Discuss any potential conflicts with IT management. 3. Obtain job descriptions of key IT personnel, and review them for adequate and appropriate qualifications, task definitions, and responsibilities. Ensure that security and control accountability are appropriately assigned to key personnel. 4. Based on discussions within management both inside and outside the IT organization, assess whether the IT organizational structure is aligned with business strategies to ensure expected IT service delivery. 5. Review documented IT policies and selected procedures for completeness and relevance with specific emphasis on security, business continuity planning, operations, and IT customer service. 6. Inquire whether responsibilities have been assigned to keep the policies and procedures current, to educate/communicate them to staff members, and to monitor compliance with them. 7. Based on discussions with senior IT management, assess whether strategic, operational, and tactical IT plans are in place to ensure alignment with the organization’s overall business plans. 8. Determine the existence of an IT steering committee, and review this committee’s functions through a limited review of steering committee meeting minutes. 9. Assess whether IT planning and control linkages have been established through communication or reports to the audit committee. 10. Ensure that a formal methodology is used in the development of new systems or major enhancements to systems in production. The methodology should include formal steps for definition, feasibility assessment, design, construction, testing, and implementation as well as formal approvals at every stage. 11. Determine that well-controlled processes are in place for making changes to application programs in production, including testing and documentation sign-off, and formal approvals to implement the change into production. 12. Ensure that responsibility for physical and logical security has been appropriately apportioned and that appropriate documented procedures exist. 13. Review procedures in place for operating and maintaining the physical and wireless IT network, in terms of device configuration and software parameter changes, and ensure that procedures for allocating and maintaining the network configuration are performed on a scheduled basis and under proper change management. 14. Review the business continuity and IT disaster recovery plans to ensure that detailed plans for recovery of operations have been prepared, and that the plans are documented, communicated to the appropriate personnel, and properly tested on a periodic basis. 15. Review both the IT budget and actual costs as well as performance against those measured to assess financial performance. Discuss reasons for any variances. EXHIBIT 6.2 IT General Controls Preliminary Survey Review Steps reviewed on a regular basis. IT operations management often ignored these log reports, but they were useful for tracing inappropriate operator activities. Today, this console activity is still recorded onto log files, but it is not even printed but recorded on server files; the sheer volume of that data makes a periodic human review of console log reports unrealistic; other tools and controls are available to help IT auditors understand operations controls. An IT auditor should gain an understanding of the IT organization, its established control procedures, and any operation’s specialized duties and responsibilities. E1C06 09/16/2010 13:20:57 Page 161 IT Management General Controls & 161 Today, IT systems are operating in a client-server world, where powerful but very major server computers drive a wide range of terminals and storage devices. Computer operations centers are much more automated and efficient than they were not so many years ago. User terminals are protected by firewalls and are connected— through either cables or wireless connections—to complex enterprise-controlled networks or to the Internet. We have even moved to the concept of cloud computing, where the IT network is so vast and complex that it appears as if we are just connecting a device to a cloud in the sky to receive interconnections. There is more on cloud computing in Chapter 9. Our general controls discussion here covers in-house enterprise IT operations. An important step in an audit review of IT operations general controls is to clearly define the planned review’s objectives. All too often, management may ask IT audit to ‘‘review computer systems controls’’ in a data center without any clear objectives for the review. Management memories do not fade that fast and that management request for an audit may be based on IT controls as they once existed in older, legacy systems. An IT auditor should consider these questions when planning the review: & & & What is the purpose of the information system operations review? Which specific controls and procedures are expected to be in place? How can evidence be gathered to determine if these general controls work? Based on the results of this exercise, IT audit should develop a set of control objectives specifically tailored for the planned review rather than just use a standard set of internal control questions. The IT audit objectives identified always depend on the purpose of the review. If management has requested a review of the costs and efficiency of data center operations, for example, IT audit procedures might include such areas as chargeback procedures and the job-scheduling systems. Although larger-system IT general controls reviews can have a variety of purposes, they often fit into one of four review types: 1. Preliminary reviews of IT general controls for server-based IT operations. This is type of review, to gain a general understanding or overview of the IT control environment, is outlined in Exhibit 6.2. IT auditors have performed these reviews over the years, going back to legacy mainframe systems. IT audit asks questions, observes operations, and reviews documentation, but there is typically only very limited testing, if any. For example, IT audit might inquire about the procedures for updating production program libraries and might review the documents or processes used for program library approvals. However, the auditor probably would not select a sample of the programs in the production library to determine if proper library update procedures had been followed. A preliminary review can help determine the need for a more detailed general controls review, an extended control risk assessment at a later date, or can gather internal controls information for a specific applications review. This type of review is limited in scope and may not cover all aspects of IT operations. Areas where this level of review would be appropriate might include a preliminary controls review of IT operations at a new acquisition or a follow-up review after a very detailed general E1C06 09/16/2010 13:20:57 162 & Page 162 General Controls in Today’s IT Environments controls review from an earlier period; the review here would emphasize changes in control procedures as well as actions taken on prior audit recommendations. 2. Detailed general controls reviews of IT operations. A comprehensive, detailed review of IT general controls should cover all aspects of IT operations, including systems programming, wireless and telecommunications controls, and storage management, including virtualization. A detailed general controls review, including tests of controls, often requires IT audit to spend considerable fieldwork time in both the IT operations and the development functions. The preliminary review sometimes can be performed by a less experienced auditor who is developing IT audit skills, but a detailed general controls review is best performed by more senior IT audit staff members with a good understanding of IT controls and procedures. Based on a preliminary IT operations walk-through review, IT audit should develop an understanding of the control procedures over IT operations. The detailed audit procedures performed can be modified based on this preliminary information. Questions an IT auditor might pose could include: & How is work scheduled? Some server operating system procedures do little more than initiate jobs from a production job queue file, while operations personnel sometimes have considerable authority in deciding which jobs to run. In the latter situation, IT audit might want to spend time reviewing control log reports and operator instructions. If these procedures have been automated, IT audit may want to consider a specialized review of the production control software area. & How is storage media managed? Automated tools often are used here. In addition, some operations have a separate library facility where production media cartridges are mounted. Even when software has been installed, computer operators often can bypass label controls and introduce incorrect files into a production environment. & What types of operator procedures or instructions are used? Server-based systems soft- and hard-copy operations documentation can take a variety of formats; IT audit should have a general understanding of these documentation formats and content to help in the design of specific audit tests. & How is work initiated, and how does it flow through operations? In many client-server operations, IT production is initiated through remote job entry user terminals. In others, the production control function funnels all necessary input data to machine operations. Some functions rely on users to initiate most inputs through their network terminals. The type and nature of IT audit’s tests will depend on these procedures. The basic idea here is that IT audit should understand how the server-based operations function. The effective IT auditor should go through a set of these types of questions prior to each review. A larger systems operations function may install new procedures from time to time, changing or adding complexities to the control structure. The audit procedures to be performed in a detailed review of general controls for a legacy computer system can be extensive, depending on the size and scope of the audit. Exhibit 6.3 contains a limited set of control objectives for this type of review. 3. Specialized or limited-scope reviews. Because of management requests and perceived risks, IT auditors often perform limited reviews over specialized areas within E1C06 09/16/2010 13:20:57 Page 163 IT Management General Controls & 163 1. Determine the servers and other IT equipment is located in a secure, environmentally controlled facility. 2. Discuss physical and environmental control procedures with IT management to determine current policies and future plans. 3. Tour computer room facilities and observe physical security strengths and weaknesses, including: a. The existence of locking mechanisms to limit computer room access only to authorized individuals b. The placement of computer room perimeter walls and windows to limit access c. The location of power transformers, water chiller units if appropriate, and air-conditioning units to provide proper protection d. The general location of the computer room facilities within the overall building to minimize traffic e. The existence of fire detection equipment, including zone-controlled heat and smoke detectors f. The existence of a zone-controlled, overall fire protection system, including local extinguishers 4. Review computer room temperature, humidity, and other environmental controls, and assess their adequacy. 5. Briefly review maintenance records to ascertain that physical and environmental controls are regularly inspected and maintained. 6. Production processing should be scheduled to promote efficient use of computer equipment consistent with the requirements of systems users. Through interviews with operations management, develop an overall understanding of computer processing demands, including online and batch production work as well as any end user computing. 7. Also through interviews, describe the telecommunications network surrounding the computer system, including firewalls and connections to all servers, workstations, Internet links, and wireless connections. 8. Review procedures for scheduling regular production jobs including the use of automated job scheduling tools. 9. Match a limited number of scheduled production jobs against actual completion times to determine whether actual schedules are followed. 10. Determine that operating system job classes or priority codes are used to give proper priority to critical production jobs, and evaluate procedures for rush or rerun jobs. 11. Review documentation standards for production applications to determine that they provide operators with information regarding: a. Normal operations, including instructions for special forms, tape files, and report disposition b. Application restart and recovery procedures 12. Review procedures, automated or manual, for turning new applications or revisions over to production to determine there is a review by operations following standards. 13. Determine that policies prohibit IT operations personnel from performing programming tasks or running unauthorized jobs. 14. Determine that production source libraries cannot be accessed by operations personnel. 15. Assess IT procedures for periodically reviewing the contents of automated log files or otherwise monitoring improper operator use of server equipment. 16. Review and document procedures for changing production programs or procedure libraries when emergency situations require special handling. 17. Determine that all emergency processing activities are properly documented and are subject to subsequent management review. 18. Select several documented emergency program fixes and determine that the necessary changes were added to production processing libraries and were documented. EXHIBIT 6.3 Larger-System General Controls Review Objectives (Continued) E1C06 09/16/2010 13:20:57 164 & Page 164 General Controls in Today’s IT Environments 19. Determine that an automated system is in place to log all IT systems activity, including all jobs and programs run, any reruns, abnormal terminations, or operator commands and data entered through system consoles. 20. Determine that all appropriate server activity logs are reviewed periodically, that exception situations are investigated, and that the results of investigations are documented. 21. Determine that files produced from server operating system’s log monitors are retained long enough to allow investigation of unusual activities. 22. Review procedures for logging problems to determine that all abnormal software and hardware operating conditions are documented. 23. Determine that schedules exist for the submission of any critical input batch files and that procedures exist to follow up on missing data. 24. Review procedures to prohibit unauthorized input or access to production files and programs. 25. Review a limited sample of scheduled production applications to determine that appropriate systems control techniques are used. 26. Determine whether users or IT personnel are responsible for reviewing output controls, and assess whether those control reviews are being performed. EXHIBIT 6.3 (Continued ) an overall IT function. These reviews can be limited to one function, such as database administration, or a specialty area, such as systems supporting a particular business unit. Often, management will request that IT audit perform such a targeted specialized review due to some identified problem, such as the identification of a fraud. An audit of a highly specialized or technical area of IT operations often takes considerable IT auditor creativity in planning the work. Management may be concerned about the equity of the computer chargeback system and may ask the audit department to look at it. IT audit will need to gain a general understanding of the system used, spend time planning the additional procedures and tests to be performed, and then return to the actual testing. As IT resources have grown in complexity and importance to an enterprise, auditors can expect to perform more of these specialized, limited reviews. Because the IT function is a major resource in many enterprises, it may be inappropriate to attempt to review all IT general controls in all operational areas in one single detailed review. This would be the same as if internal audit attempted to perform a review of ‘‘manufacturing’’ in a major plant environment. Rather than cover all manufacturing functions, internal audit might review production control one year and receiving and inspection the next, and eventually cover other significant functions. For a specialized review of a specific IT control area, such as mass storage memory management, IT audit should expand on the procedures developed for a general controls review in that area and add additional audit tests as necessary. 4. Reviews to assess compliance with laws or regulations. One of the major objectives of internal control, as discussed in Chapter 1, is compliance with laws and regulations. IT auditors always should be aware of objectives in this area and include appropriate tests in their reviews. Auditors working with government agencies or in enterprises that do extensive government contracting may be required to perform IT-related compliance audits often to determine if appropriate E1C06 09/16/2010 13:20:57 Page 165 IT Management General Controls & 165 laws and regulations are being followed. These audits will differ very much from agency to agency and from one political division to another. A compliance-related IT review often can be combined with a preliminary or detailed general controls review, but IT auditors must be aware of the relevant procedures and regulations, such as those published by the government agency requiring the audit. Most bank examination agencies, for example, have published IT controls guidelines. When operating in this type of environment, IT auditors must be aware of the regulatory environment as well as any published procedures. Client-Server and Smaller-Systems General IT Controls IT auditors often face challenges in evaluating general controls in a smaller IT operation, ranging from networked client-server configurations to enterprise desktop systems. General controls evaluation problems arise because smaller systems often are installed with limited staffs in a more ‘‘user-friendly’’ type of environment. IT auditors, however, typically look for general IT controls in terms of the more traditional, larger mainframe IT environment discussed in the sections that follow. That is, IT auditors are looking for the strong physical security, good revision, and proper separation-of-duties controls that often do not exist or are only partially implemented in typical smaller-systems environments. This less formal approach was perhaps adequate when these small business or desktop systems were used primarily for single-office accounting or similar low–audit risk applications. The large capacity and capability of smaller systems, the growth of the Internet, and the transition to client-server computing has made these smaller systems important parts of the IT control framework. When faced with evaluating controls in these smaller computer systems settings, IT auditors sometimes revert to the traditional, almost ‘‘cookbook’’ types of controls recommendations. That is, they recommend that desktop systems be placed in locked rooms or that a small, two-person IT development staff is expanded to four in order to ensure proper separation of duties. There may be situations where such controls are appropriate, but often they are not applicable in a smaller business setting. IT audit can easily lose credibility if its control recommendations are not appropriate to the risks found in the smaller computer systems setting. Enterprises today implement increasing numbers smaller client-server systems to support business units, for specific departmental computing, or provide IT for the entire enterprise. Despite their smaller size, these systems often represent significant general control concerns. IT auditors should understand the general controls surrounding smaller computer systems. Adequate general controls are necessary in order to place reliance on specific application controls. Although some IT auditors once thought of smaller client-server systems as one generic computer system class (as opposed to larger, mainframe computers), technological changes have introduced significant differences in control procedures and IT audit concerns among them. Smaller systems can be implemented in a variety of ways, depending on the system configuration and the size of the enterprise. IT auditors should be able to recognize these differences and develop appropriate general internal control procedures to review their general controls. This section discusses these general controls E1C06 09/16/2010 13:20:57 166 & Page 166 General Controls in Today’s IT Environments in terms of smaller business Internet-connected and networked client-server systems general controls. These computer systems provide total IT support for a smaller business function or unit; they also may support unit or departmental computing functions in a larger enterprise in support of central computer systems resources. IT auditors may encounter all of these smaller computer systems in a single modern enterprise. Client-server systems are often a combination of various types and sizes of interconnected IT systems and are found in many enterprises. The term client-server first appeared in IT literature in the late 1980s. It is one of those IT terms that is difficult for non–IT specialists to understand, let alone describe. As a way of thinking about this configuration, IT auditors should remember that in a local network environment each of the workstations is a client; a centralized processor, which contains common shared files and other resources, is called the server. There also may be specialized servers for such tasks as storage management or printing. Workstation users submit requests from client machines to a server, which then serves that client by doing the necessary processing. This client-server architecture, however, goes beyond just a workstation and a server. An application that queries a centralized database can be considered the client while the database that develops the view of the database is the server to all workstations requesting database service. Similarly, an application program can request services from an operating system communications server. Exhibit 6.4 shows a client-server system sample EXHIBIT 6.4 Client-Server System Configuration E1C06 09/16/2010 13:20:59 Page 167 IT Management General Controls & 167 configuration where a single server handles requests from multiple clients across a network. This client-server configuration, though very general, represents the typical IT system of today. Small Business IT System Controls If an IT system is located in a secure facility and has a relatively large IT support staff, IT auditors probably should consider it to be a ‘‘larger’’ computer system for purposes of IT audit planning and should review for appropriate larger system general control procedures. This definition is not particularly precise, but it would cover the typical major IT system. This same type of attribute-based description can be more difficult in the smaller-system environment. A strict computer hardware architecture definition often does not help IT audit to decide when to apply smaller-system internal control review procedures. For example, smaller desktop computers can be coupled together with attached peripheral devices to provide more computer power than many traditional mainframe machines. When reviewing controls in such an environment, IT audit should consider these linked computers to be the same as the larger systems discussed earlier in this chapter. Another problem in identifying smaller computers is that they often look like larger processors. For example, IBM first implemented a small business system in 1988 called the AS/400. Their AS/400 product line and the individual machine capacities have been expanded many times to make these systems effectively operate beyond classic mainframe systems. Today’s desktop or laptop systems have had a rapid growth curve. Starting with hobbyists building their own microcomputers using integrated circuit chips, things really got started in the late 1970s when Apple Computer Corporation was formed and produced the Apple II microcomputer. Although the machine was initially viewed as a toy by many, a spreadsheet software package, VISICALC, introduced about a year later, made the Apple II a serious tool for business decision making. Several years later, in the early 1980s, IBM introduced its personal computer and legitimized the microcomputer as a serious business processing tool. Today, many computers are still said to be ‘‘IBM compatible,’’ even though IBM no longer manufacturers these products. Today, personal computers, often connected into networks, are used for many business IT applications. They are often the only computer system resource for a smaller enterprise and have replaced smaller ‘‘mainframe’’ systems. They also may be used for specialized departmental computing, even though there may also be a larger, mainframe computer capability within an enterprise. In particular, these specialized computers are used for such applications as research laboratory or manufacturing process control rather than for pure business IT. These same machines also may be used for some business processing applications in addition to their intended specialized purposes. Ever-increasing speed and capacity has done much to promote the use of these server systems. When the first Apple II was released, it had an internal memory of 42 k, or 42,000 memory locations. By the mid-1990s, by contrast, off-the-shelf machines typically came with 32,000,000 memory locations, or 32 mg. Today, these processors are much larger and virtually every other measure, whether processing speed, capability of running multiple tasks, or disk file memory capacity, has changed dramatically. E1C06 09/16/2010 13:20:59 168 & Page 168 General Controls in Today’s IT Environments These smaller business unit systems can cause difficulties for IT auditors who have sometimes told their audit committee that they plan to review the general controls surrounding ‘‘all’’ IT systems in the enterprise. Clearly, this type of objective covers major server systems as well as any separate divisional IT systems. However, any plans to review ‘‘all’’ enterprise IT systems can also include specialized IT workstations in the engineering laboratory used for recording test results or systems at the end of a production distribution line that weigh packages and route them to the correct shipping dock. These definition problems will only get worse as embedded systems play a greater role in controlling business processes. Embedded systems are the computers that reside behind the dashboard of a car or on the control panel of a video recorder or kitchen microwave. As consumers, we press these flat-panel screens and generally do not think we are submitting computer system commands. Although all of the systems just mentioned are computer systems, IT audit’s reviews should emphasize the computer systems used for business IT purposes. To follow the previous example, the processor at the end of the distribution line probably uses a standard set of embedded software that cannot be modified by local staff. It was very possibly purchased from an outside systems vendor, and, after initial installation and testing, it simply works, with no programmer interaction. Such a machine generally has limited business or control risk implications. IT audit often works in an environment where only smaller business systems are used, particularly when the enterprise is relatively small. An example would be a not-for-profit enterprise whose only systems needs are a server and desktop systems to support direct mailing and limited accounting-related applications. IT audit should review general controls over such a server system as if it were a classic, larger enterprise system. That is, there is still a need for systems security, integrity, and backup procedures. These types of smaller business systems generally have these common characteristics: & & Limited IT staff. The small-business IT system, whether a single Internetconnected desktop system or a series of units tied to a server, will have a very limited dedicated IT staff, if any. A desktop system to provide accounting reports for a small company may be maintained by a single person. A small-business or server system may have a manager/administrator and perhaps one or two systems administrators as its total IT department. Such a small IT operation creates a control risk because it is dependent on some separate small consulting firm for much of its IT support, and requirements for backing up critical files may be ignored. However, a small staff size will not in itself create internal controls concerns. IT audit should be able to look for compensating controls just as it does when reviewing a smaller accounting department where a classic separation of duties is lacking. Limited programming and systems development capability. The typical small business IT system makes extensive use of purchased software packages. The only ‘‘programming’’ needed may be for installing and updating purchased software, maintaining systems parameter tables, and writing simple retrieval programs. If an IT auditor finds a programming staff or extensive in-house development activity at the beginning of a review, he or she should consider some of the control procedures discussed for larger systems development functions. E1C06 09/16/2010 13:20:59 Page 169 IT Management General Controls & & & & 169 Limited environmental controls. Small business systems generally can be plugged into normal power systems and operate in a fairly wide range of temperatures. Because of their limited requirements, they sometimes are installed without important, easy-to-use environmental controls, such as backup drives or electrical power surge protectors. Some small business computer installations or file servers may be housed in formal, environmentally controlled computer rooms, but this is not a necessary attribute of these systems. Limited physical security controls. Because these systems have less need for environmental controls, often they are installed directly in office areas. The level of IT auditor concern regarding physical security controls depends on the type of equipment and the applications processed. IT audit sometimes may recommend that physical security be improved, particularly where critical applications are being processed. In many other instances, however, this lack of physical security controls should not present a significant internal control problem. Extensive telecommunications network. Virtually all desktop systems today are tied to the Internet. Data and applications can be uploaded or downloaded easily. In addition, materials can be downloaded easily through common USB devices. A combination of controls and policies should be established to protect the enterprise. An IT auditor should look for the effective use of installed antivirus software as well as systems firewalls; both of these are discussed in Chapter 19. These characteristics certainly do not define a smaller business IT system but merely explain some common attributes. However, they should help IT audit to better decide on the control procedures to be used. When in doubt, however, IT audit should consider the system to have the internal control characteristics of a larger, more complex IT system. Auditing General Controls for Smaller IT Systems Some smaller IT systems may be separate operating units of a larger enterprise and provide support for the total enterprise. Such systems may have many of the attributes of a larger, mainframe computer system, including a limited but formal IT enterprise, production schedules, and a responsibility for implementing new applications. However, the smaller IT system enterprise often has no other specialized functions. IT audit will encounter a variety of computer hardware brands or product names in smaller-systems environments, but most will be open systems with a common operating system that can operate no matter what brand of hardware is used. This is different from the classic mainframe computers of an earlier era, where the manufacturer generally built the computer hardware as well as the operating system. Numerous vendors supply such small business computer systems with both improved functionality and price performance, and IT auditors will be more effective in reviewing small business IT system controls if they have an overall knowledge of some of their capabilities. Despite the more informal nature of a typical small-business IT system, it still should have some similar general control objectives with these internal control concerns: & Smaller system controls over access to data and programs are often weak. When unauthorized persons are allowed to access and modify computer E1C06 09/16/2010 13:20:59 170 & & & Page 170 General Controls in Today’s IT Environments files, general controls are very much weakened. IT audit should consider access to data and programs to be the major general controls objective when reviewing the smaller IT enterprise. This is true whether the IT department uses packaged software products or spreadsheet or database applications developed in-house. Controls over access to data can be considered in terms of both specific applications and general controls. However, in smaller IT systems, general controls often have a greater importance than specific application data access controls because applications operating on a single small business computer system typically all operate under the same set of data access controls. In a small system, data can be improperly accessed and modified through user terminals, unauthorized use of specialized utility programs, or invalid IT requests. Improper data access through user workstations. Small systems—whether a series of laptops connected through a wireless system or a powerful server system— often do not have the sophisticated security controls found on the older, large, mainframe-type server systems. Rather, these smaller systems have a user logon/ password identification coupled with menu-based information security. A systems user typically enters the assigned logon code onto the terminal and receives a menu screen with the applications available to that code. Only then can the user access the applications assigned to that menu. These menu-based security systems can provide a fairly effective control against improper access attempts but also can break down due to the informality of many smaller enterprises. Logon codes are often not changed on a regular basis, one general menu is given to virtually all employees, or terminals with more privileged IDs are left on for virtually all to use. Because users are often not aware of potential data vulnerabilities, management may give only minimal attention to such security issues. In order to review general controls in this area, IT audit should first gain an understanding of the data security system installed; it may range from a good password-based system to a highly structured set of procedures. Because a small-business IT system may not have installed logging mechanisms to monitor invalid access attempts, IT audit should review the overall administration procedures covering the security system. These can include reviewing how often logons are changed, who has access to the system administrator’s menu, and what local management’s general appreciation is of IT access controls. Unauthorized use of utility programs. Modern small systems are often equipped with powerful utility programs, designed to be used for special problem-solving situations that can easily change any application data file. All too often, these utilities serve as substitutes for normal production update programs or are used by an IT manager for these special updates; sometimes they even are given to users. For example, an enterprise may have installed an inventory status system. Although the system normally provides proper stock-keeping records, the inventory status may become misstated from time to time for a variety of reasons. In order to help users correct their inventory status record-keeping problems, the IT administrator may have developed the practice of correcting inventory balances by using a utility program. The IT manager may be following proper management direction in the normal use of such a program, but there may be no audit trails over E1C06 09/16/2010 13:20:59 Page 171 IT Management General Controls & & 171 its use. IT audit can assess the usage of any such program through inquiry and observation. Improper IT data and program access requests. The informality of smaller enterprises often allows data to be accessed improperly through normal IT operations procedures. For example, someone affiliated with the IT function may initiate a special computer run, which results in an improper access to confidential data. In larger, more formal enterprises, such a request often requires special management permission, but smaller, more informal enterprises often waive such requirements. This type of access may be a greater control risk than access through use of improper programs. IT audit should look for controls to prevent such casual IT requests through the use of a request-for-data-services type of form, approved by management. In addition, logs should be maintained listing all production IT activities as well as the name of the requester and the report recipient. Many of the control concerns over improper access to data also apply to small-system program libraries. Small-business systems typically have menu-based systems that offer some security types of controls. Without such a proper menu type of security system to limit improper access, often it is relatively easy for someone with a little knowledge to locate and potentially modify program library files. IT audit also may find weak controls over program library updates. The one or two IT personnel in a smaller IT department who act as network administrators typically can update program libraries with little concern for documenting those changes or for obtaining any type of upper management authorization. Some of these changes may be justified in order to respond to user emergency requests, but others may not be properly authorized. It is difficult, if not impossible, to install separation-of-duties enterprise controls over small-business system program libraries. In addition, it probably will not work for IT audit to suggest that management formally review and approve all program library updates—management will neither be interested in performing nor have the technical skills to perform such reviews. The best control method here might be to install procedures that require the logging of all changes or software package updates to the production program library, with such logs subject to periodic IT auditor reviews. This type of control takes advantage of the fact that many small-business IT systems maintain hash1 total counts of program sizes in bytes and also can retain some form of date or version number within the program name. IT audit might then suggest this small-business computer system program library control: & & Establish program-naming conventions that include the date or version number included with the program name. When not available in commercially purchased software, a separate control file with this data can be established. This feature is becoming increasingly common; for example, it can be implemented within Microsoft Windows XP or Windows 7 operating systems. Have the persons authorized to make program table or parameter changes log in the version number, date, program size, and reason for the change in a manual listing subject to periodic management. If the application was developed in-house, the source code should contain comments explaining the change. E1C06 09/16/2010 13:20:59 172 & & & & Page 172 General Controls in Today’s IT Environments Maintain at least one backup copy of the program library, and rotate a copy of the program library file to a secure portable disc drive in an off-site location at least once per week. Strengthen access controls such that nonauthorized personnel cannot easily access program library files. Perform an IT audit review of the library change log on a periodic basis. That review should match logged program versions, dates, and sizes with data reported on the program library file. These steps will not provide IT audit with complete assurance that all program changes have been authorized; however, if IT audit periodically reviews logged changes and questions any discrepancies, IT personnel at that facility probably will take care to document and log any production program changes. The message throughout this section is that there are or should be some general IT internal control concerns for all smaller systems, whether a network of laptops coupled to a server over wireless links or a free-standing office desktop system. There are many variations in the types of small-system IT configurations, but IT auditors should use some general internal control objectives to review general controls in these IT environments. Exhibit 6.5 outlines some internal control considerations for an IT audit review of a smaller-business IT system. In many instances, these review steps should be included as part of a larger, more comprehensive review in other areas of business operations. Nonbusiness Specialized Processor IT Systems In many enterprises today, systems can be found in areas beyond IT operations. They may be located in engineering laboratories, manufacturing control operations, marketing departments, and many other areas. These systems may be used for process control, automated design work, statistical analysis processing, or many other applications. Some are totally dedicated to specific applications while others may be used for a variety of tasks within their assigned functions. This multitude of IT machines has come about because of their relatively low cost, the familiarity of many professionals with IT techniques, and of the inability of traditional IT departments to support specialized IT needs. Although these systems are not used for traditional business information needs, such as maintaining accounts receivable records, they often support critical applications for the enterprise. For example, an engineering department computer may support new product computer-aided design (CAD) work. Systems backup and integrity concerns in this environment may be as great as in the typical business IT center. IT audit’s role in regard to specialized IT operations will vary with both management’s direction and review objectives. Some internal audit organizations will have little involvement with reviews of specialized computer systems, but the internal controls reviewed here often can play an important role in support of IT audit’s understanding of control procedures and in other operational audit activities. Before attempting any review of such a specialized computer system, IT audit should obtain a rough familiarity with the functions of that operation. For example, an IT auditor who plans to review a dedicated computer-aided design and manufacturing E1C06 09/16/2010 13:20:59 Page 173 IT Management General Controls & 173 1. Establish and document responsibilities for the IT function. Identify persons responsible for all IT operations functions and document their authorities and responsibilities. 2. Determine if there is a complete and current inventory of systems hardware, including servers, firewalls, printers, and network controllers, as well as a complete inventory of application and systems software. 3. The network hardware inventory report should contain model and identification numbers. Review a limited sample of these items to determine whether the equipment is installed as described. 4. Trace a sample of the listed application and systems software to determine that current versions are installed, that appropriate documentation is in place, and that vendor licenses are current. 5. Review file and data backup procedures, and determine whether this data is backed up regularly to secure locations. 6. Observe computer server facilities, and verify that equipment is located in limited-access secure facilities with adequate power and environmental controls. 7. Observe key file storage backup processes to determine that media are regularly backed up to secure off-site locations. 8. Assess the adequacy of access control security procedures to determine that key systems and files are adequately protected by passwords that are regularly changed. 9. Review procedures in place for restricting, identifying, and reporting on unauthorized users of the network environment, and assess the adequacy of processes to investigate and correct security violations. 10. Assess the adequacy of systems security monitoring processes as well as new employee training practices in place to emphasize application security. 11. Determine that adequate antivirus software has been installed and is regularly updated. 12. Review the adequacy of procedures for installing new software in the systems environment, and assess that controls are in place to prevent the introduction of unauthorized software products. 13. Review a sample of key applications, and verify that they are supported by adequate continuity plans for disaster recovery purposes. Also, determine that continuity plans are tested periodically. 14. Interview persons responsible for network security to determine that adequate firewall tools have been installed and are monitored regularly. 15. Review records of systems downtime over a recent period, and determine that adequate shortand long-range measures are in place to promote continual processing improvements. 16. If available, obtain application operating schedules covering key financial and operational applications, and determine the adequate attention is given to application internal controls. 17. Interview systems manager/administrator to assess whether this person is knowledgeable and properly trained. Also, if the system is managed by outside consultants, review the adequacy of systems support efforts. 18. Interview a sample of systems users, and determine if they are satisfied with system performance, including response times and availability. EXHIBIT 6.5 Review Objectives for Smaller-Business IT System General Controls (CAD/CAM) computer operation needs a general understanding of the terminology, general workings, and objectives of CAD/CAM. Reviews of these specialized IT systems are not recommended for the less experienced IT auditor. In order to find control analogies from normal business IT situations and translate them to specialized controls environments, an auditor must be fairly experienced in reviewing business IT operations. Over time, IT audit will encounter more of these specialized computer operations. The creative IT auditor can make E1C06 09/16/2010 13:21:0 174 Page 174 & General Controls in Today’s IT Environments increasing contributions to management by performing operational reviews over these computer centers on a periodic basis. IT TECHNICAL ENVIRONMENT GENERAL CONTROLS The last three levels of IT general controls in the IT controls hierarchy diagram in Exhibit 6.1 describe more technical IT general controls: systems software, systems development, and application-based general controls. Systems software and system development general IT controls are discussed in Chapter 8. Application development general controls are discussed in Chapter 10. This chapter has introduced the high-level process of auditing IT general controls, the pervasive controls whose effectiveness is essential for strong IT internal control environments. Whether working with a large enterprise with vast IT resources or a smaller business system, IT auditors should identify and then assess the effectiveness of their IT general controls. If IT general controls are weak, IT auditors can expect to find deficiencies in application and other internal controls throughout the organization. Even though some IT auditors find reviews of IT technical areas, such as cybersecurity of operating systems procedures, to be more interesting from an audit perspective, enterprise IT general controls should be reviewed on a regular and ongoing basis. If an area has been reviewed in a past period, the results of those reviews should be repeated with appropriate updates and adjustments for new management policies, standards, and IT hardware and organization changes. IT auditors always should recognize that the strengths and weaknesses of many of other areas reviewed depends on these IT general controls. NOTE 1. A hash total is a summation of the numeric and alphabetic values for some computer value. It is often used as a control total. E1C07 09/16/2010 13:31:13 Page 175 7 CHAPTER SEVEN Infrastructure Controls and ITIL Service Management Best Practices C H A P T E R 6 D I S C U S S E D A U D I T procedures for reviews of information technology (IT) general controls; these are the persuasive types of controls that are installed throughout an enterprise’s IT systems operations and provide protection for all systems and applications. Examples of general controls might be physical locks and other security controls for a hardware server center or a common IT password security system covering all enterprise IT operations. As Chapter 6 emphasized, weak IT general controls will impact all IT applications that are part of those systems operations. In today’s world of pervasive IT processes and systems, installed throughout the enterprise and ranging from an application, to control of an accounting general ledger, to the all-pervasive Internet, IT auditors should have a strong understanding of IT internal control techniques. Although the lines of separation are sometimes difficult, we generally can think of IT controls on two broad levels: application controls that cover a specific process (such as an accounts payable application to pay invoices from purchases) and general IT controls. This latter category covers many controls that go beyond those discussed in Chapter 6; they do not relate just to specific IT applications and are important for all aspects of an enterprise’s IT operations. The concept of IT general controls goes back to the early days of centralized, mainframe computers. Today we often think of the set of processes that cover all enterprise IT operations as the IT infrastructure. This IT infrastructure is very different across enterprises, large and small, due to the relative size of their operations and the overall nature of their business. Because of the many possible variations in the types and sizes of IT systems and facilities that may be needed, there is really no one single set of 175 E1C07 09/16/2010 13:31:13 176 & Page 176 Infrastructure Controls and ITIL Service Management Best Practices control procedures here. Rather, an enterprise should implement a set of best practices that will guide it to establish its own IT general controls best practices. An important internal control concept here often goes beyond how IT applications reports and other IT outputs are delivered to business users. Every business IT function supports a wide range of IT service management processes, which include such areas as problem management (i.e., how IT resolves issues with its business users) or configuration management (i.e., how IT keeps track of installed software and equipment versions). IT service management covers a wide range of internal control issues, and there are some well-recognized best practices that an enterprise should install. This chapter looks at IT infrastructure general controls based on the set of worldwide recognized best practices called the Information Technology Infrastructure Library (ITIL). These ITIL-recommended service management best practices outline the type of framework an internal audit should consider when reviewing IT internal control risks and recommending effective IT general controls improvements. Because there is never a single definition for what is considered best, some refer to these as just good practices. We prefer to use the term best practices. ITIL SERVICE MANAGEMENT BEST PRACTICES ITIL is an abbreviation for an actual set of technical publications, a set of best practices first developed in the 1980s by the British government’s Office of Government Commerce (OGC), formerly called the Central Computer and Telecommunications Agency. It is an independent collection of best practices that was first widely recognized in IT operations first in the United Kingdom, followed by the European Union (EU), then Canada and Australia. It is now increasingly common in the United States. ITIL is a detailed framework of significant IT best practices, with comprehensive checklists, tasks, procedures, and responsibilities designed to be tailored to any IT organization. Dividing key service delivery processes between those covering IT service delivery and those for service support, ITIL has become the de facto standard for describing many fundamental processes in IT service management, such as configuration and change management. ITIL is a formal ‘‘library’’ of technical publications, all published by the British OGC.1 The publications are tightly controlled, similar to the International Standards Organization publications discussed in Chapter 18. IT auditors should be aware of ITIL and should determine if their IT functions have embraced any adopted ITIL best practices as part of their IT internal controls reviews. Our intent here is not to provide a detailed description of ITIL’s service delivery components but to give internal auditors a highlevel understanding of some of its components. An understanding of ITIL will allow IT auditors to better evaluate key processes and to make more effective recommendations when reviewing IT general controls. ITIL service delivery best practices cover what is frequently called the IT infrastructure—the supporting processes that allow IT applications to function and deliver their results to systems users. All too often, IT auditors have focused their attention on the application development side of IT processes and ignored important supporting service delivery processes. An enterprise can put massive efforts, for example, in building E1C07 09/16/2010 13:31:14 Page 177 ITIL Service Management Best Practices & 177 and implementing a new budget forecasting system, but that application will be of little value unless there are good processes in place, such as problem and incident management, to allow users to report and resolve systems difficulties. Also needed are good capacity and availability processes to allow the new application to run as expected. These ITIL processes are all part of the IT infrastructure, and a well-designed and controlled application is of little value to its users without such strong service support and delivery processes in place. IT auditors should have a good understanding of these enterprise processes and then develop an appropriate test of controls. These may have been covered in an IT general controls review, but ITIL provides a good general best practices model to follow. Although they were fairly common elsewhere in the world, ITIL best practices are now becoming widely recognized in the United States as well but have not been adequately recognized by many internal auditors. The Web site of the Information Systems Audit Control Association (ISACA) has numerous reference materials that tie ITIL best practices with the Control Objectives for Information and related Technology (CobiT) framework discussed in Chapter 2. Unfortunately, a search on the Institute of Internal Auditors’ Web site at the time of this publication contains no references to ITIL. All internal auditors who perform reviews that touch IT infrastructure areas should understand internal control procedures following ITIL best practices. The next sections provide an overview of some ITIL service delivery processes important for an IT auditor, including capacity or service-level management best practices. This should give an IT auditor some guidance on how IT functions, such as a help desk, should have effective processes in place in these very important areas of IT operations. ITIL does not specify standards for building and managing IT controls; it suggests new ways to implement and operate infrastructure general controls that should have already been in place. ITIL service delivery strategies can be viewed as a continuous activity life cycle, sometimes shown as three embedded process activity rings. The outer ring defines continuous service improvement processes. That is, an ITIL-ready organization should have continuous service processes in place that encompass all other service management processes and receive inputs from outside IT customer sources. There are three independent, linked processes within the continuous service improvement ring—service design, transition, and operations best practices; each is discussed in later sections. In the center of these concentric rings is the service strategy process. This core process includes the IT organization policies and practices that were described in the COSO internal controls framework control environment element introduced in Chapter 1. Exhibit 7.1 shows this same service delivery model as a feedback flowchart process. ITIL processes traditionally have been split between those covering service support and those for service delivery. Service support processes help make IT application operate in an efficient and customer-satisfying manner, while service delivery processes improve the efficiency and performance of IT infrastructure elements. There are five ITIL service support best practice processes, ranging from release management best practices, for placing an IT component into production, to incident management, for the orderly reporting of IT problems or events. ITIL service support processes cover good practices for any IT enterprise, whether a centralized operation using primarily E1C07 09/16/2010 13:31:14 178 & Page 178 Infrastructure Controls and ITIL Service Management Best Practices EXHIBIT 7.1 ITIL Continuous Feedback Loop classic legacy mainframe systems as its IT central control point, to highly distributed client-server operations. Because of the many variations possible in an IT operations function, ITIL does not prescribe the details of ‘‘how’’ to implement service support processes, such as their configuration or change management. Rather, it suggests good practices and ways to manage inputs and relationships between these processes. There is no order or precedence among each of these best practices. They can be considered and managed separately, but all of them are somewhat linked to one another, providing a linkage between the business operations, IT technology, and infrastructure management. Although there are many separate but interrelated elements to ITIL, we discuss only those service life cycle components that are more important for an IT auditor performing a general controls review. These ITIL best practices suggest preferred IT operations approaches to operate IT production systems in a manner that will promote efficient operations and will deliver quality services to the ultimate user or customer of these services. These best practices are particularly useful for an IT auditor performing a review and making recommendations in an IT operations area. When IT audit is observing and reviewing IT operations internal controls, often it is useful to think of the area being reviewed in terms of the separate ITIL processes discussed in the next sections. For example, the ITIL process called incident management, or what has traditionally been called the ‘‘help desk’’ is a facility where systems users or customers call in to IT operations with a question or problem. Although a help desk function can be very useful, it is often a source of grousing when, for example, similar problems are called in repeatedly with no evident efforts to initiate a solution to the problem. Going beyond just a casual help desk and thinking of these activities as an overall process where matters are reported to other supporting processes will improve E1C07 09/16/2010 13:31:15 Page 179 ITIL’s Service Strategies Component & 179 performance here and improve the overall quality of IT operations. When an IT auditor observes deviations from ITIL best practices, IT audit recommendations for performance improvement can become a major service to management. ITIL’S SERVICE STRATEGIES COMPONENT The upper left corner of Exhibit 7.1 of ITIL feedback loop processes shows a function called service strategies. This component describes the ITIL service management policies, strategies, and standards that provide input and direction to the other ITIL service design, transition, and operation processes. Those latter three components also provide inputs to service strategies to establish continuous process improvements. As a best practice, ITIL suggests that an IT management team should first ask some hard questions about the quality of their IT service function, including: Which Which Which Which & & & & of of of of our our our our IT services or service offerings are the most distinctive? services are the most profitable? customers and stakeholders are the most satisfied? activities are most different and effective? These are not the types of questions that IT management typically asks; they are seldom questions from management when assessing IT resources, and they are questions very seldom raised by IT auditors. Nevertheless, IT audit should consider these types of questions when performing a general IT controls review. The idea is to encourage the enterprise IT function to move from being from a resource that maintains IT processes and to one that provides valuable and cost-effective services to the overall enterprise. Exhibit 7.2 is a list of additional questions to help IT to improve its strategic capabilities and offerings. & & & & & & & What IT services should we offer and to whom? That is, do we serve all enterprise units, a limited sample, or outside customers? How do we differentiate ourselves from competing alternatives? Outside service providers offer alternative services, but what are the unique costs or values that make this IT function a better alternative? How can we truly create value for our customers? Too often, IT handles almost redundant, required services, such as month-end financial reports that receive little attention. IT should look to see how it can better service users. How can we make a case for strategic investments? Rather than regularly just submitting budget requests for such matters as software upgrades, IT should carefully justify such requests. How should we define service quality? Through surveys and collaborative work, all interested parties should recognize how to identify quality IT services. How do we efficiently allocate our resources across our defined portfolio of offered services? How can we resolve conflicting demands for shared services? EXHIBIT 7.2 Questions for Developing ITIL Strategic Capabilities E1C07 09/16/2010 13:31:15 180 & Page 180 Infrastructure Controls and ITIL Service Management Best Practices A multidisciplinary approach would be required for the questions in this exhibit because ITIL suggests that the IT organization should work with other functions, such as operations, finance, quality assurance, and internal audit, to better understand and define these key IT strategies for the enterprise. The whole idea is that an IT department or group should decide what it is in regard to the overall enterprise and what services it can offer. This type of introspective review may result in a service portfolio or catalog that defines IT’s capabilities and service offerings. ITIL service strategies introduce a best practices process that has been often ignored by both IT auditors and financial managers: financial management for IT services. Many IT auditors avoid this area, arguing that they are not accountants and do not need to worry about accounting-related issues. Classic financial internal auditors often see IT services as an issue that is too technical or of no interest. However, this is an important internal control area of potential concern and an ITIL best practice. Several other best practice areas under service strategies, such as organizational development, are not discussed here. In its earlier days, the IT function in most enterprises was operated as a ‘‘free’’ support service with its expenses handled through central management and costs allocated to users with little attention given to IT-related costs. If a user department wanted some new application, it would pressure management to purchase the software package and add any additional necessary people to manage it. Over time, IT enterprises began to establish chargeback processes, but these were too often viewed as a series of ‘‘funny-money’’ transactions where no one paid too much attention to the actual costs of IT services. Today, the costs and pricing of IT services are or should be a much more important consideration. The well-managed IT function should operate more as a business, where ITIL financial management is an important and key ITIL process to help manage the financial controls for that business. The objective of the service strategy financial management process is to suggest guidance for the cost-effective stewardship of assets and resources used in providing IT services. IT should be able to account fully for its spending on IT services and to attribute the costs of services delivered to the enterprise’s customers. There are three separate subprocesses associated with ITIL financial management: 1. IT budgeting is the process of predicting and controlling the spending of money for IT resources. Budgeting consists of a periodic, usually annual, negotiation cycle to set overall budgets along with the ongoing day-to-day monitoring of current budgets. Budgeting ensures that there has been planning and funding for appropriate IT services and that IT operates within this budget. Other business functions will negotiate periodically with IT to establish expenditure plans and agreed investment programs; these ultimately set the budgets for IT. 2. IT accounting is the set of processes that enable IT to account fully for the way its money is spent by customer, service, and activity. IT functions often do not do a good job in this area. They have a wide variety of external costs, including software, equipment lease agreements, telecommunications costs, and others, but these costs often are not well managed or reported. They have enough data to pay the bills and evaluate some specific area costs, but IT functions often lack the level of detailed accounting that can be found in a large manufacturing enterprise, as an example. E1C07 09/16/2010 13:31:15 Page 181 ITIL Service Design & 181 The manufacturing cost accounting or activity based accounting model has applicability there. 3. Charging is the set of pricing and billing processes to charge customers for the services supplied. This requires sound IT accounting and needs to be done in a simple, fair, and well-controlled manner. The IT charging process sometime breaks down in an IT function because the billing reports of IT services are too complex or technical for many IT service customers to understand. IT needs to produce clear, understandable reports of the IT services used such that customers can verify details, understand enough to ask questions regarding service, and negotiate adjustments if necessary. Financial management for IT supports the service strategy process through defined IT costing, pricing, and charging procedures. Although generally not operated as a profit center, the financial management process allows both IT and its customers to better think of IT service operations in business terms. The financial management process may allow IT and overall management to make decisions about what, if any, functions should be retained in-house or outsourced to an external provider. The financial management process allows accurate cost-benefit analyses of the IT services provided and allows the IT enterprise to set and meet financial targets. It also provides timely reporting to the service-level management process, such that customers can understand the charging and pricing methods used. Of all of the ITIL service support and delivery processes, financial management is one ITIL best practice that frequently gets short shrift. IT people have a technical orientation and tend to think of financial management as an accounting issue. On the other side of the coin, finance and accounting professionals tend to look at these issues as too technical and beyond such transactions as equipment lease accounting or facility space charges. IT auditors should develop some financial skills as well as using their IT knowledge to review and assess financial management process internal controls. Exhibit 7.3 provides procedures for an internal audit review of the costs and pricing of IT processes. This is often not a common review area for internal audit, but given the large costs distributed to customers as well as the importance of an enterprise’s IT resources, it can be an important audit area. ITIL SERVICE DESIGN ITIL service strategies support three other process areas, starting with the first phase in the service life cycle, service design. IT processes for service design cover areas more closely aligned with the smooth and efficient operation of the overall IT infrastructure. ITIL has identified five aspects of service design: 1. The design of each IT service offered, including their functional requirements, resource needs, and anticipated capabilities 2. The design of service management systems and tools, often presented through a formal portfolio, for the management and control of these services through their life cycles E1C07 09/16/2010 13:31:15 182 & Page 182 Infrastructure Controls and ITIL Service Management Best Practices 1. Develop and document a general understanding of the cost structure for IT operations, including costs of equipment supplies and salaries. 2. Review and understand costing philosophy for IT operations: Is it an overhead function, cost recovery, or revenue generating? 3. Review processes for costing and pricing IT services: a. Are all IT costs covered? b. Based on interviews with IT users, does the costing and pricing system appear to be understandable? c. Is there a process in place to administer the costing process and to make adjustments if necessary? 4. Review the negotiation process with IT users to understand pricing process: Are expected costs included in service-level agreements (SLAs)? 5. Select pricing reports during a period for several processes and check to determine the prices are included in SLAs. 6. Review appropriateness of the adjustment process over a period of time to determine the corrections are investigated and applied when appropriate. 7. Review a sample of data processing services billed for one accounting period, and determine whether they cover all actual IT costs. Investigate and report on any differences. 8. Review the overall budgeting process for IT operations, and determine if it appears consistent with other enterprise budgeting procedures. 9. Assess whether management reviews actual IT costs and compares them to established budgets, taking appropriate remedial actions as required. 10. For a selected accounting period, trace IT pricing charges to appropriate accounting system entries. EXHIBIT 7.3 Costs and Pricing IT Audit Review Steps 3. The design of IT architectures and management systems necessary to provide the services 4. The design of processes needed to install, operate, and improve these overall service processes 5. The design of measurement methods and metrics of the service processes and their component architectures What this really says is that every IT function installs a lot of customer services, and these services should be managed and controlled through appropriate best practice techniques. To support efficient service delivery, ITIL has specified a series of specific processes. Some of these, such as the continuity management process, have traditionally been near and dear to the hearts of many IT auditors. Others, such as service-level agreements (SLAs) that define performance and expectations between IT and its customers should be familiar to other internal auditors who encounter similar arrangements in other areas. Service Delivery Service Level Management Service Level Management is the name given to the process of planning, coordinating, drafting, agreeing, monitoring, and reporting on formal agreements between both IT and the providers and recipients of IT services. This process is managed through E1C07 09/16/2010 13:31:15 Page 183 ITIL Service Design & 183 service-level agreements (SLAs) that represent a formal agreement between IT and both providers of services to IT as well as IT end user customers. When the first ITIL servicelevel best practices materials were published in 1989, an SLA was an interesting but uncommon concept. Today many enterprises have introduced them—although with varying degrees of success—and IT auditors should be familiar with and understand the importance of SLAs when reviewing internal IT infrastructure controls. As an example of an SLA, when IT contracts with an outside provider, such as for disaster recovery backups, the arrangement will be covered by a formal contract where the disaster recovery provider agrees to provide certain levels of service, following some time-response-based schedule. The governing contract here is an SLA between IT and the provider of continuity services. SLA agreements between IT and their customers are even more important, from an internal control perspective. We have used the more current term of customer here rather than the older and still common term IT users. Many groups use IT’s services, and as customers, they have expectations of certain levels of service and responsiveness. These arrangements are defined through an SLA, a written agreement between the IT provider and its customers defining the key service targets and responsibilities of both parties. The emphasis should be on a mutual agreement, and SLAs should not be used as a way of holding one side or the other to ransom. A true partnership should be developed between the IT provider and the customer for mutually beneficial results; otherwise, the SLA could quickly fall into disrepute, and a culture of blame may prevent any true service quality improvements from taking place. In an SLA, IT promises to deliver services per an agreed-on set of schedules and understands there will be penalties if service standards are not met. The goal here is to maintain and improve on service quality through a constant cycle of agreeing, monitoring, reporting, and improving the current levels of IT service. SLAs should be strategically focused on the business and on maintaining the alignment between the business and IT. Exhibit 7.4 outlines the contents of a typical SLA. This type of document would not be found as part of a mortgage document signed at a house closing. Rather, the IT customers negotiate the IT service requirements that they are seeking, such as ‘‘average response times no more than . . . ’’ or ‘‘financial systems close processing completed by . . . ’’ or other factors. To temper expectations and show what could be available, an IT function usually provides a service offerings catalog. Customer IT service requirements should be negotiated and formal SLAs established. Performance against these SLAs should be monitored on an ongoing basis with performance reported regularly. Failure to meet these SLA standards could result in additional negotiations and SLA adjustments. This SLA process provides benefits for the business and IT, including: & & & Because IT should be working to meet negotiated standards, IT services will tend to be of a higher quality, causing fewer interruptions. The productivity of the IT customers should improve as well. IT staff resources will tend to be used more efficiently when IT provides services that better meet customer expectations. By using SLAs, the services provided can be measured and the perception of IT operations generally will improve. E1C07 09/16/2010 13:31:15 184 & Page 184 Infrastructure Controls and ITIL Service Management Best Practices While there is no one form or format for an SLA, the following are contents that should be considered for most SLAs: Agreement Introduction Pages & Parties to this agreement & Title and brief description of the agreement & Signatories & Dates: start, end, review & Scope of the agreement; what is covered and what is excluded & Responsibilities of both the service provider and the customer & Description of the services covered Service Hours & Hours that each service is normally required (e.g. 24  7, Monday to Friday 08:00–18:00) & Arrangements for requesting service extensions, including required notice periods (e.g., request must be made to the service desk by 12 noon for an evening extension, by 12 noon on Thursday for a weekend extension) & Special hours allowances (e.g., public holidays) & Service calendar Availability & Availability targets within agreed hours, normally expressed as percentages. The measurement period and method should be stipulated and may be expressed for the overall service, underpinning services, and critical components or all three. Since it is difficult to relate to simplistic percentage, availability can be measured in terms of the customer’s inability to carry out its business activities Reliability & Usually expressed as the number of service breaks, or the mean time between failures (MTBF) or mean time between system incidents (MTBSI) Support & Support hours (where these are not the same as service hours) including arrangements for requesting support extensions & Required notice periods (e.g., request must be made to the service desk by 12 noon for an evening extension) & Special hours allowances (e.g., public holidays) & Target time to respond to incidents, either physically or by other method (e.g., telephone contact, e-mail) & Target time to resolve incidents, within each incident priority—targets vary depending on incident priorities Throughput & Indication of likely traffic volumes and throughput activity (e.g., number of transactions to be processed, number of concurrent users, amount of data to be transmitted over the network) Transaction Response Times & Target times for average or maximum workstation response times (sometimes expressed as a percentage: for example, 95% within 2 seconds) Batch Turnaround Times & Times for delivery of input, and the time and place for delivery of output Changes & Targets for approving, handling, and implementing requests for changes (RFCs), usually based on the category or urgency/priority of the change EXHIBIT 7.4 Sample IT Service-Level Agreement Contents E1C07 09/16/2010 13:31:15 Page 185 ITIL Service Design & 185 IT Service Continuity and Security & Brief mention of IT service continuity plans and how to invoke them, and coverage of any security issues, particularly any responsibilities of the customer (e.g., backup of free-standing PCs, password changes) & Details of any diminished or amended service targets should a disaster situation occur (if no separate SLA exists for such a situation) Charging & Details of the charging formula and periods (if charges are being made). If the SLA covers an outsourcing relationship, charges should be detailed in an annex as they are often covered by commercial in confidence provisions Service Reporting and Reviewing & The content, frequency, and distribution of service reports, and the frequency of service review meetings Performance Incentives/Penalties & Details of any agreement regarding financial incentives or penalties based on performance against service levels. These are more likely to be included if the services are being provided by a third-party organization. It should be noted that penalty clauses can create their own difficulties EXHIBIT 7.4 & & (Continued ) Services provided by the third parties are more manageable when underpinning contracts are in place, and any possibilities of negative influence on the IT service provided is reduced. Monitoring overall IT services under SLAs makes it possible to identify weak spots that can be improved. The SLA process is an important component of IT operations. If an enterprise IT function does not use formal SLAs, IT auditors reviewing both IT operations general controls and business services applications should consider recommending the establishment of such formal SLA processes. SLAs can create a totally new environment within IT, where all parties will better understand their responsibilities and service obligations with the SLA as a basis for resolving many issues. IT audit can use the status of an enterprise’s SLAs while assessing internal controls in a variety of areas and for making strong controls improvement recommendations. Service Delivery Capacity Management ITIL capacity management ensures that the capacity of the IT infrastructure is aligned to business needs to maintain the required level of service delivery at an acceptable cost through appropriate levels of capacity. Through gathering business and technical capacity data, this process should result in a capacity plan to deliver cost-justified IT capacity requirements for the enterprise. In addition to a prime objective of understanding an enterprise’s IT capacity requirements and to deliver against them, capacity management is responsible for assessing the potential advantages new technologies could have for the enterprise. The capacity management process generally is considered in terms of three subprocesses: business, service, and resource capacity management. Business capacity E1C07 09/16/2010 13:31:15 186 & Page 186 Infrastructure Controls and ITIL Service Management Best Practices management is the long-term process to ensure that the future business requirements are taken into consideration and then planned and implemented as necessary. Service capacity management is responsible for ensuring that the performance of all current IT services fall within the parameters defined in existing SLAs. Finally, resource capacity management has more of a technical focus and is responsible for the management of the individual components within the IT infrastructure. The multiple inputs to these three capacity management subprocesses include: & & & & & & & SLAs and SLA breaches Business plans and strategies Operational schedules as well as schedule changes Application development issues Technology constraints and acquisitions Incidents and problems Budgets and financial plans As a result of these multiple inputs, the capacity management process—often under a single designated capacity manager—will manage IT processes, develop and maintain a formal capacity plan, and ensure certain capacity records are up to date. In addition, the capacity manager must be involved in evaluating all changes to establish their effect on capacity and performance. This capacity evaluation should happen both when changes are proposed and after they are implemented. Capacity management must pay particular attention to the cumulative effect of changes over a period of time that may cause degraded response times, file storage problems, and excess demand for processing capacity. Other capacity management process responsibilities include some duties of the network manager and the application and system manager. They are responsible for translating the business requirements into the required capacity to meet these requirements and to optimize IT performance. Smaller enterprises and many IT facilities, of course, may not be able to justify a fullor even a part-time capacity manager. However, some resource within the IT organization should have responsibility for capacity management issues. It is an important service design issue. The implementation of an effective capacity management process offers IT the benefits of an actual overview of the current capacity in place and the ability to plan capacity in advance. Effective capacity management should be able to estimate the impact of new applications or modifications as well as provide cost savings that are in tune with enterprise operations requirements. Proper capacity planning can significantly reduce the overall cost of ownership of an IT system. Although formal capacity planning takes time, internal and external staff resources, and software and hardware tools, the potential losses incurred without capacity planning can be significant. Lost productivity of end users in critical business functions, overpaying for network equipment or services, and the costs of upgrading systems already in production can more than justify the cost of capacity planning. This is an important ITIL process, and IT auditors should consider the capacity management processes in place when reviewing IT infrastructure general controls. E1C07 09/16/2010 13:31:15 Page 187 ITIL Service Design & 187 Service Delivery Availability Management Enterprises today are increasingly dependent on their IT services being available 24 hours a day and 7 days per week (24  7). In many cases, when those IT services are unavailable, the business stops as well. It is therefore vital that an IT function manage and control the availability of its services. This can be accomplished by defining the requirements from the business regarding the availability of the IT services and then matching them with the possibilities of the IT enterprise. Availability management depends on multiple inputs, including requirements regarding the availability of the business; information on reliability, maintainability, recoverability, and serviceability; as well as information from the other processes, incidents, problems, and achieved service levels. The objectives of the availability management process are to: & & & & & & Produce and maintain an appropriate and up-to-date availability plan that reflects the current and future needs of the enterprise. Provide service and guidance to all other areas of the enterprise on IT availabilityrelated issues. Ensure that service availability achievements meet or exceed targets, by managing service and resource-related availability performance. Assist with the diagnosis and resolution of availability-related incidents and problems. Assess the impact of all changes on the availability plan and the performance and capacity and capacity of all services and resources. Ensure that proactive measures are implemented wherever those actions are cost justifiable. Availability management activities can be described as planning, improving, and measuring actions. Planning involves determining the availability requirements to find out if and how IT can meet them. The service-level management process, discussed previously, maintains contact with the business and will be able to provide appropriate expectations to availability management. Businesses may have unrealistic expectations regarding IT systems availability when they do not understand what systems availability means in real terms. For example, business users may want 99.9% availability yet not realize that this will cost five times more than providing only 98% availability. It is the responsibility of service-level management and the availability management process to manage such expectations. Exhibit 7.5 shows this availability and costs relationship. It does not cost very much to keep basic IT systems running, but that is all the enterprise will receive. Both management and IT auditors should keep this relationship in mind when reviewing controls and making recommendations. An IT function can design for either ‘‘availability’’ or ‘‘recovery.’’ When the business cannot afford a particular service downtime for any length of time, IT will need to build resilience into the infrastructure and ensure that preventive maintenance can be performed to keep services in operation. In many cases, building ‘‘extra availability’’ into the infrastructure is an expensive task that can be justified by business 09/16/2010 13:31:15 188 & Page 188 Infrastructure Controls and ITIL Service Management Best Practices Special solutions with redundancies Highavailability system design Costs E1C07 Effective service management Systems management Base products, technology, and components Availability EXHIBIT 7.5 IT Availability and Cost Relationships needs. Designing for availability is a proactive approach to avoiding downtime in IT services. When the business can tolerate some downtime of services or when a cost justification cannot be made for building in additional resilience into the infrastructure, designing for recovery is the appropriate approach. Here, the infrastructure will be designed such that in the event of a service failure, recovery will be ‘‘as fast as possible.’’ Designing for recovery is a more reactive management approach for availability. In any event, other processes, such as incident management, need to be in place to recover as soon as possible in case of a service interruption. The main benefit of availability management is to have a structured process in place to deliver IT services that meet the agreed requirements of the customers. This should result in a higher availability of the IT services and increased customer satisfaction. Availability management covers an area where IT auditors often can ask some hard questions as part of their IT general controls reviews. Service Delivery Continuity Management As businesses are becoming ever more dependent on IT, the impact of any unavailability of IT services has increased drastically. Every time service availability or performance is E1C07 09/16/2010 13:31:15 Page 189 ITIL Service Transition Management Processes & 189 reduced, IT customers cannot continue with their normal work. This trend toward a high dependency on IT support and services will continue and increasingly influence direct customers, managers, and decision makers. ITIL continuity management emphasizes that the impact of a total or even partial loss of the IT services should be estimated and continuity plans established to ensure that the business, and its supporting IT infrastructure, will always be able to continue. ITIL calls for an appropriate strategy to be developed that contains an optimal balance of risk reduction and recovery options. ITIL also calls for some of the same business continuity and disaster recovery strategies as are discussed in Chapters 23 through 25 on IT disaster recovery and continuity management. Using the approaches outlined there, an IT organization can implement an effective set of service continuity processes. IT auditors should refer to those chapters to better understand and evaluate continuity and disaster recovery planning processes. Service Delivery Information Systems Security Management IT security management is another set of best practices, included in this book as part of Chapters 26 to 28. ITIL recognizes the need for information systems security within the corporate governance framework to provide a strategic direction for security activities and to ensure these activities are achieved. ITIL emphasizes that security is more than just an IT issue; it should be a management issue. The objectives of IT security are to protect the interests of those relying on IT information and the systems and communications that deliver it with the following ITIL information security objectives: & & & & Availability objective. Information is available and usable when required, and the systems that provide it can appropriately resist attacks and recover from or prevent failures. Confidentiality objective. Information is observed or disclosed to only those who have a right to know. Integrity objective. Information is complete, accurate, and protected against unauthorized modification. Authenticity and nonrepudiation objective. Business transactions as well as information exchanges between enterprises, or with partners, can be trusted. ITIL information security management goes on to outline best practices for a complete information security management system. A very important best practice, information security management processes, are discussed in Chapter 19. An IT auditor should have the ability both to recognize the key elements of an effective information management system and to make recommendations to improve and enhance existing systems when appropriate. ITIL SERVICE TRANSITION MANAGEMENT PROCESSES As IT auditors recognize, IT operations almost always have been subject to regular hardware or software changes, These changes may involve proper transition planning E1C07 09/16/2010 13:31:15 190 & Page 190 Infrastructure Controls and ITIL Service Management Best Practices to introduce the new components, testing and validation before any release to production, and configuration management to control the inventory and relationships of the IT equipment and services. ITIL has grouped these best practices into what it calls transition management. This area can present some significant internal control risks for IT auditors reviewing IT infrastructure operations. An IT application, for example, may be installed with solid internal controls. However, subsequent unauthorized changes to the same application or improperly configured attached equipment may introduce new control concerns. Service Transition Change Management The problem management process, discussed as part of service operations, often results in the need for IT changes ranging from program changes or process revisions to improve service or reduce costs. The goal of ITIL change management is to utilize standardized methods and procedures for the efficient and prompt handling of all changes, in order to minimize their impact on service quality and the day-to-day operations. ITIL change management processes include: & & & & IT hardware and system software Communications equipment and software All applications software All documentation and procedures associated with the running, support, and maintenance of live systems The last point is of particular concern to IT auditors. All too often, IT hardware and software is changed with little concern given to changing the supporting documentation and related software. Changes to any IT components (e.g., applications software, documentation, or procedures) should be subject to a formal change management process. IT auditors often encounter environments where the change management process is haphazard at best. Examples here are changes to applications without thinking through their implications on the overall IT infrastructure, incident management fixes that create other changes, or senior management requests for changes to solve shortterm or immediate problems. A formal change management process that reviews and approves any proposed changes will almost always improve IT and enterprise internal control processes. The ITIL change management process should be tightly linked to configuration management, discussed earlier, to ensure that information regarding the possible implications of a proposed change are made available and that any possible impacts are detected and presented appropriately. Change management processes should have high visibility and open channels of communication in order to promote smooth transitions when changes take place. To improve this process, many IT functions have instituted a formal Change Advisory Board (CAB), made up of people from IT and other IT customer functions within the enterprise, to review and approve changes. A CAB is a body that exists to approve changes and to assist in the assessment and prioritization of changes. It should be given E1C07 09/16/2010 13:31:15 Page 191 ITIL Service Transition Management Processes & 191 the responsibility for ensuring that all changes are adequately assessed from both a business and a technical perspective. To achieve this mix, the CAB should consist of a team with a clear understanding of customer business needs as well as technical development and support functions. Chaired by a responsible change manager, the CAB should be composed of IT customers, applications developers, various experts/technical consultants as appropriate, and representatives of any contractor or third party if in an outsourcing situation. Although a CAB should meet regularly to review and schedule proposed changes, it should not act as an impediment to IT operations. Its goal is to provide an orderly scheduling and introduction of all types of IT infrastructure changes. Efficient overall service management processes require a capability to change things in an orderly way, without making errors and wrong decisions. An effective change management process is indispensable for an effective IT infrastructure. When reviewing IT internal controls, internal auditors should look for an effective change management process that provides: & & & & & & & & Better alignment of IT services to business requirements Increased visibility and communication of changes to both business and service support staff Improved risk assessments Reduced adverse impact of changes on the quality of services Better assessments of the cost of proposed changes before they are incurred Fewer changes that have to be backed out, along with an increased ability to do this more easily when necessary Increased productivity of IT customers through fewer disruptive delays and higherquality services Greater ability of IT to absorb a large volume of changes. An effective ITIL change management process is an important component of IT infrastructure controls. The process also must align tightly with other key processes in the IT infrastructure: configuration, capacity, and release management. Service Transition Configuration Management Whatever their relative size, IT operations functions are complex with multiple types and versions of hardware and software components and linkages to cloud computing components (discussed in Chapter 9) that must work together in an orderly, well-managed manner. This is true for both major corporations with classic mainframe systems, farms of servers, and a multitude of storage devices and communications gear and small IT systems operation. A formal configuration management function is an important service delivery process that supports the identification, recording, and reporting of IT components, their versions, constituent components, and relationships. Items that should be under the control of configuration management include hardware, software, and associated documentation. Configuration management is not the same concept as the depreciation accounting process for asset management, although the two are related. Asset management systems maintain details on IT gear above a certain value, their E1C07 09/16/2010 13:31:15 192 & Page 192 Infrastructure Controls and ITIL Service Management Best Practices business unit and location. Configuration management also maintains relationships between assets, which an asset management process usually does not. Some enterprises start with an asset management and then move on to configuration management. A basic activity of the configuration management process is to identify various individual components in IT operations, called configuration items (CIs), and then to identify key supporting data for these CIs, including their ‘‘owners,’’ identifying data, version numbers, and systems interrelationships. This data should be captured, organized, and recorded in what is often known as a configuration management database (CMDB). The team responsible for configuration management should select and identify these configuration structures for the entire infrastructure’s CIs, including establishing relationships between each CI and connected components in the overall IT infrastructure configuration. Going beyond just a configuration status entry on the CMDB, the process should ensure that only authorized CIs have been accepted and that no CI is added, modified, replaced, or removed without an appropriate change request and an updated specification. An IT auditor can think of the importance of the configuration management process in terms of desktop applications in the audit department. Every internal auditor today probably has a laptop computer, but unless each has consistent versions of software, these systems may have difficulty communicating with one another. This is where configuration management is important. Configuration management is really important when attempting to understand the various versions or types of software and equipment in a large IT operation. The configuration management process also includes some control elements. A series of reviews and audits should be implemented to verify the physical existence of CIs and to check that they are correctly recorded in the configuration management system. Although we have used the word audit here, this is not an IT audit process but an ITIL task of the IT team responsible for the configuration management process. Configuration management should also maintain records for CI status accounting to track the status of CIs as they change from one state to another—for instance, from under development, to being tested, going live, and then being withdrawn. The CMDB does not have to be a complex, specialized application. An enterprise can establish a very basic level of CMDB by just using spreadsheets, local databases, or even paper-based systems. In today’s large and complex IT infrastructures, however, configuration management requires the use of physical and electronic libraries along with a CMDB to hold definitive copies of all software and supporting documentation. The CMDB should be based on database technology that provides flexible and powerful interrogation facilities. It should hold the relationships between all system components, including ITIL-defined incidents, problems, known errors, changes, and releases. The existence of an effective CMDB can be a good point for IT audit to understand an enterprise’s IT configuration management process and its supporting controls. If the enterprise does not have a good CMDB, IT audit can anticipate finding significant internal control problems throughout the IT infrastructure. Exhibit 7.6 outlines audit procedures for reviewing an enterprise’s configuration management process. The configuration management process interfaces directly with systems development, testing, and change and release management processes to incorporate new and E1C07 09/16/2010 13:31:15 Page 193 ITIL Service Transition Management Processes & 193 1. Review and understand existing enterprise configuration management practices as well as their interfaces to the service management processes, procurement, and development. 2. Assess the knowledge and capability of existing IT functions and its staff in terms of controls and processes for configuration, change, and release management processes. 3. Review the extent and complexity of existing configuration data, whether held in hard-copy form, in local spreadsheets, or in configuration management databases (CMDB), and develop an understanding of that database and its retrieval tools. 4. Select a production application and understand its definition on the CMDB in detail including interfaces to change management, release management, other service management processes, procurement, and development. 5. Using the installed CMDB reporting tool, define the inventory of configuration items for a selected application or system and physically trace reported CIs to actual configuration components. 6. Determine that processes are in place to link configuration management business processes and procedures with the CMDB tools. 7. Test the CMDB and other support tool(s) to determine whether key components, software, and documentation have been implemented and controlled on the CMDB. 8. Review adequacy of facilities to provide secure storage areas to manage CIs (e.g., cabinets, controlled libraries, and directories). 9. Assess adequacy of processes to communicate and train staff in the importance and use of configuration management. 10. Review problem management processes to determine the extent and appropriateness of their use of the CMDB for resolving problems. 11. Determine that appropriate access and update controls are in place to prevent unauthorized or inappropriate use of the CMDB. 12. Determine that the CMDB receives adequate backups and that it is part of the continuity plan key resources backup and recovery procedures. EXHIBIT 7.6 ITIL Configuration Management Internal Audit Steps updated product deliverables. Control should be passed from the project or supplier to the service provider at the scheduled time with accurate configuration records. In addition, the CMDB can be used by the service-level management process to hold details of services and to relate them to the underlying IT components. The CMDB also can be used to store inventory details of CIs, such as the supplier, cost, purchase date, and renewal date for a license. An additional bonus is the use of the CMDB to cover the legal aspects associated with the maintenance of licenses and contracts. Service Transition Release Management IT functions need effective processes to ensure that changes are introduced to all impacted parities in an orderly and well-controlled manner. Release management covers the transition of authorized changes to an IT service. A release typically consists of a number of problem fixes and enhancements to the service including new or changed software and hardware needed to implement the required approved changes. Releases normally are implemented as full releases, where all of the components being changed are built, tested, distributed, and implemented together. This eliminates the danger that obsolete versions of CIs are incorrectly assumed to be unchanged and used within the release. With a full release, all elements supporting some application area or system are released as a single systems component. With all new and existing E1C07 09/16/2010 13:31:15 194 & Page 194 Infrastructure Controls and ITIL Service Management Best Practices components bundled together, any problems are more likely to be detected and rectified before entry into the live environment. The disadvantage is that the amount of time, effort, and computing resources needed to build, test, distribute, and implement the full release will increase. An alternative and common approach to release management is the use of a delta or partial release approach that includes only those CIs that changed since the last full or delta release. A delta release may be more appropriate when a full release cannot be justified due to such factors as the urgency for needed facilities or the size and related resource requirements of a delta release in comparison with a full release. There is no single correct choice, and a decision to make a delta release should be made case by case based on the CAB’s recommendation. IT auditors should understand the importance of well-ordered release processes and should look for such established processes as they perform IT general controls reviews. The previous sections have outlined ITIL service support processes at a very high level. When reviewing IT general controls, an IT auditor should think of the importance of processes such as configuration management. An IT auditor does not need to be an expert in these ITIL service support areas but should keep them in mind when reviewing IT general controls. An auditor should become familiar enough with these processes to understand controls and procedures supporting IT service support. ITIL SERVICE OPERATION PROCESSES The ITIL best practices described in this chapter began with the importance of setting service strategies, including basic IT policies and such higher-level areas as IT financial management. In the natural process of launching IT resources, the next set of best practices covered service design processes that handle IT capacity and availability management as well as service-level management to help users of IT services agree with users of IT services and the services that will be provided The last of these three linked ITIL processes is known as service operations, a process that allows the business customer to see the quality of the IT services offered. Service operations cover the day-to-day service value to the overall enterprise that should be provided by IT systems and processes. The purpose of the ITIL service operation process is to help coordinate and deliver IT services to customers. This value concept often was missing in the earlier days of IT operations, but ITIL best practices bring the concept to the forefront of business and management needs. Service Operation Event and Incident Management ITIL service operations define separate service operation processes for event and incident management, where events are defined as any detectable occurrence that has significance for the management of the IT infrastructure or delivery of related IT services. ITIL defines many similarities between ITIL events and incidents, but our discussion here focuses on just incident management. Incident management processes cover the activities necessary for restoring an IT service following a disruption. ITIL defines a disruption as any type of problem that E1C07 09/16/2010 13:31:15 Page 195 ITIL Service Operation Processes & 195 prevents an IT user from receiving expected adequate services, whether it is an overall system failure, the user’s inability to access an application for any of a wide variety of reasons, a password failure due to a ‘‘fat fingers’’ typing error, or any other problem. The reported problem is called an incident, which means some type of deviation from standard operations. Although many IT operations have a help desk or a customer support group, this general function is referred to here as the service desk. The service desk is usually the owner of the incident management process, although all service support groups across IT may have a role. The objective of effective incident management processes is to restore normal operations as quickly as possible in a cost-effective manner with minimal impact on either the overall business or the user. How quickly is quickly should not be subject to interpretation, and ITIL calls for restoration time frame standards to be defined in service-level agreements. As discussed, effective SLAs are an important component of the IT infrastructure, and IT auditors should be aware of their existence. The first component of the ITIL incident management process is the detection and documentation of the incident by the service desk, as a single point of contact. These incidents can include such matters as a user calling in some specific application problem or IT operations informing the service desk of an application processing problem. Once an incident is received, the service desk should classify it in terms of its priority, impact, and urgency. The definition of a reported incident’s priority is one of the more important aspects of managing IT incidents. Every person who calls in an incident generally thinks that it is the most important. The incident management function has the difficult task of defining the relative priority of the reported incident, its importance, and its impact on the business. Exhibit 7.7 shows the typical life cycle of an incident from the initial call through resolution and closure. The point here is to help IT auditors understand service desk recommended best practices, in order for an IT auditor to ask some probing questions when reviewing IT general controls. For example, IT auditors should look for formal SLAs, as part of the service-level management process, to define the priority with which incidents need to be resolved, the effort put into their resolution, and the recovery from incidents. These SLAs should depend on: & & & The impact or criticality of the incident on the reporting entity or overall enterprise. Incident management should assess, for example, how many users will suffer as a result of a reported technical failure of a hardware component. Similarly, a call regarding a problem with the month-end accounting close process should be assigned a higher level of criticality than a problem with an application that generates purchase orders. The urgency of the reported incident. Urgency refers to the speed necessary to solve an incident of a certain impact. A high-impact incident does not, by default, always have to be solved immediately. An incident call reporting some user group cannot work at all because of some service outage often should be of greater urgency than a senior manager calling to request a functionality change. The size, scope, and complexity of the incident. The incident management team should investigate the reported incident as soon as possible to determine its extent. A reported failure of some component may just mean that a device is out of service 09/16/2010 13:31:15 196 & Page 196 Infrastructure Controls and ITIL Service Management Best Practices Incident detection and recording Ownership, monitoring, tracking, and communication E1C07 Classification and initial support Service request Service request procedure Investigation and diagnosis Resolution and recovery Incident closure EXHIBIT 7.7 ITIL Incident Management Life Cycle or might indicate a server is down. Those types of incidents are often not that complex and can be repaired relatively easily. A telecommunications failure that might impact multiple international units and thus might delay the monthly financial close can be much larger in size and scope. Once an incident has been logged in, the process of investigation and diagnosis should begin. If the service desk cannot solve the incident, it should be assigned to other IT support levels for resolution. However, all parties that work on the incident should keep records of their actions by updating a common incident log file. Some incidents can be resolved through a quick fix by the service desk, others by a more formal problem resolution, or in the case of more significant problems, by a workaround to get things back in partial operation coupled with a formal request for change (RFC) to systems, to a vendor, or to whatever parties are needed to correct such a more significant problem. In any event, efforts should be marshaled to correct the problem with the incident management function retaining ownership of the matter until resolution. Solid documentation should be maintained to track the incident until its resolution. The incident can be formally closed once matters have been fixed. If it is not easily solved, the incident should be passed to the problem management process function, as discussed in the next section. E1C07 09/16/2010 13:31:15 Page 197 ITIL Service Operation Processes & 197 All ITIL processes are somewhat related to one another, but in many circumstances, incident management represents the first line between users or customers of IT services and IT itself. Properly organized, incident management should be much more than just the help desks of an earlier time, when users called in with problems but frequently did not get much help beyond password resets. Incident management is a first point of contact between the customers—users—and the overall IT function. Incidents due to failures or errors within the IT infrastructure result in actual or potential variations from the planned operation of services. Sometimes the cause of these incidents may be apparent and can be addressed or fixed without further investigation. In other situations, there may be a need for a hardware or software repair, a matter that often takes some time to implement. Short-run solutions may be a work-around, a quick fix to get back in operation, or a formal RFC to the change management process to remove the error. Examples of short-term work-arounds might be instructing a customer to reboot a personal computer or resetting a communications line, without directly addressing the underlying cause of the incident. Where the underlying cause of the incident cannot be identified, it is often appropriate to raise a problem record for the unknown error within the infrastructure. Normally a problem record is raised only if an investigation is warranted, and its actual and potential impact should be assessed. Successful processing of a problem record will result in the identification of the underlying error, and then the record can be converted into a known error once a work-around has been developed. Service Operation Problem Management When the incident management process encounters a deviation with an unknown cause or reason, that incident should be passed on to the ITIL problem management process for resolution. The objective here is to minimize the total impact of problems through a formal process of detection and repair as well as to take actions to prevent any recurrence. The problem management process is the next step in the criticality of some reported incident and should be considered in terms of three subprocesses: problem control, error control, and proactive problem management. ITIL defines a ‘‘problem’’ as an unknown underlying cause resulting from one or more incidents. A ‘‘known error’’ is a problem that has been diagnosed successfully and for which a work-around has been identified. The idea is not to create a second administrative function in an IT enterprise to take reported help desk incidents but to identify when and how some incidents reported to the help desk should be passed on another person or authority to better diagnose the matter and treat it as a problem. An effective problem management process can do much to improve overall IT customer service. In addition to solving any single incident that was bumped up to the problem management process, IT should try to establish processes for better problem and error control, including maintaining data to help identify trends and suggesting improved procedures for the proactive prevention of problems. Data should be maintained on solutions and/or any available work-arounds for any resolved and closed problem records. In many instances, problem management may encounter a situation where it is E1C07 09/16/2010 13:31:15 198 & Page 198 Infrastructure Controls and ITIL Service Management Best Practices necessary to go a step further and file a formal RFC, either through an IT development function or through a hardware or software vendor. The problem management process focuses on finding patterns among incidents, problems, and known errors. A detailed review of these patterns allows an analyst to solve the problem by considering the many possibilities and narrowing things down to a solution; such a review is often called ‘‘root cause’’ analysis. There are many good techniques for resolving and correcting problems, which often are caused by a combination of technical and nontechnical factors. An IT auditor reviewing problem management processes should look for documented formal procedures that will support problem analysis and resolution. ITIL problem management is a good area for IT auditors to diagnose IT service delivery problems in order to better understand the overall health of IT operations. Areas where an IT auditor may ask some questions here include: & & & & The number of RFCs raised and their impact on the availability and reliability of the overall IT services covered The amount of time worked on investigations and diagnoses for various types of problems by organization unit or vendor The number and impact of incidents occurring before a root problem is solved or a known error is confirmed The plans for resolution of open problems with regard to people and other resource requirements as well as related costs and budgeted amounts The ITIL service operation problem management process is an important area for IT auditors to consider and understand when assessing the overall health of IT infrastructure operations. An efficient incident management process is necessary to receive customer calls and take immediate corrective actions, but an effective problem management process goes a step further to analyze and solve the problem, initiating RFCs where necessary and otherwise improving IT customer satisfaction. SERVICE DELIVERY BEST PRACTICES The preceding paragraphs have outlined some of the ITIL service management life cycle processes that are critical for an IT audit understanding of infrastructure controls. ITIL service management outlines processes for launching, managing, and controlling all levels of IT services with an emphasis on establishing customer satisfaction. ITIL guidance goes beyond just internal controls and includes managing IT costs, establishing measurements and metrics, and other quality improvement measures. ITIL calls for any IT function to build a program of continual service improvements to review, analyze, and make recommendations for improvements in each area of ITIL service delivery components. As a series of related processes, continuous IT improvement involves IT audit in some of these activities: & Reviewing management information and trends to ensure that services are meeting agreed-on service levels E1C07 09/16/2010 13:31:15 Page 199 Auditing IT Infrastructure Management & & & & & 199 Reviewing management information and trends to ensure that the output of established service management processes achieves desired results Periodically conducting IT audits to verify employee and process compliance Reviewing existing IT deliverables for relevance Conducting periodic customer satisfaction surveys ITIL service management life cycle processes are increasingly being adopted by IT functions worldwide. The emphasis on service increases the importance of IT resources and supporting infrastructure to the overall enterprise and its stakeholders. IT auditors should increase their knowledge of ITIL processes and build these best practices into their ongoing internal controls reviews. AUDITING IT INFRASTRUCTURE MANAGEMENT The ITIL service management processes introduce an expanded and improved approach for looking at all aspects of the IT infrastructure. These processes are not independent and freestanding. Although each ITIL process can operate somewhat by itself, they all depend on the input and support from other related processes. An internal auditor reviewing controls over any of the ITIL processes must think of them in relation to other related ITIL processes. For example, Exhibit 7.8 shows the relationship of the ITIL change management process and how it is dependent on and supports other related processes. The ITIL service management life cycle is a series of interrelated best practice processes that support the management of the IT infrastructure and of the enterprises. IT applications are in the center of this puzzle and are a key area of internal controls concerns. Our previous discussions of problem, incident, and change management ITIL Change Management Change Management Release Management Assesses Impact Authorizes change Controls release of new version of software or hardware if required to implement change Capacity Management Configuration Management Configuration Management Assesses Impact on business and IT performance Identifies areas impacted Updates records EXHIBIT 7.8 Relationship of ITIL Change Management to Other ITIL Processes Source: Robert R. Moeller, Brink’s Modern Internal Auditing, 7th ed. (Hoboken, NJ: John Wiley & Sons, 2009). Copyright ß 2009, John Wiley & Sons. Adapted with permission of John Wiley & Sons. E1C07 09/16/2010 13:31:15 200 & Page 200 Infrastructure Controls and ITIL Service Management Best Practices processes, among others, tended to call for a very large IT function with multiple levels of staff and management resources. An IT auditor might ask if these ITIL best practices also apply to a smaller enterprise. Our answer here is an emphatic yes; ITIL applies to all sizes of IT functions. In order to be ITIL compliant, an enterprise does not need multiple levels of support staff. Rather, it needs to think of the various service support and service delivery processes from an ITIL best practices perspective. A smaller IT function may not need to establish separate incident management and problem management functions, for example, but must think of each as a separate process with unique controls procedures. Even in a very small IT function, each ITIL process area should be treated as separate areas for process improvement. IT auditors should take particular care when making recommendations regarding their IT infrastructures. The size and scope of the areas being audited and the scope of operations should always be considered. This author remembers the early days of IT controls, when many applications were developed in-house for production applications. Most audit guidance materials recommended that there be a separation of duties between people who operate the computer and those who program it. Otherwise, in those days of far simpler IT systems, there was a risk that an individual with a fraudulent intent might change an application program (e.g., write himself an unauthorized check) and then produce this personal check when operating the system. This was good control in the early days of IT but is not as relevant today. Today’s IT auditors should think about the adequacy and appropriateness of IT controls in terms of how they are built into individual applications as well as the infrastructure process controls discussed in this chapter. The IT infrastructure is an important internal control review area. All too often, IT and other internal auditors have concentrated their attention on the applications controls and the IT general controls of the past. In today’s world of complex processes supporting the IT infrastructure, the ITIL processes outlined in this chapter describe some excellent areas for IT audit attention. When reviewing internal controls for any IT enterprise, whether a major enterprise-wide IT operation or the smaller function found in many of today’s smaller enterprises, the effective IT auditor should concentrate on reviewing controls over key IT infrastructure processes. NOTE 1. ITIL publications are available from the U.K. agency called The Stationery Office (TSO) and can be found through www.tsoshop.co.uk. E1C08 09/16/2010 13:27:24 Page 201 8 CHAPTER EIGHT Systems Software and IT Operations General Controls V I R T U A L L Y E V E R Y C O M P U T E R S Y S T E M has some type of master program— often called an operating system (OS)—to perform such tasks as scheduling an application program to run or saving or storing the results of an application program and outputting the results to a printer or display device. No matter what the size of the computer operations, these OS master programs and related supporting software control a computer system’s operations. Chapter 6 discussed the importance of information technology (IT) general controls, the kinds of controls covering all aspects of IT operations, and Chapter 7 discussed IT infrastructure controls, processes to better manage and control IT operations. This chapter discusses the very important area of systems software and related IT operations general controls. Whether it is a Microsoft Windows operating system on a laptop computer or Linux controlling an office server system, the OS is a key component to any computer system operation. It and the supporting systems software are essential for supporting enterprise IT operations. IT auditors need to understand the importance of the OS as well as any operations control programs in the enterprises where they are performing general controls reviews. This chapter discusses OS concepts and surveys some of the OS types and the supporting systems software that are essential in today’s typical IT operation. IT auditors should have a very general understanding of the purposes and importance of these types of IT software and should look for effective general controls when performing reviews in this important general controls area. This chapter focuses on some of the more common types of operating systems found in today’s IT systems as well as the supporting software to create well-controlled IT systems operations. 201 E1C08 09/16/2010 13:27:24 202 & Page 202 Systems Software and IT Operations General Controls IT OPERATING SYSTEM FUNDAMENTALS An IT or computer OS is the master program that serves as the prime interface between all system hardware components and the users of that computer system— the people and sometimes other automated devices. An OS is responsible for the management and coordination of the many activities necessary to run some applications on a computer system. Many IT auditors first encounter an OS while using their laptop or desktop personal computers equipped with either Microsoft or Apple Macintosh OS programs. A version of Microsoft’s Windows OS is perhaps the most familiar to many. In its multiple versions, it can be both a major help and sometimes a source of frustration. The origins of today’s OS go back to the days of mainframe computers, when the manufacturers of these devices (e.g., Burroughs, Control Data, Honeywell, IBM, Univac, etc.) each developed complex OS programs for their systems. Each was fairly unique to that computer manufacturer and its individual machine models. As an example, Univac, which is considered the founder of large-scale computer systems, had its model 1100 and 494 series of mainframe computers in the 1970s. Each Univac product line had its own OS that did not communicate with either those of other Univac machines and certainly not with those of competitors. In those early days of mainframe computers, many computer models were unique, but many of these machine OS controllers had innovative features. As example, the Univac 1100 series of computers had facilities for real-time processing in the 1970s, a feature that IBM, then the dominant computer manufacturer, did not introduce until the 1980s. The concept of a mainframe OS really changed, however, when IBM introduced its System/360 series of computer and mainframe operating systems in the mid-1960s. The IBM System/360 was a family of mainframe computers available in widely differing capacities and price points and with a single OS, called OS/360, planned for every model. This concept of a single OS spanning an entire product line was crucial for the success of IBMs System/360. IBM’s current mainframe operating systems, later called MVS and now z/OS, are distant descendants of this original system; applications written for the OS/360 can still be run on modern IBM machines. Exhibit 8.1 illustrates how an OS interacts with both computer resources and its applications. In the center, the computer system primarily consists of its central processing unit (CPU) and random access memory (RAM) as well as the external storage devices. The CPU is the set of highly integrated circuits that define the logic of how the computer will operate. The OS gives instructions to the CPU and manages the other computer resources. The next ring out is the key software components, such as an Oracle database system. The applications are found at the outer ring. Users and IT auditors primarily deal with these outer ring applications. Any modern OS, no matter the size of the computer system hardware, is a very complex set of software that goes beyond the skills of even the most expert systems programmers. While IT auditors regularly perform internal controls reviews of applications and review general control reviews of many aspects of the supporting control programs such as database processors, but too often the OS is taken as a given and ignored from an internal controls perspective. IT auditors should have a general E1C08 09/16/2010 13:27:24 Page 203 IT Operating System Fundamentals & 203 APPLICATION SOFTWARE DATABASE MANAGEMENT SYSTEMS OPERATING SYSTEM (OS) CPU, RAM, AND DISKS EXHIBIT 8.1 Role of the Computer Operating System understanding of basic OS attributes and should consider reviewing general software and infrastructure controls surrounding an enterprise’s computer systems OS resources. Microcomputer or Personal Computer Operating Systems The first microcomputers or personal computers (PCs) did not have the capacity or need for the elaborate operating systems that had been developed for mainframes or even what were called minicomputers; minimal operating systems were developed, often loaded from the computer’s very limited read-only memory (ROM). An early disk-based operating system called Control Program for Microcomputers (CP/M) was found on many of the first early microcomputers. CP/M was popular on many of the first systems and was closely imitated by Microsoft’s disk operating system (MS-DOS), which became wildly popular as the OS chosen for the IBM PC. IBM’s version of that microcomputer OS was distributed by Microsoft in its early days and called IBM DOS. Microsoft began upgrading and expanding this OS through the stream of improved versions that are found on many laptop and desktop computers today, such as Windows XP or Windows 7. Even before the IBM PC, Apple Computer Inc. (now Apple Inc.) launched a very popular microcomputer called the Apple II. A very popular machine then, the Apple II had its own primitive OS. After the introduction of the IBM PC, Apple soon launched a computer called the LISA, which had a very innovative graphical user interface (GUI) OS. LISA led to the Apple Macintosh computer with its Mac OS. Today, virtually every microcomputer OS today is based on either a Microsoft version of Windows or an Apple Mac OS. Today’s Server Operating Systems: Unix and Linux The early mainframe computers in the 1970s were massive machines requiring complex OS supports and many environmental requirements. They were just too E1C08 09/16/2010 13:27:24 204 & Page 204 Systems Software and IT Operations General Controls complex for smaller businesses or university research facilities. As the computer industry grew during that decade, several manufacturers introduced smaller, less complex computers called minicomputers. Although no longer in existence today, the prominent computer manufacturers then were Digital Equipment Corporation (DEC) and Data General (DG). Each of these had lines of computer hardware started with their own OS facilities that were far less complex than the requirements of IBM’s mainframe OS. These minicomputers were developed before the Internet, as we know it, but there was a massive professional and academic interest at that time on finding better ways to improve the use computer resources. The Unix OS was an important product of this era. Unix (or UNIX) is an OS that originated at Bell Labs in 1969 as an interactive time-sharing system, and it has evolved as a nonvendor freeware product, with many extensions and new ideas provided in a variety of versions of Unix by different companies, universities, and individuals. Because it was not a proprietary OS owned by any computer company and because it is written in the then-standard C programming language, Unix became the first open OS that could be improved or enhanced by anyone. A wide range of Unix versions were developed and used in workstation products from Sun Microsystems, Hewlett-Packard (now HP), IBM, and other companies. The Unix environment and its client-server program model were important elements in the development of the Internet and the reshaping of computing as centered on networks rather than in individual computers. Unix has been designed to give software developers a single set of what are called application program interfaces (APIs) to be supported by every Unix system. An API is a written contract between systems and application developers defining what the two sets of developers are to receive and responsible for providing. Unix has continued to be an important component of many IT systems. An IT auditor usually will have no need to review the ‘‘innards’’ of any Unix implementation, but he or she should ask some general questions about the status of any Unix system as part of any general controls review. Exhibit 8.2 contains some Unix IT general controls review questions. An IT auditor should look for implementation of one of the standard versions of Unix. Use of a standard version will allow an enterprise to better adapt and implement new applications going forward. That is, even if an enterprise is experiencing no control problems with applications currently running under a nonstandard version of Unix, problems going forward may arise if the installed versions of Unix is not compatible. Although Unix had its origins as a free software facility where all interested users could upload versions and modify them for their own requirements, manufacturers gained the rights to Unix and established controls that limited the ability to change versions of the OS software. IT auditors generally think of a controlled software OS, such as Unix, as a good thing, but its many users in facilities, such as university research labs, were looking for a more open version of Unix that they could improve and tweak. As an alternative to Unix, in 1991, Linus Torvalds, a programmer based in the Helsinki, Finland, developed a new OS, called Linux. Although similar to Unix, Linux had many features that impressed computer science professionals. The Linux OS is predominantly known for its use in servers, and it can be installed on a wide variety of E1C08 09/16/2010 13:27:24 Page 205 IT Operating System Fundamentals & 205 dkf 1. Meet with IT personnel to develop a general understanding of type and version of the installed Unix operating system (OS). a. Is the OS commercial software, such as Solaris or IBM’s AIX? b. If the OS is freeware, such as from an academic source, determine that the version installed is current. c. Determine whether the installed OS is supported by appropriate documentation and backup procedures. 2. Obtain auditor-level permission to access and audit UNIX procedures. If not familiar with basic commands, request IT management to supply a tutorial. a. Obtain a login for auditor use. b. Create a special directory for auditor use, and change the permissions on this directory so that only the allowed IT auditor has access to this directory and the files within it. 3. Particularly if a smaller installation system, review the server’s physical security to ensure that a casual outside user cannot, at a glance, see the entire setup of the server room. Ensure that the monitor is well hidden from open viewers. a. Review controls to ensure that only authorized persons can access the server. b. Determine that the server’s CPU unit has adequate security to prevent unauthorized users from resetting it or even switching it off and that there are prevention mechanisms to protect against any booting from alternate media. c. Determine that only appropriate personnel are allowed physical access to the room through the use of locks or swipe cards. 4. Use what are called the Unix eeprom parameter to ensure that the UNIX security mode is set to ‘‘full’’ and that an adequate EEPROM password is chosen to ensure that even if a malicious attacker gains physical access to the system, he or she will be unable to reboot the system without knowing the EEPROM password. 5. Obtain the hardware inventory on the system, listing all hardware devices by issuing the UNIX command usr/platform/‘uname -i’/sbin/prtdiag -v. 6. Get a list of software installed on the system through the UNIX command pkginfo to gather the details of the installed software, and resolve any differences. 7. What procedure is followed when adding new users? Ensure that there exists a welldocumented procedure for adding new users. 8. Assess the adequacy of procedures for adding new users to the system, and review IT and HR records for the names and designation of these personnel. a. Determine that users have to fill in a form or sign their acceptance of an agreement before being given a username and a password and that users are required to sign their agreement and acceptance. b. Determine what password restrictions, consistent with enterprise security policies, are imposed on Unix server users. EXHIBIT 8.2 Unix General Controls Review Procedures computer hardware, ranging from embedded devices, mobile phones, and even some watches and supercomputers. The Linux OS, installed on both desktop and laptop computers, has become increasingly popular in recent years. The feature that makes Linux unique is that it is totally ‘‘freeware.’’ That is, interested users can download a version of Linux at no cost, but they must agree to post any changes that they make to the software for all to see and use. All accepted changes become part of an enhanced Linux system. As an example of its growing acceptance, several years ago, even IBM installed Linux on several of its server systems. E1C08 09/16/2010 13:27:24 206 & Page 206 Systems Software and IT Operations General Controls IT auditors can expect to encounter more and more installations of Linux when reviewing IT general controls. Many users in a Linux environment find that it has more features and flexibility than even popular Windows-based applications. When an IT auditor includes a Linux OS as part of a general controls review, he or she should ask questions similar to these: & & & & & & & Linux users. Who can perform what functions at an OS level, and do those rights seem appropriate for the person’s duties? Services. What services, server functions, protocols, modules, and other functional components are installed and/or running on the system, and are they needed and authorized? Because of the open nature of Linux, a system can be filled with many unknown software components. Networks and connectivity. What protocols and devices are allowed to reach the Linux system? What other hosts are allowed to reach this system, and do they all appear appropriate? File systems. Are only approved file systems in use, and is Linux device utilization monitored? Logging/Auditing. Are all required Linux events recorded and analyzed? Security configuration. Are appropriate user access restrictions installed and system enforced, such as to password life controls and other parameters? Applications. Are only necessary and approved applications installed and running on the Linux system? Thousands of commands can be developed through a Linux OS, and an IT auditor should meet with enterprise IT software specialists to gain a better understanding of the implemented system. An IT auditor should look for OS system configuration standards and build procedures, jointly defined by enterprise system administrators and information security groups. Although some IT auditor technical guidance may be needed, an effective audit process would then consist of a comparison of those standards to the current configuration of the Linux system. These detailed procedures are beyond the scope of this chapter, but an IT auditor can ask a systems administrator to demonstrate compliance. IT auditors will encounter more and more Unix and Linux installations as they review general controls in larger server systems. They should take the time and effort to become familiar with these very important OS tools. FEATURES OF A COMPUTER OPERATING SYSTEM An IT OS acts as an interface between its applications and the hardware. A key element of that hardware, particularly on a personal or microcomputer, is the circuitry to operate the computer’s functions. This is known as the central processing unit (CPU), the circuit chip that controls the machine. The CPU’s logic provides services for key processes, such as assigning memory and other resources, establishing task priorities for the process, loading program code into memory, and executing programs. E1C08 09/16/2010 13:27:24 Page 207 Features of a Computer Operating System & 207 These programs then interacts with the user and/or other devices and performs its intended function. Although IT auditors typically do not need to understand the detailed functions of any OS, whether Windows XP, Linux or others, every OS has the same general functions: & & & & Interrupts. A central element of any OS, interrupts are a mechanism for the OS to interact with and react to its environment. Interrupts provide a computer’s CPU with a way of automatically running specific code in response to events. Most CPUs and their OS support hardware interrupts that allow the programmer to specify code that can be run when that event takes place. When an interrupt is received, the computer’s hardware automatically suspends whatever program is currently running, saves its status, and runs computer code previously associated with the interrupt; this is analogous to placing a place marker in a book in response to a phone call. Interrupts may come from either the computer’s hardware or the running program. When a hardware device triggers an interrupt, the OS decides how to deal with the event and establishes protocols for doing so. This is similar to how a person usually responds to what may be a false smoke detector alarm in the home before just calling for help. The processing of hardware interrupts is a task that is usually delegated to software called device drivers, which may be part of the OS, part of another program, or both. A program may also trigger an interrupt to the OS. If a program wants to access hardware, for example, it may interrupt the OS, which causes control to process the request. Memory Management. An OS is responsible for managing all system memory which is currently in use by its programs. This ensures that a program does not interfere with memory already used by another program. Since programs timeshare, each program must have independent access to memory. If a program fails, it may cause memory used by other programs to be affected or overwritten. Malicious programs, or viruses, may purposefully alter another program’s memory or may affect the operation of the OS itself. It can take only one misbehaving program to crash a computer system. Various methods of OS memory protection exist, but all require the particular and unique features of each CPU. Virtual memory. Desktop and laptop computer CPUs generally do not have enough available random access memory (RAM) to run all of the programs that most users expect at once. Virtual memory is an operating system feature that enables a system to use RAM memory address space that is independent of other processes running in the same system, and to use a space that is larger than the actual amount of RAM present, temporarily relegating some contents from RAM to a disk, with little or no overhead. Virtual memory looks at RAM for areas that have not been used recently and copies them onto spaces in the hard disk to free up space in the RAM to load other applications. Exhibit 8.3 shows this virtual memory concept. Multitasking. The important OS concept of multitasking refers to the running of multiple independent programs on the same computer, making it seem as if the computer is performing multiple tasks at the same time. Since most computers generally can only do one thing at one time, OS time-sharing concepts are used. E1C08 09/16/2010 13:27:24 208 & Page 208 Systems Software and IT Operations General Controls CPU EXHIBIT 8.3 & & & Cache RAM Virtual Memory Disk Storage Virtual Memory Management Concepts Each program uses a share of the computer’s time to execute the program, using a piece of the OS called a scheduler, which determines how much time each program will spend executing and in which order execution control should be passed to programs. Control is passed to a process that allows programs access to the CPU and memory so that another program can use the CPU. Disk access and file systems. Access to data stored on disks is a central feature of all OSs. Computers store data on disks using files, which are structured in specific ways in order to allow for faster access, higher reliability and to make better use of the drive’s available space. An OS defines the specific way in which files are stored on a disk and enables them to have names and attributes. Operating systems generally support a single type of disk drive and only one kind of file system. These file systems have been limited in their capacity, speed, and the kinds of file names and directory structures they can use. These limitations often reflected limitations in the OSs they were designed for, making it very difficult for an OS to support more than one file system. However, Unix and Linux support a technology known as a virtual file system (VFS). Unix supports a wide array of storage devices, regardless of their design or file systems to be accessed through a common API. This makes it unnecessary for programs to have any knowledge about the device they are accessing. A VFS enables the OS to provide programs with unlimited access to a number of devices with an infinite variety of file systems installed on them through the use of specific device drivers and file system drivers. Device drivers. A device driver is a specific type of software that allows interaction with hardware devices. It is an interface for communicating through what is called a computer bus or communications subsystem, providing commands to and/ or receiving data from the device and on the other end, from the requisite interfaces to the OS and software applications. It is an OS-specific hardware-dependent computer program that enables an OS, applications software package, or computer program running under the OS to interact transparently with a hardware device. It usually provides the requisite interrupt handling necessary for any necessary asynchronous time-dependent hardware interfacing needs. An OS essentially dictates how every type of device should be controlled. The function of the device driver is to translate these OS-mandated function calls into device-specific calls. In theory, a new device that is controlled in this manner should function correctly if a suitable driver is available. Networking. An OS supports a variety of networking protocols, hardware, and applications for using them. This means that computers running dissimilar OSs can participate in a common network for sharing resources such as files, printers, and scanners using either wired or wireless connections. Networks can essentially allow a computer’s OS to access the resources of a remote computer to support the same E1C08 09/16/2010 13:27:24 Page 209 Other Systems Software Tools & & 209 functions as it could if those resources were connected directly to the local computer. These network connections include everything from simple communication, to using networked file systems or even sharing another computer’s graphics or sound hardware. Client-server networking involves a program on a computer somewhere that connects via a network to another computer, called a server. Servers offer various services to other network computers and users. These services usually are provided through ports or numbered access points beyond the server’s network address. Each port number usually is associated with a maximum of one running program, which is responsible for handling requests to that port. Security. Computer security depends on a number of technologies working properly, but the OS provides access to a number of resources that are available to software running on the system and to external devices, such as networks. An OS is capable of distinguishing between requests that are allowed to be processed as well as others that should not be processed. An OS may distinguish between ‘‘privileged’’ and ‘‘nonprivileged’’ software transactions, and systems commonly have a form of requester identity, such as a username. To establish identity, there may be a process of authentication. Often a defined username must be entered, and each username may have an associated password as well. Other methods of authentication, such as magnetic cards or biometric data, might be used instead. Security features, discussed in Chapter 19, are a very important component of an OS. OTHER SYSTEMS SOFTWARE TOOLS Exhibit 8.1 shows a conceptual view of a computer system, with the CPU, RAM, and disk storage devices in the center and surrounded by the OS. The next ring outside of the OS but before user applications includes systems software tools. The processes in this ring are such things as database management systems and a wide variety of software tools, including programs that are not part of the OS but also not applications. This category of software includes utility programs for such areas as utilities for file sharing, print services, e-mail, Web sites, and what are called file transfer protocols (FTPs). Many of these utility programs have evolved because the OS for some computer systems did not have some needed services or because outside providers developed superior solutions. As an example here that dates back to the very early days of computer systems when early IBM mainframes and their COBOL program compilers did not have a very good way to sort a program’s data. Other machine vendors had built efficient data sort routines into their COBOL compilers, but the major vendor at that time, IBM, had not. To help with this weakness, a group of New York University students developed a sort program in 1968 that eventually became SyncSort (www.syncsort.com), a wellrespected utility program that became almost a standard attachment for IBM mainframe applications. Although many years have now passed, a much-enhanced SyncSort is still used frequently. A large number of utility programs—by IBM and other vendors— became common in IBM mainframe systems. The IT auditor who encounters these E1C08 09/16/2010 13:27:24 210 & Page 210 Systems Software and IT Operations General Controls utility programs should ask questions about the reasons for their use and assess whether they seem appropriate. IT auditors encounter these and many other utility programs as help or service programs for their own laptop machines. Many are excellent, and often they can be downloaded at no charge or for a small license fee. In many cases for Windows- or Macbased utility software, the actual programs today reside on the Internet, and the download only provides communication links. Perhaps the best example of such free utility software can be found with Google, with its search engine, communications software, and more. Google is a very reputable company; the software is free but includes some advertising messages. A large body of utility programs, particularly for Windows or Mac systems as well for Unix and Linux servers, have been released by small or little-known vendors that may contain security risks or other control concerns. IT auditors should always raise questions and express concerns, when they encounter any such software. Exhibit 8.4 contains review guidelines for an auditor’s search for unauthorized software. Procedures will vary by the type of OS, but an IT auditor should focus on authorization rules for installing any software as well as security rules for the software package. Many variations of utility software may be installed on any system. In the next sections we focus on several basic software types as well as some of the risks of using unauthorized versions. The text presents only a small sample of the many software types available. In our era of freeware software, university labs or similar sources may offer many versions. Such products may be technically brilliant in design but may never advance in maintenance and upgrades. IT management and certainly IT auditors should be aware of potential risks associated with such utility software. 1. 2. 3. 4. 5. 6. Review and document the software server environment, gaining an understanding of the persons or authorities responsible for authorizing and approving utility software. Determine whether there is a comprehensive inventory of installed software in place, including the authorizing authority for the software and other supporting documentation. Determine that the enterprise maintains a roster of approved supported software. a. Review documentation and controls for that supported software. b. Determine that appropriate cost-benefit analyses or other justifications have been developed to document the supported software. Determine whether there is a process in place to regularly review software licenses and updates. a. On a test basis, select several installed software packages to ensure their licenses are up to date. b. Also on a test basis, trace several installed software packages back to their original purchase justification to determine whether supporting documentation and approval appear appropriate. Identify any installed unlicensed software, and make recommendations to either remove or uninstall the software. If any software has been installed as freeware, determine that it was justified and approved, that the IT organization has assessed and improved its internal controls, and that the software is still appropriate to enterprise operations. EXHIBIT 8.4 Unauthorized Utility Software Review Guidelines E1C08 09/16/2010 13:27:24 Page 211 Other Systems Software Tools & 211 File Transfer Protocol Software FTP is a common network utility software tool used to exchange and manipulate files over an Internet-based networks. FTP is based on client-server architecture, using separate control and data connections between the client and server applications. It is also often used as an application component to automatically transfer files for program internal functions and can be used with user-based password authentication or with anonymous user access. This software was first developed in the very early days of dial-up IT telecommunications, and it was originally an unsecure method of transferring files because FTP had no method for data encryption. Thus, under earlier network configurations, usernames, passwords, FTP commands, and transferred files could be captured by anyone on the same network using what is called a packet sniffer detection tool. FTP security has since been improved, and FTP processes have been built into most Web servers. There are multiple versions of this software available for transferring files. The IT auditor should attempt to assess where versions of FTP have been installed on a system, as part of a general controls review, and also should understand that this software can be used to improperly copy and capture critical data files. Virus Protection Software Software virus detection and other cybersecurity software tools are discussed in Chapter 20. This is a major category of utility software, no matter what computer platform or OS, because of the many threats facing all systems. There are some cybersecurity tools built into some OS versions, but an IT auditor should see effective implementation of this an important category of utility software as part of any general controls review. Multiple vendors offer virus and IT security protection software. Because of the complexity of detecting cybersecurity problems and correcting them, a software vendor needs a strong staff to stay ahead in this ever-changing area. Some very effective freeware software products are available as well as multiple commercial packages. An enterprise must select a product that appears to suit its needs. Most versions of this software are effective, but the software must be updated almost constantly to counter new virus risks that have been identified. We discuss these cybersecurity risks and threats in greater detail in Chapter 20. An IT audit should verify that the current version of such software covers all systems platforms, that the software is updated on a regular basis, and that there is follow-up to correct and repair any reported violations. Automated Security Self-Evaluation Tools A variety of tools are designed not for IT auditors but for enterprise IT management to assist in improving IT operations. One example is the Automated Security SelfEvaluation Tool (ASSET) from the U.S. National Institute of Standards and Technology (NIST). This freeware software gathers data, generates reports, and provides a centralized place for the collection of system access data, as do other self-evaluation E1C08 09/16/2010 13:27:25 212 & Page 212 Systems Software and IT Operations General Controls Network and Systems Management Software Tools Network Management Products  35 Application Management Tools 39 Software Performance Management Software 30 General Network Monitoring Software 57 Bandwidth Traffic Monitoring Tools 9 Web Usage Monitors 17 Asset Managemement/Resource Inventory Tools 21 Software License Management Tools 9 Software Compliance Tools 37 Software SQL Help Tools 33 Installation and Deployment Software Tools Software Distribution Products 14 Software Packaging Products 7 Drive Imaging Tools 5 Migration and Configuration Managers 15 Server Migration Aids 15 SoftwareGroup Policy Managers 22 Software Tools for Systems Administration Disk Degragmentation and Drive Monitoring Tools 9 Remote Troubleshooting Products 23 Network Automation and Patch Processing Tools 20 Program Scribing Tools 14 Patch Management Software Tools 19 Application Sharing Software Tools 3 Application Conflict Testing Tools 5 Software OS Security Products Hardware-Based Firewall Products 23 Software-Based Firewall Products 17 Security Auditing Software Tools 40 Intrusion Detection Systems 16 Intrusion Prevention Systems 24 Smart Card/Biometric Authentication Systems 13 Spam Content Filtering Tools 67 Antivirus Software Tools 21 Antispyware Tools 17 Storage and Backup Software Software Backup Systems 40 Storage Management Software Tools 29 Disaster Recovery Products/Services 46 Clustering and Fallover Software 13 Load-Balancing Software 17  Total number of Windows-related software products in each category, per Redmond magazine (November 2009). EXHIBIT 8.5 Utility Software Product Categories for Windows-Based Products E1C08 09/16/2010 13:27:25 Page 213 Other Systems Software Tools & 213 tools. Our objective here is not to discuss this software but to describe it as a type of freeware. We have presented several examples of the many types of utility programs that an IT auditor may find installed on a server system. Exhibit 8.5 is an extensive list of these general categories of software. For example, we list five different categories of installation and deployment software tools as well as vendors offering products in various categories for each. At the time of this publication, 15 different vendors offered migration and configuration management products under the category of installation and deployment software tools. Vendors will come and go over time, but there are a large number of software products out there. In addition, our list here only includes Microsoft Windows-based product and does not include other software for the Mac, Unix, Linux, and others. Our point and warning to IT auditors is that a typical IT production system may have many of these software tools installed. Often a tool was added to a system because of a special need at one time or due to the cajoling of an overzealous salesperson. When encountering these software products as part of any IT general controls review, an IT auditor should ask: & & & & What is the purpose and function of the utility software, and who in the IT department is responsible for the software product? Is the license for the software current? Is the software regularly used at present? Are there any internal control issues or concerns associated with the software product? There are a large number of software tools installed on any server system, and some may have served a purpose at one time but may no longer be effective. Others may have been offered by vendors that are no longer in business. An IT auditor should be aware of the types and versions of OS and OS support software installed on any system as part a general controls review. He or she can provide a genuine service to management by surveying the utility programs in place as part of a general controls review and making recommendations to update or streamline these program functions. E1C09 09/16/2010 13:28:17 Page 214 9 CHAPTER NINE Evolving Control Issues: Wireless Networks, Cloud Computing, and Virtualization A N I N F O R M A T I O N T E C H N O L O G Y ( I T ) auditor, working in the profession for any length of time, will understand that the IT technologies and their supporting processes are changing on a continuous basis. Some of these changes—such as the move to laptop computers from desktop devices tied to a network—do not have a significant internal controls impact for IT auditors. Others do. This chapter discusses three areas where IT is changing and where IT auditors may need to take somewhat different approaches in their internal controls reviews. Our first example is the growth of wireless networks, both public and private. IT auditors often easily access their e-mail messages or get current business news from their laptop devices and the Internet. These facilities provide a massive convenience to all systems users, but they do present some control and security threats if an enterprise’s network is not properly protected through firewalls and other security controls. An IT auditor may be happy to be able to access e-mail messages or change an airline flight reservation over a wireless network, but those connections should be secure. This chapter briefly looks at some wireless network security and internal control issues. An IT configuration called cloud computing has been evolving rapidly in recent years. Although the term sounds almost exotic to many, cloud computing has become a significant concept today with our growing dependency on Web-based applications for many business processes rather than more traditional applications downloaded to home office servers. The concept known as Web services, service-oriented architecture (SOA), or software as a service (SaaS) was first largely promoted by Microsoft several years ago but has now been embraced by many others. Today, this concept has been 214 E1C09 09/16/2010 13:28:17 Page 215 Understanding and Auditing IT Wireless Networks & 215 broadened and is known as cloud computing, a configuration where many different Internet applications supported by multiple vendors and operating on multiple servers operate together out of what looks like a large fuzzy Internet cloud. This chapter introduces some cloud computing concepts and discusses cloud-related security and controls concerns that may impact IT auditors in their assessments of IT general controls. On a somewhat different level, storage management virtualization is another evolving IT infrastructure with general controls implications. The term storage management virtualization refers to the connections between a computer central processor unit (CPU) and connected or supporting mass storage drives or other devices. In past years, IT managers and auditors thought of a computer system in terms of its configuration— the disk drives and other peripheral devices that were connected to a computer’s CPU. At one time, these were attached through an often complex network of hard-wired cables. However, starting around the year 2001, software tools were introduced to better manage these storage connections; this is called storage virtualization. They began as a tool for large server sites, but today virtualization concepts have been introduced on all levels of computer processors. File virtualization can introduce strong efficiencies to IT operations; however, there can also be some internal control risks if mass storage relationships are not properly managed and backed up. We will briefly introduce IT virtualization concepts important to IT auditors. This chapter discusses these IT technical areas because they may impact IT auditors in their reviews and assessments of IT general controls. These three ITrelated areas may not be familiar to many IT auditors and their managers today, but they will be control issues in many IT general controls audits going forward. Of course, the landscape is always changing, and IT auditors should always be aware of newer technology-driven internal control issues and should modify their reviews to accommodate and assess these and other new internal controls areas. IT auditors should always take a strong lead in their enterprises in understanding new technology-based processes and then translating them to reviews and assessments of internal controls. UNDERSTANDING AND AUDITING IT WIRELESS NETWORKS Computer system components, such as terminals or printers, traditionally have been connected to their CPUs through cables or transmission wires. In earlier days, when an employee moved from one office to another, it was often necessary for a maintenance crew to string the necessary cables to the new office location. Although they originally could not transmit electronic data, radio-based wireless communications tools began to have a significant impact on the world during World War II. Through the use of wireless networks, information could be sent overseas or behind enemy lines easily, efficiently, and more reliably. Since then, transmission standards have been developed and wireless networks have grown significantly. Using local transmission stations, wireless networks first became common for local emergency services, such as the police or fire departments, which utilized wireless networks to communicate important information quickly. E1C09 09/16/2010 13:28:17 216 & Page 216 Evolving Control Issues By the late 1980s, local area networks (LANs) became very common, and many computer system configurations became to be based on a network of wired computer systems. In the early 1990s, processes and standards were developed for wireless LANs, where each system was a point in a local network. Starting with these early 1990s standards, tools were developed to implement wireless LANs. These LANs originally required communications cards between a central computer and local terminals, but they expanded to broad networks with connections to remote transmission stations and the Internet. Wireless systems have some vulnerability risks since their data is carried as radio signals subject to snooping. Wireless LANs, however, have their own problems. Connecting network elements by radio waves instead of wires presents many challenges. From a reliability standpoint, it is difficult to predict how dependable wireless network radio coverage inside a building will be because building construction features, such as steel beams and heavily plastered walls, severely weaken radio waves. Even outside of structures, predicting coverage accurately and dependably is difficult, owing to radio transmission propagation issues. Much more troubling is the fact that wireless LANs, by their very nature, broadcast their data into space, where the data can be intercepted by anyone with the ability to listen in at the appropriate frequency. These features also facilitate internal use of wireless LANs and enable interlopers to enter such networks with the same privileges as authorized users unless appropriate security controls are in place. Not all networks, and certainly not all wired networks, are secure. However, when a traditional LAN operates over cables within a relatively secure physical perimeter, the level of security provided by the physical construction usually is sufficient. Adding wireless transmission capabilities adds security vulnerabilities and the need for additional systems controls, such as the need to authenticate every network user. An IT auditor should look these general control and security characteristics in all wireless applications: & & & & Confidentiality. An application should contain a level of protection against interception, or eavesdropping, to provide assurance that messages sent are readable only by the intended recipients. Authenticity. Protections against spoofing or impersonation controls should be in place to ensure that messages originate from the claimed entity. Integrity. Controls should provide protection from transmission errors and/or willful modification of messages to offer assurances that a message has not changed in transmission. Availability. Assurances should be in place that application data will be available when and where it is required, as a protection against denial of service or poor reliability. Wireless networks and wireless LANs are important areas that should be included in any IT auditor’s review of general controls. The next sections discuss some key internal control characteristics of wireless networks important for IT auditors as well as considerations for reviews of IT wireless systems general controls. E1C09 09/16/2010 13:28:17 Page 217 Understanding and Auditing IT Wireless Networks & 217 Key Components of an IT Wireless System A wireless IT system is almost a generic name, and there can be some confusion about the overall configuration of any such system. There are several basic ways to configure a wireless system. In an open system configuration, any entity that can pick up the wireless signal can potentially gain access. This is the type of wireless system, built around a wireless access point (WAP), that one encounters when accessing the Internet from a hotel room or coffee shop. For purposes of definition, a WAP is a device that allows communications connections to wireless networks using standards such as Wi-Fi, Bluetooth, or related standards. The WAP usually connects to a wired network and can relay data between the wireless devices (such as computers or printers) and wired devices on the network. Many wireless systems in offices or homes are limited in their proximity to a WAP. Within their borders, however, they may connect to a variety of devices. Exhibit 9.1 shows such a wireless configuration with the wireless system existing inside a defined border, controlled by series of routers on the border edge. Routers are discussed in the next section; a large number of laptop computers, terminals, and other devices may be EXHIBIT 9.1 Wireless Network Architecture E1C09 09/16/2010 13:28:18 218 & Page 218 Evolving Control Issues attached to each router. As the exhibit shows, the wireless network is connecting to the Internet through several service providers and to some other external systems outside of the network. In addition, there are connections to local LANs, a cell phone network, and personal handheld devices. Although at a very general and high level, this exhibit illustrates the type of wireless system found in many enterprises today. From an IT auditor’s perspective, two key controls in this wireless system configuration are the placement of its routers and firewalls. Wireless System Routers A key component of all networks, including wireless networks, a router is an electronic device used to connect two or more computers to the Internet by cable or wireless signals. A router allows several computers to communicate with each other and to the Internet at the same time. A wireless router performs the functions of a router but also acts as a wireless access point to allow access to the Internet or a computer network without the need for a cabled connection. It can function as a wired or a wireless LAN. Some routers also contain wireless antennae that allow connections from other wireless devices. Wireless Firewalls A firewall is a type of one-way software door where transactions and activity can exit but cannot enter. It is a system, implemented in hardware, software, or a combination of both, that is designed to prevent unauthorized access to or from a private network. Firewalls frequently are used to prevent unauthorized Internet users from accessing private networks connected to the Internet, where all messages entering or leaving an internal network, the intranet, pass through an installed firewall. This important network control examines each message and blocks those that do not meet the specified security criteria. A firewall is considered a first line of defense in protecting private information. Wireless Network Vulnerabilities and Risks There are a variety of control risks associated with any wireless network, including the risk of eavesdropping into system activities, illicit entry into the network enabled by a failure of user authentication, and denial of services. A major systems integrity concern here is the risk of eavesdropping. By their nature, wireless LANs intentionally radiate their radio signal network traffic into space, and once the signals are emitted, it is impossible to control who can receive them. The key control here is to encrypt all such messages. Wireless message standards allow for such encryption, but such standards are not always installed. When reviewing wireless applications, IT auditors should determine that encryption standards have been installed and that they have been applied to all critical applications. The implementation of controls to ensure message integrity is also important for wireless systems. Network messages are transmitted in small packets of data that are then reassembled to deliver the correct message. Transmission software provides standards that should protect the integrity of all messages. Although the technical E1C09 09/16/2010 13:28:18 Page 219 Understanding and Auditing IT Wireless Networks & 219 details here may be beyond many readers, an IT auditor should meet with enterprise communications software specialists responsible for their enterprise’s wireless networks and discuss the implemented software default standards that emphasize message integrity and provide controls over illicit network entry. Our discussion here emphasizes the closed wireless networks that are more common for a business enterprise. Over time, the use of open wireless network connections, that are common in some public places to provide access to the Internet and other sites, will become more prevalent. Because these wireless systems are based on radio signal messages, we may see more perpetrators trying to get around security rules and attempting to gain improper accesses. Wireless Network Security Concerns Security is a major issue with many enterprise wireless networks in general and wireless LANs in particular. Anyone equipped with proper tools within the geographical network range of an open, unencrypted wireless network can ‘‘sniff,’’ or record, the network traffic, gain unauthorized access to internal network resources and to the Internet, and then possibly send spam messages or attempt other illegal actions using the wireless network’s Internet addresses. Although these threats reflect issues that have long troubled many types of wired networks (e.g., individuals can plug their laptop computers into available Ethernet communication jacks within a site and get access to a local wired network), this improper access activity usually does not pose a significant problem, since many enterprises had reasonably good physical security. However, radio signals bleed or move outside of buildings and drift across property lines, making network physical security largely irrelevant. Establishing effective wireless security procedures is a challenge to IT network administrators, enterprise management, and many IT auditors. There are strong and recognized standards to protect a wireless network, but many of those standards are defined in hardware routers—such as Cisco devices—that are wireless security options are set through controlling software with limited monitoring or controls. Good IT processes and technology often are easily confused, and particularly so with wireless information security management issues. However, many of the same business processes that establish strong risk management practices for physical assets and wired networks also work to protect wireless resources. IT auditors can use the following costeffective guidelines to enable enterprises to establish proper security protections as part of an overall wireless strategy. This list includes areas that represent good IT wireless internal control practices and objectives. An IT auditor can use these recommendations to better understand and evaluate enterprise wireless security processes. & & Wireless security policy and architecture design. Enterprise security policy, procedures, and best practices should include wireless networking as part of overall security management architecture to determine what is and is not allowed with wireless technology. Treat all wireless access points as untrusted. Access points need to be identified and evaluated on a regular basis to determine if they need to be quarantined as E1C09 09/16/2010 13:28:18 220 & & & & & Page 220 Evolving Control Issues untrusted devices before wireless clients can gain access to internal networks. This determination means the appropriate placement of firewalls, virtual private networks (VPNs), intrusion detection systems (IDSs), and authentication between access points or the Internet. Access point configuration policies. Enterprise administrators need to define their standard security settings before the wireless systems can be deployed. These settings include guidelines regarding IDs, wireless keys, and encryption. Access point discovery. Administrators should regularly search outward from a wired network to identify unknown access points. Such a search may identify rogue access points operating in the area—often a major concern in densely populated areas. Access point security assessments. An enterprise should perform regular security reviews and penetration assessments to identify poorly configured access points or defaults or easily guessed passwords. Wireless client protection. Wireless clients—typically user departments— should be examined regularly for good security practices. An enterprise IT function can establish good wireless procedures, but the actual users also should follow good practices. The overall objective of this chapter is to highlight some evolving internal control areas that impact IT auditors. IT wireless systems are nothing new, but they are becoming almost standard components in many system configurations today. Chapter 26 provides more guidance on auditing IT telecommunication processes and wireless systems. UNDERSTANDING CLOUD COMPUTING Cloud computing is a new and evolving concept that is important to many IT operations. Also closely related to the concepts of SaaS or SOA, cloud computing today is changing the way that many enterprises build and use IT applications. A cloud symbol is often used today to refer to the Internet in this book and in other published references. The idea behind this Internet cloud is that users do not need knowledge of, expertise in, or control over the technology infrastructure ‘‘in the cloud’’ that supports them. This term originated in the telephone industry where, up until the 1990s, data and even early Internet circuits were hard-wired between destinations, but later long-haul telephone companies began offering wireless VPN service for data communications. The growth of these wireless networks and the Internet’s World Wide Web concepts enhanced the way that we think of IT services. Cloud computing is more than just the Internet. It is the way we think of the services that Internet-resident applications provide. Because it is impossible to determine in advance precisely the paths of Internet traffic, the cloud symbol is used to describe that which were the responsibilities of the service providers as well as the network infrastructure. The concepts of software products or services on the Internet—SOA and SaaS—soon followed. In the early 2000s, Microsoft extended this concept of SaaS through the development of what it calls Web Services. IBM also later released what it called the Autonomic E1C09 09/16/2010 13:28:18 Page 221 Understanding Cloud Computing EXHIBIT 9.2 & 221 Cloud Computing Concepts Computing Manifesto, which described automation techniques such as self-monitoring, self-healing, self-configuring, and self-optimizing in the management of complex IT systems with heterogeneous storage, servers, applications, networks, security mechanisms, and other system elements that can be virtualized across an enterprise. Software vendors are increasingly offering their products as services on the Internet rather than as applications that are resident on individual in-house servers. A good example of this trend—and an early product type leader—is the provider of customer relationship management (CRM) software, SalesForce (www.salesforce.com/crm/products.jsp). This supplier of customer and sales tracking software tools does not sell its products as a set of programs loaded on proprietary CDs for customer use. Rather, all of the SalesForce programs and documentation is found on the Internet, and customers pay for the software only when they use the product. The SalesForce applications are used as a service to customers. Exhibit 9.2 shows this SaaS cloud computing concept. We have highlighted several vendors that currently offer SaaS products today, including Amazon, Google, Microsoft, and SalesForce. This is only a limited example of SaaS current applications, and certainly many more will follow. Some of the benefits of SaaS applications in a cloud computing environment are: & & Reduced infrastructure costs due to centralization. With the SaaS application in the cloud, there is no need to maintain application change management and other controls. Increased peak load capacities. Cloud providers, such as Amazon or Google, have massive server farms with massive capacities. Their load capacities are almost infinite. E1C09 09/16/2010 13:28:19 222 & & & & Page 222 Evolving Control Issues Efficiency improvements for systems that are often underutilized. Consistent performance that will be monitored by the service provider. Application and IT services resiliency. Cloud providers have mirrored solutions that can be utilized in a disaster scenario as well as for load-balancing traffic. Whether there is a natural disaster requiring a site in a different geographic area or just heavy traffic, cloud providers should have the resiliency and capacity to ensure sustainability during an unexpected event. IT auditors will need to take a different approach in reviewing internal controls for SaaS applications and understanding IT security in a cloud computing environment. Web and other infrastructure services today are increasingly being delivered in a cloud environment, and there is a need to rethink some audit and control considerations. Reviewing Cloud Computing Application Controls As any IT auditor should realize, even though an application is operating out of an SaaS environment, the need to assess and review its application internal controls does not go away. The SaaS-based application should continue to have the same audit trails, errorchecking procedures, and other good practices that are found with any well-controlled IT application. An IT auditor can almost expect that a business application run under a major vendor, such as Google, has adequate internal controls. Cloud computing represents a major change in the way applications are run and managed. Although only a limited number of vendors provide service-based software applications today, that number is expected to increase. Many vendors provide an implicit level of trust in the services they provide under the SaaS clouds, but an IT auditor should meet with their IT and business management to gain assurances that every SaaS application is well controlled. An IT auditor should attempt to help demonstrate, to direct and indirect cloud computing users, that they can have a strong level of trust in the software services and infrastructure that make up the cloud for an enterprise. Some of the key assurance issues that should be addressed are: & & & Transparency. Service providers must be able to demonstrate the existence of effective and robust security controls, assuring customers that their information is properly secured against unauthorized access, change, and destruction. Key questions for any service provider providing SaaS applications are: & What types of service provider employees have access to customer information? & Is a segregation of duties between provider employees maintained? & How are the files and data for different customers’ information segregated? & What controls are in place to prevent, detect, and react to any security and control breaches? Privacy. Cloud computing service providers should provide assurances that privacy controls are in place that will prevent, detect, and react to breaches in a timely manner, with strong and periodically tested lines of communication. Compliance. In order to comply with various laws, regulations, and standards, there can be cloud computing concerns that the data may not be stored in one place E1C09 09/16/2010 13:28:19 Page 223 Understanding Cloud Computing & & & 223 and may not be easily retrievable. It is critical to ensure that if authorities demand certain data, it can be provided without compromising other information. When using cloud services, there is no guarantee that an enterprise can get its information when needed or that a service provider may or may not claim a right to withhold information from authorities. Transborder information flow. With cloud-generated information potentially stored anywhere in the cloud, the physical location of the information can become an issue. This physical location dictates a jurisdiction and legal obligation. There are many legal issues here yet to be solved. Certification. Cloud computing service providers should assure their customers that they are doing the ‘‘right’’ things. In the future, independent third-party audits and/or service auditor reports will become a vital part of any cloud computing service provider assurance program. However, these facilities have not yet materialized. Strong and effective standards are needed to help enterprises gain assurances regarding their cloud computing supplier’s internal controls and security. At the time of this publication, there are no publicly available specific cloud computing standards. As no defined set of standards exists, an IT auditor reviewing applications provided by a cloud computing service provider should look for strong assurances in three key areas: 1. Events. The service provider should regularly document and communicate changes and other factors that have affected SaaS system availability. 2. Logs. A service provider should provide comprehensive information about an enterprise’s SaaS application and runtime environment. 3. Monitoring. Any such surveillance should not be intrusive and must be limited to what the cloud provider reasonably needs in order to run its facility. Cloud computing represents a new and interesting opportunity to rework security and IT controls for a better tomorrow, and internal auditing standards should soon follow. The Information Systems Audit and Control Association (ISACA) has already provided some preliminary cloud computing audit and control guidance, and we expect to see much more in the future as SaaS applications and cloud computing grows and matures. Cloud Computing Security and Privacy Challenges The use of SaaS applications operating in cloud computing environment shifts a wide range of challenges and responsibilities primarily from an enterprise’s IT function to an environment where some responsibilities are assumed by the cloud computing service provider while others remain the responsibility of the enterprise IT function. This is a challenge for IT auditors as well, who must understand the security and privacy components of their selected service providers. Cloud computing and the use of SaaS applications is a new trend and an increasing number of vendors are offering suites of SaaS applications. Vendors such as Google and Amazon are building huge, multiserver cloud computer complexes, but today no established set of recognized best practices exists across these various service providers. E1C09 09/16/2010 13:28:19 224 & Page 224 Evolving Control Issues In some respects, the trend of enterprises shifting some their in-house IT resources to cloud service providers has some elements similar to the move to IT service bureaus in the early 1980s. In the mid- to late 1970s, an increasing number of enterprises decided to move from their manual or unit-record punched-card processes to the new mainframe computer systems. Many had added systems development programming staffs and installed mainframe computer systems, but often with very disappointing results. A frequent problem was that these new systems did not have the capacity to process the volumes of enterprise data; when they did, often the systems experienced computer system maintenance or downtime problems. A solution for many at that time was to convert enterprise computer systems operations to what was called a service bureau—a large centralized computer systems resource that collected input materials for many clients, processed using common systems, and delivered the output reports. These service bureaus, however, did not work well for everyone. Many firms subscribed without fully realizing what they would be getting in terms of services and integrity and internal control monitoring for the enterprise processes. Most service bureau computer operations disappeared well before the demise of the mainframe. However, some of the same things are happening today with enterprises converting some conventional applications to a cloud environment. A major problem is that enterprises do not always ask the right questions from their service providers. When making a decision to select a service provider as part of a move to cloud computing, an enterprise should ask competing service providers some hard questions about their operations and standards. An IT auditor can take a lead in suggesting that its management should gain assurances in some of these areas when selecting a cloud computing service provider: & & & Privileged user access. Sensitive data processed outside an enterprise and in a cloud brings with it an inherent level of risk, because outsourced services in the cloud bypass conventional physical, logical, and personnel IT controls that an IT organization would exert over its in-house systems programs. Service providers should provide thorough information about the people who manage an enterprise’s data and systems in the cloud. They should supply specific information on the hiring and oversight of privileged administrators and the controls over their access. Regulatory compliance. An enterprise is ultimately responsible for the security and integrity of its own data, even when that data is held by a service provider. Cloud computing service providers should supply detailed information on their security governance policies and the results of recent external audits and security certifications. In addition, they should agree to update the enterprise on these activities on a regular basis. Data location. When an enterprise uses the cloud, it probably will not know exactly where data is hosted—not even the country. Because of data ownership laws, service providers should identify the specific jurisdictions where they will be storing and processing an enterprise’s data. Service providers also should make a contractual commitment to obey local privacy requirements on behalf of their customers. E1C09 09/16/2010 13:28:19 Page 225 Storage Management Virtualization & & & & & 225 Data segregation. Data in the cloud typically is stored in a shared environment alongside data from other customers. Cloud providers should provide detailed information on what is done to segregate enterprise data at rest or separate from other users and provide evidence that their security encryption schemes were designed and tested by experienced specialists. Encryption accidents can make data totally unusable and can complicate availability. Recovery. Even if an enterprise does not know the location of its cloud data, cloud providers should document what will happen to data and services in case of a disaster. They should provide evidence, including test results, that their recovery methods will replicate the data and application infrastructure across multiple sites. The services should assert whether they have the ability to do a complete restoration, and how long it will take. Investigative support. Investigating inappropriate or illegal activity may be impossible in cloud computing. Cloud services are especially difficult to investigate, because logging and data for multiple customers may be collocated and/or spread across an ever-changing set of hosts and data centers. Service providers should provide a contractual commitment to support specific forms of investigation, along with evidence that they have already successfully supported such activities. Long-term viability. An enterprise has no guarantee that cloud computing providers will never go broke or be acquired and swallowed up by a larger company. However, current providers such as Amazon, Google, IBM, and Microsoft will almost certainly continue to be around for a while. An enterprise should nevertheless gain assurances that its data will remain available even after such an event. All service providers should provide assurances that users will get their data back in a format that they can import into a replacement application. Cloud computing and SaaS applications are new and evolving areas as this book goes to press. We can expect to see established standards and recognized best practices in future years. Companies learned enough lessons years ago when using IT service bureaus to recognize how not to select a cloud service provider. Cloud computing and SaaS are the wave of the future, and IT auditors should see much more of them in the years going forward. STORAGE MANAGEMENT VIRTUALIZATION Virtualization is the concept of pooling IT physical storage resources from multiple network storage devices into what appears to be a single storage device that is managed from a central console. Storage virtualization helps an IT storage administrator perform the tasks of backup, archiving, and recovery more easily, and in less time, by disguising the actual complexity of the overall network of IT storage devices. This author was first introduced to storage management virtualization in 2002 when he was as part of a small consulting group for EMC Corporation helping to launch an Information Technology Infrastructure Library (ITIL) consulting practice. (ITIL best practices are discussed in Chapter 7.) At that time, EMC was a leader in storage management devices, E1C09 09/16/2010 13:28:19 226 & Page 226 Evolving Control Issues and its virtualization concepts were a major technology innovation. Virtualization has since become a widely used and important IT resource management process. To understand IT virtualization, one should visit the earlier days of computer systems—particularly the mainframes of old. Those computers had operating systems (OSs) that controlled various attached peripheral devices, including printers, tape drives, and mass storage devices. Although earlier mainframe computer systems initially made massive use of relatively inexpensive magnetic tape drives to store data, technology quickly moved to rotating magnetic drum and then to disk drive storage devices. Although they were much more expensive in the early days of IT, disk and drum drives quickly became popular. The limitation with tape drives was that to read the 100,000th record on a tape, the drive had to pass through the first 99,999 records to find that record. Tape drives and then drum drives quickly became almost historical anecdotes, and technology moved to rotating disk drives, which were much faster and had indexing schemes that located that 100,000th record almost instantaneously. As every IT auditor with a packed C drive on his or her laptop, full of records and other materials, can attest, there is strong need for mass storage space on all computers. Enterprises of all sizes need ways to manage and control their stored data. With enterprises creating so much information, IT operations have stored their data on multiple storage units or drives but also need to consolidate storage and get the most out of each storage unit. Schemes for managing all those devices soon become a headache. A solution has been storage virtualization, a technique to combine all storage drives into one centrally manageable resource. Virtualization in general is the separation of a device’s functions from its physical elements. With storage virtualization, a unit’s physical drive is separated from its functions to store data, and multiple physical disks will appear to be a single unit. Virtualization is a very efficient method to manage and control separate physical units using specialized virtualization software. With the proper software, virtualization techniques can be used with IT hardware devices beyond storage management, network components, servers, operating systems, or even applications. In such systems, the hardware application diagrams that IT auditors had to request as part of a review of general IT controls are no longer applicable. Virtualization software assumes these unitby-unit responsibilities. Virtualization concepts were, in some respects, the forerunner of cloud computing. We have only mentioned virtualization here as a new concept, with little supporting detail, that IT auditors will encounter in their general controls reviews. Although IT storage management virtualization was first introduced by just one company, many hardware and storage management vendors have picked up the concept. When an IT auditor encounters an environment in which virtualization is used heavily, he or she should meet with members of the IT staff to gain an understanding of the nature of the implementation, to ensure that it has been implemented in a manner that emphasizes IT internal controls, and that IT appears to be realizing benefits from the software tools or features. Similar to cloud computing, virtualization is a software product or concept offered by many major vendors. It is a growing and still evolving concept and IT auditors should increase their understanding of it as an evolving trend in IT operations. E1C10 09/16/2010 15:44:58 Page 227 III PART THREE Auditing and Testing IT Application Controls E1C10 09/16/2010 15:44:58 Page 228 E1C10 09/16/2010 15:44:58 Page 229 10 CHAPTER TEN Selecting, Testing, and Auditing IT Applications I N FO R M A T I O N TE C H N O L O GY ( I T ) A P P L I C A T I O N S are the tools that bring value to computer systems; they drive many if not most of today’s enterprise business processes. These IT applications range from the relatively simple, such as an accounts payable system to pay vendor invoices, to the highly complex, such as an enterprise resource management (ERM) set of multiple interrelated database applications to control virtually all enterprise business processes. Many IT applications today are based on vendor-leased or purchased software, an increasing number come from Web-based services, some are developed by in-house systems and programming teams, and many others are based on spreadsheet or database desktop processes. Although the IT general control procedures discussed in Chapters 6 and 7 cover controls and best practices over all IT operations, specific control processes apply to each installed IT application. In order to perform internal controls reviews in specific areas of enterprise operations, such as accounting, distribution, or engineering, IT auditors must have the skills to understand, evaluate, and test the controls over their supporting IT applications. Reviews of specific application controls often are more critical to achieving overall audit objectives than reviews of general IT controls. Application controls, however, are very dependent on the quality of overall IT general controls. For example, if there are inadequate controls over an IT configuration management process, as discussed in Chapter 7, it will be difficult for an IT auditor to rely on the controls built into a specific application that depends on strong configuration management processes. Even though an IT auditor, for example, may find that an IT order-entry application is properly screening sales orders for valid credit approvals, the surrounding general controls must also be considered. Without IT configuration 229 E1C10 09/16/2010 15:44:58 230 & Page 230 Selecting, Testing, and Auditing IT Applications management update controls, in this example, the order-entry system’s programs could be changed, without management’s authorization, perhaps to override established credit approval controls. A typical enterprise may use a large and diverse number of production IT applications. These applications support a wide variety of functions within the enterprise, starting with accounting applications but also including such areas as manufacturing, marketing, distribution, and others, depending on business activities. These supporting applications may be implemented using a variety of IT technologies, such as centralized systems with telecommunication networks, Internet-based network systems, client-server server-based applications, and even older mainframe batchprocessing systems. Some of these applications may have been developed in-house but increasingly large numbers of them are based on purchased software packages installed locally or accessed through Web-based service providers. In-house-developed applications may be written in a programming language such as C# (also called Csharp) or Visual Basic, a database report-generator language such as SQL, or the objectoriented language Java. Application documentation may range from very complete to almost nonexistent. Despite the best efforts of IT audit to suggest improvements, the same often can be said about application controls. Even though members of management sometimes do not have a good understanding of IT general controls issues, often they are interested in IT audit issues covering specific application controls. For example, while an IT audit report finding on general controls over IT operating systems program libraries may not generate much management interest, a finding of an incorrect discount calculation based on a foreign currency conversion problem in an accounts payable application is sure to draw attention. However, because of the relative complexity of many IT applications and because their controls often reside both within the application and in supporting user areas, audits of IT applications can be challenging. IT auditors should survey the active applications and select the more critical and appropriate ones for review. We also discuss approaches to effectively review internal accounting controls in IT applications, using several different types of applications as examples. Finally, the chapter discusses audit approaches for evaluating and testing those application controls as well as techniques for reviewing new applications under development. We focus on the internal control characteristics of different types of applications and on how to select appropriate applications in internal controls reviews. There are many differences from one application to another; this chapter focuses on how an IT auditor should select higher-risk applications as candidates for IT audit reviews, the tools and skills needed to understand and document application internal controls, and, finally, processes to test and evaluate those applications. IT APPLICATION CONTROL ELEMENTS People not familiar with IT sometimes think of a computer application just in terms of the system’s output reports or the data displayed on terminal screens. However, every application, whether a Web-based service application, an older mainframe system, a E1C10 09/16/2010 15:44:58 Page 231 IT Application Control Elements & 231 client-server application, or an office productivity package installed on a local desktop system, has three basic components: (1) the system inputs, (2) the programs used for processing, and (3) the system outputs. Each of these has an important role in an application’s internal control structure, and an IT auditor should understand these components when reviewing an IT application. Earlier IT applications could be separated easily into these three components. As an example, the traditional computerized payroll system from long ago used time cards and a personnel paymaster file as its inputs and a set of programs to calculate pay and benefits as well as to update pay history records. The outputs from that payroll system were the printed checks, payroll register reports, and updated paymaster files. Today, that same payroll system might accept inputs from an automated plant badge reader that controls accesses and tracks attendance, a shop-floor production system that performs incentive pay calculations, various other online inputs, and a human resources database. A series of computer programs, some located at a Web-based service provider and others distributed to remote workstations, would do the processing. In many cases today, much of the payroll processing may be handled by an outside service function that does most of these activities. The modern payroll system’s outputs include transactions to transmit compensation to employee bank accounts, pay vouchers mailed to employees, and input files to various tax and benefit sources, various display screens, and an updated human resources database. Although the input, output, and computer processing system components may not be all that clear to an IT auditor performing an initial review, the same three elements exist for all applications. No matter how complex the application may appear, an IT auditor should always develop an understanding of an application by breaking down its input, output, and processing components. The next sections briefly discuss the control aspects of these application components to give an overview on selecting, auditing and testing IT applications. Application Input Components Every IT application needs some form of input, whether it is data manually input from transaction vouchers or supplied from some automated system. Think of a common handheld calculator: The device will generate no results unless data of some sort is input through the key panel. Although an application’s programs process the data, determine the outputs, and have a major impact on controls, an IT auditor should understand the nature and sources of the input components. In traditional, batchoriented systems, this was a fairly easy process. Application inputs often were sequential records recorded on a magnetic tape file or 80- or 90-column punched cards. Today, inputs often are generated from various automated sources, including wireless data collection devices and specialized bar code readers. Inputs from Data Collection or Other Input Devices Most early IT applications used punched cards as their input source. A single card carried 80 or 90 columns of alphanumeric encoded data, and users entered input transactions onto data collection sheets for keypunching onto the card formats. The E1C10 09/16/2010 15:44:58 232 & Page 232 Selecting, Testing, and Auditing IT Applications original data collection sheet was the first step in the input chain, and early IT auditors were concerned that all transactions were keypunched correctly. These cards were then machine-sorted or otherwise manipulated prior to entry into a system, either read directly into a computer program or copied to magnetic tape for subsequent processing on a batch basis. That is, 500 lines of transactions may have been prepared on data collection sheets and processed as a batch. The need for all transactions to be keypunched correctly and subsequently read into the computer program made input transactions controls a key component of an application’s overall internal controls. Technology has effectively eliminated those punched-card input records today. Batch-type transactions that must be entered into an application are no longer entered by a specialized ‘‘keypunch’’ or data-entry department. Rather, operational departments use online terminals to enter their transactions for collection and subsequent processing. Following a processing schedule, these transactions may be input or collected and updated later in a batch mode. The data entry programs used to capture them often have some transaction-screening capabilities to eliminate any low-level errors common to earlier batch input systems. In many other situations, the entry of a transaction updates files in a real-time mode. Transaction input data comes from many sources. A retail store captures sales inputs through a combination of sales entries entered on the point-of-sale (POS) terminal and product sales are entered through bar code readers. Similarly, data is captured on a manufacturing shop floor through various tickets and badges that are entered in readers by workers directly on the floor. Small computer chips—radio-frequency IDs (RFIDs)—embedded on the label of a component may provide inputs as to the product’s identification and subsequent movement. All these input devices generate transactions for updating to some type of processing application. Input transactions are increasingly generated not from within the enterprise but from applications located in other physical locations and controlled by others. Enterprises today receive a wide variety of data transactions through the Internet, on older electronic data interchange (EDI) systems, or through wireless systems. In these cases, another enterprise may submit purchase order transactions, accounts payable remittances, or other significant business transactions. Individuals initiate sales transactions, trade securities, and perform other business through their home computers via the Internet. All of these represent input transactions to various IT applications, and each has its own unique control considerations. An IT auditor reviewing application input controls always should look for some basic internal control elements that should be found in all IT applications. For example, there should be some means of checking that only correct data is entered. A computer program that, through its supporting validation tables, can verify that a product part or employee number is or is not valid cannot easily verify that the current quantity should have been entered as 100 as opposed to 10. The older batch systems had hash total checks to help check for these possible errors. A hash total is a nonmonetary value, such as the ‘‘sum’’ of all account numbers. Modern systems also need reasonableness checks built into their data collection procedures, and the programs processing the transactions need controls to prevent errors or to provide warning signals. E1C10 09/16/2010 15:44:58 Page 233 IT Application Control Elements & 233 Application Inputs from Other Automated Systems Today’s IT applications often are highly integrated, with one application generating output data for processing by another. The transaction entered into one application may impact a variety of other interrelated applications. Thus an error or omission of an input at one point in a chain of applications may impact the processing of another connected application. In addition to understanding the sources of the transactions to an application, an IT auditor should understand the nature of other automated inputs to that same application. For example, a modern payroll system may receive inputs from a sales performance system to calculate commissions. The sales performance file that feeds the payroll system is another input. The controls there are based on the input, processing, and output controls of the sales performance system. If sales performance data represent a significant input to the payroll system, an IT auditor needs to be concerned about the controls over it as well as over any other supporting applications. A large network of interconnected applications can present a challenge to an IT auditor attempting to review the input controls for just one application. The IT auditor may be interested in understanding application input controls for application X. However, files from applications A, B, and C may provide inputs to X while D and E provide inputs to applications A and C, respectively. An IT auditor typically does not have the time or resources to review all of these processes and must decide on the most critical ones and assume that the other less critical supporting applications are generating appropriate transactions. Files and Database Inputs Although usually generated by some other supporting application or updated by the application under review itself, an application’s files and databases represent important inputs. In some instances, these files represent tables of data used for the validation of program data. As part of gaining an understanding of an application, an IT auditor should understand the nature and content of all supporting application files. The software that controls these files generally has various record-counting and other logical controls to determine that all transactions are correctly written onto and can be retrieved from the supporting system. Files also should have their own dating and label-checking controls to prevent them from being improperly input to a wrong processing cycle or an incorrect application. Once written as streams of sequential records on magnetic tape, today’s files are input onto higher-density disk drives or USB cartridges. However, an IT auditor needs to have a general understanding not only of the type and nature of inputs to a computer application but also of the source of the file data and any controls over it. (See ‘‘Completing the IT Application Controls Audit’’ later in this chapter for more details.) Databases can present particular challenges to an IT auditor. Although the term database often is misused to refer to almost every type of computer file, a computer system database is a method of organizing data in a format such that all important data elements point or relate to each other. In past years, many mainframe computers used what were called hierarchical databases, where data was organized in a grandparentparent-child ‘‘family tree’’ type of structure. Using it in a manufacturing enterprise, each product might be organized as a header record that would point to each of its parts. E1C10 09/16/2010 15:44:58 234 & Page 234 Selecting, Testing, and Auditing IT Applications Those components in turn would each have a hierarchy of records comprising its individual parts. File integrity was very important here; a program error that breaks one of the connecting chains would make it difficult to retrieve the lost data. Today, the relational database is a much more common file structure found on all types and sizes of computers. A relational database is like a multidimensional Excel spreadsheet. That is, the user can retrieve data across various database rows, columns, and pages rather than having to go to the head of each tree and search down through it to retrieve the desired data. Besides being very effective ways to organize input data to application systems, these databases allow for easy retrieval of reports for end users. Two common examples of the relational database model are Oracle Corporation’s database products and IBM’s DB2 database. Application Programs Applications are processed through a series of computer programs or sets of machine instructions. The traditional payroll application mentioned earlier would consist of computer programs to read employee’s time card data, store the number of hours worked, and use the employee number on that input time card to look up the employee’s rate and scheduled deductions. Based on this match, the program looks up the employee’s rate of pay and multiplies this by the number of hours worked to calculate the gross pay. A computer program is a set of instructions covering every detail of a process. A programmer writes detailed instructions for a computer system to follow. As an experiment to understand the details required to write such a larger computer program, an IT auditor who lacks programming skills should try to write down each step to follow in the morning from the time the alarm goes off until he or she arrives at the office. The next morning, the IT auditor should use these same instructions exactly as they are written to get up, wash and dress, and then go off to work. Following this program, most people will encounter program errors and arrive at work missing an item of clothing or worse. This is the difficulty of writing detailed computer programs. Usually it is not necessary for IT auditors to know how to write formal computer programs today beyond the simple audit retrieval applications discussed in Chapter 13, but the effective IT auditor should understand how computer programs are built and what their capabilities are in order to define appropriate control procedures. Traditional Mainframe and Client-Server Programs Mainframe, or what we often call legacy-type computers were used extensively for business applications since the early 1960s. These applications first were programmed in what are called first-generation actual machine languages that used binary 1s and 0s. We quickly moved to second-generation languages, what were called the assembly languages. These symbolic languages used codes to represent instructions, such as to add or store a value. Third-generation, or compiler, languages soon followed. They used actual English-like instruction statements, such as ‘‘ADD A TO B.’’ to describe the actions to be taken. Programs called compilers translated these instructions into machine language. A large variety of these compiler languages were introduced in the 1960s, but COBOL1 became the almost standard language for E1C10 09/16/2010 15:44:58 Page 235 IT Application Control Elements & 235 business data processing well into the 1980s. It is still in use today for some business applications, but specialized database and report-generator languages and objectoriented languages are now much more common. A wide range of computer languages are used today; they include Visual Basic and Java. Many applications also are developed using English-language-like report generator languages that reside on top of a supporting computer language. Other than having the skill to write an audit retrieval request, as discussed in Chapter 13, an IT auditor today does not need to be skilled in a programming language. Modern Computer Program Architectures In the mainframe computer days of years past, business applications almost always were developed in-house and often were written in COBOL. Most enterprises today generally purchase or lease their software packages or access them through a Web service provider, although some IT functions still do develop their own applications. In-house development normally occurs when an enterprise has business requirements where no commercial software packages seem correct or, more significantly, when an enterprise has plans for some new strategic software-based initiative. An IT auditor today, even with a fundamental knowledge of a language such as Visual Basic, COBOL, or C, may have some initial difficulties understanding how object-oriented applications are programmed and constructed. Often these newer applications consist of many very small program code modules that pass data to one another, sometimes over remote telecommunication lines. While it is certainly not a typical IT audit need, Exhibit 10.1 describes some object-oriented high-level programming concepts. Java and C++ are two of the programming languages of today’s Web-based applications.2 An IT auditor should rely on the overall application program standards in place as well as on other programming development and maintenance controls. Rather than looking for these application programming standards in each given application reviewed, he or she should review the general systems development controls in the IT enterprise. These might be included in a general review of IT operations, as discussed in Chapter 6. When an enterprise plans to build and launch in-house a major new or revised software application, IT audit should request the right to perform a preimplementation review of the new application development project. Preimplementation IT audit reviews are most effective for large development efforts that cover an extended span of time and primarily components developed in-house. Exhibit 10.2 contains IT audit procedures for a review of a new application systems development controls. These control processes are closely linked with the IT general controls discussed in Chapter 6 and an IT auditor should look for them in each application selected for review. Today many new application development projects do not consist just of in-house-developed new programs. Many modern applications are constructed by building data reference tables as part of purchased software applications as well as building interfaces between these purchased applications and other existing components. Proper attention must be devoted to preserving internal controls and performing adequate testing in these situations, and the IT audit preimplementation review approach can provide service to the enterprise. E1C10 09/16/2010 15:44:58 236 & Page 236 Selecting, Testing, and Auditing IT Applications Object-oriented programming (OOP) programming languages, such as JAVA or C++, model organized around ‘‘objects’’ of data rather than logic-based ‘‘actions.’’ Programs using languages, such as COBOL, were based on logical procedures that took input data, processed it, and produced output data. These older programming approaches described the processing logic but did not define the data. Object-oriented programming focuses on the data objects we want to manipulate rather than the logic required to manipulate them. Examples of objects range from human beings (described by name, address, etc.) to buildings and floors (whose properties can be described and managed) or the individual parts in a manufactured product. The first step in OOP is to go through a data modeling exercise and identify all the objects you want to manipulate and how they relate to each other. Think of all of the furniture in the board of directors’ meeting room. There will be a major table for board meetings and side tables for the support staff. The chairs in that room will be objects with each director, around the table, having one class of chair. The support staff, another class, and the CEO at the end of the table with still another. These objects are then generalized into classes of objects. OOP defines the logical sequences of these classes of objects. The directors’ chairs are arranged around the conference table, the CEO at the end, and support staff off to the sides. OOP provides computer instructions, based on the relevant data in the class object characteristics, to allow the objects and their characteristics to communicate with each other in well-defined interfaces called messages. For example, the CEO’s chair will be at the head or the table, and if the CEO is present, messages will be delivered to other board members. The concepts and rules used in object-oriented programming provide these important benefits: & & & & The concept of a data class makes it possible to define subclasses of data objects that share some or all of the main class characteristics. Called inheritance, this property of OOP forces a more thorough data analysis, reduces development time, and ensures more accurate coding. Since a class defines only the data it needs to be concerned with, when an instance of that class (an object) is run, the OOP program will not be able to access other program data accidentally. This characteristic of data hiding provides greater system security and avoids unintended data corruption. The definition of a class is reusable both by the program for which it is initially created and also by other object-oriented programs (and, for this reason, can be more easily distributed for use in networks). The concept of data classes allows a programmer to create new data types that is not already defined in the language itself. The OOP languages C++ and JAVA are perhaps the most popular object-oriented languages today, with JAVA frequently used in distributed applications on corporate networks and the Internet. EXHIBIT 10.1 Object-Oriented Programming Language Concepts Vendor-Supplied Software Today most IT applications are based on vendor-supplied software. An outside vendor will supply the basic, often Web-based, system elements, and the enterprise’s IT development function is responsible only for building custom tables, file interfaces, and output report formats around the purchased or licensed application. Often the vendor protects the actual program source code for the purchased software to prevent improper access and changes. IT auditors should be concerned that the software vendor has a reputation for quality, error-free software. Often smaller, entrepreneurial software suppliers offer very cost-effective solutions, but there can be risks in using undercapitalized software developers. If there is any doubt about the software vendor’s E1C10 09/16/2010 15:44:58 Page 237 IT Application Control Elements & 237 1. All requests for new or revised applications should follow IT standards and receive prior authorization. 2. The application development process should include sufficient requesting user interviews to develop a firm understanding of needs. 3. All new application projects should receive a detailed statement of requirements along with a formal cost benefit analysis. 4. Project plans should be prepared for all IT department development work as well as for individual application development projects. 5. Care should be given to ensuring that application development projects meet the long-range objectives of the enterprise. 6. The responsibilities for applications development work should be assigned with adequate time allowed to complete development assignments. 7. The applications development process should include sufficient user interviews to obtain a full understanding of requirements. 8. Attention must be given to internal controls, audit trails, and continuity procedures. 9. Adequate resource and capacity planning should be in place to ensure that all hardware and software is sufficient when the application is placed in production. 10. Sufficient attention, including test procedures, must be paid to backup, storage, and continuity planning for the new application. 11. Adequate controls must be installed to provide strong assurances regarding the integrity of the data processes and outputs from the application. 12. The application should be built with adequate controls for the identification and correction of processing errors. 13. All application processing data and transactions should contain strong audit trails. 14. Adequate documentation should be prepared on a technical as well as an application user level. 15. Test data should be prepared following a predetermined test plan that outlines expected results and satisfies user expectations. 16. When data is converted from an existing application, strong control procedures should be established over the conversion process. 17. If a critical application, internal audit should be given an opportunity to participate in a formal preimplementation audit. 18. There should be a formal sign-off and approval process as part of the completion of the application development process. EXHIBIT 10.2 Internal Audit IT Application Development Review Guidelines stability, arrangements should be made at the time of the software purchase contract to place a version of the vendor’s source code in escrow in the event of its business failure. A bank or some other agency would hold a version of the protected source code for release to customers if the software vendor were to fail. The decision to license, lease, or purchase a software package too often is based on an IT manager meeting a software salesperson at a trade show, establishing a need, and acquiring the software package without a full analysis of its costs and benefits. While lacking any form of the traditional IT preimplementation review, IT auditors can play a strong consulting-level role supporting IT management in the acquisition of a new software package. There are often many internal control issues that should be considered beyond descriptions in vendor sales brochures. Exhibit 10.3 presents an IT audit review procedure to use both when providing consulting help and when reviewing E1C10 09/16/2010 15:44:58 238 & Page 238 Selecting, Testing, and Auditing IT Applications Audit Internal Controls Review Procedure Workpaper Reference 1. Determine that the requirements and objectives for the new application have been clearly approved and defined. ______ 2. Assess whether application requirements have been clearly defined and whether they can be satisfied by modifications of current application. ______ 3. If requirements call for a new application, determine whether an IT analysis has been performed to determine if it may be more cost-effective to develop in-house or to purchase. ______ 4. If a search for a potential purchased application must be undertaken, determine that detailed requirements have been defined through a request for proposal (RFP) approach. ______ 5. Determine that the RFP requirements for the application clearly match the existing enterprise IT environment. ______ 6. Review procedures for the distribution of RFPs to ensure that this distribution covered all appropriate vendor candidates, and raise questions if a known vendor appears missing. ______ 7. Assess whether IT enterprise documentation is in place to review all vendor proposals on a consistent basis. ______ 8. For application software vendors that appear to meet preliminary requirements, determine that the software has been effectively demonstrated through appropriate testing. ______ 9. Where multiple vendors are presenting competing software products, consistent evaluation procedures should be in place. ______ 10. Enterprise financial and legal resources should be in place to participate in software selection. ______ 11. Determine that the selected software product has adequate documentation, help facilities, and a regular update program in place. ______ 12. Determine that an implementation plan is in place to convert either data or an existing application to the new software application. ______ 13. Where appropriate, develop preliminary plans for CAATT procedures covering the new application. ______ 14. Establish internal audit workpaper records for the new, purchased software application. ______ EXHIBIT 10.3 Purchased Software Internal Controls IT Audit Review Procedure the decision to purchase a major new software package. An IT auditor should understand the internal controls of major purchased software applications as well as he or she understands any in-house-developed application. Large, integrated packages, such as ERP systems, can have a major impact on all aspects of an enterprise. These database application packages may include production, purchasing, inventory, human resources, accounting, and all other business applications implemented as a linked series of databases. Data introduced to one application component, such as a revised standard cost for a manufactured part, will connect to other connected systems as necessary. For example, that revised standard cost will be reflected in inventory and financial systems, among others. E1C10 09/16/2010 15:44:59 Page 239 Selecting Applications for IT Audit Reviews & 239 IT Application Output Components No discussion of an application system would be complete without a description of its output components. These key application components usually consist of output screens, updated files, or even printed reports. These are important areas to survey in any application review, and IT audit should be concerned about controls contained on the output screens and control files. Older applications produced large volumes of output reports indicating the results of the processing and any control or error problems. The sheer volume and frequency of those reports often prevented users from paying adequate attention to control problems, and IT auditors frequently found control concerns that users could have identified just by reviewing their output reports. Today’s applications produce far fewer (if any) paper-based output reports; instead, results are reported on online data retrieval screens. In some cases, special online reports signal control problems and data errors; in others, users are responsible for calling up the appropriate screen to review problems. All too often, users ignore this step, and processing errors can go undetected. IT auditors always should review the scope of application output reports, screen messages and their user dispositions. Reports or screens are not the only application outputs. Transactions or updated files typically are passed to a variety of other integrated applications. Just as a modern IT application may receive inputs from a highly integrated set of input systems, it may be one link in a chain to other applications. Again and always, an IT auditor should develop a good understanding of the application reviewed as well as all of its inputs and outputs. SELECTING APPLICATIONS FOR IT AUDIT REVIEWS Although all significant IT operations and key applications should be subject to regular internal control reviews, IT audit typically does not have the resources or time to regularly review the controls for all enterprise IT applications. In addition, many IT applications represent a minimal level of control risk and are not essential audit candidates. As part of a specific operational review or a general IT controls review, IT audit should select only its more critical applications for review. An IT auditor should use the risk assessment techniques discussed in Chapter 4 to identify the more critical application vulnerabilities pertaining to an enterprise’s reporting, operational, and compliance requirements. Based on a high-level understanding of potential application review candidates, an effective approach is to rate all applications on a scale from 1 to 5 for each category according to these criteria: & & & & & Does the application contain primary enterprise controls or functions? Based on IT audit’s preliminary review, what is the design effectiveness of the application’s internal controls? Is the application primarily based on vendor-supplied, prepackaged software, or was it developed in-house? Does the application support more than one critical business process? Has this application been changed frequently or is it stable from period to period? E1C10 09/16/2010 15:44:59 240 & & & & Page 240 Selecting, Testing, and Auditing IT Applications What is the complexity of making application changes (e.g., table changes versus program code changes)? What is the financial impact of the application controls? What is the overall effectiveness of the IT general controls that support the application (e.g., change management, logical security, and operational controls)? Each application reviewed should be scored with higher numbers representing a create-audit criticality concern. For example, if application changes require updating program code, the score might be 5; changes through external tables might receive a 1 or 2. These scores, of course, are very arbitrary, but they provide a measure for each application considered. Exhibit 10.4 shows an example of an application’s control risk assessment using these scoring criteria. We have listed three example applications, A, B, and C, and have displayed them on a spreadsheet form with sample scorings. We have assigned a weighting for each of these criteria. For example, column 1 on application primary controls is given a weighting of 20 while column 6 on the complexity of changes is given a 10. Based on individual scores and their weights, a criticality score can be computed for each application. For example, for application A, this would begin as (5  20) þ (110) þ (510) þ . . . ¼ 375. The application that has the highest criticality scores would be the most appropriate, higher risk applications for planning an IT audit. Although a criticality score ‘‘wins’’ as a candidate for application review, IT audit also must consider other factors when selecting IT applications for review. Because IT applications are often so critical to enterprise operations, IT auditors often receive specific requests from their audit committee or management to review specific application controls. Some of the factors that may further impact IT audit’s decision to select one specific application over another may include: & & & Management requests. Management often asks IT audit to review the controls in newly installed or other significant IT applications due to reported problems or their perceived strategic importance to the enterprise. Sometimes these management requests are not made for the correct reasons, however. For example, sales analysis reports may appear to be incorrect due to bad data submitted from a reporting division, but management may consider the incorrect reports to be a ‘‘computer problem’’ and request an IT audit application review. Initially IT audit may not be aware of such user input problems and would perform normal review procedures. When IT audit is aware of such mitigating circumstances, audit test strategies should be modified prior to starting the review. Preimplementation reviews of new applications. In many instances, IT audit should become involved in reviewing new applications before they are placed in production. This is true for in-house-developed applications and purchased software packages. Strategies for IT audit preimplementation reviews are discussed in the section titled ‘‘Application Review Case Study: Client-Server Budgeting System’’ later in this chapter. Postimplementation application reviews. For some critical applications that are subject to a risk analysis, IT auditors also may want to perform a detailed 1 2 1 5 2 2 Application Control Risk Assessment Example 5 Application C EXHIBIT 10.4 5 1 Application A Application B Application Application Design contains primary effectiveness controls or of the functions application 1 1 5 10 Supports more than one critical business process Vendor supplied, prepackaged, or in-house developed 10 15 5 1 3 5 1 3 5 4 5 Frequency of Complexity of Application application application financial changes changes impact 10 Application Selection Risk Factor Assessment 10 2 2 2 Effectiveness of IT general controls support 15 245 170 275 Composite score 15:44:59 10 09/16/2010 20 E1C10 Page 241 241 E1C10 09/16/2010 15:44:59 242 & & Page 242 Selecting, Testing, and Auditing IT Applications application review sometime shortly after the actual system implementation. If an application has sufficient financial and operational controls significance, IT audit may want to schedule at least limited controls reviews on an ongoing basis. Internal control assessment considerations. Chapter 1 discussed the need for evaluating and testing internal controls as part of the Sarbanes-Oxley (SOx) Section 404 internal controls review process, and an application controls assessment is an important part of that evaluation. The results of understanding, documenting, and testing specific IT application controls by IT audit may provide a basis for the external auditor reviews in their SOx attestation processes. IT audit typically is faced with requests for reviews of a large number of application candidates at any time. Auditors must take care to document why one application is select over another. IT auditors often perform reviews of the specific applications that support an overall functional area. For example, the enterprise IT audit may schedule a combined operational and financial review of the purchasing department. This also may be the appropriate time to review the application controls for the major automated purchasing systems supporting that department. In this integrated audit approach, IT auditors can concentrate on the more technical issues surrounding the applications and on other supporting operational controls. PERFORMING AN APPLICATION CONTROLS REVIEW: PRELIMINARY STEPS Once an application has been selected for review, IT audit should gain a more detailed understanding of the purpose or objectives of that application, the technology approaches used, and the relationship of the application to other related processes. It may be necessary for the assigned IT auditor to do some background reading and study special technical aspects of that application. Auditors often can acquire this knowledge by reviewing past audit workpapers and applications documentation and by interviewing IT and user personnel. As an early step in this review process, IT audit should perform a walk-through of the application to better understand how it works and how its controls function. These preliminary steps will allow an IT auditor to develop specific audit tests of the application’s more significant controls. In the early days of enterprise-developed IT applications, documentation often consisted of detailed system flowcharts with supporting record layouts and little else. This documentation helped the programmer but usually was of little use to application users or IT auditors attempting to understand the application’s controls. In addition, these early flowcharts were often hand-prepared and became out of date quickly. When one relatively small change was made to a complex system flowchart, designers often were reluctant to redraw their charts. Although they might have remembered the changes, other persons reviewing the documentation would not be aware of them. Over time, application documentation evolved into a format that was oriented more toward text and functional charts. Decision tables and logic charts described the E1C10 09/16/2010 15:44:59 Page 243 Performing an Application Controls Review: Preliminary Steps & 243 functions of individual programs while text described the overall system. Although this type of documentation was more functional and less technical, it also quickly became outdated. Programmers and system designers often did not incorporate changes into this systems documentation. Today, powerful documentation tools, such as flowchart generators, are available. A real strength of these automated documentation tools is that detailed flowcharts can be combined into summary versions with changes introduced on one chart updating all others. IT auditors can expect to find various types and amounts of application documentation depending on the relative age and complexity of the application. Applications developed in-house sometimes have very limited documentation; in contrast, vendorsupplied applications that often have extensive documentation, including many dozens of volumes of descriptive text. Users will sometimes treat such documentation as almost encyclopedic reference materials. A review of the published documentation should be a first step to gaining an audit understanding of an application. If aspects of the documentation are missing or out of date, the IT auditor probably will have an audit report finding at the conclusion of the review. However, lack of documentation should not necessarily prevent an IT auditor from performing an application review. When performing the review, IT audit normally should look for these documentation elements: & & & & Systems development methodology (SDM) initiating documents. These documents include initial project requests, any cost/benefit justifications, and the general systems design requirements. Although many initial assumptions may have changed during the systems design and implementation process, these documents often help IT audit understand why the application was designed and controlled in the manner it is. Functional design specifications. This documentation should describe the application in some detail, including each of the program elements, database specifications, and systems controls. If major changes have been made to the application since its original implementation, the design documentation should reflect these changes. The purpose of this documentation is to allow an IT analyst to make changes or respond to user questions regarding the application. Application and program change histories. There should be some type of log or documented record listing all program changes within an application. Some IT departments keep such a log with the application documentation; others maintain it in a central file cross-referenced to the program source code. This type of documentation is an essential element to control program changes; it also provides IT audit with some feeling for the application’s relative stability. A large number of ongoing change requests for a given application may mean that the application system is not achieving user objectives. Revision service support controls should follow information technology infrastructure library (ITIL) service support best practices as discussed in Chapter 7. User documentation manuals. Along with technical documentation, appropriate user documentation should be available for any application. In a modern, Web-based system, much of this user documentation may be in the form of ‘‘HELP’’ or ‘‘READ ME’’ online screens. However, this documentation should be sufficiently E1C10 09/16/2010 15:44:59 244 & Page 244 Selecting, Testing, and Auditing IT Applications comprehensive to answer a wide range of user questions. It also should be supported by evidence of a user training program, as appropriate. IT audit should review selected application documentation to gain an understanding of the controls to be reviewed and perhaps use these materials to develop questions for later audit review interviews. Auditors also should take copies of key or representative sections for workpaper documentation. However, normally an IT auditor should not attempt to copy the entire documentation file for workpaper purposes. Many times auditors copy too much, adding considerable bulk to workpaper files but does little to accomplish audit objectives. Conducting an Application Walk-Through Review Once IT audit has reviewed prior workpapers, the application’s documentation, and interviewed both users and IT personnel to clarify any questions raised by the documentation review, a next step is to verify the application controls and processes by a walk-through review. For an IT application, a walk-through review is similar to an initial review of an operational facility where the auditor tours a facility, such as a production floor. The purpose of this walk-through is to confirm IT audit’s general understanding about how the IT application operates. During the walk-through, the auditor preliminarily tests application controls through sample transactions. As an example of an application walk-through review process, assume that IT audit has been asked to review the controls over an older in-house-developed accounts payable application operating on an in-house server system. The enterprise is a manufacturing firm with other fairly sophisticated IT applications. Assume that this accounts payable application was installed several years earlier and was never reviewed when it was under development. Now management has asked IT audit to review the application’s internal controls due to integrity concerns. Based on the review of application documentation, IT audit has determined that the application receives inputs from these sources: & & & & & Purchase order commitments from the manufacturing material requirements planning purchasing system Notifications of goods received from the materials-receiving system Various online terminal payment transactions for indirect goods and services that are not recorded through the materials-receiving system Payment approval transactions entered through an input screen Miscellaneous payables journal transactions entered as batch data Application data is recorded on a relational database along with tables of values for validating purchase terms, including the calculation of any purchase discounts. Based on the documentation review, application outputs include the accounts payable electronic fund transfer transactions, paper checks, transactions to the general ledger and to cost accounting applications, and various control and accounting summary screens and reports. The prime system users here are personnel from the general accounting and purchasing departments, who set up automatic vendor payments under preagreed E1C10 09/16/2010 15:44:59 Page 245 Performing an Application Controls Review: Preliminary Steps 1. 2. 3. 4. 5. 6. & 245 Develop a general understanding of the application, its inputs and outputs, and any procedures requiring manual or other system interactions. For an application with a large number of steps requiring manual processing procedures, select a sample of key transaction types to be processed from a normal production cycle. For workpaper documentation purposes, document identifying control numbers or other characteristics to trace transactions through application processes. Observe or use software tools to monitor the processing of each module or workstation step, noting situations where the walk-through transaction has: a. Inputs to another application or supporting process are passed on through the node processing module. b. Transactions are held for further cycles in process or rejected as errors during the specific processing module. Follow selected transactions through each processing module step, documenting instances where the documented control procedures are not being followed or where the transaction causes application errors or manual operator difficulties. At the end of the walk-through, discuss with appropriate IT or user administrators any unusual or unexpected problems and document internal control status. For an automated application with essentially no paper trail, follow essentially the same procedures but make appropriate inquiries and use software query tools to determine if the application is processing with appropriate controls and as expected. EXHIBIT 10.5 Application Walk-Through IT Audit Review Steps terms. The example application flowchart in Exhibit 10.5 describes general IT audit steps for an application walk-through review, where P/O stands for purchase order and the other key document is the purchase requisition. The steps to performing an application walk-through for the example accounts payable application include: 1. Briefly describe the application in the audit workpapers. Based on its review of the documentation, IT audit should prepare a brief description of the application for later inclusion in the audit workpapers. This workpaper documentation follows the general format of the walk-through description except it provides greater detail, identifying key subsystems, input screen formats, key data file names, and output report formats. (For a discussion of IT audit workpapers, see Chapter 5.) 2. Develop a block diagram description of the application. A block diagram represents an abbreviated auditor-level systems or functional-level flowchart for the application. It should reflect the description cited in step 1 and also illustrate some application flow concepts. Numerous laptop system software tools are available to create the diagram; or a flowchart can even be a hand-drawn document to increase an auditor’s understanding of the application reviewed. Exhibit 10.6 is an example of such an application process block diagram that an IT auditor can use to confirm his or her understanding of the application with key IT and user personnel. 3. Select key application transactions. Based on the previous steps, the IT auditor should select one or more representative transactions to trace through the application. This selection would be based on discussions with users and fellow members of the internal audit team. In this example of an accounts payable system, the IT auditor may select automated transactions that the receiving system should match against the payables purchase order records to initiate payment. E1C10 09/16/2010 15:44:59 246 & Page 246 Selecting, Testing, and Auditing IT Applications Start Resolve P/O Issue P/O Requisition No Procurement System Approved? Purchase Requisition Convert to Purchase Order Send P/O to Supplier Receive Goods per P/O Receive Vendor Invoice EXHIBIT 10.6 Match Invoice to P/O Prepare Vendor Payment End Application Process Block Diagram 4. Walk a selected transaction through the system processes. In the older days of manual or simpler IT applications, a walk-through amounted to just that. That is, an auditor would take an input transaction form and walk it through each of the clerical desks or steps normally used to process the transaction in order to verify the processing procedures. In a modern application, this walk-through process typically requires recording ‘‘screen shot’’ prints of a transaction as it is entered into a terminal and then prints that follow the transaction through subsequent steps. In our example, the walk-through transaction is a receiving report entry indicating that a valid open purchase commitment had been received. IT audit then would review the open commitments module of the system to determine whether the transaction was recorded on a transaction report or screen, then trace the transaction to a properly computed accounts payable check or to a funds-transfer transaction and then to general ledger system transactions for the correct amount. This type of application testing is called compliance testing. That is, IT audit is verifying that the application is operating in compliance with preestablished control procedures. Substantive testing or a test of financial statement balances involves a comparison of account balances or other methods; this is used if IT audit wishes to verify that all accounts payable checks have been input to the general ledger. Tests in support of SOx Section 404 controls typically tie a single-item test to financial statement general ledger accounts. E1C10 09/16/2010 15:44:59 Page 247 Performing an Application Controls Review: Preliminary Steps & 247 5. Modify the system understanding as required. The purpose of an application walk-through is to develop a basic understanding of the application’s functions and controls; thus, a walk-through review does not allow IT audit to determine whether all transactions are working as described. However, if IT audit discovers that the selected walk-through transactions are not working as assumed, the preliminary auditor-prepared application documentation may need to be revised. Once that documentation has been revised, IT audit may want to repeat the previous steps to determine that the auditor has a proper understanding of system transaction flows. Walk-throughs allow IT audit to gain a preliminary understanding not only of the application and its controls, but also of its relationship with other automated systems. Limited compliance testing allows the IT auditor to confirm that the application is operating as described. Although it is not a substitute for detailed, substantive application testing, a walk-through allows an IT auditor to identify major internal control weaknesses and gain a sufficient understanding of the application to define control objectives for subsequent, detailed audit testing and evaluation procedures. Developing Application Control Objectives After the review of documentation and walk-through compliance testing, an IT auditor should develop detailed audit objectives and procedures for completing the application review. This definition of audit objectives depends on the type of review planned, the characteristics of the application, and the results of the preliminary review steps. A particular review might be concerned with the level of control risk and the ability of the application to support financial statements correctly. The procedures associated with these audit objectives would be tests of the financial statement balances built up from detailed application transactions. An IT auditor also could have other objectives in reviewing an application. Management may have asked IT audit to review an application to determine if users have received sufficient training to operate it or to determine if related discount and interest calculations associated with accounts payable are performed correctly. The walk-through compliance testing may have identified significant problems, and an IT auditor may want to do little more than confirm those preliminary but troubling observations. Specific application review audit objectives should be clearly defined. An IT auditor responsible for the detailed review might wish to summarize these objectives for appropriate members of management to review and approve. Doing this may help prevent an IT auditor from devoting resources to testing an area not considered significant. In the above-mentioned accounts payable system, an IT auditor may have established several specific objectives for this review: & & The accounts payable system should have adequate internal controls, such that all receipts recorded from the receiving system are correctly matched to vendor files before the preparation of disbursements. Vendor terms should be computed correctly with controls to eliminate potential duplicate payments. E1C10 09/16/2010 15:44:59 248 & & & Page 248 Selecting, Testing, and Auditing IT Applications Controls should be in place to prevent or at least flag improper or unusual disbursements. All systems-generated disbursements should be recorded on general ledger files using correct account numbers and other descriptive codes. Depending on management’s direction, IT audit might develop other objectives for performing such a review. For example, the review could focus on database integrity or on control procedures over miscellaneous disbursements. Any review may have multiple objectives. For example, if management had asked IT audit to review the accounts payable system to ensure that no illegal or improper payments have been made, IT audit probably would also want to add a general objective to assess control risk and to determine that the system of internal controls is adequate. Before starting any detailed application review, IT audit should document the specific objectives of the review and discuss them with those managers who had requested the review to determine if the planned approach is on target and will satisfy the audit request. This same procedure should take place even if the application review has been initiated by the internal audit department as part of a total review of an IT function. Exhibit 10.7 contains some suggested control procedures and audit objectives for an IT application 1. Develop a general understanding of the application to be reviewed: its principal business purposes, inputs, outputs, and technology environment. 2. Based on the general understanding of the application, develop a general process flowchart that identifies key decision points, inputs, outputs, and internal controls. 3. Develop an understanding of the general controls surrounding the application and its processing environment, with an emphasis on ITIL service support and service delivery general controls. (See Chapter 7.) 4. Discuss the application and its performance with key system users and IT to understand any concerns or outstanding issues. 5. Develop a testing plan for the application that emphasizes: a. Identification of significant transactions, accounting, and business-related controls within and surrounding the application’s environment. b. Identify control objectives covering each of those significant controls as well as areas of concern that should satisfy the auditor that key controls are effective. c. Develop testing and sampling approaches for each of the key controls. 6. Gather evidence to perform tests of identified key controls, including: a. Copies of key files and extracts of transactions to reperform application functions. b. Special application transactions to test key or critical application controls. c. CAATT procedures or package software functions, if appropriate, to review application transactions and special functions. d. Manual or paper-based documentation to support the applications controls testing. 7. Schedule and perform tests of key application controls using the gathered test materials. 8. Evaluate all test results using a pass/fail approach, and communicate testing results with key systems users and IT to verify and validate the testing approach and its results. 9. Maintain copies of all testing plans and evidence, documenting the results in internal audit workpapers. 10. Develop an appropriate corrective action plan, where appropriate, to correct any problems encountered in the testing or application review. EXHIBIT 10.7 IT Application Review Control Objectives and Audit Procedures E1C10 09/16/2010 15:44:59 Page 249 Completing the IT Application Controls Audit & 249 review. Because there are so many types and variations of IT applications, the exhibit only lists high-level audit review approaches, and an IT auditor should make other changes as appropriate. COMPLETING THE IT APPLICATION CONTROLS AUDIT Usually more difficult for an IT auditor to define than the objectives for an IT audit of general controls, the specific audit objectives supporting a detailed IT application audit can vary depending on whether the review covers a single application or is a module of a larger business process, such as an ERP system. The IT auditor’s review strategy depends on whether (1) the application primarily uses purchased or in-house-developed software components; (2) the application is integrated with others or is a separate process; (3) it uses Web-based service providers, client-server or older, legacy computer system methods; and (4) its controls are largely automated or require extensive human intervention actions. The exact nature of an application also can vary considerably. Although the emphasis of audits once primarily was over controls in accounting-related applications, IT auditors today may review applications implemented in other areas as well such as manufacturing production planning or loan portfolio analysis. Any of these areas requires knowledge of the application’s specific attributes as well as the supporting technologies. That is, an IT auditor should understand and document how the application works, then define specific audit test objectives, and finally perform a series of audit tests to verify that the application controls are in place and working as expected. Besides the review of documentation and the walk-through, discussions with key user and responsible systems personnel can aid in the IT auditor’s understanding. The amount of effort spent here depends both on the type of application reviewed and the number of users who can be of help. For example, a capital budgeting decision support application probably will have a small group of key users who have a thorough understanding of its procedures. A logistical support system, such as factory floor data collection, may be used by a large group, where it may be difficult to identify just the key system users. The next step is to complete the documentation of the application for audit purposes. IT audit should have been making workpaper notes throughout. The documentation procedure here is largely one of summarizing where workpapers describe the understanding gained and include notes for potential follow-up review work. Clarifying and Testing Audit Control Objectives The previous section discussed the importance of establishing test objectives as part of an application review—the types of controls an IT auditor would expect to be in place for an application. IT audit sometimes can fail in this important next step of defining the specific objectives of the review. For example, management may expect IT audit to review an application’s internal accounting controls, but an IT audit may emphasize logical security controls with minimal attention given to other established control objectives. This misunderstanding of IT audit objectives becomes especially critical when the review E1C10 09/16/2010 15:44:59 250 & Page 250 Selecting, Testing, and Auditing IT Applications is not in an IT auditor’s more common realm of or related accounting or related applications. For example, if management has asked IT audit to review a new manufacturing resource planning system, its objectives could include validating internal accounting controls, reviewing for materials parts flow efficiencies, checking for system compliance with applicable regulations, or a combination of these. These objectives should be summarized briefly and discussed with both audit management and application-user management. Although the need for a clear statement of review objectives may appear an obvious early step, IT auditors too often omit this important step. Of course, the objectives of an application review may change if IT audit encounters evidence of other control problems during the course of a review. In a manufacturing resource planning review, for example, the initial objective of affirming the adequacy of the MRP application’s internal controls might change to one of fraud detection if potentially invalid transactions were encountered. Having defined these objectives, IT audit should next test the key control points within the application. Because, as part of gaining an understanding, the auditor already has done limited compliance testing and through the walk-through, these test procedures can now be expanded to make a more definitive assessment of the application’s controls. In older and simpler batch-oriented systems, this task was fairly easy. IT audit looked for input data acceptance controls, for any computer-processing decision points, and for output data verification controls. Since there are only a few processes associated with such an older batch application, this identification of test procedures often could be accomplished with minimal analysis. Modern applications today with online updating, close integration with other applications, and sophisticated programming techniques all combine to make identifying test procedures difficult. Other factors include: & & & & & Inputs to the application may have been generated by external sources, such as Web linkages, or from other applications at partner enterprises. Controls once performed by data input personnel are now usually built into programs. Modern optical scanning input devices and output documents with multidimensional bar codes make visual inspection difficult. Database files may be shared with other applications, making it difficult to determine where a change or transaction originated. The application may make extensive use of Web interfaces and will appear to be paperless to IT audit. There are numerous other reasons why an IT auditor may have difficulty initially identifying IT application audit test procedures. The application’s description, along with key user discussions, should help to identify some of these controls. As a rule of thumb, an IT auditor should look for points where system logic or control decisions are made within an application and then develop test procedures to verify that those decision points are correct. These points represent the key controls within an application, such as checks on the completeness of transactions or the accuracy of calculations. Exhibit 10.8 lists some typical areas of audit concern, control objectives, and test procedures for a review of a client-server application. E1C10 09/16/2010 15:44:59 Page 251 Completing the IT Application Controls Audit & 251 The following are test procedures that an IT auditor might use when assessing application internal controls. Based on the nature and objectives of the application, these may not apply to all applications, and the auditor will need to have a detailed understanding of the application and its key internal controls before developing a test plan. Audit Concern Control Potential IT Audit Tests 1. File data input validation Files for processing are available and complete. Review validation process procedures and test operations. 2. Process functionality and calculations Specific calculations conducted on one or more inputs and stored data elements produce appropriate data elements. Existing data tables or master file rating tables should be used to validate data. Compare input values and output values for all scenarios by walkthroughs or reperformance tests. Review table maintenance and controls, and assess adequacy of change edits and tolerance level controls. 3. Correct functionality and calculations Automated tracking of changes made to data, associating the change with a specific user. Automated tracking and highlighting of overrides to normal processes. Review a sample of reports for evidence of appropriate reviews. Review access to override normal processes. 4. Data extraction, filtering, and reporting Extract routine outputs are assessed for reasonableness and completeness. Evaluation of data used to perform estimation for financial reporting purposes. Review design of an extract routine against data files used. Review supervisory assessment of output from extract routine for evidence of regular review and challenges. Review sample of allocations for appropriateness. Review process to assess extracted data for completeness and validity. 5. Process to process balancing Automated checking of data received from feeder systems into or ledger systems to assure validity. Automated checking that balances on both systems match: or if not, an exception report is generated and used. Inspect interface error reports, and search for evidence of appropriate correctibe actions. Inspect validity and completeness application parameters and settings. Review access to set and amend configurable parameters on interfaces. Inspect evidence of matching reports, checks, and error file processing to determine whether controls are operating effectively. 6. Automated functionality and aging File extracts from supporting applications are available to provide management with data on aged transactions. Test sample of listing transactions to validate appropriateness of aging processing. 7. Checks for duplicate transactions Comparison of individual transactions to previously recorded transactions to match fields. Comparison of individual files to expected dates, times, sizes, and other values. Review access to set and amend configurable parameters on duplicate transactions or files. Review processes for handling rejected files or transactions. EXHIBIT 10.8 IT Application Review Processing Controls E1C10 09/16/2010 15:44:59 252 & Page 252 Selecting, Testing, and Auditing IT Applications This exhibit some outlines areas of IT auditor concern, potential control objectives to support those areas, and test procedures to determine if the controls are operating. These steps are oriented to an almost generic financial application that IT auditors frequently encounter. However, because there are so many different types and variations of IT applications, an IT auditor will have to custom build a review approach with consideration given to these issues: & & & Tests of application inputs and outputs Test transaction evaluation approaches Other application review techniques Tests of Application Inputs and Outputs In the very early days of IT auditing, many audit-related tests were little more than checks to verify that all inputs to a program were accounted for correctly and that the correct number of output transactions were produced based on these inputs. An auditor’s review of an automated payroll system is an example of such a set of tests of inputs and outputs. The IT auditor, of days past, would perform tests to determine that all time cards input were either accepted or rejected and that the number of output payroll checks produced could be reconciled to those system input time cards. This was a very basic test of system inputs and outputs. Although automated applications have become much more complex, many audit test procedures today are little more than those same tests of inputs and outputs. An IT auditor should examine the outputs generated from an application, such as invoices produced by a billing system, to determine that the input data and automated computations are correct. This type of audit test is limited in nature and often will not cover all transactions or functions within an application. The purpose of a control risk assessment or compliance test is to determine if application controls appear to be working. If all transactions or supporting data are to be reviewed, substantive testing procedures or tests of financial statement balances should be used. The extent of this testing depends on the audit objectives. For example, an IT auditor usually performs compliance tests over those aspects of an application that cover internal accounting controls related to financial statements. An IT auditor may want to perform compliance tests over other areas, such as the efficiency of administrative controls. For older applications, tests of inputs and outputs often are quite easy to perform and are not nearly as easy for today’s applications, where the auditor often does not encounter a one-to-one relationship between inputs and outputs. Test transaction approaches, discussed next, are often much easier to perform and even more meaningful. Nevertheless, tests of inputs and outputs are sometimes useful for reviews of applications. Compliance test IT audit procedures for an example automated purchasing application are outlined in Exhibit 10.9. Test Transaction Evaluation Approaches An IT auditor may want to ascertain that transactions entered into a system are processed correctly. For example, when reviewing a plant floor manufacturing application, an IT E1C10 09/16/2010 15:44:59 Page 253 Completing the IT Application Controls Audit & 253 1. Select a series of purchase orders generated by the application reviewed and trace them back to the requirements generated through either the procurement system or authorized manual purchase inputs, determining that all new purchase orders have been properly authorized. 2. From the sample selected in step 1, trace the purchase orders back to established records for vendor terms and prices, resolving any differences. 3. Select and trace a cycle of automated purchase orders to appropriate Web control logs to determine that all documents were transmitted without error and on a timely basis. 4. Using a sample of purchase orders received from log files, determine that vendors are documented through current, signed purchase agreements. 5. Select a sample of receiving reports, and determine that the application is working properly by matching receipts to open purchase orders and accounts payable records. 6. Select a sample of recent accounts payable vouchers and any actual checks generated for parts and materials, tracing the payments to valid receiving reports and purchase orders. 7. Using sample transactions that were either held upon receipt for noncompliance with terms or improper timing, verify that transactions are handled correctly and per established procedures. 8. Balance a full cycle sample of purchase transactions, from the input system providing inputs to the control logs and printed purchase order documents. 9. Investigate any balancing differences and verify the reported errors are valid. 10. Document all exceptions found as potential audit report items. EXHIBIT 10.9 Automated Purchasing System Compliance Tests Example auditor might record several shop materials transactions as they are entered on manufacturing floor terminals. After an overnight processing cycle, an IT auditor can verify that those transactions have correctly made adjustments to inventory records and that work-in-process cost reports have been properly updated. This verification can take place by reviewing special retrieval reports against data files. As part of the test transaction process, an IT auditor also can test whether error-screening controls are operating as described. The emphasis here should be on the testing of the error-verification routines within the application. IT audit can select transactions input to an application that appear to be invalid and then trace them through the application to determine that they have been properly reported on exception reports. IT audit also can consider submitting test error transactions to a system to verify that they are being properly rejected by the application. Other Application Review Techniques The computer-assisted audit tools and techniques (CAATTs) discussed in Chapter 13 can be useful in reviewing application controls. All too often, IT auditors use these tools to test some accounting control, such as an accounts receivable billing calculation, but do not to evaluate other application controls. Audit software can match files from different periods, identify unusual data items, perform footings and recalculations, or simulate selected functions of an application. Other useful techniques are: & Reperformance of application functions or calculations. This type of test is applicable for both the automated and the manual aspects of application systems. For example, if a fixed-assets application performs automatic depreciation E1C10 09/16/2010 15:44:59 254 & & & & Page 254 Selecting, Testing, and Auditing IT Applications calculations, IT audit can use CAATT processes to recalculate depreciation values for selected transactions as a compliance test. Reviews of program source code. For applications developed in-house, IT audit can verify that a program performs a certain logic check by verifying the source code. However, this type of compliance test should be used with only the greatest amount of caution. Because reading and understanding program source code is complex, it is very easy to miss a program branch around the area being tested. An IT auditor should consider the specialized programs available to compare program source code with the compiled versions in production libraries. Continuous audit monitoring approaches. IT audit sometimes can arrange to build embedded audit procedures into production applications to allow those applications to flag control or other application exception problems. These techniques also are discussed in Chapter 13 on CAATTs and in Chapter 14 on continuous auditing techniques. This approach goes beyond just auditing an application adding procedures to make it self-auditable on a continuous basis. Observation of procedures. Observations may be useful when reviewing both automated and manual applications. For example, a remote workstation receiving downloaded data from a central IT system may require extensive manual procedures in order to make the proper download connection. IT audit can observe this on a test basis to determine if the manual procedures are being performed correctly. Completing the Application Controls Review Although compliance tests are powerful methods to test application controls, IT audit should be aware that this level of assurance is not absolute. There is a risk that an IT auditor may test an application control and find it to be working when, in fact, it does not normally work as tested. Because of the risks associated with such compliance tests, therefore, IT audit always should be careful to condition any audit reports to management with a comment or warning about the risks of incorrect results due to the limited audit tests. Sometimes the controls tested do not appear to be working correctly because IT audit does not understand some aspects of the application system. IT audit may want to review the application description and identify controls to verify that they are correct. It may be necessary to revise IT audit’s understanding of the application controls and then reperform the audit risk assessment procedures. If IT audit finds that, through compliance testing, the application controls are not working, it probably will be necessary to report these findings. The nature of this report depends very much on the severity of the control weaknesses and the nature of the review. For example, if the application is being reviewed at the request of the audit committee or senior financial management, the identified control weaknesses may prevent them from placing any reliance on the financial results produced by the application. If the control weaknesses are primarily efficiency related or operational, IT audit may want to report them just to IT management for future corrective action. Applications can be primarily financial or operational. They can be implemented using purchased software, can be custom-developed applications located on in-house E1C10 09/16/2010 15:44:59 Page 255 Application Review Case Study: Client-Server Budgeting System & 255 systems with extensive database and telecommunications facilities, can operate in a client-server environment, or can exist in numerous other variations. As noted, this diversity makes it difficult to provide one set of audit procedures for all applications. While IT audit can develop a general approach to reviewing most data processing applications, usually it is necessary to tailor that approach to the specific features of a given application. The next section describes how an IT auditor might perform a review of a capital budgeting application operating in a client-server environment with telecommunications links through a network linked to a larger server machine. APPLICATION REVIEW CASE STUDY: CLIENT-SERVER BUDGETING SYSTEM As an application review example, assume IT audit has been asked to assess the internal controls over an in-house-developed client-server architecture capital budgeting system. Assume that a financial planning department has developed the capital budgeting analysis portion of this example application using a popular desktop spreadsheet software package. Although the application has been built around a purchased software spreadsheet package, the business users have coded a series of macro instructions for running the programs. The workstation portion of this example system communicates with a server file containing mainframe budgeting system data. Assume that IT audit has been asked by management to review general controls over both local networks and their client-server computer operations. Following the IT general controls review audit procedures discussed in Chapter 6, IT audit found that general controls in these areas were adequate. That is, users documented their desktop applications; adequate backups of files and programs were performed on server files; password procedures limited access only to authorized personnel; and other good internal control procedures were followed. Among IT audit’s recommendations was to place stronger controls over telecommunications access to the local network and to install virus-scanning procedures. Sometime after that general controls review, this example capital budgeting system was implemented on the enterprise administrative office network. Because this system provides direct input to the corporate budgeting system, management has asked IT audit to review its application controls. After discussing this review request with senior and IT management, IT audit developed these high-level review objectives: & & & & & The spreadsheet capital budgeting system should have good internal accounting controls that are consistent with other enterprise control processes. The application should properly make capital budgeting decisions based on both the parameters input to the system and programmed macro formulas. The system should provide accurate inputs to the central or corporate budgeting system through the local file server. IT security and integrity controls should be secure. The capital budgeting system should promote efficiency within the financial planning department. E1C10 09/16/2010 15:44:59 256 & Page 256 Selecting, Testing, and Auditing IT Applications These objectives are at a very high level but represent the general format of the audit objectives that an IT auditor should formulate for this type of application review. Management typically states its objectives in words that emphasize application performance and integrity features objectives. Although these management-initiated audit objectives often are not IT technical issues, IT audit should translate them to more detailed IT control issues for the next general IT audit steps. Review Capital Budgeting System Documentation IT audit’s first step should be to review the documentation available for this example application. Since this example is built around a commercial desktop software product, IT audit might expect to find or should ask for: & & & Documentation for the capital budgeting software package, including descriptions of the spreadsheet macro procedures, and formulas Procedures for uploading capital budget data to the central system budgeting application through network server files as well as procedures for accepting the input data to the mainframe IT function Procedures to ensure the integrity of the data resident on server files IT audit probably will not find documented procedures covering all of these elements, but there should be documentation covering the software product used, the interfaces with other applications, and the necessary manual procedures. IT audit should review these materials to determine that they are complete enough that an auditor can gain a general understanding of the overall application. Then, after reviewing this documentation and discussing it with its financial planning users, IT audit should describe the allocation for audit workpaper documentation purposes. Since this example application is built around a spreadsheet software product, this description would primarily cover its manual interfaces. Control descriptions over file server applications and their network connections to client systems should have been covered as part of the previously mentioned general controls review. IT auditors often find it convenient to describe such an application in the form of a flowchart, although a written process description may be adequate. The purpose of this type of description is to provide IT audit with workpaper documentation of the application and to provide a basis for the identification of significant control points. Identify Capital Budgeting Application Key Controls Although a rather simple but compact application, this example capital budgeting application has some critical control points. For example, if the spreadsheet macro procedures are calculating capital costs correctly, present values, and such related factors, management may very well take incorrect actions regarding investment decisions. If data is incorrectly transmitted to the mainframe budgeting system, financial statement records may be incorrect. If the application is not documented properly, a change of key users in the financial planning department may make the system nearly inoperable. E1C10 09/16/2010 15:44:59 Page 257 Application Review Case Study: Client-Server Budgeting System & 257 1. Develop a detailed understanding of all significant automated and manual application input transactions: their nature, timing, and source. 2. Develop a strong understanding of transaction error correction procedures, both the nature of the tables or rules used for verification as well as any built-in system logic; determine that formal turnaround procedures exist to control any initial error items. 3. Using documentation or database descriptions, trace all input to output data flows with the application, showing how input elements (e.g., orders from the inventory application) will change or modify other system elements. Document this understanding in workpapers through audit data process flow diagrams. 4. Determine that controls exist for comparing the number of items input to those that have been either accepted or rejected; review error identification procedures to assess whether users can easily understand the cause and nature of any errors. 5. Review procedures for the correction and resubmission of rejected items; determine if errors are held in suspense files for analysis and corrective actions. 6. Develop a detailed understanding of all significant system output control totals, and review the nature of supporting controls for a selected single application update processing cycle and from cycle to cycle. 7. Select an input update cycle for review, and determine that the number of items input, less any rejected errors, ties to application output control totals. 8. For the selected test cycle reviewed, determine that all error items from the cycle have been corrected, resubmitted, or received proper disposition. 9. Review control totals in the subsequent processing cycle to determine that file totals have remained consistent from one cycle to the next, investigating any discrepancies. 10. Review existing error suspense files to ascertain that all error items have been investigated and corrected in a timely manner; investigate any items remaining in the error cycle to determine the reasons for any delays. 11. Review any preliminary concerns or errors with IT and responsible management to make any necessary changes to audit test procedures. 12. Document all audit review and testing activity in workpapers. EXHIBIT 10.10 Capital Budgeting Application Input and Output IT Audit Tests Based on IT audit’s understanding of this example system, key system controls have now been defined and documented. Here, because IT audit has recently performed a general controls review, it is not necessary to reconsider those general controls during the application review. IT audit review procedures can now be developed similar to those shown in Exhibit 10.10. Perform Application Tests of Compliance For the final step in this application review, IT audit should perform tests of the established audit procedures. Depending on management’s and IT audit’s relative interest in the application, it may not be necessary to test all of the controls as listed. Many are related to one another. If no problems or weaknesses are identified in one control area, IT audit may decide to pass on the related control areas. Some of the tests of application controls might include: & Reperformance of computations. Capital budgeting is based on some very specific and often complex computations, such as the estimation of the present E1C10 09/16/2010 15:44:59 258 & & & Page 258 Selecting, Testing, and Auditing IT Applications value of future cash flows based on discount factors. Using another spreadsheet or other desktop system tools, IT audit could select one or several present value computations generated by the system and recalculate them to determine the reasonableness of system processes. Any major differences should be resolved. Comparison of transactions. IT audit can select several sets of application budget schedules and trace them through the file server budget system to determine that they have been correctly transmitted. Proper approval of transactions. Before any system-generated budget schedule is transmitted to the central budget system, it should have had proper management approvals. IT audit should select a sample of them for review. Numerous similar compliance tests can always be performed. The imaginative IT auditor will choose which tests to perform based on the nature of the audit and management objectives. Control weaknesses should be reported to management for corrective action. AUDITING APPLICATIONS UNDER DEVELOPMENT It is often much more efficient for an IT auditor to review an IT application for its internal controls while it is being developed and implemented rather than after it has been placed into production. The role of the IT auditor here is similar to that of a building inspector reviewing a new construction project: It is difficult to make constructive recommendations regarding the completed building. Even if some problems were found, the inspector would be under considerable pressure not to identify ones that would require significant portions of the building to be torn down and rebuilt. Rather, the building inspector identifies problems during construction and suggests how they can be corrected before completion. Similarly, the effective IT auditor should suggest corrective actions to improve system controls along the way. It is easier to implement changes during an application implementation process than after it has been completed and the system has been placed into production. To continue with the analogy, an IT auditor must be careful not to take responsibility for designing the new application’s controls. The building inspector points out problems but certainly does not take responsibility for fixing them. The discussion on performing effective IT audits in Chapter 5 emphasizes that it is IT audit’s task to review and recommend but not to design or build the controls in any area reviewed. When reviewing new applications under development, an IT auditor should point out internal control weaknesses to the application developers but only recommend that they implement those recommendations. Application development groups, user management, and auditors all tend to agree that, in reviewing new IT applications under development, IT audit provides another set of eyes to look at new and soon-to-be-implemented applications. This section offers approaches to reviewing new applications under development and discusses some of the pitfalls IT audit may encounter when attempting to audit them. E1C10 09/16/2010 15:44:59 Page 259 Auditing Applications under Development & 259 Objectives and Obstacles of Preimplementation Auditing The concept of preimplementation reviews was first proposed by the then-new profession of what was called EDP auditing in the early 1970s; at that time, many traditional internal auditors were opposed to this approach. Traditionalists argued that if an auditor reviewed an application in advance of its implementation, it would be difficult to come back later and review that same application after implementation. The argument was that if an IT auditor had ‘‘blessed’’ the internal controls of a system under development, how could that same auditor come back later and perform a critical review? Over the years, there have been many changes to the application development process. New applications frequently are based on vendor-supplied software components, and internal auditing standards, as discussed in Chapter 3, allow an internal auditor also to act as a consultant. IT auditors have also grown to accept preimplementation reviews, acting as auditors and not consultants. IT auditors, however, face four major obstacles when reviewing new applications under development: 1. ‘‘Them versus us’’ attitudes. Although IT audit and general management both may accept the concept, IT management often expresses wariness or even resentment when IT audit announces its plan to review an application that is under development and still has many details yet to be worked out. The announcement ‘‘Hello, I’m from IT audit, and I am here to help you’’ may not be received favorably. Good preimplementation review procedures can establish respect for IT audit’s role and add value in the development process. An IT auditor who spends many hours reviewing a complex new application with some potential control-related issues and who concludes only that ‘‘Documentation needs to be improved’’ will not be viewed as having added much value to the process. 2. IT auditor role problems. The IT auditor’s role must be clearly understood by all parties and might be defined as: & An extra member of the implementation team. The systems design team invites the IT auditor to design review meetings. However, that IT auditor will be more of an observer than a typical member of that team. The auditor’s objective is to gather data regarding key controls and processing procedures for a subsequent audit report. & A specialized consultant. Sometimes an IT auditor can become so involved in the systems design and development process that he or she is viewed as just another design team consultant making recommendations during the course of the implementation process. IT audit should take care to not be viewed in that light. Following the standards for an IT auditor as an enterprise consultant, as discussed in Chapter 3, an IT auditor should act primarily as an independent reviewer providing help to the team, not as a specialized consultant who is part of the design process. & An internal controls expert. In any review, IT audit always should make certain that a review of internal controls is included in the new project. However, the auditor should not be the primary designer of those controls. Otherwise, he or she may have problems reviewing the completed application and its controls at some later date. E1C10 09/16/2010 15:44:59 260 & & & & Page 260 Selecting, Testing, and Auditing IT Applications An occupant of the ‘‘extra chair.’’ Sometimes an IT auditor does not do a proper level of preparatory work during a preimplementation review. Systems management may request an auditor to review various materials and attend design review meetings. An IT auditor who does not prepare but simply attends these meetings too frequently provides no real contributions. Nevertheless, if problems occur in the future, management may say ‘‘But IT audit was there!’’ State-of-the-art awareness needs. New systems applications often involve new technologies or business processes. IT auditors should be involved in their own continuous education programs to better understand state of the are issues. A general understanding of new technologies may require some additional IT auditor homework—reading vendor manuals and other documentation. Many and varied preimplementation candidates. The typical larger enterprise may have a significant number of new application projects that are potential candidates for preimplementation reviews. These projects will all have different start times, durations, and completion dates. An IT auditor needs to perform an ongoing risk assessment to select the most appropriate new review candidates. Despite these potential obstacles, there are strong reasons for IT auditors to become actively involved in preimplementation reviews of major or critical new applications. This is particularly true in today’s era of major enterprise-wide applications, such as ERPs, that require detailed planning and testing in all areas of the enterprise. Preimplementation Review Objectives A key objective of application preimplementation auditing is to identify and recommend control improvements such that they can be potentially installed during the application development process. However, rather than just assuming that a new IT project is a given and then reviewing its controls, IT audit also should aim to review the justification and definition of the new development project. There should be a good project management system in place that properly plans development steps and measures actual progress against those planned steps. For major projects, IT audit can evaluate the adequacy of project development controls used for the particular application. This preimplementation phase is also an excellent time for an IT auditor to gain an understanding of the new application sufficient to design future automated audit tests and to define the CAATTs as discussed in Chapter 13. Whether the implementation involves a vendor software package or an application developed inhouse, IT auditors should gain overall understandings of all aspects of those application projects. In addition, IT auditors often must comply with statutory requirements for reviewing new applications under development. Several U.S. states and other countries have legislation requiring that all new significant state agency applications be reviewed by their audit departments for IT controls prior to implementation. Auditors in many U.S. state governments can expect such legislation to appear in their own states in the future. E1C10 09/16/2010 15:44:59 Page 261 Auditing Applications under Development & 261 Preimplementation Review Problems Preimplementation reviews often present IT audit with some very serious review and scheduling problems, including too many review candidates given limited IT audit resources. IT auditors sometimes make the mistake of announcing their intent to review all new applications and all major modifications prior to implementation. In a larger enterprise, dozens or even hundreds of user requests for new or major revision projects may be initiated regularly. IT audit will have no time for comprehensive preimplementation reviews and only time for little more than nominal rubber-stamp approval signatures. To overcome these difficulties, IT audit should consider these issues: & & & Select the right applications to review. Auditors are faced with the problem of selecting only those applications of audit significance. Rather than relying on a simple value judgment or an arbitrary process, IT auditors should follow a riskbased, structured selection method for identifying those applications to review. A development group, for example, may be working on applications A, B, and C. Given the relative application risks as well as limited audit time and resources, IT audit may decide to perform preimplementation reviews only for application B. However, if significant postimplementation problems appear in C, management might later second-guess IT audit and ask why system C had not been selected for review. An IT auditor with a consistent selection approach will be able to justify the decision to review B rather than C. Determine the proper IT auditor’s role. As discussed, when an application has been selected for preimplementation review, IT audit all too often can become overly involved in its systems development and implementation processes. Particularly for applications based on vendor software products, many rapidly in-house, new IT projects require extensive user and systems development team efforts, with numerous design review meetings. IT audit often is asked to participate in these design review meetings, but doing so may cause an auditor role problem. If an auditor is actively involved in the typical design review meetings where design compromises may be negotiated, he or she may find it difficult to comment on these same decisions later as audit points. However, if excluded from design meetings, IT audit may have a hard time performing the review. To be effective in reviewing new applications under development, the IT auditor’s role needs to be carefully defined. IIA Standards allow an auditor to act as an internal consultant, as discussed in Chapter 3, as long as this role is carefully defined. Review objectives can be difficult to define. It is often easy to define audit review objectives at a very high level but then difficult to translate them into reality. When an auditor informs the IT department that a given application has been selected for preimplementation review and requests supporting documentation, the IT auditor may receive folders or files containing hundreds of pages of requirements studies, general design review documentation, meeting minutes, and other materials. IT audit may then be asked to review and comment on this mass of materials. An audit objectives and control procedures approach can help an auditor to choose the relevant materials to review. E1C10 09/16/2010 15:44:59 262 & Page 262 Selecting, Testing, and Auditing IT Applications Multiple implementation projects and new technologies present some major challenges to IT audit to perform effective preimplementation reviews of IT applications. However, whether for new applications developed in-house or purchased software, IT audit preimplementation reviews will usually add value to the internal controls environment in the enterprise. In addition, through preimplementation reviews, IT auditors who have been accused in an old joke as being the ones who ‘‘join the battlefield after the action is over to shoot the wounded’’ can now play a proactive role in the application development process. Preimplementation Review Procedures Many of the same audit procedures discussed in other chapters for IT internal controls reviews can also be followed for reviews of new applications under development. All too often, IT auditors argue that applications under development are somehow ‘‘different.’’ However, as fluid and subject to ongoing developmental change as applications under development are, many of the same control objectives and procedures discussed previously for IT applications are quite appropriate for these reviews. IT auditors should tailor their preimplementation reviews along the various phases a new project’s development, starting with initial project initiation, to requirements definitions definition, to development and testing, and finally to implementation. These same basic steps apply whether the application is a major in-house-developed one, a vendor service-based software package, or a user-led set of desktop applications. The only difference is on emphasis depending on the application development approach. When IT audit has selected a given application for preimplementation review, an important first step is to review the overall planned audit program steps with IT management so that there is an understanding of what IT audit expects to find as well as the review approach. Some procedures may be tailored to fit a given application, but the next objectives should apply for most preimplementation reviews. Application Requirements Definition Objectives When possible, IT audit should get involved in a preimplementation review early in the development phase. Here, IT audit should review the detailed requirements study to determine the overall control status of the new application. If IT audit can identify control concerns during this phase of the application’s development, it will be relatively easy for system designers to address and correct them. Exhibit 10.11 is a set of IT audit procedures for the requirements definition phase of any project. IT audit should look for similar requirements no matter how the new application is developed. Some of these procedures, of course, may require modification if the application under review is composed of specialized technologies or will be a major modification to an existing system. However, IT audit should perform control procedures necessary to satisfy all of the control objectives listed here. IT audit may need to decide if any special skills are required to complete the review. If the application involves the use of new or unique systems technologies, a new vendor-supplied application package, or specialized supporting software, IT audit may want to enroll in trainingonthe softwareproduct tobeused—suchasthrough classes offeredbythevendorto the development staff—or IT audit may bring in someone with specialized skills or training. E1C10 09/16/2010 15:45:0 Page 263 Auditing Applications under Development Audit Step 263 & Workpaper Reference 1. Obtain a general understanding of the IT department’s system development methodology (SDM) standards for both developing new applications and installing purchased software to assure an appropriate requirements definition study. ___ 2. Obtain user-approved request documentation authorizing the detailed application development or purchase. ___ 3. Review detailed project plans for the new application and ascertain, through discussions with IT and requesting users, that estimates of time and resources seem reasonable and achievable. ___ 4. Determine if there was an appropriate analysis, including cost and timing considerations, to determine whether the application should be built in-house or purchased/leased. ___ 5. Determine if any special skills are needed to review application internal controls, such as RFID wireless connections or an understanding of ERP databases. If appropriate, arrange for IT audit staff members to learn new skills through seminars or other training. ___ Identify and review significant internal controls surrounding the new application. Discuss controls with both key users and IT to develop testing procedures. ___ 7. If significant portions of the application involve in-house-developed modules, assess whether appropriate consideration has been given to purchased software alternatives. ___ 8. Assess whether the impact of manual aspects of the application has been given proper consideration as part of the requirements definition, such as training needs or process changes. ___ 9. If the application appears to be a candidate for CAATT procedures, begin preliminary audit planning for installation. ___ 10. Review the extent of user sign-offs on the requirements study; based on selected interviews, assess whether users understand the new application and any internal control or procedure ramifications. ___ 6. EXHIBIT 10.11 Preimplementation Review Requirements Definition Checklist For example, with some large projects that take years to develop and implement, adding a specialist to the staff to review just that project can be effective. At the completion of this phase, IT audit might write an informal audit report outlining any preliminary observations and concerns. In addition, workpapers should be started to document the new application control’s procedures. Detailed Design and Program: Development Objectives Defining detailed design objectives is typically the longest phase of any new application’s development process. IT audit may want to schedule several reviews during this phase. Each periodic review should focus on a specific area of the new application development project, but the overall purpose should be to answer some of these questions: & & Doesthedetaileddesigncomplywiththeobjectives of the generalrequirementsdefinition? Do users understand the controls and objectives of the new application under development? E1C10 09/16/2010 15:45:0 264 & & & & & Page 264 Selecting, Testing, and Auditing IT Applications Has proper consideration been given to application controls and security? Is the application being developed according to the IT department’s own systems development standards? Is the application development process supported by a well-organized project plan, similar to IT audit planning discussed in Chapter 5? Have any earlier audit recommendations been incorporated into the detailed design? During this phase, care should be taken not to become buried in detail. Some IT enterprises may attempt to use IT audit as a quality assurance function for the project. However, overall audit effectiveness will be diminished if IT audit’s time is spent reviewing such things as compliance with detailed programming standards. Reviews of this nature should be limited to periodic testing. Any control-related concerns encountered should be brought to the attention of management so that corrective action can be taken in a timely manner. If the new application is purchased software, often there are limited in-house design and programming requirements. However, the IT enterprise may have to build file conversion programs or interfaces with existing systems or table files or report generator definitions. These can represent major efforts, and IT audit still should review controls over the purchased software before it is installed and implemented. Application Testing and Implementation Objectives This phase includes testing of the new application and completion of documentation, user training, and conversion of data files. IT audit often is able to evaluate whether system controls appear to be working as expected and will want to test any audit modules incorporated into the application. Exhibit 10.12 is a preimplementation review application testing checklist for this preimplementation phase to help IT audit recommend whether the new application is ready for final or production implementation. Significant system control problems, coupled with management pressures to implement the application as soon as possible, can make this phase difficult. IT often promises to correct control problems in the new application during a ‘‘phase two.’’ Auditors often find that because of other priorities, this promised phase two never seems to occur. IT audit should consider the severity of such control problems and either document them for follow-up review or inform management of the need for corrective action during the current implementation. At the conclusion of the application testing and implementation phase, the responsible IT auditor should prepare a final report that documents the identified significant control issues and subsequently corrected by the IT development function. This report also should outline any outstanding control recommendations that have not been implemented. Reports up to this point have been informal; this final report, however, should follow normal internal audit department reporting standards. Postimplementation Review Objectives and Reports Although the new application is no longer in development, the postimplementation phase of a preimplementation audit often is still important. The postimplementation review should take place shortly after a new application has been implemented and has E1C10 09/16/2010 15:45:0 Page 265 Auditing Applications under Development Audit Preimplementation Steps 1. & 265 Auditor Init Determine if a formal test plan exists, including an outline of key application modules detailing expected data conditions, the business rules tested, the type of test, and the expected results for each module condition tested. ___ 2. Review the results of several recent unit tests to determine if results have been mapped to the test plan, exceptions researched, and errors corrected as appropriate. ___ 3. Determine if the application being tested satisfies original application design requirements; if exceptions exist, determine that they are properly documented, reviewed, and approved by key users. ___ 4. Interview representative key users to understand their participation in the testing process; where participation appears to be lacking, discuss and document the need for user participation to ensure successful implementation. ___ Review the extent of overall system testing, including key interfaces with other applications and outside service providers. ___ 6. If any original requirements have not been achieved by the completed application, assess procedures in place to determine whether to add procedures later or to otherwise allow for discrepancies. ___ 7. If appropriate, initiate a series of internal audit–developed test transactions that emphasize key controls, defined in earlier review steps; review the test results and assess application performance. ___ 8. Summarize the results of preimplementation application testing activities, and make internal audit recommendations regarding the appropriateness of the application implementation. ___ 5. EXHIBIT 10.12 Preimplementation Review Application Testing Checklist had time to settle down. In other words, IT audit should perform the review after the users have had an opportunity to understand the application and information systems have had time to resolve any final implementation problems or errors. Here the postimplementation review determines if application design objectives have been met and if established application controls are working. It also should look at project controls to determine if the application was completed within budget. Ideally, this review should be performed by another member of the audit staff to provide an independent assessment of the new application. Many IT audit departments have a fairly formal procedures for issuing audit reports. Draft reports are prepared, auditees prepare their responses after some discussion and negotiation on the draft, and a final audit report is issued, with copies distributed to various levels of management. Sometimes this audit report format is inappropriate for reviews of new applications under development. An individual internal controls problem with a particular program or output report, which may be identified by an IT auditor when performing a preimplementation review, can be corrected by the application developer almost at once. There is little need to discuss such a finding in a formal audit report draft. The control concern should have been corrected long before the audit report was issued. Audit and general management, who might expect the E1C10 09/16/2010 15:45:0 266 & Page 266 Selecting, Testing, and Auditing IT Applications more formal audit report with its findings and recommendations, should understand the special report format used for preimplementation reviews. Informal, memo-type reports should be issued after each phase of the preimplementation reviews. These memo reports should discuss the scope of review activities and document any audit concerns. If some of the prior concerns have been corrected, the actions taken and current status of the control issue should be discussed. IT audit should also develop workpaper documentation covering these review activities, which will serve both to document preimplementation activities and to provide a basis for later application reviews. At the conclusion of the preimplementation review, IT audit should issue a formal audit report following internal audit’s department standards. Where appropriate, this report can discuss preimplementation audit findings and corrective actions taken. However, the main function of this final report is to highlight outstanding control issues that still need to be corrected within the new application system. IMPORTANCE OF REVIEWING IT APPLICATION CONTROLS An IT auditor should place a major emphasis on reviewing the supporting IT applications when performing reviews in other areas of the enterprise. Even though good general or interdependent IT control procedures may be in place, individual application controls may not all be that strong. An enterprise’s applications may have been developed through a series of compromises among users or without any level of proper quality assurance. To evaluate IT application controls properly, IT audit needs a good understanding of both IT procedures and the specific control and procedural characteristics of each application area. The effective IT auditor should spend a substantial amount of audit effort reviewing and testing controls over specific IT applications and new applications in the development process. Such reviews will provide assurance to general management that applications are operating properly and to IT management that their design and controls standards are being followed, allowing them to place greater reliance on the output results of such applications. An understanding of application control reviews is a key skill requirement for all IT auditors. NOTES 1. Developed in the 1960s, the computer programming language COBOL stands for Common Business Oriented Language. It is still used today as a key programming language. 2. Numerous textbooks and references describe object-oriented programming. A search on the Internet will provide many references. E1C11 09/16/2010 13:36:56 Page 267 11 CHAPTER ELEVEN Software Engineering and CMMi T H E I N F O R M A T I O N T E C H N O L O G Y ( I T ) applications discussed in Chapter 10 are based on software programs, some purchased from outside vendors or downloaded from Web sources, others purchased and then customized by the enterprise’s IT systems development staff, and still others built by in-house programmers and systems developers. In all instances, planning and good procedures are needed to build and implement successful IT software products. Although the process to build these applications once was called computer programming, it is increasingly known today as software engineering. This chapter provides a high-level introduction to both software engineering and the Software Engineering Institute’s Capability Maturity Model1 for Integration (CMMi1), an important assessment tool for evaluating the capabilities of an IT function on many levels. CMMi has strong links to the Control Objectives for Information and related Technology (CobiT) framework, discussed in Chapter 2, and is useful for evaluating IT quality assurance as discussed in Chapter 31. It is also a measure to allow IT audit to assess how well they are doing in building and implementing the computer-assisted audit tools and techniques (CAATTs) discussed in Chapter 13. An understanding of CMMi and software engineering concepts also will help an IT auditor to better review both IT general and application controls. SOFTWARE ENGINEERING CONCEPTS Many IT auditors know a little about computer programming from college classes or from writing simple retrieval programs to gather audit data. While those routines are often fairly simple and easy to launch, most software development projects are much more complex. The enterprise resource planning (ERP) applications discussed 267 E1C11 09/16/2010 13:36:56 268 & Page 268 Software Engineering and CMMi in Chapter 10 on auditing IT applications are examples of very comprehensive IT systems and programs. In order to build and maintain these IT systems and programs, application developers generally need better tools to build, maintain, and operate them. Software engineering—the application of engineering concepts to software development—is a useful tool. It is the application of systematic, disciplined, and quantifiable approaches to software development, operation, and maintenance. Software engineering is more than just designing and writing computer programs; it often involves several other IT skills and disciplines, including: & & & & & & & & & Software requirements analysis. This is the process of systems needs analysis, the development of systems and program specifications, and validation of software requirements. Software design. Software often is designed with specialized modeling and development tools, such as the Unified Modeling Language (UML)1. Software development. This is the construction of software through the use of programming languages. Software testing. Before any software is implemented, it needs to be tested in a thorough and comprehensive manner, often using specialized testing tools. Software maintenance. Software systems often have problems and need enhancements long after they are first completed. Tools need to be in place to install revisions in an efficient and comprehensive manner. Software configuration management. Since software systems often are very complex with many related components, their versions and overall configurations have to be managed in a standardized and structured way. Software engineering management. The management of software systems borrows heavily from project management, as discussed in Chapter 16, but software development processes often include nuances not seen in other project management disciplines. Software development processes. There is a wide range of formal approaches for designing and building application systems and their supporting systems; two popular process names are called agile and waterfall software development processes. Software quality assessments. Software engineering includes a variety of tools and approaches to improve the overall quality of all aspects of software development. Some of the software engineering approaches summarized here are discussed briefly in other chapters. For example, Chapter 10 talks about a system development and requirements process called the systems development life cycle (SDLC), and Chapter 7 reviews the set of information technology infrastructure library (ITIL) best practices processes. However, the typical IT auditor does not need to be an expert in software engineering processes. Rather, he or she should understand these general concepts and to apply them when appropriate when reviewing IT internal control processes. E1C11 09/16/2010 13:36:57 Page 269 CMMi: Capability Maturity Model for Integration & 269 CMMI: CAPABILITY MATURITY MODEL FOR INTEGRATION Effective IT development processes and a strong supporting IT function are necessary elements for good software engineering processes. However, over the years, often many IT software development functions were not that effective in building and delivering IT systems that met established requirements following an efficient and on-budget schedule. With frequent late deliveries, chaotic change and revision control processes, and no effective budget controls, IT systems commitments were often consistently missed. Often poor-quality deliveries of the IT systems resulted in too much rework, system functions that did not work correctly, and massive complaints from software product customers—the users—after delivery. These IT application development problems are nothing new. As far back as the late 1960s, a series of well-publicized major U.S. software development projects failed dramatically for being massively late, not meeting specified requirements, or going way over budget. These events were known as the software crisis; a search on Google or other search engines will provide good background information on that software crisis era. The U.S. federal government, and of course its taxpayers, were major victims of this software crisis. Over the years, the U.S. government has been a major contractor for a wide number of IT projects, and it generally purchased or arranged for this work from a variety of contractors. However, government customers frequently found that the contractors selected both missed cost or schedule estimates and delivered poor-quality software products. Faced with continuing software vendor performance frustrations over the years, the federal government entered into a contract with the Software Engineering Institute (SEI) of Carnegie Mellon University (CMU; www.seo.cmu.edu) to develop a better approach and process to measure and assess software development groups. SEI took a design engineering approach to this software development challenge. That is, an engineering team designing a new tool, such as a military weapons system, takes a very structured approach in developing the product. The design steps are tightly controlled and documented with good procedures over such areas as revision control. The well-run design engineering organization will have its processes so well defined and managed that it can easily pick up a new assignment and deliver it in a high-quality manner. SEI contrasted the well-ordered procedures in many design engineering functions with the almost chaotic processes often found with software development. Initially it developed a model, the Capability Maturity Model (CMM), that allows an organization to assess themselves and for others to determine where they stand in their software development maturity. That initial CMM was a model for software development organizational improvement. CMM initially defined what is called software process maturity in terms of five distinct levels of development maturity with a series of processes operating at each of these levels. The CMM model starts with what is called Level 1, where processes are unpredictable, poorly controlled, and incomplete. IT auditors have seen this type of maturity in many enterprises where almost nothing is documented, where systems development controls are weak, and where the software development group is operating in some form of near chaos. E1C11 09/16/2010 13:36:57 270 & Page 270 Software Engineering and CMMi 5. Optimizing Focus on Process Improvement 4. Managed Process Measured and Controlled 3. Defined Process Characterized Fairly Well Understood 2. Repeatable Can Repeat Previously Performed Tasks 1. Initial Unpredictable and Poorly Controlled EXHIBIT 11.1 CMMi Maturity Levels Model The initial CMM said that if a system development organization installs some better key processes, it will improve its operations to move to a next higher level of maturity. That is, an organization can move from the incomplete and unpredictable processes of Level 1 up to Level 2, where processes tend to be repeatable. These five levels of CMM processes are shown in Exhibit 11.1. The sections to come describe each process in greater detail. IT management often is interested in where it stands on these CMM evaluation scales, and internal auditors will find this a useful measure to assess. CMM is a process model; it can be viewed as the glue that ties people and technology together. It represents a collection of best practices and a framework for organizing and prioritizing activities. Since its initial release, the CMM five-level model has been used by many companies. The SEI’s official CMM guidance materials have been updated and expanded to multiple versions for such areas as IT organization services processes (CMMi-SVC) and the software acquisition process (CMMi-ACQ). Today, the model usually is referred to as CMMi to describe its role for IT integration-related processes. This chapter refers to the model as CMMi, although many references in software development literature refer to just the original CMM model. CMMi is a technical process for improved IT software and other process development. Some IT auditors may ask why it is important. However, CMMi concepts are closely tied to the CobiT framework discussed in Chapter 2, ITIL in Chapter 7, and to other internal audit concepts throughout these chapters. On one level, CMMi is a set of software development processes although many business enterprises today do not do E1C11 09/16/2010 13:36:57 Page 271 CMMi: Capability Maturity Model for Integration & 271 that much of their own IT systems development work. However, it is also a very good model to measure the overall processes in place in an IT organization. Many IT functions set their own internal goals to move their organizations up through the levels of CMMi maturity. An IT auditor working with IT organizations should develop an understanding of this very important CMMi model. CMMi Level 1: Incomplete, Unpredictable, and Poorly Controlled Processes An incomplete process is one that either is not performed or is only partially performed. One or more of the specific goals of the process area are not satisfied, and no generic goals exist for this level since there is no reason to institutionalize a partially performed process. In the early days of IT, virtually all systems development efforts and other procedures were run on an ad hoc, seat-of-the-pants basis with very few formal IT development procedures. In those days of mainframe computing, a data processing manager might run into a key user at the water cooler and informally plan some systems change. Rather than using a formal process for requesting new IT projects, the user might ask data processing for a new system or report. Often little was documented. The data processing manager may have made a quick note regarding the request, assigned it to a programmer or analyst, and never discussed the new project until it was complete. The requested report or system may or may not have met the user’s needs; it almost never met the user’s completion expectations. In addition, generally there were no development standards in place. This type of scenario was a characteristic of many early IT functions. There were often no established procedures, very little documentation, minimal revision controls, creating what was often an almost chaotic overall IT environment. This was the era of the mid-1960s going through the 1970s when IT was new with few established standard practices. Seemingly, everyone was trying to accomplish lots of things with limited resources. Over the years, things got better. For example, IBM published a concept it called the systems development life cycle (SDLC) for building new applications. Although not widely adopted at first, this 1960s era standard procedure became the accepted way for designing and developing many new applications. The SDLC terminology and steps have varied over time, and Chapter 10 discusses this application development process from an IT audit perspective. Other procedures, such as revision control, programming standards, and quality assurance standards, were developed later and became widely accepted as the IT application development profession matured. Unfortunately, some IT functions have not matured much. Although today’s new systems are no longer requested and designed over the water cooler, standards and software revision controls often are not good in some IT organizations. These IT functions lack many, if not most, good operating procedures, and systems are developed and operated in an almost chaotic manner. Using CMMi terminology, these are described as Level 1 organizations with incomplete, unpredictable, and poorly controlled processes. The overall philosophy in this type of IT organization is to ‘‘just do it’’ to E1C11 09/16/2010 13:36:57 272 & Page 272 Software Engineering and CMMi produce results rather than going through any level of planning or established development processes. IT organizations today often like to brag that they are CMMi Level 2, Level 3, or higher. While no one wants to admit their IT function is operating at a CMMi Level 1 chaotic state, an IT auditor can often quickly assess when an IT function is operating at such a Level 1. The function will have few standardized procedures, and those that are documented are often not regularly followed. CMMi Level 1 is the initial phase of the overall CMMi model. The term incomplete is appropriate for an IT development group to start at this initial level and improve to higher levels in the overall CMMi model. Each CMMi level is described from a systems development context in terms of (1) the activities performed by the organization, (2) the activities performed by the IT projects, and (3) the resulting process capabilities of the systems development group. These descriptions better describe an IT systems development group at each level of its relative maturity. Level 1 Activities Performed by the Systems Development Organization & & The systems development group lacks sound management practices with decisions often made on a day-to-day basis. Good software engineering and development practices are undermined by ineffective planning and reaction-driven commitments to deliver requested services. Level 1 Activities Performed in Systems Development Projects & & During any type of crisis, project leadership tends to abandon any planned procedures. The lack of sound systems development management practices defeats even strong software engineering development processes. Level 1 Resulting Software Process Capabilities & & Software processes are ad hoc with unpredictable results because development processes constantly are changed or modified as the work progresses. There are few stable software development processes in evidence. IT auditors of another age sometimes loved the CMMi Level 1 type of organization because their audit report findings and recommendations were so easy. It did not take much work for an IT auditor to write up a set of findings and recommendations in this environment. Often these were merely repeated from an earlier audit. The auditor knew things were chaotic, IT management knew something had to change, but the proper corrective action procedures were never implemented. The world today is changing. With much information published about CMMi and its recommended approaches, IT managers and even non-IT senior enterprise managers often ask questions about the quality and maturity of their IT organization. Many enterprises have tried to get around this chaotic, unpredictable IT environment by ending all in-house development and relying on purchased software. This approach, E1C11 09/16/2010 13:36:57 Page 273 CMMi: Capability Maturity Model for Integration Level 1: Just do it. Level 2: Plan before activity, complete process, then evaluate results with objective of future improvements. Activity to produce & 273 Results Planning Activity to produce Results input to Evaluation to improve EXHIBIT 11.2 CMMi Level 1 to Level 2 Differences Source: Robert R. Moeller, Brink’s Modern Internal Auditing, 6th ed. (Hoboken, NJ: John Wiley & Sons, 2005). Copyright ß 2005, John Wiley & Sons. Used with permission of John Wiley & Sons. however, really does not solve the issue. Purchased software packages can eliminate some problems, but those same packages must be implemented and maintained in an orderly manner. Although many IT groups may never rise much above Level 2 or some aspects of Level 3, most organizations do strive to get above Level 1. Through recommendations and advice, internal audit can help an IT organization to make its processes controlled and predictable. CMMi Level 2: Repeatable and Consistent Processes A systems development function should strive to move from the chaotic, unpredictable Level 1 to Level 2. The key defining word for this level is repeatable. Rather than doing things in an ad hoc manner, a systems development organization should begin to establish repeatable operating practices. Exhibit 11.2 shows the differences between CMMi Level 1 and Level 2. Rather than just reacting to crises, the systems development organization should devote more attention to planning its activities and then should evaluate results with an objective of process improvement. Systems development organizations typically do not make an immediate jump from Level 1 to 2 but essentially go through a slow process of adapting various Level 2 processes. The idea with CMMi is to improve systems development processes gradually as an organization becomes more mature. Following the three points raised previously, Level 2 can be described in this way. Level 2 Activities Performed by the Systems Development Organization & & The systems development organization should establish effective software development policies and procedures. Although various specific systems development projects may differ, the systems development function should institutionalize effective project management processes to allow the repetition of successful project management practices developed in earlier projects. E1C11 09/16/2010 13:36:57 274 & Page 274 Software Engineering and CMMi Level 2 Activities Performed in Systems Development Projects & & & Realistic IT project commitments should be made based on previous project results and current requirements. Processes should be in place to track software costs, schedules, and functionality to identify any problems meeting commitments. Control requirements and work products should ensure that project standards are being followed. Level 2 Resulting Software Process Capabilities & & The software development process capability should be disciplined such that there are stable project planning and tracking processes in place and that earlier successes can be repeated. The project management process should be under the effective control of a project management system that follows realistic plans. Level 2 and beyond requires the effective installation of a series of what CMMi calls key process areas (KPAs). These define the detailed processes necessary in each level and to get to Level 2 and beyond, Level 2’s KPAs are: & & & & & & CMMi CMMi CMMi CMMi CMMi CMMi Level Level Level Level Level Level 2 2 2 2 2 2 KPA KPA KPA KPA KPA KPA requirements requirements requirements requirements requirements requirements management processes for software project planning for software tracking and oversight for software quality assurance for software configuration management for software subcontract management CMMi Level 2 KPA Requirements Management Processes This software development process covers both the technical and customer requirements of software about to be developed and implemented. It is not unusual for an application development group to implement a software application without a full understanding of what the application is supposed to accomplish as well as its overall functional specifications. The CMMi requirements management process calls for IT developers to install formal processes defining the requirements of IT development efforts with these KPA objectives: & & & To ensure that the requirements for software products—both new applications being developed and other software tools—are defined and understood To establish and maintain agreement on the requirements with all information services requestors: users, customers, and other interested parties To ensure that the requirements are met Requirements should be documented and controlled to establish a basis for software development and project management use. Changes to requirements E1C11 09/16/2010 13:36:57 Page 275 CMMi: Capability Maturity Model for Integration & 275 should be documented and controlled to ensure that plans, deliverables, and all related activities are consistent with these requirements. Good project management practices are discussed as part of the process description later in this chapter and in Chapter 16. CMMi Level 2 KPA Requirements for Software Project Planning In addition to the project management processes discussed in Chapter 16, the KPAs here call for documented estimates of the size, cost, and schedule for use in planning and tracking all software development projects. All affected groups or individuals involved in a software development effort should receive information on commitments and agreements regarding the project. In addition, the project should follow a formal management process for project planning with adequate tracking and status reporting, including measurements for the completion of milestones. This CMMi project management KPA calls for a much higher level of detailed project planning and tracking than is used by many IT organizations. IT auditors should consider using this KPA as a standard for assessing the progress of IT development when reviewing the management of selected IT projects. Exhibit 11.3 summarizes audit procedures for an IT audit review of IT project management key processes. CMMi Level 2 KPA Requirements for Software Tracking and Oversight The purpose of a formal project tracking process is to monitor a project’s actual progress against its plan. Monitoring is accomplished by collecting significant information about schedule, resources, costs, features, and quality and comparing this information to the 1. Determine that documented estimates are in place for planning and tracking all IT systems development projects. 2. Processes should be in place for documenting activities and commitments surrounding all significant project activities. 3. All affected groups involved in project development processes should provide commitment agreements regarding their projects. 4. Project planning should follow documented and approved organizational project-planning policies. 5. Management processes should be in place to track the status of all project-planning activities. 6. All project costs and activity schedules actual results should be tracked and compared with estimates in project plans. 7. Processes should be in place to initiate corrective actions when results differ significantly from project plans. 8. Project progress should be regularly tracked against the planned schedules, effort, and budgets. 9. Senior management should review project-tracking status on a regular basis. 10. Ongoing programs of corrective actions, based on lessons learned, should be in place. EXHIBIT 11.3 KPA Status Review IT Audit Procedures Source: Robert R. Moeller, Brink’s Modern Internal Auditing, 6th ed. (Hoboken, NJ: John Wiley & Sons, 2005). Copyright ß 2005, John Wiley & Sons. Adapted with permission of John Wiley & Sons. E1C11 09/16/2010 13:36:57 276 & Page 276 Software Engineering and CMMi original and currently approved project plan. The objectives of the CMMi software tracking and oversight KPA are: & & & The process should include the information needed to conduct periodic project planning status meetings and reviews. Project managers and management should be provided with sufficient information to make data-based business decisions. The tracking process should provide information to assist future projects in their estimation and planning efforts. In many respects, the project tracking process is a by-product of the project management review KPA process. CMMi Level 2 KPA Requirements for Software Quality Assurance The purpose of this KPA is to provide management with appropriate visibility into the processes used and the software products being built. This KPA involves reviewing and auditing software products and activities to ensure that they comply with applicable procedures and standards. The objectives of the software quality assurance KPA are: & & & All software quality assurance activities should be planned with their plans reviewed and approved by appropriate levels of management. Software products and activities should adhere to applicable standards, procedures, and requirements. Software quality noncompliance issues that cannot be resolved with a given project should be addressed by senior management. CMMi Level 2 KPA Requirements for Software Configuration Management This KPA relates to establishing and maintaining the integrity of all software process products throughout their life cycle. The scope of configuration management involves identifying and controlling configuration items and units. This is similar to the ITIL configuration management process discussed in Chapter 7. In addition, the scope of this KPA includes systematically controlling all changes to active configurations as well as maintaining their integrity and traceability throughout their software life cycles. The goals of this KPA are: & & & Software configuration activities should be planned. Changes to software work products should be formally identified and controlled. Configuration baselines should be established and affected groups and individuals should be informed of their status and content. CMMi Level 2 KPA Requirements for Software Subcontract Management The purpose of the subcontract management KPA is to select qualified subcontractors and to manage them effectively. The scope of this KPA includes processes for selecting E1C11 09/16/2010 13:36:57 Page 277 CMMi: Capability Maturity Model for Integration & 277 appropriate software subcontractors, establishing commitments with those subcontractors, and tracking and reviewing subcontractor performance and results. An IT organization should have the following KPA goals and activities: & & & Select only qualified software subcontractors. The prime contractor and the subcontractors should formally agree to their commitments and obligations to each other. The prime contractor should track actual results and performance against commitments. CMMi Level 3: Defined and Predictable Processes As we move up the systems development process improvement chain, CMMi Level 3 calls for am IT organization to use lessons learned to provide inputs to the planning and evaluation processes as well as to improve results. Called the CMMi defined level, this improved systems development level has the following characteristics. Level 3 Activities Performed by the Systems Development Organization & & & Strong documentation should be in place for the organization’s standard processes for maintaining and developing software. Processes should be in place to integrate project management and software engineering/systems development activities to exploit effective IT system development. There should be ongoing support for each of the KPAs within this and other levels, including a training program to ensure skills development. Level 3 Activities Performed in Systems Development Projects & & Projects should tailor standard software processes to develop an organization’s own defined project software processes. Because the software process has become well defined, management should have good insights into the technical progress of all projects. Level 3 Resulting Software Process Capabilities & & This software process capability should be standard and consistent because both software engineering and management activities are stable and repeatable. Costs, schedule, and functionality should be under control with the software quality tracked. Although Level 3 here calls for IT systems development activities such as cost and schedule management, a move from Level 2 to Level 3 is often difficult. CMMi Level 3 calls for a software development organization to achieve a much higher level of organization coordination. Few organizations are able to achieve a CMMi Level 3 that contains the next SEI-defined KPAs. E1C11 09/16/2010 13:36:57 278 & Page 278 Software Engineering and CMMi CMMi Level 3 KPA Requirements for an Organizational Process Focus This KPA calls for the organization to raise the overall importance of the software development process. All too often, IT auditors are faced with answers like ‘‘I don’t know; that was an IT management decision’’ when asked why certain application controls are not performing with adequate control considerations. The scope of this KPA involves developing and maintaining an understanding of software development and related project processes throughout the enterprise. To operate at CMMi Level 3, there should be coordination throughout the organization to assess, develop, maintain, and improve these processes. CMMi Level 3 KPA Requirements for an Organizational Process Definition Very similar to the process focus of most KPAs, the purpose of this KPA is to integrate project software engineering and management activities that improve software process performance and provide a basis for cumulative, long-term benefits. The goal is to develop and maintain a standard software development processes where the data surrounding those activities are collected, reviewed, and adjusted for ongoing improvements. As a step beyond the normal software development process, attention should be devoted to ongoing improvements. CMMi Level 3 KPA Requirements for Training Programs and Intergroup Coordination This KPA calls for an ongoing software development training program to enhance the skills and knowledge of individuals so they can perform their roles effectively and efficiently. It also requires disciplined interaction and coordination of all project and software engineering groups within the organization. This Level 3 training KPA should be planned to support training needs of IT projects, the organization, and individuals. This intergroup coordination KPA calls for customer or end-user requirements to be recognized and agreed to by all groups. All related issues should be identified, tracked, and resolved on an intergroup basis. CMMi Level 3 KPA Requirements for Peer Reviews Sometimes difficult to launch, peer reviews involve the methodical and systematic examination of work products by other common or peer-level members of a software development team to identify defects and areas where changes are needed. The goal here is to establish a process that will identify defects in the software development process early and efficiently. CMMi Level 3 KPA Requirements for Integrated Software Management This KPA calls for aggressively integrating the organization’s software engineering and management activities into coherent and defined software processes, tailored to the organization’s software assets. Although defined as a separate CMMi KPA, this really says an organization should give particular attention to software development planning and coordination. E1C11 09/16/2010 13:36:57 Page 279 CMMi: Capability Maturity Model for Integration & 279 CMMi Level 3 KPA Requirements for Software Product Engineering The software development process should consistently perform software development activities as a well-defined engineering process that integrates all software development activities in a manner to produce correct, consistent software products effectively and efficiently. Although CMMi is tailored to the consulting groups or software development type of organization that wishes to release a range of quality products to outside customers, these same concepts should exist for internal IT groups. The goal should be to develop and implement high-quality software in a consistent manner. CMMi Level 4: Managed, Measured, and Controlled Processes CMMi compliance becomes a very difficult challenge to a software development organization as it moves up to Levels 4 and 5. Level 4 calls for an organization to begin predicting the results needed and creating opportunities to get those results. Exhibit 11.4 describes CMMi Level 4 activities where a software development organization should attempt to predict needed and expected results and then to create opportunities to get those results. Level 4 Activities Performed by the Systems Development Organization & & & The systems development organization should set quality goals for both software products and processes. There should be measures of productivity and quality for all important software process activities across all projects as part of an organizational measurement program. The systems development organization should provide a foundation for quantitative systems development evaluations. Level 4 Activities Performed in Systems Development Projects & & Projects should achieve control over their products and processes by narrowing the variation in their process performance to fall within acceptable boundaries. The risks of moving up the learning curve if a new application domain are known and carefully managed. input to Planning Activity Standards to forecast to produce Results input to input to Evaluation to improve EXHIBIT 11.4 CMMi Level 4 Activities Source: Robert R. Moeller, Brink’s Modern Internal Auditing, 6th ed. (Hoboken, NJ: John Wiley & Sons, 2005). Copyright ß 2005, John Wiley & Sons. Adapted with permission of John Wiley & Sons. E1C11 09/16/2010 13:36:57 280 & Page 280 Software Engineering and CMMi Level 4 Resulting Software Process Capabilities & & Software process capability should be predictable because the process is measured and operates within measurable limits. Allowances should be made for predictive trends in process and quality within quantitative bounds to allow for corrective action when any limits are exceeded. Two specific KPA areas have been identified at this level: qualitative process management and software quality management. Their emphasis is to integrate software development activities throughout the organization. CMMi Level 5: Optimizing Processes Level 5 is the maximum, best practices level of CMMi. It is difficult to implement and achieve. When an enterprise has reached Level 5, it is often a case for a press release. Few enterprises have honestly achieved this level to date. Level 5 calls for an enterprise to install self-correcting mechanisms that are implemented in such a way that activities are improved constantly and continuously. The emphasis is very much on defect prevention throughout the organization and the aggressive and active management of changes. Level 5 Activities Performed by the Systems Development Organization & & & The entire organization should be focused on continuous process improvement with the goal of defect prevention. Data should be used for cost-benefit analysis of new technology and new process changes. Innovations in systems development and software engineering practices should be transferred to the entire organization. Level 5 Activities Performed in Systems Development Projects & & Project teams should analyze defects and determine their causes. Project teams should evaluate processes to prevent known types of defects from recurring. Level 5 Resulting Software Process Capabilities & & Software process capability should be improving continuously because the organization improves the range of capability and process performance of its software development projects. Improvements should occur both by incremental advancement of existing processes and by innovations using new technologies and methods. CMMI BENEFITS CMMi had its origins as a tool for improving systems development processes. This chapter has described the original IT systems development CMMi model and its levels E1C11 09/16/2010 13:36:57 Page 281 IT Audit, Internal Control, and CMMi & 281 and KPAs. Many enterprises today are not involved in heavy software development initiatives, and the other CMMi models for services or acquisition may be more appropriate. No matter what the overall enterprise objectives, however, the CMMi model provides a place to start improving processes as an enterprise moves from the unstructured Level 1 to an environment of more refined processes. It is a framework to help an enterprise develop a better shared vision, a common language, and a way to define process improvement throughout an organization. Although CMMi got its start with government contract software developers, it has been implemented by many IT development organizations today. CMMi’s roots are very much tied to the American Society for Quality quality assurance standards discussed in Chapter 31. The SEI maintains CMMi standards and publishes guidance and training materials and process-related standards. As an example of CMMi potential benefits, this author became heavily involved with CMMi several years ago at a large U.S. insurance company where he was working on a contract basis as a project manager. The project involved a large systems implementation involving some new development work as well as extensive modification to existing systems. The company had published extensive IT procedures, but because of a very casual management approach, those procedures often were not followed. For example, this author found breakdowns in such relatively simple but important tasks as software revision control. Two programmers, for example, sometimes attempted to make changes to copies of the same program source code version. The result, of course, would be two incompatible software versions. Although there are other ways to remedy such problems, this author suggested that the insurance company adopt CMMi processes. The effort began with some internal seminars to the IT staff on the CMMi process improvement model, explaining its benefits and outlining that the organization was in a Level 1 chaotic state of operations. With lots of training and guidance, we were able to bring the operating unit from an unstructured Level 1 to a solid Level 2 with some Level 3 KPAs completed during the author’s project period. IT AUDIT, INTERNAL CONTROL, AND CMMI The CMMi model describes a process for improved quality software development for organizations. Although some aspects of CMMi and supporting published materials often seem tailored to the software development organization developing products for external customers, with an emphasis on major government contractors, the core CMMi model is an effective way to think about how an organization is performing and what steps it should take to improve processes. Even though a development group may not be directly focused on CMMi, an IT auditor can use this model to evaluate current performance and to recommend areas for future improvements. What does an organization need to do to claim that it is operating at some CMMi level? The chapter describes some of the specific and important KPAs required to make any such claim. However, an organization must go through some extensive process improvement along with supporting documentation to assert that it is operating at a E1C11 09/16/2010 13:36:57 282 & Page 282 Software Engineering and CMMi specific CMMi level. An organization should contract with certified registrars to review their processes and certify their CMMi level. This is particularly important if an organization needs to document its status for contract purposes. The typical IT auditor will not have the knowledge to appraise whether the IT organization being audited is operating at CMMi Level 2 or 3, or whether it has installed and implemented the correct mix of KPAs. However, an IT auditor should have a general understanding of CMMi, as described in this chapter, and be able to ask appropriate questions if the organization reviewed is operating with CMMi processes. Perhaps most important, when an IT auditor reviews an IT development group with very poor internal controls that are really operating at a CMMi Level 1, he or she might recommend that the IT group look at CMMi as a method to improve processes and develop better internal controls. The effective implementation of CMMi is a good way for an organization to improve its internal controls. Although most organizations will never reach CMMi Level 5, each level points to important processes for improved internal controls. NOTE 1. Unified Modeling Language (UML) is a standardized general-purpose modeling language in the field of software engineering. UML includes a set of graphic notation techniques to create visual models of software-intensive systems. For more information, consult www. uml-forum.com E1C12 09/16/2010 13:39:35 Page 283 12 CHAPTER TWELVE Auditing Service-Oriented Architectures and Record Management Processes A N Y P R O F E S S I O N A L W H O H A S been working with information technology (IT) hardware and software for more than a few years knows that IT technologies and techniques are always changing and evolving. What was a hot new concept just a few years ago sometimes disappears to be replaced by something new and different. In other cases, concepts that once seemed too advanced or even strange evolve into normal accepted practices. Client-server computer system configurations are an example of the latter. What was once a new and advanced concept in the 1990s is now a standard 1 IT process. Managing IT applications through service-oriented architecture (SOA) is another relatively new concept that will soon become part of the standard language of IT. SOA is an IT systems approach where an application’s business logic or individual functions are modularized and presented as services for consumer/client applications. A key concept is that the actual IT services provided are loosely coupled and independent of the actual application implementation. As a result, developers or system integrators can build applications by composing one or more services without knowing the underlying implementations of those services. For example, a service can be implemented in an advanced development language such as what is called dot-NET or Java, and the application consuming the service can be on a different platform or language. This chapter introduces SOA concepts for the IT auditor and discusses internal control and IT auditor issues surrounding the development and operations of IT applications using this technology. SOA is not yet that much of an IT audit professional hot-button issue, with the Websiteofthe InformationSystemsAuditandControl Association(ISACA)onlyoffering one other author’s book on the subject but the current Web site of the Institute of Internal 283 E1C12 09/16/2010 13:39:35 284 & Page 284 Auditing Service-Oriented Architectures and Record Management Processes Auditors (IIA) offers none, although it is a strongly evolving concept for IT applications development and implementations. IT auditors should have a general understanding of the controls and concepts surrounding SOA implementations. The chapter also reviews the importance of effective records management systems in today’s enterprises and IT environments. Today, almost all business records are created and most live their entire lives electronically. Failure to manage electronic records and physical records in accordance with established records management principles is to turn a blind eye to potential risk. This chapter also concludes with an overview of records management issues from an internal controls and IT audit perspective. SERVICE-ORIENTED COMPUTING AND SERVICE-DRIVEN APPLICATIONS Hardware and software vendors, as well as high-level software application developers, often use similar but slightly different terms to describe IT system concepts. We may encounter such expressions as service-oriented architectures, service-oriented design, and object-oriented design when hearing about the attributes of some new IT application. These expressions sound alike and generally reflect similar approaches to the manner in which IT applications are built and launched in our Web-oriented world today. In this chapter, we generally use the expression service-oriented architecture or SOA to describe this concept even though computer science purists may differ on some terminology details. SOA is an IT architectural style whose goal is to achieve loose coupling among interacting software agents. It is an approach that allows interoperability between different IT systems and programming languages, providing the basis for integration between applications on different platforms through messages across network communication links. The software is built following both common and industry-specific components that are granular and modular. A service is a unit of work done by a service provider to achieve desired end results for a service consumer. Software agents play both provider and consumer roles on behalf of their owners. The Web is filled with many often complex and sometimes differing definitions of SOA, and the definition here may be too abstract, but an IT auditor should not consider SOA as a difficult concept. Music CDs and CD players model SOA concepts. If you want to play a CD, for instance, you put your CD into a CD player, and the player broadcasts it for you. The CD player offers a CD music-playing service. When the CD ends, you can replace one music CD for another. You can also play the same music CD on a portable player or on a unit in your automobile. Both offer the same CD-playing service, but the quality of service may be different. The results of the CD music-playing service may result in a change of state for the listening consumer, perhaps a mood change from depressed to happy. We are the service client, the CD player is the service provider, and our music library of CDs acts as service broker. Just as the CD player gives us prerecorded music, we hire someone else to provide services or do the work for us in other areas because they have the available resources and are experts. Consuming a service is usually cheaper and more effective than doing E1C12 09/16/2010 13:39:35 Page 285 Service-Oriented Computing and Service-Driven Applications & 285 Service Provider Service Requester Provider Aggregator Service Aggregator Service Provider EXHIBIT 12.1 Role of the Service Aggregator the work ourselves. Most of us realize that we are not smart enough to be experts in everything. The same rule applies to building software systems. This concept of using the music service of a personal CD player from a library of music CDs leads to SOA concepts. The service is the basic building block of SOA; it is a way of accessing repeatable business capabilities, organized as simple software interfaces available for all providers and consumers. Exhibit 12.1 shows this basic concept of an SOA configuration with a service aggregator. The service requestor requests services from a provider who is an aggregator with a catalog of available service offerings, who will pull the request from any of multiple service providers, who return the service to the requestor and then to the provider. If an SOA is to be effective, we need a clear understanding of the term service. A service is a function that is well defined, self-contained, and does not depend on the context or state of other services. Services are the building blocks of SOA and can be snapped together to make other services or assembled in sequences to make processes. In SOA, services are usually organized in what is called a registry, where separate services can be snapped together to create composite applications and then fitted together into what is called an SOA blueprint. IT auditors generally encounter SOA efforts when performing an overall review of enterprise general controls, as discussed in Chapter 6, or reviewing specific application controls. While our objective is not to make an IT auditor into a computer scientist or software developer, several important SOA concepts that an IT auditor may encounter when discussing SOA processes are discussed next. & & SOA component granularity describes the size of the components that make up a system. An SOA implementation would try to achieve larger, or coarsely grained, components known as business services. These usually are be built out of smaller, or fine-grained, components as well as preexisting technical services. This set of components matters because chunks of larger granularity make SOA services easier for an enterprise to understand, reuse, and manage. Interface versus implementation concepts separate what a service does from how it does it. This issue matters because it simplifies the concept that the business E1C12 09/16/2010 13:39:35 286 & & & Page 286 Auditing Service-Oriented Architectures and Record Management Processes user’s view of SOA should focus on what the service will do rather than the details of how the technology works under the hood (to use an automobile analogy). Contracts define the obligations between the service provider and service consumer or requestor. Obligations can include service expectations such as availability, reliability, and key performance indicators, as well as service cost and support. This definition of obligations matters because it helps business users make rational decisions about which services they can rely on. These concepts are similar to the service-level agreements (SLAs) discussed in Chapter 7. Loose coupling is a way of designing services that are more flexible and less dependent on each other. This method helps services to snap together or couple and recombine. An SOA environment with loose coupling matters because it is faster to assemble business solutions out of premade blocks than it is to write every new business functions from scratch. Adopting SOA in an enterprise is not a one-application-at-a-time process but describes an overall architecture where all IT processes are organized together and connected. Services provided with Web applications where a user just double-clicks to pull up an HTML reference are an example of an SOA concept. Exhibit 12.2 illustrates a hypothetical enterprise-wide SOA configuration. At the lower lever of the exhibit are enterprise data sources and other multiplatform systems. All elements here would be defined as services such that a custom application, for example, might be built of a combination of mainframe/legacy and Web application Enterprise Custom Applications Service Enablement Processes Portals to Enterprise Processes Enterprise Service Bus Service Interface Service Interface Distributed Query Engine Application Adapters Enterprise Data Repositories EXHIBIT 12.2 Enterprise Applications Service Interface Service Interface Web Services WebLogic dot-NET and Others Enterprise-Wide SOA Example Configuration Service Interface Gateway Processors Java Applications Mainframe Legacy Applications E1C12 09/16/2010 13:39:36 Page 287 Service-Oriented Computing and Service-Driven Applications & 287 components and assembled with the requests through a service bus to the proper adaptor and application. When any of those application services need an element of data—another service—a request would go up to the communication bus and back to the appropriate data source. Such an overall application would be built using some of the newer application software development tools, such as Microsoft’s dot-NET1 or Oracle’s BEA WebLogic2 and would be configured and orchestrated in clearly specified processes. An IT auditor does not find a full SOA environment in most enterprises. Rather, an enterprise IT function, with management support, typically implements their SOA environment on a step-by-step basis following an overall blueprint to build that model. With the growing use of Web services surrounding most IT environments and software sold and delivered as individual program components rather than as full applications, IT auditors can expect to see partial and even full SOA implementations going forward. SOA Internal Control Issues and Risks An enterprise does not just move to an SOA IT environment by purchasing a SOA featured software package, training the IT staff, and then assuming they are off and running. SOA is more than just a new way of organizing existing IT systems. Detailed planning and new IT policies and procedures are required to move to that new environment. Even more important, such a transition requires a gradual and very structured implementation where IT processes are brought into an SOA environment in a controlled manner. In some cases, the advantages of SOA operations will be obvious to senior managers and others. For example, an enterprise that has moved a large segment of its applications to a Java or dot-NET Web-based environment often can easily justify the advantages of moving its legacy applications and other processes to an SOA environment. IT auditors can play a strong role in reviewing processes and helping their enterprise management and IT functions transition to SOA processes. Part of doing this requires that an IT auditor understand the preimplementation review processes, as discussed in Chapter 10, be comfortable with auditing Chapter 7’s IT infrastructure controls in an SOA environment, and understand good project management controls, as discussed in Chapter 16. An IT auditor needs to understand the somewhat unique characteristics of SOA and its control issues and risks. This chapter briefly discusses implementing effective SOA processes from an IT audit perspective. The next sections discuss the importance of building a planning blueprint for an SOA implementation, cleaning up existing applications and processes in advance, establishing appropriate SOA policies and procedures, and testing and implementing SOA in an effective manner. Planning and Building an SOA Implementation Blueprint The third letter of SOA stands for architecture, and the building of an effective architecture is an essential early step to launch SOA in an enterprise. Even if the enterprise chief information officer (CIO) has been exposed to SOA concepts or if the enterprise has implemented an SOA-like single application that gained enthusiastic comments and praise, an enterprise still needs to build an overall framework to launch any SOA process implementation. E1C12 09/16/2010 13:39:36 288 & Page 288 Auditing Service-Oriented Architectures and Record Management Processes The key to an effective SOA is to identify an existing plan or build this service set of procedures using blocks that will define the SOA similar to a series of LEGO1 building blocks.3 The blocks of these popular children’s toys can be interconnected in a variety of elaborate structures and then disassembled to build a different structure. However, SOA services are more complex than LEGO blocks that are built in cubical and triangular shapes; SOA services are much broader and generally may have many different dimensions. A key process here is to define the various service elements and the service stakeholders, including the flow of activities and decision points between the stakeholders involved in the process, in this way: & & & & Business owner. The business owner provides the application requirements for a new business capability, solution, or process to be implemented. A business owner or supporting IT staff members should develop process models to make any understanding of IT implementation service requirements easier. The business owner also needs to define nonfunctional requirements, such as expected quality of the service, for the capability, solution, or process. SOA architect. In order to launch an SOA initiative, an enterprise needs to designate a skilled SOA architect to analyze the business requirements and break them into service and process designs. The SOA architect may decide, for example, to reuse existing components rather than create new ones. When such an architect proposes a new or changed services or process implementations, this SOA architect also should deliver design specifications in terms of state diagrams, process models, and interface designs. The SOA architect formalizes the nonfunctional requirements of the component to be implemented, including its availability, security, and performance requirements. Application developer. A developer implements components based on the design specifications delivered by the SOA architect and creates test plans based on these specifications. To aid in the convergence of technology and methodology, an application developer should use parts generated by the SOA architect for implementation, including code generation or model refinements. Quality manager. A team member designated as the quality manager uses the input provided by the business owner, architect, and developer to review the correctness of the service or process that was implemented. This topic is discussed in more detail in Chapter 31. A quality manager then uses the test plans provided by the developer to execute a solution test in a staging environment and validates quality metrics, side effects, and nonfunctional characteristics. Once a key team had been identified, the enterprise should develop an SOA blueprint that indicates a target state or a complete picture of what the SOA implementation should look like when it is completed. The SOA blueprint should outline a complete list of these necessary components: & & & Business services Service description requirements Service performance metrics E1C12 09/16/2010 13:39:36 Page 289 Service-Oriented Computing and Service-Driven Applications & 289 Improve Business Visibility. Integrate systems and aggregate data for a consistent, accurate views of customers, including: & Up-to-the-minute information for improved customer service & Cross-enterprise information for targeted 1:1 activities & Consistent, accurate, and more comprehensive information for better decision making Achieve Business Flexibility. Create an integrated, agile software infrastructure for quickly responding to business needs: & Provide rapid delivery of new business capabilities. & Reduce impact of business and technology changes. & Protect investments while creating new functionality. Gain Business Efficiency. Streamline, automate, and enable the better tracking and visibility of enterprise business processes: & Securely share business processes inside and outside the IT systems firewall. & Bridge silos of data to better ensure data integrity. & Proactively manage business decisions with key performance indicators from other sources. EXHIBIT 12.3 & & & & Service-Oriented Architecture Key Benefits Interoperability standards Data schemas Policies Service discovery and classification requirements Such a blueprint provides an overall outline of where enterprise and general IT management want to go with SOA. Of course, an enterprise should not begin to implement or even approve an SOA strategy unless there is a good understanding of the benefits to be achieved from the approach. Exhibit 12.3 describes some of the key benefits that an enterprise should realize from moving to an SOA environment. Transitioning Current Applications and Processes to SOA The transition of existing processes or applications to an SOA environment, whether older legacy systems or more recent Web-based applications, contains some of the same challenges older professionals went through when we approached the Y2K problem in the late 1990s. Although some may have now forgotten, many COBOL-based programs had system calendar date fields in a yymmdd format and many program decisions based on taking action by sorting on dates. The year 2000 caused those date-based program decisions to be recorded as 0000. The Y2K concern was that many systems would fail because of bad, out-of-sequence program dates, and at the time, many older applications were poorly documented. IT functions and IT auditors went through major efforts to get ready for Y2K, and we face some of these same concerns when reviewing and cleaning up existing processes to transition to SOA operations. The concern with a transition to SOA is not undocumented yymmdd-format COBOL dates; rather, the typical IT function has layers of IT applications and programs, redundant and often mutually inaccessible systems, and much jumbled point-to-point E1C12 09/16/2010 13:39:36 290 & Page 290 Auditing Service-Oriented Architectures and Record Management Processes integrations for its applications. These problems represent major challenges to implementing effective SOA processes. If any team implementing enterprise-wide SOA does not understand their existing IT systems, processes, and organization structures, there is a strong possibility that a poorly executed SOA implementation will fail. SOA Policies and Procedures The overall concept and beauty of SOA is its use of service units that can be assembled and reassembled like LEGO blocks. However, SOA will not work very well for an enterprise if one or another business decides it wants to be bit different and that its service units do not really fit with others. Using our LEGO block example, in this case some unit would want its blocks to be of a size that does not fit with any others. There are often reasons why one or another unit needs to be different (e.g., high security issues). Nevertheless, an enterprise normally needs some type of standardssetting authority to establish policies and set the rules for its SOA services. For many enterprises, governing bodies that create and enforce SOA policies and enterprise standards are typically called the SOA centers of excellence or SOA competency centers. These centers often consist of representatives from each business unit affected by the enterprise’s SOA blueprint and plans. Almost every part of an enterprise SOA blueprint—including which services will be built, how they will be defined, and how they will interoperate—implicitly defines the SOA policies for an enterprise. Because the SOA blueprint typically contains many implicit policies, the first act of a newly constituted SOA competency center should be to ratify this blueprint as a shared goal. Of course, it is important for each affected enterprise group to understand and agree to the implications of the SOA blueprint as it impacts their daily work activities. For that reason, ratifying an SOA blueprint should not be just a rubber-stamp activity. Everyone involved should think through how this SOA vision will affect them. An enterprise should develop compliance procedures to monitor and assess the enforcement of its SOA policies and processes. Doing this establishes rules of the road and provides governance for SOA activities, based on the approved blueprint. Automated enforcement processes, where they can be established, are preferable to manual ones. However, not all policies can be automated, and some steps may require human judgment and intervention. Proper SOA governance covers multiple enforcement points across the entire service life cycle. Exhibit 12.4 describes the roles and responsibilities of key persons involved in implementing SOA and being part of its life cycle. SOA policies and processes usually are divided into two categories: Design-time governance policies should ensure that SOA artifacts meet the design requirements set forth in the SOA blueprint, and runtime governance policies should ensure that SOA services meet the runtime requirements negotiated between service provider and consumer. These SOA governance processes are somewhat like the service-level agreements (SLAs) discussed in Chapter 7, where IT operations formally agrees to provide a user group input transactions according to an agreed-on schedule and IT delivers file updates and completed processes according to the contracted schedule. With SOA, we are making a series of small agreements between service providers and consumers. E1C12 09/16/2010 13:39:36 Page 291 Service-Oriented Computing and Service-Driven Applications & 291 The SOA life cycle describes and implements a flow of activities and decision points between the stakeholders involved in this process. It should recognize these key participants: & Business owner. The business owner provides the requirements for a new business capability, solution, or process to be implemented. The best way to express these requirements is to define a supporting process model covering the service activity. Using process models provides an environment that makes understanding IT implementation requirements easier. The business owner also needs to define nonfunctional requirements (such as quality of service) for the capability, solution, or process. & SOA architect. The SOA architect, often part of the IT team, analyzes the business requirements and breaks them into service and process design elements. The architect may decide to reuse an existing component rather than create a new one, in which case he or she will decide to turn the requirement into an element in the SOA life cycle to reuse the existing component. When a new or changed service or process implementation is identified, the SOA architect delivers design specifications in terms of state diagrams, process models, and interface designs. The SOA architect formalizes the nonfunctional requirements (NFRs) of the component to be implemented including availability, security, and performance. & SOA developer. The developer implements components based on the design specifications delivered by the SOA architect and also creates test plans based on the specifications. To aid the convergence of technology and methodology, the developer uses the parts generated by the SOA architect for implementation, including code generation and model refinement tools. & SOA quality manager. The quality manager uses the input provided by the business owner, architect, and developer to review the correctness of the service or process that was implemented. The quality manager then uses the test plans provided by the developer to execute a solution test in a staging environment and validates quality metrics, side effects, and nonfunctional characteristics. & SOA operator. An SOA operator receives the tested and validated solutions and implements them within the standard IT processes in order for the solution to be made available to solution users and consumers. He or she uses the formalized NFR specifications to operate a virtualized solution that complies with the SLAs required by the consumers. SOA runtime governance solutions provide these kinds of capabilities by enforcing the NFRs and SLAs. EXHIBIT 12.4 SOA Life Cycle Stakeholders SOA Design-Time Policies SOA design-time policies and processes are designed to ensure that services are built to meet the specifications outlined in the SOA blueprint. In particular, policies should be tailored to constrain the behavior of service designers and developers on behalf of the whole SOA effort in these broad areas: & & & Interoperability. An SOA blueprint should declare a uniform method of providing interoperability among services, typically by ratifying a set of standards. Discoverability. Services may require specific attributes such as a businessfriendly description and information regarding the location of the service within an established classification or registry catalog. These elements make it possible to discover services that can be further defined through policy. Security. The SOA blueprint should declare a uniform means to provide security across SOA services. The style and parameters of this security should be consistent with overall enterprise IT security practices, as discussed in Chapter 19. E1C12 09/16/2010 13:39:36 292 & & & & & Page 292 Auditing Service-Oriented Architectures and Record Management Processes Uniqueness. Services should not have the same name as preexisting services. This area often is governed by an SOA attribute or mechanism called a namespace. Policies can help ensure that groups do not run into this problem of duplicate names. Interface compliance. All SOA services should be used or initiated in a uniform way. Although SOA services are more than just IT program elements, this process is similar to a Microsoft Windows RUN command. This standard form of interface should be mandated by policy. Data format compliance. As discussed, a major objective of SOA should be the reusability of its service elements, and a way to endure reusability is to establish common data formats known as schemas. Doing so ensures that an address field in one service can be used properly by another service, even if differences exist in how the services store the data. Common schemas should be mandated by policy. Metrics. Statistical information and reporting of service design issues also should be set by policy. An enterprise, and certainly IT audit, cannot measure the effectiveness of SOA operations unless some metrics have been established as both goals and minimum operating performance standards. Design-time SOA processes should typically connect with the enterprise’s system development life cycle (SDLC). The SDLC process was discussed in Chapter 10, and a similar service development life cycle can be developed here. SOA adoption presents challenges to enterprises that are accustomed to using IT implementations to address application requirements. New structures and processes, often referred to as the SOA life cycle, are required that provide the basis for organizational agility and promote successful SOA adoption. SOA life cycle processes, combined with effective organizational structures, become key elements in launching effective SOA processes. Most modern IT departments are under pressure to deliver cost-effective and timely solutions to their business. To achieve these goals, they use shared technical and organizational components and functions as well as cross-project initiatives to strengthen synergies across departments. When these solutions are combined with an enterprise mind-set to deliver services (as in a valuable service and not as in technology), an IT function can find itself on a path to effective SOA processes. As part of this path to SOA, adoption requires that all parties—IT and management— think in terms of value chains and understand that service is something that exists to deliver consumer satisfaction. Admittedly easier said than effectively implemented, an enterprise needs to break its application-centric thinking by implementing structured processes that cross project boundaries and life cycles. When an open mind-set, methodology, people and organization, and technology are combined successfully, SOA adoption can lead to great benefits for an IT function and the overall enterprise in terms of scale, efficiency, and especially agility. SOA Runtime Policies and Processes Runtime governance policies will reduced political friction in an enterprise because they mostly constrain IT systems on behalf of SOA service consumers. For the most part, E1C12 09/16/2010 13:39:36 Page 293 Service-Oriented Computing and Service-Driven Applications & 293 runtime policies exist to ensure that services behave in the ways they are supposed to, based on expectations of the service consumer. This set of policies and processes includes: & & & & & & & Service-level agreements. SOA providers and consumers should agree on performance expectations as well as on measurements that confirm that the services are performing as expected. Authentication. Providers and consumers also should agree on how service consumers should identify themselves. Based on runtime governance standards, authentication should include such issues as the identity systems and security tokens used. Authorization. Processes should be in place to determine if a provider is allowed to invoke a service. Encryption. There should be encryption standards to keep messages coded or scrambled so they will not be read by the wrong people. Signatures. Digital signature mechanisms should be considered to ensure that messages are sent by valid providers and consumers and are not tampered with during transit. Some of the challenges of digital signatures are discussed in Chapter 17. Alerts and notifications. Conditions should be established to trigger alerts with procedures to send them to proper authorities. Alerts can signify both business and technical conditions. Metrics. Runtime key performance indicators and measurements should be established to drive decisions are set by policy. Runtime policies typically constrain the IT operations team and IT systems on behalf of the service consumer. These processes can include support requests and responses to real-time alerts and notifications. Enabling a more responsive request to changing runtime conditions is an important value in an SOA environment. SOA Policy Enforcement Checkpoints Like the customs checkpoint at a border that checks passports and luggage, SOA governance sets up checkpoints to ensure that agreements between organizations are enforced. These checkpoints should include: & & SOA registry repository. A facility should be in place to serve as the enforcement point for SOA design-time policies and processes. SOA runtime management system. The controlling software system should have facilities to serve as the enforcement point for SOA runtime policies and processes. Virtual services are an ideal place to implement operational requirements or quality of service features, such as: & Message validation. SOA system messages should be well-formed and conform to the expectation of the service interface. & Authentication and authorization. Service messages should be organized to identify the service consumer to ensure that they are authorized to invoke the service and other important operational requirements. & Message encryption and signature operational requirements. Processes should decrypt messages, as required, and verify signatures. E1C12 09/16/2010 13:39:36 294 & Page 294 Auditing Service-Oriented Architectures and Record Management Processes Failover and load balancing. As an important control facility that should be of particular interest to IT auditors, there should be sufficient capacity to support transaction load and service availability. & Message routing. The managing SOA software should have strong facilities for forwarding messages to different service implementations based on the content or context of messages. Monitoring and SLAs. We have emphasized the importance of SLAs in several places in our discussion of SOA processes. Well-defined and monitored SLAs keep an eye on service health and performance and ensure that services are delivered as promised to consumers. & & The SOA requirements mentioned here change far more frequently than just the functional logic of a service. IT auditors should realize that a core mission of SOA governance is to promote and ensure desirable behavior among its participant people and systems. IT management must clearly communicate its SOA policies and then must enforce those policies consistently throughout the SOA life cycle. In the early days of SOA, IT architects would spend weeks or even months painstakingly documenting policies in fat books that no one read. This is similar to the early IT disaster recovery plans discussed in Chapter 23. If getting participants to be aware of these policies was not hard enough, communicating changes to policies was even harder. Manual reviews and approvals had to be put into place to force everyone to read and comply with the latest rules. Such reviews quickly became bottlenecks and encouraged people to go around policies, defeating the core mission of SOA governance. Strong and effective SOA policy management practices are particularly important. Although it really is IT management’s responsibility, IT auditors, through their reviews and audit recommendations, should help ensure that enterprise SOA policies are expressed in declarative formats that can be easily defined, changed, and removed as needed. In addition, processes should be in place to actively enforce these policies throughout the SOA life cycle. Participants here should receive immediate feedback with policy management as a critical component of these governance processes. Effective SOA policy management removes hurdles and objections to SOA governance by providing clear guidance regarding what should be compliant with the established and approved SOA blueprint. Policy management solutions improve accountability and ensure consistent outcomes. IT AUDITING IN SOA ENVIRONMENTS We have discussed many aspects of SOA but only a few of its attributes and features. With its goal of breaking down software elements and functions into independent and interchangeable components that can be connected or redefined easily, more and more SOA-related functions are likely. The major database software vendors today, such as Oracle and HP, have implemented SOA principles in their enterprise resource planning database structures, and we should see a greater SOA emphasis in these products going forward. E1C12 09/16/2010 13:39:36 Page 295 IT Auditing in SOA Environments EXHIBIT 12.5 & 295 Web Services Application Model However, the typical enterprise using SOA tools will almost certainly not begin with the full implementation shown in Exhibit 12.2. Rather, many enterprises have converted one or another key application to a Web services model. That process of converting an existing application into a Web-based and dependent environment is shown in Exhibit 12.5, describing a Web services application model. An example of this type of application can be found in one of the customer relationship management software solutions offered by SalesForce.com (www.salesforce.com/form/sem/crm), a very successful software provider that sells its software products only via the Web. Rather than individual, separate applications, Web services are application components that communicate with other applications using open protocols within the Web and with other applications using standard communications protocols. They are selfcontained, self-describing and can be used by other applications. XML, discussed in Chapter 14 on continuous auditing, and XBRL are used for Web services communications as well as for many SOA implementations. IT auditors will encounter increasing numbers of applications based on Web services approaches in the future. Various resources will be located in what we frequently today call an IT cloud, including, storage connections to multiple files, data sources, and other resources. When considering this cloud concept, we can think of Google; a user can get multiple search results no matter the topic area requested. Web services applications may be linked to this Internet cloud as well as to their own storage resources with ongoing connections to local systems, whether local networks or wireless. E1C12 09/16/2010 13:39:37 296 & Page 296 Auditing Service-Oriented Architectures and Record Management Processes IT auditors face two review areas in such an SOA environment: 1. If an enterprise is moving to a total SOA environment, as depicted in Exhibit 12.2, the IT general controls covering the SOA environment must be audited throughout the SOA project. 2. Whether the application is an element of the enterprise SOA or a separate Web services application, IT audit may need to consider reviewing the application separately. Many of these review procedures follow our discussion of IT general controls reviews in Chapter 6 and application controls review from Chapter 10, but SOA-based systems also have some unique audit characteristics, as discussed next. Auditing SOA Governance General Controls An SOA environment may consist of a largely implemented total system environment or a project in development that is converting some existing or new applications to SOA. Both face some major general controls concerns, and IT auditors who have reviewed general controls following a more traditional IT systems approach will need to rethink their IT general control review approaches. Following our earlier LEGO blocks example, an IT auditor will need to review the controls and procedures for many separate components, including the controls and permissions surrounding each individual component and their interconnections. Perhaps the major general controls issue in this SOA environment is the need for a comprehensive enterprise-level SOA blueprint covering all service elements agreed to and approved by all participating parties. These service unit considerations often go beyond just one operating division, beyond the overall enterprise, and may branch to other service providers or suppliers. A strong system of permissions as well as controls within each service element is needed. Again going back to our LEGO blocks example, each service unit should be considered as one single block, perhaps in a combination of program elements or data resources. A business unit should make the nonproprietary elements of each block available to others through clear disclosures and permissions. Some level of SOA architect function should be in place to review these service elements and establish supporting rules and procedures. Strong infrastructure best practice procedures, as discussed in Chapter 7, are essential in an SOA environment. SOA platforms are not launched by implementing everything all at once. Rather, service and processes typically are added, modified, or deleted on an ongoing basis. The challenge is far more complex than individual program library controls, however, where the concerns focus on one program element in an application. Under SOA, these changes are to a service unit that can be part of numerous other processes, and the owner of a given service may not understand who else is using that service element. A control problem in one service may have an impact on many others. Strong testing and quality controls are necessary. SOA quality resources, as described in Exhibit 12.4 and in Chapter 31, are essential. An IT auditor reviewing SOA general controls should expect to see some strong policy guidance, educational offerings and other materials to introduce users and other developers into these SOA processes. All parties should understand the rules and advantages of operating in an SOA environment, but they also need to understand E1C12 09/16/2010 13:39:37 Page 297 IT Auditing in SOA Environments & 297 the required supporting rules and procedures. An IT auditor should look for strong monitoring processes in place to alert users and developers when SOA rules and procedures have been violated. When appropriate, ideal monitoring processes should be self-correcting, where the user is informed and the matter is fixed. Many of the general IT control procedures discussed in other chapters here are applicable in an SOA environment. For example, controls over systems software, discussed in Chapter 8, or the security management general controls, discussed in Chapter 19, are equally appropriate in an SOA environment. These other controls are appropriate in a variety of environments. Exhibit 12.6 outlines IT general controls in an SOA environment. The steps outline an environment where the enterprise is not 100% 1. Based on discussions with the chief information officer (CIO), has the enterprise developed and approved a SOA strategy for IT and business operations? a. If there is an approved strategy, what is its completion status at present? b. If there is no formal SOA strategy, what are future plans for any SOA implementations? c. Have any education programs been delivered to introduce the advantages of SOA to both business and IT users? 2. Has a responsible manager or coordinator been chosen to lead enterprise SOA efforts? 3. Has the enterprise obtained or is it evaluating any SOA software? a. Have formal evaluation criteria been established to select software? b. For any software product in place, has there been a formal testing and evaluation program to analyze it for full implementation? 4. Have mission-critical business applications been identified as part of the SOA strategy? a. For any critical applications, particularly that are not Web services based, are there plans in place to convert them to an SOA environment? b. Has a formal mission-critical business process layer been defined and documented for SOA applications? c. Is there evidence that the application services layer focuses on integration interoperability, based on database, component, and infrastructure services? 5. Has a formal service director or management team been established? a. Have the objectives and risks of this service director been reviewed with management? b. Is there evidence that enterprise SOA plans have been explained and discussed with members of the management team beyond just IT? 6. Is there evidence that SOA plans have been fully integrated with enterprise continuity plans? 7. As the enterprise moves to an SOA environment, are applications initially converted from their regular status to SOA on a test basis? a. As part of any test conversion to SOA, are existing application controls reviewed and assessed? b. Does any part of an application SOA conversion include testing the component’s plug-andplay features that are the purported benefits of SOA operations? c. Is there evidence that the costs and benefits of any SOA conversion are formally evaluated? 8. Based on testing and evaluations, are all applications converted to SOA consistent with enterprise IT application and system security requirements? 9. On a test basis, have audit procedures been established and understood by both external and internal auditors for any SOA-converted applications? 10. Have the costs and benefits of any SOA conversions been developed and evaluated prior to any full-scale planned conversions? EXHIBIT 12.6 IT General Controls in an SOA Environment E1C12 09/16/2010 13:39:37 298 & Page 298 Auditing Service-Oriented Architectures and Record Management Processes SOA consistent but is moving there through an ongoing implementation project that follows an approved blueprint. Auditing Web Services Applications Just as auditing general controls in an SOA environment is very similar to general controls IT audit reviews in many other modern IT systems, internal controls reviews of individual Web services applications have many of the same aspects as other IT application control reviews. However, an IT auditor should understand the unique internal controls characteristics of Web services applications. For background, the Web services architecture contains three components, as shown in Exhibit 12.7: the service requester, service broker, and service provider. A service requester requests a Web service through a service broker to the service provider. The service provider then can directly request for another service to fulfill this service request from any other service provider and thereby create a chain of such additional requests. Such a provider functions as an intermediary during the process. When applications with different access levels are invoked at runtime, they run the risk of weakening existing security levels. Unlike a portal where just minimum access rights are implemented to afford easy security management, a Web services scenario can be effective only if a certain amount of access rights have been given with an appropriate level of access rights; these rights are often negotiated dynamically. Many of the internal controls associated with a Web services application are similar to those in any other IT application, as discussed in Chapters 10 and 11. For example, Web services application security controls for verifying users through authentication controls, having authorization controls access certain application functions, and the maintenance of user data and security policy and audit trail mechanisms are similar to normal IT applications. However, because of the organization of Web services architecture, security management controls can be an area of significant internal control vulnerabilities. EXHIBIT 12.7 Web Services Architecture Components E1C12 09/16/2010 13:39:38 Page 299 IT Auditing in SOA Environments & 299 Web services security management increases internal control risks because of its loosely coupled and often ambiguous service unit environment, along with a number of intermediary links, where a service function can jump across related processes. The number of elements in a Web services application can multiply rapidly; security management, therefore, becomes complex. While reviewing the general controls in an enterprise’s SOA environment is a relatively easy IT auditing task, with its emphasis on appropriate policies and strong infrastructure controls, reviews of individual Web services applications often can be complex. Any application can have multiple messages and transactions back and forth through its individual service units and other Web services components. Such an application can perform and operate with many different variations. Fully reviewing, understanding, and testing a Web services application often requires the development of continuous audit procedures, as discussed in Chapter 14. Exhibit 12.8 outlines IT audit 1. Develop and document a general understanding of the Web services application, not whether it is fully configured in an HTML format. a. Identify and document essential Web services components, including providers and Web services consumers. b. Develop an understanding of Web services standards, such as the use and definition of XML-based Web Services Definition Language (WSDL) to enable customers to invoke commands, and assess the use of these features by IT application developers. 2. Review the project management documentation covering the Web services application to determine if it is consistent with good control procedures. 3. Interview a selected sample of application users to assess whether they are familiar with this Web services application, including its portability features. 4. If the Web services application reviewed is one of several applications using the same vendor software, review any control weaknesses identified in other similar application reviews and determine that they have been addressed in this application review. 5. Based on discussions with IT developers, assess whether the application follows application and industry standards such that multiple partners can be added to the application’s Web services. a. Review test results of any patching of other customer Web partners into the services framework. b. Determine that proper attention is given to application internal controls whenever patched into the application’s Web services. 6. Review application security sign-on procedures, and determine whether they are consistent with industry and enterprise standards. a. Understand the security procedures in place for multiple customer Web partners. b. Based on discussions with the IT security functions, assess whether adequate protective measures have been installed for the Web services applications. 7. Assess the adequacy of logging tools installed for the application, including time-stamped facilities and the adequacy of audit trails for both system users and IT audit. 8. Determine that there are adequate change control procedures over the entire Web services application. 9. Assess whether Web services application development efforts are consentient with good systems development life cycle (SDLC) procedures. 10. Interview several Web services application users to determine if application performance is meeting their expectations. EXHIBIT 12.8 Review Procedures of a Web Service Application E1C12 09/16/2010 13:39:38 300 & Page 300 Auditing Service-Oriented Architectures and Record Management Processes review procedures for an individual Web services application. These steps will help the IT auditor gain a general understanding, of internal control processes, but he or she will need to develop a more detailed understanding of the individual application in order to fully test and review it. Eventually, the SOA environment for IT operations and Web services applications will almost certainly become or define IT standards. The concept of approaching IT services and processes as multiple building blocks that can be reused and reassembled in multiple configurations is complex but will raise questions and raise eyebrows by many in enterprise and IT management who may not initially fully understand and trust these new processes. IT auditors should build their knowledge of SOA and Web services IT processes. ELECTRONIC RECORDS MANAGEMENT INTERNAL CONTROL ISSUES AND RISKS Today all levels of enterprises are literally awash with electronic records—receipts for sales transactions, purchase orders and specifications for purchased materials, employee pay and personnel records, and much more. What were once paper-based records— sometimes even with carbon copies—have almost universally been replaced with IT electronic records. Although facilities usually exist to produce paper copies of electronic documents (e-documents), the IT-based versions are almost always the prime document repositories today. All too often, enterprises do not manage their supporting e-documents consistently. Backup policies and e-document retention periods are often established on an application-by-application basis with no consistent understanding of document retention needs. In some instances, document backup records are not kept long enough to allow an enterprise to reconstruct transactions. In other instances, e-documents are retained far too long. With IT storage media very cheap today, it is easy to keep e-documents almost in perpetuity. Although it is good to have these historical records, sometimes there are good reasons to destroy records after some legally defined maximum periods. E-documents also must be organized in a manner that they can be retrieved when necessary. We have all seen changes in IT technology where storage media formats and supporting equipment changes. As time passes, it does not make sense for an organization to retain readers or other equipment to retrieve that old stored data; even finding an outside source to retrieve the materials may be difficult. The importance of document management can be seen in a recent American Institute of Certified Public Accountants (AICPA) top 10 technologies survey. In 2009, this list contained two items, numbers 9 and 10, very pertinent to IT document management: 9. Document, Forms, Content and Knowledge Management. The ‘‘paperless’’ office is the process of electronically capturing, indexing, storing, protecting, searching, retrieving, managing, and controlling information using scanning, forms recognition, optical character recognition (OCR), centralized data repositories, and the management of PDFs and other document formats. Knowledge management brings structure and control to E1C12 09/16/2010 13:39:38 Page 301 IT Audits of Electronic Records Management Processes & 301 information, allowing organizations to harness the intellectual capital contained in the underlying data. 10. Electronic Data Retention Strategies. Involve technologies that enable appropriate archiving and retrieval of key information over a given (statutory) period of time. Strategies include policies and processes to ensure destruction of information from storage and archival media in a timely and consistent manner, as well as the impact of eDiscovery rules and regulations regarding retained data.4 This AICPA list changes from year to year; 2008 had similar entries. The point of this is that an enterprise needs a set of e-document standards covering retention periods, record formats, and backup procedures for all of its business records. These targeted items do not tell an external auditor how to audit internal controls in these areas but highlight evolving areas that should be of concern in any review of internal controls. Electronic records management became a major issue just prior to the enactment of the Sarbanes-Oxley Act (SOx), discussed in Chapter 1. In the late 1990s, a major motivator for the enactment of SOx was the then high-flying trading company Enron. Questions also were raised about Enron’s financial soundness in the early years of this century. When the U.S. Securities and Exchange Commission announced that it was going to review Enron’s financial operations, Enron’s public accounting firm, Arthur Andersen, announced that all ‘‘nonessential’’ documentation should be destroyed, per that firm’s policy. The press was then treated to a major effort by Enron’s accounting firm, shredding massive amounts of documentation on a worldwide basis and in what seemed like a destruction of documentary evidence. SOx now has rules about retaining financial documentation, and those rules have been a strong motivator in encouraging enterprises to adopt e-document management processes. IT AUDITS OF ELECTRONIC RECORDS MANAGEMENT PROCESSES Appropriate electronic records management processes should be part of any IT application review and also should be compliant with SOx standards. Although an IT auditor should assess whether records retention controls are appropriate for any applications reviewed, it is useful to review the overall electronic records management process in an enterprise. This general controls review is both IT based, with a reliance on storage management controls, and business process based, with standards to encourage the proper identification and archiving of electronic records. In cooperation with their financial and operational internal auditors, IT audit periodically should review an enterprise’s overall electronic management processes. This process is much easier today than it was before the turn of this century. The reason is that mass storage memory devices have become so low in cost and small in size that it is not unusual for a laptop computer to be equipped with a USB device plugged into the computer capable of holding several gigabytes5 of mass storage capacity. Of course, our needs for increased memory have grown massively. Earlier documents primarily consisted of text and numbers, while many documents today are based on drawings, E1C12 09/16/2010 13:39:38 302 & Page 302 Auditing Service-Oriented Architectures and Record Management Processes elaborate graphics, and digital photographs. The IT storage requirements for even a small digital picture typically require many, many bytes or words of data storage needs when compared to a text based document. It is relatively easy to modify IT systems such that key records are backed up and stored in an IT data retention facility. Another key aspect of electronic records retention is that some records should be converted from paper to electronic formats. While it is relatively easy to store such documents electronically, a clear process is needed to retrieve them easily. Anyone who has done a Google search and has received multiple retrieval responses, none of which is quite right, knows this problem. In addition to ease document retrieval, e- documents of all types should be part of an enterprise disaster recovery and continuity plan, as discussed in Chapter 23. It is relatively easy to back up key files and records from an IT application so that they can be retrieved as part of continuity processes; often it is more difficult to recover all types and formats of e-documents. As part of their overall risk management processes, IT and others in the internal audit organization should perform periodic general control reviews of enterprise e-document storage and retrieval processes. Exhibit 12.9 outlines IT audit procedures for a review of e-document records management controls. If IT audit finds, through reviews of processes and tests, that the records management internal controls appear 1. 2. 3. 4. 5. 6. 7. 8. Does the enterprise have a formal document management system in place? a. Does document management include both electronic and paper-based records? b. Is enterprise document management supported by procedures governing such matters as retention, retrieval tests, and document reference libraries? c. Are there appropriate, well-controlled tools in place for managing electronic documents? Is there a program in place for moving existing paper-format document processes and their documents to electronic formats? For all e-document conversion processes: a. Are logs maintained documenting conversion processes? b. Are databases retaining converted documents subject to strong backup and security controls? c. Are catalogs published describing backup contents? d. Are there adequate security controls over access to e-document repositories? Are e-document files part of enterprise business continuity processes, and do they appear to be adequate? Are software management processes in place to ensure the continued availability of stored edocuments in light of any database software or other changes? On a test basis, can IT audit easily retrieve a sample of older e-documents? Are there processes in place to remove or delete older shared e-documents? a. Are document deletion processes consistent with the document’s legal requirements? b. Has the document retention policy been reviewed with and received approval from the audit committee? Are e-document processes reviewed by IT management on a regular basis with the objective of adding continuous improvements? EXHIBIT 12.9 Controls Procedures for a Review of Electronic Records Document Management E1C12 09/16/2010 13:39:38 Page 303 Notes & 303 adequate, internal audit can rely on these processes for subsequent individual application reviews. NOTES 1. The .NET or dot-NET framework is Microsoft’s comprehensive and consistent programming model for building applications. See www.microsoft.com/net/. 2. BEA is a unit of Oracle Corporation. Information on its WebLogic application can be found at www.oracle.com/bea/index.html. 3. While this is more of a marketing site, information on LEGO blocks can be found at: http://shop.lego.com/ByTheme/Leaf.aspx?cn¼392&d¼104. 4. AICPA Information Technology Center, 2009 Top Technology Initiatives and Honorable Mentions//infotech.aicpa.org/Resources/TopþTechnologyþInitiatives/ 2009þTopþTechnologyþInitiativesþandþHonorableþMentions.htm. 5. A gigabyte is a unit of computer memory or data storage capacity equal to 1,024 megabytes (230 bytes) of storage capacity. A byte is effectively one character or number. E1C13 09/16/2010 16:11:39 Page 304 13 CHAPTER THIRTEEN Computer-Assisted Audit Tools and Techniques I N F O R M A T I O N T E C H N O L O G Y ( I T ) A U D I T O R S — O F T E N working with their financial and operational internal audit counterparts—gather evidence from an enterprise’s books and records to support their conclusions. This audit evidence includes any actual paper-based documents, evidence that these documents or supporting transactions were properly recorded in a timely manner, and appropriate authorizing signatures or notations. Today, most of those documents are IT based and paperless, and IT auditors have a challenge to review and understand those paperless documents and procedures to support their audit conclusions when older traditional paper-based documents have gone away. Although IT auditors test and review the internal controls surrounding those IT systems, they often need tools to better understand and evaluate the completeness and accuracy of the data stored in the files and databases of IT applications. It is almost always more efficient to use IT techniques to examine all recorded items on the supporting computer files. IT auditors can also act with greater independence by developing their own specialized file retrieval methods. There are many approaches to retrieving data through computer-assisted audit tools and techniques (CAATTs), which are independent auditor–controlled software to assist internal audit efforts. An IT auditor must obtain evidence on the validity of accounting and operational data. However, large volumes of data or the lack of paper documents often makes this review of audit evidential materials difficult or almost impossible. This chapter describes IT audit approaches to testing, analyzing, and gathering detailed evidence from data contained on IT applications through the use of CAATTs controlled by IT auditors. These techniques allow an IT auditor to review the contents of computerized applications data in files, ranging from accounting systems on large database repositories to smaller systems residing on departmental desktop systems. Although some CAATTs 304 E1C13 09/16/2010 16:11:39 Page 305 Understanding Computer-Assisted Audit Tools and Techniques & 305 require specialized data processing skills, there are many powerful audit software tools that the typical IT auditor, with no particular programming skills, can use. There are many tools and techniques available to help make audit reviews of IT-supported systems more efficient and effective. All IT auditors should have a basic understanding of the general use of CAATTs to access and review automated data to support IT audits. If an IT auditor does not have the skills to execute a particular CAATT process, he or she should have a sufficient level of knowledge to describe it and request help from another member of the IT audit team. UNDERSTANDING COMPUTER-ASSISTED AUDIT TOOLS AND TECHNIQUES A CAATT is a specialized computer program or process, controlled by IT or other auditors, that is used to test or otherwise analyze data on computer files. Terminologies change over time, and an IT auditor sometimes will see the term CAAT or CAAP rather than CAATT; the first refers to just ‘‘techniques’’ and the second is an abbreviation for ‘‘procedure.’’ All of these expressions refer to similar techniques and can be used interchangeably. The American Institute of Certified Public Accountants (AICPA) uses CAATTs, the term preferred for this chapter, although the Information Systems Audit and Control Association (ISACA) and the Institute of Internal Auditors (IIA) still uses the name CAAT. In the early years of data processing systems, auditors typically relied on printed outputs from IT systems and used conventional audit procedures to read, test, and analyze these computer-generated reports. As IT systems became more pervasive with ever larger data files, auditors needed better approaches to evaluate the documentation and records stored in large IT systems and files. In the early days, a few pioneer IT auditors developed CAATTs to read and analyze financial data. However, many auditors continued to use conventional manual techniques, relying primarily on the printed report results of IT systems. The necessity for CAATT procedures first became evident with the Equity Funding fraud in the early 1970s. Equity Funding Corporation, a U.S. insurance company, was reporting very significant growth and earnings from the late 1960s up through the early 1970s. It was later determined, however, that Equity Funding’s growth and earnings were based on a massive management fraud in which fictitious insurance policies were entered on the company’s computerized records. At that time, the external auditors relied on the printed report outputs generated by the Equity Funding computer systems rather than on the data recorded on their computer files. Had the external auditors looked at the contents of those computer files, they might have detected the fraud. Equity Funding did not have a significant internal audit function, and an Equity Funding employee and then their external auditors eventually revealed the fraud. Equity Funding’s external auditors then independently reviewed computer procedures to analyze the contents of computerized records. The Equity Funding fraud launched what was then called computer auditing—now IT auditing—and the use of CAATTs. E1C13 09/16/2010 16:11:39 306 & Page 306 Computer-Assisted Audit Tools and Techniques A CAATT is an auditor-controlled computer program or process that can be used to read production IT files to analyze, summarize that data, and perform other audit tests. In the days of legacy mainframe computer systems and before today’s powerful desktop software tools, a CAATT often was considered to be an advanced audit technique. End users typically relied on their data processing departments to write special retrieval programs to give them various requested output reports. Later both internal and external auditors began to use what was called generalized audit software to develop their own programs independently for testing and analyzing data. This generalized software became the basis for CAATTs, a term used to define specialized IT audit systems and procedures. An example might better clarify the concept of a typical CAATT. Assume that IT audit is interested in testing the accuracy of account agings from an automated accounts receivable system; however, most calculated data for that system is stored only on computer files, with no significant paper reports describing these calculations. Financial and IT auditors are concerned that the receivables, as reported on the aged trial balance report, may not be properly aged as to the number of days due. Thus, the receivables account balances may be over- or understated. IT audit can test these agings using any of three approaches. First, IT audit could use traditional, manual approaches where items are selected from an IT output report and then are traced back to any original source documents that may exist. IT audit can then determine if the items selected are entered properly on IT system records and if the aging calculations are correct. This method will work if paper records are available. However, because of the volume of receivable records in typical IT systems, IT audit can trace and test these items only selectively. Some exception conditions may be missed with such a manual test. In addition, IT audit might not be able to determine easily if the dates of transaction-based agings are functioning correctly. A second approach is to perform an internal controls review over the automated accounts receivable system. The idea is that if internal controls over the application are found to be good, IT audit can rely on system output reports. IT application internal controls reviews are discussed in Chapter 10. A review of systems documentation will determine whether the system is properly aging receivables. IT audit would then test those controls by, for example, running some test transaction into the system, either through manual transactions or another CAATT. Properly performed, this review can detect significant internal control problems as well as determine whether the system is generally working in a correct, well-controlled manner. However, IT audit would be able only to estimate the total extent of the financial statement adjustments necessary due to any account aging errors. Thus, in conjunction with this test, IT audit must determine that controls over data entry and error correction are adequate. A third and perhaps better approach is to use a CAATT application to recalculate independently all of the agings in the accounts receivable system, develop totals for the accounts receivable balance, and produce a listing of any unusual exception items. IT audit might perform this CAATT-oriented approach in five steps: 1. Determine CAATT objectives. IT audit should never just ‘‘use the computer’’ to test a system without a clear set of starting audit objectives for any CAATT. In the E1C13 09/16/2010 16:11:39 Page 307 Understanding Computer-Assisted Audit Tools and Techniques 2. 3. 4. 5. & 307 previous examples, IT audit would have an objective of determining if accounts receivable agings are correctly stated. Understand the supporting IT systems. IT audit should review IT systems documentation to determine how accounts receivable agings are calculated, where this data is stored in the system, and how items are described in system files. Develop CAATT programs. Using generalized audit software, other retrieval packages, or a computer language processor, IT audit would write its own programs to recalculate accounts receivable agings and to generate totals from accounts receivable files. Test and process the CAATT. After testing the programs, the IT auditor would arrange to have the CAATTs processed against production accounts receivable files. Develop audit conclusions from CAATT results. Similar to any audit test, audit conclusions would be drawn from the results of the CAATT processing, documented in the workpapers and discussed in the audit report, as appropriate. This is the general approach to developing and processing CAATTs. It follows the same steps IT audit would use for establishing audit objectives and performing appropriate tests on any system or process. As discussed, a CAATT is a specialized set of computer programs or procedures that are under the control of IT audit. The CAATT can be developed through generalized audit software programs run on the production computer system, specialized software run on the auditor’s own laptop computer, or specialized auditor-use-only program code embedded in an otherwise normal production application. With our major reliance on IT processes in all areas of an enterprise today, CAATTs can enhance IT audit processes in some of these areas: & & & & & & Increase audit coverage. CAATTs can allow an IT auditor to review and analyze such components as massive financial databases where IT auditors do not have easy access to online screen reports and certainly not paper reports. Focus on risk areas. Similar to the last point and our example of testing accounts receivable agings, CAATTs often allow an IT auditor to review and investigate areas that have not received a high level of IT audit scrutiny. Increase cost effectiveness. Although CAATTs may require some incremental time and cost to develop, they can be very effective for analyzing large volumes of IT-resident data over multiple periods. Improve audit credibility. CAATTs provide IT auditors with the ability to independently look at complex databases and provide detailed analyses and recommendations; that type of analysis can very much improve IT auditor credibility. Improve coordination of both IT and financial and operational auditors. CAATTs often are used to analyze financial and operational processes using IT processes. They will cause both IT and other internal auditors to better talk and coordinate their audit objectives and needs. Encourage auditor independence from IT operations. IT auditors do not have to be heavily dependent on the IT systems and infrastructure to operate their CAATTs. Although strong coordination is essential, IT auditors can operate in a fairly independent manner. E1C13 09/16/2010 16:11:39 308 & Page 308 Computer-Assisted Audit Tools and Techniques IT auditors should have a good understanding of when CAATTs should be used to enhance the audit process, the types of software tools available to an IT auditor, and how to use a CAATT in an audit. Although some CAATTs require an IT auditor to have specialized programming knowledge, most can be implemented by any internal auditor with only a general understanding of information systems. DETERMINING THE NEED FOR CAATTS CAATTs are powerful tools that can enhance both the audit process and auditor independence. However, these procedures sometimes can be time consuming to develop and will not always be cost effective unless properly planned and designed. IT audit needs to understand when a CAATT might increase overall audit efficiencies and when it will not. This section discusses areas where CAATTs will enhance an audit and areas to consider when developing and implementing a CAATT. Other sections discuss alternative CAATT approaches and procedures for implementing them as well as some problems with this approach. Before developing a specific CAATT, an IT auditor should first determine if the planned approach is appropriate. All too often, a member of management or even the chief audit officer may have attended a seminar about audit efficiencies and then asks the IT audit team to ‘‘do something’’ to improve audit efficiency by using IT resources as part of IT audits. This was particularly true several years ago, when management expressed strong concerns about all levels of audit costs associated with Sarbanes-Oxley Act (SOx) Section 404 reviews, as discussed in Chapter 1. This type of improved audit efficiency directive often results in disappointments for all parties. Similarly, a highly technical IT auditor sometimes develops a ‘‘technically interesting’’ CAATT as part of an audit even though it really does not support the overall objectives of that review. The result may be interesting but will not contribute to the overall effectiveness of the IT audit’s objectives. The decision to develop and implement a CAATT in support of an IT audit will depend on the nature of the data and production programs being reviewed in the audit, the CAATT tools available to IT audit, and the objectives of the audit. IT audit needs an overall understanding of CAATT procedures in order to make this decision, and should consider: & & Audit nature or objectives. IT audit initially should evaluate the materials to be reviewed in a planned audit and consider the size and format of any IT-based data. Audits based on values or attributes of computerized data are typically good candidates for CAATTs. For example, the above-mentioned accounts receivable audit is a good CAATT candidate because there is generally a large volume of transactions but few paper records. Many of the operational and financial audit areas discussed throughout this book are also good candidates for CAATTs. Nature of the data to be reviewed. CAATTs are most effective when both data and decision-dependent information about that data are based on automated systems. For example, a manufacturing inventory system will have most of the descriptive information about its inventory on IT system databases or files. Inventoryrelated data is input directly, and inventory status information is based on system E1C13 09/16/2010 16:11:39 Page 309 Determining the Need for CAATTs & & 309 reports on output screens. Often only limited paper-based original records exist. IT audit procedures for inventory here might include an analysis of manufacturing costs, and inventory system attributes can be summarized and analyzed through a CAATT. Other computer systems are comprised of little more than log files that organize otherwise manual records. An engineering project authorization system might have summary data stored in a systems file, but most of the information about the projects may be in manual, paper-based files. CAATTs might not be very effective in these areas because IT audit also would need to review the manual data. Only audits over areas where there is heavy dependence on IT data are good potential candidates for a CAATT. Available CAATT tools and audit skills. IT audit must develop its CAATTs using the automation tools available within the audit department or IT function. If IT audit does not have or has not budgeted for specialized CAATT software, an IT auditor cannot develop CAATTs that require such software. IT audit needs to consider the types of audit software available before embarking on any CAATT projects. That availability may be based on both audit budget constraints and product limitations. Auditor skills also must be considered. Although training materials are available, the in-charge auditor must assess whether technical audit specialists are needed and are available for the CAATT development project. The last three points are stated in very general terms, but they are areas to be considered when planning the overall strategy for using CAATTs. These comments point to many areas where a CAATT will be difficult or not particularly cost effective. However, IT audit should keep an open mind and always consider using CAATTs to enhance IT audit effectiveness. Given the lack of paper-based audit trails in many IT systems, an IT auditor has little choice but to use computer-assisted audit procedures. The challenge to IT audit is to identify appropriate areas for CAATTs. Computer technology has changed extensively over the years. The batch-oriented systems of the past have been replaced by online, database-oriented systems. Large centralized computer hardware has been replaced, in many respects, by networked client-server workstations. Despite these changes, however, the auditor’s basic approach for defining CAATTs has not really changed. For example, in 1979, the AICPA published an audit guide, Computer-Assisted Audit Techniques that provided some basic direction on the use of CAATTs. Although now long out of date and out of print, that guide contained a list of the types of audit procedures that can be performed through the use of CAATTs. Adapted for IT auditors, this set of procedures includes: & Examining records based on criteria specified by IT audit. Because the records in a manual system are visible, IT audit can scan for inconsistencies or inaccuracies without difficulty. For records in systems files, IT audit can specify audit software instructions to scan and print records that are exceptions to the criteria, so that follow-up actions can be taken. Examples of specified areas are: & Accounts receivable balances for amounts over the credit limit & Inventory quantities for negative and unreasonably large balances E1C13 09/16/2010 16:11:39 310 & Page 310 Computer-Assisted Audit Tools and Techniques Payroll files for terminated employees Bank demand deposit files for unusually large deposits or withdrawals Testing calculations and making computations. IT audit can use software to perform quantitative analyses to evaluate the reasonableness of auditee representations. Such analyses might be for: & Extensions of inventory items & Depreciation amounts & Accuracy of sales discounts & Interest calculations & Employees’ net pay computations Comparing data on separate files. When records on separate files should contain compatible information, software can determine if the information agrees. Comparisons could be: & Changes in accounts receivable balances between two dates, comparing the details of sales and cash receipts on transaction files & Payroll details with personnel files & Current and prior period inventory files to assist in reviewing for obsolete or slow-moving items Selecting and printing audit samples. Multiple criteria may be used for selection, such as a judgmental sample of high-dollar and old items and a random sample of all other items, which can be printed in the auditor’s workpaper format or on special confirmation forms. Examples are: & Accounts receivables balances for confirmations & Inventory items for observations & Fixed-asset additions for vouching & Paid voucher records for review of expenses & Vendor records for accounts payable confirmations Summarizing and resequencing data and performing analyses. Audit software can reformat and aggregate data in a variety of ways to simulate processing or to determine the reasonableness of output results. Examples are: & Totaling transactions on an account file & Testing accounts receivables aging & Preparing general ledger trial balances & Summarizing inventory turnover statistics for obsolescence analysis & Resequencing inventory items by location to facilitate physical observations Comparing data obtained through other audit procedures with IT system data files. Audit evidence gathered manually can be converted to a machinereadable form and compared to other data files. Examples are: & Inventory test counts with perpetual records & Creditor statements with accounts payable files & & & & & & & Although many of these procedures originally were developed by external auditors before integrated database files existed, these techniques are generally applicable for today’s IT auditors. The number and sophistication of CAATTs will increase as the individual IT auditor becomes more experienced in their use. E1C13 09/16/2010 16:11:39 Page 311 CAATT Software Tools & 311 CAATT SOFTWARE TOOLS In the early days of computer systems, IT users often had to submit a request to the programming department for any type of special report or analysis program. Writing computer programs was often a difficult and time-consuming process performed by IT specialists. IT auditors were often suspicious of that approach. Just as an auditor who was interested in the account balance for some population of manual records would not ask the auditee for the balance but would examine the records to calculate an auditordeveloped total, auditors often preferred to use their own programs controlled by IT audit to analyze computer-based data. This led to the development of CAATT processes with several common categories or types of computer audit software: & & & & & & Generalized audit software products Report generator languages Test data techniques Specialized audit test and analysis software Expert systems and inference-based software Embedded audit procedures Depending on the overall IT environment and IT audit’s objectives, one or more of these CAATTs may be used in a given audit situation. Some such expert systems and inference-based software require specialized IT audit technical skills, but most can be implemented by the generalist IT auditor. In today’s highly computerized environments, any effective IT audit department should use some type of audit software. With technology advances, some approaches that were common in the past are seldom used or available today. Exhibit 13.1 contains general guidelines for developing a CAATT. The exhibit is based on ISACA guideline G3 that discusses developing CAATs, modified to make the materials more consistent with other chapters in this book. ISACA audit standards and guidelines are discussed in Chapter 3 along with references to supporting ISACA Web sites. The next sections describe various types of CAATTs. All are IT auditor–based software routines to aid the application review and IT audit process. IT auditors should use these tools to assess internal controls in their application reviews. Often more important, an IT auditor should use CAATTs to support the objectives of the financial and operational internal auditors in an enterprise. Although we believe that all internal auditors should have the skills to develop their own CAATT applications, far too many non–IT auditors are reluctant to do so. Therefore, IT auditors should work with their financial and operational audit teams to develop effective CAATTs. The next paragraphs outline some basic CAATT procedures. Generalized Audit Software In the early days of IT auditing, most business applications were written in programming languages such as COBOL. Auditors at that time usually had neither the technical skills nor the time required to write their own retrieval programs to access data. When an IT auditor wanted to test or review the contents of a large data file written with a E1C13 09/16/2010 16:11:39 312 & Page 312 Computer-Assisted Audit Tools and Techniques Note: This Guideline is based heavily on the ISACA Auditing Standard Guideline G3 on the use of computer-assisted audit techniques. We have made minor editing changes, such as our preferred use of CAATT rather than CAAT; have slightly modified some sections for consistency with book materials; and have eliminated some of the text from ISACA Guideline G3 for brevity. 1. Background 1.1 Linkage to Standards and Guidelines. This ISACA standard has direct linkages to ISACA standards S5 through S14 and other standards and guidelines as discussed in Chapter 3. 1.3 Linkage to COBIT. The ISACA standard has references to CobiT as introduced in Chapter 2. For example, an IT auditor should consider CobiT DS5 Ensure Systems Security guidelines to ensure the achievement of IT objectives and compliance with IT-related laws and regulations by focusing on monitoring the internal control processes for IT-related activities and identifying improvement actions. 2. Planning a CAATT 2.1 Decision Factors for Using a CAATT. When planning an audit, an IT auditor should consider using an appropriate combination of manual and automated techniques. In determining whether to use the CAATT, the auditor factors to be considered include: & Computer knowledge, expertise, and experience of the IT auditor & Availability of suitable CAATT software tools and IT facilities & Efficiency and effectiveness of using the CAATT over a manual technique & IT auditor time constraints & Integrity of the information system and IT environment & Level of audit risk 2.2 CAATT Planning Steps. IT auditor steps for preparing for a CAATT include: & Set the audit objectives for the planned CAATT. & Determine the accessibility and availability of the organization’s IT facilities, programs/systems, and data. & Clearly understand the composition of data to be processed, including quantity, type, format, and layout. & Define the procedures included in the CAATT (e.g., statistical sampling, recalculation, confirmation). & Define output requirements. & Determine expected CAATT resource requirements, including personnel needs and IT resource requirements, & Obtain access to the organization’s IT facilities, programs/systems, and data, including supporting documentation. & Document the planned CAATT, including its objectives, high-level flowcharts, and run instructions. 2.3 IT Auditor Arrangements with the Auditee. Schedule review time with data owners to enhance the design of the CAATT and interpret the data. In addition, the auditee should understand the purpose, scope, timing, and goals of the CAATT. Clear expectations at the outset of the CAATT should be set and communicated. In addition, the IT auditor should arrange for: & The retention of data files, such as detailed transaction files, covering the appropriate audit time frame & Access to IT facilities, programs/systems, and data well in advance of the needed time period to minimize the effect on the organization’s production environment, if possible The IT auditor should assess the effect that changes to the production programs/systems may have on the use of the CAATT. In doing so, the IT auditor should consider the effect of these EXHIBIT 13.1 Auditing Guidelines for Developing a CAATT Source: ISACA E1C13 09/16/2010 16:11:39 Page 313 CAATT Software Tools & 313 changes on the integrity and usefulness of the CAATT and on the programs/systems and data used by the IT auditor. 2.4 Testing the CAATT. It is critical that the IT auditor obtain reasonable assurance of the integrity, reliability, usefulness, and security of the CAATT through appropriate planning, design, testing, processing, and review of supporting documentation. This should be done before audit reliance is placed on CAATT results. The CAATT should receive sufficient review and testing to ensure it is operating as expected. 2.5 Security of Data and CAATT. Where a CAATT used to extract information for data analysis, the IT auditor should verify the integrity of the information system and IT environment from which the data are extracted. & When a CAATT is used to extract sensitive program/system information and production data that should be kept confidential, the IT auditor should clearly understand company data classification and data-handling policies to properly safeguard the information and data with an appropriate level of confidentiality and security. & The IT auditor should consider the level of confidentiality and security required by the organization owning the data and any relevant legislation, and should consult others, such as legal counsel and management, as necessary. & The IT auditor should use and document the results of appropriate procedures to provide for the ongoing integrity, reliability, usefulness, and security of the CAATT. For example, the IT auditor should review program maintenance and program change controls over embedded audit software to determine that only authorized changes have been made to the CAATT. & When the CAATT in an environment not under the control of the IT auditor, an appropriate level of control should be in effect to identify changes to them. & When the CAATT is changed, the IT auditor should obtain assurance of its integrity, reliability, usefulness, and security through appropriate planning, design, testing, processing, and review of documentation before reliance is placed on it. 3. Performance of Audit Work 3.1 Gathering Audit Evidence. The responsible IT auditor should initiate controls on use of the CAATT to provide reasonable assurance that the audit objectives and the detailed specifications of the CAATT have been met. The IT auditor should: & Perform a reconciliation of control totals if appropriate. & Review output for reasonableness. & Perform a review of the logic, parameters, or other characteristics of the CAATT. & Review the organization’s general internal security controls, such as program change controls and system access controls, which may contribute to the integrity of the CAATT. When using test data, the IT auditor should be aware that test data only point out the potential for erroneous processing; this technique does not evaluate actual production data. The IT auditor should also be aware that test data analysis can be extremely complex and time consuming, depending on the number of transactions processed, the number of programs tested, and the complexity of the programs/systems. Before using test data, the IT auditor should verify that the test data will not permanently affect the live system. 3.2 Generalized Audit Software. When using generalized audit software to access production data, the IT auditor should take appropriate steps to protect the integrity of the organization’s data. 3.3 Utility Software 3.3.1 When using utility software, the IT auditor should confirm that no unplanned interventions have taken place during processing and that the utility software has been obtained from the appropriate system library. The IT auditor should also take appropriate steps to protect the integrity of the organization’s system and files since these utilities can easily damage the system and its files. EXHIBIT 13.1 (Continued ) E1C13 09/16/2010 16:11:39 314 & Page 314 Computer-Assisted Audit Tools and Techniques 3.4 Customized Software Queries or Scripts 3.4.1 Customized queries or scripts allow the IT auditor to specifically target desired information for analysis. Customized scripts are highly useful for environments where other CAATTs are not available, but usually specific technical skill sets are required to create these scripts. Therefore, the IT auditor should obtain assurance of the CAATT’s integrity, reliability, usefulness, and security through appropriate planning, design and testing before placing reliance on it and should ensure that proper source data are used and that output from scripts and queries is in the proper format. Customized query and script code should be maintained in a secure location to prevent unauthorized changes from occurring. 3.5 Application Software Tracing and Mapping. When using application software tracing and mapping procedures, the IT auditor should confirm that the source code being evaluated has generated the object program currently being used in production. The internal security auditor should be aware that application software tracing and mapping only points out the potential for erroneous processing; it does not evaluate actual production data. 3.6 Continuous Monitoring and Assurance 3.7.1 Continuous assurance monitoring is an uninterrupted approach that allows IT auditors to monitor controls on a continuous basis and to gather selective audit evidence through the computer. This process is discussed in Chapter 14. 3.7.2 Continuous assurance monitoring enables IT auditors to report on subject matter within a very short time frame. In some environments, it should be possible to shorten the reporting time frame to provide almost instantaneous or truly continuous assurance. 3.7.3 By definition, continuous assurance requires a higher degree of reliance on an auditee’s information system than does traditional auditing. This is a result of the need to rely on systemgenerated information versus externally produced information as the basis for audit testing. Hence, auditors need to make judgments on both the quality of the auditee’s systems and the information produced by the system itself. Systems that are of lower quality or that produce less reliable information are often less conducive to continuous assurance than those that are of high quality and produce reliable information. 3.7.4 Environments that are of a higher quality and produce reliable information are better suited to reporting periods of a short to continuous duration. Environments that are of a lower quality or produce less reliable information should use longer reporting periods to compensate for the period of time that must pass for users to review and approve or correct information processed by the system. 4. CAATT Documentation 4.1 CAATT Workpapers. CAATT processes should be sufficiently documented to provide adequate audit evidence. The audit workpapers should contain sufficient documentation to describe the CAATT application and should include: & CAATT objectives & CAATT to be used & Controls to be exercised & Staffing and timing & CAATT preparation and testing procedures and controls & Details of the tests performed by the CAATT & Details of inputs including data used, file layouts, testing periods, high-level flowcharts, and CAATT outputs, including log files and reports & Listings of relevant parameters or source code 4.4 Audit Evidence. CAATT documentation should include: & Outputs produced & Description of the audit analysis work performed on the output EXHIBIT 13.1 (Continued ) E1C13 09/16/2010 16:11:39 Page 315 CAATT Software Tools & & & & 315 Audit findings Audit conclusions Audit recommendations Data and files used should be stored in a secure location. In addition, temporary confidential data used as part of the audit should be disposed of in accordance with corporate data-handling procedures. 5. Reporting CAATT Results The objectives, scope, and methodology section of the report should contain a clear description of the CAATT used. This description should not be overly detailed, but it should provide a good overview for the reader. The description of CAATT used should also be included in the body of the report, where the specific findings relating to the use of CAATTs is discussed. If the description of the CAATT used is applicable to several findings or is too detailed, it should be discussed briefly in the objectives, scope, and methodology section of the report, and the reader should be referred to an appendix with a more detailed description. EXHIBIT 13.1 (Continued ) COBOL program, he or she usually had to submit a request to the data processing department to produce that report. The auditor would depend on a programmer to produce the requested report and could not fully act independently. This lack of auditor independence problem was solved in the early 1970s by the major public accounting firms, which developed simple audit retrieval programs. In addition to convenient data retrieval capabilities, this software often contained other common audit functions, such as sequence number gap detection or audit sampling procedures. This software eventually began to be marketed as generalized audit software (GAS). It was originally based on mainframe legacy systems; several good client-server products are still on the market today. GAS offers IT auditors some of these advantages: & & & Increased independence from information systems. GAS allows IT audit to perform tests of an application without asking the IT function to write the retrieval software, giving auditors an extra level of independence. Increased audit efficiencies. GAS software can perform such routines as confirming accounts receivable records and producing confirmation letters more efficiently than traditional audit procedures. In addition, such a CAATT should almost certainly be used over multiple years, so its development costs can be spread over time. Opportunity to observe other controls. By using an independent set of programs on the auditee’s systems operations, IT audit can observe and develop a better understanding of other IT controls. For example, IT audit may observe procedural weaknesses in work schedules or tape cartridge retrievals from the data center library. While not related to the planned tests of given data files, these observations often point to areas for subsequent audit work. Exhibit 13.2 outlines the typical work steps for planning and building a CAATT using generalized audit software. These tools were originally introduced in the E1C13 09/16/2010 16:11:39 316 & Page 316 Computer-Assisted Audit Tools and Techniques 1. Define overall CAATT objectives: What do we want to test, and why? 2. Identify key applications, database files, key transactions, and cycle times that will test the audit objective. 3. Identify available audit software tools for performing the test. 4. Identify specific files and other data elements that will be tested. 5. Using audit software tools, develop procedures to perform the desired audit tests. 6. Test the CAATT against a sample set of production files, and modify the process until it is working properly. 7. Determine timings and key planned updates, and perform the actual audit CAATT test. 8. Follow up on any unusual or unexpected results, and make further corrections as necessary. 9. Report CAATT audit results. 10. Document overall audit results and the CAATT process. EXHIBIT 13.2 Programming Steps for Developing a CAATT Application Source: Robert R. Moeller, Brink’s Modern Internal Auditing, 7th ed. (Hoboken, NJ: John Wiley & Sons, 2009). Copyright ß 2009, John Wiley & Sons. Used with permission of John Wiley & Sons. mainframe computer era where there were few other easy-to-use retrieval packages available. Today, IT auditors also can use other software retrieval language tools available on many computer systems. Report Generator Languages In days past when IT departments developed applications using compiler-based languages such as COBOL, end users relied on the IT department for all of their reports and applications. This dependency on the IT department has changed very much in today’s era of powerful user-controlled tools. Today, end users with minimal training regularly produce special reports or perform complex file manipulations in addition to using conventionally programmed applications with r