Homeland Security Exercise and Evaluation Program (HSEEP) April 2013 PRE-DECISIONAL DRAFT Intro-1 Contents INTRODUCTION AND OVERVIEW ....................................................................................... INTRO-1 Purpose .......................................................................................................................................... Intro-1 Role of Exercises ........................................................................................................................... Intro-1 Applicability and Scope ................................................................................................................ Intro-2 Supersession .................................................................................................................................. Intro-2 How to Use This Document .......................................................................................................... Intro-2 Revision Process............................................................................................................................ Intro-3 1. HSEEP FUNDAMENTALS ............................................................................................................. 1-1 Overview ............................................................................................................................................. 1-1 Fundamental Principles ....................................................................................................................... 1-1 Exercise Program Management ........................................................................................................... 1-1 Exercise Methodology ......................................................................................................................... 1-2 Exercise Design and Development ............................................................................................... 1-3 Exercise Conduct .......................................................................................................................... 1-3 Exercise Evaluation ...................................................................................................................... 1-3 Improvement Planning .................................................................................................................. 1-3 2. EXERCISE PROGRAM MANAGEMENT ................................................................................... 2-1 Overview ............................................................................................................................................. 2-1 Engage Elected and Appointed Officials............................................................................................. 2-1 Multi-year Exercise Program Priorities ............................................................................................... 2-1 Training and Exercise Planning Workshop .................................................................................. 2-2 Multi-year Training and Exercise Plan................................................................................................ 2-3 Progressive Approach ................................................................................................................... 2-3 Discussion-Based Exercises.......................................................................................................... 2-4 Operations-Based Exercises.......................................................................................................... 2-5 Rolling Summary of Outcomes ........................................................................................................... 2-6 Manage Exercise Program Resources ................................................................................................. 2-7 Exercise Budget Management ...................................................................................................... 2-7 Program Staffing ........................................................................................................................... 2-7 Other Resources ............................................................................................................................ 2-7 3. EXERCISE DESIGN AND DEVELOPMENT .............................................................................. 3-1 Overview ............................................................................................................................................. 3-1 Exercise Foundation ............................................................................................................................ 3-1 Exercise Planning Team and Events ................................................................................................... 3-2 Exercise Planning Team Considerations....................................................................................... 3-2 Exercise Planning Team Positions ................................................................................................ 3-3 Planning Activities ........................................................................................................................ 3-4 Exercise Design ................................................................................................................................... 3-9 Scope........................................................................................................................................... 3-10 Exercise Objectives..................................................................................................................... 3-11 Evaluation Requirements ............................................................................................................ 3-12 Scenario ...................................................................................................................................... 3-12 Exercise Documentation ............................................................................................................. 3-13 Media or Public Affairs Guidance .............................................................................................. 3-19 Exercise Development ....................................................................................................................... 3-20 Planning for Exercise Logistics .................................................................................................. 3-20 Planning for Exercise Control ..................................................................................................... 3-22 Homeland Security Exercise and Evaluation Program i Planning for Exercise Evaluation................................................................................................ 3-25 4. EXERCISE CONDUCT ................................................................................................................... 4-1 Overview ............................................................................................................................................. 4-1 Exercise Play Preparation .................................................................................................................... 4-1 Setup for Discussion-Based Exercises .......................................................................................... 4-1 Setup for Operations-Based Exercises .......................................................................................... 4-1 Briefings........................................................................................................................................ 4-2 Exercise Play ....................................................................................................................................... 4-2 Participant Roles and Responsibilities .......................................................................................... 4-3 Conduct for Discussion-Based Exercises ..................................................................................... 4-4 Conduct for Operations-Based Exercises ..................................................................................... 4-5 Contingency Process ..................................................................................................................... 4-7 Wrap-Up Activities ............................................................................................................................. 4-7 Debriefings.................................................................................................................................... 4-7 Player Hot Wash ........................................................................................................................... 4-7 Controller/Evaluator Debriefing ................................................................................................... 4-8 5. EVALUATION .................................................................................................................................. 5-1 Overview ............................................................................................................................................. 5-1 Evaluation Planning ............................................................................................................................ 5-1 Evaluation Team ........................................................................................................................... 5-1 Exercise Evaluation Guide Development ..................................................................................... 5-2 Recruit, Assign, and Train Evaluators .......................................................................................... 5-3 Evaluation Documentation............................................................................................................ 5-3 Pre-Exercise Evaluator Briefing ................................................................................................... 5-4 Exercise Observation and Data Collection .......................................................................................... 5-4 Observation ................................................................................................................................... 5-4 Data Collection ............................................................................................................................. 5-4 Data Analysis ...................................................................................................................................... 5-5 After-Action Report Draft ................................................................................................................... 5-5 6. IMPROVEMENT PLANNING ....................................................................................................... 6-1 Overview ............................................................................................................................................. 6-1 Corrective Actions ............................................................................................................................... 6-1 After-Action Meeting .......................................................................................................................... 6-1 After-Action Report/Improvement Plan Finalization .......................................................................... 6-2 Corrective Action Tracking and Implementation ................................................................................ 6-2 Using Improvement Planning to Support Continuous Improvement .................................................. 6-2 GLOSSARY OF TERMS .................................................................................................... GLOSSARY-1 ACRONYMS AND ABBREVIATIONS .............................................................................ACRONYM-1 Homeland Security Exercise and Evaluation Program ii Introduction and Overview Purpose The Homeland Security Exercise and Evaluation Program (HSEEP) provides a set of guiding principles for exercise programs, as well as a common approach to exercise program management, design and development, conduct, evaluation, and improvement planning. Exercises are a key component of national preparedness—they provide elected and appointed officials and stakeholders from across the whole community with the opportunity to shape planning, assess and validate capabilities, and address areas for improvement. Through the use of HSEEP, exercise program managers can develop, execute, and evaluate exercises that address the priorities established by an organization’s leaders. These priorities are based on the National Preparedness Goal, strategy documents, threat and hazard identification/risk assessment processes, capability assessments, and the results from previous exercises and real-world events. These priorities guide the overall direction of a progressive exercise program, where individual exercises are anchored to a common set of priorities or objectives and build toward an increasing level of complexity over time. Accordingly, these priorities guide the design and development of individual exercises, as planners identify exercise objectives and align them to core capabilities 1 for evaluation during the exercise. Exercise evaluation assesses the ability to meet exercise objectives and capabilities by documenting strengths, areas for improvement, core capability performance, and corrective actions in an After-Action Report/Improvement Plan (AAR/IP). Through improvement planning, organizations take the corrective actions needed to improve plans, build and sustain capabilities, and maintain readiness. In this way, the use of HSEEP—in line with the National Preparedness Goal and the National Preparedness System—supports efforts across the whole community that improve our national capacity to build, sustain, and deliver core capabilities. Role of Exercises Exercises play a vital role in national preparedness by enabling whole community stakeholders to test and validate plans and capabilities, and identify both capability gaps and areas for improvement. A well-designed exercise provides a low-risk environment to test capabilities, familiarize personnel with roles and responsibilities, and foster meaningful interaction and communication across organizations. Exercises bring together and strengthen the whole community in its efforts to prevent, protect against, mitigate, respond to, and recover from all hazards. Overall, exercises are cost-effective and useful tools that help the nation practice and refine our collective capacity to achieve the core capabilities in the National Preparedness Goal. 1 Core Capabilities are distinct critical elements necessary to achieve the specific mission areas of prevention, protection, mitigation, response, and recovery. Capabilities provide a common vocabulary describing the significant functions required to deal with threats and hazards that must be developed and executed across the whole community to ensure national preparedness. Homeland Security Exercise and Evaluation Program Intro-1 Applicability and Scope HSEEP exercise and evaluation doctrine is flexible, scalable, adaptable, and is for use by stakeholders across the whole community. 2 HSEEP doctrine is applicable for exercises across all mission areas—prevention, protection, mitigation, response, and recovery. Using HSEEP supports the National Preparedness System 3 by providing a consistent approach to exercises and measuring progress toward building, sustaining, and delivering core capabilities. HSEEP doctrine is based on national best practices and is supported by training, technology systems, tools, and technical assistance. The National Exercise Program (NEP) is consistent with the HSEEP methodology. Exercise practitioners are encouraged to apply and adapt HSEEP doctrine to meet their specific needs. Supersession This 2013 iteration of HSEEP supersedes the 2007 HSEEP Volumes. The current version reflects the feedback, lessons learned, and best practices of the exercise community, as well as current policies and plans. How to Use This Document This document serves as a description of HSEEP doctrine. It includes an overview of HSEEP fundamentals that describes core HSEEP principles and overall methodology. This overview is followed by several chapters that provide exercise practitioners with more detailed guidance on putting the program’s principles and methodology into practice. The doctrine is organized as follows: • Chapter 1: HSEEP Fundamentals describes the basic principles and methodology of HSEEP. • Chapter 2: Exercise Program Management provides guidance for conducting a Training and Exercise Planning Workshop (TEPW) and developing a Multi-year Training and Exercise Plan (TEP). • Chapter 3: Exercise Design and Development describes the methodology for developing exercise objectives, conducting planning meetings, developing exercise documentation, and planning for exercise logistics, control, and evaluation. • Chapter 4: Exercise Conduct provides guidance on setup, exercise play, and wrap-up activities. • Chapter 5: Evaluation provides the approach to exercise evaluation planning and conduct through data collection, analysis, and development of an AAR. • Chapter 6: Improvement Planning addresses corrective actions identified in the exercise IP and the process of tracking corrective actions to resolution. 2 The whole community includes individuals, families, communities, the private and nonprofit sectors, faith-based organizations, and Federal, State, local, tribal, and territorial governments. 3 The National Preparedness System includes identifying and assessing risks; estimating the level of capabilities needed to address those risks; building or sustaining the required levels of capability; developing and implementing plans to deliver those capabilities; validating and monitoring progress; and reviewing and updating efforts to promote continuous improvement. Homeland Security Exercise and Evaluation Program Intro-2 Revision Process The Department of Homeland Security (DHS) Federal Emergency Management Agency (FEMA) will review HSEEP doctrine and methodology on a biennial basis, or as otherwise needed, to make necessary modifications and incorporate lessons learned. Homeland Security Exercise and Evaluation Program Intro-3 1. HSEEP Fundamentals Overview HSEEP doctrine consists of fundamental principles that frame a common approach to exercises. This doctrine is supported by training, technology systems, tools, and technical assistance, and is based on national best practices. It is intended to enhance consistency in exercise conduct and evaluation while ensuring exercises remain a flexible, accessible way to improve our preparedness across the nation. Fundamental Principles Applying the following principles to both the management of an exercise program and the execution of individual exercises is critical to the effective examination of capabilities: • Guided by Elected and Appointed Officials. The early and frequent engagement of elected and appointed officials is the key to the success of any exercise program. They provide the overarching guidance and direction for the exercise and evaluation program as well as specific intent for individual exercises. • Capability-based, Objective Driven. The National Preparedness Goal identifies a series of core capabilities and associated capability targets across the prevention, protection, mitigation, response, and recovery mission areas. Through HSEEP, organizations can use exercises to examine current and required core capability levels and identify gaps. Exercises focus on assessing performance against capability-based objectives. • Progressive Planning Approach. A progressive approach includes the use of various exercises aligned to a common set of exercise program priorities and objectives with an increasing level of complexity over time. Progressive exercise planning does not imply a linear progression of exercise types. • Whole Community Integration. The use of HSEEP encourages exercise planners, where appropriate, to engage the whole community throughout exercise program management, design and development, conduct, evaluation, and improvement planning. • Informed by Risk. Identifying and assessing risks and associated impacts helps organizations identify priorities, objectives, and core capabilities to be evaluated through exercises. • Common Methodology. HSEEP includes a common methodology for exercises that is applicable to all mission areas—prevention, protection, mitigation, response, and recovery. This methodology enables organizations of divergent sizes, geographies, and capabilities to have a shared understanding of exercise program management, design and development, conduct, evaluation, and improvement planning; and fosters exerciserelated interoperability and collaboration. Exercise Program Management Exercise program management involves a collaborative approach that integrates resources, organizations, and individuals in order to identify and achieve program priorities. Through the Homeland Security Exercise and Evaluation Program 1-1 management of an exercise program, stakeholders provide oversight to specific training and exercise activities sustained over time. An effective exercise program maximizes efficiency, resources, time, and funding by ensuring that exercises are part of a coordinated and integrated approach to building, sustaining, and delivering core capabilities. Key elements of HSEEP’s approach to exercise program management include: • • • • • • Engaging Elected and Appointed Officials to Provide Intent and Direction. Elected and appointed officials must be engaged early and often in an exercise program. They provide both the strategic direction for the program as well as specific guidance for individual exercises. Routine engagement with elected and appointed officials ensures that exercises have the support necessary for success. Establishing Multi-year Exercise Program Priorities. These overarching priorities inform the development of exercise objectives, ensuring that individual exercises evaluate and assess core capabilities in a coordinated and integrated fashion. Using a Progressive Approach. A progressive exercise program management approach includes exercises anchored to a common set of objectives, built toward an increasing level of complexity over time, and involves the participation of multiple entities. Developing a Multi-year Training and Exercise Plan. A TEP, developed through a TEPW, aligns exercise activities and supporting training to exercise program priorities. Maintaining a Rolling Summary of Exercise Outcomes. A rolling summary report provides elected and appointed officials and other stakeholders with an analysis of issues, trends, and key outcomes from all exercises conducted as part of the exercise program. Managing Exercise Program Resources. An effective exercise program utilizes the full range of available resources for exercise budgets, program staffing, and other resources. Exercise Methodology HSEEP uses a common methodology for planning and conducting individual exercises. This methodology applies to exercises in support of all national preparedness mission areas. A common methodology ensures a consistent and interoperable approach to exercise design and development, conduct, evaluation, and improvement planning, as depicted in Figure 1.1. The following chapters contain more detailed descriptions of each phase. Figure 1.1: HSEEP Exercise Cycle Homeland Security Exercise and Evaluation Program 1-2 Exercise Design and Development In designing and developing individual exercises, exercise planning team members are identified to schedule planning meetings, identify and develop exercise objectives, design the scenario, create documentation, plan exercise conduct and evaluation, and coordinate logistics. At key points in this process, the exercise planning team engages elected and appointed officials to ensure their intent is captured and that the officials are prepared to support the exercise as necessary. Exercise Conduct After design and development activities are complete, the exercise is ready to occur. Activities essential to conducting individual exercises include preparing for exercise play, managing exercise play, and conducting immediate exercise wrap-up activities. Exercise Evaluation Evaluation is the cornerstone of an exercise and must be considered throughout all phases of the exercise planning cycle, beginning when the exercise planning team meets to establish objectives and initiate exercise design. Effective evaluation assesses performance against exercise objectives, and identifies and documents strengths and areas for improvement relative to core capabilities. Improvement Planning During improvement planning, the corrective actions identified during individual exercises are tracked to completion, ensuring that exercises yield tangible preparedness improvements. An effective corrective action program develops IPs that are dynamic documents, which are continually monitored and implemented as part of the larger system of improving preparedness. Homeland Security Exercise and Evaluation Program 1-3 2. Exercise Program Management Overview Exercise program management is the process of overseeing and integrating a variety of exercises over time. An effective exercise program helps organizations maximize efficiency, resources, time, and funding by ensuring that exercises are part of a coordinated, integrated approach to building, sustaining, and delivering core capabilities. This approach—called multi-year planning—begins when elected and appointed officials, working with whole community stakeholders, identify and develop a set of multi-year exercise priorities informed by existing assessments, strategies, and plans. These long-term priorities help exercise planners design and develop a progressive program of individual exercises to build, sustain, and deliver core capabilities. Effective exercise program management promotes a multi-year approach to: • Engaging elected and appointed officials • Establishing multi-year exercise program priorities • Developing a multi-year TEP • Maintaining a rolling summary of exercise outcomes • Managing exercise program resources Through effective exercise program management, each exercise becomes a supporting component of a larger exercise program with overarching priorities. Exercise practitioners are encouraged to apply and adapt HSEEP doctrine on exercise program management to meet their specific needs. Engage Elected and Appointed Officials Engaging elected and appointed officials in the exercise process is critical because they provide both the strategic direction for the exercise program, as well as specific guidance for individual exercises. As representatives of the public, elected and appointed officials ensure that exercise program priorities are supported at the highest level and align to whole community needs and priorities. Elected and appointed officials should be engaged early and often in an exercise program, starting with the development of exercise program priorities at the TEPW. In developing individual exercises, the exercise planning team should continue to engage their appropriate elected and appointed officials throughout the exercise planning cycle in order to ensure the leaders’ vision for the exercise is achieved. Multi-year Exercise Program Priorities An exercise program should be based on a set of strategic, high-level priorities selected by an organization’s elected and appointed officials. These priorities guide the development of exercise objectives, ensuring that individual exercises build and sustain preparedness in a progressive and coordinated fashion. Exercise program priorities are developed at the TEPW, as described in the following sections. Homeland Security Exercise and Evaluation Program 2-1 Training and Exercise Planning Workshop Purpose The TEPW establishes the strategy and structure for an exercise program. In addition, it sets the foundation for the planning, conduct, and evaluation of individual exercises. The purpose of the TEPW is to use the guidance provided by elected and appointed officials to identify and set exercise program priorities and develop a multi-year schedule of exercise events and supporting training activities to meet those priorities. This process ensures whole community exercise initiatives are coordinated, prevents duplication of effort, promotes the efficient use of resources, avoids overextending key agencies and personnel, and maximizes the efficacy of training and exercise appropriations. TEPWs are held on a periodic basis (e.g., annual or biennial) depending on the needs of the program and any grant or cooperative agreement requirements. Participation When identifying stakeholders, exercise program managers should consider individuals from organizations throughout the whole community, including but not limited to: • Elected and appointed officials responsible for providing direction and guidance for exercise program priorities and those responsible for providing resources to support exercises; • Representatives from relevant disciplines that would be part of the exercises or any realworld events, including appropriate regional or local Federal department/agency representatives; • Individuals with administrative responsibility relevant to exercise conduct; and • Representatives from volunteer, nongovernmental, nonprofit, or social support organizations, including advocates for children, seniors, individuals with disabilities, those with access and functional needs, racially and ethnically diverse communities, people with limited English proficiency, and animals. Once a comprehensive set of stakeholders has been identified, exercise program managers can include them in the exercise program by having them regularly participate in TEPWs. Conduct of the TEPW When developing exercise program priorities and the multi-year schedule at the TEPW, stakeholders should engage organizational elected and appointed officials early in the process to obtain their intent and guidance. TEPW participants also review and consider various factors such as: • Jurisdiction-specific threats and hazards (e.g., Threat and Hazard Identification and Risk Assessment [THIRA], local risk assessments); • Areas for improvement identified from real-world events and exercises; • External requirements such as State or national preparedness reports, homeland security policy (e.g., the National Preparedness Goal), and industry reports; and • Accreditation standards (e.g., hospital accreditation requirements), regulations, or legislative requirements. Homeland Security Exercise and Evaluation Program 2-2 Figure 2.1 illustrates some of the specific factors for consideration in developing exercise program priorities. Figure 2.1: Factors for Consideration in Developing Exercise Program Priorities Drawing on the above factors and core capabilities, the workshop facilitator leads a group stakeholder discussion to review exercise program priorities and outline training and exercise priorities shared across multiple organizations. The group should also develop a multi-year schedule of training and exercise activities designed to meet those priorities. At the conclusion of the TEPW, program managers will have a clear understanding of specific multi-year training and exercise program priorities, and any available information on previously planned training and exercises that align to those priorities. This combined set of information is used to develop a multi-year TEP. Multi-year Training and Exercise Plan Once the training and exercise program priorities have been outlined, stakeholders develop the multi-year TEP. The TEP identifies a combination of exercises—along with associated training requirements—that address the priorities identified in the TEPW. Progressive Approach A progressive, multi-year exercise program enables organizations to participate in a series of increasingly complex exercises, with each successive exercise building upon the previous one until mastery is achieved. Regardless of exercise type, each exercise within the progressive series is linked to a set of common program priorities and designed to test associated capabilities. Homeland Security Exercise and Evaluation Program 2-3 Further, by defining training requirements in the planning process, organizations can address known shortfalls prior to exercising capabilities. This progressive approach, with exercises that build upon each other and are supported at each step with training resources, will ensure that organizations do not rush into a full-scale exercise too quickly. Effective planning of exercises and integration of the necessary training will reduce the waste of limited exercise resources and serve to address known shortfalls prior to the conduct of the exercise. The different types of exercises that may be included in the multi-year plan are described in the following sections. A progressive exercise program is a series of exercises tied to a set of common program priorities. Each exercise builds on previous exercises using more sophisticated simulation techniques or requiring more preparation time, personnel, and planning. Discussion-Based Exercises Discussion-based exercises include seminars, workshops, tabletop exercises (TTXs), and games. These types of exercises can be used to familiarize players with, or develop new, plans, policies, agreements, and procedures. Discussion-based exercises focus on strategic, policy-oriented issues. Facilitators and/or presenters usually lead the discussion, keeping participants on track towards meeting exercise objectives. Seminars Seminars generally orient participants to, or provide an overview of, authorities, strategies, plans, policies, procedures, protocols, resources, concepts, and ideas. As a discussion-based exercise, seminars can be valuable for entities that are developing or making major changes to existing plans or procedures. Seminars can be similarly helpful when attempting to assess or gain awareness of the capabilities of interagency or inter-jurisdictional operations. Workshops Although similar to seminars, workshops differ in two important aspects: participant interaction is increased, and the focus is placed on achieving or building a product. Effective workshops entail the broadest attendance by relevant stakeholders. Products produced from a workshop can include new standard operating procedures (SOPs), emergency operations plans, continuity of operations plans, or mutual aid agreements. To be effective, workshops should have clearly defined objectives, products, or goals, and should focus on a specific issue. Tabletop Exercises A TTX is intended to generate discussion of various issues regarding a hypothetical, simulated emergency. TTXs can be used to enhance general awareness, validate plans and procedures, rehearse concepts, and/or assess the types of systems needed to guide the prevention of, protection from, mitigation of, response to, and recovery from a defined incident. Generally, TTXs are aimed at facilitating conceptual understanding, identifying strengths and areas for improvement, and/or achieving changes in perceptions. During a TTX, players are encouraged to discuss issues in depth, collaboratively examining areas of concern and solving problems. The effectiveness of a TTX is derived from the energetic Homeland Security Exercise and Evaluation Program 2-4 involvement of participants and their assessment of recommended revisions to current policies, procedures, and plans. TTXs can range from basic to complex. In a basic TTX (such as a Facilitated Discussion), the scenario is presented and remains constant—it describes an emergency and brings discussion participants up to the simulated present time. Players apply their knowledge and skills to a list of problems presented by the facilitator; problems are discussed as a group; and resolution is reached and documented for later analysis. In a more advanced TTX, play advances as players receive pre-scripted messages that alter the original scenario. A facilitator usually introduces problems one at a time in the form of a written message, simulated telephone call, videotape, or other means. Players discuss the issues raised by each problem, referencing established authorities, plans, and procedures for guidance. Player decisions are incorporated as the scenario continues to unfold. During a TTX, all participants should be encouraged to contribute to the discussion and be reminded that they are making decisions in a no-fault environment. Effective TTX facilitation is critical to keeping participants focused on exercise objectives and associated capability targets. Games A game is a simulation of operations that often involves two or more teams, usually in a competitive environment, using rules, data, and procedures designed to depict an actual or hypothetical situation. Games explore the consequences of player decisions and actions. They are useful tools for validating plans and procedures or evaluating resource requirements. During game play, decision-making may be either slow and deliberate or rapid and more stressful, depending on the exercise design and objectives. The open, decision-based format of a game can incorporate “what if” questions that expand exercise benefits. Depending on the game’s design, the consequences of player actions can be either pre-scripted or decided dynamically. Identifying critical decision-making points is a major factor in the success of evaluating a game. Operations-Based Exercises Operations-based exercises include drills, functional exercises (FEs), and full-scale exercises (FSEs). These exercises can be used to validate plans, policies, agreements, and procedures; clarify roles and responsibilities; and identify resource gaps. Operations-based exercises are characterized by actual reaction to an exercise scenario, such as initiating communications or mobilizing personnel and resources. Drills A drill is a coordinated, supervised activity usually employed to validate a specific function or capability in a single agency or organization. Drills are commonly used to provide training on new equipment, validate procedures, or practice and maintain current skills. For example, drills may be appropriate for establishing a community-designated disaster receiving center or shelter. Drills can also be used to determine if plans can be executed as designed, to assess whether more training is required, or to reinforce best practices. A drill is useful as a stand-alone tool, but a series of drills can be used to prepare several organizations to collaborate in an FSE. For every drill, clearly defined plans, procedures, and protocols need to be in place. Personnel need to be familiar with those plans and trained in the processes and procedures to be drilled. Homeland Security Exercise and Evaluation Program 2-5 Functional Exercises FEs are designed to validate and evaluate capabilities, multiple functions and/or sub-functions, or interdependent groups of functions. FEs are typically focused on exercising plans, policies, procedures, and staff members involved in management, direction, command, and control functions. In FEs, events are projected through an exercise scenario with event updates that drive activity typically at the management level. An FE is conducted in a realistic, real-time environment; however, movement of personnel and equipment is usually simulated. FE controllers typically use a Master Scenario Events List (MSEL) to ensure participant activity remains within predefined boundaries and ensure exercise objectives are accomplished. Simulators in a Simulation Cell (SimCell) can inject scenario elements to simulate real events. Full-Scale Exercises FSEs are typically the most complex and resource-intensive type of exercise. They involve multiple agencies, organizations, and jurisdictions and validate many facets of preparedness. FSEs often include many players operating under cooperative systems such as the Incident Command System (ICS) or Unified Command. In an FSE, events are projected through an exercise scenario with event updates that drive activity at the operational level. FSEs are usually conducted in a real-time, stressful environment that is intended to mirror a real incident. Personnel and resources may be mobilized and deployed to the scene, where actions are performed as if a real incident had occurred. The FSE simulates reality by presenting complex and realistic problems that require critical thinking, rapid problem solving, and effective responses by trained personnel. The level of support needed to conduct an FSE is greater than that needed for other types of exercises. The exercise site for an FSE is usually large, and site logistics require close monitoring. Safety issues, particularly regarding the use of props and special effects, must be monitored. Throughout the duration of the exercise, many activities occur simultaneously. Rolling Summary of Outcomes To help ensure that exercise program priorities are adequately addressed, exercise program managers should periodically develop and distribute a rolling summary of exercise outcomes, or rolling summary report. A rolling summary report provides stakeholders with an analysis of issues, trends, and key outcomes from all exercises conducted as part of the exercise program. This report is designed to: The rolling summary report is an analysis of exercise trends, which guides the development of future exercises. • Inform elected and appointed officials on the progress of the exercise program; • Provide data to support preparedness assessments and reporting requirements; and • Enable exercise planners to modify objectives and the exercise schedule to reflect knowledge gathered from the exercises. The rolling summary report is not a collection of AARs, but rather an analysis of trends across exercises. It is developed periodically throughout the series of exercises covered in a multi-year TEP (e.g., quarterly or biennially, depending how many exercises are conducted). This report is Homeland Security Exercise and Evaluation Program 2-6 intended to serve as an exercise program management and communications tool, which informs stakeholders and guides the development of future exercises. Manage Exercise Program Resources An effective exercise program should utilize the full range of available resources. Program managers should ensure that they have planned for an exercise budget, program staffing, and other resources. Exercise Budget Management Effective budget management is essential to the success of an exercise program, and it is important for exercise managers to maintain awareness of their available resources and expected expenditures. In developing and maintaining an exercise program budget, program managers should work with the full range of stakeholders to identify financial resources and define monitoring and reporting requirements as required by individual exercises. Program Staffing Program managers should identify the administrative and operational staff needed to oversee the exercise program. The TEP can be one basis for determining exercise program staffing needs in addition to grant funds or other programmatic considerations. Program managers should also identify gaps between staffing availability and staffing needs. Exercise program managers can consider alternative means of procuring staff members, such as adding volunteers, students from universities (e.g., student nurses or emergency management students), or interns. Other Resources Exercise program managers should also consider other resources that can support exercises. Such resources can include: • Information technology (e.g., modeling and simulation capabilities) • Exercise tools and resources (e.g., document templates) • Materials from previous exercises • Training courses • Mutual aid agreements, memoranda of understanding, and memoranda of agreement • Technical assistance • Equipment or props (e.g., smoke machines) Homeland Security Exercise and Evaluation Program 2-7 3. Exercise Design and Development Overview In the design and development phase, exercise practitioners use the intent and guidance of their elected and appointed officials and the exercise program priorities developed in Program Management to plan individual exercises. Exercise planning teams apply this guidance to shape the key concepts and planning considerations for an individual exercise or series of exercises. The eight key steps of exercise design and development include: • Setting the exercise foundation by reviewing elected and appointed officials’ guidance, the TEP, and other factors; • Selecting participants for an exercise planning team and developing an exercise planning timeline with milestones; • Developing exercise-specific objectives and identifying core capabilities based on the guidance of elected and appointed officials; • Identifying evaluation requirements; • Developing the exercise scenario; • Creating documentation; • Coordinating logistics; and • Planning for exercise control and evaluation. Exercise practitioners are encouraged to apply and adapt HSEEP doctrine on exercise design and development to meet their specific needs. Exercise Foundation The exercise foundation is a set of key factors that drive the exercise design and development process. Prior to the beginning of its design, exercise program managers should review and consider the following items: • Elected and appointed officials’ intent and guidance • Multi-year TEP • Relevant AAR/IPs from real-world events and exercises • THIRA or other risk, threat, and hazard assessments • Organizational plans and procedures • Grant or cooperative agreement requirements. By reviewing these elements, exercise program managers adhere to the progressive approach to exercises, and ensure the exercise builds and sustains a jurisdiction’s capabilities while taking prior lessons learned into account during the exercise design process. Homeland Security Exercise and Evaluation Program 3-1 Exercise Planning Team and Events Exercise Planning Team Considerations The exercise planning team manages, and is Whole community stakeholders include: ultimately responsible for, exercise design,  All levels of government development, conduct, and evaluation. Using the  Volunteer organizations exercise program priorities and guidance from  Community groups elected and appointed officials, the team determines  Private entities exercise objectives and core capabilities to be  Nonprofit organizations assessed; creates a realistic scenario to assess them;  Faith-based groups and develops supporting documentation, processes,  Groups working with individuals and systems that are used in evaluation, control, and with disabilities or access and simulation. Planning team members also help with functional needs developing and distributing pre-exercise materials, and conducting exercise planning meetings, briefings, and training sessions. An Exercise Director with authority to make decisions for the sponsoring organization provides direction to, and oversight of, the exercise planning team. The exercise planning team should be of manageable size yet represent the full range of participating organizations as well as other relevant stakeholders. For multi-jurisdictional exercises, planning team members should include representatives from each jurisdiction and participating functional areas or relevant disciplines. The membership of an exercise planning team should be modified to fit the type or scope of an exercise, which varies depending on exercise type and complexity. Usually the exercise planning team is managed by a designated team leader. To design and develop exercises most effectively, exercise planning teams should: • Adhere to a clear organizational structure, with a distinct chain of command, roles and responsibilities, and accountability to the exercise planning team leader; • Use proven management practices, processes, and tools, such as project plans and timelines, status reports, and other communications; • Identify and understand the desired objectives and associated core capabilities for the exercise, and design and develop the exercise accordingly; • Incorporate evaluation planning from the start of exercise design and development; and • Use subject-matter experts (SMEs) to develop a realistic and challenging scenario. Support agencies/organizations including advocates for children, seniors, individuals with disabilities, those with access and functional needs, diverse communities, and people with limited English proficiency should also be included throughout the planning process. In doing so, exercise planners can better understand their perspectives and promote early understanding of roles, responsibilities, and planning assumptions. Generally, planning team members are not exercise players. When resources are limited, exercise planning team members who act as both planners and players should be especially careful not to divulge sensitive exercise information to other players. Homeland Security Exercise and Evaluation Program 3-2 Exercise Planning Team Positions Regardless of the scale and complexity of an exercise, the exercise planning team can be most effective if it adheres to a coherent organizational structure that clearly delineates roles and responsibilities. In developing a structure for the planning team, exercise planners may use ICS principles, as established in the National Incident Management System (NIMS). This structure can expand or contract to reflect the scope of the exercise and the available resources and personnel of the participating organizations; depending on available resources, the same personnel can be used to execute multiple functions. This structure may include the following, which is illustrated in Figure 3.1: Figure 3.1: Sample Exercise Planning Team • Command Section. The Command Section coordinates all exercise planning activities. The Command Section includes the exercise planning team leader, who assigns exercise activities and responsibilities, provides guidance, establishes timelines, and monitors the development process. • Operations Section. The Operations Section provides most of the technical or functional expertise for scenario development and evaluation. This includes development of the Master Scenario Events List (MSEL). • Planning Section. The Planning Section is responsible for compiling and developing all exercise documentation. The Planning Section collects and reviews policies, plans, and procedures that will be assessed in the exercise. This group is also responsible for planning exercise evaluation. During the exercise, the Planning Section may be responsible for developing simulated actions by agencies not participating in the exercise and for setting up a SimCell as required. • Logistics Section. The Logistics Section provides the supplies, materials, facilities, and services that enable the exercise to function smoothly without outside interference or disruption. This section consists of two subsections: service and support. The service subsection provides transportation, barricading, signage, food and drinks, real-life medical capability, and exercise security. The support subsection provides Homeland Security Exercise and Evaluation Program 3-3 communications, purchasing, general supplies, management of very important persons (VIPs) and observer processing, and recruitment and management of actors. • Administration/Finance Section. The Administration/Finance Section provides financial management and administrative support throughout exercise development, including exercise registration support and scheduling. Planning Activities This section describes the types of planning activities—often in the form of planning meetings 4—most useful in exercise design and development. The exercise planning team members decide the type and number of planning activities needed to successfully plan a given exercise, based on its scope and complexity. When arranging meeting and exercise site locations, the planning team should take into consideration those individuals who require assistance or accommodations during attendance. Concept and Objectives Meeting Primary Focus A Concept and Objectives (C&O) Meeting is the formal beginning of the planning process. It is held to identify the scope and objectives of the exercise. For less complex exercises and for organizations with limited resources, the C&O Meeting can be conducted in conjunction with the Initial Planning Meeting (IPM). Elected and appointed officials, representatives from the sponsoring organization, participating organizations, and the exercise planning team leader typically attend the C&O Meeting. The C&O Meeting helps planners determine the exercise program priorities to be addressed based on elected and appointed officials’ guidance, design objectives based on those priorities, align exercise objectives to core capabilities, and identify exercise planning team members. Discussion Points Topics or issues generally covered during a C&O Meeting include the following: • Exercise scope • Proposed exercise objectives and their aligned core capabilities • Proposed exercise location, date, and duration • Participants and anticipated extent of play for exercise participants • Exercise planning team • Exercise assumptions and artificialities • Exercise control and evaluation concepts • Exercise security organization and structure • Available exercise resources • Exercise logistics 4 HSEEP uses the term “meetings” to indicate smaller events focused on a specific topic (exercise planning), rather than “conferences,” which are generally larger gatherings with broader agendas. Homeland Security Exercise and Evaluation Program 3-4 • Exercise planning timeline and milestones • Local issues, concerns, and sensitivities Tools The primary tools for the C&O Meeting are an agenda and the background and rationale for conducting the exercise. A briefing is useful for presenting the exercise background and rationale, as well as exercise methodology for persons unfamiliar with HSEEP. Outcomes The following outcomes are expected from the C&O Meeting: • Agreement regarding exercise concept (scope, type, mission area[s], exercise program priorities to be addressed), exercise objectives, and aligned core capabilities; • Consensus on the target exercise timeframe; • Anticipated extent of participation; • Identification of exercise planning team members; and • Exercise planning timeline with milestones, including the date of the next planning meeting. Initial Planning Meeting Primary Focus The IPM marks the beginning of the exercise development phase. Regardless of whether a C&O Meeting is held, an IPM should be conducted for all exercises. Its purpose is to determine exercise scope by getting intent and direction from elected and appointed officials, and gathering input from the exercise planning team; and to identify exercise design requirements and conditions (e.g., assumptions and artificialities), exercise objectives, participant extent of play, and scenario variables (e.g., time, location, hazard selection). The IPM is also used to develop exercise documentation by obtaining the planning team’s input on exercise location, schedule, duration, and other relevant details. During the IPM, exercise planning team members are assigned responsibility for activities associated with designing and developing exercise documents, such as the Exercise Plan (ExPlan) and the Situation Manual (SitMan), and coordinating exercise logistics. Discussion Points Topics or issues generally covered during an IPM include the following: • Clearly defined exercise objectives and aligned core capabilities; • Evaluation requirements, including EEG capability targets and critical tasks; • Relevant plans, policies, and procedures to be tested in the exercise; • Exercise scenario; • Modeling and simulation planning; • Extent of play for each participating organization; Homeland Security Exercise and Evaluation Program 3-5 • Optimum duration of the exercise; • Exercise planners’ roles and responsibilities; • Decision to record exercise proceedings (audio or video); • Local issues, concerns, or sensitivities; • Any discussion points typically covered during a C&O Meeting if a C&O Meeting was not conducted; and • Consensus regarding the date, time, and location for the next meeting. Tools The primary tools for the IPM are the read-ahead packet, agenda, core capabilities, threat and hazard information (if applicable), a proposed room layout (if applicable), and the exercise planning timeline with milestones. A briefing is useful for presenting an overview of the exercise and meeting discussion points. Outcomes The IPM results in desired outcomes, such as: • Any outcomes listed in the C&O Meeting section above if a C&O Meeting was not conducted; • Clearly defined exercise objectives and aligned core capabilities; • Initial capability targets and critical tasks, which will be reviewed and confirmed prior to the next planning meeting; • Identified exercise scenario variables (e.g., threat scenario, scope of hazard, venue, conditions); • A list of participating exercise organizations and anticipated organizational extent of play; • Draft SitMan or ExPlan; • Identification and availability of all source documents (e.g., policies, plans, procedures) needed to draft exercise documents and presentations; • A refined exercise planning timeline with milestones; • Identification and availability of SMEs, as necessary, for scenario vetting and/or expert evaluation; • Determination of preferred communication methods among the exercise planning team; • Clearly identified and assigned responsibility for exercise logistical issues; • A list of tasks to be accomplished by the next planning meeting with established dates for completion and responsible planning team members identified; and • An agreed-upon date, time, and location for the next planning meeting and the actual exercise. Homeland Security Exercise and Evaluation Program 3-6 Midterm Planning Meeting Midterm Planning Meetings (MPMs) provide additional opportunities to engage elected and appointed officials and to settle logistical and organizational issues that may arise during exercise planning. Primary Focus The MPM is a meeting to discuss exercise organization and staffing concepts, scenario and timeline development, scheduling, logistics, and administrative requirements. It is also held to review draft documentation. If only three planning meetings are scheduled (i.e., IPM, MPM, and Final Planning Meeting [FPM]), a portion of the MPM should be devoted to developing the MSEL, as needed. See the next section, MSEL Meeting, for more information. Prior to the MPM, the exercise team leader should engage elected and appointed officials to provide awareness of the planning process, address any questions, and ensure alignment with guidance and intent. Discussion Points Possible topics or issues for an MPM include the following: • Comments on draft exercise documentation • Construction of the scenario timeline—usually the MSEL—if an additional MSEL Planning Meeting will not be held • Identification of exercise venue artificialities and/or limitations • Agreement on final logistical items • Assignment of additional responsibilities Tools MPM tools include, but are not limited to, an agenda, IPM minutes, draft scenario timeline, draft documentation (e.g., ExPlan, Controller/Evaluator [C/E] Handbook), and other selected documentation needed to illustrate exercise concepts and provide planning guidance. Outcomes The following outcomes are expected from the MPM: • Fully reviewed SitMan or ExPlan; • Draft Facilitator Guide or C/E Handbook, including EEGs; • A fully reviewed exercise scenario timeline, which is typically the MSEL (if an additional MSEL Meeting will not be held); • Well-developed scenario injects (imperative if an additional MSEL Planning Meeting is not scheduled); • Agreement on the exercise site; and • Finalization of date, time, and location of the MSEL Planning Meeting and/or FPM. Homeland Security Exercise and Evaluation Program 3-7 Master Scenario Events List Meeting For more complex exercises, one or more additional planning meetings—or MSEL Meetings— may be held to review the scenario timeline. If not held separately, topics typically covered in a separate MSEL Meeting can be incorporated into the MPM and FPM. Primary Focus The MSEL Meeting focuses on developing the MSEL, which is a chronological list that supplements the exercise scenario with event synopses, expected participant responses, objectives and core capability targets to be addressed, and responsible personnel. It includes specific scenario events (or injects) that prompt players to implement the plans, policies, procedures, and protocols that require testing during the exercise, as identified in the capabilitiesbased planning process. It also records the methods that will be used to provide injects (e.g., phone call, radio call, e-mail). Discussion Points In developing a MSEL, the exercise planning team should first consider the critical tasks, conditions, and standards set forth by each exercise objective. A condition is the environment in which a task is performed; it can be provided by the scenario or through the MSEL. If scenario conditions do not trigger performance of the appropriate critical task, the exercise planning team should develop a MSEL entry to simulate the desired situation. A wellwritten entry considers the following questions: MSEL entries are tied to the Exercise Evaluation Guide critical tasks to ensure the critical tasks and core capabilities can be demonstrated during the exercise. • Is the event key (i.e., is it directly related to meeting an exercise objective)? • What is the desired critical task? Who will demonstrate the critical task? • What will stimulate the behavior (e.g., course of play, phone call, actor, video)? • Who originates the stimulant? Who receives it and how? • What action is the player expected to complete? • Should a contingency entry be developed for injection into the exercise in case the players fail to demonstrate the critical task? Tools MSEL Meeting tools include, but are not limited to, previous planning meeting minutes, draft exercise documentation, and an agreed-upon MSEL template. Outcomes Following a MSEL Meeting, the level of MSEL completion may vary. At a minimum, key events and the time of their delivery are identified, and responsibility for constructing the remaining events is assigned. Homeland Security Exercise and Evaluation Program 3-8 Final Planning Meeting The FPM is the final forum for reviewing exercise processes and procedures. Both before and after the FPM, the exercise team leader should engage elected and appointed officials to ensure that the exercise is aligning with their intent, address any questions, and receive any last-minute guidance. Primary Focus An FPM should be conducted for all exercises to ensure that all elements of the exercise are ready for conduct. Prior to the FPM, the exercise planning team receives final drafts of all exercise materials. No major changes to the exercise’s design, scope, or supporting documentation should take place at or following the FPM. The FPM ensures that all logistical requirements have been met, outstanding issues have been identified and resolved, and exercise products are ready for printing. Discussion Points The following items are addressed during the FPM: • Conduct a comprehensive, final review and approve all remaining draft exercise documents (e.g., SitMan, MSEL, C/E Handbook, EEGs) and presentation materials; • Resolve any open exercise planning issues and identify last-minute concerns; and • Review all exercise logistical activities (e.g., schedule, registration, attire, special needs). Tools The primary tools for the FPM include IPM and/or MPM minutes, an agenda, and previously finalized and/or drafted exercise documents. Outcomes The FPM should not generate any significant changes. The following outcomes are expected: • Exercise documents and materials for production are approved; • Attendees understand and approve exercise processes and procedures; • Last-minute issues are identified and resolved; and • Logistical elements, including equipment, facilities, and schedule, are confirmed. Follow-Up The exercise planning team finalizes all publications, prepares all supporting materials, rehearses presentations and briefings, and prepares to conduct the exercise. Prior to the exercise, documentation and any additional instructions should be disseminated to the appropriate personnel (e.g., presenters, facilitators, controllers, evaluators, simulators). Exercise Design The exercise planning meetings serve as the principal mechanism for executing the major steps of exercise design. The core components of design include establishing the scope of the exercise, setting exercise objectives, creating an exercise scenario, developing exercise documentation, and determining media and public relations guidance. Homeland Security Exercise and Evaluation Program 3-9 Scope Determining exercise scope enables planners to “right-size” an exercise to meet the objectives while staying within the resource and personnel constraints of the exercising organizations. Key elements in defining exercise scope include exercise type, participation level, exercise duration, exercise location, and exercise parameters. Some of these elements are determined, or initially discussed, through program management activities or grant requirements. However, the exercise planning team finalizes the scope based on the exercise objectives. Alterations to the scope are reviewed with the exercise objectives in mind; planners must consider whether a change in the scope will improve or impede the ability of players to meet the objectives. Exercise Type A first step in defining exercise scope is determining what exercise type to conduct. The exercise type is selected based on the purpose of the exercise. If the intent is to review and discuss a new policy, plan, or set of procedures, a discussion-based exercise may be appropriate. If the intent is to assess the responders’ knowledge of a plan, policy, or set of procedures, an operations-based exercise may be appropriate. Participation Level Active participation by appropriate entities and key leaders is paramount to meeting the exercise objectives successfully. Participation level refers to the organizations and level of personnel (e.g., tactical operators, line supervisors, agency directors) participating in the exercise, as well as the general number of personnel who will participate in the exercise. At times, scheduling conflicts, real-world events, or other competing requirements will limit an organization’s or key players’ ability to participate in an exercise. In this case, exercise designers will need to simulate the decisions and actions of those participants through an exercise SimCell. An Extent of Play Agreement (XPA) defines the level of participation. Exercise Duration When selecting the exercise duration, the planning team should determine how long it will take to address the exercise objectives effectively. Discussion-based exercises and some drills are generally shorter, ranging from a couple of hours to a full day. FEs and FSEs may take longer. Prevention-focused FEs that exercise the intelligence and information sharing core capability may last up to 30 days with limited duration of play each day. Resource constraints, including the opportunity cost of having employees away from their primary roles, should be factored into determining duration. Exercise Parameters Exercise parameters clearly outline what should be included in an exercise scenario based on the objectives and scope, and what should not be exercised. Often there is a desire to add exercise activities that fall outside of the scope of the exercise in order to meet diverse planning and training requirements. While these activities may be useful to an organization, they may impact the ability of players to meet exercise objectives or may reduce the benefit of the exercise by diluting its focus. Clearly defining the exercise scope early in the design process will help exercise planners keep the exercise to a manageable and realistic level. Homeland Security Exercise and Evaluation Program 3-10 Exercise Objectives Based on direction from elected and appointed officials, the exercise planning team selects one or more exercise program priorities on which to focus an individual exercise. These priorities drive the development of exercise objectives, which are distinct outcomes that an organization wishes to achieve during an exercise. Exercise objectives should incorporate elected and appointed officials’ intent and guidance, and exercise participants’ plans and procedures, operating environment, and desired outcomes. Generally, planners should select a reasonable number of specific, measurable, achievable, relevant, and time-bound (SMART) exercise objectives to facilitate effective scenario design, exercise conduct, and evaluation. Table 3.1 depicts guidelines for developing SMART objectives. SMART Guidelines for Exercise Objectives Specific Objectives should address the five Ws- who, what, when, where, and why. The objective specifies what needs to be done with a timeline for completion. Measurable Objectives should include numeric or descriptive measures that define quantity, quality, cost, etc. Their focus should be on observable actions and outcomes. Achievable Objectives should be within the control, influence, and resources of exercise play and participant actions. Relevant Objectives should be instrumental to the mission of the organization and link to its goals or strategic intent. Time-bound A specified and reasonable timeframe should be incorporated into all objectives. Table 3.1: SMART Guidelines for Exercise Objectives The exercise planning team aligns each exercise objective to one or more core capabilities. Figure 3.2 shows the relationship between exercise program priorities, exercise objectives, and core capabilities. Figure 3.2: Priorities, Objectives, and Core Capabilities Aligning objectives to a common set of capabilities enables: • Systematic tracking of progress over the course of exercise programs and/or cycles; • Standardized exercise data collection to inform preparedness assessments; and • Fulfillment of grant or funding-specific reporting requirements. Homeland Security Exercise and Evaluation Program 3-11 Evaluation Requirements It is important to develop exercise evaluation requirements early in the design process, as they will guide development of the exercise scenario, discussion questions, and/or MSEL. Evaluation requirements clearly articulate what will be evaluated during the exercise and how exercise play will be assessed. This information is documented in the EEGs. Once the exercise planning team aligns objectives to core capabilities, it identifies which capability targets and critical tasks for each core capability are being addressed by the exercise. Capability targets are the performance thresholds for each core capability; they state the exact amount of capability that players aim to achieve. Generally, these targets are based on targets identified as part of an organization’s or jurisdiction’s THIRA or other threat and hazard identification or risk assessment process. Critical tasks are the distinct elements required to perform a core capability. Critical tasks may be derived from Mission Area Frameworks, organizational operations plans or SOPs, or discipline-specific standards. Scenario A scenario is an outline or model of the simulated sequence of events for the exercise. It can be written as a narrative or depicted by an event timeline. For discussion-based exercises, a scenario provides the backdrop that drives participant discussion, and is contained in a SitMan. For operations-based exercises, a scenario provides background information about the incident catalyst(s) of the exercise. The overall scenario is provided in the C/E Handbook, and specific scenario events are contained in the MSEL. Exercise planners should select and develop scenarios that enable an exercise to assess objectives and core capabilities. All scenarios should be realistic, plausible, and challenging; however, designers must ensure the scenario is not so complicated that it overwhelms players. A scenario consists of three basic elements: (1) the general context or comprehensive story; (2) the required conditions that will allow players to demonstrate proficiency and competency in conducting critical tasks, demonstrating core capabilities, and meeting objectives; and (3) the technical details necessary to accurately depict scenario conditions and events. The exercise planning team ensures that the design effort is not characterized by a fixation on scenario development; rather, the scenario facilitates assessment of exercise objectives and core capabilities. Because of this, exercise planners should refrain from developing the scenario until after the scope and objectives of the exercise have been clearly defined. Furthermore, scenarios should avoid any sensitivity that may arise, such as the use of real names of terrorist groups or sensitive venues. Threat or Hazard The first step in designing a scenario is determining the type of threat or hazard on which the exercise will focus. Each type of emergency has its own strengths and weaknesses when it comes to evaluating different aspects of prevention, protection, mitigation, response, and recovery. The exercise planning team should choose a threat or hazard that best assesses the objectives and core capabilities on which the exercise will focus. The identification of this threat or hazard scenario should also be based on the organization’s threat/hazard identification and risk assessment. 5 Developing and maintaining these risk analyses is an essential component of 5 For further guidance on identifying and assessing risks and associated impacts, please refer to the DHS Comprehensive Preparedness Guide 201: Threat and Hazard Identification and Risk Assessment Guide, First Edition, April 2012. Homeland Security Exercise and Evaluation Program 3-12 the National Preparedness System, as they enable organizations to identify potential events that would stress their core preparedness capabilities. Modeling and Simulation When incorporated into the development of the scenario Benefits of using modeling and and overall exercise design, modeling and simulation can simulation in exercises include: bring versatility, cost savings, and fidelity to exercises.  Enhanced realism A model is a representation of a system at a point in time  Efficiency or space intended to expand an understanding of the real  Ability to exercise situations system. Simulation is a method of implementing the that cannot be safely or performance of a model, or combination of models, over realistically replicated time. Modeling and simulation supports decisionmaking processes by providing human and/or computer feedback to players during exercise play, thus dynamically representing the impact of their decisions. For example, human-based simulation during exercises is often manifested through the SimCell, which represents nonparticipating entities. An example of a computer-based simulation could include wind damage and storm surge forecasting models developed by the National Oceanic and Atmospheric Administration, which enable simulation of a hurricane’s effects on coastal communities. Modeling and simulation can also be applied in situations where reality cannot be achieved. For example, for safety reasons a bioterrorism exercise cannot be conducted by releasing a deadly virus into the environment. However, it is still important to exercise the capabilities necessary to respond to this type of scenario. The use of modeling and simulation can realistically replicate variables such as disease propagation, radiation, and chemical attacks. Exercise Documentation Comprehensive, organized exercise documentation is critical to ensure an accurate account of the exercise is preserved. This in turn allows organizations to leverage past documentation to support future exercises and, more importantly, ensures that all critical issues, lessons learned, and corrective actions are appropriately captured to support improvement efforts. While most exercise materials are not sensitive or classified, some materials (e.g., scenario details) may necessitate restrictions on distribution. It is important for the exercise sponsor(s) to understand the specific requirements for security marking rules and requirements, access and dissemination, storage, disposal, and incident reporting of sensitive documents. Consideration should also be given to the accessibility of presentations and documents, such as making information available in alternative formats (e.g., large print, compact disc, Braille), closed captioning or another form of text display, or the provision of sign language interpreters. Homeland Security Exercise and Evaluation Program 3-13 Table 3.2 lists the key exercise design and development documents identified by the exercise type and relevant audience. Document Title Exercise Type Distribution Audience Situation Manual (SitMan) Seminar (Optional), Workshop (Optional), TTX, Game All Participants Facilitator Guide Seminar (Optional), Workshop (Optional), TTX, Game Facilitators Multimedia Presentation Seminar (Optional), Workshop (Optional), TTX, Game All Participants Exercise Plan (ExPlan) Drill, FE, FSE Players and Observers Controller and Evaluator (C/E) Handbook Drill, FE, FSE Controllers and Evaluators Master Scenario Events List (MSEL) Drill, FE, FSE, Complex TTX (Optional), Game (Optional) Controllers, Evaluators, and Simulators Extent of Play Agreement (XPA) FE, FSE Exercise Planning Team Exercise Evaluation Guides (EEGs) TTX, Game, Drill, FE, FSE Evaluators Participant Feedback Form All Exercises All Participants Table 3.2: Exercise Design and Development Documents Situation Manual SitMans are provided for discussion-based exercises as the core documentation that provides the textual background for a facilitated exercise. The SitMan supports the scenario narrative and serves as the primary reference material for all participants during conduct. The introduction provides an overview of the exercise—including scope, objectives and core capabilities, structure, rules, and conduct—as well as an exercise agenda. The next section of the SitMan is the scenario, which may be divided up into distinct, chronologically sequenced modules. Each module represents a specific time segment of the overall scenario, based on exercise objectives and scenario requirements. Each module is followed by discussion questions, usually divided by organization or discipline. Responses to the modules’ discussion questions are the focus of the exercise, and reviewing them provides the basis for evaluating exercise results. These discussion questions should be derived from the exercise objectives and associated core capabilities, capability targets, and critical tasks documented in each EEG. The SitMan generally includes the following information: • Exercise scope, objectives, and core capabilities • Exercise assumptions and artificialities • Instructions for exercise participants • Exercise structure (i.e., order of the modules) • Exercise scenario background (including scenario location information) • Discussion questions and key issues Homeland Security Exercise and Evaluation Program 3-14 • Schedule of events SitMan reference appendices may include, but are not limited to: • Relevant documents regarding plans, SOPs, etc. • Jurisdiction- or organization-specific threat information • Material Safety Data Sheet 6 or agent fact sheet, when applicable • A list of reference terms Facilitator Guide A Facilitator Guide is designed to help facilitators manage a discussion-based exercise. It usually outlines instructions and key issues for discussion during the event and provides background information to help the facilitator answer questions from participants or players. This guide may also include an evaluation section that provides evaluation staff members with guidance and instructions on evaluation or observation methodology to be used as well as essential materials required to execute their specific functions. Multimedia Presentation Multimedia presentations are often used to illustrate the general scenario for participants. They are given at the Start of Exercise (StartEx) and support the SitMan. The presentation should concisely summarize information contained in the written documentation. Like the SitMan, the multimedia presentation is also divided into distinct, chronologically segmented modules that, when combined, create the entire scenario. This presentation typically contains, at a minimum, the following information: • Introduction • Exercise scope, objectives, and core capabilities • Exercise play rules and administrative information • Modules that describe the scenario The presentations are intended to help focus and drive the exercise as well as add realism. A/V enhancements to a presentation include video or sounds that convey information to participants. Exercise Plan ExPlans are general information documents that help operations-based exercises run smoothly by providing participants with a synopsis of the exercise. They are published and distributed to the participating organizations following development of most of the critical elements of the exercise. In addition to addressing exercise objectives and scope, ExPlans assign activities and responsibilities for exercise planning, conduct, and evaluation. The ExPlan is intended to be seen by the exercise players and observers—therefore, it does not contain detailed scenario information that may reduce the realism of the exercise. Players and observers should review all elements of the ExPlan prior to exercise participation. 6 Material Safety Data Sheet (MSDS) or Product Safety Data Sheet (PSDS) is intended to provide emergency personnel with procedures for handling or working with a substance in a safe manner and includes information such as toxicity, health effects, first aid, storage, disposal, protective equipment, and handling procedures. Homeland Security Exercise and Evaluation Program 3-15 An ExPlan typically contains the following sections: • Exercise scope, objectives, and core capabilities • Participant roles and responsibilities • Rules of conduct • Safety issues, notably real emergency codes and phrases, safety controller responsibilities, prohibited activities, and weapons policies • Logistics • Security of and access to the exercise site • Communications (e.g., radio frequencies or channels) • Duration, date, and time of exercise and schedule of events • Maps and directions Player Handout The Player Handout provides key information to exercise players. A Player Handout can supplement the SitMan or ExPlan by providing a quick-reference guide to logistics, agenda or schedule, and key contact data for players. Controller and Evaluator Handbook The C/E Handbook describes the roles and responsibilities of exercise controllers and evaluators and the procedures they should follow. Because the C/E Handbook contains information about the scenario and about exercise administration, it is distributed to only those individuals designated as controllers or evaluators. The C/E Handbook may supplement the ExPlan or be a standalone document. When used as a supplement, it points readers to the ExPlan for more general exercise information, such as participant lists, activity schedules, required briefings, and the roles and responsibilities of specific participants. Used as a standalone document, it should include the basic information contained in the ExPlan, and detailed scenario information. The C/E Handbook usually contains the following sections: • Assignments, roles, and responsibilities of group or individual controllers and evaluators • Detailed scenario information • Exercise safety plan • Controller communications plan (e.g., a phone list, a call-down tree, instructions for the use of radio channels) • Evaluation instructions The Controller portion of the C/E Handbook, sometimes known as Control Staff Instructions (COSIN), provides guidelines for control and simulation support and establishes a management structure for these activities. This section provides guidance for controllers, simulators, and evaluators on procedures and responsibilities for exercise control, simulation, and support. The Evaluation portion of the C/E Handbook, sometimes known as the EvalPlan, provides evaluation Homeland Security Exercise and Evaluation Program 3-16 staff members with guidance and instructions on evaluation or observation methodology to be used, as well as essential materials required to execute their specific functions. Controller and Evaluator Packets While C/E Handbooks contain detailed information that should be read and understood well in advance of the exercise, Controller Packets and Evaluator Packets are provided immediately prior to an exercise to controllers and evaluators respectively. The packets contain key information from the C/E Handbook and additional information specific to the functional area in which the given controller or evaluator will be working. This information is needed during exercise play in order to carry out control and evaluation responsibilities. Both Controller Packets and Evaluator Packets should contain the following: • Essential C/E Handbook information • Ground truth document, detailing key elements of the exercise scenario (primarily used for prevention-focused exercises) • MSEL, including injects and events for each responsible controller and evaluator • Appropriate EEGs • Maps and directions Master Scenario Events List A MSEL is typically used during operations-based or complex discussion-based exercises and contains a chronological listing of the events that drive exercise play. Each MSEL entry should contain the following at a minimum: • Designated scenario time • Event synopsis • Controller responsible for delivering the inject, with controller or evaluator special instructions (if applicable) • Intended player (i.e., agency or individual player for whom the MSEL event is intended) • Expected participant response (i.e., player response expected upon inject delivery) • Objective, core capability, capability target, and/or critical task to be addressed (if applicable) • Notes section (for controllers and evaluators to track actual events against those listed in the MSEL, with special instructions for individual controllers and evaluators) Scenario timelines listed in a MSEL should be as realistic as possible and based on input from SMEs. If the activity occurs sooner than the MSEL writers anticipated, then controllers and evaluators should note the time it occurred, but play should not be interrupted. Controllers delivering MSEL injects will either be co-located with players in the venue of play, or they will reside in a SimCell. A SimCell is a location from which controllers deliver messages representing actions, activities, and conversations of an individual, agency, or organization that is not participating in the exercise but would likely be actively involved during Homeland Security Exercise and Evaluation Program 3-17 a real incident. Prior to StartEx, the mechanisms for introducing injects into exercise play should be tested to ensure that controllers are aware of the procedures for delivering MSEL injects and that any systems that will be used to deliver them are functioning properly. The three types of descriptive MSEL events that support exercise play include: 1. Contextual injects introduced to a player by a controller help build the exercise operating environment and/or keep exercise play moving. For example, if the exercise is designed to test information-sharing capabilities, a MSEL inject can be developed to direct an actor to portray a suspect by behaving suspiciously in front of a law enforcement player. 2. Expected action events reserve a place in the MSEL timeline and notify controllers when a response action would typically take place. For example, during an FSE involving a chemical agent, establishing decontamination is an expected action that the players will take without the prompting of an inject. 3. Contingency injects are provided by a controller or simulator to players to ensure play moves forward to adequately evaluate performance of activities. For example, if a simulated secondary device is placed at an incident scene during a terrorism response exercise, but is not discovered, a controller may want to prompt an actor to approach a player and state that he or she witnessed suspicious activity close to the device location. This should prompt the responder to discover the device, resulting in subsequent execution of the desired notification procedures. MSELs are typically produced in long formats, short formats, or both. Short-form MSELs usually list injects in a single row in a spreadsheet format. These can be used as a quickreference guide during exercise play or projected onto a large screen in a control cell or SimCell. Long-form MSELs are used when greater detail is necessary; they include more detailed descriptions, exact scripting language for actors and simulators, and more detailed descriptions of expected actions. Extent of Play Agreements XPAs can be used to define the organizations participating in the exercise as well as their extent of play (e.g., one fire station for 8 hours, county Emergency Operations Center [EOC] activated at level A for 24/7 exercise operations). These agreements are formed between exercise participants and the exercise sponsor, and can be vital to the planning of an exercise, recruitment of evaluators, and development of support requirements. Exercise Evaluation Guides EEGs are intended to help evaluators collect relevant exercise observations. These documents are aligned to objectives, and document the related core capability, capability target(s), and critical tasks. Each EEG provides evaluators with information on what they should expect to see demonstrated or hear discussed. For more information on EEGs, see Chapter 5: Evaluation. Participant Feedback Form At the end of an exercise, participants may receive a Participant Feedback Form that asks for input regarding observed strengths and areas for improvement that players identified during the exercise. Providing Participant Feedback Forms to players during the exercise wrap up activities allows them to provide their insights into decisions made and actions taken. A Participant Feedback Form also provides players the opportunity to provide constructive criticism about the Homeland Security Exercise and Evaluation Program 3-18 design, control, or logistics of the exercise to help enhance the planning of future exercises. At a minimum, the questions on the Participant Feedback Form solicit the following: • Strengths and areas for improvement pertaining to the implementation of participating agencies and organizations’ policies, plans, and SOPs; and • Impressions about exercise conduct and logistics. Information collected from feedback forms contributes to the issues, observations, recommendations, and corrective actions in the AAR/IP. Feedback forms can be supplemented by the conduct of a Hot Wash immediately following the exercise, during which facilitators, controllers, and evaluators capture participant perspectives on the key strengths and areas for improvement identified during the exercise. Waiver Forms Each actor should receive a waiver form prior to the exercise. Signing this form waives liability for all exercise planners and participants. Exercising entities should use discretion when recruiting actors under the age of 18 because of additional challenges and concerns related to liability. If the exercise requires volunteers younger than 18-years-old, parents or legal guardians must sign their waiver forms. Weapons and Safety Policy All exercises, where applicable, should employ a written weapon and safety policy that is in accordance with applicable State or local laws and regulations. Exercise sponsors should coordinate the application of this policy with the appropriate safety and/or legal departments as necessary. Media or Public Affairs Guidance Members of the media have the unique ability to fulfill an important function before, during, and after an exercise. Prior to an exercise, they inform the public that an exercise will take place, and raise public awareness that the community is preparing for disasters. During an exercise, they can facilitate the validation of public information plans and procedures. Following an exercise, the media may release details to the host community on the state of its preparedness, if the exercise planning team leader provides such information. Therefore, exercise sponsors should work to incorporate media-related issues into exercise planning. Press Release Prior to an exercise, the exercise planning team should develop a written press release to disseminate to media outlets, including web-based and/or social media outlets, as appropriate. This release informs the media and the public about general exercise information. Additionally, this information can be distributed to observers, elected and appointed officials, and other VIPs. This release should not contain detailed scenario information, such as the type of threat or hazard, nor should it contain information that might hinder meeting exercise objectives if a participant were to see it. Typically, the contents of a media or public information release include the following: • Introduction, including sponsor and exercise program information • Exercise scope and objectives Homeland Security Exercise and Evaluation Program 3-19 • General scenario information • Participating agencies or disciplines Public Announcement Public announcements should be made prior to any exercise involving public space or space that will be viewable by the public. This precaution helps avoid confusion on the part of the public. It will also help the public avoid congestion near the exercise site by providing suggestions for alternate routes. Announcements can be made through local media, through mass mailings or pamphlets, and/or on signs near the exercise site. Media Policy The agency or organization sponsoring the exercise should decide whether to invite media representatives to the exercise. If invited, media representatives should have an opportunity prior to the exercise to conduct interviews with key planners and participants. At discussion-based exercises, media representatives should not be present during the discussion of any potentially sensitive information, and filming exercise conduct should be avoided so as not to inhibit or hinder discussion or the flow of play. During operations-based exercises, media representatives may be allowed to film certain activities but should be cautioned not to interfere with exercise play or film any sensitive operations. Unless media representatives are invited to participate in the exercise, a guide— typically a public information officer or designee—should escort media representatives at all times. If mock media or exercise controllers simulating the real-world media are employed during an exercise to test public affairs training, they should be kept completely separate from any real-world media representatives who may be observing the exercise. Exercise Development Exercise development involves planning for the critical elements of exercise conduct: logistics, control, and evaluation. Planning for Exercise Logistics Logistical details are important, but often overlooked, aspects of an exercise. They can make the difference between a smooth, seamless exercise and one that is confusing or even unsafe. Venue Facility and Room Meetings, briefings, and exercises should be conducted in facilities that are appropriate for the exercise scope and attendance. Planners should also ensure that all environmental and historical preservation documentation is completed if required. Facilities should be reserved solely for exercise purposes and should be accessible to all participants and free from distractions. When selecting a facility and room for exercise planning or conduct, planners should account for the following considerations: • Ensure there are enough tables and chairs for every relevant participant. • Arrange tables to best suit the meeting or exercise (e.g., U-shaped layout for exercises requiring facilitation and participant interaction). Homeland Security Exercise and Evaluation Program 3-20 • Select a facility with room acoustics that facilitate ease of discussion. • Select a facility with accessibility of parking and restrooms for all participants. Audio/Visual Requirements A/V requirements are identified during the design phase including individuals assigned to ensure equipment is properly functioning. Supplies, Food, and Refreshments Exercise planners should not assume participants will bring necessary supplies with them. Writing utensils, notepads, easels, copies of plans and procedures, name badges, and any other equipment deemed necessary should be procured prior to exercise conduct and provided to participants. The exercise planning team should also consider whether food and refreshments can be provided for participants and observers, in accordance with applicable funding guidance or venue policies. For discussion-based exercises, it is often beneficial to have a working lunch provided to minimize disruption to play. For operations-based exercises, hydration of participants is an important consideration. Badging and Identification For security purposes, all exercise participants should wear some form of identification. Although some players may wear their uniform...
Purchase answer to see full attachment