USACE Experience in Implementing Risk Analysis for Flood Damage Reduction Projects

For about a decade now, the U.S. Army Corps of Engineers (USACE) has performed its flood damage reduction project development and improvements studies in an expanded risk analysis (RA) framework. This practice is a substantial change from policies and practices of the past, and was achieved despite significant impediments, both internal and external to the USACE. Sustaining the RA policy requires continued improvements in concepts and methods, as well as acceptance by the USACE professional Communities of Practice and the customers and stakeholders served by USACE. Hurricane Katrina and the devastation of New Orleans due to the failure of the protection system have brought increased attention by the nation to notions of risk and uncertainty in project performance. In this context, a broader embracing of RA in managing the nation's critical infrastructure may help sustain the implementation of these procedures within the USACE and the profession. This paper provides a summary of the impetus for USACE adopting RA for flood-risk-reduction project evaluation and its consequent maturing and applications over the past decade. Several papers have been published that summarize concepts and applications (e.g. Dotson 1994, Burnham 1995, Davis 2003). The present state of the RA effort within the USACE is described and illustrated here with two key applications. External challenges and concerns about the USACE adoption of RA were significant and culminated in two formal reviews by the National Research Council of the National Academy of Sciences (National Research Council 1995, 2000); the National Research Council findings are summarized here. This paper outlines deficiencies and weaknesses in present policy and methods, summarizes research needs, and presents our perception of the way forward. It should not be surprising to informed observers that for a project-focused, action agency such as the USACE, the initial impetus for embracing a more formal framework for risk analysis for project development was stimulated by a specific project. Circa late 1980's/early 1990's, a levee project in the Northeast U.S. had been authorized by Congress for implementation, and USACE proposed the construction start in accordance with its normal budget cycle. The project was to take a few years to construct and was estimated to cost approximately $30 M. The "new start" failed to be included in budget legislation for that and the next year's budget cycle, but was again proposed in the following budget cycle – a common practice. When the project was proposed again, however, there was a change in the cost estimate; the project was now to cost $45 M. A 50% increase in cost in just a few years (construction had not yet begun) was noteworthy and questioned by Office of Management and Budget (OMB) and USACE budget officials. The expectation among those questioning the cost increase was that there must be an increase in proposed project protection, or that some significant change in construction methods or materials had become necessary, and thus the cost increase. The explanation by the engineers responsible for the project was that they had concluded that the freeboard criterion used for the project with the $30 M estimate was not appropriate; the freeboard had been increased, resulting in the increased cost. By way of explanation, freeboard is an increment of height added to the design flood stage for a levee to account for uncertainty in the design flood stage and to provide a buffer for other engineering uncertainties. As this project illustrates, small increments of levee height can result in large increases in cost because a height increase translates to a substantial increase in the footprint of the levee, and thus significantly more material and right-of-way are required (Figure 1)). Conceptual description of levee freeboard. Further questioning of the engineers was met with staunch defense of the "design criteria" as needed to ensure project performance, and insistence that there was no creditable improvement in the project's expected performance and economic benefits. A stand-off ensued: budget and senior officials were adamant that an increase in levee height must result in more protection and benefits; the engineers insisted that the issue was simply a choice of design criteria. In the late 1980s, freeboard and its contribution to project protection had been a point of contention for some time within the USACE with engineers maintaining that it is simply a design issue to ensure that the design flood can be contained, and economists and project formulation professionals arguing that some benefit (additional protection) should be credited for the increased height. A compromise was adopted wherein benefits would be taken for protection afforded by half the freeboard height (Moser 1991). That debate continued until the USACE adopted the expanded risk analysis procedures, as explained below. The upshot of this dialogue was that future flood damage reduction project proposals forwarded through OMB were required to be cast in a risk and uncertainty framework. OMB, and analysts within USACE, pointed out to policy makers that RA concepts had been included in the Principles and Guidelines for Water and Related Land Resources – P&G (Water Resources Council 1983), and thus, ample precedent existed for specific policies to evolve, better representing uncertainties pertaining to project performance during the planning process. Moreover, the evaluation of flood risk reduction projects has always been an exercise in risk analysis, as the occurrence of large flood events is random and described probabilistically. At this time, new American Society of Civil Engineers model building codes included a form of probabilistic loading for structural analysis, and the time had simply come to modernize policy and methods for flood damage reduction. The form of RA policy began to take shape during a seminar on levee freeboard held in August 1991 in Monticello, Minnesota. Two papers, (Davis 1991 and Moser 1991) proposed that instead of refining the economic justification of freeboard, the principles of risk and uncertainty analysis be adopted, thus eliminating the need for explicitly specifying a freeboard allowance. These proposals formed the basis for subsequent policy development. A policy advisory in the form of an Engineer Circular (draft Engineer Regulation) was sent to the USACE field offices shortly thereafter (1992) stating the primary components of the risk analysis policy. There was no accompanying technical guidance nor analytical tools and methods. This policy issuance generated reams of comments, literally, from USACE district and division offices. Few comments were supportive. Over the ensuing five years, concepts were sharpened, tools and methods developed, manuals and documentation prepared, and training sessions held. The formal documentation of the policy was a joint Engineer Regulation (ER 1105-2-101) issued by USACE Planning and Engineering directorates (USACE 1996a– updated and revised in January 2006), followed shortly thereafter by publication of an Engineer Manual (EM 1110-2-1619) (USACE 1996c). Technical and policy refinements were issued over the following ten years. ER 1105-2 101 stated that: The ultimate goal is a comprehensive approach in which the values of all key variables, parameters, and components of flood damage reduction studies are subject to probabilistic analysis. Flood damage reduction projects are planned and constructed by Federal, state, and local government agencies, and private businesses. Federal agencies undertake water resource investments for such measures as dams or levees within the broad confines of the Principles and Guidelines – P&G (WRC 1983). The P&G generally requires that Federal projects contribute to national economic development, which means that they must be economically justified because benefits exceed costs. USACE has planned and constructed many of the nation's major flood damage reduction and coastal protection projects, including the majority of levee systems protecting major urban areas across the U.S. USACE implementation of the P&G requires that flood damage reduction projects be planned and designed so that the project scale maximizes the net national economic development benefits; however, other considerations can suggest larger or smaller projects than the project that maximizes national economic development (Yoe and Orth 1996). For example, levee heights are often compared to that which would satisfy the Federal Emergency Management Agency (FEMA) National Flood Insurance Program base flood level for excluding the floodplain from mandatory flood insurance. This National Flood Insurance Program base flood, referred to as a 100-year protection level, is often mistaken by local communities as the Federal standard for urban flood protection (Davis 2007). In fact, there is no Federal standard for flood protection. In those urban settings where these principles suggest levee height less than National Flood Insurance Program base flood protection, the heights are generally increased to that level so that the levees may be certified for National Flood Insurance Program purposes. The resulting project must still be economically justified and the local sponsor may be required to pay the entire cost for the increment of levee height between the National Economic Development project and the National Flood Insurance Program base flood protection project. For non-Federal levee projects, the target selected by local agencies and the private sector is often simply to provide protection from the National Flood Insurance Program base flood so that the levee system may be certified and the protected floodplain is free of development controls. (Levee certification is discussed again later in this paper.) It is important to note that project development principles and policies, such as maximization of national economic development, are not changed by USACE adoption of RA as the analysis framework for flood damage reduction projects. The purpose of the 1992 USACE risk analysis policy is to improve decision making and engender confidence in the project formulation and evaluation process by quantifying risk and disclosing uncertainty in key data and parameters. The fundamental tenants of the policy may be summarized as: make accurate and unbiased estimates of the probability and consequences of flooding, publish and communicate those findings, and make such information part of the deliberative process by the professionals and residents of the community; acknowledge uncertainties associated with project performance and quantify, publish, and communicate that information, thus making it a meaningful component of the deliberative process. (The key information items that must be addressed are the uncertainties in discharge-frequency, stage-flow, geotechnical and structural performance, project operations, and project costs and benefits); and emphasize residual risk (probability and consequence of the exceedance of project capacity to public safety, lifeline security, and local and regional economic impact) by conducting residual risk analysis, and by documenting and communicating those findings to the deliberative project development process. The way RA is implemented by USACE districts for project development studies is only modestly changed from past practice without RA. Historically, information is gathered and developed documenting the flood threat and properties at risk. The local community is engaged in better defining the objectives and offering potential solutions. The study floodplain is parsed into "reaches" to facilitate computations (Figure 2)). For each reach, data are developed to quantify frequency of flooding and threatened properties. The approach for the analysis (in this example, a riverine reach) is to develop relationships between flow and frequency, stage vs flow, and stage vs damage. Flood damage reduction measures such as levees, floodwalls, relocation and evacuation, dams, and bypasses are formulated, their costs computed, and resulting changes to the above relationships defined. For alternative formulations, the relationships are conjoined to compute expected damage, damage reduced, and thus benefits. The overall output of the analysis for an alternative is 1) the project cost and benefits, and thus net benefits, and 2) the residual flood risk, typically annual exceedance probability with the project in place, and long-term risk, which can be compared to the "without–project" risk. 3-4B provide illustrations of these analysis principles. Floodplain parsed into reaches. Conjoining of relationships between flow, probability, stage and damage. H&H relationships altered by presence of a Reservoir. H&H relationships altered by presence of a Levee. New USACE risk analysis procedures bring several elements of uncertainty directly into the analytical computations, resulting in a more complete and deliberate analysis of project costs and performance, and consequences of capacity exceedance. Instead of single-valued relationships described in 3-4B, the uncertainty in each is estimated and incorporated into the analysis, as illustrated in Figure 5). Again using the riverine setting as an example, uncertainty in the relationship between flow and frequency is estimated by developing confidence bands as prescribed in "Guidelines for Determining Flood Flow Frequency" (IACWD 1982) for analytical flow-frequency relationships, or invoking "order statistics" as documented in Engineer Technical Letter ETL 1110-2-537 (USACE 1997) for non-analytic frequency curves. For the stage-flow uncertainty, a variance in flood stage is estimated from several sources: historic floods, high water marks, sensitivity analysis on hydraulic parameters, analysis of gauged rating data, or perhaps reference to published values. For the stage-damage relationship, parameters of structure first-floor-elevation, structure value, and damage fragility curves are sampled in a Monte Carlo framework to develop a function and its associated variance. Suggested methods for developing the uncertainty data are documented in EM 1110-2-1619. For structures such as levees that might be aged or in degraded condition, flood stage versus probability-of-failure relationships are developed by methods as suggested in "Risk-Based Analysis in Geotechnical Engineering for Support of Planning Studies" (USACE 1999). Uncertainty in flow, stage, damage and probability relationships. Conjoining the evaluation functions to compute flood risk reduction performance (an uncertainty-weighted annual exceedance probability), expected damage, expected damage reduced, expected benefits, and ultimately, expected net benefits is again performed. With RA, instead of single estimates for these important decision parameters, probability density functions are developed. Information required for "without project" conditions and each proposed alternative (ER 1105-2-101) includes probability density functions of annual exceedance probability, benefits, and net benefits. The computations are performed in a Monte Carlo framework with computer software "HEC Flood Damage Analysis." The sampling and computation scheme is described in the Flood Damage Analysis User's Manual (USACE 1998). Decisions about project scale and subsequent investment are essentially made as in the past, in an open, transparent, collaborative environment in which USACE engages the affected community, stakeholders, other Federal, state, and local agencies, and interested public and non-governmental organizations in a fluid give-and-take that is the essence of the U.S. democratic process. In the context of the present USACE risk analysis policy, the decision process can best be characterized as "risk informed" rather than be thought of as a formal risk-based decision process; there are not yet any prescribed risk-based decision criteria. As noted above, however, there are some clear boundaries as reflected in Federal law (environmental statutes, for example), regulations and policies (P&G and agency rules), and some notions fundamental to Federal investments such as public safety (project should improve public safety and not increase risk from flooding), equity (all citizens are treated in a balanced and fair manner), and requirement that project have multiple transparent beneficiaries (not enrich special interests). FEMA administers the National Flood Insurance Program, a Federal program that seeks to stimulate wise floodplain land use decisions and create a fund to indemnify owners of properties in the floodplain from flood damage they might incur. The National Flood Insurance Program is described in various FEMA documents that may be retrieved from the website: www.floodsmart.gov. Flood inundation maps are an important component in administration of the National Flood Insurance Program. A key element in drawing flood maps, and hence setting insurance rates, is that of determining levees should be credited with protecting associated from the base certification is a technical for the floodplain in there is that the levee protecting the the base USACE that their adopted risk analysis policy and methods had for floodplain and associated FEMA levee certification and engaged FEMA in approach to levee certification that risk analysis was between USACE and FEMA in and published in a policy to Corps of Engineers field offices (USACE The elements of the risk-based levee certification guidance 1) the elements and principles of FEMA regulations levee certification (e.g. a of of the base 2) the freeboard components of the certification with a probability to as as that they at of the base flood but protection need not be than that of the base flood may for levees than and to the quantify and uncertainty in the performance of the geotechnical and structural of the levee system in the analysis. The risk-based levee certification policy has been in in USACE issuance of the policy in of the RA policy during studies and the American late National Research Council 1995, Davis 2007). The of is at the of the and American in and has been to from both The and are by a of dams and other protection but the flood threat from the American (Figure A flood in the system and again to flooding from the American new of studies to improve protection. of was from new major to an to changes in for as well as various of increases and levee system of the from to and for new and American to the of the of the many large with as well as the of the and there is significant uncertainty in the system and it would various were The RA framework parsed the system into components for which and associated uncertainty be project and consequences of challenges were addressed to a risk analysis of the developing curves with representing with uncertainty to develop flow frequency the of the levee system with its associated and economic benefits and associated uncertainty for the of The approach taken and studies performed are documented in project and summarized in a paper at the on in in 1) summarizes from analysis of key the expected values for key performance parameters. Uncertainty information for these parameters was developed and in project documents but are not here. This an where the risk analysis information the Corps and other agencies better performance of a the This was by a local to the such that the the not and thus would the local performance RA, the project would be characterized as the level of flood protection and would certification for the FEMA standard 100-year this would water to the of the levees, however, its protection is In this there is for uncertainty is an important The RA the of the performance of the by a more accurate expected exceedance estimate in and that the of the 100-year FEMA flood was less than a RA, the of the alternative have been but its would not have been The of risk analysis for this project was not without its and At the time it to communicate the concepts of uncertainty and performance to decision makers to of protection with no of In the decade some has been but of risk analysis findings is still a with using the risk analysis a local the of the probability for of the to be only a of the to a The sent out to the the performance of this alternative as to to an that had a of still with to flood protection for in are for a of and the levees are to project are including better use of With all these measures at some time in the still have less protection than New Orleans Hurricane Katrina The of risk analysis to a levee project in is a that adopting a new policy is not always or in decision USACE a levee project for following P&G policy of the project to national economic benefits National Economic Development subject to public safety and other policy. The National Economic Development levee height was compared to the new USACE levee certification adopted for risk analysis and to be to USACE was to issue the certification the local sponsor was not that the levee should be certified it not the local of or the National Flood Insurance Program freeboard criteria. The information is in 2) and in Figure levee heights for (not to The reflected here is that of a with and uncertainty, less than the to ensure a level of protection on risk analysis. The local sponsor that the consequence was a by Congress for the National Research Council of the National Academy of Sciences to the USACE risk analysis policy. The of the National Research Council is summarized below. the local sponsor Congress to the levee project at the than the of and it is was the of RA in this as a failure in that the findings were the other the of the technical in this USACE, is to decision makers in a balanced and unbiased and the democratic decision process were better informed about the performance of the project than would have been the Two National Research Council studies the 1992 and Uncertainty This served on both In a National Research Council was to the USACE for on the American the developed RA procedures had been adopted for the of that National Research Council it to on the The concluded that the new procedures were an and and that of uncertainty should result in a better of the uncertainty of flood risk and damage reduction the had a of concerns about risk and uncertainty were and they were supportive. important to this is the National Research Council (National Research Council which the of a explicitly to provide a of the new RA were that new are a significant and the Corps should be for embracing but and for from a approach that has been by the a of that it needed to be including the used to the computations better between and uncertainty, the of and other uncertainties including levee performance, and the methods used to uncertainties in into the overall of project performance. With to the levee certification the the of risk and uncertainty and their with design and policy to be they that levee certification program not some level of of the 100-year flood, but rather exceedance probability – the probability that an protected by a levee system be by any potential As they annual exceedance probability of flooding should uncertainties from both and The this would provide a more level of flood protection. that this not be implemented the suggested that: the of annual exceedance probability is adopted as the key criterion for levee the that the Corps and FEMA a single probability for levee certification has to this to provide a of the that would help the public and the engineering the of the new methods, and would the computations to be more accurate and The USACE risk analysis policy for flood damage reduction is essentially the as it was issued in The of was in and the There is among its that decisions are now better informed thus, are expected to be better than adopting the RA policy. That the adoption of the policy and its by USACE field offices to be a in of RA principles and is not There has been of to normal and of training and technical guidance important development is the of within the USACE that has come about as USACE its and division offices. the by USACE in the safety that are in a risk analysis context have in increased attention to and of concepts of risk and uncertainty within the The in New Orleans has attention to flood risk, in the coastal This has a of to USACE effort to risk analysis procedures for flood damage reduction projects. A levee safety program in the of Katrina is developing new tools for levee safety in an and expanded risk analysis framework. A or in the RA policy is its failure to decisions in the of risk and uncertainty in the of levee certification for the policy that risk and uncertainty information be developed and in but not suggest to policy and guidance on to with information about the residual risk, uncertainty in project performance, and net benefits would in this information to and that need to be addressed for risk analysis procedures to and that can better and on functions to sampling the of the probability of each to risk and New analytical methods and to better the performance of levee systems as an of components floodwalls, methods for uncertainty for non-analytic frequency curves. to other performance functions and their associated uncertainties. A more for risk and for of methods, and are key that should be in with the adoption of an expanded and more complete evaluation of project performance and uncertainties. analysis safety and the of our analysis. need to make accurate and unbiased estimates of the probability of flooding and of and communicate that analysis on to acknowledge the uncertainty associated with a project and its performance, and to quantify, and communicate that analysis should emphasize the residual risk that a project is includes both the probability of flooding and the consequences of such flooding, which the of a In that that the USACE effort to an expanded risk analysis framework is the critical tools the USACE and other agencies and professionals need to flood risk reduction projects in the Davis from USACE in where had served for years as the of the Engineering is now a of the USACE for Water Resources in the areas of flood risk levee and for Water USACE, in engineering from has for USACE at its Engineering for on the use of risk analysis in planning and design and frequency analysis. Engineering USACE, has been a at for years during which time with the USACE and research on operations, risk analysis, and a of to and flood frequency analysis. of Civil and