On formally integrating science and policy: walking the walk

The contribution of science to the development and implementation of policy is typically neither direct nor transparent. In 1995, the U.S. Fish and Wildlife Service (FWS) made a decision that was unprecedented in natural resource management, turning to an unused and unproven decision process to carry out trust responsibilities mandated by an international treaty. The decision process was adopted for the establishment of annual sport hunting regulations for the most economically important duck population in North America, the 6 to 11 million mallards Anas platyrhynchos breeding in the mid-continent region of north-central United States and central Canada. The key idea underlying the adopted decision process was to formally embed within it a scientific process designed to reduce uncertainty (learn) and thus make better decisions in the future. The scientific process entails use of models to develop predictions of competing hypotheses about system response to the selected action at each decision point. These predictions not only are used to select the optimal management action, but also are compared with the subsequent estimates of system state variables, providing evidence for modifying degrees of confidence in, and hence relative influence of, these models at the next decision point. Science and learning in one step are formally and directly incorporated into the next decision, contrasting with the usual ad hoc and indirect use of scientific results in policy development and decision-making. Application of this approach over the last 20 years has led to a substantial reduction in uncertainty, as well as to an increase in transparency and defensibility of annual decisions and a decrease in the contentiousness of the decision process. As resource managers are faced with increased uncertainty associated with various components of global change, this approach provides a roadmap for the future scientific management of natural resources. The contribution of science to the development and implementation of policy typically occurs in two distinct steps. Investigations believed to be relevant to policy are planned and carried out by scientists, who then present their results to those who make policy and decisions. The actual integration of science and policy then occurs via the mental processes of policymakers and decision-makers, who are expected to use the new scientific information. This approach can be inefficient, frequently leading to complaints by scientists that their results are not used or appreciated, and by policymakers that the science is not directly relevant to decisions. For complex natural resources problems, the inability of this approach to cope with uncertainty can also jeopardize the science–policy interface, resulting in management failures (van der Sluijs 2005). Adaptive resource management (ARM) is a decision process developed to use management actions themselves to simultaneously achieve objectives and learn about system responses to actions, permitting better decisions through time (Walters 1986). Unfortunately from a communication perspective, the ARM label has been applied to a variety of processes, many of which bear little resemblance to the process that we describe here (Rist, Campbell & Frost 2013; Westgate, Likens & Lindenmayer 2013). The original ARM approach was developed for recurrent decisions, and learning occurs via the incorporation of a scientific process within the larger decision process. Specifically, uncertainty can be characterized by models representing competing hypotheses of system response to management actions. At each decision point, an action is selected based on objectives, available actions, specified models of system behaviour, weight of evidence for each model and the current estimate of system state (from monitoring), using an approach such as dynamic optimization. Conditional on the system state and selected action, each model provides a prediction about expected system state at the next period of monitoring. The comparison of predictions with the new estimate of system state provides a basis for updating relative degrees of confidence in the models, with confidence accruing to models that predict well and decreasing for those predicting poorly. The revised degrees of confidence and the new estimate of system state are then used to select a new action at the next decision point, and the process continues. This process differs in key respects from the more commonly used two-step approach described above. The process requires substantial collaboration between scientists and managers, as well as a blurring of their traditionally distinct roles. The step of comparing estimated system responses with model-based predictions as a means of discriminating among competing models is the scientific approach to learning (Platt 1964), and embedding a formal scientific process within the larger decision process ensures both maximal relevance of the science and direct use of new scientific results in each decision. Increases in transparency and defensibility result from use of dynamic optimization to compute the optimal decision directly, rather than relying on the complex mental processes of decision-makers. Development of theory underlying ARM occurred primarily in the 1980s (Walters 1986), and its logic and conceptual appeal have led to many recommendations for its adoption. However, actual, sustained applications are rare and prior to 1995, were virtually non-existent (Westgate, Likens & Lindenmayer 2013). The process of managing the sport harvest of waterfowl is a trust responsibility of FWS under the Migratory Bird Treaty Act, is continental in scope and involves the federal governments of the USA and Canada, every state and province, and an active community with hunting and conservation interests. The economic impact of waterfowl hunting is substantial (>1 million hunters; $1·8 billion for migratory bird hunting in the USA; U.S. Department of the Interior, U.S. Fish & Wildlife Service, & U.S. Department of Commerce, U.S. Census Bureau 2011). The history of this management problem has been marked by substantial scientific uncertainty, adversarial relationships among various stakeholder groups, and frequent distrust of decision-makers. There was a clear need for a process that was capable of reducing uncertainty, while providing decisions that complied with international treaties, satisfied diverse stakeholders, and were transparent and defensible. In 1992, one of us (FAJ) led the formation of an ad hoc working group of waterfowl managers and biologists from federal and state agencies to consider use of ARM for the setting of waterfowl hunting regulations. The group met multiple times between 1992 and 1995 first to discuss the basic concepts underlying ARM and then to consider how such an approach might be implemented for harvest management of mid-continent mallards, the largest mallard population in North America and the subject of substantial controversy over hunting regulations (Johnson et al. 1993; Nichols 2000). The 1994–1995 hunting season was especially frustrating for many waterfowl stakeholders, as a political intervention led to regulations that differed from those recommended via the established regulatory process. It was clear that a more objective process for establishment of waterfowl hunting regulations was highly desirable. Thus, in 1995 at the advice of her technical experts, the Director of FWS took the unprecedented step of formally adopting an ARM process for the annual establishment of sport hunting regulations for mid-continent mallards. Despite the conceptual clarity and substantial theoretical development of ARM (Walters 1986), in 1995, there was no blueprint for implementing such a programme. The 'deliberative' or 'set-up phase' (Williams, Szaro & Shapiro 2007) for this problem entailed numerous decisions and substantial effort. Development of an explicit statement of objectives required multiple meetings of FWS scientists with stakeholders who included representatives of multiple state wildlife and natural resource agencies and the Canadian government. The objective was specified to maximize weighted mallard harvest over a long time horizon, with weights designed to devalue harvest when expected population size does not exceed specified values. Specifically, weights are maximal when predicted mallard abundance exceeds a specified threshold (approximately 8 500 000 breeding birds, derived from the goal specified in the North American Waterfowl Management Plan, U.S. Department of the Interior & Environment Canada 1986) and decrease as expected population size drops further below the threshold. Four management alternatives were specified as sets of hunting regulations that varied from very conservative (smaller expected harvest) to very liberal (greater expected harvest; see U.S. Fish & Wildlife Service 2013). Four competing models of population response to hunter harvest were developed as the combinations arising from two models for reproductive rate and two for survival rate. Reproductive rate was modelled as a function of a dynamic environmental covariate (number of wetlands) and size of the mallard breeding population, with the two models differing in the strength of negative density dependence (weak vs. strong; see Johnson et al. 1997). The two models predicting annual survival rate as a function of harvest rate reflect maximal (additive mortality) and minimal (compensatory mortality) effects of hunting (Anderson & Burnham 1976; Cooch et al. 2014). Knowledge of the best approximating model in this set is important to annual regulatory decisions, with optimal hunting regulations being most restrictive under the 'weakly density-dependent reproduction + additive mortality model', and most liberal under the 'strongly density-dependent reproduction + compensatory mortality' model (Johnson et al. 1997). Continental monitoring programmes permit inference about duck population size, mortality and reproduction (Smith et al. 1989). Software for solving the stochastic dynamic optimization problem for multiple models was developed (Lubow 2001). This substantial effort was completed in time to establish regulations for the 1995–1996 hunting season, mainly by a small group of FWS scientists. Results of the effort were summarized and published to promote transparency and openness (Johnson et al. 1997). Annual hunting regulations for mid-continent mallards have been established by FWS using the ARM process for the past 20 years, and annual reports are published summarizing results from each hunting season (e.g. U.S. Fish & Wildlife Service 2013). The 'iterative phase' of the process begins each spring, with a large-scale, aerial survey providing an estimate of the number of breeding mallards. This estimate is compared with the predictions of each model made the previous year, and relative degrees of confidence in the models are modified accordingly (using Bayes' theorem). Stochastic dynamic optimization is then used to select the regulations for the upcoming hunting season. Population size changes as a result of regulations and environmental factors occurring throughout the year. The new abundance is estimated via monitoring the following spring, and the process continues. Over this 20-year period, mallard abundance has fluctuated (Fig. 1), but has been maintained near the desired level specified in the North American Waterfowl Management Plan (U.S. Department of the Interior & Environment Canada 1986). Selected regulations have been optimal, conditional on the specified objectives, regulations packages, model set, evolving model weights and year-specific estimates of mallard abundance. The scientific process of comparing each year's estimate of abundance with predictions of each of the four competing models has led to evolution of relative degrees of confidence (referred to as model 'weights', numbers between 0 and 1 that sum to 1 for the members of the model set) in the different models (Fig. 1). Although science is typically described as a cumulative endeavour, its practice can be very different, with most studies citing, but not formally building on, results of previous work. Few programmes of management or science have produced the sort of formal, evidentiary learning apparent in Fig. 1. This evolution of model weights has been accompanied by the evolution of the optimal policy (Johnson 2011), which has become increasingly more restrictive over time (Fig. 2), as expected given the two models that have accumulated the highest model weights and that consequently now have the greatest influence on optimal decisions. Specifically, predictions based on the models with weakly density-dependent reproduction have been substantially better than those of the two models with strongly density-dependent reproduction. Predictions of the models representing additive mortality have performed somewhat better than those representing a compensatory process. The programme has led to a reduction in contention in annual discussions about hunting regulations because stakeholders with competing views can see their models incorporated into decisions each year. Some models are now more heavily weighted in the decision process, but this change in emphasis has resulted not from political influence and adversarial action, but from the ability of these models to withstand the predictive demands of the scientific process. The process is transparent, as all process components have been published and are available for public scrutiny (Johnson et al. 1997; U.S. Fish & Wildlife Service 2013). Annual decisions can be justified and defended as demonstrably optimal. Cost increases associated with ARM have been modest. Monitoring mallards at a continental scale is expensive, but monitoring programmes were initiated to inform management decades before the implementation of ARM. Similarly, annual estimation and modelling based on monitoring results were a component of the previous management process, and ARM entailed redirection of some of this effort. The term 'double-loop learning' is used to indicate a step within ARM in which experience with the iterative process leads to recognition of the need to return to the 'set-up' or 'deliberative' phase in order to revise one or more process components (Williams & Brown 2012). In 2002, the basic models and formal methods for updating model weights were revised in order to better deal with process uncertainty (Runge et al. 2002). The FWS programme is now entering a second phase of double-loop learning in which consideration is being given to possible changes in objectives, actions and models. In some years, the models have predicted mallard abundance well, whereas in other years, predictions have not matched estimates as well as might be hoped (Fig. 1). In response to this inconsistent predictive ability, more mechanistic survival models (e.g Cooch et al. 2014) based on new data analyses are being considered. In addition, consideration is now being given to expansion of this program to potentially include habitat management actions as well as harvest regulations (Runge et al. 2006). The qualified success of this decision process for harvest management of mid-continent mallards has led to the development of similar FWS programmes for the harvest management of other mallard stocks (eastern and western) as well as of other species, including American black duck Anas rubripes, northern pintail Anas acuta and both lesser scaup Aythya affinis and greater scaup A. marila (U.S. Fish & Wildlife Service 2013). Development of ARM programmes for harvest management of specific goose populations is also underway. The ARM approach was developed for recurrent decisions characterized by potentially reducible uncertainty, and such decisions are pervasive in conservation, extending well beyond waterfowl harvest management. In recent years, the U.S. Department of Interior has formally endorsed use of ARM for decision problems faced by its land management agencies, of which FWS is one (Williams, Szaro & Shapiro 2007; Williams & Brown 2012). Such decision problems can be very local, such as the regulation of hiking activity in Denali National Park, with the dual objectives of permitting human enjoyment of the park and minimizing hiker disturbance of nesting golden eagles Aquila chrysaetos in order to ensure adequate eagle reproductive success (Martin et al. 2011). Decisions can also be regional in scale, as in the harvest management of horseshoe crabs Limulus polyphemus in Delaware Bay, in such a manner as to not only benefit fishermen but to also ensure adequate crab eggs for consumption by migrating shorebirds (e.g. red knots Calidris canutus) (Smith et al. 2013). FWS harvest management of mid-continent mallards in North America is the longest running ARM programme of which we are aware. However, this basic process of scientific management can be adapted to a wide variety of conservation problems. We also note that the focus of the ARM process on 'recurrent' decision problems is not as restrictive as some might believe. Consider 'one-time' decisions such as the purchase of land parcels for a conservation reserve, or the establishment of wind turbines at a location. Once made, such decisions may be difficult to revoke or revisit, seemingly eliminating the utility of anything that is learned about system responses. However, although knowledge of system responses may be of limited use on the area to which such a decision is applied, similar decisions in other locations can be informed by such learning. In this manner, it is possible to treat sequences of 'one-time' decisions using an ARM framework. And of course, an ARM approach can be used for the more traditional recurrent decisions that follow these 'one-time' decisions, for example, habitat management within reserves and regulation of wind turbine speeds at certain seasons or even times of day. Uncertainty in system response to management actions is often an impediment to implementing policy decisions in conservation and land management. Application of science is the natural response to reducing such uncertainty. However, in most existing management processes, integration of science and policy is neither direct nor maximally effective. The FWS programme described herein provides a blueprint for improved integration of science and policy. In this ARM process, the scientific step of comparing predictions with estimates of system response leads directly to changes in degrees of confidence for system models and thus in the influence of each model in the next decision. The process leads to good decisions in the present, as well as to learning and better decisions in the future. The FWS experience with harvest management of mid-continent mallards demonstrates that this conceptually attractive approach can actually be implemented by a government management agency, despite the various difficulties that typically impede such efforts, including institutional inertia, the perception of process complexity and multiple stakeholder groups with competing values. As resource managers face increased uncertainty associated with global change, the need for effectively integrating science and policy will become more pressing, and ARM processes of the sort described here provide a natural response to this need. We thank Evan Cooch, Sarah Converse and two anonymous referees for providing constructive comments on an earlier draft of this manuscript. We would also like to acknowledge the members of the Harvest Management Working Group, whose efforts and commitment over the last two decades have resulted in the successful development and continued implementation of adaptive harvest management. This article does not contain data. Jim Nichols is a senior scientist with the U.S. Geological Survey whose research focuses on animal population dynamics and conservation decision-making. Fred Johnson is a research biologist who is interested in the application of decision science for conservation problems and who regularly acts as instructor and consultant to governmental agencies. Ken Williams is executive director of The Wildlife Society and previous chief of the U.S. Geological Survey Cooperative Research Units, where he has been involved for many years in developing and applying decision science in natural resources. Scott Boomer is a wildlife biologist within the Division of Migratory Bird Management, U. S. Fish and Wildlife Service where he coordinates the Adaptive Harvest Management program and the Harvest Management Working Group.