Comment

Berton Lee Lamb is branch chief, Policy Analysis and Science Assistance Branch, Fort Collins Science Center, U.S. Geological Survey. He developed the Legal-Institutional Analysis Model (LIAM). He has published widely in leading journals in his field and is a member of the Editorial Board of Sustainability Science and the International Journal of Sustainable Societies.

E-Mail: lambl@usgs.gov

Nina Burkardt is a research social scientist in the Policy Analysis and Science Assistance Branch, Fort Collins Science Center, U.S. Geological Survey. Widely published in leading journals in her field, her research focuses on environmental negotiation, conflict resolution, and institutional analysis. She is a former president of the Western Social Science Association.

E-Mail: burkardtn@usgs.gov

hen Linda Pilkey-Jarvis and Orrin Pilkey state in their article, "Useless Arithmetic," that "mathematical models are simplified, generalized representations of a process or system," they probably do not mean to imply that these models are simple. Rather, the models are simpler than nature and that is the heart of the problem with predictive models. We have had a long professional association with the developers and users of one of these simplifications of nature in the form of a mathematical model known as Physical Habitat Simulation (PHABSIM), which is part of the Instream Flow Incremental Methodology (IFIM). The IFIM is a suite of techniques, including PHABSIM, that allows the analyst to incorporate hydrology, hydraulics, habitat, water quality, stream temperature, and other variables into a tradeoff analysis that decision makers can use to design a flow regime to meet management objectives (Stalnaker et al. 1995). Although we are not the developers of the IFIM, we have worked with those who did design it, and we have tried to understand how the IFIM and PHABSIM are actually used in decision making (King, Burkardt, and Clark 2006; Lamb 1989).

What we have learned from these investigations is that the authors of "Useless Arithmetic" are correct in asserting that policymakers should be wary of mathematical models. But the question remains: How should policymakers approach the question of a model's accuracy and utility? Our experience suggests that the ten points presented by the authors are themselves a bit simple and miss at least one lesson about how mathematical models are used. The prescriptions are a bit too simple because Pilkey-Jarvis and Pilkey do not give clues to policymakers about how to evaluate the quality of model predictions. How can an administrator or policymaker break through the technology and bureaucracy that often surrounds these models to ask the good questions and understand the answers? A lesson that could be drawn from an examination of mathematical models used in decision making is that the choice of models actually frames the policy debate in ways that limit options. The model we have worked with so long, PHABSIM, provides good examples for both of these observations.

We were surprised several years ago to learn that the IFIM and PHABSIM had achieved international status as the methods of choice for assessing the level of environmental flows. Environmental flow—known as "instream flow" in the U.S.—is the amount of water flowing in a stream that is necessary to protect environmental amenities such as fish habitat and recreation (Lamb and Doerksen 1990). While presenting papers at a water conference in Italy, we met experts who were using the IFIM and PHABSIM in Europe and Asia (Manciola and Mearelli 2000; Tamai and Chibana 2000). Over the years, we have maintained a network of users, and these were people we did not know. They were using the IFIM and PHABSIM strictly based on the literature.

We report this encounter to illustrate that PHABSIM is one of those mathematical models that is "out there." It has escaped the gravitational pull of its founders. In fact, PHABSIM is so much in its own orbit that it has spawned clones, spin-offs, and hybrids. Lamb, Sabaton, and Souchon (2004) note that the IFIM was intended to determine how much flow is required by riverine fish species to maintain a sustainable habitat. It was first thought that knowing the flow-habitat relationship would suggest the flow regime (including the timing and duration of flow events) needed for fish production, but nature has proven to be more complicated. Even the larger questions the IFIM was intended to answer—as well as questions that have arisen from use of PHABSIM—continue to puzzle scientists. This basic question—What is the relation among flow, habitat, and fish populations?—is the subject of ongoing investigations (Reiser, Wesche, and Estes 1989; Souchon and Capra 2004). Despite warnings from the founders (Stalnaker et al. 1995), practitioners frequently use only simple, intermediate output to argue for a minimum, rather than dynamic, flow standard. That intermediate output relies on PHABSIM. As Jarvis-Pilkey and Pilkey observe about other mathematical models, PHABSIM is often a black box and adding more of the pieces of the IFIM (i.e., more variables) results in complexity that is difficult to understand (Sabaton 2002).

How can policymakers approach the question of achieving quality output from such models? When the authors write about "policymakers," it is unclear who they are describing. It might be legislators, mayors, or agency heads. Our experience with the IFIM and PHABSIM shows that many times these questions emerge in local, site-specific disputes. In those cases, it is mid-level administrators such as office supervisors, district managers, and sometimes regional directors who must answer this question. On the one hand, these mid-level administrators are often close to the technical questions involved. They are likely to have more knowledge of the situation. On the other hand, the questions are sufficiently nuanced and the IFIM so complex that even the most knowledgeable mid-level administrator is going to rely on experts to gather the data and run the models.

As Pilkey-Jarvis and Pilkey observe, a consultant is frequently hired to take on these tasks under the direction of a knowledgeable employee. This presents the mid-level administrator with two types of problems. First is the problem of selecting and guiding the knowledgeable employee so that the consultant receives proper instruction and supervision. Here, one important type of information needed from the consultant is a description of the analytical steps and preliminary model products. This is vitally important because the way the consultant collects data and controls the models will affect the decision process by limiting options. Second is the problem of overseeing the evaluation of model outputs. Again, the mid-level manager is probably not going to do this directly but through subordinates. Selecting the best qualified subordinate is important. It is also essential to clearly set out management objectives so that the people to whom these tasks are delegated know the direction mid-level administrators wish to go (Gillette and Lamb 2005; Stalnaker et al. 1995).

The mid-level administrator, of course, may not be the only one setting the overall direction. Other parties to the decision process will be aware of—and often participate in—data collection and model implementation. Model outputs certainly will be shared with the other parties. Our examination of cases in which the IFIM was used indicates that the parties tend to organize themselves into specialized technical committees. Those committees guide model selection, data collection, and interpretation. The quality of model output depends on the work of those committees, which is influenced by the knowledge and skill of the employee to whom the mid-level administrator has delegated authority (Lamb, Burkardt, and Taylor 2001).

The issues notwithstanding, and contrary to Pilkey-Jarvis and Pilkey's argument, mathematical models are a key to framing environmental disputes. Still, care must be taken in selecting mathematical models that can actually deliver information useful to decision makers. In cases we have studied, the parties often do not realize that choosing which models to use is a significant factor in setting the frame of the negotiation (Lamb, Burkardt, and Taylor 2001). "Framing is about focusing, shaping and organizing the world around us…defining the reality…by selecting some elements as central and others as peripheral…." (Davis and Lewicki 2003, 200). Selecting a mathematical model frames the problem because it provides a clue to expectations about possible outcomes. One of the most useful things that models can do is help the parties see a way through a dispute. If the right model is chosen, it can define the problem in a positive light amenable to dispute resolution.

In the cases we studied, negotiations that struggled were marked by "technical options and results [that] were presented in a way that cast the negotiations in a negative light and emphasized the significance of loss" (Lamb, Burkardt, and Taylor 2001, 231). We found that negotiators often started conducting studies without fully considering other analytical approaches. Thus, we agree with Pilkey-Jarvis and Pilkey that negotiators sometimes select the simpler of output options so that the results are not a good fit to the problem at hand.

In the least successful cases we studied, the parties were hoping that a course of action would emerge from the data and did not recognize that the decision to choose a particular mathematical model constrained the decision space. In the successful cases, parties worked hard to frame the negotiation by defining the nature and scope of the technical issues, determining appropriate studies, and coming to at least tentative agreement on how management actions might proceed based on model results. By conducting modeling studies without resolving where they wanted to go with the negotiation, the parties often obscured the underlying value differences that ultimately reemerged as obstacles to a solution (Burkardt et al. 1995).

Pilkey-Jarvis and Pilkey also contend that "quantitative mathematical models are problematic and that an uncritical acceptance of them by policymakers may actually have exacerbated society's ENR [environmental and natural resources] problems." We agree to an extent. Our case studies of situations in which the IFIM was used underscore the importance of avoiding uncritical acceptance of models. However, mathematical models are not "useless." Oftentimes in environmental flow disputes a deep understanding of nature is not available. Some have argued that such deep understanding requires very long-term study (Elliott 1994). However, decisions have to be made and both statutes and regulations limit the amount of available time to make them. As Elliott's (1994) study has illustrated, a full understanding of sustainable fish populations might require more than 10 years. Mathematical models that rely on the current state of knowledge will be used in these decisions. Moreover, mathematical models in ecology are part of the scientific conversation. In addition, understanding how to use and interpret them evolves over time. Consequently, users become more sophisticated and policymakers more familiar with the results.

The authors also suggest that planners and policymakers "must stop turning to science and mathematical models for answers and instead come up with their own solutions aided by scientific observations." Yet given that decision makers are required by law to make policy based on "best available science," this seems less than helpful advice. Moreover, one of the ways mathematical models are most helpful is to evaluate alternatives that we would not want to test in the real world. Examples include testing the effects of prolonged low flow releases from reservoirs or testing the effects of occasional very high flow events. Other examples include actions that might extirpate an endangered species or result in significantly reduced production of hydroelectric power. Gard's (2006) study using the River2D model (a spin-off from PHABSIM) illustrated the value of mathematical models in designing a stream restoration project where it would have been impossible to build and test alternative designs in the field.

When the discussion in "Useless Arithmetic" turns to the role of adaptive management, the authors discuss how management actions can be modified based on real-time observation of the effects of current practices. Limits on the harvesting of fish stocks are one example of a potential adaptive management arena in which they argue that models are not needed. Although this makes some sense, without modeling the system that produces and takes fish—including the social and economic system—it might be too late to change course or reset catch limits based on annual field observations (see Elliot 1994). Consequently, modeling the system in an attempt to better understand the interplay of variables can lead to the development of hypotheses that are essential for adaptive management.

In problems such as environmental flow, hypothesis testing as part of an adaptive management approach could be accomplished through a well-designed monitoring protocol (Souchon et al. 2008). Hundreds of water management decisions are being made by governments around the world and many have been modeled using PHABSIM. But few have included long-term monitoring studies (Souchon et al. 2008). Railsback, Blackett, and Pottinger (1993) found that a monitoring program is most likely to be successful if based on assessments derived from the original modeling efforts. However, as Souchon and his colleagues point out, adopting a monitoring strategy as part of multi-party negotiations "…is not always easy, due to differing interests of the actors…duration of monitoring, nature of funding and differential timetables between facilities managers and researchers" (2008, 12). Framing the problem as one of adaptive management at the outset of negotiations can help structure the solution to include mathematical models that allow tradeoffs and guide monitoring.

In sum, we agree with several of the points made by the authors. Users need to be aware of the assumptions used, but they should be even more aware that the models frame the inevitable negotiations by constraining options and signaling whether there is a way out of the problem. Relatedly, we argue that the authors set up a false dichotomy: relying on mathematical modeling or direct scientific observation to make policy. In practice, the two are often combined. However, even if they are not combined, decisions in the ENR policy arena are the product of bargaining and negotiation, are conducted under time constraints that often preclude long-term observations, and are sometimes preferable to observing natural resource decline before one can decide what to do about it. Finally, once policymakers pursue the adaptive management prescribed by the authors, mathematical models can give greater transparency for policymakers to the tradeoffs involved in decision making. As such, rather than "useless," mathematical modeling done with an appreciation of its strengths and limitations may be helpful in enabling sound ENR decisions.

References

Burkardt, Nina, Berton L. Lamb, Jonathan G. Taylor, and Terry Waddle. 1995. Technical Clarity in Inter-Agency Negotiation: Lessons from Four Hydropower Projects. Water Resources Bulletin 31(2): 187-98.

Davis, Craig B., and Roy J. Lewicki. 2003. Environmental Conflict Resolution: Framing and Intractability—An Introduction. Environmental Practice 5(3): 200-06.

Elliott, J. Malcolm. 1994. Quantitative Ecology and the Brown Trout. Oxford: Oxford University Press.

Gard, Mark. 2006. Modeling Changes in Salmon Spawning and Rearing Habitat Associated with River Channel Restoration. International Journal of River Basin Management 4(3): 201-11.

Gillette, Shana C., and Berton L. Lamb. 2005. Core Competencies for Natural Resource Negotiation. Environmental Practice 7(3): 155-64.

King, M. Dawn, Nina Burkardt, and Brad T. Clark. 2006. Watershed Management Councils and Scientific Models: Merging Diffusion Literature to Explain Adoption. Environmental Practice 8: 125-34.

Lamb, Berton L. 1989. Comprehensive Technologies and Decision‑Making: Reflections on the Instream Flow Incremental Methodology. Fisheries 14(5): 12‑6.

Lamb, Berton L., and Harvey R. Doerksen. 1990. Instream Flow Uses of Water—Water Laws and Methods for Determining Flow Requirements. In National Water Summary 1987: Hydrologic Events and Water Supply and Use, edited by Jerry E. Carr, Edith B. Chase, Richard W. Paulson, and David W. Moody, 109-16. Washington, DC: U.S. Geological Survey.

Lamb, Berton L., Nina Burkardt, and Jonathan G. Taylor. 2001. The Importance of Defining Technical Issues in Interagency Environmental Negotiations. Public Works Management & Policy 5(3): 220-32.

Lamb, Berton L., Catherine Sabaton, and Yves Souchon. 2004. Use of the Instream Flow Incremental Methodology: Introduction to the Special Issue. Hydroécology Appliquée14(1): 1-7.

Manciola, Piergiorgio, and Mario Mearelli. 2000. Contribution to the Defining of the Minimum Acceptable Flow: Tiber River Basin Authority Experience. In New Trends in Water and Environmental Engineering for Safety and Life, edited by Ugo Maione, Beatrice M. Lehto, and Rossella Monti, 28. Rotterdam, Netherlands: A. A. Balkema.

Railsback, Steven F., Roger F. Blackett, and N. Diane Pottinger. 1993. Evaluation of the Fisheries Impact Assessment and Monitoring Program for the Terror Lake Hydroelectric Project. Rivers 4(4): 312-27.

Reiser, Dudley W., Thomas A. Wesche, and Chistopher Estes. 1989. Status of Instream Flow Legislation and Practices in North America. Fisheries 14: 22-9.

Sabaton, Catherine. 2002. Development and Use of Fish Habitat and Population Dynamics Models as Management Tools for Hydropower Plants: Overview of Electricité de France Experience. Enviro Flows 2002, Proceedings of the International Conference on Environmental Flows for River Systems, Cape Town, South Africa, March.

Souchon, Yves, and Herve Capra. 2004. Aquatic Habitat Modelling: Biological Validations of IFIM/Phabsim Methodology and New Perspectives. Hydroécology Appliquée 14(1): 9-25.

Souchon, Yves, Catherine Sabaton, Robert Deibel, Dudley Reiser, Jeffrey Kershner, Mark Gard, Christos Katopodis, Paul Leonard, N. Leroy Poff, William J. Miller, and Berton L. Lamb. 2008. Detecting Biological Responses to Flow Management: Missed Opportunities; Future Directions. River Research and Applications 24: 1-13 (in press).

Stalnaker, Clair B, Berton L. Lamb, Jim Henriksen, Ken Bovee, and John Bartholow. 1995. The Instream Flow Incremental Methodology: A Primer for IFIM. Biological Report 29. Washington, DC: U.S. Geological Survey.

Tamai, Nobuyuki, and Takeyoshi Chibana. 2000. Suitability Curves of Fishes for Riverine Habitat Evaluation. In New Trends in Water and Environmental Engineering for Safety and Life, edited by Ugo Maione, Beatrice M. Lehto, and Rossella Monti, 11. Rotterdam, Netherlands: A. A. Balkema.