Monitoring and Evaluation – July 2010 concepts

Monitoring and evaluation link past, present, and future actions and results to help improve performance and achieve goals. We collect and analyze observations and measurements to test assumptions, evaluate changes in condition, and progress towards meeting resource or management objectives. Successful monitoring tells us whether management actions are being implemented, if they are being done correctly, and if they are making a difference.

A common theme we heard in comments and our roundtable discussions is the importance of a strong monitoring program for successful land management planning. We have always recognized the critical need for monitoring, but today it is more vital than ever before, to help us take the pulse of rapidly changing ecological and socio-economic conditions. So we are proposing that the 2011 planning rule take a two-tiered approach to ensure that changes across the broader landscape are evaluated along with the ability of management actions at local levels to produce desired outcomes.

The NFS does have many great examples of effective monitoring currently being used to inform management decisions, like the National Visitor Use Monitoring, Forest Inventory and Analysis and many others. A strong planning rule with monitoring and evaluation at its core can help ensure that we continue to build on those successful efforts.

What kinds of monitoring programs would be emphasized under the new rule?

The proposed planning rule would recommend that each planning unit develop a land management plan monitoring strategy using a two-tiered approach: (1) monitoring at the planning unit level, and (2) monitoring at the broader landscape scale.

Unit-level monitoring would be focused on detecting changes on the unit and determining whether management actions are being implemented, if they are being done correctly, and if we are effectively achieving objectives and moving toward desired conditions. Unit-level monitoring would also be designed to test assumptions made in the assess or revise/amend phases, as well as gather information to help predict future conditions. Each NFS unit would be responsible for creating and implementing the unit-level monitoring plan, in conjunction with partners and scientists. 

Landscape-scale monitoring plans and strategies would be developed by forest supervisors, one or more regional foresters or Forest Service research station directors, in consultation with other government entities, private landowners, and other partners, and would be designed to detect changes caused by stressors and other factors outside the control of an individual NFS unit. Each NFS unit would be responsible for contributing to the monitoring goals of the landscape monitoring program, and participating in its development where appropriate. Landscape level and unit level monitoring would complement each other and would be focused on questions related to land management on NFS units.

In order to keep monitoring plans focused and achievable, responsible officials would work with the public to target practical, specific questions about management or changes on the landscape. This will give managers data to test assumptions, evaluate management actions and make adjustments to both projects and to the land management plan, where needed, feeding back into the assess-revise/amend-monitor framework.

Through this phase of the framework, building on the other phases, we hope to: 

  1. Progress toward more meaningful, implementable, and accountable monitoring strategies.
  2. Provide a structured, collaborative process for responding to change.
  3. Improve our approach for addressing and tracking issues that transcend National Forest System boundaries.

So, how would it work?

Under the proposed 2011 planning rule, plans would be expected to establish meaningful and accountable monitoring strategies that allow for responsive management through an open, structured, collaborative public process. Specifically:

When assessing  trends and conditions, we would apply scientifically credible methods to assess information. We would consider conservation agreements, information from state assessments, and other sources of relevant existing data as appropriate, and we would maintain dialogues with communities, scientists, and other partners so that we can benefit from the experience of others with existing monitoring programs. We might identify the kinds of data that are useful for understanding conditions and trends relevant to management, gaps in needed information that could be informed by a monitoring program, and changing conditions that we might need to track or assumptions that we might need to test.

The revise or amend  phase is where the Forest Service, working with the public, will develop the monitoring programs described above: the unit level program to test conditions, assumptions and management action; and contributions to a landscape scale program. In this phase, we would design the questions we want to measure or answer, identify outcomes or conditions to test, and determine appropriate measures for evaluating data and performance.  The public would be involved in informing and developing the programs (see collaboration post).  We would also provide public access to past monitoring plans, reports, and supporting information in the spirit and letter of open government policies.

In the monitoring phase, we would implement the monitoring programs outlined in the plan revision or amendment.  We would use collaborative processes to: identify what to monitor based on the questions identified in the monitoring program, do the monitoring, evaluate the data, and determine what to feed back into the planning framework, triggered by new information or changing conditions or needs. We would also seek to gather information to assist in predicting future conditions.  Data will be collected in accordance with a carefully designed program and shared with partners and the public. The Forest Service will conduct periodic evaluations and share those results with partners and the public as well.

 The goal is to have monitoring that:

  • Is clear and transparent, based on collaborative approaches.
  • Uses scientifically credible methods that support our capacity to learn and adjust.
  • Builds understanding of conditions and trends on the unit and the landscape.
  • Tracks management actions and provides data-based feedback to improve accountability and support successful management for desired outcomes.
  • Supports a structured approach to adaptive management, triggered by changed information or conditions.
  • Is efficient, practical and focused, working across the Agency and with partners to leverage knowledge and resources.
  • Is innovative and flexible.
  • Is achievable; with realistic staff and budget expectations.

Monitoring plans would be designed or adjusted to support multiple ways of learning and enable plans to be adaptable over time. The monitoring phase is a means for answering key questions to evaluate progress toward desired conditions and objectives under the plan, and to assess changing conditions and inform future actions. The monitoring program should also measure to the extent which the unit is fulfilling the unit’s expected contributions to the local area, state or region, and Nation. Annual evaluation of these results may lead the responsible official to conclude that changes to the plan may be necessary and that an assessment is needed for a plan amendment or revision.

Will the Forest Service be able to implement this new direction for monitoring?  What will it take?

As many stakeholders suggested at the roundtables, the goal of monitoring would not be to conduct exhaustive research– taken too far, this new approach could overwhelm managers and the public.  The goal is to design a workable, practical, and achievable two-tiered (unit and landscape level) monitoring strategy to understand changes that could impact conditions or practices on the unit, to test assumptions, to evaluate and refine management actions and hold managers accountable for achieving progress towards desired conditions, and to use data to trigger the planning framework and inform future actions.  Managers of each unit would work with the public to 1) identify what monitoring is the most important and could be the most useful in informing the need for and direction of future plan amendments or revisions, and 2) determine what the unit can achieve based on existing and foreseeable budgets.

Even with a focused monitoring program, it’s true that we will need more funding, greater contributions from research, and increased time and money spent on implementation and on partnership development at the local, regional, and national levels to pull this off.

In a time of constrained budgets, it is difficult to anticipate the level of resources that might be available.  But we are confident that this approach responds to stakeholders’ desire for a systematic, unified, monitoring; will give us critical information about conditions and management; ties in well with the all-lands approach; gives us a way to address monitoring for larger scale issues; and meets the public demand for increased credibility and accountability.

6 Responses to “Monitoring and Evaluation – July 2010 concepts”
  1. John Lehmkuhl says:

    The monitoring requirements seem well thought out and take a practical approach. My concern is that the FS has talked about monitoring since the old Forest Planning rule (30+ years), but they have never systematically done it. The organization is not structured to do monitoring (i.e., “not my job”), there are no performance incentives for line officers, and there never is funding. There have been some successful monitoring programs, such as the Pacific Northwest Forest Plan, but these have been large-scale remote-sensing programs that are fairly easily and cheaply implemented. What I am talking about is monitoring the effectiveness and validity of on-the-ground actions. For example, are the restoration treatments we are implementing designed properly and achieving the objectives? Such things are never monitored except as ad hoc grassroots efforts by interested individuals that scrape funds together. Finally, the FS does not have an effective integration of their research and management units to accomplish monitoring as described. Research functions as a sort of quasi-academic organization that has no top-down leadership to organize and fund meaningful integration of scientists and managers to efficiently address management issues. Scientists hesitate to do things for which they are not rewarded (i.e., advise on monitoring); yet proper monitoring/adaptive management can have powerful knowledge discovery (i.e. science) outputs and there are plenty of interested scientists. Basically, I think the agency needs to commit to this goal in terms of organizational structuring, incentives to line officers, and funding, or this part of the new planning rule will be meaningless.

    Like or Dislike: Thumb up 0 Thumb down 0

  2. As an investigative and activist blogger, I have been writing a lot about both government and corporate censorship, transparency and accountability. Of all the goals you have posted, your goal to have monitored, clear and transparent collaborative approaches is by far the most important.

    This approach should promote new trust and inspire cooperation from the public, hopefully creating some real positive whether management and possibly effectively achieving the desired conditions.

    Too many people do not realize how important this is to our entire ecosystem. I would also like to see some form of laymen introduction to the public.

    I would love to write a post on my new activist blog sometime soon, explaining more about what you do, as well as what needs to be done, in order to help drum up support for you and OUR cause.

    Like or Dislike: Thumb up 0 Thumb down 0

  3. slucero says:

    Currently the concepts for “Plan Content” do not include a commitment to tribal interests but only addresses “social, economic, and ecological elements of sustainability, the landscape character and recreation settings, maintenance or restoration of ecosystem resilience, maintenance or restoration of water resources, designations that exist in the plan area, and the NFMA timber requirements. The Plan Content must include reference commitment to respect tribal trust, treaty and other rights.

    Like or Dislike: Thumb up 0 Thumb down 0

  4. slucero says:

    The draft concepts for the planning rule reference Tribal Ecological Knowledge as part of the consultation and collaboration processes. The Planning Rule MUST require inclusion of Tribal Ecological Knowledge in every aspect of the Forest Service’s assessment of management plans, determining “desired conditions,” monitoring the effectiveness of plans/projects, and in determining how to revise plans/projects.

    Like or Dislike: Thumb up 0 Thumb down 0

  5. Fotoware says:

    Since we’ll have so much trouble even agreeing with assessments, I really believe that there will not be much consensus. Preservationists are saying that 22 million acres of dead forests is no big deal. That there are no such things as “catastrophic wildfires”. Yet, those same people have decided that global warming is solely behind all this forest mortality, and their solution is to do nothing and claim that whatever replaces those dead forests will be “resilient, natural and desirable”.

    Sadly, this impasse has no solution on the horizon, and we are well past the “tipping point” for our forests. The multitude of issues are clearly insurmountable unless we can mandate “good faith” collaboration, consensus and compromise, BEFORE lawsuits can be allowed to be filed. If you don’t fully participate in the collaborative process, you cannot file a lawsuit.

    Like or Dislike: Thumb up 1 Thumb down 0

  6. Micheal Hamptyon says:

    Initially as I read the Planning rule concept paper language regarding monitoring I was concerned that the agency may again be about to promise far more than history indicates is acheiveable. This concept paper related to monitoring however, seems to attempt to reel in those expectations a bit with discussion about focused monitoring, designed to be workable and achieveable. Having worked for the Forest Service in various line and staff positions for 33 years, and having invested at least 2/3 of that career trying to provide effective conservation leadership specifcially related to Forest Plan monitoring, I have given this topic considerable thought. I believe the key to more succssful Forest Plan monitoring in the future will not be more funding for monitoring which has been a continual frustration for many in the agency and the public for decades. Rather more focused guidance and accountability of how and where the monitoring funds (NFIM) are invested, will be the key. Unfortunately, such questions are often internal, with limited opportunity for the public to engage in agency policy and adminstrative questions such as scale, criteria, and reporting mechanisms for expenditure of monitoring dollars. Currently and in the past, national efforts to standardize monitoring approaches to address broad scale questions have fallen short. However, successful efforts are numerous at the ecological region scale, as evidenced by efforts in the Sierra Nevada, Pacific Northwest and southern Appalachians, all tied to broad scale ecological assessments and research. Finding the appropriate investment balance for our forest plan monitoring dollars between identifying questions that need to be addressed at the Forest or eco-region scale will be the key to success in the future. Most species viability concerns questions will need attention at a broad scale, while many questions related to local economic related outputs still may be appropriate to address at the Forest scale. In either case, the Forest Service will need an overhaul of our accomplishment reporting system related to monitoring funds to assure broadscale funding and accountability consistency. Currently, the performance critieria is not clear enough to assure consistent reporting or monitoring approaches between regions or even between Forests within a region.

    Like or Dislike: Thumb up 1 Thumb down 0