Monitoring and Evaluation – July 2010 concepts
Monitoring and evaluation link past, present, and future actions and results to help improve performance and achieve goals. We collect and analyze observations and measurements to test assumptions, evaluate changes in condition, and progress towards meeting resource or management objectives. Successful monitoring tells us whether management actions are being implemented, if they are being done correctly, and if they are making a difference.
A common theme we heard in comments and our roundtable discussions is the importance of a strong monitoring program for successful land management planning. We have always recognized the critical need for monitoring, but today it is more vital than ever before, to help us take the pulse of rapidly changing ecological and socio-economic conditions. So we are proposing that the 2011 planning rule take a two-tiered approach to ensure that changes across the broader landscape are evaluated along with the ability of management actions at local levels to produce desired outcomes.
The NFS does have many great examples of effective monitoring currently being used to inform management decisions, like the National Visitor Use Monitoring, Forest Inventory and Analysis and many others. A strong planning rule with monitoring and evaluation at its core can help ensure that we continue to build on those successful efforts.
What kinds of monitoring programs would be emphasized under the new rule?
The proposed planning rule would recommend that each planning unit develop a land management plan monitoring strategy using a two-tiered approach: (1) monitoring at the planning unit level, and (2) monitoring at the broader landscape scale.
Unit-level monitoring would be focused on detecting changes on the unit and determining whether management actions are being implemented, if they are being done correctly, and if we are effectively achieving objectives and moving toward desired conditions. Unit-level monitoring would also be designed to test assumptions made in the assess or revise/amend phases, as well as gather information to help predict future conditions. Each NFS unit would be responsible for creating and implementing the unit-level monitoring plan, in conjunction with partners and scientists.
Landscape-scale monitoring plans and strategies would be developed by forest supervisors, one or more regional foresters or Forest Service research station directors, in consultation with other government entities, private landowners, and other partners, and would be designed to detect changes caused by stressors and other factors outside the control of an individual NFS unit. Each NFS unit would be responsible for contributing to the monitoring goals of the landscape monitoring program, and participating in its development where appropriate. Landscape level and unit level monitoring would complement each other and would be focused on questions related to land management on NFS units.
In order to keep monitoring plans focused and achievable, responsible officials would work with the public to target practical, specific questions about management or changes on the landscape. This will give managers data to test assumptions, evaluate management actions and make adjustments to both projects and to the land management plan, where needed, feeding back into the assess-revise/amend-monitor framework.
Through this phase of the framework, building on the other phases, we hope to:
- Progress toward more meaningful, implementable, and accountable monitoring strategies.
- Provide a structured, collaborative process for responding to change.
- Improve our approach for addressing and tracking issues that transcend National Forest System boundaries.
So, how would it work?
Under the proposed 2011 planning rule, plans would be expected to establish meaningful and accountable monitoring strategies that allow for responsive management through an open, structured, collaborative public process. Specifically:
When assessing trends and conditions, we would apply scientifically credible methods to assess information. We would consider conservation agreements, information from state assessments, and other sources of relevant existing data as appropriate, and we would maintain dialogues with communities, scientists, and other partners so that we can benefit from the experience of others with existing monitoring programs. We might identify the kinds of data that are useful for understanding conditions and trends relevant to management, gaps in needed information that could be informed by a monitoring program, and changing conditions that we might need to track or assumptions that we might need to test.
The revise or amend phase is where the Forest Service, working with the public, will develop the monitoring programs described above: the unit level program to test conditions, assumptions and management action; and contributions to a landscape scale program. In this phase, we would design the questions we want to measure or answer, identify outcomes or conditions to test, and determine appropriate measures for evaluating data and performance. The public would be involved in informing and developing the programs (see collaboration post). We would also provide public access to past monitoring plans, reports, and supporting information in the spirit and letter of open government policies.
In the monitoring phase, we would implement the monitoring programs outlined in the plan revision or amendment. We would use collaborative processes to: identify what to monitor based on the questions identified in the monitoring program, do the monitoring, evaluate the data, and determine what to feed back into the planning framework, triggered by new information or changing conditions or needs. We would also seek to gather information to assist in predicting future conditions. Data will be collected in accordance with a carefully designed program and shared with partners and the public. The Forest Service will conduct periodic evaluations and share those results with partners and the public as well.
The goal is to have monitoring that:
- Is clear and transparent, based on collaborative approaches.
- Uses scientifically credible methods that support our capacity to learn and adjust.
- Builds understanding of conditions and trends on the unit and the landscape.
- Tracks management actions and provides data-based feedback to improve accountability and support successful management for desired outcomes.
- Supports a structured approach to adaptive management, triggered by changed information or conditions.
- Is efficient, practical and focused, working across the Agency and with partners to leverage knowledge and resources.
- Is innovative and flexible.
- Is achievable; with realistic staff and budget expectations.
Monitoring plans would be designed or adjusted to support multiple ways of learning and enable plans to be adaptable over time. The monitoring phase is a means for answering key questions to evaluate progress toward desired conditions and objectives under the plan, and to assess changing conditions and inform future actions. The monitoring program should also measure to the extent which the unit is fulfilling the unit’s expected contributions to the local area, state or region, and Nation. Annual evaluation of these results may lead the responsible official to conclude that changes to the plan may be necessary and that an assessment is needed for a plan amendment or revision.
Will the Forest Service be able to implement this new direction for monitoring? What will it take?
As many stakeholders suggested at the roundtables, the goal of monitoring would not be to conduct exhaustive research– taken too far, this new approach could overwhelm managers and the public. The goal is to design a workable, practical, and achievable two-tiered (unit and landscape level) monitoring strategy to understand changes that could impact conditions or practices on the unit, to test assumptions, to evaluate and refine management actions and hold managers accountable for achieving progress towards desired conditions, and to use data to trigger the planning framework and inform future actions. Managers of each unit would work with the public to 1) identify what monitoring is the most important and could be the most useful in informing the need for and direction of future plan amendments or revisions, and 2) determine what the unit can achieve based on existing and foreseeable budgets.
Even with a focused monitoring program, it’s true that we will need more funding, greater contributions from research, and increased time and money spent on implementation and on partnership development at the local, regional, and national levels to pull this off.
In a time of constrained budgets, it is difficult to anticipate the level of resources that might be available. But we are confident that this approach responds to stakeholders’ desire for a systematic, unified, monitoring; will give us critical information about conditions and management; ties in well with the all-lands approach; gives us a way to address monitoring for larger scale issues; and meets the public demand for increased credibility and accountability.