Incorporating shared decision making in forest management planning: an evaluation of Ontario's Resource Stewardship Agreement process.

The RSA process is also compared to "best practices" criteria from the shared-decision-making literature, collaborative theory, and participatory-democracy theory. The decision to evaluate against shared-decision-making theory, collaborative theory, and participatory democracy theory was based on three key points. First, the Ontario government promotes the Tourism and Forestry Industry Memorandum of Understanding and the RSA process as a form of shared decision making (Browne 2006). Second, the core of the RSA process is a negotiated agreement between the forest industry and resource-based tourism operators, a stakeholder group that has not traditionally had direct input into the forest management planning process. Third, there is a trend in natural resource management towards participatory approaches to government decision making. If the goal of resource managers is to have more participatory approaches to management, then new policies like the RSA process should be evaluated against standards for such processes.

Criteria for Evaluating a Shared Decision-Making Process

The criteria for evaluating the RSA process are drawn from work in land-use planning theory, negotiation theory, and policy-evaluation theory. The 56 criteria chosen are an amalgamation of criteria proposed by Innes and Booher (1999), Wondolleck (1988), Frame et al. (2004), Lawrence et al. (1997), Smith and McDonough (2001) and Conley and Moote (2003).

Innes and Booher (1999) provide a list of process and outcome criteria they deem essential to a good consensus building process. Their criteria are derived from both empirical research and practical experience within the environmental planning field and reflect the principles of complexity science and communicative rationality. Wondolleck (1988) lists five key attributes that should be present in a land-use decision-making process. Her attributes come from years of studying national forest planning processes used by the United States Forest Service. Frame et al. (2004) list a comprehensive set of fourteen process criteria and eleven outcome criteria they used to evaluate the success of collaboration in British Columbia's Land and Resource Management Planning process. Their criteria are derived mostly from the collaborative planning and evaluation literature. Lawrence et al. (1997) and Smith and McDonough (2001) conducted research on procedural justice and how it could be incorporated into natural resource decision making. Both sets of authors list Leventhal et al.'s (1980) criteria for ensuring fairness when public participation is included in natural resource decision making. Building on the concept of procedural justice, Smith and McDonough (2001) develop a list of attributes they deem necessary for a natural resource decision-making process to be perceived as fair. These attributes are developed based on a study in which participants in the Northern Lower Michigan Ecosystem Management Project were asked their opinion regarding the fairness of natural resource agency decisions. Finally, Conley and Moote (2003) provide a list of typical criteria that are used for evaluating collaborative natural resource management programs. Their criteria (see below) come from several authors including Blumberg (1999), Born and Genskow (2000), D'Estree and Colby (2000), Innes (1999), KenCairn (1998), and the Lead Partnership Group (2000).

Data Collection and Analysis

Data used to evaluate the RSA process were obtained in two ways: a mail survey of tourism operators and a review of published and unpublished documents related to the RSA process. The mail survey provided the opinions of RSA process participants, while the review of published documentation on the RSA process provided key technical information, and a review of unpublished information from government, academic, and industry sources provided additional information.

The mail survey targeted those owning a resource-based tourism business in northern Ontario. The survey was sent to a total of 444 resource-based tourism businesses that had the potential to be involved in the RSA process. To maximize the response rate, multiple mail contacts were made with each operator, an approach suggested by Dillman (2000). The mail survey was conducted in March and April of 2005. At that time, two rounds of RSA agreements, those required for 2004 and 2005 forest management plans, should have been completed, while operators with 2006 and possibly 2007 plans would have gained some experience with the process. The questionnaire asked participants their opinions on whether the RSA process was achieving the evaluative criteria. To minimize questionnaire length, the questions focused on those criteria that could not easily be evaluated from a review of policy documents and related literature alone. A total of 116 operators returned completed questionnaires for a response rate of 26%. Of these 116 operators, 61 had commenced participation in the RSA process. Seventeen operators had signed one or more RSAs.

For the second part of the RSA evaluation, an exhaustive library and internet search was conducted to collect available published and unpublished documentation on the RSA process. This included official policy manuals and legislation, as well as unpublished government, industry, and academic reports. These documents were reviewed to find information pertaining to each of the evaluative criteria. This information was then incorporated with the survey responses to establish a rating for each criterion. In addition, one researcher attended an RSA summit sponsored by NOTO, held in November 2004. The researcher documented first hand the experiences of tourism operators, forest industry representatives, and agency officials with the RSA process.

The responses of tourism operators to the questionnaire were analysed using basic descriptive statistics. Based on these responses, and on the review of the RSA policy documents, each criterion was assigned a qualitative rating of 'met', 'somewhat met', 'neutral', or 'not met'. For a criterion evaluated solely on questionnaire responses, the rating was determined based on the mean score of all respondents on a 5 category response scale, where an 'agree' was valued at +2.0 and a 'disagree' was valued at -2.0. If the mean response score was greater than +1.5 the criterion was considered 'met'. It was considered 'somewhat met' between +0.51 and +1.5, 'neutral' between -0.5 and +0.5 and 'not met' if the mean response score was less than -0.5. Where there was pertinent policy information, as well as questionnaire response data, the criterion was rated using a combination of the two sources. For criteria where the use of participants' opinions was impractical, ratings were assigned using the researchers' best judgement based on the review of the RSA policy documents.

Study Limitations

This study has several limitations. First, only the opinions of the tourism operators were investigated and therefore the results reflect only the views of one set of stakeholders. A future study might seek to assess the opinions of the forestry industry as well as the opinions of other users of the Crown land. Second, this study examined the RSA process in its infancy. By the study date of spring 2005, less than half of the forest management plans in Northern Ontario had incorporated RSAs. Therefore, the results are preliminary. Respondents' views may vary with increased experience with RSAs. Despite these limitations, the study provides a comprehensive evaluation of a new collaborative process in forest management planning that explicitly includes resource-based tourism operators. By focusing on the tourism operators' perspective of the process, the evaluation reflects the opinions of the primary audience for which the process is designed.

Results and Discussion

Policy Goal Evaluation

The RSA process is meeting some of its goals at this point in implementation. The RSA process somewhat met three goals and received a neutral rating on two additional goals (Table 1). The remaining four goals could not be evaluated because they either pertained primarily to the forest sector, which was not surveyed, or it was too early in the process to assess whether they have been met. Although it appears to be too early in the implementation of the RSA process to definitively evaluate goals achievement, it should be noted that to date the RSA process is not failing to meet any of the policy goals.

Best Practices Criteria Evaluation

The evaluation of the RSA process with respect to best practices criteria from shared-decision-making theory, collaborative theory, participatory democracy theory, and other research presented in the academic literature revealed both strengths and weaknesses (Table 2). Thirty-one criteria were rated as 'met' or 'somewhat met', 14 were rated as 'neutral' while eight were rated as 'not met'. Three criteria were given an 'undetermined' rating due to insufficient data. Below we group and discuss the evaluative criteria by areas of strength and areas for potential improvement.

Strengths

Inclusion of Tourism