Wilderness Risk-Management Survey Introduction
This study was a replication of the original Wilderness Risk-Management Survey that was completed by the University of Utah and NOLS in 2003. At that time, NOLS wanted to conduct a study that would accomplish two things. It would create a taxonomy for risk management, and it would assess how a variety of organizations manage risk (Paisley, Sibthorp & Szolosi, 2003). The first goal was important because it had become obvious that outdoor expeditionary programs (OEPs; defined as spending two or more nights in the field) had a variety of definitions in language around risk management. It was hard to assess how (dis)similarly they managed field hazards because they assigned different definitions to the same words. The second goal came about in response to the fact that there had been little industry-wide research done on how OEPs employed risk-management strategies to address field hazards. Because there was little empirical research available, organizations could not easily compare themselves to one another.
Given that more than a decade had passed since the first survey, it seemed appropriate to repeat it to understand if risk-management strategies had changed. The original survey began with a Delphi panel to name and define both risk-management strategies and categories of field hazards that would apply to a broad range of OEPs. A Delphi process is used to build consensus among a panel of experts around a given topic, and was used in this case to design a survey that would represent a variety of risk-management views (Szolosi, Sibthorp, Paisley & Gookin, 2003). After three rounds, the panel identified 21 risk-management strategies and 15 field hazards. Using these strategies and hazards, a survey was created and sent to OEP organizations. The survey provided a hazard and asked respondents to identify the primary risk-management strategies they used to address the hazard.
The 2003 survey results showed that regardless of OEP size field staff training, policies and procedures, field staff (instructor) judgment, supervision of participations and pre-course communication were the most frequently marked risk-management strategies. Perhaps more significantly, the results showed that larger organizations overall employed more risk-management strategies (Paisley, Sibthorp & Szolosi, 2003). Large organizations were defined as having more field days, more staff and larger budgets. The implication of this finding was that organizations should compare themselves to similar organizations rather than to the largest and most visible OEPs due to differences in available resources. In addition, the results showed that the size of the organization was related to the particular strategies they reported as being most important. As an example, large organizations employed internal and external safety reviews more often than small organizations, which, as an example, depended on pre-course communication more often than large organizations. This finding, again, likely points to resources as being the differentiating factor.
The purpose of the present study was twofold. First, to determine whether and how risk-management strategies have changed for OEPs over the last decade. Second, to identify contemporary concerns and issues.
The 2016 study began by reviewing the 21 risk-management strategies and 15 field-based hazards identified through a Delphi process in the original 2003 project (Szolosi et al., 2003). Based on expert feedback, minor changes were made to the wording of five field hazards in order to reduce bias: “Participant misbehavior” was changed to “Participant behavior;” “Staff incompetence” was changed to “Staff performance;” “Inappropriate staff to participant interaction/contact” was changed to “Staff to participant interaction/contact;” “Environmental” was changed to “Environment;” “Hygiene” was changed to “Inadequate Hygiene.” Two of the original hazards were deleted from the survey because they were thought to be adequately captured through other, more encompassing, hazards. For example, “Competition with other institutions” was considered a subset of “Public Interactions.” One risk-management strategy was changed to reflect more common terminology: “Critical Incident Stress Debriefing” was changed to “Psychological Stress Discussion.” See Tables 2 and 3 for definitions of each risk-management strategy and hazard.
Contact was made with four organizations to which OEPs commonly belong: the Association for Experiential Education (AEE), the Association for Outdoor Recreation and Education (AORE), the Wilderness Education Association (WEA), and the Wilderness Risk Management Conference (WRMC). Each of these organizations distributed an invitation letter and link to the online survey to their members. The survey asked participants to rank risk-management strategies for each hazard.
For the primary analysis, the objective was to compare risk-management strategies by empirically generated groups of organizations. The groups were formed using cluster analysis, a statistical technique that groups organizations into like groups based on organizational descriptors such as number of field days, remoteness of operating area, and organizational mission. To determine which strategies are most used by each cluster, we tabulated the overall reliance of each strategy across the content area of the 13 hazards. That is, the specific hazards themselves are not of direct interest in this analysis. They represent context areas for study participants to consider how they manage risks. By collapsing across these areas, it is possible to understand how reliant each organization is on specific risk-management strategies. This organizational reliance is then averaged for each cluster (or group) above. This process gives us a profile of how each cluster, and the organizations as a whole, manage their risks.
In addition, for the secondary analysis, we tabulated concerns for the upcoming field season and risk-management strategies of focus for 2017. Risk-management strategies for each hazard are sorted in tables in Appendix A.
A convenience sample of 262 was used for analysis after removing incomplete surveys. Taking a subset of the descriptives and using cluster analysis, four “clusters” or groups with somewhat distinct profiles were created (see Table 1). Organizations that fit clearly into one cluster or another might find cluster-based comparisons more useful than comparing to the overall average.
Survey respondents reported their programs offer the following disciplines:
- 89% backpacking
- 71% climbing
- 70% paddling
- 59% winter sports
- 45% rafting
- 37% mountaineering
- 16% sailing
Other reported disciplines included cycling, trapping, caving, stand-up paddleboarding (SUP), surfing, high ropes, horse-packing, canyoneering, primitive skills, trail maintenance, dog sledding, scuba, and cultural immersion.
A graph of research strategies by the frequency of use by the cluster is presented in Figure 1.
In general, the most relied on strategies are:
- Field Staff Training
- Policies and Procedures
- Field Staff (Instructor) Judgment
- Supervision of Participants
- Field Staff Screening
The least relied on strategies are:
- External Safety Reviews (ongoing as part of, for example, accreditation)
- External Incident Reviews (after an incident)
- Psychological Stress Debriefings
- Venue Evaluation or Location Scouting
- Internal Review of Safety Management Protocol
The strategies that vary the most by cluster are:
Cluster 1, Camps and Campus Recreation use:
- Less Participant Screening
- Less Participant Training
- This group uses the fewest number of risk-management strategies overall
Cluster 2, Large OEPs (e.g., NOLS, Outward Bound) use:
- More Course Documentation
- More Course Debriefings
- More Internal Safety Reviews (on-going)
- More Internal Incident Reporting and Review (after an incident)
- More Emergency Action Plans
- This group uses the largest number of risk-management strategies overall
Cluster 3, Guides use:
- Less Field Staff Training
- More Field Staff Screening
Cluster 4, Therapeutic Programs use:
- More Mentoring and Apprenticeship
- Less Emergency Action Planning
- More Field Staff Supervision
- More External Incident Reviews
- More Psychological Stress Debriefings
Survey respondents reported that they are most concerned in the coming season with managing the following hazards:
- Risk Inherent in the Program
- Lack of Participant Supervision
- Staff Performance
Survey respondents reported that in the coming (2017) season they will rely most on the following risk-management strategies:
- Field Staff Training
- Policies and Procedures
- Field Staff (Instructor) Judgment
- Formal Wilderness Medicine Training Requirement of Staff
- Pre-Course Communication
For more specific information on how each field hazard was most commonly managed, see Appendix A. As an example, OEP organizations relied most commonly on field staff (instructor) judgment to manage the environment whereas driving/transportation was most commonly managed by using policies and procedures but rarely managed by using a psychological stress debriefing. As one might expect, medical issues were most commonly managed by formal wilderness medical training, and poor instruction was managed by field staff training.
Perhaps the most significant finding from this study was that OEPs are more alike than different in regards to how they employ risk-management strategies. Regardless of their organizational characteristics, the same top five risk-management strategies were reported in 2003 and 2016. The main differences that do exist between cluster types seem to be related to the organizational mission and/or the size of the organization. While there are some notable and significant differences, we obtained very little information from the survey to interpret why findings varied between clusters. Therefore, the explanations below are speculative.
Camps and campus recreation are more recreationally oriented, have less experienced staff, and use less participant screening. When considering a summer camp or campus recreation program, these findings might be attributed to the fact that the programs they offer are designed to reach a broad audience that is not necessarily skilled in any particular sport. Programs in this cluster can train staff who are hired with less experience, and they are similarly less concerned with screening participants because the programs are designed to be appropriate for the general population, regardless of experience.
Large OEPs use more course documentation, course debriefs, emergency action plans and internal safety reviews as well as internal incident reporting. This may stem from at least two different causes. The first, as previously mentioned, is that they have more resources in terms of finances and staff, and can afford to use these risk-management strategies. The increased documentation helps large OEPs ensure that they have systems in place to support courses in the field, and to be consistent with common practices for conducting adventure activities. Dissecting incidents through the use of these strategies helps OEPs meet their educational objectives, which often involve using risk while maintaining the health and well-being of students and staff. Secondly, for some organizations, the focus on documention may also stem from a desire to be prepared in the event of a lawsuit. Finally, the data also show that large OEPs operate in more remote conditions than other cluster types. It is possible that they have a greater reliance on emergency action plans because staff cannot depend on being able to reach a base camp or field office, which has led to more developed and complex protocols than other organizational types.
Guiding companies hired more experienced employees and used less staff training. In comparison to other clusters, they operated in more remote sites, and their focus was on recreation. As one might imagine, a guiding company that takes participants backcountry skiing will hire employees who have a obtained industry certifications and who have a long resume of experience as backcountry skiers because their staff need not only to be excellent skiers, but they must also be able to manage a group in backcountry terrain. Because their staff must come in with this degree of experience, staff training is more likely to be focused on the company’s individual policies whereas a campus recreation program might be training staff on policies and procedures, but also focuses on developing technical proficiency that guiding employees already have.
Finally, therapeutic programs, logically enough, had therapeutic missions, longer staff training, more field days, a higher staff to student ratio, and a more selective enrollment process for participants. Their staff training, while longer in initial duration, also included more mentorship than other types of programs, which might be attributed to the need for staff to develop the ability to manage a variety of either physical or mental health conditions that the participants might have. In general, therapeutic programs screen their participants more closely because they need to ensure that they will be well served by the program, meaning that whatever needs the participants have can be adequately addressed in the field.
One limitation of this study is that we know very little about the specific content addressed through the various risk-management strategies. Field staff training remains a top strategy employed by OEPs as do policies and procedures. Their ranking in the top five remains the same between the 2003 and 2016 surveys. However, the study does not explain how OEPs use field staff training nor does it capture how their specific policies may have changed. As an example, certain technologies have either become newly available (the iPhone did not exist until 2007) or more readily available (personal locator beacons are now owned by many weekend backpackers). OEPs are almost certainly responding to these changes by teaching instructors how they want such tools to be used, both through staff training and policies and procedures.
Similarly, when an organization looks at how it manages risk today when compared to 2003, it may see that it has dramatically changed, a finding that contrasts one of the main takeaways from this survey (that how OEPs manage risk has remained the same). It is important to realize, however, that the survey captures industry trends rather than change at the level of the individual organization. A particular company may have grown from taking 50 students out each summer a decade ago to now taking out 1,000 students each summer. Consequently, it will almost certainly have modified its risk-management strategies. Those modifications are likely to have brought the organization more in line with the larger organizations with which it now shares more commonalities. In other words, when looking at the data obtained from the survey, organizations should compare themselves to similar organizations rather than simply assessing whether their risk-management strategies are different today than when the original study was completed.
The finding that the use of risk-management strategies has not dramatically changed over the last decade does not come as a tremendous surprise nor does the finding that regardless of organizational mission, OEPs are more alike than dissimilar in how they manage risk. However, the fact that this is so does not necessarily mean that it should be so. In other words, the results of this survey could be a useful starting place for OEPs to coalesce around the question of whether the ways we manage risk are the best ways to manage risk. How much of what we see being done today stems from it being the way it has always been done? It seems a worthwhile conversation to have even if its conclusion is, “Yes, we have found the best risk-management strategies.” As Dallat, Salmon, and Goode (2015) suggest, risk-management strategies have focused on the meeting point between subjective and objective hazards (the equipment, environment and participant) whereas it might be useful to look at the entire delivery system of an OEP from the permitting authorities on to the specifics of, for example, field staff training.
While beyond the scope of this study, there might also be utility in drilling down into the specifics of each risk-management strategy. What policies and procedures result in better risk-management? How can a staff training be structured so that newly hired employees have the tools they need to manage risk in the field? The details of each risk-management strategy are likely to have changed between the 2003 and 2016 survey, and it is similarly likely that they vary by cluster type as well. Having access to that type of detailed information could be a way for organizations to better understand how they compare to their peers, and offer them insight into how they could change their risk-management strategies.
Ultimately, this survey offers a snapshot of risk-management, and a starting point for further conversations. Risk is an inherent part of OEPs, and is something many organizations see as central to their programming and the participant experience. How OEPs manage risk will always be a moving target. But, in order to hone in on the ideal, the industry should continue to assess how it employs strategies to manage risk and continue to discuss about what changes would lead to improvements.
Table 1. Descriptive characterization of clusters
Cluster 1: Camps and Campus Recreation
Cluster 2: Large OEPs
Cluster 3: Guiding companies
Cluster 4: Therapeutic Programs
Table 2. Risk-Management Strategies and Definitions
|Field-based risk-management strategies||Provided Examples & Definitions|
|1. Field Staff Screening||Verification of employee skills and certifications, medical screening of employees|
|2. Formal Wilderness Medical Training Requirement of Field Staff||Wilderness First Aid, Wilderness First Responder, or EMT/WEMT|
|3. Mentoring & Apprenticeship||A formal procedure for staff evaluation and promotion including working with more experienced field staff|
|4. Field Staff Training||A formal period of training conducted on a regular basis and attended by the majority of field staff for the purpose of developing more effective field staff|
|5. Field Staff (Instructor) Judgment||The practice of allowing field staff to make decisions based on personal expertise for a given situation|
|6. Supervision of Field Staff||Proximity of supervisory personnel to assist staff directly responsible for field operations|
|7. Participant Screening||Medical screening, making sure expectations are consistent with the program goals, and verifying participant skills match program requirements|
|8. Pre-Course Communication||Disclosure of program risks and participant expectations through marketing materials, pre-course forms and paperwork, and phone or other verbal communication|
|9. Participant Training||Structured training of the program participants so that they might operate more independently and/or safely in the course environment|
|10. Supervision of Participants||Proximity and ratios of field staff to participants, access to instructor assistance|
|11. Ratios of Field Staff to Participants|
|12. Emergency Action Plan||A formal plan that includes evacuation, communication, first aid, reporting, and debriefing procedures for field based emergencies|
|13. Policies and Procedures||A formal document(s) that detail field staff responsibilities, equipment protocols, field outlines and checklists, and standard operating practices|
|14. Psychological Stress Debriefing||Trained personnel, or access to trained personnel, to handle post emergency debriefing with participants, families, and staff|
|15. Internal Incident Review Procedure||Formal procedure to review field incidents with input from participants, field staff, and administration|
|16. External Incident Review Procedure||Formal procedure to review field incidents with input from participants, field staff, administration, and an external party with specialized expertise|
||Formal and regular process of reviewing field-based safety management procedures|
|18. External Review of Safety Management Protocols||Formal and regular process of reviewing field-based safety management procedures including an external party with specialized expertise|
|19. Course Documentation||Formal reports completed at regular intervals with input from participants, field staff, and administration to address existing and potential program issues|
|20. Course Debriefings||Post course discourse between field staff and a member of the administrative team to evaluate possible hazards and changes for future courses|
|21. Venue Evaluation or Location Scouting||A systematic process of evaluating course locations, hazards, access, etc|
Table 3. Field Based Hazards
Specific field-based hazards
|1. Risk Inherent in the Program Activity Itself||Climbing, boating, ropes course, etc.|
|2. Environment||Weather, animals, terrain, facilities|
|4. Participant Behavior||Participant behavior impacting staff, group, or other participants|
|5. Staff Performance||Staff skill, experience, maturity, or judgment to effectively lead group|
|6. Medical Management||Diagnosis, medical response|
|7. Lack of Participant Supervision||Proportion of “free” or unsupervised time, participant solos, non-activity based incidents or accidents|
|8. Poor Instruction|
|9. Equipment Malfunction|
|10. Staff to Participant Interaction/Contact|
|11. Public Interactions||Interactions with other institutions, outfitters, law enforcement or land management agencies, or local citizens|
|12. Poor Nutrition and Dehydration|
|13. Inadequate Hygiene|
Dallat, C., Salmon, P. M., & Goode, N. (2015). All about the teacher, the rain and the backpack: The lack of a systems approach to risk assessment in school outdoor education programs. Procedia Manufacturing, 3, 1157-1164.
Paisley, K., Sibthorp, J. & Szolosi, A. (2003). Building a risk-management lexicon. The Outdoor Network 14(4). 1, 30-32.
Szolosi, A., Sibthorp, J., Paisley, K., and Gookin, J. (2003). University of Utah/NOLS Risk Management Study. Tech. Rep. Retrieved from http://www.health.utah.edu/parks-recreation-tourism/docs/nols/UU_NOLS_RMstudy.pdf.