Chapter 3: Synthesizing Results

Key Questions Matrix
Findings and Lessons Learned

Key Questions Matrix

Table 6 - Key Questions Matrix

Findings and Lessons Learned

1. Scouts are willing and competent in collecting data, even though they don't really understand why they are collecting data.

My interviews with experienced village scouts revealed a high level of comfort with using field patrol, safari hunting, and crop damage dataforms. Scouts unanimously stated that the dataforms were clear and easy to fill out. Scouts further stated that using the dataforms did not interfere with their other primary responsibilities on field patrols and safari hunts.

Conversations with the data entry staff at Nyamaluma also revealed that the scouts who submit dataforms rarely make errors, and the errors made are usually minor in nature. ADMADE's two main strategies for developing competent data collectors, training and a weeding out process, appear to have been effective in producing a network of scouts who are willing and able to record data during field operations. In 1999, Nyamaluma has also started giving certificates in monitoring to all scouts who attend an advanced monitoring workshop. If this monitoring certification becomes adopted program wide, and Unit leaders require/encourage all data recorders to be certified, then this step will serve to increase even further the level of competence of monitoring officers.

However despite their understanding of data collection, the scouts I spoke with didn't fully understand the role monitoring plays or can play at the Unit or project level. Although most were familiar with the quota setting exercises, they had only a fuzzy appreciation that data could be useful in other ways and by who.

2. Scouts and their field supervisors see monitoring primarily for the benefit of higher level administration. Exercises such as land-use planning and quota setting can increase the perceived value and relevance of monitoring activities at the community level.

When asked why they were collecting data on field patrols, safari hunts, or investigating crop damage, the most common answer from village scouts could be paraphrased as "because it's my job". Scouts in general do not have a clear understanding of why collecting this information is important. Many thought the information was needed by higher level officers, one remarking to the effect, "we don't need to write these things down for ourselves, we already know where the poachers and animals are." Another believed that there was someone in the command headquarters that was reviewing his dataforms as they were submitted.

Involving scouts in community-level exercises such as quota setting meetings land-use planning workshops can increase their appreciation and understanding of information-based decision making. Unfortunately however not all scouts are able to attend such functions. Frequently only those scouts based at the Unit headquarters, or who happen to be at the Unit headquarters during a meeting, have the opportunity to see how their fellow residents use information.

3. CBNRM monitoring should encompass both the status of the resource as well as other important aspects of the program, such as management and public attitudes. Failure to monitor all of the critical linkages in the program can erode the foundation of the CBNRM approach.

The various elements of CBNRM projects can be thought of as the links of a very long chain, with each link critical for holding all the parts of the program together. Conserving wildlife is certainly an important element of wildlife based CBNRM, but in and of itself does not guarantee that the program will continue to be sustainable. All of the other links in the chain, including public awareness, education, improved food security, equitable sharing of benefits, transparent and accountable management, support from traditional leaders, a sustainable market of safari hunters, responsible professional hunters, etc., are equally important to achieving the final goal. A monitoring system needs to keep encompass of all of the pieces of the puzzle or the whole thing can fall apart.

As an example, one of the Units in the Luangwa Valley was doing well for several years, with hunting revenue steadily increasing, and community initiatives, such as a new theater group and participation in public meetings, on the rise. In 1998, a financial scandal was unearthed, eventually leading to the loss of the Unit leader. As a result of this blow, staff morale plummeted, confidence in the program fell, poaching skyrocketed, the number of hunting clients fell by more than half, and hunting revenue dropped by 40% in a single year. A strong Unit leader, good financial management, and community support for the program are important links in the chain of success, links which did not hold up in this case. If monitoring of these elements had been more sensitive and responsive, it is possible that the damage from this tragic event could have been lessened.

4. Monitoring wildlife with indirect indicators is highly economical and is adequate for most management purposes if done properly. Measures of sampling effort are important when using indicators.

ADMADE's use of indirect indicators for assessing wildlife population, as opposed to direct counting methods, is almost cost-free. Wildlife scouts would still conduct anti-poaching operations and accompany safari clients even if they weren't recording data. The amount of additional investment to add data collection to these operations is miniscule, being mostly training costs, stationary, and the salaries for 2-3 support staff at Nyamaluma. Nor does monitoring seem to impinge upon the ability of scouts to perform the primary functions of their field operations.

Although this method of assessing wildlife is far from perfect, when observations are made in a consistent manner, multiple indicators are used, and a conservative approach is made to setting harvest levels, then sustainable management of the population is possible. The fact that many ADMADE areas in different parts of the country have been able to support safari hunting fairly consistently for the past ten years is in itself compelling evidence that this strategy of monitoring can work. While monitoring with indices can not answer many other important questions, such as absolute population abundances or the conservation impact on broader taxa, they seem to have proven adequate for management.

A second benefit, which may be even more significant than the financial savings, is that monitoring wildlife with indices can be achieved by ZWA field officers and community leaders themselves . Direct methods of censusing wildlife can provide important data as well, but are unlikely to be within the capacity of field staff due anytime soon due to the cost and technical expertise required. Developing monitoring methods that can function at least partially without dependence on support from donors or central government should be an objective of all CBNRM programs.

One way ADMADE's monitoring data can be made even more robust for management, and also shed light on some of the more scientific questions, is to make greater use of measures of sampling effort in analyses. Although field patrols and even safari hunts do not represent random samples either spatially or temporally, recording when, where, and for how long scouts are making observations in the field would increase the validity of results. The research Unit at Nyamaluma is moving in this direction, for example introducing in 1999 a column for 'day light hours in grid' on the field patrol dataform. They have also introduced a new database system that for the first time allows entry of all observational data from field patrols and safari hunts, including the 'daylight hours spent hunting' and number of baits used for large cats. These new fields of information will allow more accurate measures of hunting effort and help control for the effects of these variables in other analyses.

5. Initial capital investment and centralized training are not sufficient by themselves to establish a community based data collection system. Support visits and assistance with data processing are required.

The ADMADE Units near Kafue National Park present an interesting natural experiment on the importance of field support to CBNRM. Four of these areas were among the nine GMAs selected by USAID for support in the early 90s. They received the same vehicles, radios, uniforms, and training at Nyamaluma as did the remaining five areas, which are all in the Luangwa Valley. The only real difference between the support of these areas and those in Luangwa Valley was the frequency of visits from extension staff from Nyamaluma. Visits to these areas occurred much later and less frequent than Units closer to Nyamaluma. Indeed our trip to Kasonso-Busanga and Lunga-Luswishi in early 1999 was the first visit from an ADMADE inspection visit since the program began.

Although these areas received the same capital investment and sent officers to the same Nyamaluma-based courses, they were clearly behind in many aspects of the program. The greatest gaps could be noted in the amount and structure of community participation in ADMADE. However even the monitoring activities in these areas were substantially less organized. The largest gaps were not with the capabilities of the scouts, but data management practices at the Unit Headquarters. Data management is a topic covered in many courses at Nyamaluma, but has always required additional field support and is a never-ending focus of inspection teams throughout country.

This discrepancy is not all that surprising but highlights the importance of field support for communities in CBNRM programs, particularly in areas where new skills are involved, such as monitoring and building partnerships between communities and government. Although capital investment and foundation training are important may be even sufficient for some aspects of CBNRM, other skills clearly are imparted more slowly and require a longer-term presence from support staff. Discrepancies between the east and west GMAs also poses some questions about ADMADE's centralized training model, and whether or not some skills would be more effectively and economically imparted through regional extension services.

6. For uniformity and project level analyses, the design of a monitoring system needs to be centralized. However implementation and analysis are most effective when driven at the field level.

It is somewhat tenuous to claim that ADMADE has a truly "community-based" monitoring system. Although the principal actors in data collection are community residents, they are working under a government command and control structure and are following top-down instructions for monitoring. There is certainly nothing wrong with this approach, and indeed if you want to be able to generate project-wide summaries and analyses, there really is no alternative strategy to using a top-down methodology to ensure consistent and uniform data collection.

However the top-down approach to monitoring can falter when you examine which levels are best served by the monitoring system. Because village scouts and their Unit supervisors are basically following instructions they were given by inspection teams and during workshops at Nyamaluma, they may not immediately understand the rationale of the monitoring program and the techniques used for data analysis. Consequently, at least initially they may not see much benefit from monitoring, other than appeasing those in higher authority. Many problems in monitoring at the field level, such as poor data management, poor supervision, and just simple neglect, can be traced to a fundamental lack of appreciation of the importance of monitoring and understanding its role in management. As scouts and Unit leaders become more experienced with using information for decision making, these data management issues tend to become less of a problem.

This recurring pattern suggests that an important strategy to strengthening monitoring programs is to raise the level of appreciation and understanding of monitoring at the field level by involving officers not only in data collection but also analysis and interpretation. Devolving these functions, which are currently performed almost entirely at Nyamaluma, to the community level would likely result in strengthened data collection and management. A competing hypothesis is that local people actually have very little use for systematically collected data, in which case strengthening the command and control structure would be a more appropriate strategy for solving monitoring problems.

Although Nyamaluma has not made a concerted effort to devolve data analysis, it has taken some steps to increase the capacity of community members to summarize and interpret monitoring data. Monitoring is covered as one of the topics during many workshops and courses, and Unit staff are frequently involved in data interpretation during inspection visits. Undoubtedly these efforts will continue, as the leadership of ADMADE aspires to reduce support to some areas so they can strengthen others and maybe even introduce the program to new areas.

7. Building the capacity of field staff to collect data can be achieved relatively quickly and cheaply. Building the capacity of field staff and community leaders to analyze/disseminate monitoring data is a slower process.

Data collection has proven to be the easiest set of skills to teach to scouts and unit staff. Data collection primarily involves recording on paper those observations which scouts are already familiar with, and doesn't require any understanding of data management systems and the ultimate role data can play. Data recording skills can usually be successfully developed through attendance at a single course at Nyamaluma followed by periodic feedback on dataforms. Data management, which essentially consists of collecting, certifying, and filing dataforms, builds upon data collection and is next most difficult skill to develop. This requires practices which previously may have been either unfamiliar or not perceived to be important. Data management in a Unit requires attention to detail, cooperation among Unit staff, and strong leadership. Although some Unit leaders have quickly understood the importance and practices of managing data, many need more than just attendance at a course at Nyamaluma. Office management systems are a frequent focus of inspection teams from Nyamaluma.

Data analysis, which builds off of both data collection and proper management, is the last and most difficult skill to develop. Training is critical for teaching Unit leaders and community members how to analyze and interpret monitoring data, but needs to be supplemented with on-the-job learning. This aspect of monitoring has proven the most difficult to develop, partly because it builds off of several other skills. However some areas have very capable Unit Leaders who have learned what patterns and conclusions can be extracted from monitoring data, and as the training and outreach programs at Nyamaluma continue, these skills will certainly continue to grow.

8. The spatial aspect of monitoring data is essential in a CBNRM monitoring program, and should be integrated into data collection, processing, and outputs from the very beginning. Maps are a powerful tool for focusing dialog and communicating results to local communities.

The architects of ADMADE's monitoring program appreciated from the very beginning the importance of incorporating spatial references in all datasets at an appropriate scale. Initially each Unit was divided into 10 km2 grids, and base maps prepared so that each grid could be given a number that could be written on dataforms. As this system was tested, it became apparent that scouts could locate themselves within a quadrant within the grid, so the base maps were redesigned with 5 km2 grids. This coarse scale sacrifices precision for the sake of accuracy, but seems adequate for the management-oriented uses of the data. More importantly, scouts feel comfortable that they can locate themselves on a map at this scale with a high degree of confidence.

Nyamaluma's use of GIS technology allows not only the ability to enter and analyze spatial data but also present results in a variety of map formats. Maps have proven an excellent tool for crystallizing dialogue around the key resource issues in a community, and can present complex spatial relationships in a format that can be understood by most people with a minimum amount of education. There are numerous examples of successful community-initiated land use resolutions based on spatial relationships, such as the relocation of fishing camps, gardens, and even households away from wildlife areas. These initiatives would not likely have been possible without Nyamaluma's ability to quickly and accurately generate large maps of monitoring results.

9. Data collection programs should be designed in consideration of information management capacity. A well-constructed computerized database at the project level is an effective way of storing, analyzing, and disseminating information.

Like many conservation and development programs, ADMADE's monitoring program has generated a lot of data, perhaps more than can be practically analyzed. At the Unit level, the capacity to study and summarize data is quite low, and most data forms remain in the filing cabinet until field staff from Nyamaluma come to collect them. Even at Nyamaluma, constraints on staff time and software have prevented some datasets from being fully entered, resulting in some dataforms sitting unanalyzed for years. Hence an important caveat for all CBNRM programs is to not collect more data than can be processed. While everyone, in particular project managers, would like more data for their reports and assessments, collecting data that can't be used is inefficient and can lead to disillusionment when those involved in monitoring fail to see the fruits of their labor.

Over the history of the program Nyamaluma has been able to process an impressive amount of data streaming in the from the Units. The main tool for this task has of course been the computerized database system. Nyamaluma's database allowed thousands of dataforms to be entered and summarized in a consistent manner. The 1999 database upgrade, which was a component of this research (see Chapter 4), increased the capacity of Nyamaluma to enter data, analyze it, and create summaries for a variety of stakeholders. The database has also, for the first time, largely removed the technical barriers to disseminating data electronically. Nevertheless, as the number of dataforms introduced into the program continues to grow, ADMADE will need to continue to ask itself - both at the community and project level - when does it become too much.

10. All links in the information flow are critical. Constriction or breakage at any point in the information flow can render all other efforts useless.

In the same way that it is a shame to see data collection efforts go to waste when there is insufficient capacity for data processing, it also sad to see data collection wasted when there is a breakdown in the information flow. Although loss of data in ADMADE is not the norm, and does not seem to be linked to any single cause, it has happened all too frequently. Munyamadzi Unit lost all their 1997 data for both field patrols and safari hunting, possibly it was taken by a department biologist and never returned. Other areas have sent data to Nyamaluma through the regional command, only to have it lost en route. Many other dataforms have been lost either at the scout camps or in Unit offices, many of which are poorly equipped and organized. Even at Nyamaluma, data has occasionally been lost either because of a hardware crash, operator error, or disorganized filing (although in many cases backups allow lost data to be recovered). Hence when planning or evaluating a monitoring system, all links in the information flow should be treated as equals, and assumptions that unplanned parts of the puzzle will simply fall into place later should be avoided at all costs.

11. Disseminating results to stakeholders is required for adaptive management to take place, yet too frequently receives little attention. Village scouts and Unit Leaders are in general not very good at disseminating results to community members.

The ultimate aim of monitoring in adaptive management programs is to provide feedback to the program for improving planning and management. Performing analyses, creating summaries, printing wall maps, reports, etc. are all well and nice, but one should never forget that unless monitoring results are disseminated and result in a better program, monitoring will not have fulfilled it purpose. This is true both at the community and project level.

Dissemination of results is a critical prerequisite for this feedback process to occur. At the community level, people who need monitoring results include scouts, community leaders, safari operators, professional hunters, the Unit Leader and his staff. At the project level, the relevant stakeholders include training and research staff at Nyamaluma, senior officers at ZWA Headquarters, and ADMADE's institutional partners. These are the parties responsible for planning and implementing ADMADE at the various levels, and all need access to monitoring data if they are to make informed decisions.

ADMADE has been more successful in disseminating monitoring results to some parties than to others. The research and extension staff at Nyamaluma are of course familiar with the results of monitoring, because they are intimately involved in the analysis and interpretation. Unit leaders and their deputies are probably the second-best informed, as they have the most contact with Nyamaluma officers and are recipients of printed maps and summaries. ADMADE has relied heavily upon Unit leaders and their deputies to inform the public about monitoring results, however the extension staff and scouts I interviewed did not indicate such exchanges are the norm. Information flow within the Unit will most likely improve as the community management committees step up and play a bigger role.

The audience which has perhaps been most poorly reached by ADMADE's dissemination system for monitoring results is the "Lusaka crowd", in particular senior officers at ZWA headquarters, and the donor and NGO community. This gap in the information flow is acknowledged at Nyamaluma, and has been caused at least in part by insufficient middle-level management. The ADMADE newsletter, which is the one update published at Nyamaluma and comes out on a regular basis, is oriented more towards news items and promoting the "ADMADE Vision," as opposed to reporting results which can be used to guide policy makers. The challenge for ADMADE is to find mechanisms to keep all of its constituents informed, without compromising the level of feedback to the people who need it the most - the rural communities.

12. Community ownership of data is important to recognize in a CBNRM monitoring system.

One of the underlying tenets of ADMADE's approach to wildlife management is that the communities are the de-facto owners of the wildlife and the program. This basic principle, upon which all CBNRM is based, applies to monitoring through the ownership of data. Village scouts, in their role as local residents and representatives of the community, collect the majority of monitoring data. Although communities are by no means homogenous entities and there are multitudes of distinct interests and agendas, because the monitoring data is collected by a local person it has a level of credence not likely possible if outsiders were the primary data collectors. Dataforms also remain the property of the community after they have been processed at Nyamaluma. Thus even though Units are heavily dependent on outside support to design and implement their monitoring activities, ownership of the data lies very much with the communities in both design and practice.

13. Pilot testing dataforms is important.

With nearly ten years experience in publishing dataforms, the Nyamaluma research unit has learned, sometimes the hard way, the importance of pilot testing dataforms. Some of the common mistakes on dataforms have been traced to layout problems, confusing terminology, or ambiguous wording. Even simple design elements, such as using the # sign to stand for 'number' have been misunderstood and caused errors. Training is of course an important element in using dataforms, however because not all scouts can attend training when changes are made to dataforms, and there is no substitute for intuitive design, clear instructions, and pilot testing.

It often will take several attempts to work all the bugs out in a dataform. Reviewing the mistakes on dataforms is an ongoing exercise by the extension and research staff, providing feedback which is then used when dataforms are reprinted. At other times, design changes have been introduced even after a dataform is printed, whereby scouts are asked to pencil in the new changes. Other design changes have been implemented in the field. Aside from being confusing for scouts, having multiple versions of dataforms in circulation can make certain types of analyses difficult or impossible.

Nyamaluma has also learned that dataforms should be designed primarily for recording very specific data. Tabular data entry sections that prompt the recorder for specific pieces of information tend to work better than open-ended descriptive sections . Open-ended sections are prone to being overlooked by the scout or generating incomplete or irrelevant details that can not be analyzed. Examples of comments from the first field patrol dataform included the irrelevant "It was a good patrol, only too many mosquitoes", or ambiguous "we found some footprints". The newer dataforms still have a comments section for unusual observations or problems, but the majority of observational data is entered under discrete columns such as "grid", "snares found", "fresh poacher camps", etc.

14. Integrating data collection with other activities is more sustainable than making data collection a separate program.

ADMADE's accomplishments in data collection are in large part due to the use of village scouts as human sensors. Low budget CBNRM programs such as ADMADE are unlikely to ever have the financial or human resources to maintain a separate network of field monitors in such a vast project area. Instead of being a separate activity on its own, data collection at the Units has basically been tacked onto existing operations, such as anti-poaching patrols, escorting safari clients, and problem animal control. There are advantages and disadvantages to this approach. When monitoring is merged with other activities, the observations may be biased by non-random sampling methods and limited time available for observations and measurement. However these drawbacks are vastly offset by the single most important advantage to integrating data collection with other activities: it is more likely to get done . The primary role of village scouts, both on paper and in their own perception, is law enforcement. They would conduct field patrols and accompany safari hunters whether they had dataforms or not. If monitoring became a completely additional assignment for scouts, instead of a supplementary activity, it would be much more challenging to get scouts to use their limited resources for monitoring operations.

In addition to introducing data collection as a supplemental and not separate activity, ADMADE's monitoring design also merges recording of critical data, with less-critical data. Unit leaders and scouts naturally have greater interest in recording data that could potentially get them into or exonerate them from trouble, such as safari hunting license numbers or the amount of ammunition consumed on a patrol. By combining this type of information on the same forms as less immediately pressing data, such as observations of live animals or bush fires, ADMADE has ensured that all data will be recorded properly and diligently.

15. Successful monitoring is dependent on a conducive management environment

It is important to remember that monitoring in CBNRM programs is but one element of a much larger and complex system. Monitoring can only be effective when all the other critical pieces are also functioning. Monitoring can not compensate for the lack of an economically marketable resource. Monitoring can not overcome institutional and legal frameworks that fail to empower communities to manage their resources. Monitoring can not be community-based when there are no resources for training or field support. See Appendix III for a more comprehensive framework for community-based monitoring.

Thus, if the expressed aim of a monitoring program is to help develop a self-perpetuating community based resource program, then one would be wise to examine the feasibility of the project as a whole before investing many resources in monitoring. If, on the other hand, monitoring information is desired as an end in itself irregardless of the outcome of the project, then it may be worth supporting whether the project is successful or not.

Acknowledgements | About the Report | Acronyms | Introduction | Monitoring in ADMADE | Synthesizing Results | Interventions | Conclusion | Bibliography | Monitoring Framework | ADM Menu System | Data Analysis Conceptual Framework | Monitoring Workshop Notes | Additional Research Readers' Comments | Make Comments