Strategies for Effective Monitoring: A Case Study of ADMADE

Acknowledgements | About the Report | Acronyms | Introduction | Monitoring in ADMADE | Synthesizing Results | Interventions | Conclusion | Bibliography | Monitoring Framework | ADM Menu System | Data Analysis Conceptual Framework | Monitoring Workshop Notes | Additional Research
Readers' Comments | Make Comments

Chapter 5: Conclusion

The Way Forward

ADMADE's monitoring system has steadily improved itself since its inception using very few resources. The amount of data collected, the quality of that data, and the number of applications for the data have all been on the rise. While the monitoring system functions very well at many levels, plans for the future should (1) focus on the chronic problems that continue to limit the effectiveness of monitoring to improve the ADMADE program, and (2) address new information needs that have arisen from recent structural changes in the program. Some broad recommendations are presented below.

Improve financial monitoring
The most glaring information gap for ADMADE, commonly acknowledged both within the program and from the perspective of outsiders, focuses on revenue flow within the project. At the national level, total revenue from safari license sales is fairly well known and accessible, thanks largely to the computerized licensing system at the Wildlife Conservation Revolving Fund . However revenue flows within the WCRF, between the WCRF and local communities, and within local communities, are far less transparent.

Lack of transparency in ADMADE's financial flows is an ongoing problem which breeds confusion and mistrust, and creates opportunities for mismanagement of resources. Both department field staff and leadership of local communities commonly suspect the WCRF is intentionally withholding their money and delaying disbursement because it is used for other purposes. Delays in disbursement is disruptive and costs the communities money because they don't receive any generate interest generated from their revenue. This has interfered with program activities at the field level on several occasions, and erodes the foundation of trust between government and communities upon which the program is based. More than one chief has suspended or threatened to suspend all activities in their area until the department releases their funds.

The WCRF in turn withholds payments from communities because there are suspicions that previous impresses have not been used and cleared properly, which in many cases is true. In some Units, revenue flows through the regional command, adding another layer to the transaction and creating more doubts in the community as to whether they are receiving everything they are supposed to. In almost all Units, planning field operations and community development projects is hampered by a lack of knowledge of how much money the area has earned or is projected to earn. Clearly the lack of transparent accounting and limited dissemination of financial data is causing damage at all levels.

Unlike some monitoring problems, which can be addressed satisfactorily by interventions of Nyamaluma alone, improving financial monitoring will require an integrated approach with leadership from the highest levels of the department. Although improved information flow will certainly need to be part of the solution, communities also need to be trained in and responsible for accounting procedures, with closer field support and auditing from ADMADE. The WCRF needs to strengthen its reporting of revenue use and educate people about the policies and formulas used for distribution, and perhaps computerize analysis of community financial reports which only they receive. Most importantly, mechanisms for sharing financial data between all parties need to be strengthened.

Monetary issues are by their nature highly charged political hot potatoes, however failing to address and improve financial monitoring may ultimately prove to be ADMADE's Achilles heel.

Monitor project impacts
The impact of community development projects has not been well studied. Information that has been systematically collected about projects so far includes the start and completion dates, amount of money spent on the project, and the source. However data on the number of beneficiaries, as well as the value and nature of the goods or services from the project, has not been institutionalized and is only collected on an ad-hoc basis.

Monitoring project impact is important not only to meet donor reporting requirements and determine whether a community has used its financial benefits efficiently, but also to assess whether the community development efforts are indeed complementary the other objectives of ADMADE. Recently there has been a growing realization that many of the most popular community development projects, such as schools and clinic, do not directly address the number one threat to wildlife in many areas - poaching driven by hunger, and may not represent the collective will of the community. This type of finding can be used to better plan community development needs, and prioritize those projects which address food security and land use issues which are more important to ADMADE's long term survival.

Improve dissemination of results
Nyamaluma, in its role as ADMADE's central nervous system for monitoring data, has long recognized a need to improve communication of results to stakeholders, particularly to organizations working at the national level. Sharing findings with other groups will likely play an even greater role in the future as ADMADE attempts to expand the portfolio of services it can offer to communities, develop partnerships with other organizations, and diversify its funding base. Each type of structural change in the program is accompanied by new information needs for planning and evaluation.

In the past, ADMADE's monthly newsletter has been its primary link to the outside world, even to the point of using it as a substitute for quarterly reports to USAID. However if ADMADE wishes to maintain support within the new ZWA leadership, donor community, and wildlife sectors domestically and internationally, there is clearly a need for more in-depth and regular results reporting.

One of the strategies ADMADE has employed in the past, and should continue to pursue, is taking advantage of the advances in information technology to disseminate results. As described in Chapter 4, the program now has a and state-of-the-art database and web site which could serve as a clearinghouse for monitoring data. The web site is currently under utilized, however there are numerous evaluation reports, monitoring summaries, and special studies that could be posted with little effort.

Another strategy ADMADE would be wise to pursue is to utilize the coordinating office in Chilanga as a national repository of data on ADMADE. Some of the most important audiences for monitoring results at the program level are officers within ZWA, the donor and conservation organizations in Lusaka, and the media. The coordinating office should serve as a liaison with these groups, and have at their deposal materials for talks, presentations, and program planning. Many of these audiences do not require the latest, cutting-edge analyses from the field, but simply very basic information on what-where-why-when-how of ADMADE. Misperceptions of ADMADE and its approaches are common, even with the professional conservation and development communities.

Coordinate with other national monitoring initiatives
As described in the stakeholder analysis section, there are several parallel projects and organizations in Zambia directly involved in wildlife monitoring. These include the National Environmental Monitoring and Information Network, and the Wildlife Resources Unit at Environmental Council of Zambia. ADMADE has yet to solidify its relationship with these other initiatives, and define whether it will be an active or passive partner with other monitoring programs. ADMADE has much to contribute to other monitoring programs, not only in terms of data but also in methodology and experiences. Conversely, ADMADE could also benefit from other monitoring programs, through improved dissemination of ADMADE's accomplishments, complementary datasets in adjacent protected areas, validation of findings, cooperation in organized censuses such as aerial surveys, and sharing experience with monitoring methods that ADMADE has yet to develop, such as vegetation, agricultural, and socio-economic monitoring. As a program of the government of Zambia, ADMADE also has a certain degree of obligation to coordinate with and support other government monitoring programs. At a very minimum, coordination with other environmental monitoring initiatives is needed to avoid conflict and duplication of efforts.

Revisit incentives for data collection
From the very beginning, ADMADE's policy towards incentives for data collection has been to discourage material incentives in order to reduce the likelihood of data falsification. The fear is that scouts will make up nonexistent field patrols in order to collect an additional bonus for data collection. As an alternative to material incentives, the ADMADE model predicts that scouts will be motivated to do additional work in data recording from a sense of pride emanating from the belief that their dataforms will ultimately help their community.

Units are encouraged however to recognize and praise scouts who are involved in data recording in non-material ways. Nyamaluma itself at one time sponsored a national competition for the best data recorder, however it was eventually abandoned under the rationale that Units should be responsible for providing awards for data collection. Invitations to attend additional training courses at Nyamaluma is also thought to be an incentive for scouts to excel in monitoring.

Not surprisingly, the lack of incentives for monitoring is not popular with scouts. Because some scouts are involved with monitoring and others are not, it is seen as an additional function above and beyond the primary role of law enforcement. The concern about data falsification may be warranted, however the benefits of providing scouts with incentives for monitoring need to be further explored. This issue may resurface on its own as scouts have recently been asked to conduct specialized field patrols specifically for the purpose of collecting data (e.g., snare survey, fish camp patrols, water holes patrols).

Village scout salaries average between $20 and $40 a month, a rate which forces many to live with their families in poverty, unable to afford to send their children to school, buy food during the hungry season, or live in decent housing. One could even argue that scouts are indeed being exploited by the government and their own communities for their labor, as their civil servant counter parts do the same work and get paid two or three times as much. Providing incentives for additional monitoring duties may help alleviate the condition of village scouts, and rather than increase the likelihood of unethical behavior actually reduce the chances that they will be susceptible to accept bribes or resort to poaching for their own survival.

Prioritize data needs
Under the new wildlife act, ADMADE has recently introduced several new structures at the community level: the community resource board, the village area group, peer groups, and three technical committees. Introducing these new structures comes with its own data issues and a new layer of monitoring, needed to ensure that the changes are having the desired effect. Community participation in meetings, community development needs, and allocation of project revenue are all examples of monitoring needs that have only recently been introduced. Additional calls for monitoring the impacts of community development projects, conducting snare surveys, fish camp and waterhole patrols, household demography, and collecting data on food security issues, are other examples of monitoring activities that have recently come aboard.

ADMADE/Nyamaluma needs to prioritize their data needs in light of their capacity to collect, process, and analyze data, else they may find themselves struggling to swim in an ocean of monitoring data. Nyamaluma has in the past introduced new dataforms or changes to dataforms that it was unable to process or analyze. Although Nyamaluma's new information system significantly removes many of the technical barriers to managing data, time and manpower are still limiting factors.

Many of the new dataforms introduced in 1999 were designed to used primarily within the community. It remains to be seen whether communities will have the capacity and interest in using so many new dataforms. ADMADE may also wish to examine the community-based monitoring methods in other rural development projects, such as the community self-monitoring ledgers in the CARE Livingstone Food Security Project, or the methodology for developing local level business plans in CLUSA Zambia's Rural Group Business Program. Although ADMADE has substantial experience in monitoring wildlife and law enforcement, other NGOs have greater experience in community mobilization and development reporting, lessons which may be of benefit to ADMADE.

Increase community capacity in data analysis
Nyamaluma's 1999 workshops in monitoring skills were a good step in developing the capacity of communities to analyze their own data. These efforts need to be continued and expanded in order for communities to be equipped with the skills to perform functions such as quota setting and running land use planning workshops. Increasing the capacity of communities to use their own data would also reduce the turn around time between data collection and useful results, relieve the pressure on Nyamaluma's extension service so that it can focus more on improving its training programs.

The capacity building process would likely be hastened if ADMADE supplemented its centralized training programs at Nyamaluma with a field-based network of support staff. Developing skills such as managing a filing system, summarizing data spatially and tabularly, interpreting results, and presenting findings to others are best taught 'on the job' through trial and error. Visits from Nyamaluma staff are very beneficial, however their frequency and rapid pace limits the amount of interaction and support possible.

Monitor the monitoring
Lastly, making the monitoring system a subject of itself would provide needed feedback for improving how information is being used at all levels. Monitoring the monitoring would not necessarily require introducing a new dataform. Simple measures can be made of the amount, frequency, and spatial representation of each of the datasets. Many of these analyses can already be performed with the new database. A standard questionnaire or interview guide could also be developed to evaluate how adequately ADMADE is meeting the information needs of its primary stakeholders.

Other Research Needs

Like all proper inductive research, this study identified as many new areas for research as it was able to illuminate. This section describes some of the broad topics that need further work, while research studies on several more specific topics are outlined in Appendix VI.

Data Quality Framework for Indices
Measures of data quality in the context of using indirect indicators are still relatively undeveloped. Developing a methodology for measuring the quality of data of indices would allow comparisons of monitoring data over time, as well as give users an idea of which data is more trustworthy . The variables which impact data quality for indices are already well known, and include:

What remains is to develop a framework and coding system which operationalizes each of these elements into a data quality index for a dataset. ADMADE, with its heavy use of indicators, would be the perfect setting to develop and test such a framework.

Using Information for Local Resource Plans
ADMADE would also be an excellent testing ground for developing and testing a framework for using information to negotiate community-level co-management plans, and land use plans. Community level resource planning is one of the ultimate aims of CBNRM, but the process is often directed by outsiders. The land-use planning workshops in ADMADE provide a natural experiment to measure how information is made available, presented, understood, trusted, interpreted, ignored, etc. by the various stakeholders in community-level resource planning workshops. Compiling case studies and developing a framework would guide the planning and facilitation of other resource planning efforts, including those addressed to non-wildlife resources such as forest products, fisheries, and mining.

Cost-Benefit Curve of Monitoring
ADMADE's present challenge to prioritize its data needs in light of its limitations offers another opportunity for research on the cost-benefit curve of monitoring. Donors, government agencies, and projects alike eventually have to deal with this issue at one scale or another: when do the additional investments in monitoring no longer warrant the additional benefits? Monitoring costs are not only financial, but can also be measured in terms of staff time and the number of new skills/activities that can be realistically absorbed by communities within a finite timespan. Developing a methodology for calculating a cost index for each type of dataset, as well as a way to measure the importance of each type of data, could help programs like ADMADE strategically plan their data needs. This in turn may help avoid the unfortunately too common de-facto method of data prioritization - continuing to add new monitoring elements until the program can no longer cope with all the training, supervision, and data processing requirements, eventually resulting in one or more datasets haphazardly dying from neglect.

Acknowledgements | About the Report | Acronyms | Introduction | Monitoring in ADMADE | Synthesizing Results | Interventions | Conclusion | Bibliography | Monitoring Framework | ADM Menu System | Data Analysis Conceptual Framework | Monitoring Workshop Notes | Additional Research
Readers' Comments | Make Comments