Evaluation of Mainstreaming Sustainable Land Management in Ngamiland Productive Landscape

Report Cover Image
Evaluation Plan:
2015-2016, Botswana
Evaluation Type:
Mid Term Project
Planned End Date:
07/2016
Completion Date:
12/2016
Status:
Completed
Management Response:
Yes
Evaluation Budget(US $):
25,000

Share

Document Type Language Size Status Downloads
Download document UNDP-GEF-MTR-TOR-NGAMILAND SLM.pdf tor English 439.79 KB Posted 265
Download document Ngamiland SLM MTR Report_30 Nov 2016_final.pdf report English 2103.27 KB Posted 356
Download document UNDP-GEF-MTR-TOR-NGAMILAND SLM.pdf tor English 439.79 KB Posted 98
Title Evaluation of Mainstreaming Sustainable Land Management in Ngamiland Productive Landscape
Atlas Project Number:
Evaluation Plan: 2015-2016, Botswana
Evaluation Type: Mid Term Project
Status: Completed
Completion Date: 12/2016
Planned End Date: 07/2016
Management Response: Yes
Focus Area:
  • 1. Others
Corporate Outcome and Output (UNDP Strategic Plan 2014-2017)
  • 1. Output 1.3. Solutions developed at national and sub-national levels for sustainable management of natural resources, ecosystem services, chemicals and waste
Evaluation Budget(US $): 25,000
Source of Funding: GEF
Evaluation Expenditure(US $): 32,697
Joint Programme: No
Joint Evaluation: No
Evaluation Team members:
Name Title Email Nationality
Maria Onestini Evaluator rponesti@criba.edu.ar ARGENTINA
GEF Evaluation: Yes
GEF Project Title: Mainstreaming SLM in Rangeland Areas of Ngamiland District Landscapes for Improved Livelihoods
Evaluation Type: Mid-term Review
Focal Area: Land Degradation
Project Type: FSP
GEF Phase: GEF-5
GEF Project ID: 4751
PIMS Number: 4629
Key Stakeholders: Ministry of Environment Wildlife and Tourism, Ministry of Agriculture
Countries: BOTSWANA
Comments:

Project MTE delayed however TORs are being finalised with support from UNDP-GEF regional coordination unit. Project currently experiencing challenges with only 1 active staff member covering the entire Ngamialand disctrict for the past 3 months. 

Lessons
1.

1. The value of a robust, realistic design cannot be overestimated. A proper design is strongly linked to eventual successful implementation and obtaining developmental sustainable results. A strong design implies thorough knowledge of the policy context of a country and it should be realistic in relation to what can be achieved through implementation in relation to the national context.


2.

2. Furthermore, if design of projects does not include issues that are crucial to achieve outcomes, these will not occur routinely. Design should include developmental and equity issues deeply imbedded in analysis, expected outcomes, and expected outputs/products in order to make certain that these do occur. Issues such as gender, livelihoods in a context of sustainable development, and equity need to be interweaved into a project’s expected outputs and expected results.


3.

3. Robust design also implies the development of robust methodological tools that can aid and further implementation and sustainability (such as indicators, log frames, means of verification, planning tools, and the like).


4.

4. Projects that do not have a ‘road map’ with time bound achievements and time bound indicators to meet tend to strain with implementation programming. This is in detriment of seeking results since tardiness in achieving products takes time away from implementing outputs and seeking sustainability of achievements.

 


5.

5. Partners that are included in whatever capacity need to straightforwardly analyse and acknowledge their capacity to be involved before committing to a project. If a partner cannot implement, direct, or take part in whatever outputs, outcomes, or processes they are committed to, they should not be involved in a project since this hinders the intervention as a whole.

 


6.

6. Stakeholder participation, including participation from non-governmental organizations and intended beneficiaries, should be a process that begins in design and is fulfilled throughout all of the stages of an intervention, and this can only improve a project. Paternalistic approaches with beneficiaries hinder effect since at times this implies that their points of view and needs are not truly heeded and attempts to deal with root causes of a problem are hindered.


Findings
1.

DESIGN:

The design of the Project follows standard structure for these sorts of interventions with intended outcomes and outputs within a framework of expected goal and expected objective. Moreover, the formal logic of the Bio – Chobe project identifies threats and plans to endeavour to act upon them in order to obtain products, processes and results.
Outcomes are clearly established as intended short and medium term effects of the intervention. That is, expected Outcome 1 (Collaborative governance framework in place in PAs and buffer zones resulting in reduced threats to biodiversity and economic growth) and expected Outcome 2 (Management effectiveness and financial sustainability in core protected areas strengthened to address existing and emerging threats to biodiversity) are well expressed in the sense that they are established as expected results that would stem from the Project. At the output level these are also adequately expressed. That is, outputs (i.e. the products and services which should result from the completion of activities within a development intervention) are also expressed adequately in that manner. 6 The overall approach, nevertheless, is satisfactory, in the sense that barriers are identified and ways to overcome these are recognised. That is, the design identifies the barriers and delineates processes/activities that to some extent could conceivably breach the gaps needed to create conditions for effective management of protected areas in Botswana. The baseline analysis contains a thorough and strong amount of data and analysis, particularly of the condition of biodiversity in the area and draws upon several commissioned analyses in order to provide reference information regarding the planned Project.
That being said however, there is a prevailing weakness and vagueness at the most basic level of conceptualization. That is, there is wide-ranging imprecision on the products that need to be developed and implemented in order to achieve the outputs and outcomes needed for the Project to succeed.

The imprecision has resulted in disagreement between the Project and partners in what is to be done. In the Project Document, perhaps due to these intrinsic imprecisions, there is a list of outputs per outcome (yet since they are an addendum they are not integrated fully into the project design/log frame, etc.) and there is an indication that “a detailed activity list and a chronogram of activities per output is under development and will be finalised upon project inception”. However, this has never been developed nor acknowledged by stakeholders.

READ MORE ON THE DOCUMENT 


2.

RESULTS FRAMEWORK/LOGFRAME:

The Project established baseline indicators9 in the Project Document (Logical Framework) at the level of Objective, the two expected Outcomes level, and for several –not all-- of the expected outputs. For each indicator, baseline figures are established to a great degree and target figures are indicated in most cases. No midterm targets/indicators are established.
For the most part, the comprised indicators – that is, if they are included in the Log Frame and expressed as such-- are specific and measurable, they are expressed as numbers or percentages or through another definite format. For some expected outputs, however, no baseline indicators are presented (i.e. this is the case for expected Output 1.2 Integrated land use plans reducing threats and expanding economy, and also for expected Output 1.3 Tourism revenue exploited and diversified in priority areas including FRs and CBNRM/CHA). Therefore, since no baseline indicators are established for these outputs, the target indicators for these same outputs cannot be analysed as a result of the intervention. When presented, the indicators are time bound given that they are expected to be achieved by the end of the intervention. In some cases, they are not specific nor measurable, for example, “Habitats stable or improving”. In this case (as in others) this phrase cannot be construed as an indicator since is not presented in a specific nor measurable format.

However, some of the target indicators are not expressed thoroughly as such, they are stated as products or processes but not as expected results. For example, in the indicator “Stakeholder forummeeting regularly, tracking progress against project indicators and ensuring timely decision making and corrective action” is not truly a target indicator of a result.

READ MORE ON THE DOCUMENT 


3.

Project Implementation and & Adaptive Management

Management Arrangements:
Project Management: The Project Steering Committee (PSC) is operating as well as could be expected given the issues of the poor rate of implementation and achievements of the whole project, attributable to the staffing issues (see Remaining barriers to achieving the project objective” above), with regular meetings – and regular comments complaining that the project is failing to achieve on-the-ground.

Implementing Partners:

Informants to the MTR from the key departments involved in the project (DFRR and DAP) have very different perspectives of the project, some much more positive than the others. The department with more positive impression of the project is more closely involved in current activities – the other is frustrated, claiming that the TRG is blocking activities they feel the project should be supporting. Both departments have appointed official counter-parts for the project, but do not always send the same staff to meetings – thus there are information gaps. The new NPC, the NPO and these key implementing partners should be working much more closely together, the project supporting the departments to implement activities or where they lack the expertise or need more support, through consultants or other partners.

GEF Partner Agency: It is regrettably the conclusion drawn from the MTR review of documents and the MTR mission that the quality of support provided by the GEF Partner Agency (UNDP) has not met the standards which would be expected. Many of the project implementation issues can be traced back to: (1) a the decision to base the NPC I in Gaborone at the early stages of the project (Sept 2014 – Feb 2015), where he was also head of the UNDP CO Environment Unit; and (2) the under-staffing of the project from October 2015, when UNDP Country Office (CO) in Gaborone took the decision to release NPC I from his contract due to his lack of focus on project activities. Clearly, this was a difficult decision to make, but it then led to a gap of 12 months until at the third round of advertising the CO recruited a second NPC in mid-September 2016. From the perspective of the implementing partners and other stakeholders in Maun and across Ngamiland, the NPC I had been building-up momentum, awareness and expectations of the project and he suddenly left. UNDP perhaps should have sent senior staff to Ngamiland to explain the dilemma and put in place more technical support and advice on project implementation for the new NPO, whose contract began only a week before the contract of NPC I was terminated

READ MORE ON DOCUMENT 


4.

MANAGEMENT ARRANGEMENTS

The management arrangements established in the design and inception phases are the characteristic organizational provisions of this sort of intervention. The Project is to be managed at the local level by a Project Manager, and a Finance and Administration Officer will be in charge of overseeing financial management of the project. These are the local management arrangements established at design and no changes have been introduced. In practice, the PMU is at the time of the midterm review a highly-committed team yet they are overextended and there is a need for strengthening the Unit.

READ MORE ON THE DOCUMENT 


5.

WORK PLANNING

The Project has had critical delays in project start up and implementation, and continues to be lagging in achieving what are to be many of the planned activities, documents, processes and therefore results at its mid-point. Staffing issues (mainly delay in PMU staffing), unclear roles and unclear partnership agreements, nationally lingering ownership of some partners, the unrealistic design, as well as the Project not leveraging the financial resources it was intended to have, have all hindered implementation. Some of these causes for delays have been resolved (at least at the time of evaluation), for instance some staffing issues. Others continue to be true and continue to pose problems in the future expected achievements of results.

READ MORE ON THE DOCUMENT 


Recommendations
1

Reduce area of land to under improved management (from a baseline of zero) from 1,000,000 ha to a more realistic 200,000 ha

2

The PMU and implementing partners need to work more closely together, which will increase ownership, facilitating post-project sustainability and scaling-up.

3

The non-beef livestock products should be taken to include small-stock and work begun urgently to enable women to benefit  from the project. 

4

The TRG should return to being the technical advisory system for the PSC, reducing the frequency of meetings and the number of members.

5

PSC / TRG and others should review whether to continue project activities on cropland areas.  If support is being continued, the emphasis should shift away from solely on conservation agriculture to a wider range of SLM technologies. 

6

PSC need to define missing targets in results framework now that the IRA has been completed. 

7

The UNDP CO need to make concerted efforts to fully support the project.

1. Recommendation:

Reduce area of land to under improved management (from a baseline of zero) from 1,000,000 ha to a more realistic 200,000 ha

Management Response: [Added: 2016/12/18] [Last Updated: 2017/03/17]

There appears to have been an over estimation for the size of the communal rangeland utilized by livestock farmers which was estimated at 800,000 hectares. Large parts of the project pilot area, though it technically qualifies at communal rangeland does not in fact have any cattle because of the poisonous plant Mogau. Large tracks of land in NG2 does not have any cattle. The recommendation to revise down the targeted area from 1 million to 200,000 hectares is therefore in order

Key Actions:

Key Action Responsible DueDate Status Comments Documents
Present the recommendation to the PSC for ratification.
[Added: 2016/12/18] [Last Updated: 2017/03/17]
PMU & Project Manager 2016/11 Completed Recommendation to be presented to the PSC meeting scheduled for 6th December 2016 History
2. Recommendation:

The PMU and implementing partners need to work more closely together, which will increase ownership, facilitating post-project sustainability and scaling-up.

Management Response: [Added: 2016/12/18] [Last Updated: 2017/03/17]

The strategy to engage more closely with Implementing Partners is to have bilateral and multilateral activities that foster continual contact. Recently, the PMU held an equipment handing over event with DFRR and DCP and the PMU also organized a joint SLM workshop with DEA. This allowed for a closer interaction between the PMU and implementing agencies.

Key Actions:

Key Action Responsible DueDate Status Comments Documents
There are a series of one-on-one activities between the PMU and implementing agencies in the 2017 AWP. These include: Automation of the irrigation systems at the DFRR Maun and Shakawe Nurseries Production of seedlings at these Nurseries Distribution of seedlings to CA farmers to make live fences Joint Beef measles campaign with DAP & DVS
[Added: 2016/12/18] [Last Updated: 2017/03/17]
PMU & Project Manager 2017/01 Completed The 2017 AWP will give ample opportunities for close cooperation and collaboration between PMU and Implementing Partners History
3. Recommendation:

The non-beef livestock products should be taken to include small-stock and work begun urgently to enable women to benefit  from the project. 

Management Response: [Added: 2016/12/18] [Last Updated: 2017/03/17]

Women make up the majority of small stock owners. The Project has through DAP been approached by Nhabe Agricultural Management Association (NAMA) to assist in preparing a feasibility study for a community abattoir in Sehithwa. Since there currently three cattle abattoirs in the District (BMC & two private ones), the new abattoir should be for small stock. The activity is planned to be undertaken in 2017.

Key Actions:

Key Action Responsible DueDate Status Comments Documents
Undertake a feasibility study for a community abattoir in Sehithwa for NAMA
[Added: 2016/12/18] [Last Updated: 2017/03/17]
PMU 2017/02 Completed The feasibility study is planned and budgeted for in the 2017 AWP History
4. Recommendation:

The TRG should return to being the technical advisory system for the PSC, reducing the frequency of meetings and the number of members.

Management Response: [Added: 2016/12/18] [Last Updated: 2017/03/17]

The frequency of meetings of the TRG (fortnightly) was intended to generate interest and commitment from IPs and ensure a thorough knowledge and understanding of the Project Document and their respective roles. There was also a sustainability strategy to have alternate representatives from each Department / institution to allow for continuity in the event that a particular member is no longer to attend the TRG for whatever reason. Whilst it is agreeable that the frequency of the meetings should reduce to quarterly (to be in line with the PSC), there is need to retain the two member attendance per Department / Institution to ensure sustainability. The meetings are held based on the need for technical input considering the nature of the activity. They can be fewer or more throughout the year.

Key Actions:

Key Action Responsible DueDate Status Comments Documents
Present the recommendation to the PSC for ratification.
[Added: 2016/12/18] [Last Updated: 2017/03/17]
PMU & Project Manager 2016/12 Completed Recommendation to be presented to the PSC meeting scheduled for 6th December 2016 History
5. Recommendation:

PSC / TRG and others should review whether to continue project activities on cropland areas.  If support is being continued, the emphasis should shift away from solely on conservation agriculture to a wider range of SLM technologies. 

Management Response: [Added: 2016/12/18] [Last Updated: 2017/03/17]

There has been a large investment made on conservation agriculture in terms of equipment. It is only logical to continue supporting the CA farmers with skills improvement to ultimately reach the target of producing yields of one tonne per hectare. Other SLM technologies will nonetheless be pursued such as live fencing, composting and bunding.

Key Actions:

Key Action Responsible DueDate Status Comments Documents
Other SLM technologies included in the 2017 AWP.
[Added: 2016/12/18] [Last Updated: 2017/03/17]
PMU 2017/01 Completed The 2017 AWP will give ample opportunities for other SLM technologies History
6. Recommendation:

PSC need to define missing targets in results framework now that the IRA has been completed. 

Management Response: [Added: 2016/12/18] [Last Updated: 2017/03/17]

The missing targets have been defined and included in the 2017 AWP.

Key Actions:

Key Action Responsible DueDate Status Comments Documents
Present the missing targets to the PSC for ratification.
[Added: 2016/12/18] [Last Updated: 2017/03/17]
PMU 2016/12 Completed Recommendation to be presented to the PSC meeting History
7. Recommendation:

The UNDP CO need to make concerted efforts to fully support the project.

Management Response: [Added: 2016/12/18] [Last Updated: 2017/03/17]

The recommendation is noted. However, UNDP CO is continuously supporting the project through participating in Project Steering Committees and special forums where invitations has been extended to the office such as addressing Full Council meetings. The country office further conduct field visits to project sites to determine the level of progress being made. In 2015, seven (7) of such visits to the project were made in an effort to support the project.

Key Actions:

Key Action Responsible DueDate Status Comments Documents
UNDP CO support to the project
[Added: 2016/12/18] [Last Updated: 2017/03/17]
UNDP No due date Initiated The Quality Assurance of the project is being done by UNDP CO. Resources are specifically allocated for this purpose in the AWP and has been continuing since inception of the project. History

Latest Evaluations

Contact us

1 UN Plaza
DC1-20th Floor
New York, NY 10017
Tel. +1 646 781 4200
Fax. +1 646 781 4213
erc.support@undp.org