Terminal Evaluation of the Adaptation in the Coastal Zones of Mozambique Project

Report Cover Image
Evaluation Plan:
2017-2020, Mozambique
Evaluation Type:
Final Project
Planned End Date:
12/2017
Completion Date:
11/2018
Status:
Completed
Management Response:
Yes
Evaluation Budget(US $):
40,000

Share

Document Type Language Size Status Downloads
Download document ToRs Terminal Evaluation - Adaptation in the Coastal Zones of Mozambique.pdf tor English 953.13 KB Posted 14
Download document TE Adaptation in the Coastal Zones of Mozambique - Final Report.pdf report English 2397.80 KB Posted 13
Title Terminal Evaluation of the Adaptation in the Coastal Zones of Mozambique Project
Atlas Project Number: 00062383
Evaluation Plan: 2017-2020, Mozambique
Evaluation Type: Final Project
Status: Completed
Completion Date: 11/2018
Planned End Date: 12/2017
Management Response: Yes
Corporate Outcome and Output (UNDP Strategic Plan 2018-2021)
  • 1. Output 1.4.1 Solutions scaled up for sustainable management of natural resources, including sustainable commodities and green and inclusive value chains
  • 2. Output 2.4.1 Gender-responsive legal and regulatory frameworks, policies and institutions strengthened, and solutions adopted, to address conservation, sustainable use and equitable benefit sharing of natural resources, in line with international conventions and national legislation
Evaluation Budget(US $): 40,000
Source of Funding: GEF
Evaluation Expenditure(US $): 32,000
Joint Programme: No
Mandatory Evaluation: No
Joint Evaluation: No
Evaluation Team members:
Name Title Email Nationality
GEF Evaluation: Yes
GEF Project Title: Adaptation in the coastal zones of Mozambique (LDCF)
Evaluation Type: Mid-term Review
Focal Area: Climate Change
Project Type: FSP
GEF Phase: GEF-5
GEF Project ID: 4276
PIMS Number: 4069
Key Stakeholders:
Countries: MOZAMBIQUE
Lessons
1.

The quality of the project implementation team and effective project management are a critical success factor for project performance: The best persons for the job should be attracted to lead the PMU; the blend of talent you can attract my render it necessary to hire staff not previously foreseen in the project document. For example, if you hire a PM who has strong project management, M&E and knowledge management capacity you may not need a M&E specialist. Or vice-versa. It is important to review the team at time of project inception. Hired staff should probably undergo a two to three months’ probation time prior to the inception workshop and be assessed for performance at that time.


2.

During inception, build a joint vision amongst the project team and ensure everyone understands their roles and responsibilities: It may be worthwhile to invest into upfront training in project management related skills development of the project team; orientations on roles and responsibilities are needed for all staff and project partners including the project board. Probably it would be a good decision to increase Project Management cost to include upfront investments into building a good team. While the inception workshop is a very helpful institution in this regard, some additional mandatory and well delivered upfront project management basics should be shared amongst the wider project team.


3.

Identifying the most suitable implementation arrangements: Countries take a lot of pride in NIM projects, and ownership usually is very good. This is an important asset. However, it is also clear that there are numerous barriers and pitfalls including that huge government bureaucracies take away the possibility for nibble and flexible project execution – often so important in demonstration project, and for adaptive management. Further thinking on how the most flexible yet well owned and integrated project delivery can be achieved needs to be part of every project design process.


Findings
1.

Project design and formulation

The design of the project was generally found to be sound and of good quality, responding to the thorough situation analysis presented in the project document. The project objective is clear, and outcomes were generally well formulated, mostly following the SMART criteria.  It is clear that at programming stage, the GEF guidelines in terms of clarity of project objectives and components; existence of capacity of the executing institutions; incorporation of relevant lessons from other project design; clarity of partnership arrangements; inclusion of counterpart resources (funding, staff, and facilities); enabling legislative environment for project implementation; appropriate project management arrangement and well-articulated assumptions and risks under each project component were considered. Indicators were very ambitious and hard to measure, especially with a view of assessing project impacts and undertaking surveys to assess beneficiary levels of know-how and adaptive capacity. Although the project document contains all elements that make up a strong ToC, it was not an integral part of the project document at the time, as this was not a programming requirement. The design seemed to be very ambitious, and probably overly ambitious in the country context of Mozambique. The project design involved relevant stakeholders, and consultation meetings were conducted at national, subnational levels, and at project implementation sites. While stakeholder participation was effective in the project design phase, as well as a stakeholder engagement plan was foreseen in the design, during project implementation this was not equally realised at all project sites. The approach was well thought through, while it is noted that the project budget did not make allocations for knowledge management and sharing of lessons learnt, as is nowadays common practice in GEF programming. Some limited funds were allocated to print and production of reports/ maps and films, however the amounts budgeted were modest.


2.

Project implementation

 Following the MTR adjustments were made to the SRF, and more importantly the management arrangements were strengthened. Decentralisation was increased and on-the-ground actions under outcome 2 were specifically “unlocked”. The non-realization of critical co-financing from UNCDF of the micro-finance component of the project had to be addressed, and was seemingly well handled as part of an adaptive management response after the MTR, in line with the new efforts to deliver on outcome 2.

While responsible project implementing partners were informed/trained about the project during the project inception, it is clear that additional trainings maybe needed in future. While the project had some established reporting lines and responsibilities, these were poorly followed through on. Staff in the provinces were disengaged from management at central level, and adaptive management was reportedly disjoint.  Regarding the finance, the LDCF project was designed to build on existing parallel investments at the project sites. As most of these investments were associated or close to UNDP, this was a good strategy. However, especially the USD 8 Mio investment by UNCDF was discontinued and did not realize as co-financing. This has serious implications for the LDCF project, which had to rethink and reposition its micro-finance strategy. The M&E budget for the project was reasonable. While reporting during quarterly reports and PIRs gave some feedback, the reporting was not strongly aligned with the SRF. It should be noted that the End of Project report which was availed to the evaluation consultants looked very impressive. However, the consultants found that the presented data was partially not in line with the evaluation findings, and seemed to be an overly positive assessment of end of project delivery. The report is a good overview of field activities implemented, but provided little reflection on impact and sustainability.


3.

Project results

Some level of triangulation took place during the TE, however certain outputs were not accessible and could not be reviewed for quality. Note that performance on outputs is not rated in the overall assessment. While the TE team did not receive any formal data that would enable a fair assessment of performance on the objective indicator(s), it can be said that the wide range of stakeholders consulted during the TE were knowledgeable about the project, climate change risks and potential adaptation options - all attained or ameliorated through activities of the project. Especially technical staff at the sub-national level, a wide range of community members, and an unknown number of individuals who were reached by information sharing through community radio clearly were touched by the project. The project has been relevant in the national development context and addressed key national priorities as set out in the NAPA. In terms of effectiveness, the project objective was only partially met. While good momentum has been gained amongst the target communities. Country ownership was generally good, and ownership on the provincial level clearly improved since the MTR. The project was formulated addressing key UNDP priorities such as climate resilience building and poverty reduction, as well as (in hind-sight) various SDGs. Gender was not very explicitly addressed, however, it is clear that especially the local level pilot interventions mainstreamed gender considerations in their approaches. Gender work could have been undertaken more systematically, however, as this was not a programming priority previously. Sustainability of the project interventions beyond the project phase is a difficult issue, and especially so in the context of this project where implementation activities on the pilot site level only really started after the MTR.


Recommendations
1

Check on outstanding payments to service providers

During the field consultations it was brought to the attention of the evaluators that several institutions seemed to think that they still needed to be paid for some of their work. For example, the team at CEPAM in Pemba was under the impression that payments were not done. It is important to service all project agreements or at least to communicate clearly what certain payments may not have been made.

2

Invest into distilling some case studies from the project

This project piloted a great diversity of interventions. On all levels, but specifically with regards to the adaptation options on the community level some very interesting demonstrations have been set up. However, due to the poor knowledge management aspects of the project no systematic documentation of the investments, processes and performance of the demonstrations are available. It is recommended that this be done esp. with the view that UNDP has prepared a new LDFC project with MITADER, which could benefit from a thorough analysis of previous efforts.

3

At least analyze the micro-finance innovation

As the absolute minimum document and analyze the micro-finance component of the project well. This approach may provide some excellent innovative ideas for strengthening communities’ resilience, however, at this point no independent and detailed review of project inputs and results is available.

4

Still undertake the endline surveys

The lack of endline surveys limit the scope to replicate/ scale-up the project. Especially in light of preparing for another large LDCF investment, it should be a worthwhile investment to provide a deeper analysis of this pilot project.

1. Recommendation:

Check on outstanding payments to service providers

During the field consultations it was brought to the attention of the evaluators that several institutions seemed to think that they still needed to be paid for some of their work. For example, the team at CEPAM in Pemba was under the impression that payments were not done. It is important to service all project agreements or at least to communicate clearly what certain payments may not have been made.

Management Response: [Added: 2018/12/10] [Last Updated: 2018/12/10]

Noted. However, for this recommendation it is important to clarify that all corresponding measures have already been taken through the project Implementing Partner (under the leadership of the Ministry Permanent Secretary), in which the concerned parties have been adequately informed on the completion of the project, as well as clarifications and completion of all corresponding administrative procedures were made.

Key Actions:

Key Action Responsible DueDate Status Comments Documents
1.1. Follow-up on the formal communication from MITADER Permanent Secretary to the project involved parties to ensure that they are all informed on the completion of the project.
[Added: 2018/12/11]
Env., NRM, CC & Resilience Unit. 2018/12 Completed
2. Recommendation:

Invest into distilling some case studies from the project

This project piloted a great diversity of interventions. On all levels, but specifically with regards to the adaptation options on the community level some very interesting demonstrations have been set up. However, due to the poor knowledge management aspects of the project no systematic documentation of the investments, processes and performance of the demonstrations are available. It is recommended that this be done esp. with the view that UNDP has prepared a new LDFC project with MITADER, which could benefit from a thorough analysis of previous efforts.

Management Response: [Added: 2018/12/11]

Acknowledged. The country office will take advantage from the upcoming LDCF project PPG to draw up a summary of lessons from this closing project to better inform the design of the new project. It is important to mention that at MTR it was recommended a substantial change of the project management unit recruited by the IP, which was under performing. The CO made necessary due diligences that resulted in changes at project administrative and financial assistant only, remaining the project manager on duty, which had strong support from Micro-finance expert hired by UNDP to ensure delivery on the project results for the remaining years. This also served as a lesson for the other projects that the recruitment of PMU should be done by UNDP to ensure effective control over the performance of each project management unit.

Key Actions:

Key Action Responsible DueDate Status Comments Documents
2.1. Invest some resources from CO regular resources to complement LDCF PPG funding in order to capture lessons learned that will inform the new LDCF and other projects in the subject of CCA, with innovative approach on Micro-finance.
[Added: 2018/12/11]
Env., NRM, CC & Resilience 2019/12 Initiated
2.2. Recruitment and performance assessments of the project management unit under UNDP’s responsibilities.
[Added: 2018/12/11]
Env., NRM, CC & Resilience 2019/12 Initiated
2.3. Ensure that all projects effectively allocate resources on M&E and communication related activities.
[Added: 2018/12/11]
Env., NRM, CC & Resilience and Management Support Unit 2019/12 Initiated
3. Recommendation:

At least analyze the micro-finance innovation

As the absolute minimum document and analyze the micro-finance component of the project well. This approach may provide some excellent innovative ideas for strengthening communities’ resilience, however, at this point no independent and detailed review of project inputs and results is available.

Management Response: [Added: 2018/12/11]

Recommendation well noted. Through its diverse portfolio, the CO will ensure that an analysis on the micro-finance innovation under the climate change adaptation umbrella is conducted to better inform future investments in this thematic area.

Key Actions:

Key Action Responsible DueDate Status Comments Documents
3.1. Conduct a quick analysis on the impact of micro-finance initiatives under the adaptation projects in 2 out of the 3 project pilot sites and drawn up key recommendations.
[Added: 2018/12/11]
Env., NRM, CC & Resilience 2019/12 Initiated
4. Recommendation:

Still undertake the endline surveys

The lack of endline surveys limit the scope to replicate/ scale-up the project. Especially in light of preparing for another large LDCF investment, it should be a worthwhile investment to provide a deeper analysis of this pilot project.

Management Response: [Added: 2018/12/11]

Noted. As indicated above, the CO will take the advantage of the upcoming LDCF PPG and invest its regular resources to complement PPG funding in order to capitalize on the lessons from this current project and better inform the new project and other related initiatives that could benefit from this project best practices and innovative approaches.

Key Actions:

Key Action Responsible DueDate Status Comments Documents
4.1. Invest some resources from CO regular resources to complement LDCF PPG funding to conduct necessary surveys that will provide necessary analysis of the current project to better inform the new LDCF and other upcoming/related projects; ensuring that good practices are appropriately replicated.
[Added: 2018/12/11]
Env., NRM, CC & Resilience 2019/12 Initiated
4.2. Ensure that future projects provide necessary budget allocations dedicated for such kind of surveys, which can be costly to be included under PM charges
[Added: 2018/12/11]
Env., NRM, CC & Resilience and Management Support Unit. 2019/12 Initiated

Latest Evaluations

Contact us

220 East 42nd Street
20th Floor
New York, NY 10017
Tel. +1 646 781 4200
Fax. +1 646 781 4213
erc.support@undp.org