Terminal Evaluation of the Adaptation in the Coastal Zones of Mozambique Project

Report Cover Image
Evaluation Plan:
2017-2021, Mozambique
Evaluation Type:
Final Project
Planned End Date:
12/2017
Completion Date:
11/2018
Status:
Completed
Management Response:
Yes
Evaluation Budget(US $):
40,000

Share

Document Type Language Size Status Downloads
Download document ToRs Terminal Evaluation - Adaptation in the Coastal Zones of Mozambique.pdf tor English 953.13 KB Posted 463
Download document TE Adaptation in the Coastal Zones of Mozambique - Final Report.pdf report English 2397.80 KB Posted 1198
Title Terminal Evaluation of the Adaptation in the Coastal Zones of Mozambique Project
Atlas Project Number: 00062383
Evaluation Plan: 2017-2021, Mozambique
Evaluation Type: Final Project
Status: Completed
Completion Date: 11/2018
Planned End Date: 12/2017
Management Response: Yes
UNDP Signature Solution:
  • 1. Sustainable
Corporate Outcome and Output (UNDP Strategic Plan 2018-2021)
  • 1. Output 1.4.1 Solutions scaled up for sustainable management of natural resources, including sustainable commodities and green and inclusive value chains
  • 2. Output 2.4.1 Gender-responsive legal and regulatory frameworks, policies and institutions strengthened, and solutions adopted, to address conservation, sustainable use and equitable benefit sharing of natural resources, in line with international conventions and national legislation
SDG Goal
  • Goal 13. Take urgent action to combat climate change and its impacts
  • Goal 15. Protect, restore and promote sustainable use of terrestrial ecosystems, sustainably manage forests, combat desertification, and halt and reverse land degradation and halt biodiversity loss
SDG Target
  • 13.2 Integrate climate change measures into national policies, strategies and planning
  • 15.2 By 2020, promote the implementation of sustainable management of all types of forests, halt deforestation, restore degraded forests and substantially increase afforestation and reforestation globally
Evaluation Budget(US $): 40,000
Source of Funding: GEF
Evaluation Expenditure(US $): 32,000
Joint Programme: No
Joint Evaluation: No
Evaluation Team members:
Name Title Email Nationality
Dr. Juliane Zeidler International Consultant and Team Leader)
Ermínio Jocitala National Consultant)
GEF Evaluation: Yes
GEF Project Title: Adaptation in the coastal zones of Mozambique (LDCF)
Evaluation Type: Mid-term Review
Focal Area: Climate Change
Project Type: FSP
GEF Phase: GEF-5
GEF Project ID: 4276
PIMS Number: 4069
Key Stakeholders:
Countries: MOZAMBIQUE
Lessons
1.

The quality of the project implementation team and effective project management are a critical success factor for project performance: The best persons for the job should be attracted to lead the PMU; the blend of talent you can attract my render it necessary to hire staff not previously foreseen in the project document. For example, if you hire a PM who has strong project management, M&E and knowledge management capacity you may not need a M&E specialist. Or vice-versa. It is important to review the team at time of project inception. Hired staff should probably undergo a two to three months’ probation time prior to the inception workshop and be assessed for performance at that time.


2.

During inception, build a joint vision amongst the project team and ensure everyone understands their roles and responsibilities: It may be worthwhile to invest into upfront training in project management related skills development of the project team; orientations on roles and responsibilities are needed for all staff and project partners including the project board. Probably it would be a good decision to increase Project Management cost to include upfront investments into building a good team. While the inception workshop is a very helpful institution in this regard, some additional mandatory and well delivered upfront project management basics should be shared amongst the wider project team.


3.

Identifying the most suitable implementation arrangements: Countries take a lot of pride in NIM projects, and ownership usually is very good. This is an important asset. However, it is also clear that there are numerous barriers and pitfalls including that huge government bureaucracies take away the possibility for nibble and flexible project execution – often so important in demonstration project, and for adaptive management. Further thinking on how the most flexible yet well owned and integrated project delivery can be achieved needs to be part of every project design process.


Findings
1.

3. FINDINGS

3.1 Project design and formulation

3.1.1 Analysis of strategic results framework

The design of the project was generally found to be sound and of good quality, responding to the thorough situation analysis presented in the project document. The project objective is clear, and outcomes were generally well formulated, mostly following the SMART criteria outlined in Table 2, above. 

It is clear that at programming stage, the GEF guidelines in terms of clarity of project objectives and components; existence of capacity of the executing institutions; incorporation of relevant lessons from other project design; clarity of partnership arrangements; inclusion of counterpart resources (funding, staff, and facilities); enabling legislative environment for project implementation; appropriate project management arrangement and well-articulated assumptions and risks under each project component were considered. The MTR identified that certain indicators and set targets were very ambitious, and recommended a revision at MTR stage (see Table 8). The revision focused on outcome indicators. Detailed information on the suggested revisions and the rationale are presented in the following: Table 8: Revised indicators proposed at MTR. Note that these were not updated in the SRF prior to TE, but are included in this report.

Notably, indicators remained very ambitious and hard to measure, especially with a view of assessing project impacts and undertaking surveys to assess beneficiary levels of know-how and adaptive capacity. This will be further discussed in the M&E section below. Although the project document contains all elements that make up a strong ToC, it was not an integral part of the project document at the time, as this was not a programming requirement. Designing projects with an outcome and impact focus – and getting implementation teams to fully embrace that thinking – is only just emerging, but certainly should be encouraged. Overall, as already identified at MTR stage, the design seemed to be very ambitious, and probably overly ambitious in the country context of Mozambique. Mozambique is one of the least developed LDC’s and faces significant challenges in terms of governance and implementation capacities at all levels – governmental and non-governmental, policy coordination, implementation and enforcement, research – to name just a few. Designing complex and ambitious projects will require the inclusion of sufficient practical support i.e. to the project management unit. All project partners need to understand the whole intend of the project to ensure that it can be managed for maximum results. Interviews with key stakeholders and project beneficiaries clearly highlight that the designed project was seen to be of critical importance to generating climate change adaptation know-how in Mozambique and to spark innovative interventions and approaches to deal with CC risks in coastal zone. Especially a decentralized focus on community empowerment, community investments and microfinance products and services were highly regarded. 


Tag: Climate Change Adaptation Climate change governance Disaster Risk Reduction Global Environment Facility fund Results-Based Management Risk Management Theory of Change

2.

3.1.4 Planned stakeholder participation

The project design involved relevant stakeholders, and consultation meetings were conducted at national, subnational levels, and at project implementation sites. Consultations at site level sought to assess the coastal erosion baseline conditions, the Vulnerability and Capacity Assessment (VCA) to establish the baseline of Communities’ vulnerability towards CC, Sea Level Rise (SLR) and induced coastal erosion, the CC Capacity Assessment (CCA), and to establish the baseline for stakeholder’s capacity. Besides, the project document contained the framework to guide two-way engagement between key implementing partners (MITADER and responsible parties) and relevant stakeholder with whom the project had to engage throughout project implementation.

This engagement aimed to: (i) ensure a general vision and understanding of the project and its expected outcomes by all concerned stakeholders; (ii) engage key stakeholders in planning, implementing and monitoring of specific interventions; (iii) ensure consistent, supportive and effective communication (information, documentation, sharing, learning and feedback) processes with key interaction groups and the wider public and (iv) influence and ensure strategic level support for project implementation from state and non-state organizations and international agencies through engagement in effective community, private sector and donor forums or platforms.

While stakeholder participation was effective in the project design phase, as well as a stakeholder engagement plan was foreseen in the design, during project implementation this was not equally realised at all project sites. Especially in Pebane (Zambezia), stakeholder participation was reported as poor (see below). Additionally, the strategy employed to facilitate stakeholder engagement was, in hindsight, probably ineffective. By making MITADER national office the central piece for coordination and budget flows certain barriers to decentralisation and stakeholder engagement were induced instead of effectively removed. This will be further discussed below, under Project Implementation. 

3.1.4 Replication approach

The project was exclusively designed to learn lessons on how investments in climate-resilient livelihoods can be profitable; including promoting the extension of microfinancing services; integration of climate risk information into land-use guidelines, coastal zone management regulations and development plans at national, provincial and community level. It was anticipated that lessons from the project would be replicated into coastal provinces of Mozambique. This would be achieved through political awareness of the need for adaptation, promotion of dialogue among policy makers for the other coastal provinces and, public awareness, field demonstration, sharing of project results and lessons through communication media and knowledge network. The approach was well thought through, while it is noted that the project budget did not make allocations for knowledge management and sharing of lessons learnt, as is nowadays common practice in GEF programming. Some limited funds were allocated to print and production of reports/ maps and films, however the amounts budgeted were modest. 


Tag: Climate Change Adaptation Climate change governance Global Environment Facility fund Project and Programme management Strategic Positioning Sustainability Capacity Building Technical Support

3.

3.2 Project implementation

3.2.1 Adaptive management

Up until the MTR, and a coinciding mission by the project’s Regional Technical Advisor , the project clearly underperformed. The absence of adaptive management up to that point was noted by the mission and criticized at MTR stage, and the PMU was primarily made responsible for project failure, both by the reviewer and interviewed project stakeholders. In reaction to the poor project implementation performance systematically documented and demonstrated at MTR stage, the PB, and more specifically UNDP, took up their oversight responsibilities and started enacting adaptive management. A strong management response was crafted and implementation of it was seen through determinedly (see Annex 6). Following the MTR adjustments were made to the SRF, and more importantly the management arrangements were strengthened. Decentralisation was increased and onthe-ground actions under outcome 2 were specifically “unlocked”. The non-realization of critical cofinancing from UNCDF of the micro-finance component of the project had to be addressed, and was seemingly well handled as part of an adaptive management response after the MTR, in line with the new efforts to deliver on outcome 2. While responsible project implementing partners were informed/trained about the project during the project inception, it is clear that additional trainings maybe needed in future. The importance of ensuring that the responsible implementing partners and the project management team are fully understanding of the project per se, its designed intensions cannot be overstated. Further key project staff and oversight personnel need to have a full understanding of results-based (project) management (RBM). This is often being taken for granted. Adaptive management is only one element of such RBM. In a complex multi-partner intervention such as this LDCF project, strong coordination and management capacities are a prerequisite for project performance. With sub-hubs in three provinces, away from the central PMU office, this played an even greater role. While the project had some established reporting lines and responsibilities, these were poorly followed through on. Staff in the provinces were disengaged from management at central level, and adaptive management was reportedly disjoint. Up to MTR, almost no actions were reported from the provincial level at all. At the national level, the evaluators found that there was reasonable planning and reporting, in line with GEF and UNDP guidelines, however, this apparently was not internalized as adaptive management tool. Project reporting such as through the PIRs and quarterly reports did not strictly follow best practice, and reports did not align with the project SRF, for example. According to stakeholders on the provincial level, adaptive management particularly improved after the MTR management response, and local level facilitators/ field staff were positively viewed in facilitating demand led and adaptive management. However, it was mentioned that such field officers were not always supported effectively in implementing their work by MITADER and the PMU, e.g. in the processing of contracts and disbursement of funds, and UNDP had to step in to facilitate some of these support actions. 


Tag: Efficiency Implementation Modality Monitoring and Evaluation Partnership Project and Programme management Coordination

4.

3.3 Project results

3.3.1 Achievement of project outputs The achievements of project results were based on the various PIPRS reviewed, and substantiated further with the information gathered from interviews and site visits. As this project was not managed on the SRF basis, a review of the output achievement as reported by project staff. Some level of triangulation took place during the TE, however certain outputs were not accessible and could not be reviewed for quality. Note that performance on outputs is not rated in the overall assessment. 

Output 1.1: A dynamic monitoring system for dunes, beaches, mangroves and sea level rise established to measure topographic, oceanographic, chemical and biological indicators.  Maps and platforms on climate change risk management of the three coastal zones were developed. Local community members (Pemba- 75; Inharrime- 22; and Pebane-45) were involved as volunteers in primary data collection. As part of the establishment of a monitoring system for climate change risk management system, a consultant was hired to establish topographic, oceanographic and biological baseline indicators. Public employees (Pemba- 12; Inharrime- 12; and Pebane- 17) drawn from relevant institutions were trained in GIS Mapping to enable them undertake assessment of climate change risk and vulnerability in the project sites. In addition, the project supported the purchase and installation of automatic weather equipment for the three weather stations (Pemba; Inharrime and Pebane). To ensure operational sustainability and impacts of the new stations, INAM technical staff (a total of 12) from the three project sites attended a training about the use and regular maintenance of the equipment and link with the local radio stations. Note by the TE consultants: While we are able to access study containing georeferenced analysis and maps, achievement of this output is not guaranteed. The study is still in its draft form and will need to be validated through a consultative process/ workshop. Several training materials were availed in the form of PowerPoint presentations. Indicators were not communicated. The automatic weather station could be visited in Pemba and Pebane. Stakeholder interviews in Inharrime confirmed the existence of the automatic weather station financed by the project. 


Tag: Climate Change Adaptation Climate change governance Disaster Risk Reduction Natural Resouce management Knowledge management Monitoring and Evaluation Partnership Policies & Procedures Capacity Building

5.

3.3 Project results

3.3.1 Achievement of project outputs (continuation)

Output 2.1: Micro-financing extended to each of the seven project sites in Pemba, Pebane and Inharrime, to disburse adaptation financing and capacity development for livelihood enhancement and diversification, to reduce vulnerability to climate change.

Microfinance Service Providers were hired through a competitive process; CCOM in Pemba, FDM in Inharrime and Tseco MF in Pebane to ensure diversification and improvement of communities’ livelihoods in response to climate vulnerability, through 3 segments of activities: agriculture, fishery and small business. In Pebane, the Ministry took the decision to terminate the contract with the Microfinance service provider Tseco MF for administrative reasons and misuse of project funds. This has directly affected negatively 1,903 households that were deprived of financial and non-financial services. However, some activities were carried out in this component such as development of skills for generation and diversification of incomes in agriculture. In Pemba, 2,381 HH directly benefiting from these project products: (Solidarity credits – 352HH; Individual credits – 144HH, Saving Groups - 188members/HH, training and technical assistance – 958HH, specialized associations–257HH). This corresponds to 53.3% of (6,446HH and 70,74% of the 3,223HH - target established in the project) with income sources and livelihoods diversified (53.38% women headed HH, and a total of 1,217 involved women). 


Tag: Climate Change Adaptation Climate change governance Communication Knowledge management Resilience Jobs and Livelihoods Micro-credit Capacity Building Vulnerable

6.

3.3.2. Overall results (attainment of objective)

Objective While the TE team did not receive any formal data that would enable a fair assessment of performance on the objective indicator(s), it can be said that the wide range of stakeholders consulted during the TE (some 75 individuals in additional to several community meetings, bringing the number to approximately 130) were knowledgeable about the project, climate change risks and potential adaptation options - all attained or ameliorated through activities of the project. Especially technical staff at the sub-national level, a wide range of community members (of which 10,718 were targeted by the project), and an unknown number of individuals who were reached by information sharing through community radio clearly were touched by the project. It has been entirely impossible for the TE team to get any meaningful data about the overall achievement of the project objective, and especially the project impact (see below ROTi assessment). The PIR 2017 report presents the cumulative progress since project start. In the absence of the formal indicator assessment, a description of progress towards the objective is given and used for the assessment. In summary it conveys:

• Although a quantitative evaluation of the project has not been completed, in general, work has been done in view of ensuring the ownership of all the stakeholders involved in the project implementation including the community members. • The implementation of the various Memorandums’ of Understanding that were signed between MITADER and a suite of partners were successful and supported the communities in the adaptation and management of the climate impacts. • Information was produced and important mechanisms were established to support communities in the process of adapting and managing climate change impacts and to assist them in the integration and orientation of land use planning in the coastal zones • For the of adaptation measures at household level, two financial service providers were hired (CCOM and FDM) who have received specialized trainings on climate and disaster risk management. This has resulted in the access of informed CC (climate change) financial services and consequently improvement of household incomes and livelihoods options and resilience to Climate Change. • In Pemba the project has directly reached 4,716 households (which 52% are women and 48% men), In Pebane, 2,650 households were covered (of which 57 % are women and 43% are men) and in Inharrime 511 households of which 80% are women and 20 % are men). This amounts to 74,4 % of the households initially targeted (10,718 HH). Overall the achievement rating of attainment of results is less positive than the ratings provided in the PIR 2017. Based on the limited quantitative data available, the achievement could probably be rated as overall moderately satisfactory. 


Tag: Climate Change Adaptation Climate change governance Effectiveness Monitoring and Evaluation Partnership Data and Statistics

7.

3.3.3. Relevance, Effectiveness & Efficiency

Relevance: The project has been relevant in the national development context and addressed key national priorities as set out in the NAPA. Considering that i.e. Inhambane province was hit by a devastating cyclone during 2017, investing into resilience building and adaptive capacities clearly has been highlighted as a must. While the cyclone did not affect the project area significantly, upper level provincial decision makers could make the linkage. Piloting adaptation options and approaches also still is highly relevant, especially now at project end when results are forthcoming after a delayed project implementation. Notably, sharing of such knowledge has not taken place, but would be relevant. The project was further aligned with the GEF OP on CCA and addressed LDCF priorities.

Effectiveness: The project objective was only partially met. While good momentum has been gained amongst the target communities especially in Pemba and Inharrime, it is not entirely clear in how far they have internalized adaptation know how and interventions. So, it is unclear if the objective would be met in the future. It seems unlikely to happen without further and sustained support efforts to the same target communities and support services. However, an opportunity seems to be that the micro-finance institutions the project worked with seem to stay in the project communities. 

Efficiency: In the absence of relevant financial information it is impossible to assess this criterium. 


Tag: Climate Change Adaptation Climate change governance Effectiveness Efficiency Relevance Global Environment Facility fund Gender Mainstreaming Ownership Risk Management Poverty Reduction

8.

3.3.6. Sustainability

Sustainability of the project interventions beyond the project phase is a difficult issue, and especially so in the context of this project where implementation activities on the pilot site level only really started after the MTR. While some impressive site level investments have been made, it is entirely unclear how they will continue. Already during the TE site visits, which took place a couple of months after project closure, many of the community pilots were not functional. A pig rearing project in Inharrime had suffered from an epidemic and all but three piglets survived. The very innovative mariculture pilots in Pemba suffered from the fact that the initially sued building material, bamboo, was not durable, and much of the initial investment had faltered. These are just a few examples of less convincing evidence for sustainability. On the other hand, the micro-finance investments all seem to continue. While it has been controversial to “donate” project funds to such microfinance institutions (they should be self-running and any cash provided should strictly be retained as cashflow), their continued presence in the project sites will likely lead to some lasting impacts. Further follow-up on this would be desirable. It is noted that the lack of undertaking the project endline surveys limit the scope to replicate/ scale-up the project results, including with regards to other newly formulate GEF LDCF investment for Mozambique. It is therefore recommended that such surveys still should be done. 


Tag: Climate Change Adaptation Climate change governance Sustainability Risk Management

9.

3.3.7. Impact

The Review of Outcomes to Impacts approach (ROtI) approach is used to assess the likelihood of impact by building upon the concepts of Theory of Change (Section 2). The ROtI approach requires ratings to be determined for the outcomes achieved by the project and the progress made towards the ‘intermediate states’ at the time of the evaluation. The rating system is presented in Table 13 below and the assessment of the project’s progress towards achieving its intended impacts is presented in Table 13. Table 13: Rating Scale for Outcomes and Progress towards Intermediate States.

There are uncertainties about the achievement of project outputs and outcomes, already described above. Rating of progress towards Outcomes is rated “C”. However, at least on the project site specific level the intermediate states have partially been started or achieved. Rating of progress towards the Intermediate States is rated “B”. Based on the above, the aggregate rating is “CB”. In the long-term, creating a foundation for knowledge and evidence-based planning and management with regards to CC will lead to improvements in local livelihoods and ecosystems - which means that environmental changes are expected to be positive. The Project, with an aggregated rating of CB can therefore be rated as “Likely” to achieve/ contribute to the expected Impact.


Tag: Impact Knowledge management Project and Programme management

10.

(Continuation from Finding 3)

3.2.4. Project Finance - Co-financing The LDCF project was designed to build on existing parallel investments at the project sites. It was intended to work in parallel with a number of site based projects underway and to add incremental climate change adaption know-how in them. As most of these investments were associated or close to UNDP, this was a good strategy. However, especially the USD 8 Mio investment by UNCDF was discontinued and did not realize as co-financing. This has serious implications for the LDCF project, which had to rethink and reposition its micro-finance strategy. It should be noted that UNDP Country Office (CO) was able to leverage a significant amount of project co-financing through TRAC resources. Especially at the end of the project, UNDP CO mobilized needed cash inputs to finalize the building/ rehabilitation of the Paquite/ Pemba drainage canal, which is considered a key project success. Table 11: Planned versus realized co-financing by source. Table 12: Actual spending as per outcome.


Tag: Operational Efficiency Results-Based Management Climate Change Adaptation Climate change governance Environment Policy Efficiency Implementation Modality Monitoring and Evaluation

11.

Effectiveness (Continuation from Finding 4)

Output 1.5: Toolkit developed outlining methodologies used to assess climate change risks, adaptation planning, cost effectiveness analysis and a replication plan for Mozambique. In accordance with the laws of Mozambique, the project supported INGC establish CLGRC in the seven project communities. These committees were trained in CCA and they provide relevant community information of climate change. Additionally, the committees were provided with emergency kits to support vulnerability reduction and community awareness campaign. Note by the TE consultants: Apparently, there was no clear guidance on the toolkit, and project management discussions with INGC suggested that the project could support existing efforts in strengthening preparedness measures at community level by creating, training and equipping local disaster management committee as part of their mandate. This work was done, as verified by stakeholders at all three pilot sites. It is also important here to mention the integration of this measure with involvement of Community Radios in broadcasting specific awareness messages in local languages, reported on below. 


Tag: Agriculture Climate Change Adaptation Climate change governance Disaster Risk Reduction Effectiveness Communication Operational Efficiency Partnership Risk Management Institutional Strengthening Knowledge management

12.

Project Results -Achievement of Project Outputs (Continuation from Finding 5)

Output 2.2: Adaptation investment plan developed for each of the seven pilot sites in Pemba, Pebane and Inharrime for community-level CCA measures such as small-scale infrastructure and ecosystembased measures. VCA consultations were conducted in seven target communities; major investments priorities were identified. An amount of 170,000 US$ was allocated to each community for building small-scale infrastructure and ecosystem based measures. Note by the TE consultants: None of the Adaptation Investment Plans were accessible for review by the evaluators, but a Needs Assessment Report was shared by UNDP CO. The evaluators did not access financial information on how these funds were spent. It was difficult at the time of the TE to assess value for money, cost effectiveness and sustainability. USD 170,000 per site and even more for Pemba where the construction/ rehabilitation of the drainage canal will bring this figure up to USD 800,000, is a sizeable investment. At this point, impacts are undocumented, and in Pebane questionable.


Tag: Effectiveness Impact Knowledge management Climate Change Adaptation Climate change governance Environment Policy

Recommendations
1

4.2 Actions to follow up or reinforce initial benefits from the project A small set of recommendations emerges from the TE:

Recommendation 1. Check on outstanding payments to service providers

During the field consultations it was brought to the attention of the evaluators that several institutions seemed to think that they still needed to be paid for some of their work. For example, the team at CEPAM in Pemba was under the impression that payments were not done. It is important to service all project agreements or at least to communicate clearly what certain payments may not have been made.

2

Recommendation 2. Invest into distilling some case studies from the project

This project piloted a great diversity of interventions. On all levels, but specifically with regards to the adaptation options on the community level some very interesting demonstrations have been set up. However, due to the poor knowledge management aspects of the project no systematic documentation of the investments, processes and performance of the demonstrations are available. It is recommended that this be done esp. with the view that UNDP has prepared a new LDFC project with MITADER, which could benefit from a thorough analysis of previous efforts.

3

Recommendation 3. At least analyze the micro-finance innovation

As the absolute minimum document and analyze the micro-finance component of the project well. This approach may provide some excellent innovative ideas for strengthening communities’ resilience, however, at this point no independent and detailed review of project inputs and results is available.

4

Recommendation 4. Still undertake the endline surveys

The lack of endline surveys limit the scope to replicate/ scale-up the project. Especially in light of preparing for another large LDCF investment, it should be a worthwhile investment to provide a deeper analysis of this pilot project.

5

4.3 Proposals for future directions underlining main objectives

Before the new LDCF project will be incepted it will be extremely useful to process the actual technical results in some more depth and discuss them with the key stakeholders and partners in a learning event. 

4.4 Best and worst practices in addressing issues relating to relevance, performance and success

Lessons #1: The quality of the project implementation team and effective project management are a critical success factor for project performance 

The best persons for the job should be attracted to lead the PMU; the blend of talent you can attract my render it necessary to hire staff not previously foreseen in the project document. For example, if you hire a PM who has strong project management, M&E and knowledge management capacity you may not need a M&E specialist. Or vice-versa. It is important to review the team at time of project inception. Hired staff should probably undergo a two to three months’ probation time prior to the inception workshop and be assessed for performance at that time. 

Lessons #2: During inception build a joint vision amongst the project team and ensure everyone understand their roles and responsibilities

It may be worthwhile to invest into upfront training in project management related skills development of the project team; orientations on roles and responsibilities are needed for all staff and project partners including the project board. Probably it would be a good decision to increase Project Management cost to include upfront investments into building a good team. While the inception workshop is a very helpful institution in this regard, some additional mandatory and well delivered upfront project management basics should be shared amongst the wider project team. 

Lessons #3: Identifying the most suitable implementation arrangements

Countries take a of pride in NIM projects, and ownership usually is very good. This is an important asset. However, it is also clear that there are numerous barriers and pitfalls including that huge government bureaucracies take away the possibility for nibble and flexible project execution – often so important in demonstration project, and for adaptive management. Further thinking on how the most flexible yet well owned and integrated project delivery can be achieved needs to be part of every project design process.  

4.5 Ratings

As part of the TE, a table with the summary ratings of the project’s results and performance are provided in a TE Ratings & Achievement Summary Table(pg. 53). 

1. Recommendation:

4.2 Actions to follow up or reinforce initial benefits from the project A small set of recommendations emerges from the TE:

Recommendation 1. Check on outstanding payments to service providers

During the field consultations it was brought to the attention of the evaluators that several institutions seemed to think that they still needed to be paid for some of their work. For example, the team at CEPAM in Pemba was under the impression that payments were not done. It is important to service all project agreements or at least to communicate clearly what certain payments may not have been made.

Management Response: [Added: 2018/12/10] [Last Updated: 2020/11/29]

Noted. However, for this recommendation it is important to clarify that all corresponding measures have already been taken through the project Implementing Partner (under the leadership of the Ministry Permanent Secretary), in which the concerned parties have been adequately informed on the completion of the project, as well as clarifications and completion of all corresponding administrative procedures were made.

Key Actions:

Key Action Responsible DueDate Status Comments Documents
1.1. Follow-up on the formal communication from MITADER Permanent Secretary to the project involved parties to ensure that they are all informed on the completion of the project.
[Added: 2018/12/11]
Env., NRM, CC & Resilience Unit. 2018/12 Completed
2. Recommendation:

Recommendation 2. Invest into distilling some case studies from the project

This project piloted a great diversity of interventions. On all levels, but specifically with regards to the adaptation options on the community level some very interesting demonstrations have been set up. However, due to the poor knowledge management aspects of the project no systematic documentation of the investments, processes and performance of the demonstrations are available. It is recommended that this be done esp. with the view that UNDP has prepared a new LDFC project with MITADER, which could benefit from a thorough analysis of previous efforts.

Management Response: [Added: 2018/12/11] [Last Updated: 2020/11/29]

Acknowledged. The country office will take advantage from the upcoming LDCF project PPG to draw up a summary of lessons from this closing project to better inform the design of the new project. It is important to mention that at MTR it was recommended a substantial change of the project management unit recruited by the IP, which was under performing. The CO made necessary due diligences that resulted in changes at project administrative and financial assistant only, remaining the project manager on duty, which had strong support from Micro-finance expert hired by UNDP to ensure delivery on the project results for the remaining years. This also served as a lesson for the other projects that the recruitment of PMU should be done by UNDP to ensure effective control over the performance of each project management unit.

Key Actions:

Key Action Responsible DueDate Status Comments Documents
2.1. Invest some resources from CO regular resources to complement LDCF PPG funding in order to capture lessons learned that will inform the new LDCF and other projects in the subject of CCA, with innovative approach on Micro-finance.
[Added: 2018/12/11] [Last Updated: 2020/04/30]
Env., NRM, CC & Resilience 2019/12 Completed The CO committed 200,000 USD from regular resources to reinforce management capacity to capture lessons learned into the portfolio. History
2.2. Recruitment and performance assessments of the project management unit under UNDP’s responsibilities.
[Added: 2018/12/11] [Last Updated: 2020/04/30]
Env., NRM, CC & Resilience 2019/12 Completed Upon project's closure, the CO has carried out a series of reviews including in the SOPs for recruitment and performance assessment. This was also verified in the 2019 Country Office audit. History
2.3. Ensure that all projects effectively allocate resources on M&E and communication related activities.
[Added: 2018/12/11] [Last Updated: 2020/04/30]
Env., NRM, CC & Resilience and Management Support Unit 2019/12 Completed Projects budget design all consider monitoring budget lines per output as per Prodoc standards. Independent evaluations and communications also have standalone budget lines. CO has reinforced its processe for QA with the reestablishment of the Project Management Support Unit. History
3. Recommendation:

Recommendation 3. At least analyze the micro-finance innovation

As the absolute minimum document and analyze the micro-finance component of the project well. This approach may provide some excellent innovative ideas for strengthening communities’ resilience, however, at this point no independent and detailed review of project inputs and results is available.

Management Response: [Added: 2018/12/11] [Last Updated: 2020/11/29]

Recommendation well noted. Through its diverse portfolio, the CO will ensure that an analysis on the micro-finance innovation under the climate change adaptation umbrella is conducted to better inform future investments in this thematic area.

Key Actions:

Key Action Responsible DueDate Status Comments Documents
3.1. Conduct a quick analysis on the impact of micro-finance initiatives under the adaptation projects in 2 out of the 3 project pilot sites and drawn up key recommendations.
[Added: 2018/12/11] [Last Updated: 2020/04/30]
Env., NRM, CC & Resilience 2019/12 Completed The project conducted assessment to re-evaluate the viable options in each of the 3 project target sites. The successful experience witnessed in Závora (Inhambane Province) proved to be a successful approach to be replicated. History
4. Recommendation:

Recommendation 4. Still undertake the endline surveys

The lack of endline surveys limit the scope to replicate/ scale-up the project. Especially in light of preparing for another large LDCF investment, it should be a worthwhile investment to provide a deeper analysis of this pilot project.

Management Response: [Added: 2018/12/11] [Last Updated: 2020/11/29]

Noted. As indicated above, the CO will take the advantage of the upcoming LDCF PPG and invest its regular resources to complement PPG funding in order to capitalize on the lessons from this current project and better inform the new project and other related initiatives that could benefit from this project best practices and innovative approaches.

Key Actions:

Key Action Responsible DueDate Status Comments Documents
4.1. Invest some resources from CO regular resources to complement LDCF PPG funding to conduct necessary surveys that will provide necessary analysis of the current project to better inform the new LDCF and other upcoming/related projects; ensuring that good practices are appropriately replicated.
[Added: 2018/12/11] [Last Updated: 2020/04/30]
Env., NRM, CC & Resilience 2019/12 Completed The CO invested resources to complement the development of the new GEF-LDCF PPG to ensure that the local consultation process was inclusive in the selected Districts (Moamba, Guijá, Vilankulo, Machanga and Angoche) History
4.2. Ensure that future projects provide necessary budget allocations dedicated for such kind of surveys, which can be costly to be included under PM charges
[Added: 2018/12/11] [Last Updated: 2020/04/30]
Env., NRM, CC & Resilience and Management Support Unit. 2019/12 Completed The CO is developing a PPG Development for GEF LDCF on “Scaling up local adaptation and climate-risk informed planning for resilient livelihoods – Mozambique” and the recommendation has already been considered into its planning. History
5. Recommendation:

4.3 Proposals for future directions underlining main objectives

Before the new LDCF project will be incepted it will be extremely useful to process the actual technical results in some more depth and discuss them with the key stakeholders and partners in a learning event. 

4.4 Best and worst practices in addressing issues relating to relevance, performance and success

Lessons #1: The quality of the project implementation team and effective project management are a critical success factor for project performance 

The best persons for the job should be attracted to lead the PMU; the blend of talent you can attract my render it necessary to hire staff not previously foreseen in the project document. For example, if you hire a PM who has strong project management, M&E and knowledge management capacity you may not need a M&E specialist. Or vice-versa. It is important to review the team at time of project inception. Hired staff should probably undergo a two to three months’ probation time prior to the inception workshop and be assessed for performance at that time. 

Lessons #2: During inception build a joint vision amongst the project team and ensure everyone understand their roles and responsibilities

It may be worthwhile to invest into upfront training in project management related skills development of the project team; orientations on roles and responsibilities are needed for all staff and project partners including the project board. Probably it would be a good decision to increase Project Management cost to include upfront investments into building a good team. While the inception workshop is a very helpful institution in this regard, some additional mandatory and well delivered upfront project management basics should be shared amongst the wider project team. 

Lessons #3: Identifying the most suitable implementation arrangements

Countries take a of pride in NIM projects, and ownership usually is very good. This is an important asset. However, it is also clear that there are numerous barriers and pitfalls including that huge government bureaucracies take away the possibility for nibble and flexible project execution – often so important in demonstration project, and for adaptive management. Further thinking on how the most flexible yet well owned and integrated project delivery can be achieved needs to be part of every project design process.  

4.5 Ratings

As part of the TE, a table with the summary ratings of the project’s results and performance are provided in a TE Ratings & Achievement Summary Table(pg. 53). 

Management Response: [Added: 2020/11/24] [Last Updated: 2020/11/29]

Key Actions:

Latest Evaluations

Contact us

1 UN Plaza
DC1-20th Floor
New York, NY 10017
Tel. +1 646 781 4200
Fax. +1 646 781 4213
erc.support@undp.org