Evaluación final del proyecto Manglar Vivo

Evaluation Plan:: 2020-2024, Cuba
Evaluation Type:: Final Project
Planned End Date:: 09/2020
Completion Date:: 09/2020
Status:: Completed
Management Response:: No
Evaluation Budget(US $):: 20,000

EXECUTIVE SUMMARY

Brief description of the project

The project entitled "Reducing vulnerability to coastal flooding through Ecosystem-based Adaptation (EBA) in the south of Artemisa and Mayabeque provinces", better known as Manglar Vivo (Living Mangrove), aimed to increase the resilience of coastal communities of six municipalities in the south of these two provinces to coastal erosion, flooding and marine intrusion caused by climate change primarily through the recovery and restoration of mangroves. The project was financed by the Adaptation Fund (AF), with an AF budget of USD 6,067,320. It was implemented by UNDP Cuba and executed by the country's Ministry of Science, Technology and Environment (CITMA by its initials in Spanish) and Ministry of Agriculture (MINAG by its initials in Spanish) for a period of 6 years, from 1 October 2014 to 30 September 2020 (the project, initially lasting 5 years, was extended by one year).

Objectives and scope of the evaluation

The objective of this consultancy is to carry out the final evaluation of Manglar Vivo. This evaluation analyses the relevance, design, effectiveness, efficiency, sustainability and impact of the project. It also identifies lessons learned and provides recommendations. The conclusions of the document are based on the review of relevant documentation and interviews with key stakeholders. The evaluation team consists of three evaluators. Only one of them was able to make field visits, and these were limited due to the pandemic caused by COVID-19. The evaluation team has triangulated the data collected to answer the evaluation questions.

Overall Project Rating

The evaluation concludes that Manglar Vivo was relevant, very effective and efficient. Monitoring and evaluation was moderately satisfactory. Implementation by the implementing agency was very satisfactory, while the performance of the executing agency was satisfactory. Sustainability is likely in financial, socio-political, institutional and political terms, and moderately likely from an environmental point of view.

Document	Type	Language	Size	Status	Downloads
1. TORS FINAL EVAL PROJECT MANGLAR VIVO.pdf	tor	English	1225.98 KB	Posted	242
1. INFORME FINAL TE Manglar Vivo - Final Evaluation Report (Sept 9 2020)..pdf	report	English	988.98 KB	Posted	3188
2.- EXECUTIVE SUMMARY.pdf	summary	English	1130.10 KB	Posted	196

Title

Evaluación final del proyecto Manglar Vivo

Atlas Project Number:

69416

Evaluation Plan:

2020-2024, Cuba

Evaluation Type:

Final Project

Status:

Completed

Completion Date:

09/2020

Planned End Date:

09/2020

Management Response:

Yes

UNDP Signature Solution:

1. Resilience
2. Sustainable

Corporate Outcome and Output (UNDP Strategic Plan 2018-2021)

1. Output 1.4.1 Solutions scaled up for sustainable management of natural resources, including sustainable commodities and green and inclusive value chains

SDG Target

13.2 Integrate climate change measures into national policies, strategies and planning
14.2 By 2020, sustainably manage and protect marine and coastal ecosystems to avoid significant adverse impacts, including by strengthening their resilience, and take action for their restoration in order to achieve healthy and productive oceans
15.1 By 2020, ensure the conservation, restoration and sustainable use of terrestrial and inland freshwater ecosystems and their services, in particular forests, wetlands, mountains and drylands, in line with obligations under international agreements

Evaluation Budget(US $):

20,000

Source of Funding:

Fondo de Adaptación al cambio climático

Evaluation Expenditure(US $):

20,000

Joint Programme:

Joint Evaluation:

Evaluation Team members:

Name	Title	Email	Nationality
Jon Garcia	Team Leader	jon.garcia@basstel.com
Joanna Acosta Velazquez	Team member	joanna.acosta@gmail.com	MEXICO
Daysi Vilamajo	National Evaluator	dvilamajo@ceniai.inf.cu	CUBA

GEF Evaluation:

Yes

GEF Project Title:	Reduction of vulnerability to coastal flooding through ecosystem-based adaptation in the south of Artemisa and Mayabeque provinces
Evaluation Type:	Terminal Evaluation
Focal Area:	Climate Change
Project Type:	EA
GEF Phase:	GEF-5
GEF Project ID:
PIMS Number:	PIMS no: 5090

Key Stakeholders:

Countries:

CUBA

Comments:

Informe final concluido y subido a ERC

Lessons
1.	Effective actions of Manglar Vivo: From the point of view of relevance, sustainability and impact, it is essential that the project is aligned with international, national, provincial and municipal priorities. In this sense, it is key to articulate the project with strategic, long-term national policies and plans, with resources and visibility, such as the economic and social development programme and the national adaptation plan (Tarea Vida). From the perspective of relevance, effectiveness, sustainability and impact, it is important to try to identify all relevant actors in the design, but it is fundamental to have an inclusive, open and collaborative approach during implementation, integrating those strategic actors that were not identified in the design. From the angle of effectiveness, sustainability and impact, it is essential to promote the connectivity of coastal ecosystems, working simultaneously on mangrove, swamp and bordering forests, and combining the elimination of IAS and the planting of native species with hydrological restoration. For relevance, effectiveness and efficiency, it is key to coordinate closely with provincial and municipal governments, as well as with all complementary projects present in the intervention area and the surrounding ecosystems, identifying and exploiting synergies, including joint activities. For effectiveness, efficiency, sustainability and impact, it is essential to promote multi- and interdisciplinary teams, with the active presence of research institutes and academia, and favour a collaborative attitude and permanent exchange of knowledge. In this respect, it is important to implement an approach that integrates theory with practice, with a high degree of ownership by local governments and communities. From the point of view of effectiveness, sustainability and impact, it is important to develop an identity manual and to undertake communication in a professional manner, with the help of experts, for example, the Design Institute and the Faculty of Communication of the University of Havana, respectively. From the perspective of effectiveness, it is necessary to adjust the data sheets of the agroforestry companies to ensure efficient forestry work and adequate remuneration, and thus a sufficient and motivated workforce. From the perspective of efficiency, a solid technical basis and a fluid dialogue between the PMU, the executing agency (AMA) and the implementing agency (UNDP) are essential for the efficient implementation of an international project. From the point of view of relevance, effectiveness and efficiency, it is essential to ensure the commitment of national institutions in order to be able to face difficulties as they arise (such as the provision of equipment or more manpower when the inputs provided with international funding have not arrived). From the point of view of sustainability and impact, cost-benefit analyses are an interesting analytical exercise and a useful tool for generating ownership and promoting sustainability, replication and scaling up. From the point of view of efficiency, sustainability and impact, the restoration of ecosystems is cost-effective: for every CUP invested in the restoration of coastal ecosystems, a gain of more than 6.8 CUP was obtained. EbA is also more cost-effective than adaptation through the construction of grey infrastructure. That said, EbA and adaptation with grey infrastructure are complementary and sometimes the latter is irreplaceable. From the point of view of sustainability, insurance can be important in providing continuity to the results of a project if a disaster occurs. From the point of view of effectiveness, sustainability and impact, it is essential to define and adopt measures to disseminate public goods (particularly the knowledge created), including training, demonstration sites, publications of methodological guides, and knowledge management systems that link the generators of information and knowledge and the users and propagators of that information For a greater impact, it is strategic to apply during the project its approach in other areas of the country, with similar and different ecosystems to the project, and with information and data on observed and projected climate variability and change, in order to finetune the approach. For greater impact, it is important to use the lessons learned during the implementation of one project in the design of other projects, of different scale.
2.	Areas of opportunity: It is important to avoid confusion in the project document and to follow international guidelines and good practice (e.g. by defining fewer outcomes than outputs). From the point of view of relevance, effectiveness and impact, it is important to have a strong climate information component. From the angle of effectiveness, sustainability and impact, it is key to strengthen ecological connectivity, working simultaneously on coastal, terrestrial (inland watershed) and marine ecosystems, considering the management effects of upstream and downstream areas of the intervention area. From the angle of effectiveness, sustainability and impact, it is essential to consider the built environment and human settlements, and to make strategic interventions in hard infrastructure, even when it comes to EbA projects, since they are complementary and not exclusive measures. Significantly reducing the vulnerability to climate change of some populations may require in some cases hard interventions (in some cases EbA may be insufficient to reduce vulnerability to acceptable levels). From the perspective of relevance, sustainability and impact, projects that involve protection and/or restoration of ecosystems must directly promote, at a certain scale and strategically (with a value chain vision), alternative livelihoods to those that result in the degradation of these ecosystems. The improvement in the provision of ecosystem goods and services as a result of protection and/or restoration actions is mainly manifested in the medium and long term. From the perspective of relevance, sustainability, and impact, it is essential to involve the productive actors that degrade ecosystems in a less direct way. In the case of coastal ecosystems, it is not enough to involve those who deforest, but also those who negatively affect the health of these ecosystems due to excessive water extraction (farmers) or fishing methods that degrade marine ecosystems. From the point of view of efficiency, when defining the duration of international projects, both technical and administrative issues must be taken into account, in particular the volume and speed of the contracting and procurement processes. In Cuba, attention must be paid to the need to import a large volume of goods and the long time this requires, due to the US economic, financial and commercial blockade. In this sense, projects in Cuba may require more time than in other countries. From an efficiency point of view, it is necessary to strengthen the capacity of importing companies on the specificities of the equipment to be imported and to strengthen the transition processes from the beginning. From the management perspective, all targets must be feasible and realistic within the time frame of a project From a management point of view, it is essential that the results framework allows the achievement of the objective and the outcomes to be measured. In this regard, it is essential to define SMART indicators of vulnerability. Defining robust indicators of ecosystem health is also essential in EbA projects. From a management perspective, risk identification and analysis must be realistic, recognizing the probability and potential impact of each risk. From the perspective of effectiveness, it is important to ensure sufficient labour from the outset, promoting adequate wages. From the perspective of effectiveness, efficiency, sustainability and impact, in EbA projects it is very important to have a robust M&E system from the beginning to monitor and evaluate the impacts of restoration actions on the ecosystems and the vulnerability of the population in a concrete and holistic way, considering the different ecosystems. The report has to be clear and concise and respond directly to all the elements of the indicator. The M&E system must be an instrument that supports planning and decision making during the course of the project. From an impact perspective, at the international level, it is important to establish systems to identify, systematise and disseminate lessons learned during project implementation, for example through South-South forums in the Caribbean.
Findings
1.	Main findings In terms of relevance[1], Manglar Vivo is consistent with the United Nations conventions on climate change, wetlands, and biodiversity, the international guidelines on EbA, and the objective, results, and outputs of the AF. The project is also in line with UNDP priorities at global, regional and national levels and Cuba's United Nations Development Assistance Framework 2014-2018. Furthermore, the project is in tune with national strategies and priorities in the areas of economic and social development, climate change and environment, and responds to the problems and needs of the provinces and municipalities where it focuses. All stakeholders were actively involved in the design and implementation of the project. The project design[2] formulated a fairly clear and well-integrated structure, with a few exceptions. However, there are important gaps in relation to climate information; the connectivity of coastal ecosystems with terrestrial and marine ecosystems; the built environment; and the promotion of alternative livelihoods and the modification of practices of productive sectors other than forestry. These limitations are relatively understandable, given the relatively limited financial resources available, the time frame and the pilot nature of this project. The targets are feasible and realistic within the budget, but not within the timeframe of the project. The results framework included in the project document does not allow the achievement of the goal or the key intermediate result to be measured. Overall, 80% of the indicators in the results framework are not specific and/or consistent. The identification of risks is moderately adequate, but their analysis is inappropriate. The project document does not clearly integrate lessons learned from other projects. The project document does a good job at identifying and analysing complementary international projects and identifying synergies. During its implementation, the project had a high level of coordination with other international cooperation interventions and with work and research initiatives carried out by Cuban institutions. In terms of effectiveness[3], at the end of the project, all the final targets of the results framework in the project document have been met, and 8 or 50% have been exceeded. All the FA targets have also been met, and 5 or 25% have been exceeded. This analysis is based on important assumptions. Section 3.6 examines impacts in terms of vulnerability and ecosystem health based on available information. To achieve these results, Manglar Vivo had to overcome some significant challenges. The risk mitigation strategies identified in the project document were adequate, although the strategy with regard to the import of goods was insufficient. During the implementation of the project, the actions to mitigate the risks that arose were appropriate. The project showed a high capacity for adaptive management. From the point of view of efficiency[4], the project has spent the budget foreseen in the project document. Financial performance improved over time. There are important differences in the financial implementation by component, as the cost of goods and services was not accurate in the design. Project management costs are and are expected to be slightly lower than planned. Manglar Vivo was able to mobilize 382 percent of the co-financing committed in the project document. The co-financing, all in kind, helped mitigate the impact of the delay in importing some goods and exceed some of the targets. The project produced financial reports and audits with the required regularity, with room for improvement in terms of their quality. The cost-effectiveness of Manglar Vivo was probably intermediate. Its management costs (6.5% of total costs) are below the FA ceiling (9.5%), but above the GEF and GCF ceilings for projects of this size (5%). Available information indicates that ecosystem restoration was cost-effective and that EbA is more cost-effective than adaptation through the construction of grey infrastructure. An appropriate M&E plan is included in the project document. As indicated, the results framework has major shortcomings. During implementation, especially from the mid-term evaluation, the project strengthened the M&E system. Reporting has been appropriate in terms of quantity, but its quality is average: often reporting does not respond completely, directly or clearly to the system of indicators. The project established effective partnerships with relevant actors. The Steering Committee, the Project Management Unit (PMU), the Environmental Agency (AMA by its initials in Spanish) and UNDP played their roles well and had a fluid dialogue. Despite all this, the project was extended by one year, at no cost. The sustainability6 strategy is sound, although more attention should have been paid to other connected ecosystems, the integration of sustainability into productive sectors other than forestry, and the promotion of alternative livelihoods. From the point of view of the policy, regulatory and institutional framework, the necessary conditions have been established to make the project's results sustainable in the short, medium and long term. From the financial point of view, the provinces of Artemisa and Mayabeque and the project's municipalities have already secured substantial resources to give continuity to the results of Manglar Vivo, especially those related to ecosystem restoration. In addition, the forests are insured. Furthermore, there is progress in mobilizing international resources. The project has provided equipment that will facilitate the continuity of the project's results. From a socio-cultural perspective, the project has strengthened the awareness and technical capacity of almost all relevant actors. There is also a strong political will to give continuity to the project's results and technical capacities and knowledge transfer mechanisms to do so. From an environmental perspective, the project results are subject to significant risks, including the occurrence of major extreme climate events; the expansion of Invasive Alien Species (IAS); and the degradation of connected ecosystems. In terms of impact[5], in the short term, pressures on ecosystems have been considerably reduced, but are not negligible. These pressures are likely to be limited in the medium to long term. The economic blockade of the country and the COVID-19 do not help to reduce these pressures. There is no comprehensive information on the health of coastal ecosystems. Available information suggests an improvement. In addition, available information indicates an improvement in the health of marine and terrestrial ecosystems. The health of these ecosystems is expected to improve over time. There is little scientific evidence on the impact of the project in reducing vulnerability to coastal flooding. It is reasonable to think that the restoration of coastal ecosystems, the cleaning of ditches and channels, and the strengthening of planning, management and response capacities have reduced the vulnerability of target populations to these aspects. There is anecdotal evidence in this regard. Those who have benefited most are the populations immediately on the coast. A AMA study will assess vulnerability reduction more rigorously in 2021. Manglar Vivo contributed to the Sustainable Development Goals (SDGs), had socio-economic benefits, respected environmental and social safeguards, and promoted gender equity and the inclusion of youth. The evaluation team has identified only positive unexpected outcomes. The project provided public goods in the form of new knowledge, approaches and technologies and took steps to disseminate these public goods. There are excellent prospects in terms of replication and/or scaling up. The results of the project have informed the development of policies and strategies. During the project, the project approach was applied in other areas of the country. There are prospects for replication in the municipalities and, to a greater extent, the project provinces, and other provinces of the country. In addition, the lessons learned during the implementation of this project are being used in the design of other projects to be financed with international resources, of different scales. At the international level, there has been no concrete progress in replicating the lessons learned during the implementation of the project. [1] For details, see section 3.1 [2] For details, see section 3.2 [3] For details, see section 3.3. [4] For details, see section 3.4. 6 For details, see section 3.5. [5] For details see section 3.6.

Recommendations
1	Recomendación 1: Con base en los resultados del proyecto, la UMP y AMA deberían elaborar un documento que describa los aspectos a tener en cuenta en la elaboración de un plan de manejo integral de las cuencas costeras que drenan a los manglares del sur de Artemisa y Mayabeque, con el fin de promover el buen manejo del caudal ecológico en el corto, medio y largo plazo y contribuir de esta forma a que las acciones de restauración permanezcan en el tiempo. La AMA debería presentar este documento al Consejo Nacional de Cuencas Hidrográficas (CNCH), quien en el país es responsable del manejo de cuencas y la elaboración de sus planes de manejo.
2	Recomendación 2. La UMP, AMA y el PNUD deberían organizar un taller tan pronto la situación del COVID-19 lo permita para identificar y caracterizar las lecciones aprendidas durante la implementación del proyecto. Este ejercicio debería tener en cuenta las lecciones aprendidas recogidas en los PPR y esta evaluación final, pero deberá tener la flexibilidad suficiente para integrar las lecciones identificadas por todos los actores relevantes. Después del taller, la UMP, AMA y PNUD deberían consolidar las lecciones, integrarlas en un documento y diseminarlas, incluyendo su integración en el sistema de gestión del conocimiento adoptado.
3	Recomendación 3. La AMA, el PNUD y el FA deberían usar esas lecciones en el desarrollo y la implementación de nuevos proyectos. A ese respecto, AMA debería continuar con sus esfuerzos para iniciativas nacionales e internacionales, mientras que PNUD y FA deberían reforzarlos, por ejemplo, organizando webinars que reúnan varios proyectos en el Caribe.
4	Recomendación 4. La AMA debería promover que el estudio PVR planeado para la zona de intervención en 2021 realmente se realice, teniendo en cuenta proyecciones climáticas futuras. La AMA debería cerciorarse de que el estudio tenga en cuenta los términos de Manglar Vivo. En ese sentido, el estudio debería responder las preguntas sobre salud de los ecosistemas y la vulnerabilidad de los beneficiarios directos e indirectos de Manglar Vivo que esta evaluación final no ha podido responder cabalmente por su alcance en términos de equipo y de días y por la imposibilidad de realizar trabajo de campo debido al COVID-19. Los resultados del PVR deberían considerarse como una evaluación de los resultados de Manglar Vivo, con más tiempo transcurrido desde su finalización. La AMA debería asegurar que esto es explícito, por ejemplo, con un anexo dedicado específicamente a ello. Este anexo debería incluir lecciones aprendidas. La AMA debería procurar que esas lecciones se tienen en cuenta en el diseño de nuevos proyectos. La AMA también debería compartir los resultados de la PVR, las conclusiones en términos de los resultados de Manglar Vivo y las lecciones aprendidas con PNUD Cuba y Panamá, de cara a que estos puedan incorporar las lecciones aprendidas en el diseño e implementación de nuevos proyectos.

Evaluación final del proyecto Manglar Vivo

EXECUTIVE SUMMARY

Share

Evaluations by Unit

Latest Evaluations

Evaluation Snap Shot

Contact us