Final Evaluation Japan Caribbean Climate Change Partnership

Report Cover Image
Evaluation Plan:
2018-2021, RBLAC
Evaluation Type:
Final Project
Planned End Date:
11/2019
Completion Date:
03/2020
Status:
Completed
Management Response:
Yes
Evaluation Budget(US $):
50,000

Share

Document Type Language Size Status Downloads
Download document TOR JCCCP Final Evaluation - IC-Lead Evaluator.pdf tor English 309.44 KB Posted 38
Download document TOR JCCCP Final Evaluation - Regional Evaluator.pdf tor English 273.41 KB Posted 31
Download document Final Evaluation_Japan Caribbean Climate Change Partnership (J-CCCP).pdf report English 3074.51 KB Posted 66
Download document JCCCP TE Management Response.docx related-document English 38.16 KB Posted 26
Title Final Evaluation Japan Caribbean Climate Change Partnership
Atlas Project Number: 00088096 JCCCP
Evaluation Plan: 2018-2021, RBLAC
Evaluation Type: Final Project
Status: Completed
Completion Date: 03/2020
Planned End Date: 11/2019
Management Response: Yes
Corporate Outcome and Output (UNDP Strategic Plan 2018-2021)
  • 1. Output 2.1.1 Low emission and climate resilient objectives addressed in national, sub-national and sectoral development plans and policies to promote economic diversification and green growth
Evaluation Budget(US $): 50,000
Source of Funding: Japan
Evaluation Expenditure(US $): 44,000
Joint Programme: No
Joint Evaluation: No
Evaluation Team members:
Name Title Email Nationality
a. John Owanga Team Leader jogwang1@gmail.com
b. Saudia Rahat Regional Evaluator saudia_rahat@yahoo.com/saudia.rahat@gmail.com
GEF Evaluation: No
Key Stakeholders: Beneficary institutions in Belize, Dominica, Grenada, Guyana, Jamaica, Saint Lucia, St Vincent and the Grenadines, and Suriname; Local Authorities, Civil Society Organisations and other public and private organisations in all the beneficary countries
Countries: BARBADOS
Lessons
1.

Lesson #1: Existence of technical capacities, political buy-in and ownership of the process by Government Ministries / Departments / Agencies and beneficiaries is critical for successful implementation and sustainability of project results. Ownership of J-CCCP driven intervention and their successful implementation at community levels results from the project ensuring that local communities understand climate change, its catastrophic effects, and the need to do something about it, as well as what can actually be done within their local context to promote buy-in and sustainability. 


2.

Lesson #2: Well-coordinated and sustainable partnerships within government and across borders are important elements for the success of regional and national level initiatives. J-CCCP successfully built and maintained a number of local and international partnerships, and were a contributing factor to project successes.


3.

Lesson #3: It is important to maintain flexibility in planning and the implementation of complex, multi-sectoral projects. This should be supported by adaptive management practices. Capacity of project proponents regarding technical quality and reporting on pilot projects was mitigated through hiring of technical experts in the specific focal areas to assist with the proposal development and ultimately build capacity in that area.


4.

Lesson #4: It is important that the design of a project ensures linkages to national policies, priorities and programmes of each of the participating countries, as well as their collective (regional) agendas. This ensures mainstreamed implementation, easily fitting into and contributing to the established national priorities and their institutionalized implementing structures, as well as ultimate sustainability of interventions and benefits. It is also important to build on what is already existing and not re-invent the wheel.


5.

Lesson #5: Conducting wide dissemination and training maximizes project benefits beyond immediate beneficiaries. In the same vein, it is important to incorporate participatory learning activities in the project e.g. Quiz competition and environmental fair.


6.

Lesson #6: Direct Implementation Modality (DIM) strategy using contractual arrangements with Implementing Partners is effective. The project management structure offered by the UNDP is transparent and allows for impartial distribution of benefits. All stakeholders can be trained under this system.


Findings
1.

1: The overall performance rating of J-CCCP in relation to the applied evaluation criteria is satisfactory


2.

2: The development of the NAMAs and NAPs is a key and scalable component of the project for advancing climate resilience from policy and planning – to implementation.


3.

3: Investments in the community pilot projects are contributing towards the advancement of technologies that are low-emission and help advance climate risk management.


4.

4: To a large extent, the J-CCCP has demonstrated that through deliberate promotion, supporting and working with stakeholders at regional, national and community levels, it is possible to inculcate and mainstream CC into national strategies, policies and community level actions.


5.

5: The J-CCCP strengthened knowledge networks. This and wider and earlier implementation of South-South and North-South Cooperation could have been better leveraged to maximize the benefits of knowledge sharing in other viable sectors for advancing the overall objective of the project.


6.

6: The sustainability of the project’s emerging benefits will largely depend on the willingness of stakeholders to adopt and out-scale interventions, and continue normal use beyond the duration of the project, and the long-term political and financial commitment of policy-makers to provide enabling investment environments for scaling up of successful adaptation and mitigation measures.


7.

7: The J-CCCP achievements have refocused the “thinking” and practice of leaders at all levels, policy makers, supporting development partners, economic investors and communities in climate smart production use of energy, water, other related resources (e.g land) and infrastructures. This lays a strong foundation for institutionalized and locally led country and region wide central focus on integrating medium to long-term planning for adaptation to and mitigating climate change. It should spur countries to mandate low-emission risk- resilient development in medium to long-term planning that supports increased investment in, and the application of Climate Change Resilient economic growth, which will contribute to expected impact in which targeted countries achieve inclusive low-emission risk-resilient sustainable development.


8.

8: To a large extent, the J-CCCP has demonstrated that through deliberate promotion, supporting and working with stakeholders at regional, national and community levels, it is possible to inculcate and mainstream into CC resilience; the strategies, policies and community level actions and the wider national economic planning, those proven initiatives and innovations for mitigation and adaptation to extreme climate variabilities. The J-CCCP has also put in place building blocks for sustained capacities across sectors in the eight countries to provide for, invest in, plan for and implement climate change requirements across all public and private development investments. It was noted that many of the community projects will positively impact the capability of beneficiaries in CC resilient income generation, and as a result of the experience gained in continued application of practices learnt in the pilots, financial and economic sustainability will be strengthened.


Recommendations
1

Full adoption and future benefits to be realised.

The national governments will need to put in more deliberate and well publicised work on creating market incentives and policies to attract more investment in climate change resilient resource use and economic production technologies, especially in relation to water, land and energy resources.  Governments themselves will need to put in more public resources on further policy and institutional stimuli to make optimal use of what J-CCCP has put in place, while removing any inequalities in all peoples’ vulnerabilities to and capacities to cope with CC and DDR.

2

Enhancing South-South and North-South knowledge exchange:

There was better South-South than North-South cooperation, especially in relation to community projects. This limited the scope of knowledge and information exchange that took place especially given the multi-sectoral emphasis supported by the J-CCCP.  Involvement of Japanese technologies could have been scaled up from project inception, for example, the Japanese private sector could have been invited to install and demonstrate some of their technologies in some of the beneficiary communities. That is, a matchmaking between available technologies in Japan and current problems in the Caribbean (especially with climate resilient infrastructure) could have been explored for implementation of unique and technologically advanced pilot demonstrations.

3

Building on the successes registered, and the momentum created:

The successes realised, and momentum created should not be permitted to dissipate. The national governments, and if UNDP has another programme to provide support, should further strengthen the evidence base created to date to make the case for greater levels of investments in adaptation, and to develop national understanding of which policies and strategies have the optimum feasibility, technically, socially, environmentally, and can be expected to provide overall net benefits to economic growth in different sectors. The J-CCCP PMU through their monitoring and project analysis as well as knowledge management work have accumulated rich data and information to enable continuous economic and other analyses of the proven technologies and practices. This data and information need to be put into use by the national Governments and at Regional level through an appropriate existing regional umbrella body, and with continued support of UNDP. The analysed data needs to be packaged in a manner attractive to the potential investors and financiers, both public and private, and disseminated widely. It was already noted that involvement of the private sector in the project could have been improved. It is critical that the private sector as the main consumer of energy and generator of goods, services and employment world over is attracted to switch to more renewable energy.  This for instance can be by making cost of renewable energy cheaper than energy from fossil fuel.   

4

Enhancing GE and HR considerations in a project (i) Countries could have benefited from orientation and awareness building as it relates to human rights-based approach to programming and gender mainstreaming to better include activities/strategies to address same in the pilot projects (ii) reporting templates/guidelines for pilots/projects need to reinforce the importance of reporting on how GE and HR issues were addressed. That is, data captured should go beyond simple disaggregation of beneficiary information by sex and to include other parameters such as age, ethnicity, religion etc. Details on strategies and challenges towards mainstreaming GE and HR considerations need to be documented to enhance mainstreaming of GE and HR on an on-going basis in project implementation.

5

Enhancing project planning, participation and management of resources: (i) An initial assessment of the capacity of the community groups and a strategy to address weaknesses should have been part of the preparatory processes for pilot projects, this coupled with a clear criteria for membership eligibility, and a cap on the number of community-based projects for each country, would optimize productive participation of communities in their own projects, and speed up implementation; (ii) Hiring of an independent quantity surveyor might prove to be valuable for projects with larger construction components. He/she would enhance the budgeting process and elucidate the comprehensive list of items needed to inform better procurement planning; (iii) Contracts with external companies (e.g. construction companies) should have smaller units of deliverables to allow more flexibility during project implementation, especially when changes are required, and to provide contractors with more frequent, albeit smaller payments. The latter is important for the contractors to be able to cover expenses incurred (iv) There is need for an initial assessment of the institutional framework and plausible scope and quantity of community-based projects that can be sustainably supported by the PMU; (v) Partnering with contributing stakeholders and seeking in-kind contribution is highly recommended. In J-CCCP, there was partnership with the UNFCCC for support in training related to the NAMAs with in-kind contributions, and IICA supported the apiculture project in Saint Lucia. The 4H was instrumental in implementing community projects in Jamaica. With this, the intensity of work was stretched across all supporting parties.

6

Ensuring timely completion of project as designed: There should be year zero for preparation work for the entire project similar to J-CCCP, including identifying the scope of work, preparing the implementing teams and preparing the necessary procurement documents for better planning. A more strategic intervention during year zero would be a deliberate process of intensive sensitisation and orientation of the targeted population on for instance climate change and the need for adaptation and mitigation, as well as possible mechanisms for achieving these.

1. Recommendation:

Full adoption and future benefits to be realised.

The national governments will need to put in more deliberate and well publicised work on creating market incentives and policies to attract more investment in climate change resilient resource use and economic production technologies, especially in relation to water, land and energy resources.  Governments themselves will need to put in more public resources on further policy and institutional stimuli to make optimal use of what J-CCCP has put in place, while removing any inequalities in all peoples’ vulnerabilities to and capacities to cope with CC and DDR.

Management Response: [Added: 2020/06/10]

Countries in the region face resource challenges across several sectors. However, the multiple co-benefits of resilience and adaptation investments in protecting and enhancing other development investments make the case for their prioritisation in public expenditure as well as integration in all activities and funding sources. UNDP’s programming will continue to advocate and implement this integrative approach, with efforts focusing on the reducing inequalities and supporting the most vulnerable.

Key Actions:

2. Recommendation:

Enhancing South-South and North-South knowledge exchange:

There was better South-South than North-South cooperation, especially in relation to community projects. This limited the scope of knowledge and information exchange that took place especially given the multi-sectoral emphasis supported by the J-CCCP.  Involvement of Japanese technologies could have been scaled up from project inception, for example, the Japanese private sector could have been invited to install and demonstrate some of their technologies in some of the beneficiary communities. That is, a matchmaking between available technologies in Japan and current problems in the Caribbean (especially with climate resilient infrastructure) could have been explored for implementation of unique and technologically advanced pilot demonstrations.

Management Response: [Added: 2020/06/10]

While it can be rationalised and agreed that the North-South Cooperation could have commenced earlier in the project, it should also be understood that it was always the intention of the project to be driven by the community needs and expressed goals.  As such, the project sought to understand these needs through vulnerability assessments under Outcome 2 of the project where community projects were submitted.  The aim was to support the intended impacts outlined in pilot project submission with targeted North-South cooperation where directed or targeted matchmaking opportunities could be created between technologies in Japan that related specifically to the problems outlined in the pilot project.  This process took more time than anticipated as focus was on the design and development of the pilots for ensuring alignment to project outputs as well as beneficiary considerations relating to gender and other vulnerabilities.  However, as soon as pilots were established the study tour to Japan was organised. This tour included both implementers from the pilot projects as well as technical officers in government related to the subject matter so that adoption of practices learnt could be more wide-reaching. Based on the reports from attendees, a knowledge sharing tour was organised with the Japanese private sector with expertise beneficial to several projects including nature farming and seed saving techniques for sustainable agriculture and agro-ecosystem management. The technologies were presented at the national level in Dominica and then at a regional workshop for broader uptake. 

 

South-South cooperation was also established in a targeted way among in-country implementers. A technical focal point group was created as a forum for learning and best practices where members from each country queried and guided on implementation processes and gave technical guidance for quality assurance of interventions.

 

The Technical Specialists of the PMU also lent to the creation of this South-South support and cooperation. For instance, one Specialist led a study tour with project implementers from St. Vincent to Grenada to provide direct support and guidance for the unique bio-digester project. The project had other examples of this throughout.

Key Actions:

3. Recommendation:

Building on the successes registered, and the momentum created:

The successes realised, and momentum created should not be permitted to dissipate. The national governments, and if UNDP has another programme to provide support, should further strengthen the evidence base created to date to make the case for greater levels of investments in adaptation, and to develop national understanding of which policies and strategies have the optimum feasibility, technically, socially, environmentally, and can be expected to provide overall net benefits to economic growth in different sectors. The J-CCCP PMU through their monitoring and project analysis as well as knowledge management work have accumulated rich data and information to enable continuous economic and other analyses of the proven technologies and practices. This data and information need to be put into use by the national Governments and at Regional level through an appropriate existing regional umbrella body, and with continued support of UNDP. The analysed data needs to be packaged in a manner attractive to the potential investors and financiers, both public and private, and disseminated widely. It was already noted that involvement of the private sector in the project could have been improved. It is critical that the private sector as the main consumer of energy and generator of goods, services and employment world over is attracted to switch to more renewable energy.  This for instance can be by making cost of renewable energy cheaper than energy from fossil fuel.   

Management Response: [Added: 2020/06/10]

Much of the country data gathered in the project which can help propel activities under the NAPs and NAMAs are outlined in these documents and do indeed provide much pertinent information on how best to move forward these processes – either nationally or through the new project.  The project also collected post-evaluation data on training as well as lessons learnt and some of these data have been used to inform future training.

Key Actions:

Key Action Responsible DueDate Status Comments Documents
Creation of an MCO central document repository that teams and future staff can reference lessons learnt from previous projects
[Added: 2020/06/10]
Operations team 2020/05 Completed The EnGenDER project is actively building on the policies articulated (NAPs, NAMAs) to implement sustainable, inclusive and gender-responsive approaches across vulnerable sectors. Coupled with targeted sector-level gender-based analyses, the project will design and support investments in resilience and adaptation, building on JCCCP’s results.
4. Recommendation:

Enhancing GE and HR considerations in a project (i) Countries could have benefited from orientation and awareness building as it relates to human rights-based approach to programming and gender mainstreaming to better include activities/strategies to address same in the pilot projects (ii) reporting templates/guidelines for pilots/projects need to reinforce the importance of reporting on how GE and HR issues were addressed. That is, data captured should go beyond simple disaggregation of beneficiary information by sex and to include other parameters such as age, ethnicity, religion etc. Details on strategies and challenges towards mainstreaming GE and HR considerations need to be documented to enhance mainstreaming of GE and HR on an on-going basis in project implementation.

Management Response: [Added: 2020/06/10]

It is agreed that this was a weakness from the onset, however, subsequent to having a Monitoring and Evaluation Analyst on board, the project closely engaged the UNDP MCO Gender Focal Point.  This engagement led to a more gender-focused approach in many of the pilot projects where the PMU endeavoured to incorporate basic gender analysis (e.g. Dominica Morne Prosper pilot project) and determined how best to engage the community based on gender roles. There was also gender training undertaken for the PMU team and pilot project proponents in St. Lucia. While the project was still unable to go beyond sex-disaggregation and basic gender analysis, this still allowed for the project to capture specific gaps which can be addressed in future interventions.

Additionally, this weakness is indeed acknowledged and led specifically to the development of the EnGenDER project, specifically targeting gender-responsiveness and inclusion in addressing climate change and disaster risk. The project’s results framework, monitoring and reporting templates all reflect this approach. Outputs are designed to analyse and address gender-based vulnerabilities and impacts.

Key Actions:

Key Action Responsible DueDate Status Comments Documents
Project design will involve the gender and M&E functions from the MCO to strengthen the quality of the analysis, results and indicators. These functions will also support the QA and LPAC processes. Project development will also include gender analysis as far as possible.
[Added: 2020/06/10]
Programme 2020/06 Completed This mechanism has been in operation and will function on an ongoing basis. This has been seen in the recently developed GEF 6 portfolio, EnGenDER and Spotlight multi-country projects.
5. Recommendation:

Enhancing project planning, participation and management of resources: (i) An initial assessment of the capacity of the community groups and a strategy to address weaknesses should have been part of the preparatory processes for pilot projects, this coupled with a clear criteria for membership eligibility, and a cap on the number of community-based projects for each country, would optimize productive participation of communities in their own projects, and speed up implementation; (ii) Hiring of an independent quantity surveyor might prove to be valuable for projects with larger construction components. He/she would enhance the budgeting process and elucidate the comprehensive list of items needed to inform better procurement planning; (iii) Contracts with external companies (e.g. construction companies) should have smaller units of deliverables to allow more flexibility during project implementation, especially when changes are required, and to provide contractors with more frequent, albeit smaller payments. The latter is important for the contractors to be able to cover expenses incurred (iv) There is need for an initial assessment of the institutional framework and plausible scope and quantity of community-based projects that can be sustainably supported by the PMU; (v) Partnering with contributing stakeholders and seeking in-kind contribution is highly recommended. In J-CCCP, there was partnership with the UNFCCC for support in training related to the NAMAs with in-kind contributions, and IICA supported the apiculture project in Saint Lucia. The 4H was instrumental in implementing community projects in Jamaica. With this, the intensity of work was stretched across all supporting parties.

Management Response: [Added: 2020/06/10]

(i) GEF SGP has a history of working with and supporting capacity building of grantees, as did the UNDP ICCAS project in Grenada. The PMU therefore collaborated with SGP in the development of pilot project application.  As part of implementing lessons learnt from the above, capacity assessment criteria were integrated within the grant application form. This allowed for the support of weaknesses seen in both the application and the proposed implementing agency. In many cases the PMU insisted on establishment of partnerships with other entities (e.g. Guyana agreement and partnering with the National Water Authority) that could support the specific weaknesses noted. In some cases, it was noted that funding would be needed to be transferred to an assessed government department who would provide guidance and support to the community organisation responsible for implementation (e.g. St. Vincent – Mayreau Provision of rainwater harvesting systems project).

The form was generally quite robust as the intention was to ensure data for baseline information were captured; beneficiary information (relating to targeting of the most vulnerable); monitoring component for quality assurance.  This assisted in ensuring that community groups would have the capacity to undertake and achieve the project goals. The PMU provided training in this process through 2 webinars at the onset as well as one-on-one counsel during implementation.

It can be agreed that there indeed needed to be a cap on the number of pilot projects accepted and implemented.  This was noted as a lesson learnt and shared with future interventions such as EnGenDER.

(ii) Part of the process for ensuring national ownership of the pilots and increasing resource efficiency was engaging the competent technical agencies in country in providing technical specifications and quantities and quality assurance for construction works in particular. Where it was found to be necessary (e.g. due to alignment to timelines of the project) independent surveyors were used (e.g. Dominica Morne Rachette pilot project).

(iii) This concern relating to cash flow of smaller contractors has been observed, particularly towards the end of the activities and where issues at a single site would inhibit payment for a large batch of work. It was highlighted in the lessons learnt from the project. This adjustment will be made in future procurement processes.

(iv) Indeed, in hindsight, the approach to the pilot projects could have had stricter definitions to ensure a more manageable quantity of activities for the PMU. This experience has led specifically to an approach in EnGenDER for larger (in value) and fewer sector-led activities.

(v) UNDP continues to believe that leveraging the expertise and capacities of other partners will be the most effective approach to implementing integrated sustainable solutions. The private sector is one which is currently under-engaged in programming, which will be actively increased as future programming evolves.

Key Actions:

Key Action Responsible DueDate Status Comments Documents
Incorporate private sector engagement as key part of standard TOR for stakeholder analysis and action plans in project design.
[Added: 2020/06/10]
Programme 2020/09 Not Initiated COVID-19 response is currently directly engaging and supporting recovery of micro, small and medium-sized enterprises (MSMEs).
6. Recommendation:

Ensuring timely completion of project as designed: There should be year zero for preparation work for the entire project similar to J-CCCP, including identifying the scope of work, preparing the implementing teams and preparing the necessary procurement documents for better planning. A more strategic intervention during year zero would be a deliberate process of intensive sensitisation and orientation of the targeted population on for instance climate change and the need for adaptation and mitigation, as well as possible mechanisms for achieving these.

Management Response: [Added: 2020/06/10]

The year zero concept for effective detailed scoping of the project environment, stakeholder and beneficiary analysis and engagement and planning for the implementation phase is indeed one which continues to be observed as a need across all projects. Having a more realistic expectation of what can be achieved in period which is primarily taken with staff recruitment, re-familiarising stakeholders and establishing governance and coordination mechanisms would help to relieve the pressures of attempting to deliver with partial information. This needs to be a broader development community discussion, as well as flexibility with donors for more flexible and adaptive management processes, which is not applied across the board.

Key Actions:

Latest Evaluations

Contact us

1 UN Plaza
DC1-20th Floor
New York, NY 10017
Tel. +1 646 781 4200
Fax. +1 646 781 4213
erc.support@undp.org