Final Evaluation: Cambodia Climate Change Alliance Phase 2 Project

Report Cover Image
Evaluation Plan:
2019-2023, Cambodia
Evaluation Type:
Final Project
Planned End Date:
07/2019
Completion Date:
05/2019
Status:
Completed
Management Response:
Yes
Evaluation Budget(US $):
30,000

Share

Document Type Language Size Status Downloads
Download document Signed_ToR_CCCA_FinalEvaluation_International_final.pdf tor English 317.68 KB Posted 312
Download document Final Evaluation CCCA-II Report.pdf report English 1229.57 KB Posted 615
Title Final Evaluation: Cambodia Climate Change Alliance Phase 2 Project
Atlas Project Number: 00059036
Evaluation Plan: 2019-2023, Cambodia
Evaluation Type: Final Project
Status: Completed
Completion Date: 05/2019
Planned End Date: 07/2019
Management Response: Yes
UNDP Signature Solution:
  • 1. Poverty
Corporate Outcome and Output (UNDP Strategic Plan 2018-2021)
  • 1. Output 2.1.1 Low emission and climate resilient objectives addressed in national, sub-national and sectoral development plans and policies to promote economic diversification and green growth
SDG Goal
  • Goal 1. End poverty in all its forms everywhere
  • Goal 13. Take urgent action to combat climate change and its impacts
SDG Target
  • 1.5 By 2030, build the resilience of the poor and those in vulnerable situations and reduce their exposure and vulnerability to climate-related extreme events and other economic, social and environmental shocks and disasters
  • 13.1 Strengthen resilience and adaptive capacity to climate-related hazards and natural disasters in all countries
  • 13.2 Integrate climate change measures into national policies, strategies and planning
Evaluation Budget(US $): 30,000
Source of Funding: Project Budget
Evaluation Expenditure(US $): 25,430
Joint Programme: No
Joint Evaluation: No
Evaluation Team members:
Name Title Email Nationality
Alan Ferguson International Consultant for CCCA Programme Final Evaluation alan@regionalconsulting.ca
GEF Evaluation: No
Key Stakeholders:
Countries: CAMBODIA
Lessons
1.

Firstly, integrating climate change into Cambodian government operations and developing an effective knowledge management system is a long, slow process. CCCA has been working for ten years to build a strong, comprehensive policy framework. But there are still significant gaps in the process that require continued support, and uncertainties about the timetable for full implementation of Cambodia’s Climate Change Strategic Plan.


2.

Secondly, capacity development for enhanced climate resilience and reduced GHG emissions requires not only training human resources, but also fundamental changes in the institutions within which they work. Individuals can make a difference, particularly and sometimes only if they can influence how their organisation prioritizes and delivers on national commitments to addressing climate change. Implementation of climate change policy and strategy requires more than training and demonstration but also challenging modification of institutional processes. For example, ministry responses to climate change commitments need to be built into their organisational structure, priority-setting, workplans and job descriptions. 


3.

Thirdly, the introduction of appropriate technologies can be complicated. There are a host of factors, as shown by the CCCA projects, which affect the acceptance and adoption of new climate-resilient and low carbon technologies and practices. Technologies such as rice husk briquettes fuel substitutes, or no-tillage/cover crop farming, or bio-digestor electricity, or solar electrical micro-grids can prove to be viable alternatives but their wider adoption depends upon many social, financial, organisational and human factors and the readiness and incentives to shift to new methods and practices. The drivers of viable innovations and the strategies to facilitate them are poorly known in the CCCA projects. 


Findings
1.

The direct involvement of CCCA with government policies and line ministry programmes ensures that the activities are aligned with government priorities. CCCA Window 1 Grant projects are specifically designed to implement Ministry’s CCAPs under the umbrella of the national CCCSP. This makes the programme highly relevant to national objectives. At the subnational level, commune authorities and local people are involved in selecting the priorities for small-scale climate resilience activities.


Tag: Climate Change Adaptation Climate change governance Disaster Risk assessments Disaster Risk Reduction Resilience building Vulnerable Relevance Civic Engagement Local Governance Programme Synergy Country Government

2.

The annual budgets and expenditures are presented in Table 5 based on the overall CCCA budget of $11.7 M USD. Almost $10M has been expended to the end of 2018, or 85% of the total budget. Result-1 accounted for 48.4% of total expenditures and Result-3 30.6%. The annual expenditures relative to budgets started slow in year 1 at 75% but increased rapidly to over 90%. Result-2 involves more flexibility and responsiveness in activities funded and therefore the delivery rate varies from year to year.


Tag: Efficiency Knowledge management Operational Efficiency Project and Programme management Country Government

3.

The cost-effectiveness and value for money questions are harder to answer since it is too early to determine the extent to which climate change investments from CCCA have led to reduced vulnerability or increased resilience. CCCA staff point to the baseline conditions that had limited policy and government framework and action and the progress that has now been made today after ten years of CCCA support. There has been significant development of the climate change policy and planning systems, and subjective assessments of readiness that suggest great improvements, but there is no reliable empirical basis at this stage for knowing whether the benefits of CCCA will outweigh the costs.


Tag: Climate Change Adaptation Climate change governance Climate finance Resilience building Vulnerable Effectiveness Policy Advisory

4.

The effectiveness of CCCA-II is reflected in the policy developments that have been accomplished and the large number of activities and outputs that have been completed (Annex 7). A summary of Logframe progress in Annex 10, from the 2018 CCCA Annual Report, show the substantive level of outputs results. The downstream effects on government programmes and budgets and on actual climate change adaptation and GHG emissions reduction is less certain.


Tag: Climate Change Adaptation Climate change governance Crisis Mitigation Vulnerable Effectiveness Multi Donor Trust Funds Resource mobilization Monitoring and Evaluation Policy Advisory

5.

Long-term viability of the technologies piloted
The evidence of CCCA-II impact on adoption of new adaptation and mitigation technologies (Table 3) remains to be considered in many of the Grant project final completion reports. While uncertainties about acceptance by prospective users are often apparent (see Annex 8 field notes), some important experiences have helped to identify the key factors influencing the shift toward climate resilient technologies and practices in the CCCA projects. A few examples of these factors are presented below from the interviews and site visits.


Tag: Climate Change Adaptation Crisis Mitigation Impact Technology

6.

The incremental contribution of CCCA to advancing clean technologies in industry is minor in the national context. Partnerships with larger initiatives were not explored. For example, industrial energy efficiency was addressed in a 2011-2015 GEF/UNIDO project, “Reduce Greenhouse Gas Emission through Energy Efficiency in Industrial Sector in Cambodia”. The main activities were aimed at developing site specific pilots that demonstrated the cost and energy savings from clean technologies. The project trained many enterprises, energy experts that work with those enterprises, and on the regulatory end, developed policies that would support enterprises in search of clean technologies. Some exceptional results were reported. The original target was ‘40 IEE project quick scans’ and ‘cumulative savings of 45,000 TOEs28 of energy savings over the technology lifetime’. The project reached this output target but also demonstrated a lifetime GHG reduction of 436,870 tonnes CO2 - over ten times the original project estimate. On the institutional strengthening side, many of the outputs focused on training government officials on energy software, energy audits and on IEE which were for the most part achieved. However, the project failed to work with financial institutions on IEE investments which was one of the biggest shortcomings.29 The evaluator noted nevertheless that most of the cost-benefit for the companies in the project was high and most investments paid back in less than 2 years.30 Lessons from such sector experiences can usefully inform CCCA programming and government extension services to provide technical support for industry modernization. The value-added contribution of CCCA project to sector mitigation policies and government programme development remains poorly defined.


Tag: Energy Environment Policy Green Economy Renewable energy Knowledge management Programme Synergy Programme/Project Design Country Government Technology

7.

Development effects on communities

The livelihood, income, water supply and related community capacity building impacts have been generally positive but cumulative programme effects can only be gleaned in a general way from the output list and field notes in Annexes 7 and 8. The community development impacts are primarily associated with local results from the Grant projects that had livelihood development activities, and the small-scale works undertaken by PDOEs for climate change integration into Commune Investment Plans.


Tag: Climate Change Adaptation Climate change governance Crisis Response Vulnerable Effectiveness Impact Small Grants Programme Local Governance Programme Synergy Strategic Positioning Country Government Capacity Building

8.

Impact on climate change responses

CCCA-II primary impact has been on strengthening the government commitment and policy and planning systems toward greater climate change orientation. Integrating climate change at a high level in NSDP and economic growth considerations is a significant accomplishment over the situation at the close of CCCA-I. According to interview respondents, CCCA support has given climate change a much more prominent role in key ministries – particularly MEF, MOP, MRD and MPWT. There has also been growing recognition of the challenges and complexities in CCAP implementation in ministries and perhaps under-estimation of the difficulties in operationalizing the 2014 Climate Financing Framework. CCCA-II also increased awareness about the need to present a sound economic or business case for climate resilient infrastructure, low carbon development options and energy management investment. The experience has also raised the profile on integrating climate change risk reduction at the village level, and the importance of linking increased national budgets for climate change response to identified risks and vulnerabilities on the ground.


Tag: Rural development Climate change governance Disaster Risk Reduction Vulnerable Impact Knowledge management Country Government Policy Advisory

9.

The CCCA Grant projects are required to have exit/sustainability strategies. Following comments from the MTR, each grant was asked to update its exit and sustainability strategy and indicate any needs for CCCA support. This was recorded and the CCCA team was to monitor implementation.31 Further documentation of this monitoring is warranted. The interviews at sampled project sites (Annex 8) asked questions about the user group management of the community infrastructure that had been established – solar pumping, tube wells, solar electricity system, rehabilitated irrigation canals, school solar electricity, etc. While local committees and user groups had in most cases been reported as established, evidence of actual management functions and user fee collection in place were sparse. (There are established rules for water user groups in Cambodia but we could not find them being implemented at the few sites visited). Some of these facilities, e.g., school budgets for maintaining solar system batteries, will have difficulties unless external financial support can be found.


Tag: Sustainability Resource mobilization Civic Engagement Local Governance Monitoring and Evaluation Project and Programme management Sustainability

10.

The Final Evaluation generally concurs with conclusions of the mid-term review: “The sustainability has to be assessed against the level of empowerment within line ministries and/or the ownership of results for involved communities. Within line ministries, CCCA2 is not fully sustainable: there is still occasional resistance from key decision makers within line ministries for climate change mainstreaming, evidencing the need for further awareness raising activities. …The institutional strengthening is well under way under CCCA2 with the enhancement of the capacity of NCSD. CCCA2 has relied heavily on the provision of consultants for the delivery of some key products. This approach has produced mixed results with a lower than expected line ministry capability to be empowered.”32


Tag: Climate change governance Sustainability Knowledge management Sustainability Country Government Private Sector Policy Advisory

11.

During the evaluation sites visits, a government official was asked about the percentage of hand pumps that were functioning properly in his district. He estimated that only about 60% of hand pumps under MRD responsibility were functional due to lack of government resources or lack of water. Studies of water management in Cambodia also point to major issues in the operations and maintenance of irrigation schemes. For example, a survey of the 2,525 irrigation schemes in Cambodia, determined that 1574 or 62% did not function at all; 802 or 32% of schemes function partly and only 149 schemes 6% function well.33 This is a central issue that is not yet fully recognized in CCCA strategies – providing more funding to line ministries for climate change action also needs to come with obligations for significant institutional reform, capacity building and community empowerment. Further assessment of sustainability in all the CCCA-II field projects is warranted in the completion and evaluation reports. For example, the MOWRM project in Oddar Meanchey province installed extensive equipment and protocols for monitoring meteorological, groundwater, stream discharge and water supply conditions, and the creation of a Farmer Water Users Committee/Group at Chong Kal. The arrangements for continued operations, maintenance and sustainability including usability of the water resource data should be verified in the completion report, to be submitted soon.


Tag: Climate Change Adaptation Sustainability Civic Engagement Rule of law Implementation Modality Monitoring and Evaluation Programme/Project Design Sustainability Civil Societies and NGOs Country Government Capacity Building Policy Advisory

12.

Coherence/Complementarity

As noted under the Objectives achievement discussion in Section 2.1, it is difficult to identify and assess the complex relationships between the CCCA-II Results (expected outcomes) and the many project and activities funded under the Grant Facility Windows 1, 2 and 3. Many of the results from the Grant projects fall within Output 1.3 but also within other programme Outputs or none at all. This means that synergies and complementarities between different ministry and partner activities in contributing to programme results are not explicit and apparent.


Tag: Coherence Harmonization Programme Synergy Results-Based Management Country Government

13.

Partnerships and Grantee Capacity Development

Capacity assessment of all grantees was carried out at the project design stage, and relevant capacity development measures were integrated. This sometimes included training and mentoring of the grantee’s administrative and management staff, more frequent spot checks, or provision of detailed operational procedures when the grantee’s own procedures were not fully up to standards.35 Oversight of the Grant projects has been a key function of CCCA Secretariat staff and CCD. The cursory field review of nine Grant projects and discussions with staff from five other projects, as summarized in Annex 8, suggested that further capacity development support is needed in some cases, especially in regard to documentation of results, field inspections and sustainability arrangements. There have been exceptional efforts by CCCA, especially in the case of assisting universities to develop climate change research priorities and proposals. Further assistance in the final stages of closing Phase II is needed to assess the uptake of the technologies and practices that have been promoted and the potential for sustainability and replication.


Tag: Climate Change Adaptation Sustainability Public administration reform Oversight Programme/Project Design Sustainability Civil Societies and NGOs Capacity Building

Recommendations
1

CCCA and CCD should implement targeted measures to finalize line ministries’ commitments to climate change mainstreaming in government programmes.

2

CCCA and CCD should fully monitor and report on, in the programme completion report, the results of the project activities at integrating climate change into the pilot communes and assess the effectiveness and lessons for subnational climate change action building upon other previous work on local government climate change mainstreaming in Cambodia.

3

CCCA should develop and apply a method to assess the leadership, capability and sustainability of water and energy user groups that have been established in the CCCA Grant projects, drawing upon available standards and guidelines for sustainable community-based facilities management.

4

CCCA Grant project agencies that have promoted selected technologies should be required to determine the level of adoption of the technologies, the necessary conditions for adoption, or reasons for rejection by the targeted user group/beneficiaries and the specific factors that influence the technology uptake and potential replication.

5

CCCA should collaborate with UNIDO or other mitigation expert organisations on a review of government and private sector programmes involved with energy management in selected industrial sectors, drawing on the experiences and recommendations from the CCCA-II projects and the recent GEF industrial energy efficiency project in Cambodia.

6

The Cambodia National Biogas Programme (NBP) should develop a business partnership with appropriate livestock supply companies to expand pig farm biogas technology with modern livestock management systems, particularly in areas where water quality improvements are needed near farm operations.

7

CCCA should adopt quality standards for project reporting and case studies and enhance capacity of grantees and CCD staff for concise, objective description of key results and useful evidence emerging from CCCA projects.

8

CCCA, NCDD and CRDB should develop a practical decision support tool and related dashboard to assist local government staff in analyzing climate-resilient infrastructure options, drawing upon the extensive climate change project databases in Cambodia.

9

CCCA and DCC should strengthen MOE’s strategic focus on ecosystem-based adaptation in their primary role to maintain the protected area system at the local level.

 

10

CCCA should develop a programme-wide Gender Action Plan that addresses measurable cross-cutting objectives and results from the multiple projects that are funded.

 

11

CCCA and UNDP should develop a CCCA-III M&E Plan that includes external evaluation of projects by sector on an annual basis, appointing a designated M&E Officer, and ensuring that the CCCA-III Results Framework has well-defined, measurable outcomes and reliable, pre-tested indicators.

12

CCCA should increase the monitoring and reporting on government, private sector and beneficiary contributions to the programme results.

Management Response Documents
1. Recommendation:

CCCA and CCD should implement targeted measures to finalize line ministries’ commitments to climate change mainstreaming in government programmes.

Management Response: [Added: 2019/06/11] [Last Updated: 2020/11/25]

Agreed

Key Actions:

Key Action Responsible DueDate Status Comments Documents
This is already incorporated in Phase 3 design. CCCA Phase 3 will deepen CC mainstreaming in 5 priority ministries.
[Added: 2019/06/11]
Project Developer (UNDP CO) 2019/07 Completed
2. Recommendation:

CCCA and CCD should fully monitor and report on, in the programme completion report, the results of the project activities at integrating climate change into the pilot communes and assess the effectiveness and lessons for subnational climate change action building upon other previous work on local government climate change mainstreaming in Cambodia.

Management Response: [Added: 2019/06/11] [Last Updated: 2020/11/25]

Agreed

Key Actions:

Key Action Responsible DueDate Status Comments Documents
It will be included in the final project report.
[Added: 2019/06/11] [Last Updated: 2019/12/19]
Climate Change Department (CCD) of General Secretariat of National Council for Sustainable Development (GSSD) 2019/12 Completed The final report for the project produced with the recommended areas incorporated History
3. Recommendation:

CCCA should develop and apply a method to assess the leadership, capability and sustainability of water and energy user groups that have been established in the CCCA Grant projects, drawing upon available standards and guidelines for sustainable community-based facilities management.

Management Response: [Added: 2019/06/11] [Last Updated: 2020/11/25]

Agreed

Key Actions:

Key Action Responsible DueDate Status Comments Documents
It is already included in the issues under review by the consultancy on lessons learned from CCA grants (May 2019)
[Added: 2019/06/11]
CCD/GSSD 2019/06 Completed
4. Recommendation:

CCCA Grant project agencies that have promoted selected technologies should be required to determine the level of adoption of the technologies, the necessary conditions for adoption, or reasons for rejection by the targeted user group/beneficiaries and the specific factors that influence the technology uptake and potential replication.

Management Response: [Added: 2019/06/11] [Last Updated: 2020/11/25]

Agreed. 

Key Actions:

Key Action Responsible DueDate Status Comments Documents
It is already implemented. This is part of the final project report requirements and will also be a focus of the knowledge event organized on 28-29 May 2019.
[Added: 2019/06/11]
CCD/GSSD 2019/06 Completed
5. Recommendation:

CCCA should collaborate with UNIDO or other mitigation expert organisations on a review of government and private sector programmes involved with energy management in selected industrial sectors, drawing on the experiences and recommendations from the CCCA-II projects and the recent GEF industrial energy efficiency project in Cambodia.

Management Response: [Added: 2019/06/11] [Last Updated: 2020/11/25]

Agreed

Key Actions:

Key Action Responsible DueDate Status Comments Documents
UNIDO was consulted and included as an advisor for the CCCA project with MIH on clean production. Lessons learnt from the CCCA project will be communicated to UNIDO as MIH is not a priority ministry for Phase 3.
[Added: 2019/06/11]
CCD/GSSD 2019/06 Completed “GGGI and UNIDO will cooperate on larger projects in green industry and lessons learnt from the CCCA MIH project have been shared with GGGI (June 2019).”
6. Recommendation:

The Cambodia National Biogas Programme (NBP) should develop a business partnership with appropriate livestock supply companies to expand pig farm biogas technology with modern livestock management systems, particularly in areas where water quality improvements are needed near farm operations.

Management Response: [Added: 2019/06/11] [Last Updated: 2020/11/25]

Agreed

Key Actions:

Key Action Responsible DueDate Status Comments Documents
This recommendation applies to NBP not CCCA. It will be communicated with NBP. They already have a partnership with one large livestock supplier.
[Added: 2019/06/11]
NBP 2019/03 Completed
7. Recommendation:

CCCA should adopt quality standards for project reporting and case studies and enhance capacity of grantees and CCD staff for concise, objective description of key results and useful evidence emerging from CCCA projects.

Management Response: [Added: 2019/06/11] [Last Updated: 2020/11/25]

Agreed

Key Actions:

Key Action Responsible DueDate Status Comments Documents
This already done standards, the issue is staff capacity. CCCA will provide additional training/mentoring on this under Phase 3.
[Added: 2019/06/11] [Last Updated: 2020/08/09]
CCD/GSSD 2020/03 Completed The training plan is developed and integrated under Phase 3 of the project which is currently in its implementation stage. History
8. Recommendation:

CCCA, NCDD and CRDB should develop a practical decision support tool and related dashboard to assist local government staff in analyzing climate-resilient infrastructure options, drawing upon the extensive climate change project databases in Cambodia.

Management Response: [Added: 2019/06/11] [Last Updated: 2020/11/22]

Agreed

Key Actions:

Key Action Responsible DueDate Status Comments Documents
This is best done by NCDD-S through their climate change projects, with technical support from NCSD/CCCA as needed. CCCA will share this recommendation with NCDD-S for consideration.
[Added: 2019/06/11] [Last Updated: 2019/10/16]
CCD/GSSD, NCDD-S 2019/09 Completed This has been shared with NCSD for consideration History
9. Recommendation:

CCCA and DCC should strengthen MOE’s strategic focus on ecosystem-based adaptation in their primary role to maintain the protected area system at the local level.

 

Management Response: [Added: 2019/06/11] [Last Updated: 2020/11/25]

Agreed 

Key Actions:

Key Action Responsible DueDate Status Comments Documents
This focus will be taken into account for cooperation with MoE under Phase 3
[Added: 2019/06/11] [Last Updated: 2020/08/09]
CCD/GSSD, MoE 2020/12 Completed Phase 3 of the project is approved and currently in its implementation stage. History
10. Recommendation:

CCCA should develop a programme-wide Gender Action Plan that addresses measurable cross-cutting objectives and results from the multiple projects that are funded.

 

Management Response: [Added: 2019/06/11] [Last Updated: 2020/11/25]

Agreed

Key Actions:

Key Action Responsible DueDate Status Comments Documents
It is implemented and Gender Action Plan requirement included in Phase 3 project document
[Added: 2019/06/11]
Project Developer (UNDP CO) 2019/06 Completed
11. Recommendation:

CCCA and UNDP should develop a CCCA-III M&E Plan that includes external evaluation of projects by sector on an annual basis, appointing a designated M&E Officer, and ensuring that the CCCA-III Results Framework has well-defined, measurable outcomes and reliable, pre-tested indicators.

Management Response: [Added: 2019/06/11] [Last Updated: 2020/11/25]

Agreed

Key Actions:

Key Action Responsible DueDate Status Comments Documents
Points 2 and 3 already incorporated in Phase 3 design. The first point is noted and CCCA grants under Phase 3 will be evaluated in a cross-cutting manner instead of grant-specific evaluations.
[Added: 2019/06/11] [Last Updated: 2019/10/16]
CCD/GSSD 2020/12 Completed Cross-cutting evaluation of grants included in CCCA3 budget (but not annual) History
12. Recommendation:

CCCA should increase the monitoring and reporting on government, private sector and beneficiary contributions to the programme results.

Management Response: [Added: 2019/06/11] [Last Updated: 2020/11/25]

Agreed

Key Actions:

Key Action Responsible DueDate Status Comments Documents
It is integrated in the results framework of new Phase 3 of CCCA (indicators 3.3.3 and 3.1.1)
[Added: 2019/06/11]
Project Developer (UNDP CO) 2019/06 Completed

Latest Evaluations

Contact us

1 UN Plaza
DC1-20th Floor
New York, NY 10017
Tel. +1 646 781 4200
Fax. +1 646 781 4213
erc.support@undp.org