Terminal Evaluation of the Generating, Accessing and Using Information and Knowledge Related to the Three Rio Conventions Project

Report Cover Image
Evaluation Plan:
2016-2018, Cambodia
Evaluation Type:
Final Project
Planned End Date:
02/2019
Completion Date:
12/2018
Status:
Completed
Management Response:
Yes
Evaluation Budget(US $):
24,000

Share

Document Type Language Size Status Downloads
Download document 3Rio_Final Evaluation_TOR.pdf tor English 901.95 KB Posted 1176
Download document 3Rio_Final Evaluation_Report.pdf report English 1544.53 KB Posted 1189
Download document Disagreement from the IP on the Terminal Evaluation Report_3Rio.docx related-document English 16.17 KB Posted 1196
Title Terminal Evaluation of the Generating, Accessing and Using Information and Knowledge Related to the Three Rio Conventions Project
Atlas Project Number: 00083830
Evaluation Plan: 2016-2018, Cambodia
Evaluation Type: Final Project
Status: Completed
Completion Date: 12/2018
Planned End Date: 02/2019
Management Response: Yes
Focus Area:
  • 1. Others
Corporate Outcome and Output (UNDP Strategic Plan 2018-2021)
  • 1. Output 2.1.1 Low emission and climate resilient objectives addressed in national, sub-national and sectoral development plans and policies to promote economic diversification and green growth
SDG Goal
  • Goal 1. End poverty in all its forms everywhere
  • Goal 13. Take urgent action to combat climate change and its impacts
  • Goal 15. Protect, restore and promote sustainable use of terrestrial ecosystems, sustainably manage forests, combat desertification, and halt and reverse land degradation and halt biodiversity loss
SDG Target
  • 1.5 By 2030, build the resilience of the poor and those in vulnerable situations and reduce their exposure and vulnerability to climate-related extreme events and other economic, social and environmental shocks and disasters
  • 13.1 Strengthen resilience and adaptive capacity to climate-related hazards and natural disasters in all countries
  • 13.2 Integrate climate change measures into national policies, strategies and planning
Evaluation Budget(US $): 24,000
Source of Funding: Project budget
Evaluation Expenditure(US $): 16,702
Joint Programme: No
Joint Evaluation: No
Evaluation Team members:
Name Title Nationality
Dr. Amal Aldababseh
GEF Evaluation: Yes
GEF Project Title: Generating, Accessing and Using Information and Knowledge Related to the Three Rio Conventions
Evaluation Type: Terminal Evaluation
Focal Area: Multifocal Areas
Project Type: MSP
GEF Phase: GEF-5
GEF Project ID: 5295
PIMS Number: 5222
Key Stakeholders: Naitonal Council for Sustainable Development
Countries: CAMBODIA
Lessons
1.

Lesson learned 1: The Project was able to deliver significant achievements despite the challenging circumstances it faced at the inception phase including its small size in relation to the planned deliverables and being the first project to follow the national implementation modality. The Project was able to make good progress in establishing a strong foundation on which future and further work on accessing environmental data in Cambodia can be carried out. It can be noted that this was achieved by the dedication, flexibility, and determination with which the GSSD/MoE, PMU, UNDP Cambodia, UNDP/GEF, and other key stakeholders have implemented the project activities. Thus, this Project demonstrated that even in challenging project implementation contexts, with dedication and flexibility, important results can be achieved.


2.

Lesson learned 2: When technical capacity is considered as a challenging situation at the national level, it is very critical to define a clear project management structure that embeds the services of international technical advisors at various levels of project implementation. In Cambodia, this Project required the involvement of technical advisors to support the PMU in project implementation. However, that was not well-articulated in the Project Document, hence, it was not fully discussed during the Project Inception Phase. Furthermore, the Project has faced several problems in mobilizing the needed technical advisors on time. As a result, the involvement of external international specialists was late which has caused further delay in project implementation. The presence of a technical advisor from the early beginning of the project implementation is crucial to ensure the successful implementation of the project’s activities. The UNDP/GEF team may develop a list of qualified technical advisors, who can be easily mobilized by country officers to support CCCD projects implementation.


3.

Lesson learned 3: Based on the review of the technical deliverables produced by the Project, the effectiveness of relatively small amounts of assistance mobilized by the GEF initially was highly effective in supporting the national efforts and shows that even in a complex situation, the institutional capacity can be sustained with this initial stimulation. The successful implementation of the Project depends on the fund's availability, strong political support, and the mobilization of technical expertise needed. Hence, an exit strategy is very crucial to ensure the integration of the Project’s deliverables in governments work plan and strategies to ensure projects’ results sustainability.


4.

Lesson learned 4: Timely support to the PMU is needed from the UNDP/GEF and UNDP Country Offices to ensure the development of the needed adaptive management measures to be undertaken during the project inception phase. This could have helped the Project to avoid delay and support the project to utilize whatever opportunities arising that would lead to improved cost-efficiency, and/or offers solutions to a problem. Although this Project is very relevant to Cambodia and was entirely based on its NCSA Project, different operational issues resulted in slowing down the project implementation and have caused uncertainty with respect to the project’s sustainability. The UNDP/GEF technical support is very crucial during the inception phase to ensure that operational risks are clearly analyzed at the project inception stage as well as regularly during project implementation with concrete mitigation measures to be identified as part of the adaptive management.


5.

Lesson learned 5: A key to ensuring successful implementation of the project’s activities and the sustainability of its key deliverables is to ensure the country ownership. Hosting the PMU within the government premises is a very effective mean of fully engaging with government and local stakeholders. The project was hosted at the GSSD/MoE premises. The project was able to get the needed political, technical, financial, and logistical support. It has enhanced the ability of the PMU to coordinate and collaborate with other initiatives implemented in the fields of climate change, land degradation, and biodiversity conservation.


Findings
1.

3.1 Project Design/ Formulation


The project design is considered highly relevant to the Government of Cambodia’s global environmental obligations, development plans and strategy, to UNDAF and UNDP Country Programme, and to the GEF objectives. The Project Document included specific outcomes, outputs, listed activities under each output and defined targets and indicators.


The Project Document failed to include the required level of details concerning the project log-frame (LFA). Although it included the project’s components and outputs, it failed -in many cases- to make a proper link to the local context. The Project Document was successful in addressing five main cross-cutting capacity issues and defining the way to deliver sustainable impact by addressing the critical need to provide better environmental knowledge in Cambodia, building cross-cutting capacities to respond to the needs of the three conventions, and in defining the reporting requirements and developing the needed capacity to coordinate and generate needed information. Furthermore, on-going public awareness on linkages of the global environment to national socio-economic development priorities designed and partially implemented in some provinces and at the national level in Cambodia. These activities contributed to improving the implementation of the Rio Conventions in Cambodia through the development of individual and institutional capacities to better coordinate and generate data and information related to the implementation of the Rio Conventions.


The Project is operating in a policy framework that includes, among others: “Cambodia Rectangular Strategy-Phase III (2013-2018)”, the “Cambodia Climate Change Strategic Plan 2014-2023"; the “National Policy on Green Growth”; the “National Strategic Plan on Green Growth 2013-2030”; and the “National Protected Area System Strategic Management Framework (2014)”.


Tag: Relevance Global Environment Facility fund Programme/Project Design UNDP management

2.

3.1.1 Analysis of the LFA/Results Framework (Project logic/ Strategy, Indicators)

According to the project’s inception report, the LFA has been reviewed, some outputs and indicators were amended, and one output was added to the second outcome. However, the PMU and UNDP CO kept using the old LFA in their quarterly, annual progress, and annual implementation reports where output 2.4 was not in place and hence no reporting was done concerning this output.
Essentially the LFA followed the GEF format but it did not include targets at the outcome level. This resulted in some weaknesses in the LFA in defining targets and indicators at the outcomes level at the TE. Therefore, the LFA has led to some confusion concerning the project’s strategy. Table 2 provides an overview of the TE assessment of the project’s LFA and how “SMART” the achievements are compared to the defined end-of-project targets.

 


Tag: Programme/Project Design Results-Based Management

3.

3.1.2 Assumptions and Risks
The review of the risks to the Project, during the Project formulation phase, indicated that “these risks are manageable through the project’s learn-by-doing approach” mainly that “the Project is a direct response to national priorities identified through the NCSA process; as a result, there is a strong national ownership and willingness to succeed, hence low risks that key stakeholders will not participate in the project and lack of political will”. Furthermore, the project was designed to be hosted at the GSSD/MOE, which was expected to contribute to managing any operational risks which would also contribute to a better prospect for long-term sustainability of Project results.


Tag: Ownership Programme/Project Design Risk Management

4.

3.1.3 Lessons from other relevant projects incorporated into project design.


After a careful review of the Project Document, no clear signs for incorporating lessons learned from other relevant projects into this Project design. However, government officials indicated that they have learned several lessons from the implementation of the Project, as the first project to be fully implemented following the NIM modality, which they would like to utilize and benefit from in the design of other GEF and UN projects. To share some (a) learning-by-doing is very crucial for the sustainability of all initiatives and to ensure the national and government ownership even if the capacity is limited. The Government’s officials have faced many complex issues in implementing this project, yet, they consider it as a better way of learning, and sustainable method to enhance capacity, (b) a chief technical advisor should be hired from the early beginning of the project and to be stationed in the country so that the team learns more and gets the needed technical support on time, and (c) innovative ways to be developed to provide support to government’s staff who are participating in the Project implementation.


Tag: Implementation Modality Ownership Programme/Project Design

5.

3.1.4 Planned stakeholder participation
To ensure the Project efficient and effective implementation, the main Project’s stakeholders and the direct beneficiaries – the government institutions – should be fully engaged and supportive of the project’s intervention. The project has managed to develop some of the critical partnerships with stakeholders at the national mainly with the Ministries of Environment (hosts the UNFCCC and UNCBD focal points) and Agriculture, Forestry and Fisheries (hosts UNCCD focal point), where relationships appear to be pleasant and there is considerable support. However, considering the strategic scope of this Project to build national capacities in data gathering, accessing, sharing, and reporting, the TE would have expected to see more evidence of partnerships with organizations involved in different field in relation to the Rio Conventions, such as the academic sectors, private sectors, and national and international non-governmental organizations and development partners.


Tag: Communication Ownership Partnership Project and Programme management Sustainability

6.

3.1.5 Replication approach


The Project’s results such as the developed capacity, enhanced public awareness, and the developed guidelines and tools would ensure the sustainability of global environmental benefits and outcomes’ replicability of the key principles. The implemented approach for replicability included the following main elements:
The project was designed to address national priorities that were identified through the NCSA process. The project would support Cambodia in having access to more accurate and timely environmental information as well as a better coordination mechanism for the implementation of the Rio Conventions. The capacity developed by this Project will equip government’s teams with the needed skills, tools and knowledge to develop and implement other key environmental initiatives.


Tag: Communication Sustainability Capacity Building

7.

3.1.6 UNDP comparative advantage
UNDP, the GEF Implementing Agency, for this Project was selected based on its experience and expertise in supporting capacity developments efforts in Cambodia, the regional, and the global level. Furthermore, the comparative advantage lies in its experience in integrating policy in national processes, policies, and frameworks, and the lessons learned and best practices that it could bring to bear from their experience in other countries.


The Government of Cambodia and UNDP CO have worked jointly on implementing the NCSA project and its follow up initiatives. This Project is complex due to its multi-sectoral, and multi-stakeholders nature, hence, UNDP’s ability to provide the needed technical expertise in designing and implementing activities, its in-country presence, its key role with regards to advocacy, “advocating a sustainable development agenda with Government and other partners14,” and its experiences in developing environmental indicators and monitoring and evaluation tools15 - which is extremely necessary in implementing these kind of complex initiatives impacts on the ground - all these comparative advantages helped UNDP to be in a prime position to provide Cambodia with the needed support.


Tag: Advocacy Technical Support

8.

3.1.7 Linkages between the project and other interventions within the sector
The Project positively collaborated with several national initiatives and projects funded by international donors and development partners. Those include UNDP/GEF, Asian Development Bank (ADB), UN Environment/GEF, GIZ, and the Government of Cambodia. In addition, the project was hosted at the DBD at the Ministry of Environment. The DBD is managing several biodiversity-related projects and this has facilitated the work of the Project by sharing lessons learned, sharing financial and technical resources, and providing the needed logistical and technical support. Also, the Project was implemented under the UNDP Environment Unit which is currently responsible for implementing six other ongoing initiatives. The Project’s team members were collaborating with their colleagues from other projects.


 


Tag: Global Environment Facility fund Small Grants Programme Donor relations Multilateral Partners Project and Programme management International Financial Institutions UNDP management Civil Societies and NGOs

9.

3.1.8 Management arrangement


The Project is being implemented under the NIM. The DBD/GSSD/MoE is the designated Executing Agency (EA) and main beneficiary. UNDP is the Senior Supplier and the GEF Implementing Agency responsible for transparent practices and appropriate conduct. UNDP has the Project Assurance role, which supports the Project Board Executive by carrying out objective and independent project oversight and monitoring functions. All project’s activities are developed in close cooperation with the MAFF as it hosts the focal point for the UNCCD. The Executive is represented by a senior official of GSSD/MoE, as an individual representing the project ownership to chair the group. The Senior Beneficiaries are the Departments of Biodiversity Conservation (DBC) and the Department of Climate Change, GSSD/MoE, and the Department of International Cooperation, MAFF.


The NIM modality was based on the HACT assessment and agreed with the Government of Cambodia. However, the selection of this Project to be the first national project ever that fully implements the NIM modality was a brave decision due to the complex nature of the project, and the limited experiences at the national level (for the UNDP CO and the Government) on how to implement a project under the NIM modality. The management arrangements were developed in the Project Document, presented and agreed during the inception workshop. However, no changes were proposed during the inception phase, and hence, the Project has followed the proposed structure despite the 22 months delay in the project implementation.


Tag: Implementation Modality Oversight Quality Assurance Coordination

10.

A Project Board (PB) was to provide strategic decisions and management guidance to implement the project. The PB reports to the National Council for Sustainable Development (NCSD). The PB is made up of representatives of relevant ministries and government departments (the three Rio Conventions focal points and a representative from the Ministry of Women’s Affairs), and UNDP, and chaired by the GSSD/MOE Secretary-General. A National Project Director (NPD) was nominated by the Government of Cambodia to follow up on the Project activity, who is the Deputy Secretary General, GSSD/MOE. The project was monitored by the PB. The Project Document stated that the project board should meet at least (2) times per year. Based on the approved annual work plan (AWP), the PB reviews and approves project quarterly plans when required and authorizes any major deviation from agreed quarterly plans. The NPD is supervising operational management and guidance for execution and implementation within the constraints laid down by the Project Board and subcontracts specific components of the project to specialized government agencies. The NPD is actively responsible for financial management and disbursements with accountability to GSSD/MoE and UNDP.


Tag: Oversight Country Government

11.

A Capacity Development Technical Advisor (CDTA) position was proposed in the Project Document. This CDTA was intended to join the PMU from the early beginning of project implementation to provide relevant expertise and capacity building support to the PMU and the NPD. The assumption that recruiting a CDTA would help in bringing together delivery of the key deliverables and oversees the capacity development strategy. However, the Project faced a big challenge in mobilizing the needed CDTA. The Project hired several CDTAs, who were unable to deliver – for many reasons beyond the project’s capacity- which has contributed to a further delay in the project implementation. The forth CTA hired mid-2018, was able to support the PMU but with considerable delay in submitting the project’s deliverables. Hence, the limited time left for project implementation makes it very difficult for the PMU to finalize all project’s deliverables.
 


Tag: Challenges Implementation Modality Oversight Capacity Building Technical Support

12.

3.2 Project Implementation
The Project implementation arrangement and its adaptive management have been reviewed and assessed. The following aspects of project implementation have been assessed20: Adaptive management (changes to the project design and project outputs during implementation); Partnership arrangements (with relevant stakeholders involved in the country); Feedback from M&E activities used for adaptive management; Project finance; Monitoring and evaluation; design at entry and implementation*, and UNDP and Implementation Partner Implementation/ execution coordination, and operational issues*.


Achievements of project implementation and adaptive management have been rated in terms of the criteria above at a six-level scale as follows21: Highly satisfactory (HS) - the project has no shortcomings; Satisfactory (S) - minor shortcomings; Moderately satisfactory (MS)- moderate shortcomings; Moderately unsatisfactory (MU)-significant shortcomings; Unsatisfactory (U)- major shortcomings; and Highly unsatisfactory (HU) - severe shortcomings.
The following paragraphs provide a complete review and justifications for the rating of the results. The rating and a description of that rating are summarized in the TE Ratings & Achievements table 1, Page 6.


Tag: Implementation Modality Project and Programme management

13.

Inception Phase:
The inception phase is considered as an opportunity to unite the project management team, to define the current and near-future status of the project, to discuss and review the project strategy with stakeholders, to put in place the necessary logistics, to develop the first Annual Work Plan (AWP) and to review and refine the Project Logical Framework (LFA)22. The major output of the inception phase should be the Inception Report (IR)23 and the first AWP, which, on an agreement with the Project Board, will form a necessary flexible basis for implementation.


Tag: Programme/Project Design Project and Programme management Results-Based Management

14.

3.2.1 Adaptive Management
Adaptive management means that the PMU with the support of the PB should constantly keep referring to the Project’s goal and objectives and critically assessing how the activities are contributing to the outputs and how those outputs are leading to the objective. Although the project started one year later than the planned date and had witnessed another major delay for more than six months due to the difficulties in hiring the project team, the international consultants, and the national consultant, the TE did not observe any major adaptive management measures.
 


Tag: Implementation Modality Challenges Human and Financial resources

15.

3.2.2 Partnership arrangements
The Project has been successful in arranging partnerships with the main stakeholders (GSSD/MOE and MAFF) for the implementation of the project. The project was hosted at the GSSD/MOE, this has helped the project to be very close to other projects and initiatives led by the DBD/GSSD/GSSD/MOE. As a result, the Project was able to closely monitor the implementation of other initiatives developed/ supported by key international donors (including the UN Environment, UNDP/GEF Projects, and GIZ and ADB). However, as stated in the MTR, the project should establish key partnerships to facilitate its implementation and extend its work to cover all beneficiaries. For example, for the piloting, the MTR recommended that “Begin conceptual work on the CHM Pilot Education work at the municipality level. To implement the pilot, the project management should enter into an implementing partnership agreement with a municipality to develop and pilot test a CHM Three Rios training package for NGOs to disseminate and carry forward the learning about the CHM and Three Rios through its network of communities. This pilot work with GSSD/MOE and project should result also in a guide of the Three Rio Conventions for schools and public.”24 However, this did not take place.


Tag: Donor International Financial Institutions UN Agencies Partnership Project and Programme management Country Government

16.

3.2.3 Feedback from M&E activities used for adaptive management
The M&E framework of the Project is a perfect example of the missing adaptive management framework that should have been applied to address a management challenge. The PMU did not have any sufficiently developed adaptive management framework and did not fully understand the project’s strategy. However, the PMU had an honest desire to get on with the job and get some of the project’s activities in place.

 


Tag: Communication Monitoring and Evaluation Results-Based Management

17.

It is noted that the Project did not go under major changes as a result of the MTR, however, the Project through the PMU and with the support of the PB was able to demonstrate a set of adaptive management in response to the Recommendations of MTR:


- Evaluation Recommendation 1: GSSD/MoE formally links project implementation to the GSSD/MoE/MAFF processes related to information and knowledge management. In response, a project work plan and approach to move toward common information system considered the fact that each of the conventions is currently at the different levels of capacity. The work plan was produced and endorsed during the project board meeting in Q1 2018.


Tag: Knowledge management Monitoring and Evaluation

18.

It is noted that the Project did not go under major changes as a result of the MTR, however, the Project through the PMU and with the support of the PB was able to demonstrate a set of adaptive management in response to the Recommendations of MTR:

Evaluation Recommendation 2: GEF/UNDP/GSSD-MoE reschedule project in line with the late implementation start, i.e. Oct. 2016. The Project, therefore, needs to be extended for 1.5 years (31 December 2018). In response, a project extension request was submitted to the UNDP GEF for official approval. The project was officially extended to the 31st of December 2018.


Tag: Monitoring and Evaluation Policies & Procedures

19.

It is noted that the Project did not go under major changes as a result of the MTR, however, the Project through the PMU and with the support of the PB was able to demonstrate a set of adaptive management in response to the Recommendations of MTR:

- Evaluation Recommendation 3: PMU develops an infographic vision statement to articulate the expected outcomes in a short guide to communicating the overall expected outcomes and road map with two components to all stakeholders. The project implementation strategy is limited in the area related to technical oversight and guidance on interpretation of the results and is missing important inclusions of stakeholders and institutional linkages for sustainability and uptake of IMS and CHM toward results between output one and output two in addition to the events already organized such as IDB. In response to this, a Roadmap and implementation strategy and institutional linkage were reflected in the Project work plan developed and endorsed during the project board meeting in the first quarter of 2018.


Tag: Monitoring and Evaluation Results-Based Management

20.

It is noted that the Project did not go under major changes as a result of the MTR, however, the Project through the PMU and with the support of the PB was able to demonstrate a set of adaptive management in response to the Recommendations of MTR:

- Evaluation Recommendation 4: UNDP immediately hires a qualified (chief) Technical Specialist (CTS) to support technical implementation, develop an implementation roadmap in line with this MTE report, set up the suggested implementation platforms and oversee processes linking work between the two outputs and outcome. The CTS should engage in all project results in monitoring and oversight. In response, a technical specialist was hired and was on board in November 2017. However, the consultant was unable to deliver until April 2018. A new CTS was hired in June, who is currently helping the PMU in finalizing all project deliverables.


Tag: Monitoring and Evaluation Oversight Technical Support

21.

It is noted that the Project did not go under major changes as a result of the MTR, however, the Project through the PMU and with the support of the PB was able to demonstrate a set of adaptive management in response to the Recommendations of MTR:

- Evaluation Recommendation 5: PMU continue to encourage the knowledge management learning officer and recruit partnership, gender mainstreaming strategy consultants and IT expert/team to support the Project Director and GSSD, Coordinator for developing and advancing the Three Rios Partnership/networking strategy and operationalizing project GSSD/MoE social/networking, cross-sectoral and public learning agenda and development of partnership engagement and gender mainstreaming strategies, and website(s). In response, the Project hired a group of national and international consultants to support the Project Director and Coordinator in delivering the project’s outputs.


Tag: Human and Financial resources Knowledge management Monitoring and Evaluation

22.

It is noted that the Project did not go under major changes as a result of the MTR, however, the Project through the PMU and with the support of the PB was able to demonstrate a set of adaptive management in response to the Recommendations of MTR:

- Evaluation Recommendation 6: The project/knowledge management learning officer become focal point for the strategic project communication, i.e. newsletter and broader uptake of the CHM through making linkages to the future CHM pilot project, dissemination of learning guides and working with partners for the CHM training of trainers, environmental education in general and project activities. In response, an agreement was reached, and the concept was produced regarding the CHM for the RIO conventions. Each of the Rio Conventions teams agreed to be part of the CHM and to share the information on the two common thematic areas under RIO Conventions including (i) Sustainable land used management and restoration of the ecosystem; and (ii) CEPA (Communication, Education and Public Awareness). The draft of the Praka to ensure that each of the Rio conventions continues their collaboration is ready and is in the process to be endorsed.

 


Tag: Communication Monitoring and Evaluation Shared Services

23.

3.2.4 Project Finance
In line with the UNDP/GEF TE guide, the TE has assessed the differences between the actual expenditure and the leveraged financing and co-financing during the TE mission presented in Table 4, which provides an overview of the budgeted expenditures of the GEF Project of US$990,000. As of September 2018, US$ 673,137.85 about (68%) of the project total budget, has been dispersed. However, around US$293,319.59 about (29.63%) are committed to be disbursed during the period of October to December 2018 (encumbrance). This amount will be used to finalize the work of the project technical advisor, the TE, and the production of the project’s deliverables.

 


Tag: Efficiency Resource mobilization Operational Efficiency

24.

3.2.5 Monitoring and evaluation: design at entry and implementation (*)
M&E Design at Entry

The standard UNDP/GEF budgeted monitoring and evaluation plan was included in the Project document with identified responsible parties, allocated budget, and specified time frame for each M&E activity. The M&E plan is supposed to be conducted in accordance with the established UNDP and GEF procedures. A total of US$ 35,500, about 3.5% of the total GEF grant was allocated for the M&E activities. Supposedly, this amount is enough to conduct the proposed M&E plan except for the MTR which is not a requirement for this size of projects, however, due to the necessity to conduct it, allocations were mobilized from other resources. The TE consultant could not get an update on the total budget utilized for the M&E activities and hence is unable to define if the originally planned budget was enough or not.


All standard UNDP/GEF M&E tools were included in the project document, including the log-frame, indicators, targets, inception workshop, and inception report, terminal evaluation, the quarterly and annual progress reports and board meetings. The MTR was also planned in the Project Document although it is not required for this project, however, no budget was allocated for MTR as well as for the TE.


Tag: Global Environment Facility fund Resource mobilization Small Grants Programme Monitoring and Evaluation

25.

Implementation of M&E
The TE reviews the UNDP role and considers that it has been correctly applied to this project, based on the following notes:
- The UNDP/GEF Regional Unit and UNDP Cambodia’s provisions of financial resources have also been in accordance with project norms and in the timeframe.
- The UNDP CO has been active in (i) preparing all project progress reports including the annual and quarterly reports, (ii) preparing, discussing, and finalizing annual work plans in line with the UNDP/GEF guidelines, (iii) follow up in all financial payments and transactions, and (iv) providing the needed support to mobilize international consultants/advisors to support project implementation.
- The project’s M&E activities followed the UNDP/GEF established procedures as the UNDP CO and the UNDP/GEF RTA have conducted several monitoring exercises including preparation and review of the project progress reports, and participation in the project board meetings.
- The UNDP CO has helped the PMU in recruiting the needed international consultants in line with the established Rules and Regulations of the United Nations.


However, the M&E framework could have been strengthened due to the presence of several key challenges including; the absence of a clear adaptive management framework, the project’s slow and complicated start, and the absence of strategic guidance from the UNDP CO and UNDP/GEF during the project inception phase.


Tag: Monitoring and Evaluation Results-Based Management

26.

The following elements are identified in the project document as the principal components of monitoring and evaluation:
A Project Inception Phase (Inception Workshop and Inception Report): The Inception Phase is a key activity at the beginning of any UNDP/GEF project. The IW is used to introduce an understanding and ownership of the project’s goals and objectives among the project stakeholder groups. As mentioned above, a Project IW was organized almost 22 months after the signing of the Project Document, however, during the IW, the management structure was not discussed or nor modified, some changes were introduced to the Project Log-Frame26 , the limited discussion was done on the project’s annual work plan. Furthermore, the IR was a copy of the project document, it did not capture the discussion took place during the workshop, and hence, the Inception Phase (Workshop and Report) represent a substantial weakness in the project cycle.


Tag: Monitoring and Evaluation Ownership

27.

Annual Progress Reports (APRs) and Project Implementation Report (PIR). The APRs are UNDP monitoring tool while the PIR is a UNDP/GEF monitoring and reporting tool. The project has prepared and submitted three APRs for 2015, 2016, and 2017, and one PIR for the year 2017/2018. The APRs are used as a critical analysis of the project’s status and are submitted to the PB for review, discussion, and endorsement.
There have been limited revisions to the project LFA despite significant delay encountered by the project over its lifetime. Indeed, the quarterly and annual reports have been based upon the 7 Outputs rather than on 8 outputs as suggested during the IW. The project is focusing on delivering a few specific products, rather than addressing the issues of improving access to environmental information related to the Rio Conventions. This has affected the projects ability to link activities with outcomes and progress towards the project objective.


Tag: Communication Monitoring and Evaluation Quality Assurance

28.

Quarterly Progress Monitoring (QPRs); the QPRs are used to report on progress made based on the UNDP Enhanced Results-Based Management Platform. The TE evaluator did review all Project’s QPRs (3 for 2016, 3 for 2017 and 2 for 2018) and has observed that although the project has managed to submit all needed QPRs, key information was missing like the updated risks logs, and the reporting was at the outputs level using the original outputs (7 outputs instead of 8) with a focus on describing the activities to a great extent.


Day-to-day monitoring of implementation progress is the responsibility of the PMU based on the project’s AWP and its indicators.
Mid Term Review and Final Evaluation: the MTR was concluded in August 2017 as a result of the monitoring visit conducted by the UNDP/GEF RTA in 201727. The TE was organized to take place during the last three months of the project’s operation in accordance with UNDP and GEF requirements.


Project Terminal Report (PTR). The PTR is a full report that summarized all activities, achievements, and outputs of the project, lessons learned, the extent to which objectives have been met, structures and mechanisms implemented, capacities developed, among others. This report should be prepared during the last three months of the project implementation and to be discussed during the terminal review meeting. Ideally, this report should be prepared by the Project team who has overseen all project’s operational issues since its inception. However, the TE consultant noted that the CTS is preparing it and the PTR is scheduled to be delivered by mid-December, to be presented to the PB for the terminal review.


Tag: Monitoring and Evaluation Quality Assurance Results-Based Management

29.

Terminal review meeting. The terminal reviewing meeting will be organized by the project board, with the participation of its members, in December 2018. The terminal review meeting will discuss the PTR that is under preparation by the CTS.


Tag: Monitoring and Evaluation Oversight

30.

3.2.6 UNDP and Implementing Partner implementation/execution coordination, and operational issues (*)
UNDP implementation
The Senior Supplier is UNDP as GEF Implementing Agency. UNDP has the Project Assurance role, which supports the Project Board Executive by carrying out objective and independent project oversight and monitoring functions. The key aspects of the UNDP implementation are as follows:
- UNDP followed up on the Project and continuously examined if it is being implemented with a focus on project activities.
- The UNDP support to the PMU is regarded as highly satisfactory and, in many cases, timely:
• There have been significant number of monitoring and review exercises conducted by the UNDP Cambodia including preparation of the Annual Project Review /Project Implementation Review reports, and production of the Combined Delivery Report.
• The UNDP has also been active in reviewing and following up on the project’s QPRs, financial reports, and project AWPS.


Tag: Efficiency Operational Efficiency Oversight Coordination

31.

GSSD/ MOE Execution
The project followed the full NIM modality; executed by the GSSD/MOE and implemented by the UNDP CO in Cambodia through a PMU with the support of a group of national and international consultants.


The DBD/GSSD/MOE was appointed to serve as Executing Agency. A National Project Director (NPD) was appointed and is actively responsible for financial management and disbursements with accountability to GSSD/MoE and UNDP. According to the Project Document, the Executive is represented by a senior official of GSSD/MoE, as an individual representing the project ownership to chair the group. The Senior Beneficiary is the Departments of Biodiversity and Climate Change, GSSD/MoE and the Department of International Cooperation, MAFF act as the Senior Beneficiary of the Project.

 


Tag: Implementation Modality Oversight Country Government

32.

3.3 Project Results
3.3.1 Overall Results (attainment of objectives) (*)

The achievements of expected results were evaluated in terms of attainment of the overall objective as well as identified project’s outcomes and outputs, according to the UNDP/GEF evaluation guidelines. For this the performance by outcome is analyzed by looking at (i) general progress towards the established baseline level of the indicators; (ii) actual values of indicators by the end of the Project vs. designed ones; (iii) evidences of relevance, effectiveness, and efficiency of the results as well as how this evidence was documented.


The summary of an evaluation of the attainment of objective and outcomes of the Project are presented in Table 6. The assessment of progress is based on observations, findings, and data collected during the field mission in Cambodia, interviews with key stakeholders, data provided in the annual reports, technical reports reviewed. The progress at the output level is provided in Annex 9.


The Capacity Development Monitoring and Evaluation Scorecard was conducted in 2014 during the formulation of Project. The results of the assessment were considered as a baseline in the revised Log-Frame. The MTR discussed the capacity development in the scorecards but did not provide any rating. The rating of the assessment of achievement at the time of the TE is presented in Annex 10.


Tag: Results-Based Management

33.

3.3.2 Relevance (*)
All evidence – during the TE mission- showed that the project is very relevant to the government and addressed the highly regarded topic. The key stakeholders interviewed during the mission expressed the added value of the project and emphasized that another phase or a second phase to follow up on the Piloting site (Orchid Gardens) and to use properly the environmental economic valuations is critical. To the TE consultant opinion, the Project managed to improve national capacity and awareness pertaining to biodiversity, land degradation, and climate change and the relevant international conventions. It also managed to develop a system that integrates all info related to the RIO Conventions into a single system and that would contribute to increase and enhance effective analysis, reporting, and information sharing and data access.

 


Tag: Climate Change Adaptation Biodiversity Natural Resouce management Relevance Communication SDG monitoring and reporting

34.

3.3.3 Effectiveness and efficiency (*)
Effectiveness

The Project has made tangible progress towards the achievements of its overall objective “to improve access to environmental information related to the Rio Conventions through the harmonization of existing environmental management information systems and improvement of coordination of the implementation of these conventions in Cambodia. It specifically helped in “improving the implementation of the Rio Conventions in Cambodia through the development of national capacities to better coordinate and generate better information related to the implementation of these conventions.” The Project objective and main outputs have been achieved; the most of established targets have been met. However, not all targets were achieved within the implementation period, and some of them are planned to be achieved by project closure.
Considering the above-mentioned facts, Effectiveness was rated Moderately Satisfactory.
 


Tag: Effectiveness Communication Capacity Building Coordination

35.

 

The rating for project Efficiency is Moderately Satisfactory for the following reasons:

• Major project results have been achieved in 4 years, however, the quality of some of the project’s results was not vetted nor endorsed by national and international experts as many of them will be delivered during the last month of project implementation.
• The hosting of the project within the BDB/GSSD/MOE premises with other UNDP and other donors funded projects enhanced the projects’ efficiency and facilitated its work and cooperation with different projects and their stakeholders like GIZ, and UN Environment.
• The international consultants and the project technical specialists were able to provide the needed technical backstopping and develop some critical outputs during the project implementation, however, the timeline is not in line with the original plans.


Furthermore, the Project has managed to leverage around 73% of in-kind financial resources (from the Government), and more than the planned UNDP cash contribution (147.67%).


Tag: Efficiency Resource mobilization Operational Efficiency

36.

3.3.4 Country Ownership
As per the project document, “Cambodia ratified the United Nations Convention on Biological Diversity (UNCBD) on February 9, 1995, and the United Nations Framework Convention on Climate Change (UNFCCC) on December 18, 1995, and the United Nations Convention to Combat Desertification and Drought (UNCCD) on August 18, 1997.”30 Furthermore, Cambodia ratified other important protocols under the Rio Conventions in later years, namely “The Cartagena Protocol on Biological Safety to protect biodiversity from the potential risks posed by genetically modified organisms that are the product of biotechnology; The Kyoto Protocol; Cambodia acceded to the Nagoya-Kuala Lumpur Supplementary Protocol on Liability and Redress on remedial measures arising from damages caused by the transboundary movement of living modified organisms on August 30, 2013.; and Cambodia signed the Nagoya Protocol on Access to Genetic Resources and the Fair and Equitable Sharing of Benefits Arising from their Utilization on February 1, 2012.”31

 


Tag: Natural Resouce management Sustainability

37.

3.3.5 Mainstreaming
Based on project documents reviews and the observations collected during the TE mission to Cambodia, it was evident that the Project addresses UNDP priorities of developing the Government’s capacity to comply with the Rio Conventions implementation and obligations in national plans. The Project was able to positively mainstream several UNDP priorities. In particular:


- Some frameworks and tools have been developed and endorsed such as the EIMS, CHM, a 5-year capacity development plan to enhance synergy among the Rio Conventions implementation, a communication and partnership strategies for the 3 Rio conventions, and the Roadmap and action plan to implement partnership and communication strategy.
- The Project developed the capacity and provided support to Cambodian delegation by enhancing evidence-based papers for negotiations in international meetings
- The Project objectives conform to agreed priorities in the UNDAF and National Development Plans.
- The Project managed to target provinces in order to pilot the produced tools. In the piloting site.
- The project accounts in a limited manner for gender differences when developing and applying project activities; however, as the primary focus of the project is improving access to information which is gender neutral, it is not inappropriate to limit gender considerations to those project components which do have a gender impact.
- International and national consultants included both women and men (43% of the consultants were women).
- Around 25% of the project leadership position were women.
- The Project targeted both women and men in its capacity building and public awareness components. The project document stated that “the project will take steps to ensure that women account for at least 40% of all training and capacity building in the project.” Based on data provided by PMU, it was noticed that the project was almost successful in achieving this target by including 38.8% women in its training and capacity building initiatives. Lists of all project’s activities indicating the total number of women and men are included in Annex 7.


Tag: Gender Mainstreaming Gender Parity Communication Capacity Building

38.

3.3.6 Sustainability (*)
The Project’s main approach to sustainability is to strengthen capacities to implement and manage global convention guidelines and enhance generation, accessing and use of information and knowledge. The project’s exit strategy should be dependent on the continuation of commitments and activities without the need for long-term international financing. However, it was noticed that the Project does not have an exit strategy. A set of activities are defined to ensure the sustainability of the Project’s results after the project life, but those were not developed as a comprehensive strategy as stated in the Project Document “An exit strategy will be prepared 6 months before the end of the project to detail the withdrawal of the project and provide a set of recommendations to the government to ensure the long-term sustainability and the up-scaling of project achievements to other parts of Cambodia.”


As stated in the UNDP-GEF guideline for TE, sustainability is generally considered to be the likelihood of continued benefits after the project ends. Consequently, the assessment of sustainability considers the risks that are likely to affect the continuation of project outcomes.


Tag: Sustainability Communication Knowledge management

39.


Financial risks
There is only one financial risk related to mobilizing needed resources to ensure the finalization of key project’s activities such as the finalization of the work related to the CHM, EIMS and the work at the piloting sites which needs substantial financial resources to ensure data collection, screening, and sharing with the CHM and IMS. The operationalization of key project’s deliverables, which are supposed to be finalized during the last month of the project implementation, needs financial resources to ensure implementation. Nevertheless, the project established two cooperation mechanisms with the RUPP and the DBD/GSSD/MOE in order to continue the work in the piloting site and make sure that the tools developed is going to be utilized and tested.


Tag: Sustainability Resource mobilization

40.

Socio-economic risks
No significant social risks were not identified by the project, or in the project document. However, as some of the project’s deliverables concerning the environmental economic valuation tools might be implemented in the piloting site, this might have an impact on the stakeholders and local communities in the piloting area. Thus, a detailed assessment of the socio-economic impacts should be taken into consideration.


Tag: Impact Sustainability Service delivery

41.

Institutional framework and governance risks
As identified in the Project MTR report, the Project’s “deliverables need to link to the new GSSD/MOE structure and be anchored for institut8ioanl sustainability”33. Although DBD/GSSD/MOE is interested to continue the work of the project and the Project’s outcomes have already established the needed institutional capacities and infrastructure that would ensure the project’s outcomes on sustainability, the need to link these outcomes/deliverables to the DBD/GSSD/MOE work is still missing.

Environmental risks to sustainability
There are no activities that may pose any environmental threats to the sustainability of the project’s outcomes.


Tag: Sustainability Results-Based Management Service delivery

42.

3.3.7 Impact
The Project has made major advances in the deployment of EIMS and CHM in Cambodia and building national capacities to advance the work on the Rio Conventions at the national level. Many outputs and Project’s results were first time achieved in Cambodia. The successful impact of the project is evident through;

  •  The concept on the Information Management System aiming at sharing environmental information among the RIO Conventions was designed, developed, and endorsed by the RIO Conventions Focal Points;
  • A comment web-portal playing a role as Clearing House Mechanism (CHM) is designed and launched. To support the rollout of the common web-portal, known as clearing house, the project also supports to produce (http://rio.picdaro.com/en/):
  • - Roadmap and action plan to implement partnership and communication strategy;
  • - 5-year capacity development plan to enhance synergy among the three Rio conventions and its implementation
  • - Communication and partnership strategies for the 3 Rio Conventions have been drafted.
  • ? The role of key stakeholders is upgraded through a Prakas (joint declaration between the institutions of the Rio Conventions focal points). This Prakas is being developed and expected to be endorsed by the end of this year. The Prakas aims at upgrading the role of the member of the technical working group (The technical working group composes of key stakeholders from line ministries and the Rio conventions). It is also expected to bring in more collaboration specifically on the Rio conventions thematic areas;
  • ? A technical paper was produced with focuses on the analysis of the environmental economic valuation tools produced. The relevant documents were used in some initiatives launched by the MoE/NCSD namely in the Ecotourism sector with a focus on the Orchid Research and Conservation Centre/ Siem Reap Province.
  • ? The project has mobilized technical support to Cambodia delegation, enhancing evidence-based paper for negotiations in more than 3 international meetings as listed below:
  • - 15 interventions on Cambodia positions related to biodiversity conservation and management were presented at SBSTTA 21, SBSTTA 22, 10th meeting of Working Group on Article 8j (WG 8(j)-10) and the 2nd meeting of Subsidiary Body on Implementation of the Conventions on Biological Diversity (SBI-2)
  • - 5 Cambodia statements were intervened in the 2nd, 3rd and 4th sessions of the Intergovernmental Science-Policy Platform on Biodiversity and Ecosystem Services (IPBES-2, IPBES-3, IPBES-4).
  • - Interventions were made during UNFCCC and UNCCD’s meetings and COPs; and
  • - Involvement in negotiation processes for 3 Rio Conventions’ Ad-Hoc, SBSTTA meetings and COPs such as CBD COP 13 in Cancun, Mexico in 2016; UNFCCC COP 23 in Bonn, Germany 2017; and UNCCD COP 13, Ordos, China in 2017 and their related meetings.

Tag: Ecosystem services Impact Communication Capacity Building Technical Support

Recommendations
1

As nine major Project’s products will be finalized by early December 2018 (a month before the project official closure) and will be discussed and endorsed at a national workshop during the third week of December, an urgent and clear plan of action needs to be developed to ensure the utilization of these products after 2018 to ensure Project’s outcomes sustainability. These products include the 5-years capacity development plan, the strategy to bring the synergy among the Rio conventions in its implantation, the Roadmap, and action plan to implement partnership and communication strategy, as well as the guidebook to produce a joint report for the Rio Conventions (UNDP, GSSD/MOE, and MAFF)

2

The Project has managed to produce a set of valued public awareness products on the Rio Conventions and is currently in the process to finalize the CEPA (Communication, Education, Public Awareness) package on the synergy among the Rio Conventions.  It is recommended to develop a dissemination plan for those public awareness and outreach tools as part of the DBD, DCC, and CCD future work, to ensure that future initiatives would build on the Project activities and results and will incorporate the project’s products in its work. (UNDP, GSSD/MOE and MAFF).

3

Despite the fact the TE was unable to review several Project’s products as they are not finalized yet, it is recommended that the remaining project training and piloting activities be completed as soon as possible, including piloting the testing of the economic valuation tools in Siem Reap Province that incorporating environmental economic valuations for ecosystem services, which aimed to demonstrate measurable indicators of delivering global environment benefits. (UNDP, and GSSD/MoE).

4

The work to improve access to data and enhance national capacities for Rio conventions implementing has just begun through this Project. It still at the early stages hence other UNDP and Government of Cambodia initiatives and projects should continue working on the upgrading of the national capacity, the infrastructure, and project’s deliverables produced to ensure that the Country will achieve the Project’s Objective (GSSD/MOE, UNDP, development partners, and donor agencies).

1. Recommendation:

As nine major Project’s products will be finalized by early December 2018 (a month before the project official closure) and will be discussed and endorsed at a national workshop during the third week of December, an urgent and clear plan of action needs to be developed to ensure the utilization of these products after 2018 to ensure Project’s outcomes sustainability. These products include the 5-years capacity development plan, the strategy to bring the synergy among the Rio conventions in its implantation, the Roadmap, and action plan to implement partnership and communication strategy, as well as the guidebook to produce a joint report for the Rio Conventions (UNDP, GSSD/MOE, and MAFF)

Management Response: [Added: 2019/07/23] [Last Updated: 2020/12/02]

Disagree

The National Partner has fundamental disagreement on the assessment conducted for Output 2.1 (find detail in the document named Disagreement from IP on the Terminal Evaluation Report). Hence the Project National Partner rejects the Terminal Evaluation, and all its findings.

Key Actions:

2. Recommendation:

The Project has managed to produce a set of valued public awareness products on the Rio Conventions and is currently in the process to finalize the CEPA (Communication, Education, Public Awareness) package on the synergy among the Rio Conventions.  It is recommended to develop a dissemination plan for those public awareness and outreach tools as part of the DBD, DCC, and CCD future work, to ensure that future initiatives would build on the Project activities and results and will incorporate the project’s products in its work. (UNDP, GSSD/MOE and MAFF).

Management Response: [Added: 2019/07/23] [Last Updated: 2020/12/02]

Disagree

The National Partner has fundamental disagreement on the assessment conducted for Output 2.1 (find detail in the document named Disagreement from IP on the Terminal Evaluation Report). Hence the Project National Partner rejects the Terminal Evaluation, and all its findings.

Key Actions:

3. Recommendation:

Despite the fact the TE was unable to review several Project’s products as they are not finalized yet, it is recommended that the remaining project training and piloting activities be completed as soon as possible, including piloting the testing of the economic valuation tools in Siem Reap Province that incorporating environmental economic valuations for ecosystem services, which aimed to demonstrate measurable indicators of delivering global environment benefits. (UNDP, and GSSD/MoE).

Management Response: [Added: 2019/07/23] [Last Updated: 2020/12/02]

Disagree

 

The National Partner has fundamental disagreement on the assessment conducted for Output 2.1 (find detail in the document named Disagreement from IP on the Terminal Evaluation Report). Hence the Project National Partner rejects the Terminal Evaluation, and all its findings.

Key Actions:

4. Recommendation:

The work to improve access to data and enhance national capacities for Rio conventions implementing has just begun through this Project. It still at the early stages hence other UNDP and Government of Cambodia initiatives and projects should continue working on the upgrading of the national capacity, the infrastructure, and project’s deliverables produced to ensure that the Country will achieve the Project’s Objective (GSSD/MOE, UNDP, development partners, and donor agencies).

Management Response: [Added: 2019/07/23] [Last Updated: 2020/12/02]

Disagree

The National Partner has fundamental disagreement on the assessment conducted for Output 2.1 (find detail in the document named Disagreement from IP on the Terminal Evaluation Report). Hence the Project National Partner rejects the Terminal Evaluation, and all its findings.

Key Actions:

Latest Evaluations

Contact us

1 UN Plaza
DC1-20th Floor
New York, NY 10017
Tel. +1 646 781 4200
Fax. +1 646 781 4213
erc.support@undp.org