Mid-Term Evaluation of Generating, Accessing and Using Information and Knowledge Related to the Three Rio Conventions

Report Cover Image
Evaluation Plan:
2016-2018, Cambodia
Evaluation Type:
Mid Term Project
Planned End Date:
09/2017
Completion Date:
10/2017
Status:
Completed
Management Response:
Yes
Evaluation Budget(US $):
30,000

Share

Document Type Language Size Status Downloads
Download document Rio_MTR_TOR.pdf tor English 1951.52 KB Posted 108
Download document Rio project_MTR report 2017_Final.pdf report English 2414.69 KB Posted 145
Title Mid-Term Evaluation of Generating, Accessing and Using Information and Knowledge Related to the Three Rio Conventions
Atlas Project Number: 00083830
Evaluation Plan: 2016-2018, Cambodia
Evaluation Type: Mid Term Project
Status: Completed
Completion Date: 10/2017
Planned End Date: 09/2017
Management Response: Yes
Focus Area:
  • 1. Environment & Sustainable Development
  • 2. Others
Corporate Outcome and Output (UNDP Strategic Plan 2014-2017)
  • 1. Output 1.3. Solutions developed at national and sub-national levels for sustainable management of natural resources, ecosystem services, chemicals and waste
Evaluation Budget(US $): 30,000
Source of Funding: Project budget
Evaluation Expenditure(US $): 19,000
Joint Programme: No
Mandatory Evaluation: No
Joint Evaluation: No
Evaluation Team members:
Name Title Email Nationality
Stephanie J. Hodge Independent Evaluation Expert shodge1@gmail.com
GEF Evaluation: Yes
GEF Project Title: Cambodia : Generating , Accessing and Using Information and Knowledge Related to the Three Rio Convention
Evaluation Type: Terminal Evaluation
Focal Area: Climate Change
Project Type: MSP
GEF Phase: GEF-5
GEF Project ID: 5295
PIMS Number: 5222
Key Stakeholders:
Countries: CAMBODIA
Lessons
1.

Some notable good practices and lessons learned in addressing issues relating to effectiveness, efficiency and relevance include the following:

  • The necessity and importance of having technically competent advisor support (with the right mixed skill set including legal and policy) at the original inception report and workshop. For the project, the need was for a TS with competencies in information management systems, interoperability and information system development linking data management to the standards and reporting requirements of the three Rio Conventions and especially legal skill and knowledge. Thus there was a need for competency on IMS, gender mainstreaming, and IT and capacity building/knowledge management for three Rio’s (MEA) data management. Such competencies were needed to clarify the expected results to all stakeholders at the project onset;
  • Importance of informal ways to highlight strategic partnerships for implementation, e.g. MOU with sectors for implementation;
  • Lesson on the importance of strategic learning approaches, knowledge management and communications for capacity building goals and longer term sustained results;
  • Leveraging synergetic (other) UNDP international project expertise for system level results.

Findings
1.

Project Strategy:

Finding 1 - Highly relevant

Cambodia ratified the United Nations Convention on Biological Diversity (UN-CBD) on 9 February 1995, the United Nations Framework Convention on Climate Change (UNFCCC) on 18 December 1995, and the United Nations Convention to Combat Desertification (UNCCD) on 18 August 1997. The project goals are linked to the national development plan and to UNDP’s global work environmental sustainability and localization of SDGs.

Although the UNDP/GOC CPD program target year of 2015 has passed, the project is contributing to UNDAF priority 1 – Sustainable, Inclusive Growth and Development. It was directly in line with CPAP Country Programme Output 1.2 Scaled-up action on national program for climate change adaptation and mitigation across sectors that is funded and implemented, targeting the most vulnerable poor populations.

The project is contributing to CPD/CPAP 2016–18

This project feeds neatly into compliance with global processes. Many of its 17 Sustainable Development Goals and 169 targets are relevant to the three Rio Conventions. The government has created a working group for localizing SDG in the government development plan. This project directly contributes to these objectives. With synergies enacted (a problem noted to date with implementation) and it can make a valuable contribution of the environmental indictors with these projects’ expected process and outputs.


2.

Project Strategy:

Finding 2- Design, Project outputs and outcome linkages are not clear.

The project strategy has been described in the inception report. The document describes the outputs and indicators and justifies them, but it does not make a strategy with interlinkages between outcomes and outputs. The strategy is missing the story of the interlinkages and road map toward overarching vision. Project strategy is planned to address priority actions identified through the NCSA (2005–2006), including targets on instilling the broader public use of environmental information in decision-making and policy-making, better coordination, mainstreaming environmental management into national policies, and plans and programs to advance Cambodia’s national interests on environmental matters in international forums. Stakeholders generally meet through project board meetings, and the evaluator found that a common understanding toward the outcome vision and mechanisms for broader goals and a description of work processes are not commonly understood across stakeholders interviewed. A short strategy can be prioritized with support of a competent Technical Advisor/Specialist TA as a guide and a presentation of the project concept, vison and change process road map is needed. The inception workshop and report did not succeed to do this, i.e. bring everybody together with a clear understanding of the way forward and end result. While the target beneficiaries are the Three Rio focal points as immediate benefiters of the capacity building and systematized CHM and information gathering, project goals are much broader. They should address the need for capacity building of all decision makers with stake in environmental information management issues, including the public. The project implementation is currently limited to a discussion with the direct beneficiaries and does not have a strategy or mechanism in place for the broader learning goals (intersectoral and public).


3.

Project Strategy:

Finding 3 - Technical advice gaps and conceptual issues and use of language are confusing, i.e. to the stakeholder platform, CHM and/or information management (IM) systems, etc.

The strategy and implementation to date is lacking technical guidance. The day-to-day work also requires clear technical language and support, for instance, translating how the project deliverable will add value to Cambodia’s environmental information management and environmental decision-making beyond the targeted focus on capacity strengthening support to the convention focal points. There are conceptual understanding gaps. The project framework needs to be clear for anybody to understand. The understanding of expected project outcome was not clear, based on the stakeholders interviewed. This limited vision of information management for Three Rio’s vs Cambodia’s environmental management system and the link to decision-makers in the context of environmental information management systems is expected to be corrected when a Technical Advisor/ Specialist is on board and a technically vetted implementation plan is developed.


4.

Progress Towards Results:

The will to work toward and support project results was high across all the core project stakeholders interviewed, i.e. members of the steering committee and others. This project has two focused outputs - IMS/CHM and sets out a course toward environmental information management systems, and learning to support environmental management and environmental literacy are cases in point. While the project just started implementation and the Technical Advisor/Specialist has yet to be recruited, there were achievements, including MOE/GSSD capacities gained to implement GEF projects through the NIM modality. Four baseline deliverables were produced (two from the MEA expert and two from the CDA), but these need to be technically vetted before they can be advanced. The capacity development work in particular is still missing a strategy and training plan. Evaluation noted that certain areas in the assessment are weak, i.e. legal scoping of existing policies and laws on information management and legalities of the cross-sectoral obligations for information management. It also needs anchoring to the new GSSD/MoE processes.

Finding 1 - First full NIM project delay in operational Start

The project experienced a long delay, and according to the time line (annex) is at least one year and half late. This project needs to be rescheduled. Early after the project signing, UNDP conducted a HACT assessment. The delay issue, however, is understandable, given that was the first full UNDP/GEF NIM project with the DBD-GSSD/MoE and had a steep learning curve and time consuming negotiations over work planning and the quality assurance role of UNDP. Evaluator learned that this role was not discussed and understood before proceeding. The startup delays were resolved. UNDP recruited a more conducive focal point for the monitoring role and for relationship building. The joint monitoring and oversight arrangement is currently working. Eleven meetings have taken place and fundamental activities have transpired.


5.

Progress Towards Results:

Finding 2- Project Management Unit need for technical guidance and support

While the project coordinator is doing well to advance the project, he requires technical guidance and support to work with high level stakeholders and the broader stakeholders’ convening and strategies. While implementation is advancing, there are concerns raised by the board members and knowledgeable stakeholders as to the rigor of the overall implementation plan, the products emerging from contracts and the sequencing of activities. For instance, while three baseline deliverables are completed: review on capacity building tools, scoping of needs of capacity, and MEA inventory, their quality needs vetting by CDA. Also, there is a need for a discussion on producing indicators for management and environmental literacy. For outcome level results, interlinkages need to be made between the two outcomes for sustainability: outcome 1 - system development and outcome 2 - capacity development.

 

The board members interviewed project consensus on vision of the overarching expected outcome and guidance on how to best interact (use of theirs and others’ time). This project needs technical guidance including an implementation strategy/road map that outlines the linkages between outputs and outcomes, including a mechanism for the technical deliverables, broader stakeholder engagement and partnership/capacity building strategy, i.e. scheduling of activities towards outcome level results and system changes. While the focus is to build a system that supports implementation of three conventions, the title and targeted focus as to the uses and capacity of the three focal points are restrictive as to what the project can contribute to Cambodia’s environmental management and sustainable development. The scoping of indicators, for example, should be a multi-stakeholder and technically led process linked to the state of the environmental reporting processes and the SDG planning.


6.

Progress Towards Results:

Finding 3: The project institutional context has changed positively and supports project goals.

This project needs to link to the work of the GSSD/MOE. The project context has changed and is benefiting from a new institutional context. In 2015, the GSSD/MOE was instituted by government decree, and this project can feed neatly into the needs for the GSSD/MOE. The project needs more visibility aligned with the GSSD/MOE institutional mandate and the overarching technical intersector committee can be led at the level of the NSCD executive.

In May 2015, responding to the GSSD/MOE’s structural reform, Department of Climate Change (DCC) substituted for CCD has been moved under the National Council for Sustainable Development-General Secretariat (GSSD). It is mandatory to promote sustainable development aimed at ensuring economic, environmental,


7.

Progress Towards Results:

Finding 4- Product vs process towards institutionalized systems is not apparent.

Outcome 1 includes two key deliverables, including information management system design with recommendations for a scientifically and legally vetted suggested cross-sectoral indicators for information management collection and use and the development of Three Rio CHM, a one stop shop harmonizing work that is already advanced. The legal ramifications, cost of the new information management, data systems use and a road map for expansion and maintenance over the next decade, for example, can also be helpful in the exit strategy. The project needs an exit strategy. This needs thinking at MTE.


8.

Progress Towards Results:

Finding 5 - Completed CHM inventory work needs to be vetted technically

The quality of the products produced, including the inventory, need to be technically vetted, i.e. cross-sectoral legalities and a costs/action plan of IM still need to be scoped through an intersectoral consultation process. The project has produced an inventory, but as a product vs a process towards a future institutionalized system. The work is not linked to an institutional enhancement change process, i.e. GSSD/MoE. The current understanding of the project end goals is limited to a CHM, which is the interface and information dissemination container for existing content as opposed to a government-led and -mandated data collection system. (This is a future goal).


9.

Progress Towards Results:

Finding 6- Expected results are not significantly advanced. There is no CB strategy linked to output or overall expected outcome.

While a rudimentary capacity needs assessment has been produced, it is lacking analytical analysis and practical project recommendations about the situation of environmental informational management capacities and what to do next for learning and uptake of the information by the primary beneficiaries and the broader public sector for their role in decision-making. The technical specialist will develop an action-oriented capacity development strategy with an exit strategy linked to the stakeholder engagement/partnership strategy.

A capacity needs assessment was implemented as Draft, Version 2c, May 2017


10.

Project Implementation & Adaptive Management:

Finding - NIM implementation is the optimal modality for learning by doing but needs technical support and learning to implement projects.

The project is implemented as first GEF/UNDP/Royal Government of Cambodia NIM with limited UNDP support. There were two previous NIM with embedded UNDP support, TA and procurement. As such, the agencies are learning how to work together in a new way. The CCCA operational manual, while agreed to be a guide for implementation during the inception meeting, was questioned by UNDP.. To date the project team has not been able to leverage the UNDP comparative advantages to actively support the technically monitoring of  results, including providing consultants and engaging in oversight services and having the ability to bring the best international experiences, promoting south south experiences and hiring  technical support ..

On a good note, based on the review of the activities and program logic, there have been some capacity building and MAFF and  GSSD/MoE collaboration, including improving access and generation of information related to the three Rio Conventions. A project team is in place and the project board is functioning. The inventory is produced and has provoked the discussions among the board and the management team around the technical leadership need to guide the board. With the arrival of a Technical Advisor/Specialist TS, this outcome is expected to advance. For the inventory and indicators, the CBD localization work has advanced, as has the CCCA; however, evaluator finds that the project may need to do more work with the UNCCD supported by TS.


11.

Sustainability:

Finding - The expected outputs/deliverables (including the Partnership CB/KM Learning aspects) need to be linked to the new GSSD/MoE  structures and anchored for institutional sustainability.

Currently the lack of integration of project output goals to the  GSSD/MoE  institutional processes is a risk. With the TS on board, it is expected that this risk will be reduced substantively.

The project beneficiaries are currently being restricted to the focal points of the three Rio Conventions, and while the targeted beneficiaries of the training and support system include the CHM, an improved environmental information management system requires building capacity of all stakeholders to engage in development of and learning about the conventions. Ultimately project actions are expected to support the broader environmental management goals of the  GSSD/MoE/MAFF. This project needs a strong multi-stakeholder partnership strategy and a capacity development plan for engaging partners and a broader group of stakeholders in implementation. This can be linked to the project learning, knowledge management and information sharing goals (see evaluator’s recommendation on knowledge management and strategic partner networking).

The project needs an exit strategy linked to the longer-term vision of the system, including the intersectoral data collection and knowledge sharing implications, and linked to the environmental code and the state of the environmental reporting.


Recommendations
1

Recommendaton 1: GSSD/MoE formally links project implementation to the GSSD/MoE/MAFF processes related to information and knowledge management.

2

Recommendatoin 2: GEF/UNDP/GSSD-MoE reschedule project in line with the late implementation start, i.e. Oct. 2016. The Project, therefore, needs to be extended for 1.5 years (31 December 2018).

3

Recommendation 3: PMU develop an infographic vision statement to articulate the expected outcomes in a short guide to communicate the overall expected outcome and road map with two components to all stakeholders. The project implementation strategy is limited in the area related to technical oversight and guidance on interpretation of the results and is missing important inclusions of stakeholders and institutional linkages for sustainability and uptake of IMS and CHM toward results between output one and output two in addition to the events already organized such as IDB

4

Recommendation 4: UNDP immediately hire a qualified (chief) Technical Specialist to support technical implementation, develop an implementation road map in line with this MTE report, set up the suggested implementation platforms and oversee processes linking work between the two outputs and outcome; the Technical Specialist take full responsibility for outcome 1 - concrete deliverables including the development of the IMS and the CHM and outcome 2 - ensuring the comprehensive capacity development and international and national training plans and strategy for a CHM and learning pilot. The Technical Specialist should engage in all project results monitoring and oversight.

5

Recommendation 5: PMU continue to encourage the knowledge management learning officer and recruit partnership, gender mainstreaming strategy consultants and IT expert/team to support the Project Director and GSSD, Coordinator for developing and advancing the Three Rios Partnership/networking strategy and operationalizing project GSSD/MoE social/networking, cross-sectoral and public learning agenda and development of partnership engagement and gender mainstreaming strategies, and website(s). This work includes managing activities for broader government and public learning about the Rio Conventions. The project KM learning officer can be made focal point for the strategic project communication, i.e. newsletter and broader uptake of the CHM through making linkages to the future CHM pilot project, dissemination of learning guides and working with partners for the CHM training of trainers, environmental education in general and project activities

6

Recommendaton 6: The project KM learning officer become focal point for the strategic project communication, i.e. newsletter and broader uptake of the CHM through making linkages to the future CHM pilot project, dissemination of learning guides and working with partners for the CHM training of trainers, environmental education in general and project activities.

1. Recommendation:

Recommendaton 1: GSSD/MoE formally links project implementation to the GSSD/MoE/MAFF processes related to information and knowledge management.

Management Response: [Added: 2017/12/18]

Agree

Key Actions:

Key Action Responsible DueDate Status Comments Documents
Detailed plans (Roadmap) to achieve this recommendation will be drafted, validated by PMU, 3 Rio focal persons and project management, and implemented
[Added: 2017/12/18] [Last Updated: 2018/05/18]
PMU/GSSD and MAFF with support from Project Management 2018/01 Completed Project detail workplan and approach to move toward common information system needs to consider the fact that each of the convention is currently at the different levels of capacity was produced and endorsed during the project board meeting in Q1 2018. History
2. Recommendation:

Recommendatoin 2: GEF/UNDP/GSSD-MoE reschedule project in line with the late implementation start, i.e. Oct. 2016. The Project, therefore, needs to be extended for 1.5 years (31 December 2018).

Management Response: [Added: 2017/12/18]

Agree

Key Actions:

Key Action Responsible DueDate Status Comments Documents
Submit the request for project expanded timeframe up to December 2018; - Finalized MTR report to be used as the supporting document to the request for timeframe expansion.
[Added: 2017/12/18]
PMU/GSSD, UNDP, Project Management 2017/11 Completed MTR report finalized and project extension approved.
3. Recommendation:

Recommendation 3: PMU develop an infographic vision statement to articulate the expected outcomes in a short guide to communicate the overall expected outcome and road map with two components to all stakeholders. The project implementation strategy is limited in the area related to technical oversight and guidance on interpretation of the results and is missing important inclusions of stakeholders and institutional linkages for sustainability and uptake of IMS and CHM toward results between output one and output two in addition to the events already organized such as IDB

Management Response: [Added: 2017/12/18] [Last Updated: 2017/12/18]

Agree

Key Actions:

Key Action Responsible DueDate Status Comments Documents
Develop draft of the required document; - Validate the draft with PMU/GSSD, 3Rio focal Points and project management; - Disseminate the document to the key project beneficiaries, specifically each of the Rio convention focal point.
[Added: 2017/12/18] [Last Updated: 2018/08/15]
PMU/GSSD with support from Technical Expert, Project Management 2018/01 Completed Roadmap and implementation strategy and institutional linkage were reflected in the Project detail workplan developed and endorsed during the project board meeting in Q1 2018. History
4. Recommendation:

Recommendation 4: UNDP immediately hire a qualified (chief) Technical Specialist to support technical implementation, develop an implementation road map in line with this MTE report, set up the suggested implementation platforms and oversee processes linking work between the two outputs and outcome; the Technical Specialist take full responsibility for outcome 1 - concrete deliverables including the development of the IMS and the CHM and outcome 2 - ensuring the comprehensive capacity development and international and national training plans and strategy for a CHM and learning pilot. The Technical Specialist should engage in all project results monitoring and oversight.

Management Response: [Added: 2017/12/18]

Agree

Key Actions:

Key Action Responsible DueDate Status Comments Documents
Draft ToR and launch recruitment of Technical Specialist.
[Added: 2017/12/18] [Last Updated: 2018/05/18]
PMU/GSSU/UNDP, Project Management 2017/11 Completed The Technical Specialist was recruited and on board in November 2017. History
5. Recommendation:

Recommendation 5: PMU continue to encourage the knowledge management learning officer and recruit partnership, gender mainstreaming strategy consultants and IT expert/team to support the Project Director and GSSD, Coordinator for developing and advancing the Three Rios Partnership/networking strategy and operationalizing project GSSD/MoE social/networking, cross-sectoral and public learning agenda and development of partnership engagement and gender mainstreaming strategies, and website(s). This work includes managing activities for broader government and public learning about the Rio Conventions. The project KM learning officer can be made focal point for the strategic project communication, i.e. newsletter and broader uptake of the CHM through making linkages to the future CHM pilot project, dissemination of learning guides and working with partners for the CHM training of trainers, environmental education in general and project activities

Management Response: [Added: 2017/12/18] [Last Updated: 2017/12/18]

Partially Agree

Key Actions:

Key Action Responsible DueDate Status Comments Documents
PMU produces the draft project with support from the TS and with involvement from 3 Rio focal points as well as project management. - Draft and finalize national consultants ToRs; - Launch recruitment process; - Implement the project in line with the agreed implementation plans
[Added: 2017/12/18] [Last Updated: 2018/05/18]
Technical Specialist/PMU/GSSD, Project Management 2018/01 Completed All positions were recruited. History
6. Recommendation:

Recommendaton 6: The project KM learning officer become focal point for the strategic project communication, i.e. newsletter and broader uptake of the CHM through making linkages to the future CHM pilot project, dissemination of learning guides and working with partners for the CHM training of trainers, environmental education in general and project activities.

Management Response: [Added: 2017/12/18]

Partially Agree

Key Actions:

Key Action Responsible DueDate Status Comments Documents
Develop knowledge and information sharing plan at the scale of the project; - PMU develop the plan, get endorsement from project management. And seek for guidance and support from project management to identify the focal person/department; - Take action to formally link to the focal person/department; - Maintain engagement with the focal person/department.
[Added: 2017/12/18] [Last Updated: 2018/08/15]
Technical Specialist, PMU/GSSD, Project Management 2018/08 Completed The agreement in principle was reached and the concept was produced in regards to the CHM for the 3 Rio conventions. Each of the 3Rio focal team agreed to be part of the CHM and to share the information on the two common thematic areas under 3Rio including 1) Sustainable land used management and restoration of ecosystem; and 2) CEPA (Communication, Education and Public Awareness). The draft of the Praka to ensure that each of the Rio convention continue their collaboration. It is expected that the Prakas will be endorsed by the end of Q3 2018. History

Latest Evaluations

Contact us

220 East 42nd Street
20th Floor
New York, NY 10017
Tel. +1 646 781 4200
Fax. +1 646 781 4213
erc.support@undp.org