Integrating Rio Conventions Provisions into Ukraine’s National Policy Framework

Report Cover Image
Evaluation Plan:
2018-2022, Ukraine
Evaluation Type:
Final Project
Planned End Date:
03/2018
Completion Date:
03/2018
Status:
Completed
Management Response:
Yes
Evaluation Budget(US $):
12,000

Share

Document Type Language Size Status Downloads
Download document ToR_ TE Rio project Ukraine.docx tor English 69.04 KB Posted 70
Download document PIMS 4478 Evaluation of RIO Conventions Project_FINAL1.docx report English 2150.25 KB Posted 74
Title Integrating Rio Conventions Provisions into Ukraine’s National Policy Framework
Atlas Project Number: 00074532
Evaluation Plan: 2018-2022, Ukraine
Evaluation Type: Final Project
Status: Completed
Completion Date: 03/2018
Planned End Date: 03/2018
Management Response: Yes
Focus Area:
  • 1. Environment & Sustainable Development
  • 2. Others
Corporate Outcome and Output (UNDP Strategic Plan 2014-2017)
  • 1. Output 1.4. Scaled up action on climate change adaptation and mitigation across sectors which is funded and implemented
Evaluation Budget(US $): 12,000
Source of Funding: Project budget
Evaluation Expenditure(US $): 9,000
Joint Programme: No
Joint Evaluation: No
Evaluation Team members:
Name Title Email Nationality
Elinor Bajraktari
Viktor Karamushka
GEF Evaluation: Yes
GEF Project Title: Capacity Development: Integrating Rio Convention Provisions into Ukraine’s National Environmental policy Framework
Evaluation Type: Terminal Evaluation
Focal Area: Climate Change
Project Type: MSP
GEF Phase: GEF-5
GEF Project ID: 4913
PIMS Number: 4478
Key Stakeholders:
Countries: UKRAINE
Lessons
Findings
Recommendations
1

Evaluation Recommendation 1: Strengthen Engagement with SDGs at the Sub-national Level

The Rio Project has established good foundations for further UNDP engagement with the SDGs, which the CO has already begun to build on. It has also contributed to UNDP’s partnership with local communities and governments in the pilot areas. The project has showcased that the combination of SDGs and local development carries significant potential for UNDP, as has been argued in more detail in Chapter 4 of the report. UNDP should continue to strengthen its support for public institutions, particularly at the sub-national level, to adapt SDG targets and indicators to local circumstances, establish monitoring systems and create databases for monitoring progress, and report results nationally and internationally. It should also continue to explore ways of getting international donors more closely engaged with the SDGs. Furthermore, efforts should be made to link all this work more closely with the broader reform agenda led by the National Reform Council.

2

Evaluation Recommendation 2: Strengthen Synergies and Linkages between Projects

Drawing on lessons from the experience of the Rio Project, UNDP should further strengthen collaboration between projects, and where feasible establish integrated frameworks for project planning and implementation. The latter is certainly difficult, given the projects’ short timeframes and varying funding modalities, but it might be possible at the sub-national level if a sufficient number of projects are up and running. This will require the application of development of principles and methods for integrated ecosystem-based management. Using the example of collaboration between the Rio and SGP projects, UNDP could explore the establishment of a common platform for managing aspects of projects that share similar objectives. Such platform may combine not only elements related to information sharing, data systems, monitoring and evaluation, but also implementation tools such as systems for procurement, recruitment, awareness raising, etc. If such a platform is established across projects, it should be fully integrated with the CO’s results-based management system.

3

Evaluation Recommendation 3: Using the M&E System to Track Important Parameters

UNDP should examine how projects use their M&E systems to track important aspects of their work with a view to improving the availability of information for management purposes. Measuring some of these dimensions was a challenge in this evaluation. The following are a few worth considering.

  • Uptake of project outputs (studies, training, etc.) and the degree to which they serve their intended purpose – Projects should monitor the extent to which research and analytical documents they produce get incorporated into national policies and programmes.
  • Capacity of beneficiaries – Projects should also try to track the degree to which the capacity of participants taking part in the various training programmes organized by the project has improved.
  • Replication effects of pilot initiatives, the lessons they generate during the piloting stage and the extent to which the get scaled up – One key characteristic of pilots is that they serve to produce lessons which when shared lead to replication. They are key vehicles for transmitting experience and play a crucial role for upscaling and replication. However, it is not clear how the lessons are collected, analyzed, synthesized and shared. UNDP should develop a tracking mechanism for pilot initiatives, including documenting results, lessons, experiences and good practices. The tracking of pilots should be fully integrated into the CO’s results-based management systems and lessons learned and best practices should be managed as an important component of the CO’s knowledge management strategy.
  • Co-financing – The CO should strive for a more standardized definition of co-financing and monitor it more effectively by developing a tracking system at the project level.
1. Recommendation:

Evaluation Recommendation 1: Strengthen Engagement with SDGs at the Sub-national Level

The Rio Project has established good foundations for further UNDP engagement with the SDGs, which the CO has already begun to build on. It has also contributed to UNDP’s partnership with local communities and governments in the pilot areas. The project has showcased that the combination of SDGs and local development carries significant potential for UNDP, as has been argued in more detail in Chapter 4 of the report. UNDP should continue to strengthen its support for public institutions, particularly at the sub-national level, to adapt SDG targets and indicators to local circumstances, establish monitoring systems and create databases for monitoring progress, and report results nationally and internationally. It should also continue to explore ways of getting international donors more closely engaged with the SDGs. Furthermore, efforts should be made to link all this work more closely with the broader reform agenda led by the National Reform Council.

Management Response: [Added: 2018/07/13]

UNDP fully agrees with these recommendations and certain actions have already been made.

Specifically, UNDP has an agreement with two oblast administrations to prepare oblast-specific reports with the list of SDGs targets and indicators identified in the process of their adaptation the context. UNDP is developing its strategy on SDGs localization in the rest of oblasts of Ukraine, learning from experience in the first two oblasts. The local presence in Ukraine that UNDP has due to its recent projects (e.g., CBA), is used as an asset – namely, the contacts with local administrations, the newly signed Memoranda of Cooperation with many oblast administrations, other partner networks and resource centers established with support from UNDP in the past.

Furthermore, UNDP will explore cooperation with other donor-funded projects operating at the oblast level, especially those on decentralization.

Specific actions proposed below.

Key Actions:

Key Action Responsible DueDate Status Comments Documents
1.1 Develop a strategy for SDGs localization using the experience of localization in Volyn and Dnipro oblasts
[Added: 2018/07/13]
SDGs DST, ENV portfolio 2018/05 Completed Completed
1.2 Support the Government (MEDT and Statistics Service of Ukraine) in identifying best platform and mechanisms to organize regular monitoring of SDGs implementation at the national and oblast levels.
[Added: 2018/07/13] [Last Updated: 2019/05/02]
UN SDG task force, UNDP CD, SDGs DST, SDG advisor 2019/11 Initiated The consultant to design the platform has been hired, will hold consultations with the Government during June-July and will complete the design in August- September 2019. History
1.3 Provide to the Government (MEDT and Statistics Service of Ukraine) expert support on development on SDGs indicators which can be used at oblast level as an alternative to those national SDGs indicators which are not applicable at the oblast level
[Added: 2018/07/13]
GIZ/UNDP project, SDGs DST 2018/05 Completed Through the GIZ/UNDP
1.4 Intensify talks with donors who can provide technical assistance to support national SDGs process. Two-fold task: resource mobilization or integration of SDGs into current donor-funded projects
[Added: 2018/07/13] [Last Updated: 2018/12/18]
UNDP CD, Strategic Unit, SDGs DST 2018/11 Completed Per 2018/11, over $1.5 million mobilized from Sida, GIZ and SlovakAid on e-learning on SDGs, local SDG implementation, sharing of international practices on SDG implementation, and strengthening the work of Parliament of SDGs. History
1.5 Intensify advocacy for inclusion of SDGs into the agenda of Ukraine’s development priorities
[Added: 2018/07/13] [Last Updated: 2018/10/26]
UNDP CD, SDGs DST 2018/10 Completed MAPS mission conducted in March – August 2018 gave a significant burst to this process. The cabinet of Ministers instructed in July 2018 the line ministries to align their annual plans to SDGs. The Ministry of Regional Development has requested inclusion of UNDP in working group on regional development indicators, with an intention to align regional development priorities (and monitoring indicators) to SDGs. History
1.6 Strengthen cooperation with development partners by going beyond information sharing and forging collaboration at the level of project activities
[Added: 2018/07/13] [Last Updated: 2018/12/28]
CO, SDGs DST 2019/11 Initiated Started with EBRD on 12/12/2017 History
2. Recommendation:

Evaluation Recommendation 2: Strengthen Synergies and Linkages between Projects

Drawing on lessons from the experience of the Rio Project, UNDP should further strengthen collaboration between projects, and where feasible establish integrated frameworks for project planning and implementation. The latter is certainly difficult, given the projects’ short timeframes and varying funding modalities, but it might be possible at the sub-national level if a sufficient number of projects are up and running. This will require the application of development of principles and methods for integrated ecosystem-based management. Using the example of collaboration between the Rio and SGP projects, UNDP could explore the establishment of a common platform for managing aspects of projects that share similar objectives. Such platform may combine not only elements related to information sharing, data systems, monitoring and evaluation, but also implementation tools such as systems for procurement, recruitment, awareness raising, etc. If such a platform is established across projects, it should be fully integrated with the CO’s results-based management system.

Management Response: [Added: 2018/07/13]

UNDP mostly agrees with this recommendation, however establishing this platform and its effective management is a challenging. The CO will consider what are the opportunities.

In order to avoid silos and move beyond a projectized, ad hoc modality, UNDP has gradually moved towards a programme approach in its support to countries worldwide, including Ukraine. The next step in this regard is the switch towards a full area-based approach in supporting the roll-out of the sustainable development agenda at the subnational level where relevant. Specific actions are listed below.

Key Actions:

Key Action Responsible DueDate Status Comments Documents
2.1 Undertake a review of the operational processes in the UNDP Ukraine portfolios where common platforms and mechanisms can be applied. Assess which processes are efficient and which need improvement (procurement, grant management, interns engagement etc.). Clarify how human resources (project staff and consultants) in projects can be used more effectively, where they can contribute to multiple projects and or co-funded from peer projects.
[Added: 2018/07/13] [Last Updated: 2019/05/02]
UNDP CD, SDGs DST 2019/04 Completed In the follow-up to this recommendation and CO Audit in 2018, the office implemented a new SOP on grant management that is coherent across all portfolios. The capacities of the procurement and HR units have been strengthened with additional personnel to avoid operational bottlenecks. The portfolio has undergone significant restructuring and managerial and assistant positions are now shared between the projects, increasing overall efficiency of the implementation. History
2.2 Ensure regular review of the list of projects partners – to see where different projects work with the same beneficiaries and how to improve coherence of UNDP programmes
[Added: 2018/07/13] [Last Updated: 2018/10/26]
UNDP CD, SDGs DST, All portfolios 2018/09 Completed UNDP Ukraine has established the new SDG area based portfolio with a solid network of UNDP Oblast SDGs Coordinators, aimed at planning and implementing UNDP Ukraine activities at oblast-level. The proposed new way of operating together with existing area based networks with government, civil society, and other partners will allow UNDP Ukraine to act as a unified force, as a coordinated advocacy and implementation structure, as a central convening agency. A mapping of different UNDP projects and their implementation geography has been conducted and can form the basis for further area-based planning and implementation. History
2.3 Review the process through which new projects are designed and negotiated with partners to ensure that new projects built on experience, expertise and partnerships of earlier and ongoing UNDP projects in Ukraine
[Added: 2018/07/13]
Strategic unit 2018/02 Completed Regularly as part of a project concept development
2.4 Use SDGs as a cross cutting topic to be integrated into results frameworks for all UNDP projects. This will allow building synergies between the projects.
[Added: 2018/07/13] [Last Updated: 2018/10/26]
M&E and Strategic unit 2018/09 Completed UNDP Ukraine has established the new SDG area based portfolio with a solid network of UNDP Oblast SDGs Coordinators, aimed at planning and implementing UNDP Ukraine activities at oblast-level. The proposed new way of operating together with existing area based networks with government, civil society, and other partners will allow UNDP Ukraine to act as a unified force, as a coordinated advocacy and implementation structure, as a central convening agency. A mapping of different UNDP projects and their implementation geography has been conducted and can form the basis for further area-based planning and implementation. History
2.5 Improve communications activities by more frequent engagement with mass media and posting articles and op-eds about UNDP activities where UNDP’s activities are presented in integrated manner (as ‘delivering as one’ instead of delivering fragmented activities). SDGs, climate change, human rights, gender equality etc. can be a common underpinning topic which consolidates all activities under one logically consistent umbrella
[Added: 2018/07/13] [Last Updated: 2018/10/26]
Communications Unit, all projects 2018/09 Completed UNDP media presence has increased with more article and op-eds are published, covering many cross-functional topics at once. UNDP CO moved all projects websites to one UNDP Ukraine site. History
3. Recommendation:

Evaluation Recommendation 3: Using the M&E System to Track Important Parameters

UNDP should examine how projects use their M&E systems to track important aspects of their work with a view to improving the availability of information for management purposes. Measuring some of these dimensions was a challenge in this evaluation. The following are a few worth considering.

  • Uptake of project outputs (studies, training, etc.) and the degree to which they serve their intended purpose – Projects should monitor the extent to which research and analytical documents they produce get incorporated into national policies and programmes.
  • Capacity of beneficiaries – Projects should also try to track the degree to which the capacity of participants taking part in the various training programmes organized by the project has improved.
  • Replication effects of pilot initiatives, the lessons they generate during the piloting stage and the extent to which the get scaled up – One key characteristic of pilots is that they serve to produce lessons which when shared lead to replication. They are key vehicles for transmitting experience and play a crucial role for upscaling and replication. However, it is not clear how the lessons are collected, analyzed, synthesized and shared. UNDP should develop a tracking mechanism for pilot initiatives, including documenting results, lessons, experiences and good practices. The tracking of pilots should be fully integrated into the CO’s results-based management systems and lessons learned and best practices should be managed as an important component of the CO’s knowledge management strategy.
  • Co-financing – The CO should strive for a more standardized definition of co-financing and monitor it more effectively by developing a tracking system at the project level.
Management Response: [Added: 2018/07/13]

This recommendation is fully accepted as it is fully in line with what is required for quality assurance and ROAR reviews. RBM approach should be used to a larger extent - at the stage of project formulation and, with adjustments, at the stage of implementation and monitoring. Specific additional actions are proposed below.

Key Actions:

Key Action Responsible DueDate Status Comments Documents
3.1 To provide guidelines, advice and assistance on M&E frameworks and reporting to project staff on a regular basis
[Added: 2018/07/13] [Last Updated: 2018/12/13]
M&E unit, ENV Portfolio 2018/11 Completed Full-time M&E Officer (UNV) has been deployed to assist the projects within the portfolio with developing M&E frameworks, collecting outcome and output level data, reviewing reports and proposal to ensure they are result-oriented and comply with UNDP RBM standards. A comprehensive RBM training for the new portfolio team was conducted in November 2018. History
3.2 To monitor the extent to which research and analytical documents produced get incorporated into national policies and programmes
[Added: 2018/07/13] [Last Updated: 2018/12/13]
ENV Portfolio, M&E unit 2019/07 Initiated Instruments for tracking the extent to which research and analytical documents get incorporated into policies and programmes will be developed in 2019. History
3.3 Reinforce a tracking system for assessing capacity of participants taking part in the training programmes
[Added: 2018/07/13] [Last Updated: 2018/12/13]
M&E unit, ENV Portfolio 2019/10 Initiated Sample tools, following Kickpatrick’s model for the assessment of knowledge and skills of participants of capacity building events will be presented and relevant staff will be trained on it’s application in 2019. History
3.4 Develop a mechanism to analyse and disseminate lessons learnt at all UNDP projects based on analysis of M&E indicators, projects results, grant initiatives etc.
[Added: 2018/07/13] [Last Updated: 2018/12/13]
M&E unit, ENV Portfolio 2018/11 Completed Portfolio carries biannual workplan reviews, that include discussion of lessons learnt, progress on indicators and project results, grant initiatives and results of pilots, Meetings are held for the entire portfolio to allow for exchange of information between the projects and facilitation the discussion on building stronger synergies and linkages between the projects. In addition reinforced annual (at minimum) board meetings also provide a platform for discussion and lessons learnt dissemination with external stakeholders and final beneficiaries. History
3.5 Develop guidance to the projects on monitoring co-financing from beneficiaries (against co-financing set in the Project Documents; and against co-financing indicated for individual initiatives, e.g., which UNDP receives from applicants for grants)
[Added: 2018/07/13] [Last Updated: 2018/07/27]
M&E unit, ENV Portfolio 2018/06 Completed History
3.6 Improve M&E monitoring system by introducing M&E indicators which are available from external sources, not those produced by the projects implementing the activities. This will allow to get more reliable and less subjective data for analysis of projects effectiveness
[Added: 2018/07/13] [Last Updated: 2018/12/13]
M&E unit, ENV Portfolio 2019/11 Completed All ongoing projects within the environment portfolio now have M&E frameworks that are in line with UNDP RBM standards. The indicators rely on a mix of sources, including project data, as well as national and regional external sources. New UNDP Strategic Plan, Country Programme Document and UNDAF frameworks for 2018-2022 ensure regular collection of data that demonstrated progress at output and outcome levels. The indicators enable management to make evidence-based decisions. History
3.7 Develop a tracking mechanism for grant and pilot initiatives, including documenting results, lessons, experiences and good practices. The tracking of grants and pilots should be fully integrated into the CO’s RBM systems.
[Added: 2018/07/13] [Last Updated: 2018/12/13]
CO, M&E unit, ENV Portfolio 2019/06 Initiated UNDP CO reviewed grant policy and developed a unified template for grant holders, including a reporting template to track their progress. The tool to review and consolidate the information will be developed in 2019. History

Latest Evaluations

Contact us

220 East 42nd Street
20th Floor
New York, NY 10017
Tel. +1 646 781 4200
Fax. +1 646 781 4213
erc.support@undp.org