Evaluation of Country Programme Action Plan (CPAP) 2012-2016

Report Cover Image
Evaluation Plan:
2012-2016, Mongolia
Evaluation Type:
Country Programme Evaluation
Planned End Date:
02/2016
Completion Date:
02/2016
Status:
Completed
Management Response:
Yes
Evaluation Budget(US $):
50,000

Share

Document Type Language Size Status Downloads
Download document Final Report_CPAP Evaluation.pdf report English 1178.86 KB Posted 298
Download document TOR_Outcome Evaluation.pdf tor English 265.42 KB Posted 216
Title Evaluation of Country Programme Action Plan (CPAP) 2012-2016
Atlas Project Number:
Evaluation Plan: 2012-2016, Mongolia
Evaluation Type: Country Programme Evaluation
Status: Completed
Completion Date: 02/2016
Planned End Date: 02/2016
Management Response: Yes
Focus Area:
  • 1. Poverty and MDG
  • 2. Democratic Governance
  • 3. Crisis Prevention & Recovery
  • 4. Environment & Sustainable Development
  • 5. Cross-cutting Development Issue
  • 6. Others
Corporate Outcome and Output (UNDP Strategic Plan 2014-2017)
  • 1. Output 7.3. National development plans to address poverty and inequality are sustainable and risk resilient
Evaluation Budget(US $): 50,000
Source of Funding: UNDP Mongolia Engagement Facility
Evaluation Expenditure(US $): 45,100
Joint Programme: No
Joint Evaluation: No
Evaluation Team members:
Name Title Email Nationality
Abhijit Bhattacharjee Dr. abhijit@results-matter.co.uk INDIA
Natsagdorj Batjargal natsagdorj_batjargal@yahoo.com
Uranbileg Jamba juranbileg@gmail.com
Battulga holbooinc@yahoo.com
GEF Evaluation: No
Key Stakeholders: Government of Mongolia, line Ministries, SCOs, development partners
Countries: MONGOLIA
Comments:

Overall findings: UNDP’s role as a trusted partner to the Government of Mongolia (GoM) on complex and politically sensitive issues of governance and development as well as its convening role creates space for engagement with the GoM on development and governance issues. Development partners see it as a unique role of UNDP. This has enabled development of various legal and policy instruments in the country, though implementation and enforcement of these lag behind. UNDP has been instrumental in developing capacity of various institutions mostly at the national level, namely: National Human Right Commission of Mongolia (NHRCM), International Think Tank for Landlocked Developing Countries (ITT-LLDC), National Emergency Management Agency (NEMA) and the Ministry of Finance (MoF), in particular. A key challenge in the country is to ensure focus on poverty; this will require moving away from
a conventional notion of poverty and a new construct which UNDP is now developing linking poverty and vulnerability to climate change, disaster risk reduction and overall resilience of the economy. Future programming in the areas of poverty and environment will demand ability to bring cross-thematic linkages at all levels, underpinned by a well-articulated theory of change.
With competition for resources getting tougher, it will require UNDP to demonstrate strong
comparative advantage and innovative approaches in resource mobilization.

Lessons
Findings
Recommendations
1

UNDP’s country programme needs to be informed by a well-articulated theory of change (TOC) which identifies critical issues, drivers and inhibitors of change, assumptions underlying intended interventions, outputs and outcomes. The TOC will need to provide an overarching framework and a new construct for addressing the issues of economic development and resilience, poverty, climate change and disaster risk reduction, based on SDGs.

2

UNDP will need to develop its staff capacity and competence to work across thematic areas by establishing lateral linkages and demonstrating ability for holistic analysis as well distinctive competence in programmatic areas with clear focus that directly links to the TOC. In this regard, the following areas of expertise will be of particular relevance:

  • Social protection programmes, especially targeting the poor and vulnerable
  • Participatory rangeland management in semi-nomadic pastoralist system
  • Linking vulnerability, DRR and CCA in programme design and delivery
  • Urban poverty and governance.
3

As a general principle, where UNDP resources are involved, it should invest in helping the government to develop policies and new legislations only when it has undertaken a satisfactory analysis of political commitment, change drivers, resource commitments and  implementation/ enforcement capacity of the government. Otherwise, its future focus ought to be on enabling the government at both central and local levels to implement/enforce the existing legal and policy frameworks.

4

Besides partnership with conventional DPs, innovative approaches to working with IFIs (World Bank and ADB) and private sectors will be needed in future, as funding environment gets restricted.

5

While developing the results framework and M&E plan for the new CPD, special attention needs to be paid to identifying data points that will be required to be generated in order to objectively attribute overall outcome results to UNDP’s outputs.

6

Working with the World Bank and ADB, develop capacity building strategy for the MoF and line ministries to undertake the necessary cost-benefit and feasibility analysis of national policies before these are proposed for adoption.

7

Building on the work of CSLSGB, in the next phase focus on strengthening community groups and local civil society groups through introduction of social accountability mechanism (like social audit, accountability scorecard). In this, UNDP may explore partnership with initiatives like MASAM (initiated by World Bank) where it could bring in voices of women and other vulnerable sections of society in functioning of the social audit system.

8

In partnership with SDC, undertake a thorough study of issues in rangeland management in different zones of the country and, working with local authorities and herder groups, design scalable system to be implemented in one or two zones in the country. This could integrate UNDP’s current work related to ecosystem-based adaptation.

9

Working with the Deputy Prime Minister’ office, MoF, NEMA and Ministry of Environment, explore an inter-Ministerial coordination group which could provide a      platform for a whole-of-government approach for working on disaster risk reduction and developing resilience framework for the country.

10

Review implementation of the NIM and capacity gaps within MoF and implementing partners and put in place support mechanisms to address current implementation challenges.

1. Recommendation:

UNDP’s country programme needs to be informed by a well-articulated theory of change (TOC) which identifies critical issues, drivers and inhibitors of change, assumptions underlying intended interventions, outputs and outcomes. The TOC will need to provide an overarching framework and a new construct for addressing the issues of economic development and resilience, poverty, climate change and disaster risk reduction, based on SDGs.

Management Response: [Added: 2016/12/20] [Last Updated: 2016/12/20]

UNDP Mongolia CO agrees with the recommendation and it takes several measures to ensure that the programme is informed by TOC.

Key Actions:

Key Action Responsible DueDate Status Comments Documents
New CPD has the well-articulated TOC: a. Discussions on TOCs for each outcome b. CPD formulation incorporates the discussion and the chart
[Added: 2016/12/20]
DRR 2016/03 Completed HQ PAC commended that it is good quality country programme document. History
All new project documents have the TOC which outlines clear change pathway describing how the project will contribute to outcome level change as specified in the CPD. The requirement is incorporated in the quality assurance of the new project documents, which is included in the QA tools. QA tools are launched and CO started applying the QA standards.
[Added: 2016/12/20]
TLs and M&E 2016/12 Completed As part of the corporate requirement for the QAs, the assessment reports and the actions are entered into the system for monitoring and follow-up activities.
Project QA framework as part of the corporate PPM reform, is adapted and applied. It contains assessment of the project TOCs.
[Added: 2016/12/20]
POs and M&E 2016/12 Completed
2. Recommendation:

UNDP will need to develop its staff capacity and competence to work across thematic areas by establishing lateral linkages and demonstrating ability for holistic analysis as well distinctive competence in programmatic areas with clear focus that directly links to the TOC. In this regard, the following areas of expertise will be of particular relevance:

  • Social protection programmes, especially targeting the poor and vulnerable
  • Participatory rangeland management in semi-nomadic pastoralist system
  • Linking vulnerability, DRR and CCA in programme design and delivery
  • Urban poverty and governance.
Management Response: [Added: 2016/12/20] [Last Updated: 2016/12/20]

UNDP Mongolia CO agrees with the capacity and competence building across thematic areas.

Key Actions:

Key Action Responsible DueDate Status Comments Documents
Lateral linkages are established across thematic areas in new CPD. The CO applies an issues based approach to programming. The thematic areas recommended in the evaluation are incorporated in the programme formulation.
[Added: 2016/12/20]
DRR 2016/03 Completed
CO Transformation Plan contains a new business model where the technical areas emphasized, namely SDGs and Governance; Climate Change, Natural Resources Management and Environmental Governance. It provides an integrated management of the programme portfolio to create synergies.
[Added: 2016/12/20]
DRR 2016/09 Completed
CO Learning Plan incorporates learning sessions and opportunities to build staff competence to work across thematic areas.
[Added: 2016/12/20] [Last Updated: 2017/09/04]
HR and Learning committee 2017/02 Completed Talent Development committee has been established and the plan includes the learning sessions. History
3. Recommendation:

As a general principle, where UNDP resources are involved, it should invest in helping the government to develop policies and new legislations only when it has undertaken a satisfactory analysis of political commitment, change drivers, resource commitments and  implementation/ enforcement capacity of the government. Otherwise, its future focus ought to be on enabling the government at both central and local levels to implement/enforce the existing legal and policy frameworks.

Management Response: [Added: 2016/12/21]

CO agrees that the analysis of political and resources commitments, change drivers and implementation capacity of the government and the IPs is important.

Key Actions:

Key Action Responsible DueDate Status Comments Documents
The IP assessment (with above mentioned analysis) is incorporated in the business processes. a. Business process for new project approval/start contains the assessment
[Added: 2016/12/21] [Last Updated: 2017/09/04]
M&E 2017/03 Completed it is incorporated in the business process History
New project documents contain the relevant analysis (it is also one item in the new prodoc approval checklist).
[Added: 2016/12/21] [Last Updated: 2017/09/04]
Programme team 2017/06 Completed New project document checklist contains the analysis part. History
Project QA framework (appraisal, implementation and closure) is applied. QA contains relevant questions under its Effectiveness and Sustainability sections.
[Added: 2016/12/21] [Last Updated: 2017/11/19]
Programme team and M&E 2017/12 Completed As part of the corporate QA exercise, the project QA framework is fully integrated to the CO business process and the relevant project QAs are completed. Access to the report is : https://app.powerbi.com/groups/me/dashboards/9ece5a0f-229d-4aa9-9d64-87c9429759a5/reports/764160b4-9d01-4e12-a301-b630c370ef51/ReportSection3 History
HACT macro assessment is completed
[Added: 2016/12/21] [Last Updated: 2017/11/19]
Finance and M&E 2017/12 Completed HACT macro assessment is commissioned with KPMG Global LTA and the report submitted end Nov. History
HACT micro assessments of the IPs are completed
[Added: 2016/12/21] [Last Updated: 2018/09/18]
Finance and M&E 2018/08 Completed Micro assessments of Ministry of Construction, Ministry of Environment and Tourism, National Emergency Management Agency, Ministry of Food, Agriculture and Light Industry and Civil Service Council completed. History
4. Recommendation:

Besides partnership with conventional DPs, innovative approaches to working with IFIs (World Bank and ADB) and private sectors will be needed in future, as funding environment gets restricted.

Management Response: [Added: 2016/12/21]

CO agrees with the recommendation.

Key Actions:

Key Action Responsible DueDate Status Comments Documents
Cooperation MOU is signed with UB Mayor’s Office
[Added: 2016/12/21]
HD and MDG team 2016/06 Completed
Dialogue with IFIs
[Added: 2016/12/21] [Last Updated: 2017/09/04]
DRR 2017/06 Completed History
Urban planning/development discussion with ADB (seeking potential partnership)
[Added: 2016/12/21] [Last Updated: 2017/09/04]
DRR 2017/06 Completed History
5. Recommendation:

While developing the results framework and M&E plan for the new CPD, special attention needs to be paid to identifying data points that will be required to be generated in order to objectively attribute overall outcome results to UNDP’s outputs.

Management Response: [Added: 2016/12/21]

CO agrees with the recommendation.

Key Actions:

Key Action Responsible DueDate Status Comments Documents
M&E plan for the new CPD is formulated and approved with data reference points.
[Added: 2016/12/21]
M&E 2016/03 Completed Approved as part of the new CPD. History
CO Monitoring Strategy and M&E plan is updated with reference to measures on data collection.
[Added: 2016/12/21] [Last Updated: 2017/09/04]
M&E 2017/03 Completed Monitoring strategy and M&E plan updated History
6. Recommendation:

Working with the World Bank and ADB, develop capacity building strategy for the MoF and line ministries to undertake the necessary cost-benefit and feasibility analysis of national policies before these are proposed for adoption.

Management Response: [Added: 2016/12/21]

Accepted.

Key Actions:

Key Action Responsible DueDate Status Comments Documents
SDG project preparatory phase and full SDG program will address this issue including: - Conduct the Development Finance Assessment -Development of a roadmap for Implementing the SDV based on the assessment of institutional options.
[Added: 2016/12/21] [Last Updated: 2017/11/19]
Programme team 2017/12 Completed SDG project preparatory phase has been implemented with number of important interventions. Development of full SDG program is underway. -the Development Finance Assessment conducted and final review is on-going. -Roadmap development is also started for Implementing the SDV based on the assessment of institutional options. History
7. Recommendation:

Building on the work of CSLSGB, in the next phase focus on strengthening community groups and local civil society groups through introduction of social accountability mechanism (like social audit, accountability scorecard). In this, UNDP may explore partnership with initiatives like MASAM (initiated by World Bank) where it could bring in voices of women and other vulnerable sections of society in functioning of the social audit system.

Management Response: [Added: 2016/12/21]

During the CSLSGBs project implementation very useful resources have been created as a result of the grants programme for NGOs, including meeting procedures, standard code of conduct, guidance for oversight functions, methodology for participatory monitoring etc. The new project on Strengthening Representative Bodies will continue this grant scheme to NGOs. The project will aim to create an accountability link between the local Hural representatives and the marginalized goups such as youth, women and urban and rural poor.

Key Actions:

Key Action Responsible DueDate Status Comments Documents
A new project document is approved with the outputs focusing on strengthening the community engagement with the local Hural such as development of detailed procedure for public consultations on draft decisions through a consultative process, grants criteria for those engaging the target groups such as women, young people and the poor and vulnerable groups, introduction of performance indicators for Hural performance etc.
[Added: 2016/12/21]
Programme unit 2016/11 Completed Project document approved with the planned elements focusing on community. History
8. Recommendation:

In partnership with SDC, undertake a thorough study of issues in rangeland management in different zones of the country and, working with local authorities and herder groups, design scalable system to be implemented in one or two zones in the country. This could integrate UNDP’s current work related to ecosystem-based adaptation.

Management Response: [Added: 2016/12/21]

New CPD integrates planning for initiatives on rangeland management and working with the local communities and authorities to resolve issues related land, pasture and water management. CO will develop a project proposal on improving adaptive capacity and risk management of rural communities in Mongolia, which incorporates the ecosystem-based adaptation to restore and maintain ecosystem functions.

Key Actions:

Key Action Responsible DueDate Status Comments Documents
A project proposal is formulated that integrates the rangeland management and UNDP’s work related to ecosystem-based adaptation. It plans to support an up-scaled integrated adaptation approaches through development of local pasture management plans with participation of local government officials and herders’ representatives; training on land use/pasture land use planning, oversight and enforcement; knowledge sharing and awareness raising activities on sustainable pasture management etc.
[Added: 2016/12/21] [Last Updated: 2017/09/04]
Programme team 2017/07 Completed A proposal developed and the stakeholder discussions held. GCF project proposal is completed and validation workshop held. History
9. Recommendation:

Working with the Deputy Prime Minister’ office, MoF, NEMA and Ministry of Environment, explore an inter-Ministerial coordination group which could provide a      platform for a whole-of-government approach for working on disaster risk reduction and developing resilience framework for the country.

Management Response: [Added: 2016/12/21]

UNDP will work with the National Emergency Management Agency on establishing a national platform to mainstream disaster risk reduction into development planning and improving cross-sectoral collaboration and coordination.

Key Actions:

Key Action Responsible DueDate Status Comments Documents
Programme priorities include activities to increase resilience of the poor and vulnerable against disasters. New country programme document incorporates UNDP’s work on disaster risk reduction including community-level preparedness, targeted measures and reducing urban vulnerability to disasters.
[Added: 2016/12/21]
Programme team 2016/03 Completed
10. Recommendation:

Review implementation of the NIM and capacity gaps within MoF and implementing partners and put in place support mechanisms to address current implementation challenges.

Management Response: [Added: 2016/12/21]

CO agrees with the recommendation

Key Actions:

Key Action Responsible DueDate Status Comments Documents
Review meeting is organized to assess the NIM implementation.
[Added: 2016/12/21]
Operation Manager 2016/06 Completed
HACT macro and micro assessments are done. Implementation modality is identified for each IP based on the risks.
[Added: 2016/12/21] [Last Updated: 2018/09/18]
Finance and M&E 2018/06 Completed HACT macro and micro assessment are done. Modalities with IP is discussed and agreed based on the risks. History
NPDs and stakeholder workshop on the implementation modality and quality assurance of the programme .
[Added: 2016/12/21] [Last Updated: 2019/01/01]
DRR, OM and M&E 2018/12 Completed Through UNDAF and CPD results workshop[, the operational change and implementation modalities are informed/discussed among the key stakeholders including NPDs. Due to the frequent changes in the government (following the elections) which had impact on the change of the NPDs, a specific workshop for NPDs were not held. However, a working lunch with the NPDs to discuss the various issues including implementation modality organized in Jan, 2017 History

Latest Evaluations

Contact us

1 UN Plaza
DC1-20th Floor
New York, NY 10017
Tel. +1 646 781 4200
Fax. +1 646 781 4213
erc.support@undp.org