- Evaluation Plan:
- 2012-2016, Mongolia
- Evaluation Type:
- Country Programme Evaluation
- Planned End Date:
- 02/2016
- Completion Date:
- 02/2016
- Status:
- Completed
- Management Response:
- Yes
- Evaluation Budget(US $):
- 50,000
Evaluation of Country Programme Action Plan (CPAP) 2012-2016
| Document | Type | Language | Size | Status | Downloads |
|---|---|---|---|---|---|
 Final Report_CPAP Evaluation.pdf |
report | English | 1178.86 KB | Posted | 298 |
| TOR_Outcome Evaluation.pdf |
tor | English | 265.42 KB | Posted | 216 |
| Title | Evaluation of Country Programme Action Plan (CPAP) 2012-2016 | ||||||||||||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| Atlas Project Number: | |||||||||||||||||||||
| Evaluation Plan: | 2012-2016, Mongolia | ||||||||||||||||||||
| Evaluation Type: | Country Programme Evaluation | ||||||||||||||||||||
| Status: | Completed | ||||||||||||||||||||
| Completion Date: | 02/2016 | ||||||||||||||||||||
| Planned End Date: | 02/2016 | ||||||||||||||||||||
| Management Response: | Yes | ||||||||||||||||||||
| Focus Area: |
|
||||||||||||||||||||
| Corporate Outcome and Output (UNDP Strategic Plan 2014-2017) |
|
||||||||||||||||||||
| Evaluation Budget(US $): | 50,000 | ||||||||||||||||||||
| Source of Funding: | UNDP Mongolia Engagement Facility | ||||||||||||||||||||
| Evaluation Expenditure(US $): | 45,100 | ||||||||||||||||||||
| Joint Programme: | No | ||||||||||||||||||||
| Joint Evaluation: | No | ||||||||||||||||||||
| Evaluation Team members: |
|
||||||||||||||||||||
| GEF Evaluation: | No | ||||||||||||||||||||
| Key Stakeholders: | Government of Mongolia, line Ministries, SCOs, development partners | ||||||||||||||||||||
| Countries: | MONGOLIA | ||||||||||||||||||||
| Comments: | Overall findings: UNDP’s role as a trusted partner to the Government of Mongolia (GoM) on complex and politically sensitive issues of governance and development as well as its convening role creates space for engagement with the GoM on development and governance issues. Development partners see it as a unique role of UNDP. This has enabled development of various legal and policy instruments in the country, though implementation and enforcement of these lag behind. UNDP has been instrumental in developing capacity of various institutions mostly at the national level, namely: National Human Right Commission of Mongolia (NHRCM), International Think Tank for Landlocked Developing Countries (ITT-LLDC), National Emergency Management Agency (NEMA) and the Ministry of Finance (MoF), in particular. A key challenge in the country is to ensure focus on poverty; this will require moving away from |
| Lessons | |
|---|---|
| Findings |
| Recommendations | |
|---|---|
| 1 | UNDP’s country programme needs to be informed by a well-articulated theory of change (TOC) which identifies critical issues, drivers and inhibitors of change, assumptions underlying intended interventions, outputs and outcomes. The TOC will need to provide an overarching framework and a new construct for addressing the issues of economic development and resilience, poverty, climate change and disaster risk reduction, based on SDGs. |
| 2 | UNDP will need to develop its staff capacity and competence to work across thematic areas by establishing lateral linkages and demonstrating ability for holistic analysis as well distinctive competence in programmatic areas with clear focus that directly links to the TOC. In this regard, the following areas of expertise will be of particular relevance:
|
| 3 | As a general principle, where UNDP resources are involved, it should invest in helping the government to develop policies and new legislations only when it has undertaken a satisfactory analysis of political commitment, change drivers, resource commitments and |
| 4 | Besides partnership with conventional DPs, innovative approaches to working with IFIs (World Bank and ADB) and private sectors will be needed in future, as funding environment gets restricted. |
| 5 | While developing the results framework and M&E plan for the new CPD, special attention needs to be paid to identifying data points that will be required to be generated in order to objectively attribute overall outcome results to UNDP’s outputs. |
| 6 | Working with the World Bank and ADB, develop capacity building strategy for the MoF and line ministries to undertake the necessary cost-benefit and feasibility analysis of national policies before these are proposed for adoption. |
| 7 | Building on the work of CSLSGB, in the next phase focus on strengthening community groups and local civil society groups through introduction of social accountability mechanism (like social audit, accountability scorecard). In this, UNDP may explore partnership with initiatives like MASAM (initiated by World Bank) where it could bring in voices of women and other vulnerable sections of society in functioning of the social audit system. |
| 8 | In partnership with SDC, undertake a thorough study of issues in rangeland management in different zones of the country and, working with local authorities and herder groups, design scalable system to be implemented in one or two zones in the country. This could integrate UNDP’s current work related to ecosystem-based adaptation. |
| 9 | Working with the Deputy Prime Minister’ office, MoF, NEMA and Ministry of Environment, explore an inter-Ministerial coordination group which could provide a platform for a whole-of-government approach for working on disaster risk reduction and developing resilience framework for the country. |
| 10 | Review implementation of the NIM and capacity gaps within MoF and implementing partners and put in place support mechanisms to address current implementation challenges. |
1. Recommendation:
UNDP’s country programme needs to be informed by a well-articulated theory of change (TOC) which identifies critical issues, drivers and inhibitors of change, assumptions underlying intended interventions, outputs and outcomes. The TOC will need to provide an overarching framework and a new construct for addressing the issues of economic development and resilience, poverty, climate change and disaster risk reduction, based on SDGs.
Management Response: [Added: 2016/12/20] [Last Updated: 2016/12/20]
UNDP Mongolia CO agrees with the recommendation and it takes several measures to ensure that the programme is informed by TOC.
Key Actions:
| Key Action | Responsible | DueDate | Status | Comments | Documents |
|---|---|---|---|---|---|
| New CPD has the well-articulated TOC:
a. Discussions on TOCs for each outcome
b. CPD formulation incorporates the discussion and the chart
[Added: 2016/12/20] |
DRR | 2016/03 | Completed | HQ PAC commended that it is good quality country programme document. History | |
| All new project documents have the TOC which outlines clear change pathway describing how the project will contribute to outcome level change as specified in the CPD. The requirement is incorporated in the quality assurance of the new project documents, which is included in the QA tools. QA tools are launched and CO started applying the QA standards.
[Added: 2016/12/20] |
TLs and M&E | 2016/12 | Completed | As part of the corporate requirement for the QAs, the assessment reports and the actions are entered into the system for monitoring and follow-up activities. | |
| Project QA framework as part of the corporate PPM reform, is adapted and applied. It contains assessment of the project TOCs.
[Added: 2016/12/20] |
POs and M&E | 2016/12 | Completed |
2. Recommendation:
UNDP will need to develop its staff capacity and competence to work across thematic areas by establishing lateral linkages and demonstrating ability for holistic analysis as well distinctive competence in programmatic areas with clear focus that directly links to the TOC. In this regard, the following areas of expertise will be of particular relevance:
- Social protection programmes, especially targeting the poor and vulnerable
- Participatory rangeland management in semi-nomadic pastoralist system
- Linking vulnerability, DRR and CCA in programme design and delivery
- Urban poverty and governance.
Management Response: [Added: 2016/12/20] [Last Updated: 2016/12/20]
UNDP Mongolia CO agrees with the capacity and competence building across thematic areas.
Key Actions:
| Key Action | Responsible | DueDate | Status | Comments | Documents |
|---|---|---|---|---|---|
| Lateral linkages are established across thematic areas in new CPD. The CO applies an issues based approach to programming. The thematic areas recommended in the evaluation are incorporated in the programme formulation.
[Added: 2016/12/20] |
DRR | 2016/03 | Completed | ||
| CO Transformation Plan contains a new business model where the technical areas emphasized, namely SDGs and Governance; Climate Change, Natural Resources Management and Environmental Governance. It provides an integrated management of the programme portfolio to create synergies.
[Added: 2016/12/20] |
DRR | 2016/09 | Completed | ||
| CO Learning Plan incorporates learning sessions and opportunities to build staff competence to work across thematic areas.
[Added: 2016/12/20] [Last Updated: 2017/09/04] |
HR and Learning committee | 2017/02 | Completed | Talent Development committee has been established and the plan includes the learning sessions. History |
3. Recommendation:
As a general principle, where UNDP resources are involved, it should invest in helping the government to develop policies and new legislations only when it has undertaken a satisfactory analysis of political commitment, change drivers, resource commitments and implementation/ enforcement capacity of the government. Otherwise, its future focus ought to be on enabling the government at both central and local levels to implement/enforce the existing legal and policy frameworks.
Management Response: [Added: 2016/12/21]
CO agrees that the analysis of political and resources commitments, change drivers and implementation capacity of the government and the IPs is important.
Key Actions:
| Key Action | Responsible | DueDate | Status | Comments | Documents |
|---|---|---|---|---|---|
| The IP assessment (with above mentioned analysis) is incorporated in the business processes.
a. Business process for new project approval/start contains the assessment
[Added: 2016/12/21] [Last Updated: 2017/09/04] |
M&E | 2017/03 | Completed | it is incorporated in the business process History | |
| New project documents contain the relevant analysis (it is also one item in the new prodoc approval checklist).
[Added: 2016/12/21] [Last Updated: 2017/09/04] |
Programme team | 2017/06 | Completed | New project document checklist contains the analysis part. History | |
| Project QA framework (appraisal, implementation and closure) is applied. QA contains relevant questions under its Effectiveness and Sustainability sections.
[Added: 2016/12/21] [Last Updated: 2017/11/19] |
Programme team and M&E | 2017/12 | Completed | As part of the corporate QA exercise, the project QA framework is fully integrated to the CO business process and the relevant project QAs are completed. Access to the report is : https://app.powerbi.com/groups/me/dashboards/9ece5a0f-229d-4aa9-9d64-87c9429759a5/reports/764160b4-9d01-4e12-a301-b630c370ef51/ReportSection3 History | |
| HACT macro assessment is completed
[Added: 2016/12/21] [Last Updated: 2017/11/19] |
Finance and M&E | 2017/12 | Completed | HACT macro assessment is commissioned with KPMG Global LTA and the report submitted end Nov. History | |
| HACT micro assessments of the IPs are completed
[Added: 2016/12/21] [Last Updated: 2018/09/18] |
Finance and M&E | 2018/08 | Completed | Micro assessments of Ministry of Construction, Ministry of Environment and Tourism, National Emergency Management Agency, Ministry of Food, Agriculture and Light Industry and Civil Service Council completed. History |
4. Recommendation:
Besides partnership with conventional DPs, innovative approaches to working with IFIs (World Bank and ADB) and private sectors will be needed in future, as funding environment gets restricted.
Management Response: [Added: 2016/12/21]
CO agrees with the recommendation.
Key Actions:
| Key Action | Responsible | DueDate | Status | Comments | Documents |
|---|---|---|---|---|---|
| Cooperation MOU is signed with UB Mayor’s Office
[Added: 2016/12/21] |
HD and MDG team | 2016/06 | Completed | ||
| Dialogue with IFIs
[Added: 2016/12/21] [Last Updated: 2017/09/04] |
DRR | 2017/06 | Completed | History | |
| Urban planning/development discussion with ADB (seeking potential partnership)
[Added: 2016/12/21] [Last Updated: 2017/09/04] |
DRR | 2017/06 | Completed | History |
5. Recommendation:
While developing the results framework and M&E plan for the new CPD, special attention needs to be paid to identifying data points that will be required to be generated in order to objectively attribute overall outcome results to UNDP’s outputs.
Management Response: [Added: 2016/12/21]
CO agrees with the recommendation.
Key Actions:
| Key Action | Responsible | DueDate | Status | Comments | Documents |
|---|---|---|---|---|---|
| M&E plan for the new CPD is formulated and approved with data reference points.
[Added: 2016/12/21] |
M&E | 2016/03 | Completed | Approved as part of the new CPD. History | |
| CO Monitoring Strategy and M&E plan is updated with reference to measures on data collection.
[Added: 2016/12/21] [Last Updated: 2017/09/04] |
M&E | 2017/03 | Completed | Monitoring strategy and M&E plan updated History |
6. Recommendation:
Working with the World Bank and ADB, develop capacity building strategy for the MoF and line ministries to undertake the necessary cost-benefit and feasibility analysis of national policies before these are proposed for adoption.
Management Response: [Added: 2016/12/21]
Accepted.
Key Actions:
| Key Action | Responsible | DueDate | Status | Comments | Documents |
|---|---|---|---|---|---|
| SDG project preparatory phase and full SDG program will address this issue including:
- Conduct the Development Finance Assessment
-Development of a roadmap for Implementing the SDV based on the assessment of institutional options.
[Added: 2016/12/21] [Last Updated: 2017/11/19] |
Programme team | 2017/12 | Completed | SDG project preparatory phase has been implemented with number of important interventions. Development of full SDG program is underway. -the Development Finance Assessment conducted and final review is on-going. -Roadmap development is also started for Implementing the SDV based on the assessment of institutional options. History |
7. Recommendation:
Building on the work of CSLSGB, in the next phase focus on strengthening community groups and local civil society groups through introduction of social accountability mechanism (like social audit, accountability scorecard). In this, UNDP may explore partnership with initiatives like MASAM (initiated by World Bank) where it could bring in voices of women and other vulnerable sections of society in functioning of the social audit system.
Management Response: [Added: 2016/12/21]
During the CSLSGBs project implementation very useful resources have been created as a result of the grants programme for NGOs, including meeting procedures, standard code of conduct, guidance for oversight functions, methodology for participatory monitoring etc. The new project on Strengthening Representative Bodies will continue this grant scheme to NGOs. The project will aim to create an accountability link between the local Hural representatives and the marginalized goups such as youth, women and urban and rural poor.
Key Actions:
| Key Action | Responsible | DueDate | Status | Comments | Documents |
|---|---|---|---|---|---|
| A new project document is approved with the outputs focusing on strengthening the community engagement with the local Hural such as development of detailed procedure for public consultations on draft decisions through a consultative process, grants criteria for those engaging the target groups such as women, young people and the poor and vulnerable groups, introduction of performance indicators for Hural performance etc.
[Added: 2016/12/21] |
Programme unit | 2016/11 | Completed | Project document approved with the planned elements focusing on community. History |
8. Recommendation:
In partnership with SDC, undertake a thorough study of issues in rangeland management in different zones of the country and, working with local authorities and herder groups, design scalable system to be implemented in one or two zones in the country. This could integrate UNDP’s current work related to ecosystem-based adaptation.
Management Response: [Added: 2016/12/21]
New CPD integrates planning for initiatives on rangeland management and working with the local communities and authorities to resolve issues related land, pasture and water management. CO will develop a project proposal on improving adaptive capacity and risk management of rural communities in Mongolia, which incorporates the ecosystem-based adaptation to restore and maintain ecosystem functions.
Key Actions:
| Key Action | Responsible | DueDate | Status | Comments | Documents |
|---|---|---|---|---|---|
| A project proposal is formulated that integrates the rangeland management and UNDP’s work related to ecosystem-based adaptation. It plans to support an up-scaled integrated adaptation approaches through development of local pasture management plans with participation of local government officials and herders’ representatives; training on land use/pasture land use planning, oversight and enforcement; knowledge sharing and awareness raising activities on sustainable pasture management etc.
[Added: 2016/12/21] [Last Updated: 2017/09/04] |
Programme team | 2017/07 | Completed | A proposal developed and the stakeholder discussions held. GCF project proposal is completed and validation workshop held. History |
9. Recommendation:
Working with the Deputy Prime Minister’ office, MoF, NEMA and Ministry of Environment, explore an inter-Ministerial coordination group which could provide a platform for a whole-of-government approach for working on disaster risk reduction and developing resilience framework for the country.
Management Response: [Added: 2016/12/21]
UNDP will work with the National Emergency Management Agency on establishing a national platform to mainstream disaster risk reduction into development planning and improving cross-sectoral collaboration and coordination.
Key Actions:
| Key Action | Responsible | DueDate | Status | Comments | Documents |
|---|---|---|---|---|---|
| Programme priorities include activities to increase resilience of the poor and vulnerable against disasters. New country programme document incorporates UNDP’s work on disaster risk reduction including community-level preparedness, targeted measures and reducing urban vulnerability to disasters.
[Added: 2016/12/21] |
Programme team | 2016/03 | Completed |
10. Recommendation:
Review implementation of the NIM and capacity gaps within MoF and implementing partners and put in place support mechanisms to address current implementation challenges.
Management Response: [Added: 2016/12/21]
CO agrees with the recommendation
Key Actions:
| Key Action | Responsible | DueDate | Status | Comments | Documents |
|---|---|---|---|---|---|
| Review meeting is organized to assess the NIM implementation.
[Added: 2016/12/21] |
Operation Manager | 2016/06 | Completed | ||
| HACT macro and micro assessments are done. Implementation modality is identified for each IP based on the risks.
[Added: 2016/12/21] [Last Updated: 2018/09/18] |
Finance and M&E | 2018/06 | Completed | HACT macro and micro assessment are done. Modalities with IP is discussed and agreed based on the risks. History | |
| NPDs and stakeholder workshop on the implementation modality and quality assurance of the programme .
[Added: 2016/12/21] [Last Updated: 2019/01/01] |
DRR, OM and M&E | 2018/12 | Completed | Through UNDAF and CPD results workshop[, the operational change and implementation modalities are informed/discussed among the key stakeholders including NPDs. Due to the frequent changes in the government (following the elections) which had impact on the change of the NPDs, a specific workshop for NPDs were not held. However, a working lunch with the NPDs to discuss the various issues including implementation modality organized in Jan, 2017 History |