ICPR- Outcome evaluation: Combined programme

Report Cover Image
Evaluation Plan:
2017-2021, Mongolia
Evaluation Type:
Outcome
Planned End Date:
05/2021
Status:
Deleted
Management Response:
No
Evaluation Budget(US $):
1,000
Document Type Language Size Status Downloads
Title ICPR- Outcome evaluation: Combined programme
Atlas Project Number: 00085915
Evaluation Plan: 2017-2021, Mongolia
Evaluation Type: Outcome
Status: Deleted
Planned End Date: 05/2021
Management Response: Yes
UNDP Signature Solution:
  • 1. Poverty
  • 2. Governance
  • 3. Resilience
  • 4. Sustainable
Corporate Outcome and Output (UNDP Strategic Plan 2018-2021)
  • 1. Output 1.1.1 Capacities developed across the whole of government to integrate the 2030 Agenda, the Paris Agreement and other international agreements in development plans and budgets, and to analyse progress towards the SDGs, using innovative and data-driven solutions
SDG Goal
  • Goal 1. End poverty in all its forms everywhere
  • Goal 13. Take urgent action to combat climate change and its impacts
  • Goal 15. Protect, restore and promote sustainable use of terrestrial ecosystems, sustainably manage forests, combat desertification, and halt and reverse land degradation and halt biodiversity loss
  • Goal 16. Promote peaceful and inclusive societies for sustainable development, provide access to justice for all and build effective, accountable and inclusive institutions at all levels
  • Goal 17. Strengthen the means of implementation and revitalize the Global Partnership for Sustainable Development
  • Goal 8. Promote sustained, inclusive and sustainable economic growth, full and productive employment and decent work for all
SDG Target
  • 1.2 By 2030, reduce at least by half the proportion of men, women and children of all ages living in poverty in all its dimensions according to national definitions
  • 1.5 By 2030, build the resilience of the poor and those in vulnerable situations and reduce their exposure and vulnerability to climate-related extreme events and other economic, social and environmental shocks and disasters
  • 13.1 Strengthen resilience and adaptive capacity to climate-related hazards and natural disasters in all countries
  • 13.2 Integrate climate change measures into national policies, strategies and planning
  • 15.3 By 2030, combat desertification, restore degraded land and soil, including land affected by desertification, drought and floods, and strive to achieve a land degradation-neutral world
  • 16.3 Promote the rule of law at the national and international levels and ensure equal access to justice for all
  • 17.17 Encourage and promote effective public, public-private and civil society partnerships, building on the experience and resourcing strategies of partnerships
  • 8.4 Improve progressively, through 2030, global resource efficiency in consumption and production and endeavour to decouple economic growth from environmental degradation, in accordance with the 10-Year Framework of Programmes on Sustainable Consumption and Production, with developed countries taking the lead
  • 8.6 By 2020, substantially reduce the proportion of youth not in employment, education or training
Evaluation Budget(US $): 1,000
Source of Funding: Country Programme M&E budget
Evaluation Expenditure(US $): 1,000
Joint Programme: No
Joint Evaluation: No
GEF Evaluation: No
Key Stakeholders:
Countries: MONGOLIA
Comments:

Due to the COVID 19 pandemic situation, the plan is moved. In consultation with IEO, it is now planned to be conducted later in 2020. The IEO team is working on the final report with all the submissions and interviews done with the CO and the stakeholders.

Lessons
1.

Overview of outcome 1 (“By 2022, poor and vulnerable people are more resilient to shocks and benefit from inclusive growth and a healthy ecosystem”) progress and ratings:

UNDP’s contribution to meeting the outcome level targets was rated as “moderate”. In other words, UNDP moderately influences the resilience of poor and vulnerable people to shocks and their benefitting from inclusive growth and a healthy ecosystem in Mongolia. The work to mainstream the 2030 agenda in Mongolia is on track to meet the indicator target (which however does not appropriately reflect the anticipated results at output level). Efforts to mitigate climate change and effectively manage natural resources to support the most vulnerable are at risk. Significant contributions were made to outcome indicator 1.3 (size of protected areas). Contributions to indicator 1.4 (economic loss from natural hazards) and 1.5 (annual greenhouse gas emissions) are not directly measured but estimates based on pilot initiatives suggest modest potential for reducing greenhouse gas emissions. UNDP’s role in achieving targets for indicators 1.1. (poverty reduction) and 1.2 (reducing unemployment) was weak, as would be expected for such high-level indicators. The review noted with concern the absence, over the period reviewed, of contributions that directly tackle poverty and unemployment.

 

Overview of outcome 2 (“By 2022, governing institutions are more responsive and accountable to citizens, while ensuring effective participation of young people and realization of the rights of the poor and marginalized”) progress and ratings:

 

UNDP’s contribution to meeting outcome level targets was rated as “moderate plus”. In other words, UNDP influences improvement of voice, participation and accountability in Mongolia to a moderate or higher degree. UNDP is on track to meet targets for the outputs on women’s leadership, boosting civil service effectiveness and extractive industries governance. It is however off track to reach the indicator target on citizen engagement. Significant contributions were made to indicator 2.2.1 (number of women benefiting from measures supporting their leadership role), indicator 2.3.1 (number of government agencies adopting informed anti-corruption plans), indicator 2.4.1 (number of complaints received) and indicator 2.4.2 (HR procedures meeting minimum benchmarks). UNDP’s role in achieving the targets for indicators 2.1.1. (number of public hearings) and 2.1.2 (effective mechanisms to engage civil society) was weak. While outputs are appropriately linked to the outcome level, and there is a line of contribution from UNDP to changes in the UNDAF outcome and associated indicators, the contribution of other factors is hugely significant (i.e. the political, social and cultural factors contributing to good governance). Although outcome indicators include a focus on youth participation, there was no corresponding UNDP effort or output indicator that targeted young people (or marginalized groups).  


Findings
1.
  1. Finding 1 – Hampered by the delayed receipt of a major grant, the projects grouped under Outcome 1 consist of seemingly unrelated, mostly small-scale initiatives in the environment, poverty, energy and resilience realms. These projects do not add up to a coherent, stand-alone portfolio. This significantly hampered the CO’s ability to demonstrate results at the outcome level.
  2. UNDP Mongolia’s outcome area 1 covers a large breadth of topics, with fragmented interventions. To align the country programme to the UNDAF RRF, two portfolios of the 2012-2016 Country Programme (inclusive growth and environment; climate change and disaster risk reduction) were combined, creating an outcome with 10 underlying outputs and a plethora of projects.[1] These are loosely tied together in the CPD narrative and draft theory of change (dated 1 March 2016) as “a new approach on poverty reduction which integrates resilient development, disaster risk reduction and climate change mitigation/adaptation”.[2] This review however found that outcome 1 did not provide a workable framework in principle nor did it yield the expected results in practice. This is partly due to unforeseen delays in securing a $23M GCF project which could have potentially contributed to output 1.3, and outputs 1.5 through 1.8. The resulting fragmentation[3] made the portfolio labour-intensive and hard to manage (see figure 15 in the statistical annex). In the context of the shrinking fiscal space of the Government of Mongolia (GoM)[4] and the sharp decrease in Overseas Development Assistance (ODA) since 2017[5], this has contributed to UNDP’s difficulties in making the business case for its outcome area 1 and mobilizing required resources on time and on target. 
  3. At design, there were insufficient linkages between the projects under outcome 1 to achieve output indicators, as well as between the combined outputs to achieve outcome indicators. For instance, only one small project, “Activated2030 at #Hub: A Youth Enterprising Lab,” related directly to the outcome indicator on employment. However, this activity did not feature any associated output indicators, meaning that measurement of the results chain from UNDP activities to outputs to outcomes is interrupted[6] All in all, many different monitoring and measurement approaches were found across outcome area 1, with no visible attempt at harmonization across projects. This complicated demonstration of aggregate or higher-level results, and cross-fertilization between projects.
  4. Finding 2 UNDP achieved mixed results in SDG mainstreaming and localization. While the (unambitious) indicator target was achieved, sustainability and scale-up of results are at risk. UNDP capitalized on its strong relationship with the Government of Mongolia and displayed some degree of thought leadership and analytical excellence but support for the SDG agenda appears to be diminishing in some quarters of society.
  5. Comprehensive support to SDG mainstreaming, implementation, monitoring and reporting has emerged as a flagship offering of UNDP Mongolia, alongside its governance portfolio pitched as an ‘enabler’. UNDP made a significant impact in Mongolia with regard to making the SDGs known and used in policy planning and implementation, including at the local level. SDG localization initiatives in the capital and three provinces are considered a good practice by the regional bureau, first and foremost for their indirect benefits: i.e. positioning UNDP as a service provider worthy of local government cost-sharing (instead of using the private sector) and opening the door for other more sensitive programmes, and for a nascent SDG financing initiative. The single most important success factor here was UNDP’s strong and long-standing relationship with the GoM at all levels from central to aimag (provincial). This meant that the GoM routinely turned to UNDP for technical assistance on key development policies and strategies, providing an opportunity to input and align with international best practice and gender and human rights standards. As a consequence, key policies such as the long-term “Vision 2050”, “Mongolia Sustainable Development 2030”[7], “Long-Term Sustainable Green Development Goal 2025” and the government’s COVID-response bear the imprint of UNDP to varying but clearly perceptible degrees, which is a major achievement.[8] UNDP also supported the Government of Mongolia to produce and present its first Voluntary National Review report on SDG implementation to the United Nations High-level Political Forum in July 2019. The report applied the lens of “leaving no one behind”, identifying vulnerable groups and highlighting the risks of disparities in preventing, or even reversing development gains.
  6. An emerging challenge are signs of growing discontent with the SDGs as an organizing principle for development in Mongolia, regarded by some in government and civil society as not sufficiently aligned with shifting priorities due to the COVID-19 pandemic. The strategy document “Mongolia Sustainable Development 2030” was approved in 2017 but withdrawn in 2020[9] and partially repackaged in the longer-term “Vision 2050”. The latter is however not fully aligned with the SDGs.[10] The national SDG commission exists on paper but has not met during the period under review, giving UNDP little to align with or anchor its niche in support of a nationally owned agenda.[11] Key informants suggested that UNDP’s main added value in mainstreaming the SDGs lies in its capacity for research, analysis and thought leadership.[12]  
  7. One highly appreciated analytical effort is UNDP support to national SDG budgeting processes by conducting expenditure analysis and integrating results indicators that operationalize policies of the Ministry of Finance and line ministries. This is important to boost implementation of SDG-aligned activities (which has so far lagged behind policy development) within the budgetary constraints of Mongolia's fiscal adjustment framework. Specifically, UNDP promoted the integration between financing and budgeting and their early incorporation during planning stages. At the sectoral level, UNDP in collaboration with UNICEF, ADB and the EU piloted SDG-informed budgeting initiatives in the National Development Agency, Ministry of Health, Ministry of Environment and Tourism, and Ministry of Education and Science. While expansion to the employment sector is ongoing, scale-up to other government sectors and provinces remains uncertain.[13] The review notes anecdotal evidence of changes to the mindsets of policy makers in support of a result-based budgeting reform and more medium-term public finance management reforms, sparked by the impetus given by UNDP and others.
  8. A key factor that enabled UNDP to influence national development policy was the effective use of the agency’s vast network, including multilateral and bilateral agencies, international and national expert consultants, UNDP’s own regional office, and others. In the example above, UNDP teamed up with ADB as a key new partner. The Regional UNDP Bureau in Bangkok was brought in to share international best practice and policy advice on a seminal paper on air quality, nascent ideas around social protection and debt swaps, the draft UN Socio-Economic Response to COVID-19, and other emerging analytical issues. However, multi-stakeholder collaboration that included civil society  and private sector actors were rare during the period assessed, which appears to be a weakness.[14] Another key concern was the CO’s tendency for overambitious planning which contrasted with external challenges such as changes in government and government institutional capacity constraints more generally, and the absence of a realistic and integrated costing and financing strategy at the national level. Weak local governance systems, low sub-national (vertical) policy coherence, and a M&E system that is not fully integrated into government decision-making further hampered performance.
  9. Finding 3Results with regard to building the resilience of the poor to shocks were very limited, not least due to a delayed start of work in this area. UNDP has contributed to equitable, sustainable and low carbon urban development, but progress lacked institutionalisation and scale-up through Mongolia’s Nationally Determined Contributions framework.
  10. UNDP achieved several results toward building the resilience of the poor to shocks, from developing a smart phone application[15], to providing assistance to herder households affected by dzud, and was instrumental in the UN’s response to COVID-19. However, the primary objective of the Disaster Risk Reduction (DRR) programme seems to have slipped, i.e. to support the creation of subnational disaster management plans. Progress toward set indicators is off track or at risk, questioning whether achievements of past programme cycles can be sustained.
  11. A key challenge was the CO’s inability to secure funding on time, in particular due to prolonged approval processes by the Green Climate Fund (GCF) which finally signed its agreement with the CO in November 2020. Rather than revising its RF, the CO relied on the “Nationally Appropriate Mitigation Actions” (NAMA) project alone for target achievement.[16] The project which helped lay the basis for market transformation for energy efficiency in the construction sector, was thus measured against unattainable indicators such as “investments leveraged for climate change adaptation” (indicator 1.6.1) and reduction of greenhouse gas emissions (indicator 1.10.1). Continuity with past work was also hampered by external factors such as changes in government and staff turnover and by internal factors such as low UNDP staff morale and weak monitoring data overall.
  12. Other projects, such as the eight small-scale initiatives tackling livelihoods at local level (output 1.7), made commendable efforts in participatory planning and implementation (e.g. with pasture user groups) and inclusion of women (e.g. collection of gender-disaggregated data for a baseline study,  gender-balanced participation at trainings). These projects did not however achieve their stated aim of boosting the incomes of affected communities and are a salient example of programme fragmentation (see finding 1) and high managerial cost. Taking a step back, these small projects throw up the question of balancing downstream work, potentially bringing tangible benefits to rights-holders in a shorter time-frame, and upstream work, tackling longer-term policy and institutional changes, within outcome area 1 and for UNDP Mongolia on the whole. This may require rethinking. 
  13. Air pollution in Ulaanbaatar is a major concern. UNDP aimed to foster equitable, sustainable and low carbon urban development through support to Ulaanbaatar City’s SDG Road Map and the NAMA project mentioned above. The SDG Road Map was implemented on time[17] but – in the view of this ICPR – has several flaws, including lack of focus on poverty reduction, employment and associated gender issues, a confusing M&E framework, and low feasibility in terms of implementation. The NAMA project has successfully addressed part of the barriers to adopting energy efficiency technologies in the construction sector but its contribution to actual emission reduction over the course of the current planning cycle is limited.[18] Going forward, as noted by its terminal evaluation, future NAMA like interventions should be embedded in the Nationally Determined Contributions framework rather than be “separately institutionalized”, ensuring a common understanding of priorities among stakeholders. Lesson learning from this innovative effort is crucial in addressing future long term constraints in promoting energy efficiency in the construction sector in Mongolia.    
  14. Finding 4National capacity building to implement and manage green development in Mongolia yielded mixed results. Despite a favourable legal and policy environment, declining development finances and a lack of political will significantly hindered progress. This presents significant challenges to maintaining the critical ecosystem services needed to support the livelihoods of vulnerable groups.
  15. Mongolia is experiencing increased land and pasture degradation, some desertification due to mining and overgrazing in rural areas, as well as water deficits due to climate change. The legal environment for sustainable management and use of natural resources is strong, and aligned with international norms and national priorities.[19] Mongolia’s weak environmental performance in spite of this reflects the limited capacity of national and local stakeholders to implement and enforce the legal framework, including the lack of transparency in licensing allocation and permits enforcement, and limited civil society participation in natural resource management processes.[20]
  16. During the period under review, UNDP  exceeded its target of expanding the areas of state and local protected areas (SPA; LPA) in Mongolia by 0.6 per cent – this through the cumulative effort of three interventions which started between 2012-2015 in the past CPD and ended in 2017-2019.[21] IEO accounted for an expansion of 4.99 million ha (4.82 million ha of LPAs and 0.17 million ha of SPAs) through the “Land Degradation”[22],  the “Managed Resource Protected Area”[23], and the “Ecosystem-Based Adaptation”[24] projects. Relative to the total land area of Mongolia (1,553,560 km2)[25], this represents a 3.21% increase of specially protected areas which is higher than the 0.6% targeted by the Country Programme (see output 1.5 in annex 1).
  17. UNDP contributed to improving planning capacities of local governments to implement land degradation offset and mitigation initiatives, as well as ecosystem-based adaptation frameworks.[26] There is some evidence that capacity building for officials of the Ministry of Environment and Tourism improved draft legislation regarding biodiversity offsetting, better management of natural resources and rehabilitation of degraded land after mining activities and the promotion of human rights. Similarly, efforts were noted in strengthening the rule of law and mutual responsibility of stakeholders in decision making processes related to social inequality, environmental degradation and human rights violations in the mining sector. [27] UNDP also achieved moderate results with regard to enhanced community participation in the management of local protected areas and improved stakeholder engagement in support of the promotion of sustainable value chains, notably through the establishment of the sustainable cashmere platform, which provides a framework for leveraging private sector resources and to link herders to fair markets.[28] Factors that enabled the achievement of those results were UNDP’s holistic approach to natural resource management; the application of cross-sectoral and landscape-level planning; the existence of an adequate policy and regulatory framework; the use of learning-by-doing approaches in technical capacity development; and the increasing participation of national and local stakeholders, including local governments, community-based organizations, and the private sector, notably mining companies (cf. supporting evidence in output 1.3-1.5 in annex 1).
  18. Other planned UNDP interventions did not yield the expected results, or only to a limited degree. This includes national capacity building in the area of biodiversity conservation (Nagoya Protocol and the Biodiversity Finance Plan) and resource mobilization in support of inclusive and sustainable livestock value chains, namely the targeted private equity investment. The small-scale nature of training activities[29] and limited evaluation of trainings were also of concern, both in terms of efficiency and in terms of potential for scale-up. Other challenges included frequent changes in government leadership, the limited fiscal space of the government of Mongolia due to instability in the mining sector recently exacerbated by the COVID-19 pandemic, and lack of political will, especially in support of activities addressing overgrazing in rural areas, as manifested in slow vertical funds implementation by the GoM and delays in approving the laws, guidelines and strategies generated through UNDP support. Further, limited cooperation with other UN-agencies (e.g. with Food and Agriculture Organization (FAO) on the promotion of a sustainable livestock value chain) and the limited availability of human resources[30] at the CO level were hindering factors. There is a substantial risk for UNDP to lose ground in one of its key and traditional areas of work with the GoM, given the overall reduction of international development financing flow to Mongolia[31] and the overall drop in GEF funding (which accounts for three of the five projects in this portfolio).
  19. Outcome 2 - Voice, participation and accountability

RQ1: What progress has UNDP made towards planned country programme outputs, and how is this contributing to UNDP/UNDAF outcomes in the current programme period?

 

Overview of outcome 2 (“By 2022, governing institutions are more responsive and accountable to citizens, while ensuring effective participation of young people and realization of the rights of the poor and marginalized”) progress and ratings:

 

UNDP’s contribution to meeting outcome level targets was rated as “moderate plus”. In other words, UNDP influences improvement of voice, participation and accountability in Mongolia to a moderate or higher degree. UNDP is on track to meet targets for the outputs on women’s leadership, boosting civil service effectiveness and extractive industries governance. It is however off track to reach the indicator target on citizen engagement. Significant contributions were made to indicator 2.2.1 (number of women benefiting from measures supporting their leadership role), indicator 2.3.1 (number of government agencies adopting informed anti-corruption plans), indicator 2.4.1 (number of complaints received) and indicator 2.4.2 (HR procedures meeting minimum benchmarks). UNDP’s role in achieving the targets for indicators 2.1.1. (number of public hearings) and 2.1.2 (effective mechanisms to engage civil society) was weak. While outputs are appropriately linked to the outcome level, and there is a line of contribution from UNDP to changes in the UNDAF outcome and associated indicators, the contribution of other factors is hugely significant (i.e. the political, social and cultural factors contributing to good governance). Although outcome indicators include a focus on youth participation, there was no corresponding UNDP effort or output indicator that targeted young people (or marginalized groups).  

  1. Finding 5UNDP is uniquely positioned for governance work in Mongolia. However, UNDP currently has a relatively narrow governance portfolio. Unlike in the past, it is not supporting wider initiatives on government accountability (citizen and civil society engagement or anti-corruption measures).
  2. UNDP in Mongolia is recognized as a key government partner and a facilitator of multi-stakeholder dialogue on governance reform. UNDP has a very strong reputation in the country,[32] especially among government officials, based on a legacy of heavy support and engagement for 40 years, especially during the transition to a market economy and multi-party system after the fall of the Soviet Union. The Partnership Survey 2019[33] found that governance support is the most valued UNDP contribution in the development of the country. UNDP is therefore well-placed to pass some tough messages, in line with its human rights-based approach to development: i.e. serving citizens, the rights holders, and being “downward accountable” for its actions is the duty of the Mongolian government. Citizens of all walks of life have a right to be heard and participate in decisions that affect them.[34] The review detected some path dependency in the CO’s engagement with the GoM[35], focusing somewhat narrowly on local governance, capacity development, women’s political leadership, citizen engagement and civil service reform and avoiding more sensitive issues. Key informant interviews (UN Agencies and civil society representatives) and a scan of UNDP public statements made in Mongolia testify to UNDP’s limited engagement when it comes to promoting the United Nations’ human rights-based agenda, including speaking out against corruption and giving voice to those that are silenced.
  3. Civil society voice can be a major conduit for enhancing government accountability and is an area where UNDP has significant experience globally. Improving engagement with civil society in Mongolia has been a recurring recommendation from past evaluations.[36] Recent CO management responses suggested that the office would embark on new initiatives focusing on youth and citizen engagement and implement a whole-of-society approach, for which a strategy for engagement of CSOs in SDGs implementation was to be developed. However, this review was unable to identify a CSO engagement strategy, platform or framework. CSO-related initiatives such as the “Sounding Board”, the small grant scheme at local level, the “model session procedures” for citizen-representative hurals (CRH)[37], support to preparing the CSO report to the Voluntary National Review and youth engagement for building synergies between institutions, private sector and youth are commendable but do not add up to a coherent or strategic whole.
  4. The civil society sounding board met in September 2020 to comment on seven ‘transformative missions’ that UNDP Mongolia hopes to embark on in the future (as identified by the CO’s reference group). While this engagement is clearly a step in the right direction, the feedback received was that speaking time allocated to sounding board members was too short to make significant proposals for change, suggesting that validation rather than critical comment was sought. The small grant scheme (as part of output 2.1) in principle could be an important contribution to fostering civil society engagement in decision-making processes at local level – thus creating precedents for participatory decision-making, budgeting and oversight. However, it is very limited in scale and has not performed strongly. In total, 23 grants (24 percent of the total) were implemented by NGOs over the last 4 years (2017-2020), and challenges were encountered in implementation processes and pairing NGOs with citizen-representative hurals (local councils).[38] Interviews with civil society representatives also noted limited space and knowledge to engage with the government, at both national and local levels. There was a general perception among key informants that UNDP has been less actively engaged with civil society in recent years than other UN Agencies.  
  5. Fighting corruption was singled out in key informant interviews with government and partners as an area where UNDP’s normative voice could be louder. The CO’s only direct intervention is focused on corruption in the extractive industries, while work on ethics in the civil service is more indirectly concerned with corruption, among other issues of professional integrity. UNDP’s policy work has effectively informed the GoM Action Plans in this particular area, but is not fully capitalizing on past work to prevent corruption across government and society on the whole to sustain development gains.[39] According to key informants (partners, donors, government and civil society), future initiatives could support other decision-making processes that are susceptible to corruption, e.g. in health and management of resources, and bolster digital transformation of government services.  Current UNDP efforts on accountability for a professionalized, ethical and performance-based civil service, the development of gender-sensitive human resource procedures and the establishment of Ethics Councils in the view of this ICPR are strategic and need to be pursued and strengthened further. There is momentum in the GoM to strengthen enforcement of laws and procedures for the ethical behaviour of public officials. This is timely as the inflow of COVID-19 related response and recovery funds, too, will require transparent and accountable governance that is able to deliver services effectively.
  6. Finding 6UNDP has progressed, with good results, in strengthening legislation, reforms, and capacity strengthening of the civil service, and enhancing the voice and representation of women in political processes. These programmes benefitted from solid partnerships and UNDP’s positive legacy and reputation in the governance arena. Achievement of results was hampered by the long-term nature of envisioned change paired with a lack of intermediate level progress indicators and absence of experimentation.
  7. UNDP support has been instrumental for the GoM to progress towards strengthening the legal framework for local governance. UNDP’s technical assistance on the Constitutional Review (2015 and 2016) informed amendments[40] to the Constitution passed in 2019. CO is on track to achieve intended results relating to strengthening the legal and policy framework for an independent, professionalized, and performance-based civil service. Some achievements include UNDP’s technical assistance to drafting the Regulation on Developing Job Descriptions in 2019, and improvements to public scrutiny of draft laws and public administration performance. UNDP has reached its targets with regard to enhancing the capacity of local councils through training for local officials to fulfil their functions. The development of guidelines for building procedures for dialogues with citizens, trainings, small grant programmes for citizen-representative hurals, technical support and knowledge management and, tailored and dedicated interventions for women are also on track. Affected changes in the functions of local councils represent important developments against the backdrop of low levels of citizens' trust in political institutions.[41] Some citizen-representative hurals are also cost-sharing training costs which is an indication of the commitment and future sustainability of the initiative. [42]
  8. Leadership training for female elected local representatives[43] focused on problem-solving by engaging different local actors and served as a catalyst to address gender issues. For example, UNDP’s nation-wide initiatives in the realm of women’s political empowerment were crafted around capacitating and giving confidence to locally elected women to effect change, promoting their achievements and encouraging them to re-run for elections or compete for higher level positions. 2000 elected women representatives participated in total. The review found some degree of evidence of women acquiring greater confidence and influence in their decision-making role at local council level, of better networking and creation of alliances between women across different levels and across parties.[44]
  9. Success factors included political consensus on the relevance of legal reform and administrative decentralization, a solid legal framework, and growing interest nationally in citizens’ voice, including responsible corporate citizenship. UNDP’s reputation on governance issues and its long-standing partnerships with key governance donors, such as the Swiss Agency for Development and Cooperation (SDC),[45] were also instrumental. Factors hindering achievement of results included the lack of a national, all-party consensus on priorities for civil service reform (such as how far to push in terms of a gender-transformative agenda), high staff turnover especially at local (hural) level, low legal and policy expertise at the level of the parliament secretariat, and continued politicization of the civil service. In addition, a male-dominated political culture and persistent gender stereotypes, as well as unavailability of funds and a lack of performance reporting on outcome level changes (economic, political, leadership status of women, intended or unintended effects of norms and rules, etc.) undermined progress.
  10. Partners and government officials pointed toward the need for piloting and demonstrating the impact of policies at local level for the governance programme. While policy related support at central level was deemed critical in advancing reforms, translating policies into action and tangible results on people’s lives was also seen as a crucial component that could be strengthened. One good example of policy innovation and implementation was noted under the policy innovation challenge which contributed to doubling the voluntary pension enrolment of soum herders during the challenge period.

Programme adaptation in response to the Covid-19 pandemic

RQ3: To what extent is the adapted programme in sync with relevant local coordination systems and with the efforts of other key actors?

UNDP Mongolia’s country programme has not been significantly adapted in response to COVID-19. Rather, some specific activities have been added by fitting them into the existing programme logic. The programme thus continues to be in sync with relevant coordination systems, chiefly with the UN Country Team through the UNDAF. The broader question (which is beyond this review to answer) is whether a more significant overhaul might have been called for given the depth of the current crisis. Through its COVID-19 response, and with new leadership, UNDP has diversified its partnerships with other key development actors: UNDP recently collaborated with UNICEF, ADB, FAO and others and a civil society ‘sounding board’ has been created. While adding some initial confusion, delinking from the UNRC role may have contributed to enhanced relations with partners (see also finding 9).

  1. Finding 7UNDP’s financial and technical assistance to the national COVID-19 response was timely and supported evidence-based decisions, including support to vulnerable groups.
  2. The CO responded to COVID-19 with new projects and, to a lesser degree, by rethinking its strategy for ongoing initiatives. The delivery rate stands at 49 percent at the time of writing.[46] Following the lockdown of borders at the end of January 2020 to reduce transmission from China, the Deputy Prime Minister’s Office requested UNDP to commission a “Rapid Socioeconomic Impact Assessment of COVID-19 on Vulnerable Groups and Value Chains in Mongolia”[47] which became a key guiding document. Stakeholders unanimously commended UNDP Mongolia for its rapid response and thought leadership on this exercise. There is some evidence that the COVID-19 Law passed in April 2020 was influenced by UNDP’s draft initial assessments and advocacy, which profiled the impact of the crisis on those “left behind” such as the elderly, women-headed households and the families of herders. Through this initiative, and its strong history of poverty research, UNDP is now well positioned to engage more fully in social protection work in Mongolia – a remark and request made by several stakeholders interviewed.
  3. Within the UN system, UNDP contributes to the Socio-Economic Response Plan (SERP) as technical lead for the socio-economic response, under the guidance and coordination of the UN Resident Coordinator. UNDP effectively collaborated with many other entities in this role, including UNFPA, UNICEF, FAO, International Labour Organization and International Organization for Migration, but some criticized UNDP’s tendency to support the government directly, rather than through a concerted UN effort. As of September 2020, Mongolia had received over $500M in COVID-related support.[48] By May 2020, UNDP had received over US$2.5M in additional funds for its COVID-response and the funding gap for further planned activities under the SERP stood at US$24M.[49]
  4. Two aspects stand out from UNDP’s engagement in the COVID response. First, the agency was able to sharpen its profile as a thought leader and analyst on “big picture” work and identifying effective levers of change. Going forward, and building on this experience, COVID may have been a turning point helping UNDP Mongolia move away from project implementation (other than to demonstrate concepts) and raise its engagement to a more strategic level. It is noteworthy that UNDP Mongolia successfully leveraged policy capacity from the UNDP regional hub and headquarters as part of the analytical and assessment work, thus demonstrating UNDP’s knowledge exchange capacity. Second, UNDP Mongolia used the COVID crisis as an opportunity to rethink its “strategic argument” and programmatic approach. Proposing a “transformation journey”, the CO draws on systems thinking (including “sensemaking”) and portfolio approaches to identify seven “transformative missions” to address the development challenges faced by Mongolia in the COVID-recovery and beyond. The seven missions highlight a “green recovery” focused on low carbon transition, 21st century value chains and right-sized herding, supported by integrated technology platforms and next generation governance to achieve a rebalancing of the economy and public interest and dignified lives for all.[50] The recently raised GCF funding ($23M) can support this agenda.
  5. Finding 8No notable efforts were made to keep country programme documentation (theory of change, results framework) current, despite stark changes in context/assumptions and crippling resource constraints.
  6. Corporate mechanisms so far suggest that COVID-19 is a further layer in UNDP programming (through additional financing, reporting in a Mini-ROAR and on UN/UNDP indicators) and no call has come to question what the organization does elsewhere. It is therefore perhaps not surprising that the forward thinking described in the previous paragraphs is not matched with full-blown adaptive management at UNDP Mongolia with regard to the existing country programming approach. Small programmatic adjustments have been made due to COVID-19, including in the biodiversity project ENSURE and the cashmere platform, but no major overhaul was seen since the programme was first designed in 2017 – including when resource mobilization was stalled for several years on some output areas. In this process, country programme documentation was not kept fully current: This review found no evidence of rethinking (or finalizing) the current theory of change or CPD RRF for instance, or improvements and updates to the M&E system (removing finished projects, revising and updating indicators and targets). Recently, the approach taken was to add COVID-related results under existing outputs – in some cases to beef up performance for outputs otherwise “off track” (e.g. output 1.8[51]). The ICPR found limited evidence of COVID reprogramming adversely affecting existing programmes but this should be watched closely.
  7. In 2020, the CO was selected by HQ to conduct a ‘sense making exercise’ to assess the validity and relevance of the programme document and interventions to contribute to the CPD results framework. The exercise recommended UNDP Mongolia to move from a solution provider to a solution facilitator and to embed innovative approaches in its programme design and implementation strategies. This should include adoption of a systems thinking approach to inform the upcoming Common Country Assessment and ToC discussion, ahead of formulating the new country programme and its results framework. It is not clear whether or how the results of the exercise are being integrated and utilized in the current programming cycle. The seven transformative missions put forward in the ‘strategic argument evolution’ appear to relate to the next programme cycle starting 2023.

 


[1] Further, the Poverty portfolio was discontinued and merged with the Governance portfolio, leaving some smaller projects ‘floating’ that were also added to outcome area 1, e.g. ‘Activated2030 #Hub: A Youth Enterprising Lab’ in outputs 1.1 and 1.9. This contributed to the measurement challenges described in paragraph 19.

[2] UNDP Mongolia, ‘Theory of Change for UNDP Country Programme Document 2017-21 in Mongolia’ (DRAFT), UNDP, 2016, p. 6. Similarly, the CPD notes: “The nexus of economic growth, social development and environmental sustainability will be the focus of the programme, with governance as a key enabler”; UNDP Mongolia Country Programme Document, 2017-2021.

[3] Figure 15 (Annex) shows that the overall number of projects active with UNDP Mongolia decreased slightly from the past programme cycle (2012-2016) while average expenditure per year per project increased. However, this is due to effective consolidation of efforts under outcome area 2; outcome 1 remains highly fragmented.

[4] Fiscal space is the budgetary room that allows a government to provide resources for public purposes without undermining fiscal sustainability. In Mongolia this space is small as government cannot raise spending or lower taxes without endangering market access and putting debt sustainability at risk.

[5] Net ODA received as a percentage of GNI and government expenditure decreased by 21 and 8 percent respectively between 2017 and 2018 (see figure 3, Annex).

[6] In other cases, there is no coherence between project indicators and CPD RF output indicators making project dashboards appear “green” and assessments against the CPD RF appear “red” which looks like a contradiction.

[7] “Mongolia Sustainable Development 2030” was approved in 2017 but subsequently rescinded, in line with the amendments to the Law on Development Policy, Planning, and Administration of 2020.

[8] The main concepts and principles of the SDGs as propagated by UNDP are included. Specifically, one input from 2015 that still resonates is the “Gender Baseline Analysis against SDGs in Mongolia” which supported national target setting for “Mongolia Sustainable Development 2030”. Ulaanbaatar City's Development Vision (now part of Vision 2050) also incorporates the main SDGs concepts.

[9] Mostly for reason beyond UNDP such as changes in the government and political interests.

[10] UNDP Mongolia Annual Business Plan 2020.

[11] In 2017, the Sub-Committee on Sustainable Development Goals under the Standing Committee on Social Policy, Education, Culture, and Science of the State Great Khural was established. The committee is in charge of monitoring the implementation of the Mongolia Sustainable Development Vision-2030 and SDGs. In addition to the sub-committee, a National Council for sustainable development was established, also in 2017. The council is mandated to provide technical policy support to the GoM in mainstreaming the SDGs into strategies and ensure the implementation. Although the establishment of the council and the committee took over a year, there is no evidence of either having met or started work. According to stakeholders, members’ commitment is insufficient to effectively mainstream the SDGs.

[12] It should be noted that the project to establish the International Think Tank for Landlocked Developing Countries – government-run and fairly successful in its own terms – did not further UNDP’s standing as a thought leader as the think tank is external and independent and has by now graduated from UNDP support. There is no evidence of contribution to SDG integration in Mongolia beyond project closure in 2018 and limited evidence on contributions to South-South cooperation in key UNDP mandate areas.

[13] United Nations Mongolia, “2017-2019 UN Mongolia Country Results Report (UNDAF 2017-2021)”, 2020, p.14. According to key informants, the MoF is likely to extend the pilot projects into the social welfare sector.

[14] NGO/CSO engagement was mainly through the small grants and the sounding boarding meetings. Public consultation processes on legal framework revisions were the main platforms for multi-stakeholder engagement. The two main private sector coordination bodies, the Business Council of Mongolia and the Mongolian National Chamber Of Commerce And Industry, seem not to have been considered as UNDP partners. They are large membership-based coordination mechanisms aiming to create an enabling environment for inclusive economic growth by leveraging laws and policies and policy dialogue. 

[15] The smart phone application, developed in cooperation with the National Emergency Management Agency, aimed to raise awareness and improve stakeholder skills for disaster risk reduction and response.

[16] The CPD outputs against which NAMA was assessed against are output 1.6 (Setting up the institutional and legal framework for the implementation of mitigation and DRR measures) and output 1.10 (fostering the adoption of low carbon and energy efficient technology by city governments).

[17] It is the review’s understanding that Ulaanbaatar City’s SDG Roadmap was approved but subsequently put on hold.

[18] While the potential lifetime energy saving and GHG emission reduction of the five demonstrations projects was estimated at 134 Gwh and 48,140 tCO2eq for a conservative 14-year project life time, actual potential GHG reduction for the three years between the end of this project and the current country programme is about 10% of the target set under output indicator 1.10.1.

[19] In the past 25 years, Mongolia has approved more than 30 environmental laws and several hundred environmental regulations, bylaws and policies, all the result of decade-long efforts of UNDP and other agencies such as FAO, World Bank and SDC. See: Final Evaluation of Strengthening Environmental Governance In Mongolia - Phase II, UNDP, 2014.

[20] Evaluation of UNDP’s development cooperation in middle income countries; Mongolia case study note (2020).

[21] Terminal Evaluation Report, Land Degradation Offset and Mitigation in Western Mongolia, 2019; Terminal Evaluation Report, Ecosystem Based Adaptation Approach to Maintaining Water Security in Critical Water Catchment in Mongolia, 2017; Terminal Evaluation Report, Mongolia’s Network of Managed Resource Protected Areas, 2018.

[22] Terminal Evaluation Report, Land Degradation Offset and Mitigation in Western Mongolia, 2019.

[23] Terminal Evaluation Report, Mongolia’s Network of Managed Resource Protected Areas, 2018.

[24] Terminal Evaluation Report, Ecosystem Based Adaptation Approach to Maintaining Water Security in Critical Water Catchment in Mongolia, 2017.

[25] See: https://data.worldbank.org/indicator/AG.LND.TOTL.K2?end=2018&locations=MN&start=2018&view=bar

[26] Refer to supporting evidence of output 1.3-1.5 in Annex 1.

[27] Environmental Governance for Sustainable Natural Resource Management (SEPA EGP).

[28] Official Launch of Mongolian Sustainable Cashmere Platform on 20 November 2020.

[29] BIOFIN capacity development activities were limited to (a) a pilot training on "Environmental planning and financing" in cooperation with the National Academy of Governance and led to the development of an online training module; (b) a study tour on inclusive meat business for representatives from two locally protected areas; and (c) a training session for officials from the National Centre for Livestock Gene Bank on improving the nucleus herd.

[30] While available human resources are adequate for the implementation of current projects in the natural resource and climate change portfolio, there is a lack in leadership, at the cluster level, to push for resource mobilisation in this area and support upstream activities.

[31] See figure 2 in the Statiscal Annex.

[32] Key informants noted that “UNDP is central to Mongolia’s identity”, “everybody takes UNDP seriously”, and “no door is shut to UNDP”.

[33] UNDP Partnership Survey 2019, Mongolia respondents 28 in total, government 8, civil society 6, UN nations 4, partners, private sector 4, bilateral agencies 2, non-UN 2, IFI 2. 

[34] “A human rights-based approach is not only about expanding people’s choices and capabilities but above all about the empowerment of people to decide what this process of expansion should look like”. “It not only defines the subjects of development, largely confirming UNDP present policy priorities, but it also translates people’s needs into rights, recognising the human person as the active subject and claim-holder. It further identifies the duties and obligations of those against whom a claim can be brought to ensure that needs are met.” UNDP, A Human Rights-based Approach to Development

Programming in UNDP – Adding the Missing Link. UNDP 2015.

[35] KIIs noted that UNDP’s way of interacting with the GoM was entrenched in certain patterns of behaviour and around certain governance topics, without much room for innovation. 

[36] ADR Mongolia 2011, CPAP Outcome Evaluation 2015.

[37] The model CRH session procedure is a good step towards institutionalization: It has articles on engaging the public, and most CRHs adopted the model session procedure after the 2020 elections. Moving from procedure to widespread routine practice remains challenging. 

[38] There are plans for hurals to be paired with appropriate NGOs or local resource persons to support project implementation. It is unclear whether this engagement would add up to an established platform and how progress would be monitored. 

[39] Project “Support to Integrity and Transparency Efforts in Mongolia”, 2008-2011; comprehensive study/review of the constitution to inform the parliamentary working group; support to drafting of legislation and policy, including the 2015 national anti-corruption strategy.

[40] The amendments supported by UNDP ensure consistency of development policies and stability of planning processes despite changes in government. Two out of three amendments relate to local governance. These were informed by studies commissioned by UNDP, as well as an implementation review of a Law on Administrative and Territorial Units and their Governance.

[41] SRBM Mid Term evaluation 2018

[42] CHRs sharing 11 % of the training costs

[43]Women’s Leadership Training (WLT) was organized for the 2,164 female CRH representatives across the country. conducted in all hurals: 21 aimags and their soums, and city and district hurals.                                             

[44] MIC Evaluation 2019, SRBM Mid-Term Evaluation 2018.

[45] SDC has been funding the local governance programme since 2008 but will leave Mongolia in 2023.

[46] Source: Atlas-Power-Bi 2020; accessed 22 December 2020. Besides COVID-19, general and local elections may have slowed down delivery in 2020.

[47] UNDP Mongolia, Rapid Socio-Economic Impact Assessment of COVID-19 Prevention Measures on Vulnerable Groups and Value Chains in Mongolia, UNDP, June, 2020; UNDP Mongolia (fliphtml5.com)  

[48] Mongolia received an estimated $563.4M in COVID-related support (including loans) from the WB, International Monetary Fund, ADB, EU, United States Agency for International Development, UN and others. The majority of the funding was from IFIs and earmarked for new programmes, in addition to some repurposing of existing programmes. Some loans may be concessional, which may lead to debt issues later. https://breathemongolia.org/news-article/a-summary-of-covid-19-relief-and-support-received-by-mongolia/

[49] Funds received include: Roughly US$1.1 million from the Japanese Supplementary Budget Fund (JSBF) to support the social welfare information system and to digitally empower social workers that are in the frontline of response; US$836,954 from the Joint SDG Fund with UNICEF in July 2020 for SDG financing and development of an Integrated National Financing Framework for Mongolia to maintain and accelerate the progress on SDGs in and beyond COVID-19; $50,000 from Trac3 for a rapid socio-economic impact assessment; $120K from Trac2 to address disrupted value chains; $192,600 of Rapid Response Funding to assist the Government in developing a national COVID-19 recovery strategy that comprehensively responds to economic and social dimensions based on socio-economic impact assessments; A US$350,000 proposal to support a green recovery by providing technical assistance to develop new green financial Instrument-Debt for Climate Swap was submitted under the Rapid Recovery Framework. UNDP, Covid-19 ROAR (Mongolia: UNDP, May 2020).

[50] UNDP Mongolia, ‘Strategic Argument Evolution’, Version 14, UNDP Mongolia, September 2020.

[51] UNDP Mongolia - Engagement Facility -COVID-19 Socio-Economic Assessment and UNDP Mongolia - Engagement Facility-Covid19 - Rapid Response Facility (2020-2020).

 

  1. Finding 9UNDP’s vision for how it intends to add value to development in Mongolia needs updating, clearly bringing out its niche and added value in a changing development landscape.
  2. As an international organization, UNDP is known for intellectual leadership on human development paradigms; a history of universal presence; and strong institutional networks with multiple ministries per country, enabled by its broad mandate. In Mongolia, the path forward is muddled by many conflicting demands. On the one hand, the UNDP Mongolia is yet to fully live up to its middle-income country mandate of being an analyst and thought leader regarding the ‘big picture’ of Mongolian development challenges and landscapes. This would include the capacity to identify real needs, effective levers and mechanisms for change and tackle some strategic areas that are perhaps left unaddressed by other actors in Mongolia, bringing in policy capacity from regional hubs/HQ as well as independent international expertise and best practice. This vision is at odds with UNDP being an implementer of (micro-)projects or an overall coordinator of the UN system and suggests some degree of decoupling from government as a sole and privileged partner. On the other hand, progressive positioning at subnational and local levels to support last-mile challenges, and a renewed emphasis on inequality, vulnerability and exclusion have also emerged as key priorities, recognizing that many challenges remain despite middle income status, including the distribution of wealth and the environmental consequences of accumulated wealth. UNDP Mongolia’s decisive move into the innovation space, with the Resident Representative part of UNDP’s NextGen leadership, an A-Lab slated to open in 2021,[1] and ‘sensemaking’ and visioning (strategic argument evolution) exercises ongoing, will need to be supported by experimentation and small-scale demonstration projects on the ground. This requires risk-taking and appreciating failure as a strategy for achieving success in the long run.
  3. Decoupling from government and embracing a wider partnerships portfolio is also challenging. UNDP has long been the “go to” agency for the GoM, a reliable implementer committed to supporting country-led efforts for achieving the 2030 Agenda, and sourcing its revenue from projects that directly involve partnership with the GoM. It is unclear from this review if government would be open to receiving more innovative and potentially disruptive support from UNDP, even if free of charge, and raise the level of engagement to a more strategic level. For instance, one idea floated was about UNDP facilitating macro-economic advisory to the government, from a “leave no one behind” perspective. While UNDP’s COVID-related support suggests such an opening, other evidence, including key informant interviews, suggest otherwise. Finally, to leverage resources, for instance loans received from IFIs or grants in the environment/climate realm, UNDP remains tied to the government as line ministries would need to ‘outsource’ funds to UNDP.
  4. Last but not least, UNDP is still finding its niche as an ‘integrator’ for development in Mongolia and seems to struggle to communicate the concept and operationalize what it means in implementation. UNDP was thus far unable to address stakeholders’ confusion about the role of the UNDP Resident Representative versus the UN Resident Coordinator. Multiple key informants noted that “the divorce from the RC role has been brutal”, and more so since it happened during a pandemic where, because of urgency and habit, Mongolian officials continued to “go straight to UNDP”. 
  5. While generally collaborating well with other bilateral and multilateral actors, full-blown partnerships with other UN agencies have not been frequent during the programme cycle examined. Rather, UNDP had a tendency to “go vertical” and work straight with Government. At the strategic level, UNDP thus operated mostly on its own in its traditional domains of comparative advantage such as governance and environment, despite the presence of other relevant agencies. The review found some evidence of UNDP working with sister UN agencies at the project level, e.g. the joint UNDP-UNEP work on the Poverty-Environment Initiative, with UN Women and UNFPA on the governance portfolio, with UNEP and FAO on UNREDD+, and with UNICEF on air pollution The SDG implementation work requires partnerships with more (and perhaps different) UN agencies. Some movement in that direction was noted over the course of 2020, such as collaboration with both UNICEF and UNEP on SDG budgeting.
  6. Finding 10The programme cycle was characterized by instability in terms of financial and human resources which hindered full realization of the planned programme. UNDP Mongolia funds only 9% of its programme from core resources and receives limited government cost-sharing. It thus remains heavily dependent on donor funding, Vertical Funds in particular, which is not always disbursed on time. Staffing fluctuated greatly with downsizing at the beginning at the programme cycle and expansion expected for 2021/2022 with the arrival of long-awaited non-core resources.
  7. Resource mobilization in the context of decreasing ODA (see figure 3, Annex 2) and a tightening donor landscape[2] has been a challenge for the office early on in the current programme cycle.[3] The CO has planned for a budget of $50.47M across the two outcomes over the CPD cycle. In the first 3 ½ years of the six-year cycle, it was able to secure only 30% of the planned cycle funds and expensed just under $5M annually, the vast majority of which from non-core funding (see figure 4, Annex 2). UNDP Mongolia receives the lowest amount of core resources in the region ($350,000 per year) which funds 9% of the programme.
  8. There is an expectation for government cost-sharing, given Mongolia’s middle-income country status and the huge mineral wealth in the country, in line with UNDP’s business model (national implementation modality, NIM). NIM gives greater ownership to the GoM which can enhance uptake of policy advisory and sustainability of programmes, as clearly noted in this review with local government cost-sharing in Mongolia. Of concern is the size of cost-sharing overall which stands at just 4% of the CO’s total expenditure to date.[4] While there are many reasons for the slow shift into this direction in Mongolia – including pressures related to the debt crisis, shrinking fiscal space, and now the COVID-19 pandemic – it is also an indication that UNDP’s business case may not yet be entirely convincing. This is a vicious circle as an office “busy reacting to donor priorities”, as one key informant put it, will have trouble defining its niche and raising resources for its own agenda. UNDP Mongolia thus remains heavily dependent on Vertical Funds for its programme, particularly from the GEF and the GCF. These funds come with strings attached, including parallel M&E systems and minimal staffing, and funds agreements can involve lengthy negotiations and preparations, as seen in case of the GCF funds ($23M) approved only in late 2020.
  9. However, new funding opportunities were successfully seized in 2020.[5] First, related to COVID-19, the CO received roughly $1.4M for response and recovery. Second, UNDP’s SDG financing proposal under the UN Joint SDG fund, which includes government cost sharing, was successful and will be implemented with UNICEF and UNFPA. Third, two large initiatives in pipeline were signed in 2020, to include a €7.4M EU-funded project on public financial management, to be implemented with FAO and ILO, as well as a $23M GCF project on climate adaptation. Other projects in the pipeline are KOICA on gender equality in political decision-making estimated at $4.8M and smaller initiatives with ILO (debt swap and eco-tourism). This is emblematic of the current momentum on resource mobilization – which was however not yet used to leverage ethical and appropriate private sector partnerships and funding directly from International Financial Institutions. There is also room for improvement in donor coordination, along the lines of the Cashmere Platform which set a positive example for systems thinking.
  10. An important downsizing exercise took place at UNDP Mongolia at the end of the last programme cycle, in 2016.[6] The CO currently has 34 personnel – 14 fixed-term appointments, 16 on service contracts funded by projects, and an additional 4 personnel on service contract for administrative backstopping, including to the Resident Coordinator’s Office and non-resident agencies. The programme portfolio is carried by only three programme officers at NOA/B level; with the Assistant Resident Representative post at NOC level frozen. The review found that UNDP Mongolia is not adequately staffed to effectively play its role as an integrator: while more junior staff bring enthusiasm and can drive the CO’s innovation agenda, the office lacks mid-level/senior officers who can interact with other intergovernmental organizations, leverage resources, and above all provide high-impact policy advice. Partners in government and other international organizations noted that UNDP seems overstretched and that this may be affecting effective programme management, especially M&E and reporting. Delivering on the current portfolio, additional COVID-19 related work, and portfolio expansion e.g. in the area of SDG financing, will affect staff capacity and UNDP Mongolia’s business model.[7] The CO is aware of these challenges and has been proactive in requesting a functional review to ensure that office structure and capacity respond to the changes ahead. As a small office in a middle-income country, the shrinking budget has kept the management efficiency ratio high (24% on average over the period 2017-2020[8]). However, with the additional funds in the pipeline, this is likely to decrease in the near future, providing a potential opening for bolstering the programme team.
  11. At 65%, women represent a majority of UNDP Mongolia’s staff, including in senior leadership and among those holding stable contracts. Given the relatively young demographic of female staff in the CO, and prevailing gender norms in the country (women’s roles as care givers and low awareness of women’s contributions in politics, economy and society), it is crucial that the office allows for flexible working arrangements and continues to rank female staff commensurate with the posts they occupy, and their experience. The CO has not had a dedicated Gender Officer since late January 2018.
  12. Finding 11UNDP Mongolia’s work on governance as well as recent COVID-related initiatives are largely rights-based in design and emphasize “no one left behind” principles. In contrast, most interventions on the environment-poverty nexus have a more indirect focus on gender and support to vulnerable populations.
  13. UNDP Mongolia is missing an office-wide gender strategy, Gender Equality Seal[9], or outcome-specific gender analysis. Entry points for gender-transformation and “leaving no one behind” are therefore not clearly identified. UNDP’s gender marker system (GEN0-GEN3)[10] indicates that 60 percent of UNDP Mongolia’s CPD budget is allocated to projects for which gender equality is a significant objective (GEN2), contrasted with 38 percent of budget intended to contribute to gender equality in only a limited way (GEN1). Most projects rated GEN2 or higher are clustered in the governance area (outcome 2), including one output dedicated entirely to the empowerment of women in political leadership (output 2.2). This output (marked GEN2) is appropriate in the national context, given the low percentage of female elected representatives contrasted with high female educational attainment.[11] This ICPR assessed gender-responsiveness of programmes by independently validating the gender marker ratings  assigned by the CO, based on gender-related results reported in the ROARs and project level reporting and highlighted in key informant interviews. The analysis confirms that the environment-poverty portfolio is lagging behind: in many instances, there were no gender-related indicators in place and no evidence of contributions made to gender equality and the empowerment of women (Annex 5 contrasts the CO’s GEN marker allocation with the IEO assessment).   
  14. Gender thus appears not to be strategically mainstreamed across outcome 1, hampering the achievement of transformative results. Some projects contributed to change at the output level (output 1.4, 1.5; COVID-19 socio-economic assessment). At the level of programme effectiveness, it remains impossible to ascertain who benefited from UNDP’s interventions as data disaggregation is minimal. In addition to missing sex-disaggregation (available only for certain projects such as ENSURE and BIOFIN), the absence of disaggregation for age and disability means that intersectionality cannot be assessed. Gender-specific indicators are overall scarce and of low quality (e.g. Ulaanbaatar City SDG Road Map), and not clearly aligned with official statistics. Available documentation indicates that gender targeting is fairly well applied (e.g. encouraging participation of female participants; outputs 1.4, 1.5, 1.6, 1.7, 1.10), that community-led activities may have some empowering effect on women (e.g. building a reservoir for safe drinking water; participation in decision making in land use and natural resources management; output 1.4) and that the gender-specific data  and policy advocacy in the context of the COVID had a significant effect, pushing the GoM to mainstream gender perspectives in planning and pandemic response. Overall, measurement of UNDP’s contribution to the lives of women and vulnerable groups remains sketchy, with repercussions for decision-making, corporate accountability and learning.      
  15. Under outcome 2, in contrast, gender equality and women’s empowerment are squarely positioned as accelerators of development: gender-sensitive governing institutions and empowerment of female elected officials are seen as hallmarks of good governance. The outcome addressed capacity gaps and engagements for elected women to better conduct their leadership roles in local councils. The outcome more clearly addressed power imbalances in decision-making and access to resources, e.g. through workplace harassment training and investment in gender action plans (civil service), or by providing small grants and involving women in designing solutions (citizen-representative hurals). Efforts to enhance governance in extractive industries however yielded no measurable or perceivable results regarding gender or vulnerable groups.  
  16. Finding 12The CO has implemented an effective approach to monitoring and evaluating the performance of its programme. However, some impacts of UNDP’s work in Mongolia are at risk of staying out of focus due to unfit or restrictive indicator and target formulations, failure to consolidate achievements of related projects and limited harmonization between monitoring frameworks within the same outcome area.  
  17. According to the UNDP Partner Survey, programme implementation and reporting are strong areas of CO performance.[12] Indeed, the ICPR found that all projects reviewed have M&E plans in place – albeit of uneven quality. Further, the CO evaluation plan was fully implemented (13 project evaluations to date completed against 13 planned), with 57 percent of evaluations rated as satisfactory and above, which is well above the regional and global averages. The CO has a good coverage of project level evaluations across the resilience and ecosystem workstream but only one project evaluation pertaining to governance. No outcome evaluations were conducted besides this ICPR, making synthesis work (while not mandatory) an area for further growth and improvement. The CO also assists with M&E training and promotes results-based management practices at government and partner levels. More efforts could be made to implement post-workshop surveys to assess the quality of trainings and track results not captured by current indicators (which are usually at the level of participant numbers by sex).  
  18. The harmonization between project- and donor-specific M&E systems remains challenging. The review uncovered examples where UNDP failed to consolidate monitoring and reporting on its achievements across initiatives and therefore could not showcase the full extent of its results. For instance, three projects on specially protected areas (SPAs) closed in 2018/9 but failed to report on integrated results, including late or spillover effects of the interventions.[13] Similarly, the civic engagement output measured by number of public hearings and mechanisms for engagement is reported as exceeding its targets 6-fold[14] – indicating weak planning and/or poorly set targets. These indicators fail to measure and provide a reliable indication of the CO’s progress towards outputs which, as a result, negatively affects the capacity of UNDP to present a compelling and credible narrative of its achievements. While reporting was largely complete (only one project had significant lacunae), it overemphasized the positive and tended not to discuss – and demonstrate learning from – challenges encountered. The ‘all green’ dashboard initially provided in the self-assessment for this very review is an indication.
  19. Coherence between indicators across the results chain is also limited, i.e. several output indicators do not directly relate to output statements or lack short term and realistic targets. Weak interlinkages between outputs and lack of harmonization between their monitoring frameworks hamper measurement of contribution to outcomes. Many outcome level indicators are not specific to UNDP’s contribution[15] and lack adequate focus on outcome level behavioural changes. More clarity and detail are required on the tools and methods for data collection (means of verification) and how these have been adjusted over time. There is also lack of consideration for gender dynamics (e.g. data disaggregation and specific gender indicators) and for age (e.g. to measure impacts on youth or the elderly).

 

 


[1] Even if initially time-bound and coming with three professional staff but no funds, it is expected to make a dent given the fertile environment (educated population, easy access to Chinese tech, etc.).

[2] Some of the big donors such as USAID have left the country following its graduation to middle-income/stable democracy. With GIZ and the SDC slated to leave by 2023, only the EU and Canada are left as major donors locally.

[3] Key contributors to date were the Global Environment Fund (27%), the Swiss Agency for Development and Cooperation (17%), The Canadian Department for Foreign Affairs, Trade and Development (15%), UNDP regular resources and funding windows (13%) and the UN Multi-Partner Trust Fund Office (11%); see figure 6 in Annex.

[4] The trend is upward: In 2019, the CO has mobilized $200,000 through government-cost sharing under the SDG localization project; in 2020, it has mobilized government resources for SDG financing.

[5] UNDP Mongolia, ‘Annual Business Plan’, accessed August 2020. 

[6] According to key informants, staff experienced the restructuring as demoralizing, particularly the downgrading of some positions and the departure of international staff.

[7] The incoming vertical funding in particular poses a problem: UNDP Direct Project Costing means that the CO can charge only minimal staffing costs to vertical funds and may struggle to recover the costs entirely. The CO has been absorbing, with its financial reserves, the cost of developing projects funded with vertical funds.

[8] Source: PowerBi-Atlas (2021). https://app.powerbi.com/groups/me/reports/65efb00e-89c1-4ad3-bd7c-374ea1af5ee5/ReportSection14b2b9318521e16cca50

[9] Gender Equality Seal is a certification provided to CO when they complete a set of specific gender standards.

[10] The gender marker system is UNDP’s corporate instrument enabling the monitoring of programmatic intent and expenditure with regard to gender equality and the empowerment of women.

[11] In the recent elections, women won 17 percent of parliament seats. In contrast, women surpass men in attainment of educations and Science at all levels, e.g. 71.1 percent of women complete upper education compared to 63.8 percent of men and 61 percent of all university students are female. Source: World Bank Open Data: Country Profile Mongolia.

Mongolia NSO 2019.

[12] 2020 UNDP Partner Survey. Question 8: 67 percent of respondents strongly agree that UNDP has quality reporting and Question 6: 74 percent of respondent were in agreement with the statement that UNDP brings most value in programme and project implementation.

[13] As of 2018, 3.02 million ha of SPAs were still pending approval (2.45 million ha) or submission to parliament (0.57 million ha). Terminal Evaluation Report, Ecosystem Based Adaptation Approach to Maintaining Water Security in Critical Water Catchment in Mongolia, 2017; Terminal Evaluation Report, Mongolia’s Network of Managed Resource Protected Areas, 2018.

[14] Target 2.1.1 is set at 50 public hearings, cumulatively. In 2020, the CO reports 6 national public hearing and 421 public hearings by CRHs, 79 by aimag, and 342 by soum. Under this output, the CO also reports on extensive public consultations on proposed amendments to the Constitution (2019) on sections related to local governance – but it is not clear if these are being measured with the current indicator.

[15] E.g. for outcome 2, the voice index is not an accurate indicator to measure success of the programme as reflected in the CPD.


Recommendations
1

UNDP Mongolia should refine and communicate its business case for 2021/2022 by clarifying its offer and specifying how it intends to contribute to UNDAF objectives as well as COVID response and recovery in the immediate future.   
UNDP should intensify its efforts to help the government mitigate the socioeconomic impact of COVID-19, specifically on groups severely affected by the pandemic. This may involve institutional strengthening of the Cabinet Secretariat’s Office as main influencer during the crisis, as well as the provision of country-specific analyses and solutions, focusing on the most sensitive governance and economic issues only a trusted partner like UNDP can tackle. To clarify its offer for 2021/2022, the CO should produce a concise theory of change diagram visualizing in an attractive format who UNDP Mongolia is and how it makes a difference. A short accompanying narrative should identify new intervention areas, synergies and coherence across portfolios, linkages with other actors, opportunities for innovation and new partnerships, as well as the main underlying assumptions and risks. Such a document can serve as UNDP’s business case positioning the office as a key partner with a distinct profile in Mongolia’s dwindling donor landscape. It can also guide the CO in identifying activities with sustainability potential and serve as communication tool to clarify UNDP’s role vis-a-vis the UN Resident Coordinator’s Office, which has been a source of confusion.

2

[Linked to findings 3, 4, 10 and 12, RQ 2 and crosscutting RQs on COVID-19 and positioning]: UNDP Mongolia should immediately review its staffing structure and tweak its results framework to ensure it is realigned with recent adjustments and fit for purpose. 
Staffing for programme delivery needs to be reviewed as soon as possible. The office may require additional headcount to implement pipeline projects starting in 2021 which is challenging given a relatively high existing management efficiency ratio. In addition, programme leads need more headspace to engage in innovation, strategic and thought leadership work, including on UNDP’s normative position to be conveyed through advocacy and policy advisory (see recommendations 3 and 5). As a preliminary step to act on recommendation 1, the results chain for the vast existing outcome 1 must be reviewed to ascertain that the CO can prove contribution. To ensure that progress can be fully demonstrated by the end of the programme cycle, adjustments to the measurement of results (e.g. tweaking indicators, revising targets, including disaggregation where possible) should be made as soon as possible and in line with the updated theory of change. New programmes (GCF, KOICA, EU) will also need to be reflected adequately in both outcome areas and should not be made to fit under existing headings where contribution lies elsewhere. This will help demonstrate the full extent of the CO’s results and position UNDP as the partner of choice on a wide range of issues for the 20232027 period. 

3

[Linked to findings 1-9 and 11, and RQ 1] – In the next programme cycle, UNDP Mongolia should further enhance its focus on knowledge brokering, analysis and technical leadership, specifically in the realm of socio-economic analysis. An emphasis on ‘upstream’ work is expected from UNDP in middle-income countries. Delinking from the UN
coordination role has freed space for UNDP Mongolia to focus on its key strengths and take on a more substantive leadership role, as demonstrated in the socio-economic response to COVID-19. In its 2023-2027 CPD, UNDP needs to position itself clearly as a thought leader and promote its added value in policy advisory and knowledge brokering across the UN system and beyond. UNDP has the necessary clout to lead transformative, green and gender-responsive change in Mongolia and must now take decisive steps to model a whole-of-society approach engaging broadly beyond the government. Programme area leads need to adapt and operationalise the latest corporate guidance for the context of Mongolia, build on recently initiated socioeconomic and poverty analysis, and invest in demonstration projects to ensure proof of concept by collaborating with the upcoming A-Lab. The office is well-positioned for analytical work in some of the programmatic areas that were left aside in 2017-2022 such as skills/employment and youth and could consider re-engaging more strongly if a consolidated portfolio allows. International and national partners are available to collaborate, including the National Statistical Office, ILO, UNFPA and various civil society platforms (see recommendation 5). 

4

[Linked to findings 1-6 and 11, and RQ 1] – Programmatically, UNDP Mongolia should add depth to its governance programme and consolidate its portfolio on the environment-poverty nexus. There are significant opportunities to push for a “green recovery” from COVID-19 that UNDP could capitalize on. UNDP Mongolia must make some tough choices regarding its highly fragmented portfolio on sustainable development, streamlining it and perhaps emphasizing some new areas going forward (see recommendation 3). A particularly promising area to bundle some existing work and partnerships is “green recovery” from COVID19 where significant national and donor interest is noted. Such a focus would
need to be operationalized further but could build on various assets in the office (history of engagement around climate and environmental issues; incoming GCF funding; etc.). In the
governance area, UNDP’s portfolio could be more ambitious and normatively based, to include a strong focus on anti-corruption, gender and human rights. Emphasis on citizens’ voice
needs to be reinstated as a core area of what UNDP does, believes in and stands for. Development financing is an additional growth area where UNDP’s expertise is sought and valued. Mongolia CO is beginning to make a name for itself as a lab for experimentation in this area. This should be continued and knowledge, including of failed pilots and experiments, should be documented and shared widely across the region and globally. 

5

[Linked to findings 5-7 and 9, RQ 2 and crosscutting RQs on positioning] – UNDP Mongolia should branch out its partnerships, with particular focus on civil society. 
Being a thought leader and knowledge broker includes broad engagement and consultation, moving beyond servicing the government of Mongolia, and promoting multidisciplinary solutions. Conducting a comprehensive landscape analysis of development partners and resources to ‘build forward better’ could provide entry points. Such a piece could build on the work of this review and its by-products such as the ‘formative analysis’. Partnership with civil society, or indeed, strengthening civil society per se in its role to hold government
accountable and to accelerate the achievement of the SDGs should be a key focus of UNDP’s work in Mongolia. This implies support beyond grant management to include areas such as improvement of legal frameworks for civil society engagement, capacity development in the realms of advocacy, policy analysis and international fundraising, as well as facilitating direct engagement with government authorities at different levels. Partnerships with other non-state actors, such as foundations or the private sector, should also be examined and pursued where strategic, ethical and feasible. 

Latest Evaluations

Contact us

1 UN Plaza
DC1-20th Floor
New York, NY 10017
Tel. +1 646 781 4200
Fax. +1 646 781 4213
erc.support@undp.org