Independent Country Programme Evaluation: Kazakhstan

Report Cover Image
Evaluation Plan:
2018-2021, Independent Evaluation Office
Evaluation Type:
ICPE/ADR
Planned End Date:
12/2019
Completion Date:
12/2019
Status:
Completed
Management Response:
No
Evaluation Budget(US $):
35,000

Share

Document Type Language Size Status Downloads
Download document ICPE Kazakhstan - Annexes.pdf related-document English 512.85 KB Posted 49
Download document ICPE Kazakhstan - Annexes ToR.pdf tor English 593.51 KB Posted 28
Download document ICPE_Kazakhstan_full report.pdf report English 6414.22 KB Posted 63
Download document ICPE_Kazakhstan _EvaluationBrief.pdf summary English 290.84 KB Posted 30
Title Independent Country Programme Evaluation: Kazakhstan
Atlas Project Number:
Evaluation Plan: 2018-2021, Independent Evaluation Office
Evaluation Type: ICPE/ADR
Status: Completed
Completion Date: 12/2019
Planned End Date: 12/2019
Management Response: No
UNDP Signature Solution:
  • 1. Poverty
  • 2. Governance
  • 3. Sustainable
  • 4. Energy
Corporate Outcome and Output (UNDP Strategic Plan 2018-2021)
  • 1. Output 1.1.1 Capacities developed across the whole of government to integrate the 2030 Agenda, the Paris Agreement and other international agreements in development plans and budgets, and to analyse progress towards the SDGs, using innovative and data-driven solutions
  • 2. Output 1.2.1 Capacities at national and sub-national levels strengthened to promote inclusive local economic development and deliver basic services including HIV and related services
  • 3. Output 1.2.2 Enabling environment strengthened to expand public and private financing for the achievement of the SDGs
  • 4. Output 1.4.1 Solutions scaled up for sustainable management of natural resources, including sustainable commodities and green and inclusive value chains
  • 5. Output 2.2.1 Use of digital technologies and big data enabled for improved public services and other government functions
  • 6. Output 2.2.3 Capacities, functions and financing of rule of law and national human rights institutions and systems strengthened to expand access to justice and combat discrimination, with a focus on women and other marginalised groups
  • 7. Output 2.5.1 Solutions developed, financed and applied at scale for energy efficiency and transformation to clean energy and zero-carbon development, for poverty eradication and structural transformation
  • 8. Output 3.2.1 National capacities strengthened for reintegration, reconciliation, peaceful management of conflict and prevention of violent extremism in response to national policies and priorities
Evaluation Budget(US $): 35,000
Source of Funding:
Evaluation Expenditure(US $): 35,000
Joint Programme: No
Joint Evaluation: No
Evaluation Team members:
Name Title Email Nationality
Yogesh Kumar Bhatt Senior Evaluation Specialist and Lead Evaluator
Lilit Melikyan ,Zehra Kacapor-Dzihic Evaluation Consultant,Evaluation Consultant ,
Tianhan Gui Research Consultant
GEF Evaluation: No
Key Stakeholders:
Countries: KAZAKHSTAN
Lessons
Findings
1.

Finding 1. UNDPs programme in Kazakhstan is highly relevant to the country’s national priorities and Kazakhstan’s vision of becoming a global player and provider of development assistance to countries in the region and globally. The sustainability of results, however, is often affected by the frequently changing government priorities and reform directions, requiring ongoing adaptation of UNDP interventions and programmes to fit the changing needs and policy landscape. This is further compounded by the country’s UMIC status, the shrinking landscape of donors and development partners and unpredictable government cost sharing.


Tag: Energy Relevance Sustainability Government Cost-sharing Resource mobilization Justice system Public administration reform Donor relations Human and Financial resources Implementation Modality Procurement International Financial Institutions Capacity Building Inclusive economic growth Policy Advisory Technical Support Agenda 2030 SDG Integration SDG monitoring and reporting

2.

Finding 2. Multitude of activities, many of which are small interventions and a lack of detailed theory of change compromises the consolidation of results of UNDP’s interventions. The CO has re-structured its programme portfolios, but there have been limited synergies and coherence among them. Programme planning and implementation is informed by monitoring and reporting at the level of inputs, activities and outputs, and outcome level assessment is mainly approached by commissioned external outcome evaluations.


Tag: Coherence Effectiveness Gender Equality Women's Empowerment Country Support Platform Implementation Modality Innovation Monitoring and Evaluation Programme Synergy Theory of Change Inclusive economic growth Micro-credit

3.

Finding 3. UNDP has made notable contributions in promoting joint efforts and collaboration with other UN agencies in Kazakhstan, although there is a perception that UNDP tends to lead and then take over the joint initiatives. The cooperation has mostly been within the framework of the UN Country team working groups, and projects addressing regional disparities and social development as well as partly supporting the SDG nationalization process. However, joint efforts are seldom when it comes to using UN leverage in addressing strategic issues such as human rights and disaster risk reduction where a joint response would yield greater results


Tag: Disaster Risk Reduction Coherence Effectiveness Impact Gender Equality Gender Mainstreaming Women's Empowerment Human rights Integration Partnership UN Country Team Policy Advisory Technical Support Agenda 2030 Mainstreaming, Acceleration and Policy Support (MAPS) Rapid Integrated Assessment SDG Integration

4.

Finding 4. UNDP has made significant efforts to integrate and mainstream gender across its programmes and operations by investing in strengthening its internal gender policies and strategies. Its programmatic work has been gender-responsive and has succeeded in incorporating gender equality and women’s empowerment considerations at institutional and policy level frameworks and national laws. 


Tag: Effectiveness Impact Gender Equality Gender Mainstreaming Women's Empowerment Results-Based Management Capacity Building Disabilities Inclusive economic growth Social Protection Data and Statistics

5.

2.2 Inclusive and Sustainable Growth

Outcome 18. Diversification of the economy provides decent work opportunities for the underemployed, youth, and socially vulnerable women and men. 

Finding 5. The outcome has been ambitiously defined with outputs partially contributing to it. There has been some discrepancy between the definition of the outcome and actual projects implemented under it. Interventions to improve social protection system for people with disabilities and interventions in health sector only have a very indirect or no linkages to the outcome. 


Tag: Effectiveness Gender Equality Women's Empowerment Local Governance Access to Medicines Results-Based Management Theory of Change Disabilities Inclusive economic growth Jobs and Livelihoods Social Protection Youth

6.

Finding 6. The support provided for advancement of rights of Persons with Disabilities in Kazakhstan has led to improvement in legislative and institutional foundations for protection of rights of PWD and improved provision of social services. These efforts bring important investments in strengthening the system of response and its transparency. If continued, the results achieved thus far have a high potential for impact.


Tag: Effectiveness Impact Sustainability Civic Engagement e-Governance Human rights Parliament Innovation Programme Synergy Disabilities Jobs and Livelihoods Social Protection Policy Advisory Technical Support

7.

Finding 7. UNDP has contributed to promotion and provision of decent work opportunities for the unemployed, youth, and socially vulnerable women and men. It has also assisted expansion of entrepreneurship and employability, most notably through ‘green jobs’, by provision of micro-loan schemes and training. The contribution, however, is limited to certain regions and communities with no clear strategy for scaling-up and replicability. 


Tag: Agriculture Rural development Vulnerable Green Economy Effectiveness Relevance Sustainability Gender Equality Women's Empowerment Local Governance Partnership Private Sector Inclusive economic growth Jobs and Livelihoods Micro-credit Youth

8.

2.3 Environment and Energy

Outcome 19. Ecosystems and natural resources are protected and sustainably used, and human settlements are resilient to natural and human-induced disasters and climate change. 

Finding 8: UNDP has laid the necessary groundwork for strengthening environmental governance in protected areas and reduction of pressure on the biodiversity of desert and semi-desert ecosystems and wetlands through its work on improving regulatory environment for forest, wildlife and protected area management. Once approved, the legislative and institutional frameworks have the potential to ensure mainstreaming of economic valuations of eco-systems in natural resources management. 


Tag: Fishery Forestry Disaster Risk Reduction Biodiversity Environmental impact assessment Natural Resouce management Protected Areas Effectiveness Impact Sustainability Micro-credit Technology

9.

Finding 9. UNDP has made a significant contribution towards climate change mitigation/greenhouse gas reduction by promoting energy efficiency in residential sector and renewable energy. However, the highly subsidized energy tariffs continue to be a strong disincentive impacting the scale of effectiveness of efforts. 


Tag: Energy Effectiveness Efficiency Sustainability

10.

Finding 10. UNDP has used innovative market-based mechanisms for sustainable urbanization, including through the promotion of Public-Private Partnerships and Small and Medium Enterprises, under urban Nationally Appropriate Mitigation Action (NAMAs). However, the success of the overall efforts largely depends on the assumption that integrated urban transport planning is carried out in conjunction with overall urban planning at the country level targeting urban, semi-urban and rural areas as part of homogenous development. 


Tag: Rural development Energy Effectiveness Gender Equality Women's Empowerment Innovation Partnership Inclusive economic growth Urbanization

11.

Finding 11. UNDP has made an important contribution towards climate change adaptation, piloting models for sustainable use and management of land and water resources. Piloted green technologies demonstrate they can not only reduce pressure on natural resources, but also improve profitability and mitigate disaster risks. The needs, however, are large requiring deepened engagement in these areas and a focus on upscaling of these initiatives with the government. 


Tag: Agriculture land resouces Climate Change Adaptation Disaster Risk Reduction Water resources Effectiveness Sustainability Country Government Capacity Building

12.

Finding 12. UNDP has helped the government to improve monitoring and liquidation of persistent organic pollutants and medical waste. However, the low prices of old combustion technologies for healthcare medical waste management remain a key risk for further promotion of good quality nonburn technologies in Kazakhstan.


Tag: Adaptation Fund Environmental impact assessment Waste management Effectiveness Sustainability Global Environment Facility fund Private Sector Financing Partnership Technology

13.

Finding 13. UNDPs contributions in Disaster Risk Reduction during the current programme cycle have been limited in scope and scale. While initial steps have been taken to transition from ‘response and recovery actions’ to ‘prevention and risk reduction’ as per the Sendai Framework for Disaster Risk Reduction, systemic barriers impede the effectiveness of DRR efforts in the country. 


Tag: Disaster Risk assessments Disaster Risk Reduction Effectiveness Impact Relevance Sustainability Gender Equality Women's Empowerment Data and Statistics

14.

2.4 Democratic governance

Outcome 20. Judicial and legal systems, and public institutions, are fair, accountable and accessible to all people.

Finding 14. UNDP has helped the government strengthen foundations for improved public service delivery through investment in public administration and civil service reform. Investments on the improvement of meritocracy across the civil service have resonated well with the civil service, however, sustainability is still weak and depends on continued political commitment to scaling up and maintaining the reform results and policy direction.


Tag: Effectiveness Relevance Sustainability Government Cost-sharing Resource mobilization Anti-corruption Civic Engagement Human rights Justice system Parliament Public administration reform Rule of law Change Management Human and Financial resources Knowledge management Policy Advisory

15.

Finding 15. UNDP has laid the necessary groundwork for improved local governance towards fiscal decentralization. However, the results have been limited in terms of system changes and uptake of approaches and mechanisms owing to insufficient capacity of local governments and resource constraints.


Tag: Effectiveness Efficiency Impact Sustainability Local Governance Public administration reform Monitoring and Evaluation Service delivery Technical Support

16.

Finding 16. UNDP has made a notable contribution in promoting civil society and its role in decisionmaking processes, which have the potential of bringing transformative changes from the perspective of democratization of the society. 


Tag: Vulnerable Relevance Sustainability Anti-corruption Civic Engagement e-Governance Human rights Parliament HIV / AIDS Pandemic Disabilities Social Protection SDG Integration

17.

Finding 17. UNDP’s support to improve government systems for procurement of medical goods and services has helped overcome risk of corruption and promoted improved access to quality treatment, particularly for the most vulnerable population like people living with HIV and tuberculosis. 


Tag: Vulnerable Impact Sustainability Global Fund Government Cost-sharing Anti-corruption Civic Engagement Access to Medicines HIV / AIDS Pandemic Procurement UN Agencies Technology

18.

Finding 18. UNDPs support has made important inroads in promoting human rights, prevention of gender-based violence and access to justice. However, the efforts remain fragmented without a longterm goal or strategy for transformative change. 


Tag: Coherence Effectiveness Relevance Sustainability Gender Equality Gender-Based Violence Women's Empowerment Human rights Justice system Rule of law Knowledge management Theory of Change Capacity Building Policy Advisory Technical Support

19.

Outcome 21. The Government, together with partners, promotes achievement of sustainable development goals in the region, and leads in promotion and implementation of United Nations principles, standards and conventions. 

Finding 19. UNDP has played a key role in supporting Kazakhstan’s regional and global leadership ambitions by supporting the establishing of KazAid (Kazakhstan’s ODA system), providing institutional support to the Astana Regional Hub for Civil Services and in promoting South-South cooperation. While this has brought recognition and helped share Kazakhstan’s experience, the sustainability is fragile due to lack of institutionalisation of the KazAid and the Astana Regional Hub. 


Tag: Green Economy Relevance Sustainability Gender Equality Women's Empowerment Civic Engagement Public administration reform Knowledge management Strategic Positioning Capacity Building South-South Cooperation Agenda 2030 SDG Integration

20.

Finding 20. UNDP was instrumental in operationalisation of Kazakhstan’s aid policies as stipulated in the Official Development Assistance (ODA) Law in 2014. However, the sustainability of these contributions is limited, due to resource constraints and political environment. 


Tag: Efficiency Relevance Sustainability Aid Coordination Public administration reform Strategic Positioning Capacity Building South-South Cooperation Technical Support

21.

Finding 21. UNDP has been instrumental in promoting and sharing of Kazakhstan’s experience on civil service reform with professionals and peers from Central Asia and beyond through the Astana Regional Hub for Civil Service. Concrete solutions for the institutionalization of the Hub and its sustainability, however, remain unclear at the time of the evaluation.


Tag: Effectiveness Relevance Sustainability Gender Equality Gender Mainstreaming Women's Empowerment Anti-corruption Civic Engagement Public administration reform Knowledge management Project and Programme management Capacity Building South-South Cooperation Technical Support

Recommendations
1

Recommendation 1. UNDP needs to consolidate its country portfolio and ensure support is provided in the areas where it has strongest comparative advantage. Demand driven nature will remain, but responsiveness to government demands needs to happen within the parameters of outcome level results and a well-defined theory of change with proper analysis of causal assumptions and risks. UNDP should ensure that piloting and modeling of services and/or approaches is done within this larger institutional framework to ensure sustainability of results. 

Given the dwindling donor resources and country’s UMIC status, UNDP needs to ensure that its support is relevant and contributes to system level changes in a limited number of sectors where UNDP has most visible comparative advantage. Thus far, UNDP has engaged across multiple sectors. In many of them this engagement lacked depth and was fragmented into one-off interventions without scaling and transformative potential. This was possible given broad country programme framework and loose results framework. UNDP needs to invest efforts in elaborating its country level theory of change based on a thorough assessment of the sectors with strongest comparative advantage in terms of expertise and strongest strategic positioning and commitment by the government. Assessment of risks and assumptions and detailed mitigation strategies need to be developed, particularly for areas where modelling/ demonstration pilots will happen as part of system level intervention. It is also advisable to avoid embarking on new themes and sub-themes and areas of intervention that loosely belong to wider programme’s intervention framework.

2

Recommendation 2. UNDP should invest strongly on gathering and utilizing evidence towards improved programming and knowledge management in the country office. This should be supported by a cross-portfolio knowledge platform for regular reflection on results, synergy and coherence within and across the CO portfolio. 

UNDP is following standard monitoring and reporting procedures, and commissions external project and outcome level evaluations. However, weaknesses are found in the manner in which UNDP gathers, analyses and uses the monitoring data, particularly from piloting and modelling for internal decision making but also for promotion of best practices to draw attention and commitment of the government to scale up working models. Reporting is mainly output oriented with little analysis and reflection on outcome level results, and with little use of externally-available research studies, analytical papers or reports to support this analysis. It is strongly advised to further invest in knowledge management within the office, with emphasis on outcome level monitoring, reporting and analysis by thematic areas and/or programme clusters and to establish a knowledge sharing platform for regular reflection on results, synergy and coherence within and across the CO portfolio. 

3

Recommendation 3. UNDP should continue investing in social, public administration and civil service reforms, particularly in the areas in which support was initiated during this country programme cycle, such as meritocracy, anti-corruption and social service delivery.

Investments in strengthening foundations for active delivery of social services, particularly for PWD as well as investments in public administration and civil service demand long-term un-dwindling support and attention to ensure the changes take root towards changing mindsets and public service delivery habits. UNDP is strongly positioned in these sectors, so policy and technical leverage should be utilised to achieve change in an incremental manner, based on a clear theory of change and resource mobilisation plan. 

4

Recommendation 4. UNDP needs to continue spearheading initiatives for empowerment of women, the most vulnerable groups and civil society to take more active role in decision making processes based on lessons learned and results of interventions in these areas achieved thus far. 

UNDP has achieved important results in terms of empowerment of vulnerable groups, notably women and PWD, and engaged strongly with civil society in various sectoral decision-making processes (e.g. health, human rights). This support should continue and build up on lessons learned gathered so far to ensure that partnering with government in different sectors is inclusive of various interest groups. Ongoing support needs to be based on human rights and leave no-one behind principles, to ensure full inclusiveness and transparency of decision making and service delivery. 

5

Recommendation 5. UNDP should build on the achievements in the energy and environment sector and deepen its engagement to upscale and institutionalize the results by strengthening existing partnerships and engaging with non-traditional partners.

UNDP should seek to create more depth and expertise in the areas that it currently covers under the energy and environment portfolio. Climate change mitigation and adaptation, transboundary waters, environmental education, sustainable agriculture, renewable energy and DRR are among areas where UNDP CO could play a larger role. Given the cross-cutting nature, DRR activities should be integrated into other thematic areas (climate change adaptation in particular), with a focus on disaster prevention and risk reduction. In doing so, UNDP should engage with both the government partners and institutions, and the private sector and other non-traditional partners. 

6

Recommendation 6. UNDP should develop an exit strategy and engage with the Government to discuss the sustainability, institutionalisation and future of services and approaches modelled by UNDP across its portfolio, most notable of all being the Astana Regional Hub and support to the national ODA system. 

UNDP has modelled a number of services, mechanisms and approaches across all the targeted sectors. However, apart from the support in drafting of laws and regulatory instruments, the focus has mainly been on demonstration pilots and models, only some of which have been institutionalized. While the pilots have been helpful in demonstrating the ‘proof of concept’, significantly more sustained work is required to ensure institutionalizing and sustainability of results. Of particular relevance are UNDPs interventions in Outcome 21. UNDP has achieved important results through the work of the Astana Regional Hub and modelling Kazakhstan’s ODA interventions. Both have been successful in increasing the positive image of Kazakhstan and its civil service among regional and wider network of countries and civil service peers. At the same time, the Hub has served as an important resource for domestic civil service reform efforts. However, administrative and legal identity of the Hub remains within UNDP in a form of a project, funded mainly by the government with funds expiring in 2020; while the KazAid Agency has not been formally established yet. Considering the circumstances and investments of UNDP and development partners in the Hub, the Agency and the models, it is urgent for UNDP to engage in discussions with the government on future legal and administrative structures of the Hub and the Agency in particular. These discussions should lead to an exit strategy for UNDP, with measures to ensure sustainability and institutionalisation of the modelled institutions, services and approaches. 

7

Recommendation 7. UNDP should develop and proactively pursue a long-term resource mobilization strategy and implementation plan that includes a predictable government cost sharing agreement and innovative financing mechanisms including partnerships with the private sector and IFIs. 

Given Kazakhstan’s UMIC status, resource mobilization will remain a challenge for the CP. Funding in the new CPD cycle will very much depend on Government Cost Sharing (GCS) and vertical funds, due to the shrinking ‘traditional’ donor environment, as well as the ability of the CO to engage better with the IFIs and the private sector, building on its competitive advantages, and above all being creative and innovative. UNDP should develop a long-term resource mobilization strategy with clearly defined targets and proactively pursue resource mobilization efforts both from traditional and non-traditional sources. Cooperation with the IFI’s and the government in utilization of loans and engaging with the private sector on market-based mechanisms should be integral parts of this strategy. Given the fact that Government will remain UNDPs largest contributor, the CO and the Government should agree on making the GCS more predictable by setting aside a set amount of funds to be ‘reserved’ for the CO in the next CPD cycle based on the Government’s strategic priorities. This will make UNDP programme less volatile and vulnerable from the standpoint of financial stability. This should also help the CO mobilize a bigger amount from other donors to match/co-finance the Government funds. 

Latest Evaluations

Contact us

1 UN Plaza
DC1-20th Floor
New York, NY 10017
Tel. +1 646 781 4200
Fax. +1 646 781 4213
erc.support@undp.org