Independent Country Programme Evaluation: North Macedonia

Report Cover Image
Evaluation Plan:
2018-2021, Independent Evaluation Office
Evaluation Type:
ICPE/ADR
Planned End Date:
12/2019
Completion Date:
12/2019
Status:
Completed
Management Response:
No
Evaluation Budget(US $):
30,000

Share

Document Type Language Size Status Downloads
Download document ICPE North Macedonia-Annexes TOR.pdf tor English 298.49 KB Posted 195
Download document ICPE_North Macedonia-Annexes.pdf related-document English 707.12 KB Posted 256
Download document N Macedonia_Eval brief.pdf summary English 191.13 KB Posted 259
Download document N Macedonia full report.pdf report English 2760.17 KB Posted 478
Title Independent Country Programme Evaluation: North Macedonia
Atlas Project Number:
Evaluation Plan: 2018-2021, Independent Evaluation Office
Evaluation Type: ICPE/ADR
Status: Completed
Completion Date: 12/2019
Planned End Date: 12/2019
Management Response: No
Corporate Outcome and Output (UNDP Strategic Plan 2018-2021)
  • 1. Output 1.1.2 Marginalised groups, particularly the poor, women, people with disabilities and displaced are empowered to gain universal access to basic services and financial and non-financial assets to build productive capacities and benefit from sustainable livelihoods and jobs
  • 2. Output 2.1.1 Low emission and climate resilient objectives addressed in national, sub-national and sectoral development plans and policies to promote economic diversification and green growth
  • 3. Output 3.1.1 Core government functions and inclusive basic services4 restored post-crisis for stabilisation, durable solutions to displacement and return to sustainable development pathways within the framework of national policies and priorities
Evaluation Budget(US $): 30,000
Source of Funding:
Evaluation Expenditure(US $): 30,000
Joint Programme: No
Joint Evaluation: No
Evaluation Team members:
Name Title Email Nationality
Alan Fox Lead Evaluator
Paul Georis Evaluation Consultant
Daniele Pedretti Evaluation Consultant
Silvana Rusi Evaluation Consultant
Alan Fox Evaluation Consultant
Tianhan Gui Research Consultant
Elizabeth Wojnar Research Consultant
GEF Evaluation: No
Key Stakeholders:
Countries: MACEDONIA, THE FORMER YUGOSLAV REPUBLIC OF
Lessons
Findings
1.

2.1 Employment and livelihoods

Finding 1. UNDP’s involvement in implementing Active Labour Market Measures (ALMM) has contributed to improving the effectiveness of employment policy in North Macedonia, and reduced unemployment.

UNDP has been providing valuable support to the Ministry of Labour and Social Affairs and the Employment Agency since 2007 with the design and implementation of active labour market policies to promote self-employment (including among persons with disabilities, or PwD), stimulate the growth of small and medium enterprises and social enterprises and bring disadvantaged groups (Roma, youth, women) to the labour market through activation measures.

The oldest and most important measure (in terms of financing) is the ongoing Self-Employment Programme, which strengthens entrepreneurship skills among unemployed people willing to start their own businesses through training and equipment. The results are very positive with an impressive number of job creations and a business survival rate after three years: in the period from 2016 to 2018, the self-employment measure attracted a total of 8,114 unemployed persons to apply for the business start-up scheme, from which 3,063 were women (37.75 percent) and 3,162 (39 percent) by young people under age 29. Of these, 4,743 unemployed candidates underwent and successfully completed the training to strengthen their entrepreneurship and management skills (1,860 women or 39.22 percent and 1,900 or 40 percent youth). With UNDP support, 4,236 of the certificate holders prepared a business plan with the support of the Agency for Entrepreneurship Promotion (1,642 women or 38.76 percent and 1,702 or 40 percent youth) and 3,422 established their new small businesses (1,303 women or 38 percent and 1,368 or 40 percent youth). The programme has been contributing to economic growth and the promotion of entrepreneurship. According to data from the central register, 12-15 percent of newly created businesses in the country are established as a result of the programme. Moreover, special support consisting of coaching and mentoring services has been implemented in pilot municipalities to increase Roma participation in the Self-Employment Programme, with very encouraging results.


Tag: Effectiveness Efficiency Impact Strategic Positioning Country Government Jobs and Livelihoods Poverty Reduction Technical Support Vulnerable Women and gilrs Youth

2.

2.1 Employment and livelihoods

Finding 1. UNDP’s involvement in implementing Active Labour Market Measures (ALMM) has contributed to improving the effectiveness of employment policy in North Macedonia, and reduced unemployment. (continuation)

While UNDP has encouraged the sustainability of newly created services by building the capacity and ownership of stakeholders, including by helping them to identify long-term sources of funding, the limited financial capacity of municipalities remains an obstacle for maintaining and rolling out services created thanks to these projects. In order to address the skills mismatch in the labour market, UNDP recently tested a new measure to improve coordination with private companies to identify training needs and facilitate the search for qualified workers, including through the creation of youth information clubs, which allow students to meet with company representatives and explore internship, volunteer and entrepreneurial opportunities within companies.


Tag: Human rights Rule of law Policies & Procedures Country Government Jobs and Livelihoods Poverty Reduction Capacity Building Vulnerable

3.

2.2 Democratic governance

Finding 2. The CPD framework for democratic governance proved overall well adapted to the needs of the sector. The emphasis on reinforcing governance at the local level was fully justified by the country’s situation and needs.

 


Tag: Effectiveness Anti-corruption Local Governance Service delivery Institutional Strengthening National Institutions

4.

Finding 3. The democratic governance portfolio consisted of a coherent group of interventions in line with CPD priorities that built on previous experiences and followed the same bottom-up approach, also addressing the needs of smaller, less developed municipalities.

UNDP interventions in the democratic governance outcome area share the same goal of enhancing local governance despite the different sources of funding and geographical scope, contributing directly to the outputs identified in the CPD in terms of more open, transparent and accountable public policies and more effective and equitable services delivery at the local level.Through the early stages of the EU-funded ReLOaD programme, UNDP helped set the stage for improved trust between local authorities and the civil society in five North Macedonian municipalities, by seeking agreement on a few development priorities, and spurring interest from local nongovernment organizations (NGOs) to participate in service delivery efforts through the programme.


Tag: Coherence Effectiveness Anti-corruption Civic Engagement Local Governance Policies & Procedures Institutional Strengthening National Institutions

5.

Finding 3. The democratic governance portfolio consisted of a coherent group of interventions in line with CPD priorities that built on previous experiences and followed the same bottom-up approach, also addressing the needs of smaller, less developed municipalities. (continuation)

Both donors and beneficiaries appreciated the bottom-up approach adopted by UNDP, the ability to build on previous efforts and the emphasis laid on developing innovative solutions through pilot and replication phases, providing decision-makers with new tools for local policy development. For example, citizen satisfaction surveys were conducted to assess the quality of municipal services and improve decision-making and planning. The e-Knowledge Platform on Local Development is another innovative instrument which integrates training materials, best practices and information generated by project activities. A new Internet platform was created, providing open and transparent access to data on several development areas contributing to the quality of life in municipalities. These efforts contribute to evidence-based policies which are being promoted by the Ministry of Local Self-Government, as well as planning, reporting and coordination mechanisms. The capacity of the ministry was also enhanced in conducting comparative analyses of LSGUs by gathering evidence factors influencing their performance.

UNDP enabled the piloting of new approaches before replicating them on a larger scale, giving the administration the possibility to get acquainted with new methods and the time to adapt internal procedures and rules. For example, the Quality of Life reports were first tested in ten municipalities with EU funding before being expanded to cover all 81 municipalities.


Tag: Effectiveness Efficiency Sustainability Local Governance Rule of law Communication Donor relations Knowledge management Policies & Procedures Strategic Positioning Institutional Strengthening National Institutions

6.

Finding 4. The country office responded to emergency needs linked to the 2015 migration crisis, supplementing the humanitarian aid provided by other UN agencies with actions seeking to enhance the resilience of two particularly affected municipalities.

 

 


Tag: Effectiveness Local Governance UN Agencies Humanitarian development nexus Migration Resilience Coordination Institutional Strengthening National Institutions

7.

Finding 5. While the sustainability of results remains a significant question, there is also a great demand for further cooperation on the part of municipalities.

The sustainability of UNDP interventions in the area of local governance remains a major concern. The political crisis of 2016-2017 affected all levels of government, resulting in a significant turnover of municipal staff, many of whom the project had trained. UNDP had to deploy additional resources to restore relations between local actors, often by resuming/rearranging activities to ensure the sustainability of results. The absence of public financing mechanisms to maintain local services established through UNDP interventions is another major problem. Low ownership is also a challenge for the sustainability of outputs and outcomes, as many municipalities lack a strategic long-term vision and cannot maintain effective administrative capacity to reap the benefits from the decentralization process.


Tag: Effectiveness Efficiency Sustainability Local Governance Human and Financial resources Operational Efficiency Ownership Policies & Procedures Coordination

8.

2.3 Social inclusion

Finding 6. The rationale for a separate outcome area for social inclusion is not clear given the mismatch between the budget and expected outputs and the overlap with the employment and livelihood outcome area, which has a strong social inclusion dimension. 

With a planned budget of approximately $1 million, it was clear from the start that the CPD outputs would not be achieved. Over the reporting period, only three projects and one micro-project were implemented, making social inclusion the smallest portfolio of the CPD. As indicated in Chapter 1, there was no intervention implemented for three CPD  outputs.


Tag: Efficiency Civic Engagement Local Governance Partnership Project and Programme management Results-Based Management Theory of Change Disabilities Inclusive economic growth Jobs and Livelihoods Social Protection

9.

Finding 7. UNDP support substantially improved the pre-school educational opportunities for the Roma population in the municipality of Shuto-Orizari, providing design and construction management for a new kindergarten.

With assistance from the Norwegian cooperation programme, UNDP helped construct the only kindergarten in the Roma municipality of Shuto-Orizari, one of the most deprived areas of Skopje. The building replaces the old school that burned down in 2017. When completed, it will give access to pre-school education to 200 Roma children from 2-5 years old in a modern space of 1,225 m2. The new kindergarten complies with high levels of energy efficiency standards (energy class A) and provides a new nursery service for children aged 9-24 months.


Tag: Effectiveness Sustainability Local Governance Promotion of dialogue Education Operational Services Vulnerable Youth

10.

Finding 8. In the area of gender equality, a single UNDP intervention contributed to strengthening the institutional and legal framework for combating gender-based violence and domestic violence, contributing to the country’s early ratification of the Istanbul Convention.

 


Tag: Effectiveness Sustainability Gender Equality Gender Mainstreaming Gender-Based Violence Women's Empowerment Human rights Justice system Rule of law Social Protection Women and gilrs

11.

2.4 Environmental sustainability and resilience

Finding 9. The CPD was broadly aligned with national environmental priorities and the portfolio of projects covered most of the environmental issues identified in the CPD. Missing were intended interventions to encourage industries to adopt and comply with environmentally friendly standards and norms.

Overall, the UNDP environment portfolio responds to national needs and government priorities. Interventions are well aligned with the CPD and the Sustainable Development Agenda. UNDP helped build national capacities for greenhouse gas (GHG) inventories and mitigation modelling and analyses and supported national authorities in fulfilling their communication obligations in line with international climate change agreements providing relevant information and data on investment priorities across sectors. 


Tag: Energy Environment Policy Coherence Effectiveness Relevance Policies & Procedures Country Government Resilience SDG Integration

12.

Finding 10. UNDP implemented important interventions in line with CPD outcomes in the fields of disaster management and nature protection. Despite efforts to enlist civil society and local community support, there are concerns about the sustainability of results of some projects due to weak institutional capacity and lack of appropriate funding mechanisms. 

North Macedonia is highly vulnerable to natural disasters, including earthquakes, heatwaves, forest fires and floods. Through the successful implementation of two large EU-funded Floods Recovery Programmes, UNDP has demonstrated its ability to manage large-scale water and transport infrastructure works, establishing itself as the country’s ‘go-to organization’ on flood risk management. UNDP support was also instrumental in strengthening disaster risk management, with Flood Risk Assessments prepared for almost all sub-river basins in the country and Flood Risk Management Plans developed for Strumica river basin, Polog region, Upper Vardar River in line with the EU Flood Directive. The largest project in the portfolio ‘Improving Resilience to Floods in the Polog Region’ is considered a model for disaster risk management on a regional level.


Tag: Disaster Risk Reduction Natural Resouce management Effectiveness Efficiency Civic Engagement Local Governance Human and Financial resources Project and Programme management

13.

Finding 10. UNDP implemented important interventions in line with CPD outcomes in the fields of disaster management and nature protection. Despite efforts to enlist civil society and local community support, there are concerns about the sustainability of results of some projects due to weak institutional capacity and lack of appropriate funding mechanisms. (continuation)

Long-term interventions involving the local population and all relevant institutions at local and central levels tend to achieve higher impact and sustainability as illustrated by the 15-year ‘Prespa Lake Ecosystem Restoration Programme’ aimed at introducing integrated watershed management of the Prespa Lake to deliver better water quality to the population with a more balanced use of natural resources. Over the years, the project has contributed to significant behavioural and organizational changes in the targeted area, with UNDP promoting a clear division of responsibilities between municipalities, CSOs and central authorities. Understanding the importance of its role in protecting natural resources, the municipality of Resen has established a Sector for Environment Protection consisting of two departments, one for Nature Protection and the second one for Waste Management and Integrated Pollution Prevention and Control. The municipality has also assumed the responsibilities of management authority for two protected areas within its territory (Ezerani Nature Park and Lake Prespa Monument of Nature). The creation of a permanent structure, the Prespa Lake Watershed Management Council, representing all important interests55 is another important mechanism for guaranteeing the sustainability of results. A measure of the success of the intervention is the fact that UNDP decided to discontinue its support, considering that counterparts have now enough capacity to carry on activities on their own.


Tag: Disaster Risk Reduction Natural Resouce management Effectiveness Impact Sustainability Civic Engagement Local Governance Strategic Positioning

14.

Finding 10. UNDP implemented important interventions in line with CPD outcomes in the fields of disaster management and nature protection. Despite efforts to enlist civil society and local community support, there are concerns about the sustainability of results of some projects due to weak institutional capacity and lack of appropriate funding mechanisms. (continuation)

Through carefully designed activities implemented in the framework of the ‘Improving the Management of Protected Areas’ project, UNDP helped to trigger transformational change by creating scalable and replicable examples for 25 protected areas. The project included a significant grant component for NGOs and management bodies of the protected areas. A comprehensive capacity-building programme accompanied the grant application processes. The economic and social benefits of the intervention are evident in the case of the three National parks involved. The management bodies have used the grants mainly to enhance their management capacities and expand their tourism offer (for example, by creating an arboretum or reintroducing autochthone species), while NGOs have improved their capacity to monitor and protect the environment and to promote these areas as tourism destinations.


Tag: Disaster Risk Reduction Natural Resouce management Effectiveness Efficiency Sustainability Local Governance

15.

Finding 11. UNDP positively contributed to North Macedonia’s climate change efforts, helping establish a nationwide monitoring, reporting and verification system in accordance with the Paris Agreement.

 


Tag: Climate Change Adaptation Climate change governance Environment Policy Partnership Policies & Procedures Country Government SDG Integration SDG monitoring and reporting

16.

2.5 Results management

Finding 12. Although effective monitoring practices are generally in place for projects, assessing their effectiveness and cumulative impact is more difficult with a results framework that does not easily capture the contribution of UNDP interventions to CPD outputs and outcomes as well as progress towards the SDGs.

Overall, UNDP interventions appear to be well monitored during their implementation, with project implementation measured at the output, outcome and impact level, with UNDP staff carrying out their monitoring tasks competently with the support of the M&E officer. On the downside, only a few interventions were evaluated at the end of their implementation period.


Tag: Effectiveness Efficiency Human and Financial resources Monitoring and Evaluation Project and Programme management Results-Based Management Theory of Change SDG Integration SDG monitoring and reporting

17.

2.6 Portfolio management

Finding 13. While there is a high degree of focus under the employment and livelihoods and the democratic governance portfolios, the environment and resilience portfolio is more fragmented. Overall, the level of funding appears insufficient to contribute significantly to the ambitious outputs and outcomes identified in the CPD.

In total, the CPD numbered 41 projects. While there is a high degree of focus in the employment and livelihoods and the democratic governance portfolios, the environment and resilience portfolio is more fragmented with a total of 19 projects. However, the portfolio focuses on three main areas (climate change, sustainable management of natural resources and disaster risk management) and also includes large-scale projects of above $500,000. As already noted, the social inclusion portfolio is clearly underfunded, although this is an issue of design rather than implementation. Targets appear overambitious, especially in the environment outcome area (e.g. reduction of GHG emissions) considering the budget and scope of the interventions. Overall, further strengthening of synergies across portfolios is needed, to reduce the number of projects.


Tag: Climate change governance Environment Policy Effectiveness Efficiency Local Governance Migration Jobs and Livelihoods

18.

2.7 Joint programming

Joint programmes have proven difficult to establish with UN agencies seeing limited added value in inter-agency cooperation beyond access to new funds. Little progress has been made in deepening inter-agency cooperation. While two joint programmes were implemented in the previous cycle, there was only one example of inter-agency cooperation in the current CPD on a project involving UNDP, UN Population Fund (UNFPA), UN Women and UN Children’s Fund (UNICEF) supporting deinstitutionalization reforms for disabled people.


Tag: Efficiency Local Governance Partnership Project and Programme management UN Agencies

19.

2.8 Gender

Finding 14. While the CPD did not tackle gender issues through one of its outcomes, the performance of the country office in mainstreaming gender issues is satisfactory, with several interventions directly targeting women or integrating the principles of gender equality.

Between 2016 and 2018, the country programme budget was spent predominantly on projects designed to contribute to gender equality and women’s empowerment, with almost 60 percent of project expenditure classified under the GEN-2 marker. The employment programme was strong on gender mainstreaming, with special measures focusing on improving job opportunities for women. The CWP targeted mainly women (over 80 percent of programme participants are women) with valuable training, on-the-job learning and work experience, enhancing their skills, increasing their self-confidence and improving their employability.


Tag: Environment Policy Effectiveness Efficiency Gender Equality Gender Mainstreaming Gender-Based Violence Women's Empowerment Inclusive economic growth Jobs and Livelihoods Social Protection Capacity Building SDG Integration Women and gilrs

Recommendations
1

While the current CPD outcome areas remain largely relevant, the emphasis should be on integrated cross-sectoral interventions promoting balanced, inclusive and sustainable development across the country.

In terms of funding, the current CPD focused on employment, environment and local governance. These issues remain important and closely linked priorities for the country’s development. North Macedonia continues to suffer from low activity levels and high unemployment, which contribute to poverty and exclusion. Slow growth and job shortages are driving more and more people, especially young and educated ones, to seek better living conditions abroad, jeopardizing the country’s economic future. With climate change, already vulnerable local communities are facing an increasing risk of environmental disasters whose costs can hinder their development.

In this context, the priority for future interventions should be to improve people’s income and employment opportunities to raise living standards, targeting firstly the poorest and most vulnerable, within the overall objective of promoting balanced, inclusive and sustainable development. Given the interrelated nature of the country’s challenges, the efforts made during the current period to better integrate UNDP interventions across the various portfolios must be continued and strengthened as well as advice and expertise to improve systems and policies at the national level. This is particularly true in the field of social inclusion where addressing the needs of the most disadvantaged groups require targeted measures that integrate education, social protection and employment aspects.

2

The results framework and evaluation practices should be strengthened to better capture programme achievements and better assess contributions to SDG targets.

There is a need to evaluate more frequently the results and impact of completed projects in order to capture and share lessons learned, inform new policy directions and decide on the best ways to implement future projects. More generally, there is a need to strengthen the results framework to better reflect the contribution of the CPD to the SDGs and to improve the design of gender-specific indicators at outcome and output levels.

3

Support for effective employment policies should continue with a focus on women and young people and complementary measures in the field of social inclusion.

The challenges of the labour market call for a wider range of employment measures. While the unemployment rate has fallen from 32 percent in 2010 to 15.5 percent in the first half of 2019, the downward trend obscures the low participation rates, especially among women, and the large informal economy that mainly employs low-skilled workers. Youth unemployment remains excessive. The vocational education and training and adult education systems are not dynamic and responsive enough to close the gap between labour market supply and demand. In this context, UNDP should provide additional support with the design and testing/evaluation of new employment measures to better meet the needs of hard-to-employ groups, with a focus on young people and women. Efforts to improve the relevance of the educational offer should also be intensified to help address the growing skills mismatch.

4

UNDP should scale up subnational interventions, encouraging inter-municipal cooperation and reinforcing the role of regional planning in the context of EU integration.

Given its strong relationships with local stakeholders across the country and responding to needs from different perspectives, UNDP is well-positioned to continue supporting subnational governments in the decentralization process, promoting the principles of partnership and project-based management and helping local authorities become drivers of change in cooperation with civil society and other local actors.

Efforts to introduce e-governance and innovation into local governance should be intensified to improve service delivery and combat corruption, examining all aspects of local policies and service delivery from a technological, regulatory and/or administrative perspective and testing new approaches in pilot municipalities before replicating them on a larger scale.

UNDP should encourage municipalities to pool their resources and capacities to address more efficiently common development challenges. In the environment sector, this could mean working to promote greater synergies between local economic development and nature protection. To that end, UNDP should consider working with the Government to establish a Global Environment Facility Small Grants Programme in the country, learning from successful SGP programmes in neighbouring countries (e.g. Albania, Armenia, Romania).

North Macedonia has established regional centres to promote the natural and economic potential of its eight planning regions. UNDP could provide support, for example, by mobilizing stakeholders in the development of strategic projects, at the level of several municipalities or even the entire planning region, to target funding from the EU and international financial institutions.

Finally, UNDP should support capacity development at the national level to link with local governments, channelling funding for new services and showcasing the most effective programmes.

5

UNDP should continue to focus its environment portfolio on climate change and disaster risk reduction, with greater emphasis placed on helping the Government address the causes of air pollution.

Improving air quality in the main cities in North Macedonia will feature high on the Government’s agenda. Given UNDP’s track record in this area, it is recommended that the country office use data from the study of Household Heating to plan and implement innovative pilot projects to experiment possible measures to reduce the use of plastic and cheap coal for heating purposes by low-income families in Skopje’s suburbs. As the issue has a strong social dimension, it is also recommended that the UNDP environment and social inclusion sectors work in close cooperation.

Although the Government is showing greater commitment to promoting energy efficiency and the use of renewables, climate change still does not rank high in the country’s priorities. In this context, UNDP should support the Government with public awareness campaigns to shift the knowledge of ecological threats from a narrow group of experts and policymakers to the broader public and to encourage behavioural changes needed to fight climate change. Efforts to build institutional and technical capacity of national authorities for measuring and reporting on emissions, mitigation and adaptation activities in line with the Paris Agreement should be pursued in the next CPD.

Given that a large part of North Macedonia still lacks a multisectoral approach to disaster reduction, UNDP should continue to work on DRR, building on the achievements of the current period to promote integrated disaster risk reduction systems addressing all forms of risks, moving away from purely reactive interventions. The emphasis should be on the establishment and integration of effective prevention and response systems, taking into account the different needs of the population, particularly those most at risk of exclusion (the elderly, women, people with disabilities), including by integrating a gender perspective into DRR interventions to promote the equitable participation of men and women in environmental risk management and disaster prevention.

 

6

UNDP should seek opportunities to build on its strategic positioning in North Macedonia to further harmonize EU accession and SDG priorities.

UNDP is well-positioned to deepen its support to North Macedonia on the twin priorities of EU accession and SDG implementation, especially with the EU approximation and negotiation process underway. The EU is the main international donor in the country, and the Government has prioritized approximation of national laws, rules and procedures with the EU acquis.

For example, in the area of disaster management, UNDP support has been instrumental in dealing with the immediate consequences of the 2015 flash floods and facilitating the recovery process. Still, some project results are at risk in the absence of effective national policies to maintain water and sanitation infrastructure and enforce controls. Building on the expertise it has accumulated in recent years in this field, UNDP can play an important role at national and subnational levels helping to advance the country’s level of compliance with the EU Flood Directive, which in turn can help to achieve targets within SDG 6 (clean water and sanitation).

Management response not available

Latest Evaluations

Contact us

1 UN Plaza
DC1-20th Floor
New York, NY 10017
Tel. +1 646 781 4200
Fax. +1 646 781 4213
erc.support@undp.org