Independent Country Programme Evaluation: Somalia

Report Cover Image
Evaluation Plan:
2018-2021, Independent Evaluation Office
Evaluation Type:
ICPE/ADR
Planned End Date:
12/2019
Completion Date:
12/2019
Status:
Completed
Management Response:
No
Evaluation Budget(US $):
150,000

Share

Document Type Language Size Status Downloads
Download document ICPE Somalia - ToR.pdf tor English 339.49 KB Posted 70
Download document ICPE Somalia - Annexes.pdf related-document English 1158.00 KB Posted 184
Download document ICPE_Somalia_2019.pdf report English 4848.65 KB Posted 158
Download document ICPE_Somalia_EvalBrief.pdf summary English 203.56 KB Posted 54
Title Independent Country Programme Evaluation: Somalia
Atlas Project Number:
Evaluation Plan: 2018-2021, Independent Evaluation Office
Evaluation Type: ICPE/ADR
Status: Completed
Completion Date: 12/2019
Planned End Date: 12/2019
Management Response: Yes
UNDP Signature Solution:
  • 1. Poverty
  • 2. Governance
  • 3. Resilience
  • 4. Energy
  • 5. Gender
Corporate Outcome and Output (UNDP Strategic Plan 2018-2021)
  • 1. Output 1.2.1 Capacities at national and sub-national levels strengthened to promote inclusive local economic development and deliver basic services including HIV and related services
  • 2. Output 1.6.1 Country-led measures accelerated to advance gender equality and women’s empowerment
  • 3. Output 2.2.2 Constitution-making, electoral and parliamentary processes and institutions strengthened to promote inclusion, transparency and accountability
  • 4. Output 2.2.3 Capacities, functions and financing of rule of law and national human rights institutions and systems strengthened to expand access to justice and combat discrimination, with a focus on women and other marginalised groups
  • 5. Output 2.6.1 Capacities strengthened to raise awareness on and undertake legal, policy and institutional reforms to fight structural barriers to women’s empowerment
  • 6. Output 3.1.1 Core government functions and inclusive basic services4 restored post-crisis for stabilisation, durable solutions to displacement and return to sustainable development pathways within the framework of national policies and priorities
  • 7. Output 3.3.1 Evidence-based assessment and planning tools and mechanisms applied to enable implementation of gender-sensitive and risk-informed prevention and preparedness to limit the impact of natural hazards and pandemics and promote peaceful, just and inclusive societies
  • 8. Output 3.3.2 Gender-responsive and risk-informed mechanisms supported to build consensus, improve social dialogue and promote peaceful, just and inclusive societies
  • 9. Output 3.5.1 Energy access re-established for crisis-affected populations, with a focus on gender-sensitive, risk-informed and sustainable recovery
  • 10. Output 3.6.1 Energy access re-established for crisis-affected populations, with a focus on gender-sensitive, risk-informed and sustainable recovery
Evaluation Budget(US $): 150,000
Source of Funding:
Evaluation Expenditure(US $): 147,000
Joint Programme: No
Joint Evaluation: No
Evaluation Team members:
Name Title Email Nationality
Elisa Calcaterra Evaluation Advisor and Lead Evaluator
Amanuel Zerihoun Evaluation Specialist and Associate Lead Evaluator
Andrew Rathmell Evaluation Consultant
Phillip Rawkins Evaluation Consultant
GEF Evaluation: No
Key Stakeholders:
Countries: SOMALIA
Lessons
Findings
1.

Finding 1. UNDP has gradually been building a portfolio of work which starts to address some key environmental challenges in Somalia, events, particularly droughts and floods, to provide access to sustainable energy and to support the establishment of building blocks for a functioning environmental governance, in partnership with some of the key institutional counterparts and other United Nations agencies. UNDP has also demonstrated that it can adapt to emerging priorities and emergency situations. However, long-term support for one of the key national priorities, disaster risk reduction, is not yet in place, beyond the support provided within the framework of the ECC project as a response to the 2017 emergency.


Tag: Disaster Risk Reduction Natural Disaster Renewable energy Efficiency Relevance Technical Support

2.

Finding 2. The portfolio supported policy processes, awareness-raising and project implementation. There is some evidence of successful technical support provided, but several policies are still pending validation/approval/implementation. There are signs of increased awareness of key environmental problems (e.g., charcoal production) and the need to move towards long-term solutions (RRF and at project scale). Project implementation, which has taken place in Somaliland and Puntland only, has seen a range of interventions with mixed results. Infrastructure rehabilitation and construction at this stage stands out as the area with the potential for providing tangible benefits, but monitoring and adaptive management were not always adequate. Overall, while the programme is growing, the response has so far only been able to address a small part of the country’s needs through small-scale interventions and sustainability of the interventions is at risk.


Tag: Challenges Effectiveness Technical Support Policies & Procedures Environment Policy

3.

UNDP and its partners have supported the development of several policies and strategies, as well as some communications related to environmental conventions. These were facilitated by close cooperation with some key institutional actors at the Federal Government of Somalia (e.g., Ministry of Planning, Investment and Economic Development (MOPIED), Directorate General of Environment), in Puntland (e.g., Ministry of Environment, Agriculture and Climate Change) and Somaliland (Ministry of Environment and Rural Development), and by a clarity of roles and responsibilities when implementation was carried out in partnership with other United Nations agencies (e.g., PROSCAL). Several are however still at some stage of the approval process and there is no evidence yet of implementation. This is not unique to environmental policies and strategies and is partially due to the complex and fragmented political environment in the country, specifically the diverging opinions of the Federal Government of Somalia and Federal Member States, as well as among Federal Member States and the international community. While there is no estimate of the time required to complete a policy process, with very few having been completed, the table below summarizes the most recent developments and the time lapses, particularly between technical development and the next steps.


Tag: Jobs and Livelihoods Youth Stabilization Environment Policy Effectiveness Efficiency UN Agencies Technical Support

4.

Finding 4. UNDP has been engaged in addressing the humanitarian and development divide on the issue of IDPs in Somalia through the lens of durable solutions. Through its participation in flagship joint United Nations programmes, UNDP is affirming its positioning as a development actor and developing a differentiated response in what has been a field dominated by humanitarian actors. However, evidence of results is not yet available.


Tag: Joint UN Programme Strategic Positioning Displacement Humanitarian development nexus Coherence Relevance Technical Support

5.

Finding 5. Through the Innovation Facility, UNDP is proposing a differentiated and positive contribution leveraging technological solutions to address social problems and stimulating the emergence of a digital economy in Somalia. While at an early stage to observe tangible results and potential for job creation and employment, it appears particularly relevant to the Somali context in responding to the demands and aspirations of youth.


Tag: Jobs and Livelihoods Youth Innovation Effectiveness Technical Support

6.

Finding 6. Effective institutions. The three main projects within the effective institutions portfolio formed the core of the UNDP investment in State-building in Somalia, and together contributed substantially to the formation of State structures and governance processes at three levels of government. All three projects made timely and effective contributions in the sphere of the production and adoption of relevant high-level policies and strategies. The portfolio also showed the ability to adapt and change course as circumstances required or as new, urgent priorities became apparent; and stakeholders confirmed that consultations took place at design and implementation stage.


Tag: Anti-corruption Public administration reform Effectiveness Technical Support

7.

Finding 7. Effective institutions – stabilization. Support to stabilization has been a valuable component of the international efforts to stabilize newly liberated areas. UNDP has played a central, if low-key, role in enabling the Somali Government to progress the roll-out of local governance in liberated areas. Its added value has been the injection of management capacity into the Somali Government. UNDP adapted well during the life of S2S to changing context. However, the evaluation cannot rigorously assess the impact of UNDP programming on actual stabilization outcomes because of a lack of evaluation evidence and monitoring information.


Tag: Capacity Building Strategic Positioning Stabilization Effectiveness Technical Support

8.

Finding 8. Inclusive politics. The portfolio targeted support to institutions and political processes which are critical to the formation of a fully equipped modern State, providing timely and effective contributions in the sphere of the production and adoption of relevant high-level policies and strategies (e.g., electoral law). There was a consistent effort to ensure that consultations took place with Somali counterparts, but efforts were stymied by major political barriers and the portfolio struggled to respond to urgent needs of government counterparts and to maintain close relations with donors.


Tag: Election Rule of law Effectiveness Relevance Technical Support

9.

Finding 9. Gender in effective institutions and inclusive politics. The interventions in the effective institutions portfolio have attempted to mainstream gender equality with dedicated activities geared to enhancing women’s capacities and their representation in decision-making positions. However, results accomplished have been limited largely to policies adopted, rather than their implementation, and to enhanced representation without an improved presence for women in decision-making, except for more recent interventions under JPLG, which is now addressing the issue. Inclusive politics has achieved limited success in terms of integration of gender equality concerns into project plans and efforts to support gender mainstreaming with government partners, with one exception, WPP, which is the only project dedicated to gender equality and women’s empowerment and is highly valued by women’s organizations. The projects providing support to constitutional review and Parliament paid very limited attention to matters related to gender equality and women’s empowerment and demonstrated weak integration of gender equality into overall project planning. Like WPP, election support is rated highly for effort and for effective integration of gender equality and women’s empowerment into project plans and projected results.


Tag: Gender Equality Gender Mainstreaming Women's Empowerment Effectiveness Technical Support

10.

Finding 10. Inclusive politics. After a very positive beginning in the first part of the evaluation period, relations between UNDP and the UNSOM Political Affairs and Mediation Group have become difficult, except in the sphere of elections support. Relations with UN-Women are also challenging. This has been to the detriment of effective project implementation (see finding 9) and has affected the relations of joint programmes with Somali partners.


Tag: Election Human rights Gender Mainstreaming Challenges UN Agencies Coordination Technical Support

11.

Finding 11. The long duration of the rule of law programme has meant that medium-term initiatives have had time to bear fruit, particularly in Somaliland and Puntland. The Joint Rule of Law Programme delivered tangible improvements at the activity and output levels in access to justice, and judicial and police capacity. However, the absence of data means that it is not possible to rigorously assess the overall impact of rule of law programmes, despite the large sums spent on them.


Tag: Justice system Rule of law Joint UN Programme Challenges Technical Support

12.

Finding 12. The Joint Rule of Law Programme helped to progress political agreement over structures for policing and justice across Somalia. UNDP justice and police programmes adapted well to changing political and strategic circumstances. The New Policing Model, while still contested by some in the federal police, has provided a structure to enable cooperation supported by the Joint Police Programme.


Tag: Justice system Rule of law Joint UN Programme Effectiveness Technical Support

13.

Finding 13. The original design and management of the Joint Rule of Law Programme proved to be flawed. The new joint programmes in the portfolio are better designed and more manageable but further work is needed to ensure that they deliver effectively for all concerned. However, in Somaliland, engagement on rule of law has been problematic, and engagements in South-Central have sometimes been difficult. Some key areas are currently not covered. This is understandable due to the need to focus the support provided by the portfolio but they remain a priority for Somalia in the long term.


Tag: Justice system Rule of law Joint UN Programme Programme/Project Design Challenges UN Country Team Technical Support

14.

Finding 14. The Joint Human Rights Programme has successfully raised the prominence of human rights and has the potential to help Somalia fulfil its commitments. However, a lack of political will on the side of the Somali authorities, the United Nations and donors has led to slow progress.


Tag: Disabilities Human rights Gender Mainstreaming Joint UN Programme Effectiveness Advocacy

15.

Finding 15. The UNDP project on PCVE only began in September 2018, making it too early to assess its contribution to outcome-level results. However, the evaluation positively assesses the approach adopted by UNDP.


Tag: Crisis prevention Extremism Challenges Technical Support

16.

Finding 16. UNDP work on security sector governance seeks to take a more systematic approach to capacity-building across central and member state security ministries and has made initial progress. However, it is not yet possible to assess outcomes-level results and sustainability is a real risk.


Tag: Capacity Building Security Challenges Relevance Technical Support

17.

Finding 17. Programme coherence. The programme is not structured around a coherent theory of change which highlights the causal pathways from project-level contributions to programme outcome and impact levels. In addition, and as a result of the above, there is limited coordination among interventions, compounded by a lack of systematic information-sharing, knowledge management and coordination of activities, as well analysis of trends and common risks. Additionally, the engagement in the various locations of the country is detailed only at project level without an overarching strategy and guiding framework, clearly communicated to partners, leading to misunderstanding with counterparts about what UNDP can deliver in each location.


Tag: Knowledge management Programme Synergy Theory of Change Challenges Technical Support

18.

Finding 18. Capacity development versus capacity injection. UNDP development programming in Somalia is mostly, though not exclusively, about capacity development. Exceptions include interventions focusing on the rehabilitation and development of infrastructure and livelihoods support (e.g., development of value chains). Yet what is notable is the absence of a coherent capacity development strategy and the reliance on capacity injection to support key governance process through the provision of external expertise. While it has not worked in all cases (see section 2.2), this approach has enabled UNDP to achieve several results under the governance programmes (see sections 2.3 and 2.4), but raises the questions of long-term sustainability and efficiency as well as value for money. When UNDP support is withdrawn as a project ends abruptly, this leaves institutions not fully capable of exercising their mandates. Equally, pressure to expand to new geographic areas may lead to the spreading of resources too thinly at the expenses of quality in delivery.


Tag: Capacity Building Programme Synergy Reconstruction Coherence Effectiveness Efficiency Sustainability Technical Support

19.

Finding 19. Management and operations. There is appreciation by partner agencies for the key role played by UNDP operations in supporting their engagement in Somalia, with several noting that they would not be able to operate otherwise. However, government counterparts, donors and some UNDP staff raise the problem of speed in delivery and quality of processes, including reporting and level of responsiveness.


Tag: Strategic Positioning Efficiency Relevance Country Government UN Country Team Operational Services

20.

Finding 20. Management and operations. In all three outcome areas, the financial execution rate has been above 70 percent. Financial delivery only, especially in a complex and fragile context like Somalia, may however not necessarily lead to the delivery of results. The management efficiency ratio reached 27 percent196 in 2018, but was otherwise stable at around 10-15 percent. While these costs are higher than the average (4-6 percent), they are justified by the operational context of Somalia. Attention needs to be directed in case the 2018 situation is repeated and causes need to be identified.


Tag: Service delivery Efficiency Sustainability UN Country Team Operational Services

21.

Finding 21. Aid coordination. UNDP has been central to the design of the first aid coordination architecture in the country, manages one of the three trust funds (MPTF) and through the effective institutions portfolio (SIP project) supported the establishment of the Aid Coordination Unit. While the concept was valid and the need for coordination is undisputed by stakeholders, conflicts and political disputes among stakeholders have interfered with the effectiveness of the Unit and the utility of the aid architecture. The performance of the Aid Coordination Unit, which is entirely staffed by technical consultants, has not delivered the expected results. While the MPTF is meant to increase transparency and facilitate the coordination of aid flows to the country, stakeholders report several challenges with its operations.


Tag: Human and Financial resources Aid Coordination Efficiency Coordination

22.

Finding 22. Joint programming and joint implementation. Approximately 75 percent of the UNDP programme over the period 2015 to present has been jointly implemented. The type of arrangements varies and not all joint programmes have the same characteristics (see sections 2.1, 2.2, 2.3, 2.4). The more recent joint programmes have been designed based on lessons learned from the first generation of such programming and are assessed as fitter for the purpose (see finding 12). Results over the period vary in quality and degree of sustainability, which remains a risk for several interventions (see sections 2.1, 2.2 and 2.4 for specific findings). Excellence in delivery is recurrently referred to as the only UNDP tool to make progress in the complex context of Somalia and the evaluation observed an effort to learn and adapt from experience in joint implementation.


Tag: Joint UN Programme Sustainability Coordination

23.

Finding 23. Gender. UNDP in Somalia is moving away from the implementation of interventions directed only at gender, as recommended in the 2015 ADR, to an approach which mainstreams relevant aspects in the entire programme. This was supported by a restructuring of the function but is not yet institutionalized and formalized through a dedicated strategy and buy-in by staff and counterparts. While the adoption of such a direction is vital, the difficulty of getting there should not be underestimated. It should also be noted that there remains one gender–specific project, WPP, which is greatly valued by women’s organizations and should be continued (see finding 9).


Tag: Gender Equality Gender Mainstreaming Efficiency Operational Services

24.

Finding 24. Design, monitoring and evaluation. There is some evidence of learning from evaluations and reviews, specifically for strategic decision-making. However, monitoring is identified as a challenge, with shortcomings observed both in terms of quality, frequency and accuracy of direct monitoring by UNDP and ineffective use of TPM. Quality of the involvement of partners and donors in the project design phase is mixed.


Tag: Monitoring and Evaluation Challenges Efficiency Operational Services

Recommendations
1

In framing its next Country Programme, UNDP should determine its added value in the Somali aid ecosystem, review its communication strategies with partners and donors, and explicitly articulate its contributions to overall development results through theory of change approaches. The Country Programme development process should ultimately strengthen the strategic focus of the programme, develop synergies across portfolios, and ensure sustainability

2

Ensure that sustainability in relation to capacity development, finances and projects is at the center of the project design, monitoring and adaptive management efforts, in cooperation with partners in the Country.

3

A rights-based approach is fundamental to UNDP approaches and should be a central part of the mutual accountability framework.

4

The office should strengthen its results-based systems and practices. In particular, solid evidence, systematically collected, used for adaptive management, and communication with donors and partners, is central to success of UNDP efforts in Somalia.

5

The reorganization of the gender function should be supported through the establishment of incentives and measures which will embed it into the implementation of the programme.

6

Programme management and operations: while understanding the complexity of the context, UNDP has to make sure that it has the best possible systems in place to ensure efficient delivery of its programme.

7

Aid Architecture: UNDP in coordination with UN RCO and UNSOM should clarify the roles and responsibilities of UNDP and the other actors in the management and support of the aid coordination system, in line with the roles and responsibilities of each actor under the UN Reform, with a view to streamline and simplify the process, improve operational efficiency and ensure transparent aid flows to both the FGS and the FMS.

8

Joint Programming: a critical weakness of the JPs is the absence of provision for collective oversight and for vesting authority in the Programme Management Board. It is recommended that this deficiency in the regulations and guidance provided to Joint Programs be addressed as a matter of urgency. It is recognized that this may well require a formal agreement among agency heads at HQ and that it should be supported by joint monitoring and evaluation (see recommendation 4)

Latest Evaluations

Contact us

1 UN Plaza
DC1-20th Floor
New York, NY 10017
Tel. +1 646 781 4200
Fax. +1 646 781 4213
erc.support@undp.org