Final evaluation of the Support to Public Administration Project

Report Cover Image
Evaluation Plan:
2019-2021, South Sudan
Evaluation Type:
Final Project
Planned End Date:
01/2020
Completion Date:
08/2020
Status:
Completed
Management Response:
No
Evaluation Budget(US $):
50,000

Share

Document Type Language Size Status Downloads
Download document IGAD-Final Evaluation Report - 10 April 2020_.pdf report English 817.18 KB Posted 40
Title Final evaluation of the Support to Public Administration Project
Atlas Project Number: 00072642
Evaluation Plan: 2019-2021, South Sudan
Evaluation Type: Final Project
Status: Completed
Completion Date: 08/2020
Planned End Date: 01/2020
Management Response: Yes
UNDP Signature Solution:
  • 1. Poverty
Corporate Outcome and Output (UNDP Strategic Plan 2018-2021)
  • 1. Output 3.1.1 Core government functions and inclusive basic services4 restored post-crisis for stabilisation, durable solutions to displacement and return to sustainable development pathways within the framework of national policies and priorities
SDG Target
  • 1.2 By 2030, reduce at least by half the proportion of men, women and children of all ages living in poverty in all its dimensions according to national definitions
  • 1.a Ensure significant mobilization of resources from a variety of sources, including through enhanced development cooperation, in order to provide adequate and predictable means for developing countries, in particular least developed countries, to implement programmes and policies to end poverty in all its dimensions
Evaluation Budget(US $): 50,000
Source of Funding: Norway, UNDP
Evaluation Expenditure(US $): 50,000
Joint Programme: No
Joint Evaluation: No
Evaluation Team members:
Name Title Email Nationality
Richard Chiwara Dr mahachi.rc@gmail.com
Geoffrey Batali Mr. bataligk@gmail.com
GEF Evaluation: No
Key Stakeholders: UNDP, government of South Sudan, Donors - Norway, Kenya, Uganda, Ethiopia
Countries: SOUTH SUDAN
Lessons
1.

CSSOs can be twinned with more than one individual. The evaluators recorded numerous instances where the CSSOs mentored/coached the whole department or even the whole Ministry in basic tasks such as office procedures, work ethics and using computers. In Rumbek for example, key informants at the state MLPS&HRD said the CSSOs showed them how to keep the office environment clean and developed the filing system that they still use to this date.


Tag: Sustainability

2.

CSSOs cannot deliver standard comparable results if their contribution is based only on their individual initiative. Some CSSOs were considered to have been very successful and helpful by their respective twins, while others were considered not so successful indicating a need for CSSOs to have specific Terms of Reference when they are deployed.


3.

Practical level coaching/mentoring may not be enough to build effective capacity. Key informants, especially at senior level of Director and Director-General level felt that there was a need for their staff to get advanced training in order to establish a cadre of national civil service professionals who may be able to mentor their colleagues.


4.

Embedding the PMU in government enhances ownership of project processes and results. There was demonstrable institutional memory of the project within government during the evaluation as they were able to identify and track the deployment of twins in the states.


5.

Capacity involves developing the individual skills as well as retaining skilled personnel. Evidence obtained in visits to the states was that there is still a huge capacity gap across all MDAs as a number of Ministries were losing trained staff to NGOs and other organisations, largely because of low remuneration levels in the civil service. For example, the Kiir Mayardit hospital in Rumbek had one doctor, one anesthetist and no nursing staff, down from 14 doctors when it was established.


6.

Project processes and results are more likely to be sustainable if they have synergy with other initiatives within UNDP or other organisations. Some of the initiatives that were started by CSSOs collapsed when they left. Key informants at state level said they could not continue with the initiatives due to lack of funding as government was not funding operational budget for activities. The initiatives that have continued to survive are typically those that were linked to other ongoing initiatives, as for example in Torit where some of the Cooperatives were linked to the Food and Agriculture Organisation (FAO) and World Food Programme (WFP) interventions.

 

 


7.

Effectiveness can be compromised if direct beneficiaries are not engaged and involved in project processes from the beginning.  Directors of some of the hospitals that were visited said if they had been consulted, they would have recommended CSSOs with different professions from the ones that were deployed.


Findings
1.

Relevance

Based on evaluators’ observation of conditions at subnational levels, the project was highly consistent with beneficiaries’ requirements and needs. CSSOs not only provided coaching and mentoring in the substantive areas of work. They also supported their counterpart twins in other tasks such as how to use computers, how to write minutes of meetings, and how to write reports. There was evidence in the states that the CSSOs were highly appreciated across all the sector Ministries to which they were deployed.

The project was consistent with triangular South-South cooperation, as well as UNDP’s signature solution for Governance for peaceful, just, and inclusive societies as outlined in its Strategic Plan 2018 – 2021. By seeking to enhance institutional capacity at subnational levels, the IGAD initiative was therefore highly relevant and aligned to UNDP’s CPD, specifically the United Nations Cooperation Framework (UNCF)/CPD Outcoem1: Strengthened peace infrastructures and accountable governance at the national, state and local levels.

The project’s strategy for ‘on-the-job’ training through mentoring and coaching, whereby CSSOs were involved in direct service delivery within the institutions to which they were seconded directly aligned it to ten (10) Sustainable Development Goals (SDGs) – 1, 2, 3, 4, 5, 8, 9, 11, 12 and 16.


2.

Effectiveness

Three out of five indicators were exceeded, while one of the remaining two was satisfactory, while the fifth indicator was not rated due to insufficient data.

Indicator 1. Number of institutional policies developed to enhance operations. This indicator was achieved. At central government level, a number of policies were developed, including for example, Social Protection Policy and national Technical and Vocational Education and Training (TVET) policy and strategic framework.  CSSOs deployed at state level also contributed to development of various policies, including for example the state Employment Policy in Rumbek.

However, the evaluation also noted that the individual capacities of the twins as it relates to whether or not they were now independently capable to develop and produce these institutional policies was still limited. The evaluators observed that any outputs that were initiated by the CSSOs but not completed by the time of their departure have remained incomplete. For example, in the Rumbek state Ministry of Finance, the CSSOs had started to lead the development of the state Development Plan, but left before it was completed. This was still not completed by February 2020, and the responsible twins said that they did not have the capacity to complete the exercise independently.

Indicator 2. Number of strategic plans/frameworks developed to implement established policies. This indicator was achieved. 53 strategic plans/frameworks were developed against a planned target of five strategic plans/frameworks.  The evaluation independently verified that an Annual Operational Plan, Human Resources Strategy Plan, as well as Regulatory Audit Manual, Performance Audit Manual, and a Communications Audit Manual were developed at the National Audit Chamber (NAC). At state level, the evaluation verified that Annual Strategic Plans were developed with CSSO support in the following ministries, departments and agencies (MDAs): (a) state Ministry of Labour, Public Service and Human resources Development (MLPS&HRD) in Rumbek, (b) state Ministry of Agriculture and Rural development (MARD) in Torit and Yambio; (c) state Ministry of Trade in Yambio; and (d) state Ministry of Gender and Social Welfare (MOGSW) in Wau.

Indicator 3. Proportion of targeted institutions reporting improved work-related performance by the twins. This indicator was achieved. The evaluation noted that work-related performance of the ‘twins’ was increased and there was greater appreciation for the CSSOs in those sectors where they were also involved in direct service delivery, in particular in the hospitals. There were however two challenges that affected performance by twins. Firstly, there were cases of mismatched skills between the CSSO and twins, such that there was limited direct coaching/mentoring. Secondly, the project was undertaken during a period of austerity where the government only provided Chapter 1 budget (salaries), so there were no operational resources to carry out activities on which to base the coaching/mentoring.

Indicator 4. Proportion of twins expressing satisfaction over the twinning arrangement. This indicator was partially achieved. The evaluation noted that the indicator measured ‘the proportion of twins expressing satisfaction with twinning arrangements’ while the baseline and target measured ‘proportion of twins that gained knowledge from their coach/mentor’. This inconsistency could potentially give rise to significant variance in measurements. Majority of the twins interviewed during the evaluation were satisfied with the twinning arrangements. At the hospital in Torit for example, the local staff did not know how to use the Complete Blood Count (CBC) machine and Oxygen machine until after the CSSOs taught them.

Indicator 5. Number of targeted institutions rated as offering improved. There was no end line survey undertaken to determine the exact number of institutions that were providing improved services. Based on key informants interviewed, public service delivery was declining due to budgetary constraints. While this is not attributable to the project, is does however make it difficult to objectively assess improvement of service delivery in targeted institutions.

On the other hand, there were also institutions that could demonstrably attribute improved service delivery to the CSSOs. In Torit for example, the evaluation noted that when the state Ministry of Gender and Social Welfare was established by separating it from the Ministry of Youth, Culture and Social Welfare, the CSSOs helped to develop capacity of all ministry personnel as well as developing institutional policies and code of conduct.


3.

Efficiency

Overall, the evaluation found that the project’s quantitative outputs were delivered efficiently and could be directly attributable to its activities. The project delivery of available resources was high at 90.8 percent, which means management of the project by the Project Management Unit (PMU) in terms of activity implementation was commendable in spite of the hostile operating environment characterised by conflict and limited access to some areas.


4.

Sustainability

Although the project was anchored in national institutions, some of the civil service personnel had since left government in search of ‘greener pastures’. Secondly, the government has consistently lacked financial capacity to sustain the project’s results and processes as exemplified in Torit where Community Action Plans (CAPs) developed in the state Cooperatives Directorate were not been implemented due to funding challenges. Furthermore, a number of non-governmental organisations (NGOs) stopped actively supporting project activities when the CSSOs left, demonstrating a lack of confidence in government’s capacity to deliver as well as its accountability to partners and stakeholders.


Recommendations
1

UNDP should consider the following adjustments to the project design and strategy: Ensure that there are clear TORs and specific deliverables for the CSSOs upon their deployment so that capacity development is uniform laterally across states, as well as horizontally between central, state and county governments.

2

UNDP should establish a mechanism whereby CSSOs maintain contact with their twins via email and/or social media as part of continuous mentoring/coaching. The twins had developed personal and professional relations with their respective CSSOs, and a number of them would wish to continue the relationship.

3

UNDP should ensure that programme staff have more involvement and oversight of the project in order to retain institutional memory after the exit of project staff.

4

The Government should ensure that targeted institutions have appropriate staff that will be twinned with the CSSOs. This should include ensuring that there is clarity with regards to the grade level of the respective CSSO/twin as well as ensure that the twinned personnel remain in position for the duration of the project.

5

The Government should honour its commitments, especially with regards to welfare of CSSOs in terms of accommodation and transport. This should also include taking appropriate measures to ensure that trained personnel are retained in their positions.

Latest Evaluations

Contact us

1 UN Plaza
DC1-20th Floor
New York, NY 10017
Tel. +1 646 781 4200
Fax. +1 646 781 4213
erc.support@undp.org