Final evaluation of the Public Financial Management Project

Report Cover Image
Evaluation Plan:
2019-2022, South Sudan
Evaluation Type:
Final Project
Planned End Date:
11/2020
Completion Date:
11/2020
Status:
Completed
Management Response:
Yes
Evaluation Budget(US $):
30,000

Share

Document Type Language Size Status Downloads
Download document UNDP Support to Public Financial Management Summative Evaluation ToRs.pdf tor English 107.94 KB Posted 1068
Download document FINAL EVALUATION OF PUBLIC FINANCE MANAGEMENT PROJECT.pdf report English 1077.63 KB Posted 1136
Title Final evaluation of the Public Financial Management Project
Atlas Project Number: 86376
Evaluation Plan: 2019-2022, South Sudan
Evaluation Type: Final Project
Status: Completed
Completion Date: 11/2020
Planned End Date: 11/2020
Management Response: Yes
Focus Area:
  • 1. Others
Corporate Outcome and Output (UNDP Strategic Plan 2018-2021)
  • 1. Output 1.2.1 Capacities at national and sub-national levels strengthened to promote inclusive local economic development and deliver basic services including HIV and related services
SDG Target
  • 1.2 By 2030, reduce at least by half the proportion of men, women and children of all ages living in poverty in all its dimensions according to national definitions
  • 1.a Ensure significant mobilization of resources from a variety of sources, including through enhanced development cooperation, in order to provide adequate and predictable means for developing countries, in particular least developed countries, to implement programmes and policies to end poverty in all its dimensions
Evaluation Budget(US $): 30,000
Source of Funding: Japan. Africa Development Bank
Evaluation Expenditure(US $): 33,691
Joint Programme: No
Joint Evaluation: No
Evaluation Team members:
Name Title Email Nationality
Momodou Foon External evaluation msfoon@yahoo.com
GEF Evaluation: No
Key Stakeholders: National and state ministries of Finance, Revenue authorities Japan, AfDB
Countries: SOUTH SUDAN
Lessons
1.
  1. Lesson Learned in terms of the CPD's Relevance 

The extent to which the CPD’s intended output and outcomes are consistent with national and local policies and priorities and the needs of targeted beneficiaries.

Best practices:

  • The CPD is well aligned with South Sudan’s NDS structure and related governmental policy and planning frameworks
  • The CPD’s pillar design articulates a solid and robust model of a nexus-inspired mix of strategic interventions
  • The project initiation plan (PIP) modality is a promising avenue for the South Sudanese context and could be scaled up to blossom into full-fledged programmes if the ideas prove successful.
  • UNDP’s Recovery and Resilience Programme proved highly useful in providing affirmative action specifically to vulnerable women and youth by providing specialized skills development training focusing on the development of agricultural value chains and entrepreneurship (about 69% of 4,581 youth whose skills sets were developed in 22 trades were women). This programme could be scaled up to address the lack of skills of women and raise the profile and engagement of women in decision-making processes
  • The successful GEWE partnership of UNDP with the University of Juba could be scaled up and possibly extended beyond the “Women’s transformational leadership programme“ and the ”ICT project to support young women and girls in programming and IT skills“
  • The peanut butter-making agricultural value chain project in Aweil (Lake Region) was designed to boost grassroots-level economic cohesion. As a best practice it could serve as a model for similar projects in other communities.
  • The policy discussion held by UNDP in 2020 on the global human development report discussing the applicability of the global theme on how disasters impact inequality sparked interest among members of the Economic cluster on the importance of considering the significance of the environment for economic development. This kind of event could be repeated and expanded with related follow-up meetings to inform on practical implications for stakeholders including the private sector

       2. Lesson Learned CPD Coherance 

The compatibility of the CPD with other interventions in South Sudan to understand whether they support or undermine the programme, and vice versa; incl. internal coherence and external coherence.

Best practices:

  • The CPD echoes the HDP nexus approach and presents an integrated ensemble of programmatic outputs that are coherently interlinked with relevant strategic frameworks including the NDS, UNCF, HRP.
  • UNDP’s activities under CPD Outcome 1 are well articulated with UNMISS’ substantive thematic remit and the working relationship with the Mission is well developed.
  • The revitalization of UNDP’s environmental portfolio filled a previously existing gap.
  • In terms of UNDP’s engagement in addressing the triple nexus with partners other than UNEP, the practice of community level peace agreements between migrant and sedentary communities marks an innovative practice of managing conflict dynamics between cattle herders and peasants. These agreements spare countless lives, given that increasingly cattle herders, vigilante groups and youth militias are carrying arms. This is an area to be considered for continued investment and upscaling.

 

3. Lesson Learned in terms of CPD Effectivness 

Effectiveness: Progress against planned results at output and outcome level.

Best practices:

  • UNDP’s approach to fulfilling is role as chief (co-)convenor/coordinator of crucial processes such as the peace process, constitutional process, NDS planning/design process, humanitarian funding mechanism etc. garnered high marks among respective stakeholders.
  • UNDP is suited as convenor to help UNMISS mobilize a vast array of stakeholders to participate in peacebuilding negotiation track processes.
  • UNDP set up productive partnerships - across both CPD Outcomes and related CPD pillars - with several other UNCT entities including UN Women, UNEP, UNESCO, UNFPA, UNICEF and UNMISS.  
  • Inter-agency partnerships made possible collaborative efforts to ensure PBF re-eligibility.
  • The CPD is effective in reaching or surpassing Gender-specific sub-targets.

4. Lesson Learned in terms of the CPD's Efficiency 

Efficiency: The degree to which resources (funds, expertise, equipment, time, etc.) provided as inputs were economically used and converted into results.

Best practices:

  • Resource mobilization capacity is strong.
  • Time-bound targets are quite often met early and performance levels not seldom exceed set (final) targets.
  • Gross and net budget absorption capacity reach more than satisfactory rates pretty much across the board.

5. Lesson Learned in terms of the CPD's Sustainability 

Sustainability: The likelihood of successful continuation of programme/project implementation after external development assistance has come to an end.

Best practices:

  • The underlying logic of the capacity building approach used for the GEMS (cascade approach rather than “mirroring”/one-on-one tutoring) appears promising
  • The PIP approach qualifies as best practice that could be replicated on a larger scale to test out innovative solutions
  • Overall, the strategic choice of linking a strong anti-poverty and skills building approach under Outcome 2 with the peacebuilding focus (social cohesion, conflict mitigation and resolution, reconciliation, A2J) of Outcome 1 constitute a sustainable recovery package.

6. Lesson Learned in terms of the CPD's Impact 

Impact: Transformative, structural change at the level of knowledge, attitude, behaviours and practices going deeper than, and beyond, mere surface-level adjustments.

Best practices:

  • Support to the National Dialogue as successful first-ever grass roots consultative process in South Sudan.
  • Support to strengthening the peace infrastructure including the establishment of a ground-breaking conflict early warning and response system.
  • Support to the NDS design leading to the desired result.
  • Improvement of public financial management landscape.
  • UNDP successfully filled its role as convenor of the SSHF.
  • The Aweil Safety Net is a promising example of an HDNP project model and a key candidate for upscaling.. 
  • Initiation of important gender equality and women’s empowerment advancements.
  • UNDP is instrumental in orchestrating the integrated response to the COVID-19 pandemic through coordinating the design and implementation of the SERP.

Findings
1.

Key Findings and Conclusions
A. Key achievements and best practices under the evaluation criterion of CPD relevance included:

  • The CPD is well aligned with South Sudan’s NDS structure and intent;
  • The CPD’s pillar design articulates a solid and robust model of a nexus-inspired mix of strategic interventions;
  • Using the project initiation plan (PIP) modality seems a promising avenue for the South Sudanese context to test out ideas that could be scaled up and blossom into full-fledged programmes if they prove successful.

Related key challenges to be mentioned include: 

  • Conditions are still such that a potential switch from a DIM to NIM modality cannot be considered anytime soon and requires massive investments (way beyond the GEMS initiative designed to support the Revitalised Transitional Government of National Unity (R-TGoNU) during the transitional period mandated;
  • The difficult terrain and recurrent insecurity are clear operational limitations in terms of geographical/territorial reach among NGOs, CSOs and Government services;
  • The limited geographical area(s) of coverage in terms of rule-of-law, security, justice and corrections, as well as basic social services’ delivery and presence of the Government Authorities is, at the same time, a cause and an effect of the absence of the rule-of-law resulting in lawlessness and protracted armed violence.

B. Coherence-related key achievements and best practices were: 

  • The CPD echoes the HDP nexus approach and presents an integrated ensemble of programmatic outputs that are coherently interlinked with relevant strategic frameworks including the NDS, UNCF, HRP;
  • UNDP’s activities under CPD Outcome 1 are well articulated with UNMISS’ substantive thematic remit and the working relationship with the Mission is well developed;
  • The revitalization of UNDP’s environmental portfolio filled a previously existing gap.

In terms of key challenges, the following was noted:

  • Internal programmatic coherence and related work processes (within UNDP, across internal work units) are not optimal throughout and there seems to be some potential for streamlining programmatic design, in-house communication and the coordination of work flows.
  • While progress in this regard has been registered in the recent past, UNDP still has a tendency to work in internal silos, even within the same project.
  • UNDP is still in the process of rebuilding its former strategic position and reputation as key player in the area of environmental/climate change/renewable energies; and the UN and the country as a whole lost valuable time due to the UNDP CO management’s decision in 2014, to close down its related portfolio and leave the entire responsibility to UNEP.
  • While UNDP has positioned its CPD to build resilience both systemically and socio- economically, structurally the “aid machinery” and related pattern or even culture of funding has been biased in favour of humanitarian activities which limited the relative availability of financial resources for investments into sustainable development.

C. With regard to CPD effectiveness, key achievements and best practices comprised:

  • Overall, the CPD registered above-average progress in implementing activities and achieving set targets at more than satisfactory levels, on average;
  • UNDP’s convenor role was borne out in its ability to help UNMISS mobilize a vast array of stakeholders to participate in peacebuilding negotiation track processes;
  • Across both CPD Outcomes and related CPD pillars, UNDP entered into productive partnerships with a number of other UNCT entities including UN Women, UNEP, UNESCO, UNFPA, UNICEF and UNMISS;
  • PBF re-eligibility was recently ensured thanks to collaborative efforts made possible through inter-agency partnerships; v. Gender-specific sub-targets were mostly reached if not surpassed.

Effectiveness-related key challenges included:

  • Implementation progress of interventions under CPD Outcome 1 (“Strengthened peace infrastructure and accountable governance at the national, state and local levels”) were significantly hampered by contextual delays to the constitutional and peacebuilding processes beyond the direct influence of the UN, such as the fact that a Government of national Unity could only be formed after lengthy delays, in early 2020, while intercommunal tensions would regularly flare up over recent years and levels of violence were on the rise across the country, in 2020.9
  • The Covid-19 pandemic slowed down the implementation of activities across the board and required adaptive partial retooling of interventions and repurposing of resources, specifically affecting Outputs 1.3. (“Key governance institutions are enabled to perform core functions in line with the New Deal and the outcome of the peace process”) and 1.4.(“Capacities developed across the whole of government to integrate the 2030 Agenda, the Paris Climate Agreement and other international agreements, and to analyse progress towards the Goals, using innovative and data-driven solutions”), in a negative way.
  • The remoteness and lacking infrastructure in the countryside make it difficult for UNDP to ensure an equal reach of its interventions in the geographical sense, and results in a bias towards urban centres and their vicinities. Transport by road or air is difficult and costly due to the poor road network, underdeveloped and expensive air travel, heavy rains causing floods etc. All these factors combined make some of the geographical locations inaccessible, especially. during the rainy season. Furthermore, these factors negatively affect the ability of the UN as a whole to provide services to remote locations which is hence a general challenge in the South Sudanese context.

D. Key achievements and best practices under the evaluation criterion of efficiency include the following observations: 

  • Resource mobilization capacity proved to be strong;
  • Time-bound targets were quite often met early and performance levels often met or in some cases even exceeded set targets;
  • Gross and net budget absorption capacity reached more than satisfactory rates pretty much across the board.

Key Challenges include: 

  • Overall, longer-term development work is still in direct competition with short-term humanitarian support, and there is need for a deeper articulation of HRP design and implementation processes as well as funding flows, vis-à-vis a more sustainable HDPN approach.
  • The South Sudanese operational context is characterized by high costs of doing business due to the compounded effects of high levels of insecurity, the pronounced need to import goods and services including human capacity, the remoteness of many targeted locations compounded by inaccessibility during the rainy season, etc.
  • There are some (relatively minor) concerns regarding the SMARTness of CPD indicator statements and the horizontal coherence between indicator formulations, baseline data and targets.

E. Under the evaluation criterion of sustainability, key achievements and best practices included:

  • The underlying logic of the capacity building approach used for the GEMS (cascade approach rather than “mirroring”/one-on-one tutoring) appears promising;
  • The Project Initiation Plan (PIP) project design modality qualifies as best practice that should be replicated on a larger scale to test out innovative solutions;
  • Overall, the strategic choice of linking a strong anti-poverty and skills building approach under Outcome 2 with the peacebuilding focus (social cohesion, conflict mitigation and resolution, reconciliation, A2J) of Outcome 1 constitute a sustainable recovery package.

Related key challenges were:

  • In the recent past, there was an unfortunate pattern of complete overhaul of civil service staff following changes in political leadership, which negatively affected institutional memory and capacity.
  • The actual level of capacities is sometimes lower than originally assessed, by UNDP, which results in too high a threshold of training interventions that exceeds basic demand and thus created a gap between the bulk of capacity training needs and expectations and the higher-level management training on offer.
  • Limited understanding of programme/project design processes results in limited actual involvement of Government institutions during the design phase, resulting in false expectations and subsequent frustrations among GoSS staff and entities about construed “unmet promises” (examples: GEMS and trade projects). This lowers levels of buy-in and appropriation of GoSS stakeholders and represents a missed opportunity in terms of capacity building. In addition, this creates unnecessary, actually unfounded and thus avoidable friction and frustration between GoSS and UNDP staff.

Recommendations
1

The design of the project should be expanded to include taxpayers with a focus on taxpayer education, ministries of finance as the central policy arm and driver of the tax system and address fiduciary risks in the Public Financial Management (PFM) system.  Revenue Authority enterprise risks; taxpayer compliance risks, and fiduciary risks in budget execution to enthrone transparency, accountability and probity in PFM should be addressed.The design of the project should be expanded to include taxpayers with a focus on taxpayer education, ministries of finance as the central policy arm and driver of the tax system and address fiduciary risks in the Public Financial Management (PFM) system.  Revenue Authority enterprise risks; taxpayer compliance risks, and fiduciary risks in budget execution to enthrone transparency, accountability and probity in PFM should be addressed.

2

In a post-conflict country like South Sudan with weak institutions and depressed work environments, for optimal impact, capacity ought to take a wider view by refocusing from capacity building restricted to training alone, but extended to human and institutional productivity which involves support to enhance the work environment through addressing - energy, furniture and equipment needs; strengthening processes through business process model and digitalization; elaborating strategic plan and administering staff appraisal frameworks; supporting wellness – staff health insurance plans; and, of course, training to upscale knowledge and skills.

3

The Project management unit should provide a one-stop-shop for data and information management in accordance with M&E tracking results requirement recording the project’s activities and results, reports, outcomes, outputs, main stakeholders, principal beneficiaries, budgets and expenses.

4

Given the quantum of social services delivered at the local government levels, counties and payams, there is need to start a development planning process to drive the budgeting process in the states from the payam level. All efforts must be mustered to elaborate integrated development plans at every level which would lead to the fiscal budgets at the state level.

5

Gender mainstreaming and women empowerment should be enhanced by appropriately designed training programmes to sharpen the competences of CSOs and women’s groups to represent and advocate for the interests of children and women as well as human rights in the budget process.

1. Recommendation:

The design of the project should be expanded to include taxpayers with a focus on taxpayer education, ministries of finance as the central policy arm and driver of the tax system and address fiduciary risks in the Public Financial Management (PFM) system.  Revenue Authority enterprise risks; taxpayer compliance risks, and fiduciary risks in budget execution to enthrone transparency, accountability and probity in PFM should be addressed.The design of the project should be expanded to include taxpayers with a focus on taxpayer education, ministries of finance as the central policy arm and driver of the tax system and address fiduciary risks in the Public Financial Management (PFM) system.  Revenue Authority enterprise risks; taxpayer compliance risks, and fiduciary risks in budget execution to enthrone transparency, accountability and probity in PFM should be addressed.

Management Response: [Added: 2020/11/19]

UNDP has taken note of this recommendation and as PFM reform is a process, the project (Support to PFM in SS) seeks to further employ frameworks to ensure that risks in revenue collection and budget appropriation are reduced. Tax education is already part of the first phase of the project and the States would be supported to put in place further robust systems

Key Actions:

Key Action Responsible DueDate Status Comments Documents
1.1 Design and employ framework that reduce risk in revenue collection and budget appropriation
[Added: 2020/11/19] [Last Updated: 2021/08/07]
PFM Manager 2022/12 Not Initiated UND Supported for the establishment of a unified tax system consists of three key pillars: (1) the enactment of the state revenue authority bills (SRA), (2) existence of legal framework, and (3) existence of SRA governance structures. UNDP also supported the SRA establishment and conducted market and tax-related assessments, which informed the design of the SRA bills and tax laws. These pillars has harmonized revenue collection and administration within the three states resulting in enhanced accountability and public finance management and oversight functions ( Reference is made on page 12 and 13 of the attached report). History
1.2 Expand taxpayer’s education
[Added: 2020/11/19]
PFM Manager 2022/12 Not Initiated Accepts recommendation fully
2. Recommendation:

In a post-conflict country like South Sudan with weak institutions and depressed work environments, for optimal impact, capacity ought to take a wider view by refocusing from capacity building restricted to training alone, but extended to human and institutional productivity which involves support to enhance the work environment through addressing - energy, furniture and equipment needs; strengthening processes through business process model and digitalization; elaborating strategic plan and administering staff appraisal frameworks; supporting wellness – staff health insurance plans; and, of course, training to upscale knowledge and skills.

Management Response: [Added: 2020/11/19]

The PFM project accepted the recommendation and will seeks to build both human and institutional capacities as well as influence both policy and legal environment for accountability, transparency and collaboration between national and states.

Key Actions:

Key Action Responsible DueDate Status Comments Documents
2.1 Capacity building for State Revenue Authority (SRA) staff/management and State Legislative Authority (SLA) members
[Added: 2020/11/19] [Last Updated: 2021/08/07]
PFM Manager 2021/06 Completed This is to ensure that revenue collection and management are enhanced. To this end, 350 tax officers (made of men and women from the various States) were trained on tax policy and non-oil revenue administration to ensure that tax revenues are collected effectively and efficiently for enhanced service provision ( reference is made on the attached report page 6) History
2.2 Provision of equipment and renovation of office space to SRA
[Added: 2020/11/19] [Last Updated: 2021/08/07]
PFM Manager 2021/06 Completed The project Completed construction of the State Revenue Authority office in Western Bahr el Ghazal. UNDP also supported, installation of ICT equipment in three target states commenced in 2021. Contractors installed the equipment in a pre-determined sequence in the three target locations ( reference is made on page 6 of the attached report) History
2.3 Review laws, legislations and policies to support the revenue mobilization in the states and ensure accountability, transparency and collaboration between national and states.
[Added: 2020/11/19]
Senior Economist 2022/12 Not Initiated Aligned to Peace Agreement.
3. Recommendation:

The Project management unit should provide a one-stop-shop for data and information management in accordance with M&E tracking results requirement recording the project’s activities and results, reports, outcomes, outputs, main stakeholders, principal beneficiaries, budgets and expenses.

Management Response: [Added: 2020/11/19]

The SPACE team take note of this recommendation and will work to further strengthen the M&E and data management system to truck results.

Key Actions:

Key Action Responsible DueDate Status Comments Documents
3.1 Provide training on taxpayers’ data collection system for SRA staffs
[Added: 2020/11/19]
PFM Manager 2022/12 Not Initiated
3.2 Strengthen and expanded ICT component of the project to aid quality tax collection, management and reporting system of SRA
[Added: 2020/11/19]
PFM- ICT Specialist 2022/12 Not Initiated
Conduct routine and periodic monitoring to truck the project progress and store all the data in UNDP South Sudan Share point
[Added: 2020/11/19]
PFM Manager 2022/12 Not Initiated
4. Recommendation:

Given the quantum of social services delivered at the local government levels, counties and payams, there is need to start a development planning process to drive the budgeting process in the states from the payam level. All efforts must be mustered to elaborate integrated development plans at every level which would lead to the fiscal budgets at the state level.

Management Response: [Added: 2020/11/19]

The  project will build capacities at all levels – national and states – to enhance the preparation and execution of development plans and broadening the budget process. This is one of the key focus of the SDG Fund PFM projects were resources have been secured to support three states in revenue mobilization with a focus on expenditure on service delivery and state level development planning.

Key Actions:

Key Action Responsible DueDate Status Comments Documents
build capacity of State MOF staff in budget preparation and appropriation processes
[Added: 2020/11/19] [Last Updated: 2021/08/07]
PFM Manager 2021/06 Completed Trainings of State Ministry of Finance Officials on the PFM cycle took place during Q1 2021. Topics discussed included planning, budgeting, revenue, expenditure, internal control, procurement, external audit, among others ( Reference is made on page 15 of attached report) History
5. Recommendation:

Gender mainstreaming and women empowerment should be enhanced by appropriately designed training programmes to sharpen the competences of CSOs and women’s groups to represent and advocate for the interests of children and women as well as human rights in the budget process.

Management Response: [Added: 2020/11/19]

The PFM team fully accept  this recommendation and will work to strengthen gender mainstreaming and women empowerment in the project execution, especially on enhancing gender budgeting.

Key Actions:

Key Action Responsible DueDate Status Comments Documents
5.1 Mainstream gender and human rights in the budget process at the state level through gender budgeting.
[Added: 2020/11/19]
PFM Manager 2022/12 Not Initiated
Collect and report gender disaggregated data during UNDP and donor reporting
[Added: 2020/11/19]
PFM Manager 2022/12 Not Initiated

Latest Evaluations

Contact us

1 UN Plaza
DC1-20th Floor
New York, NY 10017
Tel. +1 646 781 4200
Fax. +1 646 781 4213
erc.support@undp.org