Terminal Evaluation Selected government institutions have the capacity to effectively and efficiently deliver services to all

Report Cover Image
Evaluation Plan:
2013-2016, Eritrea
Evaluation Type:
Outcome
Planned End Date:
09/2016
Completion Date:
08/2016
Status:
Completed
Management Response:
Yes
Evaluation Budget(US $):
30,000

Share

Document Type Language Size Status Downloads
Download document TERMS OF REFERENCE Capaity DevelopmnetVER1.docx tor English 36.99 KB Posted 253
Download document FINAL EVALUATION REPORT - OUTCOME 4.pdf report English 2344.64 KB Posted 364
Title Terminal Evaluation Selected government institutions have the capacity to effectively and efficiently deliver services to all
Atlas Project Number:
Evaluation Plan: 2013-2016, Eritrea
Evaluation Type: Outcome
Status: Completed
Completion Date: 08/2016
Planned End Date: 09/2016
Management Response: Yes
Focus Area:
  • 1. Others
Corporate Outcome and Output (UNDP Strategic Plan 2014-2017)
  • 1. Output 4.1. Country led measures accelerated to advance women's economic empowerment
  • 2. Output 4.3. Evidence-informed national strategies and partnerships to advance gender equality and women's empowerment
  • 3. Output 4.4. Measures in place to increase women's participation in decision-making
Evaluation Budget(US $): 30,000
Source of Funding: UNDP
Evaluation Expenditure(US $): 18,410
Joint Programme: No
Joint Evaluation: No
Evaluation Team members:
Name Title Email Nationality
Dr. Hamid R. Chauhry International Outcome Evaluation Consultant hamid_chaudhry@yahoo.ca
GEF Evaluation: No
Key Stakeholders:
Countries: ERITREA
Lessons
1. Main lessons learned • National institutions need the human and institutional capacity to cope with the systems and procedures of donor community, international organizations and to meet the global obligations. There are mainly human resources and institutional capacity constraints which seem to cut across most partner institutions, particularly governmental IPs. This situation imposes considerable pressure on institutions to do the maximum with what is available in terms of human, financial and other resources. Capacity problems, high turnover and low retention of workers have thus been common for implementing institutions. ? • Human capacity alone cannot achieve the objectives of sustainable economic development, efficient service delivery and sustainable development goals (SDGs). Institutional capacity needs to be strengthened simultaneously that encompasses institutional reforms and incentives, policies and procedures, internal structure, organizational transformation, accountability framework, mandates, vertical and horizontal hierarchy, coordination/networking, ToR and job description. • The procedures and regulations for disbursement of funds were cited by many IPs as the main constraints to effective implementation of programmes (e.g. NCHE), and adversely affected the performance. Also, procurement procedures both local and foreign also contributed to delays. ? • When dealing with various levels of government in programme and activity delivery, care must be taken to endeavour to engage with a broad range of executive decision makers, especially the apex management. This is critical for the success and sustainability of programme interventions. For example, involvement of MoND and sector ministries in the preparation of SPCF 2013 – 2016 promoted the integration between the participating ministries. • Well-defined set of quantitative baseline indicators and targets of outcomes and outputs are vital for measuring the periodical progress, monitoring and evaluation of programmes. Also, inputs, activities, outputs, outcomes and goals are interlinked through results chain. Government policies of allowing limited access to IPs have hampered joint, timely and inclusive planning and M&E. Similarly, government's restrictions on inclusive and independent evaluative and analytical surveys/studies, and timely release of reports continues to limit availability of reliable baseline data for results monitoring and reporting. • NUEW - Providing training on an integrated approach created great opportunities for the participants and sharing their experiences and discovered great potential for the campaign. Supporting poor students will be a path to their participation in economic development activities and income stability. Recommendations The following set of recommendations for outcome 4 evaluation is provided based on the analysis and the implementation progress outlined in this report. • UNDP, with its strong partnership with the Government, is in a unique position to galvanize development efforts in Eritrea. UNDP should leverage this strength to developing its programmes based on national development needs and on its own mandates. Many activities of outcome are either in progress or could not be started due to reasons mentioned in the previous sections. UNDP may continue support to complete the unfinished agenda. • Outcome 4 support was mainly focused on individual level capacity building through training while little was done to develop institutional capacity. Institutional capacity is a little broader area that embraces institutional reforms, review and adjust mandates, hierarchy, establish a system of horizontal and vertical coordination within the institutions, review and readjust ToR of staff, accountability, performance measurement system and incentives. UNDP may, therefore, endeavour to conduct a comprehensive institutional capacity assessment to identify the real needs for institutional capacity development of the national institutions without which the desired results would not be achieved.. • In the context of resources efficiency, generally UNDP’s main focus remains on the maximization of spending to achieve the financial delivery targets. This is not the only criteria for measuring performance of the programme. Emphasis should equally be placed on measuring programme performance and change, and on the results of programme interventions. • UNDP's indicators in its current programme are not necessarily the optimal indicators for the particular development interventions that UNDP is pursuing. Good quality results—that is, well formulated impacts, outcomes, outputs, activities and indicators of progress—are crucial for proper monitoring, evaluation and measuring results. If results are unclear and indicators are absent or poorly formulated, monitoring and evaluating progress will be challenging, making it difficult for staff and decision makers to know how well plans are progressing and when to take corrective actions. The next SPCF should be based on improved indicators and baseline data and in particular it needs to be strengthened and take into account local and international standards. Too often in the design of UNDP projects and programs throughout the world, there is a tendency to develop indicators as an afterthought to the narrative rationale, contents and change management theories of programme documents. Following the SMART criteria, UNDP may design appropriate and detailed indicators that are properly scaled, measurable and achievable during the next programme cycle. • Flexible and simpler rules are required to avoid delays and for day to day and routine activities and key agenda related to policy issues, budget revisions and other important issues remain to be the responsibility of MoND that needs to be strengthened. UNDP may provide technical assistance to MoND to develop a guidebook on a harmonized process of project management cycle to be followed by all national institutions. The guidebook should provide guidelines on the process, producers and requirements of project identification and preparation, annual work plans, budgets, approval and implementation. Each activity should be assigned a timeline. To follow these procedures and timelines should be made mandatory for the all national institution. The institutional capacity of MoND may be developed to meet these requirements. • UNDP could consider strengthening its results-based (RB) M&E capacities, and support the Government to do the same. This would help in supporting UNDP and government institutions to establish RB M&E systems that would be based on SMART indicators to assess results of development interventions and improvements in institutional capacity. Similarly, UNDP may continue to promote improvements in data collection and analysis, especially since the lack of disaggregated statistical data remains an obstacle for more sophisticated, targeted interventions. Efforts can build on the Social Card – Social Welfare Information System (SWIS), which generates data for analysis, reporting, and monitoring the impact of social policies, and strengthens evidence-based policy decision making.

Tag: Operational Efficiency Sustainability

Findings
1. Relevance: Relevance of the outcome 4 objectives and the strategy to the national and UN priorities was strong. Within the framework of SPCF 2013 – 2016The support provided under Outcome 4 is need-based and has targeted the weak areas of human and institutional capacity development of national institutions. This outcome provided support to GoSE to accelerate progress in achieving Millennium Development Goal (MDG) 3 and 8 on national capacity development and gender equality. Capacity of MoF, MoJ, EIT and MoMR is also being strengthened, and the protection and provision of essential services to refugee communities in Eritrea are also taken into account. Efficiency: Outcome 4 was highly focused on demand-driven and results-based approach to its interventions and covered a wide category of institutions. The activities undertaken and outputs achieved were efficient both in terms of financial and human resources utilization. Delivery rate remained nearly 100% throughout the implementation period of Outcome 4. The Project maintained robust financial management systems and strong accounting practices and the rule of law as key principles specified in the UNDP Automatically Tuned Linear Algebra Software (ATLAS) System and the UNDP Standard Operating Procedures. In certain cases transfer of funds was delayed that affected progress of projects. The national execution (NEX) modality adopted to implement outcome activities ensured maximum cost-effectiveness, flexible and tailored to the capacity development of the national institutions. Procurement of equipment and delivery was effective as implementation partners (IPs) were satisfied with the quality of equipment. NUEYS and National Confederation of Eritrean (NCEW) exceeded their targets in 2015. M&E system was not in place within IPs. Effectiveness: Overall effectiveness of the activities and programmes for outcome 4 shows mixed results. Some of the activities exceeded the targets (e.g. NUEYS trained 985 persons against a target of 900, Similarly, NCEW has completed majority of the tasks ahead of stipulated time.) while others either did not start or could not be completed due to various reasons, mainly the delayed approvals and transfer of funds (e.g. MoF), lack of availability/shortage of professional staff (e.g. NSO). Quantitative indicators show that the Outcome 4 projects’ support to IPs has significantly contributed to the capacity building, improved working environment and enhanced the self-confidence level of staff. Impact: At this stage it was too early to expect the real impact of outcome interventions on the national institutions, as the results appear after few years. However, through review of documents, extensive discussions with UNDP and government IPs’ staff, interviews with key informants and other UN agencies, the mission has nevertheless availed the opportunity to have an approximation of the degree of programme impact. The outcome projects have strengthened capacity of national institutions in priority areas enabling them to implement their annual work plans and mandates more efficiently. By constantly supporting GoSE in difficult times UNDP has emerged as a trusted partner of the government. An inclusive and joint approach adopted by UNDP in implementing Outcome 4 interventions has not only enhanced the credibility and image of UNDP among IPs but also of those IPs among the public and international communities. For instance, the conference and symposia organized by NCEW has not only elevated the image of Eritrea in international community but also opened the doors for ILO technical assistance. There was high praise by the IPs’ representatives for UNDP support. Extensive training programme undertaken by UNDP in collaboration with UNFPA and UNICEF and provision of equipment has made significant impact on the national institutions. The capacity of the staff has been strengthened in their respective areas of specializations, improved the working environment and raised the confidence of individual staff. Particularly, impact was reflected in the achievements of MoFA, NCEW, NUEW, NUEYS, MoJ and NSO. Also, capacity of staff was built in research techniques and data collection that will help institutions in M&E of their programmes. Sustainability: Commitment and ownership of the programmes initiated under outcome 4 are strong in the participating national institutions. However, financial sustainability of the interventions will be a challenge unless firm financial commitments are made by the donors and government. Sustainability factor was strong in NCEW, NUEW, NUEYS, MoJ and MoH. However, sustainability of the programme is a major concern for MoF as the training programme planned could not be implemented due to non-availability of budget. Rigorous training programme has built the capacity of the staff of national institutions to carry on their work more efficiently after the termination of UNDP support. Experience has been acquired to conduct national surveys and prepare country level reports. This contributes to sustainability of NSO and the line ministries capacity in producing and sharing data. Commitment of the NUEYS management shown to continue expanding the programme is good sign of sustainability. Rehabilitation of vocational schools has not only ensured the sustainability of the programme but also the expansion of programme activities following the termination of UNDP assistance. Inspired by UNDP interventions, NUEYS has planned to establish a revolving fund for micro-credit for small enterprises. Sustainability is not a likely problem in the case of digital library of the MoJ because it has the human and technical capacity to run the digital library and to digitalize its archives. It is less threatened by staff turnover due to incentive systems that have been introduced to retain its staff. Main focus of the outcome 4 has been on developing individual capacity and provision of equipment. Sustainability is also dependent to a large degree on the establishment of an institutional and organizational structure that is dynamic and conversant with modern technological changes.
Recommendations
1

UNDP, with its strong partnership with the Government, is in a unique position to galvanize development efforts in Eritrea. UNDP should leverage this strength to continue developing its programmes based on national development needs and on its own mandates.

2

UNDP, with its strong partnership with the Government, is in a unique position to galvanize development efforts in Eritrea. UNDP should leverage this strength to continue developing its programmes based on national development needs and on its own mandates.

 

3

Outcome 4 support was mainly focused on building capacity at the individual level through training and little was done to develop institutional capacity. Developing individual capacity alone cannot improve the efficiency and service delivery by the national institutions. Institutional capacity little is a broader area that encompasses institutional reforms, review and adjusts mandates and hierarchy, establish a system of horizontal and vertical coordination with a ministry/institution, review and readjust ToR of staff, performance measurement system and incentive system. UNDP may, therefore, endeavour to conduct a comprehensive institutional capacity assessment study to identify the real needs for institutional capacity development of the national institutions.

4

In the context of financial resources efficiency, generally UNDP’s main focus remains on the maximization of spending to achieve the targets of delivery. This is not the only criteria for measuring performance of the programme. More emphasis should be placed on measuring programme performance and change, and results of interventions.

5
6

Delays in approval and transfer of funds were cited as major constraint to AWP implementation and for not achieving the targets. Flexible and simpler rules are required for day to day and routine activities and key agenda related to policy issues, budget revisions and other important issues remain to be the responsibility of MoND that needs to be strengthened through capacity building. UNDP may provide technical assistance to MoND to develop a guidebook on a harmonized procedure of project management cycle to be followed by all national institution. The guidebook should cover guidelines on the process and producers of project identification and preparation, annual work plans, budgets, approval and implementation.  Each activity should be assigned a timeline. To follow these procedures and timelines should be made mandatory for the all national institution. The institutional capacity of MoND may be developed to meet these requirements.

 

7

UNDP’s support to digitalization and e-services was hugely successful and can be a model for the entire country programme.  Increasingly, UNDP should strive to exploit its comparative strengths and become a knowledge-based organization. 

8

UNDP could consider strengthening its results-based (RB) M&E capacities, and support the Government to do the same. This would help in supporting UNDP and government institutions to establish RB M&E systems that would be based on SMART indicators to assess results of development interventions and improvements in institutional capacity. Similarly, UNDP could continue to promote improvements in data collection and analysis, especially since the lack of disaggregated statistical data remains an obstacle for more sophisticated, targeted interventions. Efforts can build on the Social Card – Social Welfare Information System (SWIS), which generates data for analysis, reporting, and monitoring the impact of social policies, and strengthens evidence-based policy decision making.

9

Refocusing the efforts of its governance program with a greater orientation to decentralization at all levels (see above notes on streamlining local governance) and addressing the inter – ministerial and intergovernmental overlap and duplication of authority and roadblocks to the effective delivery of services to citizens at the local level taking a socio-– legal perspective. UNDP must do a better job of emphasizing the logic of interdependence of policy, institutions and finance and continue to support synergies.

1. Recommendation:

UNDP, with its strong partnership with the Government, is in a unique position to galvanize development efforts in Eritrea. UNDP should leverage this strength to continue developing its programmes based on national development needs and on its own mandates.

Management Response: [Added: 2016/11/16] [Last Updated: 2017/12/14]

Fully agree.

Key Actions:

Key Action Responsible DueDate Status Comments Documents
Efforts will be made to continue developing UNDP programmes based on national development priorities and UNDP mandates.
[Added: 2017/12/14] [Last Updated: 2018/08/20]
Senior management and all programme managers 2017/01 Completed History
2. Recommendation:

UNDP, with its strong partnership with the Government, is in a unique position to galvanize development efforts in Eritrea. UNDP should leverage this strength to continue developing its programmes based on national development needs and on its own mandates.

 

Management Response: [Added: 2016/11/21]

We fully agree with the recommendation

Key Actions:

Key Action Responsible DueDate Status Comments Documents
UNDP will continue to enhance its strong partnership with the Government
[Added: 2016/11/21] [Last Updated: 2018/08/22]
UNDP senior management, programme heads and project managers. 2018/01 Completed Ongoing: UNDP's engagement and partnership with the government will continue for the next programme cycles. History
3. Recommendation:

Outcome 4 support was mainly focused on building capacity at the individual level through training and little was done to develop institutional capacity. Developing individual capacity alone cannot improve the efficiency and service delivery by the national institutions. Institutional capacity little is a broader area that encompasses institutional reforms, review and adjusts mandates and hierarchy, establish a system of horizontal and vertical coordination with a ministry/institution, review and readjust ToR of staff, performance measurement system and incentive system. UNDP may, therefore, endeavour to conduct a comprehensive institutional capacity assessment study to identify the real needs for institutional capacity development of the national institutions.

Management Response: [Added: 2016/11/21]

We fully agree with the recommendation

Key Actions:

Key Action Responsible DueDate Status Comments Documents
UNDP will discuss with the Government to conduct institutional capacity assessment in the country.
[Added: 2016/11/21] [Last Updated: 2018/08/22]
Country Office senior management and programme heads 2018/01 Completed Ongoing: The Country Office senior management and programme heads will make more efforts to convince the government about the importance of conducting institutional capacity assessment in the national development process. History
4. Recommendation:

In the context of financial resources efficiency, generally UNDP’s main focus remains on the maximization of spending to achieve the targets of delivery. This is not the only criteria for measuring performance of the programme. More emphasis should be placed on measuring programme performance and change, and results of interventions.

Management Response: [Added: 2016/11/21]

We totally disagree with the recommendation

Key Actions:

Key Action Responsible DueDate Status Comments Documents
UNDP measures programme performance and its impacts on the socio-economic and livelihood of the beneficiaries and their environment.
[Added: 2016/11/21]
UNDP senior management, programme heads, implementing partners and the beneficiaries. 2016/08 Completed UNDP programme staff monitors projects implementation and carries out evaluations to measure results of implemented projects.
5. Recommendation:
Management Response: [Added: 2016/11/22]

We partially agree with agreement. The quality, availability and access to adequate and accurate data in Eritrea continues to be a major challenge. This makes it very difficult to establish beelines and measurable indicators. However, after 10 years, for the first time Eritrea produced an MDG Report in 2015 with support from UNDP. The MDGs Report makes available data to almost all Government Ministries and Departments for their planning and as a reference in the preparation of the National Development Plan. In addition, with UNDP support a Study Report was produced that reviewed and analysed capacity gaps in the management information systems on meeting the evolving needs of the departments in Ministry of Finance and the government in general.  UNDP supported the National Union of Eritrean Youth and Students  in conducting a needs assessment and market analysis study and in establishiment of baseline on employment/labour in order to tailor skills trainingprogrammes and enhanecement of the vocational skill training centers to appropriate needs and job markets. Similarly, with UNDP support the government is undertaking a labour market survey for the fisrt time in many years.

 

Key Actions:

Key Action Responsible DueDate Status Comments Documents
UNDP will keep on supporting the implementing partners to conduct more baseline surveys.
[Added: 2016/11/22] [Last Updated: 2018/08/20]
Senior management and programme heads. 2018/01 Completed UNDP will keep on discussing with the government for the establishment of data and allow access to development partners. History
6. Recommendation:

Delays in approval and transfer of funds were cited as major constraint to AWP implementation and for not achieving the targets. Flexible and simpler rules are required for day to day and routine activities and key agenda related to policy issues, budget revisions and other important issues remain to be the responsibility of MoND that needs to be strengthened through capacity building. UNDP may provide technical assistance to MoND to develop a guidebook on a harmonized procedure of project management cycle to be followed by all national institution. The guidebook should cover guidelines on the process and producers of project identification and preparation, annual work plans, budgets, approval and implementation.  Each activity should be assigned a timeline. To follow these procedures and timelines should be made mandatory for the all national institution. The institutional capacity of MoND may be developed to meet these requirements.

 

Management Response: [Added: 2016/11/22]

We don’t agree with the recommendation. UNDP in collaboration with other UN Agencies organized  a - two day workshop to all Implementation Partners, including the Ministry of National Development staff, on the preparation of AWP, financial reports, project document, and result oriented progress reports. UNDP also has prepared Local NIM Manual which will be used as guideline by the Ministry of National Development and Implementing Partners to implement UNDP supported projects.

Key Actions:

Key Action Responsible DueDate Status Comments Documents
UNDP will continue organizing trainings and workshops to improve capacity of the Implementing Partners and Ministry of National Development.
[Added: 2016/11/22] [Last Updated: 2018/08/20]
UNDP Programme heads. Ministry of National Development and Implementing partners. 2018/12 Completed Training manuals on HACT, financial management and procurement have been prepared to be used as toolkits in implementing UNDP funded projects. History
7. Recommendation:

UNDP’s support to digitalization and e-services was hugely successful and can be a model for the entire country programme.  Increasingly, UNDP should strive to exploit its comparative strengths and become a knowledge-based organization. 

Management Response: [Added: 2016/11/22]

We fully agree with the recommendation

Key Actions:

Key Action Responsible DueDate Status Comments Documents
UNDP Eritrea will continue implementing the UNDP Knowledge Based Strategy by documenting best practices and lessons learned and encourage staff to participate in community and knowledge management platforms.
[Added: 2016/11/22] [Last Updated: 2018/08/22]
Unit heads and project officers. 2018/12 Completed Ongoing: The Country Office has already implemented the UNDP Knowledge Based Strategy by documenting best practices and lessons learned and will continue doing so. History
8. Recommendation:

UNDP could consider strengthening its results-based (RB) M&E capacities, and support the Government to do the same. This would help in supporting UNDP and government institutions to establish RB M&E systems that would be based on SMART indicators to assess results of development interventions and improvements in institutional capacity. Similarly, UNDP could continue to promote improvements in data collection and analysis, especially since the lack of disaggregated statistical data remains an obstacle for more sophisticated, targeted interventions. Efforts can build on the Social Card – Social Welfare Information System (SWIS), which generates data for analysis, reporting, and monitoring the impact of social policies, and strengthens evidence-based policy decision making.

Management Response: [Added: 2016/11/22] [Last Updated: 2017/12/14]

We do not agree with the recommendation.  To strengthen its M & E capacity UNDP has established a Program Support Unit and organized various training on M & E to its staff. UNDP is also supporting the Governmnte in the generation and dissemination of data including various evaluative and analytical surveys and studies, such as the Eritrean Population and Health Survey (EPHS); the utilization of sex- disaggregated data and the introduction of a comprehensive civil and vital registration system. In addition, assistance is being provided for the establishment of a socio-economic indicator database and a comprehensive and gender responsive national data management system. UN also will continue promoting gender equality by supporting the government to produce baseline data on gender, support gender mainstreaming, budgeting and advocacy. Strengthen human and institutional capacities at sector level to plan, coordinate, implement, monitor and evaluate national development programmes.

 

   

 

Key Actions:

Key Action Responsible DueDate Status Comments Documents
No action is required.
[Added: 2017/12/14]
Not applicable 2017/01 No Longer Applicable [Justification: The recommendation is not acceptable.]
9. Recommendation:

Refocusing the efforts of its governance program with a greater orientation to decentralization at all levels (see above notes on streamlining local governance) and addressing the inter – ministerial and intergovernmental overlap and duplication of authority and roadblocks to the effective delivery of services to citizens at the local level taking a socio-– legal perspective. UNDP must do a better job of emphasizing the logic of interdependence of policy, institutions and finance and continue to support synergies.

Management Response: [Added: 2016/11/22]

We fully agree with the recommendation: However, UNDP is supporting the Government for effective and efficient service delivery by designing projects/programs and advocating for and support the development of a strategy and institutional framework to facilitate policy coherence and information sharing. However, it is not priority of the government.

 

Key Actions:

Key Action Responsible DueDate Status Comments Documents
No action is required because it is not the priority of the government.
[Added: 2017/12/14]
Not applicable. 2017/01 No Longer Applicable [Justification: It is not government's priority. ]

Latest Evaluations

Contact us

1 UN Plaza
DC1-20th Floor
New York, NY 10017
Tel. +1 646 781 4200
Fax. +1 646 781 4213
erc.support@undp.org