Mid-term evaluation of Korea Facility for Poverty Alleviation

Report Cover Image

The RoK-UNOSSC Facility Phase 2 has three components, the Consortium (6 Institutions), the Platform  (3 institutions) and the Scaled-Up Project with one institution. It is funded by the Ministry of Science  and ICT (MSIT) of the Republic of Korea (RoK) and implemented under the Direct Implementation (DIM) arrangement by the United Nations Office for South-South Cooperation. While two countries were selected for piloting the integrated project led by the Consortium (Cambodia and Indonesia); the scaled- up project under the Regional Cooperative Agreement Regional Office (RCARO) has 14 participating countries in the Asia-Pacific Region. The Facility budget of $4 million is made through annual contributions to UNOSSC by the government of the Republic of Korea. The Ministry of Science and ICT requested  the Steering Committee of the Facility to appoint one of the implementing institutions (STEPI) to represent the Ministry in the coordination of the participating institutions in RoK. It is a triangular cooperation capacity building Programme. 
A mid-term evaluation for the RoK-UNOSSC Facility Phase 2 (2016 – 2020) was planned and budgeted  for in compliance with the UNDP evaluation policy, relating to the scale of the project ($4 million),  and the provision in the monitoring and evaluation framework included in the Phase 2 Facility Document. The mid-term evaluation is intended to provide a comprehensive assessment of the project progress at its mid-way point to strengthen implementation for the remaining duration of the project.  
The methodology for the evaluation covered the following areas: • Desk study review of all relevant project documentation; • Consultations with Programme Management in UNOSSC, the Science and Technology Policy Institute (STEPI) as a coordinating institution in the Republic of Korea (RoK), all ten (10) participating RoK institutions, and  government counterparts in Cambodia and Indonesia. 
 
Overall, this evaluation captures the successes, challenges, and lessons learnt from implementation  of the project from 2017 and 2018. The evaluation ends with a list of recommendations generated from the analysis. 

Share

Document Type Language Size Status Downloads
Download document Approved TOR_Korean Facility_Mid-term evaluation.pdf tor English 89.21 KB Posted 111
Download document Cleaned_final Mid-term Evaluation Report of the RoK-UNOSSC Facility Phase 2.pdf report English 465.25 KB Posted 136
Title Mid-term evaluation of Korea Facility for Poverty Alleviation
Atlas Project Number:
Evaluation Plan: 2018-2021, UN Office for South-South Cooperation
Evaluation Type: Mid Term Project
Status: Completed
Completion Date: 01/2019
Planned End Date: 01/2019
Management Response: Yes
Corporate Outcome and Output (UNDP Strategic Plan 2014-2017)
Evaluation Budget(US $): 14,300
Source of Funding: Project budget
Evaluation Expenditure(US $): 11,200
Joint Programme: No
Joint Evaluation: No
Evaluation Team members:
Name Title Email Nationality
Ms. Michaelle Tauson Evaluation Consultant mmtauson@gmail.com
GEF Evaluation: No
Key Stakeholders: Government of the Republic of Korea, the Ministry of Science and ICT (MSIT)
Countries: GLOBAL
Lessons
Findings
1.

The RoK-UNOSCC programme, or the Facility Phase 2, can be seen as a unique project, as it was designed in recognition of the complex, multi-faceted issues that countries face when attempting to achieve development goals. As such the programme aims to deliver one collective goal through leveraging the comparative advantage of multiple specialised RoK agencies through triangular cooperation. In addition, the main purpose of triangular cooperation, according to the Buenos Aires Plan of Action for Promoting and Implementing Technical Cooperation among Developing Countries (TCDC), is to foster self-reliance amongst developing countries through the exchange and/or transfer of knowledge, resources, and capacity. Therefore, the RoK-UNOSSC programme aimsto achieve sustainable and scalable results through the passage of knowledge and insight. It is ultimately a capacity building initiative.


Tag: Monitoring and Evaluation Trade and Development Capacity Building Institutional Strengthening South-South Cooperation

2.

Benefits of the Programme and Working as One

This project is important for the government of Indonesia and a true model of how triangular cooperation can be done smoothly with many stakeholders.

- Government Representative from Indonesia

Analysis from project documents and interview transcripts demonstrates an overall positive view of the programme. Empirical data have shown that projects are moving in the right direction. Project documents and reports and the responses from interviewees show on track outputs and positive outcomes. All reports showed that deliverables were, more or less, on time and delivered at the budgeted cost. In addition, a number of successes and benefits were relayed in interviews and project reports. For example, representatives from Korean institutions reported that they felt that much was gained through knowledge and capacity transfer. According to some participants, this model of triangular cooperation and South-South cooperation is useful because countries have a lot to share with one another, due to their direct experience of development challenges. In this sense, representatives felt that knowledge transfer could be done easily and with insight.


Tag: Agriculture Knowledge management Partnership Technology Trade and Development Capacity Building Institutional Strengthening South-South Cooperation Women and gilrs

3.

Communication and Collaboration

As mentioned previously, the RoK-UNOSSC programme is highly complex as it aims to achieve high level change across developing countries and is attempting to tackle development issues from a holistic perspective. As such efficient and effective communication and coordination are paramount for the success of the programme. At the same time, communication and collaboration, in this context of triangular cooperation are some of the most complex aspects of the programme as reported by interview respondents and as addressed in mid-term reports. This complexity was recognised from the outset of implementation by UNOSSC and STEPI. As a result, a coordinating institution was selected by peers in the Platform Group and Consortium respectively. The role of the Coordinator was to carry out effective communication with UNOSSC and STEPI in the case of the Consortium, in which Canaan was selected. STEPI assumed this role for the Platform Group. Furthermore, STEPI was appointed by the Government of the Republic of Korea as the local coordinator, representing the Ministry of Science and ICT (MSIT).


Tag: Trade and Development Capacity Building Coordination South-South Cooperation

4.

Communication and Collaboration

As mentioned previously, the RoK-UNOSSC programme is highly complex as it aims to achieve high level change across developing countries and is attempting to tackle development issues from a holistic perspective. As such efficient and effective communication and coordination are paramount for the success of the programme. At the same time, communication and collaboration, in this context of triangular cooperation are some of the most complex aspects of the programme as reported by interview respondents and as addressed in mid-term reports. This complexity was recognised from the outset of implementation by UNOSSC and STEPI. As a result, a coordinating institution was selected by peers in the Platform Group and Consortium respectively. The role of the Coordinator was to carry out effective communication with UNOSSC and STEPI in the case of the Consortium, in which Canaan was selected. STEPI assumed this role for the Platform Group. Furthermore, STEPI was appointed by the Government of the Republic of Korea as the local coordinator, representing the Ministry of Science and ICT (MSIT).


Tag: Communication Trade and Development Capacity Building Coordination South-South Cooperation

5.

Communication

Overall, there are some excellent examples of communication taking place in the programme. For example, consortium members all reported that they have strong working relationships with other consortium members and that they feel they can engage with other institutions across the consortium fairly easily; a strong rapport has been built among both institutions and individuals which has enabled this type of an environment. Due to this, they use one another to obtain needed information, establish relationships on the ground in Cambodia and Indonesia, and even troubleshoot from time to time. At the same time, some issues in communication were highlighted and all respondents suggested that improved communication would lead to improved project outcomes exponentially. One example expressed during interviews was that of communicating planned activities in advance. For instance, when institutions are planning a workshop in Korea, they should give advance notice to the other institutions, so they can work together, perhaps provide a session if needed, and not overlap in activities. Another example given was that of field visits to or workshops in Cambodia and Indonesia. Sometimes visits are 11 scheduled at the same time or directly after another, thus overwhelming project participants such as government officials, community members, and school officials. Interviewees mentioned that it was the lack of advance notice that caused these issues.


Tag: Effectiveness Efficiency Communication Knowledge management Programme/Project Design Education Institutional Strengthening

6.

Collaboration

Issues in collaboration were discussed frequently during interviews. Although, according to one respondent, few, if any, of the constraints were mentioned by coordinators during the Steering Committee Meeting in which all Facility Component Coordinators were in attendance and all had an opportunity to review issues with their respective groups before the Meeting. During interviews, respondents expressed that, although they had a great rapport and a friendly working relationship, they still felt that they were working independently with only minimal overlap and collaboration.

 All the consortium team, we all have good relationships, but in terms of implementation, it feels really independent rather than a consortium. We have a great relationship, but that does not mean anything. We have to have a physical relationship. - Respondents from Korean Institutes.

Many interview participants pointed to the initial programme implementation process as the reason why sound, collaborative relationships were not built, especially in regard to the consortium. According to participants, institutions initially applied to be part of Phase 2 and put together proposals independently. After their submitted proposals, institutions were asked to find partners, figure out how they could work together and to submit a joint proposal. The joint proposals were then appraised, and one consortium was selected. However, the Steering Committee wanted energy and entrepreneurship which were not included in the selected consortium. Thus, two members from another consortium were selected to join the selected members. Because this was in the initial stages of the programme, some institutions expressed that they were not able to clearly see ways in which to overlap and synergise until projects were implemented and ways of working together became clearer.


Tag: Effectiveness Sustainability National Human and Financial resources Implementation Modality Knowledge management Programme Synergy Advocacy Coordination Institutional Strengthening

7.

Project Implementation: Timelines, Delays, and Risk Assessment

As this project has many stakeholders, objectives, and target countries, which all include a number of underlying, dependent factors, the programme is highly susceptible to delays. Firstly, many respondents noted that the programme has been slow to be implemented, and at the country level programmes have to be designed and refined, which can take some negotiation and compromise with governments. The Facility Document was signed in June 2016, and implementation earnestly commenced in 2017. However, because the planning portion of the programme was comparatively short (for those institutions co-opted into the Consortium), contingencies were not worked into the timelines. Much more time is required to consider potential risks during the planning process and the existing risk framework needs to be reviewed and updated during the annual planning. Some of the delays which occurred, may have been mitigated with better awareness and planning. In Indonesia for example, Ramadan was not considered when planning and some institutes reported getting behind during this month and having to push back activities. In Cambodia, institutes were not aware of the severity of the rainy season and the extent to which floods can impact their projects. These are some considerations when making the annual and project plans and should be considered as a part of a larger risk assessment.


Tag: Implementation Modality Programme/Project Design Project and Programme management Risk Management

8.

Project Implementation: Disbursement

After communication and collaboration, the second most discussed issue amongst interview respondents was funding disbursement procedures and consequent delays. Under Phase 2, UNOSSC switched from the United Nations Office for Project Services (UNOPS) disbursement system to the Harmonised Approach to Cash Transfers (HACT). UNOSSC staff as well as the implementing institutions had to learn the requirements of the new system. This resulted in delays during the initial disbursements, which led to some project level delays.


Tag: Human and Financial resources Implementation Modality Project and Programme management Cash Transfers South-South Cooperation

9.

Monitoring and Evaluation

A well-defined monitoring and evaluation framework is extremely useful for development programming. Monitoring is used to ensure that programme delivery is happening as expected and as planned; whereas evaluation is often used to measure effectiveness, efficiency, and sustainability, based on the achievement of programme delivery. As such, they go hand in hand. Monitoring and evaluation frameworks are almost always established at the beginning of a project to delineate tasks; set up a timeline of deliverables; and provide specific indicators in which to capture progress, outputs, and outcomes. Overall, they guide programmes and show whether or not they are on track and producing the expected results. One notable finding is that the overarching RoK-UNOSSC project does not have a logistical framework (log frame) for monitoring and evaluation. This is due to the experimental nature of the project and its need for an iterative and responsive process. This is also due to the extremely different components and goals of the project. However, it has been noted that (if the project is to move into a third phase) such a framework would be useful to delineate tasks and ensure the project is delivering as intended.


Tag: Effectiveness Efficiency Sustainability Monitoring and Evaluation Trade and Development Coordination South-South Cooperation

Recommendations
1

Planning and Coordination

  • Immediate actions aimed at stronger coordination should be initiated by UNOSSC and the Steering Committee.
  • The Steering Committee should revisit allocation of coordination responsibilities between UNOSSC and STEPI.
  • RoK institutions are expected to align programmes with government requirements and to be flexible.
  • Institutions should spend time together in the form of a workshop getting to know the expertise of each institution, in particular, organised communication workshops in the RoK would be helpful for institutions to understand each other and also develop an understanding of the technical capacity of the other institutions.
  • Joint planning as called for by UNOSSC and STEPI should be done under the guidance of the Facility component coordinators and the coordinators must ensure that there is a clearly communicated delineation of duties across the programme.
  • The participating institutions should prioritize coordination meetings over individual plans as more difficult to get all institutions together in one place.
  • Proactive coordination is necessary to ensure effective communication across all 10 Korean institutions, the UNOSSC and Indonesia and Cambodia counter-parts towards achievements of Facility outputs.
2

Results Assessment and Administration

Regarding monitoring and evaluation, UNOSSC should provide alternative ways of monitoring South-South and triangular cooperation projects and provide training on the basic principles of monitoring and evaluation – how to capture change in development

A greater effort has to be made to achieve understanding; translators should be identified including students, volunteers, interns, and online volunteers.

3

Ongoing Project Support and Considerations:

UNOSSC and STEPI should ascertain, through surveys or consultations, what project management training is required for more efficient project delivery, UNOSSC and STEPI would then conduct such training.

Project planning assistance should be required by institutions as needed, specifically those who may not have experience working in development previously.

More information should be provided on the UNOSSC portal to ensure communication, for instance, o all scheduled tasks from each organisation are on an online/shareable calendar, for example, a monthly/biweekly communique is sent to update institutions on ongoing activities and update on any changes. o manage/oversee a forum of communication, where institutions can troubleshoot and express concerns.

1. Recommendation:

Planning and Coordination

  • Immediate actions aimed at stronger coordination should be initiated by UNOSSC and the Steering Committee.
  • The Steering Committee should revisit allocation of coordination responsibilities between UNOSSC and STEPI.
  • RoK institutions are expected to align programmes with government requirements and to be flexible.
  • Institutions should spend time together in the form of a workshop getting to know the expertise of each institution, in particular, organised communication workshops in the RoK would be helpful for institutions to understand each other and also develop an understanding of the technical capacity of the other institutions.
  • Joint planning as called for by UNOSSC and STEPI should be done under the guidance of the Facility component coordinators and the coordinators must ensure that there is a clearly communicated delineation of duties across the programme.
  • The participating institutions should prioritize coordination meetings over individual plans as more difficult to get all institutions together in one place.
  • Proactive coordination is necessary to ensure effective communication across all 10 Korean institutions, the UNOSSC and Indonesia and Cambodia counter-parts towards achievements of Facility outputs.
Management Response: [Added: 2020/03/05] [Last Updated: 2021/01/31]

UNOSSC management accepts this recommendation and has taken more proactive measures to strengthen coordination among Korean institutions and partner countries. UNOSSC management has led and facilitated coordination especially for six Korean institutions under the Consortium component of the Facility. UNOSSC management ensures that communications with partner countries are channeled through the focal government agency for the Facility in each country.

Key Actions:

Key Action Responsible DueDate Status Comments Documents
1.1 Proactive engagement with all stakeholders to consult and prepare for 2019 and 2020 work plan activities aligned with national development priorities in partner countries. a. Post-midterm evaluation report workshop (28 Jan – 1 Feb 2019) b. Workshop on Formulation the 2019 Workplan for the Indonesian Village Innovation Development Programme (21- 23 Apr 2019) c. Consultative Workshop on 2020 Workplan Formulation in Cambodia (24 December 2019)
[Added: 2020/03/05]
UNOSSC ROAP 2020/12 Overdue-Initiated
1.2. Strengthened communications among Korean institutions, Facility Management and partner countries a. Leveraged ICT applications to enhance regular communications among all stakeholders especially among working groups (activity specific task holders): Facility Components (Platform, Consortium, Scaled-up) b. Improved communications in partner countries by channeling information through focal agency (e.g. Ministry of Planning (Bappenas) coordinates all Korean Facility work in Indonesia. Encouraged all institutions to report and liaise with Bappenas apart from local partner institutions
[Added: 2020/03/05]
UNOSSC ROAP 2020/12 Overdue-Initiated
1.3. Provide advisory services to partner countries and Korean institutions to identify strategic actions to be taken throughout the year aligned with Facility objectives and country demands. Especially when a new team took over the liaison function of the Science and Technology Policy Institute (STEPI) in June, as Facility Manager, UNOSSC staff went on a mission together with STEPI to partner countries to learn about project activities in Cambodia and Indonesia. The UNOSSC management team will continue to coordinate project activities. a. GS-NSTC-STEPI Meeting (18 – 19 July) b. STEPI-Bappenas-MoV Technical Meeting (29 – 30 July)
[Added: 2020/03/05]
UNOSSC ROAP 2019/12 Completed
1.4. Based on joint annual work planning and formulation for Indonesia, Facility Management has been supporting Korean institutions to deliver requirements made by partner countries in a coordinated manner. a. Consultation on Indonesia’s Village Index Harmonization (25 – 26 Sep) b. Joint review workshop on “Village Empowerment Project to Promote Sustainable Community Development through Innovative Integration of Science & Technology” (27 Sep) c. Joint Monitoring Mission and Consultative Workshop on 2020 Workplan Formulation in Cambodia (23-24 December) d. Facilitate arrangement of upcoming joint events in 2020 **Note: Steering Committee recommended all institutions to complete project activities by September 2020.
[Added: 2020/03/05] [Last Updated: 2020/11/09]
UNOSSC ROAP 2020/09 Completed Completed History
2. Recommendation:

Results Assessment and Administration

Regarding monitoring and evaluation, UNOSSC should provide alternative ways of monitoring South-South and triangular cooperation projects and provide training on the basic principles of monitoring and evaluation – how to capture change in development

A greater effort has to be made to achieve understanding; translators should be identified including students, volunteers, interns, and online volunteers.

Management Response: [Added: 2020/03/05] [Last Updated: 2021/01/31]

UNOSSC management accepts this recommendation. Project monitoring and evaluation should be strengthened with a log framework tailored to the project but often not all SS and TrC activities can be quantified. This year, UNOSSC management in cooperation with one Korean institution under the Platform will assess achievements of the Facility and stakeholders’ understanding on SS & TrC through a survey questionnaire addressed to implementing institutions and partner governments before the final project evaluation. It may be difficult to organize a separate training for implementing institutions as their other project priorities before its closure in December 2020. But finding from the assessment will be useful for final evaluation of Phase 2 and in designing Phase 3.

Key Actions:

Key Action Responsible DueDate Status Comments Documents
Design of the next phase of the Korean Facility, according to UNDP guidelines, and result-based framework taking into account relevant recommendations from the midterm and final evaluation
[Added: 2020/03/05] [Last Updated: 2020/06/10]
UNOSSC ROAP 2020/12 Overdue-Initiated History
3. Recommendation:

Ongoing Project Support and Considerations:

UNOSSC and STEPI should ascertain, through surveys or consultations, what project management training is required for more efficient project delivery, UNOSSC and STEPI would then conduct such training.

Project planning assistance should be required by institutions as needed, specifically those who may not have experience working in development previously.

More information should be provided on the UNOSSC portal to ensure communication, for instance, o all scheduled tasks from each organisation are on an online/shareable calendar, for example, a monthly/biweekly communique is sent to update institutions on ongoing activities and update on any changes. o manage/oversee a forum of communication, where institutions can troubleshoot and express concerns.

Management Response: [Added: 2020/03/05] [Last Updated: 2021/01/31]

UNOSSC management accepts this recommendation and has been providing support in preparing annual workplan of each institutions as required.

Key Actions:

Key Action Responsible DueDate Status Comments Documents
Provided guidance to Korean institutions in preparing annual workplans, if necessary.
[Added: 2020/03/05] [Last Updated: 2020/06/13]
UNOSSC ROAP 2020/02 Completed Completed History

Latest Evaluations

Contact us

1 UN Plaza
DC1-20th Floor
New York, NY 10017
Tel. +1 646 781 4200
Fax. +1 646 781 4213
erc.support@undp.org