- Evaluation Plan:
- 2011-2019, DPR Korea
- Evaluation Type:
- Final Project
- Planned End Date:
- 10/2019
- Completion Date:
- 10/2019
- Status:
- Completed
- Management Response:
- Yes
- Evaluation Budget(US $):
- 20,000
End of Project evaluation: Pilot Project to Support Socio-Economic Development of Rural Areas in the DPRK (SED)
| Document | Type | Language | Size | Status | Downloads |
|---|---|---|---|---|---|
 ToR_SED_23May2019.pdf |
tor | English | 8844.44 KB | Posted | 107 |
| DPRK_SED_TerminalEvaluation_Report_Oct2019.pdf |
report | English | 2008.72 KB | Posted | 148 |
| DPRK_SED_TerminalEvaluation_Summary_Oct2019.pdf |
summary | English | 301.58 KB | Posted | 54 |
| Title | End of Project evaluation: Pilot Project to Support Socio-Economic Development of Rural Areas in the DPRK (SED) | ||||||||
|---|---|---|---|---|---|---|---|---|---|
| Atlas Project Number: | 00074805 | ||||||||
| Evaluation Plan: | 2011-2019, DPR Korea | ||||||||
| Evaluation Type: | Final Project | ||||||||
| Status: | Completed | ||||||||
| Completion Date: | 10/2019 | ||||||||
| Planned End Date: | 10/2019 | ||||||||
| Management Response: | Yes | ||||||||
| Corporate Outcome and Output (UNDP Strategic Plan 2018-2021) |
|
||||||||
| Evaluation Budget(US $): | 20,000 | ||||||||
| Source of Funding: | TRAC | ||||||||
| Evaluation Expenditure(US $): | 25,400 | ||||||||
| Joint Programme: | No | ||||||||
| Joint Evaluation: | No | ||||||||
| Evaluation Team members: |
|
||||||||
| GEF Evaluation: | No | ||||||||
| Key Stakeholders: | |||||||||
| Countries: | DPRK -DEMOCRATIC PEOPLE'S REPUBLIC OF KOREA |
| Lessons | |
|---|---|
| 1. | Conclusion #1: Significant External Factors/Challenges Severely Affected the Project
Significant external factors/challenges beyond the control of the UNDP DPRK CO were encountered throughout the entire SED project implementation, and severely affected the timely delivery of project outputs and eventual achievement of results.
In particular, the evaluation highlights below the 3 external factors/challenges as the main constraints.
The SED Project Team has done their best to implement the project despite encountering the significant external factors/challenges beyond the control of the UNDP DPRK CO throughout the entire SED Project. However, improvements could still be further strengthened in the following areas:
|
| Findings | |
| 1. | The SED PRODOC indicated that the earliest commencement of the SED Project formulation was a joint UNDP/UNIDO fact-finding mission (led by UNIDO) in May 2011. As one of the “firstgeneration” UNDP DPRK projects since the re-opening of the UNDP DPRK CO in 2009, the UNDP DPRK identified a UNIDO expert to develop the SED Project according to the needs and priorities and constraints at the local province/county level in DPRK. Tag: Ownership Programme/Project Design Project and Programme management Country Government |
| 2. | The project took extensive consideration to stakeholder participation in project design, decision making, planning, implementation and monitoring. For example, the National Counterparts (MoLI, CSoST, CBS) and Local Counterparts (CPCs in Unryul, Unchon and Hoechang Counties) were invited to contribute to designing of project interventions and technical discussions on the output activities. This translated to an increase in confidence and ownership of project activities in the SED Project implementation. Tag: Local Governance Ownership Programme/Project Design Results-Based Management |
| 3. | The SED PRODOC had appropriate risk assessments with impact and probability ratings, and prepared corresponding counter-measures/management responses which were appropriate at that point of time and for the project duration (2013 to 2016). The SED Project identified a total of 7 risks: • 1 security/political risk • 3 operational risks • 1 environmental risk • 2 technical risks Tag: Results-Based Management Risk Management |
| 4. | 3.1.4 Lessons from Other Relevant Projects Incorporated into Project Design The evaluation did not find substantive evidence of lessons from other relevant projects being considered and taken directly to support the SED Project design.
• ‘Strengthening the Resilience of Communities through Community-based Disaster Risk Management’ Project (CBDRM Project) • ‘Sustainable Energy Solutions for Rural Livelihoods’ Project (SES Project) One example of this alignment was the SED Project Team learned key lessons from the SES Project to develop innovative energy solutions such as Solar PV Panels to provide sustainable and reliable energy supply to the Spirulina and Pistia Centres in Unryul and Unchon Counties (South Hwanghae Province). Tag: Renewable energy Knowledge management |
| 5. | 3.1.5 Planned Stakeholder Participation The SED Project generated strong stakeholder interest, especially at the DPRK national/central government ministries and Local Counterparts such as CPCs, CNTDAs and STSs in Unchon, Unryul and Hoechang Counties. In terms of project design, the proxy indicators would be the number of stakeholders involved in planning and attendance during the project formulation/planning meetings. The evaluation interviews with National and Local Counterparts indicated sufficient evidence of direct involvement based on detailed accounts of the project outputs. The minutes of the PSC meetings recorded perfect attendance and representations from the National Counterparts. The proxy indicators from M&E Field Monitoring Visits for participation at the project implementation stage indicated high project output ownership, perfect attendance at project field site meetings, capacity development/knowledge dissemination activities, and the visible evidence of construction/installation taking place. During the evaluation interviews, there were high levels of project output-ownership as the Local Counterparts and beneficiaries were able to provide extensive technical details of their project outputs. Tag: Knowledge management Programme/Project Design Civil Societies and NGOs |
| 6. | 3.1.6 Replication Approach Replication and up-scaling are fundamental to the SED Project as it provides the opportunity to build on best practices and lessons learned, and expand the reach and impact of its project outputs. As such, UNDP, government agencies and the private sector would utilize these given opportunities to support the replication and up-scaling of the most successful projects and practices through their networks and contacts. Tag: Sustainability Food Security |
| 7. |
Tag: Implementation Modality Country Government |
| 8. | Despite the early and recurring setbacks, the evaluation reviewed that the SED Project Team displayed good project management abilities and effectively utilised appropriate project management tools to implement the SED Project to the best of their abilities. Tag: Project and Programme management |
| 9. | There was evidence of strong interest and commitment at the local county level with the signing of the Exchange of Letter documents between UNDP DPRK, MoLI and CPCs of Unchon, Unryul and Hoechang Counties on the stakeholder contributions (both financial and in-kind), roles and responsibilities to implement the SED Project Activities. Tag: Country Government UN Agencies UN Country Team |
| 10. | However, the evaluation reviewed that there were inconsistencies (and inconsistent templates) in the SED Project Team’s financial reporting processes due to different reporting requirements given. Tag: Operational Efficiency Project and Programme management |
| 11. | However, the M&E process at the project level by the SED Project Team could be further improved in 2 key areas:
Tag: Monitoring and Evaluation Sustainability |
| 12. | 2. Field data collection to measure effectiveness and impact on completed project activities Tag: Effectiveness Impact Monitoring and Evaluation Sustainability |
| 13. | The evaluation established that there were strong working relationships between the UNDP DPRK CO, the SED Project Team and National/Local Counterparts and project beneficiaries at the county level. These working relationships were frequently tested by the slow progress of the SED Project. Key representatives of the National/Local Counterparts expressed numerous disappointments at the prolonged delays and unsuccessful implementation of the SED Project. Tag: Implementation Modality Country Government UN Agencies UN Country Team |
| 14. | The evaluation observed that: • all outputs (1.1, 2.1, 2.2, 3.1) that UNIDO was fully responsible for as stated in the LOA did not deliver and achieve the desired project results and hence received the lowest rating The evaluation further noted that UNDP had done its best to deliver and achieve the desired project results despite encountering significant external factors/challenges, mainly due to the 6 UN Sanctions in 2016 and 2017 and the recurring banking channel disruption/closure that prevented funds transfer into DPRK) during the SED Project duration. Tag: Impact Results-Based Management |
| 15. | While the SED PRODOC appropriately addressed the problems/needs identified and collectively presented a comprehensive solution to strengthen the national and local capacity for improved nutrition and food security, the evaluation reviewed that SED Project was too comprehensive and ambitious with 130 activities, comprising mainly complex technical design specifications, procurement of complex technical equipment and materials, and customized installation and construction to local requirements over the initial 3-year project duration Tag: Relevance Knowledge management Programme/Project Design |
| 16. | Due to external factors/environment beyond the control of the SED Project team severely affected the desired project results. Hence the SED Project did not fully achieve the intended outcome. Out of the 7 project outputs: • 5 outputs (1.1, 1.2, 2.2, 2.2, 3.1) were not fully effective • 2 outputs (1.3 and 3.2) were substantively effective Tag: Effectiveness Monitoring and Evaluation |
| 17. | 3.3.4 Efficiency Achievement Rating: 2/6 (Unsatisfactory – Major Shortcomings) Tag: Efficiency |
| 18. | 3.3.5 National Ownership Achievement Rating: 5/6 (Satisfactory – Minor Shortcomings) Tag: Ownership Country Government Capacity Building |
| 19. | 3.3.6 Sustainability Sustainability Rating: 2/4 (Moderately Unlikely - Significant Risks) Tag: Sustainability Implementation Modality Risk Management |
| 20. | Despite the above major setbacks encountered, the basic human needs were potentially achieved through concrete examples of how the Spirulina and Pistia Centres considered the needs of women and young children. Tag: Agriculture Livestock Nutrition Inclusive economic growth |
| Recommendations | |
|---|---|
| 1 | Develop PRODOCS that take into close consideration the issues faced in special country context like DPRK PRODOCs should be developed to mitigate issues faced in special country context with the following governance framework: • Partnership arrangements and the governance modality should be simplified and appropriately led by UNDP with an agency partner or technical working/advisory group (preferably with in-country office presence) to minimize partnership complications. If any partnership agreement is required, this should be concluded with clear roles and responsibilities for accountability purposes, signed and attached as an annex to the signed PRODOC before the commencement of any projects. |
| 2 | Improve financial reporting processes For improved financial accountability and transparency purposes, UNDP DPRK CO should improve their project financial reporting processes to track and report progress of consistent financial figures i.e. budget and actual expenditure for consistent comparisons between budget and actual expenditure, as per project outputs, based on project CDRs, for submissions of all relevant project reports (including annual progress reports), to demonstrate the efficient use of funding on project output-based activities. |
| 3 | Review and update extensively the country office policies and procedures with long-term scenario planning Numerous external factors/challenges occurred in the period of 2013-2018 which severely constrained the UNDP DPRK CO in successfully delivering the desired results. It is therefore recommended that the UNDP DPRK CO should: R3.1) work with UNDP Regional HQ to extensively review and update all operational, procurement and financial management policies and procedures to account for all that happened within the 2013-2018 period and appropriately mitigate any future constraints |
| 4 | Develop a robust M&E system at project level R4: Developing a robust M&E system at project level |
| 5 | Develop and implement a Sustainable Production and Consumption Supply Chain e.g. for the Pistia and Spirulina production and wild fruits processing
The pilot projects in producing Pistia and Spirulina, together with the potential to produce wild fruits, have the potential to be replicated and upscaled to other provinces within DPRK to improve nutrition and food security. They further demonstrated the need for continuity to enable a complete agriculture/food supply chain for sustainable production and consumption. |
| 6 | Build capacity continuously in sustainable agriculture, farming and food production practices To further improve nutrition and food security, any future projects in DPRK should continue to include capacity building activities at local county and village (Ri) levels such as developing and implementing: R6.1) foundational and advanced hands-on/practical courses in sustainable agriculture, farming and food production practices to overcome and adapt to severe climate change |
1. Recommendation:
Develop PRODOCS that take into close consideration the issues faced in special country context like DPRK
PRODOCs should be developed to mitigate issues faced in special country context with the following governance framework: • Partnership arrangements and the governance modality should be simplified and appropriately led by UNDP with an agency partner or technical working/advisory group (preferably with in-country office presence) to minimize partnership complications. If any partnership agreement is required, this should be concluded with clear roles and responsibilities for accountability purposes, signed and attached as an annex to the signed PRODOC before the commencement of any projects.
• Any technical design specifications/requirements should be appropriately identified and formulated during the fact-finding mission prior to developing the PRODOC.
Management Response: [Added: 2019/11/21] [Last Updated: 2020/11/21]
Agreed
Key Actions:
| Key Action | Responsible | DueDate | Status | Comments | Documents |
|---|---|---|---|---|---|
| 1.1 Simplify partnership arrangements and the governance modality appropriately led by UNDP with an agency partner or technical working/advisory group (preferably with in-country office presence) to minimize partnership complications. If any partnership agreement is required, this will be concluded with clear roles and responsibilities for accountability purposes, signed and attached as an annex to the signed PRODOC before the commencement of any projects.
[Added: 2019/11/21] [Last Updated: 2020/04/22] |
CO, BRH | 2020/04 | Completed | Project closed History | |
| 1.2 Appropriately identify and initially formulate technical design specifications/requirements during the fact-finding mission prior to developing the PRODOC.
[Added: 2019/11/21] [Last Updated: 2020/04/22] |
CO, BRH | 2020/04 | Completed | Project closed History |
2. Recommendation:
Improve financial reporting processes
For improved financial accountability and transparency purposes, UNDP DPRK CO should improve their project financial reporting processes to track and report progress of consistent financial figures i.e. budget and actual expenditure for consistent comparisons between budget and actual expenditure, as per project outputs, based on project CDRs, for submissions of all relevant project reports (including annual progress reports), to demonstrate the efficient use of funding on project output-based activities.
Current project progress reports only contain a CDR as an annex which does not provide a clear picture for comparison of output / activity based progress / expenditures against the plan / budget.
Management Response: [Added: 2019/11/21] [Last Updated: 2020/11/21]
Agreed
Key Actions:
| Key Action | Responsible | DueDate | Status | Comments | Documents |
|---|---|---|---|---|---|
| 2.1 Report consistent financial figures for comparisons of actual expenditures versus planned budget as per project outputs in submissions of all important reports, to demonstrate the efficient use of funding as well as financial accountability and transparency.
[Added: 2019/11/21] [Last Updated: 2020/04/22] |
CO, project team | 2020/04 | Completed | Project closed History |
3. Recommendation:
Review and update extensively the country office policies and procedures with long-term scenario planning
Numerous external factors/challenges occurred in the period of 2013-2018 which severely constrained the UNDP DPRK CO in successfully delivering the desired results. It is therefore recommended that the UNDP DPRK CO should: R3.1) work with UNDP Regional HQ to extensively review and update all operational, procurement and financial management policies and procedures to account for all that happened within the 2013-2018 period and appropriately mitigate any future constraints
R3.2) incorporate extensive long-term scenario planning processes with appropriate risk assessments and counter-measures to ensure that suitable policies and procedures can be implemented to resolve and minimize issues in the event of unforeseen circumstances
Management Response: [Added: 2019/11/21] [Last Updated: 2020/11/21]
Agreed
Key Actions:
| Key Action | Responsible | DueDate | Status | Comments | Documents |
|---|---|---|---|---|---|
| 3.1 Work with RBAP and BRH to extensively review and update all operational, procurement and financial management policies and procedures responding to the changed situations for appropriately mitigating any future constraints
[Added: 2019/11/21] [Last Updated: 2020/04/22] |
CO, BRH, RBAP | 2020/04 | Completed | Project closed History | |
| 3.2 Strengthen long-term scenario planning processes with appropriate risk assessments and counter-measures to ensure that suitable policies and procedures can be implemented to resolve and minimize issues in the event of unforeseen circumstances
[Added: 2019/11/21] [Last Updated: 2020/04/22] |
CO, project team | 2020/04 | Completed | Project closed History |
4. Recommendation:
Develop a robust M&E system at project level
R4: Developing a robust M&E system at project level
The UNDP DPRK CO should develop a robust M&E system at project level with effective monitoring and accountability mechanisms to: • collect and report real/reliable data during project implementation, including field implementation visits as follow up to programme field monitoring visits, in order to show the results achieved and the impact.
• continuously monitor and report in the project annual reports on the use of the assets and delivered items, after handover to project beneficiaries, to see the full operation/production in its entirety and also to determine the expected impact results.
Management Response: [Added: 2019/11/21] [Last Updated: 2020/11/21]
Agreed
Key Actions:
| Key Action | Responsible | DueDate | Status | Comments | Documents |
|---|---|---|---|---|---|
| 4.1 Strengthen the monitoring of operations of project supported facilities including the use of the assets and delivered items, after handover to project beneficiaries, and collection of reliable data, during project implementation visits in addition to program monitoring, in order to have evidence-based project results reporting and replication
[Added: 2019/11/21] [Last Updated: 2020/04/22] |
CO program and project teams | 2020/04 | Completed | Project closed History |
5. Recommendation:
Develop and implement a Sustainable Production and Consumption Supply Chain e.g. for the Pistia and Spirulina production and wild fruits processing
The pilot projects in producing Pistia and Spirulina, together with the potential to produce wild fruits, have the potential to be replicated and upscaled to other provinces within DPRK to improve nutrition and food security. They further demonstrated the need for continuity to enable a complete agriculture/food supply chain for sustainable production and consumption.
In line with the UN SDG #12 (Ensure sustainable consumption and production patterns) which significantly focuses on “operating on supply chain, involving everyone from producer to final consumer”6, it is recommended that: R5.1) future UNDP DPRK projects should develop and complete the full agriculture/food supply chain, incorporating climate change adaption/resilience capabilities to overcome severe climate change conditions, to upscale and fully commercialize the production of agricultural and food products for increased nutrition and food security in DPRK
R5.2) UNDP DPRK CO should facilitate knowledge/operational transfer of successful pilot projects (Pistia and Spirulina Centers as key examples) with procedural, operational and hands-on training manuals should be replicated in close partnership with National/Local Counterparts
Management Response: [Added: 2019/11/21] [Last Updated: 2020/11/21]
Agreed
Key Actions:
| Key Action | Responsible | DueDate | Status | Comments | Documents |
|---|---|---|---|---|---|
| 5.1 Develop and complete the full agriculture/food supply chain for UNDP projects in such area, incorporating climate change adaption/resilience capabilities to overcome severe climate change conditions, to upscale and fully commercialize the production of agricultural and food products for increased nutrition and food security in DPRK
[Added: 2019/11/21] [Last Updated: 2020/04/22] |
CO, BRH | 2020/04 | Completed | Project closed History | |
| 5.2 Facilitate knowledge/operational transfer of successful pilot initiatives (Pistia and Spirulina Centers as key examples) with procedural, operational and hands-on training manuals for wider replications in close partnership with national/local counterparts.
[Added: 2019/11/21] [Last Updated: 2020/04/22] |
CO, national and local counterparts | 2020/04 | Completed | Project closed History |
6. Recommendation:
Build capacity continuously in sustainable agriculture, farming and food production practices
To further improve nutrition and food security, any future projects in DPRK should continue to include capacity building activities at local county and village (Ri) levels such as developing and implementing: R6.1) foundational and advanced hands-on/practical courses in sustainable agriculture, farming and food production practices to overcome and adapt to severe climate change
R6.2) gender mainstreaming activities to assess the capacity needs according to gender requirements, and foundational and advanced hands-on/practical courses specifically relating to enhancing gender equality and improving the women’s living and livelihood standards
R6.3) train-the-trainer courses to transfer knowledge gained from the courses in R6.1 and R6.2 to national/local research institutes, technology and dissemination centres, and vocational skills training schools to increase the training impact in other provinces/counties in DPRK.
R6.4) study tours for increased exposure to acquiring knowledge in global trends and best practices in other countries of similar context and/or culture to DPRK
Management Response: [Added: 2019/11/21] [Last Updated: 2020/11/21]
Agreed
Key Actions:
| Key Action | Responsible | DueDate | Status | Comments | Documents |
|---|---|---|---|---|---|
| 6.1 Develop and deliver foundation and advanced hands-on/practical courses in sustainable agriculture, farming and food production practices overcoming / adapting to severe climate change.
[Added: 2019/11/21] [Last Updated: 2020/04/22] |
CO, project team | 2020/04 | Completed | Project closed History | |
| 6.2 Develop and deliver foundation and advanced hands-on/practical courses specifically relating to enhancing gender equality and improving the women’s living and livelihood standards.
[Added: 2019/11/21] [Last Updated: 2020/04/22] |
CO, project team | 2020/04 | Completed | Project closed History |