Management Response Detail

Sustainable Rural Energy Development (SRED)

Commissioning Unit:	DPR Korea
Evaluation Plan:	2011-2019
Evaluation Type:	Outcome
Completion Date:	11/2013
Unit Responsible for providing Management Response:	DPR Korea

1. Recommendation: Evaluation Recommendation or Issue 1. The implementation of the SRED programme exhibits important deviations from the original strategy and budget. This is mainly due to the impact of changes in the project context (primarily related to international political issues), which have led to: (i) a long suspension of activities; (ii) changes in budget distribution; (iii) the suspension of certain activities; and (iv) the inability to leverage additional financial resources from other agencies and/or financiers. These deviations imply an alteration of the anticipated project strategy away from achieving its ultimate objective (i.e. to prepare DPRK to implement a nation-wide rural energy programme). It is recommendable to make explicit the present objectives that SRED wants to achieve.

Management Response: [Added: 2014/02/26]

Partially agree, as SRED Project Team had raised all the issues on inception and previous PSC meetings and the objectives were discussed and fine-tuned according to the circumstances. # of guidelines in place that must be considered for future programming in the field of energy. While some guidelines are directly due to the international political context for UNDP in DPRK, others are based on lessons drawn over time. The issuance of new UNDP guidelines has implications for the strategy and scope of an intervention, as is the case for the SRED programme. UNDP guidelines and principles not mentioned in the UNSF and CP include: (i)a focus away from grid-connected energy technologies (even small grids); (ii) a focus away from direct support to policy development; (iii) a limitation of project funds available for hardware to 20% of the total project sum funded by UNDP; and (iv) a halt on capacity building activities for high-level Government staff, including international study tours. Updates on internal UNDP policy and principles are communicated with the Government of DPRK (NCC).

Key Actions:

Key Action	Responsible	DueDate	Status	Comments	Documents
1.1 To share approved Stocktaking Report with the key stakeholders. [Added: 2014/02/26]	DRR, PM	2013/12	Completed	Stocktaking report was shared with key stakeholders after approval by DRR in December 2013.
1.2 To agree on final revised outcomes before end of the project. [Added: 2014/02/26]	DRR, PM	2013/12	Completed	The 6th Joint PSC meeting took place on 12 December 2013.

2. Recommendation: Evaluation Recommendation or Issue 2: By consequence, progress of SRED is very much in terms of delivered activities. It is unlikely that the envisaged improvements in development conditions (as pursued by the Outputs 2, 3 and 4) will actually be achieved. As a result, the expected end-of-project situation will not reached. According to the Consultant, this is not only a result of the changes in international context but also due to flaws in the identification and validation of the underlying assumptions for the SRED Programme. The Programme is highly ambitious in its objective, time frames are extremely short, roles of national counterparts are not always clear and their technical and managerial skills would need substantial strengthening. These are valuable lessons learned that were not available in 2005.

Management Response: [Added: 2014/02/26]

Fully agree. These valuable lessons must be considered for future programming in the field of energy. The political context directly affects project implementation and execution. As a result, policy-related activities under SRED were suspended. UNDP?s DPRK specific guidelines can have repercussions on the overall project strategy and limit direct investment in equipment. A negative side-effect is that sometimes expectations are created among rural beneficiaries, which cannot be met. Project activities generally need more time than anticipated. The renewable energy pilots under SRED are affected by external factors (among others: the long, idle winter period) and inadequate local managerial skills. Also, the complexity and implications of technology development and quality assurance are underestimated.

Key Actions:

Key Action	Responsible	DueDate	Status	Comments	Documents
2.1 To share approved Stocktaking Report with the key stakeholders. [Added: 2014/02/26]	DRR, PM	2013/12	Completed	Stocktaking report was shared with key stakeholders after approval by DRR in December 2013
2.2 Further discuss these issues with national and international consultants/partners, on the following PTC and PSC meetings [Added: 2014/02/26] [Last Updated: 2016/06/01]	PM	2013/12	Completed	The 6th Joint PSC meeting took place on 12 December 2013. Following action to be taken by DRR and P

3. Recommendation: Evaluation Recommendation or Issue 3: SRED has made substantial progress in terms of expenditures and outputs since 2011. As of December 2012, about 56% of total resources have been spent. It is anticipated to terminate the Programme by end 2013. This is only feasible if the implementation of the pending renewable energy projects (one hydropower plant, one pig farm digester, one rice husk gasifier) evolves without delay. Three more digesters are programmed that are to be produced in DPRK. This involves a process of technology transfer (which has already started) and the availability of production facilities. Based on earlier experiences with the SWEDPRA project, the Consultant is not optimistic that these three systems can be produced and installed in the course of 2013.

Management Response: [Added: 2014/02/26]

Fully agree. Project was granted no cost extension until end of June 2014. International procurement in DPRK has become increasingly complex, time-consuming, and costly, putting a large burden on project staff and UNDP procurement officers. The main causes are the need for strict compliance with international sanctions on DPRK, difficult international logistics, the lack of response by international providers of goods and services, and the high prices quoted. PM had suggested no cost extension for one year and final decision of the Senior Management was 6 months.

Key Actions:

Key Action	Responsible	DueDate	Status	Comments	Documents
3.1 PM to prepare justification on the realistic date of project termination. [Added: 2014/02/26]	DRR, PM	2013/11	Completed	Action taken by PM and UNDP CO
3.2 Before receiving of formal approval of the Stocktaking Report UNDP Senior Management has taken decision for no cost extension of SRED project until June 2013. [Added: 2014/02/26]	DRR, PM	2013/11	Completed	Action taken by UNDP Senior Management in consultation with UNDP HQ

4. Recommendation: Evaluation Recommendation or Issue 4: The country situation provides a strong rationale for UNDP to address the energy situation in the rural areas in DPRK and contribute to the Millennium Development Goals (MDGs). The choice for the Cooperative Farms (COFs) as an exclusive target group seems not fully justified by human development or vulnerability criteria since other groups are equally vulnerable. From a practical point of view, the COFs provide a good starting point to implement renewable energy and energy efficiency technologies and extract valuable lessons. The COFs operate as more or less independent socioeconomic units, which facilitates assessing the baseline situation and achieved impacts. On the other hand, the focus on COFs may imply that potential synergies at a higher level are not recognized or exploited

Management Response: [Added: 2014/02/26]

Partially agree. UNDP is constrained by its current mandate from engagement at levels higher than rural/cooperative farm level.These valuable lessons learned must be considered for future programming in the field of energy. National partners and local people have little experience with renewable energy projects and lack the managerial and technical skills to implement them. Up-scaling will not be possible if project implementation is not successfully transferred to a national entity with substantial executing capacity.

Key Actions:

Key Action	Responsible	DueDate	Status	Comments	Documents
4.1 To share approved Stocktaking Report with the key stakeholders. [Added: 2014/02/26]	DRR, PM	2013/12	Completed	Stocktaking report was shared with key stakeholders after approval by DRR in December 2013
4.2 To reflect the point on selection of entry points for future new UNDP rural energy intervention in ToRs of local and international consultants for development new successor programme [Added: 2014/02/26] [Last Updated: 2016/06/01]	DRR, PM	2015/10	Completed	Action to be taken by PM and DRR New project formulated reflecting evaluation recommendations

5. Recommendation: Evaluation Recommendation or Issue 5: Technology demonstration and barrier removal activities are more difficult than anticipated. The linear approach proposed by SRED is subject to external factors and risks that may not have been acknowledged as such at design stage. In practice, the project rather follows a learning-by-doing approach. In many occasions, the PM recalls earlier experiences to expedite subsequent demonstration pilots. The Programme?s tight time schedule is not well matched to this more practical approach. By consequence, project implementation is continually perceived as ?delayed?, while a more positive perception could be that many useful lessons are being learnt within a short time period (since 2011).

Management Response: [Added: 2014/02/26]

Fully agree. These valuable lessons learned must be considered for future programming in any UNDP projects in DPRK. The limited background information and knowledge available to UNDP make it more difficult to validate assumptions and propose an adequate project strategy. The project context can change quickly due to international issues, as canproject mandates and staffing of national counterparts.

Key Actions:

Key Action	Responsible	DueDate	Status	Comments	Documents
5.1 To reflect in the project progress reports more positive perception that many useful lessons are being learnt within a short time period since SRED project implementation started in 2010. [Added: 2014/02/26] [Last Updated: 2016/06/01]	PM	2014/10	Completed	Action to be taken by PM Project closed in October 2014
5.2 To collect more information through conducting development and institutional framework assessment and household survey for new UNDP rural energy initiative development. [Added: 2014/02/26] [Last Updated: 2016/06/02]	PM	2016/05	Completed	Action to be taken by PM The new Sustainable Energy Solutions project is undertaking detail energy demand and resource assessment studies in 15 Ris (survey started in May 2016)

6. Recommendation: Evaluation Recommendation or Issue 6: The activities proposed in the Strategic Results Framework are not always appropriate and/or sufficient. Study tours, curriculum building and international workshops are not effective to increase technical and management skills at farm level (and are targeted at higher-level persons). The stocktaking mission could observe the need for improved technical and managerial competence for implementing the demonstration pilots. Unfortunately, the context in DPRK offers little opportunities for nationals to acquire hands-on experience. DPRK can benefit from experiences in other countries to develop low-cost energy technologies, improve building design, introduce more efficient agricultural practices, and learn about sustainable management and delivery models. Conservation agriculture and combined food and energy production are pursued in various countries, in some cases with GEF support. While rural energy users would be the ultimate beneficiaries, partnerships with foreign peer organizations can help strengthening the knowledge base in DPRK and induce a process towards more international integration. A good example of South-South partnerships under SRED is the biogas plant from the Kyrgyz Republic. Priority should be given to strengthening of capabilities for operation, maintenance and repair of energy technologies at the farm level.

Management Response: [Added: 2014/02/26] [Last Updated: 2014/02/26]

Partially agree, as some tours and such have been found to be useful. These valuable lessons learned must be considered for future programming in any UNDP projects in DPRK.

Key Actions:

Key Action	Responsible	DueDate	Status	Comments	Documents
6.1 To share approved Stocktaking Report with the key stakeholders. [Added: 2014/02/26]	PM	2013/12	Completed	Stocktaking report was shared with key stakeholders after approval by DRR in December 2013.
6.2 To discuss these issues during consultation meetings with NCC and other key stakeholders. [Added: 2014/02/26] [Last Updated: 2016/06/01]	DRR	2015/10	Completed	Action to be taken by PM and DRR New project formulated reflecting evaluation recommendation

7. Recommendation: Evaluation Recommendation or Issue 7: The SRED Programme team does presently not have a comprehensive understanding of the social, geographic, and economic organization of the rural areas in DPRK. It is suggested to strengthen inhouse knowledge on these aspects, as it will contribute to justify the choice for the COF as an entry point made for SRED. Representatives from the national government (NCC) and international agencies (WFP) indicate that the cooperative farms are not the most vulnerable groups in the country. NCC would like to see a broader approach altogether.

Management Response: [Added: 2014/02/26]

Partially agree. These valuable lessons learned must be considered for future programming in any UNDP projects in DPRK. On-going work as done under SRED provides valuable information and knowledge for UNDP about the organization of life in rural DPRK, as well as the mechanisms (such as barter and local markets) to cope with scarcity of food and materials. This information is useful for future programming and to support a differentiated approach with respect to gender and vulnerable groups.

Key Actions:

Key Action	Responsible	DueDate	Status	Comments
7.1 To share approved Stocktaking Report with the key stakeholders. [Added: 2014/02/26]	DRR, PM	2013/12	Completed	Stocktaking report was shared with key stakeholders after approval by DRR in December 2013
7.2 To collect more information through conducting development and institutional framework assessment and household survey for new UNDP rural energy initiative development. [Added: 2014/02/26] [Last Updated: 2016/06/02]	DRR, PM	2016/05	Completed	Action to be taken by PM and DRR The new Sustainable Energy Solutions project is undertaking detail energy demand and resource assessment studies in 15 Ris (survey started in May 2016)
7.3 To discuss these issues during consultation meetings with NCC and other key stakeholders. [Added: 2014/02/26] [Last Updated: 2016/06/02]	DRR, PM	2016/05	Completed	Action to be taken by PM and DRR Discussion made and the new Sustainable Energy Solutions project is undertaking detail energy demand and resource assessment studies in 15 Ris (survey started in May 2016)

8. Recommendation: Evaluation Recommendation or Issue 8: The technology pilots are valuable elements of a learning process for UNDP and its Government partners. It is important to distinguish between energy products and renewable energy projects. The challenge regarding products is mostly related to product development and testing to ensure performance, durability, ease of installation and maintenance; and to efficient mechanisms to reach the end-user. Systematic testing is needed to certify that proposed technical solutions are reliable, effective, and safe.

Management Response: [Added: 2014/02/26]

Partially agree. These valuable lessons learned must be considered for future programming in any UNDP projects in DPRK. Due to tight time frame, it is very challenging to collect information based on M&E for developing good lessons learned and good practice information.

Key Actions:

Key Action	Responsible	DueDate	Status	Comments	Documents
8.1 To share approved Stocktaking Report with the key stakeholders. [Added: 2014/02/26]	PM	2013/12	Completed	Stocktaking report was shared with key stakeholders after approval by DRR in December 2013.
8.2 Continue M&E as it is planned until end of project by June 2014. [Added: 2014/02/26] [Last Updated: 2016/06/01]	PM	2014/10	Completed	Action to be taken by PM Actions taken, until project closed

9. Recommendation: Evaluation Recommendation or Issue 9: Projects require capital investment (equipment), skilled human resources, proper planning, management and supervision. Project implementation processes are tedious, with UNDP being closely involved with the supervision, logistics and operational management. The used contract modalities are basically focused on procurement of equipment and do not fully specify the roles of the actors involved in project implementation. National partners and local people have little experience with renewable energy projects and lack the managerial and technical skills to implement them. Up-scaling will not be possible if project implementation is not successfully transferred to a national entity with substantial executing capacity

Management Response: [Added: 2014/02/26]

Fully agree. These valuable lessons learned must be considered for future programming in any UNDP projects in DPRK. Output 6 was discussed on PSC meetings and changed to ?New Successor Sustainable Rural Energy Development Programme formulated and designed?

Key Actions:

Key Action	Responsible	DueDate	Status	Comments
9.1 To share approved Stocktaking Report with the key stakeholders. [Added: 2014/02/26]	PM	2013/12	Completed	Stocktaking report was shared with key stakeholders after approval by DRR in December 2013
9.2 To discuss these issues during consultation meetings with NCC and other key stakeholders. [Added: 2014/02/26]	DRR, PM	2013/12	Completed	Action taken by PM and DRR.
9.3 To highlighted changes of Output 6 on PSC meeting and discuss it with concerned parties. [Added: 2014/02/26]	DRR, PM	2013/12	Completed	Action taken by PM and DRR.

10. Recommendation: Evaluation Recommendation or Issue 10: The energy projects requested by the Cooperative Farms (specifically the capital-intensive small hydro power plants) are not always justified from an economic perspective, or are not the least-cost solution. As observed during the mission, end-users are eager to see electricity service established or restored, which is understandable. Electricity usage in the rural areas however is very inefficient from a chain perspective as a result of obsolete distribution lines, inefficient appliances and ineffective processes. End-users and intermediaries must learn to become not only energy- but also resourceefficient. It is recommended to apply appropriate analytical tools (such as life-cycle analysis) to energy solutions, once initial experiences have been gained with the demonstration pilots. The national counterparts must be trained to apply these concepts and tools to specific situations.

Management Response: [Added: 2014/02/26]

Partially agree. These valuable lessons learned must be considered for future programming in any UNDP projects in DPRK. With respect to institution building, it is clear that efficient institutions are the key to supplying capital inputs (including energy solutions) to rural communities. As indicated before, these institutions are not sufficiently well identified, described and understood. Voids exist in the national institutional framework, in a sense that required functions are not adequately recognized and covered. Barriers in this aspect are systemic and beyond the scope and mandate of UNDP?s work in DPRK.

Key Actions:

Key Action	Responsible	DueDate	Status	Comments	Documents
10.1 To collect more information through conducting development and institutional framework assessment and household survey for new UNDP rural energy initiative development. [Added: 2014/02/26] [Last Updated: 2016/06/02]	PM	2016/05	Completed	Action to be taken by PM The new Sustainable Energy Solutions project is undertaking detail energy demand and resource assessment studies in 15 Ris (survey started in May 2016)

11. Recommendation: Evaluation Recommendation or Issue 11: SRED is paying little attention to the design of sustainable delivery models for energy solutions in the rural areas. However, energy products produced in DPRK (including efficient coal and biomass stoves, and thermal isolation blocks) are presently sold to UNDP. One would expect these to be marketed to end-users instead (which seem to have some purchasing capacity). The signal presently received by the national counterparts is that sales are guaranteed at a high price, which consolidates a comfortable niche market. There is no stimulus to reduce product costs and optimize transport logistics, and to supply the product in large volumes (at a reduced margin per product). This situation is not in the interest of the beneficiaries targeted by SRED and does not generate any leverage on the resources provided by UNDP

Management Response: [Added: 2014/02/26]

Partially agree, as initially main goal of SRED project was to demonstrate modern renewable energy technologies. In absence of market mechanisms it is difficult to expect to find any incentive/stimulus to reduce product costs and optimize transport logistics, and to supply the product in large volumes. The situation partially can change for better if it would be possible to transfer the project results for up scaling to a national entity with substantial executing capacity.

Key Actions:

Key Action	Responsible	DueDate	Status	Comments	Documents
11.1 To share the Stocktaking Report with concerned partners. [Added: 2014/02/26] [Last Updated: 2016/06/01]	PM	2014/10	Completed	Action to be taken by PM.

12. Recommendation: Evaluation Recommendation or Issue 12: The lack of appropriate delivery mechanisms is linked to institutional and policy barriers. SCST has the mandate to deliver technology to society but may lack the necessary human resources, or may not have a vision on how to collaborate with lower-level authorities (provinces, counties, farms). UNDP is still in the process of understanding the role of national actors, which limits its possibilities to identify key partners and processes for promoting effective delivery mechanisms and supportive policy measures. Hopefully, on-going work with NCC and the national partners can strengthen this knowledge base. A specific policy barrier may be, that energy products for the rural areas (except coal) are not mainstreamed in the national planning system. A systemic barrier for most national partners is likely the lack of familiarity with modern logistics and supply services as a paradigm to follow.

Management Response: [Added: 2014/02/26]

On 4th PSC meeting (5 September 2012) it was commented on Output i.e. #4 ?Policies and Mechanisms Put in Place to Remove Barriers to Identification and Implementation of Energy Demonstration Projects?. It is noticed that policy, regulatory or institutional change is important to address structural and systemic barriers to energy access but project has not been able to do much in this area due to the political context. It was recommended small steps could be taken in this regard i.e. survey of policies and regulations and gap analysis as a starting point. In the ensuing discussion, it was agreed that within the DPRK context, it would be better to use the term ?national guideline? and a deeper understanding is required before any inroads. PSC meeting concluded to revisit and address the above issues through a series of discussion after reviewing the detailed workplan submitted by PM by the end of September

Key Actions:

Key Action	Responsible	DueDate	Status	Comments	Documents
12.1 To share the Stocktaking Report with concerned partners. [Added: 2014/02/26]	PM	2013/12	Completed	Action taken by PM
12.2 To collect more information through conducting development and institutional framework assessment and household survey for new UNDP rural energy initiative development [Added: 2014/02/26] [Last Updated: 2016/06/02]	DRR, PM	2016/05	Completed	The new Sustainable Energy Solutions project is undertaking detail energy demand and resource assessment studies in 15 Ris (survey started in May 2016)

13. Recommendation: Evaluation Recommendation or Issue 13: The Consultant holds to the opinion that SRED must do a large effort to collect and systemize lessons learned from the present demonstration pilots and actually produce guidelines, manuals and best practices. This work should be done with great detail and include an assessment of the maturity of the various energy solutions demonstrated under SRED. To enable these activities, the Project horizon should be extended beyond December 2013. A differentiated approach can be followed, including ongoing support of promising (but not yet mature) technologies (component 2) if sufficient institutional capacity is available.

Management Response: [Added: 2014/02/26]

Disagree as this is already part of project Output 5. Compilation and dissemination of lessons learned and "Good Practices". Due to tight time frame and limitations on available data and applicable methodologies, it is very challenging to collect information based on M&E for developing good lessons learned and good practice information

Key Actions:

Key Action	Responsible	DueDate	Status	Comments	Documents
13.1 To share the Stocktaking Report with concerned partners. [Added: 2014/02/26]	PM	2013/12	Completed	Project has received no cost extension until June 2014. Although current developments might prevent a possibility to finish project by this time

Management Response: [Added: 2014/02/26]

Key Actions:

Management Response: [Added: 2014/02/26]

Key Actions:

Management Response: [Added: 2014/02/26]

Key Actions:

Management Response: [Added: 2014/02/26]

Key Actions:

Management Response: [Added: 2014/02/26]

Key Actions:

Management Response: [Added: 2014/02/26] [Last Updated: 2014/02/26]

Key Actions:

Management Response: [Added: 2014/02/26]

Key Actions:

Management Response: [Added: 2014/02/26]

Key Actions:

Management Response: [Added: 2014/02/26]

Key Actions:

Management Response: [Added: 2014/02/26]

Key Actions:

Management Response: [Added: 2014/02/26]

Key Actions:

Management Response: [Added: 2014/02/26]

Key Actions:

Management Response: [Added: 2014/02/26]

Key Actions:

Evaluations by Unit

Latest Evaluations

Evaluation Snap Shot

Contact us