Mid-Term Evaluation: Strengthening the Resilience of Communities through Community-Based-Disaster Risk Management (CBDRM)

Report Cover Image
Evaluation Plan:
2011-2019, DPR Korea
Evaluation Type:
Mid Term Project
Planned End Date:
09/2018
Completion Date:
08/2018
Status:
Completed
Management Response:
Yes
Evaluation Budget(US $):
12,000

Share

Document Type Language Size Status Downloads
Download document TOR_SES_CBDRM_Project_Final21Feb18.pdf tor English 488.26 KB Posted 291
Download document FinalMTRReport_CBDRM_August 2018.pdf report English 1248.91 KB Posted 396
Download document FinalMTRReport_CBDRM_Summary_August2018.docx summary English 1172.72 KB Posted 267
Title Mid-Term Evaluation: Strengthening the Resilience of Communities through Community-Based-Disaster Risk Management (CBDRM)
Atlas Project Number: 00091747
Evaluation Plan: 2011-2019, DPR Korea
Evaluation Type: Mid Term Project
Status: Completed
Completion Date: 08/2018
Planned End Date: 09/2018
Management Response: Yes
Focus Area:
  • 1. Environment & Sustainable Development
  • 2. Others
Corporate Outcome and Output (UNDP Strategic Plan 2014-2017)
  • 1. Output 5.4. Preparedness systems in place to effectively address the consequences of and response to natural hazards (e.g. geo-physical and climate related) and man-made crisis at all levels of government and community
Evaluation Budget(US $): 12,000
Source of Funding: TRAC
Evaluation Expenditure(US $): 10,000
Joint Programme: No
Joint Evaluation: No
Evaluation Team members:
Name Title Email Nationality
Denika Blacklock Independent Evaluation Consultant djbkarim@gmail.com
GEF Evaluation: No
Key Stakeholders:
Countries: DPRK -DEMOCRATIC PEOPLE'S REPUBLIC OF KOREA
Lessons
Findings
1.

Section 3: Analysis of Findings based on the OECD DAC for Development Evaluations

Project Strategy: To what extent is the project strategy relevant to country priorities, country ownership and the best route towards expected results ?

Relevance of the project design. Although one of the challenges faced by UNDP and all humanitarian actors in DPRK is a lack of concrete information on government priorities, recent policy and strategy developments related to disaster management in the country, as well as its commitment to international frameworks such as the Sendai Framework and the Paris Agreement, demonstrate that the government recognizes, at the very least, the impact that disaster and climate change are having on the country and the need to take action to reduce disaster and climate risks. Specifically, Chapter 4, page 93, Second National Communication on Climate Change 2012 states: “ensure investment for the work to minimize loss of life and property, and build the national capacity for prevention of disastrous events through establishment of the national real-time monitoring system, early warning system and enhancement of corresponding capacity of central and Government bodies to natural disasters such as flood, drought, forest fire, landslide, typhoon and tidal wave, etc., caused by climate change.” In this respect, the CBDRM project is highly relevant to the needs of the country. Although the project refers to disaster risk management (DRM), in practice, the project focuses primarily on preparedness and coping, with some risk reduction measures included. It does not systematically address the entire cycle of DRM, so it is important to keep that in mind when assessing the project’s results against its intended outcome related to improving community resilience. Nonetheless, as an initiative to introduce the concept and demonstrate CBDRM approaches to reduce threats to human life during disaster, it has proven to be – and continues to be even more so – a highly relevant initiative for the country.


Tag: Natural Disaster Relevance Policies & Procedures Programme/Project Design Country Government

2.

Continuing

Section 3: Analysis of Findings based on the OECD DAC for Development Evaluations

Project Strategy: To what extent is the project strategy relevant to country priorities, country ownership and the best route towards expected results ?

Based on desk review and field visits, the selection of sites to pilot the CBDRM project was undertaken with due consideration to constraints of efficiency, but also with a commitment to inclusion and effectiveness in mind. With 80% of the population living in mountainous areas at risk of flooding and landslides (the two hazards initially covered by the project based on the impact of disasters over the past 10-15 years), it would have been easy and justifiable to select communities that were easy to access for the project team. However, sites visited were remote, and in some cases, inaccessible due to weather conditions. Site selection was therefore based on Ris which were at high risk and unlikely to receive support through other projects, given the increased cost in both time, human resources and money to
implement activities in remote communities. This goes to the heart of disaster risk management and the importance of targeting the most vulnerable. With respect to effectiveness, a number of Ris were selected to maximize synergy with the SES project, which improved both management and implementation effectiveness, as well as providing synergies and increased benefits to the target communities.


Tag: Natural Disaster Vulnerable Operational Services

3.

Continuing

Section 3: Analysis of Findings based on the OECD DAC for Development Evaluations

Project Strategy: To what extent is the project strategy relevant to country priorities, country ownership and the best route towards expected results ?

That said, one of the challenges faced by the project was addressing issues of vulnerability and the differing needs of men, women and children (among others) in DRM. As iterated by other UN agencies and the IFRC during the MTR mission to DPRK, the concept of equality and equity (‘what one person receives, all receive’) colours perceptions of vulnerability (or lack thereof) within society. As also experienced during the MTR mission, collecting data or information on the varying needs of different groups in the community is challenging and often limited to the number of men and women, and number of school children. However, by using international good practice and information related to disaster planning (at the Ri level), early warning and evacuation, the project was able to ensure that elements of targeting the most vulnerable (children, pregnant women, the elderly) were included as standard practice in the community without overstating the issue and undermining local ideology.


Tag: Vulnerable Data and Statistics

4.

Continuing

Section 3: Analysis of Findings based on the OECD DAC for Development Evaluations

Project Strategy: To what extent is the project strategy relevant to country priorities, country ownership and the best route towards expected results ?

Appropriateness of the RRF. The original RRF was revised during the inception phase of the project in order to make it more relevant to the DPRK operating context. Based on the evaluator’s experience in other countries, this is not uncommon in UNDP, where projects are developed by people unfamiliar with a specific country context and are attempting to replicate successful ideas from one country to another, or do not engage the appropriate UNDP staff members to develop an RRF that is in line with UNDP planning and monitoring requirements, thus making results management a risky and tricky undertaking. The CBDRM project team did well to revise the RRF to focus on what was reasonably achievable in the operating context of DPRK and the budget constraints of the project. However, the output indicators are entirely quantitative and are very input (activity) oriented, limiting capacity to analyse any change effected by the project in the target Ris at the output level. This MTR has attempted to evaluate the outputs from a qualitative perspective and provides suggestions for additional indicators and revised targets to improve the level and quality of monitoring by the project team.

The activities identified to support the achievement of the outputs are critical but will not necessarily result in the achievement of the stated outputs. This is a result of poor conceptualization of outputs at the planning stage and beyond the control of the project team. At most, the activities will result in a partial achievement of the outputs (see below in section on Progress Towards Results).


Tag: Relevance Results-Based Management Theory of Change

5.

Section 3: Analysis of Findings based on the OECD DAC for Development Evaluations

Progress towards results: To what extent have the expected outputs and outcome of the project been achieved so far?

Assessment of progress towards targets. This MTR provides an assessment of progress towards current output targets based on monitoring data provided and supplementary qualitative information captured during the MTR country mission.

Factors contributing to progress. The primary factor contributing to the progress of the project to date has been the effectiveness and management capacity of the project team. In particular, the ability of the project team to continually readjust the work plan so as to take advantage of opportunities for activity implementation when they arise, and to identify and utilize efficiencies wherever possible. This includes taking advantage of international consultants for both training and knowledge management purposes.


The secondary factor in implementation progress to date has been the commitment at the Ri level to improve understanding of disaster management and make use of the knowledge and materials provided through CBDRM. However, implementation progress has been severely impeded by the sanctions regime on DPRK, limiting the capacity of the UNDP Country Office to undertake procurement in a timely fashion, if at all, as well as the Country Office’s Internal Control Framework (ICF) which requires the project and programme staff to spend significant amount of time on verification of activities and delivery of materials beyond normal monitoring processes, as well as requiring international staff members to be part of all monitoring activities, which is both expensive and time consuming. This increases the project management costs significantly and decreases the amount of time that can be allotted to the qualitative monitoring of the project at the output level.


Tag: Effectiveness Procurement Project and Programme management

6.

Continuing

Section 3: Analysis of Findings based on the OECD DAC for Development Evaluations

Progress towards results: To what extent have the expected outputs and outcome of the project been achieved so far?

Barriers to achieving project outputs. It goes without saying that the Security Council Resolution 2397 (2017) is the biggest barrier to project implementation to date. Although the project was designed within the framework of the previous sanctions regime, building on proven concepts and the procurement of approved materials, the application of more stringent sanctions in 2017 has made project implementation near to impossible, if not for the management capacities and commitment of the project team and Country Office. Although initial project targets were set within the previous sanctions regime and were highly likely to be achieved, given the fluidity of the current international environment regarding DPRK in mid-2018, it is difficult to determine whether or not the project will be able to achieve its present targets by the end of 2019 when the project is scheduled to close. Recommendations related to how to make the most of the capacities of the project in the current environment in order to effect change on the ground that will have a sustainable humanitarian impact are provided below.


Tag: Procurement Project and Programme management

7.

Project Implementation and Adaptive Management

Project efficiency. The CBDRM project is being implemented through the Direct Implementation Modality (DIM). In light of the geo-political context and the ICF for the Country Office, this is the most appropriate implementation arrangement. It also reduces the opportunity for misuse of funds and can, in the right context, improve the speed of implementation. Should there be an opportunity for sanctions exemptions to be granted for UNDP, and a review of the ICF towards easing some of the restrictions on the Country Office, ease of implementation and possibility of scale-up of the project will lead to significant impact in the target communities.


Tag: Procurement Project and Programme management Efficiency Implementation Modality

8.

Continuing

Project Implementation and Adaptive Management
Project efficiency.

Given UNDP’s precarious operating circumstances – implementing projects without a Country Programme Document, which lapsed in 2016 – it is very difficult for the project to formalize partnerships with relevant agencies in order to capitalize on shared knowledge, community relationships and complementary activities. During the country mission for this MTR, it was suggested by FAO, WFP and IFRC that joint programming would be beneficial to each organization, to leverage the work being done at the community level to increase capacity for DRM, while simultaneously reducing the risk of landslides though agro- forestry and improving food security and nutrition. UNDP should give serious and immediate consideration to the development of a new CPD for UNDP DPRK, within the framework of longer-term humanitarian intervention detailed above, so as to leverage partnerships for improved project efficiency, effectiveness and impact in target communities.


Tag: Efficiency Operational Efficiency Partnership UN Agencies

9.

Project Implementation and Adaptive Management

Financial Controls and In-kind Contributions. As noted routinely in this report, the project team has been extremely capable at managing its resources and adjusting planning in order to ensure that the project makes the most efficient use of its time and money in light of the many procurement challenges it faces. There is excellent planning, and in a ‘normal’ operating context, this would result in excellent financial planning and management. However, the project team is faced with many constraints, not least the issue of the banking channel, which impact how quickly it can access funds. In the view of this MTR, the project team is doing an excellent job within the constraints that it is implementing activities and should not be reviewed against issues beyond its direct control. The project team also makes good use of the in-kind contributions of communities related to both structural and non-structural works. While it was not possible to assess the in-kind contribution of each community, it was evident from the communities which were visited that the structural interventions which have already been completed would not have been without community participation to make reasonable initial preparations for structural interventions (ie: contributions that would equate to less than 15% of total activity cost), or their enthusiastic participation in activities such as the PDRA or the simulations.


Tag: Efficiency Human and Financial resources Procurement

10.

Project Implementation and Adaptive Management

Project priority: value for money or quality and inclusion. As noted above, the project selected sites which were most in need of support, as opposed to ones that were easiest to reach for implementation and monitoring purposes. In light of the global trend driven by a number of influential donors to guarantee value for money, it is refreshing to see a project focus reaching the most vulnerable rather than reaching as many people as possible. While it is more expensive (time, logistics and human resources) to reach remote areas, in the spirit of the SDGs and ‘leaving no one behind’ this project, as well as the SES project, should be held up as an example of what that means in practice. Moreover, it is noted in this review that remoteness of implementation has not undermined the quality of implementation, or the timeliness when implementation was not constrained by procurement issues and is likely to have a far more meaningful impact in the target communities than in other communities where risks were lower, although more people would have been reached through the activities.


Tag: Disaster risk management Efficiency Project and Programme management

11.

Project Implementation and Adaptive Management

Monitoring and Reporting. The Country Office has a comprehensive monitoring system at project and programme level, with guidelines on the roles and responsibilities of staff at both levels. Moreover, the ICF requires monitoring at both levels to be undertaken by international staff, in particular for verification of any materials procured through the projects. However, while guidelines for monitoring projects are in place, there are no specific tools to support standardized monitoring at the activity and output level. The issue of field visit reports and field monitoring reports was clarified during the country mission and understood by the evaluator to be the equivalent of Back to Office Reports (BTORs). However, because there is no standardized format and reports are not individualized as in other country offices, there is confusion related to purpose, content and follow-up. The Country Office needs to improve the tools used for monitoring, and who uses what tool, to clarify roles and responsibilities in monitoring, as well as monitoring for results.


Tag: Monitoring and Evaluation Quality Assurance Data and Statistics

12.

Continuing

Project Implementation and Adaptive Management

Monitoring and Reporting.

What is even more evident is that the rigourous activity implementation monitoring has allowed for excellent channels of communication between the project team and community stakeholders, and informal communications (if not formal) is very good, based on feedback from community leaders interviewed. The project strategy facilitating the use of international tools for disaster risk assessment and planning have resulted in a highly participatory decision-making process on what types of interventions were needed, where they were needed, and how they would be prioritized and implemented. This process has built trust between the communities and project team and leads to good information sharing when/where feasible in light of the restrictions on data collection mentioned above.

Normally, an MTR would assess how well a project uses monitoring data to communicate results to a wider audience, however, within the context of project implementation in DPRK limits what data can be used and where. As such, the MTR will not be evaluating this aspect of M&E within the project. However, with the potential implementation of more qualitative monitoring of results at the output level, it is recommended that more effort be put on internal communications of results within the wider UN system in order to support the justification for continued UNDP operations in DPRK, and to provide evidence for the need to ease procurement challenges for more effective project implementation.


Tag: Communication Monitoring and Evaluation Operational Efficiency

13.

Project Implementation and Adaptive Management

Stakeholder engagement and partnership management. The project document provided a detailed plan on how and with whom the project would engage in order to facilitate efficient project implementation and leverage the knowledge resources of partners. In reality, the lapsing of the CPD in 2016 has meant that the partnership strategy is being implemented only in part, with government partners. Partnerships with other UN agencies and organizations such as IFRC are informal at the information sharing level but are not sufficient to be considered having an impact on project efficiency and effectiveness. Partnerships with the government are limited to information sharing only, with no decision-making authority on the part of government. CBDRM’s government partners are SCEDM, SHMA, MoLEP and CBS, coordinated by the NCC. They participate in quarterly project steering committee (PSC) meetings, where progress to date, challenges and plans for the next quarter are presented and discussed, with final decisions taken by the UNDP DRR based on input and advice from project, programme and government stakeholders. SCEDM is
the focal point for the project, despite being a new entity within the government created 13 November 2014. Compared to SHMA and MoLEP, which have a long history of involvement in UNDP projects, SCEDM does not possess the leadership and coordination capacities necessary to take ownership of DRM processes initiated by the project, once the project is complete. This can be viewed as a criticism but also an opportunity. The project can support SCEDM to improve its knowledge base and understanding of CBDRM as a concept and leverage the experiences of the target villages to improve its own planning and management across a number of agencies involved in DRM. Although UNDP is restricted from providing technical support to government, using the CBDRM Framework supported by the project as a way to help SCEDM to facilitate its leadership on DRM could be a possible action area for scale up if procurement issues surrounding structural interventions at the Ri level persist.

Nonetheless, all partners interviewed during the MTR country mission, be they UN agencies, village beneficiaries or PSC members, said that they felt the project team and UNDP more generally was effective at communicating progress and challenges, even if the persistence of challenges around procurement created some friction and feeling of unmet expectations at national and Ri level.


Tag: Challenges Effectiveness Knowledge management Partnership Project and Programme management

14.

Project Implementation and Adaptive Management

Sustainability. One of the most critical aspects of projects that focus on knowledge transfer, and to some extent skills transfer, is that knowledge is rarely lost. Even when it is not used, and can be recalled when necessary, even in parts. To that end, the results of the CBDRM project in relation to raising awareness of and changing attitudes towards disaster risk management at the community level is likely to be very sustainable. While most knowledge products and other tools have already been handed over to the beneficiaries, with the CBDRM Programme Framework still under discussion and somewhat less likely to be moved forward with the government if current SCEDM coordination capacities are not improved, the major challenge for the project will be to encourage the dissemination of the information and skills provided so that organic roll-out (as opposed to scale-up) can take place in surrounding counties. One option is the continuing support of national consultants by UNDP to continue to collect disaster data, including damage and loss, which would support the generation of lessons learned within and between counties. In light of the commitment shown to the CBDRM activities through community mobilization for structural measures and agro-forestry, incremental changes using community resources are possible, with the encouragement of SCEDM and other partners.


Tag: Sustainability Communication Knowledge management

15.

Continuing

Project Implementation and Adaptive Management

Sustainability. In terms of potential for scale-up, in light of the current geopolitical environment and precarious operational context for UNDP, it is unlikely that any large-scale funding for CBDRM (as an issue) will be forthcoming in the short and medium terms. Despite the fact that the project has already demonstrated impact in relation to attitudes towards disaster preparedness and (some aspects of) disaster reduction, which will lead to a reduction in the loss of life and livelihood assets during crises in the future, it is not an obvious choice to channel humanitarian funding at the moment. One option that UNDP may want to consider supporting potential scale-up (or roll-out depending on the intensity) is to work with other development partners to have information and tools shared in counties where other disaster management and food security related projects are being implemented, in order to introduce concepts and potentially change mind sets on disaster management in the community.

 


Tag: Disaster risk management Sustainability Knowledge management Project and Programme management

16.

Continuing

Project Implementation and Adaptive Management

Sustainability.

Currently, the biggest risk to the sustainability of the changes effected to date is if the project or country office was to be closed due to operational constraints. With 17 months remaining in project implementation, there is sufficient time and resources to consolidate the gains made in knowledge and skills in the target communities, and to provide supplementary training and awareness sessions to deepen understanding of CBDRM and stimulate interest in the issue in surrounding communities. However, if viewed from the aspect of the communities, a major risk is the inability of UNDP to complete the procurement of materials for structural interventions for DRR and agro-forestry, which are highly regarded in the community and are important for both moral and demonstration effect. The gully check dams constructed in Chuma Ri which was built in 2016 (leading to a dramatic reduction in asset loss during the rainy season in 2017) is an excellent example. Increased moral leads to increased commitment to and ownership of a new idea. Further, with the onset of the rainy season, the delay in structural and agro-forestry interventions could lead to further environmental risks to the project should there be any severe flooding and resulting landslides, reversing the work already undertaken by the communities to prepare for those interventions.


Tag: Natural Disaster Sustainability Project and Programme management Operational Services

17.

Continuing

Project Implementation and Adaptive Management

Sustainability. One of the challenges for the project management team is to determine the way forward based on current levels of uncertainty in the operating environment. Should they cut their losses in relation to procurement and focus on soft interventions like capacity building and knowledge transfer through in-country study tours (the success of which was evidenced in the SERCARB project) or take a risk to see if procurement requests are approved, albeit very late in the season which impacts when structural interventions can be put in place. At the moment, the project is aiming to make a final decision by the end of Q3 2018 on where to place its emphasis in terms of consolidating gains already made during the final months of the project. Although risky in terms of planning and delivery, it is important to note that the project team has a firm ‘red line’ at which point it adapts to a new scenario for implementation. It demonstrates a responsible approach to adaptive management in a highly fluid and risk-prone implementation environment.


Tag: Sustainability Procurement Project and Programme management

18.

Assessment of Cross-cutting issues

The focus of capacity building assistance. The CBDRM project does not have a capacity building strategy per se, with activities more ad hoc in nature based on needs identified during the PRNA process, with a number of trainings to improve knowledge and skills around disaster preparedness and coping at the Ri level. The focus of these trainings is at the individual level, and by default at the Ri leadership. Unlike in other contexts where a heavy focus on capacity building of the individual, rather than the system or institution, undermines the effectiveness and sustainability of capacity building activities, in the DPRK context where there is far less movement of people within and between institutions and roles in the community, there is far more retention of knowledge and skills gained, although without in-depth qualitative monitoring, or the opportunity to monitor how the community uses its new tools and skills in a crisis, it is difficult to have a clear view of the impact of the training. However, a specific capacity development strategy, which clarifies why and how such activities will take place, would be beneficial to the project and also underscore the need for more qualitative monitoring in terms of tracking knowledge retention and use.

Nonetheless, as noted above and routinely throughout this MTR, the weakness of monitoring from a qualitative perspective limits the monitoring of capacity building to numbers of people trained and types of tools/skills provided. This does not lend itself to adequate analysis of the effectiveness of the activities, nor to the sustainability of the results, regardless of findings of the MTR. Keeping in mind possibilities for scale-up of CBDRM and other similar projects in the future, it is important that some qualitative monitoring of knowledge transfer and retention be undertaken in order to determine what works and what doesn’t in the DPRK context.


Tag: Effectiveness Knowledge management Capacity Building

19.

Assessment of Cross-cutting issues

The focus on inclusion. The project document did not detail a specific gender mainstreaming or social inclusion strategy, however, within the overall strategy of the project activities aim to promote the idea of targeting the most vulnerable in the community in preparedness and coping with disaster in line with the Sendai Framework. The main groups targeted including young children, pregnant women and the elderly. Despite the view that all community members are equal, and none are more or less vulnerable than others, the concept that perhaps some groups need more help than others in times of crises has taken hold in target Ris, which demonstrates impact at an awareness level. No specific gender-based budgeting or dedicated budget to support gender mainstreaming in the project, which could add value if the project is required to focus on soft-activities for the remainder of the project. Similarly, the absence of specific gender mainstreaming tools in use by the project is notable, but with project activities being guided by the principles of the Sendai Framework and global good practice in CBDRM, the gap in gender-sensitive and inclusive planning and budgeting is not a wide as would be expected in the absence of the use of such tools. However, at the monitoring stage, gender and inclusion data collected is very weak, providing only sex disaggregated data and not digging deeper to understand the differing views and experiences of men and women in the implementation of project activities. If the project is able to implement additional qualitative indicators, as recommended above, it is anticipated that this gap can be partially filled.


Tag: Vulnerable Efficiency Gender Mainstreaming Youth Data and Statistics

Recommendations
1

Improved Monitoring

The MTR has frequently noted that limiting monitoring and data collection to quantitative approaches only undermines the ability of the project to capture the qualitative change created and the potential impact of the project in the short and medium term. Providing analysis of qualitative change can also demonstrate the importance of the project despite the significant operational challenges, not least procurement challenges, which have caused delay in the implementation of some activities. The MTR recommends including a number of qualitative indicators at the sub-output level.

2

Standardized monitoring tools

A standardized quarterly monitoring report should be used to consolidate data from the BTORs on a quarterly basis only, providing ease in data analysis. The report should be completed by the project team (lead by the Project Manager), with quality assurance of the data and analysis undertaken by M&E Specialist. This also provides a clear delineation between the role of the project and programme in monitoring and reporting at the projectlevel.

3

Communication of project results

With the inclusion of more qualitative indicators at the output level, it is hoped that more meaningful analysis of the humanitarian importance of the project will be captured, and it is recommended that the UNDP Country Office put significantly more effort into communicating these results within the wider UN system in order to reinforce why UNDP’s presence in DPRK is essential.

4

Managing community expectations

While plans for structural interventions were agreed with target communities, delays in procurement undermine community commitment and ownership to the initiatives. The project needs to find a way to better manage community expectations related to structural interventions, perhaps by only discussing these plans once procurement is approved based on previous needs assessments.

5

Consolidating CBDRM commitment at the national level

The sustainability of current results and possible future scale-up of CBDRM relies heavily on the capacity of SCEDM to take ownership of DRM coordination in the country. It is recommended that the project team facilitate more knowledge transfer and leadership skills to SCEDM, using the CBDRM Programme Framework as a launching point for improved coordination of the cross-cutting sector.

6

Exit strategy

It is recommended that UNDP identify an agency to take over the responsibility for coordinating the CBDRM Programme Framework after the project is complete as it is unlikely that SCEDM capacity to take on that role will be sufficient by the time the project ends, as well as work closely with participating counties for the formal handover of products such as the DLDD and CBDRM Programme Framework for the improved ownership and continued learning of county officials related to risks, vulnerabilities and community-based disaster risk management.

1. Recommendation:

Improved Monitoring

The MTR has frequently noted that limiting monitoring and data collection to quantitative approaches only undermines the ability of the project to capture the qualitative change created and the potential impact of the project in the short and medium term. Providing analysis of qualitative change can also demonstrate the importance of the project despite the significant operational challenges, not least procurement challenges, which have caused delay in the implementation of some activities. The MTR recommends including a number of qualitative indicators at the sub-output level.

Management Response: [Added: 2018/10/03] [Last Updated: 2020/12/13]

It is important some qualitative output indicators are considered to improve reporting on the qualitative changes. Providing analysis of qualitative change can also demonstrate the importance of the project despite the significant operational challenges, not least procurement challenges, which have caused delay in the implementation of some activities.

Key Actions:

Key Action Responsible DueDate Status Comments Documents
1.1. In consultation with counterparts following indicators will be monitored by CBDRM project. These will be reported in the QPRs/APR. a) DRM Plans are reviewed annually by the communities and financially supported for implementation b) Usefulness of Disaster Loss and Damage Database (DLDD) reports for DRM planning at three counties
[Added: 2018/10/03] [Last Updated: 2019/09/25]
CBDRM Project team SCEDM 2019/09 Completed DRM Plans reviewed annually by the communities and implemented as needed and Disaster Loss and Damage Database (DLDD) in finalization process which will inform better disaster preparedness. History
1.2. At the programme level, Programme staff report qualitative changes of the projects in ROAR through captured qualitative results from projects.
[Added: 2018/10/03] [Last Updated: 2019/09/15]
Programme Analyst, M&E Specialist 2019/09 Completed As of September 2019, the project produced 9 stories capturing qualitative results from the project activities on ground. History
2. Recommendation:

Standardized monitoring tools

A standardized quarterly monitoring report should be used to consolidate data from the BTORs on a quarterly basis only, providing ease in data analysis. The report should be completed by the project team (lead by the Project Manager), with quality assurance of the data and analysis undertaken by M&E Specialist. This also provides a clear delineation between the role of the project and programme in monitoring and reporting at the projectlevel.

Management Response: [Added: 2018/10/03] [Last Updated: 2020/12/13]

It is recommended that instead of having joint reports following field visits, whether it is for implementation and/or monitoring purposes, team members should submit individual BTORs separately for project and programme.

Key Actions:

Key Action Responsible DueDate Status Comments Documents
2.1 Projects and programme team will submit separate BTORs upon field missions.
[Added: 2018/10/03] [Last Updated: 2019/09/15]
CBDRM Project team 2019/09 Completed BTORs are submitted separately History
2.2 Quarterly Progress Reports (QPRs) should be completed by the project team (led by the Project Manager), with quality assurance of the data and analysis undertaken.
[Added: 2018/10/03] [Last Updated: 2019/09/25]
CBDRM Project team/PA/M&E Specialist 2019/09 Completed Implemented by project team History
2.3 Report qualitative changes of the projects in ROAR through captured qualitative results from projects.
[Added: 2018/10/03] [Last Updated: 2019/09/15]
CBDRM Project team/PA/M&E Specialist 2019/09 Completed CBDRM project is capturing qualitative changes on ground through its activities. These were reported in 2018 ROAR As of September 2019, the project produced 9 stories capturing qualitative results from the project activities on ground. History
3. Recommendation:

Communication of project results

With the inclusion of more qualitative indicators at the output level, it is hoped that more meaningful analysis of the humanitarian importance of the project will be captured, and it is recommended that the UNDP Country Office put significantly more effort into communicating these results within the wider UN system in order to reinforce why UNDP’s presence in DPRK is essential.

Management Response: [Added: 2018/10/03] [Last Updated: 2020/12/13]

With the inclusion of more qualitative indicators at the output level, it is hoped that more meaningful analysis of the humanitarian importance of the project will be captured, bearing in mind the sensitivities in sharing project results publicly due to the complex geopolitical context under which UNDP operates in DPRK. Following are some of the key actions that will be taken to improve the reporting of qualitative changes that the project is leading on the ground.

Key Actions:

Key Action Responsible DueDate Status Comments Documents
3.1 CBDRM project to share communication material (videos/brochures…) with relevant parties including BRH
[Added: 2018/10/03] [Last Updated: 2019/09/25]
CBDRM Project team/PA/ UNDP DPRK CO Communications focal point 2019/12 Completed The project team prepared communication document and will make public when condition allows History
3.2 Conduct a painting competition for school children to deepen their awareness of DRM among people and select most innovative and creative ideas.
[Added: 2018/10/03] [Last Updated: 2019/09/15]
Programme and CBDRM Project team 2019/06 Completed Completed: all the shortlisted paintings were used in 2019 calendar. History
4. Recommendation:

Managing community expectations

While plans for structural interventions were agreed with target communities, delays in procurement undermine community commitment and ownership to the initiatives. The project needs to find a way to better manage community expectations related to structural interventions, perhaps by only discussing these plans once procurement is approved based on previous needs assessments.

Management Response: [Added: 2018/10/03] [Last Updated: 2020/12/13]

The project needs to find a way to better manage community expectations related to structural interventions, perhaps by only discussing these plans once procurement is approved based on previous needs assessments.

Key Actions:

Key Action Responsible DueDate Status Comments Documents
4.1 Until sanctions issue is resolved, CBDRM project is expected to be engaged in soft activities rather than hard/structural interventions
[Added: 2018/10/03] [Last Updated: 2019/09/25]
PA and CBDRM Project team 2019/09 Completed More soft activities are on-going and international procurement in progress History
4.2 If it comes to the point when procurement is no longer possible, exit strategy would be enforced as had been recommended
[Added: 2018/10/03] [Last Updated: 2019/09/25]
PA and CBDRM Project team 2019/09 Completed International procurement in progress and exit strategy in preparation History
4.3 Project team must be in constant touch with local communities and keep them updated of the project progress in procurement
[Added: 2018/10/03] [Last Updated: 2019/09/25]
Programme and CBDRM Project team 2019/09 Completed Implemented History
5. Recommendation:

Consolidating CBDRM commitment at the national level

The sustainability of current results and possible future scale-up of CBDRM relies heavily on the capacity of SCEDM to take ownership of DRM coordination in the country. It is recommended that the project team facilitate more knowledge transfer and leadership skills to SCEDM, using the CBDRM Programme Framework as a launching point for improved coordination of the cross-cutting sector.

Management Response: [Added: 2018/10/03] [Last Updated: 2020/12/13]

It is recommended that the project team facilitate more knowledge transfer and leadership skills to SCEDM, using the CBDRM Programme Framework as a launching point for improved coordination of the cross-cutting sector

Key Actions:

Key Action Responsible DueDate Status Comments Documents
5.1 SCEDM and partners endorse the CBDRM Programme Framework as the main tool for the coordination of DRM activities at community level. It’s included as part of DRR/DRM strategy of DPRK.
[Added: 2018/10/03] [Last Updated: 2019/09/25]
SCEDM MOLEP UNDP 2019/09 Completed SCEDM and partners endorsed the CBDRM Programme Framework which contributed to the DRR/DRM strategy History
6. Recommendation:

Exit strategy

It is recommended that UNDP identify an agency to take over the responsibility for coordinating the CBDRM Programme Framework after the project is complete as it is unlikely that SCEDM capacity to take on that role will be sufficient by the time the project ends, as well as work closely with participating counties for the formal handover of products such as the DLDD and CBDRM Programme Framework for the improved ownership and continued learning of county officials related to risks, vulnerabilities and community-based disaster risk management.

Management Response: [Added: 2018/10/03] [Last Updated: 2020/12/13]

As the CBDRM project will end by December 2019 according to the Prodoc, it’s the right time to start deploying its exit strategy to meaningfully consolidate the results achieved till date and ensure, there is a sustainability in the project activities and results.

Key Actions:

Key Action Responsible DueDate Status Comments Documents
6.1. CBDRM project shall organize a National Partners Meet to assess what was done better and what else need to be completed in fulfilling the needs of the communities. a) At this meet, CBDRM project must encourage the local communities to make in-kind contributions in the absence of procurement activities. b) Consolidating CBDRM project results till date
[Added: 2018/10/03] [Last Updated: 2019/09/15]
Programme and CBDRM Project team 2019/03 Completed Completed: National Partners Meet was conducted during 20-22 March. Follow up activities are being implemented in best possible way, despite of deadlock in procurement History

Latest Evaluations

Contact us

1 UN Plaza
DC1-20th Floor
New York, NY 10017
Tel. +1 646 781 4200
Fax. +1 646 781 4213
erc.support@undp.org