Disaster Risk Reduction final evaluation

Report Cover Image
Evaluation Plan:
2013-2017, Jordan
Evaluation Type:
Mid Term Project
Planned End Date:
02/2014
Completion Date:
03/2014
Status:
Completed
Management Response:
Yes
Evaluation Budget(US $):
16,714

Share

Document Type Language Size Status Downloads
Download document ToR - Evaluation of the Project 10 Oct 2013 - final.docx tor English 56.90 KB Posted 603
Download document Evaluation_Report_FINAL (3).docx report English 175.72 KB Posted 973
Title Disaster Risk Reduction final evaluation
Atlas Project Number: 00080351
Evaluation Plan: 2013-2017, Jordan
Evaluation Type: Mid Term Project
Status: Completed
Completion Date: 03/2014
Planned End Date: 02/2014
Management Response: Yes
Focus Area:
  • 1. Others
Corporate Outcome and Output (UNDP Strategic Plan 2014-2017)
  • 1. Output 5.4. Preparedness systems in place to effectively address the consequences of and response to natural hazards (e.g. geo-physical and climate related) and man-made crisis at all levels of government and community
Evaluation Budget(US $): 16,714
Source of Funding: C/S
Joint Programme: No
Joint Evaluation: No
Evaluation Team members:
Name Title Email Nationality
Fadi H Hamdan Mr. fhamdan@drmcentre.com
GEF Evaluation: No
Key Stakeholders: ASEZA, PDTRA, JNBC & JCD
Countries: JORDAN
Lessons
Findings
Recommendations
1 Awareness raising of local officials and building demand at the local level: ASEZA and PDTRA have established DRR unit, with varying degrees of staffing and legislation. More work and awareness raising is required to create and consolidate demand at the local level regarding the importance of working on the following issues: o The creation of a DRM district committee. o Mainstreaming DRM and CCA into the activities of the authority, while accounting for variations in exposure, vulnerability and risk according to gender considerations. o Modifying budget lines to account for expenditure on various aspects of DRR under various budget categories o Strengthening linkages between DRM (including collation and analysis of extensive and intensive disaster losses), CCA, and poverty analysis and poverty reduction efforts.
2 Importance of institutions and legislation at the national level: The Jordan Civil Defense is mandated with all matters related to DRM, including the drafting of a national strategy for Disaster Risk Management. It is envisaged that the strategy drafted by the JCD, or any other institution in case of a change of mandates since carrying out the interviews, will have national, institutional, sectoral and local / governorate components. The UNDP could engage the JCD, and other relevant mandated institutions, as an implementing partner in future projects in order to provide technical assistance in the refinement of the strategy before it is issued
3 Importance of the roles of a Chief Technical Advisor (CTA) at the national and local levels: whether in the design of the project document or during its implementation, the project document (and the outcome, outputs and activities within) could have benefitted from the presence of a Chief Technical Advisor (CTA), able to advise staff at the national or local levels as challenges arise and necessary modifications to the original design of the project document become inevitable. Such an approach during the design and implementation of the project document could have contributed to improved design and implementation of the project document
4 Importance of the role of Project Assurance: the project could have benefitted from the presence of a personnel related to Project Assurance to ensure accuracy of results reporting, monitoring and evaluation. However, due to staff changes this post left vacant during a critical part of the project progress
5 Importance of the role of Project Technical Committee: the project technical committees for the three outputs played a very important role in taking decisions as the project progressed. These decisions included analyzing progress, approving terms of reference for consultants and analyzing and identifying needs for deviating from original set of activities. Many of the challenges identified in this report are related to decisions taken by the project technical committee, perhaps in response to emerging challenges. In order to better scrutinize the decision making process of the project technical committees, there is a need to keep minutes of the discussion which take place during meetings of these committees including the various alternatives which are considered and the reasoning for selecting a particular alternative
6 The design of the project document should ensure that future projects working on linkages between DRR, CCA, poverty analysis and poverty reduction have an integral element related to the collation of disaster losses and their analysis based on extensive and intensive disaster loss considerations
7 The training should be informed by a training needs assessment which would identify the training needs required in order to adapt and implementing best practices worldwide including those related to tracking and allocating funds for DRR, development of balanced DRM portfolios between corrective, prospective and compensatory DRM strategies, balanced portfolio for financing DRM strategies, private sector role and practices in risk transfer, amongst others.
8 The project design should link as much as possible, and try to influence if possible, the national DRM strategy being pioneered by the JCD. This would help ensure direct correspondence between local and national priorities and would help enlist the help of the JCD, as the main body mandated with DRM, in case of resistance to change from local bodies and authorities
9 The project design should as much as possible try to link to corrective disaster risk management strategies in national critical infrastructures (i.e. education, civil Defence and health) as well as illegal settlements and poor neighborhoods where recent research has shown the corrective risk management strategies are most effective when applied to the most vulnerable 20% of the population
10 There is a need for a stronger role for a CTA during project implementation, and participation in the different Project technical Committees, thus providing valuable assistance and support to the roles of the PA and the NPMA
11 The project modifications and Terms of Reference (ToRs) to consultants other than the Implementing partners should be reviewed by a CTA
12 Thorough follow-up on the main activities that can have an impact on the successful achievement of outputs and outcomes of the project, which should already have been identified by the Risk Matrix in the project document itself. For example, the project under consideration could have benefited from a more detailed risk matrix identifying that the lack of adoption of special ordinances and lack of setup of a DRM district committee would jeopardise efforts to integrate and mainstream DRR and CCA activities into the functioning of ASEZA and PDTRA while accounting for gender considerations.
13 Thorough follow up is required during project implementation to ensure that initiatives for DRR and CCA linkages to poverty analysis and poverty reduction are linked in an effective manner to improving resilience of livelihoods especially in poorer and most vulnerable communities while accounting for gender considerations
14 Thorough follow up is required during project implementation to ensure that corrective disaster risk management strategies are effectively targeting the most vulnerable sectors, communities and urban neighbourhoods
15 Stronger linkages within outputs for the protection of development and poverty reduction gains, through the focus on corrective strategies targeting the most vulnerable communities and improved capabilities for disaster loss collation and analysis. In this regard, it should be recognized that international experience demonstrates that corrective disaster risk management strategies, while costly in comparison to prospective disaster risk management strategies, are most cost effective when targeting the most vulnerable 20% of the population and corresponding infrastructure in terms of human settlements and their infrastructures and livelihoods.
16 Stronger linkages within outputs for disaster risk governance which delineates roles and responsibilities for disaster risk into four main stages (risk pre-appraisal, societal and technical risk assessments, risk evaluation and risk management all pillared around a two way risk communication strategy). Indeed recent international experience by UN agencies among other international NGOs working on DRM reveals that vulnerability can only be reduced by addressing inequality in exposure, vulnerability, risk transfer mechanisms all of which are related to inequality in access to decision making process related to DRM (e.g. for more information the GAR 13 [ ] report addresses this issue in detail).
17 Stronger focus within outputs on understanding risk transfer mechanisms by the private sector, including in particular the real estate and construction sectors. Indeed best practice guidelines as embodied by the GAR 13 report [i] stresses the importance of focusing on understanding risk transfer mechanisms to better comprehend how the decision making process related to DRM can be adapted to ensure more equal risk transfer practices.
18 Minimal if any resources are currently allocated for disaster recovery, as defined in the UNISDR terminology and reproduced in the definitions section at the beginning of this report. Indeed, examination of answers on recovery preparations within the national and local (Petra and Aqaba) HFA reporting templates shows that differences between recovery and response are not properly understood. In particular, the linkages of recovery to livelihoods and the development process in general are not well understood even though these are necessary in order to improve synergies between DRR, CCA, poverty analysis and poverty reduction efforts. Indeed, in the absence of comprehensive corrective strategies for DRM (which is in line with international best practice that recommends targeting the 20% most vulnerable settlements, communities and sectors), recovery planning becomes more important to prevent risk from accumulating again in the wake of any disaster. Developing and delineating responsibilities for recovery strategies and policies will have a direct impact on poverty reduction efforts, and may be addressed by considering the following activities: o Development of explicit mandates at the national, sectoral and local level for the development and updating of recovery plans. o Building capacities at the national, sectoral and local levels for the development of recovery plans, including plans for recovery in the wake of losses corresponding to extensive risks. o Allocation for funds for the development of recovery plans. o Creation of a recovery category for budget expenditure under DRM
19 Progress in all the above areas is dependent to a large extent on the ability of the project document and related staff to engage the JCD so that the above is reflected within the national strategy for DRM currently being prepared by the JCD.
20 Ensure that quality criteria and quality methods identified in the quality management framework for project activity results do measure the impact of activities and not solely related to input-based deliverables (e.g. the reports delivered, the strategies and policies developed, etc)
21 Provide a more detailed analysis of the impact of various risks within the risk analysis matrix both within the initial project document and in the course of the monitoring and ongoing evaluation of project activities
22 Update the training needs assessment at several junctures during project implementation to ensure that capacities required for carrying out activities necessary for improving project impact as listed above are being acquired in a suitable manner
23 Ensure that training and capacities needs assessment, referred to above, are carried out in a participatory manner that accounts for variations in exposure, hazard and risk according to gender considerations
24 Follow up on the implementation of the roles and responsibilities of any DRM and DRR units at both local and national levels, together with DRM district committees at the local level, to ensure that DRR and CCA integration is being carried out in a manner that accounts for gender consideration and provide advice on how best to ensure that when necessary
1. Recommendation: Awareness raising of local officials and building demand at the local level: ASEZA and PDTRA have established DRR unit, with varying degrees of staffing and legislation. More work and awareness raising is required to create and consolidate demand at the local level regarding the importance of working on the following issues: o The creation of a DRM district committee. o Mainstreaming DRM and CCA into the activities of the authority, while accounting for variations in exposure, vulnerability and risk according to gender considerations. o Modifying budget lines to account for expenditure on various aspects of DRR under various budget categories o Strengthening linkages between DRM (including collation and analysis of extensive and intensive disaster losses), CCA, and poverty analysis and poverty reduction efforts.
Management Response: [Added: 2014/04/23] [Last Updated: 2014/04/23]

Agree to a certain extent

Key Actions:

2. Recommendation: Importance of institutions and legislation at the national level: The Jordan Civil Defense is mandated with all matters related to DRM, including the drafting of a national strategy for Disaster Risk Management. It is envisaged that the strategy drafted by the JCD, or any other institution in case of a change of mandates since carrying out the interviews, will have national, institutional, sectoral and local / governorate components. The UNDP could engage the JCD, and other relevant mandated institutions, as an implementing partner in future projects in order to provide technical assistance in the refinement of the strategy before it is issued
Management Response: [Added: 2014/04/23] [Last Updated: 2014/04/23]

agree

Key Actions:

3. Recommendation: Importance of the roles of a Chief Technical Advisor (CTA) at the national and local levels: whether in the design of the project document or during its implementation, the project document (and the outcome, outputs and activities within) could have benefitted from the presence of a Chief Technical Advisor (CTA), able to advise staff at the national or local levels as challenges arise and necessary modifications to the original design of the project document become inevitable. Such an approach during the design and implementation of the project document could have contributed to improved design and implementation of the project document
Management Response: [Added: 2014/04/23] [Last Updated: 2014/04/23]

agree

Key Actions:

4. Recommendation: Importance of the role of Project Assurance: the project could have benefitted from the presence of a personnel related to Project Assurance to ensure accuracy of results reporting, monitoring and evaluation. However, due to staff changes this post left vacant during a critical part of the project progress
Management Response: [Added: 2014/04/23] [Last Updated: 2014/04/23]

Agree Usually the project assurance is the responsibility of the project board, and many cases this is delegated to a staff from the CO

Key Actions:

5. Recommendation: Importance of the role of Project Technical Committee: the project technical committees for the three outputs played a very important role in taking decisions as the project progressed. These decisions included analyzing progress, approving terms of reference for consultants and analyzing and identifying needs for deviating from original set of activities. Many of the challenges identified in this report are related to decisions taken by the project technical committee, perhaps in response to emerging challenges. In order to better scrutinize the decision making process of the project technical committees, there is a need to keep minutes of the discussion which take place during meetings of these committees including the various alternatives which are considered and the reasoning for selecting a particular alternative
Management Response: [Added: 2014/04/23] [Last Updated: 2014/04/23]

agree

Key Actions:

6. Recommendation: The design of the project document should ensure that future projects working on linkages between DRR, CCA, poverty analysis and poverty reduction have an integral element related to the collation of disaster losses and their analysis based on extensive and intensive disaster loss considerations
Management Response: [Added: 2014/04/23] [Last Updated: 2014/04/23]

Will adopt the approach in future interventions

Key Actions:

7. Recommendation: The training should be informed by a training needs assessment which would identify the training needs required in order to adapt and implementing best practices worldwide including those related to tracking and allocating funds for DRR, development of balanced DRM portfolios between corrective, prospective and compensatory DRM strategies, balanced portfolio for financing DRM strategies, private sector role and practices in risk transfer, amongst others.
Management Response: [Added: 2014/04/23] [Last Updated: 2014/04/23]

Future projects will enforce this methodology

Key Actions:

8. Recommendation: The project design should link as much as possible, and try to influence if possible, the national DRM strategy being pioneered by the JCD. This would help ensure direct correspondence between local and national priorities and would help enlist the help of the JCD, as the main body mandated with DRM, in case of resistance to change from local bodies and authorities
Management Response: [Added: 2014/04/23] [Last Updated: 2014/04/23]

We agree to this recommendation, and will factor it into a new possible phase

Key Actions:

9. Recommendation: The project design should as much as possible try to link to corrective disaster risk management strategies in national critical infrastructures (i.e. education, civil Defence and health) as well as illegal settlements and poor neighborhoods where recent research has shown the corrective risk management strategies are most effective when applied to the most vulnerable 20% of the population
Management Response: [Added: 2014/04/23] [Last Updated: 2014/04/23]

Will pay due diligence in future projects/ interventions

Key Actions:

10. Recommendation: There is a need for a stronger role for a CTA during project implementation, and participation in the different Project technical Committees, thus providing valuable assistance and support to the roles of the PA and the NPMA
Management Response: [Added: 2014/04/23] [Last Updated: 2014/04/23]

agree

Key Actions:

11. Recommendation: The project modifications and Terms of Reference (ToRs) to consultants other than the Implementing partners should be reviewed by a CTA
Management Response: [Added: 2014/04/23] [Last Updated: 2014/04/23]

Agree, was adhered by partially in the subject project and will insure full compliance in the future

Key Actions:

12. Recommendation: Thorough follow-up on the main activities that can have an impact on the successful achievement of outputs and outcomes of the project, which should already have been identified by the Risk Matrix in the project document itself. For example, the project under consideration could have benefited from a more detailed risk matrix identifying that the lack of adoption of special ordinances and lack of setup of a DRM district committee would jeopardise efforts to integrate and mainstream DRR and CCA activities into the functioning of ASEZA and PDTRA while accounting for gender considerations.
Management Response: [Added: 2014/04/23] [Last Updated: 2014/04/23]

not necessarily

Key Actions:

13. Recommendation: Thorough follow up is required during project implementation to ensure that initiatives for DRR and CCA linkages to poverty analysis and poverty reduction are linked in an effective manner to improving resilience of livelihoods especially in poorer and most vulnerable communities while accounting for gender considerations
Management Response: [Added: 2014/04/23] [Last Updated: 2014/04/23]

Agree, we believe that follow up has been and will continue to be an important role of the project management team supported by the country office, however in the absence of a national entity that is mandated by DRM and mainstreaming with a lens on linkages with CCA impact would be difficult to achieve

Key Actions:

14. Recommendation: Thorough follow up is required during project implementation to ensure that corrective disaster risk management strategies are effectively targeting the most vulnerable sectors, communities and urban neighbourhoods
Management Response: [Added: 2014/04/23] [Last Updated: 2014/04/23]

agree with the above reservation

Key Actions:

15. Recommendation: Stronger linkages within outputs for the protection of development and poverty reduction gains, through the focus on corrective strategies targeting the most vulnerable communities and improved capabilities for disaster loss collation and analysis. In this regard, it should be recognized that international experience demonstrates that corrective disaster risk management strategies, while costly in comparison to prospective disaster risk management strategies, are most cost effective when targeting the most vulnerable 20% of the population and corresponding infrastructure in terms of human settlements and their infrastructures and livelihoods.
Management Response: [Added: 2014/04/23] [Last Updated: 2014/04/23]

Agree, will adhere by this guidance in future interventions/ projects

Key Actions:

16. Recommendation: Stronger linkages within outputs for disaster risk governance which delineates roles and responsibilities for disaster risk into four main stages (risk pre-appraisal, societal and technical risk assessments, risk evaluation and risk management all pillared around a two way risk communication strategy). Indeed recent international experience by UN agencies among other international NGOs working on DRM reveals that vulnerability can only be reduced by addressing inequality in exposure, vulnerability, risk transfer mechanisms all of which are related to inequality in access to decision making process related to DRM (e.g. for more information the GAR 13 [ ] report addresses this issue in detail).
Management Response: [Added: 2014/04/23] [Last Updated: 2014/04/23]

Agree, will adhere by this guidance in future interventions/ projects

Key Actions:

17. Recommendation: Stronger focus within outputs on understanding risk transfer mechanisms by the private sector, including in particular the real estate and construction sectors. Indeed best practice guidelines as embodied by the GAR 13 report [i] stresses the importance of focusing on understanding risk transfer mechanisms to better comprehend how the decision making process related to DRM can be adapted to ensure more equal risk transfer practices.
Management Response: [Added: 2014/04/23] [Last Updated: 2014/04/23]

Agree, will adhere by this guidance in future interventions/ projects

Key Actions:

18. Recommendation: Minimal if any resources are currently allocated for disaster recovery, as defined in the UNISDR terminology and reproduced in the definitions section at the beginning of this report. Indeed, examination of answers on recovery preparations within the national and local (Petra and Aqaba) HFA reporting templates shows that differences between recovery and response are not properly understood. In particular, the linkages of recovery to livelihoods and the development process in general are not well understood even though these are necessary in order to improve synergies between DRR, CCA, poverty analysis and poverty reduction efforts. Indeed, in the absence of comprehensive corrective strategies for DRM (which is in line with international best practice that recommends targeting the 20% most vulnerable settlements, communities and sectors), recovery planning becomes more important to prevent risk from accumulating again in the wake of any disaster. Developing and delineating responsibilities for recovery strategies and policies will have a direct impact on poverty reduction efforts, and may be addressed by considering the following activities: o Development of explicit mandates at the national, sectoral and local level for the development and updating of recovery plans. o Building capacities at the national, sectoral and local levels for the development of recovery plans, including plans for recovery in the wake of losses corresponding to extensive risks. o Allocation for funds for the development of recovery plans. o Creation of a recovery category for budget expenditure under DRM
Management Response: [Added: 2014/04/23] [Last Updated: 2014/04/23]

Agree, will consider adopting the mentioned activities in future interventions/ projects

Key Actions:

19. Recommendation: Progress in all the above areas is dependent to a large extent on the ability of the project document and related staff to engage the JCD so that the above is reflected within the national strategy for DRM currently being prepared by the JCD.
Management Response: [Added: 2014/04/23] [Last Updated: 2014/04/23]

Agree, UNDP will support to its best capacity the development of the national strategy

Key Actions:

20. Recommendation: Ensure that quality criteria and quality methods identified in the quality management framework for project activity results do measure the impact of activities and not solely related to input-based deliverables (e.g. the reports delivered, the strategies and policies developed, etc)
Management Response: [Added: 2014/04/23] [Last Updated: 2014/04/23]

Agree, will further adhere by this guidance in the future

Key Actions:

21. Recommendation: Provide a more detailed analysis of the impact of various risks within the risk analysis matrix both within the initial project document and in the course of the monitoring and ongoing evaluation of project activities
Management Response: [Added: 2014/04/23] [Last Updated: 2014/04/23]

Agree, will try to the best ability to adhere by this guidance within the framework of future projects, will encourage other stakeholders to follow course

Key Actions:

22. Recommendation: Update the training needs assessment at several junctures during project implementation to ensure that capacities required for carrying out activities necessary for improving project impact as listed above are being acquired in a suitable manner
Management Response: [Added: 2014/04/23] [Last Updated: 2014/04/23]

Agree, will follow this guidance in future projects/ interventions

Key Actions:

23. Recommendation: Ensure that training and capacities needs assessment, referred to above, are carried out in a participatory manner that accounts for variations in exposure, hazard and risk according to gender considerations
Management Response: [Added: 2014/04/23] [Last Updated: 2014/04/23]

Agree, will follow this guidance in future projects/ interventions

Key Actions:

24. Recommendation: Follow up on the implementation of the roles and responsibilities of any DRM and DRR units at both local and national levels, together with DRM district committees at the local level, to ensure that DRR and CCA integration is being carried out in a manner that accounts for gender consideration and provide advice on how best to ensure that when necessary
Management Response: [Added: 2014/04/23] [Last Updated: 2014/04/23]

Agree, will follow this guidance in future projects/ interventions

Key Actions:

Latest Evaluations

Contact us

1 UN Plaza
DC1-20th Floor
New York, NY 10017
Tel. +1 646 781 4200
Fax. +1 646 781 4213
erc.support@undp.org