Evaluation: "Strengthening resilience and coping capacities in the Caribbean through integrated early warning systems

Report Cover Image
Evaluation Plan:
2017-2021, Barbados
Evaluation Type:
Final Project
Planned End Date:
06/2017
Completion Date:
02/2017
Status:
Completed
Management Response:
Yes
Evaluation Budget(US $):
18,000

Share

Document Type Language Size Status Downloads
Download document EWS Final Evaluation Report .pdf report English 1374.68 KB Posted 332
Download document TOR EWS II Evaluation II revised.pdf tor English 171.74 KB Posted 328
Title Evaluation: "Strengthening resilience and coping capacities in the Caribbean through integrated early warning systems
Atlas Project Number: 00088003
Evaluation Plan: 2017-2021, Barbados
Evaluation Type: Final Project
Status: Completed
Completion Date: 02/2017
Planned End Date: 06/2017
Management Response: Yes
Focus Area:
  • 1. Crisis Prevention & Recovery
  • 2. Others
Corporate Outcome and Output (UNDP Strategic Plan 2014-2017)
  • 1. Output 5.4. Preparedness systems in place to effectively address the consequences of and response to natural hazards (e.g. geo-physical and climate related) and man-made crisis at all levels of government and community
SDG Goal
  • Goal 1. End poverty in all its forms everywhere
  • Goal 13. Take urgent action to combat climate change and its impacts
SDG Target
  • 1.5 By 2030, build the resilience of the poor and those in vulnerable situations and reduce their exposure and vulnerability to climate-related extreme events and other economic, social and environmental shocks and disasters
  • 13.1 Strengthen resilience and adaptive capacity to climate-related hazards and natural disasters in all countries
  • 13.3 Improve education, awareness-raising and human and institutional capacity on climate change mitigation, adaptation, impact reduction and early warning
Evaluation Budget(US $): 18,000
Source of Funding: Project
Evaluation Expenditure(US $): 17,295
Joint Programme: No
Joint Evaluation: No
Evaluation Team members:
Name Title Email Nationality
Dinesh Aggarwal Mr dinesh.a@rediffmail.com INDIA
GEF Evaluation: No
Key Stakeholders: National Partners
Countries: BARBADOS
Lessons
1.

The lesson learnt is that creation of a knowledge platform is not sufficient to achieve a policy level decision. The decision makers need to be engaged directly and the knowledge platform can, at best, support the process of engagement. Further, to serve its purpose, the information regarding the existence of the knowledge platform needs to be communicated and promoted amongst the target audiences.


2.

The project has suffered partially due to limited coordination amongst the multiple project participants. One of the other reasons for limited achievement has been the absence of a comprehensive and elaborate work plan and activity plan for carrying out the required tasks. The project also suffered due to the lack of human resources deployed for execution of the project. The inception meetings for the project happened for each of the countries individually, but no inception of the project that would provide a comprehensive work plan / activity plan was prepared. The basic work plan given in the project document (for the purpose of budget) was used as the work plan for project execution. The project team also made use of the work plan and the procurement plans on the UNDP Intranet. The preparation of RFPs for design and installation of CAP server could be issued only in February 2016, which is purely due to oversight, in the absence of a comprehensive work plan and activity plan for implementing the project. An early action towards procurement of hardware and software would have clearly avoided the delays in execution of outcome 3 of the project.   The lesson learnt is that to ensure proper control on the execution of the project, it is necessary to have a detailed activity wise work plan.


3.

The lesson learnt is that in the absence of a detailed work plan and inception reports (including the plan for deployment of human resources) from all of the major project participants, it becomes difficult to track and monitor the progress made by the partner agencies.


Findings
1.

The design of the project was well thought out. It had the components pertaining to the pilot / demonstration (outcome 2 and outcome 3) and aimed to replicate by creating a framework, case studies, and knowledge sharing (outcome 1). One of the issues with the project design was that it was a bit over ambitious to consider that it will be possible to obtain commitment from national governments to implement CAP EWS by simply creating a knowledge platform. Getting the national governments to commit to CAP EWS requires a persistent campaign to the target audiences and the decision makers by engaging them directly. In such scheme of things, the knowledge sharing platform could have been used only as a tool.


2.

One of the other problems with the design of the project is that the indicators have been put at the outcome level and not the output level. Generally speaking, the indicators at the output level are more quantitative in nature and related to the activities (e.g. number of persons trained, number of workshops organised etc.). It is desirable to have indicators both at the output and at the outcome level. In cases where we have indicators at both the outcome level and the output level, both the qualitative and quantitative aspects are covered.


3.

In spite of substantial work to be carried out by some of the project implementation partners, there is no inception report or work plan from these project participants. The project partners could not deliver the required deliverables due to a shortage of deployment of human resources, limited coordination (amongst themselves) and the lack of a proper work plan at their end.


Recommendations
1

In the future projects, while designing the projects, wherever possible provide the indicators both at the output and outcome levels.

2

For all future projects, preparation of inception report and a detailed work plan may be made mandatory. The project document may specify the use to project management tools like Gantt Chart etc.

3

In case of involvement of multiple agencies as project partners, it is important that each one of them prepares their inception report (including the detailed work plan and schedule for deployment of human resources). This inception report needs to be aligned with the overall timelines and work plan for the project at an aggregate level.

4

The implementing agency should carry out a realistic assessment of the human resources which would be required at different times during implementation of the project. The assessment regarding the requirement of the human resources should consider the overall duration of the project. In case of shortfall in the availability of human resources, provision should be made to hire consultants to meet the shortfall.

1. Recommendation:

In the future projects, while designing the projects, wherever possible provide the indicators both at the output and outcome levels.

Management Response: [Added: 2017/07/03]

From our point of view indicators were provided at both the output and outcome level of the project. Additionally, this project contributed to the UNDAF and SPD 2012-2016 outcomes which already had measurable indicators included.

Key Actions:

Key Action Responsible DueDate Status Comments Documents
Programme teams will continue to ensure that Output and Outcome level indicators - and associated baselines, milestones and targets - are embedded in all development project results frameworks and that funded monitoring and evaluation frameworks are included to help ensure accountability for the results.
[Added: 2019/05/21]
UNDP Sub-Regional Office Programme Team 2021/12 Initiated
2. Recommendation:

For all future projects, preparation of inception report and a detailed work plan may be made mandatory. The project document may specify the use to project management tools like Gantt Chart etc.

Management Response: [Added: 2017/07/03]

The project used tools to aid project management although a Gantt Chart wasn’t specifically used. Project documents do not specify PM tools beyond corporate requirements, as these vary by project type, resources and knowledge. While agreeing that more detail should have been provided especially through the joint work between the collaborating agencies, inception reports and joint work plans were prepared and utilized throughout the project period. The CO will be partnering with agencies through an upcoming initiative and a preliminary workshop has already been convened to flesh out the detailed work plan as well as discuss and agree on the coordination mechanism.

Key Actions:

Key Action Responsible DueDate Status Comments Documents
Detailed annual work plans will be made mandatory for all development projects and all Project Management Units will be mandated to use corporate Project Management tools including ATLAS and the Planning, Monitoring and Evaluation Toolkit
[Added: 2019/05/21]
UNDP Multi-Country Office 2021/12 Initiated
3. Recommendation:

In case of involvement of multiple agencies as project partners, it is important that each one of them prepares their inception report (including the detailed work plan and schedule for deployment of human resources). This inception report needs to be aligned with the overall timelines and work plan for the project at an aggregate level.

Management Response: [Added: 2017/07/03]

This recommendation is agreeable.  It should be noted however, that joint work plans and inception reports were evident during the project. Further the CO will be partnering with agencies through an upcoming initiative and a preliminary workshop has already been convened to flesh out the detailed work plan as well as discuss and agree on the coordination mechanism. The lead and supporting agencies have been identified for each output, and a Regional Coordinator specifically with the role of ensuring active collaboration and joint implementation by the agencies.

Key Actions:

Key Action Responsible DueDate Status Comments Documents
For joint programming, costed annual and multi-year workplans will be prepared by each involved agency and these will be collectively reviewed and monitored to support accountability and efficiency.
[Added: 2019/05/21]
UNDP Multi-Country Programme Team 2021/12 Initiated
4. Recommendation:

The implementing agency should carry out a realistic assessment of the human resources which would be required at different times during implementation of the project. The assessment regarding the requirement of the human resources should consider the overall duration of the project. In case of shortfall in the availability of human resources, provision should be made to hire consultants to meet the shortfall.

Management Response: [Added: 2017/07/03]

During the project proposal phase the National Disaster Offices (NDOs) identified focal points to lead in-country implementation.  During implementation, however, in some cases, these focal points were re-assigned leaving a gap in implementation.  This gap was recognized and observed and it was for this reason that in two cases, countries’ Education Coordinators also acted as country leads for overall implementation of in-country activities.

Key Actions:

Key Action Responsible DueDate Status Comments Documents
Capacity assessments will be prioritised during project design and the Letter of Agreement Annexes will be updated annually to ensure that costed project implementation support from UNDP is accounted for in annual project planning, particularly to reflect changes in national implementation capacity.
[Added: 2019/05/21]
UNDP Multi-Country Office Programme Team 2021/12 Initiated

Latest Evaluations

Contact us

1 UN Plaza
DC1-20th Floor
New York, NY 10017
Tel. +1 646 781 4200
Fax. +1 646 781 4213
erc.support@undp.org