Mid-term Evaluation: Early Warning System Project

Report Cover Image
Evaluation Plan:
2019-2023, Cambodia
Evaluation Type:
Mid Term Project
Planned End Date:
07/2019
Completion Date:
07/2019
Status:
Completed
Management Response:
Yes
Evaluation Budget(US $):
30,000

Share

Document Type Language Size Status Downloads
Download document ToR_MTR_EWS.pdf tor English 3264.96 KB Posted 29
Download document EWS - Final MTR Report.pdf report English 3549.60 KB Posted 20
Title Mid-term Evaluation: Early Warning System Project
Atlas Project Number: 00082718
Evaluation Plan: 2019-2023, Cambodia
Evaluation Type: Mid Term Project
Status: Completed
Completion Date: 07/2019
Planned End Date: 07/2019
Management Response: Yes
UNDP Signature Solution:
  • 1. Resilience
Corporate Outcome and Output (UNDP Strategic Plan 2018-2021)
  • 1. Output 3.3.1 Evidence-based assessment and planning tools and mechanisms applied to enable implementation of gender-sensitive and risk-informed prevention and preparedness to limit the impact of natural hazards and pandemics and promote peaceful, just and inclusive societies
SDG Goal
  • Goal 1. End poverty in all its forms everywhere
  • Goal 13. Take urgent action to combat climate change and its impacts
SDG Target
  • 1.5 By 2030, build the resilience of the poor and those in vulnerable situations and reduce their exposure and vulnerability to climate-related extreme events and other economic, social and environmental shocks and disasters
  • 13.1 Strengthen resilience and adaptive capacity to climate-related hazards and natural disasters in all countries
Evaluation Budget(US $): 30,000
Source of Funding: Project Budget
Evaluation Expenditure(US $): 12,890
Joint Programme: No
Joint Evaluation: No
Evaluation Team members:
Name Title Email Nationality
Hans van Noord International Consultant - Project Midterm Review in Cambodia
GEF Evaluation: No
Key Stakeholders:
Countries: CAMBODIA
Lessons
Findings
1.

1. The EW project is assessed as very relevant.  Although the overall objective of the project and the separate outcomes are well defined, in its design the order or sequencing of the outcomes could have better reflected a logical chronological order. The Theory of Change of the refinement report presented an improved sequencing of interventions. The results framework of the project is straightforward, with a limited number of indicators and targets.


2.

2. The initial start-up of the project has been very slow and problematic. The inability of the Royal Government of Cambodia to convene the Project Board together with confusion over management arrangements resulted in an almost frozen condition of the project in 2015 and 2016. The redesign and refinement approach and proposed changes in arrangements (a.o. NIM-DIM transition, a project manager with required technical expertise and experience) have enabled the project management team to revive the project, with a sharp increase in delivery in 2017 and especially 2018.


3.

3. The present project team is considered understaffed, considering the ambitious work load going into the final project phase (until May 2020), caused by the need to compensate for the considerable implementation delay in the initial ears, and the related shift of focus on proper documentation, extracting lessons and emerging good practices and knowledge management.


4.

4. The feedback of the stakeholders is reflecting their general appreciation for the support provided by project to enhance their capacity in monitoring (hardware), analysis (software), capacity building (HR-institutional) and generation of tailored climate and EW information, linked to the low – and fast on-set of climate-induced natural hazards (droughts and floods).


5.

5. The project is recently expanding its initial reach through a series of additional partnerships with national and international organisations (PIN (EWS1294), DCA (drought Info Hubs), SERVIR and RIMES etc.), which are considered strategic, time-efficient considering the limited remaining implementation period and partly aimed at replicating approaches that have been piloted and have proven value for enhancing EWS development.


6.

6. As a result, combined with additional activities identified per component, the output of the project has markedly increased, which will support attaining the original set outcomes with more confidence (e.g. establishment of rating curves for the AHSs, MAFF NAP support etc.).


7.

7. The progress of the project is assessed as satisfactory and on track to achieve the set outcomes. Key remaining areas of focus for the project in the remaining implementation period are related to activities aimed at drought assessment and related capacity building, application of hydro-meteo data in actual EWS (forecasting, info flow reflected in SOPs, training-testing) and enhancement of sustainability (HR, financial etc.)


8.

8. The Project Implementation & Adaptive Management rating is assessed as satisfactory.


9.

9. Based on the assessment of the categories above the overall sustainability rating is moderately likely. There are still moderate risks, especially with regards to O&M and overall human resource capacity. It is suggested to work out a concise exit strategy as phasing out plan for the project, identifying interventions to enhance lasting impact of the project and improve overall sustainability of the investments and interventions. Sustainability has been regarded as an issue and risk since the design phase. The O&M plan should build more confidence in availability of (sufficient) resources and technical capability of staff. Private sector engagement is being explored to mobilize additional resources for O&M, but this should not distract from the need for core budgeting by RGC.


10.

10. Absence of a dedicated academic curriculum on hydrology and meteorology in Cambodia remains a longer-term constraint to safeguard scientific support and HR capacity, and will require a medium- to longer-term approach in capacity building. Although outside of the shorter-term scope of this project, there is a clear need for additional support post-project to address this critical constraint.


11.

11. An effective and cost-efficient procurement process has allowed the project to purchase hydro-meteorological equipment at competitive pricing, as compared to other comparable projects in the country, without having to compromise quality. This is reflected in a considerable cost savings up to $600,000, allowing the project to invest in, e.g., additional monitoring hardware (e.g. an estimated 100 automatic rain gauges), extended warranty of stations, additional technical support by the suppliers and extra spare parts and additional training activities for those outcome areas where specific focus is needed


12.

12. It is noted that in the ProDoc relatively little attention is given to gender specific approaches to ensure a gender balanced implementation of the project. As the project is in its later phase of implementation and is progressing to implement more interventions at sub-national level, in districts and communities, attention to inclusiveness and gender balance in capacity building efforts is required. Suggested AMAT indicators could support monitoring a gender balanced approach.


Recommendations
1

Revise and adjust the following project logframe, indicators and set targets.

  • Outcome 1 - Indicator 1: (targeting only MOWRAM and MAFF).
  • Outcome 2 – Indicator 2: [% change of agricultural productivity], through ECCA and Random Control Trials..)
  • Outcome 2 – Indicator 3: Indicator 3: 7 provinces and agro-ecological zones.
2

Update and make use of the GEF Tracking Tool

 

3

Add staff to the project management team

4

Compile an exit strategy/phasing out plan

5

Be pragmatic with Private sector engagement in finalizing the O&M Plan. Regard this as a medium-term development (and ambitious) but focus on robustness of primary RGC budget allocation. Work in partnership in collaborating with the private sector in exploring opportunities to mobilize additional resource for O&M:

  • PIN-MOWRAM-UNDP to link with SMART;
  • NCDM (WB), MAFF, UNDP with insurers etc.
6

Plan for a Review Workshop, with all key partners, highlighting/showcasing progress and achievements, including sustainability perspective / gaps.

7

Explore opportunities to infuse experiences/knowledge on drought into MAFF.

  1. Drought study, national level assessment, identification of spatial distribution, zones with highest vulnerability
  2. Redefinition of NAP for DRR in agriculture (2019-2023)
  3. Application in curriculum development of ToT on DRAT,
  4. Trainings at Provincial level (staff, farmers)
  5. Agro-meteorological needs and translation into seasonal forecasting
  6. Linkage between DOM database and MAFF drought information need.
1. Recommendation:

Revise and adjust the following project logframe, indicators and set targets.

  • Outcome 1 - Indicator 1: (targeting only MOWRAM and MAFF).
  • Outcome 2 – Indicator 2: [% change of agricultural productivity], through ECCA and Random Control Trials..)
  • Outcome 2 – Indicator 3: Indicator 3: 7 provinces and agro-ecological zones.
Management Response: [Added: 2019/07/25] [Last Updated: 2019/07/25]

Agreed. The suggested changes are relevant to improving the quality of the indicators, and in addressing the correct roles of the stakeholders of the project. The specific indicator on agricultural productivity requires baseline data / study on the productivity before the start of the project, and it was not available. It would be changed to fit the context of project impact.

Key Actions:

Key Action Responsible DueDate Status Comments Documents
Revising the Outcome 1 - Indicator 1: (targeting only MOWRAM and MAFF), to include NCDM (SOP-DM Law, NAP for DRR) in target
[Added: 2019/07/25]
Project Team/ RTA / UNDP 2019/08 Overdue-Initiated The changes will be incorporated in the PIR process (July 2019). Formal process will be communicated in the next Project Review Board meeting.
Outcome 2 – Indicator 2: [% change of agricultural productivity], through ECCA and Random Control Trials..) Suggested revision: In absence of quantitative baseline assess possibility to conduct a survey in awareness level and knowledge of communities of climate/EW information and climate resilient agricultural practices.
[Added: 2019/07/25]
UNDP / Project Management / RTA 2019/09 Overdue-Initiated History
Outcome 2 – Indicator 3: Indicator 3: 7 provinces and agro-ecological zones. Suggested revision: Adjust to target 5 pilot provinces
[Added: 2019/07/25]
UNDP / Project Management / RTA 2019/09 Overdue-Initiated History
2. Recommendation:

Update and make use of the GEF Tracking Tool

 

Management Response: [Added: 2019/07/25] [Last Updated: 2019/07/25]

Agreed. The GEF tracking tool will be updated and referred for monitoring the progress of the project.

Key Actions:

Key Action Responsible DueDate Status Comments Documents
Update the GEF tracking tool
[Added: 2019/07/25]
UNDP/Project management 2019/08 Overdue-Initiated It is conducted the same time as PIR 2019 (July 2019).
3. Recommendation:

Add staff to the project management team

Management Response: [Added: 2019/07/25] [Last Updated: 2019/07/25]

Agreed. Two consultants have been hired to support monitoring the activities under the project and reporting the progress / impacts of all project activities

Key Actions:

Key Action Responsible DueDate Status Comments Documents
Recruiting monitoring specialist and communication specialist
[Added: 2019/07/25]
UNDP/Project management 2019/06 Completed The tasks of the two consultants covers the communication strategy of the project in highlighting the progress and impact of the EWS project, increasing the visibility of the EWS project’s partnership and detail / specific output of the project includes a compilation of case study capturing the scope, results but also challenges and learnings of the EWS project, to be used as one of the key documents in the project closing.
4. Recommendation:

Compile an exit strategy/phasing out plan

Management Response: [Added: 2019/07/25] [Last Updated: 2019/07/25]

Agreed. The phase-out plan is an important document to ensure that UNDP has the concrete plans on sustainability of intervention. While the extra staff supports described above is to ensure the documentation of various project intervention. A first draft of exit strategy has been developed on July 8, 2019 and has been shared to RTA in Bangkok, UNDP Resident Representative, Head of Programme and Programme Analyst. Discussion will be based on the draft provided, including mapping potential funding sources. 

Key Actions:

Key Action Responsible DueDate Status Comments Documents
Developing an exit strategy / phasing out plan
[Added: 2019/07/25]
UNDP/Project management 2019/10 Initiated The first draft has been developed and discussions will be carried out on mapping potential funding sources. A more detail document on strategy would be developed after getting comments internally.
5. Recommendation:

Be pragmatic with Private sector engagement in finalizing the O&M Plan. Regard this as a medium-term development (and ambitious) but focus on robustness of primary RGC budget allocation. Work in partnership in collaborating with the private sector in exploring opportunities to mobilize additional resource for O&M:

  • PIN-MOWRAM-UNDP to link with SMART;
  • NCDM (WB), MAFF, UNDP with insurers etc.
Management Response: [Added: 2019/07/25] [Last Updated: 2019/07/25]

Agreed. The project would look at both ways, exploring private sector engagement and supporting RGC in developing long-term O&M strategy.

Key Actions:

Key Action Responsible DueDate Status Comments Documents
Development of private sector engagement strategy of the project through a Consultant.
[Added: 2019/07/25]
UNDP/Project management 2019/10 Initiated The consultant is being hired and expected to be on board by 1st August 2019.
6. Recommendation:

Plan for a Review Workshop, with all key partners, highlighting/showcasing progress and achievements, including sustainability perspective / gaps.

Management Response: [Added: 2019/07/25] [Last Updated: 2019/07/25]

Agreed. A review workshop will be conducted before the end of the project period

Key Actions:

Key Action Responsible DueDate Status Comments Documents
Conducting a review workshop
[Added: 2019/07/25]
UNDP/Project management 2019/12 Initiated Review workshop will be conducted before the end of the year, based on agreement of government stakeholders.
7. Recommendation:

Explore opportunities to infuse experiences/knowledge on drought into MAFF.

  1. Drought study, national level assessment, identification of spatial distribution, zones with highest vulnerability
  2. Redefinition of NAP for DRR in agriculture (2019-2023)
  3. Application in curriculum development of ToT on DRAT,
  4. Trainings at Provincial level (staff, farmers)
  5. Agro-meteorological needs and translation into seasonal forecasting
  6. Linkage between DOM database and MAFF drought information need.
Management Response: [Added: 2019/07/25]

Agreed. 

Except of development for NAP DRR for Agriculture. An initial discussion has been discussed with MAFF, yet MAFF has not sent a formal request on this undertaking. 

Key Actions:

Key Action Responsible DueDate Status Comments Documents
Conducting a national drought study
[Added: 2019/07/25]
UNDP/Project management 2019/09 Overdue-Initiated Consultant is being hired, and expected to be on-board in August. The study is to be completed in September 2019.
Extend the agreement with Dan Church Aid on the DRAT, including provincial level training on DRAT
[Added: 2019/07/25]
UNDP/Project management 2019/12 Initiated The agreement is being initiated; all documents have been prepared; and expected to be starting in July 2019.
Development of farm curriculum, a translation of agro-meteorological needs of forecasting, including linkage between DOM database and MAFF drought information need.
[Added: 2019/07/25]
UNDP/Project management 2019/07 Completed To be translated into Khmer and tested in the first training in August 2019.

Latest Evaluations

Contact us

1 UN Plaza
DC1-20th Floor
New York, NY 10017
Tel. +1 646 781 4200
Fax. +1 646 781 4213
erc.support@undp.org