MID-TERM EVALUATION REPORT FOR UNDP/AF PROJECT "DEVELOPING CLIMATE RESILIENT FLOOD AND FLASH FLOOD MANAGEMENT PRACTICES TO PROTECT VULNERABLE COMMUNITIES OF GEORGIA" PROJECT

Report Cover Image
Evaluation Plan:
2011-2015, Georgia
Evaluation Type:
Mid Term Project
Planned End Date:
12/2014
Completion Date:
11/2015
Status:
Completed
Management Response:
Yes
Evaluation Budget(US $):
15,000

Share

Document Type Language Size Status Downloads
Download document TOR_mid-evaluation-Rioni flood-Nov2015.docx tor English 46.38 KB Posted 395
Download document Evaluation report-mid-Rioni Flood-Nov2015.pdf report English 985.41 KB Posted 273
Title MID-TERM EVALUATION REPORT FOR UNDP/AF PROJECT "DEVELOPING CLIMATE RESILIENT FLOOD AND FLASH FLOOD MANAGEMENT PRACTICES TO PROTECT VULNERABLE COMMUNITIES OF GEORGIA" PROJECT
Atlas Project Number: 00076540
Evaluation Plan: 2011-2015, Georgia
Evaluation Type: Mid Term Project
Status: Completed
Completion Date: 11/2015
Planned End Date: 12/2014
Management Response: Yes
Focus Area:
  • 1. Others
Corporate Outcome and Output (UNDP Strategic Plan 2014-2017)
  • 1. Output 5.4. Preparedness systems in place to effectively address the consequences of and response to natural hazards (e.g. geo-physical and climate related) and man-made crisis at all levels of government and community
Evaluation Budget(US $): 15,000
Source of Funding: Adaptation Fund
Joint Programme: No
Joint Evaluation: No
Evaluation Team members:
Name Title Email Nationality
Edward Russell Independent evaluator
Kate Skhireli Independent evaluator GEORGIA, REPUBLIC OF
GEF Evaluation: No
Key Stakeholders:
Countries: GEORGIA, REPUBLIC OF
Lessons
Findings
Recommendations
1 1. Project Design ? Revisions to logframe output indicators in line with changes suggested in Table 5 of MRER. Particularly indicators: 1.1.1; 1.4.1; 1.5.1; 2.1.2; 2.2.1; 2.4.1; 3.3.1; 3.4 and 3.4.1 are suggested to be changed.
2 2. Administration and Project Management ? Measures to address challenge associated with procurement of national service providers (like builders) should be implemented as soon as possible. These could include: The possible pairing of targeted national service providers with experienced service providers from the region; pre-targeting of potentially qualified national service providers and holding short information workshops that address UNDP procurement requirements and other elements linked to the work; and the very advanced planning of activities by the PMU to allow for delays and allow sufficient time for workshops and support processes.
3 3. Project Management ? Urgently develop an M&E Plan for the project that covers all elements including the investment elements.
4 4. Project Management ? Involve wider range of key partners more directly in assessment of technical documents produced for the project. This could be approached through short focused workshops (after circulation) that cover the technical reports.
5 5. Adaptive Management Intervention ? In addition to the already envisaged training/capacity building planned for 2015, and additional capacity building/training effort for key project partners that includes assessment element to build confidence.
6 6. Adaptive Management ? Risk log need to be updated and consideration given to the suggestions made in Table 8.
7 7. Project Implementation ? The main challenge facing the project relates to on-the-ground implementation. Therefore recommend identifying and supporting a highly regarded local project ?champion? to promote the project and its activities at local (municipal) level.
8 8. Project Implementation ? Recommend a ?dry run? simulation of a crisis that will test all elements of the system that the project is contributing to. This will clearly address the integration and demonstrate the real utility of the products developed through the project.
1. Recommendation: 1. Project Design ? Revisions to logframe output indicators in line with changes suggested in Table 5 of MRER. Particularly indicators: 1.1.1; 1.4.1; 1.5.1; 2.1.2; 2.2.1; 2.4.1; 3.3.1; 3.4 and 3.4.1 are suggested to be changed.
Management Response: [Added: 2016/01/14]

Not accepted as the project is monitored at the outcomes and objectives level

Key Actions:

2. Recommendation: 2. Administration and Project Management ? Measures to address challenge associated with procurement of national service providers (like builders) should be implemented as soon as possible. These could include: The possible pairing of targeted national service providers with experienced service providers from the region; pre-targeting of potentially qualified national service providers and holding short information workshops that address UNDP procurement requirements and other elements linked to the work; and the very advanced planning of activities by the PMU to allow for delays and allow sufficient time for workshops and support processes.
Management Response: [Added: 2016/01/14]

The project is already implementing the method of pre-targeting potentially qualified national service providers within Georgia and making them aware of the upcoming program of works. In addition, in our view, we feel that the local contractors, having been through the initial learning experience of bidding for the first contracts, are now fully aware of UNDP procurement requirements and are able to respond to tenders. UNDP procurement has held short information workshops for contractors that address UNDP procurement requirements and other elements linked to the work. This has been reflected in the relative ease of procuring the second group of sites following the initial problems. Hence, in our view, the initial problems of procuring construction firms have partially been solved and construction has already been completed on some sites. We do not agree that, in order to avoid delays in procuring builders for the remaining sites, we could look at the possibility of procuring from outside of Georgia. We think that this could, in fact, increase delays, and would additionally remove the opportunity (and project obligation) to build capacity, with international expertise input, within Georgia for contracting and undertaking FRM engineering projects of this nature.

Key Actions:

Key Action Responsible DueDate Status Comments Documents
2.1 Monitor progress of construction progress and procurement of sites (total 6) in each quarter and take action as necessary
[Added: 2016/01/14] [Last Updated: 2016/12/21]
PMU 2016/07 Completed In 1st Quarter 2015 the project had two active constructions ; Two constructions completed, add 6 ongoing by end of 2015 History
2.2 Develop list of pre-qualified regional contractors who might be of assistance if needed
[Added: 2016/01/14]
PMU 2015/03 Completed PMU keeps a list of constructors who participated in the tenders. Additionally the project collected companies contact information from the target municipalities
3. Recommendation: 3. Project Management ? Urgently develop an M&E Plan for the project that covers all elements including the investment elements.
Management Response: [Added: 2016/01/14]

Management agrees with the recommendation

Key Actions:

Key Action Responsible DueDate Status Comments Documents
3.1 Develop project M&E Plan
[Added: 2016/01/14]
Project Manager with support from PMU 2015/06 Completed The M&E plan has been developed
4. Recommendation: 4. Project Management ? Involve wider range of key partners more directly in assessment of technical documents produced for the project. This could be approached through short focused workshops (after circulation) that cover the technical reports.
Management Response: [Added: 2016/01/14]

The project has hosted a number of workshops at which technical findings have been elaborated and comments have been invited both at the workshop and on the technical document. It has been difficult to get comments on technical work outside of workshops. However, Management will make additional efforts to ensure that comments solicited by a number of different means (smaller workshops, lunchtime presentations/discussions among and between relevant agencies, questionnaires, online submission of comments, meetings with relevant staff). In addition, bi-lateral meetings will be implemented to more directly engage key stakeholders.

Key Actions:

Key Action Responsible DueDate Status Comments Documents
4.1 CTA to identify all technical documents for review and comment by key partners
[Added: 2016/01/14]
CTA & PM 2015/03 Completed Technical documents have been identified.
4.2 All technical documents of relevance to be translated and disseminated via a number of different mediums
[Added: 2016/01/14]
PM 2015/06 Completed Most technical documents are translated and disseminated among project partners.
4.3 Comments collated and translated
[Added: 2016/01/14]
PM & Comms Exp. 2015/12 Completed
5. Recommendation: 5. Adaptive Management Intervention ? In addition to the already envisaged training/capacity building planned for 2015, and additional capacity building/training effort for key project partners that includes assessment element to build confidence.
Management Response: [Added: 2016/01/14]

Additional training courses have already been conducted in GIS and two -week training courses in hydraulic modelling will be undertaken during early May and Early June. Both courses include assessment and feedback for trainees. More generally, the project will engage with MoENRP regarding their training facility and will provide ToT for the centre, will register all training provided.

Key Actions:

Key Action Responsible DueDate Status Comments Documents
5.1 Identify all additional training that can be provided under the project
[Added: 2016/01/14]
PM and CTA 2015/03 Completed Identified, GIS and hydraulic modeling trainings
5.2 Undertake already identified training in GIS
[Added: 2016/01/14]
PM and GIS Exp 2015/03 Completed
5.3 Undertake already identified training in Hydraulic Modelling
[Added: 2016/01/14]
PM and Lead Hydraulic modeller 2015/03 Completed
5.4 Second Hydrology training course
[Added: 2016/01/14]
PM & Lead Hydrologist 2015/09 Completed
5.5 Risk Model user training
[Added: 2016/01/14]
PM & GIS expert 2015/11 Completed
6. Recommendation: 6. Adaptive Management ? Risk log need to be updated and consideration given to the suggestions made in Table 8.
Management Response: [Added: 2016/01/14]

Agreed and suggestion in Table 8 accepted.

Key Actions:

Key Action Responsible DueDate Status Comments Documents
6.1 Update risk log as per Table 8 of MTE report
[Added: 2016/01/14]
PM 2015/03 Completed
6.2 Undertake quarterly review and update of risk log
[Added: 2016/01/14] [Last Updated: 2016/12/21]
PM 2016/07 Completed Risk log is quarterly reviewed History
7. Recommendation: 7. Project Implementation ? The main challenge facing the project relates to on-the-ground implementation. Therefore recommend identifying and supporting a highly regarded local project ?champion? to promote the project and its activities at local (municipal) level.
Management Response: [Added: 2016/01/14]

The project as recently recruiting a Communications Expert, who will be tasked with, among other things, communicating and promoting project achievements at all levels. The Communications Expert will develop a communications plan that will be implemented before the project completion and will consider the necessity for a local champion in each municipality and will advice on best approach for communicating the project to different audiences. In general, we agree with the suggestion to identify local project champions. However the project from its starting point of implementation selected focal points in each target municipalities who are very well familiar in the project activities and objectives and they promote and support the project on local level.

Key Actions:

Key Action Responsible DueDate Status Comments Documents
7.1 Communication Expert hired
[Added: 2016/01/14]
PMU 2015/09 Completed
7.2 Communications Plan developed and implemented
[Added: 2016/01/14]
PM and Comms Exp 2015/08 Completed Communication plan developed in August 2015, discussed and approved with PMU and CO; actions are being implemented
8. Recommendation: 8. Project Implementation ? Recommend a ?dry run? simulation of a crisis that will test all elements of the system that the project is contributing to. This will clearly address the integration and demonstrate the real utility of the products developed through the project.
Management Response: [Added: 2016/01/14]

It has been noted by the MTE that this activity was already planned under the project and included in Annex to the project inception report. Nonetheless, we accept the reminder and will undertake this activity as intended

Key Actions:

Key Action Responsible DueDate Status Comments Documents
7.1 Undertake a ?dry run? simulation of a flood emergency.
[Added: 2016/01/14] [Last Updated: 2016/12/21]
PM Supporting the coordination of the Disaster Prevention and Planning Department, Emergency Management Agency of the Ministry of Internal Affairs of Georgia. 2016/06 Completed Will be done as FFEWS completed History

Latest Evaluations

Contact us

1 UN Plaza
DC1-20th Floor
New York, NY 10017
Tel. +1 646 781 4200
Fax. +1 646 781 4213
erc.support@undp.org