Success message
error message
warn message
Strengthening Climate information and Early Warning System Terminal Evaluation
Commissioning Unit: Ethiopia
Evaluation Plan: 2017-2020
Evaluation Type: Project
Completion Date: 09/2018
Unit Responsible for providing Management Response: Ethiopia
Documents Related to overall Management Response:
 
1. Recommendation:

Project design

Identify actions to safeguard public goods

A risk analysis was undertaken during the project preparation phase and is presented in Annex I of the Prodoc. However, while NMA and HWQD emphasised a critical shortfall in O&M budgets, this sustainability risk was not included in the analysis or adequately considered in project design. Significant resources were invested in procurement and installation of hydromet observation equipment, but risk of equipment theft, vandalism and lack of routine maintenance was not addressed. While it is recognised that, once equipment is handed over to the government it becomes their responsibility, the project should have investigated and budgeted specific actions to minimise risk in the short-term, while longer-term sustainability is developed.

 

Management Response: [Added: 2018/11/28] [Last Updated: 2021/01/13]

partially agreed

Key Actions:

Key Action Responsible DueDate Status Comments Documents
Identify actions to safeguard public goods;
[Added: 2018/11/28] [Last Updated: 2020/08/03]
UNDP, NMA,and NDRMC 2019/12 Completed Completed History
Accurately identify baselines and measurable indicators;
[Added: 2018/11/28] [Last Updated: 2020/08/03]
UNDP & Government 2020/12 Completed Completed History
Include outputs and output level indicators in the PRF;
[Added: 2018/11/28] [Last Updated: 2020/08/03]
UNDP and Government 2020/12 Completed Completed History
Establish an Effective Project Management Unit;
[Added: 2018/11/28] [Last Updated: 2020/08/03]
UNDP and NMA 2020/12 Completed Completed History
Make use of the GEF tracking tool;
[Added: 2018/11/28] [Last Updated: 2020/08/03]
UNDP 2020/08 Completed Completed History
2. Recommendation:

Evaluation Recommendation or Issue 2: Actions to follow up or reinforce initial benefits from the project:

Management Response: [Added: 2018/11/28] [Last Updated: 2018/12/21]

Agreed

Key Actions:

Key Action Responsible DueDate Status Comments Documents
Identify and assess risks at hydro met sites;
[Added: 2018/11/28] [Last Updated: 2020/08/03]
UNDP, HWQD and NDMC 2019/06 Completed Completed History
Undertake community-based awareness raising of the value of public goods;
[Added: 2018/11/28] [Last Updated: 2020/08/03]
UNDP , NMA , HWQD of the MoWIE 2019/12 Completed Completed History
Undertake community-based awareness raising of the value of public goods;
[Added: 2018/11/28] [Last Updated: 2020/08/03]
UNDP, NMA , HWQD, of the MoWIE 2020/12 Completed Completed History
Maximise land-use at AWS sites;
[Added: 2018/11/28] [Last Updated: 2020/08/03]
UNDP, NMA and CEFCC 2020/12 Completed Completed History
Where feasible, employ guard at sites situated away from settlements
[Added: 2018/11/28] [Last Updated: 2020/08/03]
NMA/HWQD and NDRMC 2019/06 Completed Completed History
3. Recommendation:

Accurately identify baselines and measurable indicators

The methodology to quantify the baseline capacity assessment score, and measure progress toward the target, was not identified. While scores were assigned, they were had no meaning in the absence of a methodology. Accordingly, it was not possible to quantify progress towards the project’s objective.

Management Response: [Added: 2021/01/08] [Last Updated: 2021/01/13]

Key Actions:

4. Recommendation:

Include outputs and output level indicators in the PRF

The absence of output level indicators made it difficult to effectively monitor implementation of activities and identify adaptive management in a timely manner.

Management Response: [Added: 2021/01/08] [Last Updated: 2021/01/13]

Key Actions:

5. Recommendation:

Project implementation, monitoring & evaluation

Establish an effective Project Steering Committee

The PSC was not well attended by its appointed stakeholders and did not adequately fulfil its function. Several members never attended, two members interviewed by the evaluation team could not recollect the project and one stated they were unaware of their responsibilities, since no ToR was circulated. The PSC has not met since August 2016. The roles and responsibilities of PSC members should be clearly communicated to maximise participation and buy-in of stakeholders.

Management Response: [Added: 2021/01/08] [Last Updated: 2021/01/13]

Key Actions:

6. Recommendation:

Establish an Effective Project Management Unit

Despite budgetary allocation from LDCF, allocation of offices at NMA and procurement of office equipment, no PMU was established by the IP. Therefore, the project lacked a dedicated project manager, administrative and financial assistant, and monitoring & evaluation expert. The part-time national project co-ordinator was too busy with day-to-day activities to adequately provide project implementation oversite, and the capacity of focal points at HWQD and NDRMC was also stretched.

Management Response: [Added: 2021/01/08] [Last Updated: 2021/01/13]

Key Actions:

7. Recommendation:

Make use of the GEF tracking tool

The GEF Focal Tracking Tool was not completed at any stage of the project. It is an important M&E tool for supporting measurement of progress towards project outcomes and outputs, and for identifying requirements for adaptive management. It also supports the development of impact level indicators. Its omission from this project contributed to evaluators not being able to assign an impact rating.

Management Response: [Added: 2021/01/08] [Last Updated: 2021/01/13]

Key Actions:

8. Recommendation:

Identify and assess risks at hydromet sites

During the field mission, the evaluators learned of several incidences of theft and/or vandalism of critical hardware, including solar panels and GPRS units. Of the 32 hydrological stations with GPRS telemetry installed, only 22 were functional by the end of the project. Solar panels and/or GPRS equipment had been stolen at a minimum of four of these stations. Of the 40 AWS installed, four were not functional at the end of the project and solar panels had been stolen at a minimum of two. While the causes of malfunction of the remaining AWS is not known to the evaluators, a further two hydrological sites malfunctioned due to poor internet connectivity, and one malfunctioned due to sedimentation build up in the gauge. Multi-hazard risk assessments should be undertaken to identify broad and site-specific mitigation actions.

Management Response: [Added: 2021/01/08] [Last Updated: 2021/01/13]

Key Actions:

9. Recommendation:

Undertake community-based awareness raising of the value of public goods

Generally, there is insufficient community-level awareness of the purpose and value of hydromet observational equipment. Awareness raising campaigns should be implemented to sensitise communities to the role these public goods play in improving the accuracy of climate information and early warnings, and how this benefits civil society. By raising awareness, the incidence of theft and vandalism should be reduced. Several observers suggested an effective medium for awareness raising would be to communicate through religious and other community leaders. Inauguration ceremonies for local stakeholders have taken place at some AWS sites.  Apparently, these have been effective at raising local awareness. Opportunities for school visits to AWS and hydrological sites, and the erection of interpretative signage should also be explored.

Management Response: [Added: 2021/01/08] [Last Updated: 2021/01/13]

Key Actions:

10. Recommendation:

Where feasible, employ guard at sites situated away from settlements

For sites situated away from settlements and exposed to high risk of theft/vandalism, consideration should be given to employing guards. The minimal cost would be offset by a reduction in capital investments to replace stolen/vandalised equipment. This cost should be built in to recurrent budgets.

Management Response: [Added: 2021/01/08] [Last Updated: 2021/01/13]

Key Actions:

11. Recommendation:

Maximise land-use at AWS sites

AWS sites are situated on 1 ha land holdings and are enclosed by fencing. The majority of each plot is unused. The sites could host meteorological and climate change education programmes as an effective means of community outreach. During the field mission, the team visited Dangila AWS in Amhara. The observer, using her own resources, had established a small environmental education facility. It was extremely popular with the local community and hosted regular groups, including from the local school. There was evidence of significant impact on local awareness raising. Furthermore, by fully utilising land at AWS sites, NMA would have further leverage to push back against land claims, which have significantly increased in recent years due to escalating land prices.

Management Response: [Added: 2021/01/08] [Last Updated: 2021/01/13]

Key Actions:

12. Recommendation:

Develop a sustainable finance strategy

While the project aimed to approach the private-sector to increase fee structures with existing customers and make additional funds available from new customers, little progress was made. NMA is currently undertaking a study on payment for meteorological services. Through consultation with parallel projects, and building on global best practice, this study should be developed to identify the feasibility of implementing private-sector financial mechanisms. If private-sector financial mechanisms are found to be feasible, a sustainable finance strategy, based on these findings, should be developed and implemented.

Management Response: [Added: 2021/01/08] [Last Updated: 2021/01/13]

Key Actions:

13. Recommendation:

Develop a strategy to implement mobile phone-based agricultural advisories

The project aimed to develop a market research plan to implement mobile phone-based agricultural advisories. However, few discussions were held with Ethio Telecom — Ethiopia’s mobile telecommunications provider. A parallel ADB-Climdev funded project is undertaking a technology and needs assessment. This assessment should be developed into a strategy to identify the feasibility of implementing mobile phone-based agricultural advisories.

Management Response: [Added: 2021/01/08] [Last Updated: 2021/01/13]

Key Actions:

14. Recommendation:

Decentralise operations and maintenance to the regional level

The installation and maintenance of hydromet observational equipment is centralised at the Federal level. There is significant evidence that this is inefficient and ineffective. Capacity for O&M is weak at the regional level and federal level technicians have limited budgets to travel. Equipment failures or theft, though reported, are regularly overlooked, weakening national and regional capacity to generate climate information and early warnings. By decentralising O&M, regional authorities would be empowered as stewards of hydromet observational infrastructure and O&M costs would be minimised. Senior staff at NMA and HWQD recognise that, as Ethiopia’s climate information infrastructure continues to expand, decentralisation is a necessity. As a first step, a capacity needs assessment should be undertaken at all regional NMA and HWQD offices.

Management Response: [Added: 2021/01/08] [Last Updated: 2021/01/13]

Key Actions:

15. Recommendation:

Develop an awareness raising communications strategy

Establishing a network of hydromet observational equipment and building capacity for forecasting and early warnings are significant achievements of this project. However, influencing behavioural change to effectively use information and make informed decisions is another challenge. A communications strategy should be developed to identify opportunities for awareness raising platforms at regional, zonal and woreda levels. Attention should be paid to opportunities for engaging with schools, since building awareness in children is an effective way to communicate with adults, particularly in rural communities.

Management Response: [Added: 2021/01/08] [Last Updated: 2021/01/13]

Key Actions:

16. Recommendation:

Ensure hydrological observational equipment is fit for purpose

A number of hydrological gauge stations use shaft encoder technology which is susceptible to sedimentation build-up. Sedimentation stops these gauge stations from recording accurate measurements and regular maintenance is required during rains to ensure reliability. Since O&M capacity in Ethiopia is limited, these gauge stations may remain inactive for several months, reducing capacity to monitor river levels and issue flood advisories. Site-specific environmental conditions should be carefully considered when selecting equipment requirements.

Management Response: [Added: 2021/01/08] [Last Updated: 2021/01/13]

Key Actions:

Latest Evaluations

Contact us

1 UN Plaza
DC1-20th Floor
New York, NY 10017
Tel. +1 646 781 4200
Fax. +1 646 781 4213
erc.support@undp.org