Strengthening Climate information and early warning systems terminal evaluation

Report Cover Image
Evaluation Plan:
2016-2021, Tanzania
Evaluation Type:
Final Project
Planned End Date:
08/2018
Completion Date:
08/2018
Status:
Completed
Management Response:
Yes
Evaluation Budget(US $):
35,000

Share

Document Type Language Size Status Downloads
Download document EWS Final ToRs.pdf tor English 2613.00 KB Posted 59
Download document Final TE_EWS Report_21_Aug_2018.pdf report English 1566.17 KB Posted 74
Title Strengthening Climate information and early warning systems terminal evaluation
Atlas Project Number: 00074211
Evaluation Plan: 2016-2021, Tanzania
Evaluation Type: Final Project
Status: Completed
Completion Date: 08/2018
Planned End Date: 08/2018
Management Response: Yes
Corporate Outcome and Output (UNDP Strategic Plan 2018-2021)
  • 1. Output 2.3.1 Data and risk-informed development policies, plans, systems and financing incorporate integrated and gender-responsive solutions to reduce disaster risks, enable climate change adaptation and mitigation, and prevent risk of conflict
Evaluation Budget(US $): 35,000
Source of Funding: GEF-LDCF, UNDP
Evaluation Expenditure(US $): 24,240
Joint Programme: No
Joint Evaluation: No
Evaluation Team members:
Name Title Email Nationality
Dinesh Aggarwal INDIA
Erneus Kaijage
GEF Evaluation: Yes
GEF Project Title: Strengthening climate information and early warning systems in Tanzania for climate resilient development and adaptation to climate change
Evaluation Type: Terminal Evaluation
Focal Area: Climate Change
Project Type: FSP
GEF Phase: GEF-5
GEF Project ID: 4991
PIMS Number: 5096
Key Stakeholders: Prime Minister's Office, Tanzania Meteorological Agency, Local Government Authorities
Countries: TANZANIA (UNITED REPUBLIC OF )
Lessons
1.

It would have been useful to have included in the project design the Score Card (SC) as a means of measuring progress of the project.

The project design has considered that demonstrations of benefits at local level from an effective EW, increased information and prediction ability on floods and droughts, increased agro-meteorological information, will help replicate and upscale project outcomes to other regions. As such there is nothing wrong with this consideration. However, it needs to be appreciated that preparation of reports and knowledge products alone is not sufficient for replication. Such lessons learnt, good practices and field demonstration need to be disseminated and communicated to the target audiences to achieve the objective of replication.

The project design has provided mobile phone network as a means of communicating early, the likelihood of a hazardous event and the measures required to be taken by the community to address the event. However, in the developing world, some geographical areas (including some of the locations where pilots under the projects we carried out) are still not at the level where availability of mobile telephony is still not there. For the cases where, mobile communication network is not available, the projects in the area of early warning / disaster management, in their design, need to provide for an alternative way to transmit data and to communicate ‘Early Warning’ of a likely event to the locations where mobile communication is still not available.

Data could then be transmitted by using satellite transmission. However, the cost of establishing and managing a satellite link for transmission of the data from the weather stations is quite high. As this is the case, it could potentially be incorporated into the regular operational costs of the Water Basin Authorities/Boards which are situated in areas where mobile phone services are not available.

The limited capacity of the TMA to develop weather products constrains its ability to sustain its operations. Were TMA able to create and sell such products, this would contribute to generate more consistent revenues to finance its operations.


Findings
1.

The project has successfully installed new automated weather stations, rain gauges, river gauges and hydro-met stations. The project carried out on the ground interventions at two pilot locations. The inhabitants of the pilot project sites have benefited from the project services and receive agrometeorological information for socio-economic activities such as crop farming and livestock keeping. The project has distributed 200 smart phones to small holder’s farmers in the pilot areas in Liwale and Arumeru districts. In addition, 810 small holder farmers were connected with a farmer SMS system. These activities have increased the reach of the early warning system in the targeted (pilot) areas. Radios, faxes, televisions, meetings, social media, meeting with journalist etc. were also among the means of disseminating weather information to pilot sites and other sites in the country.

Although there are some issues (e.g. non achievement of flood modelling, actual dissemination of the information/ warning of the newly created network etc.) with the achievements of the results, the project has been able to achieve most of its objectives, ‘to strengthen the climate monitoring capabilities, early warning systems and available information for responding to climate shocks and planning adaptation to climate change in Tanzania’.

The project has strengthened the climate monitoring capabilities of the country. When it comes to ‘early warning’ the project has missed on a very important output (flood modelling), Also the performance on the front of ‘strengthen availability of information for responding to climate change’ has not been that good (in-spite of the system being in place). When it comes to the objective of ‘planning adaptation to climate change’ the project has been able to successfully introduce climate change aspects in the land use planning in the two pilot areas.

The project provided the solutions in order to address the barriers towards use of ‘Early Warning Systems’ and ‘Hydro-Met Capacity’. However, due to some shortfalls in the achievement of results, there is a bit of lacking in terms of effectiveness and efficiency of delivery due to non achievement of some of the outputs (e.g. flood modelling, simulation exercise).


Recommendations
1

Generally speaking, in case of use of a Score Card (SC) as an indicator and a means for verification, the project document should include an elaboration on, how the Scores would be determined. Also, a write up should be included in the monitoring plan regarding the determination of ‘Score’ as an activity, so that it does not get missed out.

2

The project design has considered that demonstrations of benefits at local level from an effective EW, increased information and prediction ability on floods and droughts, increased agro-meteorological information, will help replicate and upscale project outcomes to other regions. As such there is nothing wrong with this consideration. However, it needs to be appreciated that preparation of reports and knowledge products alone is not sufficient for replication. Such lessons learnt, good practices and field demonstration need to be disseminated and communicated to the target audiences to achieve the objective of replication. It is recommended that project design should include the activities targeted as dissemination of the knowledge, case studies and lessons learnt to support replication and scaling up of project outcomes.

3

The measuring instruments (like gauges, sensors in the automatic weather stations etc.) supplied under the project will require periodic calibration. The infrastructure (laboratory etc.) required for this is presently not available in Tanzania, the future projects involving supply of measuring instrument/equipment should have a strategy for periodic calibration of the instruments/equipment. It can also be a part of the procurement process, wherein the calibration services are provided by the supplier of the equipment/instruments.

4

The project design has provided mobile phone network as a means of communicating early, the likelihood of a hazardous event and the measures required to be taken by the community to address the event. However, in the developing world, some geographical areas (including some of the locations where pilots under the projects were carried out) are still not at the level where availability of mobile telephone is still not there. For the cases where, mobile communication network is not available, the projects in early warning /disaster management, in their design, need to provide for an alternative way to transmit data and to communicate ‘Early Warning’ of a likely event to the locations where mobile communication is still not available. Such alternatives, may include public broadcasting, community-based radio stations etc.

5

The project could not complete the task of development and deployment of the flood forecasting models. The consultant for this activity has already been appointed. The deliverables for this will be provided by the consultant much after the end of the project. It is recommended that the project may work out the modalities for it to ensure that the quality deliverables are received, the required demonstration and training to the officials of the basin authorities is provided regarding the use of the software.

6

It is recommended to scale up the project to cover other areas of the country and the remaining seven water basins in Tanzania so that the other parts of the country can also benefit from the early warning. However, while doing so the lessons learnt from this project may be used to ensure better results. 

7

It is proposed, and TMA would in part meet its expenses by selling weather data (or weather data products). In this regard, the country may need to have a formal data policy and regulatory mechanism to facilitate commercial use/marketing of weather data and products. In this regard, a study should (or would need to) be carried out to see the kind of provisions made in different countries and draw the lessons.

8

To sustain the operations of TMA it is necessary to have consistent revenue streams through the sale of weather products. Some of the weather products which can be tapped are:

  • Data for development of weather index-based crop insurance models
  • Data for solar radiations (Solar Atlas)
  • City/location specific weather data to facilitate Energy Efficiency measures in the building sector (Green Buildings)
9

For communication the old and proven method of community-based radio stations may be explored. Although this may be comparatively capital intensive (compared to communication on mobile phone), the coverage of the impacted population would be almost 100%. Further, from sustainability point of view this is better as no recurring cost is involved (for mobile communication the telecom companies are to be paid regularly which impacts the financial sustainability).

10

From the view point of financial sustainability, one of the regular expenses is the regular bills to be paid to the telecommunication companies. The national government may explore the possibility to have policies, wherein there are no charges by the telecom companies for providing such kind of services. For example, there can be a condition in the licensing agreement to the telecom companies, wherein the services provided for disaster management and defense of the country are not chargeable.

11

In the areas where mobile phone services are not available; the project has provided for establishment of a satellite link for transmission of the data from the weather stations. The capital cost of providing the link and the recurring cost (for managing the batteries, invertors, solar PV systems etc.) is also quite high. It is recommended that in such situations, the creation of facilities with mobile service provides (telecom companies) may be explored on a cost sharing basis. This is likely to reduce both the capital cost and the recurring cost for provision of the link for data transmission. It will have an added advantage of the development of the area.

1. Recommendation:

Generally speaking, in case of use of a Score Card (SC) as an indicator and a means for verification, the project document should include an elaboration on, how the Scores would be determined. Also, a write up should be included in the monitoring plan regarding the determination of ‘Score’ as an activity, so that it does not get missed out.

Management Response: [Added: 2018/09/24]

PMO-DMD and the project management agree with this recommendation and will seek to include a Score Card as a means of measuring progress of the project. This will include clear identification of indicators against which the progress will be measured. In the current project the Score Card was not determined as an evaluation tool at the beginning of the project design.

Key Actions:

Key Action Responsible DueDate Status Comments Documents
Include the use of Score Card in the scaled-up as well as other projects.
[Added: 2018/09/24]
Environment, CC &Resilience Pillar 2019/12 Initiated Project Scale up Concept Note development is in progress
Participate in RBM training to ensure a better understanding on how monitoring is done using SC
[Added: 2018/09/24]
Environment, CC &Resilience Pillar 2019/12 Initiated Project Scale up Concept Note development is in progress
2. Recommendation:

The project design has considered that demonstrations of benefits at local level from an effective EW, increased information and prediction ability on floods and droughts, increased agro-meteorological information, will help replicate and upscale project outcomes to other regions. As such there is nothing wrong with this consideration. However, it needs to be appreciated that preparation of reports and knowledge products alone is not sufficient for replication. Such lessons learnt, good practices and field demonstration need to be disseminated and communicated to the target audiences to achieve the objective of replication. It is recommended that project design should include the activities targeted as dissemination of the knowledge, case studies and lessons learnt to support replication and scaling up of project outcomes.

Management Response: [Added: 2018/09/24]

PMO-DMD and the project management agree with this recommendation and will include the knowledge sharing activities in the scaled-up project and any other future programming.

Key Actions:

Key Action Responsible DueDate Status Comments Documents
Include knowledge sharing activities in the scaled-up as well as other new projects.
[Added: 2018/09/24]
Environment, CC &Resilience Pillar 2019/12 Initiated Project Scale up Concept Note development is in progress
3. Recommendation:

The measuring instruments (like gauges, sensors in the automatic weather stations etc.) supplied under the project will require periodic calibration. The infrastructure (laboratory etc.) required for this is presently not available in Tanzania, the future projects involving supply of measuring instrument/equipment should have a strategy for periodic calibration of the instruments/equipment. It can also be a part of the procurement process, wherein the calibration services are provided by the supplier of the equipment/instruments.

Management Response: [Added: 2018/09/24]

PMO-DMD and the project management agree with this recommendation subject to the following strong caveats: (i) that funds be available for this activity in the scaled-up project; (ii) that adding calibration requirements as part of a procurement process to source suitable supplier of the equipment/instruments does not unduly restrict competition to the detriment of value for money as per UNDP principles.

Key Actions:

Key Action Responsible DueDate Status Comments Documents
Include calibration infrastructure and services in the scaled-up project assuming adequate availability of funds.
[Added: 2018/09/24]
Environment, CC &Resilience Pillar 2020/01 Initiated Will be followed up assuming the developing Concept Note is accepted and funded.
Ensure complete specification during procurement process to source suppliers, subject to applicable competition rules (availability of expertise and value for money).
[Added: 2018/09/24]
Environment, CC &Resilience Pillar 2020/01 Initiated Will be followed up assuming the developing Concept Note is accepted and funded.
4. Recommendation:

The project design has provided mobile phone network as a means of communicating early, the likelihood of a hazardous event and the measures required to be taken by the community to address the event. However, in the developing world, some geographical areas (including some of the locations where pilots under the projects were carried out) are still not at the level where availability of mobile telephone is still not there. For the cases where, mobile communication network is not available, the projects in early warning /disaster management, in their design, need to provide for an alternative way to transmit data and to communicate ‘Early Warning’ of a likely event to the locations where mobile communication is still not available. Such alternatives, may include public broadcasting, community-based radio stations etc.

Management Response: [Added: 2018/09/24]

PMO-DMD and the project management agree with this recommendation and shall be taken un in future programming.

Key Actions:

Key Action Responsible DueDate Status Comments Documents
Alternative ways to transmit data and to communicate ‘Early Warning’ in remote areas without mobile phone networks to be considered in future programming
[Added: 2018/09/24]
Environment, CC & Resilience Pillar 2020/12 Initiated In case the new (Scale up) project materializes
5. Recommendation:

The project could not complete the task of development and deployment of the flood forecasting models. The consultant for this activity has already been appointed. The deliverables for this will be provided by the consultant much after the end of the project. It is recommended that the project may work out the modalities for it to ensure that the quality deliverables are received, the required demonstration and training to the officials of the basin authorities is provided regarding the use of the software.

Management Response: [Added: 2018/09/24]

PMO-DMD and the project management agree with this recommendation.

Key Actions:

Key Action Responsible DueDate Status Comments Documents
Follow up closely with the consultant (BRL) to ensure quality delivery of the flood and drought forecasting model.
[Added: 2018/09/24]
Environment, CC & Resilience Pillar 2019/12 Initiated Progress updates have been given by BRL on a regular basis
Organize training for relevant officials as planned.
[Added: 2018/09/24]
Environment, CC & Resilience Pillar 2019/12 Initiated
6. Recommendation:

It is recommended to scale up the project to cover other areas of the country and the remaining seven water basins in Tanzania so that the other parts of the country can also benefit from the early warning. However, while doing so the lessons learnt from this project may be used to ensure better results. 

Management Response: [Added: 2018/09/24]

PMO-DMD and the project management agree with this recommendation.

Key Actions:

Key Action Responsible DueDate Status Comments Documents
Lessons learning will inform the scaled-up project, the consultant is being guided to include this in the concept note.
[Added: 2018/09/24]
Environment, CC & Resilience Pillar 2019/12 Initiated Project Scale up Concept Note development is in progress
7. Recommendation:

It is proposed, and TMA would in part meet its expenses by selling weather data (or weather data products). In this regard, the country may need to have a formal data policy and regulatory mechanism to facilitate commercial use/marketing of weather data and products. In this regard, a study should (or would need to) be carried out to see the kind of provisions made in different countries and draw the lessons.

Management Response: [Added: 2018/09/24]

PMO-DMD and the project management agree with this recommendation subject to TMA approval’s and commitment to its implementation, which is outside of UNDP’s unilateral control.

Key Actions:

Key Action Responsible DueDate Status Comments Documents
Advocacy/Consultation with TMA to seek their endorsement and include this recommendation in the scale-up project will take place through a hired consultant.
[Added: 2018/09/24]
Environment, CC & Resilience Pillar 2019/12 Initiated Project Scale up Concept Note development is in progress
8. Recommendation:

To sustain the operations of TMA it is necessary to have consistent revenue streams through the sale of weather products. Some of the weather products which can be tapped are:

  • Data for development of weather index-based crop insurance models
  • Data for solar radiations (Solar Atlas)
  • City/location specific weather data to facilitate Energy Efficiency measures in the building sector (Green Buildings)
Management Response: [Added: 2018/09/24]

PMO-DMD and the project management take note of this recommendation but are not able to accept it because the Tanzanian Government is implementing a strong centralized revenue collection policy, which will seriously hamper the implementation of this recommendation.  UNDP will nonetheless bring this recommendation to the attention of TMA, along with recommendation 7 above.

Key Actions:

Key Action Responsible DueDate Status Comments Documents
UNDP will bring the recommendation to the attention of TMA assuming the new developing project is approved
[Added: 2018/09/24]
Environment, CC & Resilience Pillar 2023/01 Initiated Project Scale up Concept Note development is in progress
9. Recommendation:

For communication the old and proven method of community-based radio stations may be explored. Although this may be comparatively capital intensive (compared to communication on mobile phone), the coverage of the impacted population would be almost 100%. Further, from sustainability point of view this is better as no recurring cost is involved (for mobile communication the telecom companies are to be paid regularly which impacts the financial sustainability).

Management Response: [Added: 2018/09/24]

PMO-DMD and the project management agree with this recommendation the implementation thereof will be subject to availability of funds given the assessor’s own remark that this is capital extensive. This is linked to recommendation 4.

Key Actions:

Key Action Responsible DueDate Status Comments Documents
Consult key stakeholders to include community radio in the scaled-up project and advocate for broadcasting funds for radio stations
[Added: 2018/09/24]
Environment, CC & Resilience Pillar 2020/01 Initiated Project Scale up Concept Note development is in progress
10. Recommendation:

From the view point of financial sustainability, one of the regular expenses is the regular bills to be paid to the telecommunication companies. The national government may explore the possibility to have policies, wherein there are no charges by the telecom companies for providing such kind of services. For example, there can be a condition in the licensing agreement to the telecom companies, wherein the services provided for disaster management and defense of the country are not chargeable.

Management Response: [Added: 2018/09/24] [Last Updated: 2018/09/24]

PMO-DMD and the project management disagree with this recommendation which is beyond the scope of the project and UNDP’s mandate. Introduction of new policy or policy change is a long-term process and may not happen through the influence of one project hence no action required from UNDP at this stage.

Key Actions:

11. Recommendation:

In the areas where mobile phone services are not available; the project has provided for establishment of a satellite link for transmission of the data from the weather stations. The capital cost of providing the link and the recurring cost (for managing the batteries, invertors, solar PV systems etc.) is also quite high. It is recommended that in such situations, the creation of facilities with mobile service provides (telecom companies) may be explored on a cost sharing basis. This is likely to reduce both the capital cost and the recurring cost for provision of the link for data transmission. It will have an added advantage of the development of the area.

Management Response: [Added: 2018/09/24]

PMO-DMD and the project management agree with this recommendation which will be considered as part of recommendation 4 above.

Key Actions:

Latest Evaluations

Contact us

220 East 42nd Street
20th Floor
New York, NY 10017
Tel. +1 646 781 4200
Fax. +1 646 781 4213
erc.support@undp.org