Mid-term evaluation of the environmental sustainability outcome

Report Cover Image
Evaluation Plan:
2016-2021, Tanzania
Evaluation Type:
Outcome
Planned End Date:
09/2018
Completion Date:
11/2018
Status:
Completed
Management Response:
Yes
Evaluation Budget(US $):
80,000

Share

Document Type Language Size Status Downloads
Download document Environment & Climate Change Outcome Evaluation Final ToRs.pdf tor English 366.97 KB Posted 52
Download document TZ CPD Outcome Evaluation Report Final Nov 27 2018.pdf report English 1415.19 KB Posted 45
Title Mid-term evaluation of the environmental sustainability outcome
Atlas Project Number: 94386,94384,83123,86631,61988,102185,74211,92475,61743,60996,68935
Evaluation Plan: 2016-2021, Tanzania
Evaluation Type: Outcome
Status: Completed
Completion Date: 11/2018
Planned End Date: 09/2018
Management Response: Yes
Focus Area:
  • 1. Environment & Sustainable Development
  • 2. Others
Corporate Outcome and Output (UNDP Strategic Plan 2014-2017)
  • 1. Output 2.5. Legal and regulatory frameworks, policies and institutions enabled to ensure the conservation, sustainable use, and access and benefit sharing of natural resources, biodiversity and ecosystems, in line with international conventions and national
  • 2. Output 5.2. Effective institutional, legislative and policy frameworks in place to enhance the implementation of disaster and climate risk management measures at national and sub-national levels
Evaluation Budget(US $): 80,000
Source of Funding: UN, UNDP
Evaluation Expenditure(US $): 27,229
Joint Programme: No
Joint Evaluation: No
Evaluation Team members:
Name Title Email Nationality
GEF Evaluation: No
Key Stakeholders: Vice President's Office, Ministry of Natural Resources and Tourism, UN agencies under the Outcome
Countries: TANZANIA (UNITED REPUBLIC OF )
Lessons
1.

Strong partnerships for planning and implementation of programmes and projects still remain the most cost-effective vehicles for development assistance.


2.

While mainstreaming projects into government programmes catalyses inclusion of project issues into national budgeting processes, it does not automatically guarantee availability of finances because the government budget is still limited, and many budget lines are not fully funded. Thus, the struggle for moving funding for the resilience pillar from project to systemic (budgetary) is not yet over. 


Findings
1.

The outcome is currently being delivered through thirteen projects (Table 2); six under output 1, four under output 3 and two under output 3. Five of the projects have undergone terminal evaluations; two have been through mid-term reviews; four are just starting implementation, and one has just been submitted to the GEF for final approval (Table 2). The number of projects is however likely to increase as resource mobilization is still continuing. The CPD expected the Pillar to be financed by a total of US$ 31,900,000, which included $8,900,000 to be mobilized by UNDP (from GEF and other sources), while the government and other partners would raise $23 million as co-finance. The MTR however finds that the total funds mobilized to date by UNDP, in partnership with government, civil society and NGOs is US$ 129,229,009. The amount includes $ 30,307,187 managed by UNDP directly ($ 23,787,952 from the GEF, $ 6,519,235 from UNDP core funds), US$ 1,800,000 from the Tanzania Forest Conservation Group, US$ 1,000,000 from the  Eastern Arc Mountains Conservation Endowment Fund, US$ 400,000 from the World Wide Fund for Nature, US$ 400,000 from the Wildlife Conservation Society, US$ 710,227 from the Government of the Republic of South Africa, US$7,070,000 from private donors (Association of Tanzanian Tobacco Traders – US$ 3,570,000 and Institute of Resources Assessment under REDD+ - US$ 3,500,000 ), and USD 78,955,156 from the government and its partners (as a co-finance). Although the funds for the older projects in the portfolio (those closed and some under implementation) were mobilized before the start of the current Country Programme, they have all contributed to its implementation. Indeed, at MTR, UNDP, the government and their partners have mobilized about US$ 16.9 million, which is more than double the target; they have mobilized a co-finance of $ 23 million, which is very close to the target.  Part of the additional funds is attributed to one additional project that was not in the CPD - Enhancing Climate Change Resilience in Zanzibar ($ 1 million).


2.

The overall rating for the Pillar is Satisfactory, which is explained by the individual ratings of the evaluation criteria, summarized below.


3.

Relevance (Highly Satisfactory): Overall, the three Pillar II outputs and the thirteen projects’ in the portfolio are highly relevant to the development agenda of Tanzania. The MTR concludes that they were all designed in close participation with relevant government ministries, departments and agencies, local government authorities, relevant civil society groups, international and local non-government organizations, and occasionally communities. Moreover, by and large all the projects remained relevant throughout the implementation phase, even where some changes took place in the development of the country. UNDP has, through the projects implemented within the outcome, played a key role in introducing the Government and its project partners to the best global practices to promote partnerships for the SDGs. The requirements for knowledge management and the templates for integrating it into projects and institutions managing them has improved over time; with newer projects having a mandatory budgeted outcome on it.  While newer projects are designed with an explicit ToC, the older projects did not have one; however, they all applied the logical framework analysis and had clear logic underpinning their design. Projects that did not have a ToC at design had them reconstructed as part of either the MTR or the TE.  


4.

Efficiency (Satisfactory to Highly Satisfactory): Overall, the projects under the outcome have strong arguments on cost effectiveness (efficiency), largely based on consideration for alternative options to tackle the challenges, and adequately informed by best practices generated via both UNDP and government programmes. However, in most cases, the resources mobilized through the projects are only adequate to demonstrate best practices and technologies. There is still a significant gap in resources for upscaling the demonstrated practices and technologies to increase the impacts and advance the SDGs across the three outputs of the outcome in 28 districts. 


5.

Effectiveness (Moderately Satisfactory): Evaluation of the extent to which the projects are delivering outputs that contribute to the three Pillar II outputs as well as how those outputs contribute to the CPD Outcome is challenged by the weak link between the Pillar indicators and projects’ indicators, the nature of the output indicators (too focused on quantifiable deliverables rather than the changes that occur when individuals and institutions utilize the deliverable outputs), and limited monitoring and reporting at the Pillar and CPD levels. In addition, some indicators have no baseline and/or target values and there is doubt about the accuracy of at least one indicator. The baseline value for indicator 1.4 is zero, which seems to indicate that the country has no systems for surveying wildlife or monitoring wildlife crime in the country. Yet Tanzania has had a long history of conservation and strong institutions for both tasks, with technical and financial assistance from UNDP and other Development Partners.  


6.

Despite the above challenge, there is evidence of progress towards increasing capacity of the country (national government, Local Government Authorities, and communities) to reduce the likelihood of conflict and lower risk of natural disasters, including from climate change; 86 percent of the projects evaluated scored “Moderately Satisfactory” and above on progress towards results while 100 percent scored “Moderately Satisfactory” and above on effectiveness (with 71 percent scoring “Satisfactory and above). Key deliverables include review of five policies with recommendations provided to mainstream sustainable forest management principles in the sectors; support to the development of the National Climate Change Strategy, accompanied by identification of Nationally Determined Commitments (NDCs) in which the country identifies sectors which it will tackle to reduce emissions; and development of nationally appropriate mitigation actions (NAMAs) for 3 sectors (transport, energy and waste management). Other deliverables include four District Land Use Management Plans integrating SLM - for Morogoro, Mvomero, Mkinga and Muheza District Councils, gazettement of 6 new Forest Nature Reserves, and development of tourism strategies for the southern Tourism circuit.


7.

Despite the impressive deliverables, challenges remain; UNDP projects largely demonstrate best practices, methodologies and technologies, which need to be upscaled via additional finances in order to bring about changes in the 28 districts. Even when projects deliver national, district or community level strategies, they often do not provide the additional funds required for implementation, post project; and majority of the projects do not have budgeted, financed exit strategies. While UNDP has made impressive effort in mobilizing innovative partnerships and identifying new and additional sources of funds for upscaling the proven initiatives, the additional funds are still limited in view of the magnitude of the barriers that need to be shifted to make significant progress in the relevant SDGs in the 28 districts.


8.

While UNDP, the government and all relevant partners are putting a great deal of effort to mobilize additional funding, this is constrained by several challenges:  competition for GEF resources has increased tremendously in the last 10 years, and UNDP’s share of the GEF portfolio is down to 24 percent, impacting UNDP’s ability to expand its portfolio; private sector in Tanzania is juvenile, and the private public sector partnerships (PPP) is still an emerging principle, with government opening up cautiously. Taxation is limiting private sector expansion for a meaningful CSR contribution. Declining opportunities for funding are occurring in tandem with increasing government priorities that are not matched with available resource, especially in the adaptation, energy and environment sector.


9.

The factors that contributed to achievement of the pillar II Outcome include high levels of stakeholder participation during planning and innovative implementation arrangements, close alignment of the projects to the government priorities, and UNDP’s recognized leadership in the environment, climate change and disaster risk reduction work in Tanzania. Inadequate assessment of risks, limited co-finance and weak monitoring and evaluation had negative effects on effectiveness.


10.

Sustainability (Moderately Satisfactory): Initiatives piloted by the projects under the three outputs of Pillar II generally have fewer environmental, institutional framework and governance risks to sustainability, but stronger socio-economics and financial risks to sustainability.  The sustainability of 29 percent of the seven evaluations available to this review were rated “Unlikely”, 43 percent rated “Moderately Unlikely” while only 
28 percent rated “Moderately Likely”. Strong points driving sustainability include the use of existing institutions to implement projects, which has avoided the establishment of parallel institutions and generally led to mainstreaming project initiatives into the regular government programmes. The high levels of stakeholder ownership and awareness of the projects initiatives by national and subnational governments, local communities, and where relevant, civil society and academia provide strong basis for sustainability.  Financial risks to sustainability are driven by inadequate mainstreaming of project cost into government budgeting processes, new directive on centralization of government revenue collection and general lack of project exit strategies. Even where budgets include allocations for project proven initiatives, this does not guarantee funding because many budget lines are not financed. About half of all the projects under Pillar II do not have exit strategies. However, even where the projects had exit strategies, they are not always fully financed or implemented in time to secure sustainability.  


11.

Gender mainstreaming (Satisfactory): The MTR finds that the design, implementation and monitoring of environmental sustainability, climate change and resilience projects has addressed gender issues adequately, increasingly more so for the newer projects. However, although the gender marker was introduced in 2009, only the newest projects (about to start implementation) have a gender marker and a gender strategy. Indeed, the oldest projects in the Pillar – Miombo Woodlands, Climate Information, SPANEST, CADESE - did not have gender disaggregated indicators. Nevertheless, UNDP has actively promoted the empowerment of vulnerable groups – women and youth - in all its projects and activities. It has improved gender mainstreaming and capturing and reporting its achievements in empowerment, as reflected in the newer projects.  


12.

Although the Country Programme has not reached the threshold US$ 25 million per year set by the Gender Strategy (2014-2017) as the threshold for engaging a gender advisor in the Country Office, the Country Director, the Deputy Country Director and the Head of Programme are designated as Gender Focal points. This shows the seriousness with which the Country Office takes mainstreaming gender issues into all the programmes and projects. Indeed, the Country Office’s score on Gender Equality Seal is Silver.  


Recommendations
1

The projects contributing to the Pillar outputs are complex and are delivering many results that are not captured by the indicators of the Pillar in the UNDAP. In addition, while the indicators of the Pillar provide a mechanism to monitor and capture production of project outputs, they limit analysis of the project’s contribution to Country Programme Document Outcome. It is recommended that a pillar-level strategy document be developed to spell out how the projects will contribute to the pillar outputs and the Country Programme Document outcome, linking project indicators more clearly to pillar indicators.

2

UNDP should increase effort in resource mobilization, formulate budgeted exit strategies for projects and help them capitalize

3

UNDP should pay more attention to sustainability issues at project design, and that it uses best practices from its large network of projects globally to tackle difficult challenges

4

Projects without gender action plans should formulate one as soon as possible

1. Recommendation:

The projects contributing to the Pillar outputs are complex and are delivering many results that are not captured by the indicators of the Pillar in the UNDAP. In addition, while the indicators of the Pillar provide a mechanism to monitor and capture production of project outputs, they limit analysis of the project’s contribution to Country Programme Document Outcome. It is recommended that a pillar-level strategy document be developed to spell out how the projects will contribute to the pillar outputs and the Country Programme Document outcome, linking project indicators more clearly to pillar indicators.

Management Response: [Added: 2019/05/09]

Agrees

Key Actions:

Key Action Responsible DueDate Status Comments Documents
Review pillar’s indicators to capture more results once new project approval are granted.
[Added: 2019/05/10]
Pillar Lead/Programme Specialist and M&E Specialist 2019/12 Initiated
Review projects targets and baseline values for indicators for better reporting.
[Added: 2019/05/10]
Pillar Lead/Programme Specialist and M&E Specialist 2019/12 Initiated
Develop Pillar M&E Plan and Communication Strategy to support results documentation and reporting based on CPD and UNDAP indicators.
[Added: 2019/05/10]
Pillar Lead/Programme Specialist and M&E Specialist 2019/12 Initiated
2. Recommendation:

UNDP should increase effort in resource mobilization, formulate budgeted exit strategies for projects and help them capitalize

Management Response: [Added: 2019/05/10]

Agrees

Key Actions:

Key Action Responsible DueDate Status Comments Documents
Develop an Environment and Climate Change Pillar specific Partnership & Resource Mobilization Strategy accompanied with an implementable Action Plan.
[Added: 2019/05/10]
Pillar Lead/Programme Specialist 2019/12 Initiated A solid first draft of the CO Partnerships & Communications Strategy and Action Plan has already been developed. History
Solicit government’s leadership in facilitating resource mobilization through strategic and continuous engagement of the relevant government institutions.
[Added: 2019/05/10]
Pillar Lead/Programme Specialist 2019/12 Initiated
Working with the relevant government institutions in developing projects’ budgeted exit strategies with a key focus on achieving full ownerships of projects’ interventions.
[Added: 2019/05/10]
Pillar Lead/Programme Specialist 2019/12 Initiated
3. Recommendation:

UNDP should pay more attention to sustainability issues at project design, and that it uses best practices from its large network of projects globally to tackle difficult challenges

Management Response: [Added: 2019/05/10]

Agrees

Key Actions:

Key Action Responsible DueDate Status Comments Documents
Implementation of the actions in Recommendation 2 above shall contribute to sustainability; this shall be considered in future project design processes.
[Added: 2019/05/10]
Pillar Lead/Programme Specialist 2019/06 Overdue-Initiated
4. Recommendation:

Projects without gender action plans should formulate one as soon as possible

Management Response: [Added: 2019/05/10]

Agrees

Key Actions:

Key Action Responsible DueDate Status Comments Documents
Ensure that gender is mainstreamed in all projects
[Added: 2019/05/10]
Pillar Lead/Programme Specialist and M&E & Communication specialists 2019/12 Initiated In the past, gender action plan was not mandatory, however, of recent, new projects have a gender action plan incorporated.

Latest Evaluations

Contact us

220 East 42nd Street
20th Floor
New York, NY 10017
Tel. +1 646 781 4200
Fax. +1 646 781 4213
erc.support@undp.org