Integrated approach for resilience and food security mid- term evaluation

Report Cover Image
Evaluation Plan:
2017-2020, Ethiopia
Evaluation Type:
Mid Term Project
Planned End Date:
12/2019
Completion Date:
12/2019
Status:
Completed
Management Response:
Yes
Evaluation Budget(US $):
50,000

Share

Document Type Language Size Status Downloads
Download document UNDP-GEF-MTR-TOR-IAP project-international consultant-final.docx tor English 102.79 KB Posted 325
Download document MTR report final signed.pdf report English 1572.16 KB Posted 383
Title Integrated approach for resilience and food security mid- term evaluation
Atlas Project Number: 00097070
Evaluation Plan: 2017-2020, Ethiopia
Evaluation Type: Mid Term Project
Status: Completed
Completion Date: 12/2019
Planned End Date: 12/2019
Management Response: Yes
UNDP Signature Solution:
  • 1. Poverty
Corporate Outcome and Output (UNDP Strategic Plan 2018-2021)
  • 1. Output 1.1.1 Capacities developed across the whole of government to integrate the 2030 Agenda, the Paris Agreement and other international agreements in development plans and budgets, and to analyse progress towards the SDGs, using innovative and data-driven solutions
SDG Goal
  • Goal 13. Take urgent action to combat climate change and its impacts
SDG Target
  • 13.2 Integrate climate change measures into national policies, strategies and planning
Evaluation Budget(US $): 50,000
Source of Funding: gef
Evaluation Expenditure(US $): 50,000
Joint Programme: No
Joint Evaluation: No
Evaluation Team members:
Name Title Email Nationality
Mark Anstey International Consultant
Abera Gayessa Local Consultant ETHIOPIA
GEF Evaluation: Yes
GEF Project Title: Food/IAP Integrated landscape management to enhance food security and ecosystem resilience
Evaluation Type: Mid-term Review
Focal Area: Multifocal Areas
Project Type: FSP
GEF Phase: GEF-6
GEF Project ID: 9135
PIMS Number: 5559
Key Stakeholders: Ministry of Environment and Forest, GEF
Countries: ETHIOPIA
Lessons
1.

4.3.1 Lessons Learned on design and project start up (inception) for future projects 226. Ensure Clarity of Project design and RF (standardized formatting): As discussed in the report initial sections, the MTR team would suggest that this project strategy and design, though basically adequate, lacked clarity, consistency and in some cases clear logic. As a result, the essentially fairly simple concept (strategy) was not easily evident to the average reader. Furthermore, the project Results Framework contained many issues both regarding way the components were organized and arranged (i.e. the non-standard use of Components and Outcomes together) which then complicated application of other standard UNDP tools (AWP, PIR, etc.), and the quality of indicators. These problems should not have been allowed to persist through to signature stage and thus it is strongly recommended that in future project development processes a rigorous review is applied to these aspects.

227. Ensuring a rigorous Inception Phase and Report: The inception phase of any project is critical for ensuring the successful future implementation, and usually involves a). an assessment of whether any factors have changed since project development, b). finalization of baseline / target data in RF if such is needed (as in the case of this project) and the updating / refinement of the original Multi-year workplan (plus initial AWP). The key findings and recommendations can then be presented at the Inception workshop.

228. It is unfortunate that this opportunity to deal at the start with weaknesses in the RF was not taken during the inception phase of this project as it is both simpler and more effective to do so then rather than at MTR stage, and would have made the monitoring process of the project easier. It is strongly recommended that in any future UNDP/GEF project in Ethiopia this is done carefully and systematically, even if this results in some delay in operational start up. A small delay initially is worth avoiding such problems later


Findings
1.

Project Design

Analysis of Section 2 of Project Document: Project Strategy (ToC) and Design

This project is a “child project” under the umbrella of the regional GEF Program Fostering Sustainability and Resilience for Food Security in Sub-Saharan Africa -An Integrated Approach (IAP). This program covers 12 countries, with child projects for each under different IA partners to the program. Following the commencement of the above Program in approx. mid 2016 the Ethiopian child project was developed by UNDP, the designated IA in Ethiopia. The project development process was atypical of GEF full size projects due to its status as a child project of the regional IAP program and no PIF was prepared (instead an accelerated process was applied – i.e. an “Expression of Interest” (EOI) was prepared and submitted to the GEF). Following the approval by GEF of the EOI UNDP Ethiopia employed 2 consultants (one national, one international) to develop the full-sized project document and CEO Endorsement document. Following GEF Secretariat review this was revised and finalized / approved by GEF on 21 February 2017. Thus, in relative terms the project development and approval process were completed in the unusually short time of only about 7 months.

The project context and Strategy were developed using the Resilience, Adaptation Pathways and Transformation Assessment (RAPTA) approach10. In broad terms the analysis and strategy follow closely that of the regional IAP document. Thus, as in the Regional IAP, 3 “impact pathways” are identified, namely: - Building institutional frameworks for resilient food systems - Scaling up best practices in Integrated Natural Resource Management - Understanding impacts and sharing evidence to influence policy and practice.

The ToC (see below) describes the first two of the above at the 1 st “immediate outcomes” level (but calls them Components 1 and 2) that result from a subset of 7 “immediate outcomes”, which in turn are a product of 6 “outputs”. The 3rd intervention pathway is represented as an overarching aspect on the side and is called Component 3.


Tag: Agriculture Food Security Livestock Natural Resouce management Global Environment Facility fund Rule of law Monitoring and Evaluation Programme/Project Design Project and Programme management Theory of Change Jobs and Livelihoods

2.

Analysis of Project Results section (part 3 of project document) and the Results Framework Matrix.

 Unusual and Multiple Formats: Most UNDP-supported GEF-financed projects are described in the text as having “components” but usually these are presented in the Results Framework as “outcomes” – thus the terms component and Outcome are synonymous. Thus, many projects have 2 to 3 components described in the text and 2 to 3 equivalent outcomes in the RF. This project however has 3 Components, then for each component one or more outcomes, resulting in a total of 5 outcomes altogether. This is an unusually large number of Outcomes and has the knock-on effect of resulting in an unusually large number of outputs (16).  It is not clear to the MTR team why this format/terminology was chosen for this project. Apart from the impact on the clarity of logical flow from output to outcome to objective, it also complicates the situation in terms of AWP and PIR preparation. For example, in the AWP and PIR formats there is no provision for “components” so only the 5 Outcomes are present, but in AWP they are numbered Outcome 1.1, 1.2, etc. while in PIR Outcome 1, 2, 3, etc. The choice of having separate Components and Outcomes was, therefore, clearly not standard and its use has not helped ensure clarity.  In our opinion there was no logical reason why the components could not have been represented in the RF as outcomes, the outcomes as outputs and outputs as main activities. This would have been a much clearer approach and would have help with clarity of logical flow and been compatible with existing UNDP / GEF planning and reporting instruments (AWP, PIR, etc). It is recommended that in the future this issue is avoided. 59. Apart from the above format/terminology issue, there are some curious logic chains between some outcome to output to activities. For example, in Outcome 1.2 the Outcome is about putting in place policy and incentives but then outputs are about implementing value chains (see table 3 below).


Tag: Global Environment Facility fund Communication Monitoring and Evaluation Project and Programme management Results-Based Management Value Chain

3.

Progress Towards Results

 Analysis of progress towards outcomes

 There was a short delay to the start of the project following the signing of the UNDP project document on 12 May 2017. The planned inception phase was 2 months, but the final Inception workshop only took place at the end of August (29 August 2017) approximately 1 month late. This is not a particularly long delay. 65. However, more significant than this slight delay was the lack of a through set of inception phase activities. Typically, an inception phase report should be produced at the start of any project that: a. reviews any changes in the situation since project signature and updates accordingly b. Reviews the project Results Framework and in particular the indicators, baseline and targets to ensure they are 1). An effective basis for monitoring, 2). Accurate / meaningful, c). feasibly to collect data for. In this project this was a particularly important step as many of the baselines were provisional and it was specifically stated in the RF that they needed to be confirmed at inception phase. Furthermore, as it now transpires, many of the indicators suffered serious weaknesses of various kinds. IAP PROJECT, ETHIOPIA- MTR 18 c. reviews the broad expected outcomes, outputs and activities and the multi-year workplan in the project document and “operationalizes” them (i.e. clarifies and adds detail on how they will be practically implemented). On this basis a draft AWP for the starting year can be prepared for presentation and approval during the Inception workshop and subsequent Project Steering Committee meeting (if deemed necessary). 


Tag: Agriculture Livestock Natural Resouce management Water resources Effectiveness Human and Financial resources Programme/Project Design Results-Based Management

4.

Meetings by the MTR teams with Woreda Steering Committees in 5 Woreda’s provided ample proof of their full ownership of the planning and implementation of project activities, and their strong commitment to them. project resources and activities were seen as positive additions to their normal activities that both allowed more to be done but also extended existing efforts using a new integrated approach, 74. In addition, 12 Technical Committees, 12 Gender Teams and approx. 44 Community Watershed committees were established by the project and the Woreda Steering Committees in order to best execute the activities in the field. These were not specifically identified in the project document but were established on the basis of identified need. In the case of the Community Watershed Committees these are entirely community initiatives (see more on these under Outcome 3). 75. As per the project document, there is also a national level platform – this also acts as the overall project steering committee and has met so far 4 times since project commencement. 


Tag: Agriculture Food Security Biodiversity Energy National Local Governance Awareness raising Youth

5.

Community watershed land rehabilitation:

Briefly this intervention involves helping communities in already deliniated watershed areas to a). identify areas degraded (suffering erosion, loss of vegetation, loss of topsoil, etc) due to overgrazing, tree cutting, etc., b). identifying and selecting viable community level methods to stop or reverse such degradation (closure and zero grazing, physical methods such as trenches, bunds, check-dams, biological methods such as tree and grass planting), c). providing training and limited material inputs (simple digging tools, wire netting for gabion preparation, tree seedlings and seeds) to build community capacity to implement selected methods, d). on-going technical advice and guidance to communities implementing rehabilitation activities, e). monitoring of impacts. 83. The project very realistically realized that initially many communities, without previous experience of the effectiveness and benefits of such rehabilitation activities, would lack the necessary commitment to undertake them. Thus as a first step in it first supported the small scale application of closure and rehabilitation (for example on 5 or 6 hectares) – the very remarkable levels of regeneration of grass cover that was quickly evident in only a short time was sufficient evidence for many Woreda authorities and communities to then designate significant larger areas (200-300 ha. for example).


Tag: Agriculture Food Security Livestock Biodiversity Natural Resouce management Water resources Effectiveness Capacity Building Technical Support

6.

Impact monitoring and economic analysis:

As discussed in previous sections the project RF indicators are mainly process related and even those that more directly measure impact (such as ha. land rehabilitated, etc) are rather broad. In addition, the new monitoring and information system being set up by the project (see Outcome 5) will include monitoring of important real time / real life impacts (such as changes in vegetation cover over time, etc.). However, if the full and most meaningful benefits of the activities and approaches that the project is demonstrating are to be recognized and used as basis to justify further replication, then the MTR team strongly advise more indepth assessment of impacts. 90. For example, an important aspect of the feasibility of the closure of land to grazing is that it results in fodder production, which in tern makes feasible a change to the “cut and carry” stall fed approach to livestock production. In this case it is very important to have some assessment of how much fodder is produced and what economic implication does that have i.e. how much of a benefit is it to household members of the watershed committee / communities?


Tag: Agriculture Food Security Forestry Energy Natural Resouce management Effectiveness Rural Monitoring and Evaluation Jobs and Livelihoods

7.

Need for more detailed impact and GEB monitoring and Assessment: As in the case of the Watershed land rehabilitation, the MTR main comment in regard to these activities is that there is a need to more meaningfully quantify and monitor impact in order to provide a better picture of impact and justification for replication. For example, the purpose of the exercise is mainly to reduce tree cutting and increase amount of dung being used for productive purposes rather than burnt – in this case it is important to actually quantify what impact they have in these terms i.e. how much fuel wood is actually saved (compared to households not using the technologies) and what does this mean in terms of the annual reduction of fuel wood used ?. If adoption / replication is 10% /year what does that mean in terms of number of trees saved over 10 years ?. 95. This kind of analysis provides a meaningful measure of the real and potential impact of such activities and thus a basis for government and donors to assess true environmental benefits. Ideally, assessment and monitoring can go further and quantify the impacts of additional dung on productivity, reduced mortality and costs from reduced smoke inhalation, the benefits to productivity of households/women from reduced time spent on fuel preparation/collection.


Tag: Agriculture land resouces Food Security Energy Natural Resouce management Donor relations Monitoring and Evaluation Jobs and Livelihoods Technology Value Chain Awareness raising Capacity Building

8.

It is important to note that the economic viability of many of the above has been enhanced through support to strengthening “value chains” – by identifying markets, building trade connections and supporting producers to meet specific market demands, such value chain support by the project increases the potential benefits of the diversification. This is another good example of how the project has taken the basic “ingredients” in the project document and applied it to maximum effect. Another interesting cross-linkage reported to the MTR team during field visit was the beneficial impact that beekeeping had on the production of fruit trees planted in and around households (due to better pollination presumably). This is an example of unexpected benefits that can come from a “whole system” integrated approach and from intelligent diversification of production systems.

As in the case of previous activities under this Outcome, the MTR team identify a need to more rigorously assess and monitor the meaningful impact of the project activities in terms of the project main objectives (i.e. ecosystem resilience and food security). Thus, the project needs to better evaluate a). the environmental costs/benefits and sustainability of the land use practices introduced (for example, some do have potential negative long term implications such as the solar pumps as they are increasing ground water extraction, while others such as the beekeeping seem to have only positive implications), b). the real impact in terms of food security – do the new practices increase food supply/incomes and does that translate meaningfully into beneficiary households having greater multi-year levels of food security?


Tag: Food Security Forestry Livestock Natural Resouce management Monitoring and Evaluation Value Chain Women and gilrs Youth

9.

Problems and Barriers faced with Implementation of Outcome 3:

The most obvious problem under this outcome relates to the missed target in regard to area of agro-pastoral areas under ILM (target was 30,000 ha. and MTR reported achievement 5,528 ha.). This the MTR team believes can be related to 2 issues: a). the lower general capacity at both regional and Woreda level in the mainly pastoral areas of the country, b). the lower levels of community cohesion and less established land use system in historically pastoral areas that have been experiencing a transition over past decades to agropastoral systems. As was noted in the discussion above on activities to close grazing/rehabilitate lands, the latter issue impacts effectiveness of activities and required significant adaption of approach (from community managed to cooperative managed). Such testing and subsequent adjustment clearly delayed implementation. The former issue has meant that rolling out activities in the field has been slower. It is expected that with the increased experience achievements under this target will accelerate. However, it is also recommended to review the situation and to either adjust expectations in line with realistic forecasts of impact or make other adjustments that will improve effectiveness of activities generally in these sites, and specifically in regard to agropastoral areas rehabilitation.


Tag: Agriculture land resouces Effectiveness Resource mobilization National Communication Donor relations Partnership

10.

Outcome 5:

This outcome focuses on achieving a system of evidence-based Monitoring and Assessment, Knowledge Management and Learning within which local stakeholders will be key actors. Activities under this outcome were intended to focus on monitoring and assessment of whether institutional frameworks, integrated approaches and initiatives for transformation to new livelihoods have a positive impact on resilient food systems and the generation of GEBs. Achievements to date include a). A web based, GIS embedded system for multi scale monitoring of ecosystem services and global environmental benefits (GEB) developed and training on its application and functional operation underway - system design completed and officially launched by EFCC Commission. Training for project Field Coordinators (who will then train Woreda level staff) is ongoing, b). Action research by local Universities (examples include research into new tree species for highlands by Debre Birhan University, adaptive early maturing fruit plants by Wolaita Sodo University in Dugan Fango, research by Haremaya university in Doba woreda to improve soil fertility, research by Hawassa University on adaptability of improved maize and haricot bean verities for Bilate zuria woreda, etc).

The project in collaboration with innovator of the green bag concept (nonplastic natural material bags) provided practical training on the production of green bag technology. The training was provided to environmental school clubs members in four project woredas (Angolelana Tera , Doba, Dugun Fango and Chiro woredas). Following the training school club members are making the green bag and demonstrated their product for their respective woreda community. The woreda’s are also interested to scale up the technology and practice at the enterprise level.


Tag: Food Security Forestry Green Economy Knowledge management Monitoring and Evaluation Education Jobs and Livelihoods

11.

Remaining barriers to achieving project objectives

Broadly, the project is progressing well towards achieving its objectives, as discussed above. However, the MTR identify 3 sets of potential barriers to achievement of the project results: 122. Capacity and Cultural / traditional land use experience barriers in mainly Pastoral Woreda’s: this has already been discussed in previous sections. 123. Impact monitoring: likewise, the need for more in-depth and meaningful assessment and monitoring of project activities impact has been discussed previously. Its importance lies in a). achieving an accurate picture of real impacts and, b). providing a powerful basis for advocacy and ensuring replication / investment of the ILM approach as applied by the project. 124. Lack of dedicated outputs/activities dedicated to documenting experience and lessons learned as basis for advocating replication / leveraging additional resources: As discussed briefly under the project Strategy and design section, the project lacks specifically dedicated outputs/activities related to documenting results, experience and lessons learned, and packaging these in a way that effectively communicates them to the government/donors and thereby helps ensure the adoption of the ILM model at wider scales (replication within government and donor financed support to government).


Tag: Agriculture land resouces Effectiveness Monitoring and Evaluation Programme/Project Design Sustainability Advocacy

12.

 Management arrangements 

 The implementation arrangements are described in Section 3.4. The project is being implemented in 6 different regions and 12 different Woreda’s so faced an extremely challenging task. Despite these challenging circumstances, the project is being implemented in a very effective way and the levels of stakeholder involvement seem, on the basis of the MTR team’s meetings, to be extremely high. 128. Indeed, the fact that the project is being implemented so effectively is a testament to the commitment and hard work of people at all levels involved in the implementation of the project. Commendation needs to be given in particular to the PMU, specifically the PM, who have successfully managed to “bring on board” so effectively and in such a brief period the widely distributed stakeholders at Woreda level. 129. The national Project Steering Committee has met three times over the life of the project. The PSE is proving an effective mechanism for project oversight. 130. The Project Management Unit has established itself in the EFCC Commission offices in Addis Ababa, and project coordination offices established in each of the target Woreda’s with the exception of Somali Region. In the latter case, due to Woreda capacity limitations, a single regional coordination office was established to support activities within the two Woreda’s to be targeted in the Somali Region. Each of the coordination offices is staffed by a “Field Coordination Officer” and A “Finance Officer”.


Tag: Energy Human and Financial resources Implementation Modality Procurement Programme/Project Design Project and Programme management

13.

Work planning

As is usual for UNDP-GEF projects, budgets and workplans are developed on an annual basis and are approved by the Project Steering Committee. 143. The project Manager and E&E team made a pragmatic “translation” of the Project document into a workable 1st AWP that overcame the issues discussed previously in regard to activities under Output 1.2.1 (see para. 60). 144. Additionally, the PMU are to be commended for their effective efforts to develop clear and pragmatic system of work planning at the Woreda level. It is likely that this greatly facilitated the rapid Woreda level understanding and commitment to implementation and the on-going effectiveness of field level implementation.


Tag: Effectiveness Implementation Modality Programme/Project Design Project and Programme management

14.

Project Finance and Co-finance

 At the, time of the MTR (November 2019), the project had spent a total of 5,163, 295 USD out of a total budget of 10,739,450 – in other words just under half (48%) of the total. The project Budget is divided on the basis of the Components rather than outcomes and thus has 3 implementation sections and 1 Project Management section. 147. Financial delivery (actual compared to budgeted) is 79%. This is mainly due to a significant under delivery of Component 216 - 66.5% of the originally budgeted amount, a slight under delivery of Project management costs, and an over delivery of Components 1 and 3. 148. The under delivery of Component 2 would be concerning except for the fact that the project is actually achieving (and in some cases exceeding) expected results in the field. This suggests that implementation of Component 2 has been highly costeffective and achieved savings.


Tag: Effectiveness Efficiency Human and Financial resources Policies & Procedures Operational Efficiency Project and Programme management

15.

Project-level Monitoring & Evaluation Systems 

The project’s M&E framework is similar to the majority of all UNDP-GEF projects with a USD 342,000 allocated for project monitoring. This is a substantially larger than normal figure but includes all funds allocated to Outcome 4 (knowledge management). 156. Roles and responsibilities are clearly defined in the M&E Plan. There is some lack of clarity in the way resources are allocated as some significant items are actually outputs/activities under Outcome 4 (knowledge management). In the MTR Team opinion, it would have been better to have not directly included these in the M&E plan table and to have ensure the M&E plan clearly focused on the principle M&E tasks and their budget. 157. The project is generally being monitored effectively and efficiently according to the existing indicators. As discussed elsewhere the weak RF in terms of indicators has caused challenges and, in many ways, do not well reflect actual project progress and results. 158. It is noteworthy and commendable that the project has followed through on ensuring gender related monitoring data and has developed additional gender monitoring tools during implementation that are likely to provide more meaningful measure of impact in this regard.


Tag: Effectiveness Efficiency Human and Financial resources Monitoring and Evaluation Project and Programme management

16.

Stakeholder engagement.

Stakeholder analysis is contained within the Project Document and the main stakeholders are identified, with a broad description of their mandate, as well as their identified role and responsibilities within the project. 163. As described previously, there are many stakeholders and the project is correspondingly complex (see table of key stakeholders in annex). However, stakeholder engagement and inclusion are considered by the MTR Team to be very satisfactory with the needs and concerns of stakeholders taken into account through each of the project steps and processes. 164. As described in Section 2 of this report (para. 70 onwards) an important focus of the project was the initial establishment of multi-stakeholder frameworks. The outcome is stated as “Multi-stakeholder and multi-scale platforms in support of integrated natural resources management in agricultural landscapes in place “ and the target at mid-term is “At least 12 functioning (convening and decision-making) multistakeholder platforms in place in the project sites; plus one at national-level”.


Tag: Agriculture land resouces Livestock Natural Resouce management National Civic Engagement Human and Financial resources Awareness raising Youth

17.

Reporting

The project reporting requirements are covered under Section 6 Monitoring and Assessment Plan (there is no specific section as in previous project document formats specifically addressing reporting). It is stated that “project-level monitoring and assessment will be undertaken in compliance with UNDP requirements as outlined in the UNDP POPP and UNDP Evaluation Policy. While these UNDP requirements are not outlined in this project document, the UNDP Country Office will work with the relevant project stakeholders to ensure UNDP M&A requirements are met in a timely fashion and to high quality standards. Additional mandatory GEF-specific M&A requirements (as outlined below) will be undertaken in accordance with the GEF M&E policy”. 173. As with all such projects the key project reporting requirements and responsibilities are: • An annual report based on the Annual Workplan – Project manager • Annual financial report – finance officer and UNDP CO • The UNDP/GEF Project Implementation Report (annual) – Project manager, CO and RTA. • Two periodic independent review reports (midterm and terminal) – independent consultants • Periodic reports to project steering committee (when it meets) – project manager, chair of committee, UNDP E&E representative. In addition to the above there are range of internal project reporting mechanisms.


Tag: Communication Implementation Modality Monitoring and Evaluation Project and Programme management

18.

Communication 

Through the project’s engagement with local stakeholders, there is a good understanding of the aims and objectives of the project on the ground- however, there has been no targeted communication campaigns as yet. Communications. However, to date the level of communication to national/regional stakeholders has been limited – this is perhaps justified in the initial stages of the project while results are still being generated but should now become a priority as part of the investment leverage and replication process.


Tag: National Regional Communication

19.

Sustainability

Overall, the MTR would consider the project as very sustainable in that it a). is implemented through the existing central government and Woreda system and builds on rather than separately contributes to its activities, b). it is building during implementation the capacity and know-how of local stakeholders and testing effective mechanisms for achieving results that these stakeholders can then continue post project, c). the initiatives and activities that make up the ILM approach are technically feasible, and are directly relevant to stakeholders having been selected in consultation with them, d). environmental aspects of the ILM also bring immediate livelihood benefits as well as longer-term environmental benefits, and therefore are more likely to be maintained and supported. 179. However, some sustainability issues do exist and these are discussed in the following sections, as appropriate, with the hope that these issues can be resolved before the issue grows and is identified as a significant risk at the end of the project.


Tag: Efficiency Local Governance Communication Donor relations Human and Financial resources

20.

Socio-economic Risks to Sustainability

The ILM approach, as being applied by the project, appears to have very limited socio-economic risks to sustainability. Contrary to many LD or SLM initiatives the short term economic and livelihood costs (from loss of grazing in rehabilitated lands, etc.) are mainly compensated and as a result are more likely to be maintained until full ecosystem service benefits then also materialize. 184. In most of the project sites there is a high level of social cohesion and an awareness of the need to undertake land use changes and as a result strong social commitment. The exception is perhaps in the mainly pastoral areas where there are greater challenges. 185. Socio-economic risks are further reduced by three factors: a). the project is promoting a diversity of livelihoods and thus the risks related to failure of any specific livelihood is less, b). the options selected were done so based on consultation with Woreda experts and the communities themselves, c). the gender mainstreaming aspect of the project increases the role of women and thereby reduces the risks of socioeconomic problems to the households as a whole. 186. Thus, overall the MTR team consider socio-economic sustainability of the ILM approach piloted by the project as likely.


Tag: Sustainability Gender Mainstreaming Jobs and Livelihoods Awareness raising

21.

Institutional Framework and Governance Risks to Sustainability 

This project is fully integrated into the existing Woreda and Kabele institutional and governance structure. In addition, the “in-process” training involved in implementation, plus the multiple governance and coordination structures (Woreda Steering Committee, technical committee, Gender teams, Watershed Management committees, etc) mean that the work initiated by the project is extremely sustainable in this regard. Furthermore, it should be easily transferable to other Kabele within target Woreda’s and also to other targeted Woreda’s and regions not currently targeted. 188. The possible exception is those Woreda that lack the same level of capacity as the majority (for example those in mainly pastoral areas). However, the capacity gaps in those areas are a challenge for all development efforts and perhaps beyond the scope of the project to be able to change or mitigate easily. 189. At a national level it is more difficult to judge the level of awareness and commitment outside of the EFCCC (i.e. other members of the national steering committee). However, based on the minutes of project Steering committee meetings the other national stakeholders are also well engaged and supportive of the project approach and results. Thus, the overall institutional and Governance sustainability is rated likely


Tag: National Rural Local Governance Rule of law Capacity Building

22.

Environmental Risks to Sustainability

The main point of the ILM approach is to integrate ecosystem management and land use in order to build more resilient production and livelihoods, and thence food security. Thus, intrinsically it is aiming towards environmental sustainability. 191. Nonetheless there are potential risks involved in the introduction of diversified land use. One example is the introduction of small-scale irrigation agriculture in Gursum Woreda in Somali region. This has been extremely successful and is already being replicated by other farmers. It depends principally on extraction of water from the water table directly on the irrigated territory. This does bring with it the potential risk that large-scale replication will lead to over extraction and lowering of the water table, etc. with negative impact on farming livelihoods and environment. Probably this risk is small given the likely rates of replication and fact the watertable is a superficial one (near river) and is annually recharged – however, such “unexpected” repercussions are a potential risk and any activities promoted by the project with such risks need to be carefully monitored. 192. However, the majority of the project initiatives are environmentally beneficial or neutral in impact and so overall the environmental sustainability is considered very likely.


Tag: Agriculture land resouces Food Security Natural Resouce management Water resources Sustainability Jobs and Livelihoods

Recommendations
1

Evaluation Recommendation or Issue 1: Carry out a revision/clarification of Project Results Framework Indicators, Baseline and Targets:

This includes a). revision of existing indicators/baseline/ targets (clearer language, quantitative baselines, consistent parameters, remove duplication, etc.), b). inclusion of clear and quantifiable GEB and FS impact indicators for each outcome. It is recommended this revision is completed by no later than end of 1st Quarter 2020.

2

Evaluation Recommendation or Issue 2: Strengthen Project  monitoring and assessment of GEB and FS impacts in the field and levels of replication:

It is recommended that internal project site monitoring that incorporates impact data (including controls), and measures of replication is developed by no later than end of 1st Quarter 2020. Some suggested activities and methodologies for achieving this is provided in the annex. This monitoring and assessment could be further enhanced through support to relevant targeted Action Research activities by national/regional academic institutions.

3

Evaluation Recommendation or Issue 3: Midterm Planning to consolidate initiated activities and Move forward with so far uninitiated ones

Multiyear Planning at this stage (mid-term) to ensure that all remaining aspects of project implementation are rolled out in the most feasibly way possible in the remaining period of the project duration will be critical to avoiding potential problems. Thus, the preparation of an updated internal multi-year workplan until the project EoP is recommended in 1st Quarter of 2020

4

Evaluation Recommendation or Issue 4: Trouble shooting Implementation barriers in challenging Project sites (Agropastoral) and Learning from initial experience / beneficiaries feedback:

It is recommended that at this mid-point in implementation, and in consultation with the EFCC Commission, the project needs to assess the progress and barriers faced in the agropastoral project sites and identify either ways to try and overcome those, or pragmatic adjustment of expectation / targets in these specific sites. In addition, it is recommended that Project Coordinators in each Woreda undertake a quick review with local partners and beneficiaries of the experience gained during the initial 2 years of the project and the practical lessons learned in terms improving efficiency of the further roll-out of activities during the remaining 2 years of the project. It is recommended this is done during 1st Quarter of 2020 - Any decisions in this regard can then be incorporated into the updated multiyear workplan (see above).

5

Evaluation Recommendation or Issue 5 : Enhancing Impact of the School Clubs:

As described in report text, the MTR team has some concerns on both the sustainability and impact of the school clubs, particularly in terms of what real incentive exists for the members. It is recommended that to enhance both the awareness/knowledge impact and the motivation of members, the project should introduce the addition of “Field trips” – a). to areas within the Woreda that show LD issues in practice and project initiatives to address them, b). to other project sites to experience other agroecological zones and situations. Planning for this needs to be incorporated into 2020 annual work planning and budget.

6

Evaluation Recommendation or Issue 6: Additional Support to Practical Research on high land tree species adaption and cultivation in the Highlands

It is recommended that the project focus more resources to this objective, including seeking practical expertise and knowhow in this regard, particularly within East Africa (notably Kenya).

7

Evaluation Recommendation or Issue 7: Planning in advance the Strategy and Actions Needed to effectively Communicate Project Achievements and advocating ILM approach (as applied by the project) – I.e. A Communication and Replication Plan:

It is strongly recommended that a strategy and plan for achieving this in the final 18/12 months of the project is developed (by end of 2020) and relevant activities added to the project work planning in 2021.

Recruitment of Project Communications and Advocacy Officer:

The above additional activities bring with them additional workload and the need for skill set/experience not currently available in the project. For this reason, it is recommended the project recruit a national Communications and Advocacy Officer to take direct responsibility for the overall implementation of activities and to provide support and guidance to Woreda Field Coordinators on this aspect. Ideally this officer should be recruited before preparation of the “Communication and Replication Plan” and his/her initial task would be to help in its preparation i.e. recruitment recommended mid2020.

8

Evaluation Recommendation or Issue 8: UNDP Ethiopia to apply and Advocate the ILM Model (as applied by the Project) in other Environmental and Rural Development contexts:

As highlighted in the report, the ILM approach/model as applied by the project is applicable to a wide range of natural resource use management contexts irrespective if their primary focus is environmental (as in case of GEF projects) or sustainable rural development, etc. Thus, it is recommended that the ILM approach/model is adopted into the UNDP Ethiopia “tool-box” and applied wherever relevant in other projects and programs in the future.

1. Recommendation:

Evaluation Recommendation or Issue 1: Carry out a revision/clarification of Project Results Framework Indicators, Baseline and Targets:

This includes a). revision of existing indicators/baseline/ targets (clearer language, quantitative baselines, consistent parameters, remove duplication, etc.), b). inclusion of clear and quantifiable GEB and FS impact indicators for each outcome. It is recommended this revision is completed by no later than end of 1st Quarter 2020.

Management Response: [Added: 2020/04/22] [Last Updated: 2021/01/28]

Agreed

Key Actions:

Key Action Responsible DueDate Status Comments Documents
revision of existing indicators/baseline/ targets (clearer language, quantitative baselines, consistent parameters, remove duplication, etc.), b). Inclusion of clear and quantifiable GEB and FS impact indicators for each outcome
[Added: 2020/04/22] [Last Updated: 2020/08/03]
PMU + UNDP 2020/03 Completed completed History
2. Recommendation:

Evaluation Recommendation or Issue 2: Strengthen Project  monitoring and assessment of GEB and FS impacts in the field and levels of replication:

It is recommended that internal project site monitoring that incorporates impact data (including controls), and measures of replication is developed by no later than end of 1st Quarter 2020. Some suggested activities and methodologies for achieving this is provided in the annex. This monitoring and assessment could be further enhanced through support to relevant targeted Action Research activities by national/regional academic institutions.

Management Response: [Added: 2020/04/22] [Last Updated: 2021/01/28]

Agreed

Key Actions:

Key Action Responsible DueDate Status Comments Documents
Prepare PES procedure manual
[Added: 2020/04/22] [Last Updated: 2021/09/04]
PMU 2021/12 Initiated History
3. Recommendation:

Evaluation Recommendation or Issue 3: Midterm Planning to consolidate initiated activities and Move forward with so far uninitiated ones

Multiyear Planning at this stage (mid-term) to ensure that all remaining aspects of project implementation are rolled out in the most feasibly way possible in the remaining period of the project duration will be critical to avoiding potential problems. Thus, the preparation of an updated internal multi-year workplan until the project EoP is recommended in 1st Quarter of 2020

Management Response: [Added: 2020/04/22] [Last Updated: 2021/01/28]

Agreed

Key Actions:

Key Action Responsible DueDate Status Comments Documents
Following through and functionally establish the web-based GIS monitoring system in a meaningful way
[Added: 2020/04/22] [Last Updated: 2020/11/15]
PMU 2020/12 Completed completed History
Initiating in a timely manner the Outputs and activities so far not started such as: PES, insurance, Vital Signs monitoring landscapes system, etc.
[Added: 2020/04/22] [Last Updated: 2020/11/15]
PMU 2020/12 Completed completed History
4. Recommendation:

Evaluation Recommendation or Issue 4: Trouble shooting Implementation barriers in challenging Project sites (Agropastoral) and Learning from initial experience / beneficiaries feedback:

It is recommended that at this mid-point in implementation, and in consultation with the EFCC Commission, the project needs to assess the progress and barriers faced in the agropastoral project sites and identify either ways to try and overcome those, or pragmatic adjustment of expectation / targets in these specific sites. In addition, it is recommended that Project Coordinators in each Woreda undertake a quick review with local partners and beneficiaries of the experience gained during the initial 2 years of the project and the practical lessons learned in terms improving efficiency of the further roll-out of activities during the remaining 2 years of the project. It is recommended this is done during 1st Quarter of 2020 - Any decisions in this regard can then be incorporated into the updated multiyear workplan (see above).

Management Response: [Added: 2020/04/22] [Last Updated: 2021/01/28]

Agreed

Key Actions:

Key Action Responsible DueDate Status Comments Documents
Project Coordinators in each Woreda undertake a quick review with local partners and beneficiaries of the experience gained during the initial 2 years of the project and the practical lessons learned in terms improving efficiency of the further roll-out of activities during the remaining 2 years of the project.
[Added: 2020/04/22] [Last Updated: 2020/11/15]
PMU and woredas 2020/12 Completed completed History
5. Recommendation:

Evaluation Recommendation or Issue 5 : Enhancing Impact of the School Clubs:

As described in report text, the MTR team has some concerns on both the sustainability and impact of the school clubs, particularly in terms of what real incentive exists for the members. It is recommended that to enhance both the awareness/knowledge impact and the motivation of members, the project should introduce the addition of “Field trips” – a). to areas within the Woreda that show LD issues in practice and project initiatives to address them, b). to other project sites to experience other agroecological zones and situations. Planning for this needs to be incorporated into 2020 annual work planning and budget.

Management Response: [Added: 2020/04/22] [Last Updated: 2021/01/28]

Agreed

Key Actions:

Key Action Responsible DueDate Status Comments Documents
It is recommended that to enhance both the awareness/knowledge impact and the motivation of members, the project should introduce the addition of “Field trips” – a). to areas within the Woreda that show LD issues in practice and project initiatives to address them, b). to other project sites to experience other agroecological zones and situations. Planning for this needs to be incorporated into 2020 annual work planning and budget.
[Added: 2020/04/22] [Last Updated: 2020/11/15]
PMU 2020/12 Completed Completed History
6. Recommendation:

Evaluation Recommendation or Issue 6: Additional Support to Practical Research on high land tree species adaption and cultivation in the Highlands

It is recommended that the project focus more resources to this objective, including seeking practical expertise and knowhow in this regard, particularly within East Africa (notably Kenya).

Management Response: [Added: 2020/04/22] [Last Updated: 2021/01/28]

This is not the mandate of the project. However the project will support any initiative to increase the number of adaptable and economically important tree species in the high lands.

Key Actions:

Key Action Responsible DueDate Status Comments Documents
The project will support the initiative of Debrebirhan University and Angollela Tera woreda in conducting the action research to get adaptive tree species in the high lands
[Added: 2020/04/22] [Last Updated: 2020/11/15]
PMU 2020/12 Completed Completed History
7. Recommendation:

Evaluation Recommendation or Issue 7: Planning in advance the Strategy and Actions Needed to effectively Communicate Project Achievements and advocating ILM approach (as applied by the project) – I.e. A Communication and Replication Plan:

It is strongly recommended that a strategy and plan for achieving this in the final 18/12 months of the project is developed (by end of 2020) and relevant activities added to the project work planning in 2021.

Recruitment of Project Communications and Advocacy Officer:

The above additional activities bring with them additional workload and the need for skill set/experience not currently available in the project. For this reason, it is recommended the project recruit a national Communications and Advocacy Officer to take direct responsibility for the overall implementation of activities and to provide support and guidance to Woreda Field Coordinators on this aspect. Ideally this officer should be recruited before preparation of the “Communication and Replication Plan” and his/her initial task would be to help in its preparation i.e. recruitment recommended mid2020.

Management Response: [Added: 2020/04/22] [Last Updated: 2021/01/28]

Agreed

Key Actions:

Key Action Responsible DueDate Status Comments Documents
The project has to review and document lessons learned to advocate the project ILM Model to the government or other donors as an effective approach for future sustainable rural development. This is a key need if the project is to meet its targets in terms of leveraged finances from donors, and if the projects experience is to significantly influence future government policy and programs.
[Added: 2020/04/22] [Last Updated: 2020/11/15]
UNDP EFCCC ,PMU 2020/12 Completed Completed History
8. Recommendation:

Evaluation Recommendation or Issue 8: UNDP Ethiopia to apply and Advocate the ILM Model (as applied by the Project) in other Environmental and Rural Development contexts:

As highlighted in the report, the ILM approach/model as applied by the project is applicable to a wide range of natural resource use management contexts irrespective if their primary focus is environmental (as in case of GEF projects) or sustainable rural development, etc. Thus, it is recommended that the ILM approach/model is adopted into the UNDP Ethiopia “tool-box” and applied wherever relevant in other projects and programs in the future.

Management Response: [Added: 2020/04/22] [Last Updated: 2021/01/28]

Agreed

Key Actions:

Key Action Responsible DueDate Status Comments Documents
ILM approach/model as applied by the project is applicable to a wide range of natural resource use management contexts irrespective if their primary focus is environmental (as in case of GEF projects) or sustainable rural development, etc. Thus, it is recommended that the ILM approach/model is adopted into the UNDP Ethiopia “tool-box” and applied wherever relevant in other projects and programs in the future.
[Added: 2020/04/22] [Last Updated: 2020/11/15]
UNDP and PMU 2020/12 Completed Completed History

Latest Evaluations

Contact us

1 UN Plaza
DC1-20th Floor
New York, NY 10017
Tel. +1 646 781 4200
Fax. +1 646 781 4213
erc.support@undp.org