Terminal Evaluation Mainstreaming Agrobiodiversity into Agricultural Production Systems Ethiopia project

Report Cover Image
Evaluation Plan:
2017-2020, Ethiopia
Evaluation Type:
Project
Planned End Date:
01/2017
Completion Date:
01/2017
Status:
Completed
Management Response:
Yes
Evaluation Budget(US $):
28,939

Share

Title Terminal Evaluation Mainstreaming Agrobiodiversity into Agricultural Production Systems Ethiopia project
Atlas Project Number: 00075747
Evaluation Plan: 2017-2020, Ethiopia
Evaluation Type: Project
Status: Completed
Completion Date: 01/2017
Planned End Date: 01/2017
Management Response: Yes
Focus Area:
  • 1. Poverty and MDG
  • 2. Others
Corporate Outcome and Output (UNDP Strategic Plan 2014-2017)
  • 1. Output 1.3. Solutions developed at national and sub-national levels for sustainable management of natural resources, ecosystem services, chemicals and waste
Evaluation Budget(US $): 28,939
Source of Funding: gef
Evaluation Expenditure(US $): 28,939
Joint Programme: No
Joint Evaluation: No
Evaluation Team members:
Name Title Email Nationality
Vincent Lefebvre Mr. lefevrevinc@gmail.com
Abera Gayesa ETHIOPIA
GEF Evaluation: Yes
GEF Project Title: Mainstreaming Agro-Biodiversity Conservation in to the farming system of Ethiopia
Evaluation Type: Terminal Evaluation
Focal Area: Biodiversity
Project Type: FSP
GEF Phase: GEF-4
GEF Project ID: 3736
PIMS Number: 2913
Key Stakeholders: Minstery of Agriculture , : EBI, FCF, Woredas & Kebeles
Countries: ETHIOPIA
Lessons
1.

1.1.1Lessons learned for the design, implementation, monitoring, and evaluation of the project

Design:

  • Action #1.1 - Co-financing (in-kind): the planning team had anticipated over 60% of co-financing from UNDP and GoE; there is no evidence that these pledges were based on real actions and activities and in any case, no technical support / backstopping or renting of office and GoE participating staff / per-diem can account for such vast amounts of financial resources; this points out towards a lack of proper analysis of UNDP’s added value and on the GoE/UNDP real commitment and financial capacities.

At project formulation stage, co-financing details should be laid down as for the GEF project details so that it becomes a more transparent process.

  • Action #1.2 - Gender strategy: the project design explained in detail the need to include women into the project; otherwise, there is a risk of women exclusion or less than planned / mandatory participation; this should have been assessed through a subsequent gender analysis at inception phase. This approach is not efficient as whatever comes out from these studies is difficult to integrate into a pre-existing project; this is demonstrated in exemplary fashion with small-scale actions focusing on women resulting from MTR recommendations; while these are quite laudable at activity level and indeed benefitting women, they have little impact, project-wise, as they were not integrated within an all-inclusive approach to mainstream gender within the intervention. Still, the project was highly participatory and involved all targeted communities.

A gender strategy must be devised right at formulation stage with concepts, approaches and methodologies already integrated within the log frame right down to activity level; this is a critical step of the project formulation stage as in agriculture, many activities are gender-specific and require different approaches in terms of awareness raising, participation, capacity-building…

  • Action #1.3 - Disbursement trend: the PRODOC had anticipated a disbursement rate decrease over time (high spending rate during the first years and lowest rate on Y5); this approach results in significant implementation delays and activities being scaled back to subsequent years because of lack of capacity at project start-up. Experience shows that the spending rates need to follow up a normal distribution, being (i) lowest at project start-up due to the inception phase (more emphasis on staff recruitment, purchase of initial equipment / means of transport and little or no emphasis on project implementation), (ii) maximised at midterm and (ii) minimised by project‘s end (through an effective exit strategy [e.g. scaling-up and maintenance activities]).

This issue was actually corrected during the inception workshop but the log frame planning remained the same.

  • Action #1.4 - Scaling up effect: the success of these types of projects greatly depends on a multiplication effect; otherwise, they remain one more pilot development project; the project pushed for reforms at federal level through policy reviews and at local level with some resources to be allocated for scaling up the project’s results to neighbouring areas; the interviews showed that the latter was more successful evidencing that good communication combined with exchange visits and incorporating indirect beneficiaries in trainings can create interest for neighbouring communities to become engaged in similar project activities on agrobiodiversity conservation. In any case, the level of financial resources allocated for scaling up was not enough and could be actually part of another intervention.

Future intervention should adopt a approach as for this project: (i) engage into scaling up intervention’s results to neighbouring areas through project site team and involve woreda Administration support (e.g. more funds for neighbouring local Administration to capture and replicate the project’s results) and (ii) supporting the federal level into effectively taking ownership of project results (e.g. funds for validating project’s results, launching influential permanent working groups at federal level on ).

  • Action #1.5 - Selecting the number of project sites to enhance efficiency (more value for money): the choice of selecting four crops with one site each clearly put the emphasis on the piloting nature of the project; however, the project failed to scale up significantly because lobbying at federal level requires an altogether different kind of financial effort (possibly beyond GEF’s scope) to mainstream agro-biodiversity project results within relevant ministries.

This issue points out towards the need for a more inclusive approach at local / regional level starting with several project sites per crop that could bypass the federal level and encourage (i) projects’ results validation (e.g. extensions packages, by-laws) through entire regions, (ii) increase the bargaining power of farmers (through cooperatives) with more substantial production volumes (for direct selling and/or processing) in a way that can attract (regional / international) buyers/agro-processors.

  • Action #1.6 - Supporting service providers: the project focused essentially on building up the capacity of FV cooperatives and providing pathways for commercialization and adding value through agro-processing; little attention was paid to strengthening existing service providers that could support the FV cooperatives; when the project ends, the level of support (in terms of value for money) that these will be able to provide to cooperatives will be key to cooperatives’ development (e.g. technical expertise on agro-processing, credit facilities, expertise in management and financial control…).

Future interventions’ designs should allocate resources for service providers so that cooperatives can have access to relevant services by project’s end in order to accompany their expansion and development.

 

Implementation and M&E:

  • Action #1.7 - Project staff recruitment: although internal recruitment has the advantage of contracting staff that is familiar with the corporate culture of the executing agency (hence, being more swiftly operational), development project implementation requires specific managerial skills that only external staff can master through experience with the donor community; in addition, there is always a risk that internal staff on secondment for specific projects will be still required to perform regular activities in addition to their project-specific tasks.

There is therefore a need to analyse whether internal and/or external recruitments are most appropriate for this type of project; whatever the case, strict recruitment procedures should be followed up with more emphasis on the managerial skills of the potential candidates than on their technical knowledge on agro-biodiversity (e.g. similar experience in managing development projects).

  • Action #1.8 - Exit strategy: projects should have a clear exit strategy with specific activities that will ensure continuity of the project’s achievements; this strategy should be adopted by project’s mid-term although the concept and approach should have been agreed upon right at inception stage; in any case, by mid-term, the project’s staff should be able to identify the critical pathways that would need attention for ensuring project’s sustainability (whether it is scaling up, replication or just maintaining assets) and define activities with corresponding milestones which their completion means official project results’ handing over.

 


Findings
Recommendations
1

Evaluation Recommendation or Issue 1. Recommendations for the future programming/design process:  The design of a project should encompass all key aspects including research and monitoring of effects with clear indicators and analysis to substantiate findings and conclusions. 

Gender mainstreaming should be inserted from the design itself and not as an afterthought, with indicative issues well considered from the beginning, such as gender roles differentials, impact of project on daily workload of women, impact of project on time spent by women, increase in women’s income.

Furthermore, the design of a project should be more realistic and properly resourced.

2

Evaluation Recommendation or Issue 2. Recommendations for the concluding stage of the Project: In situ Conservation: The Project needs to propose and promote clear management arrangements for in situ conservation mechanisms (gene banks, field gene banks, etc.) in order to sustain and reinforce the initial benefits from the Project.

3

Evaluation Recommendation or Issue 3. Recommendations for the concluding stage of the Project: filling the gap: The Project should ‘fill the gaps’ identified in its first stage, in particular gathering information, monitoring and researching vital issues such as productivity, green markets, organic production, as well socio – economic issues such as market value and benefits distribution.

4

Evaluation Recommendation or Issue 4. Recommendations for the concluding stage of the Project: financial sustainability: The Project should work directly with beneficiaries regarding matters of financial sustainability, indicating that the project pilots and implements a first stage of processes but that external funding eventually ends and that they should take over this issue, mainly by re investing some of the benefits they have obtained through the intervention.

5

Evaluation Recommendation or Issue 5. Recommendations for the concluding stage of the Project: gender mainstreaming: A true gender mainstreaming process should be initiated, not only seeking participation of women in activities and making sure of assessing the implications for women and men of any planned action, taking into account the concerns, needs, and experiences of women as part of the Project and making sure that the intervention does not damage women’s access to resources, and that women and men benefit equally, and inequality is not perpetuated.

6

Evaluation Recommendation or Issue 6. Recommendations for the concluding stage of the Project: extension package:  There is a strong need for the Project to push for the completion of overdue products that were supposed to be well underway by the intervention’s mid-term.  A case in point, and a main issue, are the extension packages, which not only will be an effort in meeting with expected outputs but also to secure sustainability and continuous backing for the use of farmers’ varieties for the target crops and for the incorporation of agro biodiversity components for farming in Ethiopia.

7

Evaluation Recommendation or Issue 7. Recommendations for the concluding stage of the Project: police issue:  The Project needs to rejoin the conclusions and recommendations of the gaps identification and framework analysis and provide an impetus for the application of at least some of the recommendations for national – level policies in order to impulse the setting up of upgrading of frameworks that promote the mainstreaming of agro biodiversity conservation, including the use of farmers’ varieties, and for national structures to work with local authorities and farmers’ and community groups in this subject.

8

Evaluation Recommendation or Issue 8. Recommendations for the concluding stage of the Project: knowledge management:  A knowledge management process and translation needs to take place so that the materials being developed or that will be developed in the second project phase are more ‘user friendly’

9

Evaluation Recommendation or Issue 9. Recommendations for the concluding stage of the Project: capacity building: The Project should also strengthen the capacities and provide adequate local project management support so that the project sites can have adequate technical expertise, infrastructure, and guidance to properly implement the local aspects as well as to unquestionably and systematically monitor effects, impacts and processes.

10

Evaluation Recommendation or Issue 10. Recommendations for the concluding stage of the Project: scaling up of the project results: The second phase of the Project should also be stage where its catalytic effect is harnessed in order to expand (to other crops), to other regions (within Ethiopia and at the regional African – level) as well as to upscale.

1. Recommendation:

Evaluation Recommendation or Issue 1. Recommendations for the future programming/design process:  The design of a project should encompass all key aspects including research and monitoring of effects with clear indicators and analysis to substantiate findings and conclusions. 

Gender mainstreaming should be inserted from the design itself and not as an afterthought, with indicative issues well considered from the beginning, such as gender roles differentials, impact of project on daily workload of women, impact of project on time spent by women, increase in women’s income.

Furthermore, the design of a project should be more realistic and properly resourced.

Management Response: [Added: 2017/11/30]

Management response: agreed, for the future program designing encompass all key aspects including research and monitoring of effects with clear indicators and analysis to substantiate findings and conclusions.

Key Actions:

Key Action Responsible DueDate Status Comments Documents
Raise awareness on police recommendations & marketing strategy.
[Added: 2017/11/30]
EBI,PCU, project site 2017/01 Completed Both documents (police recommendation & marketing strategy) are developed and under implementation.
2. Recommendation:

Evaluation Recommendation or Issue 2. Recommendations for the concluding stage of the Project: In situ Conservation: The Project needs to propose and promote clear management arrangements for in situ conservation mechanisms (gene banks, field gene banks, etc.) in order to sustain and reinforce the initial benefits from the Project.

Management Response: [Added: 2017/11/30]

Management response: agreed

Key Actions:

3. Recommendation:

Evaluation Recommendation or Issue 3. Recommendations for the concluding stage of the Project: filling the gap: The Project should ‘fill the gaps’ identified in its first stage, in particular gathering information, monitoring and researching vital issues such as productivity, green markets, organic production, as well socio – economic issues such as market value and benefits distribution.

Management Response: [Added: 2017/11/30]

Management response: agreed, productivity increment will be addressed by implementing the prepared Farmers’ Varieties extension package. However, addressing green market and organic production issue are time demanding as well as have controversial nature to address within short period of time. However for future project this point is well taken. 

Key Actions:

Key Action Responsible DueDate Status Comments Documents
Facilitate research works & implement the results obtained from the research.
[Added: 2017/11/30]
EIAR, EBI, PCU 2017/01 Completed The EIAR conducted the research and submitted the report as per the agreed term.
4. Recommendation:

Evaluation Recommendation or Issue 4. Recommendations for the concluding stage of the Project: financial sustainability: The Project should work directly with beneficiaries regarding matters of financial sustainability, indicating that the project pilots and implements a first stage of processes but that external funding eventually ends and that they should take over this issue, mainly by re investing some of the benefits they have obtained through the intervention.

Management Response: [Added: 2017/11/30]

Management response: Agreed.

Key Actions:

Key Action Responsible DueDate Status Comments Documents
Pave the way for sustaining the project results/achievements.
[Added: 2017/11/30]
PCU, Project site 2017/01 Completed The project strongly worked on the sustainability issues
5. Recommendation:

Evaluation Recommendation or Issue 5. Recommendations for the concluding stage of the Project: gender mainstreaming: A true gender mainstreaming process should be initiated, not only seeking participation of women in activities and making sure of assessing the implications for women and men of any planned action, taking into account the concerns, needs, and experiences of women as part of the Project and making sure that the intervention does not damage women’s access to resources, and that women and men benefit equally, and inequality is not perpetuated.

Management Response: [Added: 2017/11/30]

Management response: agreed 

Key Actions:

Key Action Responsible DueDate Status Comments Documents
Facilitate mainstreaming of gender into all activities of the project.
[Added: 2017/11/30]
PCU, Project site 2017/01 Completed This actions is completed
6. Recommendation:

Evaluation Recommendation or Issue 6. Recommendations for the concluding stage of the Project: extension package:  There is a strong need for the Project to push for the completion of overdue products that were supposed to be well underway by the intervention’s mid-term.  A case in point, and a main issue, are the extension packages, which not only will be an effort in meeting with expected outputs but also to secure sustainability and continuous backing for the use of farmers’ varieties for the target crops and for the incorporation of agro biodiversity components for farming in Ethiopia.

Management Response: [Added: 2017/11/30]

Management response:  agreed.

Key Actions:

Key Action Responsible DueDate Status Comments Documents
Facilitate the implementation of the FVs extension package.
[Added: 2017/11/30]
PCU, Project site 2017/01 Completed The extension package for four crops was validated by experts. Continuous trainings organized for concerned experts to facilitate the implementation of the package at grass root level.
7. Recommendation:

Evaluation Recommendation or Issue 7. Recommendations for the concluding stage of the Project: police issue:  The Project needs to rejoin the conclusions and recommendations of the gaps identification and framework analysis and provide an impetus for the application of at least some of the recommendations for national – level policies in order to impulse the setting up of upgrading of frameworks that promote the mainstreaming of agro biodiversity conservation, including the use of farmers’ varieties, and for national structures to work with local authorities and farmers’ and community groups in this subject.

Management Response: [Added: 2017/11/30]

Management response: agreed 

Key Actions:

Key Action Responsible DueDate Status Comments Documents
Facilitate capacity building work for framers associations /cooperatives & local institutions.
[Added: 2017/11/30]
PCU, Project site 2017/01 Completed Capacity building addressed based on the assessment of the capacity gap.
8. Recommendation:

Evaluation Recommendation or Issue 8. Recommendations for the concluding stage of the Project: knowledge management:  A knowledge management process and translation needs to take place so that the materials being developed or that will be developed in the second project phase are more ‘user friendly’

Management Response: [Added: 2017/11/30]

Management response: agreed  and translation of the key project  products will be done in Amharic, Kembategna and Oromipha

Key Actions:

Key Action Responsible DueDate Status Comments Documents
Scaling up the project results & best practices.
[Added: 2017/11/30]
PCU, Project site 2017/01 Completed Scaling up of the project achievements coordinated at all level.
9. Recommendation:

Evaluation Recommendation or Issue 9. Recommendations for the concluding stage of the Project: capacity building: The Project should also strengthen the capacities and provide adequate local project management support so that the project sites can have adequate technical expertise, infrastructure, and guidance to properly implement the local aspects as well as to unquestionably and systematically monitor effects, impacts and processes.

Management Response: [Added: 2017/11/30]

Management response: agreed . The project has got one focal person in each woreda and has been working through the existing government structure. Therefore additional capacity building effort will be made in addition to the past.

Key Actions:

Key Action Responsible DueDate Status Comments Documents
Facilitate training & workshop on agro-biodiversity conservation approaches.
[Added: 2017/11/30]
Project site 2017/01 Completed Awareness creation workshops and trainings organized to effectively mainstream Agro-biodiversity conservation.
10. Recommendation:

Evaluation Recommendation or Issue 10. Recommendations for the concluding stage of the Project: scaling up of the project results: The second phase of the Project should also be stage where its catalytic effect is harnessed in order to expand (to other crops), to other regions (within Ethiopia and at the regional African – level) as well as to upscale.

Management Response: [Added: 2017/11/30]

Management response: Agreed.

Key Actions:

Latest Evaluations

Contact us

1 UN Plaza
DC1-20th Floor
New York, NY 10017
Tel. +1 646 781 4200
Fax. +1 646 781 4213
erc.support@undp.org