Final evaluation of the InTerDev project

Report Cover Image
Evaluation Plan:
2016-2020, Kosovo
Evaluation Type:
Final Project
Planned End Date:
05/2017
Completion Date:
09/2017
Status:
Completed
Management Response:
Yes
Evaluation Budget(US $):
10,965

Share

Document Type Language Size Status Downloads
Download document Final TOR International.pdf tor English 297.39 KB Posted 100
Download document Final Evaluation Report.pdf report English 772.36 KB Posted 121
Title Final evaluation of the InTerDev project
Atlas Project Number: 00071978
Evaluation Plan: 2016-2020, Kosovo
Evaluation Type: Final Project
Status: Completed
Completion Date: 09/2017
Planned End Date: 05/2017
Management Response: Yes
Focus Area:
  • 1. Poverty and MDG
  • 2. Others
Corporate Outcome and Output (UNDP Strategic Plan 2014-2017)
  • 1. Output 1.1. National and sub-national systems and institutions enabled to achieve structural transformation of productive capacities that are sustainable and employment - and livelihoods- intensive
Evaluation Budget(US $): 10,965
Source of Funding: 11266 - ADA
Evaluation Expenditure(US $): 13,212
Joint Programme: No
Mandatory Evaluation: No
Joint Evaluation: No
Evaluation Team members:
Name Title Email Nationality
Krenar Loshi Local Consultant krenar.loshi@gmail.com KOSOVO
GEF Evaluation: No
Key Stakeholders: Municipalities of Dragash and Strpce, Local Development Fund from Suhareka and project beneficiaries
Countries: KOSOVO
Lessons
1.

These are built around the key features of the project that need to be maintained in future phases or similar context:

Physical presence of the project through focal points in both benefiting municipalities has proven to be a great value added proposition to the project, securing not only effective and direct communication and follow-up with beneficiaries, but also an added quality assurance.

The present focus on practical training combining on-site advise, study visits, B2B exchanges, with involvement of local action groups have proven to be most beneficial approach to assisting the new farmers joining the scheme.

The current levels of municipal own contributions have proven to strengthen the local ownership and a clear value for money for benefiting municipalities, which paves the way for exploring opportunities for raising the municipal share, and also the share of farmers aiming at expansion.

Based on the feedback of evaluators and performance of the selected applicants, further adjustments should be made in the application form and evaluation grid and ensure continued monitoring of application procedures and tools related to LDF calls. Any knowledge gaps of potential beneficiaries leading to incomplete applications should be addressed through training to reduce the number of rejected applications.

There’s a need to maintain training on environmental issues including training session at the outset of the LDF calls, using extracts from successful and poorer applications as example.

The fore lessons learned are built around the key features of the project that need to be maintained in future phases or similar context:

Presence in the municipalities. Physical presence of the project through focal points in both benefiting municipalities has proven to be a great value added proposition to the project, securing not only effective and direct communication and follow-up with beneficiaries, but also an added quality assurance.

New farmers. The present focus on practical training combining on-site advise, study visits, B2B exchanges, with involvement of local action groups have proven to be most beneficial approach to assisting the new farmers joining the scheme.

Municipal contributions. The current levels of municipal contributions have proven strengthen the local ownership and a clear value for money for benefiting municipalities, which paves the way for exploring opportunities for raising the municipal share, and also the share of farmers aiming at expansion.

Continue monitoring of application procedures and tools related to LDF calls. Based on the feedback of evaluators and performance of the selected applicants, further adjustments should be made in the application form and evaluation grid. Any knowledge gaps of potential beneficiaries leading to incomplete applications should be addressed through training to reduce the number of rejected applications.

Maintain training on environmental issues including training session at the outset of the LDF calls, using extracts from successful and poorer applications as example.


Findings
1.

Mid-term evaluation was carried out at the end of 2015. Recommendations were provided for the remaining project duration.

a) Renew the commitment - 2nd phase of the project. The project followed the recommendations. The ownership is of the municipalities is strong, also evidenced by the commitments in terms of own contribution during the project and also for the future.

b) Rethink the training component. The project followed the recommendations and organized no more capacity building sessions under the component 1. One of the findings of the final evaluation is that in terms of staff training, learning by doing approach (eg. working groups such as LAG, field site visits, study visits) in collaboration with the project staff is an effective way of learning.

c) Do not give up on Dragash/Dragaš Women in Business. Project has provided further support to the SE in developing a catalogue for their products and in matchmaking at many levels. However, despite of contacts with potential buyers no contracts have been established. One of the findings of the ET is that the members of the social enterprise are capable of producing traditional type of handicrafts also which could be an opportunity when connected to tourism.

d) Conduct monitoring on expected impacts. The team has done a great effort on strengthening the data collection on income generation and job creation and refers to this information in reports. Data on employment (for the household members and jobs for hired employees) is now being gathered, making distinction between the types of job created, change in monthly income, gender, ethnicity, age group and PWD, in accordance with the specific target groups. This database, the coverage of which should still be expanded, is a valuable tool for the stakeholders involved in defining the measures and targets for future initiatives, conducting participatory monitoring and evaluation on a continuous basis, based on a systematized way of presenting the results. It is worth emphasizing the importance of having a shared understanding among the stakeholders based on specified definitions for different types of work, making also the clear distinction between job creation and formalization.

d) Actions that could challenge social values and norms related to gender roles. The progress report 1-6/2016 refers to this recommendation, feeding the design of the next stage. The story of the first woman welder in Shtërpcë/Štrpce got wide publicity globally and this is a high potential in terms of future information campaigns related to TEP actions targeted to local audiences. Similarly, highlighting experiences of joint inter-ethnic cooperation between (Albanian and Serbian) raspberry growers could generate high impact with local audiences.

e) Scaling up (municipalities/national level). Recommendation has been taken into account.


Recommendations
1

Develop clear exit strategy

2

Evaluation of farmer’s associations in terms of functioning/organization and economic potential

3

Clustering of TEP actions and delivery of trainings across components

4

Deepen cooperation with central level Institutions (MLSW, MAFRD), as well as with locally based tourist vendors, events, fairs and festivals, tourism associations, ARDA East, LYACs, etc. on the local and regional level.

5

Maintain training on environmental issues

6

Enhance donor coordination to strengthen the linkages to wider development initiatives in the territory as a framework for TEPs and to further develop coordination on other segments of the selected value chains

7

Further analyze and develop possibilities of collaboration with other vocational training centers

8

Marketing and commercialization measures to enhance marketing and commercialization segment (branding, identification of markets, sales agents with social entrepreneur background, product licensing and certification, including through tourism)

9

Neighbour to neighbour know-how focusing on women and interethnic collaboration were possible, primarily targeting those neighbours who have not been able to join the scheme

10

Consider organizing mid-calls or sub-calls within the same call with modified criteria, targeting specific neighbour to neighbour assistance activities

11

Individual purchases instead of group purchases

12

Introduce a reserved budget line for 100 % grants for valid cases fulfilling the criteria

13

Open door policy 

14

Translated contracts with beneficiaries

15

Abolish additional points given for own-contributions above 10% under the LDF

16

Reduce support for expansion of business/farms and focus on new and small farmers joining the scheme

1. Recommendation:

Develop clear exit strategy

Management Response: [Added: 2017/09/06]

A valid point. In the INTERDEV phase 2, the project foresees a development of an exit strategy as one dedicated activity. Funds have been also set aside in the budget for the development of the exit strategy. It will be developed jointly with the municipal LAGs to ensure that all needs of municipalities are addressed, leading to sustainability of the project and future complementary interventions led by the municipalities themselves. In this regard, mechanisms and products done by the project and the LAGs will be embedded in the exit strategy

Key Actions:

Key Action Responsible DueDate Status Comments Documents
Develop a clear and sustainable exit strategy for InTerDev
[Added: 2017/09/06] [Last Updated: 2017/09/18]
Project Staff 2020/03 Initiated The exit strategy will be designed in the year 3 of project implementation. History
2. Recommendation:

Evaluation of farmer’s associations in terms of functioning/organization and economic potential

Management Response: [Added: 2017/09/06]

The project team supports the development of associations throughout the project, both under the LDF and the TEP components. In INTERDEV 2, the project will consider different set of evaluation criteria for associations to benefit from grant schemes, and will also assess the potential of associations in the area in terms of economic development under other project service lines (value chains development, TEP support).

Key Actions:

Key Action Responsible DueDate Status Comments Documents
Set specific criteria and evaluation measures for associations under the project grant schemes.
[Added: 2017/09/06] [Last Updated: 2017/09/18]
Project Staff 2019/12 Initiated History
Assess the potential of associations and eventual support to them under the various service lines of the project (value chains development, TEPs, etc.)
[Added: 2017/09/06] [Last Updated: 2017/09/18]
Project Staff 2019/12 Initiated History
3. Recommendation:

Clustering of TEP actions and delivery of trainings across components

Management Response: [Added: 2017/09/06]

The INTERDEV2 project does indeed envisage clustering TEP actions more closely, with a set of key actions: active labour market measures, grants and advisory services, social entrepreneurship, and organic farming. The actual design of the TEPs will take this into consideration. However, different target groups and evaluation/selection mechanisms may justify a separate division of the TEP actions.

Key Actions:

Key Action Responsible DueDate Status Comments Documents
Design TEPs with effective and efficient clustering of actions in mind, so as to ease delivery of the actions and the subsequent monitoring.
[Added: 2017/09/06] [Last Updated: 2017/09/18]
Project Staff 2017/12 Completed History
Deliver training for end – beneficiaries under the components 2 and 3 while only the training for institutions under component 1.
[Added: 2017/09/06]
Project Staff 2020/12 Initiated
4. Recommendation:

Deepen cooperation with central level Institutions (MLSW, MAFRD), as well as with locally based tourist vendors, events, fairs and festivals, tourism associations, ARDA East, LYACs, etc. on the local and regional level.

Management Response: [Added: 2017/09/06]

Indeed, a further cooperation and more linkages to the central institutions is necessary for the project, also with the view of incorporating the TEP methodology as a means of local economic development into national or local policies. Moreover, effective coordination and collaboration among local-level actors is essential for a good delivery of the project activities and the following desired impact. Therefore, the project will focus heavily on both these aspects of cooperation.

Key Actions:

Key Action Responsible DueDate Status Comments Documents
Deepen further the cooperation with MLSW and MAFRD and other central level institutions, namely with Kosovo Chamber of Commerce, MCYS, and CYAC as well as with MESP/AEP through dedicated meetings, coordination platforms, visibility, and other events
[Added: 2017/09/06]
Programme Staff 2019/12 Initiated
Strengthen further the network with locally based tourist vendors, events, fairs and festivals, tourism associations, ARDA East, LYACs, etc., all important for promotion and marketing of Sharr/Šar region products. (value chains development support, funding of fairs, coordination mechanisms)
[Added: 2017/09/06]
Project Staff 2019/12 Initiated
Work with MLSW and KAS on improving statistics on employment, particularly on the local level
[Added: 2017/09/06] [Last Updated: 2017/09/18]
Project Staff 2019/12 Initiated History
5. Recommendation:

Maintain training on environmental issues

Management Response: [Added: 2017/09/06]

Environmental elements of agriculture, eco-profit approach, biodiversity protection and other are embedded in the INTERDEV 2. The project will pay special attention on mainstreaming environmental considerations throughout the activities, either through dedicated trainings, awareness campaigns, sharing of best practices and more.

Key Actions:

Key Action Responsible DueDate Status Comments Documents
Deliver trainings and other capacity development sessions on environmental considerations of farming and agricultural production, particularly under TEPs, but also for the LDF grantees and other actors in the relevant value chains
[Added: 2017/09/06] [Last Updated: 2017/09/18]
Project Staff 2019/12 Completed History
Organize awareness campaigns, sharing of best practices, and visibility elements on environmental impact of agriculture, biodiversity protection, and more.
[Added: 2017/09/06] [Last Updated: 2017/09/18]
Project Staff 2019/12 Initiated History
6. Recommendation:

Enhance donor coordination to strengthen the linkages to wider development initiatives in the territory as a framework for TEPs and to further develop coordination on other segments of the selected value chains

Management Response: [Added: 2017/09/06]

Donor coordination and complementarity of activities is essential in the framework of the TEPs. The project will reinvigorate a platform for local level collaboration and coordination to maximize impact and eliminate overlapping. 

Key Actions:

Key Action Responsible DueDate Status Comments Documents
Organize a sharing platform with other donors in the partner areas, sharing experiences, synergizing activities and eliminating overlapping.
[Added: 2017/09/06] [Last Updated: 2017/09/18]
Project Staff 2019/12 Initiated History
7. Recommendation:

Further analyze and develop possibilities of collaboration with other vocational training centers

Management Response: [Added: 2017/09/06]

Collaboration with the VTCs of MLSW is essential in the framework of the TEPs, in particular under the almps and other skills development actions. Depending on the design of the various TEP actions and the demand for skills in the partner areas, the project team will further develop possibilities of collaboration with other Vocational training centres who offer skills development courses.

Key Actions:

Key Action Responsible DueDate Status Comments Documents
Depending on the design of the TEP actions, develop collaboration with the vocational training centres of Gjilan/Gnjilane and Ferizaj/Uroševac, also coming from the fact that an additional municipality in the south-east region of Kosovo is part of the project’s second phase (Viti/Vitina Municipality). This will depend on the vocational profiles available in the VTCs and the demand in the partner municipalities.
[Added: 2017/09/06] [Last Updated: 2017/09/18]
Project Staff 2019/12 Completed History
8. Recommendation:

Marketing and commercialization measures to enhance marketing and commercialization segment (branding, identification of markets, sales agents with social entrepreneur background, product licensing and certification, including through tourism)

Management Response: [Added: 2017/09/06]

The project will introduce measures to enhance marketing and commercialization segment (such as branding, identification of markets, sales agents with social entrepreneur background, product licensing and certification, including through tourism) in order to maximize the valorisation of the territory and improve the access of producers to markets and value chains. Moreover, further development of value chains, business-to-business networking, branding on the regional level are all embedded in the phade 2 of the project.

Key Actions:

Key Action Responsible DueDate Status Comments Documents
Explore options to facilitate inter-municipal cooperation initiatives aiming at commercialization, quality assurance and branding of Sharr/ Šar products, cross-municipal enterprises, as well as sponsoring events and festivals, e.g. Regional Gastronomic Fairs, Cultural events, DokuFest, NgomFest in Prizren and similar
[Added: 2017/09/06] [Last Updated: 2017/09/18]
Project Staff 2019/12 Initiated History
9. Recommendation:

Neighbour to neighbour know-how focusing on women and interethnic collaboration were possible, primarily targeting those neighbours who have not been able to join the scheme

Management Response: [Added: 2017/09/06]

A multiplier effect of the project’s activities whereby the impact trickles down to households that have not been direct beneficiaries of the project is desired (through creation of additional jobs, stronger local markets, etc.). The project indeed should focus on facilitating neighbour exchanges and collaboration as appropriate to maximize the impact of the interventions. 

Key Actions:

Key Action Responsible DueDate Status Comments Documents
Introduce measures to enhance neighbour to neighbour exchange (knowledge, experience, share of the new seedlings, etc.). This may be through village meetings, sharing of best practices, measures of positive deviance, and other mechanisms.
[Added: 2017/09/06] [Last Updated: 2017/09/18]
Project Staff 2019/12 Initiated History
10. Recommendation:

Consider organizing mid-calls or sub-calls within the same call with modified criteria, targeting specific neighbour to neighbour assistance activities

Management Response: [Added: 2017/09/06]

The project team will consider organizing mid-calls or sub-calls within the same call under the Phase 2 of the project. However, these calls for grant applications or expressions of interest require a heavy work load from the project team as well as additional resources in evaluation of the calls. This may hinder the effectiveness of these calls, also taking into account the agriculture season, procurement costs possibly rising, and more.

Key Actions:

Key Action Responsible DueDate Status Comments Documents
Explore options and modalities to organize mid-calls and sub-calls with modified criteria, targeting specific neighbour to neighbour assistance activities (e.g. women to women and inter-ethnic neighbour to neighbour assistance activities).
[Added: 2017/09/06] [Last Updated: 2017/09/18]
Project Staff 2018/12 Completed History
11. Recommendation:

Individual purchases instead of group purchases

Management Response: [Added: 2017/09/06]

Group purchases of agricultural inputs under the LDF grant scheme was requested by the donor as means of lowering procurement costs and effectivizing delivery. However, taking into account the feedback received from the beneficiaries, the project will explore further what would be the best mechanism of purchasing, either through groups or on an individual basis, in close collaboration with the end-beneficiaries

Key Actions:

Key Action Responsible DueDate Status Comments Documents
If group purchasing required as per donor’s request, involve representation of farmers to ensure greater compatibility of acquired inputs
[Added: 2017/09/06] [Last Updated: 2017/09/18]
Project Staff 2019/12 Initiated History
Explore options of sub-group purchases – clustering of purchasing various inputs based on beneficiary feedback and preference, allowing for more tailored, yet effective and efficient procurement.
[Added: 2017/09/06] [Last Updated: 2017/09/18]
Project Staff 2019/12 Initiated History
12. Recommendation:

Introduce a reserved budget line for 100 % grants for valid cases fulfilling the criteria

Management Response: [Added: 2017/09/06]

The project will introduce a reserved budget line for 100 % grants for valid cases fulfilling the predefined criteria. These 100% grants are dedicated to households living in extreme poverty, or in dire need of socioeconomic support. However, detailed assessment of this situation and subsequent award of the 100% grant must be well documented and justified, to prevent any misuse of this exception or complaints from other beneficiaries.

Key Actions:

Key Action Responsible DueDate Status Comments Documents
In close coordination with LDF, introduce a reserved budget line for 100 % grants for valid cases fulfilling the criteria of extreme poverty or need (to be applied in certain cases), making it clear and transparent at the outset of the call.
[Added: 2017/09/06]
Project Staff 2019/12 Initiated This practice has been already applied for vulnerable cases during 3 calls in the first stage of the project and will continue during the second stage of LDF grant calls under INTERDEV 2.
13. Recommendation:

Open door policy 

Management Response: [Added: 2017/09/06]

UNDP operates a stakeholder response mechanism in order to capture and address feedback and potential complaints from end-beneficiaries. This mechanism is operated online. However, as suggested, the project will introduce a mechanism to facilitate complaints for those who cannot articulate the complaint in writing (applications), whereby the complaint can be filled orally.

Key Actions:

Key Action Responsible DueDate Status Comments Documents
Establish a complaint response mechanism in the municipal offices, capturing any potential complaints or general feedback locally. This will be done in an anonymous, confidential way and documented.
[Added: 2017/09/06] [Last Updated: 2017/09/18]
Project Staff 2019/12 Initiated History
On the basis of the feedback received, prepare a list of Frequently Asked Questions (FAQ), and submit it to LDF to support information sessions that could possibly be organized after each call, as well as keep these lists available for distribution in the municipal offices.
[Added: 2017/09/06] [Last Updated: 2017/09/18]
Project Staff 2019/12 Initiated History
14. Recommendation:

Translated contracts with beneficiaries

Management Response: [Added: 2017/09/06]

The Contracts under the LDF with farmers will be translated in the three languages by the Project team as suggested.

Key Actions:

Key Action Responsible DueDate Status Comments Documents
Complete the contracts translated into local languages needed for the grant beneficiaries to understand the content. These will then be used for the subsequent calls throughout the project duration.
[Added: 2017/09/06] [Last Updated: 2017/09/18]
Project Staff 2017/12 Completed The project team will send for translation and will ensure quality of translation in local languages for the LDF grant beneficiaries. History
15. Recommendation:

Abolish additional points given for own-contributions above 10% under the LDF

Management Response: [Added: 2017/09/06]

The project team in close coordination with LDF will ensure to abolish additional points given for own-contributions above 10% in the LDF scheme. This may negatively affect the farmers who are in a less favourable socioeconomic situation, who are at the same time the ones the project wants to reach the most. 

Key Actions:

Key Action Responsible DueDate Status Comments Documents
Eliminate additional points given for own-contributions above 10% of the total sum for applicants, in order to ensure that farmers in less financially favourable position are not discriminated.
[Added: 2017/09/06] [Last Updated: 2017/09/18]
Project Staff 2019/12 Initiated The project team is coordinating closely with LDF to ensure the action is implemented. History
16. Recommendation:

Reduce support for expansion of business/farms and focus on new and small farmers joining the scheme

Management Response: [Added: 2017/09/06]

The project team will start to slowly reduce the support for expansion of existing businesses/farms and hand it over to Municipalities to lead with it. The project will explore the best ways to balance the support to existing and established smallholder farmers with new farmers who have not been supported.  

Key Actions:

Key Action Responsible DueDate Status Comments Documents
Explore modalities to reduce support for expansion of business/farms, which is to be slowly handed over to municipalities to coordinate on their own (the next level), and focus on new and smallholder farmers joining the scheme.
[Added: 2017/09/06] [Last Updated: 2017/09/18]
Project Staff 2019/12 Initiated History

Latest Evaluations

Contact us

220 East 42nd Street
20th Floor
New York, NY 10017
Tel. +1 646 781 4200
Fax. +1 646 781 4213
erc.support@undp.org