Promoting Inclusive Labour Market Solutions in the Western Balkans

Report Cover Image
Evaluation Plan:
2018-2021, RBEC
Evaluation Type:
Final Project
Planned End Date:
09/2018
Completion Date:
11/2018
Status:
Completed
Management Response:
Yes
Evaluation Budget(US $):
13,200

Share

Document Type Language Size Status Downloads
Download document ToR_Evaluation_ILMS_Final.pdf tor English 167.71 KB Posted 347
Download document UNDP WB ILM Final Evaluation Report 2018_final.pdf report English 1710.55 KB Posted 593
Download document Annex 4 Logframe matrix.docx related-document English 137.59 KB Posted 350
Title Promoting Inclusive Labour Market Solutions in the Western Balkans
Atlas Project Number: 00095840
Evaluation Plan: 2018-2021, RBEC
Evaluation Type: Final Project
Status: Completed
Completion Date: 11/2018
Planned End Date: 09/2018
Management Response: Yes
UNDP Signature Solution:
  • 1. Poverty
Corporate Outcome and Output (UNDP Strategic Plan 2018-2021)
  • 1. Output 1.1.2 Marginalised groups, particularly the poor, women, people with disabilities and displaced are empowered to gain universal access to basic services and financial and non-financial assets to build productive capacities and benefit from sustainable livelihoods and jobs
SDG Goal
  • Goal 1. End poverty in all its forms everywhere
  • Goal 8. Promote sustained, inclusive and sustainable economic growth, full and productive employment and decent work for all
SDG Target
  • 1.4 By 2030, ensure that all men and women, in particular the poor and the vulnerable, have equal rights to economic resources, as well as access to basic services, ownership and control over land and other forms of property, inheritance, natural resources, appropriate new technology and financial services, including microfinance
  • 8.5 By 2030, achieve full and productive employment and decent work for all women and men, including for young people and persons with disabilities, and equal pay for work of equal value
Evaluation Budget(US $): 13,200
Source of Funding: Project
Evaluation Expenditure(US $): 13,200
Joint Programme: No
Joint Evaluation: No
Evaluation Team members:
Name Title Email Nationality
Thomas Vasseur Independent Evaluator thomas.vasseur@gmail.com
GEF Evaluation: No
Key Stakeholders: Country Offices, ILO
Countries: ALBANIABOSNIA AND HERCEGOVINAKOSOVOMACEDONIA, THE FORMER YUGOSLAV REPUBLIC OFMONTENEGROSERBIATURKEY
Lessons
1.

The regional dimension of ILMS is not just an add-on but offers a strong added-value as it
allows all of the targeted countries and territories to benefit from the experience of its
neighbours and use it for its own benefit.
ILMS suffers from an unfair syndrome: Most of the great impact it creates, and while ILMS is
“everywhere”, it is not visible. This fact calls for a pro-active and creative stance in that
respect, with the help of digital technology.
ILMS is a “knowledge monster”: it generates so much great and useful knowledge from its
numerous activities. There is a clear risk of a part of it being lost or not followed upon if not
managed properly. Here again, web-based technology can be of great assistance.
While favouring a participative, two-way bottom-up and top-down approach, ILMS still is to
tackle the reconnection of its activities between the central and local institutional level.


Findings
1.

2.1.1. Relevance, clarity of objectives. Relevance to national, regional policies.
Finding Relevance 1 (FR1) The strategic and policy relevance are strong and comprehensive, obviously resulting from a long-term field experience and analysis.
The project proposal clearly explains how it is supportive of all the existing national and regional-level policy instruments in the countries/territories of intervention.

 


Tag: Relevance Strategic Positioning UN Agencies Jobs and Livelihoods

2.

Finding Relevance 2 (FR2) ILMS is lacking a clear formulation of the longer-term strategy it pursues.

While the initial proposal clearly describes its logical framework, it does not clearly describe its final or sustainable objectives; neither does it describes various phases leading to its final goal.
While ILMS proposes to enhance the capacities of institutions, its progress towards reaching a longer-term objective is, however, not marked with the necessary benchmarks paving the way until those institutions will be fully able to deliver inter-institutional collaborative and human-centred services. More than just indicators of implementation progress, the project needs to fit into a scale equipped with progress markers towards institutions are in full capacity of delivering user-centred approaches, inclusive of hard-to-employ groups.
The theory of change of the project is aiming at raising-awareness, building interest and capacities though it does not describe the process of achieving impact. Then, with reference to the previous paragraph, the transformative process leading to sustainability (i.e., the institutionalisation of introduced practices), is also missing or insufficiently described. ILMS appears as the first of several phases that will lead to durable change.


Tag: Relevance Programme/Project Design Strategic Positioning Theory of Change Capacity Building Institutional Strengthening

3.

2.1.2. Project and monitoring mechanisms
Finding Relevance 3 (FR3) The project design has a strong relevance, weakened by an excess of activities and a certain disconnection of the asylum seeker component from the rest. With an array of activities approaching multi-geographic levels (municipal, provincial, central), multi-strategic levels (policy, programme, project), multi-actor, multi-country/territory and regional, there is a perception of the project attempting to tackle all fronts at the same time.


Tag: Relevance Programme/Project Design Strategic Positioning Education Jobs and Livelihoods Private Sector Women and gilrs Youth

4.

Finding Relevance 4 (FR4) The implementation approach is highly consultative, participatory and flexible to securing interest, buy-in and future ownership of beneficiary institutions. The project approach has obviously given a strong consideration of all constraints inherent to multi-country, multi-level and multi-stakeholders interventions.
A key achievement of this project is that its participatory and consultative approach has raised awareness, obtained the interest and secured the buy-in of all institutions and stakeholders across the Western Balkans.
While this is an essential condition for institutions to deliberately adopt newly introduced practices, the evaluation clearly sees that the ILMS remains merely a first step of a longer strategy leading to institutionalisation.
The “3D” or 3-dimensional approach (i.e. involving a simultaneous intervention at local/provincial level, central level and regional level) is particularly appropriate to be inclusive of all stakeholder, including the vulnerable unemployed, combine field practice with national policy vision. More focus could have been put on the traditional weak institutional linkage between the central and local level. Though, this is a focus potential more relevant to the recommended next phase of ILMSP where piloting should involve both levels more intensively.
The consultative and participatory attitude standing behind each project activity involving institutions has been unanimously appreciated across the region.


Tag: Relevance Implementation Modality Ownership Jobs and Livelihoods

5.

Finding Relevance 5 (FR5) Both UNDP and ILO are relevant implementing agencies for this project as they combine the technical expertise and field operability required by the country-tailored and regional dimension features of the ILMS. The relevance of the respective roles and responsibilities of each agency could be further reviewed.
The two agencies offer all the expertise, experience, operational capacity, credibility that such a project as ILMS requires. However, the rationale that has guided the division of the respective roles and responsibilities has not appeared very strongly to the evaluation. In practice, UNDP and ILO have worked very well together, based on willingness and dedication. However, the external perspective on agencies’ role is that relevance and efficiency can be increased, when reviewing the level of intervention (central, local), the nature of activities, and the mobilisation of expertise. The division of responsibilities at the activity level versus broader component level has not facilitated the overall perception of the coherence of the project to local stakeholders who have not always connected, sometimes been aware, that the various activities did belong to a single project. ILMS may have provided a more formal explanation about the allocation of responsibilities between UNDP and ILO to all project stakeholders.


Tag: Relevance Programme Synergy Strategic Positioning UN Agencies

6.

Finding Relevance 6 (FR6)

Monitoring mechanisms are relevant and the implementation follow-up has been effectively performed. The outcome and activity-level indicators have been relevant to monitoring implementation progress. Project partners have been systematically provided the space and opportunities to express their satisfaction feedback.
The project would benefit from establishing a strategic-level monitoring framework, so progress towards key strategic milestones, such as the institutionalization of new practices can be measured.


Tag: Relevance Monitoring and Evaluation

7.

2.1.3. Relevance to targeted groups
Finding Relevance 7 (FR7) The project is highly relevant to local, central institutions and their need to reform their policies and practices. The concrete, TEP-based interventions have also directly answered the needs of the most vulnerable groups, with variations in sustainability of created jobs; often undermined by limited financial resources.
The involvement of the private sector in several of the ILMS activities (TEP, Business leadership networks, publications on inclusiveness of people with disabilities to the private sector…) has confirmed this stakeholder can make an instrumental contribution to making PES and CSW policy and delivery of user-centred services gain in efficiency and effectiveness.
 


Tag: Relevance Jobs and Livelihoods National Institutions Private Sector Vulnerable

8.

2.1.4. Adjustment to changes in the sector of intervention
Finding Relevance 8 (FR8) The flexible and consultative project approach has been highly relevant to adjusting to changes, starting with establishing the project during the inception phase. The elevated number of project activities has stretched the project flexibility to the maximum and has intensively and extensively taped in the human resources. Not to repeat the findings of the previous section, the evaluation has found the elasticity demonstrated by the project is the clear result of two aspect: the project approach (stakeholder consultation) and human dedication (also using in-country agency presence to get the support required).


Tag: Relevance Human and Financial resources Programme/Project Design

9.

2.2.1. Achievement of results, Improvement of beneficiary institutions, lives of beneficiaries
Finding Effectiveness 1 (FE 1) Overall most target results have been achieved, apart from some of the activities related to asylum seekers. A summary of key achievements has been listed under this Report’s Section 1.1 Project Background. Though, this could be considered as a minimal requirement, reaching activity and component targets remains a major accomplishment. ILMS is content-rich (activity-intensive), its installation requires a long and substantial preparation phases, involving, among other steps, interactions with an impressive number of stakeholders. The inception phase has absorbed close to a quarter of the time allocated for implementation and has further squeezed an already dense implementation schedule. Unsurprisingly, a project extension was necessary to implement the long list of activities, and still, the evaluation has found the timeframe tight, especially, since the project cannot independently set the pace of implementation but depends on institutions’ availability.


Tag: Effectiveness Displaced People National Institutions

10.

2.2.2. Appropriateness of delivery and management mechanisms
Finding Effectiveness 2 (FE 2) The project has made the best use of limited staffing to ensure the overall delivery of activities. However, a more appropriate division of tasks among ILO and UNDP, as well as more project staff could have been more effective.
A tight management and coordination, using personal involvement of UNDP and ILO regional and country staff has compensated a disproportion between the volume of work and the human resources allocated to conduct the numerous tasks. The distribution of activities has been logically made based on the agency’s pre-existing operational capacity presence in the country as well as on the level of intervention though not systematically. For instance, in former Yugoslav Republic of Macedonia and Montenegro, UNDP is closely working with both the Ministry of Labour and the National Employment Agency, thus at the policy and programme level but also during the project implementation. The countries/territory of the region are characterized by varying stages of developments in terms of state of reform progress, requiring tailored interventions. Notwithstanding, the ILO and UNDP presence is not uniform across the region. These elements are indicating that the distribution of management roles and responsibilities has to take into account agency and country context specifics, but also the specific technical and operational capacities at regional level.


Tag: Effectiveness Human and Financial resources Operational Efficiency Project and Programme management Jobs and Livelihoods

11.

2.2.3. Accessibility of target groups (end-users) to project services
Finding Effectiveness 3 (FE 3) The multi-level and multi-stakeholder approach introduced by the project has ensured an effective access of project activities to intended beneficiaries at various levels of implementation (hard to employ beneficiaries at the municipal level, institutions at all central and local levels). The multi-stakeholder modality of TEP-based interventions has also permitted a thorough identification, outreach and selection of vulnerable beneficiaries excluded from the labour market. Institutions, civil society organisations, employers and the unemployed involved in ILMS activities have praised the opportunities created by the project for being involved at each of their respective level, from women victims of domestic violence to Ministry of Labour representatives. The project has created the conditions to involving a wide range of actors. One challenge though is for the project to continue maintaining this access and relations with stakeholders over time. The evaluation has noted that ILMS has intelligently used each agency’s already established relationships in the country to implement project activities.


Tag: Effectiveness Gender-Based Violence Civil Societies and NGOs National Institutions Vulnerable Women and gilrs

12.

2.2.4. Adaptation to changing conditions to ensuring achievement of results
Finding Effectiveness 4 (FE 4) ILMS project team has gone out its way to maintain a high level of flexibility to adjust to the availability of the institutions’ – especially central level institutions (ministries of labour, national employment agencies) already busy agenda to obtain a high degree of involvement and interest in the promotion of innovative practices ILMS has introduced. This is especially true for regional events where bringing together representatives of institutions from 5 different countries/1 territory at the same place and at the same time represents an evident challenge and time-consuming effort.


Tag: Effectiveness Innovation Project and Programme management National Institutions

13.

2.2.5. Target group satisfaction
Finding Effectiveness 5 (FE 5) Project events - trainings, exchange workshops, self-assessments have been very positively welcome overall and qualified as “mind-changing”. The feedback recorded from participants from attending institutions and other stakeholders indicates a very high level of satisfaction across various activities. The satisfaction relates to the novelty of the approach and the tools introduced by ILMS. Institutions’ consultation during the design of ILMS but also as a constant way of interacting during the development and implementation of activities have created a genuine feeling of ownership of the project activities and processes enabling a full involvement of project partners. Central and local PES and CSW have equally appreciated the innovative approaches proposed by the self-assessment and audits tools. For most, this was a first and a striking experience where employment and social services could review their practices and develop an awareness of their gaps in terms of practices towards achieving more efficient and usercentred services.


Tag: Effectiveness Country Government National Institutions

14.

2.3.1. Efficient use of resources to achieving results
Finding efficiency 1 (FEF 1) The project scores very high when comparing the allocated financial and human resources with the number of activities delivered. However, this approach is not sustainable in the long run. The high cost per activity efficiency rate is primarily the result of tremendous commitment of staff who combines both the experience and the skills for the tasks. However, while this can be considered as a success in the short run, maintaining such a high ratio is exposing to an overload of work for the project staff in the mid-term.


Tag: Efficiency Human and Financial resources Operational Efficiency Project and Programme management

15.

Finding Efficiency 2 (FEF2)

The level of financial, human and time resources has not been adequate to the level of ambition of the project. Without entering into details, the comparison between the number, nature of activities – especially TEP and regional workshops, which are costly by nature – and the overall budget provides a strong indication that the budget was underestimated or the scope of activities were overestimated. The motto “It is sometimes expensive to go cheap” can be applied to this project. Hopefully, this is not the case, as this project has achieved key steps towards sustainability. This is mostly owing to the project team’s impressive level of effort and dedication. Without such personal involvement, there is a strong likeliness those results would not have been achieved.


Tag: Efficiency Human and Financial resources Operational Efficiency

16.

Finding Efficiency 3 (FEF 3) The territorial employment approach has brought efficient results. However, the TEP project interventions component is disproportionately smaller than the preparation effort.
As an introductory note to this finding, it is worth mentioning that the TEP model is not new and has been implemented in the past in other regions of the world. However, those earlier TEPs were larger in scale and in monetary value as they often took place in economically depressed areas where no other significant job creation investments where existing. In the context of ILMS, TEP schemes have been planned as small component since the main purpose lied in introducing the territory-based approach rather than offering a major financial investment in the municipalities of interventions. As a matter of fact, a part of the TEP process with the ILMS consisted of identifying other job-creation interventions in the selected areas so those could be included during the TEP audit phases and so TEP smaller interventions could be developed in synergy with existing programmes.


Tag: Efficiency Regional Jobs and Livelihoods

17.

2.3.2. Timeliness of activity delivery
Finding Efficiency 4 (FEF 4) A tight project duration with a long, underestimated installation phase, caught up thanks to intense staff dedication As already mentioned in the relevance findings section, the evaluation considers the project duration is too tight for a long-term change-oriented intervention such as ILMS. To start with, deploying a project with multiple activities, multiple level of interventions, multiple actors, multiple countries/territories on top the regional dimension, can be expected to require a considerable start-up phase before it can engage into activities. With limited project staffing and institution representative availability as two other constraints, 20 months or 24 months is indeed very short to achieving project goals.


Tag: Efficiency National Regional

18.

2.3.3. Optimisation of use of resources
Funding efficiency 5 (FEF 5) The project has realized a strong optimisation of resources. The possibilities of cost-efficient alternatives are usually limited when project resources are limited. While not always timely, ILMS activities have been delivered with modest costs and limited human resources. This is considered as an achievement when considering the number of external consultancy required by the nature of activities. Regional events are usually costly actions as they involve travel, accommodation and related costs for a short duration. The use of technology in remote communication sometimes offers a cheaper alternative. ILMS staff in the region did not have the opportunity to meet physically during the project implementation phase. However, since there is so much knowledge and experience accumulated within the ILMS team, the evaluation believes, provided an ILMS II comes to life, that organizing a project team meeting should be considered as a matter of priority, not only during a possible ILMS II preparation effort, but also, at least once a year. Indeed, the same way institutions benefit from exchanging experiences from the region, the project team must be provided with the opportunity to share experiences, perspectives on similar activities implemented in different contexts.


Tag: Efficiency Human and Financial resources Operational Efficiency Project and Programme management

19.

2.3.4. Project management and coordination mechanisms
Finding Efficiency 6 (FEF 6) Both internal and external coordination mechanisms have functioned well as they are needs-based. Internally, be it at country or regional level, UNDP and ILO have maintained an intense level of exchange; which has proven highly relevant in a project requiring so much flexibility, given the number, diversity of events as well as the management of numerous external consultants. Had money and time allowed, organising regional project staff experience capitalisation workshop could have been beneficial for the exchange of experience in the course of implementation.


Tag: Efficiency Operational Efficiency Project and Programme management Coordination

20.

Finding Efficiency 7 (FEF 7) The high delivery rate of the ILMS is an indicator of an efficient project management and coordination mechanisms, under tight staffing conditions. Lesson are to be learned from the division of roles and responsibilities between UNDP and ILO. This efficiency is seen as the result of intense and regular exchange between UNDP and ILO both at regional and country level. Having UNDP and ILO managing certain activities at different level have given the impression of a disconnect, with activities belonging to two distinct projects. There has been an intelligent and efficient integration of the ILMS to existing UNDP and ILO country programmes. Such is the case in Kosovo where the ADA-funded INTERDEV has provided significant expertise and transfer of knowledge as well as hands-on know-how to the Gjakova TEP. Sometimes combining ILMS resources with other interventions targeting employment has created synergy and added value. Overall, the management task has been complicated by having different staff managing different activities making it time and energy consuming to follow all activities and gather information.


Tag: Efficiency Project and Programme management UN Agencies Coordination

21.

2.4.1. Main direct impact and contribution to higher objectives
Finding Impact 1 (FI 1) “Mind-changing”: Owing to a highly participatory approach and innovative tools, ILMS has achieved a very strong “buy-in” of institutions and changed their perspectives on their own practices. ILMS has created a strong impact across the region. Often referred to as “mind-changing” or “this is the right thing”, across the region, across stakeholders, the evaluation has gathered a strong appreciation of the relevance of the ILMS approach (see section 1.1. Project Background, page 10 for a description of the ILMS approach) and the importance of the issues it tackles.


Tag: Impact Sustainability Jobs and Livelihoods National Institutions

22.

2.4.2. Indirect positive and/or negative impacts
Finding Impact 2 (FI 2) There are examples of concrete steps that go beyond “buy-in”. These sorts of impacts stand between the direct and indirect consequence of the ILMS intervention. Whatever the case, ILMS has done more than sparked an inter-institutional dialogue but has, in some instances, initiated a change in the collaboration between institutions at the local level. In Mojkovac, the project intervention has sparked a dialogue between the CSW and the employment office who have made the joint decision to increase their collaboration and share respective data on users in order to crosscheck information on the same individuals. Regional employment office in Pale is also going to take concrete steps towards a more efficient collaboration with the CSW.


Tag: Impact Regional Jobs and Livelihoods National Institutions

23.

Finding Impact 3 (FI 3) Special attention is to be paid to the TEP individual interventions Numerous findings can potentially be extracted from TEPs since these interventions are substantial and potentially projects on their own. While, the evaluation believes TEP interventions are worth a closer review, the present assessment has focused on an impact based findings since this is about concrete, employment-generating interventions. The idea behind the TEPs was to promote the model of public-private-civil society partnership as well as to support UNDP Country Offices, local communities in mobilizing additional contributions.


Tag: Impact Resource mobilization Innovation Partnership Jobs and Livelihoods National Institutions

24.

2.4.3. Broader effects and impacts, including on gender and environment
Finding Impact 4 (FI 4) Understanding exclusion to reduce discrimination. Discovering the value of individuals instead of the burden they are often perceived as, is a precondition for institutions to change the perception of their own roles. ILMS, through its self-awareness exercises, peer-review workshops and related events has brought together institutions and other stakeholders to take a closer look at the factors of social exclusion. By increasing their level of understanding, they have taken a new perspective on vulnerable communities, look at their potentials rather than just their limitations. If minds have changed overall, all vulnerable groups are not equally placed in front of prejudices. And as much as there is a link between the level of prejudice and the degree of social exclusion, there is also a relation between the level of effort to be invested and the degree of social exclusion.


Tag: Impact Gender Equality Gender-Based Violence Vulnerable Women and gilrs

25.

2.4.4. Impact of on beneficiaries’ lives
Finding Impact 5(FI 5) Most TEP interventions have demonstrated a capacity to produce positive impact, especially on local institutions and the hard-to-employ. However, there is room to increase its potential impact. The territorial approach is understood as supporting the process of enhancing a strong multistakeholder involvement in the design of inclusive labour market solutions. (Please see the section 2.1.2 Project Monitoring Mechanisms for a more detailed presentation of the TEP model).
The territorial focus of the TEP model has successfully mobilized and built interest of a wide range of stakeholders at local level. The case of the TEP in Vranje has indicated that, without a strong involvement of the municipality, and, in the absence of synergy with other jobcreation programmes, the small budget size of TEP interventions remained insufficient to create sustainable employment solutions. However, when, for instance, combined with complementary UNDP interventions, such as honey producers in Kosovo*, this model demonstrates its full potential. Probably, if TEP is able to catalyse key economic stakeholders and focus on attracting investments, an enhanced version is likely to increase the potential impact of TEP, in terms of job creation and a highly inclusive local economic development.


Tag: Impact Programme Synergy Jobs and Livelihoods National Institutions

26.

2.5.1 Sustainability of project results
Finding Sustainability 1(FF 1) Clarifying whether sustainability means institutionalisation The relevance section of this report indicates that ILMS seems to constitute the first phase of a longer intervention where achieving sustainability is the ultimate goal. This is a strategylevel decision to be made. Three central questions may help in this process. The first one is for the project: “How much of the project results, introduced approaches, will the public institution use in the long term?” And the second question to UNDP and ILO is “What is the role and space for both agencies in this process?” The third question is to donors: “What share of the process is to be financially supported by international donors and at which point Western Balkans governments will assume the cost of the process?”
Taking the example of Integrated Case Management, the feedback from interviewers indicates that ILMS has enabled PES institutions to get an understanding of the tool, perform self-assessment to identify the next steps necessary to conduct the process of building a tailored ICM. However, those steps correspond to substantial activities (functional review of services, legislation analysis…) of a process that needs to be developed. Centres for Social Work also need to undergo a similar process which implementation will require times and the continuation of the ILMS project.
To date, the evaluation has gathered enough elements confirming the key pre-conditions validating the continuation of the process towards sustainability, have been met: Not only the awareness and interest are strong, but the request for the ILMS to continue has been consistently expressed by the consulted stakeholders during the field evaluation mission.


Tag: Effectiveness Impact Sustainability National Institutions

27.

2.5.2 Sustainability factors and conditions
Finding Sustainability 2 (FF 2) The theory of change and the strong EU accession asset of the Western Balkans region The theory of change underlying ILMS is actually already guided by a well-defined and overarching process: the EU accession. This process is also coherent with and supported by the global development agenda: The Sustainable Development Goals and the specific targets established in each country/territory of the region. Indeed, there are common dynamics and deep links between negotiation chapters and SDG targets, as well as common goals of two processes, revealing a high degree of complementarity of the two development agendas. Both SDGs and EU accession process propose pathways with tailored targets, instruments and resources made available to achieve those aims. Each of the approaches proposed by the UN and EU are also supportive of one another, for instance with EU chapter making reference to UN SDGs and proposing coherent benchmarks (in terms of eradicating poverty, fighting discrimination…). Notwithstanding that the theory of change should be specific to each Western Balkan country/territory, SDGs and negotiation chapters already offer essential overarching elements guiding the theory of change.
Now that ILMS has raised a strong interest in the region, pursuing the effort in developing the practices and tools has become a factor of sustainability as, not continuing the intervention, may cause the momentum to fall down if the process is not accompanied over time.


Tag: Sustainability Theory of Change Agenda 2030

28.

2.8.3 VISIBILITY
Most institutions are aware ADA is the donor of the ILMS. Physical visibility in this project mainly exists through project events (workshops) and publications and reports. The three donor logo – ADA, UNDP and ILO – effectively appear on all products.
The evaluation has been able to verify ILMS has a consistent physical visibility as all of its products are marked with the donor and implementer logos of ADA, UNDP and ILO.
When it comes to the awareness of the various stakeholders, while UNDP and ILO were systematically mentioned, ADA was referred to in a number of instances, though slightly less systematically. 


Tag: Sustainability Donor UN Agencies Jobs and Livelihoods

Recommendations
1

The project has successfully raised awareness and has gained strong interest from local and central institutions across the region. While it has secured the “buy-in”, and local ownership of inclusive-drive labour market initiatives, many of these initiatives still need to be refined and further developed.  The policy and institutional context as well as the levels of active labour market policies and social protection provision vary in the different Western Balkans (WB) countries. This is especially the case in Kosovo*[1], a great deal of time still needs to be invested in both the policy dialogue and legal reforms, as well as the visibility of promoted models. Similarly, methodologies (self-assessment of inclusive service) has received the highest appreciation from the institutions that have practiced it. The project has also raised high expectations now that institutions have a better understanding of their needs.

Achieving the long-term objectives of the ILMS requires that the intervention should be continued. Thus, the evaluation strongly recommends a following phase of ILMS to be developed, building on the results achieved, using the established regional set-up.

The design of an ILMS Phase II will require the implementation of the following recommendations: Formalising a strategic framework and entering a Phase II preparation process.

 

[1] References to Kosovo shall be understood to be in the context of Security Council resolution 1244 (1999).

2

There shall be no ILMS phase II without a detailed long-term strategic framework. ILMS needs to place itself in time into a longer-term process but also in space vis-à-vis other related regional interventions such as the Employment and Social Affairs Platform (ESAP).

The following actions are suggested:

  • Define the limit of the ILMS intervention strategic objective (tools developed and adjusted to the specifics of each country e.g. Inclusive labour market practices integrated at the local level; inclusive labour market measures fully institutionalized at both central and local level etc.
  • Define the strategic level of interventions in coherence with ESAP strategic objective and in consultation with Regional Cooperation Council (RCC)
  • Build the strategic path of ILMS, into the possible sequence of phases – projects leading to a sustainable strategic objective

Define strategic progress indicators towards sustainability i.e. indicators of the gradual but durable integration of project results

3

Prior to review UNDP and ILO’s respective roles and responsibilities in a possible ILMS II phase, the evaluation recommends to a strategic positioning review of both ILMS and ESAP project to increase their synergy and impact.

  • Compare policy, central-institution level interventions of ILMS and ESAP, to review if there are similarities in interventions and whether some central-level activities can be conducted jointly;

Since the RCC is a Western Balkans inter-governmental body, it is recommended that ILMS uses ESAP’s “privileged” access to central-level governments to maintain a commitment towards the institutionalization of ILMS introduced methodologies and the consideration of lessons learned from ILMS field practices;

4

This involves taking stock of lessons learned, best practices in the respective countries/territories to formulate component and activity-level recommendations

- Mobilize ILMS project team to take stock of lessons learned, best practices in the respective countries/territory to formulate component and activity-level recommendations;

- Organize an ILMS staff regional workshop to review the following:

- Review the overall coherence of the continuation of project components and activities

- Review the coordination, management staffing needs and support

- Review the knowledge management mechanisms

- Review the complementarity and best distribution of UNDP-ILO country level roles

- Identify the level and type of international possible national expertise needed for ILMS PHASE II in order to establish an estimated planning of expertise needs. (if necessary seek the advice of consultants already involved in ILMS Phase I)

5

Discontinuing the support to this highly relevant intervention, with a genuine longer-term potential of institutionalization would jeopardize the gains made under phase I of this project. This will also contradict with the ultimate goal of the intervention: implementing durable reforms.

  • Further financial commitment from ADA, UNDP, ILO is recommended;

Since ILMS is highly supportive of EU-level strategic priorities but also EU strategic, policy and programmes supporting the accession process of the Western Balkans countries, ADA, UNDP and ILO, also with the involvement of the RCC, should jointly mobilize other EU member states to step in. Switzerland and Germany, for instance, are already actively engaged in the employment sector and would be very relevant potential supporters of ILMS.

6

ILMS is a tree with many branches and leaves: there 6 different country contexts, multiple levels of intervention plus the regional dimension. Addressing this complexity requires a clearer definition of roles.

The following are elements that should help both agencies in doing so:

  • Reporting lines: as the project-holder, UNDP should continue to keep the overall management and reporting role. This means that all reporting should be centralized by UNDP at country/territory level and then channeled to the regional level. This is expected to ensure field offices are fully aware of all country- level and regional-level project events. This should help fill the gap knowledge from the country perspective of all level of activities.
  • ILMS Phase II should be equipped with a full-time position in each of the WB6. Following the above-suggesting reporting line, it is recommended that a full-time ILMS Focal staff is appointed in each country/territory. This full-time position should at least cover a reporting (to regional project level) function, a coordination/facilitation function and have an overview of all ILMS activities in each country/territory. The exact job description of this position (or these positions in case the competences required are allocated between a UNDP and an ILO staff) will depend on the proposed design of the next ILMS phase
  • The general management recommendation is to again, keep the management line as simple as possible: UNDP would keep a supervision role in implementation, while ILO would fulfil and advisory role and a supplier of methods, tools and international expertise.
  • Implementation of activities: With the objective is keeping the implementation management line as simple as possible, the evaluation recommends all activities to remain under UNDP management but with varying level of ILO involvement depending on the nature of activities and the level of intervention.
  • The direct management responsibility of each activity, between UNDP and ILO, should be determined based on the agency displaying the most relevant combination of expertise and experience for that activity.

In situations where activities require the expertise of both UNDP and ILO, it is recommended to have the implementation management to be placed under one agency while the other may have an advisory role. This is to avoid confusion of project partners towards the division of roles and responsibilities between the UNDP and ILO.

7

In line with Recommendation 6, ILMS should appoint one full-time Focal Staff for each country/territory to ensure the smooth coherence, coordination and reporting of all activities at the field and regional level. At the Country/territory level, each of those six-focal staff will ensure close coordination of interventions between UNDP and ILO.

  • Internal coordination: In line with Recommendation 6, ILMS should coordinate activities through it ILMS Country/territory focal staff. In turn, the focal staff, should ensure ILO is update and involve in all relevant activities
  • It is recommended that UNDP/ILO ILMS staff meet physically once a year. As much as technology is necessary for the exchange of information, the exchange of ideas and experience among project staff – especially in case ILMS Phase II relies on full-time positions – gathering physically ILMS staff will be essential to identify internal and external lessons learn and suggest corrective actions.

External coordination: Close coordination with the RCC, and especially its ESAP project is crucial, especially since ESAP is likely to have a Phase II. In case of the ILMS Phase II, close coordination/consultation of ILMS and ESAP preparation will be needed to maximize the coherence and impact of both interventions. Since ILO is located in the same building where RCC sits, the evaluation recommends for the ILO ILMS position (who will continue coordinating the identification and supply of international expertise, as well as deal with tools and methodologies) to entertain a regular exchange of information, besides remote coordination with the UNDP ILMS Project Coordinator.

8

The recommended ILMS Phase II will continue to manage, produce and exchange a sizeable volume of information. If not systematized, managing such a task may impede implementation.

  • It is recommended that all project stakeholders have a minimum level of information about the diversity of ILMS activities. This implies for an overview ILMS implementation information is easily accessible to all, with regular and timely updates.
  • The production of short ILMS Quarterly Activity updates as a compilation of country/territory activity updates should be made available to all stakeholders at all levels.
  • UNDP should have a dedicated ILMS Webpage (possibly hosted by UNDP Website) where with reporting, events, tools, methodologies, success story sections could be developed.
  • The design of written products (audits, assessment, workshop notes…) should be made visually friendly, allowing for an easy identification and follow-up of decisions, actions;
  • Basic guidelines (where to find information, simple user-friendly formats for reports, when are reports available, who is responsible for what, follow-up on decisions….)  could be developed to keep control on the management of knowledge production and its use.

An online tool to host ILMS knowledge products could also be used by the project to monitor the progress of implementation: e.g. on online calendar of events with online confirmation of attendance, and workshop output reports available after completion;

9

TEP financial resources were too small in phase I, significantly reducing its potential. A more ambitious approach should be developed, with more resources with the objective of establishing replicable successful models.

  • Increase significantly the value of individual TEP interventions: Only select one vs multiple interventions.
  • Strengthen the economic aspect of the TEP mode: Focus the value chain to establish a financially sustainable model (organic food offers such a potential). This will require expert inputs (value chain, market access…). Subsistence farming does not generate sufficient revenues, a model building strong added value is necessary.
  • Provide a wide array of business support to vulnerable unemployed, including mentorship over a sufficient period of time (minimum 6 months).
  • Use a stronger TEP 2.0 to obtain a full cooperation between ministries, local employment offices, local CSW, private sector, civil society and will consider a more diverse range of partnerships.

Involve/invite central authorities in the process to take in possible lessons learned from TEP 2.0 and propose adequate support.

10

For institutions to use a tailor-made ICM, a substantial effort needs to be deployed, especially in context where much needs to happen on the legal and capacity side. 

The evaluation has identified some of the following key steps to be implemented in the future:

  • Continue with peer learning sessions to discuss practice from EU countries.
  • Conduct functional assessment of PES (ILO related tools and methodologies can be made available for this activity).
  • Use PES functional assessment to formulate recommendations on the required changes and the process guiding to integrating changes.
  • Strengthen the central-local level institutional interaction by inviting representatives of both levels so the central level consults and collects information on new practices from the local level.

Conduct an audit of IT systems and databases in order to design an integrated information system comprehensive of all relevant data necessary to develop user-centered counselling.

11

ILMS, apart from TEP concrete interventions is not a type of intervention where physical visibility is offered a strong place.

- Use the recommended online platform to provide strong ADA visibility.

- Use ILMS knowledge products to include a one paragraph on ADA’s commitment to inclusive employment.

- Introduce ILMS workshops and other events with a short introduction on ADA’s commitment to inclusive employment.

- Similar to ESAP, consider developing a 2 minutes film presenting ILMS and make it available online.

1. Recommendation:

The project has successfully raised awareness and has gained strong interest from local and central institutions across the region. While it has secured the “buy-in”, and local ownership of inclusive-drive labour market initiatives, many of these initiatives still need to be refined and further developed.  The policy and institutional context as well as the levels of active labour market policies and social protection provision vary in the different Western Balkans (WB) countries. This is especially the case in Kosovo*[1], a great deal of time still needs to be invested in both the policy dialogue and legal reforms, as well as the visibility of promoted models. Similarly, methodologies (self-assessment of inclusive service) has received the highest appreciation from the institutions that have practiced it. The project has also raised high expectations now that institutions have a better understanding of their needs.

Achieving the long-term objectives of the ILMS requires that the intervention should be continued. Thus, the evaluation strongly recommends a following phase of ILMS to be developed, building on the results achieved, using the established regional set-up.

The design of an ILMS Phase II will require the implementation of the following recommendations: Formalising a strategic framework and entering a Phase II preparation process.

 

[1] References to Kosovo shall be understood to be in the context of Security Council resolution 1244 (1999).

Management Response: [Added: 2018/11/23] [Last Updated: 2021/01/06]

UNDP as the Executive of the Project accept this recommendation and consider continuation of the ILMS is essential for ensuring sustainability of initiated activities aiming to support policy and institutional reform in the countries involved. While the main objective of phase I was to open dialogue and facilitate knowledge and experience exchange among professionals from the WB-6 on various barriers and possible solutions, phase II will predominantly focus on the supporting the implementation and institutionalization of inclusive approaches to employment and social service delivery. Consequently, Phase II shall be designed to respond to specific country needs, by applying a diverse and bespoke set of measures, which shall be aligned to national policy and institutional context.

Key Actions:

Key Action Responsible DueDate Status Comments Documents
1.1. Develop strategic framework on the regional and national level priority actions to tackle policy and institutional gaps for pursuing integrated provision of inclusive employment and social services
[Added: 2018/11/23] [Last Updated: 2018/12/17]
SDT IRH/ ES, UNDP COs/NPC, ILO DWT/CO ES, ILO CTA and NCs 2018/12 Completed The new strategic framework presented and endorsed by national partners at the Final Project Board Meeting. Roadmaps for national level priority initiatives finalized for 4 countries and a territory of Kosovo. Concept of the Inclusiveness Facility to support inclusive services of 7 SEE countries developed. History
1.2. Develop a project proposal for the Phase II for endorsement by the national partners and potential donors
[Added: 2018/11/23]
SDT IRH/ES, UNDP COs /NPC, ILO DWT/CO Budapest 2018/11 Completed Proposal was developed and submitted for the phase 2.
2. Recommendation:

There shall be no ILMS phase II without a detailed long-term strategic framework. ILMS needs to place itself in time into a longer-term process but also in space vis-à-vis other related regional interventions such as the Employment and Social Affairs Platform (ESAP).

The following actions are suggested:

  • Define the limit of the ILMS intervention strategic objective (tools developed and adjusted to the specifics of each country e.g. Inclusive labour market practices integrated at the local level; inclusive labour market measures fully institutionalized at both central and local level etc.
  • Define the strategic level of interventions in coherence with ESAP strategic objective and in consultation with Regional Cooperation Council (RCC)
  • Build the strategic path of ILMS, into the possible sequence of phases – projects leading to a sustainable strategic objective

Define strategic progress indicators towards sustainability i.e. indicators of the gradual but durable integration of project results

Management Response: [Added: 2018/11/23] [Last Updated: 2021/01/06]

UNDP and ILO accept the recommendation. Considering the complexity and novelty of the issue, during the phase I, the project had attempted to establish alliances with international organizations and jointly promote the concepts of inclusiveness and integrated service delivery for the marginalized groups, including with RCC/ESAP project. The issues are gaining the momentum and the possibilities for synergies and cross-fertilization shall be further explored in the Phase II.

Key Actions:

Key Action Responsible DueDate Status Comments Documents
2.1. Convene consultation and coordination meeting(s) with RCC’s ESAP Project Management to clarify the focus of the two regional initiatives and ensure coherence of proposed approaches to policy and institutional reforms
[Added: 2018/11/23] [Last Updated: 2018/12/17]
SDT IRH/TL and ES, UNDP COs /NPC, ILO DWT/CO Budapest, ILO CTA and NCs 2018/11 Completed ILO and RCC held a consultation/coordination meeting. RCC, ILO and UNDP have agreed on the policy areas/issues which shall require joint or complementary action. History
2.2. Given the complexity of envisaged policy and institutional transformation, specify sub-regional and where necessary country - specific targets, interim benchmarks, progress indicators and sequencing of actions for measuring the level of institutionalization of the project results
[Added: 2018/11/23] [Last Updated: 2018/12/17]
SDT IRH/ES, UNDP COs /NPC, ILO DWT/CO Budapest, ILO CTA and NCs 2018/11 Completed The project proposal for Phase II envisages country specific work packages with respective targets and indicators of progress. History
3. Recommendation:

Prior to review UNDP and ILO’s respective roles and responsibilities in a possible ILMS II phase, the evaluation recommends to a strategic positioning review of both ILMS and ESAP project to increase their synergy and impact.

  • Compare policy, central-institution level interventions of ILMS and ESAP, to review if there are similarities in interventions and whether some central-level activities can be conducted jointly;

Since the RCC is a Western Balkans inter-governmental body, it is recommended that ILMS uses ESAP’s “privileged” access to central-level governments to maintain a commitment towards the institutionalization of ILMS introduced methodologies and the consideration of lessons learned from ILMS field practices;

Management Response: [Added: 2018/11/23] [Last Updated: 2021/01/06]

UNDP and ILO partially agree with the evaluation finding and recommended actions and would like to reiterate that during the Phase I, the project had ensured continued UNDP and ILO coordination with RCC’s ESAP project, with the purpose of avoiding possible overlaps. Nonetheless, during the phase II, the project management intends to raise the bar and bring the principles of collaboration with RCC at higher level, thus aiming to achieve a more substantive impact and sustainability of project’s results.

Key Actions:

Key Action Responsible DueDate Status Comments Documents
3.1. Obtain detailed information about achieved results under ESAP I, in particular, regarding delivered technical assistance, available policy reports and other knowledge resources.
[Added: 2018/11/23]
SDT IRH/ES ILO DWT/CO Budapest 2018/10 Completed
3.2. Identify specific areas, which would benefit from coordinated or joint work or joint advocacy.
[Added: 2018/11/23]
SDT IRH/ES ILO DWT/CO Budapest 2018/11 Completed
3.3. Involve RCC in co-design and joint implementation of a set of regional activities related to peer-learning and policy level related activities of the project
[Added: 2018/11/23]
ILO DWT/CO Budapest SDT IRH/ES 2018/10 Completed
3.4. Keep RCC continuously informed on the progress, developed methodologies and piloted practices and explore the possibilities for joint advocacy to institutionalize the successful approaches
[Added: 2018/11/23] [Last Updated: 2018/12/17]
SDT IRH/TL and ES 2018/11 Completed RCC representatives took part at the final Project Board Meeting of the ILMS I project. The mechanisms for coordination and exchange of information on continuous basis have been embedded in the ILMS II project proposal History
4. Recommendation:

This involves taking stock of lessons learned, best practices in the respective countries/territories to formulate component and activity-level recommendations

- Mobilize ILMS project team to take stock of lessons learned, best practices in the respective countries/territory to formulate component and activity-level recommendations;

- Organize an ILMS staff regional workshop to review the following:

- Review the overall coherence of the continuation of project components and activities

- Review the coordination, management staffing needs and support

- Review the knowledge management mechanisms

- Review the complementarity and best distribution of UNDP-ILO country level roles

- Identify the level and type of international possible national expertise needed for ILMS PHASE II in order to establish an estimated planning of expertise needs. (if necessary seek the advice of consultants already involved in ILMS Phase I)

Management Response: [Added: 2018/11/23] [Last Updated: 2021/01/06]

The management agrees with the proposed recommendations and will ensure that adequate time and resources is allocated to a thorough consultation process for designing the Phase II. This is even more relevant considering the wealth of good practices and lessons learned the project had been able to generate during the initial phase. The short time frame and the limited budget had been main constrains for organizing more frequent consultations and developing effective knowledge management tools, which would complement the peer leaning events. Considering the importance of the issue for the overall success of the Project, the team will explore the possibility for funding/organizing a regional co-designing workshop.

Key Actions:

Key Action Responsible DueDate Status Comments Documents
4.1. Organize joint consultations of UNDP and ILO teams that have been engaged in the implementation and management of the ILMS Phase I to provide feedback and inputs concerning project’s content, management structure, knowledge management mechanisms and optimal distribution of UNDP-ILO country level roles. Reflect better governance arrangements and project management set-up as well as the realistic workplan in the project document as a joint product from the meeting.
[Added: 2018/11/23] [Last Updated: 2021/09/03]
SDT IRH/TL and ES UNDP COs /NPC 2021/12 Initiated Feedback and suggestions for the improved/streamlined management received from the national project teams and reflected in the ILMS II project proposal. History
5. Recommendation:

Discontinuing the support to this highly relevant intervention, with a genuine longer-term potential of institutionalization would jeopardize the gains made under phase I of this project. This will also contradict with the ultimate goal of the intervention: implementing durable reforms.

  • Further financial commitment from ADA, UNDP, ILO is recommended;

Since ILMS is highly supportive of EU-level strategic priorities but also EU strategic, policy and programmes supporting the accession process of the Western Balkans countries, ADA, UNDP and ILO, also with the involvement of the RCC, should jointly mobilize other EU member states to step in. Switzerland and Germany, for instance, are already actively engaged in the employment sector and would be very relevant potential supporters of ILMS.

Management Response: [Added: 2018/11/23] [Last Updated: 2021/01/06]

The management partially agrees with the proposed recommendations. It should be reiterated that the funding opportunities for regional level activities are becoming scarcer. In this respect, it is also worth exploring the options for mobilizing resources from national funding windows (EU, Switzerland, Germany) for the implementation and replication of activities, which are being piloted / catalyzed under the sub-regional project.

Key Actions:

Key Action Responsible DueDate Status Comments Documents
5.1. UNDP and ILO project team to develop a joint plan for approaching bilateral and multilateral donors to explore funding opportunities.
[Added: 2018/11/23] [Last Updated: 2021/09/03]
SDT IRH/TL and ES ILO DWT/CO Budapest 2021/12 Initiated At initiative of ADA, consultations regarding project’s next phase initiated, including a potential partnership with Caritas History
5.2. National teams to identify opportunities for integration of the inclusiveness agenda into national strategic priorities for collaboration with EU and bilateral donors and explore possibilities for scaling up/replication of the successful methodologies/approaches which are being piloted under ILMS
[Added: 2018/11/23] [Last Updated: 2021/09/03]
SDT IRH/ES UNDP COs /NPC, ILO CTA and NCs 2021/05 Completed The methodology on case management applied in the new UNDP/WB project on inclusive reintegration of Roma returnees. History
6. Recommendation:

ILMS is a tree with many branches and leaves: there 6 different country contexts, multiple levels of intervention plus the regional dimension. Addressing this complexity requires a clearer definition of roles.

The following are elements that should help both agencies in doing so:

  • Reporting lines: as the project-holder, UNDP should continue to keep the overall management and reporting role. This means that all reporting should be centralized by UNDP at country/territory level and then channeled to the regional level. This is expected to ensure field offices are fully aware of all country- level and regional-level project events. This should help fill the gap knowledge from the country perspective of all level of activities.
  • ILMS Phase II should be equipped with a full-time position in each of the WB6. Following the above-suggesting reporting line, it is recommended that a full-time ILMS Focal staff is appointed in each country/territory. This full-time position should at least cover a reporting (to regional project level) function, a coordination/facilitation function and have an overview of all ILMS activities in each country/territory. The exact job description of this position (or these positions in case the competences required are allocated between a UNDP and an ILO staff) will depend on the proposed design of the next ILMS phase
  • The general management recommendation is to again, keep the management line as simple as possible: UNDP would keep a supervision role in implementation, while ILO would fulfil and advisory role and a supplier of methods, tools and international expertise.
  • Implementation of activities: With the objective is keeping the implementation management line as simple as possible, the evaluation recommends all activities to remain under UNDP management but with varying level of ILO involvement depending on the nature of activities and the level of intervention.
  • The direct management responsibility of each activity, between UNDP and ILO, should be determined based on the agency displaying the most relevant combination of expertise and experience for that activity.

In situations where activities require the expertise of both UNDP and ILO, it is recommended to have the implementation management to be placed under one agency while the other may have an advisory role. This is to avoid confusion of project partners towards the division of roles and responsibilities between the UNDP and ILO.

Management Response: [Added: 2018/11/23] [Last Updated: 2021/01/06]

UNDP accepts the recommendation and finds the evaluators recommendation very pertinent for the improvement of project management effectiveness. The issue is going to be reviewed during the formulation of the next phase and joint review meeting between UNDP and ILO management and different options, with financial implications shall be explored for streamlining the project management/implementation structure. ILO supports the recommendation that recognizes importance of its advisory role and technical expertise. The ILO partially supports the recommendations in that technical issues required technical competence in both implementation, management and supervision. Also, the ILO recommends that, should staff be recruited at the local level to work full-time on the project, they should have regular contracts with pension contributions paid by the employer (the UN) and health insurance through the UN scheme.

Key Actions:

Key Action Responsible DueDate Status Comments Documents
6.1. Based on the endorsed content of the Phase II, the complexity and the distribution of agreed activities, UNDP and ILO shall propose adequate project management/ implementation structure and the corresponding cost estimates
[Added: 2018/11/23] [Last Updated: 2018/12/17]
SDT IRH/TL and ES ILO DWT/CO Employment Specialist 2018/11 Completed The newly proposed structure and management/ reporting arrangements agreed and reflected in ILMS II proposal History
7. Recommendation:

In line with Recommendation 6, ILMS should appoint one full-time Focal Staff for each country/territory to ensure the smooth coherence, coordination and reporting of all activities at the field and regional level. At the Country/territory level, each of those six-focal staff will ensure close coordination of interventions between UNDP and ILO.

  • Internal coordination: In line with Recommendation 6, ILMS should coordinate activities through it ILMS Country/territory focal staff. In turn, the focal staff, should ensure ILO is update and involve in all relevant activities
  • It is recommended that UNDP/ILO ILMS staff meet physically once a year. As much as technology is necessary for the exchange of information, the exchange of ideas and experience among project staff – especially in case ILMS Phase II relies on full-time positions – gathering physically ILMS staff will be essential to identify internal and external lessons learn and suggest corrective actions.

External coordination: Close coordination with the RCC, and especially its ESAP project is crucial, especially since ESAP is likely to have a Phase II. In case of the ILMS Phase II, close coordination/consultation of ILMS and ESAP preparation will be needed to maximize the coherence and impact of both interventions. Since ILO is located in the same building where RCC sits, the evaluation recommends for the ILO ILMS position (who will continue coordinating the identification and supply of international expertise, as well as deal with tools and methodologies) to entertain a regular exchange of information, besides remote coordination with the UNDP ILMS Project Coordinator.

Management Response: [Added: 2018/11/23] [Last Updated: 2021/01/06]

The management agrees that ensuring a streamlined vertical (regional-national level) and horizontal (UNDP-ILO-RCC-WB-EU) coordination is essential for the success of the project. Nonetheless, coordination at this scale would require additional investment in staff, tools and activities, which would improve the level of interaction at regional and field level. Based on the final project content, ILO and UNDP shall review different management and implementation options.

Key Actions:

Key Action Responsible DueDate Status Comments Documents
7.1 Based on the agreed content of project activities, UNDP and ILO will propose an elaborated project management/implementation structure, which shall ensure a streamlined coordination at country and regional level
[Added: 2018/11/23] [Last Updated: 2018/12/17]
SDT IRH/TL and ES, ILO DWT/CO ES 2018/11 Completed History
7.2. Project will allocate sufficient resources for providing online and in person communication and exchange of project ideas
[Added: 2018/11/23] [Last Updated: 2018/12/18]
SD and Communications Teams IRH 2019/01 Completed The project budget envisages sufficient resources. History
7.3. UNDP/ILO ILMS and RCC’s ESAP teams shall agree on the frequency and the format for convening regular consultations and updates on project activities
[Added: 2018/11/23] [Last Updated: 2018/12/17]
SDT IRH/TL and ES, ILO DWT/CO ES and CTA 2018/12 Completed ILMS II work plan envisages 2 in-person consultation meetings amongst the regional and project teams, including an earmarked budget allocation History
8. Recommendation:

The recommended ILMS Phase II will continue to manage, produce and exchange a sizeable volume of information. If not systematized, managing such a task may impede implementation.

  • It is recommended that all project stakeholders have a minimum level of information about the diversity of ILMS activities. This implies for an overview ILMS implementation information is easily accessible to all, with regular and timely updates.
  • The production of short ILMS Quarterly Activity updates as a compilation of country/territory activity updates should be made available to all stakeholders at all levels.
  • UNDP should have a dedicated ILMS Webpage (possibly hosted by UNDP Website) where with reporting, events, tools, methodologies, success story sections could be developed.
  • The design of written products (audits, assessment, workshop notes…) should be made visually friendly, allowing for an easy identification and follow-up of decisions, actions;
  • Basic guidelines (where to find information, simple user-friendly formats for reports, when are reports available, who is responsible for what, follow-up on decisions….)  could be developed to keep control on the management of knowledge production and its use.

An online tool to host ILMS knowledge products could also be used by the project to monitor the progress of implementation: e.g. on online calendar of events with online confirmation of attendance, and workshop output reports available after completion;

Management Response: [Added: 2018/11/23] [Last Updated: 2021/01/06]

The management recognizes the importance of this recommendation and shall pay particular attention to this issue in designing the phase II, including by allocating adequate funding and staff time.

Key Actions:

Key Action Responsible DueDate Status Comments Documents
8.3. The project team will coordinate with ESAP project about the use of existing Platform
[Added: 2018/11/23] [Last Updated: 2021/09/03]
ILO DWT ES and CTA SDT IRH/TL and ES 2021/12 Initiated The topic has been extensively discussed during the Closing Project Board Meeting and modalities for an effective used of ESAP platform agreed and also reflected in ILMS II project design. History
8.1. In consultation with UNDP’s and ILO’s communication experts, the project team shall prepare a plan with proposed tools and products for better communication of project’s results and management of generated knowledge under the project.
[Added: 2018/11/23] [Last Updated: 2019/10/02]
SD and Communications Teams/ IRH, ILO DWT/CO Budapest 2019/09 Completed The Project Document of ILMS II entails a thorough Communication plan including preferred tools, media, and format to be used in ILMS II outreach (please see page 21). History
8.2. Project’s Phase II shall provide sufficient human and financial resources for the implementation of a sound knowledge management system.
[Added: 2018/11/23] [Last Updated: 2018/12/18]
SDT IRH/ES 2019/01 Completed ILMS envisages increased budget allocation for the development and feeding in information for a sound knowledge management History
9. Recommendation:

TEP financial resources were too small in phase I, significantly reducing its potential. A more ambitious approach should be developed, with more resources with the objective of establishing replicable successful models.

  • Increase significantly the value of individual TEP interventions: Only select one vs multiple interventions.
  • Strengthen the economic aspect of the TEP mode: Focus the value chain to establish a financially sustainable model (organic food offers such a potential). This will require expert inputs (value chain, market access…). Subsistence farming does not generate sufficient revenues, a model building strong added value is necessary.
  • Provide a wide array of business support to vulnerable unemployed, including mentorship over a sufficient period of time (minimum 6 months).
  • Use a stronger TEP 2.0 to obtain a full cooperation between ministries, local employment offices, local CSW, private sector, civil society and will consider a more diverse range of partnerships.

Involve/invite central authorities in the process to take in possible lessons learned from TEP 2.0 and propose adequate support.

Management Response: [Added: 2018/12/17] [Last Updated: 2021/01/06]

The management finds the evaluation recommendation fully aligned to the lessons learned and the feedback received from the project partners. The demonstration effect from the Phase I has been very positive. The partners have recognized the strength of the local actions and several offices have been successful if mobilizing new partnerships in this area. Project teams will review the lessons learned with a purpose of developing a set of complementary regional activities (technical resources, capacity building), which could reinforce the local level actions

Key Actions:

Key Action Responsible DueDate Status Comments Documents
9.2. Develop a methodology and a set of guidelines and tools for implementation of an upgraded 2.0 TEP.
[Added: 2018/12/17] [Last Updated: 2021/09/03]
SDT IRH/ES 2021/12 Initiated TEP 2.0 methodology is being drafted History
9.1. With inputs from country offices and national partners systematize lessons learned from TEP 1.0 experience.
[Added: 2018/12/17]
SDT IRH/ES UNDP Cos /NPC 2018/11 Completed TEP 1.0 - Lessons learned document is being drafted
9.3 Use the available ILO Guidelines and training materials on TEPs/LEPs
[Added: 2018/12/17] [Last Updated: 2020/10/01]
SDT IRH/ES UNDP COs /NPC 2020/08 Completed Planned activities took place between 01.01.2020 – 30.06.2020 reported through a Mid-year Review Note for submission to the donor Austrian Development Agency by 30 August 2020 History
9.3. Agree on the particular added value and resources to be made available by the regional project in designing and implementation of TEPs
[Added: 2018/12/17] [Last Updated: 2020/02/27]
SDT IRH/ES UNDP COs /NPC 2019/12 Completed TEP related activities have been redefined. In consultation with national partners, the participating countries have agreed to refocus project activities to a) defining governance mechanisms and regulation for institutionalization of TEP approach into local employment planning and b) developing a robust SDG-based methodology for measuring and reporting the impact of TEP activities History
10. Recommendation:

For institutions to use a tailor-made ICM, a substantial effort needs to be deployed, especially in context where much needs to happen on the legal and capacity side. 

The evaluation has identified some of the following key steps to be implemented in the future:

  • Continue with peer learning sessions to discuss practice from EU countries.
  • Conduct functional assessment of PES (ILO related tools and methodologies can be made available for this activity).
  • Use PES functional assessment to formulate recommendations on the required changes and the process guiding to integrating changes.
  • Strengthen the central-local level institutional interaction by inviting representatives of both levels so the central level consults and collects information on new practices from the local level.

Conduct an audit of IT systems and databases in order to design an integrated information system comprehensive of all relevant data necessary to develop user-centered counselling.

Management Response: [Added: 2018/12/17] [Last Updated: 2021/01/06]

The management agrees with the evaluation finding that Integrated Case Management should constitute one of the main pillars of Phase II, given its potential underpinning effect on the policy and institutional reforms in inclusive employment and social service provision. In designing the phase II, the project shall build on the extensive analysis national roadmaps for action, which have been developed for the participating countries/territory.

Key Actions:

Key Action Responsible DueDate Status Comments Documents
10.1. Based on agreed national roadmaps for implementation of integrated case management, prepare a detailed and country-specific plan of actions.
[Added: 2018/12/17] [Last Updated: 2019/10/02]
SDT IRH/ES UNDP COs /NPC 2019/09 Completed Country-specific plans of action are prepared and uploaded to project’s main knowledge management repository hosted under UNDP IRH website. History
Ensure that Phase II embraces different dimensions of ICM’s implementation i.e. capacity building, the organizational/institutional changes, upgrading the IT systems; introducing comprehensive models for holistic assessment of the clients and designing personalized integration plans
[Added: 2018/12/17]
SDT IRH/ES UNDP COs /NPC 2018/12 Completed Proposed approach has been reflected in the project strategy and the RRF of ILMS II
11. Recommendation:

ILMS, apart from TEP concrete interventions is not a type of intervention where physical visibility is offered a strong place.

- Use the recommended online platform to provide strong ADA visibility.

- Use ILMS knowledge products to include a one paragraph on ADA’s commitment to inclusive employment.

- Introduce ILMS workshops and other events with a short introduction on ADA’s commitment to inclusive employment.

- Similar to ESAP, consider developing a 2 minutes film presenting ILMS and make it available online.

Management Response: [Added: 2018/12/17] [Last Updated: 2021/01/06]

The management agrees with the finding and takes note of recommended actions. During the Phase II, the issue of visibility shall be addressed from the broader perspective and the project team shall engage the expert advice of the communication experts

Key Actions:

Key Action Responsible DueDate Status Comments Documents
11.1. In consultations with ILO’s and UNDP’s communications experts, ILMS Phase II will develop a set of communication tools and products, and allocate suitable funding for the communication activities, which shall increase project’s and donor’s visibility. The examples proposed by the evaluator, constitute a good foundation for developing a systematized visibility plan
[Added: 2018/12/17] [Last Updated: 2019/10/02]
SD and Communications Teams/UNDP IRH ILO DWT/CO Budapest 2019/09 Completed In line with Key Action 8.1. necessary communication plan has been prepared and currently under implementation. Necessary funding is determined in consultation with UNDP IRH Communications Team and the amount is allocated from the project budget. History

Latest Evaluations

Contact us

1 UN Plaza
DC1-20th Floor
New York, NY 10017
Tel. +1 646 781 4200
Fax. +1 646 781 4213
erc.support@undp.org