Promoting Inclusive Labour Market Solutions in the Western Balkans

Report Cover Image
Evaluation Plan:
2018-2021, RBEC
Evaluation Type:
Final Project
Planned End Date:
09/2018
Completion Date:
11/2018
Status:
Completed
Management Response:
Yes
Evaluation Budget(US $):
13,200

Share

Document Type Language Size Status Downloads
Download document ToR_Evaluation_ILMS_Final.pdf tor English 167.71 KB Posted 44
Download document UNDP WB ILM Final Evaluation Report 2018_final.pdf report English 1710.55 KB Posted 75
Download document Annex 4 Logframe matrix.docx related-document English 137.59 KB Posted 32
Title Promoting Inclusive Labour Market Solutions in the Western Balkans
Atlas Project Number: 00095840
Evaluation Plan: 2018-2021, RBEC
Evaluation Type: Final Project
Status: Completed
Completion Date: 11/2018
Planned End Date: 09/2018
Management Response: Yes
Corporate Outcome and Output (UNDP Strategic Plan 2018-2021)
  • 1. Output 1.1.2 Marginalised groups, particularly the poor, women, people with disabilities and displaced are empowered to gain universal access to basic services and financial and non-financial assets to build productive capacities and benefit from sustainable livelihoods and jobs
Evaluation Budget(US $): 13,200
Source of Funding: Project
Evaluation Expenditure(US $): 13,200
Joint Programme: No
Joint Evaluation: No
Evaluation Team members:
Name Title Email Nationality
Thomas Vasseur Independent Evaluator thomas.vasseur@gmail.com
GEF Evaluation: No
Key Stakeholders: Country Offices, ILO
Countries: ALBANIABOSNIA AND HERCEGOVINAKOSOVOMACEDONIA, THE FORMER YUGOSLAV REPUBLIC OFMONTENEGROSERBIATURKEY
Lessons
1.

The regional dimension of ILMS is not just an add-on but offers a strong added-value as it
allows all of the targeted countries and territories to benefit from the experience of its
neighbours and use it for its own benefit.
ILMS suffers from an unfair syndrome: Most of the great impact it creates, and while ILMS is
“everywhere”, it is not visible. This fact calls for a pro-active and creative stance in that
respect, with the help of digital technology.
ILMS is a “knowledge monster”: it generates so much great and useful knowledge from its
numerous activities. There is a clear risk of a part of it being lost or not followed upon if not
managed properly. Here again, web-based technology can be of great assistance.
While favouring a participative, two-way bottom-up and top-down approach, ILMS still is to
tackle the reconnection of its activities between the central and local institutional level.


Findings
1.

Relevance
ILMS enjoys a strong policy and pragmatic relevance and fits precisely to local, national, as well as sub-regional priorities in building more efficient, adapted and responsive solutions involving the resources and expertise of all relevant actors. While ILMS is strategically fully supportive of labour-inclusive actions in the region, it lacks the clear formulation of a strategic framework that would assure all project partners (i.e. Centres for Social Welfare, Municipal Employment Agencies, National Employment Services, ministries of labour and social policies, the private sector, the civil society) that there will be a continuation of the present project so that sustainability is ensured.
With a presence on multiple frontlines in the region, from policy level to project level, ILMS has displayed a strong ambition, expressed through an impressive array of activities, but lessened by insufficient financial and human resources. The spread of activities – while all unanimously appreciated – is such that, maintaining ILMS’s coherence and the awareness that all stakeholders are part of a single project has required significant efforts.

Effectiveness
Given its limited resource and timeframe, ILMS has achieved remarkable results and built a deep interest and the willingness of institutions, the private sector and the civil society to pursue further the path towards developing inclusive services. The ILMS project team who has spared no efforts to constantly consult and mobilise project partners is to be commended for its dedication and personal commitment. However, if ILMS has won the “buy-in” of institutions, the road to the institutionalisation of collaborative, multi-stakeholders practices, tools and methods remains long.
UNDP and ILO have proven effective partners as both agencies combine the field presence, experience and expertise that ILMS requires.

Efficiency
ILMS has offered very good value for money when looking the cost per activity rate. However, this has been made possible thanks to very intense UNDP and ILO ILMS project staff commitment and is not sustainable on the longer-term.
Impact
Even if it is hard to capture, ILMS has made an immense first “mind-changing” impact on its partners, especially at the local level. It has raised expectations and desire to go further in the reform process and created a moral commitment of the project to not stop its intervention at this early stage.
ILMS has also produced very concrete impact in the field through its TEP interventions. While some sub-projects have produced promising results when judiciously integrated in existing UNDP programmes, some others have disappointed mainly because of insufficient financial resources made available by the project.

Sustainability
ILMS has secured the first but essential steps that open the way to a promising sustainability: Awareness, interest, buy-in and already some elements of ownership of innovative practices.
For this reason alone, there is a strong case for ILMS to consider a further project phase.


Recommendations
1

The project has successfully raised awareness and has gained strong interest from local and central institutions across the region. While it has secured the “buy-in”, and local ownership of inclusive-drive labour market initiatives, many of these initiatives still need to be refined and further developed.  The policy and institutional context as well as the levels of active labour market policies and social protection provision vary in the different Western Balkans (WB) countries. This is especially the case in Kosovo*[1], a great deal of time still needs to be invested in both the policy dialogue and legal reforms, as well as the visibility of promoted models. Similarly, methodologies (self-assessment of inclusive service) has received the highest appreciation from the institutions that have practiced it. The project has also raised high expectations now that institutions have a better understanding of their needs.

Achieving the long-term objectives of the ILMS requires that the intervention should be continued. Thus, the evaluation strongly recommends a following phase of ILMS to be developed, building on the results achieved, using the established regional set-up.

The design of an ILMS Phase II will require the implementation of the following recommendations: Formalising a strategic framework and entering a Phase II preparation process.

 

[1] References to Kosovo shall be understood to be in the context of Security Council resolution 1244 (1999).

2

There shall be no ILMS phase II without a detailed long-term strategic framework. ILMS needs to place itself in time into a longer-term process but also in space vis-à-vis other related regional interventions such as the Employment and Social Affairs Platform (ESAP).

The following actions are suggested:

  • Define the limit of the ILMS intervention strategic objective (tools developed and adjusted to the specifics of each country e.g. Inclusive labour market practices integrated at the local level; inclusive labour market measures fully institutionalized at both central and local level etc.
  • Define the strategic level of interventions in coherence with ESAP strategic objective and in consultation with Regional Cooperation Council (RCC)
  • Build the strategic path of ILMS, into the possible sequence of phases – projects leading to a sustainable strategic objective

Define strategic progress indicators towards sustainability i.e. indicators of the gradual but durable integration of project results

3

Prior to review UNDP and ILO’s respective roles and responsibilities in a possible ILMS II phase, the evaluation recommends to a strategic positioning review of both ILMS and ESAP project to increase their synergy and impact.

  • Compare policy, central-institution level interventions of ILMS and ESAP, to review if there are similarities in interventions and whether some central-level activities can be conducted jointly;

Since the RCC is a Western Balkans inter-governmental body, it is recommended that ILMS uses ESAP’s “privileged” access to central-level governments to maintain a commitment towards the institutionalization of ILMS introduced methodologies and the consideration of lessons learned from ILMS field practices;

4

This involves taking stock of lessons learned, best practices in the respective countries/territories to formulate component and activity-level recommendations

- Mobilize ILMS project team to take stock of lessons learned, best practices in the respective countries/territory to formulate component and activity-level recommendations;

- Organize an ILMS staff regional workshop to review the following:

- Review the overall coherence of the continuation of project components and activities

- Review the coordination, management staffing needs and support

- Review the knowledge management mechanisms

- Review the complementarity and best distribution of UNDP-ILO country level roles

- Identify the level and type of international possible national expertise needed for ILMS PHASE II in order to establish an estimated planning of expertise needs. (if necessary seek the advice of consultants already involved in ILMS Phase I)

5

Discontinuing the support to this highly relevant intervention, with a genuine longer-term potential of institutionalization would jeopardize the gains made under phase I of this project. This will also contradict with the ultimate goal of the intervention: implementing durable reforms.

  • Further financial commitment from ADA, UNDP, ILO is recommended;

Since ILMS is highly supportive of EU-level strategic priorities but also EU strategic, policy and programmes supporting the accession process of the Western Balkans countries, ADA, UNDP and ILO, also with the involvement of the RCC, should jointly mobilize other EU member states to step in. Switzerland and Germany, for instance, are already actively engaged in the employment sector and would be very relevant potential supporters of ILMS.

6

ILMS is a tree with many branches and leaves: there 6 different country contexts, multiple levels of intervention plus the regional dimension. Addressing this complexity requires a clearer definition of roles.

The following are elements that should help both agencies in doing so:

  • Reporting lines: as the project-holder, UNDP should continue to keep the overall management and reporting role. This means that all reporting should be centralized by UNDP at country/territory level and then channeled to the regional level. This is expected to ensure field offices are fully aware of all country- level and regional-level project events. This should help fill the gap knowledge from the country perspective of all level of activities.
  • ILMS Phase II should be equipped with a full-time position in each of the WB6. Following the above-suggesting reporting line, it is recommended that a full-time ILMS Focal staff is appointed in each country/territory. This full-time position should at least cover a reporting (to regional project level) function, a coordination/facilitation function and have an overview of all ILMS activities in each country/territory. The exact job description of this position (or these positions in case the competences required are allocated between a UNDP and an ILO staff) will depend on the proposed design of the next ILMS phase
  • The general management recommendation is to again, keep the management line as simple as possible: UNDP would keep a supervision role in implementation, while ILO would fulfil and advisory role and a supplier of methods, tools and international expertise.
  • Implementation of activities: With the objective is keeping the implementation management line as simple as possible, the evaluation recommends all activities to remain under UNDP management but with varying level of ILO involvement depending on the nature of activities and the level of intervention.
  • The direct management responsibility of each activity, between UNDP and ILO, should be determined based on the agency displaying the most relevant combination of expertise and experience for that activity.

In situations where activities require the expertise of both UNDP and ILO, it is recommended to have the implementation management to be placed under one agency while the other may have an advisory role. This is to avoid confusion of project partners towards the division of roles and responsibilities between the UNDP and ILO.

7

In line with Recommendation 6, ILMS should appoint one full-time Focal Staff for each country/territory to ensure the smooth coherence, coordination and reporting of all activities at the field and regional level. At the Country/territory level, each of those six-focal staff will ensure close coordination of interventions between UNDP and ILO.

  • Internal coordination: In line with Recommendation 6, ILMS should coordinate activities through it ILMS Country/territory focal staff. In turn, the focal staff, should ensure ILO is update and involve in all relevant activities
  • It is recommended that UNDP/ILO ILMS staff meet physically once a year. As much as technology is necessary for the exchange of information, the exchange of ideas and experience among project staff – especially in case ILMS Phase II relies on full-time positions – gathering physically ILMS staff will be essential to identify internal and external lessons learn and suggest corrective actions.

External coordination: Close coordination with the RCC, and especially its ESAP project is crucial, especially since ESAP is likely to have a Phase II. In case of the ILMS Phase II, close coordination/consultation of ILMS and ESAP preparation will be needed to maximize the coherence and impact of both interventions. Since ILO is located in the same building where RCC sits, the evaluation recommends for the ILO ILMS position (who will continue coordinating the identification and supply of international expertise, as well as deal with tools and methodologies) to entertain a regular exchange of information, besides remote coordination with the UNDP ILMS Project Coordinator.

8

The recommended ILMS Phase II will continue to manage, produce and exchange a sizeable volume of information. If not systematized, managing such a task may impede implementation.

  • It is recommended that all project stakeholders have a minimum level of information about the diversity of ILMS activities. This implies for an overview ILMS implementation information is easily accessible to all, with regular and timely updates.
  • The production of short ILMS Quarterly Activity updates as a compilation of country/territory activity updates should be made available to all stakeholders at all levels.
  • UNDP should have a dedicated ILMS Webpage (possibly hosted by UNDP Website) where with reporting, events, tools, methodologies, success story sections could be developed.
  • The design of written products (audits, assessment, workshop notes…) should be made visually friendly, allowing for an easy identification and follow-up of decisions, actions;
  • Basic guidelines (where to find information, simple user-friendly formats for reports, when are reports available, who is responsible for what, follow-up on decisions….)  could be developed to keep control on the management of knowledge production and its use.

An online tool to host ILMS knowledge products could also be used by the project to monitor the progress of implementation: e.g. on online calendar of events with online confirmation of attendance, and workshop output reports available after completion;

9

TEP financial resources were too small in phase I, significantly reducing its potential. A more ambitious approach should be developed, with more resources with the objective of establishing replicable successful models.

  • Increase significantly the value of individual TEP interventions: Only select one vs multiple interventions.
  • Strengthen the economic aspect of the TEP mode: Focus the value chain to establish a financially sustainable model (organic food offers such a potential). This will require expert inputs (value chain, market access…). Subsistence farming does not generate sufficient revenues, a model building strong added value is necessary.
  • Provide a wide array of business support to vulnerable unemployed, including mentorship over a sufficient period of time (minimum 6 months).
  • Use a stronger TEP 2.0 to obtain a full cooperation between ministries, local employment offices, local CSW, private sector, civil society and will consider a more diverse range of partnerships.

Involve/invite central authorities in the process to take in possible lessons learned from TEP 2.0 and propose adequate support.

10

For institutions to use a tailor-made ICM, a substantial effort needs to be deployed, especially in context where much needs to happen on the legal and capacity side. 

The evaluation has identified some of the following key steps to be implemented in the future:

  • Continue with peer learning sessions to discuss practice from EU countries.
  • Conduct functional assessment of PES (ILO related tools and methodologies can be made available for this activity).
  • Use PES functional assessment to formulate recommendations on the required changes and the process guiding to integrating changes.
  • Strengthen the central-local level institutional interaction by inviting representatives of both levels so the central level consults and collects information on new practices from the local level.

Conduct an audit of IT systems and databases in order to design an integrated information system comprehensive of all relevant data necessary to develop user-centered counselling.

11

ILMS, apart from TEP concrete interventions is not a type of intervention where physical visibility is offered a strong place.

- Use the recommended online platform to provide strong ADA visibility.

- Use ILMS knowledge products to include a one paragraph on ADA’s commitment to inclusive employment.

- Introduce ILMS workshops and other events with a short introduction on ADA’s commitment to inclusive employment.

- Similar to ESAP, consider developing a 2 minutes film presenting ILMS and make it available online.

1. Recommendation:

The project has successfully raised awareness and has gained strong interest from local and central institutions across the region. While it has secured the “buy-in”, and local ownership of inclusive-drive labour market initiatives, many of these initiatives still need to be refined and further developed.  The policy and institutional context as well as the levels of active labour market policies and social protection provision vary in the different Western Balkans (WB) countries. This is especially the case in Kosovo*[1], a great deal of time still needs to be invested in both the policy dialogue and legal reforms, as well as the visibility of promoted models. Similarly, methodologies (self-assessment of inclusive service) has received the highest appreciation from the institutions that have practiced it. The project has also raised high expectations now that institutions have a better understanding of their needs.

Achieving the long-term objectives of the ILMS requires that the intervention should be continued. Thus, the evaluation strongly recommends a following phase of ILMS to be developed, building on the results achieved, using the established regional set-up.

The design of an ILMS Phase II will require the implementation of the following recommendations: Formalising a strategic framework and entering a Phase II preparation process.

 

[1] References to Kosovo shall be understood to be in the context of Security Council resolution 1244 (1999).

Management Response: [Added: 2018/11/23]

UNDP as the Executive of the Project accept this recommendation and consider continuation of the ILMS is essential for ensuring sustainability of initiated activities aiming to support policy and institutional reform in the countries involved. While the main objective of phase I was to open dialogue and facilitate knowledge and experience exchange among professionals from the WB-6 on various barriers and possible solutions, phase II will predominantly focus on the supporting the implementation and institutionalization of inclusive approaches to employment and social service delivery. Consequently, Phase II shall be designed to respond to specific country needs, by applying a diverse and bespoke set of measures, which shall be aligned to national policy and institutional context.

Key Actions:

Key Action Responsible DueDate Status Comments Documents
1.1. Develop strategic framework on the regional and national level priority actions to tackle policy and institutional gaps for pursuing integrated provision of inclusive employment and social services
[Added: 2018/11/23] [Last Updated: 2018/12/17]
SDT IRH/ ES, UNDP COs/NPC, ILO DWT/CO ES, ILO CTA and NCs 2018/12 Completed The new strategic framework presented and endorsed by national partners at the Final Project Board Meeting. Roadmaps for national level priority initiatives finalized for 4 countries and a territory of Kosovo. Concept of the Inclusiveness Facility to support inclusive services of 7 SEE countries developed. History
1.2. Develop a project proposal for the Phase II for endorsement by the national partners and potential donors
[Added: 2018/11/23]
SDT IRH/ES, UNDP COs /NPC, ILO DWT/CO Budapest 2018/11 Completed Proposal was developed and submitted for the phase 2.
2. Recommendation:

There shall be no ILMS phase II without a detailed long-term strategic framework. ILMS needs to place itself in time into a longer-term process but also in space vis-à-vis other related regional interventions such as the Employment and Social Affairs Platform (ESAP).

The following actions are suggested:

  • Define the limit of the ILMS intervention strategic objective (tools developed and adjusted to the specifics of each country e.g. Inclusive labour market practices integrated at the local level; inclusive labour market measures fully institutionalized at both central and local level etc.
  • Define the strategic level of interventions in coherence with ESAP strategic objective and in consultation with Regional Cooperation Council (RCC)
  • Build the strategic path of ILMS, into the possible sequence of phases – projects leading to a sustainable strategic objective

Define strategic progress indicators towards sustainability i.e. indicators of the gradual but durable integration of project results

Management Response: [Added: 2018/11/23]

UNDP and ILO accept the recommendation. Considering the complexity and novelty of the issue, during the phase I, the project had attempted to establish alliances with international organizations and jointly promote the concepts of inclusiveness and integrated service delivery for the marginalized groups, including with RCC/ESAP project. The issues are gaining the momentum and the possibilities for synergies and cross-fertilization shall be further explored in the Phase II.

Key Actions:

Key Action Responsible DueDate Status Comments Documents
2.1. Convene consultation and coordination meeting(s) with RCC’s ESAP Project Management to clarify the focus of the two regional initiatives and ensure coherence of proposed approaches to policy and institutional reforms
[Added: 2018/11/23] [Last Updated: 2018/12/17]
SDT IRH/TL and ES, UNDP COs /NPC, ILO DWT/CO Budapest, ILO CTA and NCs 2018/11 Completed ILO and RCC held a consultation/coordination meeting. RCC, ILO and UNDP have agreed on the policy areas/issues which shall require joint or complementary action. History
2.2. Given the complexity of envisaged policy and institutional transformation, specify sub-regional and where necessary country - specific targets, interim benchmarks, progress indicators and sequencing of actions for measuring the level of institutionalization of the project results
[Added: 2018/11/23] [Last Updated: 2018/12/17]
SDT IRH/ES, UNDP COs /NPC, ILO DWT/CO Budapest, ILO CTA and NCs 2018/11 Completed The project proposal for Phase II envisages country specific work packages with respective targets and indicators of progress. History
3. Recommendation:

Prior to review UNDP and ILO’s respective roles and responsibilities in a possible ILMS II phase, the evaluation recommends to a strategic positioning review of both ILMS and ESAP project to increase their synergy and impact.

  • Compare policy, central-institution level interventions of ILMS and ESAP, to review if there are similarities in interventions and whether some central-level activities can be conducted jointly;

Since the RCC is a Western Balkans inter-governmental body, it is recommended that ILMS uses ESAP’s “privileged” access to central-level governments to maintain a commitment towards the institutionalization of ILMS introduced methodologies and the consideration of lessons learned from ILMS field practices;

Management Response: [Added: 2018/11/23]

UNDP and ILO partially agree with the evaluation finding and recommended actions and would like to reiterate that during the Phase I, the project had ensured continued UNDP and ILO coordination with RCC’s ESAP project, with the purpose of avoiding possible overlaps. Nonetheless, during the phase II, the project management intends to raise the bar and bring the principles of collaboration with RCC at higher level, thus aiming to achieve a more substantive impact and sustainability of project’s results.

Key Actions:

Key Action Responsible DueDate Status Comments Documents
3.1. Obtain detailed information about achieved results under ESAP I, in particular, regarding delivered technical assistance, available policy reports and other knowledge resources.
[Added: 2018/11/23]
SDT IRH/ES ILO DWT/CO Budapest 2018/10 Completed
3.2. Identify specific areas, which would benefit from coordinated or joint work or joint advocacy.
[Added: 2018/11/23]
SDT IRH/ES ILO DWT/CO Budapest 2018/11 Completed
3.3. Involve RCC in co-design and joint implementation of a set of regional activities related to peer-learning and policy level related activities of the project
[Added: 2018/11/23]
ILO DWT/CO Budapest SDT IRH/ES 2018/10 Completed
3.4. Keep RCC continuously informed on the progress, developed methodologies and piloted practices and explore the possibilities for joint advocacy to institutionalize the successful approaches
[Added: 2018/11/23] [Last Updated: 2018/12/17]
SDT IRH/TL and ES 2018/11 Completed RCC representatives took part at the final Project Board Meeting of the ILMS I project. The mechanisms for coordination and exchange of information on continuous basis have been embedded in the ILMS II project proposal History
4. Recommendation:

This involves taking stock of lessons learned, best practices in the respective countries/territories to formulate component and activity-level recommendations

- Mobilize ILMS project team to take stock of lessons learned, best practices in the respective countries/territory to formulate component and activity-level recommendations;

- Organize an ILMS staff regional workshop to review the following:

- Review the overall coherence of the continuation of project components and activities

- Review the coordination, management staffing needs and support

- Review the knowledge management mechanisms

- Review the complementarity and best distribution of UNDP-ILO country level roles

- Identify the level and type of international possible national expertise needed for ILMS PHASE II in order to establish an estimated planning of expertise needs. (if necessary seek the advice of consultants already involved in ILMS Phase I)

Management Response: [Added: 2018/11/23]

The management agrees with the proposed recommendations and will ensure that adequate time and resources is allocated to a thorough consultation process for designing the Phase II. This is even more relevant considering the wealth of good practices and lessons learned the project had been able to generate during the initial phase. The short time frame and the limited budget had been main constrains for organizing more frequent consultations and developing effective knowledge management tools, which would complement the peer leaning events. Considering the importance of the issue for the overall success of the Project, the team will explore the possibility for funding/organizing a regional co-designing workshop.

Key Actions:

Key Action Responsible DueDate Status Comments Documents
4.1. Organize joint consultations of UNDP and ILO teams that have been engaged in the implementation and management of the ILMS Phase I to provide feedback and inputs concerning project’s content, management structure, knowledge management mechanisms and optimal distribution of UNDP-ILO country level roles. Reflect better governance arrangements and project management set-up as well as the realistic workplan in the project document as a joint product from the meeting.
[Added: 2018/11/23] [Last Updated: 2019/05/17]
SDT IRH/TL and ES UNDP COs /NPC 2019/09 Initiated Due date has been set to September 2019. The joint meeting is going to be organized in September and the discussions during the meeting will provide an update to the key action. History
5. Recommendation:

Discontinuing the support to this highly relevant intervention, with a genuine longer-term potential of institutionalization would jeopardize the gains made under phase I of this project. This will also contradict with the ultimate goal of the intervention: implementing durable reforms.

  • Further financial commitment from ADA, UNDP, ILO is recommended;

Since ILMS is highly supportive of EU-level strategic priorities but also EU strategic, policy and programmes supporting the accession process of the Western Balkans countries, ADA, UNDP and ILO, also with the involvement of the RCC, should jointly mobilize other EU member states to step in. Switzerland and Germany, for instance, are already actively engaged in the employment sector and would be very relevant potential supporters of ILMS.

Management Response: [Added: 2018/11/23]

The management partially agrees with the proposed recommendations. It should be reiterated that the funding opportunities for regional level activities are becoming scarcer. In this respect, it is also worth exploring the options for mobilizing resources from national funding windows (EU, Switzerland, Germany) for the implementation and replication of activities, which are being piloted / catalyzed under the sub-regional project.

Key Actions:

Key Action Responsible DueDate Status Comments Documents
5.1. UNDP and ILO project team to develop a joint plan for approaching bilateral and multilateral donors to explore funding opportunities.
[Added: 2018/11/23] [Last Updated: 2019/05/17]
SDT IRH/TL and ES ILO DWT/CO Budapest 2019/10 Initiated The status of the action has been changed to initiated in May 2019 to be completed in October 2019. ILO and UNDP Project Team agreed to prepare the list of priority topics for potential scaling up and propose an action plan for approaching potential donors/partners for resource mobilization. History
5.2. National teams to identify opportunities for integration of the inclusiveness agenda into national strategic priorities for collaboration with EU and bilateral donors and explore possibilities for scaling up/replication of the successful methodologies/approaches which are being piloted under ILMS
[Added: 2018/11/23] [Last Updated: 2019/05/17]
SDT IRH/ES UNDP COs /NPC, ILO CTA and NCs 2019/10 Initiated The status of the action has been changed to initiated in May 2019 to be completed in October 2019. In several COs the processes are underway for mainstreaming the inclusiveness agenda into the broader social protection policies and programmes. ILO’s consultations for supporting the inclusiveness agenda have been completed with the Governments of Serbia and North Macedonia and they are ongoing in Montenegro. In North Macedonia and BIH, UNDP is scaling up the inclusiveness agenda into the new generation of employment projects. In Kosovo*, UNDP has initiated a dialogue with the Ministry of Labour for integration of inclusive service provision in the ongoing the social protection reform process. *References to Kosovo shall be understood to be in the context of Security Council Resolution 1244 (1999). History
6. Recommendation:

ILMS is a tree with many branches and leaves: there 6 different country contexts, multiple levels of intervention plus the regional dimension. Addressing this complexity requires a clearer definition of roles.

The following are elements that should help both agencies in doing so:

  • Reporting lines: as the project-holder, UNDP should continue to keep the overall management and reporting role. This means that all reporting should be centralized by UNDP at country/territory level and then channeled to the regional level. This is expected to ensure field offices are fully aware of all country- level and regional-level project events. This should help fill the gap knowledge from the country perspective of all level of activities.
  • ILMS Phase II should be equipped with a full-time position in each of the WB6. Following the above-suggesting reporting line, it is recommended that a full-time ILMS Focal staff is appointed in each country/territory. This full-time position should at least cover a reporting (to regional project level) function, a coordination/facilitation function and have an overview of all ILMS activities in each country/territory. The exact job description of this position (or these positions in case the competences required are allocated between a UNDP and an ILO staff) will depend on the proposed design of the next ILMS phase
  • The general management recommendation is to again, keep the management line as simple as possible: UNDP would keep a supervision role in implementation, while ILO would fulfil and advisory role and a supplier of methods, tools and international expertise.
  • Implementation of activities: With the objective is keeping the implementation management line as simple as possible, the evaluation recommends all activities to remain under UNDP management but with varying level of ILO involvement depending on the nature of activities and the level of intervention.
  • The direct management responsibility of each activity, between UNDP and ILO, should be determined based on the agency displaying the most relevant combination of expertise and experience for that activity.

In situations where activities require the expertise of both UNDP and ILO, it is recommended to have the implementation management to be placed under one agency while the other may have an advisory role. This is to avoid confusion of project partners towards the division of roles and responsibilities between the UNDP and ILO.

Management Response: [Added: 2018/11/23]

UNDP accepts the recommendation and finds the evaluators recommendation very pertinent for the improvement of project management effectiveness. The issue is going to be reviewed during the formulation of the next phase and joint review meeting between UNDP and ILO management and different options, with financial implications shall be explored for streamlining the project management/implementation structure. ILO supports the recommendation that recognizes importance of its advisory role and technical expertise. The ILO partially supports the recommendations in that technical issues required technical competence in both implementation, management and supervision. Also, the ILO recommends that, should staff be recruited at the local level to work full-time on the project, they should have regular contracts with pension contributions paid by the employer (the UN) and health insurance through the UN scheme.

Key Actions:

Key Action Responsible DueDate Status Comments Documents
6.1. Based on the endorsed content of the Phase II, the complexity and the distribution of agreed activities, UNDP and ILO shall propose adequate project management/ implementation structure and the corresponding cost estimates
[Added: 2018/11/23] [Last Updated: 2018/12/17]
SDT IRH/TL and ES ILO DWT/CO Employment Specialist 2018/11 Completed The newly proposed structure and management/ reporting arrangements agreed and reflected in ILMS II proposal History
7. Recommendation:

In line with Recommendation 6, ILMS should appoint one full-time Focal Staff for each country/territory to ensure the smooth coherence, coordination and reporting of all activities at the field and regional level. At the Country/territory level, each of those six-focal staff will ensure close coordination of interventions between UNDP and ILO.

  • Internal coordination: In line with Recommendation 6, ILMS should coordinate activities through it ILMS Country/territory focal staff. In turn, the focal staff, should ensure ILO is update and involve in all relevant activities
  • It is recommended that UNDP/ILO ILMS staff meet physically once a year. As much as technology is necessary for the exchange of information, the exchange of ideas and experience among project staff – especially in case ILMS Phase II relies on full-time positions – gathering physically ILMS staff will be essential to identify internal and external lessons learn and suggest corrective actions.

External coordination: Close coordination with the RCC, and especially its ESAP project is crucial, especially since ESAP is likely to have a Phase II. In case of the ILMS Phase II, close coordination/consultation of ILMS and ESAP preparation will be needed to maximize the coherence and impact of both interventions. Since ILO is located in the same building where RCC sits, the evaluation recommends for the ILO ILMS position (who will continue coordinating the identification and supply of international expertise, as well as deal with tools and methodologies) to entertain a regular exchange of information, besides remote coordination with the UNDP ILMS Project Coordinator.

Management Response: [Added: 2018/11/23]

The management agrees that ensuring a streamlined vertical (regional-national level) and horizontal (UNDP-ILO-RCC-WB-EU) coordination is essential for the success of the project. Nonetheless, coordination at this scale would require additional investment in staff, tools and activities, which would improve the level of interaction at regional and field level. Based on the final project content, ILO and UNDP shall review different management and implementation options.

Key Actions:

Key Action Responsible DueDate Status Comments Documents
7.1 Based on the agreed content of project activities, UNDP and ILO will propose an elaborated project management/implementation structure, which shall ensure a streamlined coordination at country and regional level
[Added: 2018/11/23] [Last Updated: 2018/12/17]
SDT IRH/TL and ES, ILO DWT/CO ES 2018/11 Completed History
7.2. Project will allocate sufficient resources for providing online and in person communication and exchange of project ideas
[Added: 2018/11/23] [Last Updated: 2018/12/18]
SD and Communications Teams IRH 2019/01 Completed The project budget envisages sufficient resources. History
7.3. UNDP/ILO ILMS and RCC’s ESAP teams shall agree on the frequency and the format for convening regular consultations and updates on project activities
[Added: 2018/11/23] [Last Updated: 2018/12/17]
SDT IRH/TL and ES, ILO DWT/CO ES and CTA 2018/12 Completed ILMS II work plan envisages 2 in-person consultation meetings amongst the regional and project teams, including an earmarked budget allocation History
8. Recommendation:

The recommended ILMS Phase II will continue to manage, produce and exchange a sizeable volume of information. If not systematized, managing such a task may impede implementation.

  • It is recommended that all project stakeholders have a minimum level of information about the diversity of ILMS activities. This implies for an overview ILMS implementation information is easily accessible to all, with regular and timely updates.
  • The production of short ILMS Quarterly Activity updates as a compilation of country/territory activity updates should be made available to all stakeholders at all levels.
  • UNDP should have a dedicated ILMS Webpage (possibly hosted by UNDP Website) where with reporting, events, tools, methodologies, success story sections could be developed.
  • The design of written products (audits, assessment, workshop notes…) should be made visually friendly, allowing for an easy identification and follow-up of decisions, actions;
  • Basic guidelines (where to find information, simple user-friendly formats for reports, when are reports available, who is responsible for what, follow-up on decisions….)  could be developed to keep control on the management of knowledge production and its use.

An online tool to host ILMS knowledge products could also be used by the project to monitor the progress of implementation: e.g. on online calendar of events with online confirmation of attendance, and workshop output reports available after completion;

Management Response: [Added: 2018/11/23]

The management recognizes the importance of this recommendation and shall pay particular attention to this issue in designing the phase II, including by allocating adequate funding and staff time.

Key Actions:

Key Action Responsible DueDate Status Comments Documents
8.1. In consultation with UNDP’s and ILO’s communication experts, the project team shall prepare a plan with proposed tools and products for better communication of project’s results and management of generated knowledge under the project.
[Added: 2018/11/23] [Last Updated: 2019/05/17]
SD and Communications Teams/ IRH, ILO DWT/CO Budapest 2019/09 Initiated Due date has been set to September 2019. UNDP has prepared an outline of the Inclusive Employment Digest series – a knowledge management tool that synthesizes the specific knowledge being produced under the ILMS II project. In consultation with ILO to be finalized and first issue released by September 2019. History
8.2. Project’s Phase II shall provide sufficient human and financial resources for the implementation of a sound knowledge management system.
[Added: 2018/11/23] [Last Updated: 2018/12/18]
SDT IRH/ES 2019/01 Completed ILMS envisages increased budget allocation for the development and feeding in information for a sound knowledge management History
8.3. The project team will coordinate with ESAP project about the use of existing Platform
[Added: 2018/11/23] [Last Updated: 2018/12/18]
ILO DWT ES and CTA SDT IRH/TL and ES 2020/12 Initiated A follow up trilateral consultation (UNDP/ILO/RCC) has taken place to discuss the concrete modalities for utilizing the ESAP platform as a communication and knowledge management tool. Once cleared by the QA team, UNDP shall share the relevant reports & knowledge products developed under ILMS I. History
9. Recommendation:

TEP financial resources were too small in phase I, significantly reducing its potential. A more ambitious approach should be developed, with more resources with the objective of establishing replicable successful models.

  • Increase significantly the value of individual TEP interventions: Only select one vs multiple interventions.
  • Strengthen the economic aspect of the TEP mode: Focus the value chain to establish a financially sustainable model (organic food offers such a potential). This will require expert inputs (value chain, market access…). Subsistence farming does not generate sufficient revenues, a model building strong added value is necessary.
  • Provide a wide array of business support to vulnerable unemployed, including mentorship over a sufficient period of time (minimum 6 months).
  • Use a stronger TEP 2.0 to obtain a full cooperation between ministries, local employment offices, local CSW, private sector, civil society and will consider a more diverse range of partnerships.

Involve/invite central authorities in the process to take in possible lessons learned from TEP 2.0 and propose adequate support.

Management Response: [Added: 2018/12/17]

The management finds the evaluation recommendation fully aligned to the lessons learned and the feedback received from the project partners. The demonstration effect from the Phase I has been very positive. The partners have recognized the strength of the local actions and several offices have been successful if mobilizing new partnerships in this area. Project teams will review the lessons learned with a purpose of developing a set of complementary regional activities (technical resources, capacity building), which could reinforce the local level actions

Key Actions:

Key Action Responsible DueDate Status Comments Documents
9.1. With inputs from country offices and national partners systematize lessons learned from TEP 1.0 experience.
[Added: 2018/12/17]
SDT IRH/ES UNDP Cos /NPC 2018/11 Completed TEP 1.0 - Lessons learned document is being drafted
9.2. Develop a methodology and a set of guidelines and tools for implementation of an upgraded 2.0 TEP.
[Added: 2018/12/17] [Last Updated: 2019/05/17]
SDT IRH/ES 2019/12 Initiated Due date has been set to December 2019. Preparation of the TEP methodology is being initiated. Based on the agreed workplan with the country offices, TEP related activities will be rolled out during the second half of 2019. History
9.3 Use the available ILO Guidelines and training materials on TEPs/LEPs
[Added: 2018/12/17]
SDT IRH/ES UNDP COs /NPC 2019/06 Initiated ILMS II foresees LEP workshops/trainings organized by ILO
9.3. Agree on the particular added value and resources to be made available by the regional project in designing and implementation of TEPs
[Added: 2018/12/17] [Last Updated: 2018/12/18]
SDT IRH/ES UNDP COs /NPC 2019/12 Initiated Following the evaluator’s recommendation ILMS II proposal defines the resources and activities of the regional project that can add value and complement country level resources History
10. Recommendation:

For institutions to use a tailor-made ICM, a substantial effort needs to be deployed, especially in context where much needs to happen on the legal and capacity side. 

The evaluation has identified some of the following key steps to be implemented in the future:

  • Continue with peer learning sessions to discuss practice from EU countries.
  • Conduct functional assessment of PES (ILO related tools and methodologies can be made available for this activity).
  • Use PES functional assessment to formulate recommendations on the required changes and the process guiding to integrating changes.
  • Strengthen the central-local level institutional interaction by inviting representatives of both levels so the central level consults and collects information on new practices from the local level.

Conduct an audit of IT systems and databases in order to design an integrated information system comprehensive of all relevant data necessary to develop user-centered counselling.

Management Response: [Added: 2018/12/17]

The management agrees with the evaluation finding that Integrated Case Management should constitute one of the main pillars of Phase II, given its potential underpinning effect on the policy and institutional reforms in inclusive employment and social service provision. In designing the phase II, the project shall build on the extensive analysis national roadmaps for action, which have been developed for the participating countries/territory.

Key Actions:

Key Action Responsible DueDate Status Comments Documents
10.1. Based on agreed national roadmaps for implementation of integrated case management, prepare a detailed and country-specific plan of actions.
[Added: 2018/12/17] [Last Updated: 2019/05/17]
SDT IRH/ES UNDP COs /NPC 2019/09 Initiated Due date has been set to September 2019. Consultations with national partners are ongoing and expected to be finalized in September 2019. History
Ensure that Phase II embraces different dimensions of ICM’s implementation i.e. capacity building, the organizational/institutional changes, upgrading the IT systems; introducing comprehensive models for holistic assessment of the clients and designing personalized integration plans
[Added: 2018/12/17]
SDT IRH/ES UNDP COs /NPC 2018/12 Completed Proposed approach has been reflected in the project strategy and the RRF of ILMS II
11. Recommendation:

ILMS, apart from TEP concrete interventions is not a type of intervention where physical visibility is offered a strong place.

- Use the recommended online platform to provide strong ADA visibility.

- Use ILMS knowledge products to include a one paragraph on ADA’s commitment to inclusive employment.

- Introduce ILMS workshops and other events with a short introduction on ADA’s commitment to inclusive employment.

- Similar to ESAP, consider developing a 2 minutes film presenting ILMS and make it available online.

Management Response: [Added: 2018/12/17]

The management agrees with the finding and takes note of recommended actions. During the Phase II, the issue of visibility shall be addressed from the broader perspective and the project team shall engage the expert advice of the communication experts

Key Actions:

Key Action Responsible DueDate Status Comments Documents
11.1. In consultations with ILO’s and UNDP’s communications experts, ILMS Phase II will develop a set of communication tools and products, and allocate suitable funding for the communication activities, which shall increase project’s and donor’s visibility. The examples proposed by the evaluator, constitute a good foundation for developing a systematized visibility plan
[Added: 2018/12/17] [Last Updated: 2018/12/18]
SD and Communications Teams/UNDP IRH ILO DWT/CO Budapest 2019/09 Initiated Communications team has suggested an optimal communications strategy, but it is also expected that they will assist in the design of the communications/visibility products History

Latest Evaluations

Contact us

220 East 42nd Street
20th Floor
New York, NY 10017
Tel. +1 646 781 4200
Fax. +1 646 781 4213
erc.support@undp.org