Mid-term Evaluation of the Project Support to Afghanistan Livelihoods & Mobility (SALAM)

Report Cover Image
Evaluation Plan:
2015-2020, Afghanistan
Evaluation Type:
Mid Term Project
Planned End Date:
12/2018
Completion Date:
12/2018
Status:
Completed
Management Response:
Yes
Evaluation Budget(US $):
50,000

Share

Document Type Language Size Status Downloads
Download document SALAM MTE ToR.pdf tor English 461.50 KB Posted 240
Download document SALAM MTE Report_Final.pdf report English 1218.94 KB Posted 524
Title Mid-term Evaluation of the Project Support to Afghanistan Livelihoods & Mobility (SALAM)
Atlas Project Number: 00094515
Evaluation Plan: 2015-2020, Afghanistan
Evaluation Type: Mid Term Project
Status: Completed
Completion Date: 12/2018
Planned End Date: 12/2018
Management Response: Yes
UNDP Signature Solution:
  • 1. Energy
Corporate Outcome and Output (UNDP Strategic Plan 2018-2021)
  • 1. Output 3.1.1 Core government functions and inclusive basic services4 restored post-crisis for stabilisation, durable solutions to displacement and return to sustainable development pathways within the framework of national policies and priorities
  • 2. Output 3.2.1 National capacities strengthened for reintegration, reconciliation, peaceful management of conflict and prevention of violent extremism in response to national policies and priorities
  • 3. Output 3.2.2 National and local systems enabled and communities empowered to ensure the restoration of justice institutions, redress mechanisms and community security
  • 4. Output 3.3.1 Evidence-based assessment and planning tools and mechanisms applied to enable implementation of gender-sensitive and risk-informed prevention and preparedness to limit the impact of natural hazards and pandemics and promote peaceful, just and inclusive societies
SDG Goal
  • Goal 7. Ensure access to affordable, reliable, sustainable and modern energy for all
SDG Target
  • 7.1 By 2030, ensure universal access to affordable, reliable and modern energy services
Evaluation Budget(US $): 50,000
Source of Funding: Project Resources
Evaluation Expenditure(US $): 50,000
Joint Programme: No
Joint Evaluation: No
Evaluation Team members:
Name Title Email Nationality
Sue Emmott Evaluator sue.emmott@xtra.co.nz
GEF Evaluation: No
Key Stakeholders:
Countries: AFGHANISTAN
Lessons
Findings
1.

2. Relevance
2.1. Policy context

The policy context for SALAM was the massive returns of Afghan refugees and asylum seekers from Pakistan and Europe which was at its height in 2016. How to absorb the very large numbers of returnees and how to regularise the opportunities for legal labour migration were high on the agenda of the Afghan government.

At the Geneva Conference in November 2018 the Afghan Government acknowledged that a number of large-scale, multi-partner reintegration programmes are being implemented around the country in line with the Afghanistan National Peace and Development Framework (ANPDF). These efforts have been appreciated by the principal countries hosting Afghans, which have long maintained that voluntary repatriation and sustainable reintegration is the preferred solution for Afghans residing on their territories. However, it was also noted that, given the current humanitarian context and systemic development challenges facing Afghanistan, return and sustainable reintegration is a task that will require long-term interventions and investments, and which cannot be accomplished by the Governments in the region alone.


Tag: Relevance Local Governance Policies & Procedures Displaced People Refugees

2.

Afghan government policy
The overarching framework of the ANPDF (2017 to 2021) includes a vision to build a productive and broad-based economy that creates jobs. It recognises that investment will not come without peace and that, with chronic and ongoing conflict, a realistic strategy and a long timeframe are necessary. Economic Growth and Job Creation is one of four priorities, including investment in vocational education to help align skills with market demands. The potential positive contribution of returning migrants and IDPs is recognised as well as the importance of integrating the generation of Afghan youth raised in exile into the labour markets. There is a commitment to improving the quality assurance of training institutions using an outcomes-based education model and to restructure curriculum and assessment to ensure quality of learning.


Tag: Relevance Policies & Procedures Migration Jobs and Livelihoods Capacity Building Displaced People

3.

One UN for Afghanistan
SALAM is framed by the One UN for Afghanistan approach.  One UN recognises the ANPDF and the development planning system that underpins it as the single coordinating structure for development assistance and commits UN agencies to working as one program. One of the thematic areas represents the core responsibility of the UN agencies in norm and standard- setting and dissemination of public policy lessons and best practice. Of the thematic areas and outcomes, two are relevant to SALAM.

A key element of the UN effort is providing capacity support to the Government to establish the required regulatory framework and strategy and promote the transition to productive employment and decent work. The mandates of the UN partner agencies in SALAM are anchored in the 2030 Agenda for Sustainable Development and committed to the principles of universality, equality and leaving no one behind.


Tag: Relevance UN Agencies Agenda 2030 Leaving no one behind

4.

Government of Finland policy
Finland’s program in Afghanistan is delivered through a partnership agreement on bilateral cooperation signed in April 2013. The goal is to help Afghanistan develop into a country that poses no threat to international or regional security and will not be an international hub for the production of illegal narcotics. Among the factors that are important for Finland in supporting sustainable development are to develop the foundation for the Afghan economy in diverse manners according to the principles of sustainable development.

Article 6 on Migration states that the Parties will work in partnership to tackle the trafficking and smuggling of human beings and will continue to engage in close cooperation aimed at preventing illegal immigration and the illegal presence of physical persons of Afghan and Finnish nationality in the other Party’s territory and will facilitate the return of such illegally present persons to their country of origin.


Tag: Relevance Bilateral partners

5.

2.2. MOLSAMD

MoLSAMD is mandated to work on labour affairs, social protection and welfare. For labour policy the Ministry has worked with ILO on employment strategies, Labour Law reform, international labour standards, migration, skills development, child labour and social dialogue. For skill development the framework is the National Skills Development Project (NSDP), which was one of the first National Priority Programs (NPPs) commencing in 2004. It preceded but was included in the 2008 Afghanistan National Development Strategy (ANDS) and retained in the 2017 ANPDF. Government policy is that other initiatives on skills development are supposed to be anchored within NSDP. The main and ongoing support to NSDP has been the World Bank-funded Afghan Skills Development Project (ASDP). There have been various projects in MOLSAMD that have skills development as an objective. The diagram below situates SALAM among these within a 14 year timeframe starting in 2004 with the launch of NSDP up to the present in 2018 and in the context of the Afghan government’s policy framework. It also shows the World Bank-funded PLACED, which is a project in preparation on managed labour migration.

Two other projects are relevant to SALAM: Skills Assessment and Certification for Afghanistan (SACA) (2014-17) and Non-Formal Approach to Training, Education, and Jobs in Afghanistan (NATEJA) (2014-18).


Tag: Partnership Results-Based Management Country Government International Financial Institutions UN Agencies Capacity Building

6.

2.3. SALAM design

At the time of design in 2016, SALAM was a highly relevant project seeking to respond to a significant and urgent problem. It was designed jointly by UNDP, ILO, UNHCR and MOLSAMD and, at the point of signing the agreement, there was a strong sense of ownership and shared vision. This included a clarity of focus on the need to contain the problem of Afghans using informal routes of migration to Europe. The partnership between UNDP, ILO and UNHCR had the potential to address labour migration at policy level and to work on the nexus of relief and development at local level by building on UNHCR’s humanitarian base of support to returnees using UNDP’s long term approach to sustainable livelihoods. In view of the scale of the problem, the ambition for SALAM was high with a budget of US$120 million and scope across five provinces with linkages to the wider region.


Tag: Programme/Project Design UN Agencies Migration

7.

Theory of change

SALAM works in two streams (outputs):

  • Output 1 aims to formalising the institutional structures for labour migration
  • Output 2 aims to increase employment opportunities for potential migrants and returnees.

Each stream has a theory of change, summarised below and shown in greater detail in Annex 3.

The framework project had a third output which was dropped when it became clear that the project would not be funded in full. However, although available funding from Finland was only around 4 percent of the desired, there was no change to the intended outputs. Such a level of ambition was unrealistic.


Tag: Programme/Project Design Theory of Change Migration Jobs and Livelihoods

8.

Beneficiaries
The design of SALAM is centred on preventing irregular migration. The original target group was those who had returned, mainly in an involuntary way, from Europe and Pakistan who were considered high risk in terms of future irregular migration. This group are therefore the starting point for identification as SALAM beneficiaries.

At midterm, when the process of identifying beneficiaries in Nangarhar is only just starting, the target group has widened. It includes IDPs, in recognition of similar and sometimes greater need than returnees. It extends to members of the host communities, in recognition of the potential for conflict when the influx is massive. Women are an equity-based target group, further specified as IDP female heads of household, young women and girls, and women entrepreneurs. A second equity-based target group is people with disabilities. Inclusion also extends to unskilled labourers as an especially vulnerable group. The two recently issued contracts for delivery of job placement and TVET include quotas for women, disabled and youth. The additional rationale for inclusion of youth is that extreme vulnerability increases the risk of radicalization and recruitment for violent extremism.

All these rationales are valid, and they are in line with donor commitments to invest in durable solutions for refugees, internally displaced people and sustainable support to migrants, returnees and host/receiving communities, as well as for other situations of recurring vulnerabilities. However, the result is that the criteria are so inclusive that almost everyone has the potential to be included. At the point that beneficiaries are to be selected, and in a context where opportunity and income are highly desired and contested, the process of selection is more likely to be mired in compromise than if it was more tightly defined. The challenge of identifying beneficiaries is exacerbated because the total number (originally 2,400 reduced to 1,400) is very small in relation to demand (estimated at more than 100,000 families). This will inevitably undermine the desired result of job creation and reduce the relevance of SALAM in the area that is most important to the donor and the provincial authorities.


Tag: Migration Displaced People Refugees Vulnerable Youth Gender Equality Programme/Project Design

9.

Labour market analysis
Underpinning the aim of creating jobs is assessment of the labour market. Originally an activity of the project was to conduct its own assessment. However, with the various delays, it was agreed to use a 2017 Oxfam study as the basis for planning. This describes an increasingly fragile situation in Nangarhar, with extremely high levels of stress in the casual labour market and a significant decrease in daily wages. Although returnees, especially those who were in Pakistan long term, often have a higher level of skill, their access to the labour market is limited because it is heavily mediated through family and social connections. Skill possession alone, even where there is demand, does not guarantee access to jobs. For women, access to jobs is additionally severely limited due to historic cultural and structural inequities.


Tag: Challenges Programme/Project Design Jobs and Livelihoods Trade and Development

10.

Donor engagement
At the time of design a significant proportion of the $120 million budget was expected to come from the European Union with IOM as a partner alongside ILO, UNDP and UNHCR. However, the EU decided to fund IOM and no other funds were forthcoming. In supporting SALAM, Finland had hoped that their investment would catalyse additional funding if results could be showcased. The issue of resource mobilization and fundraising was raised at the June 2018 Project Board. Although participants considered SALAM to be the most significant project aiming to link humanitarian and development assistance, donors continued to be cautious about supporting it. In the absence of visible results it is certainly difficult to engage donors actively.


Tag: Resource mobilization Donor relations Partnership

11.

3. Effectiveness
3.1. Results to date
Results at output level

During 2017 - the inception phase - all activities related to start-up and there was no progress on outputs. At the time of the MTE in late 2018 the project still cannot demonstrate results at output level although there are a number of activities underway that are heading in that direction. The slow progress, especially for beneficiaries is Nangarhar, is a source of frustration for all partners.
Output 1: Formalized institutional structures in support of regular labour migration for afghan women and men are established

For 2018 the indicators were refined and the number reduced. Targets were also revised downwards to be more realistic but were set as a cumulative total for 2019.

Output 2: National and international employment opportunities for women and men potential migrants and returnees in Nangarhar are increased

As with output 1, there was minimal progress on output indicators and targets set for 2017 were not reached.

Likewise, indicators were revised for 2018 and new targets set for 2019. For indicator 2.1 the wording about completion of training reflects the fact that, although beneficiaries may graduate from SALAM training, there is no official certification process for the skills they attain.


Tag: Effectiveness Capacity Building

12.

3.2. Activities
Status of activities

At activity level, under pressure to demonstrate results, activities have finally gathered momentum. The table shows their status.

The activities in the Annual Workplan were not revised during inception and have not been revised since. Establishment of a TVET Board is no longer appropriate as the government has established a TVET Commission in its place. Although a Board is still on the agenda, GIZ is the main and most appropriate partner along with an EU-funded investment to follow this through. Also not active is conducting a labour market review as the Oxfam assessment is fit for purpose.


Tag: Donor relations Project and Programme management

13.


Activities in Nangarhar

The activity which most interests the donor and the provincial authorities (DOLSAMD and the Governor’s Office) is delivery of competency-based training programs for priority trades. This activity has only just started with contracts recently issued to private sector providers for job placement (200 beneficiaries) and TVET (600 beneficiaries). When time was running short and the HACT assessment risk rating for procurement was significant, UNDP undertook procurement on the government’s behalf. Once the contractors were engaged, UNDP continued to manage the contractors rather than handing programmatic responsibility back to MOLSAMD to be managed by DOLSAMD at provincial level where the activities are taking place.

At the time of the MTE the Director of DOLSAMD had not been officially informed of the activities and had not wished to meet the contractors until the necessary information and authority was provided. The most recently contracted consortium AVTI/PADO has not worked in Nangarhar previously and had not registered with the Department of Economy. This is important because, regardless of whether there is capacity to do so, DOLSAMD has responsibility for monitoring skills provision which is designed and delivered outside of government-supported systems to ensure that it does not fall short of national standards. In fact, due to shortage of time before the end of SALAM, the duration of training is too short to deliver the quality expected by MOLSAMD. But, as one of the National Technical Assistants (NTA) in Nangarhar has the title of Field Monitoring and Reporting Officer, there should be capacity to take on this role.

These were very immediate factors that pose a high risk to successful completion of activities unless they are addressed with urgency (see factors affecting achievement). Outstanding are training activities for a further 600 beneficiaries under ILO’s management. At the time of the MTE it was not clear what form they would take.


Tag: Local Governance Procurement Jobs and Livelihoods Capacity Building

14.


Changes to activities

It is clear that there have been some shifts in project focus during implementation although these are difficult to track. At the higher level there has been a shift from emphasis on migration to emphasis on reintegration with a focus on areas of high return. The Annual Report of 2017 indicates that activities oriented to promotion of outward migration as a primary response to unemployment were replaced by those oriented to in-situ job-creation. There was also mention of increasing opportunities for host communities as well as returnees as an important conflict prevention consideration. However, one year on, job creation is receiving far less attention than training. This matters because a guiding principle of TVET is that it should be matched by job creation to mitigate against the potential to increase frustration among job-seekers who believe training will lead to a job. In the design of SALAM there is mention of strengthening the capacity of DOLSAMD’s Employment Service Centres (ESCs) because they serve the public as well as SALAM beneficiaries. It is not clear whether, or why, this has been dropped.


Tag: Jobs and Livelihoods Capacity Building

15.

3.3. Factors affecting achievement
Clarity of purpose

The main factor affecting achievement is clarity of purpose. The intended outputs are set at an unrealistic level. Some of the activities are individually useful but they are spread widely and it is difficult to see how they connect to each other and what they add up to. In other words, SALAM lacks the kind of focus that would enable progress on results to be tracked.

In output 2 there is a lack of clarity about whether the project is working on job creation or simply on vocational training. In fact most of the activities are about training but it is not easy to understand whether this has happened because job creation was assessed to be too difficult or inappropriate or whether, in the face of pressure to demonstrate results, it was simply missed.


Tag: Challenges Effectiveness Sustainability Project and Programme management

16.

Capacity

A key contribution of the project is supposed to be capacity development. The approach taken is to add capacity to the ministry at national and subnational level in the form of NTA. The number of people, and their job descriptions, seem to have been taken from the framework project without consideration of the exact capacity problem that the NTA are supposed to be a solution to. The framework design does not include a capacity assessment and, although the subproject has implicit and explicit objectives of capacity development, there is no capacity development plan. The result is that the NTA have not been provided with clear direction and some have not been able to establish roles for themselves that are likely to increase the effectiveness of the project. Consequently NTA appear to be under-utilised.


Tag: Risk Management Jobs and Livelihoods Private Sector Challenges Effectiveness Results-Based Management

17.

Certification and standards

The project discovered very late, during the development of the TVET tender, that a national assessment and certification system does not yet exist. National Occupational Skills Standards (NOSS) and course curricula have been many years in the making, including under the project NATEJA, but are not at the stage that they can be used to certify any training conducted under SALAM. This means that anyone who completes a course under SALAM will not have a recognised qualification. MoLSAMD will only be able to certify that they attended and completed training but not that they are proficient or reach the approved NOSS quality standards. In fact, as the duration of training is less than MOLSAMD consider necessary to acquire the skills to do the job, the training provided through SALAM is unlikely to result in proficiency.


Tag: Challenges Operational Efficiency Capacity Building

18.

Gender
SALAM aims to deliver results for women as well as men. At policy level, the signed and draft Bilateral Labour Agreements have moved forward without consideration of gender sensitivity. ILO raised the issue and suggested that gender integration should be a dedicated session of the National Labour Migration Strategy Inter-Ministerial Task Force but at a point when extensive negotiations and high- level endorsement had already taken place and the BLAs were ready to be signed. ILO and UNDP may not have been able to influence the process to incorporate a gender perspective but it is clear that the effort to influence came too late. It is unlikely that there will be any updates to the BLAs within the timeframe of SALAM. The lack of a gender perspective can partly be explained by the fact that the vast majority of Afghans working in Gulf countries are male and the structural barriers to female entrants are such that they cannot be overcome in the foreseeable future.


Tag: Effectiveness Gender Equality Women's Empowerment

19.

SALAM is a very ambitious project in scope and far too ambitious within the timeframe.

Output 1 is essentially about institutional development. Global experience is that institutional capacity development is a long term endeavour which can only be led from within government as part of the political process of reform.  For that reason it is not easily influenced by development partners within the constraints of a timebound project.

This output seeks to develop an institutional framework for labour migration. ILO research shows that Afghanistan is 10–15 years behind other South Asian countries such as Bangladesh, Nepal and Pakistan in instituting a proper system for governance of formal labour migration. Recent World Bank research, undertaken as part of the preparation process for the PLACED, shows that countries such as the Philippines and Vietnam have taken two to three decades to build a fully functioning labour- sending system. ILO-led activities in this output are therefore unrealistic within a three year timeframe. This was already known as, for example, the activity relating to training on labour migration is an extension of work to develop a national training course that had not been completed in 2013 when the project ended and no further funding was available.


Tag: e-Governance Migration

20.

Output 2 relates to job creation and skill development. The experience of NSDP is instructive in terms of what is reasonable to achieve and within what timeframe. Many factors beyond the control of the implementing partners (MOLSAMD and MoE) limited achievement: the elections in 2009; increasing insecurity; entrenched bureaucratic structures; a high degree of centralisation of the public TVET sector; and high turnover in the leadership of the implementing ministries. One of the sticking points - essential if the quality of TVET was to be improved - was granting financial autonomy to public, project-supported TVET institutes, which did not have buy in at political level.

The original Project Development Objective of the second ASDP was to increase the potential for employment and higher earnings of graduates from Vocational Education and Training institutions through improvements in the skills delivery system. When progress stalled in 2015, the project was restructured to focus more narrowly on improving TVET teacher competencies and curriculum in selected priority trades.  In that case the reasons for slow implementation were: insufficient coordination between implementing agencies; poor procurement and contract management; and high staff turnover. Within that project was a sub component to develop a system for benchmarking and certification of competencies, led by ILO.

In the light of all this experience it is not surprising that SALAM is not achieving as intended. Institutional capacity development takes a very long time. Not least, to be successful a range of relationships of trust must be built and that cannot be done quickly. When individuals move on, that relationship building has to start over. For international staff, who are used to delivering in tight timeframes this can be challenging and, when under pressure to demonstrate results, they may forego the relationship in favour of speed, as in the case of managing contracts. But this is a high-risk approach, which may damage the relationship with DOLSAMD beyond repair and have a knock on effect on the delivery of training.


Tag: Operational Efficiency Procurement Capacity Building Coordination

21.

4. Efficiency
4.1. Delivery

At midterm none of the partners – MOLSAMD, UNDP, ILO, UNHCR and Finland - were happy with the progress made by SALAM. By the end of 2017, delivery was only 19 percent against target. For 2018 official figures are not yet available but the guesstimate for expected delivery at the time of the MTE is 30 percent for ILO and 80 percent for UNDP (actual delivery plus commitment). In order to protect funding for 2018 this includes expenditure on activities that have been initiated in 2018 but will not be delivered until 2019.

Common concerns are the length of the inception phase, the slow pace of initiating results-oriented delivery, and the difficulty establishing a collaborative working relationship between partners. In Nangarhar, DOLSAMD has expressed concern about the delay in implementation in the province since at least March 2018. For MOLSAMD, an additional and major concern has been the proportion of the budget allocated to international staff costs.

The inception phase lasted much longer than intended. On the side of UNDP, processes for recruitment and establishing the fund transfer mechanism between partners were prolonged. On the side of MOLSAMD, the change of leadership led to delays in some important approvals necessary to move forward. In part, the nature of some processes is that there are several steps, each of which takes time. But the failure to establish consistent and full time project management to follow through on start-up activities and to ensure timely delivery was a significant cause of delay in delivery.


Tag: Efficiency Donor relations Partnership

22.

4.2. Project management
Overall project management has been weak. The critical position of Project Manager took time to fill and the recruit resigned after six months. Thereafter a UNDP Program Officer acted in the role, alongside his other work, from March to September 2018, when the ILO Project Coordinator took over, also in an acting capacity. This has meant that the project has never had a dedicated manager who could consistently cultivate and maintain relationships with MOLSAMD and coordinate between UN partners. Critically, there was no-one who could provide consistent direction and support to the NTA teams in Kabul and Jalalabad.

Recruitment of other staff has also been slow. International positions were not filled until September 2017 for the Planning, Monitoring and Reporting Specialist (PMRS) after two rounds and December 2017 for the Chief Technical Adviser (CTA) after three attempts.

Eight positions for NTAs are allocated. Filling the four in MOLSAMD was not easy but was eventually successful. In Nangarhar the Provincial Coordinator and the Senior Inter-Agency Liaison Officer (based at the Governor’s Office) were filled but both resigned and the posts are currently vacant.


Tag: Project and Programme management

23.

4.3. National implementation
SALAM is implemented by MOLSAMD through the National Implementation Modality (NIM). This means that ownership and responsibility for all programming decisions rests with the government. UNDP manages the risk through the mechanism Harmonized Approach to Cash Transfers (HACT), which includes a rigorous assessment of ministry capacity to devolve responsibility for project financial management to MOLSAMD in a way that is accountable. HACT also establishes a simplified set of procedures for manging financial transfers between UN agencies. The Micro Capacity Assessment Report of the HACT process was submitted in January 2018 and subsequent time was needed to deal with issues arising.


Tag: Efficiency Implementation Modality Ownership Procurement Private Sector

24.

4.4. Institutional and governance arrangements Governance

At strategic level the Project Board is the governance mechanism. Two Boards have been held, in December 2017 and June 2018.

When the gap in project management due to the lack of a Project Manager became apparent, a Technical Working Group (TWG) was initiated. This has been successful in raising and dealing with important matters requiring decisions. However, in the absence of routine project management meetings that would normally be convened by a project manager, the TWG acted as a substitute. There does not seem to have been any control over who attended. In the June 2018 TWG there were
12 participants. Alongside Embassy of Finland, senior UN partners and the MOLSAMD Director of Skills Development, the four NTA also attended. Given some of the subject matter discussed, which included disclosure of financial and procurement information, this is inappropriate. There were also concerns among some members that disagreements within the project were being aired inappropriately and that some behaviour was unprofessional.


Tag: e-Governance Project and Programme management

25.

Strategic coordination
An important element of the design of SALAM is high level coordination on displacement. This starts with the government-led DiREC (Displacement and Return Executive Committee) and is supported by the Durable Solutions Working Group (DSWG), which ensures the linkage with reintegration platforms and coordination activities.  The DSWG is chaired by the Ministry of Refugees and Repatriations (MoRR) and co- chaired by UNHCR, UNDP and IOM on a rotational basis. During inception SALAM used DSWG as a platform to position itself and is subsequently represented at meetings for the purpose of coordination.

Another coordination forum is the Labour Migration Working Group (LMWG), hosted by the World Bank to bring together partners and experts working to support the development of a labour migration management system.

At provincial level it was hoped that there would be a P-DiREC. SALAM aimed to promote its success by placing a Coordinator in the Office of the Governor in Jalalabad. However, the recruit was never clear about the purpose of the role and there was insufficient support from Kabul to make it a reality. Subsequently the person resigned and there has been no further action. Nangarhar does have a Provincial DSWG in which UNHCR is very active. However, after energetic coordination in 2016 and 2017 when there were massive returns, the group has reportedly settled back into a less active role.

ILO has coordinated closely with the World Bank during the project preparation for PLACED because of the close relationship between Output 1 and PLACED.


Tag: Donor relations Coordination Displaced People

26.

4.5. Factors affecting efficiency
Project structure

The way the project structure (organogram) is presented (see Annex 5) does not reflect the NIM modality. Normally the top of the organogram would be the Project Director who, in SALAM, is the MOLSAMD Director of Skill Development. The Project Manager would report to him as well as, for accountability purposes, to UNDP. However, in SALAM, the PM reports only to the Head of the Livelihoods Unit in UNDP, as do the international CTA and PMRS. This, combined with the PM mainly being performed by a UN staff member in an acting capacity, has distorted the ownership of SALAM. International staff believe that they are working for UNDP, not the government. Even though the NTA are on government contracts, the fact that their positions are funded by SALAM has created the impression that they too work for UNDP.

In the absence of a PM it would be reasonable to expect that the CTA might take a technical lead and the PMRS might take the lead on planning. This has not happened and there is a vacuum of leadership. The impression given is that everyone has a hand in everything but no-one is in charge. Combined with ambiguity around the outputs (see effectiveness section), the result is that SALAM lacks direction.

The reason SALAM has lacked a permanent PM is supposedly because of the difficulty of recruitment. However, especially after the closure of several very large USAID-funded projects after the security transition in 2014, the kind of senior person who can manage a project and cultivate appropriate and respectful relationships with government can be found in the market relatively easily. Other agencies do not face the same difficulties that UNDP claims to have.


Tag: Efficiency Implementation Modality Project and Programme management UNDP Regional Bureaux

27.

4.5. Factors affecting efficiency

Capacity

Capacity development has been covered in the effectiveness section. It is also an important influencing factor for efficiency. Like many ministries, MOLSAMD is still developing its capacity after the turmoil of the years of conflict and the sometimes capacity depleting effects of projectized donor funding. Since 2002, a solution to capacity weakness has been to add capacity in the form of international and national TA. If the added NTA capacity is strong it can, to some extent, compensate for weakness in the tashkeel (civil service).However, years of experience in Afghanistan and globally have shown that NTA is rarely successful at developing capacity and tends to perpetuate substitution. Often, NTA do not themselves have capacity due to the political economy of recruitment, which tends to favour nepotism.


Tag: Efficiency Implementation Modality Operational Efficiency Oversight Capacity Building

28.

Insecurity

Insecurity does not directly impact on SALAM activities. However, ongoing and worsening insecurity in Nangarhar and mandatory UN security restrictions on travel does reduce the number of times and length of visits of the international staff. This affected the assessment of VTCs, which was done late and less thoroughly than desirable. It currently makes it difficult to monitor activities and to mitigate the risk of the inexperience of the NTA.

Time
As well as affecting effectiveness, as already discussed, time also affects efficiency. Some of the delays experienced in SALAM are normal and, in the Afghan context, few projects deliver on agreed targets within the initial timeframe and even in extension. For this reason, a three year timeframe to deliver outputs, especially when they start from scratch rather than building on a previous intervention, is very short. Therefore, Finland’s assumption that the small size of SALAM relative to the EU and World Bank projects may be advantageous in terms of faster delivery is likely false.


Tag: Effectiveness Efficiency Partnership Procurement Security

29.

5. Sustainability

As detailed in the effectiveness section, activities are still in their early stages. This means that no comment can be made on sustainability. For activities relating to institutional development, the timeframe needed is much longer than the three years of SALAM. In fact, some of the activities of ILO are building on work that was started but not completed several years ago and, especially that relating to the normative role of the UN, is work that will continue indefinitely.

For the activities directly benefiting beneficiaries, sustainability of the results of training and job placement will depend very much on the quality of experience the trainees receive. If it is high quality they may have improved job opportunities. However, since access to jobs is so heavily influenced by connections and networks, even high quality training may not lead to jobs. Low quality training almost certainly will not result in jobs. As previously discussed, the timeframe and the inexperience of the contractors means that the quality of training is compromised.


Tag: Sustainability Jobs and Livelihoods

Recommendations
1

Confirm the leadership role of the Project Director and recruit a Project Manager.

2

Develop an Annual Workplan for 2019 with realistically achievable indicators of activities so that SALAM can come to a satisfactory conclusion.

3

Revisit the governance arrangement especially the purpose and membership of the Technical Working Group, paying attention to appropriate hierarchy.

4

Revisit the role of the international staff so that MOLSA can re-establish ownership

5

Develop an exit strategy and consider the future beyond SALAM

1. Recommendation:

Confirm the leadership role of the Project Director and recruit a Project Manager.

Management Response: [Added: 2019/04/25]

Agreed. The leadership role of the Project Director is confirmed under National Implementation Modality (NIM) and the project implementation is on-going. The Project Manager recruitment is also under process.  

Key Actions:

Key Action Responsible DueDate Status Comments Documents
Conduct monthly meeting with Project Director to ensure government ownership
[Added: 2019/04/25] [Last Updated: 2019/06/09]
Acting Project Manager and Chief of Livelihoods and Resilience unit 2019/06 Completed Monthly meetings are being conducted with the national project director. History
Conduct recruitment of the shortlisted candidates (Project Manager)
[Added: 2019/04/25] [Last Updated: 2019/06/09]
Acting Project Manager and Chief of Livelihoods and Resilience unit 2019/05 No Longer Applicable [Justification: Based on donor's decision and official letter, no new recruitment in 2019, will be carried out by the project. ]
History
2. Recommendation:

Develop an Annual Workplan for 2019 with realistically achievable indicators of activities so that SALAM can come to a satisfactory conclusion.

Management Response: [Added: 2019/04/25] [Last Updated: 2019/04/25]

Agreed. The 2019 AWP with two new indicators and realistic annual targets has been developed in close consultation with the implementing partner .

Key Actions:

Key Action Responsible DueDate Status Comments Documents
Add two new indicators under the second output of the project
[Added: 2019/04/25]
UNDP Programme Analyst 2018/12 Completed Minutes of project board meeting. 2 indicators added and approved by the project board.
Add realistic achievable annual targets
[Added: 2019/04/25]
UNDP Programme Analyst 2018/12 Completed 2019 AWP, revised RRF and Log Frame History
Conduct a Project Board Meeting to approve the 2019 AWP with revised indicators and annual targets
[Added: 2019/04/25]
UNDP Programme Analyst 2018/12 Completed Minutes of project board meeting of 3rd Dec 2018 History
Sign the 2019 Annual Workplan by the Project Board members
[Added: 2019/04/25]
UNDP Programme Analyst 2019/03 Completed
3. Recommendation:

Revisit the governance arrangement especially the purpose and membership of the Technical Working Group, paying attention to appropriate hierarchy.

Management Response: [Added: 2019/04/25]

Agreed. The governance arrangement with the purpose and membership of the Technical Working Group will be re-visited jointly with Ministry of Labor Social Affairs (MoLSA).

Key Actions:

Key Action Responsible DueDate Status Comments Documents
Initiate monthly meetings with project director to ensure the ownership of government
[Added: 2019/04/25] [Last Updated: 2019/06/09]
Acting Project Manager and Chief of Livelihoods and Resilience unit 2019/06 Completed Monthly meetings initiated with the project director. History
Invite senior staff of MoLSA, UNDP, ILO, UNHCR and Finland Embassy to participate in the meetings of the technical working group (TWG)
[Added: 2019/04/25] [Last Updated: 2019/06/09]
Planning Monitoring and Reporting Specialist 2019/05 Completed Senior staff of MoLSA, UNDP, ILO, UNHCR and Finland Embassy are regularly participating in TWGs. History
4. Recommendation:

Revisit the role of the international staff so that MOLSA can re-establish ownership

Management Response: [Added: 2019/04/25]

Agreed. The role of international staff was and will continue be revisited. In view of the no cost extension granted exceptionally by the donor, the Chief Technical Advisor (CTA) was not extended after December 2018. Technical support will be sought through ad hoc consulting and a higher involvement of the project national technical advisers.  The Planning, Monitoring and Reporting Specialist (PMRS) will provide technical support to MoLSA and build the monitoring capacity of NTA staff to monitor the project. Project implementation remains the responsibility of Ministry of Labour and Social Affairs as the UN project implementing partner.

Key Actions:

Key Action Responsible DueDate Status Comments Documents
Monitoring Tracking Tools to be developed for SALAM project
[Added: 2019/04/25]
Planning Monitoring and Reporting Specialist and Programme Analyst 2019/03 Completed Tracking tools for contract management added to the existing project M&E system
Capacity Development plan for NTA staff to carry out the implementation of the project
[Added: 2019/04/25] [Last Updated: 2019/09/24]
Planning Monitoring and Reporting Specialist 2019/08 Completed This action completed on August 2019. History
Start Implementation of the capacity development plan
[Added: 2019/04/25] [Last Updated: 2019/12/16]
Planning Monitoring and Reporting Specialist and PSRT 2019/12 Completed This activity is completed. History
5. Recommendation:

Develop an exit strategy and consider the future beyond SALAM

Management Response: [Added: 2019/04/25]

Agreed. In the project document, there is already exit strategy meanwhile, the project in consultation with all stakeholders will consider the future beyond the project. The exit strategy also includes collection and inclusion of project lessons of experience into the current UNDP research on population movements with UNHCR and IOM.

Key Actions:

Key Action Responsible DueDate Status Comments Documents
Prepare an exit strategy plan
[Added: 2019/04/25] [Last Updated: 2019/12/16]
Programme Analyst, Planning Monitoring 2019/12 Completed The Exit strategy plan that focuses on an improved capacity and understanding of project national technical advisory staff in the implementing agency (the Ministry of Social Affairs) to sustain project results as per RRF and logical framework and develop future interventions on reintegration and return based on lessons of experience from SALAM. History
Organise a project board meeting to focus on exit strategy
[Added: 2019/04/25] [Last Updated: 2019/06/09]
Programme Analyst, Planning Monitoring and Reporting Specialist 2019/06 Completed The Project board meeting decided to close the project by December 2019, as per the plan. History
Closure of contracts, transfer or relocation of assets
[Added: 2019/04/25] [Last Updated: 2019/12/16]
Programme Analyst, Planning Monitoring and Reporting Specialist 2019/12 Completed This can be agreed in the 2019 July project board meeting. History

Latest Evaluations

Contact us

1 UN Plaza
DC1-20th Floor
New York, NY 10017
Tel. +1 646 781 4200
Fax. +1 646 781 4213
erc.support@undp.org