Mid-Term evaluation of "Innovation for Service Delivery (ISDA)" project

Report Cover Image
Evaluation Plan:
2017-2021, Albania
Evaluation Type:
Mid Term Project
Planned End Date:
12/2019
Completion Date:
12/2019
Status:
Completed
Management Response:
Yes
Evaluation Budget(US $):
10,000

Share

Document Type Language Size Status Downloads
Download document ISDAsp-National Senior Expert for Final Project Assessment ToRs.docx tor English 22.64 KB Posted 236
Download document Evaluation report ISDA SUPPORT PROJECT - D. Jashari, November 2019.pdf report English 675.58 KB Posted 319
Title Mid-Term evaluation of "Innovation for Service Delivery (ISDA)" project
Atlas Project Number: 00068697
Evaluation Plan: 2017-2021, Albania
Evaluation Type: Mid Term Project
Status: Completed
Completion Date: 12/2019
Planned End Date: 12/2019
Management Response: Yes
UNDP Signature Solution:
  • 1. Governance
Corporate Outcome and Output (UNDP Strategic Plan 2018-2021)
  • 1. Output 1.1.1 Capacities developed across the whole of government to integrate the 2030 Agenda, the Paris Agreement and other international agreements in development plans and budgets, and to analyse progress towards the SDGs, using innovative and data-driven solutions
SDG Goal
  • Goal 16. Promote peaceful and inclusive societies for sustainable development, provide access to justice for all and build effective, accountable and inclusive institutions at all levels
SDG Target
  • 16.6 Develop effective, accountable and transparent institutions at all levels
Evaluation Budget(US $): 10,000
Source of Funding: Project Budget
Evaluation Expenditure(US $): 5,276
Joint Programme: No
Joint Evaluation: No
Evaluation Team members:
Name Title Email Nationality
Dalina Jashari External Evaluator djashari@idmalbania.org ALBANIA
GEF Evaluation: No
Key Stakeholders: Ministry of Innovation and Public Administration
Countries: ALBANIA
Lessons
Findings
1.

Relevance

The ISDA support project is highly relevant and responds to the reformations and challenges of the Albanian Government at transforming the public service delivery in the country through the citizens- centric approach. Such approach and vision are part of the a) Cross-Cutting Public Administration Reform Strategy (PAR) 2015–2020 which constitutes the overall framework for the modernization and transformation of institutions and practices of public administration in the country, with the vision of providing “high quality services for citizens and businesses in a transparent, effective, and efficient way through the use of modern technologies and innovative services, and that complies with the requirements of European integration through impartial, professional and accountable civil servants, part of efficient setups”; b) of the National Strategy for Development and Integration (NSDI) 2015–2020. Furthermore, this vision is supported c) by the Cross-Cutting Strategy “Digital Agenda for Albania" (DAA) 2015–2020 under the strategic priority 2 “Policy for the development of electronic governance and delivery of interactive public services for citizens and businesses”. The project’s key strength relies in addressing the public service delivery challenges focused on the citizens, in an innovative and consistent manner in close partnership with the government at central and local level. The evaluation confirms that the project has critically contributed at the implementation of the government’s priorities on public service delivery since the beginning of this efforts from the Albanian government in 2014 and in expanding geographically the pool of these services focused on quality and inclusiveness. Additionally, stakeholders involved in this assessment report stated that UNDP is in a unique position as a donor and as an international agency with constructive cooperation with government, other donors and international organizations as well as with Albanian non-state stakeholders (CSOs, media, businesses, academia etc.)


Tag: Relevance Anti-corruption Local Governance Public administration reform Agenda 2030

2.

Effectiveness and Efficiency

The ISDA Support Project implementation (2014-2019) has gone through two main milestones, the first one mainly focused on policy formulation and institutional capacity building and the second one mainly focused on adopting best practices and promoting innovative solutions. Intervention during both phases is categorized by a clear logic and tailored to the government challenges and changes on public service delivery system. Over the 5 years of implementation the government institution in charge of the Public Administration Reform (PAR) has been changed 2 times, however, given the commitment and the professionalism of the project management, the UNDP and involvement of all the stakeholders, the ISDA Support Project managed to overcome and to achieve the intended outcomes in time. This project was a multi-year project including combination of different types of activities at strategic and technical level. The general framework of the project is well-detailed by specifying for each of the expected outputs respectively the output indicators, data source, baseline targets and the data collection methods. The project has achieved to implement the envisaged activities, to deliver the outputs and specifically to achieve the development results. All interviewed stakeholders, expressed satisfaction with both achievement and project performance. All of the set indicators have been achieved and reflection on the level of the overall impact is already in place by the project management.


Tag: Effectiveness Efficiency Public administration reform Project and Programme management Institutional Strengthening Policy Advisory

3.

Public Service Delivery Policy Development, Implementation and Front Office - Back Office separation and the establishment of the service delivery standards for citizens and businesses

The efforts on policy formulation has started since the beginning of the project implementation in 2014 and during the first phase of the project till 2016 the project has substantially contributed to the development of the key policy document “Long term policy document on the delivery of citizen centric services by central government institutions in Albania”, approved by the Decision of the Council of Ministers no. 384, dated 25 May 2016. Key contributions of the project under these two first results consist also on the expertise offered for the process of separation of the front office (FO) from the back office (BO) in central government institutions. Key stakeholders involved in this assessment report highlights that the starting phase of the reform in itself was challenging and the assistance provided by the ISDA Support Project critical. This, due to the fact that “it was not only about changing the public administration, it was about changing a whole system’s approach and culture”. The commitment of the reform leading institution at that time and the support of the project management exceeded some major problems related to the resistance from change of the administration or even to the resistance of the government agencies and offices as regards the front-back office separation process as “a threat” to their authority. Still these perceptions are evident in the work of ADISA, however (to date) the government actors are generally committed to facilitate ADISA’s work as a unique state institution which ensures that citizens and businesses in the country receive quality public service delivery and easy access to the administrative services.


Tag: Effectiveness Efficiency National Public administration reform Implementation Modality Innovation Quality Assurance Service delivery

4.

Agency for Delivery of Integrated Services of Albania and promotion of Innovative solutions promoted sustainably for the public service delivery reform implementation

The Agency for Delivery of Integrated Services of Albania- ADISA is certainly the best practice as regards the public service delivery reform. Such statement is underlined by all the stakeholders interviewed in this report. ADISA, established in 2014 under MIPA10’s authority and by Decree no. 693 of the Council of Ministers with the mandate of managing the citizen service centers, currently operates based on the law no. 13/2016, dated 18 February 2016 “On the way of delivering public services at front office level in the Republic of Albania”, which provides for expanded responsibilities in citizen centric public service delivery. In frame of ISDA Support Project, ADISA has gained organizational and functional support. During the project implementation, through international and local technical assistance, the project supported ADISA on Front Office – Back Office separation, on management of the delivery of integrated services, on customer care service, management and supervision. Furthermore, assistance is given at infrastructure level as well as on institutional capacity building for ADISA and its branches outside Tirana. The support given to ADISA and its growth over the year has been recognized by all the stakeholders which suggests that this assistance is still relevant in order to ensure the sustainability of the agency.

The establishment of the Innovation Lab in 2017 based on the successful practices supported by UNDP in South Asia and Europe is another important achievement of the project under the result “Innovative solutions promoted sustainably for the public service delivery reform implementation”. The first of its kind in the Balkans, ADISA Lab introduces the use of innovation and technology on public service delivery in Albania. This Lab, approved as a selfstanding unit under ADISA’s Executive Director as of January 2019, operates based on the contribution of a diverse group of people within the agency, bringing together different skills and expertise, who are committed in advancing the innovation agenda. The project assisted the ADISA Lab in operational issues (such as operational manuals) as well communication issues, capacity building by offering relevant expertize.


Tag: Effectiveness Efficiency National Public administration reform Implementation Modality Coordination Institutional Strengthening

5.

E-government agenda activities and management support to the reform leader institution

Projects outputs for these envisioned results consists on providing E-Government expertise on the preparation of a pre-feasibility study on broadband development in Albania. This expertize is completed in 2017 and supported the Government of Albania strategic investment plans as well as the digital integration pillar of the Western Balkans’ regional economic area. Experts interviewed in this report considers the implementation of IT solutions are important at enabling public service delivery and increasing benefits to citizens and businesses. The project has also provided e-government expertise, including on guidelines related to planning, implementation and oversight for public services, and standards for increased service automation level. Current activities under this result are related to supporting ADISA in its initiative to increase capabilities of E-Albania usage by the citizens through the availability of double screen service windows in its centers and co-location locations. Important highlights are given from the interviewers as regards the expertize offered by the project on standard compliance and quality assurance in public service delivery, reengineering and deregulation processes, front-back office separation and support to ADISA. Additionally, support is given to the reform leading institution which has been highly important since the beginning of the public service reform in 2014. The ISDA Support Project has continued in doing so during its implementation, however, changes in the government after the elections in 2017 have produced some challenges particularly related to the staff flow and need for sustainability of technical staff in the field.


Tag: Effectiveness Efficiency National e-Governance Technology Institutional Strengthening Technical Support

6.

USE OF RESOURCES AND RISKS

The project is implemented in accordance with UNDP’s National Implementation Modality, whereby the Office of the Deputy Prime Minister is the National Implementing Partner, on behalf of the Government of Albania. The highest management body for the project is the Project Steering Committee (PSC) which includes on the Government side: the reform leader, the appointed project director (who as of January 2019 is ADISA’s Executive Director), ADISA, AKSHI (National Agency for Information Society), Delivery Unit / Prime Minister’s Office, Deputy Minister of the Interior & Agency for Support of Local Self-Government (ASLG) /, and on the donor side: all contributing partners’ representatives - the Italian Agency for Development Cooperation (AICS), the Austrian Development Agency (ADA), and UNDP – as well as EU Delegation and the World Bank- all partners in supporting the Albanian Public Service reform. The UNDP is recognized in this report as a strategic partner of the Albanian Government which has contributed to address the development changes of the country and to support relevant interventions with sound coordination with the donor’s community. The project have involved both long-term and short -term staff, respectively composed by National project manager and a Project assistant and by local and international experts with a strong background in the various sub-areas of public service delivery and innovation.

 


Tag: Efficiency Human and Financial resources Implementation Modality Risk Management Country Government International Financial Institutions

7.

Impact and Sustainability

Different stakeholders shared their views in frame of this assessment report on the role that the ISDA Support Project has played at developing the Albanian’s government reform to improve the public service delivery. All of the individuals unanimously agreed that the project impact is high at policy level as well as at capacity building level. This, due to the political commitment, the individual commitment of the reform leading institution and of the commitment of the UNDP project management. In regard to the impact and the sustainability, the pool of expertise offered by the project since the beginning of the government reform is another key contribution of the project. Such statement is confirmed by all the stakeholders who emphasize it as a must, not only at the beginning of the reform (central government institutions required capacities), but even in the future, particularly focused on innovative solutions.


Tag: Impact Sustainability National Anti-corruption Public administration reform Institutional Strengthening Policy Advisory

Recommendations
1

The reform leading institution, given its important role, should improve capacities on technical expertise/staff in order to better respond to the dynamics of the reform’s implementation The reform leading institution, given its important role, should improve capacities on technical expertise/staff in order to better respond to the dynamics of the reform’s implementation.

2

Following the abundant support given to ADISA during these years, it is important to build its ownership, to consolidate its capacities and the self-sustainability. Interventions in this regard should be a mixture of capacity building to the existing and future staff (on topics related to the service quality, performance monitoring and evaluation), boost use of innovative tools (through the innovation lab) and assistance to ADISA’s organizational structure aiming to sustain its capacities and staff.   

3

The extent to which the reform has been implemented so far in Albania is based also on the valuable contribution from different donors support, such as the Italian and Austrian governments, United Nations, the European Union, and the World Bank. However, the donor support over the next years is still very crucial in order to consolidate even more the results achieved so far, to accelerate the processes and to respond on time to the needs particularly at  increasing the services also from other central institutions and expanding these services all over the country.

Conclusions, lessons and recommendations suggest that (due to their nature and complexity), some of the project outputs needs to be consolidated in the following years, such as ADISA’s business model and its geographic enlargement, government efforts in advancing and supporting its digital agenda as well as further assistance to the vulnerable groups such as Roma community and people with disabilities.

4

Based on the citizen-centric approach, it is recommended that additional focus should be paid to the citizens in remote areas. The ADISA Mobile Office Facility is an important initiative which should continue and expand in the future in all remote areas of the country. Based on the success of the co-location model, the Mobile Office Facility can make use of local capacities on the ground. 

5

As stated in this report, the project has paid special attention to the vulnerable groups and to gender issues. In this regard, it is recommended that the future activities should be focused on ICT literacy for these groups, development of relevant training materials targeting different age groups (especially youth in remote areas). Such initiatives can be coordinated with the education institutions in the targeted areas.

6

The success of the public Service Delivery reform is two-sided, depending both on the government’s and the citizen’s commitment. It is recommended that in the future, additional awareness raising campaigns should be focused on citizens, in order to increase their involvement as beneficiaries of the public services (i.e continue promotion of the Citizens Charter) and as monitoring agents of the public service’s quality.

1. Recommendation:

The reform leading institution, given its important role, should improve capacities on technical expertise/staff in order to better respond to the dynamics of the reform’s implementation The reform leading institution, given its important role, should improve capacities on technical expertise/staff in order to better respond to the dynamics of the reform’s implementation.

Management Response: [Added: 2020/03/03] [Last Updated: 2021/01/25]

The recommendation addresses on ongoing challenge. Recognizing the limited staff at the level of the public services reform leader in Albania, ISDA Support Project has been responsive in providing needed technical expertise. Notwithstanding, ongoing engagement in this direction should remain on focus, also considering the impact of this challenge on the pace on the planned delivery of assistance in support of the reform.

Key Actions:

Key Action Responsible DueDate Status Comments Documents
Continue to provide technical support in reform coordination until the closure of the project
[Added: 2020/03/03]
UNDP CO Management 2020/03 Overdue-Initiated
Ensure an ongoing engagement with reform leadership to anticipate needs and plan for timely support as appropriate
[Added: 2020/03/03]
UNDP CO Management 2020/12 Overdue-Not Initiated To begin upon the start of the new project
2. Recommendation:

Following the abundant support given to ADISA during these years, it is important to build its ownership, to consolidate its capacities and the self-sustainability. Interventions in this regard should be a mixture of capacity building to the existing and future staff (on topics related to the service quality, performance monitoring and evaluation), boost use of innovative tools (through the innovation lab) and assistance to ADISA’s organizational structure aiming to sustain its capacities and staff.   

Management Response: [Added: 2020/03/03] [Last Updated: 2021/01/25]

This is a valid recommendation. As the reform enters its consolidation phase, strengthening institutional sustainability especially with reference to ADISA as a reform champion is a key requisite to ensure that impetus for ongoing citizen-centric improvements in public service delivery is firmly rooted. ISDA Support Project has been a strong support in embedding innovation as an overarching approach to solving problems and addressing citizen needs providing the assistance to launch in mid 2017 of the first government innovation lab in Albania, then build capacity, and support innovation project work. As ADISALab has become in 2019 a structured unit within the agency, under its Executive Director, the assistance has graduated to making ADISALab 2.0 capable of providing innovation capacities to other institutional service providers in Albania, But such efforts are only at a starting point, and ongoing support is required.

Key Actions:

Key Action Responsible DueDate Status Comments Documents
Enable ongoing assistance for innovation in public service delivery in Albania
[Added: 2020/03/03]
UNDP CO Management DPM 2020/12 Overdue-Initiated
3. Recommendation:

The extent to which the reform has been implemented so far in Albania is based also on the valuable contribution from different donors support, such as the Italian and Austrian governments, United Nations, the European Union, and the World Bank. However, the donor support over the next years is still very crucial in order to consolidate even more the results achieved so far, to accelerate the processes and to respond on time to the needs particularly at  increasing the services also from other central institutions and expanding these services all over the country.

Conclusions, lessons and recommendations suggest that (due to their nature and complexity), some of the project outputs needs to be consolidated in the following years, such as ADISA’s business model and its geographic enlargement, government efforts in advancing and supporting its digital agenda as well as further assistance to the vulnerable groups such as Roma community and people with disabilities.

Management Response: [Added: 2020/03/03] [Last Updated: 2021/01/25]

Relevance of this recommendation is recognized. ISDA Support Project has been a valued contributor in the recognized progress of the public services reform in Albania. It has also played an important role in assuring that reform benefits reach women and the more vulnerable, such as inhabitants of remote areas, and the Roma and Egyptian communities. This becomes especially relevant considering the declared focus by the government on online public services. ‘Leave no one behind’ is, as such, embedded strongly in the design of proposed successor project, CSDA. Recognizing that at this stage the reform is increasingly facing the need to harmonize the pull towards greater efficiency, as well as effectiveness, in terms of public service access and quality, CSDA addresses the need for continuing development partner assistance, present from reform onset. As a harmonized effort with shared partner objectives as well as ongoing and planned interventions, this assistance is considered critical especially in helping to push forward the change agenda with the focus on women and the most vulnerable, build support, break resistance, raise capacity and promote sustainability

Key Actions:

Key Action Responsible DueDate Status Comments Documents
Continue working with development partners and in line with government priorities towards providing ongoing assistance for the consolidation of the public services reform in Albania
[Added: 2020/03/03]
UNDP CO Management 2020/04 Overdue-Initiated
4. Recommendation:

Based on the citizen-centric approach, it is recommended that additional focus should be paid to the citizens in remote areas. The ADISA Mobile Office Facility is an important initiative which should continue and expand in the future in all remote areas of the country. Based on the success of the co-location model, the Mobile Office Facility can make use of local capacities on the ground. 

Management Response: [Added: 2020/03/03] [Last Updated: 2021/01/25]

Relevance of recommendation to help promote solutions that are agile and effective and implemented in an integrated fashion is acknowledged. ISDA Support Project has been working with the reform leadership and champion institutions to ensure that identified challenges are analyzed in detail and addressed comprehensively. Hence, in terms of access to customer-care services throughout the territory, the project has supported an integrated approach that answers needs of different groups, rather than aggregations attributed to the average citizen. For the future, given the nature of traditional government, necessary bespoke solutions would require outside backing to be realized.

Key Actions:

Key Action Responsible DueDate Status Comments Documents
Incorporate assistance for experimenting, piloting and roll-out assistance for customized solutions that meet needs of citizens
[Added: 2020/03/03]
UNDP CO management 2020/12 Overdue-Not Initiated Ref: CSDA project document
5. Recommendation:

As stated in this report, the project has paid special attention to the vulnerable groups and to gender issues. In this regard, it is recommended that the future activities should be focused on ICT literacy for these groups, development of relevant training materials targeting different age groups (especially youth in remote areas). Such initiatives can be coordinated with the education institutions in the targeted areas.

Management Response: [Added: 2020/03/03] [Last Updated: 2021/01/25]

This is a valid recommendation. While the ISDA Support Project has focused on the digital agenda from the government perspective, it is essential to dedicate attention to supporting the development of digital skills in Albania, so that citizens can take advantage of the benefits of information technology, and are able to engage effectively in shaping how online services are implemented. ISDA Support successor project incorporates such an imperative, and also looks at a regional perspective to address this matter collectively within the Western Balkans.

Key Actions:

Key Action Responsible DueDate Status Comments Documents
Work with development partners to help develop and implement mechanisms to improve digital literacy in Albania
[Added: 2020/03/03]
UNDP CO Management 2020/12 Overdue-Not Initiated Begin with the start of the new project
6. Recommendation:

The success of the public Service Delivery reform is two-sided, depending both on the government’s and the citizen’s commitment. It is recommended that in the future, additional awareness raising campaigns should be focused on citizens, in order to increase their involvement as beneficiaries of the public services (i.e continue promotion of the Citizens Charter) and as monitoring agents of the public service’s quality.

Management Response: [Added: 2020/03/03] [Last Updated: 2021/01/25]

The relevance of this recommendation is recognized. Support for ongoing engagement and communications has been an important component of ISDA Support Project, which has increased progressively in weight. Considering their role, in the new project, such activities are designed as even more prominent.

Key Actions:

Key Action Responsible DueDate Status Comments Documents
Allocate needed resources in support of citizen engagement and communication in the assistance for the consolidation of the public services reform in Albania
[Added: 2020/03/03]
UNDP CO Management 2020/12 Overdue-Initiated

Latest Evaluations

Contact us

1 UN Plaza
DC1-20th Floor
New York, NY 10017
Tel. +1 646 781 4200
Fax. +1 646 781 4213
erc.support@undp.org