Final Evaluation of Access to Information programme II

Report Cover Image
Evaluation Plan:
2017-2020, Bangladesh
Evaluation Type:
Final Project
Planned End Date:
09/2019
Completion Date:
09/2019
Status:
Completed
Management Response:
Yes
Evaluation Budget(US $):
50,000

Share

Document Type Language Size Status Downloads
Download document [Final] TOR_A2i Phase II_Final Evaluation.pdf tor English 250.12 KB Posted 64
Download document [Final] Final Evaluation Report_A2i-II.pdf report English 2311.23 KB Posted 341
Title Final Evaluation of Access to Information programme II
Atlas Project Number: 00045483
Evaluation Plan: 2017-2020, Bangladesh
Evaluation Type: Final Project
Status: Completed
Completion Date: 09/2019
Planned End Date: 09/2019
Management Response: Yes
UNDP Signature Solution:
  • 1. Poverty
Corporate Outcome and Output (UNDP Strategic Plan 2018-2021)
  • 1. Output 1.1.1 Capacities developed across the whole of government to integrate the 2030 Agenda, the Paris Agreement and other international agreements in development plans and budgets, and to analyse progress towards the SDGs, using innovative and data-driven solutions
  • 2. Output 1.1.2 Marginalised groups, particularly the poor, women, people with disabilities and displaced are empowered to gain universal access to basic services and financial and non-financial assets to build productive capacities and benefit from sustainable livelihoods and jobs
  • 3. Output 2.1.2 Capacities developed for progressive expansion of inclusive social protection systems
  • 4. Output 2.2.1 Use of digital technologies and big data enabled for improved public services and other government functions
SDG Goal
  • Goal 1. End poverty in all its forms everywhere
  • Goal 4. Ensure inclusive and equitable quality education and promote lifelong learning opportunities for all
  • Goal 8. Promote sustained, inclusive and sustainable economic growth, full and productive employment and decent work for all
SDG Target
  • 1.3 Implement nationally appropriate social protection systems and measures for all, including floors, and by 2030 achieve substantial coverage of the poor and the vulnerable
  • 4.4 By 2030, substantially increase the number of youth and adults who have relevant skills, including technical and vocational skills, for employment, decent jobs and entrepreneurship
  • 8.10 Strengthen the capacity of domestic financial institutions to encourage and expand access to banking, insurance and financial services for all
Evaluation Budget(US $): 50,000
Source of Funding: Project budget
Evaluation Expenditure(US $): 28,672
Joint Programme: No
Joint Evaluation: No
Evaluation Team members:
Name Title Email Nationality
Naimur Rahman International Consultant
Salahuddin A Aminuazzaman Senior National Consultant
Nazneem Ahmed National Consultant
GEF Evaluation: No
Key Stakeholders:
Countries: BANGLADESH
Lessons
1.

Broadly speaking a2i, in effect, is not only a digitalization but could also be seen as a package of pro-people, pro-poor public administration reform programmes.

It appears that a2i is already in a process of institutionalization. The Government of Bangladesh has by now designated a senior official of the ministries as Chief Innovation Officer/ Innovation Officer, as the "reform champions" to guide and support various innovations and take active initiative towards institutionalization of such innovation culture in the ministries and departments/agencies.

The project has managed to develop and maintain a partnership and cooperation arrangement with media for the dissemination of e-services and create awareness and the demand from below.

a2i has adopted an inclusive, bottom-up approach for the promotion of innovation culture in public service delivery.


Findings
1.

The National Portal is currently serving more than 1.5million users where they are enjoying e-services with 90 million hits per month (KOICA 2018). Different evaluations observed that the Portal had reduced the time and costs of the citizens in accessing government services. However, the National Portal is still more information- centric than service-oriented.


2.

The e-Nothi has expedited the delivery of public services.and has significantly reduced the time and costs and also enhanced citizens’ trust in the service delivery system of the government. However, the active users of e-nohti are still very low and it demands further expansion.


3.

e-Mutation has made the land record management efficient, hassle-free and transparent. But However, sustainability and replication of e-Mutation is still a challenge. The functional/institutional ownership needs to be transferred to the Ministry of Land and Land Reform Board immediately.


4.

Union Digitial Centres (UDCs) Evidences from the field suggest that the Union Digital Centres (UDCs) have significantly transformed the lives of the people of the underserved areas. The UDCs are providing as many as 116 different types of services and have reduced TCV significantly. It could be recognized as a dependable "service delivery hub" for the un- served and under-served population. Despite being its functional importance, the UDC’s financial sustainability appears to be a challenge.


5.

G2P Payment is a remarkable move to reach the poorest of the poor who are the beneficiaries of the social safety net programs. The G2P has transformed the manual, paper-based system for transferring the payments of Social Safety Net Programs through a much user-friendly agent banking approach based at the UDCs. Furthermore, as an inclusive approach, G2P has created an entry point for the SSNP program beneficiaries to get involved in the formal banking system. Most importantly, G2P has sealed the scope of any corruption /leakage of the SSNP services.


6.

Multi-Media Classroom (MMC) is an interesting innovation to address the digital and education divide. The MMCs has helped and enabled the teachers to use of ICT tools for class room teaching. Around 9.5 million students have been benefitted by the MMC. However, monitoring the effective functioning of the MMCs and collection of real time data on the use of MMCs are inadequate.


7.

Empathy Training is a unique approach to develop a responsive and proactive service to the citizens. However, there is hardly any evidence-based research on the overall impact of the empathy training and the innovation being practiced.


8.

Service Innovation Fund (SIF) has contributed significantly to the innovation eco-system. Out of 53 projects of SIF, 17 projects have been scaled up. It is estimated that cumulatively about 0.9 million people have been benefited from the SIF proto-types till June 2018. Like some other innovations, under a2i, there is also an absence of evidence-based research on the overall impact and implications of the SIF generated innovation.


9.

Changing values and attitudes of public officials. The a2i’s efforts to capacity development of civil servants have significantly changed the values, attitudes and responsiveness of the civil servants, which in effect have contributed significantly to the building of an innovation culture in all tiers of government.


10.

a2i has leveraged its success through partnerships: a2I has established a number of local and global partnerships across public, private and development organizations for achieving its multiple goals and objectives. By June 2018, a2i has made active partnerships with 96 organizations of which 83 are national and 13 are international. A2i has also established partnership with academia and research organizations for drawing new innovation, development research and process analysis.


11.

South-South and Triangular Cooperation: a2i has successfully contributed towards building the South-South Cooperation platform to facilitate the exchange of ideas, knowledge and experiences on innovation in public service delivery among the countries of the global South. Meanwhile, Bangladesh government has already set up a ‘South-South Cooperation Cell" at ERD.


12.

a2i has made impact beyond the national boundaries: a2i has also disseminated its innovations/ products to countries like Maldives, Bhutan, Nepal, Somalia, Fiji. Furthermore,a2i has contributed to Bangladesh’s advancement in global e-government development ranking (115th position in 2018 compared to the 150th position in 2012). Besides, a2i has earned international recognition by receiving a good number of national and international awards on use of ICT in public service delivery.


13.

a2i emerged as an influential policy advocacy entity: a2i has actively provided technical assistance for the designing/ drafting the policies/ regulations/ guidelines related to ICT and digital management. According to a2i inventory, the programme has supported directly/indirectly in the formulation /revision of 39 policies and laws (some of the noteworthy policies include - e.g., Digital Bangladesh Vision 2021, the Seventh Five-Year Plan (2016- 2020) and the National ICT Policy 2015. However, so far a2i has not been able to play an active policy advocacy role in broader issues related to development management and e- governance through its in-house research and communication and dissemination tools.


Recommendations
1

The a2i project has managed to achieve remarkable results over past years across many domains and strategic areas, even beyond their core mandate of ensuring easy access to simplified and digitised public services. While such diversification of efforts is commendable, such spreading out of activities portfolio has impeded in certain ways the managerial oversight that is important for ensuring the quality of project interventions and outcome. This also got reflected in the efforts of a2i team in collating relevant follow-up information that the Evaluation team for carrying out their mandate. It would be helpful for the a2i project team to focus on consolidating the gains towards institutionalisation and mainstreaming of successful high- impact initiatives (such as e-Nothi, e-Mutation and P2G payment system).

2

This should be complemented with nuanced efforts of capacitating a2i partners at all levels: Ministries / Divisions, Districts and sub-District levels, through planned effusion of organisational competences and good practices of a2i to its partners – so that the initiatives ideated and spearheaded by the project could eventually be transferred for mainstreaming by respective Ministries and Divisions. The Evaluation team would also recommend conducting a methodologically robust study to better understand the scale of capacity development requirements of mid-level civil servants and lower bureaucracy over the next five year. The project should also explore the feasibility of building on the accomplishments of change-agents at all levels of government, encouraging them to become the ambassadors of change.

3

The End-line Evaluation did not manage to gather sufficient evidence with regard to programme level emphasis of a2i on gender-specific targets; and there is an urgent need of taking corrective and affirmative measures for achieving different goals of a2i’s gender strategy. To begin with, the gender strategy needs internalization and appreciation among a2i staff. The curriculum of training package on gender should be reviewed and new issues related to a2i projects should be included in the orientation and training package. Other three critical areas of interventions could be

  • Integration of gender considerations in all programmatic interventions of a2i. The Gender Task Force, which was created in 2015 should be made functional. The project should also consider developing gender specific compliance l markers for decisions on any new initiatives under a2i. A framework should be developed for gathering gender disaggregated data; and this should be used by all project interventions under a2i. Affirmative support for scaling up of gender sensitive innovation may also be considered.

  • Promotion of gender-sensitive partnerships for women’s economic empowerment through entrepreneurship development in UDCs. Initiatives like Ek-Shop and Agent Banking should be made more accessible to women with affirmative bias toward women’s participation.

  • a2i should consider augmenting its Facilitate development of gender-responsive government institutions.

4

The Evaluation team would strongly recommend employing the expertise of an Organizational Development Specialist to appraise the present ways-of-working of a2i team and present plausible scenarios on next-generation operating model for a2i; and on the basis of these inputs propose appropriate structuring of a more efficient implementation support mechanism for the new Aspire to Innovate (a2i) programme. The OD expert could build on the insights derived from the portfolio sense-making session conducted by UNDP with the a2i team, which recognized the need of articulating core capabilities of a2i in terms of process rather than project-oriented outcomes. The OD appraisal should also include an enterprise wide informational audit, to be followed by appropriate corrective measures to ensure that the programme level data-system of a2i is more robust with strong analytical insight and capabilities.

5

Organisational, operational management and regulatory strengthening of the functioning of Union Digital Centres to become the unified channel for delivering digital service (and products) of government as well as private sector at the grassroots. Studies carried out by Transparency International Bangladesh and Institute of Business Administration, University of Dhaka have a number of relevant suggestions in this regard. The project might consider constituting a task-force / working group comprising of members from both the study teams to holistically look at,

  • the business model under the 4P (Public-Private-People Partnership) principle for financial viability of Digital Centres as well as to achieve the goals of societal wellbeing and value of money;

  • appropriate legislative framework and policy guidelines for UDCs to function as one-stop-shop for electronic delivery of all the important public services.

6

a2i has initiated a number of public service innovations and also contributed towards formulating a number of policies, rules and legislative mandates; but the process documentations of such innovations are sketchy and not properly recorded. Going forward the project should definitely lay emphasis on documenting its processes of sparking / supporting innovations and policy reform efforts, as well as its subtle role of advocating within the government. Simultaneously, the a2i team should consider communicating externally – sharing their ideas and learning near real time through blog posts and other social media posts; and undertaking policy oriented longitudinal research that builds upon its rich repository of knowledge on public sector performance and innovation in Bangladesh.

1. Recommendation:

The a2i project has managed to achieve remarkable results over past years across many domains and strategic areas, even beyond their core mandate of ensuring easy access to simplified and digitised public services. While such diversification of efforts is commendable, such spreading out of activities portfolio has impeded in certain ways the managerial oversight that is important for ensuring the quality of project interventions and outcome. This also got reflected in the efforts of a2i team in collating relevant follow-up information that the Evaluation team for carrying out their mandate. It would be helpful for the a2i project team to focus on consolidating the gains towards institutionalisation and mainstreaming of successful high- impact initiatives (such as e-Nothi, e-Mutation and P2G payment system).

Management Response: [Added: 2019/09/29]

Agreed. This recommendation has already been taken into account during the designing of the new project. Component 1 of the new project focuses on this recommendation and implementation of focusing towards institutionalization and mainstreaming of high-impact initiatives.

Key Actions:

Key Action Responsible DueDate Status Comments Documents
Initiate the discussion on the given recommendation during the first project steering committee meeting
[Added: 2019/09/29] [Last Updated: 2020/06/26]
A2i 2020/12 Initiated History
Introduce a clear support structure at organizational level for institutionalization and mainstreaming
[Added: 2019/09/29] [Last Updated: 2020/06/26]
a2i, ICT Division, Cabinet Division 2022/06 Initiated History
Initiate the formation and implementation of a2i-Bangladesh Innovation Trust for institutionalization and mainstreaming of overall impactful initiatives of a2i
[Added: 2020/06/26]
a2i, ICT Division, Cabinet Division 2021/06 Initiated The Trust Bill is now at Cabinet Division for review. Later, it will be sent to the Parliament to enact an act for approval of the Trust.
2. Recommendation:

This should be complemented with nuanced efforts of capacitating a2i partners at all levels: Ministries / Divisions, Districts and sub-District levels, through planned effusion of organisational competences and good practices of a2i to its partners – so that the initiatives ideated and spearheaded by the project could eventually be transferred for mainstreaming by respective Ministries and Divisions. The Evaluation team would also recommend conducting a methodologically robust study to better understand the scale of capacity development requirements of mid-level civil servants and lower bureaucracy over the next five year. The project should also explore the feasibility of building on the accomplishments of change-agents at all levels of government, encouraging them to become the ambassadors of change.

Management Response: [Added: 2019/09/29]

Agreed. UNDP has already taken steps to infuse a2i’s organizational competences and good practices across the ministries and institution during the new project cycle.

Key Actions:

Key Action Responsible DueDate Status Comments Documents
Finalize designing the implementation initiative that will focus on capacity development of civil servants that goes beyond public service innovation
[Added: 2019/09/29] [Last Updated: 2020/06/26]
a2i, ICT Divsion 2020/10 Initiated History
Start implementing the new initiative across the ministries
[Added: 2019/09/29] [Last Updated: 2020/06/26]
a2i, Cabinet Division 2020/12 Initiated History
Hire a research institute/academic institute/consulting firm which will conduct a relevant study
[Added: 2019/09/29] [Last Updated: 2020/06/26]
A2i 2020/08 Not Initiated The process of hiring a research institute/academic institute/consulting firm is in progress. History
Share the findings with respected ministries & department
[Added: 2019/09/29] [Last Updated: 2020/06/26]
A2i 2021/01 Not Initiated History
3. Recommendation:

The End-line Evaluation did not manage to gather sufficient evidence with regard to programme level emphasis of a2i on gender-specific targets; and there is an urgent need of taking corrective and affirmative measures for achieving different goals of a2i’s gender strategy. To begin with, the gender strategy needs internalization and appreciation among a2i staff. The curriculum of training package on gender should be reviewed and new issues related to a2i projects should be included in the orientation and training package. Other three critical areas of interventions could be

  • Integration of gender considerations in all programmatic interventions of a2i. The Gender Task Force, which was created in 2015 should be made functional. The project should also consider developing gender specific compliance l markers for decisions on any new initiatives under a2i. A framework should be developed for gathering gender disaggregated data; and this should be used by all project interventions under a2i. Affirmative support for scaling up of gender sensitive innovation may also be considered.

  • Promotion of gender-sensitive partnerships for women’s economic empowerment through entrepreneurship development in UDCs. Initiatives like Ek-Shop and Agent Banking should be made more accessible to women with affirmative bias toward women’s participation.

  • a2i should consider augmenting its Facilitate development of gender-responsive government institutions.

Management Response: [Added: 2019/09/29]

Agreed. UNDP will take measures to achieve the different goals of a2i gender strategy. The project will consider gender issues in designing new programmatic interventions. Moreover, the following actions will be taken to address the recommendation.

Key Actions:

Key Action Responsible DueDate Status Comments Documents
Hire a gender specialist (consultant) who will provide strategic guideline and monitor a2i’s gender-specific targets and activities
[Added: 2019/09/29] [Last Updated: 2020/06/26]
A2i 2020/12 Initiated Hiring a gender specialist is ongoing. History
Institutionalize a Gender Tasks force which will mitigate gender-related issues within a2i
[Added: 2019/09/29] [Last Updated: 2020/06/26]
A2i 2020/12 Initiated A Gender Tasks force already exists. With the process of hiring a gender specialist ongoing, the institutionalization work will accelerate further once the gender specialist is on board. History
Ensure gender-specific compliance is established in all current and new initiatives
[Added: 2019/09/29] [Last Updated: 2020/06/26]
A2i 2021/06 Initiated Gender Action Plan has already been on board and gender-specific compliances are being ensured with the help of a2i Dashboard, the data monitoring dashboard of a2i. History
Continue to ensure gender-disaggregated data are collected for all a2i initiatives
[Added: 2019/09/29] [Last Updated: 2020/06/26]
A2i 2024/06 Initiated Project has been doing this and will continue to do so for the entire project life cycle. Gender-disaggregated data are collected and incorporated in a specific gender action plan, which have already been submitted to UNDP Bangladesh in order to take corrective and affirmative measures for achieving different goals of a2i’s gender strategy and resolve gender-related issues. History
4. Recommendation:

The Evaluation team would strongly recommend employing the expertise of an Organizational Development Specialist to appraise the present ways-of-working of a2i team and present plausible scenarios on next-generation operating model for a2i; and on the basis of these inputs propose appropriate structuring of a more efficient implementation support mechanism for the new Aspire to Innovate (a2i) programme. The OD expert could build on the insights derived from the portfolio sense-making session conducted by UNDP with the a2i team, which recognized the need of articulating core capabilities of a2i in terms of process rather than project-oriented outcomes. The OD appraisal should also include an enterprise wide informational audit, to be followed by appropriate corrective measures to ensure that the programme level data-system of a2i is more robust with strong analytical insight and capabilities.

Management Response: [Added: 2019/09/29] [Last Updated: 2019/09/29]

Agreed. a2i has already hired an OD expert, who will propose a skills matrix needed within the organizational structure to effectively implement the project during its next stage.

Key Actions:

Key Action Responsible DueDate Status Comments Documents
Hire an international consultant of OD for a2i to develop an organizational structure and skills matrix
[Added: 2019/09/29]
A2i 2019/07 Completed Dr. Subash Masters came on board as a consultant to oversee this task.
Create an implementation plan based on the OD report
[Added: 2019/09/29] [Last Updated: 2020/06/26]
A2i 2020/08 Initiated A draft OD implementation plan has been developed. Finalization of the report is on process. It is expected to be finalized by August 2020. History
5. Recommendation:

Organisational, operational management and regulatory strengthening of the functioning of Union Digital Centres to become the unified channel for delivering digital service (and products) of government as well as private sector at the grassroots. Studies carried out by Transparency International Bangladesh and Institute of Business Administration, University of Dhaka have a number of relevant suggestions in this regard. The project might consider constituting a task-force / working group comprising of members from both the study teams to holistically look at,

  • the business model under the 4P (Public-Private-People Partnership) principle for financial viability of Digital Centres as well as to achieve the goals of societal wellbeing and value of money;

  • appropriate legislative framework and policy guidelines for UDCs to function as one-stop-shop for electronic delivery of all the important public services.

Management Response: [Added: 2019/09/29]

Agreed. UNDP will be taking following initiatives to explore the findings from the researches more in details.

Key Actions:

Key Action Responsible DueDate Status Comments Documents
Host a joint session with Institute of Business Administration (IBA), University of Dhaka and Transparency International Bangladesh (TIB) to discuss this recommendation
[Added: 2019/09/29] [Last Updated: 2020/06/26]
A2i 2020/08 Not Initiated History
Discuss an appropriate legislative framework/ policy guideline on UDCs in the Board Meeting (PSC) of a new project
[Added: 2019/09/29] [Last Updated: 2020/06/26]
A2i 2020/09 Not Initiated History
Create an action plan for possible next steps if necessary, based on the session
[Added: 2019/09/29]
A2i 2020/12 Not Initiated
6. Recommendation:

a2i has initiated a number of public service innovations and also contributed towards formulating a number of policies, rules and legislative mandates; but the process documentations of such innovations are sketchy and not properly recorded. Going forward the project should definitely lay emphasis on documenting its processes of sparking / supporting innovations and policy reform efforts, as well as its subtle role of advocating within the government. Simultaneously, the a2i team should consider communicating externally – sharing their ideas and learning near real time through blog posts and other social media posts; and undertaking policy oriented longitudinal research that builds upon its rich repository of knowledge on public sector performance and innovation in Bangladesh.

Management Response: [Added: 2019/09/29]

Agreed. The new project will be looking into establishing a strategic learning unit, which will oversee the knowledge management and knowledge dissemination of the learning inside and outside a2i.

Key Actions:

Key Action Responsible DueDate Status Comments Documents
Create a process document outline that will clearly highlight and record programmatic and policy-level changes taking place as a result of a2i intervention
[Added: 2019/09/29] [Last Updated: 2020/06/26]
A2i 2020/12 Not Initiated History
Appoint one process documentation focal in each team of A2i, who will coordinate with the A2i’s Project Management and Data team to ensure proper documentation process
[Added: 2019/09/29] [Last Updated: 2020/06/26]
A2i 2020/08 Initiated a2i has developed a robust data archiving and knowledge management repository, named a2i Dashboard. Each team of a2i has been coordinating with the Project Management and Data team and appointed a specific focal point to ensure proper documentation and knowledge management with the help of a2i Monitoring Dashboard. History
Capacitate all a2i teams on knowledge management and process documentation
[Added: 2019/09/29] [Last Updated: 2020/06/26]
A2i 2020/12 Initiated A hands-on training based on the data update at a2i Dashboard has been provided to the Data focal for data & documentation of each team to capacitate them for proper data collection, data input and process documentation. History

Latest Evaluations

Contact us

1 UN Plaza
DC1-20th Floor
New York, NY 10017
Tel. +1 646 781 4200
Fax. +1 646 781 4213
erc.support@undp.org