Global Libraries Bulgaria Programme Final Evaluation

Report Cover Image
Evaluation Plan:
2010-2013, Bulgaria
Evaluation Type:
Final Project
Planned End Date:
05/2013
Completion Date:
06/2013
Status:
Completed
Management Response:
Yes
Evaluation Budget(US $):
71,200

Share

Document Type Language Size Status Downloads
Download document TOR GLB Independent Exernal Evaluation (2).doc tor English 131.00 KB Posted 352
Download document GLB Independent Evaluation FINAL 20may13.docx report English 253.29 KB Posted 1051
Title Global Libraries Bulgaria Programme Final Evaluation
Atlas Project Number: 71115
Evaluation Plan: 2010-2013, Bulgaria
Evaluation Type: Final Project
Status: Completed
Completion Date: 06/2013
Planned End Date: 05/2013
Management Response: Yes
Focus Area:
  • 1. Democratic Governance
Corporate Outcome and Output (UNDP Strategic Plan 2014-2017)
  • 1. Civil society, including civil society organizations and voluntary associations, and the private sector contribute to the MDGs in support of national planning strategies and policies
  • 2. Access to information policies support accountability and transparency
Evaluation Budget(US $): 71,200
Source of Funding: Project Budget
Joint Programme: No
Mandatory Evaluation: No
Joint Evaluation: No
Evaluation Team members:
Name Title Email Nationality
Edmond Gaible PhD
Dotcho Mihailov PhD
GEF Evaluation: No
Key Stakeholders: Ministry of Culture, Ministry of Information Technology and Communications, Bulgarian Library and Information Association, National Association of Municipalities, 260 municipalities, 28 district libraries, 942 Chitalishte and municipal libraries
Countries: BULGARIA
Lessons
Findings
Recommendations
1 Increase focus of support and activities relevant to pensioners
2 Develop support for local-level training provided by librarians to different user groups (especially to pensioners) through provision of adequate training materials such as: ? An agenda and trainers guide; ? Participant resources to hand-out such as a glossary of terms; ? Screen animations of common tasks, and; ? Poster-templates for outreach to local groups ? Introduction to basic techniques for facilitating group training.
3 Improve education services for schoolchildren/adults (vocational education) through additional efforts in building partnership with MoEYS and other relevant agents in GoB for the development of technology-supported educational opportunities ( tele-collaborative projects; game-based learning; school-related group blogs; MoEYS-sponsored e-learning).
4 Support virtual and face-to-face networking among GLB librarians.
5 Refine approaches to e-services through defining more precisely understanding of e-services. A clear description of the expectations placed on librarians is needed with a focus on the idea that they should facilitate access to information, opportunities and existing services.
6 Enhance e-services through advocacy and building appropriate partnerships aimed at: - raising awareness of e-government among the general public and librarians - contributing to the development of municipal and other e-governance services that require lower levels of security, such as references to information, checking processing of statements and forms, checking tax status, etc.
7 Establish ?reciprocal? partnership agreements, negotiate symmetrical, effective, transparent and accountable partnerships that maintain and increase the value of the GLB network. Future partnership agreements should engage both partners in activities that are mutually beneficial and commensurate.
8 Improve support for e-services by raising GLB librarians? awareness of programs initiated by partnering institutions through preparation and distribution of fact sheets, conducting webinars and improving training.
9 Intensify advocacy for complete fulfillment of staffing commitment.
10 Initiate advocacy for full-time GLB district coordinator positions at the GLB program?s 39 training bases and seek to secure regional services sustainability through ensuring small project funding or long-term support by MoC and by national and EU funds.
11 Initiate advocacy for changes to co-financing and project partnerships with the aim to make the most appropriate programs open to proposals offered independently by local governments or Chitalishte when those proposals request support for activities using the GLB installation. Bearing in mind the ongoing preparation for the new EU programming period (2014-2020), systematic advocacy pressure on behalf of the Chitalishte and the GLB program can help GLB libraries benefit from EU funds to a greater extent.
12 Initiate advocacy for changes to the Complementary Subsidy from the MoC to the Chitalishte - adding a proposal category for technology-related or GLB-related activities, especially technology-supported arts and cultural activities. Thus MoC could create a channel for funding to the GLB libraries that could support sustainable participation in training, procurement of hardware (e.g., PIDs, replacement computers, etc.) and other relevant goods and services.
13 Increase the reliability of reporting on citizen training results through a revision of the quarterly reporting, as appropriate, to better reflect local-level activities.
14 Increase project-preparation capacity on local level through: ? additional training, if possible; work aids (guides / handbooks); incentives (training, networking, hardware, etc.) ? additional opportunities for project-based funding (including reissue of the call for local-content projects under the GSFIP) ? Encouragement of partnerships with CSOs
15 Increase decentralized training and networking activities on regional level through: ? Support for independent contracting of experienced trainers; pursuit of sector-specific training funds (e.g., rural development, e-inclusion, etc.); demand-based training, such as: community development, e-culture; ? Micro-grant opportunities for collaborating libraries and organization of interest-focused meetings.
16 Ensuring long-term sustainability through separation of financial and technical-assistance functions: ? ?return? of financial-management to MoC ? Technical Assistance Entity to fulfill specific functions?training, networking support, advocacy, support for project-based funding (NGOs/ private-sector service providers/ consulting companies). TAE should be selected based on competition among potential providers. Board support should not ensure this entity?s ?institutionalization? as a monopoly service provider for GLB libraries, nor should such membership restrict the entity?s operations only to GLB libraries.
1. Recommendation: Increase focus of support and activities relevant to pensioners
Management Response: [Added: 2013/08/01]

The project agrees with the recommendation. Within the planned activities in 2013 the PMU will focus on this specific user group and will create incentives for target libraries to involve pensioners more actively by providing services tailored to their specific needs.

Key Actions:

Key Action Responsible DueDate Status Comments Documents
Introduce target librarians to impact data for pensioners level of usage and current ICT practices (both from the Interim and the Final Impact Assessment studies); identify on potential niches for launching new activities/services focused on this specific user group (within the framework of planned GLB forums for local development ).
[Added: 2013/08/01] [Last Updated: 2013/12/30]
PMU 2013/12 Completed
Place special focus on lifelong learning activities for pensioners within the planned activities for development of e-education services in target libraries.
[Added: 2013/08/01] [Last Updated: 2013/12/30]
PMU 2013/12 Completed
A more strategic approach for promoting the inclusion of pensioners will be incorporated in the project proposal for a new grant to the BMGF.
[Added: 2013/08/01]
GLB Foundation 2014/12 Overdue-Not Initiated
The recommendation will be addressed through planned regular up-grades of the E-learning platform with relevant training resources.
[Added: 2013/08/01] [Last Updated: 2013/12/30]
PMU 2013/12 Completed
2. Recommendation: Develop support for local-level training provided by librarians to different user groups (especially to pensioners) through provision of adequate training materials such as: ? An agenda and trainers guide; ? Participant resources to hand-out such as a glossary of terms; ? Screen animations of common tasks, and; ? Poster-templates for outreach to local groups ? Introduction to basic techniques for facilitating group training.
Management Response: [Added: 2013/08/01]

The project agrees with the recommendation.

Key Actions:

Key Action Responsible DueDate Status Comments Documents
This recommendation will be incorporated as a separate activity under the training component of the project proposal for a new grant from BMGF.
[Added: 2013/08/01]
GLB Foundation 2014/12 Overdue-Not Initiated
3. Recommendation: Improve education services for schoolchildren/adults (vocational education) through additional efforts in building partnership with MoEYS and other relevant agents in GoB for the development of technology-supported educational opportunities ( tele-collaborative projects; game-based learning; school-related group blogs; MoEYS-sponsored e-learning).
Management Response: [Added: 2013/08/01]

The project agrees with the recommendation and will seek to address it through the planned elaboration of an e-education concept and the development of relevant e-education services to be offered by the target libraries.

Key Actions:

Key Action Responsible DueDate Status Comments Documents
Include a special focus on the improvement of educational services for schoolchildren/adults in the development of e-education services in target libraries. Research and advice by the e-education consultant on relevant programs of the Ministry of Education and Science (MoES) and opportunities for effective partnership between the target libraries and the schools.
[Added: 2013/08/01] [Last Updated: 2013/12/30]
PMU 2013/12 Completed
4. Recommendation: Support virtual and face-to-face networking among GLB librarians.
Management Response: [Added: 2013/08/01]

The project will continue to support GLB librarians networking both through the regional thematic meetings and through the continuous upgrade of the web portal, including through upgrading the quality of the existing communications tools.

Key Actions:

Key Action Responsible DueDate Status Comments Documents
Develop an online course on using social media for promoting library activities and for connecting with other librarians; offer the online course to target libraries through the GLB e-learning platform. Provide advice to librarians on how to best utilize their own (and their libraries?) presence in social media.
[Added: 2013/08/01] [Last Updated: 2013/12/30]
PMU 2014/12 Overdue-Initiated
Start a Best Practices wiki for librarians and promote it among libraries in the country, incl. through training on how to use it.
[Added: 2013/08/01]
GLB Foundation 2014/12 Overdue-Not Initiated
Start a group blog for librarians to share best practices and promote it among libraries in the country, incl. through training on how to use it.
[Added: 2013/08/01]
GLB Foundation 2014/12 Overdue-Not Initiated
Based on feedback from previous years, one of the most valuable elements of the regional meetings is the opportunity for the librarians to meet and exchange best practices. In 2013, the PMU has developed and introduced an enriched concept for local meetings, including two types of meetings for each district ? one focused on the exchange of best practices and experience sharing among librarians and the other aimed at presenting the role and demonstrating the impact of libraries on their local communities in front of local authorities and stakeholders.
[Added: 2013/08/01] [Last Updated: 2013/12/30]
PMU 2013/12 Completed
5. Recommendation: Refine approaches to e-services through defining more precisely understanding of e-services. A clear description of the expectations placed on librarians is needed with a focus on the idea that they should facilitate access to information, opportunities and existing services.
Management Response: [Added: 2013/08/01]

The role of the library and the librarian in the area has always been understood as identifying and facilitating the access of the users to online information and existing e-services. This understanding has been shared with and by the librarians. The actual access to e-services and online information is directly linked to the stage of development and the quality of the existing e-services and online content.

Key Actions:

Key Action Responsible DueDate Status Comments Documents
Continue the ongoing dialogue with the librarians on their role in identifying and facilitating access to existing e-services and online information, through various communication tools - GLB web portal, direct mailing, local meetings, etc.
[Added: 2013/08/01] [Last Updated: 2013/12/30]
PMU 2013/12 Completed
Encourage librarians to partner with the local structures of the partner institutions (Ministry of Health, Employment Agency, etc.) in identifying and facilitating the access new e-services and to existing online information.
[Added: 2013/08/01]
PMU, GLB Foundation 2014/12 Overdue-Initiated
6. Recommendation: Enhance e-services through advocacy and building appropriate partnerships aimed at: - raising awareness of e-government among the general public and librarians - contributing to the development of municipal and other e-governance services that require lower levels of security, such as references to information, checking processing of statements and forms, checking tax status, etc.
Management Response: [Added: 2013/08/01]

The project agrees with the recommendation and will continue promote e-government services that are available and offered by the governmental institutions. The GLB is currently supporting an information campaign for personal identification code (PIC) of the National Revenue Agency. A number of e-services could be accessed through the use of PIC by the citizens in the libraries.

Key Actions:

Key Action Responsible DueDate Status Comments Documents
Information campaign on e-government services.
[Added: 2013/08/01]
PMU, GLB Foundation No due date No deadline established
Training of librarians how to promote and offer e-government through libraries.
[Added: 2013/08/01]
PMU, GLB Foundation No due date No deadline established
7. Recommendation: Establish ?reciprocal? partnership agreements, negotiate symmetrical, effective, transparent and accountable partnerships that maintain and increase the value of the GLB network. Future partnership agreements should engage both partners in activities that are mutually beneficial and commensurate.
Management Response: [Added: 2013/08/01]

The PMU agrees with this recommendation. As a first step, the main focus of GLB has been to identify and establish partnerships with national institutions that raise the awareness and demonstrate the capacity and the potential of the libraries. The next logical step would be, building on the experience and on the recognition of libraries as valuable partners, to identify and formulate institutional partnership agreements that would increase the value and will guarantee a direct return on the library.

Key Actions:

Key Action Responsible DueDate Status Comments Documents
The continuation of the existing partnerships, their upgrade as well as the establishment of new ?reciprocal? partnerships based on symmetry, effectiveness, transparency and accountability will be one of the key priorities of the new GLBG legacy entity.
[Added: 2013/08/01]
GLB Foundation 2014/12 Overdue-Not Initiated
8. Recommendation: Improve support for e-services by raising GLB librarians? awareness of programs initiated by partnering institutions through preparation and distribution of fact sheets, conducting webinars and improving training.
Management Response: [Added: 2013/08/01]

The project will continue to raise GLB librarians? awareness of partnering institutions? initiatives and stimulating them to participate actively in such activities. The project will continue to use all its networking resources such as posting information on the web portal, informing librarians for new opportunities via e-mail, direct contacts with the regional managers or through the provision of specialized e-services training.

Key Actions:

Key Action Responsible DueDate Status Comments Documents
The project will continue to utilize all available resources ? forums for local development, web portal, e-mailing, webinars and specialized e-services training in order to constantly upgrade librarians? awareness of partnering institutions? initiatives and stimulating them to participate actively in such activities.
[Added: 2013/08/01] [Last Updated: 2013/12/30]
PMU 2013/12 Completed
9. Recommendation: Intensify advocacy for complete fulfillment of staffing commitment.
Management Response: [Added: 2013/08/01]

The project agrees with the recommendation.

Key Actions:

Key Action Responsible DueDate Status Comments Documents
Within the planned 2013 activities all GLB partners will seek to draw the attention of the new government on the necessity and importance of fulfilling the staffing commitment. These efforts will be continued by the new GLB legacy entity.
[Added: 2013/08/01] [Last Updated: 2013/12/30]
GLB Programme Board 2013/12 Completed
10. Recommendation: Initiate advocacy for full-time GLB district coordinator positions at the GLB program?s 39 training bases and seek to secure regional services sustainability through ensuring small project funding or long-term support by MoC and by national and EU funds.
Management Response: [Added: 2013/08/01]

The PMU agrees with the recommendation and will seek opportunities for ensuring long-term sustainability on regional level through measures related to the GLB district coordinator positions.

Key Actions:

Key Action Responsible DueDate Status Comments Documents
This recommendation will be addressed by the GLB legacy entity, which will explore opportunities to sustain the existing network of district and to ensure sustainability of the GLB 39 training bases, including through securing funding for libraries through national and EU funds.
[Added: 2013/08/01]
GLB Foundation 2014/12 Overdue-Not Initiated
11. Recommendation: Initiate advocacy for changes to co-financing and project partnerships with the aim to make the most appropriate programs open to proposals offered independently by local governments or Chitalishte when those proposals request support for activities using the GLB installation. Bearing in mind the ongoing preparation for the new EU programming period (2014-2020), systematic advocacy pressure on behalf of the Chitalishte and the GLB program can help GLB libraries benefit from EU funds to a greater extent.
Management Response: [Added: 2013/08/01]

The project agrees with this recommendation and will incorporate it as one of the key priorities of the new GLBG legacy entity.

Key Actions:

Key Action Responsible DueDate Status Comments Documents
Advocacy will be a key priority for the GLBG legacy entity, which will seek to support the GLB libraries? network and expand GLB libraries? opportunities to benefit from EU funds.
[Added: 2013/08/01]
GLB Foundation No due date No deadline established
12. Recommendation: Initiate advocacy for changes to the Complementary Subsidy from the MoC to the Chitalishte - adding a proposal category for technology-related or GLB-related activities, especially technology-supported arts and cultural activities. Thus MoC could create a channel for funding to the GLB libraries that could support sustainable participation in training, procurement of hardware (e.g., PIDs, replacement computers, etc.) and other relevant goods and services.
Management Response: [Added: 2013/08/01]

The project agrees with the recommendation and will advocate for such changes to be agreed upon by the relevant institutions.

Key Actions:

Key Action Responsible DueDate Status Comments Documents
The MC as GLB lead partner will explore possible options for granting a percentage of the complementary subsidy for such technology-oriented activities.
[Added: 2013/08/01]
MC 2014/12 Overdue-Not Initiated
Advocacy activities in the same direction will be initiated also by the GLBG legacy entity as the complementary subsidy is an important channel for funding to the GLB libraries that could ensure the sustainability of the GLB network in the post-grant period.
[Added: 2013/08/01]
GLB Foundation No due date No deadline established
13. Recommendation: Increase the reliability of reporting on citizen training results through a revision of the quarterly reporting, as appropriate, to better reflect local-level activities.
Management Response: [Added: 2013/08/01]

Reporting on citizen training results has always followed the BMGF requirements. The PMU will initiate revision of the reporting instructions related to citizen training in order to better clarify them to librarians and stimulate them to report data in line with the updated BMGF Performance metrics.

Key Actions:

Key Action Responsible DueDate Status Comments Documents
Quarterly reporting instruction revision in line with the updated BMGF Performance metrics definitions on citizen training and consultation (officially announced since April, 2013).
[Added: 2013/08/01]
PMU 2013/06 Completed
14. Recommendation: Increase project-preparation capacity on local level through: ? additional training, if possible; work aids (guides / handbooks); incentives (training, networking, hardware, etc.) ? additional opportunities for project-based funding (including reissue of the call for local-content projects under the GSFIP) ? Encouragement of partnerships with CSOs
Management Response: [Added: 2013/08/01]

The project agrees with the recommendation and will address it within the training component activities planned for 2013, which envisage the utilization of various capacity building tools. These efforts will be continued by the new GLBG legacy entity.

Key Actions:

Key Action Responsible DueDate Status Comments Documents
The local thematic meetings and forums for local development will be used as tools for promoting partnership between libraries, local authorities and other local actors. When applicable, special focus will be placed on funding opportunities.
[Added: 2013/08/01] [Last Updated: 2013/12/30]
PMU 2013/12 Completed
The Young Librarians Workshop planned for September 2013 will focus on this topic as well. Continuation of this initiative is foreseen.
[Added: 2013/08/01] [Last Updated: 2013/12/30]
PMU 2013/09 Completed
New materials will be published on the e-learning platform.
[Added: 2013/08/01] [Last Updated: 2013/12/30]
PMU 2013/12 Completed
The implementation of the Micro-Grant Scheme for funding small-scale projects of target libraries will help librarians to develop further their to upgrade librarians? project preparation and project implementation capacity by providing an opportunity for ?learning by doing? and gaining practical experience.
[Added: 2013/08/01] [Last Updated: 2013/12/30]
PMU 2013/12 Completed
15. Recommendation: Increase decentralized training and networking activities on regional level through: ? Support for independent contracting of experienced trainers; pursuit of sector-specific training funds (e.g., rural development, e-inclusion, etc.); demand-based training, such as: community development, e-culture; ? Micro-grant opportunities for collaborating libraries and organization of interest-focused meetings.
Management Response: [Added: 2013/08/01]

The project agrees with the recommendation.

Key Actions:

Key Action Responsible DueDate Status Comments Documents
Through the implementation of the Micro-grant scheme for funding small-scale projects of target libraries the project will support in practice library projects that envisage such activities - hiring of experienced trainers, needs assessment activities in order to prepare demand-based training for the community, etc.
[Added: 2013/08/01] [Last Updated: 2013/12/30]
PMU 2013/12 Completed
16. Recommendation: Ensuring long-term sustainability through separation of financial and technical-assistance functions: ? ?return? of financial-management to MoC ? Technical Assistance Entity to fulfill specific functions?training, networking support, advocacy, support for project-based funding (NGOs/ private-sector service providers/ consulting companies). TAE should be selected based on competition among potential providers. Board support should not ensure this entity?s ?institutionalization? as a monopoly service provider for GLB libraries, nor should such membership restrict the entity?s operations only to GLB libraries.
Management Response: [Added: 2013/08/01]

The Program Board of the project has already decided on establishing a new non-governmental organization as a GLBG legacy entity to ensure the long-term sustainability of the project?s results. The project partners will take into consideration the risks, identified by the evaluation team highlighted when formulating the GLB sustainability strategy. As to the recommended separation of financial and technical-assistance functions, the financial management of the library sector has always been within the competencies of MC. In this situation the new entity is expected to support MC and to help sustain and strengthen the vitality of the GLB network through the provision of technical assistance, through advocacy and through fund raising for the library network from other financial sources.

Key Actions:

Key Action Responsible DueDate Status Comments Documents
Establish GLB Foundation.
[Added: 2013/08/01] [Last Updated: 2013/12/30]
GLB Programme Board 2013/09 Completed

Latest Evaluations

Contact us

220 East 42nd Street
20th Floor
New York, NY 10017
Tel. +1 646 781 4200
Fax. +1 646 781 4213
erc.support@undp.org