Green Urban Lighting Mid Term Evaluation

Report Cover Image
Evaluation Plan:
2016-2020, Armenia
Evaluation Type:
Mid Term Project
Planned End Date:
12/2016
Completion Date:
07/2016
Status:
Completed
Management Response:
Yes
Evaluation Budget(US $):
10,595

Share

Document Type Language Size Status Downloads
Download document Project - Lighting - Mid-term evaluation - 2016.pdf report English 1989.52 KB Posted 480
Download document UNDP-GEF-MTR-TOR-Lighting-2016_v1-final.docx tor English 97.58 KB Posted 210
Download document Lighting - MTE - Management Response.docx related-document English 30.73 KB Posted 81
Title Green Urban Lighting Mid Term Evaluation
Atlas Project Number: 00074869
Evaluation Plan: 2016-2020, Armenia
Evaluation Type: Mid Term Project
Status: Completed
Completion Date: 07/2016
Planned End Date: 12/2016
Management Response: Yes
Focus Area:
  • 1. Environment & Sustainable Development
  • 2. Others
Corporate Outcome and Output (UNDP Strategic Plan 2014-2017)
  • 1. Output 1.5. Inclusive and sustainable solutions adopted to achieve increased energy efficiency and universal modern energy access (especially off-grid sources of renewable energy)
SDG Goal
  • Goal 13. Take urgent action to combat climate change and its impacts
  • Goal 7. Ensure access to affordable, reliable, sustainable and modern energy for all
SDG Target
  • 13.2 Integrate climate change measures into national policies, strategies and planning
  • 7.a By 2030, enhance international cooperation to facilitate access to clean energy research and technology, including renewable energy, energy efficiency and advanced and cleaner fossil-fuel technology, and promote investment in energy infrastructure and clean energy technology
Evaluation Budget(US $): 10,595
Source of Funding: UNDP
Joint Programme: No
Joint Evaluation: No
Evaluation Team members:
Name Title Email Nationality
GEF Evaluation: Yes
GEF Project Title: Green Urban Lighting Mid-term Review
Evaluation Type: Mid-term Review
Focal Area: Climate Change
Project Type: FSP
GEF Phase: GEF-5
GEF Project ID: 74869
PIMS Number: 4669
Key Stakeholders:
Countries: ARMENIA
Lessons
Findings
Recommendations
1

 Legislation framework is improving, but focus is needed to achieve adoptions of new standards and rules for green public procurement to support the market penetration of EE lighting.

  • The development of a national phase-out policy for incandescent and other inefficient lighting fixtures will be a key milestone to make a shift towards EE lighting on the Armenian market happen. Project shall therefore keep track and support the government in developing the strategy.
  • The Russian Standard for Illumination from 2011, which is to be updated in 2016, is one of the tracks to be followed up and considered for possible adaptation/adoption in Armenia.
  • Project shall further take into account the required quality control and affordability constraints, and elaborate measures and policies, such as adequate quality control and social support schemes, by building on the experiences and lessons learned from other countries.
  • Regarding adoption of new rules for procurement of energy efficient lighting, experiences from similar work done in other countries shall be incorporated into the Armenian case

   

2

Ensure that municipal stakeholders are able to take energy efficiency criteria forward into their daily operations

  • Procurement advice and specific guidelines on formulation of technical specifications and selection criteria based on ‘most-advantageous-tender’ concepts shall be developed and introduced to municipal procurement staff, including safeguarding elements, illumination quality levels, product guarantees and environmental aspects. In this respect, specific procurement guidelines for the replacement of old CFLs and LFLs and other types of luminaires and lighting fixtures (as deemed necessary) shall be considered.
  • Project team shall further consider introducing lifecycle costing approaches in public tenders organized in the frame of the demonstration projects. Procurement and technical staff dealing with public tenders shall be trained on this concept.
  • Methodological guidance/manual prepared for audits of public lighting systems shall be relatively straight forward by building on the audits already conducted in the frame of the GUL project as well as on the information that can be drawn from the comprehensive EBRD studies done for the preparation of EBRD Street Lighting Project.
  • Support for private, international, and innovative municipal financing sources shall be given high priority under the remaining project implementation period. In order to attract further EE urban lighting programs, promotional road shows could be considered to attract more donor funds for the capitalisation of the municipal revolving funds.
  • Given the large replication potential for energy efficiency in indoor lighting (public and private residential buildings), remaining project duration shall be used to select a few representative public buildings of different type for more detailed energy audits not addressing only lighting, but their energy consumption in general (so as to contribute to the formulation of broader municipal EE programs later on).
  • With the help of dedicated financing experts, specific guidelines and templates for municipalities to develop model contracts for different kind financing modalities (direct procurement, EPCs, ESCOs etc.) shall be prepared. Additional supporting instruments, such as analytical tools to conduct lighting system audits, or technical, economic and financial feasibility assessments could be made available from other similar projects.
  • Municipal programs for EE public lighting will need to be enhanced in terms of broader coverage (e.g. overall lighting infrastructure retrofit strategy) and especially for the municipalities involved addressing the plans for retrofitting other remaining streets not included among the covered by UNDP/GEF or other IFIs (e.g. EBRD) support. Lighting programs shall also consider indoor EE lighting retrofit needs in public buildings.
3

Maintain high level of public outreach and institutionalize public awareness measures in the long term

  • The Project shall maintain the high level of dissemination and public awareness creation activities throughout the remaining project period. Public outreach expert, preferably with international background and relevant expertise shall be nominated to implement dedicated awareness activities.
  • Establishment of a testing laboratory for lighting equipment and educational laboratory in a university is already underway. However, it shall be assessed whether any complementary outreach is required e.g. for universities to encourage the effective use of the laboratory for research and educational purposes by a variety of stakeholders.
  • Lighting campaigns and awareness programs shall be widened up and addressing the potential for EE in lighting for different end-user groups (users of public and residential buildings). Reinventing the wheel is not necessary, since there is a vast experience to learn from other countries’ program experiences[1]. Project shall further start measuring impact of campaigns allowing to measure the contribution of EE lighting retrofits towards the change of energy demand in the country, and towards the project finalisation introduce specific monitoring indicators into its M&E strategy.
  • In terms of more effective and visible awareness and campaigning, a dedicated website (separate web domain) shall be introduced in Armenia. This website – eventually combined with social media functionality – shall provide the major information hub on energy efficient lighting in Armenia.
  • Experiences and lessons learnt from Armenian GUL project shall be referred to in the UNEP’s en.light programme website (http://www.enlighten-initiative.org/), thus becoming part of the international cooperation network and exchange platform. Further exchange with other countries implementing similar activities (e.g Russia, Kazakhstan, Belarus) shall be maintained.

It is recommended that the project team invest to compile “lessons learnt” from the project to contribute to the project’s knowledge management, learning and information dissemination strategy. An analytical, thorough and, as required, also critical report summarizing experience and findings of this project would highly benefit the country activities and UNDP in general. As a part of that, an analysis of the pilot project tender results and their evolution over the time in line with the capacity built shall be considered in such review.

 


[1] Just to mention a few initiatives: the European Green Light Programme (http://iet.jrc.ec.europa.eu/energyefficiency/sites/energyefficiency/files/greenlightfolder_final_2013.pdf), European LED Quality Charter (http://iet.jrc.ec.europa.eu/energyefficiency/sites/energyefficiency/files/files/documents/eu_led_quality_charter.pdf) or Australian Energy Efficiency Lighting Program (http://www.energyrating.gov.au/products/lighting)

4

Monitoring & evaluation of GHG mitigation levels and project impacts to be reviewed

  • Monitoring of GHG emission reductions and correlating energy savings is to be refined and beneficiaries supported in building own M&E capacities.
  • While direct emission reductions are likely to be achieved, indirect targets require additional action for their achievement; complementary efforts to be made during the second half of the project to address areas with large replication potential are indoor lighting in public and private residential buildings.
  • The targeted GHG reduction impact of the project was reassessed and slightly amended at the project inception phase, but neither the project document nor the inception report was calculating the direct project impact over the entire lifetime of the investment, which would make the assessment consistent with the recommended GEF methodology and other GEF funded climate change mitigation projects. Calculation methodology shall be thus reviewed and updated within the MTR and changes adopted within the GEF CC Mitigation Tracking Tool.
1. Recommendation:

 Legislation framework is improving, but focus is needed to achieve adoptions of new standards and rules for green public procurement to support the market penetration of EE lighting.

  • The development of a national phase-out policy for incandescent and other inefficient lighting fixtures will be a key milestone to make a shift towards EE lighting on the Armenian market happen. Project shall therefore keep track and support the government in developing the strategy.
  • The Russian Standard for Illumination from 2011, which is to be updated in 2016, is one of the tracks to be followed up and considered for possible adaptation/adoption in Armenia.
  • Project shall further take into account the required quality control and affordability constraints, and elaborate measures and policies, such as adequate quality control and social support schemes, by building on the experiences and lessons learned from other countries.
  • Regarding adoption of new rules for procurement of energy efficient lighting, experiences from similar work done in other countries shall be incorporated into the Armenian case

   

Management Response: [Added: 2016/11/28] [Last Updated: 2016/11/29]

 Identify institutional gaps and legislative drawback to ensure easy adoption of the new illumination code in Armenia and promote green procurement rules in the municipalities.

Key Actions:

Key Action Responsible DueDate Status Comments Documents
1.1 Develop recommendations for phase-out of the incandescent lamps, including the supportive legal framework as well as social impacts, and experience of countries with comparable economic situation and submit it to the Government.
[Added: 2016/11/29] [Last Updated: 2016/11/30]
UNDP Project team, Project Board, 2017/12 Completed 1.1 The amendments to the Law on RE and EE of Armenia was adopted in May 2016, it can help in promoting the new decision on phase-out UNDP Project Team History
1.2 The construction norms on natural and artificial illumination updated
[Added: 2016/11/29] [Last Updated: 2017/10/25]
UNDP Project Team Technical Advisory Board 2017/12 Completed Construction norms RACN 22-03-2017 “Artificial and Natural Lighting” were approved and enacted by ordinance of the Chairman of State Urban Development Committee under the Government of the Republic of Armenia (#56-N of April 13, 2017). The norms were registered by the Ministry of Justice of the Republic of Armenia on April 26, 2017, with state registration code #11717171. History
1.3 Analyze the experience of other countries which was adopted procurement rules based on principles of energy efficiency and life-cycle cost
[Added: 2016/11/29] [Last Updated: 2016/12/01]
UNDP Project Team 2017/12 Completed The report was developed and shared with the stakeholders on 30 November 2016. History
2. Recommendation:

Ensure that municipal stakeholders are able to take energy efficiency criteria forward into their daily operations

  • Procurement advice and specific guidelines on formulation of technical specifications and selection criteria based on ‘most-advantageous-tender’ concepts shall be developed and introduced to municipal procurement staff, including safeguarding elements, illumination quality levels, product guarantees and environmental aspects. In this respect, specific procurement guidelines for the replacement of old CFLs and LFLs and other types of luminaires and lighting fixtures (as deemed necessary) shall be considered.
  • Project team shall further consider introducing lifecycle costing approaches in public tenders organized in the frame of the demonstration projects. Procurement and technical staff dealing with public tenders shall be trained on this concept.
  • Methodological guidance/manual prepared for audits of public lighting systems shall be relatively straight forward by building on the audits already conducted in the frame of the GUL project as well as on the information that can be drawn from the comprehensive EBRD studies done for the preparation of EBRD Street Lighting Project.
  • Support for private, international, and innovative municipal financing sources shall be given high priority under the remaining project implementation period. In order to attract further EE urban lighting programs, promotional road shows could be considered to attract more donor funds for the capitalisation of the municipal revolving funds.
  • Given the large replication potential for energy efficiency in indoor lighting (public and private residential buildings), remaining project duration shall be used to select a few representative public buildings of different type for more detailed energy audits not addressing only lighting, but their energy consumption in general (so as to contribute to the formulation of broader municipal EE programs later on).
  • With the help of dedicated financing experts, specific guidelines and templates for municipalities to develop model contracts for different kind financing modalities (direct procurement, EPCs, ESCOs etc.) shall be prepared. Additional supporting instruments, such as analytical tools to conduct lighting system audits, or technical, economic and financial feasibility assessments could be made available from other similar projects.
  • Municipal programs for EE public lighting will need to be enhanced in terms of broader coverage (e.g. overall lighting infrastructure retrofit strategy) and especially for the municipalities involved addressing the plans for retrofitting other remaining streets not included among the covered by UNDP/GEF or other IFIs (e.g. EBRD) support. Lighting programs shall also consider indoor EE lighting retrofit needs in public buildings.
Management Response: [Added: 2016/11/29]

Enhance coordination with municipalities on building corresponding capacity within administration related to the lighting system energy efficiency for conducting:  energy audits, retrofit strategy, procurement rules, negotiation with donors and financial institutions, ESCOs, and energy management in general.

Key Actions:

Key Action Responsible DueDate Status Comments Documents
2.1 Develop and introduce procurement Guidelines for out-door and in-door lighting fixtures based on technical specifications which considers minimum energy efficiency requirements, quality safeguards, life-cycle costing. The experience of the Project gained during implementation of pilot projects will be used and consulted with partner municipalities. The Guidelines will be shared with Yerevan Municipality, partner 13 municipalities, as well as with Ministry of Territorial Administration and Development.
[Added: 2016/11/29] [Last Updated: 2017/10/25]
UNDP Project team 2017/12 Completed Green procurement guidelines developed, agreed with the key stakeholders and circulated accordingly, including to the Ministry of Nature Protection, Ministry of Territorial Administration and Development, Ministry of Energy Infrastructures and Natural Resources. History
2.2 Develop methodological guidance/manual on energy audit and organise training programme for municipality specialists.
[Added: 2016/11/29] [Last Updated: 2017/10/25]
UNDP Project Team 2017/12 Completed Methodology for performing an energy audit in residential and public buildings with its Buildings Energy Performance Calculation Tool was developed and approved earlier in the frames of a parallel project on February 25, 2016, and registered as AST371-2016 by the National Institute of Standards. It contains section of lighting issues that was enriched in accordance with the experience and expertise of this project and in line with the best international experience History
2.3 Develop the overview of the Municipal Energy Efficiency Revolving Funds and share with stakeholders
[Added: 2016/11/29] [Last Updated: 2018/11/02]
UNDP CO, UNDP Project Team 2018/11 Completed The overview developed and shared during the Donor Coordination meeting in UN on 28 June 2016 and within numerous relevant events. History
2.4 In consultation with Yerevan Municipality, Ministry of Labor and Social Affairs, donors involved in social support assistance programs select representative buildings for in-door light retrofitting projects. Analyze the project results and ensure appropriate dissemination of the outcomes.
[Added: 2016/11/29] [Last Updated: 2017/10/25]
UNDP Project Team, Project Board 2017/12 Completed Six indoor lighting demonstration projects were implemented in the buildings selected jointly by the Ministry of Nature Protection, Ministry of Labor and Social Affairs, World Vision Armenia, Yerevan Municipality, National Academy of Sciences has identified potential areas for the demonstration of indoor lighting retrofit projects. History
3. Recommendation:

Maintain high level of public outreach and institutionalize public awareness measures in the long term

  • The Project shall maintain the high level of dissemination and public awareness creation activities throughout the remaining project period. Public outreach expert, preferably with international background and relevant expertise shall be nominated to implement dedicated awareness activities.
  • Establishment of a testing laboratory for lighting equipment and educational laboratory in a university is already underway. However, it shall be assessed whether any complementary outreach is required e.g. for universities to encourage the effective use of the laboratory for research and educational purposes by a variety of stakeholders.
  • Lighting campaigns and awareness programs shall be widened up and addressing the potential for EE in lighting for different end-user groups (users of public and residential buildings). Reinventing the wheel is not necessary, since there is a vast experience to learn from other countries’ program experiences[1]. Project shall further start measuring impact of campaigns allowing to measure the contribution of EE lighting retrofits towards the change of energy demand in the country, and towards the project finalisation introduce specific monitoring indicators into its M&E strategy.
  • In terms of more effective and visible awareness and campaigning, a dedicated website (separate web domain) shall be introduced in Armenia. This website – eventually combined with social media functionality – shall provide the major information hub on energy efficient lighting in Armenia.
  • Experiences and lessons learnt from Armenian GUL project shall be referred to in the UNEP’s en.light programme website (http://www.enlighten-initiative.org/), thus becoming part of the international cooperation network and exchange platform. Further exchange with other countries implementing similar activities (e.g Russia, Kazakhstan, Belarus) shall be maintained.

It is recommended that the project team invest to compile “lessons learnt” from the project to contribute to the project’s knowledge management, learning and information dissemination strategy. An analytical, thorough and, as required, also critical report summarizing experience and findings of this project would highly benefit the country activities and UNDP in general. As a part of that, an analysis of the pilot project tender results and their evolution over the time in line with the capacity built shall be considered in such review.

 


[1] Just to mention a few initiatives: the European Green Light Programme (http://iet.jrc.ec.europa.eu/energyefficiency/sites/energyefficiency/files/greenlightfolder_final_2013.pdf), European LED Quality Charter (http://iet.jrc.ec.europa.eu/energyefficiency/sites/energyefficiency/files/files/documents/eu_led_quality_charter.pdf) or Australian Energy Efficiency Lighting Program (http://www.energyrating.gov.au/products/lighting)

Management Response: [Added: 2016/11/29] [Last Updated: 2016/11/29]

The Project outreach strategy implementation will be continued and upgraded to the current project stage needs. 

Key Actions:

Key Action Responsible DueDate Status Comments Documents
3.1 Enhance project results dissemination and public awareness activities throughout the remaining project period
[Added: 2016/11/29] [Last Updated: 2018/11/02]
UNDP CO, UNDP Project Team, Project Board 2018/11 Completed Public awareness activities will be concluded with the finalizing seminar in November 2018, and project results dissemination will be continued in the frames of Climate Change Program activities. History
3.2 Assist in effective use of the lighting testing laboratory for research and educational purposes by a variety of stakeholders.
[Added: 2016/11/29] [Last Updated: 2017/10/25]
Project Board, Yerevan Municipality 2017/12 Completed 71 products were tested and certified accordingly during the two years of the laboratory’s full-fledged operation. History
3.3 Work-out approaches for measuring awareness campaigns impact assessment.
[Added: 2016/11/29] [Last Updated: 2017/10/25]
UNDP-GEF RTA, UNDP Project Team 2017/12 Completed 1. Public survey implemented in 2015, another one upcoming 2. Import data of LED and luminaires of other types received from Customs Service of the State Revenue Committee of the RA. 3. Green Lighting Week is organized in Yerevan as an informational campaign in a series of thematic events from 23 to 27 October 2017. History
3.4 Create the dedicated web-page for sharing the lighting energy efficiency information and establish the link with UNEP’s en.light programme web-site.
[Added: 2016/11/29] [Last Updated: 2017/10/25]
UNDP-GEF RTA, UNDP Project Team 2017/12 Completed The web-page is created and updated regularly and continuously. History
4. Recommendation:

Monitoring & evaluation of GHG mitigation levels and project impacts to be reviewed

  • Monitoring of GHG emission reductions and correlating energy savings is to be refined and beneficiaries supported in building own M&E capacities.
  • While direct emission reductions are likely to be achieved, indirect targets require additional action for their achievement; complementary efforts to be made during the second half of the project to address areas with large replication potential are indoor lighting in public and private residential buildings.
  • The targeted GHG reduction impact of the project was reassessed and slightly amended at the project inception phase, but neither the project document nor the inception report was calculating the direct project impact over the entire lifetime of the investment, which would make the assessment consistent with the recommended GEF methodology and other GEF funded climate change mitigation projects. Calculation methodology shall be thus reviewed and updated within the MTR and changes adopted within the GEF CC Mitigation Tracking Tool.
Management Response: [Added: 2016/11/29] [Last Updated: 2016/11/30]

The monitoring and evaluation of the project impact on GHG mitigation will be reviewed to make it consistent with the recommended GEF methodology.

Key Actions:

Key Action Responsible DueDate Status Comments Documents
4.1 The indoor lighting energy efficiency retrofit projects results evaluated, as well as their replication potential assessed
[Added: 2016/11/29] [Last Updated: 2018/11/02]
UNDP Project team 2018/11 Completed The findings of indoor lighting energy efficiency retrofit projects’ implementation are summarized and incorporated in the Lessons Learned Report and in the final PIR. History
4.2 The project GEF CC Mitigation Tracking Tool revised
[Added: 2016/11/29] [Last Updated: 2017/10/25]
UNDP Project team 2017/12 Completed The mitigation impact methodology is revised to make it consistent with GEF Tool History

Latest Evaluations

Contact us

1 UN Plaza
DC1-20th Floor
New York, NY 10017
Tel. +1 646 781 4200
Fax. +1 646 781 4213
erc.support@undp.org