Energy Efficiency in Public Buildings in Ukraine

Report Cover Image
Evaluation Plan:
2018-2022, Ukraine
Evaluation Type:
Mid Term Project
Planned End Date:
01/2020
Completion Date:
12/2019
Status:
Completed
Management Response:
Yes
Evaluation Budget(US $):
30,000

Share

Document Type Language Size Status Downloads
Download document MTR_EEBuilding_TOR_Final.docx tor English 120.53 KB Posted 124
Download document MTR Final Report EEPB for download to ERC.docx report English 2402.77 KB Posted 152
Title Energy Efficiency in Public Buildings in Ukraine
Atlas Project Number: 00088958
Evaluation Plan: 2018-2022, Ukraine
Evaluation Type: Mid Term Project
Status: Completed
Completion Date: 12/2019
Planned End Date: 01/2020
Management Response: Yes
Focus Area:
  • 1. Environment & Sustainable Development
  • 2. Others
Corporate Outcome and Output (UNDP Strategic Plan 2014-2017)
  • 1. Output 1.5. Inclusive and sustainable solutions adopted to achieve increased energy efficiency and universal modern energy access (especially off-grid sources of renewable energy)
SDG Goal
  • Goal 13. Take urgent action to combat climate change and its impacts
  • Goal 7. Ensure access to affordable, reliable, sustainable and modern energy for all
  • Goal 9. Build resilient infrastructure, promote inclusive and sustainable industrialization and foster innovation
SDG Target
  • 13.2 Integrate climate change measures into national policies, strategies and planning
  • 7.1 By 2030, ensure universal access to affordable, reliable and modern energy services
  • 9.4 By 2030, upgrade infrastructure and retrofit industries to make them sustainable, with increased resource-use efficiency and greater adoption of clean and environmentally sound technologies and industrial processes, with all countries taking action in accordance with their respective capabilities
Evaluation Budget(US $): 30,000
Source of Funding: Project budget; donor
Evaluation Expenditure(US $): 27,400
Joint Programme: No
Joint Evaluation: No
Evaluation Team members:
Name Title Email Nationality
Louis-Philippe Lavoie Mr l.philippe.lavoie@gmail.com
Igor Komendo Mr komendo@gmail.com UKRAINE
GEF Evaluation: Yes
GEF Project Title: Removing Barriers to increase investment in Energy-Efficiency in Public Buildings in Ukraine through the ESCO Modality in Small and Medium Sized Cities
Evaluation Type: Mid-term Review
Focal Area: Climate Change
Project Type: FSP
GEF Phase: GEF-5
GEF Project ID:
PIMS Number: 4114
Key Stakeholders:
Countries: UKRAINE
Lessons
Findings
1.

Project Design- Strategy and Outputs-based Rating  

 Main Findings

Strategy update:

The PIF has been drafted in early 2013 and many adjustments to social, economic and political environments have been made since that time. It is recommended that after a period of six years or less after the PIF stage a period should be allocated for reflecting and updating the project strategy.

Despite the PIR 2018 call for the development of a new strategy for the EEPB implementation (cf. page 11 PIR 2018) to support and improve the general implementation strategy highlighted in Section 2 of the Project Document, the Evaluation Team has not been informed nor do they have access to any document pertaining to the integrated strategy.

The EEPB Project Manager drafted some activity-based strategy documents, but there has not been a document that deals with the project as a whole through an integrated strategy, which is geared toward the successful implementation in terms of effectiveness and identified barriers removal.

The Evaluator must mention, however, that the Project Manager made a presentation at the Project Steering Committee meeting in December 2018 related to his priority topics that must be dealt in 2019. But at midterm, the PM did not outline the integrated strategy with the aim of linking the project Outcomes as a whole.

A few strategic elements are mentioned in the Project Document (page 24, section 2: Strategy). Once again, the EEPB project should have improved or upgraded the general strategy highlighted in the Project Document by taking into consideration the actual situation and the existing barriers to EPC development. It is essential to outline a strategy during the midterm.


Tag: Energy Implementation Modality Programme/Project Design Infrastructure Technology Awareness raising Capacity Building

2.

Energy efficiency in public buildings through the installation of solar panels:

Grant 49,700 USD

  • Energy Management and Monitoring Systems were introduced in 4 municipal objects in Puscha-Voditsa city.
  • 4 energy managers were trained to oversee the EE measures in above-mentioned objects.
  • In addition, some EE measures based on renewables were introduced to reduce GHG footprint. (Sanatorium for Children).

Training of Energy Managers and EMIS: Grant 22,530 USD

  • Set up of a Training Centre for energy managers in Cherkassy Oblast.
  • 17 energy managers were assigned.
  • In addition, EMIS has been introduced in 17 public buildings and EE measures using renewable solar energy were implemented in the Training Centre.
  • 1100 people have been trained in energy management and energy saving. The Center of Practitioners are now able to monitor the energy consumption of public buildings in the district (17 objects) and promptly report deviations from the optimally predicted level.

Setting up of the EE Centre as a platform for supporting community development: Grant 49,950 USD

  • The Energy Efficiency Centre has been set up in the city of Zhytomyr, which now works in close cooperation with the Zhytomyr oblast administration.
  • The EE Centre provides trainings for energy managers for the surrounding cities (so far 72 energy managers were trained and employed). In addition, EE improvement/repair works in the premises of Centre were achieved. 

Tag: Energy Environment Policy Education Awareness raising Capacity Building Institutional Strengthening

3.

EMIS&C Implementation:

The EEPB did exemplary work in terms of implementing the EMIS&C at the local level in selected buildings where EPC pilots and replication projects were implemented. The EEPB project duly signed 27 MoUs with selected municipalities rather than 10 as stated in the Project Document. The MoU requires the involvement of a designated Energy Manager, a conditionality, for getting a Grant for Pilot Projects as well that the development of the Sustainable Energy Action Plan and the implementation of the EMIS&C: 10 SEAPs have been submitted until now and 10 EMIS Plans have been developed in the selected municipalities. This work is a work-in-progress and a considerable effort is required to extend it nation-wide.

Again, the Evaluator recalls that EMIS is, as a priority, a centralized task to be managed by the SAEE. At the local level, the priority should be given to EMIS and Control with the aim of reducing the energy consumption and demand and feeding data to central EMIS (SAEE).

Nation-wide Energy Consumption Database and EMIS at the local level:

The nation-wide extension of the EMIS is a significant challenge. To extend the nation-wide EMIS (database) the database must be centralized. There must be mandatory reporting of energy consumption data as well, an assigned energy manager for each municipality, among others, to fill out data into the web-based EMIS. At this point in time, there are 3 operational databases implemented out of the project framework: (i) the Kiev Oblast Energy Efficiency Center (KOEEC); (ii) the State Agency for EE (SAEE), and iii) the Odesa Energy Management Agency (OEMA). Some other municipalities have also rolled out their own ‘’platform’’ for data gathering in their buildings. Based on information gathered during site visits and from documents, only OEMA has connected its database with the distance metering equipment installed in 23 buildings by EPC projects. It is a good start, but the EEPB project is not on the verge of extending the EMIS Database countrywide.


Tag: Energy Resource mobilization National Local Governance Technology

4.

Crosscutting issues

Gender equality

The Project Document stipulates achieving gender equality through the empowerment of women to fully participate in all project activities and specifically those related to capacity development under the various components through working, for example, with NGOs like “Krona”, the Ukrainian Women’s Fund, La-Strada, School of Equal Opportunities, All-Ukrainian Women Centre of Information and Social-Economic Adaptation, Association of Energy Auditors. 

It was found that project didn`t conduct any gender assessment in the target areas and all the activities carried out were predominantly gender blind. However, the PMU pays special attention in assessing gender balance while implementing each of the Pilot projects (Component 3).

Taking to account that the majority of Pilot projects have been implemented on the basis of kindergartens, schools, gymnasiums and hospitals, the majority of final beneficiaries of the project are children (particularly in schools, gymnasiums and kindergartens). On average, the number of girls in these institutions reaches 60%, however, given the individuality of each institution, there were opposite examples. Teachers occupy a niche less than 10% in the gender balance and given the fact that Ukrainian schools adhere to traditional values (that women are more responsible and careful when it comes to teaching and raising children) the distribution between men and women is about 10-15% and 85-90% respectively. It is hard to assess gender distribution in hospitals, due to large and unstable turnover of people. Analyzing the whole project and taking into account people who worked with the project and who are direct users of public buildings involved in the project, the gender distribution was 54% of women and 46% of men, which shows to be identical to the gender situation in Ukraine.

The implementation of other Components of the project (EMIS introduction and awareness raising, legal framework development, FSM creation) does not foresee any specific strategy on sustaining gender balance. 


Tag: Energy Gender Equality Women's Empowerment National

5.

Human rights

The project design does not address or demand directly any specific human rights references. The PMU demonstrated no special strategy towards human rights aspect in the delivery of project Components. However, good human health is the key to a high quality of life and high productivity. The health quality is influenced by most of the equivalent factors, one of which is a safe working environment. In the given context, the project, by introducing ESCO mechanism and EMIS, provided the municipal buildings` operators with the opportunity to create a comfortable and safe temperature regime, thereby creating favorable working conditions for teachers, students, children, doctors, patients and other employees of institutions supported by the project. Thus, the project addresses predominantly the “right to education” supporting one of its main pillars “Availability” which stipulates that educational buildings should meet both safety and sanitation standards.

 

SDGs 

One of the key elements of the project is the introduction of the ESCO mechanism and EMIS in public buildings making possible for its operators: (1) provide the most efficient use of energy in heating and hot water supply systems by increasing the efficiency of thermal installations and the rational use of energy recourse; (2) use advanced renewable energy sources and energy installations; (3) produce budget savings; (4) reduce greenhouse gas emissions.

This contributes to the advancement of SDG 7: “Providing access to affordable, reliable, sustainable and modern energy sources for all”, and SDG 13: “Taking urgent action to combat climate change and its effects.” on the country level.

All of the abovementioned gives an opportunity to predict even more impact on two SDGs by the end of the project.

 


Tag: Climate change governance Human rights Health Sector Human and Financial resources Education Agenda 2030

6.

Project Design and Strategy Outline              

Strategy outlined in 2015 is still relevant

The objective of this project is, among others, to assist the Government of Ukraine, as outlined in the “Energy Strategy of Ukraine to 2030”.

In the business as usual scenario, implementation of energy efficiency meaures in public buildings with reliance on budgetary resources and without the participation of the private sector, will take an extended period of time to materialise. In accordance with the project design, the EEPB will support the Government of Ukraine in:

  • Creating attractive and competitive business terms and conditions for investors/ESCOs, such as initial support in the preparation of feasibility studies and in terms of an investment grant for 10 pilot projects, which will give developers long-term stability and provide a sufficient ROI
  • Establishing an Innovative Financing Mechanism to make it easier for ESCOs to obtain commercial financing in order to implement EPC contracts 
  • Development of financial incentives is to be provided to ESCOs to invest in Energy Efficiency in public buildings such as income tax holiday for a specific period, duty and tax exemptions on equipment and services
  • Providing capacity development to ESCOs in order to promote investment in support of Energy Efficiency in public buildings
  • Support for improved energy management in public buildings in Ukraine, through more metering and energy management information systems, as tools to encourage and promote additional investments in energy efficiency

At midterm of the EEPB project, as pointed out in the Project Document (designed in 2015-2016, and approved in March 2016), the economic situation is still remaining difficult and the high cost of capital remains a key barrier to capital-intensive investments in EE projects. The strategy promoted by the EEPB aiming to involve the private sector and the Energy Performance modality remains a relevant strategy to face the current and mid-term economic situation in Ukraine. Although most of EE-EPC investments are low capital-intensive, one can say that the ‘’pump has been primed’’ through a series of EPC projects implemented by ESCOs because of the EEPB project.

 


Tag: Energy Business Model Programme/Project Design Jobs and Livelihoods Capacity Building

7.

Project Design and Logframe

The project’s goal is to reduce GHG emissions by creating a favorable legal, regulatory and market environment and strengthening institutional, administrative and technical capacities to promote the implementation of the ESCO model through EPC contracts model and by supporting the involvement of energy managers in public buildings. The Project Document goes in that direction. Its four components encompass the required basic support to Ukraine and local authorities (small to medium-sized cities) for attracting the ESCO business model and private investment in EE project initiatives under the EPC modality.

In addition, the combined focus on Energy Management Systems and ESCO market development was somewhat innovative in the public sector. Other donors worked on ESCO market development or on energy management systems, but there was no project supporting a combined and integrated approach to ESCO market development and energy management. The project design was appropriate and relevant, taking into consideration the economic and social contexts.

Despite the good project design, the EEPB project faced some problems under the Component 3 (Pilot Projects) while the budget line (BL) 72 200 (the most important BL: 1,250,000 USD) was not clearly dedicated to grants for pilot or demonstration projects because of its general title ‘’Equipment and Furnitures’’. On the other hand, the objective and budget limitation (50,000 USD per project, for a series of 10 Pilot projects) of that ‘’crypto-grant’’ BL were defined in the Project Document (cf.: Project Document, page 16): ‘’Creating attractive and competitive business terms and conditions for investors/ESCOs, such as initial support in the preparation of feasibility studies and in terms of an investment grant for 10 pilot projects, which will give developers long-term stability for an appropriate ‘’return on investment’’. Within the same BL, the SGP has been involved (11 projects, see Appendix 9) for supporting a series of training deliveries mainly related to EMIS and Energy Managers. In addition, the SGP implemented a few EE projects with the aim of demonstrating the relevance and performance of RE and EE measures.


Tag: Emission Reduction Energy Resource mobilization National Local Governance Partnership Programme/Project Design Project and Programme management Urbanization

8.

Results Framework Coherency and Performance           

Through its four components, the whole EEPB Project framework is coherent with the approved objectives: ‘’Accelerating the implementation of EE measures in public buildings through the ESCO business model utilizing the EPC modality.”

In term of quantitative results, the EEPB project is performing well, and the Evaluator is confident that objectives and outcomes should be achieved, with only minor shortcomings under Outcome 2:

  • Output 2.1: Financial Support Mechanism (FSM) established and capitalized to support private investment (ESCO) in Public Buildings in Ukraine toward a non-realistic target (21 million USD) as mentioned in the previous paragraph
  • Still under Outcome 2, the EEPB project did not implement the necessary FSM to face the high cost of capital in the banking system in Ukraine. Because of the EEF regulation, which does not allow financing EE projects in public buildings, and in the absence of a donor or a third party willing to invest on a long run commitment (e.g.: a dedicated fund) a significant amount of money for providing soft-loans to ESCOs or other EE investors in the public sector, the design of a comprehensive FSM is just a theoretical exercise

However, with the aim of foreseeing further development, the Evaluator makes a recommendation (#1) to soundly support the SAEE at the first stage of the Green Development Funds and through the establishment of a sustainable mechanism helpful to ESCO business model development and EPC modality. For that purpose, the EEPB must intensively cooperate with the SAEE.

On the other hand, the Evaluator would like to point out the basic FSM put in place and already operational. The ‘’ESCO Factoring’’ modality can allow ESCOs more fluidity for dealing with the banks’ requirements related to collateral for obtaining a loan from the bank. This is in line with the international best practice but it does not resolve the issue of the high cost of capital.   


Tag: Energy National Rule of law Business Model Monitoring and Evaluation Partnership

9.

Progress towards Results Outcomes / Outputs Analysis

This Section of the MTR report is the most important one for evaluating and rating the Outcomes/Outputs-based project implementation. The Evaluator met with key decision-makers and stakeholders, as well as ESCOs’ project beneficiaries in 4 cities (Ref.: Mission Agenda Appendix 6). In addition, the evaluation team met many times with the EEPB PMU and the UNDP CO for collecting the reliable data and cross-checking information with regard to the implementation process and results.

Key Achievements

In practice, the Project was initiated in April 2017 (involvement of the PM) the EEPB Project has improved its performance, and to a certain extent recovered for the lost time (Dec. 2016 - April 2018). The lack of performance during the first year is mainly related to NIM scheme as defined in the Project Document and the lack of interest of the MinRegion, the key implementation partner.

In 2018 the project shifted to DIM implementation modality. Most of tasks were launched in 2018-2019, especially the series of Pilot Projects, EMIS at local level, training deliveries, the setup of the ‘’Intergovernmental Technical Working Group Working Group’’ to develop the National Energy Management and Monitoring System as well than other activities described and rated in Section 4.3 in terms of result, efficiency and effectiveness.

The Evaluator is confident that the objective/outcome is expected to achieve most of its end-of-project targets, with only minor shortcomings, although more intensive work must be done with regard to Outcome 4 to ensure a fully operational countrywide EMIS Database

 


Tag: Energy Environment Policy Effectiveness Local Governance Rule of law Partnership

10.

Results Outcomes / Outputs Analysis Ratings

The following table is very important to rule on the EEPB implementation performance and achievements. In addition, the results review toward targets and objectives, Table 4.2.1 rates the efficiency and effectiveness of each Outcome. The rating is in line with the UNDP-GEF MTR Evaluation Guidelines highlighted in the MTR TORs The corresponding weight (between 1 to 6) of each Output is based on results and achievements toward targets. The Outcome is rated on the average rating of the build-in Outputs under each project component (Outcome).

 


Tag: Effectiveness Efficiency

11.

Remaining barriers to achieving the project objective

The Evaluator reviewed all barriers pointed during the initial stages of the project design 4 years ago. At midterm, the Evaluator is confident that critical barriers will be removed, with the exception of one, which is related to the cost of capital.


Tag: Programme/Project Design Project and Programme management

12.

Project Implementation and Adaptive Management   

GEF projects tend to be overdesigned due to an inability to adapt to a new situation or context in the target country or the project design needs minor improvements to be in line with the approved objectives. Adaptive Management softens the common criticism that sometimes Pro Docs are too rigid. On the other hand, it is important to be aware of the changes allowed and levels of authority required for approval.

To shorten implementation delays with regards to TA and EE improvement projects, with the weakness of the national taxation system (inefficient VAT recovering), and to facilitate the implementation of EE projects in public buildings in small communities, the UNDP CO and the PMU asked the SGP Unit to carry out a series of EE activities through the involvement of some NGOs knowledgeable of issues and context of the community-based development. This is a good example of adaptive management, because the whole EEPB project was delayed during the first two years in regard to its planning, and also due to the SGP Unit requiring time to become fully operational and knowledgeable of the requirements and community-based commitment. 


Tag: Energy Efficiency Human and Financial resources Implementation Modality Project and Programme management

13.

Management Arrangements              

As mentioned in the previous paragraph, the Project is being implemented under the UNDP Direct Implementation Modality (DIM) rather than the NIM as planned in the Project Document. The UNDP Ukraine is fully responsible for the overall management of the project. Full-time Project Manager (PM) is responsible for the day-to-day management and decision-making for the Project’s planning, reporting and supervision of of the Project experts and other Project staff.

Overall rating of the Management Arrangement is rated satisfactory because of the quality involvement of the UNDP CO-Officer, his close working relationship with the SAEE (especially in regard to the on-going nationwide EMIS-database), the dedication of the Project Manager and most importantly the direct link established by the PMU to the municipal sector through cooperation agreements with cities.

The project Management Arrangement is seemingly efficient and collaborative for planning and making the follow-up on going activities:

-              The EEPB Project holds Project Board annual meetings: one on Dec 11th, 2017, second on Dec 20th, 2018. The meetings were carried out with the participation of the key ministries and state agencies, international and national experts, municipalities, business representatives, and UNDP CO and RTA. 


Tag: National Local Governance Implementation Modality Partnership Project and Programme management

14.

Work planning 

The EEPB project has been duly approved on December 2016, but for many reasons out of the control of the UNDP CO (e.g. the lack of involvement of the key implementation partner), the project effectively began in December 2017 after the first Project Board Meeting. That is the reason why only two AWPs were drafted: AWP 2018 and AWP 2019. The Inception Workshop has been held in June 2017.

Despite the slow start, the EEPB project made up for the lost time. At midterm, key expected results were reached or surpassed, although a large share of work has yet to be completed in order to support the SAEE in implementing the nationwide EMIS Database during the upcoming two years.

Work planning is being carried out in a manner, which is consistent with the Project Document and GEF-UNDP guidelines. In particular, it is conducted on the basis of annual work plans (AWPs), which are reviewed and approved by the Project Board. The detailed Annual Work Plan is formulated in the form of a Project Implementation Plan that incorporates key features.


Tag: Human and Financial resources Implementation Modality Monitoring and Evaluation Risk Management Sustainability

15.

Finance and co-finance

The table below highlights a worrisome situation in regard to GEF financing disbursement. At midterm, the total GEF funding disbursement is achieved only at 33%. The EEPB activity planning must be significantly accelerated during the upcoming two years. The implementation of the nationwide EMIS Database, as well as the EMIS & Control at the local level needs more TA and equipment. Because pilot projects are among probably the best way to demonstrate the efficiency and relevance of the ESCO business model, another series of low-cost EPC Pilot Projects must be implemented in public buildings. The budget resource is available to go further in that direction.


Tag: Energy Efficiency Global Environment Facility fund Resource mobilization Human and Financial resources Policies & Procedures

16.

Project-level monitoring and evaluation systems        

The PMU and the UNDP CO managed to issue the AWPs by the end of the year, and the PIRs in line with the Project Document and GEF requirements in July.

As a rule, the UNDP CO and the PM provided all available data on the results in the form of energy savings and GHG emission reduction. The EEPB designed a comprehensive Excel file for all EPC (Pilots and replications), making the follow-up of results easier for external parties such as the Evaluator (MTR). In addition, the EEPB used the CCM Tracking Tool (last update July 2019). Although the EPC excel file is very informative, the file must be improved because the evaluator found out some minor errors.

According to the Evaluator, there might be a lack of quality information with regard to the actual impact of the EMIS & Control in term of energy savings. Based on the result at midterm, most of the energy savings (+/- 60%) results of the EMI&C. The impact of EMIS&C is difficult to define while the EPC Project is also implementing some EE measures. In other words, the EEPB should take action for figuring the actual impact of solely EMIS&C. The Evaluator makes a recommendation (#2) to this end.

Finally, and not the least, the EEPB project carried out the MTR as scheduled in July 2019. An independent National Evaluator supported the international Evaluator. Both Evaluators carried out the two-week MTR mission in July 2019. The UNDP CO as well as the PMU provided a constant and transparent support to the MTR Evaluation team.

The M&E task is rated Satisfactory (S).


Tag: Emission Reduction Energy National Monitoring and Evaluation

17.

Stakeholders’ engagement              

The EEPB Project was not in a position to undertake a project of this magnitude and make up for the last time without the collective and cooperative efforts of all parties involved. The list of participants at the Project Board Meeting (PBM) confirms the effective engagement of stakeholders and decision-makers from: (i) GoU, including SAEE, (ii) Parliament representatives; (iii) UNDP and RTA; and (iv) implementing Partners. The PBM 2017 involved 20 participants, and the second PBM 2018 involved 26 participants. This is an indicator of interest and engagement of key stakeholders.

The Stakeholders Engagement is rated Highly Satisfactory (HS).


Tag: Civic Engagement

18.

Reporting

Such as it is for other GEF-UNDP projects, the Project Manager is preparing PIRs (early July). PIRs follow the standard UNDP/GEF format and provide general ratings and comments on the Project’s progress by the Implementing Partner, Project Manager UNDP Country Office Programme Officer, Project Implementing Partner and Regional Technical Advisor. The reports provide information on planned and implemented activities. However, the PIR template deals with progress and issues pertaining solely to outcomes. The Evaluator is required to schedule two long meetings with the Project Manager to evaluate the implementation details and 28 output results.

Communications

The communication between the Project Manager, the UNDP CO as well as with the key Project Partner (SAEE), is a continuous and efficient process. However, in order to speed up the implementation of the nationwide EMIS Database, the Project should work more closely and strengthen its communication link with the SAEE, which is responsible for the countrywide database.

As mentioned in Sections 4.3.1 and 4.3.5 the EEPB Project arranged round-table meetings with priority stakeholders to ensure their cooperation for the project components, especially the ones under Outcome 2, 3 and 4. The list of participants to PBM shows that none of the key stakeholders were left out during communication and information dissemination. The Evaluator confirms that all the stakeholders interviewed (site visits in 4 cities) are aware not only of the objectives and project strategy but also of current situation and further project initiatives.

The project website is comprehensively documented and very useful to stakeholders, and even to UNDP CO staff and the RTA to follow up the progress and evidence of achievements.


Tag: Communication Implementation Modality Partnership Project and Programme management

19.

Sustainability

In this section of the MTR report, the Evaluator is required to deal with four priority topics as follows:

  • Financial risks to sustainability
  • Socio-economic risks to sustainability
  • Institutional framework and governance risks to sustainability
  • Environmental risks to sustainability

In accordance with the GEF MTR guidelines, the rating scale of Sustainability components is as follows: 4 (likely), 3 (moderately likely), 2 (moderately unlikely) and 1 (unlikely).

Based on the desk review of various project documents (Appendix 5), meetings with stakeholders and the EEPB Project Team and the UNDP CO, the Evaluator is required to answer the following question:

‘’Sustainability: To what extent are there financial, institutional, socio-economic, and/or environmental risks to sustaining long-term project results?’’.


Tag: Sustainability Global Environment Facility fund Human and Financial resources Operational Efficiency

20.

Socio-economic risks to sustainability    

Social or political risks that may jeopardize the sustainability of project outcomes, the risk that the level of stakeholder ownership, including ownership by governments (municipalities) and other key stakeholders (banks and ESCO) will be insufficient to allow for the project outcomes/benefits to be sustained.

It is Moderately Unlikely that stakeholders` ownership may be jeopardized as the Project Beneficiaries are publicly owned and operated by municipalities, the geo-political and/or social risk is minimal. Municipalities are public institutions operating on an annual budget-base and monetary savings (EPC payments) are quite accurately known in advance.

Is there sufficient public / stakeholder awareness in support of the long-term objectives of the project and its replication? Are the lessons learned being documented by the Project Team on a continual basis and shared/ transferred to appropriate parties who could learn from the project and potentially replicate and/or scale it in the future?

The potential replication and scaling is Likely to happen as Financial institutions (banks) make money because of interest rates, ESCOs make money because of the cost-effectiveness of the EE measures and relevant services, and municipalities save money on a long run, incrementally during the payback period (10%) and an exponential saving (100%) after the payback period. All parties find their interests to be sustainable through this accord. Municipalities met the MTR team and discussed their intent to implement more EPC with ESCOs on a grander scale and in greater volume. The lesson learnt is related to the cost - when interest rates rise significantly, investment decisions are postponed or the ESCOs’ investment amount reduced.


Tag: Sustainability Local Governance Human and Financial resources Awareness raising

21.

Institutional framework and governance risks to sustainability

The legal frameworks; policies; governance structures and processes pose risks that may jeopardize the sustenance of project benefits.

It is Unlikely that current legal frameworks, policies, governance structures may jeparis the sustenance of the project benefits. About fifteen years ago while the first UNDP-ESCO project has been implemented in Ukraine (Rivne) and at that time, the national and regulation framework related to ESCO-EPC was not as well developed as it is in 2019. Consequently (2000-2010) the UNDP-ESCO Rivne project did not perform, among others, due to a lack of relevant regulations related to ESCO-EPC development in the municipal sector.

However, the ESCO business model took off after due to a major improvement in national regulation. To a certain extent, most of regulation barriers are on the verge of being lifted because of the EEPB project.

On the other hand, the Evaluator pointed out in many sections of the MTR, that the cost of money in Ukraine impeded the ESCOs’ cost-intensive investment decision. It is quite true but at the same time, the low cost-intensive investment lessens the risk of payment as well as the risk of technical failures. At the stage of barrier removal (purpose of the current project), such a situation drastically mitigated the risk due to the implemented basic EE improvements, and the financial risk was reduced due to the low capital investment.

The extent to which the project managed to improve or contribute to legal frameworks related to ESCO-EPC development in Ukraine?

It is Likely that the project management contribute to legal framework related to ESCO – EPC development in Ukraine as it is mentioned in a previous rationale, there are many lessons learnt from the failure of the Rivne project (ended in 2011) especially in term of regulation and legal frameworks. During the most recent years the majority of the regulatory barriers have been removed:

  • The Parliamentary Committees and members of parliament (MPs) have approved the Draft Law No. 9386 as of 10.12.2018 “On the improvement of the ESCO mechanism” and recommend it to be passed in the first Parliamentary reading. The document was not voted yet due to dissolution of the Verkhovna Rada (June 2019).
  • Law on New Investment Opportunities in Energy Efficiency (“ESCO Law”) #327 adopted in 2015, amended in January 2019.
  • Law “On Energy Efficiency in Buildings” #2118 was adopted in 2018.

The core legislation is set. EEPB is working still on amendments to improve the law “On the energy efficiency of public buildings.” Work is currently underway towards the mandatory involvement of trained Energy Managers in all cities until the end of the project time horizon.

 


Tag: Energy Sustainability Rule of law Business Model Risk Management

22.

Environmental risks to sustainability    

The environmental risks that may jeopardize the sustenance of the project outcomes.

It is Moderately Unlikely that the project may face environmental risks. As a rule, EE measures implemented by ESCOs are environment-friendly. But not all levels of risk can be negated especially in regard to pipes cleaning. Pipe cleaning is one of the basic and low-cost EE measures for making the hot water distribution in buildings more efficient in term of the heat transfer. Mineral acid cleaning is used for the removal of metallic scale and corrosion products. It is usually applied in a three-stage operation: degreasing, metal collection, and passivation. This method must be applied with care due to a variety of reasons, including environmental, safety, and disposal issues. ESCOs are aware of the risk and manage to mitigate that risk in the appropriate way.


Tag: Environmental impact assessment Sustainability

Recommendations
1

New FSM and Project Financing Enhancement with the aim of establishing a relevant and efficient FSM on a long-term basis, the EEPB project should make an arrangement to work alongside the SAEE at the design stage of the Feasibility Study of the ‘’Green Fund’’, which was envisaged to finance EE activities in various economic sectors. The Government of Finland expects to finance the Feasibility Study. The main purpose of the recommended involvement is to make sure that the likely ‘’Green Fund’’ will have a built-in component to deal with ESCO/EPC financing needs. To this end, the EEPB should provide the financial TA for to the SAEE.

Who:

The International or National Expert with an extended understanding and experience in financial mechanisms dedicated to ESCOs, e.g.: Loan Guarantee Fund, Factoring, forfaiting[1] or Buyout mechanism. The Financial Mechanism Expert will closely work with the SAEE.

When and How:

Step 1: Sign a MoU with the SAEE for confirming their mutual interest and commitment.

Step 2: The EEPB will draft the National/International Expert’s ToRs in coordination with the SAEE. The expert should be jointly selected by the Project and the SAEE.

Step 3:  The Expert must have a workstation within the SAEE’s premise.

 

2

Countrywide EMIS extension - Whilst not undervaluing the importance of ESCO development, the priority for the upcoming two or three years should be transferred to the development of the countrywide EMIS-Database.

The Evaluator recommends much more intensive support in the direction of SAEE and the Working Group, with the aim of outlining a detailed methodology and selecting the most appropriate web-based tool(s). Such a support is essential for the rollout of a platform for web-based data collection and analysis at a centralized level (SAEE).

In addition to the Working Group mentioned above, the Evaluator recommends setting up a sort of Task Force dedicated to the development and implementation of the countrywide EMIS from January 2020 to the end of the project timeframe.

In many municipalities some officials mentioned a barrier at the level of city council related to investment in EMIS. This recommendation aims at extending the EMIS to a broader spread of municipalities, countrywide, while simultaneously extending the EMIS&C at the local building level (ref.: Recommendation 4). For extending the EMIS to all cities, the project must continue and enhance its effort for enforcing a new secondary regulation to this end.

Who:

The EEPB already involved an international expert for dealing with the web-based tool. The expert should increase his involvement with the SAEE with the aim of accelerating the full rollout of the web-based EMIS Database. It has been done in many countries and the EEPB could learn more from the web-based EMIS in operation in the sub-region. In addition to a more intensive involvement of the international expert, the EEPB should involve a National Expert for supporting the reputable International Expert involved since 2018. The EMIS National Expert should also be responsible for meeting with municipalities, pushing them to embark on the EMIS and EMIS&C.

3

Strengthen Awareness and Dissemination of Achievements

At the end of the project timeframe, the EEPB should plan an international conference (regional) on ESCO EPC implementation in public buildings supported by a high quality EEPB Final Report with the aim of sharing its experience and raising the awareness of the donor community. Other countries in the region where projects related to ESCO/EPC have been implemented should be made to participate. Such a requirement has already been pointed out in the Project Document, but the Evaluator would like to stress again the importance of the regional information dissemination based on quality reporting.

Who:

The Project Manager is responsible for drafting the EEPB Final Project report. He technical and institutional extended knowledge will be valuable for strengthening the technical, financial and organizational priority topics to be highlighted in the report. On the other hand, with the aim of issuing a high-quality report, the EEPB should also involve a professional editor.

When:

The Final report should be prepared about three months prior to the ESCO Ukraine international Conference.

4

Implementation of a series of ESCO EMIS&C pilot projects

The Project must take action to implement a series of EMIS&C[1] pilot projects. The EMIS&C pilot projects aim at demonstrating the performance, in terms of energy savings, resulting in a better control of energy consumption and demand. By demonstrating the relevance of energy management and control the EEPB project should speed up the extension of the EMIS-Database and later on the replication of EMIS&C in a large spread of municipalities (countrywide) with the aim of reaching an annual GHG emissions reduction of 5,000 tons/yr rather than 1,238 t/yr which is the current achieved rate by the EEPB.

To this end, the EEPB should implement a series of 10 pilot ESCO EMIS&C projects and additional training sessions related the operation of the energy control equipment (black box), 

where new metering and control equipment will have been installed in selected public buildings.

As a first step, the EEPB project should select up to 10 buildings, (5 buildings preferably in a single large city, e.g. Odesa, and other 5 building in other cities) and proceed with the required FS and baseline analysis. These 10 ESCO EMIS&C pilot projects should not be combined with any EE measures that could be implemented later on by an ESCO. The aim of such Pilot Projects is to find out the actual energy savings resulting directly from the implementation of the EMIS&C. The cost of equipment should be paid for by the EEPB, as a grant. Such a series of EMIS&C projects and the information dissemination of results should help to remove any existing barriers at the city level and also aid in promoting the involvement of Energy Managers.

At midterm, the EMIS (local) works in 10 cities (498 buildings) and 2 regions (25 small cities, 23 villages), which includes the installation of meters in public buildings. As a result, 56% of energy savings (EEPB) came from the EMIS already implemented and 44% from pilot and replicated EE projects. It is an important lesson that is to be learned.

EMIS equipped with the needed electronic device allows the building manager to get the real-time data and take the appropriate action for a better control of demand and consumption. When the EMIS is supported by a control system (EMIS&C) at the building level, the benefit in term of energy saving is estimated at 5-10% a year, and perhaps more. Such energy savings are reached through a low-cost investment.

The Evaluator recommends implementing at least 10 EMIS&C pilot projects ASAP, and proceeding with an outreach campaign for disseminating the positive and proven result spread on a period of one year. The EMIS&C pilot projects could be appropriate for testing an incentive mechanism for the municipal Energy Manager.

 

1. Recommendation:

New FSM and Project Financing Enhancement with the aim of establishing a relevant and efficient FSM on a long-term basis, the EEPB project should make an arrangement to work alongside the SAEE at the design stage of the Feasibility Study of the ‘’Green Fund’’, which was envisaged to finance EE activities in various economic sectors. The Government of Finland expects to finance the Feasibility Study. The main purpose of the recommended involvement is to make sure that the likely ‘’Green Fund’’ will have a built-in component to deal with ESCO/EPC financing needs. To this end, the EEPB should provide the financial TA for to the SAEE.

Who:

The International or National Expert with an extended understanding and experience in financial mechanisms dedicated to ESCOs, e.g.: Loan Guarantee Fund, Factoring, forfaiting[1] or Buyout mechanism. The Financial Mechanism Expert will closely work with the SAEE.

When and How:

Step 1: Sign a MoU with the SAEE for confirming their mutual interest and commitment.

Step 2: The EEPB will draft the National/International Expert’s ToRs in coordination with the SAEE. The expert should be jointly selected by the Project and the SAEE.

Step 3:  The Expert must have a workstation within the SAEE’s premise.

 

Management Response: [Added: 2020/04/22] [Last Updated: 2021/01/27]

Accepted

Key Actions:

Key Action Responsible DueDate Status Comments Documents
Sign a MoU with the SAEE for confirming their mutual interest and commitment.
[Added: 2020/04/22] [Last Updated: 2021/05/04]
UNDP CO, Project Manager 2021/09 Not Initiated Subject to: (i) re-confirmed interest of State Agency for Energy Efficiency after its restructuring in December 2019, (expected) in developing FS; and (ii) re-confirmed commitment of the Government of Finland. Acting Head of State Agency for Energy Efficiency was appointed on 2 July 2020. The MoU was not signed also due to temporary suspension of the Project activities since 26.11.2020. Therefore, UNDP did not develop FS of the Green Fund. It is envisaged to be completed by September 2021 in case of positive outcome of pre-start activities of the project after the suspension was lifted.. History
2. Recommendation:

Countrywide EMIS extension - Whilst not undervaluing the importance of ESCO development, the priority for the upcoming two or three years should be transferred to the development of the countrywide EMIS-Database.

The Evaluator recommends much more intensive support in the direction of SAEE and the Working Group, with the aim of outlining a detailed methodology and selecting the most appropriate web-based tool(s). Such a support is essential for the rollout of a platform for web-based data collection and analysis at a centralized level (SAEE).

In addition to the Working Group mentioned above, the Evaluator recommends setting up a sort of Task Force dedicated to the development and implementation of the countrywide EMIS from January 2020 to the end of the project timeframe.

In many municipalities some officials mentioned a barrier at the level of city council related to investment in EMIS. This recommendation aims at extending the EMIS to a broader spread of municipalities, countrywide, while simultaneously extending the EMIS&C at the local building level (ref.: Recommendation 4). For extending the EMIS to all cities, the project must continue and enhance its effort for enforcing a new secondary regulation to this end.

Who:

The EEPB already involved an international expert for dealing with the web-based tool. The expert should increase his involvement with the SAEE with the aim of accelerating the full rollout of the web-based EMIS Database. It has been done in many countries and the EEPB could learn more from the web-based EMIS in operation in the sub-region. In addition to a more intensive involvement of the international expert, the EEPB should involve a National Expert for supporting the reputable International Expert involved since 2018. The EMIS National Expert should also be responsible for meeting with municipalities, pushing them to embark on the EMIS and EMIS&C.

Management Response: [Added: 2020/04/22] [Last Updated: 2021/01/27]

Accepted

Key Actions:

Key Action Responsible DueDate Status Comments Documents
The EEPB project should establish the EMIS Support Unit (or Task-Force) in the SAEE. The national expert (to be recruited) and the international expert (Mr. Goran) should be intensively involved with the EMIS unit and along with the related SAEE’s staff members. The mandate of the EMIS Unit must be to finalise the EMIS data collection methodology and tools, and provide the training to energy managers at the municipal level and the central EMIS website Administrator. The rollout of the countrywide EMIS is critical because it is a major deliverable of the EEPB.
[Added: 2020/04/22]
Project Team 2022/12 Not Initiated
3. Recommendation:

Strengthen Awareness and Dissemination of Achievements

At the end of the project timeframe, the EEPB should plan an international conference (regional) on ESCO EPC implementation in public buildings supported by a high quality EEPB Final Report with the aim of sharing its experience and raising the awareness of the donor community. Other countries in the region where projects related to ESCO/EPC have been implemented should be made to participate. Such a requirement has already been pointed out in the Project Document, but the Evaluator would like to stress again the importance of the regional information dissemination based on quality reporting.

Who:

The Project Manager is responsible for drafting the EEPB Final Project report. He technical and institutional extended knowledge will be valuable for strengthening the technical, financial and organizational priority topics to be highlighted in the report. On the other hand, with the aim of issuing a high-quality report, the EEPB should also involve a professional editor.

When:

The Final report should be prepared about three months prior to the ESCO Ukraine international Conference.

Management Response: [Added: 2020/04/22] [Last Updated: 2021/01/27]

Accepted

Key Actions:

Key Action Responsible DueDate Status Comments Documents
At the end of the project timeframe, the EEPB should plan an international conference (regional) on ESCO EPC implementation in public buildings supported by a high quality EEPB Final Report with the aim of sharing its experience and raising the awareness of the donor community.
[Added: 2020/04/22]
Project Team 2020/12 Overdue-Not Initiated
4. Recommendation:

Implementation of a series of ESCO EMIS&C pilot projects

The Project must take action to implement a series of EMIS&C[1] pilot projects. The EMIS&C pilot projects aim at demonstrating the performance, in terms of energy savings, resulting in a better control of energy consumption and demand. By demonstrating the relevance of energy management and control the EEPB project should speed up the extension of the EMIS-Database and later on the replication of EMIS&C in a large spread of municipalities (countrywide) with the aim of reaching an annual GHG emissions reduction of 5,000 tons/yr rather than 1,238 t/yr which is the current achieved rate by the EEPB.

To this end, the EEPB should implement a series of 10 pilot ESCO EMIS&C projects and additional training sessions related the operation of the energy control equipment (black box), 

where new metering and control equipment will have been installed in selected public buildings.

As a first step, the EEPB project should select up to 10 buildings, (5 buildings preferably in a single large city, e.g. Odesa, and other 5 building in other cities) and proceed with the required FS and baseline analysis. These 10 ESCO EMIS&C pilot projects should not be combined with any EE measures that could be implemented later on by an ESCO. The aim of such Pilot Projects is to find out the actual energy savings resulting directly from the implementation of the EMIS&C. The cost of equipment should be paid for by the EEPB, as a grant. Such a series of EMIS&C projects and the information dissemination of results should help to remove any existing barriers at the city level and also aid in promoting the involvement of Energy Managers.

At midterm, the EMIS (local) works in 10 cities (498 buildings) and 2 regions (25 small cities, 23 villages), which includes the installation of meters in public buildings. As a result, 56% of energy savings (EEPB) came from the EMIS already implemented and 44% from pilot and replicated EE projects. It is an important lesson that is to be learned.

EMIS equipped with the needed electronic device allows the building manager to get the real-time data and take the appropriate action for a better control of demand and consumption. When the EMIS is supported by a control system (EMIS&C) at the building level, the benefit in term of energy saving is estimated at 5-10% a year, and perhaps more. Such energy savings are reached through a low-cost investment.

The Evaluator recommends implementing at least 10 EMIS&C pilot projects ASAP, and proceeding with an outreach campaign for disseminating the positive and proven result spread on a period of one year. The EMIS&C pilot projects could be appropriate for testing an incentive mechanism for the municipal Energy Manager.

 

Management Response: [Added: 2020/04/22] [Last Updated: 2021/01/27]

Accepted

Key Actions:

Key Action Responsible DueDate Status Comments Documents
10 EMIS&C Pilot Projects should be implemented and fully operational
[Added: 2020/04/22]
Project Team 2020/12 Overdue-Not Initiated
ESCO will proceed with energy management during the initial year. The ESCO will be required to submit the final report at the end of energy management period and will transfer the energy management task to the local Energy Manager.
[Added: 2020/04/22]
Project Team 2020/12 Overdue-Not Initiated
The EEPB will organize a national EMIS Database/EMIS&C national conference in July 2021 for disseminating the result and promoting its replication in other cities.
[Added: 2020/04/22] [Last Updated: 2021/05/10]
Project Team 2021/10 Not Initiated Rescheduled due to project suspension. History

Latest Evaluations

Contact us

1 UN Plaza
DC1-20th Floor
New York, NY 10017
Tel. +1 646 781 4200
Fax. +1 646 781 4213
erc.support@undp.org