ESCO Moldova - Transforming the market for Urban Energy Efficiency in Moldova by introducing Energy Service Companies

Report Cover Image
Evaluation Plan:
2018-2022, Moldova
Evaluation Type:
Final Project
Planned End Date:
11/2018
Completion Date:
11/2018
Status:
Completed
Management Response:
Yes
Evaluation Budget(US $):
22,000

Share

Document Type Language Size Status Downloads
Download document TOR - ESCO TE - International_2018.pdf tor English 465.41 KB Posted 841
Download document TOR - ESCO TE - National_2018.pdf tor English 442.98 KB Posted 581
Download document UNDP-GEF_ESCO_Moldova_TE_Final_17.12.2018-TCh_20.12.2018.pdf report English 911.25 KB Posted 723
Title ESCO Moldova - Transforming the market for Urban Energy Efficiency in Moldova by introducing Energy Service Companies
Atlas Project Number: 00079687
Evaluation Plan: 2018-2022, Moldova
Evaluation Type: Final Project
Status: Completed
Completion Date: 11/2018
Planned End Date: 11/2018
Management Response: Yes
UNDP Signature Solution:
  • 1. Energy
Corporate Outcome and Output (UNDP Strategic Plan 2018-2021)
  • 1. Output 1.4.1 Solutions scaled up for sustainable management of natural resources, including sustainable commodities and green and inclusive value chains
  • 2. Output 1.5.1 Solutions adopted to achieve universal access to clean, affordable and sustainable energy
  • 3. Output 2.3.1 Data and risk-informed development policies, plans, systems and financing incorporate integrated and gender-responsive solutions to reduce disaster risks, enable climate change adaptation and mitigation, and prevent risk of conflict
SDG Goal
  • Goal 7. Ensure access to affordable, reliable, sustainable and modern energy for all
SDG Target
  • 7.1 By 2030, ensure universal access to affordable, reliable and modern energy services
Evaluation Budget(US $): 22,000
Source of Funding: Project (donor) funds / GEF
Evaluation Expenditure(US $): 18,720
Joint Programme: No
Joint Evaluation: No
Evaluation Team members:
Name Title Email Nationality
Paata Janelidze Team Leader
Veaceslav Vladicescu Team Member
GEF Evaluation: Yes
GEF Project Title: ESCO Moldova - Transforming the market for Urban Energy Efficiency in Moldova by introducing Energy Service Companies
Evaluation Type: Terminal Evaluation
Focal Area: Climate Change
Project Type: MSP
GEF Phase: GEF-5
GEF Project ID: 5157
PIMS Number: 5135
Key Stakeholders: Ministry of Economy of Republic of Moldova
Countries: MOLDOVA, REPUBLIC OF
Lessons
Findings
1.

3. Findings

(As requested by the ToR, in addition to a descriptive assessment, all criteria marked with (*) must be rated)

3.1 Project Design / Formulation 

As recommended by the Guidance for Conducting Terminal Evaluations of UNDP-Supported, GEF Financed Projects the findings of this chapter are based on the analysis whether or not: - The ESCO Moldova Project objectives and components were clear, practicable and feasible within its time frame - The capacities of the Executing Agency and its counterparts were properly considered when the project was designed - Lessons from other relevant projects (if any) were properly incorporated in the project design - The partnership arrangements were properly identified, and roles and responsibilities negotiated prior to project approval - Counterpart resources (funding, staff, and facilities), enabling legislation, and adequate project management arrangements were in place at project entry - The project assumptions and risks were well-articulated in the ProDoc.

An additional important point in terms of project formulation is to consider whether the planned outcomes were "SMART" (S - Specific: Outcomes must use change language, describing a specific future condition; M - Measurable: Results, whether quantitative or qualitative, must have measurable indicators, making it possible to assess whether they were achieved or not; A - Achievable: Results must be within the capacity of the partners to achieve; R - Relevant: Results must make a contribution to selected priorities of the national development framework; T - Time- bound: Results are never open-ended. There should be an expected date of accomplishment).

Project objectives

The ESCO Moldova Project is focused on creation of a functioning, sustainable and effective ESCO market for scaling up mitigation efforts in the whole municipal building sector in Moldova in line with the GUDP. This goal was supposed to be achieved through the implementation of four components: (i) GUDP for Chisinau; (ii) Creation and operation of ESCOs; (iii) ESCO market operation; and (iv) Replication and dissemination. The ESCO Moldova Project thus has been designed to provide technical assistance and investment in demonstration activities and thereby reduce existing financial, technical, institutional and awareness barriers. As stated in the ProDoc, the ESCO Moldova Project supports a broader ESCO market development approach, as opposed to the original concept (in PIF), which supported the creation of a single company, a public private partnership (PPP), which would act as a catalyst to the development of the ESCO market. The rationale for this was that neither the City of Chisinau no commercial banks or financial institutions were willing to invest or finance in the PPP, and all ESPs met during the preparatory phase were not interested in partnership under the PPP. Therefore, more ambitious objective has been identified. However, the implementation of the ESCO Moldova Project clearly showed that ESPs are not interested in EPC modality as well, despite the awareness about ESCO model capacity building. The political and financial risks has driven to uncertainties of practical matter. Companies, willing to embark on the EPC model were cautious and were reluctant to make the first step, because of the multiple bank frauds and corruption scandals registered in the energy efficiency sector in 2014-2017. It has influenced at the end also the potential beneficiaries, making them resistant to the new financing modality. As soon as the business was considered the professional service provider, and which could not convince the public sector of the success of the model, the latter reacted consequently.

The project team was the only ambassador of ESCO model and its benefits, which was however not sufficient enough to play the role of ESCO driver and pro-active information. In order to find the large number of credible investment proposals there was a need of a larger team, that would mostly work in the field, identify willing partners, identify possible energy efficiency investments and constantly oversight the level of partners’ empowerment to implement the project, rather than focus only on desk work analysis. For this purpose, an expanded multi-skilled specialists project team (engineer, finance expert, communication, community engagement, etc.) should have been designed in ProDoc. The ProDoc concluded that the Ministry of Economy, with the support of USAID financed SYNENERGY, developed the draft regulation on energy services in the public sector and created a basis for the removal of legislative and non-legislative barriers to the implementation of EPCs in Moldova. Therefore, no outcome was planned related to the legal and regulatory framework. The assumption that the legislative barriers will be removed didn’t materialized and lack of legal provisions for operating of ESCO Market created issues for project implementation. Project didn’t envisaged work on supporting the Government policies to support ESCO activities, including regulations, economic incentives, information policies, etc. The fact that even at the EU level there were different interpretations on recording of Energy Performance Contracts (EPCs) in government accounts until September 2017, when Eurostat guidance notice was issued, shows how uncertain was to reach the objectives for ESCO Moldova Project. Not mentioning that the necessary national legal provisions for EPC operationalization ( new Law on Energy Efficiency and Law on Condominium) were passed by Moldovan Government in late 2018. 


Tag: Energy Relevance Global Environment Facility fund Private Sector Financing Programme/Project Design

2.

3.1 Project Design / Formulation (continuation)

3.1.1 Analysis of Logical Framework (Project logic /strategy; Indicators)

The Logical Framework (LogFrame) is a key basis for planning of detailed activities under the implementation framework that was defined in the ProDoc. The LogFrame shall in principle serve to monitor and evaluate the overall project achievements – based on defined targets and indicators to measure these targets. The LogFrame, at some level, is lacking internal logic, clarity and consistency; planned activities are not always appropriate and/or adequate towards the identified targets; targets are not consistent throughout the ProDoc. For instance:

- Project Objective: in the Log-frame, the GHG reduction targets are confusing. Indeed, the target for cumulative (2014-2038) direct reductions is 68 ktons of CO2, which is consistent with the Appendix 2: Calculation of GHG Emissions Reductions (in Table 11 direct project emission reductions during 2014-2018 equals to 6.28 ktons and direct post-project reductions during 2018-2038 – 61.6 ktons), i.e. the lifetime of investments is more than 20 years. If correctly calculated, the cumulative target for 20-year lifetime should be 3.08 (annual GHG savings) x 20 = 61.6 ktons of CO2. Further, it is assumed that TA provided by the ESCO Moldova Project to Chisinau City could include feasibility analyses and replication of building retrofitting design and that a number of additional EE retrofit projects would be carried out by ESCOs. Those projects were supposed to achieve the same savings per EPC as demo projects implemented during the ESCO Moldova Project timeframe. Based on these assumptions the direct postproject emission reductions were calculated in the Appendix 2, as 3.08 x 15 = 46 ktons of CO2 (40 ktons in the LogFrame). It must be noted that firstly, such emission reductions can be accounted more to consequential (formerly determined as indirect) reductions rather than direct ones; and secondly, it is unclear, where the 15-year period comes from. Indirect emissions in the Appendix 2 are estimated as 694 ktons (bottom-up approach) and 278 ktons (Top-down approach) while in the LogFrame target for indirect emissions equal to 240 ktons of CO2. And finally, it is unclear what is meant under the “Total emissions” (381 ktons of CO2 as per the LogFrame).

- Outcome 1: as mentioned above, this outcome doesn’t look as an integral part of the ESCO Moldova Project. As stated in the MTR report, the consultant suggested (and UNDP comments on the draft of this report concurred) that reference to a GUDP for Chisinau should be removed from the logframe. Instead, the MTR recommended (recommendation 4) to consider a GUDP for a city other than Chisinau as part of Component 1. However, this recommendation hasn’t been followed for the same reason: Outcome 1 is not strongly related to other outcomes and requires more project resources than envisaged in the ProDoc. 

- Output 2.4: EE projects implemented using the EPC modality, clearly refers to the completed implementation and thus the established targets, Target 1: 20 EE projects selected and documented, Target 2: 20 EE projects using EPC modality are under implementation, are inadequate

- LGF related outputs (Outputs 3.5: LGF Management and Accountability Arrangements; Output 3.6: LGF Operations Monitoring and Reporting mechanism; and Output 3.7: LGF Exit Strategy and New Regulation Framework): there are two targets established, (i) number of Loan Guarantee approvals (20 in total out of which 5 in year 1; 10 in year 2; and 5 in year 3); and (ii) size of LGF and loans (at least USF 2.7 million). It seems not realistic to expect 5 loan guarantee approvals in one year, when there are no ESCOs existing, no energy audits conducted, no FIs identified for LGF, etc. 

In general, it must be noted that there are a large number of indicators and targets established for the planned outcomes; some of them might be not included in the LogFrame (e.g. selection of 20 projects and then implementation of the same projects, are established as separate targets). The original LogFrame has not been updated during the inception phase; the MTR of the ESCO Moldova Project didn’t recommend any changes in the LogFrame. While analyzing LogFrame the MTR proposed to use two more SMART indicators: (i) Number of known non-Project EPCs taking place in Moldova; and (ii) Estimated value of non-Project EPCs taking place in Moldova. However, this wasn’t included into the recommendations. With the above mentioned log frame design inconsistences on outputs and indicators, it was of outmost importance to pay a better attention to project implementation and exert the risk management functions. 


Tag: Energy Project and Programme management Results-Based Management Risk Management Theory of Change

3.

3.1 Project Design / Formulation (continuation)

3.1.3 Lessons from other relevant projects incorporated into project design Before the ESCO Moldova Project start some donor-funded projects with the objectives, related to EEin buildings, were either completed or under the implementation. Among them: - UNIDO-implemented, GEF-financed project Reducing Greenhouse Gas Emissions through improved Energy Efficiency in the Industrial Sector in Moldova (2011-2013) - EBRD’s two sustainable energy financing facilities (MoSEFF and MoREEFF, implemented during 2012-2017), under which credit lines to local banks were created for on-lending to corporate and residential borrowers for sustainable energy investments. It is also stated in the ProDoc that the municipal sector already takes advantage of those facilities for majorretrofitting of public buildings, including EE components - GIZ-funded project Modernization of Local Public Services in the Republic of Moldova, one of the focus of which was on investment attraction in EE in public buildings. There ProDoc dos not provide how the lessons learned (LL) from the above projects were considered in the design of the ESCO Moldova Project.

3.1.4 Planned stakeholder participation Key stakeholders of the ESCO Moldova Project, including governmental agencies and ministries, and first of all, the Ministry of Environment as the Executing Agency of the project, have been identified during the design phase. Planning of the stakeholder participation has started from the early stages of the ESCO Moldova Project development. The planning was based on clear understanding of the features of Moldova regarding the building sector, namely roles played by key institutions. Based on the abovementioned text it is the TE Team’s opinion that stakeholder participation has been planned adequately. Another key stakeholder in the project should have been the EBRD who have lots of activities in Moldova on green urban development and on funds for financing for energy efficiency in public buildings. There is no evidence that the project actively engaged with the EBRD. Another key stakeholder in the project were ESPs or Energy Service Providers who were expected to transform into ESCOs. Aside from their participation in the training workshops on ESCO market development, there is no evidence to suggest that ESPs were strongly involved in the implementation of this project.


Tag: Effectiveness Global Environment Facility fund Strategic Positioning

4.

3.1 Project Design / Formulation (continuation)

3.1.7 Linkages between project and other interventions within the sector 

As mentioned above, there are a number of projects in Moldova, with the similar objectives, either already implemented or being under the implementation. ESCO Moldova Project design calls for cooperation with those activities in the sector, to consider their lessons and avoid overlapping/duplication of activities. However, the linkages with them are fragmental. The ProDoc does not really provide details about the linkages and co-operation opportunities with other interventions within the sector nor does explain how the ESCO Moldova Project complements the results of the earlier projects. During the implementation, there were week linkages between the ESCO Moldova Project and other ongoing interventions (e.g. with the EBRD-funded MoREEFF)


Tag: Emission Reduction Energy Global Environment Facility fund Human and Financial resources Implementation Modality Programme Synergy Programme/Project Design Project and Programme management Results-Based Management Country Government Coordination

5.

3.2 Project Implementation

As recommended by the Guidance for Conducting Terminal Evaluations of UNDP-Supported, GEF Financed Projects, TE findings in this chapter are based on assessment of implementation approach, whether or not: (i) The logical framework is used during implementation as a management and M&E tool; (ii) Effective partnerships arrangements are established for implementation of the project with relevant stakeholders involved; (iii) Lessons from other relevant projects are incorporated into project implementation; and (iv) Feedback from M&E activities used for adaptive management.

3.2.1 Adaptive management (changes to the project design and project outputs during implementation) The implementation of the ESCO Moldova Project can be divided into two periods, before and after the MTR. Before the MTR (March 2017) the PMU (Project Manager) was in a position that all planned activities of ProDoc are logical and feasible; the ESCO Moldova Project is on track and there were no critical risks, which would decrease the likelihood of achievement of the planned outcomes. The reports prepared by the International Consultants (L.P.Lavoie, K.Konstantinou, T.Crawshaw) also didn’t flag issues, addressing of which would require application of the adaptive management with the exception when the Consultant requested adjustments to the financial mechanism (Loan Guarantee Fund) because of not willingness of the commercial banks to provide loans (and in this case the LGF was becoming useless). The change of the Board Chair (Ministry of Economy instead of the Ministry of Environment), initiated by the ESCO Moldova Project, can be considered as the first application of the adaptive management, which really improved effectiveness of the Board. 

Under the Component 1, which was aimed at development and adoption of Green Urban Development Plan (GUDP) of Chisinau, it was initially planned, to take the existing UDP as a basis, add “green” elements and arrive at a Green UDP. Instead, the adaptive management has been applied and a report on essential elements of a Green UDP has been prepared. The Project Board and Chisinau Municipality have both confirmed in writing that they are satisfied with this outcome. Nevertheless, Outcome 1 per se hasn’t been achieved and thus, it cannot be counted as a proper application of adaptive management. At the same, it must be noted that there were issues beyond the control of the ESCO Moldova project, which didn’t allow to achieve the target (for instance, the position of Chisinau municipality: to have updated UDP by integrating “green elements”, rather than developing a new GUDP). In PIR 2016 the PM reported that LGF couldn’t been put with the local banks and thus, the decision was made (adaptive management) for its location within the Energy Efficiency Fund and make USD 900,000 available for this purpose. In fact, this appeared not a right decision because the EEF took on the triple-role of grantor, lender and guarantor to enable private sector ESCOs to invest in municipal EE projects using Energy Performance Contracts (EPC). The Project Document does not specify whether the pilot project should be residential or public buildings (only in Appendix 3, 14 public and 6 residential buildings are mentioned), and therefore, the ESCO Moldova Project has focused only on public sector energy efficiency investments. This represents a sensible adaptive management considering the Moldovan circumstances (underheating due to the low affordability).

According to the MTR report, Project was very well-managed, has used adaptive management intelligently to overcome major hurdles and has credible proposals for recovery and therefore, Project implementation and Adaptive Management was rated as Satisfactory (S). However, given that the project fell into crisis only a few months after the MTR one can only conclude that the MTR was quite optimistic and that it took place at a time in late 2016 and early 2017 before the real problems of the project were fully known. At the same time, the MTR underlined a need of the extensive application of the adaptive management in the future. In response to this challenge the UNDP CO has initiated the adaptive management procedure which consisted of: (i) MTR Assessment; and (ii) Revision of the ESCO Moldova Project design and strategy following the corruption scandals of April 2017. The MTR Assessment analyzed the extent to which recommendations from the MTR have been considered, and the extent to which the recommendations are still relevant in 2018. In the Revised Design and Strategy report different options for adaptive management were proposed, but none of them were selected by the UNDP Moldova CO Management due to not proven commitments (on co-financing of the pilot projects), from the project partners’ side and diverging opinion between UNDP-GEF RTA, Moldova CO and national partners. The decision was grounded also on existing country implementation context challenges, such as unstable political and financial situation, weak institutional capacities of EEF and EEA, as well as immature capital markets. Adopting a strategy option for further project implementation and extension of project by 2020 ( 1,5 years in addition), was considered as not opportune, given the administrative reform of energy institutions in 2018-2019 and upcoming parliamentary and City Hall elections in 2019. Bearing in mind this risks, UNDP has requested reimbursement of LGF amounts, transferred in 2015 to EEF and go ahead with project terminal evaluation, which would allow to close the project by the end of 2018. Therefore, not too much have been achieved after the MTR and none of the adaptive management options as per the Report #2 “Revised Project Design and Strategy” was practically not applied.


Tag: Energy Green Climate Green Economy Global Environment Facility fund Human and Financial resources Implementation Modality Monitoring and Evaluation Partnership Policies & Procedures Project and Programme management Urbanization South-South Cooperation

6.

3.2 Project Implementation (continuation)

1.1.1 Project Finance

For the evaluation of ESCO Moldova Project finance, the key financial aspects of the actual costs and leveraged and financing have been assessed. Differences between planned and actual expenditures also were assessed and explained: Table 1: Planned and Actual Co-financing (in USD million).

Since no pilot project has been implemented, no grant was provided by the partner Agency (EEF); no and co-financing was provided by the Municipality of Chisinau. Financial oversight of the project is provided by UNDP under the National Implementation Modality (NIM). Combined Delivery Reports (CDRs) have been prepared thoroughly, on a timely basis, and in a manner consistent with regulations on financial reporting. The annual disbursements amounted to: - USD 53,365 in 2018 - USD 42,674 in 2017 - USD 134,693 in 2016n  - USD 1,094,1198 in 2015 - USD 10,492 in 2014 - Total (as of 31 July 2018): USD 435,343 (considering that USD 900,000 has been returned by the EEF)

1.1.2 Monitoring and evaluation: design at entry and implementation

(*) M&E Design at Entry The ProDoc among other includes description of the budgeted Monitoring and Evaluation (M&E) plan with identified responsible parties for M&E activities, allocated indicative budget, and specified time  frame for each M&E activity. According to M&E plan, M&E should be conducted in accordance with established UNDP and GEF procedures. The indicative M&E budget was USD 51,400 or about 4% of the total GEF grant. M&E activities among others include measurement of means of verification for project indicators and measurement of means of verification for project progress and performance (measured on an annual basis), etc. Standard M&E tools include LogFrame (contains performance and impact indicators as well as means of verification), Inception Report, Mid-Term Review, Terminal Evaluation as well as standard UNDP and GEF project progress reports. At the same time, as mentioned above in Chapter 3.1.1, the original LogFrame, at the certain level, was lacking internal logic and consistency and therefore, some indicators and targets had to be redefined to better and more specifically reflect project outputs. Based on the above mentioned the M&E design at project start up is rated as Moderately Satisfactory (MS).

Implementation of M&E

In general, the actual implementation of M&E is in compliance with the M&E plan, because: - The ESCO Moldova Project is subject to regular review of the UNDP CO and has been supervised regularly - Project implementation has been reviewed by the PB. AWPs have been regularly developed and submitted for approval to the PB. Four meetings of the PB were organized in total (No.1: 30.03. 2015; No.2: 31.05.2016; No.3: 28.12.2016; No.4: 07.03.2017). At each PSC meeting the achievements to date were discussed, assessed and agreed. The PB should play a critical role in M&E by quality assurance of the activities and outputs; it should ensure that required resources are committed and negotiates solutions to any problems with external parties. Unfortunately, the PB was unable to provide adequate guidance to ensure the above mentioned. - Inception Workshop was held on April 1, 2015 with participation of UNDP CO, UNDP RTA, National Implementing Partner and key stakeholders. - The MTR mission was conducted in November 2016, MTR report delivered in March 2017. 

Evaluation of the monitoring results was not always adequate. For instance: - Overall Project Achievement and Impact was rated by the MTR as Moderately Unsatisfactory (MU). Rating for project design & strategy was Moderately Satisfactory (MS); Project Objective: Indicator 1: Satisfactory (S); Indicator 2: not rated; Indicator 3: S; Indicator 4: MS; Outcome 1: Unsatisfactory (U); Outcome 2 – S; Outcome 3 – Highly Satisfactory (HS); Outcome 4: MS; Project implementation and adaptive management – S; Sustainability – MU. These ratings seem not always logical. Indeed, if the achievement of the Objective was satisfactory and of outcomes satisfactory on average (except of Outcome 1), then it is not clear why the sustainability was rated as unsatisfactory. In addition, the above mentioned Adaptive Management Review, Report #2 - Revised Project Design and Strategy, which was produced by the MTR reviewer just a year after the MTR, concluded: The Project today - no strategy, plenty of budget, not enough time. - Project Manager and partially, UNDP CO were overestimating progress in achievement of development objective and implementation progress, especially before the MTR. In Table 2 ratings by the Project manager, CO and RTA are sourced from PIRs. Table 2: Ratings for the Progress of Development Objective and Implementation


Tag: Energy Efficiency Government Cost-sharing Monitoring and Evaluation

7.

3.2 Project Implementation (continuation)

1.1.3 UNDP and Implementing Partner implementation / execution coordination, and operational issues (*)

UNDP (Implementing Agency) implementation 

This ESCO Moldova Project was implemented under the National Implementation Modality (NIM) and thus the role of UNDP in the implementation is not as high as under the Direct Implementation Modality (DIM). Nevertheless, a number of direct responsibilities of the UNDP was described in the ProDoc and first of all, UNDP (along with RTA and PM) was responsible for Measurement of Means of Verification of project results. The key aspects of the UNDP implementation are as follows: 

- UNDP was continuously looking at whether the ESCO Moldova Project is being implemented based on the Results Based Management with focus on established targets

- UNDP CO has been conscientious since the project started on national context and the existing barriers for implementing ESCO Moldova project though EPC modality and has constantly addressed them in the risk logs, PIRs, etc. UNDP has started to address the identified and/or updated risks through multiple meetings with project stakeholders (Ministry of Finance, Ministry of Economy, EEF, etc.), building the capacities of potential ESCO companies, as well as addressing them at the PB meetings and requesting the Board the guidance and support.

- UNDP developed the Management Response to the MTR recommendations; it took practical steps to address them. When it became obvious that there is a high likelihood that the planned outcomes wouldn’t be achieved (After the MTR and PIR 2017), an International Consultant on adaptive management was hired in late 2017 and worked on adjusting of the implementation strategy. Unfortunately, it was too late to change the situation (there was no evidenced indication for that), and after reviewing the proposed implementation strategy, UNDP CO correctly concluded that it would not work even in case of the no-cost extension of the ESCO Moldova Project and took the decision in early 2018 not to ask for the extension of the ESCO Moldova Project duration beyond November 2018.

UNDP CO, RTA regularly were warning project manager about the delays, worsened implementation environment and high risks. For instance, in PIR 2017 the RTA stated that “the external situation (Mayor left, problems with EEF) have unfortunately had a major impact on the implementation progress of the project. Situation is difficult due mainly to the changes that are taking place in both the EEF and City of Chisinau and a reluctance of private companies to work with them given current scandals. This requires innovative thinking and strong adaptive management”. There were frequent changes in the UNDP CO environment team during the lifetime of the project (3 environment team leads in 4 years), which at the certain level might affect the effective monitoring and evaluation from the UNDP side.


Tag: Human and Financial resources Implementation Modality Monitoring and Evaluation Operational Efficiency Project and Programme management Results-Based Management Coordination

8.

1.2 Project Results

In this chapter ESCO Moldova Project results including direct project outputs, short- to medium-term outcomes, and longer-term impact including global environmental benefits, replication effects and other local effects are evaluated. The detailed milestones of the ESCO Moldova Project, are as follows: 14.03.2013 - ESCO Moldova preparation grant and concept note approved by GEF 15.08.2014 - ESCO Moldova project approved for implementation by GEF 21.08.2014 - UNDP-GEF ESCO Moldova Local Project Appraisal Committee (LPAC). 22.09.2014 - UNDP started recruitment processes to staff the Project 28.11.2014 - UNDP-GEF ESCO Moldova Project start date (duration 4 years) December 2014 - Moldovan banking fraud crisis - three banks lost 1 billion USD. 13.03.2015 - UNDP-GEF ESCO Moldova call for a International Green Urban Development Consultant 29.03.2015 - 1st mission of STE Konstantinos KONSTANTINOU for stakeholder training 29.03.2015 - 1st mission of STE Louis-Philippe LAVOIE, Outcome 2 and stakeholder training 01.04.2015 - Official launch of UNDP-GEF ESCO Moldova (Inception workshop) 09.04.2015 - LGF Operational Guidelines issued (by L-P.Lavoie) 30.03.2015 - First Project Board Meeting 23.06.2015 - STE Louis-Philippe LAVOIE - ESCO training for candidate ESCO, AEE and FEE. 27.07.2015 - STE Konstantinos KONSTANTINOU - two days ESCO training for stakeholders 08.11.2015 - STE company ENVIROS conducted two days of ESCO training for stakeholders 26.11.2015 - EEF issues a technical proposal to UNDP to manage the $900,000 GEF Loan Guarantee Fund and to finance selected EE projects in Chisinau. 17.12.2015 - ESCO-FEE LGF agreement signed, contract value 900,000 USD, guarantees for 20 projects, due date 31.12.2018. 18.12.2015 - (or shortly thereafter) 900,000 USD transferred to EEF. 31.05.2016 - Second Project Board Meeting (8 Board members + Project team and others) 01.08.2016 - UNDP GEF ESCO Moldova contracts ENVIROS as consultant facilitator for 20 EPC contracts for 440 days to November 2017 (3x international and 3x local experts). Hence there were two ENVIROS contracts - the first as trainers, the second as facilitators. 28.11.2016 - ENVIROS international event for ESCOs (within a wider energy event) 07.03.2017 - MTR issued 26.02.2018 - Adaptive Management Review (AMR), Report #1 - Assessment of the Mid-Term Review, issued March 2018 - Adaptive Management Review (AMR), Report #2 – revised Project Strategy, issued

The following publications have been prepared by the ESCO Moldova Project:

- Review of all national and municipal energy efficiency programs and action plans, ongoing programs, green development trends and key barriers to green growth economic instruments implementation - Gap Analysis Report - The Status of the Current Urban Development Plan for the City of Chisinau in Relation to Green Urban Development - Chisinau General Urban Plan. A Practice Guide to Green Urban Development Planning - Sustainable Energy Action Plan. Municipality of Chisinau - Support the ESCO Market Development in Moldova. Methodology Outline - Support the ESCO Market Development in Moldova. Institutional Recommendations towards WSCO Development - Support the ESCO Market Development in Moldova. Methodology Outline on training needs assessment for the project target groups - Capacity Building and Training for Developing of ESCO Market in Moldova. Final Report by ENVIROS


Tag: Energy Effectiveness Efficiency Resource mobilization Results-Based Management Country Government Private Sector Capacity Building

9.

1.2 Project Results (continuation)

1.2.2 Relevance (*)

Relevance of the problem addressed by the ESCO Moldova Project is already demonstrated above in introduction to Chapter 2 and Chapter 3.1. During the TE mission the TE Team obtained evidence on that. After the TE mission the TE Team has spent significant time (including on additional interviews of the International Consultants involved) to understand whether or not the project strategy was relevant and provided the most effective route towards expected/intended results. Interviewed stakeholders underlined that the ESCO Moldova Project was highly relevant to the country. In fact, one of the changes attributed to the ESCO Moldova Project was the treatment of energy efficiency more broadly as a “hot topic” at the highest levels of government. Unfortunately, the ESCO Moldova Project couldn’t manage to achieve planned results. There were subjective reasons for that (inadequate managerial capacity, in some cases not fully cooperative partners) but there were also circumstances that prevented the achievement of the Overall Objective and Outcomes. Among them premature market conditions for the ESCO development needs to be mentioned. The situation in Moldova is that ESP companies are implementing building renovations under the “traditional” contracts but not EPC and they are not interested in getting paid from the energy savings unless there are many such companies at the market and due to the client prefers EPC option. Another issue is the low consumption in the baseline (at least, low pre-implementation consumption). Adequate addressing of these issues was a big challenge and required very strong results-based management in place with permanent application of the adaptive management due to the permanently changing implementation environment. Unfortunately, the ESCO Moldova Project couldn’t manage to offer acceptable solution to potential ESCOs and nether the Implementing Partner nor International Consultants involved could really assist in this. Based on the abovementioned the Relevance is rated as Relevant (R).

1.2.3 Effectiveness & Efficiency (*)

Effectiveness

The ESCO Moldova Project didn’t succeed to reach its overall Objective - to create a functioning, sustainable and effective ESCO market. The goal was ambitious and necessitated a strong partners commitment to be successful. Nevertheless it has greatly contributed to the awareness of ESCO model and launch the creation of ESCO market, be developing local capacity with a series of trainings and conducting energy audits. Due to UNDP testing of ESCO model were identified major barriers for ESCO market development in Moldova, and brought to MEI and MF attention, which in its turn have facilitated adoption of pending legal and regulatory frameworks for energy market ( EE Law and Condominium Laws adopted in November 2018). Even though the impact of ESCO project is very limited from the project objectives point of view, (implementation of EE projects in buildings), the efforts invested by UNDP project team have been catalytical for national market. A series of assessments, analysis, stakeholder consultations, collaboration mechanisms, as well as trainings and study tours helped understand the barriers and raise knowledge of relevant stakeholders As result of ESCO project the country now is better prepared for the 2nd try to establish an ESCO market. None of the ESCO Moldova Project’s Outcome has been fully achieved during project implementation timeline, neverthelessthe prospects for ESCO are rather positive, if considering the preparatory work done and legal framework approved in 2018 . Based on the above mentioned the Effectiveness is rated as Unsatisfactory (U).


Tag: Energy Effectiveness Efficiency Relevance Implementation Modality Integration Ownership Partnership Jobs and Livelihoods Urbanization

10.

1.2.6 Sustainability (*)

The ESCO Moldova Project has been designed to deliver sustainable impact in Moldova. As stated in the UNDP-GEF guideline for TE, sustainability is generally considered to be the likelihood of continued benefits after the project ends. Consequently, the assessment of sustainability considers the risks that are likely to affect the continuation of project outcomes.

Financial risks

Question13: What is the likelihood of financial and economic resources not being available once GEF grant assistance ends? (This might include funding through government - in the form of direct subsidies, or tax incentives, it may involve support from other donors, and also the private sector. The analysis could also point to macroeconomic factors.)? Answer: None of the planned outcomes has been achieved and thus the risk remains high and ESCOs still are not operating in the energy efficiency market in Moldova. At the same time, it must be noted that the LGF was supposed to be operational based on a guide/manual which would have included a strong M&E and sustainability plan; given that manual was not applied and LGF did not become operational it is very difficult to assess this risk. Therefore, the sustainability is rated as Unlikely (U).

Socio-economic risks

Question: Are there social or political risks that may threaten the sustainability of project outcomes? What is the risk for instance that the level of stakeholder ownership (including ownership by governments and other key stakeholders) will be insufficient to allow for the project outcomes/benefits to be sustained? Answer: None of the planned outcomes has been achieved and thus the risk remains high Question: Do the various key stakeholders see that it is in their interest that the project benefits continue to flow? Is there sufficient public/stakeholder awareness in support of the project’s longterm objectives? Answer: Most of the stakeholders understand the importance of improvement of EE in buildings. Key stakeholders have got very negative result and likely, will act differently in the future. Based on the above-mentioned the Socio-economic Risks are not high and the sustainability is rated as Moderately Unlikely (MU) 


Tag: Impact Sustainability Global Environment Facility fund Risk Management

Recommendations
1

UNDO CO to systematize and keep relevant materials related to the Green Urban Development Plan, developed by the ESCO Moldova project.

2

In the future, when the competitive market of the energy service providing companies will be created, the EE retrofits under the EPC likely will become the main modality (such trend is observed e.g. in Ukraine). Therefore, continuation of awareness raising campaign by the UNDP is recommended.

1. Recommendation:

UNDO CO to systematize and keep relevant materials related to the Green Urban Development Plan, developed by the ESCO Moldova project.

Management Response: [Added: 2019/01/16] [Last Updated: 2020/12/05]

Recommendation is well noted and agreed. The materials related to GUDP, are systemized in electronic files and available to:

  • projects implemented by UNDP CO in the area of  energy and environment infrastructure, as well as local development ( e.g. Green Cities)  
  • Municipality of Chisinau, Balti and other Local Public Authorities
  • interested donor institutions

Key Actions:

Key Action Responsible DueDate Status Comments Documents
1.1 UNDP/GEF Green Cities project will apply the Practice Guide to Green Urban Development Planning while developing the Sustainable Urban Mobility Plan
[Added: 2019/01/16] [Last Updated: 2021/01/25]
UNDP CO, Environment, Energy and Climate Change Cluster 2020/12 Completed The elaboration of Sustainable Urban Mobility Plan (SUMP) for Chisinau is being finalized. The Practice Guide has been taken into consideration and its parts integrated in the draft SUMP. History
1.2. UNDP CO will inform interested partner institutions in the energy sector and relevant people in Chisinau City Hall, including current Mayor, on (i) Practice Guide to Green Urban Development Planning and (ii) Public Green Procurement Guideline and advocate for their application
[Added: 2019/01/16] [Last Updated: 2020/02/07]
UNDP CO, Environment, Energy and Climate Change Cluster 2019/12 Completed The Green City Action Plan designed by EBRD and the National Program of investments in energy efficiency drafted by EIB have considered the Guides designed by the ESCO project. Moreover, UNDP has shared this documents with the new Chisinau Mayor (elected in November 2019) and new staff in the City Hall, to be considered as basis for developing the Sustainable development concept of the Chisinau Municipality. History
2. Recommendation:

In the future, when the competitive market of the energy service providing companies will be created, the EE retrofits under the EPC likely will become the main modality (such trend is observed e.g. in Ukraine). Therefore, continuation of awareness raising campaign by the UNDP is recommended.

Management Response: [Added: 2019/01/16] [Last Updated: 2020/12/05]

The recommendation is well noted and agreed. ESCO Moldova project had a catalytical role for setting the foundations and better prepare the energy market for EPC and ESCO and will continue to promote the awareness on new financial mechanisms in energy efficiency

Key Actions:

Key Action Responsible DueDate Status Comments Documents
2.1. ESCO project final report with lessons learned, as well as main findings of Terminal Evaluation report will be shared through UNDP coporate, individual country office and other web sites
[Added: 2019/01/16] [Last Updated: 2019/03/22]
UNDP CO, Environment, Energy and Climate Change Cluster 2019/03 Completed Final Report of the ESCO Project was made publicly on UNDP corporate and CO web-sites ( please see the links: https://erc.undp.org/evaluation/evaluations/detail/9655 and http://www.md.undp.org/content/dam/moldova/docs/ESCO%20Final%20Report%20with%20lessons%20learned_%2017.12.2018_.pdf ). Additionally the lessons learned on ESCO model implementation and its further potential at national level has been discussed with various donors and IFIs in Moldova (WB, EIB, NEFCO). History
2.2 Further networking with interested partner institutions on EPC and ESCO mechanism will be explored
[Added: 2019/01/16] [Last Updated: 2021/01/25]
UNDP CO, Environment, Energy and Climate Change Cluster 2020/12 Completed A study has been developed in 2020 by UNDP on the most appropriate financial mechanism to apply ESCO in residential sector, along with the draft Energy Performance Contract. The application of ESCO mechanism is being tested within 1 multistorey residential building in Chisinau within the Green City Project in 2021. If the pilot is successful, it will be promoted for scaling up at national level. History

Latest Evaluations

Contact us

1 UN Plaza
DC1-20th Floor
New York, NY 10017
Tel. +1 646 781 4200
Fax. +1 646 781 4213
erc.support@undp.org