Final Evaluation "Improving Energy Efficiency in Low Income Areas and Households of Romania"

Report Cover Image
Evaluation Plan:
2010-2015, Romania
Evaluation Type:
Final Project
Planned End Date:
06/2015
Completion Date:
07/2016
Status:
Completed
Management Response:
Yes
Evaluation Budget(US $):
30,000

Share

Document Type Language Size Status Downloads
Download document Terminal Evaluation EE Romania.pdf report English 1289.67 KB Posted 406
Title Final Evaluation "Improving Energy Efficiency in Low Income Areas and Households of Romania"
Atlas Project Number:
Evaluation Plan: 2010-2015, Romania
Evaluation Type: Final Project
Status: Completed
Completion Date: 07/2016
Planned End Date: 06/2015
Management Response: Yes
Focus Area:
  • 1. Others
Corporate Outcome and Output (UNDP Strategic Plan 2014-2017)
  • 1. Output 1.5. Inclusive and sustainable solutions adopted to achieve increased energy efficiency and universal modern energy access (especially off-grid sources of renewable energy)
Evaluation Budget(US $): 30,000
Source of Funding: GEF
Evaluation Expenditure(US $): 23,804
Joint Programme: No
Mandatory Evaluation: Yes
Joint Evaluation: No
Evaluation Team members:
Name Title Email Nationality
Roland Wong Independent Evaluator rolandwong@shaw.ca
Adil Lari Independent Evaluator office@acegroup.at
GEF Evaluation: Yes
GEF Project Title:
Evaluation Type: Terminal Evaluation
Focal Area: Climate Change
Project Type: FSP
GEF Phase: GEF-4
GEF Project ID: 4115
PIMS Number: 4289
Key Stakeholders: line ministries, local authorities, NGO
Countries: ROMANIA
Lessons
1.

Lessons Learned from work on new regulations and policies on fuel poverty and energy-efficiency

The external intervention, as the one of UNDP GEF project, for the clarification of new “concepts” in Romania

– fuel poverty and vulnerable costumers was necessary and beneficial.

The political and operational limiting factors have diverted the project from its course and have caused implementation delays. These elements were underestimated in the initial risk-assessment process and due to issues related to project design.


2.

Lessons Learned related to demonstration projects


A greater impact of the demonstration projects is obtained if these projects are sustained by other components (like building capacity, increasing awareness), also demonstrating the outcomes and benefits of using energy efficiency measures and if the selected buildings have proper energy management. The choice of energy-efficient design and materials for the public buildings, which are a significant part of the total buildings, gave the local government an important role model and a powerful influence in the building sector, having also a high replicability potential. However, it is important to ensure that the materials selected follow international best practice for energy-efficiency.

The demonstration projects show also the co-benefits linked to energy efficiency: like economic benefits (creating jobs and business opportunities, reduced energy bills) and social benefits (increased indoor comfort, improved air quality, higher capacity of low income households to pay the energy bills). The involvement of local community in the execution of the demonstration projects is beneficial and contributes to community empowerment and ownership of the project results.

,

Lessons Learned related to Energy Management

Energy Management is about people. Energy management follows the steps of measure, monitor, and mitigate. You cannot start with mitigation when you have not done proper measurement and monitoring. You cannot do energy management properly without having adequately trained and motivated staff that is monitoring energy performance on an ongoing basis and ranking, prioritizing different energy-efficiency measures. The project only started to look at the issue of energy management three years into a five year project. The lesson learned here is that energy management needs to be a priority from the start of the project and not something which starts when a project is close to finish. It is important not to underestimate the amount of work involved in designing, transferring, installing, operating, and maintaining an energy management information system (EMIS).

Lessons Learned related to National Buildings Registry


Capitalizing on existing experience from other UNDP GEF projects is welcome (i.e. – from UNDP GEF energy efficiency in Croatia), but measures have to be taken so that the core elements can be shared before the final product is transferred to national authorities. In particular, it is important that UNDP owns the source code associated with the related software and database to prevent unnecessary complications and delays. The full utility of the National Building Registry can be reached only if data entry processes and registry maintenance is enforced by relevant regulations. The main lesson learned here is that transferring a national buildings registry from one country to another takes longer than might be imagined and appropriate period of time of at least two years needs to be allowed for the transfer and for training of relevant officials.

,

Lessons Learned related to weak Project Design

The project was designed in such a way as to have a major focus on utilization of “Romanian only” energyefficiency building materials. This was despite the fact that a market for “Romanian only energy-efficiency” building materials did not exist at the start of the project and did not develop during the project lifetime. At the end of the project there was only one company in Romania that met the criteria of Romanian only certified energy efficient building materials. The experience with trying towards the end of the project to work with this Romanian certified energy-efficient building material was a failure and did not work out. The final evaluation has picked this focus on “Romanian only materials” as a major flaw in the project design. Much valuable time was lost because of this flawed project design and a key lesson learned here is to design projects taking into account the baseline situation and not to put in unrealistic challenges and obstacles into the project design.

,

Lessons Learned (other)
Management


• Reliable baseline and thorough risk analysis can build the success of a project. A poor risk analysis allows the establishment of too ambitious goals, which cannot be reached during the project lifetime. In the case of this project, risk analysis completely overlooked the problem that the project was supposed to focus on ““Romanian only energy efficient” building materials when a market for such materials did not exist.
• A stable and well skilled project team is a key element for assuring the correct and timely project implementation. The frequent changes within the project team generated delays and an unbalanced burden in different phases of the project.
• An experienced international chief technical advisor, expert in energy-efficiency in buildings, is critical for project success. This project suffered from the fact that there was no high level international chief technical advisor providing high level technical guidance, oversight, and support.

• Adaptive project management allows regular adjustments to the project outputs, and implementation calendar. But, attention should be paid because the adaptive management approach is not a universal
solution and sometimes can only partially solve the problem.
• Ownership of the project and commitment by the project partners it is instrumental in pursuing the project objectives through the project implementation period in order to quickly adapt to the challenging situations of the project.


Findings
1.

Conclusions

This project could not have been implemented without UNDP GEF support, especially for catalyzing the national efforts in addressing EE and FP issues. The project design was extremely ambitious and did not take into account all of the difficulties involved. In particular, the design unrealistically chose to focus on Romanian only energy-efficient certified building materials when such a market did not exist at the start of the project and did not develop during the project lifetime. This focus on “Romanian only” certified energy efficient building materials turned out to be impossible to achieve during the lifetime of the project.

The flexibility proven in project implementation allowed different approaches and changes as and when required and have contributed to the overall marginally satisfactory rating of the project. It allowed the project to adapt the “Romanian only” certified efficient building materials approach to local market conditions and to demonstrate energy efficiency measures using other materials and technologies.

The UNDP GEF project will be further leveraged through the existing or future planned Governmental interventions regarding the energy efficiency in buildings and in particular if the fuel poverty related legislation and regulations are put in place and adopted, this can be said to have been a big success of the project.

Recommendations

It is important that work initiated by the project continues even after the project ends at the close of June 2016. This includes:

Reiterating the EE issues at the Government level after the 2016 elections

Continuing the lobby and advocacy activities for including in the regulation framework the policy recommendations developed within the project that envisage the integration of EE and fuel poverty issues into the practice of public administrations at the national level.

Providing support to AAECR in order to continue the training program designed within the project

Streamlining the technical designs for 50 types of blocks of flats for planning and application for funding of thermal rehabilitation projects

Continuing the activity of the 50 information points for supporting the owners of individual households from low-income communities that implement projects in a “Do-it-yourself” manner

Increasing the effort to develop a market for locally produced, energy efficient sustainable insulation materials

Operating and implementing the needed adaptation/changes that occurred during the test phase of the “Building registry” and then rolling it out into full implementation phase

Undertaking the operation and maintenance of the Energy Management Information System (EMIS) and National Building Registry (NBR) beyond the project end, by the MRDAP and ensuring that the registry is maintained and updated

Developing the legal regulation that enforces the mandatory use of EMIS and NBR by all relevant stakeholders including all relevant government authorities.


Recommendations
1

Outcome 1: Romanian energy policy integrates fuel poverty issues and addresses EE needs in low income communities. Target includes 3 national-level Government institutions integrating the reduction of fuel poverty through EE/RE into their programmes and policies by EOP.

Rating and recommendation: A moderately satisfactory outcome has been achieved with regards to Project support to integrate Romanian energy policy with fuel poverty issues and addressing EE needs in low income communities. While the Project has provided robust efforts that contribute to policy analysis, policy formulation, and draft amendments, none of these legislative acts has yet been promoted by MRDAP and other ministries for legislation endorsement. Moreover, legislative approvals are unlikely for the EOP due to local and national elections to be held in June and November 2016 respectively.

2

Outcome 2: Supply of trained architects, building engineers, builders and auditors with EE experience expanded; municipalities in low-income regions have a better understanding of EE issues and are able to support auditing and weatherization projects – including disseminating information for Do-It-Yourself projects. Targets include:

  • 200 building engineers, architects and energy auditors qualified, certified and using the information in their work for the application of EE measures (and applicable Renewable Energy Technologies-RETs) and in the use of sustainable, locally available/produced building materials by EOP
  • 10% households that plan to/have already implemented EE measures due to the public information points and other public education activities of the project in the two main counties of the project at EOP;
  • 6 building materials and construction companies within the two pilot counties which are producing and selling locally produced, sustainable EE materials at EOP;
  • 3 additional counties (beyond the 2 pilot counties) which have expressed interest in replicating project activities due to the information campaign activities at EOP;
  • 2 additional countries (beyond Romania) which have expressed interest in replicating project activities due to the information campaign activities EOP

 

Rating: and recommendation:

  • A satisfactory outcome was achieved with regards to improving capacity at the local level to reduce fuel consumption in low income communities. This is reflected in the achievement of the following targets:
  • a total of 826 building engineers, architects and energy auditors trained and certified and using the information in their work for the application of EE measures (and applicable Renewable Energy Technologies-RETs) and in the use of sustainable, locally available building materials;
  • 49% of the households interviewed have already implemented EE measures according to a survey undertaken in Dolj and Hunedoara counties in 2016, while another 46% would rehabilitate their homes should the state authorities provide co-financing subsidies (as already in place via the National Building Rehabilitation Programme). The completion of a 2014 consumer awareness survey indicates that energy efficiency is a top priority of consumers in comparison with the equivalent survey in 2012 that ranked energy efficiency as a third priority amongst consumers;

Only 1 building materials and construction company, Arabesque, has been identified for the supply of sustainable EE materials. Two previous companies

  • (Mopatel using as raw material slaked lime and Izomiorita using as raw material wool) had been identified in the two pilot counties; however, the production capacity of these companies could not be scaled up to meet demand; this would have required extensive engineering and high investment costs that were beyond the scope of this Project;
  • 17 additional counties have expressed interest in replicating project activities resulting in 43 additional information points being set up within these counties with information materials produced by the project;

UNDP Armenia has expressed interest in replicating project activities during the 6th international forum on Energy for Sustainable Development organized in Yerevan, Armenia in 2015. Reports and deliverables were shared with Armenia Country Office. The project also disseminated its results and activities at various international conferences including “ESCO Moldova project - Transforming the urban energy efficiency market by introducing the energy services companies”. This is possibly due to the lack of finalized pilot projects to demonstrate energy savings from EE measures in apartment blocks and public buildings

3

Outcome 3: Energy efficient buildings reconstructed (and potentially new buildings constructed) with reduced fuel costs or using improved sustainable energy technologies in low-income communities

Rating and recommendation: A moderately satisfactory outcome has been achieved with the completion of retrofitted public buildings that have the potential for reduced fuel costs in low income communities. For many of these buildings, the actual quantification of the reduced fuel costs and GHG emission reductions from the retrofits will not be done on this Project due to late implementation of these retrofits and the subsequent lack of time to monitor these reductions.

4

Outcome 4: Data and information available for decision-makers for designing programmes to address fuel poverty. Targets include:

  • Final project report consolidating the results and lesson learnt from the implementation of the different project components and recommendations for the required next steps;
  • Project mid-term and final evaluations and other required review;

Rating and recommendation:

  • A moderately satisfactory outcome has been achieved with a number of documents containing information, data and methodologies being available to decision-makers that can be used for designing fuel poverty programs in Romania. Unfortunately, the database that will house the local registry of building stock was only completed this month, leaving little time for MRDAP to populate the database;
  • The only targets of this outcome that can be confirmed as completed is the midterm evaluation;

Other reports including final project report, lessons learned from implementation, and the final evaluation are currently in progress

1. Recommendation:

Outcome 1: Romanian energy policy integrates fuel poverty issues and addresses EE needs in low income communities. Target includes 3 national-level Government institutions integrating the reduction of fuel poverty through EE/RE into their programmes and policies by EOP.

Rating and recommendation: A moderately satisfactory outcome has been achieved with regards to Project support to integrate Romanian energy policy with fuel poverty issues and addressing EE needs in low income communities. While the Project has provided robust efforts that contribute to policy analysis, policy formulation, and draft amendments, none of these legislative acts has yet been promoted by MRDAP and other ministries for legislation endorsement. Moreover, legislative approvals are unlikely for the EOP due to local and national elections to be held in June and November 2016 respectively.

Management Response: [Added: 2018/12/19]

Management response: Agree

The project intervention in clarifying the Fuel Poverty and vulnerable costumers’ definitions was very welcomed by the Government. The use of different definitions by different institutions may create confusion at national level and UNDP GEF project proposal for an official definition for the Fuel Poverty, was necessary and beneficial. The political and operational limiting factors have diverted the project from its course and have caused implementation delays. The project experience revealed that, even with well documented proposals for developing/amending new regulations and policies on fuel poverty and energy efficiency, the adoption of these proposals remains very challenging and were not possible by the end of project. The political instability and the lack of communication between relevant public institutions are strongly influencing this work and are elements that were underestimated in the initial risk-assessment process. In the Lessons Learned Manual delivered by the project, a significant chapter deals with Recommendations for the follow up actions by the Ministry of Regional Development in order to adopt the proposed policy amendments.

Key Actions:

2. Recommendation:

Outcome 2: Supply of trained architects, building engineers, builders and auditors with EE experience expanded; municipalities in low-income regions have a better understanding of EE issues and are able to support auditing and weatherization projects – including disseminating information for Do-It-Yourself projects. Targets include:

  • 200 building engineers, architects and energy auditors qualified, certified and using the information in their work for the application of EE measures (and applicable Renewable Energy Technologies-RETs) and in the use of sustainable, locally available/produced building materials by EOP
  • 10% households that plan to/have already implemented EE measures due to the public information points and other public education activities of the project in the two main counties of the project at EOP;
  • 6 building materials and construction companies within the two pilot counties which are producing and selling locally produced, sustainable EE materials at EOP;
  • 3 additional counties (beyond the 2 pilot counties) which have expressed interest in replicating project activities due to the information campaign activities at EOP;
  • 2 additional countries (beyond Romania) which have expressed interest in replicating project activities due to the information campaign activities EOP

 

Rating: and recommendation:

  • A satisfactory outcome was achieved with regards to improving capacity at the local level to reduce fuel consumption in low income communities. This is reflected in the achievement of the following targets:
  • a total of 826 building engineers, architects and energy auditors trained and certified and using the information in their work for the application of EE measures (and applicable Renewable Energy Technologies-RETs) and in the use of sustainable, locally available building materials;
  • 49% of the households interviewed have already implemented EE measures according to a survey undertaken in Dolj and Hunedoara counties in 2016, while another 46% would rehabilitate their homes should the state authorities provide co-financing subsidies (as already in place via the National Building Rehabilitation Programme). The completion of a 2014 consumer awareness survey indicates that energy efficiency is a top priority of consumers in comparison with the equivalent survey in 2012 that ranked energy efficiency as a third priority amongst consumers;

Only 1 building materials and construction company, Arabesque, has been identified for the supply of sustainable EE materials. Two previous companies

  • (Mopatel using as raw material slaked lime and Izomiorita using as raw material wool) had been identified in the two pilot counties; however, the production capacity of these companies could not be scaled up to meet demand; this would have required extensive engineering and high investment costs that were beyond the scope of this Project;
  • 17 additional counties have expressed interest in replicating project activities resulting in 43 additional information points being set up within these counties with information materials produced by the project;

UNDP Armenia has expressed interest in replicating project activities during the 6th international forum on Energy for Sustainable Development organized in Yerevan, Armenia in 2015. Reports and deliverables were shared with Armenia Country Office. The project also disseminated its results and activities at various international conferences including “ESCO Moldova project - Transforming the urban energy efficiency market by introducing the energy services companies”. This is possibly due to the lack of finalized pilot projects to demonstrate energy savings from EE measures in apartment blocks and public buildings

Management Response: [Added: 2018/12/19] [Last Updated: 2018/12/19]

Management response: Agree

 

Almost all the results have been achieved under this Outcome and a satisfactory rating was obtained. Outputs 2.1 and 2.2 were achieved successfully. The Output 2.3: “Local building material producers and building construction companies highly qualified and capable of producing and applying, respectively, EE building materials” was not fully achieved due to the lack of an initial in-depth analysis of the local market conditions in the concept phase of the project. Consequently, although the Project Document stated that “there are active local construction companies which produce their own building materials and have some technical capacity”, in the implementation phase, the project team faced a lack of certified local producers of sustainable thermal insulation materials and, thus, with all the efforts of the team to certify at least two local companies in the field, it was impossible to overcome another important two barriers:

  • The company producing the EE building materials needed to have the production capacity to meet the demands of the pilot program for EE building materials of the project; and
  • To supply the material to the proposed retrofits Component 3 of this Project, the company would need to be successful on a public tender.

     

In order to reduce the risk of a lengthy Project delay and as an alternative solution to secure the supply of thermal insulation material for the pilot projects in Dolj and Hunedoara counties, the project team proposed the issuance of an international open tender mid-2015 with a condition that polystyrene would not be acceptable. The international tender was awarded to the lowest bidder in late 2015, Arabesque.

Key Actions:

3. Recommendation:

Outcome 3: Energy efficient buildings reconstructed (and potentially new buildings constructed) with reduced fuel costs or using improved sustainable energy technologies in low-income communities

Rating and recommendation: A moderately satisfactory outcome has been achieved with the completion of retrofitted public buildings that have the potential for reduced fuel costs in low income communities. For many of these buildings, the actual quantification of the reduced fuel costs and GHG emission reductions from the retrofits will not be done on this Project due to late implementation of these retrofits and the subsequent lack of time to monitor these reductions.

Management Response: [Added: 2018/12/19] [Last Updated: 2018/12/19]

Management response: Agree

 

Due to the bottlenecks and delays experienced during NIM regime, the delivery of the technical documentations needed for the implementation of the Outcome 3 has also been delayed until late 2015. 

Consequently, the building rehabilitations involving the installation of thermal insulation material supplied by Arabesque started in April 2016. Although all the building rehabilitations and almost all other activities under Outcome 3 were implemented, the late commencement of these installations did not provide sufficient time for the project to monitor energy savings and GHG emission reductions, or to disseminate this type of positive information that would have an impact on people’s willingness to invest in EE building materials. In order to compensate this lack of time to disseminate the best practices and positive results of the project, the Lessons Learned Manual delivered by the project team in June 2016 and disseminated during the final workshop and posted on the Ministry’s web site will play an important role along the ongoing efforts of the Ministry of Regional Development to further disseminate the project results and best practices at the national level. 

Key Actions:

4. Recommendation:

Outcome 4: Data and information available for decision-makers for designing programmes to address fuel poverty. Targets include:

  • Final project report consolidating the results and lesson learnt from the implementation of the different project components and recommendations for the required next steps;
  • Project mid-term and final evaluations and other required review;

Rating and recommendation:

  • A moderately satisfactory outcome has been achieved with a number of documents containing information, data and methodologies being available to decision-makers that can be used for designing fuel poverty programs in Romania. Unfortunately, the database that will house the local registry of building stock was only completed this month, leaving little time for MRDAP to populate the database;
  • The only targets of this outcome that can be confirmed as completed is the midterm evaluation;

Other reports including final project report, lessons learned from implementation, and the final evaluation are currently in progress

Management Response: [Added: 2018/12/19] [Last Updated: 2018/12/19]

Management response: Agree

 

The development of the national building registry commenced in late 2012 with the initial listing of the database parameters for energy efficiency in buildings and the development of methodologies for database design. The development of this building registry aligned with Government Ordinance 18/2009 that includes a methodology for the national inventory assessment of buildings in need of rehabilitation including, most importantly for this Project, low income areas of Romania. The database design would serve as a structured building registry for MRDAP to support a better allocation of resources within the future programmatic documents (programmes and strategies of the Ministry) and to prioritize building rehabilitations in low income areas and for the first time in Romania the energy management of the buildings.

 

Consultations on the design of the database were made between 2013 and 2014 with main government stakeholders including MRDAP, MoECC, MoLSP, Department of Energy under the Ministry of Economy; and the Regulatory Authority for Energy as well as energy auditors and information and communication technology (ICT) experts. Their collective interests were in setting up a comprehensive meta-database useful for integrated policy making decisions (that would include fuel poverty alleviation measures and their impacts, monitoring of completed EE measures in specific buildings, and GHG emission reductions). Moreover, the registry should be able to produce reports that can be used to prioritize public investments aimed at increasing energy efficiency in buildings (of interest to MRDAP), reducing the carbon footprint of buildings (of interest to MECC), and efficient allocation of heating subsidies based on fuel poverty assessments (of interest to MoLSP).

 

The project team finalized the development the database in December 2015, however the Ministry of Regional Development asked for a modification of the database in order bring the best added value for the Ministry. The project management unit amended the contract with the software developer in order to accommodate the requested modification from the Ministry and the final version of the building registry was submitted in June 2016. The final database was transferred to the Ministry of Regional Development according to the initial plan, however, due to the late finalization of the database, the Ministry assumed the obligation (by a MoU signed between the Ministry and UNDP) to further collect the data for the building registry and further operate and implement it at national level after the project closure at the end of June 2016.

Key Actions:

Latest Evaluations

Contact us

220 East 42nd Street
20th Floor
New York, NY 10017
Tel. +1 646 781 4200
Fax. +1 646 781 4213
erc.support@undp.org