- Evaluation Plan:
- 2016-2020, Belarus
- Evaluation Type:
- Final Project
- Planned End Date:
- 12/2018
- Completion Date:
- 06/2018
- Status:
- Completed
- Management Response:
- Yes
- Evaluation Budget(US $):
- 40,000
Terminal Evaluation of UNDP/GEF Project: Belarus: Improving Energy Efficiency in Residential Buildings
Share
Document | Type | Language | Size | Status | Downloads |
---|---|---|---|---|---|
![]() |
tor | English | 1323.60 KB | Posted | 1141 |
![]() |
tor | English | 248.00 KB | Posted | 1131 |
![]() |
report | English | 2149.07 KB | Posted | 1441 |
Title | Terminal Evaluation of UNDP/GEF Project: Belarus: Improving Energy Efficiency in Residential Buildings | ||||||||||||||
---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
Atlas Project Number: | 00061054 | ||||||||||||||
Evaluation Plan: | 2016-2020, Belarus | ||||||||||||||
Evaluation Type: | Final Project | ||||||||||||||
Status: | Completed | ||||||||||||||
Completion Date: | 06/2018 | ||||||||||||||
Planned End Date: | 12/2018 | ||||||||||||||
Management Response: | Yes | ||||||||||||||
Focus Area: |
|
||||||||||||||
Corporate Outcome and Output (UNDP Strategic Plan 2018-2021) |
|
||||||||||||||
SDG Target |
|
||||||||||||||
Evaluation Budget(US $): | 40,000 | ||||||||||||||
Source of Funding: | |||||||||||||||
Evaluation Expenditure(US $): | 32,356 | ||||||||||||||
Joint Programme: | No | ||||||||||||||
Joint Evaluation: | No | ||||||||||||||
Evaluation Team members: |
|
||||||||||||||
GEF Evaluation: | Yes
|
||||||||||||||
Key Stakeholders: | Line Ministries; Implementing Partners; UN agencies; CSOs, private sector | ||||||||||||||
Countries: | BELARUS, REPUBLIC OF |
Lessons | |
---|---|
1. | 1. The IEERB Project has provided excellent support within a period of 6 years to improve the enabling environment for increased investments into EE residential buildings in Belarus, notwithstanding the Project having a weak Project Results Framework (PRF) which did not convey to the Evaluation Team the targets that the Project needed to meet to achieve its overall objective. With strong and stable leadership of the IEERB Project from a highly experienced and top Belarussian climate change expert, the IEERB PMU was able to overcome this issue and facilitate significant achievements for energy efficiency in the building sector in Belarus: 2. The long-term sustainability of the IEERB Project of reducing energy consumption and GHG emission reductions on new residential buildings, however, is difficult to assess considering that existing heating tariff rates are too low for economic justification of the installation of energy efficient heating equipment in buildings, and the uncertainty of when these heating tariffs will be raised from a 21.4% to a 50% recovery rate; as shown on Table 7, the decrease in the number of EE buildings approved from 2016 to 2017 is an indicator of this need for a heating tariff review and amendment. There have been several Government Decrees (most recently 2016) proposing the raising of heating tariffs from 20% to 50% of the real cost by 2020, making EE measures economically feasible after 2020. There is also the expected full adoption of the TC-EPB that will make it mandatory for Belarusian building developers to implement energy efficiency measures in their buildings notwithstanding the low heating tariffs.
|
Findings | |
1. | 3.1.1 Analysis of Project Results Framework Tag: Emission Reduction Energy Effectiveness Relevance Programme/Project Design |
2. | 3.1.2 Risks and Assumptions
Tag: Energy Relevance Results-Based Management Risk Management |
3. | 3.1.4 Planned Stakeholder Participation Tag: Energy Effectiveness Relevance Bilateral partners Country Government Capacity Building Private Sector |
4. | 3.1.5 Replication Approach Tag: Energy Effectiveness Sustainability Programme/Project Design |
5. | 3.1.6 UNDP Comparative Advantage Tag: Relevance Policies & Procedures Strategic Positioning Bilateral partners Country Government Civil Societies and NGOs |
6. | 3.1.7 Linkages between IEERB Project and Other Interventions within the Sector
Tag: Emission Reduction Energy Effectiveness Relevance Global Environment Facility fund Country Government International Financial Institutions |
7. | 3.1.8 Management Arrangements Tag: Relevance Implementation Modality Oversight Project and Programme management Country Government |
8. | 3.2 Project Implementation
Tag: Relevance Implementation Modality Project and Programme management Bilateral partners Country Government Education Capacity Building |
9. | 3.2.1 Adaptive Management Tag: Global Environment Facility fund Communication Human and Financial resources Project and Programme management Results-Based Management |
10. | 3.2.2 Partnership Arrangements Tag: Energy Partnership Country Government Capacity Building Private Sector |
11. | 3.2.3 Feedback from M&E Activities Used for Adaptive Management Tag: Effectiveness Monitoring and Evaluation Project and Programme management Results-Based Management |
12. | 3.2.4 Project Finance Tag: Efficiency Global Environment Facility fund Human and Financial resources |
13. | 3.2.5 M&E Design at Entry and Implementation
Despite the weaknesses of the PRF (as explained in Para 29), the M&E design is rated as satisfactory. 56. The M&E plan was executed according to the designed as detailed in Para 55. The Evaluation Team has had access to review a Project Inception phase report, PIRs, MTR, Project audits, PSC meeting minutes, as well as Project logs (for risk, monitoring, lessons and issues). All these files especially the Project logs provide good insights into the management of the IEERB Project in the identification of risk, issues and corresponding management responses. In addition, these Project logs also documented checklists for the M&E plan on a semi-annual basis to ensure compliance with the original M&E plan. With these files having been reviewed by the Evaluation Team, M&E plan implementation is rated as satisfactory. Ratings according to the GEF Monitoring and Evaluation system27 are as follows:
Tag: Effectiveness Monitoring and Evaluation Oversight Project and Programme management |
14. | 3.2.6 Performance of Implementing and Executing Entities
Tag: Effectiveness Efficiency Human and Financial resources Implementation Modality Project and Programme management Results-Based Management |
15. | Outcome 1: Strengthened legal and regulatory framework and mechanisms to enforce legislation for improving energy efficiency of building sector 65. To achieve Outcome 1, Project resources would be used to: Tag: Energy Effectiveness Rule of law Capacity Building National Institutions |
16. | 3.3.6 Relevance Tag: Energy Relevance Country Government |
17. | 3.3.7 Effectiveness and Efficiency Tag: Energy Effectiveness Efficiency Country Government |
18. | 3.3.8 Terminal Evaluation of Improving Energy Efficiency in Residential Buildings Country Ownership and Drivenness Tag: Energy Effectiveness Ownership Country Government |
19. | 3.3.9 Mainstreaming 98. The IEERB Project was successfully mainstreamed with the Belarus UNDAF (2011-2015)46, notably:
Tag: Climate Change Adaptation Energy Effectiveness Gender Mainstreaming Capacity Building |
20. | 3.3.10 Sustainability of Project Outcomes
Tag: Energy Sustainability |
21. | 3.3.11 Impacts 103. The Project has made significant impacts within the building sector in Belarus: Tag: Energy Impact Country Government |
22. | 3.3.3 Outcome 2: Enhanced expert capacity of the Belarusian specialists to implement new energy efficiency standards and construction norms 70. To achieve Outcome 2, Project resources would be utilized to: Tag: Energy Effectiveness Education Capacity Building National Institutions |
23. | Outcome 3: Implementation of demonstration projects for energy efficient buildings To achieve Outcome 3, Project resources would be utilized to:
Tag: Emission Reduction Energy Effectiveness Sustainability Project and Programme management |
24. | Outcome 4: Documented, disseminated and institutionalized project results providing a basis for further replication To achieve Outcome 4, Project resources would be utilized to deliver the following outputs:
Tag: Emission Reduction Energy Effectiveness Oversight Country Government |
25. | To enhance the delivery of Outputs 2.7 and 2.8, the Project provided support and organized logistics for 58 decision-makers, designers, builders and other relevant specialists for their participation on 8 study visits to five European countries (Germany, Austria, Czech Republic, England and France)39 and 16 international events abroad on best practice for energy efficient residential buildings design, construction and operation as well as legal framework, standards and policies. Tag: Energy Effectiveness Knowledge management Capacity Building South-South Cooperation National Institutions |
26. | 86. Project resources were used to cover the cost of additional energy efficiency measures estimated to be around 17% of the total capital cost of these buildings. Project partners provided the necessary technical supervision during the construction of the building envelope and installation of the energy efficient equipment by subcontractors. Technical supervision reports as a part of the delivery of Output 3.2 were generated during this period including information on necessary adjustments to ensure proper installation of equipment that would perform to the intended standards and guidelines in the design documentation. Construction completion dates of the demonstration buildings were 14 December 2016 for Mahiliou, 26 January 2017 for Minsk and 20 May 2017 for Hrodna. Tag: Energy Effectiveness Project and Programme management |
27. | The Project engaged "RUE StroyTechNorm" (STN) who assembled the Technical Committee on Standardization in the Field of Architecture & Construction (TCS-14) in 2014 to lead the development of the TC-EPB; the Technical Committee consisted of 20 members (including four Project’s experts) from various entities throughout Belarus including those working under an Energy Efficiency Committee within DEE. TCS-14 with the assistance of Project resources undertook a number of activities to develop the TC-EPB including: Tag: Energy Effectiveness Capacity Building National Institutions |
28. | 91. To date, the Project has successfully implemented numerous outreach and institutional activities since late 2013: This included:
Tag: Emission Reduction Energy Effectiveness Communication National Institutions |
29. | The Project supported the training for a wide spectrum of building professionals that commenced with the preparation of appropriate training materials in 2013 and 2014. Project consultants prepared 250 technical reports of high quality for introducing new principles into design and construction of energy efficient housing as delivery of Outputs 2.1. Many of these reports have been adopted by academic institutes such as the Construction Faculty of BNTU in Minsk. Dissemination of these materials were supplemented by presentations, lectures, leaflets, brochures and guidelines through numerous 6-day training sessions in Minsk and Hrodna, dedicated to construction policies, norms and standards, recent international developments, integrated energy efficient building design principles and techniques, available technical options and correct installation of the materials and equipment used, and methodologies for assessing energy performance. Tag: Energy Effectiveness Education Capacity Building National Institutions |
30. | 82. Prior to the commencement of Component 3, there were no energy efficient residential buildings with comparable energy performance in Belarus that could be monitored and used to inform policy for energy efficient buildings, and which could be used to demonstrate to the public and relevant building stakeholders the potential and feasibility of energy efficiency improvement measures in residential buildings. Tag: Emission Reduction Energy Effectiveness Efficiency |
31. | 71. The baseline scenario of the Project was only 3 out of more than 120 design institutes in Belarus using a new design approach to energy efficiency for buildings, with little to no dissemination of this new approach beyond the local design institutes of NIPTIS, MAPID and Grodnograzhdanproekt. Prior to training being delivered (around June 2014), the Project had completed: Tag: Energy Effectiveness Education Capacity Building National Institutions |
32. | 78. As an adaptive management measure as mentioned in Para 43, the Project in early 2017 commenced a new series of trainings for professionals from construction companies, housing operation entities and utilities as well as for tenants with practical courses at the project pilot sites in Minsk, Mahiliou and Hrodna. In May 2017, three training sessions were held for tenants in the demonstration residential buildings in Minsk, Hrodna and Mahiliou. During the heating season of 2017-2018, four 1-day training workshops with on-site practical exercises were conducted for 50 specialists of municipal services responsible for operation of the pilot residential buildings in Hrodna and Mahiliou. Tag: Energy Effectiveness Knowledge management Capacity Building South-South Cooperation National Institutions |
33. | 88. During 2017 and 2018, regular visits to the pilot buildings were made by local architects, designers, builders and decision makers as delivery of Output 3.5. However, despite the Project’s target to monitor the energy consumption from these pilot buildings for a period of one year, the Project, although conducting the monitoring of these pilot buildings for a period of 12 months, has not had the opportunity to obtain continuous energy consumption data and has not delivered Output 3.4 in full due to:
Tag: Energy Effectiveness Project and Programme management |
34. | These discussions are still ongoing since the Institute of Urban and Regional Planning is reluctant to accommodate all new approaches. The Project issued 2 reports in 2017 and 2018 on EE urban development that were submitted to DEE and IURP for consideration:
Tag: Emission Reduction Energy Effectiveness Communication National Institutions |
35. | 30. The GHG emission reductions target of the IEERB Project is 220,000 tonnes CO2 that are lifetime emission reductions generated mainly from more than 80 EE new residential buildings which comply to Technical Code “Energy Efficiency of Buildings” which is actually the Technical Code for Energy Performance of Buildings (TC-EPB). This was to be calculated by taking the baseline annual specific energy consumption (SEC) for space heating demand from the average of 60 kWh/m2 (as adopted in 2010) to 20 kWh/m2 and the annual heat demand for sanitary hot water from the current average of 65 kWh/m2 similarly to 40 kWh/m2. The Ministry of Architecture and Construction (MoAC) maintain a database of buildings that are approved for construction and compliance with EEB Technical Code, complete with the actual SEC values of the new building and the building area in m2. GHG emission reductions can then be calculated by applying an average energy emissions factor in Belarus (consisting of composite electricity and heat factors). Table 1 provides a sample calculation from the IEERB ProDoc of the direct GHG reduction impact with the target of 65 kWh/m2/yr savings in final energy use compared to similar constructed buildings in accordance with the construction norms in force in 2012. This analysis would also be applicable to the detailed design of the 3 demo buildings that were implemented on this Project as well as other buildings that are approved for construction and in compliance to the TC-EPB. The objective level target in the PRF of 220 ktCO2eq for GHG emission reductions would appear to be reasonable based on 80 buildings adopting the energy efficiency norms for buildings of the EEB Technical Code, for which the design or construction has started during the project implementation. With the enforcement system in place with MoAC approving project designs that are in compliance with the TC-EPB, the PMU will be reliant on reporting from the implementing partner, DEE, of buildings that have received these design and construction approvals. The target of 80 buildings assumes approximately 800,000 m2 of floor area for each of the 80 buildings.
Tag: Emission Reduction Energy Effectiveness Relevance Project and Programme management |
Recommendations | |
---|---|
1 | To improve design of the future CCM projects including any follow-up “building energy efficiency” project |
2 | To improve implementation of future projects similar to IEERB, the PRF (strengthened through ToC analysis and ROtI) needs to be used as a guide for preparing project work plans. |
3 | Continue training workshops in oblast training centres to ensure the availability of a critical mass of certified EE building operational personnel. |
4 | (to MoHU with assistance from EED and UNDP): Set up a program that continually informs occupants of EE buildings on optimizing use of their EE systems for hot water, heating and fresh air ventilation systems. |
5 | (to the Ministry of Architecture and Construction (MoAC) with assistance from DEE and UNDP): Continue training programs to ensure the availability of a critical mass of building design personnel to enforce compliance to the TC-EPB |
6 | (to EED and MoHU): Monitor progress of Decree No.1037 of 29.12.2017 on the “Concept of Improvement and Development of Housing and Utility Services until 2025” addressing elimination of cross-subsidies and upward adjustment of low heating tariff rate towards 50% and higher that should be |
7 | (to DEE and MoAC): Disseminate monitored energy consumption data from 3 demo buildings to the Government policymakers and investors in line with Deputy Prime Minister’s Order No.04/26p of July 4, 2017, with more emphasis on the time when the pilot buildings are fully occupied and when the tenants have been in their units for a sufficient time during which their energy consumption would be normalized; this could be as long as 18-24 months when all units are occupied although the currently monitored data are being provided for decision makers.. |
8 | (to MoNREP): Provide support to appropriate institutions to help formalize protocols and methodologies developed by the Project for reporting GHG emission reductions for EE buildings and retrofits |
9 | (to EED): Support future retrofit programs or construction of new EE buildings (supported either by GoB or donors) through training of building energy operators or managers using experiences gained through the Project |
10 | Scope for improved practice: Many country offices including Belarus need assistance in the preparation of ProDocs, most notably the Project Results Framework which needs to be focused with insertion of SMART indicators that directly contribute to an intended outcome. |
Key Action Update History
To improve design of the future CCM projects including any follow-up “building energy efficiency” project
Management Response: [Added: 2018/11/09] [Last Updated: 2020/12/11]
Key Action(s) |
Time Frame |
Responsible Unit(s) |
Tracking* |
|
Status |
Comments |
|||
1.1. Future projects should contain a clear logical framework matrix with SMART indicators and measurable targets that are reflected in the project outputs. |
2018 -on |
UNDP |
On-going |
This relates to future projects and should be considered during the project preparation phase. |
1.2 Specific M&E activities should be designed that can help project teams focus on specific indicators for monitoring and respective and the respective adequate budget allocated in the project document. |
2018 -on |
UNDP |
On-going |
|
1.3 In the future projects that would focus on energy efficiency in residential sector sufficient resources should be allocated for capacity building of building owners and buildings operators to monitor energy consumption |
2018 -on |
UNDP |
On-going |
Key Actions:
To improve implementation of future projects similar to IEERB, the PRF (strengthened through ToC analysis and ROtI) needs to be used as a guide for preparing project work plans.
Management Response: [Added: 2018/11/09] [Last Updated: 2020/12/11]
Management Response: All new projects include theory of change as a standard section in the Project Document. PRF is used as a guide for ADWP preparation.
Key Action(s) |
Time Frame |
Responsible Unit(s) |
Tracking |
|
Status |
Comments |
|||
2.1. Constantly monitor that PRF is used for ADWP preparation |
2018 -on |
UNDP |
On-going |
The best timing is during the APR and ADWP preparation. |
Key Actions:
Continue training workshops in oblast training centres to ensure the availability of a critical mass of certified EE building operational personnel.
Management Response: [Added: 2018/11/09] [Last Updated: 2020/12/11]
This recommendation is addressed to the Ministry of Housing and Utilities (MoHU) and the Energy Efficiency Department (EED) capitalizing on the expert and institutional capacities built within the project.
Key Action(s) |
Time Frame |
Responsible Unit(s) |
Tracking |
|
Status |
Comments |
|||
3.1 As a post-project monitoring, request information from the MoHU and EED on the training activities after the project completion |
Jun - Jul 2019 |
UNDP. MoHU and EED |
pending |
The project ended in Jul 2018. |
3.2 Discuss with the national partners further capacity building needs which can be including in one of the future projects. |
Jul – Jan 2019 |
UNDP. MoHU and EED |
pending |
To meet with the MoHU and EED to discuss their capacity building needs to be considered for future energy efficient projects. |
Key Actions:
(to MoHU with assistance from EED and UNDP): Set up a program that continually informs occupants of EE buildings on optimizing use of their EE systems for hot water, heating and fresh air ventilation systems.
Management Response: [Added: 2018/11/09] [Last Updated: 2020/12/11]
The MoHU and EED should use the capacity built within the project
Key Action(s) |
Time Frame |
Responsible Unit(s) |
Tracking |
|
Status |
Comments |
|||
4.1 Discuss with the MoHU and EED if they need such a programme along with a very preliminary programme content. |
Jul-Jan 2018 |
UNDP. MoHU and EED |
pending |
The best option will be the government allocating their own resources to develop such a programme which can be used for other buildings, as well. |
4.2. Consider including the development of such a programme within future projects focusing on energy efficiency |
2018-on |
UNDP |
On-going |
Key Actions:
(to the Ministry of Architecture and Construction (MoAC) with assistance from DEE and UNDP): Continue training programs to ensure the availability of a critical mass of building design personnel to enforce compliance to the TC-EPB
Management Response: [Added: 2018/11/09] [Last Updated: 2020/12/11]
The MoAC should and EED should use the capacity built within the project
Key Action(s) |
Time Frame |
Responsible Unit(s) |
Tracking |
|
Status |
Comments |
|||
5.1 As a post-project monitoring, request information from the MoAC and EED on the training activities after the project completion |
Jul – Jan 2019 |
UNDP. MoAC and EED |
pending |
The project ended in Jul 2018. |
5.2 Discuss with the national partners further capacity building needs which can be including in one of the future projects. |
Jul – Jan 2019 |
UNDP. MoAC and EED |
pending |
Key Actions:
(to EED and MoHU): Monitor progress of Decree No.1037 of 29.12.2017 on the “Concept of Improvement and Development of Housing and Utility Services until 2025” addressing elimination of cross-subsidies and upward adjustment of low heating tariff rate towards 50% and higher that should be
Management Response: [Added: 2018/11/09] [Last Updated: 2020/12/11]
This issue is politicized and beyond UNDP control and influence. UNDP should however monitor the elimination of cross subsidies as this will drastically affect economic attractiveness of energy efficiency in residential sector
Key Action(s) |
Time Frame |
Responsible Unit(s) |
Tracking |
|
Status |
Comments |
|||
6.1 Monitor the elimination of cross subsidies as this will drastically affect economic attractiveness of energy efficiency in residential sector |
2018 - on |
UNDP |
On-going |
UNDP has been and will be monitoring this issues as it defines significantly the national energy efficiency policy and practice |
Key Actions:
(to DEE and MoAC): Disseminate monitored energy consumption data from 3 demo buildings to the Government policymakers and investors in line with Deputy Prime Minister’s Order No.04/26p of July 4, 2017, with more emphasis on the time when the pilot buildings are fully occupied and when the tenants have been in their units for a sufficient time during which their energy consumption would be normalized; this could be as long as 18-24 months when all units are occupied although the currently monitored data are being provided for decision makers..
Management Response: [Added: 2018/11/09] [Last Updated: 2020/12/11]
The information on pilot buildings actual energy consumption would be very valuable for estimation of the effectiveness of the energy efficiency technologies and approaches employed by the project and can be used for future energy efficiency related projects designed by UNDP and the Government.
Key Action(s) |
Time Frame |
Responsible Unit(s) |
Tracking |
|
Status |
Comments |
|||
7.1 As a post-project monitoring, request information from the MoAC and EED on the actual energy consumption from 3 demo buildings |
June 2019, June 2020 and June 2021 |
UNDP. MoAC and EED |
pending |
Data should be requested after the completion of a heating period |
Key Actions:
(to MoNREP): Provide support to appropriate institutions to help formalize protocols and methodologies developed by the Project for reporting GHG emission reductions for EE buildings and retrofits
Management Response: [Added: 2018/11/09] [Last Updated: 2020/12/11]
UNDP may provide assistance to the MoNREP to elaborate/update/tailor the protocols and methodologies for GHG emission reductions reporting.
Key Action(s) |
Time Frame |
Responsible Unit(s) |
Tracking |
|
Status |
Comments |
|||
8.1 Discuss with the MoNREP (and other relevant stakeholders) their needs with respect to the use of the protocols and methodologies developed by the Project for reporting GHG emission reductions for EE buildings and retrofits. Based on the results of these consultations, consider to assist the MoEnv in elaborating/updating/tailoring the protocols and methodologies for GHG emission reductions reporting within future climate change related projects, e.g. the MRV project. |
Jul-Jan 2018 |
UNDP. MoEnv, PIU |
On-going |
N/A |
Key Actions:
(to EED): Support future retrofit programs or construction of new EE buildings (supported either by GoB or donors) through training of building energy operators or managers using experiences gained through the Project
Management Response: [Added: 2018/11/09] [Last Updated: 2020/12/11]
The EED should use the capacity build within the project. If appropriate, activities on training of the respective specialists into the scopes of future climate change related projects.
Key Action(s) |
Time Frame |
Responsible Unit(s) |
Tracking |
|
Status |
Comments |
|||
9.1 Discuss with the EED, MoAC and other stakeholder their needs with respect to knowledge and skills building of building energy operators or managers. |
2018 -2020 |
UNDP. MoAC and EED |
On-going |
This is an on-going discussion and UNDP assistance will depend on resource availability. |
Key Actions:
Scope for improved practice: Many country offices including Belarus need assistance in the preparation of ProDocs, most notably the Project Results Framework which needs to be focused with insertion of SMART indicators that directly contribute to an intended outcome.
Management Response: [Added: 2018/11/09] [Last Updated: 2020/12/11]
During the project formulation stage seek assistance from the IRH with respect the PRF and the set of targets and indicators quality
Key Action(s) |
Time Frame |
Responsible Unit(s) |
Tracking |
|
Status |
Comments |
|||
10.1 send draft ProDocs to the IRH specialist for review of the PRF and the proposed set of targets and indicator for every new project |
2018 -on |
UNDP CO and IRH. |
On-going |
It is our usual practice to get IRH commenting on and clear new Project Documents and we will continue this practice. |
Key Actions: