Building Energy Efficiency in North West of Russia

Report Cover Image
Evaluation Plan:
2011-2017, Russian Federation
Evaluation Type:
Mid Term Project
Planned End Date:
03/2014
Completion Date:
05/2014
Status:
Completed
Management Response:
Yes
Evaluation Budget(US $):
23,000

Share

Document Type Language Size Status Downloads
Download document PIMS 4131 Mid Term Evaluation Report - Building Energy Efficiency in the North West of Russia.pdf report English 1323.48 KB Posted 1536
Download document ToR-MTE_BuildingEE_final.doc tor English 400.00 KB Posted 356
Title Building Energy Efficiency in North West of Russia
Atlas Project Number: 00074315
Evaluation Plan: 2011-2017, Russian Federation
Evaluation Type: Mid Term Project
Status: Completed
Completion Date: 05/2014
Planned End Date: 03/2014
Management Response: Yes
Focus Area:
  • 1. Others
Corporate Outcome and Output (UNDP Strategic Plan 2014-2017)
  • 1. Output 1.5. Inclusive and sustainable solutions adopted to achieve increased energy efficiency and universal modern energy access (especially off-grid sources of renewable energy)
Evaluation Budget(US $): 23,000
Source of Funding: GEF
Joint Programme: No
Joint Evaluation: No
Evaluation Team members:
Name Title Email Nationality
Vesa Rutanen Evaluator
GEF Evaluation: Yes
Evaluation Type:
Focal Area: Climate Change
Project Type: FSP
GEF Phase: GEF-4
PIMS Number: 4131
Key Stakeholders: Presidential Envoy in Nort-West Russia
Countries: RUSSIAN FEDERATION
Lessons
Findings
Recommendations
1 Project?s strategic results framework is suggested to be reviewed, updated and revised at the earliest convenience. While substantial content of the different outcomes can stay close to what they are now, the number of indicators and targets needs to be reduced and redefined with the focus on the main target(s) to be achieved under each outcome with due attention on qualitative and sustainability aspects. After that, all project outputs under each outcome can be reviewed and adjusted to effectively contribute to the revised targets of each outcome (MTE, page. 7).
2 One of the weakest elements of the project design is clearly inadequate risk analysis (MTE, page 7).
3

Another weak element of the project design is inadequate stocktaking and analysis of the realized or planned results and activities of other past, ongoing and future projects, on which the design of the activities and outputs of the new UNDP/GEF project could have been built, thereby contributing to more efficient and effective implementation of it (MTE, page 7).

4 The ongoing administrative reform and institutional changes have created challenges for efficient project implementation. As an example, the main project partner, the Specific Commission on Energy Efficiency under the auspices of the Office of the Plenipotentiary of the President of the Russian Federation in the Northwest Federal District, was dismissed at the end of 2012, which also led to the change of the Project Director at the end of 2013. No Project Steering Committee (PSC) meetings were organized either during this transition period (MTE, page 7). As a result until now the Project Steering Committee has not been able to fulfill its envisaged role in monitoring and guiding the project implementation.
5 In the PIR of 2012, the problem about the double role of the National Deputy Director (NDD) was raised, as the NDD was acting at the same time as the project manager. Based on the information provided in the PIR, the problem had been apparent since the project inception workshop, but could not have been solved. Later, a separate project manager was hired and replaced again in summer 2013 with a new project manager, while the former project manager has been continuing since then as the ?Project Implementation Coordinator?. After all these changes, however, there is still fundamental lack of clarity on who is actually managing the project. Most discussions during the project evaluation mission were conducted with the Project Implementation Coordinator, while for any substantial issues, the current project management team is frequently referring to the former NDD as the one, who continues to provide all the substantive advice and leadership for project implementation and, among others, is managing all the discussions with the project partners on any financing r elated matters. This clearly does not look like an acceptable and cost-effective way of managing projects of this or any kind (MTE, page 7).
6 The role of the project?s regional coordinators is unclear. They as opposite to their initially envisaged role to manage all regional project activities with a clear vision on what the project is expected to achieve in their particular region, have been assigned with quite specific and limited tasks only and in some cases were not even aware of some activities to be undertaken in their respective region (MTE, page 7).
7 Apart from participation in a few meetings and workshops, there is no evidence about effective coordination and partnership building with other ongoing projects and initiatives dealing with the same substance area, including the two EBRD (one jointly with IFC) projects approved for GEF funding under the same GEF EE umbrella project, the ongoing international cooperation with the St. Petersburg Construction Committee to develop new construction norms there or on the other ongoing research work on EE building design and construction in Saint Petersburg and/or Moscow (MTE, page 8).
8 The local working group established for the review, discussion and adoption of the new construction norms in Vologda is critical, but it does not compensate for those capacity constraints and lack of information that, for instance, the local design institutes at the oblast level still have in incorporating international state of the art approaches and lessons learnt for energy efficiency building design and construction into drafting of those norms. These constraints could have been overcome by more effective use of project resources allocated for capacity building and supporting international expertise (e.g. through the on the job training) and by identifying opportunities for co-operation and partnership building with both international and national expert institutions engaged in similar work in Russia (MTE, page 8).
9 Regarding co-financing, the aggregated targets are likely to be met, but in leveraging required co-financing for individual activities such as the planned demo projects in Arkhangelsk, there seems to be major problems, which are currently slowing down the project implementation.
10 The biggest and most sustainable impact of the project is likely to be made by the educational programs (Outcome 2), subject to more detailed independent expert evaluation of their content and whether in line with international state of the art approaches and good practices on EE building design and construction. This review has not been done yet, but is suggested to be done at earliest (MTE, page 9).
11 Another project subcomponent proceeding relatively well and demonstrating some initial elements for sustainability is the energy monitoring and management system developed for and planned to be tested in the Pskov oblast. The concrete implementation plan and value added of the Energy Management system developed by another consultant group for the Arkhangelsk oblast is not completely clear yet and is likely to require some further research and consultations.
12 The weakest prospects for sustainable project impact are currently with the planned demonstration projects in Arkhangelsk focusing on EE retrofits of existing buildings. At the time of this MTE, no concrete steps had been taken by the project yet to secure financing for these projects and both the regional and the core project team seemed not to be aware that the actual investments are not supposed to be funded by the GEF resources (MTE, page 9).
13 Another concern is based on the information received during the project evaluation mission that the elements included in the pilot building design have been used as a basis also for the proposed new construction norms in the Vologda region. The review of the design of the planned pilot building in Vologda at the end of 2013 by an international building energy efficiency expert contracted by the project revealed some significant shortcomings raising a question whether similar shortcomings and suboptimal requirements have remained in the new construction norms (MTE, page 9).
14 The absence of credible business plans and uncertainties in the adequate demand of services for the distant learning centers established under component 2 in Vologda and the EE design office created by the AOEEC in Arkhangelsk are raising questions about their sustainability, especially as their exact role in supporting the other subcomponents of the project dealing with design and training related activities remained unclear. A vision shared by the local stakeholders in the interviews was that the GEF is expected to pay for the first 1-2 years of operation of those centres, after which they should become self-sustaining. There are considerable uncertainties and risks with this approach, however, which may have not been effectively addressed yet (MTE, page 9).
15 Finalized report was seen as an adequate result on its own without really thinking its contribution towards reaching the ultimate goals of the project. This approach, which has been unfortunately common also for many other international technical assistance projects, may leave behind an impressive pile of reports, but may not really produce results for replication, which is raising some concerns about the sustainable project impact in general.
16 As a part of the required management changes and given the eventual difficulties to find experienced enough local project managers to run the project entirely on their own, the recruitment of an experienced international project management and technical advisor (with knowledge of the Russian language/Russian speaking staff and previous working experience in the Russian building sector) to support the project with any substantive issues, adaptive planning and management and ongoing progress monitoring is also strongly recommended with direct reporting responsibility to the project director, UNDP Regional Technical Adviser and UNDP CO (MTE, page 45).
17 All the research work and project funds invested in energy audits, energy certificates and preparation of feasibility studies are pretty much useless, if not leading to concrete energy efficiency investments. For leveraging funding for these investments, the project needs to intensify its efforts (MTE, page 45).
18 The project inception period took longer than planned partly because of the on-going administrative reform and partly because the project ?lost? its initially approved demo site for the pilot buildings in Vologda and had to negotiate with the regional authorities about the transfer of the pilot site to another location. Given the current status of implementation, the project is neither likely to be able to complete all of its envisaged activities and reach the targeted objectives by the currently planned closing date of January, 2016 nor to spend all of its remaining financial resources by then. At the same time, the project scope and planned activities have remained highly relevant even for the coming years, if effectively and professionally implemented (MTE, page 47). The above imply that consideration of project extension will be required at some point in the future.
1. Recommendation: Project?s strategic results framework is suggested to be reviewed, updated and revised at the earliest convenience. While substantial content of the different outcomes can stay close to what they are now, the number of indicators and targets needs to be reduced and redefined with the focus on the main target(s) to be achieved under each outcome with due attention on qualitative and sustainability aspects. After that, all project outputs under each outcome can be reviewed and adjusted to effectively contribute to the revised targets of each outcome (MTE, page. 7).
Management Response: [Added: 2014/09/04]

The recommendation is highly appreciated and the Project will do its best to review and update the strategic results framework with more concrete definition of each indicator.

Key Actions:

Key Action Responsible DueDate Status Comments Documents
Russian or foreign expert will be hired to review the project?s strategic results framework and to reassess project implementation risks.
[Added: 2014/09/04] [Last Updated: 2018/11/08]
Project team UNDP 2014/12 Completed An international expert, Mr. Vesa Routanen was hired to review the SRF and re-assess risks History
2. Recommendation: One of the weakest elements of the project design is clearly inadequate risk analysis (MTE, page 7).
Management Response: [Added: 2014/09/04]

The risks have already been reassessed during the preparation of the inception report. Nevertheless, following the given recommendations, the risks will be reconsidered again in the context of the current situation in Russia.

Key Actions:

Key Action Responsible DueDate Status Comments Documents
Russian or foreign expert will be hired to review the project?s strategic results framework and to reassess project implementation risks.
[Added: 2014/09/04] [Last Updated: 2018/11/08]
Project team UNDP 2014/12 Completed An international expert, Mr. Vesa Routanen was hired to review the SRF and re-assess risks History
3. Recommendation:

Another weak element of the project design is inadequate stocktaking and analysis of the realized or planned results and activities of other past, ongoing and future projects, on which the design of the activities and outputs of the new UNDP/GEF project could have been built, thereby contributing to more efficient and effective implementation of it (MTE, page 7).

Management Response: [Added: 2014/09/04] [Last Updated: 2018/11/08]

The project team has studied similar projects that were implemented in Russia by VTT, NEFCO, NCC and IFC companies/organizations. Undoubtedly, the experience of these projects is useful for the implementation of our project. Nevertheless, the analysis has shown that these projects were oriented at the solution of some narrow issues, they implied neither improvement of legislative and regulatory framework, nor replication and development of training programs on energy efficiency for education institutions of all levels (except IFC project).

Key Actions:

Key Action Responsible DueDate Status Comments Documents
Cooperation with projects/companies whose activities correspond to the project goals.
[Added: 2014/09/04] [Last Updated: 2018/11/08]
Project team Project partners 2017/09 Completed Meetings have been held with IFC to exchange experience. Joint educational seminars and trainings in the pilot regions have been organised. History
4. Recommendation: The ongoing administrative reform and institutional changes have created challenges for efficient project implementation. As an example, the main project partner, the Specific Commission on Energy Efficiency under the auspices of the Office of the Plenipotentiary of the President of the Russian Federation in the Northwest Federal District, was dismissed at the end of 2012, which also led to the change of the Project Director at the end of 2013. No Project Steering Committee (PSC) meetings were organized either during this transition period (MTE, page 7). As a result until now the Project Steering Committee has not been able to fulfill its envisaged role in monitoring and guiding the project implementation.
Management Response: [Added: 2014/09/04]

Unfortunately, as correctly stated, a number of administrative reforms were carried out in Russia during 2012-2013. As a result, many government bodies were restructured, their employees dismissed or transferred to other positions. The project team realized that until the end of the reorganizations there was no point to hold PSC meetings since many of PSC members had already been discharged from the institutions that are interested in project implementation, while new employees were not appointed yet. The last PSC meeting was held in December 2013. Among the main points of its agenda were the changing of the National Implementing Authority and discussing the corrective actions in response to the preliminary recommendations given by the MTE. The Russian Energy Agency (REA) was approved as the new National Implementing Authority. The next PSC meeting chaired by the REA will be held in the nearest future and will be dedicated to the issues of project implementation strategy under new context and in accordance with final recommendations of the MTE. Such meetings will be held annually.

Key Actions:

Key Action Responsible DueDate Status Comments Documents
The PSC meeting chaired by the new National Implementing Authority is planned to be held in the nearest future. Hereafter, such meetings will be held annually.
[Added: 2014/09/04] [Last Updated: 2018/11/08]
Project team Project partners 2017/12 Completed PSC headed by the new Project Implementing Partner (Russian Energy Agency) have been organised regularily till project completion in late 2017 History
5. Recommendation: In the PIR of 2012, the problem about the double role of the National Deputy Director (NDD) was raised, as the NDD was acting at the same time as the project manager. Based on the information provided in the PIR, the problem had been apparent since the project inception workshop, but could not have been solved. Later, a separate project manager was hired and replaced again in summer 2013 with a new project manager, while the former project manager has been continuing since then as the ?Project Implementation Coordinator?. After all these changes, however, there is still fundamental lack of clarity on who is actually managing the project. Most discussions during the project evaluation mission were conducted with the Project Implementation Coordinator, while for any substantial issues, the current project management team is frequently referring to the former NDD as the one, who continues to provide all the substantive advice and leadership for project implementation and, among others, is managing all the discussions with the project partners on any financing r elated matters. This clearly does not look like an acceptable and cost-effective way of managing projects of this or any kind (MTE, page 7).
Management Response: [Added: 2014/09/04]

UNDP agrees that the existing project governance, reporting and management arrangements have been inadequate. Since the beginning of the project implementation several attempts have been made to strengthen and improve the project management team that included replacement of project managers and engaging international consultants. However, these measures appeared to be insufficient as confirmed by the MTE. As a result and with the advice of the MTE the project governance and management arrangements have been changed entirely. The National Implementing Partner has been replaced because formerly the lack of ownership and engagement of the NIM Partner have been part of the problem with the project governance. Positions of the Deputy National Project Director and the Project Coordinator were abolished. Much stronger ownership and engagement of a new NIM Partner ? Russian Energy Agency ? have been secured. Clear reporting lines in the PMU have been set with the local technical coordinators reporting to the PM. PMU will be strengthened with several senior technical advisors working with the PM and the National Project Director. PSC will be reestablished. Effectiveness and efficiency of the new project management arrangement will be reassessed in the end of 2014.

Key Actions:

Key Action Responsible DueDate Status Comments Documents
Optimization of the project management structure. Enhancement of the project team with technical advisors.
[Added: 2014/09/04] [Last Updated: 2018/11/08]
Project team 2014/10 Completed Project restructuring conducted in 2014 that included replacement of the NIM partner, local responsible partner and the PMU (Project Manager was changed, the national CTA , as well as the find-raising adviser and the outreach coordinator were hired) History
6. Recommendation: The role of the project?s regional coordinators is unclear. They as opposite to their initially envisaged role to manage all regional project activities with a clear vision on what the project is expected to achieve in their particular region, have been assigned with quite specific and limited tasks only and in some cases were not even aware of some activities to be undertaken in their respective region (MTE, page 7).
Management Response: [Added: 2014/09/04]

The project document implies implementation of a significant number of objectives to achieve the general goal. The tasks were distributed among regional coordinators in the way to achieve specific results in each region and later replicate the achieved results in other pilot regions, as replication is one of the main project objectives. Perhaps the evaluator got wrong impression that the regional coordinators were not aware about the overall project strategic approach to the achievement of project goal. By now, the regional coordinators have been additionally updated about project strategy for the achievement of its general goal during participation in a joint project team meeting. The regional coordinators seem to have clear understanding of the issue. Hereafter joint project team meetings are to be held regularly to discuss and adjust the general project strategy. If needed, the terms of reference for the regional coordinators will be revised. Coordinators? efficiency will be assessed, and correspondingly, decisions will be made whether to extend their contracts or to replace them.

Key Actions:

Key Action Responsible DueDate Status Comments Documents
Joint project team meetings are to be held regularly to discuss and adjust the general project strategy. If needed, the terms of reference for the regional coordinators will be revised. Coordinators? efficiency will be assessed, and correspondingly, decisions will be made whether to extend their contracts or to replace them.
[Added: 2014/09/04] [Last Updated: 2018/11/08]
Project team 2014/09 Completed History
7. Recommendation: Apart from participation in a few meetings and workshops, there is no evidence about effective coordination and partnership building with other ongoing projects and initiatives dealing with the same substance area, including the two EBRD (one jointly with IFC) projects approved for GEF funding under the same GEF EE umbrella project, the ongoing international cooperation with the St. Petersburg Construction Committee to develop new construction norms there or on the other ongoing research work on EE building design and construction in Saint Petersburg and/or Moscow (MTE, page 8).
Management Response: [Added: 2014/09/04]

As mentioned in the comments to Issue 3, the analysis of similar projects had been carried out and the most notable experience applicable to the implementation of our project had been identified. For instance, IFC?s experience in fundraising for renovation of residential buildings. The project document envisages a demo-project in the Arkhangelsk Oblast on improvement of energy efficiency of already constructed typical residential buildings. One of the issues of this demo-project is identification of co-financing sources. Application of IFC?s experience will help in solving this. As for cooperation with St. Petersburg Construction Committee, in 2012 by the order of St. Petersburg Governor the Regional Methodological Document (RMD) came into effect. The RMD deals with the consideration of energy efficiency issues at the stages of building design and construction. One of RMD?s authors was engaged by the Project to develop a similar document for the Vologda Oblast. The Document has been prepared and is now under approval by a special working group under the Administration of the Vologda Oblast. One of the objectives for technical advisors who will be engaged by the project in June-July, 2014 will be the analysis of regulatory framework in Moscow/Moscow Oblast as regions with the most advanced regulations in construction sector, in order to assess the completeness of measures to improve energy efficiency in the legislation of Project?s pilot regions. If such measures are found insufficient, the project will deal with the development and promotion of additional measures based on the latest best practices in the field.

Key Actions:

Key Action Responsible DueDate Status Comments Documents
Establishment of tight cooperation with projects/companies whose activities correspond to the project goals. Continued cooperation with field-specific organizations and committees. Analysis of regulatory framework in Moscow/Moscow Oblast in order to assess the completeness of measures for energy efficiency in the building sector of the North-West Federal District.
[Added: 2014/09/04] [Last Updated: 2018/11/08]
Project team 2015/09 Completed History
8. Recommendation: The local working group established for the review, discussion and adoption of the new construction norms in Vologda is critical, but it does not compensate for those capacity constraints and lack of information that, for instance, the local design institutes at the oblast level still have in incorporating international state of the art approaches and lessons learnt for energy efficiency building design and construction into drafting of those norms. These constraints could have been overcome by more effective use of project resources allocated for capacity building and supporting international expertise (e.g. through the on the job training) and by identifying opportunities for co-operation and partnership building with both international and national expert institutions engaged in similar work in Russia (MTE, page 8).
Management Response: [Added: 2014/09/04]

We agree that one single working group is not enough to build capacity in the Vologda Oblast. The project has already undertaken measures to enhance qualification of the local specialists. The Center for Energy Efficiency was established under Vologda Technical University with Project?s support and a special training program based on best international practices was developed for the Center. In the future, the project plans organization of a number of trainings with the engagement of international experts and in partnership with other projects and organizations. The events will be organized not only for the Vologda Oblast specialists, but also for participants from other regions of Russia?s North-West.

Key Actions:

Key Action Responsible DueDate Status Comments Documents
Development of a plan of trainings for specialists from the North-West Federal District with the involvement of international experts. Search for potential partners for organization of the trainings
[Added: 2014/09/04] [Last Updated: 2018/11/08]
Project team Project partners 2014/12 Completed History
9. Recommendation: Regarding co-financing, the aggregated targets are likely to be met, but in leveraging required co-financing for individual activities such as the planned demo projects in Arkhangelsk, there seems to be major problems, which are currently slowing down the project implementation.
Management Response: [Added: 2014/09/04]

The recommendation is highly appreciated. During the joint project team meeting in July, 2014 the Arkhangelsk Oblast Coordinator was given a task to identify sources of co-financing for demo-projects on improvement of energy efficiency of already constructed typical residential buildings. Some possible ways to achieve this objective were outlined, including: - IFC?s experience in interaction with the banking sector in terms of the fulfillment of Federal Law #161 on financing buildings renovations by residential owners; - negotiations in the Oblast Administration in order to identify opportunities for financing from the regional and municipal budgets; - in cooperation with technical advisors identification of non-governmental institutions interested in project activities.

Key Actions:

Key Action Responsible DueDate Status Comments Documents
Identification of co-financing sources by above all: - Stronger engagement with the new NIM partner and the National Project Director to unleash the potential for federal and regional budget financing - Setting clear responsibilities for co-financing analysis and partnership building within the project team (national senior advisor and the PM) - Reiterating co-financing consultations with investment institutes and partners
[Added: 2014/09/04] [Last Updated: 2018/11/08]
Project team 2016/11 Completed The Project Team was looking for alternative sources of financing for the planned pilots in Arkhangelsk, including the non-governmental Fund. In cooperation with Arkhangelsk regional administration, the Project has short-listed 12 buildings that were included in the renovation program of the city of Arkhangelsk for 2015, out of which only one was finally selected for demo implementation. However, in the end the timing for this pilot to materialise and the regional co-financing to be confirmed was not adequate for the pilot to be finalised before the UNDP/GEF project completion, so the Project had to look for the alternative pilots in other Oblasts History
10. Recommendation: The biggest and most sustainable impact of the project is likely to be made by the educational programs (Outcome 2), subject to more detailed independent expert evaluation of their content and whether in line with international state of the art approaches and good practices on EE building design and construction. This review has not been done yet, but is suggested to be done at earliest (MTE, page 9).
Management Response: [Added: 2014/09/04]

The educational programs (training modules) are under approbation (piloting) by various educational institutions. Upon completion of the approbation, the training modules will be updated based upon lecturers?/teachers? recommendations. Upon completion of the approbation the training modules will also be additionally verified for their compliance with best practices and international experience. The updated modules will be published and widely disseminated in the pilot regions and all around Russia.

Key Actions:

Key Action Responsible DueDate Status Comments Documents
Upon completion of the approbation the training modules will be updated based upon lecturers?/teachers? recommendations and verified for compliance with best practices and international experience with the involvement of the Ministry of Education and Science of the Russian Federation and the Public Chamber of the Russian Federation.
[Added: 2014/09/04] [Last Updated: 2018/11/08]
Project team Project partners 2017/11 Completed History
11. Recommendation: Another project subcomponent proceeding relatively well and demonstrating some initial elements for sustainability is the energy monitoring and management system developed for and planned to be tested in the Pskov oblast. The concrete implementation plan and value added of the Energy Management system developed by another consultant group for the Arkhangelsk oblast is not completely clear yet and is likely to require some further research and consultations.
Management Response: [Added: 2014/09/04]

As mentioned in the comments to Issue 6, the tasks envisaged to achieve project objectives were distributed among the pilot regions with the view of further replication of the results achieved in all pilot regions. The energy management system (EMS) developed for the Arkhangelsk Oblast and the geo-information system (GIS) for energy monitoring developed for the Pskov Oblast have been joined together to achieve an aggregated result. The list of equipment necessary for GIS/EMS operation and program interface for various end users had been defined. The best model for the introduction of GIS/EMS is under development by the information technologies consultant. The suggested model will be tested in Arkhangelsk and Pskov oblasts.

Key Actions:

Key Action Responsible DueDate Status Comments Documents
Development of the best model for GIS/EMS introduction with its further testing in Arkhangelsk and Pskov oblasts.
[Added: 2014/09/04] [Last Updated: 2018/11/08]
Project team Project partners 2017/11 Completed Energy Management System (EMIS) put in operation in Pskov and Vologda oblasts History
12. Recommendation: The weakest prospects for sustainable project impact are currently with the planned demonstration projects in Arkhangelsk focusing on EE retrofits of existing buildings. At the time of this MTE, no concrete steps had been taken by the project yet to secure financing for these projects and both the regional and the core project team seemed not to be aware that the actual investments are not supposed to be funded by the GEF resources (MTE, page 9).
Management Response: [Added: 2014/09/04]

Indeed, co-financing and further replication are the main difficulties in the implementation of demo- projects in Arkhangelsk. As mentioned in comments to Issue 9, the Arkhangelsk Oblast Coordinator was given a task to identify sources of co-financing for demo-projects. Some possible ways to achieve this objective were outlined, including: - IFC?s experience in interaction with the banking sector in terms of the fulfillment of Federal Law #161 on financing buildings renovations by residential owners; - negotiations in the Oblast Administration in order to identify opportunities for financing from the regional and municipal budgets; - in cooperation with technical advisors identification of non-governmental institutions interested in project activities.

Key Actions:

Key Action Responsible DueDate Status Comments Documents
Identification of co-financing sources for demo-projects (see also key actions to issue 9)
[Added: 2014/09/04] [Last Updated: 2018/11/08]
Project team Project partners 2017/12 Completed All project demos duly cofinananced History
13. Recommendation: Another concern is based on the information received during the project evaluation mission that the elements included in the pilot building design have been used as a basis also for the proposed new construction norms in the Vologda region. The review of the design of the planned pilot building in Vologda at the end of 2013 by an international building energy efficiency expert contracted by the project revealed some significant shortcomings raising a question whether similar shortcomings and suboptimal requirements have remained in the new construction norms (MTE, page 9).
Management Response: [Added: 2014/09/04]

The evaluator got wrong impression. The Regional Methodological Document (RMD) and design documentation for the construction of pilot buildings in the Vologda Oblast are two different documents. In the Vologda Oblast two different types of activities are simultaneously under implementation: 1. The RMD has been developed by the project team in co-operation with specialists and is currently under consideration by the Vologda Oblast Administration. The RMD deals with the improvement of the existing regional construction norms in terms of their energy efficiency. The document has been under consideration since October 2013 and has not come into effect yet. The delay in RMD?s approval is explained by a number of stages it has to pass (architects, constructors, departments committees? approval within the oblast), as well as by staff changes in the Oblast Administration (new executives need time to get into the matter). In December 2013 the RMD was amended with recommendations of international project advisors. The project team envisages the approval of the updated RMD by the end of 2014. 2. Demonstration project on construction of energy efficient building. Within this objective design documentation of three typical 9-storey buildings was adjusted. Construction works were delayed because of the absence of the anticipated financing from the municipal budget. Besides, adjustments to the design documentation have not included all possible energy efficiency measures because the buildings? permanent structure had to remain intact, which was noted in the report of the international advisor engaged by the project in November 2013. By now, negotiations concerning other possible construction sites have been held. The construction site must satisfy the following requirements: - have clear financing sources; - give opportunity for the implementation all of the recommended energy efficiency measures; - allow starting the construction works in late 2014.

Key Actions:

Key Action Responsible DueDate Status Comments Documents
Interaction with the Vologda Oblast Administration on adoption and promotion of RMD Change of the construction site for the implementation of demo-project
[Added: 2014/09/04] [Last Updated: 2018/11/08]
Project team Project partners 2014/12 Completed History
14. Recommendation: The absence of credible business plans and uncertainties in the adequate demand of services for the distant learning centers established under component 2 in Vologda and the EE design office created by the AOEEC in Arkhangelsk are raising questions about their sustainability, especially as their exact role in supporting the other subcomponents of the project dealing with design and training related activities remained unclear. A vision shared by the local stakeholders in the interviews was that the GEF is expected to pay for the first 1-2 years of operation of those centres, after which they should become self-sustaining. There are considerable uncertainties and risks with this approach, however, which may have not been effectively addressed yet (MTE, page 9).
Management Response: [Added: 2014/09/04]

At the initial stage of the centers establishment as well as during meetings with their managers after their creation it was emphasized that the Project would make only a small financial contribution to the centers? activities, while centers? self-sustenance should be planned well in advance. The centers? managers were full aware of the Project financing policy from the very beginning. Most of the already created centers live on the organization of educational events, full-time and distant learning for specialists. The recently hired project technical advisors were given a task of analyzing capacities of the established centers and of helping them in identification of additional financing sources for their activities.

Key Actions:

Key Action Responsible DueDate Status Comments Documents
Assistance for energy efficiency centers in identification of additional financing sources for their activities
[Added: 2014/09/04] [Last Updated: 2018/11/08]
Project team Project partners 2015/09 Completed History
15. Recommendation: Finalized report was seen as an adequate result on its own without really thinking its contribution towards reaching the ultimate goals of the project. This approach, which has been unfortunately common also for many other international technical assistance projects, may leave behind an impressive pile of reports, but may not really produce results for replication, which is raising some concerns about the sustainable project impact in general.
Management Response: [Added: 2014/09/04]

The recommendation is highly appreciated. The project plans to revise the approaches that have been used before to achieve its objectives and to change its principles of target-formulation. The new approaches will be oriented at the achievement of the ultimate Project goals ? promotion of energy efficiency in the building sector of the pilot regions and the associated reduction of C02 emissions. To help with these several technical advisors have already been hired and some more technical specialists, including international ones, are planned to be engaged in the nearest future. The project will focus on the adoption of regulations in the field of building energy efficiency together with pilot regions authorities, on wide introduction of training modules, on replication of the obtained results at least in the pilot regions with prospects for their further replication throughout Russia. Also, the project has accumulated a number of reports prepared by its consultants. These will be evaluated and the most valuable and interesting ones will be published and disseminated among specialists and authorities of the pilot regions.

Key Actions:

Key Action Responsible DueDate Status Comments Documents
Revision of project strategy and communicating the project goals and objectives to partners and stakeholders. Strengthening the ownership and engagement of the new NIM partner to secure strategic governance and supervision Engagement of new technical specialists, including international ones, in the project. Systematization, evaluation, publication and dissemination of project results.
[Added: 2014/09/04] [Last Updated: 2018/11/08]
Project team Project partners 2017/11 Completed Overall Project Achievement and Impact was rated as Moderately Unsatisfactory.by the Final Evaluation. The FE confirmed a very strong ownership of and support from the new NIM partner and the National Project Director. Engagement of highly skilled technical personnel was ensured. The project did a lot to ensure adequate results dessimination and provide for replication of its most significant endeavous History
16. Recommendation: As a part of the required management changes and given the eventual difficulties to find experienced enough local project managers to run the project entirely on their own, the recruitment of an experienced international project management and technical advisor (with knowledge of the Russian language/Russian speaking staff and previous working experience in the Russian building sector) to support the project with any substantive issues, adaptive planning and management and ongoing progress monitoring is also strongly recommended with direct reporting responsibility to the project director, UNDP Regional Technical Adviser and UNDP CO (MTE, page 45).
Management Response: [Added: 2014/09/04]

The project team appreciates the evaluator?s recommendation to hire international specialists as technical advisors. Nevertheless, technical advisors? responsibilities include not only technical consulting on energy efficient construction issues, making evaluations and providing recommendations regarding project implementation. The most important part of technical advisors? work in our understanding is related to establishing and maintaining communications between the Project and authorities of different levels. Approval of changes in building norms, assistance in demo-projects implementation, replication of project results, co-financing of project activities are among the competencies of municipal, regional and federal authorities. Thus, along with professional skills and experience in energy efficiency, the technical advisors must know the structure of government institutions, be aware of changes in government organizations, personally know representatives of the authorities and be aware of protocol procedures. Considering these constraints, international advisors, non-residents of the Russian Federation are hardly capable to perform these functions. In order to accomplish these functions, the Project has hired technical advisors, residing in Moscow and is in the process of hiring some more technical advisors residing in St. Petersburg. In the future international advisors/consultants are planned to be hired to fulfill specific technical objectives of the Project.

Key Actions:

Key Action Responsible DueDate Status Comments Documents
Technical strengthening of the PMU through (i) recruiting senior technical advisors (completed) and (ii) employment of international advisors/consultants to fulfill specific technical objectives of the Project.
[Added: 2014/09/04] [Last Updated: 2018/11/08]
Project team 2014/12 Completed History
17. Recommendation: All the research work and project funds invested in energy audits, energy certificates and preparation of feasibility studies are pretty much useless, if not leading to concrete energy efficiency investments. For leveraging funding for these investments, the project needs to intensify its efforts (MTE, page 45).
Management Response: [Added: 2014/09/04]

The recommendation is highly appreciated. As mentioned above, the project vigorously analyzes other projects? experience in attracting co-financing for energy efficiency activities as well as seeks for alternative financing sources. Particularly, an agreement between IFC and the Project has been reached on sharing experience and on implementing joint activities to attract the banking sector in financing of overhauls and renovations of residential buildings in Russia. Negotiations have been held with regional administrations regarding the attraction of co-financing from regional and municipal budgets. Besides, the Project plans to identify non-governmental institutions that would be interested in financing project activities.

Key Actions:

Key Action Responsible DueDate Status Comments Documents
Analysis of current federal and regional legislation and regulatory framework on energy efficiency to identify budget lines that can be attributed to project activities. Cooperation with projects/companies that have experience in attracting co-financing for the implementation of demo-projects and energy efficiency measures/activities. See also key actions for issue 9.
[Added: 2014/09/04] [Last Updated: 2018/11/08]
Project team Project partners 2014/12 Completed History
18. Recommendation: The project inception period took longer than planned partly because of the on-going administrative reform and partly because the project ?lost? its initially approved demo site for the pilot buildings in Vologda and had to negotiate with the regional authorities about the transfer of the pilot site to another location. Given the current status of implementation, the project is neither likely to be able to complete all of its envisaged activities and reach the targeted objectives by the currently planned closing date of January, 2016 nor to spend all of its remaining financial resources by then. At the same time, the project scope and planned activities have remained highly relevant even for the coming years, if effectively and professionally implemented (MTE, page 47). The above imply that consideration of project extension will be required at some point in the future.
Management Response: [Added: 2014/09/04]

The recommendation is highly appreciated and we would like to discuss the following concerning the matter. During the years of project implementation a number of significant results were achieved (especially within the educational component, which was commended by the evaluator): - the Regional Methodological Document has been developed and is under consideration by regional authorities. The document implies changes to the construction norms in the Vologda Oblast and is expected to be adopted by the end of 2014; - a catalog of energy efficient solutions for reconstruction of typical buildings in the Arkhangelsk Oblast has been created. The catalog contains solutions that will be used during the implementation of demo-projects. The catalog has been passed to the Administration of the Arkhangelsk Oblast to be used for renovations of residential buildings. It will be uploaded to the project?s web-site and project partners? web-sites and will be passed to other pilot regions; - a geo-information system for accounting of energy resources has been launched to enable collection and analysis of the data necessary for calculation of economic effects from the implementation of energy efficiency measures in the Pskov Oblast; - seven training modules on energy efficiency for educational institutions from elementary schools to universities have been prepared and are under approbation. The modules will later be incorporated in regional educational programs; - Centers for Energy Efficiency have been established with project?s support in Vologda (under the Vologda State University) and Arkhangelsk (under the North Arctic Federal University). The Centers will provide full-time and distant learning on energy efficiency for specialists and other categories of students; - together with international specialists the current regional design and construction norms have been evaluated and a list of suggestions for their update has been prepared. Most of these project results are ready to be replicated. They will be eagerly used by specialists in education, design, construction and maintenance of residential buildings. Nevertheless, there are still shortcomings related mostly to the implementation of demo-projects in the Vologda Oblast (construction of energy efficient buildings) and Arkhangelsk Oblast (renovation of the already constructed buildings). These occurred mostly because of the following reasons: - the project actually began in 2011 (the project document was signed in November 2010), while initially it had been planned to start in 2010; - ongoing staff changes and administrative reforms noted in the MTE report were taking place simultaneously with the project implementation. The National Implementing Authority was changed. Agreements, which had been made with official representatives of administrations and ministries to support project implementation became useless because of staff changes. Much time and efforts were spent be the project team to find new contacts and to clarify project objectives to the newly appointed employees of relevant government institutions; - dependence of demo-projects implementation on co-financing from regional and municipal budgets. The project team has analyzed the factors and reasons which affected successful project implementation as well as possible ways of improving the situation. The following steps have already been taken: - project management structure was reorganized and optimized, technical advisors were hired to strengthen technical capacities of the project team; - major project priorities were reassessed and reformulated, among which identification of co-financing sources for the implementation of demo-projects, cooperation with projects/organizations with similar objectives, replication of already obtained and perspective results; - alternative sources of co-financing (different from government) for demo-projects are being actively sought for. Among possible investors banking sector and non-governmental foundations are considered; - the project has involved technical advisors who will embark on cooperation with state authorities of different levels regarding the development and promotion of new changes to federal and regional legislation and regulatory documentation for the building sector; - agreement was reached with the Ministry of Education and Science of the Russian Federation regarding publication and dissemination of training modules for schoolchildren and students in the pilot regions and all around Russia. Despite the undertaken measures, considering the current situation in Russia and taking into account the delay in project launch we assume that the project will not be able to achieve its target results specified in the strategic framework until its initial completion date at the end of 2015. Therefore, the project will apply for no-cost extension until the end of 2016/mid 2017.

Key Actions:

Key Action Responsible DueDate Status Comments Documents
Regular analysis of changing situation in Russia to ensure timely adjustment of project strategy and policies. Preparation and submission of a justified no-cost extension request with a revised work planned budget
[Added: 2014/09/04] [Last Updated: 2018/11/08]
Project team Project partners 2015/03 Completed History

Latest Evaluations

Contact us

220 East 42nd Street
20th Floor
New York, NY 10017
Tel. +1 646 781 4200
Fax. +1 646 781 4213
erc.support@undp.org