- Evaluation Plan:
- 2016-2020, Belarus
- Evaluation Type:
- Final Project
- Planned End Date:
- 04/2017
- Completion Date:
- 08/2017
- Status:
- Completed
- Management Response:
- Yes
- Evaluation Budget(US $):
- 10,000
INDEPENDENT FINAL EVALUATION: PROJECT ON “DEVELOPING AN INTEGRATED APPROACH TO A STEPPED-UP ENERGY SAVING PROGRAMME IN BELARUS”
| Document | Type | Language | Size | Status | Downloads |
|---|---|---|---|---|---|
 Final Evaluation report EE in Schools.docx |
report | English | 2619.71 KB | Posted | 1382 |
| Final Evaluation report UNDP-EU Belarus EE.docx |
report | English | 2665.92 KB | Posted | 1451 |
| Title | INDEPENDENT FINAL EVALUATION: PROJECT ON “DEVELOPING AN INTEGRATED APPROACH TO A STEPPED-UP ENERGY SAVING PROGRAMME IN BELARUS” | ||||||
|---|---|---|---|---|---|---|---|
| Atlas Project Number: | 00067546 | ||||||
| Evaluation Plan: | 2016-2020, Belarus | ||||||
| Evaluation Type: | Final Project | ||||||
| Status: | Completed | ||||||
| Completion Date: | 08/2017 | ||||||
| Planned End Date: | 04/2017 | ||||||
| Management Response: | Yes | ||||||
| Focus Area: |
|
||||||
| Corporate Outcome and Output (UNDP Strategic Plan 2018-2021) |
|
||||||
| SDG Target |
|
||||||
| Evaluation Budget(US $): | 10,000 | ||||||
| Source of Funding: | Project Budget | ||||||
| Evaluation Expenditure(US $): | 10,000 | ||||||
| Joint Programme: | No | ||||||
| Joint Evaluation: | No | ||||||
| Evaluation Team members: |
|
||||||
| GEF Evaluation: | No | ||||||
| Key Stakeholders: | |||||||
| Countries: | BELARUS, REPUBLIC OF |
| Lessons | |
|---|---|
| 1. | The project delivered almost all of its planned deliverables, and even more for some of the activities, e.g. training more people and completing the pilots not in 3 but 4 educational institutions. This is commendable given the large scope and complexity of the project compared to the short project duration and lean staffing, and the fact that it suffered delays caused by the lengthy process of hiring the staff. The pilots in their technical part were ahead of the peers in the country leading to significant interest in the innovation technologies that were piloted. Whether large scale replication will be forthcoming or not will depend on many external factors. The educational institutions had the chance to develop creative approaches in teaching energy saving especially to the lower and middle grades. The quality of the information products, including teaching aids is appreciated by the stakeholders, but – and this is a lesson learnt- in similar projects in the future it is important to secure more active participation of the Ministry of Education. The sheer quantity and variety of the information products developed under this project, as a sum of both streams (produced by the pilots and the project team) was huge; while this has advantages, the project could have had a stronger coordinating role and interface with the Ministry of Education. This is another lesson learnt, and particularly important when there are teaching aids/tools involved which require approval front eh Ministry of Education. And finally, in the part concerning the information campaign, there should have been mechanisms to capture the feedback of the recipients of the products (also a lesson learnt): timely analysis and learning from this analysis and acting upon this learning would have increased the effectiveness of the information component The PABs proved to be effective mechanisms for public monitoring of the pilot projects at the local level, but expecting that these would have an influence on the planning/investment at the local government level in EE was perhaps overly optimistic. The only activity where the project has hardly succeeded is related to the development of local EE strategies at the pilot locations, whereby, in a nutshell there was no apparent interest on their behalf in having such plans; the fact that there was no budget allocated for that activity did not help. The survey among the 23 schools, including schoolchildren, parents and teachers shows that while there is some increase in the awareness (more in the pilot schools) compared to the baseline, some are more effective than the others (and again this is more notable in the pilot schools). Awareness was raised more in relation to the potential of using solar energy than on importance of saving energy, and for the latter more in theory and not in the application to, say own household expenses. This shows that awareness raising must be well targeted and timely to affect the level of awareness and change behaviour (e.g. coincide with reforms). Another lesson is that apart from and in parallel to implementing awareness raising activities there is a need to institute mechanisms to assess the effectiveness and create feedback loops. |
| Findings | |
| 1. | 3. 1 Project formulation: relevance 3.1.1.Relevance of the Project focus The buildings’ sector has a large potential of energy savings for Belarus: it consumes 40 percent of heat used in the country, and as such represents an important potential source of greenhouse gas (GHG) emission reductions through fuel savings. More than 80 percent of the country’s residential stock, and about 95 percent of the public building stock was built before 1996. Building thermal protection standards were significantly strengthened in 1993 and updated in 2010. Pre-1996 buildings consume, on average, nearly twice as much energy per square meter as buildings constructed in the last four years, and this applies to the educational institutions also (see Figure 3). More than 90 percent of public buildings in Belarus were built before 1996, including 95 percent of kindergartens and secondary schools: 2,479 (about 94 percent) secondary schools and 2,236 kindergartens (about 96 percent) were built before 1996. Tag: Emission Reduction Energy Rule of law Technology Institutional Strengthening |
| 2. | The project is also relevant given the ongoing energy tariff reforms aimed at removing of subsidies and hence expected increases of the tariffs in the residential sector and removing cross subsidies in the public and industrial sectors, which requires better awareness of the benefits of higher EE by the population. Linked to this is the relevance of choosing schools as targets, given that they act as agents of change with children often bringing the knowledge they gain to the families, i.e. spreading the knowledge. To sum up (and also according to the previous experience in similar projects), institutional buildings, and in particular schools, kindergartens, boarding schools and vocational-technical schools are natural points of entry for EE activities at a local level for the following reasons:
Tag: Energy Relevance MDGs Partnership International Financial Institutions UN Agencies SDG Integration |
| 3. | 3.1.2 Implementation approach Barriers to EE in Public Buildings Barriers to EE in public buildings in Belarus fall into the following categories: legal, regulatory and institutional, incentive-related, and financing. The barriers are described in more detail below
Tag: Rule of law Human and Financial resources |
| 4. | Incentive-related
Tag: Energy Challenges Relevance |
| 5. |
The project pursued the promotion of energy savings in schools in participating communities with three- pronged approach, described in Figure 5. ProDoc Tag: Human and Financial resources |
| 6. | a) Public awareness and training in energy efficiency for local stakeholders in target districts. The project has a multifaced information component, including: thematic presentations for local authorities and communities; Quarterly information newsletter, thematic project web-site; Training of trainers (ToT) on EE issues; aim at introducing Energy Efficiency Extracurricular Subject in educational institutions; Information stands in pilot buildings; Information campaign, etc. This allowed to cover different segments of the society and tackle the task of increased awareness from various angles and with varying level of complexity. Tag: Energy Communication Awareness raising |
| 7. | b) Energy efficiency pilot projects implemented via a competitive grants scheme The project included:
Tag: Energy Challenges Effectiveness Technology |
| 8. | C) The use of Area Based Development (ABD) approach within the framework of sustainable energy management at the local level According to the Description of Action of the EU-UNDP Financing Agreement “…for the purposes of this project ABD is defined as an approach to development, which implies targeting specific geographical areas in a country, characterized a particular complex development problem, through an integrated. inclusive, participatory and flexible approach (UNDP, 2003)”. Specifically, the project was expected to: Tag: Relevance Local Governance Knowledge management Policies & Procedures Programme/Project Design Capacity Building |
| 9. | 3.1.3 Stakeholder participation/ownership The EED was part of the developing the project concept and this can be observed from the well drafted Component 2. However, in some aspects, especially in relation to some of the activities under Component 3 (Local EE strategies, as discussed) the design process required consultations at various level of the local government (region, oblast, city) to assess the level of interest before including this as part of the project proposal. Similarly, in the part of Component 1, a more active role of the Ministry of Education at the stage of the writing the proposal (as well as implementation, as will be discussed in Section 3.2.1) should have been assured to clarify the modes of participation and clarify the role. Tag: Ownership |
| 10. | 3.1.4 Cost-effectiveness The estimated economic performance, the low level of technological risk of the planned technical rehabilitation of the pilots and the fact that some of the main components were planned to be locally produced imply that cost effectiveness was treated as an important factor in the project design. Ideally concerns about the cost effectiveness of the awareness raising work under Component 1 should have prompted incorporation of measures that would have allowed to assess the effectiveness of various modes of work robustly. The project implementation Unit (PIU) had only 4 staff members. This has saved on administrative costs but arguably at the expense of efficiency and effectiveness to some – even if not large - extent. This is discussed more in Chapters 3.2 and 3.4). Similarly, the 3 years’ duration was overly optimistic given the ambitious goals of the project. Tag: Effectiveness Efficiency |
| 11. | 3.1.5 Replication approach and sustainability design The choice of the new technologies was informed by the desire to make them more likely to replicate. These were doomed to be easier to replicate given their estimated economic performance and them carrying low level of technological risk (important given the high visibility). In this way, the pilot buildings were meant to serve as important blueprints for other local governments and population, and provide a visible demonstration of the fuel-saving and money-savings results of the new higher standards. Piloting of other forms of EE technologies, e.g. heat recovery (a novelty, in essence), and the use of solar energy (also a relative novelty in Belarus) was supposed to trigger the interest and replication as well. To support the replication of the best practices, a series of in-country study tours and roundtable discussions for the EE pilot communities and potential stakeholders from other oblasts not involved in pilot renovations were planned (see the discussion in Section 3.3.1 and Recommendations). Thus, the design was adequate to trigger interest in replication, but hardly sufficient to bring about changes in the investment making policy in EE by the regional and local governments as was expected – perhaps too ambitiously- according to the Description of Action of the EU-UNDP Financing agreement, as discussed. Tag: Technology Policies & Procedures Education Local Governance Knowledge management |
| 12. | 3.1.6 Results framework The results framework (RF) of the project could have been better formulated. In particular, a) there are many targets in the text of the ProDoc but not in the RF, and, moreover, there are concerns related to their formulation. This list is presented below, but also in the text of the report later:
Tag: Results-Based Management Awareness raising Capacity Building |
| 13. | 3.1.6 Results framework The results framework (RF) of the project could have been better formulated. In particular, b) Plus, the outcome level results are mentioned as tasks in the ProDoc (page 9, para 2) but they are not all featured in the RF. This applies to Outcome 3, whereby the expected outcome level result, i.e. “Active participation of local governments, educational institutions and local population in energy-saving pilot initiatives at the local level, along with demonstration of best practices and replication ensured” is hard to capture with quantifiable evidence, and the Indicators from the RF (No. of representatives of local stakeholders trained in ABD participatory practices; and No. of public hearings conducted in every target region/territory) do not capture it well. Additionally, these 3 “tasks” represent actually different level of results: while “To raise awareness and build capacity within communities to carry out energy saving measures at local level that improve energy efficiency and utilize renewable energy sources” is a good outcome level expected result (for Component 1), the expected result for Component 2, namely, “To establish pilot sites in local communities demonstrating application of innovative energy efficiency technologies in school buildings” is actually an output level result. Tag: Local Governance Results-Based Management Capacity Building National Institutions |
| 14. | 3.1.7 Linkages between project and other interventions within the sector Apart from the related activities of the EU and UNDP discussed earlier, the project was also complementary to activities of some other international organizations and financing institutions which were ongoing or happened before/or approximately at the same time as the current project. In particular: Tag: Clean Energy Energy Environment Policy Partnership International Financial Institutions Awareness raising |
| 15. | 3.1.8 Management arrangements The EED was charged with the responsibility to ensure the successful implementation of the project activities, sustainability of the achieved results, as well as reporting to the Belarusian state authorities. It was expected to closely cooperate with UNDP to ensure that all the project activities are planned and implemented in a manner appropriate to the project’s goals and objectives. The EED was expected to assign a senior official as a National Project Coordinator responsible for project implementation on behalf of the EED. The Project Organization Structure (presented below in Figure 6) includes the Project Steering Committee (PSC), Project Assurance, and the Project Team. Tag: Coordination Oversight Project and Programme management Quality Assurance |
| 16. | 3.2 Results: attainment of outputs, outcomes and objectives 3.2.1 Outcome 1: Public awareness and training in energy efficiency in target districts. 78 participants took part in the Thematic presentations for local authorities and communities in October 2014 covering various aspects related to EE in buildings (including the Belarusian experience in EE, including national policies in this area). Thematic materials were developed, disseminated and placed on the project web-site There is no hard evidence to claim that at least 80 percent of the participants have improved their knowledge of all potential benefits of energy-saving technologies and the use of renewable energy - a target from the text of the ProDoc- as the feedback forms were collected but not systematically analysed. However, the interviews for this evaluation indicated satisfaction with the quality of the presentations, with a reflection that these helped them to increase their awareness related to EE measures in educational institutions. By the end of the project implementation, 10 issues of the information newsletter were published with 299 copies each and distributed to project stakeholders in target districts. 50 percent of the newsletter’s content presented by local stakeholders (local authorities, enterprises, local mass media, NGO, local population)- a target from the ProDoc - was taken as a guiding principle. No formal feedback mechanism was put in place (e.g. in the form of inserts, feedback box on the website, etc.) but the interviewed stakeholders for this evaluation commented positively on the usefulness of these in their work and, in general, for their awareness of EE. Tag: Communication Knowledge management Awareness raising |
| 17. | The Educational tools, which are covered by the list above deserve a more detailed discussion For the additional educational programme (outside the formal curricula, i.e. for coteries, elective or extracurricular classes etc.) the requirements are as follows to ensure that the certain educational aids could be used:
The educational aids under this project were produced both by the pilot educational institutions and by the project team:
Tag: Energy Knowledge management Education Awareness raising National Institutions |
| 18. | Achievement of expected Outcome: increased awareness The analysis of the results of the Energy Efficiency Awareness Survey (EEAS) 2014/2016 indicate that the awareness raising activities carried out by the Project for the period 2014-2016 did contribute to an increased level of awareness in 23 educational institutions that took part in this survey, but that the increase varied across the topics covered. In particular:
The formal indicator from the RF, namely “percent of pilot schools acknowledge improved awareness of pupils and their parents on the benefits of rational use of energy and energy efficient measures” is met. Tag: Education National Institutions |
| 19. | 3.2 Results: attainment of outputs, outcomes and objectives 3.2.2 Outcome 2: Energy efficiency pilot projects implemented via a competitive grants scheme. Outputs: Component 2 followed through the following steps: 1) Selection mechanism was developed. With Viciebsk, Hrodna and Minsk regions preselected, the following were the main principles of the selection mechanism, spelled out in the “Instructions to the applicants”:
Tag: Energy Education Capacity Building |
| 20. | 3) Training on the procedure for receiving grants. 68 representatives of schools, lyceums, vocational schools (27 participants from 18 education entities from Hrodna region, 15 participants from 8 education entities from Viciebsk region, 26 participants from 18 education entities from Minsk region) were trained on the procedure for receiving grants within the project (November 2015). A full package of methodological materials necessary for potential applicants (local governments or the educational institutions’ authorities) were developed. Project experts provided guidance (group, individual and on-line) to the applicants and issued clarifications on the application process, were necessary and on request (1 workshop in each target community was organized). Tag: Small Grants Programme Jobs and Livelihoods National Institutions |
| 21. | 5) Necessary design documentations were developed (a feasibility study, basic design, and detailed construction documents in accordance with local regulations) for the pilot projects. 6) Identification of co-funding opportunities. The Project Team supported the local governments in searching for additional co-funding sources for pilot initiatives, including in the form of preparation of necessary documents. It was expected that at least 40 percent of total pilot project costs will be covered by means mobilization of local resources in cash and in-kind (work, services, goods and etc.). This criterion was even surpassed in reality (see Table 10) 7) Implementation of EE increasing measures. The following EE measures were implemented at all four pilot educational institutions (see Figure 8):
Installation of the ventilation systems with heat recovery expected to allow heat loss reduction during air exchange by 28-30 percent. The pilots were expected to demonstrate the economic efficiency of this technology. This technology is a novelty in Belarus and the evidence suggests that even the energy experts visit the sites to see this in operation (in Viciebsk in particular);
Tag: Energy Effectiveness Resource mobilization Local Governance National Institutions |
| 22. | 9) Training of the local authorities and pilot projects implementers in the target districts in both (a) the rules of grants administration and good practices in project management (1 workshop in each target community was organized) and (b) Regular project monitoring. Site owners and local authorities were made responsible for monitoring EE and energy savings at pilot sites as a contribution to project implementation. Site owners collected and recorded day to day measurements of heat and electricity consumption, inside and outside temperature, fixing data in written form in special journals, further used for analysis by the Project Team. Monitoring relied on heat and electricity meters. The project EE expert trained 1 staff member from each pilot institution to be in charge of this. During the project implementation, regular site visits were made to pilot projects sites to check on the operation of the equipment installed and to undertake additional measurements needed. Monitoring reports on the pilot projects were made available on the project web-site. All 4 institutions had signed contracts with the suppliers as well as companies to provide services connected with maintenance and repair. Tag: Local Governance Education Capacity Building National Institutions Energy |
| 23. | Outcome Pilot sites demonstrating application of innovative energy efficiency technologies in social buildings are established. Four pilot sites demonstrating the application of innovative EE technologies in educational institutions of different types are now established and operational. The monitoring results demonstrate savings in both heat energy and electricity (see Table 8). For electricity, the achieved savings are more or close to the expected levels. As for the heating the achieved savings are somewhat less than expected, mostly due to pre-existing technical reasons, e.g. the heating systems in all four buildings not being well balanced and hence the need to overheat some of the rooms: based on the results of heat consumption monitoring recommendations on heating system balancing were provided to the management of these institutions. Also, it should be noted however, that the expected heat saving includes savings due to the use of heat recovery ventilation: taking into account, that actually old ventilation system was not used, this amount of saving was only expected, but not actual. Tag: Energy Innovation Technology |
| 24. |
3.2.3.Outcome 3: The use of an area-based development (ABD) approach within the framework of sustainable energy management at the local level Outputs UNDP and the Regional Executive Committees signed Memorandums of Understanding (MoU) at the start of the projects. These MOUs were supposed to witness the willingness and readiness to cooperate and provide mutual support within the framework of EU-UNDP project implementation. MoUs were rather generic: had they specified the roles with more level of detail and covering all activities, the lack of clarity in relation to the roles pertaining to the local EE strategies would have been clarified and sooner in the process. Also, Partnership Agreements were signed with the pilot institutions (2 each) spelling out UNDP and implementing entity roles during the pilot initiatives implementation process using the ABD approach as well as covering the responsibilities under the National Implementation Modality (NIM). Tag: Implementation Modality Results-Based Management |
| 25. | Creation and facilitation of EE Public Advisory Boards (PABs) operating under local authorities. The 4 public meetings aimed at raising awareness of the Project’s goal, objectives, activities and expected results were also used as platforms to promote the creation of the PABs: since there were representatives of the same organizations present, the project had rightly used the opportunity. Participants were invited to participate actively in the relevant project activities to be organized at the pilot sites, including PAB meetings and PR events. According to the Progress reports of the project, it took time, however, to agree with the pilot regional authorities on the responsible party (institution) to host the respective PABs: the ProDoc specifies 2 options, at the level of the Regional Executive Committees or at the level of the pilot institutions. It was finally agreed that PABs will be established on the basis of the pilot educational institutions (kindergartens in Ashmiany and Hrodna, college – in Viciebsk and secondary school – in Dziarzhynsk). Additional discussions were required to reach an agreement among all the stakeholders and this led to the delay in the formation of the PABs and lesser number of meetings (quarterly). By the time of the evaluation the PABs were present with 15 members at each pilot (13 in Ashmiany). The Project provided technical assistance in the process of formation and functioning of the PABs as well as ensured the availability of relevant information regarding the project implementation process, including with: Tag: Civic Engagement Local Governance Oversight Education Awareness raising Technical Support Civil Societies and NGOs National Institutions |
| 26. | Assistance in developing local EE strategies. Working meetings were supposed to be held with the representatives of local executive and administrative authorities and stakeholder groups with the (a) analysis of the needs of the target districts in the EE in education sphere and (b) determination of the main priorities of the districts and education sphere for reducing energy consumption in education establishments. This was supposed to be followed up next by the preparation of recommendations and proposals for the development of the local EE strategies in education sphere. This is perhaps the one subcomponent of the project which was the least successful and the reasons are multiple:
Tag: Energy Challenges Results-Based Management |
| 27. | Outcome: Active participation of local governments, educational institutions and local population in energy-saving pilot initiatives at the local level, along with demonstration of best practices and replication ensured The outcome specified in the Project Document (as one of the 3 Tasks on page 9), namely “Active participation of local governments, educational institutions and local population in energy-saving pilot initiatives at the local level, along with demonstration of best practices and replication ensured” is hard to capture with quantifiable but given the scope of the activities of the project, and the due to the existence of the PAGs, it could be said that this was achieved. Against the baseline of “Local population is not actively involved into sustainable local development planning”, the RF of the ProDoc had 2 indicators (NB: these do not capture the specified Outcome).
It was expected that the project will also work with local governments to improve their capacity to undertake energy saving investments: this is not in the ProDoc, but in the Description of Action of the EU-UNDP Financing Agreement: this would be hard to claim, given the status and level of interest in the local EE strategies.
Tag: Civil Societies and NGOs National Institutions |
| 28. | 3.3 Prospects for sustainability and replication 3.3.1.Rehabilitation with EE technologies Sustainability The high co-financing figures for the pilots (see Table 11) indicate strong local ownership and high likelihood that the provided infrastructure will be looked after. This is also supported by the fact that the 4 institutions have contracts with the supplier companies in case there is a need in spare parts as well as contracts for maintenance. One person from each institution was trained by the Project Team in carrying out routine daily monitoring function. Tag: Sustainability Ownership National Institutions |
| 29. | Replication The pilots serve as examples of an integrated approach to EE rehabilitation of public buildings: a need to shift to this approach rather than category- specific renovations (e.g. replacing the windows in certain number of buildings, then lighting, etc.) has been recognized by the EED. There is evidence that the interest on behalf of educational institutions in the regions and beyond is high in similar rehabilitation. The pilots also demonstrated that there is interest in co-funding such initiatives resulting in “pooled” funding model: with central government (EED), local government, enterprises- sponsors, the beneficiaries themselves (e.g. parents). In other words, securing half of the necessary funding from a dedicated funding (EU in this case, but it could be WB, EBRD, etc.,) may well attract other funders contributing to funding of the remaining amount. But the barriers in the form of the financial mechanisms, as well as of legislative and institutional nature that were discussed in Section 3.1.2 are significant, and therefore the potential for large scale replication with government sources is not high. There is one more barrier characteristic for the whole country: lack of technical specialists able to provide routine technical maintenance services, but there are already ideas for 2 regions on the establishment of specialized colleges. Tag: Efficiency Sustainability Human and Financial resources Reconstruction Technology |
| 30. | 3.3.2. EE as an extracurricular subject While there is no certainty, it is possible that at least for the near future the 4 educational institutions might continue in some form the creative educational activities involving other schools in the regions- provided that there is an interest on behalf of the Educational Departments of the Regional Executive Committees and financial reward for extra activities (which was present in the current project). As for teaching EE as a formal extracurricular subject, there is a good chance that the Manual for teaching EE in the school curricula will be used in the educational institutions since it has been already approved by the Ministry of Education. This was developed with the help of the experts from the Association for the Education for Sustainable Development (AESD), who have now, at the project end also expressed a keen interest in becoming the “home” for educational materials developed under the project. They have also expressed a commitment to ensure that all the products are reviewed by the Ministry of Education. This will also make it more likely that the best ones will be used more widely in Belarus (see Recommendations) The AESD unites juridical entities which are involved in the promotion of education in sustainable development. It is a non-for profit entity with the mission to coordinate the activities of the members vis a vis non-members (and in particular, the Ministry of Education). 3 of the pilots of the project are now members of the AESD (see Table 12). All 3, as well as the state educational institution "Kindergarten No. 6” in Ashmiany have already developed proposals to strengthen their role as the designated leaning centres: these proposals will need to be approved by the Educational Departments of the Reginal Executive bodies as well as the Ministry of Education (expected in August 2017). Staff of the Belarusian State Pedagogical University, the Academy of Postgraduate Education and the Republican Institute of Vocational Education are consultants of these innovative projects Tag: Education National Institutions |
| 31. | 3.3.3.Active participation of local population in sustainable development It would be perhaps too speculative to believe that the PAGs will continue to exist beyond the project. There are, however, interesting developments worth mentioning, In Hrodna for example, the director of the kindergarten plans to include some of the members of the PAG in the membership of its Board of Trustees. In Viciebsk, while the cooperation between the college and the local branch of EED was always strong, it has become even stronger as a result of the project and at least in the near horizon it looks like the college will serve as a demonstration basis for the EED to train the industry personnel and a basis for bringing together also other interested parties Tag: Civil Societies and NGOs |
| 32. | 3.3.4.Sustainability and replication potential – a summary Both EU and UNDP have more initiatives that directly or indirectly promote EE. At the time of evaluation EU had a grant competition open for non-state actors and, to the information available, the regional executive bodies which hosted the pilots were planning to apply. The EU/UNDP project "Support to Local Development in the Republic of Belarus" is implemented by UNDP in partnership with the Ministry of Economy of the Republic of Belarus. Another EU-UNDP project, “Supporting the Transition to a Green Economy in the Republic of Belarus” 2014-2017) aims at promoting “green growth” concepts and environmentally sustainable production and consumption patterns through support of local “green” initiatives and an information campaign as a support measure all over the Republic of Belarus. UNDP/GEF project (2016-2021) “Supporting Green Urban Development in Small and Medium-Sized Cities in Belarus (Green Cities)” promotes the elaboration of green urban development plans and pilot green urban development initiatives related to EE and sustainable transport in small and medium cities in Belarus. The project is implemented in the following areas:
These projects will potentially help in promoting the replication of the best practices from the current project Tag: Urbanization Sustainability Partnership |
| 33. | 3.4.1.Project planning, implementation and adaptive management Overall the project execution was satisfactory, as the project has achieved almost all the planned deliverables (with the exception of the website- based virtual network and with the caveat that the delivery of the local EE strategies could be considered as “achieved” in a rather formal sense of the word): this is noteworthy given that the project duration was too short, especially compared to its ambitious objectives. In relation Component 2, the execution of the project was indeed exemplary. In the case of Component 1, the project could have taken arguably a more active coordinating role vis a vis the Ministry of Education: as discussed this would have had positive impact on many aspects of the project, its effectiveness and sustainability potential. Another aspect where the project management could have been more efficient is in assessing the effectiveness of its multiple information products, even though this was not reflected in the formal results framework of the project. Tag: Challenges Effectiveness Ownership Project and Programme management Coordination |
| 34. | 3.4.2 Financial management The project was executed in the National execution modality (NIM). The financial management followed UNDP rules with the necessary due diligence in place. Cost effectiveness One of the project aims was to involve local companies and equipment producers from Belarus in the process of modernization of the pilot sites as much as possible. This target was reached: the majority of equipment and construction materials were purchased from local producers, for instance:
Tag: Efficiency Human and Financial resources |
| 35. | co-financing The promised co-financing materialized in full and even exceeded expectations (see Table 14) The contribution from the local governments was US$1,117 million instead of US$0,92 million exceeding the required 40 percent mark. Tag: Government Cost-sharing |
| 36. | 3.4.3.Reporting, Monitoring and RBM Project followed the bare minimum approach following the requirements of the RF (which has limited and very basic indicators). This reflects also to the quality of the RF of ProDoc which was discussed in 3.1.6 whereby there are targets specified in the body of the text of the ProDoc but not reflected in the RF (this was discussed in Section 3.1.6). In many respects, more could have been done, in particular with respect to: Capturing the effectiveness of various information products, e.g. ensuring that there are mechanisms for capturing the feedback of the users and using this feedback to adjust the next rounds of information campaign- in line with the best practices of results based management (RBM); Analysing and reflecting upon the feedback received from the participants of various workshops (feedback was collected but not analysed) Similarly, project reporting could have been better with reflections on lessons learnt as the project progressed. The lessons learned log was regularly monitored and updated by the project team, but the best practice is that lessons learnt are captured during the project implementation and published (see Recommendations). Tag: Knowledge management Results-Based Management |
| 37. | 3.4.4 Contribution of implementing and executing agencies EED provided strong support to the project especially in terms of ensuring the successful implementation of the pilots through its regional branches. EED could have assisted the project more in relation of attracting a closer involvement of the Ministry of Education. Perhaps EED could also have been more explicit in terms of ensuring the visibility of this project as a supporter of EED mandate and strategy. UNDP requirements outlined in the UNDP Programme and Operations Policies and Procedures, concerning the Project Assurance were pursued as expected. The project also benefitted from the oversight of the EU office in Belarus: there were cases where the timely observations from the EU have helped the project to address some of the issues (e.g. related to ensuring EU visibility). Tag: Oversight Project and Programme management |
| 38. | 3.4.5 Coordination and synergies The fact that the 2 UNDP projects – the current one and the UNDP/GEF project “Improving energy efficiency in residential building in the Republic of Belarus” - are co-located had perhaps helped to establish closer links between the two. For example, a conference dedicated to EE in public and residential sectors was organized jointly (and cost shared) on the 15th of October 2015 within the framework of Energy and Environment Forum. 149 representatives of local authorities, teachers and other stakeholder groups from the pilot regions learned about modern technologies and approaches to increase EE of social and residential buildings There were no formal linkages established with other international development agencies, and WB in particular. The EE expert engaged in the project has worked in the previous WB projects, and hence, informally the learning from these WB projects was transferred to the current project in the part concerning technical solutions. Closer links would have benefitted the project, e.g. in shaping the survey questions, in the light of the upcoming WB project related to EE in heating in residential buildings. Tag: Knowledge management Programme Synergy |
| Recommendations | |
|---|---|
| 1 | Provided that the idea is supported by EED, work on the handover to the Association for the Education for Sustainable Development (AESD) of all the soft copies of information products based on a MoU which will specify that (a) AESD will work with the Ministry of Education to review of all the educational tools are cleared to be used as part of the extracurricular teaching process as well as identify the ones that could be recommended to be used country wide (b) and all those that are cleared will be made available to the public free of charge. |
| 2 | Produce a Lessons Learnt report. There could be several booklets, with one of them dedicated to the analysis of economic costs and benefits of the pilots. The other one could feature the learning from the financing side of the pilots with an analysis of the potential for replication. The third one could feature the findings from the survey on awareness levels (but the findings need to be repackaged and presented better). After the booklets are produced, work with the EED to ensure its wide dissemination. |
| 3 | The quality of the information products, including teaching aids is appreciated by the stakeholders, but – and this is a lesson learnt – in similar projects in the future it is important to secure more active participation of the Ministry of Education. |
| 4 | And finally, in the part concerning the information campaign, there should have been mechanisms to capture the feedback of the recipients of the products (also a lesson learnt): timely analysis and learning from this analysis and acting upon this learning would have increased the effectiveness of the information component. |
| 5 | Given that upcoming tariff reform there might be justified need to support the population with further projects targeting public awareness raising as well as support the government with developing a subsidy scheme that will target the poor with their energy costs. |
1. Recommendation:
Provided that the idea is supported by EED, work on the handover to the Association for the Education for Sustainable Development (AESD) of all the soft copies of information products based on a MoU which will specify that (a) AESD will work with the Ministry of Education to review of all the educational tools are cleared to be used as part of the extracurricular teaching process as well as identify the ones that could be recommended to be used country wide (b) and all those that are cleared will be made available to the public free of charge.
Management Response: [Added: 2018/01/12] [Last Updated: 2021/02/19]
Discuss The project team to ensure that the proposed actions are taken. The CO to monitor the progress.
Key Actions:
2. Recommendation:
Produce a Lessons Learnt report. There could be several booklets, with one of them dedicated to the analysis of economic costs and benefits of the pilots. The other one could feature the learning from the financing side of the pilots with an analysis of the potential for replication. The third one could feature the findings from the survey on awareness levels (but the findings need to be repackaged and presented better). After the booklets are produced, work with the EED to ensure its wide dissemination.
Management Response: [Added: 2018/01/28] [Last Updated: 2021/02/19]
Ensure that the Lessons Learned log reflects all the important issues raised by the evaluation, along with lessons identified by the project team.
Key Actions:
3. Recommendation:
The quality of the information products, including teaching aids is appreciated by the stakeholders, but – and this is a lesson learnt – in similar projects in the future it is important to secure more active participation of the Ministry of Education.
Management Response: [Added: 2018/01/28] [Last Updated: 2021/02/19]
When preparing educational materials for children work closely with the Ministry of education and its agencies on the content of these materials, as well as on the use of these materials after projects end.
Key Actions:
4. Recommendation:
And finally, in the part concerning the information campaign, there should have been mechanisms to capture the feedback of the recipients of the products (also a lesson learnt): timely analysis and learning from this analysis and acting upon this learning would have increased the effectiveness of the information component.
Management Response: [Added: 2018/01/28] [Last Updated: 2021/02/19]
Consider such mechanisms to be developed and applied.
Key Actions:
5. Recommendation:
Given that upcoming tariff reform there might be justified need to support the population with further projects targeting public awareness raising as well as support the government with developing a subsidy scheme that will target the poor with their energy costs.
Management Response: [Added: 2018/01/28] [Last Updated: 2021/02/19]
Further develop projects targeting energy efficiency, including public awareness in this area, in the country reflecting the respective country’s priorities.
Key Actions: