INDEPENDENT FINAL EVALUATION: PROJECT ON “DEVELOPING AN INTEGRATED APPROACH TO A STEPPED-UP ENERGY SAVING PROGRAMME IN BELARUS”

Report Cover Image
Evaluation Plan:
2016-2020, Belarus
Evaluation Type:
Final Project
Planned End Date:
04/2017
Completion Date:
08/2017
Status:
Completed
Management Response:
Yes
Evaluation Budget(US $):
10,000

Share

Document Type Language Size Status Downloads
Download document Final Evaluation report EE in Schools.docx report English 2619.71 KB Posted 104
Download document Final Evaluation report UNDP-EU Belarus EE.docx report English 2665.92 KB Posted 55
Title INDEPENDENT FINAL EVALUATION: PROJECT ON “DEVELOPING AN INTEGRATED APPROACH TO A STEPPED-UP ENERGY SAVING PROGRAMME IN BELARUS”
Atlas Project Number: 00067546
Evaluation Plan: 2016-2020, Belarus
Evaluation Type: Final Project
Status: Completed
Completion Date: 08/2017
Planned End Date: 04/2017
Management Response: Yes
Focus Area:
  • 1. Environment & Sustainable Development
  • 2. Cross-cutting Development Issue
  • 3. Others
Corporate Outcome and Output (UNDP Strategic Plan 2014-2017)
  • 1. Output 1.5. Inclusive and sustainable solutions adopted to achieve increased energy efficiency and universal modern energy access (especially off-grid sources of renewable energy)
Evaluation Budget(US $): 10,000
Source of Funding: Project Budget
Evaluation Expenditure(US $): 10,000
Joint Programme: No
Mandatory Evaluation: No
Joint Evaluation: No
Evaluation Team members:
Name Title Email Nationality
GEF Evaluation: No
Key Stakeholders:
Countries: BELARUS, REPUBLIC OF
Lessons
1.

The project delivered almost all of its planned deliverables, and even more for some of the activities, e.g. training more people and completing the pilots not in 3 but 4 educational institutions. This is commendable given the large scope and complexity of the project compared to the short project duration and lean staffing, and the fact that it suffered delays caused by the lengthy process of hiring the staff. The pilots in their technical part were ahead of the peers in the country leading to significant interest in the innovation technologies that were piloted. Whether large scale replication will be forthcoming or not will depend on many external factors.

The educational institutions had the chance to develop creative approaches in teaching energy saving especially to the lower and middle grades. The quality of the information products, including teaching aids is appreciated by the stakeholders, but – and this is a lesson learnt- in similar projects in the future it is important to secure more active participation of the Ministry of Education. The sheer quantity and variety of the information products developed under this project, as a sum of both streams (produced by the pilots and the project team) was huge; while this has advantages, the project could have had a stronger coordinating role and interface with the Ministry of Education. This is another lesson learnt, and particularly important when there are teaching aids/tools involved which require approval front eh Ministry of Education. And finally, in the part concerning the information campaign, there should have been mechanisms to capture the feedback of the recipients of the products (also a lesson learnt): timely analysis and learning from this analysis and acting upon this learning would have increased the effectiveness of the information component

The PABs proved to be effective mechanisms for public monitoring of the pilot projects at the local level, but expecting that these would have an influence on the planning/investment at the local government level in EE was perhaps overly optimistic. The only activity where the project has hardly succeeded is related to the development of local EE strategies at the pilot locations, whereby, in a nutshell there was no apparent interest on their behalf in having such plans; the fact that there was no budget allocated for that activity did not help.

The survey among the 23 schools, including schoolchildren, parents and teachers shows that while there is some increase in the awareness (more in the pilot schools) compared to the baseline, some are more effective than the others (and again this is more notable in the pilot schools). Awareness was raised more in relation to the potential of using solar energy than on importance of saving energy, and for the latter more in theory and not in the application to, say own household expenses. This shows that awareness raising must be well targeted and timely to affect the level of awareness and change behaviour (e.g. coincide with reforms). Another lesson is that apart from and in parallel to implementing awareness raising activities there is a need to institute mechanisms to assess the effectiveness and create feedback loops.


Findings
1.

Project Results

Overall Achievement of Objective and Outcomes (Effectiveness): Are the actual project outcomes commensurate with the original or modified project objectives? In case the original or modified expected results are merely outputs/inputs then the Consultant should assess if there are any real outcomes of the project and if yes then whether these are commensurate with the realistic expectations from such a project.

Objective

S

 

Outcome 1

S

A stronger coordinating role of the project team in the information activities would have results in increased effectiveness of the project

Outcome 2

HS

The pilots are innovative and well executed

Outcome 3

MS

Only formal delivery of the local EE strategies


Recommendations
1

Provided that the idea is supported by EED, work on the handover to the Association for the Education for Sustainable Development (AESD)  of all the soft copies of information products based on  a MoU which will specify that (a) AESD will work with the Ministry of Education to review of all the educational tools are cleared to be used as part of the extracurricular teaching process as well as identify the ones that could be recommended to be used country wide (b) and all those that are cleared will be made available to the public free of charge.

2

Produce a Lessons Learnt report. There could be several booklets, with one of them dedicated to the analysis of economic costs and benefits of the pilots. The other one could feature the learning from the financing side of the pilots with an analysis of the potential for replication. The third one could feature the findings from the survey on awareness levels (but the findings need to be repackaged and presented better). After the booklets are produced, work with the EED to ensure its wide dissemination.

3

The quality of the information products, including teaching aids is appreciated by the stakeholders, but – and this is a lesson learnt – in similar projects in the future it is important to secure more active participation of the Ministry of Education.

4

And finally, in the part concerning the information campaign, there should have been mechanisms to capture the feedback of the recipients of the products (also a lesson learnt): timely analysis and learning from this analysis and acting upon this learning would have increased the effectiveness of the information component.

5

Given that upcoming tariff reform there might be justified need to support the population with further projects targeting public awareness raising as well as support the government with developing a subsidy scheme that will target the poor with their energy costs.

1. Recommendation:

Provided that the idea is supported by EED, work on the handover to the Association for the Education for Sustainable Development (AESD)  of all the soft copies of information products based on  a MoU which will specify that (a) AESD will work with the Ministry of Education to review of all the educational tools are cleared to be used as part of the extracurricular teaching process as well as identify the ones that could be recommended to be used country wide (b) and all those that are cleared will be made available to the public free of charge.

Management Response: [Added: 2018/01/12] [Last Updated: 2018/01/28]

Discuss The project team to ensure that the proposed actions are taken. The CO to monitor the progress.

Key Actions:

2. Recommendation:

Produce a Lessons Learnt report. There could be several booklets, with one of them dedicated to the analysis of economic costs and benefits of the pilots. The other one could feature the learning from the financing side of the pilots with an analysis of the potential for replication. The third one could feature the findings from the survey on awareness levels (but the findings need to be repackaged and presented better). After the booklets are produced, work with the EED to ensure its wide dissemination.

Management Response: [Added: 2018/01/28]

Ensure that the Lessons Learned log reflects all the important issues raised by the evaluation, along with lessons identified by the project team.

Key Actions:

3. Recommendation:

The quality of the information products, including teaching aids is appreciated by the stakeholders, but – and this is a lesson learnt – in similar projects in the future it is important to secure more active participation of the Ministry of Education.

Management Response: [Added: 2018/01/28]

When preparing educational materials for children work closely with the Ministry of education and its agencies on the content of these materials, as well as on the use of these materials after projects end.

Key Actions:

4. Recommendation:

And finally, in the part concerning the information campaign, there should have been mechanisms to capture the feedback of the recipients of the products (also a lesson learnt): timely analysis and learning from this analysis and acting upon this learning would have increased the effectiveness of the information component.

Management Response: [Added: 2018/01/28]

Consider such mechanisms to be developed and applied.

Key Actions:

5. Recommendation:

Given that upcoming tariff reform there might be justified need to support the population with further projects targeting public awareness raising as well as support the government with developing a subsidy scheme that will target the poor with their energy costs.

Management Response: [Added: 2018/01/28]

Further develop projects targeting energy efficiency, including public awareness in this area, in the country reflecting the respective country’s priorities.

Key Actions:

Latest Evaluations

Contact us

220 East 42nd Street
20th Floor
New York, NY 10017
Tel. +1 646 781 4200
Fax. +1 646 781 4213
erc.support@undp.org