UNDP/GEF project "Removing barriers to energy efficiency improvements in the state sector in Belarus" final evaluation

Report Cover Image
Evaluation Plan:
2011-2015, Belarus
Evaluation Type:
Project
Planned End Date:
01/2012
Completion Date:
01/2012
Status:
Completed
Management Response:
Yes
Evaluation Budget(US $):
23,940

Share

Document Type Language Size Status Downloads
Download document ToR_PFE_International_v2.doc tor English 98.50 KB Posted 679
Download document Final Evaluation Report UNDP-GEF EE Project Belarus.doc report English 2651.50 KB Posted 1571
Download document Lessons Learnt UNDP-GEF EE Project Belarus.doc summary English 787.50 KB Posted 2202
Title UNDP/GEF project "Removing barriers to energy efficiency improvements in the state sector in Belarus" final evaluation
Atlas Project Number: 00050819
Evaluation Plan: 2011-2015, Belarus
Evaluation Type: Project
Status: Completed
Completion Date: 01/2012
Planned End Date: 01/2012
Management Response: Yes
Corporate Outcome and Output (UNDP Strategic Plan 2014-2017)
Evaluation Budget(US $): 23,940
Source of Funding:
Joint Programme: No
Joint Evaluation: No
Evaluation Team members:
Name Title Email Nationality
Jeroen Ketting Team Leader
GEF Evaluation: Yes
Evaluation Type:
Focal Area: Climate Change
Project Type: FSP
GEF Phase: GEF-2
PIMS Number: 2426
Key Stakeholders: Energy Efficiency Department of Gosstandard
Countries: BELARUS, REPUBLIC OF
Lessons
Findings
Recommendations
1 Project design should be based on a recent and fresh analysis of the needs of the actual and current barriers and opportunities in EE in Belarus through consultations with state officials, bankers, financiers, EE specialists, lawyers.
2 In future cases during the conception of the project design the terminology used in the Logframe should be defined more accurately. Inconsistencies in used terminology create problems and confusion during the project execution.
3 The economic and legal environment in countries like Belarus change continuously. Designing projects with a running length of 4 years makes it very difficult to foresee all the risks and changes that may arise. Therefore adaptive management should take place on a continual basis throughout the project, not only just after the mid-term evaluation. In this project, much time was lost while the project waited for the mid-term evaluation before any changes to the project strategy or project team were made.
4 The Logframe should define clear targets for outputs and outcomes. Open ended or vague targets should be avoided. Upon commencement of the project it is advisable that the project manager and responsible person from the UNDP country office discuss, agree on the meaning of the project?s Objective and the Outcomes. Monitoring needs to be ongoing and consistent. Resulting from the discussion of the project?s Objectives and Outcomes the project manager and the UNDP country office should adjust the outputs and activities where necessary.
5 Special care should be taken by the project manager and responsible person from the UNDP country office that the work plans are harmonized with the Logframe. The Logframe should be modified, as required, as early as the Project Inception workshop.
6 Monitoring and evaluation of the project results during the project should focus more at real on and quantitative results instead of solely focusing at whether the formal administrative requirements are met. This will allow for more stringent measurement and control of project outputs.
7 It should be made sure that the members of the Project Steering Committee are actually interested in the project and see themselves as stakeholders of the project. There should also be a fixed schedule for obligatory stakeholder meetings as one stakeholder meeting per year is not frequent enough to guarantee strong stakeholder involvement.
1. Recommendation: Project design should be based on a recent and fresh analysis of the needs of the actual and current barriers and opportunities in EE in Belarus through consultations with state officials, bankers, financiers, EE specialists, lawyers.
Management Response: [Added: 2012/06/20]

Agreed. The challenge for project implementers will be two-fold: a) being able to identify in the clearest way possible the challenges in the area the project is operating in, with a clear vision of upcoming short-term changes in the enabling environment; and, b) faster project registration (which in Belarus takes time) so that the situation does not change substantially between the time the project is designed and the time the project is registered and ready to start.

Key Actions:

Key Action Responsible DueDate Status Comments Documents
1.1 Improve program staff skills in project design by strengthening training and couching, and by recruiting stronger consultants (the rosters that are operation now help a lot in this direction).
[Added: 2012/06/20] [Last Updated: 2016/05/20]
Country Office 2013/12 Completed History
1.2 Continue the discussions with the government on faster project registration. Some results on this are already visible, but the procedure should be further simplified.
[Added: 2012/06/20] [Last Updated: 2016/05/20]
Country Office 2013/12 Completed History
2. Recommendation: In future cases during the conception of the project design the terminology used in the Logframe should be defined more accurately. Inconsistencies in used terminology create problems and confusion during the project execution.
Management Response: [Added: 2012/06/20]

Agreed. Better analytical skills in the Country Offices and more training for the programme staff will tackle this issue. Also, the build-up of experience in this area will improve the efficiency.

Key Actions:

Key Action Responsible DueDate Status Comments Documents
New staff with strong analytical skills to be recruited and effective training to be made available.
[Added: 2012/06/20] [Last Updated: 2016/05/20]
Country Office 2013/12 Completed History
3. Recommendation: The economic and legal environment in countries like Belarus change continuously. Designing projects with a running length of 4 years makes it very difficult to foresee all the risks and changes that may arise. Therefore adaptive management should take place on a continual basis throughout the project, not only just after the mid-term evaluation. In this project, much time was lost while the project waited for the mid-term evaluation before any changes to the project strategy or project team were made.
Management Response: [Added: 2012/06/20]

Agreed. This project was a good lesson that showed that there is no alternative to quick and strong action in redesigning the intervention in the conditions of a fast changing environment.

Key Actions:

Key Action Responsible DueDate Status Comments Documents
Introduce more frequent internal periodic reviews of projects.
[Added: 2012/06/20] [Last Updated: 2016/05/20]
Country Office 2013/12 Completed History
Raise the visibility and understanding of this issue within the UNDP CO and the government counterparts.
[Added: 2012/06/20] [Last Updated: 2016/05/20]
Country Office 2013/12 Completed History
4. Recommendation: The Logframe should define clear targets for outputs and outcomes. Open ended or vague targets should be avoided. Upon commencement of the project it is advisable that the project manager and responsible person from the UNDP country office discuss, agree on the meaning of the project?s Objective and the Outcomes. Monitoring needs to be ongoing and consistent. Resulting from the discussion of the project?s Objectives and Outcomes the project manager and the UNDP country office should adjust the outputs and activities where necessary.
Management Response: [Added: 2012/06/20]

Agreed. The quality of project documents should be strengthened, especially when it comes to the definition of outputs and outcomes. Also, communications between projects staff and country office staff should be strengthened at every stage of the project.

Key Actions:

Key Action Responsible DueDate Status Comments Documents
Targeted training for program and project staff on project management.
[Added: 2012/06/20] [Last Updated: 2016/05/20]
Country Office 2013/12 Completed History
Introduce more effective and improved communications strategy that will improve communications between projects and the country office.
[Added: 2012/06/20] [Last Updated: 2016/05/20]
Country Office 2013/12 Completed History
5. Recommendation: Special care should be taken by the project manager and responsible person from the UNDP country office that the work plans are harmonized with the Logframe. The Logframe should be modified, as required, as early as the Project Inception workshop.
Management Response: [Added: 2012/06/20]

Agreed. Indeed, constant monitoring of the situation and respective changes in the project design (including the Logframe) are necessary. Delaying changes in the logframe or project design until a formal evaluation is conducted (i.e. Mid-Term Evaluation) is problematic because by then too much time will have passed before the project is recalibrated to address the issue.

Key Actions:

Key Action Responsible DueDate Status Comments Documents
Introduce more frequent internal periodic reviews of projects.
[Added: 2012/06/20] [Last Updated: 2016/05/20]
Introduce more frequent internal periodic reviews of projects. 2013/12 Completed History
Targeted training for program and project staff on project management.
[Added: 2012/06/20] [Last Updated: 2016/05/20]
Country Office 2013/12 Completed History
6. Recommendation: Monitoring and evaluation of the project results during the project should focus more at real on and quantitative results instead of solely focusing at whether the formal administrative requirements are met. This will allow for more stringent measurement and control of project outputs.
Management Response: [Added: 2012/06/20]

Agreed. Even though it is difficult sometimes to move away from the formal administrative requirements in the environment of Belarus, it is fully possible to measure results more effectively based on quantitative indicators and practical results.

Key Actions:

Key Action Responsible DueDate Status Comments Documents
Targeted training for program and project staff on project management.
[Added: 2012/06/20] [Last Updated: 2016/05/20]
Country Office 2013/12 Completed History
New staff with strong analytical skills to be recruited and effective training to be made available.
[Added: 2012/06/20] [Last Updated: 2016/05/20]
Country Office 2013/12 Completed History
7. Recommendation: It should be made sure that the members of the Project Steering Committee are actually interested in the project and see themselves as stakeholders of the project. There should also be a fixed schedule for obligatory stakeholder meetings as one stakeholder meeting per year is not frequent enough to guarantee strong stakeholder involvement.
Management Response: [Added: 2012/06/20]

Agreed. The role of the Steering Committee should be strengthened. Not only meetings should be more frequent, but also the role of its members should be more active and substantive.

Key Actions:

Key Action Responsible DueDate Status Comments Documents
Increase up to three the minimal number of Steering Committee meetings per year.
[Added: 2012/06/20] [Last Updated: 2016/05/20]
Country Office 2013/12 Completed History
Introduce more effective and improved communications strategy that will improve communications between projects, the country office and the project?s partners.
[Added: 2012/06/20] [Last Updated: 2016/05/20]
Country Office 2013/12 Completed History
Create systems for more effective sharing of information between the project and the Steering Committee members.
[Added: 2012/06/20] [Last Updated: 2016/05/20]
Country Office 2013/12 Completed History

Latest Evaluations

Contact us

1 UN Plaza
DC1-20th Floor
New York, NY 10017
Tel. +1 646 781 4200
Fax. +1 646 781 4213
erc.support@undp.org