Standards and labels to promote energy efficiency

Report Cover Image
Evaluation Plan:
2011-2017, Russian Federation
Evaluation Type:
Mid Term Project
Planned End Date:
03/2013
Completion Date:
06/2013
Status:
Completed
Management Response:
Yes
Evaluation Budget(US $):
20,000

Share

Document Type Language Size Status Downloads
Download document ToR-MTE SL.doc tor English 314.50 KB Posted 502
Download document MTE UNDPGEF SL for Promoting EE in the RF Final3107.docx report English 367.68 KB Posted 1204
Download document MTE SUMMARY UNDPGEF SL for Promoting EE in the RF Final3107.docx summary English 196.23 KB Posted 782
Title Standards and labels to promote energy efficiency
Atlas Project Number: 00070781
Evaluation Plan: 2011-2017, Russian Federation
Evaluation Type: Mid Term Project
Status: Completed
Completion Date: 06/2013
Planned End Date: 03/2013
Management Response: Yes
Focus Area:
  • 1. Environment & Sustainable Development
Corporate Outcome and Output (UNDP Strategic Plan 2014-2017)
  • 1. Local and national authorities have the capacities to access and integrate multiple sources of public and private environmental financing in support of sustainable human development, including gender equality and poverty reduction
Evaluation Budget(US $): 20,000
Source of Funding: GEF
Joint Programme: No
Joint Evaluation: No
Evaluation Team members:
Name Title Email Nationality
Jeroen Ketting Evaluation consultant jeroen@thelighthousegroup.ru
GEF Evaluation: Yes
Evaluation Type:
Focal Area: Climate Change
Project Type: FSP
GEF Phase: GEF-4
PIMS Number: 3550
Key Stakeholders: Ministry of Science and education
Countries: RUSSIAN FEDERATION
Lessons
Findings
Recommendations
1 Set clear Goals, Objectives and Activities in parallel and in accordance with the logframe. Clarify and simplify the project?s Goals, Objectives, Outcomes and activities without being detrimental to the Objective and Outcomes. Adjust the activities and outputs of the project design with the aim to make the project/logframe design more logical, strategically more relevant and more understandable (Section 1.03 Main conclusions, recommendations and lessons learned, (c) recommendations, p. 33) Set clear Goals, Objectives and Activities in parallel and in accordance with the logframe. Simplify the project without being detrimental to the Objective and Outcomes. Focus on a roadmap for a future voluntary or mandatory S&L Scheme as the main project outcome (Section 5.02 Recommendations, p. 81) The formulation of the outputs of the project design / logframe is often rather vague and ambiguous. (Section 5.02 Recommendations, p. 82) The formulation of the outputs of the project design / logframe should be made formulated in a clear and unequivocal manner. (Section 1.03 Main conclusions, recommendations and lessons learned, (d) , (d) Main interventions: Extension of project duration, Substantive revision of project strategy, Enforcement of project team and project management p. 37-38)
2 Part of the selected equipment types are subject to mandatory S&L and part to voluntary S&L. MTE advises to set up different implementation plans for the voluntary and mandatory S&L schemes (Section 1.03 Main conclusions, recommendations and lessons learned, (c) recommendations, p. 34)
3 Calculations of CO2 emission reduction are based on research carried out over the period 2009-2011 whereas the project?s activities largely commenced in the second half of 2011. It is not proven to the MTE that CO2 emission reductions were achieved by facilitation by the project of market transformation towards more energy efficient building equipment and appliances. MTE advises to carry out additional calculations covering the period 2011-2013. (Section 1.03 Main conclusions, recommendations and lessons learned, (c) recommendations, p. 34)
4 The project concerns equipment types that are subject to mandatory S&L and types that are subject to voluntary S&L. Mandatory and voluntary S&L schemes require different approaches. First of all the project needs to draft concise and clear action plans for both voluntary and mandatory equipment types (Plan M and Plan V). Plans M and V should describe what needs to be in place to fulfill Outcome 1 and the overall Objective of the project as stated in the logframe. Then Plans M and V should describe what the total of actions is that should be taken in general to fulfill the Outcome 1. After that Plans M and V should describe what interventions/actions out of this total of actions the project will engage in. After that the workplans should be made, the remaining budget should be contributed to the various activities and the actions should be fitted in the proper places in the logframe and agreed with the necessary bodies. This is what a normal basic constructive approach to the project should be instead of just simply focusing at individual project activities. Such an approach would also have the added benefit that the project team and project manager would have more success in communicating to the outside world what they are actually doing. (Section 1.03 Main conclusions, recommendations and lessons learned, (c) recommendations, p. 34)
5 Increase engagement of the stakeholders and engage on a continuous basis. Stakeholder engagement is not a one-off or ad-hoc activity but something that needs to be done during the entire project duration. (Section 1.03 Main conclusions, recommendations and lessons learned, (c) recommendations, p. 34), (Section 5.02 Recommendations, p. 81)
6 It is difficult to decrease the complexity of Russian legislation and regulations. Engage only in those legislative or regulative activities where actual effect can be realistically expected. Beyond that, design the EE S&L in such a way that it can function within existing legislative boundaries and reality. (Section 1.03 Main conclusions, recommendations and lessons learned, (c) recommendations, p. 34), (Section 5.02 Recommendations, p. 81)
7 Bring in foreign experts (like the people from CLASP or VTT or TNO) who have actual experience in designing and implementing EE S&L schemes to improve the logframe and function as long term project advisors. This MTE lists a number of suggestions in paragraph 4.0.3.b. (Section 1.03 Main conclusions, recommendations and lessons learned, (c) recommendations, p. 34-35), (Section 5.02 Recommendations, p. 81)
8 Complement continuous stakeholder engagement with continuous stakeholder training. (Section 1.03 Main conclusions, recommendations and lessons learned, (c) recommendations, p. 35). (Section 5.02 Recommendations, p. 81)
9 Enforce the project team and project management with additional project team members. A senior advisor to the project manager, a PR specialist and internal project analyst/consultant as soon as possible. Bring in communication, PR and networking capacity. (Section 1.03 Main conclusions, recommendations and lessons learned, (c) recommendations, p. 35) Enforce the project team and project management with additional project team members or strengthen the team in other ways. A senior advisor to the project manager, a PR specialist and internal project analyst/consultant should all be considered. (Section 5.02 Recommendations, p. 81-82) Ideally the project team and project management would be enforced with additional project team members. A part time international CTA with an S & L background, a senior advisor to the project manager, a PR specialist and internal project analyst/consultant should be considered. Project management needs to be improved. (Section 1.03 Main conclusions, recommendations and lessons learned, (d) , (d) Main interventions: Extension of project duration, Substantive revision of project strategy, Enforcement of project team and project management p. 37-38)
10 Create monthly M&E moments as the project needs more oversight and hands-on management. Set clear goals for the project team and project management and focus at clear deliverables and verify whether actual results achieved are supporting achievement of the outcomes and the final Objective. Teach the project team to manage by and focus on priorities. It is up to the project team and UNDP responsible organs to decide who should do this. (Section 1.03 Main conclusions, recommendations and lessons learned, (c) recommendations, p. 35) (Section 5.02 Recommendations, p. 82) Monthly M&E moments should be created and clear goals for the project team and project management should be set. The project team should also be steered towards focusing at clear deliverables and verification of whether actual results achieved are supporting achievement of the outcomes and the final Objective. The project team should be stimulated to manage by and focus on priorities. (Section 1.03 Main conclusions, recommendations and lessons learned, (d) , (d) Main interventions: Extension of project duration, Substantive revision of project strategy, Enforcement of project team and project management p. 37-38)
11 Bi-annual MTE type of reviews. Evaluate the results achieved more regularly and more critically. Set clear quarterly targets and deliverables for project team and project manager. (Section 1.03 Main conclusions, recommendations and lessons learned, (c) recommendations, p. 35) Bi-annual MTE type of reviews. Evaluate the results achieved more regularly and more critically. Set clear quarterly targets and deliverables for project team and project manager and make adaptive changes in case of non-delivery. (Section 5.02 Recommendations, p. 82) Considering the current state of the project, reviews should be carried out at least yearly and preferably twice a year either internally by UNDP or by an international CTA. The results achieved should be evaluated more regularly and more critically. Clear quarterly targets and deliverables must be set for the project team and project manager. (Section 1.03 Main conclusions, recommendations and lessons learned, (d) , (d) Main interventions: Extension of project duration, Substantive revision of project strategy, Enforcement of project team and project management p. 37-38)
12 Start with a good strategic overview and implementation plan that describes in detail all the steps needed for establishing an EE S&L scheme. (Section 1.03 Main conclusions, recommendations and lessons learned, (c) recommendations, p. 35). (Section 5.02 Recommendations, p. 82) It is essential to start with a good strategic overview and implementation plan that describes in detail all the steps needed for establishing an EE S&L scheme; a roadmap for an EE S&L scheme. (Section 1.03 Main conclusions, recommendations and lessons learned, (d) , (d) Main interventions: Extension of project duration, Substantive revision of project strategy, Enforcement of project team and project management p. 37-38)
13 Increase focus on capacity building inside the project team and increase attention to sustainability of capacity built outside the team. (Section 1.03 Main conclusions, recommendations and lessons learned, (c) recommendations, p. 35) (Section 5.02 Recommendations, p. 82) Capacity building inside the project team should be increased and closer attention must be paid to sustainability of capacity built outside the team. (Section 1.03 Main conclusions, recommendations and lessons learned, (d) , (d) Main interventions: Extension of project duration, Substantive revision of project strategy, Enforcement of project team and project management p. 37-38)
14 Extend the project from May 2015 to December 2016. This would give a remaining three and a half years for the project?s implementation. (Section 1.03 Main conclusions, recommendations and lessons learned, (c) recommendations, p. 35)
15 Rework the budget line items so that sufficient funds become available for hiring additional staff, bringing in foreign experts and moving to proper offices. (Section 1.03 Main conclusions, recommendations and lessons learned, (c) recommendations, p. 35)
16 Develop a replication approach foreseeing to establish knowledge transfer, expansion of demonstration projects or capacity building and training. (Section 1.03 Main conclusions, recommendations and lessons learned, (c) recommendations, p. 35)
17 MTE also questions how effective the NICB can be if it meets only once a year. The MTE also advises to hold bi-annual meetings. (Section 1.03 Main conclusions, recommendations and lessons learned, (c) recommendations, p. 35)
18 A project like this should be approached in a structured way. It requires work on the international level of the Customs Union. It requires co-ordination with: a. Ministries of Economic Development, b. Ministry of Regional Development, c. Ministry of Industry and Trade, d. Ministry of Energy, e. Rosstandart f. Government of Moscow g. Government of Nizhniy Novgorod. Doing a project like this without involving all these parties would create risks for the sustainability of its results. The reason for the need of involving all these parties is that there is a large number of diverse matters that need to be resolved. These matters for example relate to: h. harmonization of National Standards and Labels of the member states of the Custom Union, i. harmonization, co-ordination and development of the different schemes of voluntary certification that exist at this moment (answering questions such as who is the label operator, who gives the label, what are the independent laboratories), j. clarifying the instruments of stimulation of adoption of standards (for example tax incentives, subsidies, esco schemes related to labels), k. mandatory advertisement, l. EE standards and labels for buildings and EE management of buildings, m. establishment of an information system on EE equipment n. establishment of a register of EE labeled equipment, o. propaganda p. education initiatives, q. and so on. This MTE strongly recommends that the project formulates a clear and coherent policy and approach towards tackling this myriad of diverse matters that the project will have to deal with. The knowledge on what to do may exist in the brains of the different experts and contractors but the project would greatly benefit from having this approach written out and discussed with the various stakeholders. Just pointing at the Prodoc and workplans will not be sufficient to come to a satisfactory completion of the project. (Section 1.03 Main conclusions, recommendations and lessons learned, (c) recommendations, p. 35-36)
19 Extension of project duration Project implementation seems to be delayed and the project is likely to require an extension beyond its planned lifetime of five years from May 2010 till May 2015 to for example December 2016. It is unlikely that within the remaining project duration the remaining budget of USD 5,448,741.49 will be spent and all the outcomes will be fulfilled in a satisfactory manner. Reasons of delay are the slow project start, slow project procurement and moderately unsatisfactory project management for the entire duration of the project and small size of the project implementation team and poor working conditions. Extension should be granted under the condition that all the recommendations mentioned in this report are implemented. An extension granted under conditions of an enforced project team and project management and of a substantive revision of the approach to the project implementation strategy in close consultation with project stakeholders should contribute to a successful completion of the project. (Section 1.03 Main conclusions, recommendations and lessons learned, (d) , (d) Main interventions: Extension of project duration, Substantive revision of project strategy, Enforcement of project team and project management p. 36-37)
20 Substantive revision It is not guaranteed that the Activities lead to the desired Outputs and the Outputs lead to the desired Outcomes. As such, one may question to what extent the overall project Objective will be reached when maintaining the current project strategy. The changes suggested during the inception phase were mainly of a textual nature and almost exclusively related to the Final Value Targets. The current logframe may not provide the best project strategy for achieving the project objectives. For an EE S&L scheme to be developed logically the following steps need to be taken. These steps can be divided into a PLANNING PHASE and an IMPLEMENTING PHASE. The PLANNING PHASE should take about half a year. The IMPLEMENTING PHASE should take two to three years. An extension of the project with at least 18 months would be recommendable. My recommendation would be to: 1. Do a substantive revision based upon a specialist inventory of the actual results achieved and an inventory of the work still needed to be done. This work should be led by an international CTA with a standards and labelling background. 2. Extend the duration of the project with at least one and a half years 3. Make a new three and-a-half-year work-plan (from now till end of 2016) in the case extension is granted or a two year workplan in case extension isn?t granted. 4. Change the logframe by adjusting the activities and outputs while leaving the outcomes and objective unchanged. (Section 1.03 Main conclusions, recommendations and lessons learned, (d) , (d) Main interventions: Extension of project duration, Substantive revision of project strategy, Enforcement of project team and project management p. 37)
21 Enforcement of project team and project management Ideally the project team and project management would be enforced with additional project team members. A part time international CTA with an S & L background, a senior advisor to the project manager, a PR specialist and internal project analyst/consultant should be considered. Project management needs to be improved. Monthly M&E moments should be created and clear goals for the project team and project management should be set. The project team should also be steered towards focusing at clear deliverables and verification of whether actual results achieved are supporting achievement of the outcomes and the final Objective. The project team should be stimulated to manage by and focus on priorities. Considering the current state of the project, reviews should be carried out at least yearly and preferably twice a year either internally by UNDP or by an international CTA. The results achieved should be evaluated more regularly and more critically. Clear quarterly targets and deliverables must be set for the project team and project manager. It is essential to start with a good strategic overview and implementation plan that describes in detail all the steps needed for establishing an EE S&L scheme; a roadmap for an EE S&L scheme. The formulation of the outputs of the project design / logframe should be made formulated in a clear and unequivocal manner. Capacity building inside the project team should be increased and closer attention must be paid to sustainability of capacity built outside the team. (Section 1.03 Main conclusions, recommendations and lessons learned, (d) , (d) Main interventions: Extension of project duration, Substantive revision of project strategy, Enforcement of project team and project management p. 37-38)
1. Recommendation: Set clear Goals, Objectives and Activities in parallel and in accordance with the logframe. Clarify and simplify the project?s Goals, Objectives, Outcomes and activities without being detrimental to the Objective and Outcomes. Adjust the activities and outputs of the project design with the aim to make the project/logframe design more logical, strategically more relevant and more understandable (Section 1.03 Main conclusions, recommendations and lessons learned, (c) recommendations, p. 33) Set clear Goals, Objectives and Activities in parallel and in accordance with the logframe. Simplify the project without being detrimental to the Objective and Outcomes. Focus on a roadmap for a future voluntary or mandatory S&L Scheme as the main project outcome (Section 5.02 Recommendations, p. 81) The formulation of the outputs of the project design / logframe is often rather vague and ambiguous. (Section 5.02 Recommendations, p. 82) The formulation of the outputs of the project design / logframe should be made formulated in a clear and unequivocal manner. (Section 1.03 Main conclusions, recommendations and lessons learned, (d) , (d) Main interventions: Extension of project duration, Substantive revision of project strategy, Enforcement of project team and project management p. 37-38)
Management Response: [Added: 2014/05/12]

The project document is still relevant to provide the overall guidance on the project objectives and key expected outcomes. However, we agree with the evaluator that since the project was developed (2008) and the Inception Report prepared (2010) a number of important developments took place in the Russian normative and institutional framework that has to be captured by the project strategy. Also, we agree with the MTE that the Project Management Team should have a more strategic, clear and logical understanding and interpretation of the project strategy to communicate those to the key project stakeholders, beneficiaries and stakeholders. A number of key actions are under implementation to address the MTE recommendation (see below). Key actions: 1. The Project Management Team based on the consultations with the project Working Groups reviews the project document with the view to reflect recent changes in the national legal and regulatory framework, update the project strategy (outputs, activities, work plans) and present it in a strategic, clear and logical form. 2. A national advisor (CTA) for partnerships and stakeholder outreach is recruited to support the project team with above all the review of the project strategy. 3. An international S&L consultant will be recruited to review the project strategy and support the PM team with the strategic review of the project. 4. The revised project strategy and a clear set of goals, outcomes, outputs and activities to be presented to and discussed with the project stakeholders.

Key Actions:

Key Action Responsible DueDate Status Comments Documents
Key actions: 1. The Project Management Team based on the consultations with the project Working Groups reviews the project document with the view to reflect recent changes in the national legal and regulatory framework, update the project strategy (outputs, activities, work plans) and present it in a strategic, clear and logical form. 2. A national advisor (CTA) for partnerships and stakeholder outreach is recruited to support the project team with above all the review of the project strategy. 3. An international S&L consultant will be recruited to review the project strategy and support the PM team with the strategic review of the project. 4. The revised project strategy and a clear set of goals, outcomes, outputs and activities to be presented to and discussed with the project stakeholders.
[Added: 2014/05/12] [Last Updated: 2018/11/08]
Project team, NIM partner, UNDP CTA 2014/12 Completed A team of international consultants has been recruited to support the project with the strategic project review. The international team conducted several missions to Moscow and consultations with the project team/partners. The revised project strategy was presented in early 2015, duscussed with the key stakeholders and served a basis for workplanning of project activities in 2015-2016. History
2. Recommendation: Part of the selected equipment types are subject to mandatory S&L and part to voluntary S&L. MTE advises to set up different implementation plans for the voluntary and mandatory S&L schemes (Section 1.03 Main conclusions, recommendations and lessons learned, (c) recommendations, p. 34)
Management Response: [Added: 2014/05/12]

Indeed the project has been addressing S&L for different types of equipment (domestic appliances and building engineering equipment) through different strategies/activities (with a focus on mandatory labeling for the selected domestic appliances and with a focus on voluntary labeling for engineering equipment; this choice is driven by the Russian federal legislation). These implementation plans have been reflected through annual work plans without a stand-alone vision/document outlining the project strategy towards mandatory vs voluntary S&L through out the project lifetime. Such a document/implementation plan till the completion of the project will be prepared in the course of the strategic review discussed under the issue/recommendation #1 above. Key action: In the course of the review of the project strategy (see response to recommendation 1/key action) a coherent project implementation plan for the period until the completion of the project will be prepared that will above all outline the project vision towards the choice of mandatory/voluntary S&L schemes for various equipment types. In doing so that project will engage national and local advisors as outline in the response to recommendation 1 above.

Key Actions:

Key Action Responsible DueDate Status Comments Documents
In the course of the review of the project strategy (see response to recommendation 1/key action) a coherent project implementation plan for the period until the completion of the project will be prepared that will above all outline the project vision towards the choice of mandatory/voluntary S&L schemes for various equipment types. In doing so that project will engage national and local advisors as outline in the response to recommendation 1 above.
[Added: 2014/05/12] [Last Updated: 2018/11/08]
Project team UNDP CTA 2014/12 Completed Based on the experience of the UNDP/GEF project as well as of other international initiatives in Russia (e.g. EBRD labeling project) and according to the views of the international consultants who supported the strategic review, the mandatory S&L was confirmed to be a more effective strategy to secure market transformation in Russia. History
3. Recommendation: Calculations of CO2 emission reduction are based on research carried out over the period 2009-2011 whereas the project?s activities largely commenced in the second half of 2011. It is not proven to the MTE that CO2 emission reductions were achieved by facilitation by the project of market transformation towards more energy efficient building equipment and appliances. MTE advises to carry out additional calculations covering the period 2011-2013. (Section 1.03 Main conclusions, recommendations and lessons learned, (c) recommendations, p. 34)
Management Response: [Added: 2014/05/12]

Financing of EE S&L activities commenced in Russia since 2006 during the preparatory stage of the Project and they were financed by the Ministry of Education and Science and the UNDP. This work/financing by the national government stakeholders was catalyzed directly by the UNDP/GEF project development. Calculations of CO₂ emissions reduction were made for the period 2009 ? 2011 because 2009 was the year of the introduction of the 261 Law on energy effciency. These calculations have been necessary to (a) establish a baseline for the future monitoring of CO2 reductions through market transformation; (b) set up a clear methodology for the baseline and increment reductions; (c) reflect the progress achieved since the launch of the new EE legislation through the contribution of the national stakeholders (federal co-financing to the project). The project will now continue monitoring of the GHG emission reductions achieved through the introduction of S&L with the project. Calculations covering the period 2012-2013 will be done in 2014.

Key Actions:

Key Action Responsible DueDate Status Comments Documents
The project will carry out additional calculations regarding CO₂ emissions related to relevant HHA and TBE for Russian Federation and Moscow City for the period 2012 and 2013.
[Added: 2014/05/12] [Last Updated: 2018/11/08]
Project team 2015/12 Completed The project commissioned the work on the analysis of CO2 emission reductions, incl.those resulting from market transformation due to S&L. The report was commissioned by Ernst and Young towards the project completion History
4. Recommendation: The project concerns equipment types that are subject to mandatory S&L and types that are subject to voluntary S&L. Mandatory and voluntary S&L schemes require different approaches. First of all the project needs to draft concise and clear action plans for both voluntary and mandatory equipment types (Plan M and Plan V). Plans M and V should describe what needs to be in place to fulfill Outcome 1 and the overall Objective of the project as stated in the logframe. Then Plans M and V should describe what the total of actions is that should be taken in general to fulfill the Outcome 1. After that Plans M and V should describe what interventions/actions out of this total of actions the project will engage in. After that the workplans should be made, the remaining budget should be contributed to the various activities and the actions should be fitted in the proper places in the logframe and agreed with the necessary bodies. This is what a normal basic constructive approach to the project should be instead of just simply focusing at individual project activities. Such an approach would also have the added benefit that the project team and project manager would have more success in communicating to the outside world what they are actually doing. (Section 1.03 Main conclusions, recommendations and lessons learned, (c) recommendations, p. 34)
Management Response: [Added: 2014/05/12]

The Project fully understands that different equipment types (domestic appliances and building engineering equipment) require different approaches because they are subject to different EE S&L schemes (mandatory and voluntary). These different approaches have been reflected in the annual work plans but without a formal division of work plans into two parts (V and M) and without a stand-alone project lifetime implementation plan. The recommendation is appreciated and the project will prepare such a document/implementation plan with clear division between mandatory/voluntary S&L covering the period till the completion of the project in the course of the strategic review discussed under the issue/recommendation #1 above. In the course of the review of the project strategy (see response to recommendation 1/key action) a coherent project implementation plan for the period until the completion of the project will be prepared that will above all outline the project vision towards the choice of mandatory/voluntary S&L schemes for various equipment types. In doing so that project will engage national and local advisors as outline in the response to recommendation 1 above. The Project will start preparing plans of activities separately for different equipment types which require different approaches in terms of EE S&L schemes (mandatory and voluntary).

Key Actions:

Key Action Responsible DueDate Status Comments Documents
In the course of the review of the project strategy (see response to recommendation 1/key action) a coherent project implementation plan for the period until the completion of the project will be prepared that will above all outline the project vision towards the choice of mandatory/voluntary S&L schemes for various equipment types. In doing so that project will engage national and local advisors as outline in the response to recommendation 1 above. The Project will start preparing plans of activities separately for different equipment types which require different approaches in terms of EE S&L schemes (mandatory and voluntary).
[Added: 2014/05/12] [Last Updated: 2018/11/08]
Project team, CTA 2014/12 Completed A team of international consultants has been recruited to support the project with the strategic project review. The international team conducted several missions to Moscow and consultations with the project team/partners. The revised project strategy was presented in early 2015, duscussed with the key stakeholders and served a basis for workplanning of project activities in 2015-2016. History
5. Recommendation: Increase engagement of the stakeholders and engage on a continuous basis. Stakeholder engagement is not a one-off or ad-hoc activity but something that needs to be done during the entire project duration. (Section 1.03 Main conclusions, recommendations and lessons learned, (c) recommendations, p. 34), (Section 5.02 Recommendations, p. 81)
Management Response: [Added: 2014/05/12]

PMU comment: The Project implements a continuous approach in stakeholders? engagement. There are three levels of engagements: 1) annual NICB meetings (ministries and key interested parties); 2) three Project working groups which meet regularly (most qualified technical experts in relevant fields); 3) ad-hoc meetings for additional discussions when necessary with experts who are not members of the WGs. The recommendation is appreciated. See details in the ?Key actions? section below. UNDP response: UNDP agrees with the MTE that the measures taken by the PMU for stakeholder engagement and communication have been so far insufficient. UNDP believes that this has been a result of an inadequate PMU approach to stakeholder engagement and PMU capacities rather than of lacking formal instruments/venues for stakeholder consultations set up for the project. In order to address the situation UNDP suggested strengthening of the project management team through (a) recruiting an advisor for stakeholder engagement/communication, partnership buildings and (b) changing the PMU approach towards the stakeholder engagement/communication through improved transparency, a more clear and coherent communication of the project strategy and goals. Key actions: 1. The project recruits a Technical Adviser who will be responsible among other duties to improve stakeholder engagement and communication. 2. Review of the project strategy (see response to recommendation 1) and having a clear and up to date outline of the project goals, objectives, outcomes, outputs and activities for stakeholder communication will be an important instrument for stakeholder engagement.

Key Actions:

Key Action Responsible DueDate Status Comments Documents
1. The project recruits a Technical Adviser who will be responsible among other duties to improve stakeholder engagement and communication. 2. Review of the project strategy (see response to recommendation 1) and having a clear and up to date outline of the project goals, objectives, outcomes, outputs and activities for stakeholder communication will be an important instrument for stakeholder engagement.
[Added: 2014/05/12] [Last Updated: 2018/11/08]
project team, CTA 2015/06 Completed The Technical Advisor for engaging with institutional stakeholders has been recruited by the project. This has brought some improvement to the project communication and outreach. However, this measure appeared to be insufficient to secure a successful stakeholder communication and outreach. Upon consideration of this and other aspects of the project manager, UNDP and the NIM Partner took decision to replace the Project Manager. The project strategic review was completed by June 2015 History
6. Recommendation: It is difficult to decrease the complexity of Russian legislation and regulations. Engage only in those legislative or regulative activities where actual effect can be realistically expected. Beyond that, design the EE S&L in such a way that it can function within existing legislative boundaries and reality. (Section 1.03 Main conclusions, recommendations and lessons learned, (c) recommendations, p. 34), (Section 5.02 Recommendations, p. 81)
Management Response: [Added: 2014/05/12]

The Project is actually following the recommended way: in 2011 the analysis of existing legislative and normative base was done and relevant areas of legislation and regulations where actual effect can be realistically expected were identified. Thus the project is planning and implementing its normative/regulatory activities in line with the existing Russian national and sub-regional (Customs Union) framework for voluntary and mandatory S&L. The evaluator was provided with all the drafts of the legislative and normative documents developed by the Project. As for design of the EE S&L in accordance with the existing legislative boundaries and reality, the Project is working fully within this approach. All the national (GOST R) Standards developed by the Project were developed in accordance with 1) relevant EU technical regulations and 2) draft technical regulations of the Customs Union ?On informing customers on energy efficiency of electrical energy consuming appliances? which is awaiting adoption. Therefore, when this technical regulation of the Customs Union is adopted there will be no need to redevelop the GOST Standards.

Key Actions:

Key Action Responsible DueDate Status Comments Documents
No actions required
[Added: 2014/05/12]
n/a No due date No deadline established
7. Recommendation: Bring in foreign experts (like the people from CLASP or VTT or TNO) who have actual experience in designing and implementing EE S&L schemes to improve the logframe and function as long term project advisors. This MTE lists a number of suggestions in paragraph 4.0.3.b. (Section 1.03 Main conclusions, recommendations and lessons learned, (c) recommendations, p. 34-35), (Section 5.02 Recommendations, p. 81)
Management Response: [Added: 2014/05/12]

UNDP agrees with the MTE on the rationale to engage more international expertise to support the project strategy and implementation. UNDP consistently reiterates this role of the project to bridge national action with international best practice/expertise. International expertise will be engaged for the review of the project strategy (see response to recommendation 1) and further on in the project implementation based on the outputs of the strategy review/implementation planning. Key actions: 1. The project recruits international S&L advisor/consultant to support review of the project strategy/logframe and strategic planning. The review will be completed in June 2014. 2. Building on the outputs of the strategic review/planning and upon consultations with the project benevficiaries the areas for engaging state-of-the art international expertise will be identified.

Key Actions:

Key Action Responsible DueDate Status Comments Documents
1. The project recruits international S&L advisor/consultant to support review of the project strategy/logframe and strategic planning. The review will be completed in June 2014. 2. Building on the outputs of the strategic review/planning and upon consultations with the project benevficiaries the areas for engaging state-of-the art international expertise will be identified.
[Added: 2014/05/12] [Last Updated: 2018/11/08]
Project team 2015/12 Completed 1. A team of international consultants was recruited in 2014 to support the project with the strategic project review; 2. The project engaged the services of an Chief International Consultant, Mr. Necmettin Tokur to assist the PM with the project strategic supervision History
8. Recommendation: Complement continuous stakeholder engagement with continuous stakeholder training. (Section 1.03 Main conclusions, recommendations and lessons learned, (c) recommendations, p. 35). (Section 5.02 Recommendations, p. 81)
Management Response: [Added: 2014/05/12]

Recommendation is acknowledged. The project has been addressing training, capacity building and awareness needs of various stakeholder groups in different forms best tailored to the needs of these various groups ranging from technical conferences and workshop involving institutional stakeholders to professional training for building engineers and architects. Lately the project engaged also in EE education and training work with another UNDP/GEF building EE project in Russia. This work will be continued and further strengthened as recommended by the MTE.

Key Actions:

Key Action Responsible DueDate Status Comments Documents
Include activities on training, capacity building and awareness raising for various stakeholder groups in the project work plans. Inform stakeholders about planned training and awareness raising activities directly and through the publication on the Project web site.
[Added: 2014/05/12] [Last Updated: 2018/11/08]
Project team, CTA 2014/12 Completed Development of a long-term awareness raising strategy has been commissioned by the project by the end of 2014. Several cooperation agreements had been signed with major Russian retail networks (e.g. MediaMarkt, MVideo) and in partnership with them all-Russia information campaigns were organized with the coverage of 17 mln people. An information campaign to promote EE products was held in several internet shops. Training modules have been developed for some groups of stakeholders including schoolchildren and energy auditors. The training module for schoolchildren has been tested in some pilot schools and currently is under promotion among schoolteachers. The training module for energy auditors has been disseminated among energy audit self-regulatory organizations. A special expert has been hired to deliver regular trainings on EE of household appliances to students and civil servants. The total number of specialists trained since the project’s start has reached 4664 by the end of 2014. The trainings are organized in partnership with leading Russian technical universities and the largest energy distribution company Mosenergosbyt. History
9. Recommendation: Enforce the project team and project management with additional project team members. A senior advisor to the project manager, a PR specialist and internal project analyst/consultant as soon as possible. Bring in communication, PR and networking capacity. (Section 1.03 Main conclusions, recommendations and lessons learned, (c) recommendations, p. 35) Enforce the project team and project management with additional project team members or strengthen the team in other ways. A senior advisor to the project manager, a PR specialist and internal project analyst/consultant should all be considered. (Section 5.02 Recommendations, p. 81-82) Ideally the project team and project management would be enforced with additional project team members. A part time international CTA with an S & L background, a senior advisor to the project manager, a PR specialist and internal project analyst/consultant should be considered. Project management needs to be improved. (Section 1.03 Main conclusions, recommendations and lessons learned, (d) , (d) Main interventions: Extension of project duration, Substantive revision of project strategy, Enforcement of project team and project management p. 37-38)
Management Response: [Added: 2014/05/12]

Senior adviser to the project manager and PR specialist are already in the project team. Since then the project developed and started implementation of the EE awareness programme. As for internal project analyst/consultant, this function will be performed by a newly recruited staff (Chief Technical Adviser ? CTA) who reports directly to the NPD/UNDP. CTA?s key function is liaison with institutional and corporate stakeholders, support communicating project strategy and partnership building (TORs attached). The effectiveness of this PMU arrangement will be further assessed in the middle of 2014 to identify further scope for PMU strengthening in the field of stakeholder engagement and communication. Key actions: Recruitment of the CTA for stakeholder engagement, partnership building and communication. Assessment of the project performance on EE communication and awareness work planned with the project PR specialist. Internal review of the effectiveness of the strengthened PMU in the field of communication and stakeholder engagement in the middle of 2014.

Key Actions:

Key Action Responsible DueDate Status Comments Documents
Recruitment of the CTA for stakeholder engagement, partnership building and communication. Assessment of the project performance on EE communication and awareness work planned with the project PR specialist. Internal review of the effectiveness of the strengthened PMU in the field of communication and stakeholder engagement in the middle of 2014.
[Added: 2014/05/12] [Last Updated: 2018/11/08]
Project team, NIM partner, UNDP 2014/12 Completed The proejct recruited the stakeholder engagement TA and the communication/PR advisor. Several joint activities have been carried out with the marked chains, the project communication strategy has been commissioned by the end of 2014 to outline project medium-term plans for the communication, information and outreach activities in partnership with the market players and partners. History
10. Recommendation: Create monthly M&E moments as the project needs more oversight and hands-on management. Set clear goals for the project team and project management and focus at clear deliverables and verify whether actual results achieved are supporting achievement of the outcomes and the final Objective. Teach the project team to manage by and focus on priorities. It is up to the project team and UNDP responsible organs to decide who should do this. (Section 1.03 Main conclusions, recommendations and lessons learned, (c) recommendations, p. 35) (Section 5.02 Recommendations, p. 82) Monthly M&E moments should be created and clear goals for the project team and project management should be set. The project team should also be steered towards focusing at clear deliverables and verification of whether actual results achieved are supporting achievement of the outcomes and the final Objective. The project team should be stimulated to manage by and focus on priorities. (Section 1.03 Main conclusions, recommendations and lessons learned, (d) , (d) Main interventions: Extension of project duration, Substantive revision of project strategy, Enforcement of project team and project management p. 37-38)
Management Response: [Added: 2014/05/12]

PMU comments: Mechanism of M&E of the projects implemented under the National Implementation Scheme is defined by the UNDP and the Russian Government. The Project is preparing quarterly reports according to defined formats to both UNDP and the Ministry of Education and Science. Project Implementation Report (PIR) is being done annually. Project results achieved and plans are being discussed at the NICB meetings. UNDP response: UNDP agrees with the MTE on the need to strengthen strategic management of the project by the PMU. Monitoring of and guidance to the PMU has been and will be a joint effort of the National Implementing Partner (the Ministry of education and science of Russia) and UNDP. Regular communication with the project manager, CTA and NPD has been exercised by UNDP as a follow up to the MTE. PMU has been strengthened with new staff (CTA) and will be engaging international advisor for the strategic project review/planning. Further needs to strengthen the PMU will be assessed by UNDP with the NIM Partner through an internal review in mid-2014. Key action: Strengthen the PMU through new staff, strategic reviews and a more frequent monitoring/guidance by the NIM Partner and UNDP. Internal review of the effectiveness of the strengthened PMU in the middle of 2014.

Key Actions:

Key Action Responsible DueDate Status Comments Documents
Strengthen the PMU through new staff, strategic reviews and a more frequent monitoring/guidance by the NIM Partner and UNDP. Internal review of the effectiveness of the strengthened PMU in the middle of 2014.
[Added: 2014/05/12] [Last Updated: 2018/11/08]
Project team, NIM partner, UNDP 2014/12 Completed UNDP has been conducting regular meetings with the project team focusing on various strategic, operational and procedural issues. A team of international consultants has been engaged to support the PMU with the strategic review. Review of the PMU effectiveness and of the PM performance/ability to improve has been conducted jointly by the NIM partner and UNDP. As a result UNDP and the NIM Partner took decision to replace the Project Manager as of September 2014. History
11. Recommendation: Bi-annual MTE type of reviews. Evaluate the results achieved more regularly and more critically. Set clear quarterly targets and deliverables for project team and project manager. (Section 1.03 Main conclusions, recommendations and lessons learned, (c) recommendations, p. 35) Bi-annual MTE type of reviews. Evaluate the results achieved more regularly and more critically. Set clear quarterly targets and deliverables for project team and project manager and make adaptive changes in case of non-delivery. (Section 5.02 Recommendations, p. 82) Considering the current state of the project, reviews should be carried out at least yearly and preferably twice a year either internally by UNDP or by an international CTA. The results achieved should be evaluated more regularly and more critically. Clear quarterly targets and deliverables must be set for the project team and project manager. (Section 1.03 Main conclusions, recommendations and lessons learned, (d) , (d) Main interventions: Extension of project duration, Substantive revision of project strategy, Enforcement of project team and project management p. 37-38)
Management Response: [Added: 2014/05/12]

UNDP agrees with the MTE on the need to strengthen the monitoring of the project and to significantly strengthen capacities of the PMU that has to be able carry out strategic planning and management of the project beyond the level of stand-alone activities/outputs. Since the MTE several steps have been taken: more frequent monitoring/consultations with the project team, recruitment of the CTA, initiation of the project logframe/strategy review engaging an international consultant to be recruited. The strategic review (see also response to recommendation 1) to be carried out with the engagement of the international consultant will be the first practical step to implement this MTE recommendation. Key action: Recruitment of the CTA and engaging international consultant for a strategic project review/planning (in progress). Strengthened monitoring and evaluation on the side of the NIM partner and UNDP. Strengthening and empowering the PMU and assessment of the PMU effectiveness in mid-2014.

Key Actions:

Key Action Responsible DueDate Status Comments Documents
Recruitment of the CTA and engaging international consultant for a strategic project review/planning (in progress). Strengthened monitoring and evaluation on the side of the NIM partner and UNDP. Strengthening and empowering the PMU and assessment of the PMU effectiveness in mid-2014.
[Added: 2014/05/12] [Last Updated: 2018/11/08]
Project team, NIM partner, UNDP 2014/12 Completed History
12. Recommendation: Start with a good strategic overview and implementation plan that describes in detail all the steps needed for establishing an EE S&L scheme. (Section 1.03 Main conclusions, recommendations and lessons learned, (c) recommendations, p. 35). (Section 5.02 Recommendations, p. 82) It is essential to start with a good strategic overview and implementation plan that describes in detail all the steps needed for establishing an EE S&L scheme; a roadmap for an EE S&L scheme. (Section 1.03 Main conclusions, recommendations and lessons learned, (d) , (d) Main interventions: Extension of project duration, Substantive revision of project strategy, Enforcement of project team and project management p. 37-38)
Management Response: [Added: 2014/05/12]

Recommendation acknowledged and will be implemented as part of the strategic review/planning (see response to recommendation 1).

Key Actions:

Key Action Responsible DueDate Status Comments Documents
See key actions under Issue/Recommendation 1.
[Added: 2014/05/12] [Last Updated: 2018/11/08]
Project team, NIM partner, UNDP 2014/12 Completed History
13. Recommendation: Increase focus on capacity building inside the project team and increase attention to sustainability of capacity built outside the team. (Section 1.03 Main conclusions, recommendations and lessons learned, (c) recommendations, p. 35) (Section 5.02 Recommendations, p. 82) Capacity building inside the project team should be increased and closer attention must be paid to sustainability of capacity built outside the team. (Section 1.03 Main conclusions, recommendations and lessons learned, (d) , (d) Main interventions: Extension of project duration, Substantive revision of project strategy, Enforcement of project team and project management p. 37-38)
Management Response: [Added: 2014/05/12]

Recommendation is accepted and will be implemented. The wider project team (including technical specialists, team leaders and working groups) possesses very strong technical capacities in EE S&L but might be lacking an overall strategic vision/empowerment that has to be secured by the core PMU/project manager. Project team had been lacking adequate capacities in communication and stakeholder engagement. Steps taken to strengthen the core PMU with a CTA, PR expert and international consultant as well as to encourage the PMU empowering the broader project team and stakeholder groups. Key actions: 1. PR expert is already in place and started working. 2. CTA recruited. 3. Head of the working group on enabling policies and implementation of the Moscow pilot programme will be selected and recruited and will start working on a paid basis substituting voluntary head of the working group.

Key Actions:

Key Action Responsible DueDate Status Comments Documents
1. PR expert is already in place and started working. 2. CTA recruited. 3. Head of the working group on enabling policies and implementation of the Moscow pilot programme will be selected and recruited and will start working on a paid basis substituting voluntary head of the working group. 4. Capacity building to the PMU to be strengthened
[Added: 2014/05/12]
Project team, UNDP No due date No deadline established
14. Recommendation: Extend the project from May 2015 to December 2016. This would give a remaining three and a half years for the project?s implementation. (Section 1.03 Main conclusions, recommendations and lessons learned, (c) recommendations, p. 35)
Management Response: [Added: 2014/05/12]

UNDP and the project team agree with the recommendation. The project will need extension until May 2017 to accommodate above all late approval and signature of the project. The initial request for project extension has been prepared and reviewed by UNDP and UNDP/GEF. As per the MTE recommendation #1 and upon consultations with UNDP HQs it was agreed to endorse the project extension formally upon the following action by the project: (a) outputs of the project strategy/logframe/planning review, (b) substantive strengthening of the project team in particular in the areas of strategic management, planning, communication and stakeholder engagement, (c) a justified/strengthened extension request accommodating the above. Key action: Prepare a justified extension request based on the strategic project review and PMU strengthening. Initiate the procedure for Project extension up to May 2017.

Key Actions:

Key Action Responsible DueDate Status Comments Documents
Prepare a justified extension request based on the strategic project review and PMU strengthening. Initiate the procedure for Project extension up to May 2017.
[Added: 2014/05/12] [Last Updated: 2018/11/08]
Project team, UNDP 2015/04 Completed History
15. Recommendation: Rework the budget line items so that sufficient funds become available for hiring additional staff, bringing in foreign experts and moving to proper offices. (Section 1.03 Main conclusions, recommendations and lessons learned, (c) recommendations, p. 35)
Management Response: [Added: 2014/05/12]

Recommendation is accepted and will be implemented: Secure sufficient project funds for effective project management and operations while respecting GEF project management budget limits (NPD and UNDP/GEF). Ensure that the management funds are utilized effectively and the PMU is delivering value for money.

Key Actions:

Key Action Responsible DueDate Status Comments Documents
Secure sufficient project funds for effective project management and operations while respecting GEF project management budget limits (NPD and UNDP/GEF). Ensure that the management funds are utilized effectively and the PMU is delivering value for money.
[Added: 2014/05/12] [Last Updated: 2018/11/08]
Project team, NIM partner, UNDP/GEF 2016/06 Completed History
16. Recommendation: Develop a replication approach foreseeing to establish knowledge transfer, expansion of demonstration projects or capacity building and training. (Section 1.03 Main conclusions, recommendations and lessons learned, (c) recommendations, p. 35)
Management Response: [Added: 2014/05/12]

Agree with the recommendation. Key actions: Select appropriate Project Partners (business associations, Mosenergosbyt, etc.), conclude agreements and launch continuous activities for knowledge transfer, expansion of demonstration projects and training in the following fields: 1. Awareness raising campaigns including PR actions; 2. Training of sales personnel related to HHA; 3. Training of interested parties (designers, engineers, commercial buyers, etc.) related to TBE. Replicate and disseminate experience gained in the Moscow pilot region in the second pilot region (N.Novgorod), including development of regional legislative and normative base to promote EE S&L and the variety of training activities.

Key Actions:

Key Action Responsible DueDate Status Comments Documents
Select appropriate Project Partners (business associations, Mosenergosbyt, etc.), conclude agreements and launch continuous activities for knowledge transfer, expansion of demonstration projects and training in the following fields: 1. Awareness raising campaigns including PR actions; 2. Training of sales personnel related to HHA; 3. Training of interested parties (designers, engineers, commercial buyers, etc.) related to TBE. Replicate and disseminate experience gained in the Moscow pilot region in the second pilot region (N.Novgorod), including development of regional legislative and normative base to promote EE S&L and the variety of training activities.
[Added: 2014/05/12] [Last Updated: 2018/11/08]
Project team 2016/12 Completed History
17. Recommendation: MTE also questions how effective the NICB can be if it meets only once a year. The MTE also advises to hold bi-annual meetings. (Section 1.03 Main conclusions, recommendations and lessons learned, (c) recommendations, p. 35)
Management Response: [Added: 2014/05/12]

Recommendation is accepted and discussed at the NICB (Steering Committee meeting). Bi-annual NICB meetings will be conducted upon endorsement of the NIM partner (virtual meetings is an option).

Key Actions:

Key Action Responsible DueDate Status Comments Documents
Bi-annual NICB meetings will be conducted upon endorsement of the NIM partner (virtual meetings is an option).
[Added: 2014/05/12] [Last Updated: 2018/11/08]
Project team, NIM partner 2016/12 Completed History
18. Recommendation: A project like this should be approached in a structured way. It requires work on the international level of the Customs Union. It requires co-ordination with: a. Ministries of Economic Development, b. Ministry of Regional Development, c. Ministry of Industry and Trade, d. Ministry of Energy, e. Rosstandart f. Government of Moscow g. Government of Nizhniy Novgorod. Doing a project like this without involving all these parties would create risks for the sustainability of its results. The reason for the need of involving all these parties is that there is a large number of diverse matters that need to be resolved. These matters for example relate to: h. harmonization of National Standards and Labels of the member states of the Custom Union, i. harmonization, co-ordination and development of the different schemes of voluntary certification that exist at this moment (answering questions such as who is the label operator, who gives the label, what are the independent laboratories), j. clarifying the instruments of stimulation of adoption of standards (for example tax incentives, subsidies, esco schemes related to labels), k. mandatory advertisement, l. EE standards and labels for buildings and EE management of buildings, m. establishment of an information system on EE equipment n. establishment of a register of EE labeled equipment, o. propaganda p. education initiatives, q. and so on. This MTE strongly recommends that the project formulates a clear and coherent policy and approach towards tackling this myriad of diverse matters that the project will have to deal with. The knowledge on what to do may exist in the brains of the different experts and contractors but the project would greatly benefit from having this approach written out and discussed with the various stakeholders. Just pointing at the Prodoc and workplans will not be sufficient to come to a satisfactory completion of the project. (Section 1.03 Main conclusions, recommendations and lessons learned, (c) recommendations, p. 35-36)
Management Response: [Added: 2014/05/12]

Agree on this recommendation. Initiate a strategic project review coming up with a clear, strategic and coherent project strategy to be communicated to the various project institutional stakeholders. The CTA recruited to carry out consultations with all the above stakeholders and strengthen project?s stakeholder engagement work. See also response to recommendation #1. Key actions: 1. The Project Management Team based on the consultations with the project Working Groups reviews the project document with the view to reflect recent changes in the national legal and regulatory framework, update the project strategy (outputs, activities, work plans) and present it in a strategic, clear and logical form. 2. A national advisor (CTA) for partnerships and stakeholder outreach is recruited to support the project team with above all the review of the project strategy. 3. An international S&L consultant will be recruited to review the project strategy and support the PM team with the strategic review of the project. 4. The revised project strategy and a clear set of goals, outcomes, outputs and activities to be presented to and discussed with the project stakeholders.

Key Actions:

Key Action Responsible DueDate Status Comments Documents
1. The Project Management Team based on the consultations with the project Working Groups reviews the project document with the view to reflect recent changes in the national legal and regulatory framework, update the project strategy (outputs, activities, work plans) and present it in a strategic, clear and logical form. 2. A national advisor (CTA) for partnerships and stakeholder outreach is recruited to support the project team with above all the review of the project strategy. 3. An international S&L consultant will be recruited to review the project strategy and support the PM team with the strategic review of the project. 4. The revised project strategy and a clear set of goals, outcomes, outputs and activities to be presented to and discussed with the project stakeholders.
[Added: 2014/05/12] [Last Updated: 2018/11/08]
Project team, NIM partner, UNDP 2015/12 Completed History
19. Recommendation: Extension of project duration Project implementation seems to be delayed and the project is likely to require an extension beyond its planned lifetime of five years from May 2010 till May 2015 to for example December 2016. It is unlikely that within the remaining project duration the remaining budget of USD 5,448,741.49 will be spent and all the outcomes will be fulfilled in a satisfactory manner. Reasons of delay are the slow project start, slow project procurement and moderately unsatisfactory project management for the entire duration of the project and small size of the project implementation team and poor working conditions. Extension should be granted under the condition that all the recommendations mentioned in this report are implemented. An extension granted under conditions of an enforced project team and project management and of a substantive revision of the approach to the project implementation strategy in close consultation with project stakeholders should contribute to a successful completion of the project. (Section 1.03 Main conclusions, recommendations and lessons learned, (d) , (d) Main interventions: Extension of project duration, Substantive revision of project strategy, Enforcement of project team and project management p. 36-37)
Management Response: [Added: 2014/05/12]

UNDP and the project team agree with the recommendation. The project will need extension until May 2017 to accommodate above all late approval and signature of the project. The initial request for project extension has been prepared and reviewed by UNDP and UNDP/GEF. As per the MTE recommendation #1 and upon consultations with UNDP HQs it was agreed to endorse the project extension formally upon the following action by the project: (a) outputs of the project strategy/logframe/planning review, (b) substantive strengthening of the project team in particular in the areas of strategic management, planning, communication and stakeholder engagement, (c) a justified/strengthened extension request accommodating the above.

Key Actions:

Key Action Responsible DueDate Status Comments Documents
Prepare a justified extension request based on the strategic project review, implementation of the MTE recommendations and PMU strengthening. Initiate the procedure for Project extension up to May 2017.
[Added: 2014/05/12] [Last Updated: 2018/11/08]
Project team, UNDP, UNDP/GEF 2015/04 Completed History
20. Recommendation: Substantive revision It is not guaranteed that the Activities lead to the desired Outputs and the Outputs lead to the desired Outcomes. As such, one may question to what extent the overall project Objective will be reached when maintaining the current project strategy. The changes suggested during the inception phase were mainly of a textual nature and almost exclusively related to the Final Value Targets. The current logframe may not provide the best project strategy for achieving the project objectives. For an EE S&L scheme to be developed logically the following steps need to be taken. These steps can be divided into a PLANNING PHASE and an IMPLEMENTING PHASE. The PLANNING PHASE should take about half a year. The IMPLEMENTING PHASE should take two to three years. An extension of the project with at least 18 months would be recommendable. My recommendation would be to: 1. Do a substantive revision based upon a specialist inventory of the actual results achieved and an inventory of the work still needed to be done. This work should be led by an international CTA with a standards and labelling background. 2. Extend the duration of the project with at least one and a half years 3. Make a new three and-a-half-year work-plan (from now till end of 2016) in the case extension is granted or a two year workplan in case extension isn?t granted. 4. Change the logframe by adjusting the activities and outputs while leaving the outcomes and objective unchanged. (Section 1.03 Main conclusions, recommendations and lessons learned, (d) , (d) Main interventions: Extension of project duration, Substantive revision of project strategy, Enforcement of project team and project management p. 37)
Management Response: [Added: 2014/05/12]

Recommendation is accepted. The need for an ?inventory of the actual results achieved and of an inventory of the work still needed to be done? had been discussed and agreed upon between the Project team, UNDP country Office, UNDP Regional Advisor and the National Project Director. National CTA has been recruited and an international advisor is being recruited to carry out such a strategic review/planning. See response to recommendation 1 and key actions. Recommendation on the extending the project duration is accepted. See response to recommendation # 14 and 19 and key actions. Recommendation for the preparation of the 3,5 or 2 years working plan is accepted and will be completed as part of the strategic review in 2014. Changings of the logframe will be a part of the strategic review (see response to recommendation #1) and key actions. Key actions: 1. The Project Management Team based on the consultations with the project Working Groups reviews the project document with the view to reflect recent changes in the national legal and regulatory framework, update the project strategy (outputs, activities, work plans) and present it in a strategic, clear and logical form. 2. A national advisor (CTA) for partnerships and stakeholder outreach is recruited to support the project team with above all the review of the project strategy. 3. An international S&L consultant will be recruited to review the project strategy and support the PM team with the strategic review of the project. 4. The revised project strategy and a clear set of goals, outcomes, outputs and activities to be presented to and discussed with the project stakeholders. 5. Prepare a justified extension request based on the strategic project review, implementation of the MTE recommendations and PMU strengthening. Initiate the procedure for Project extension up to May 2017.

Key Actions:

Key Action Responsible DueDate Status Comments Documents
1. The Project Management Team based on the consultations with the project Working Groups reviews the project document with the view to reflect recent changes in the national legal and regulatory framework, update the project strategy (outputs, activities, work plans) and present it in a strategic, clear and logical form. 2. A national advisor (CTA) for partnerships and stakeholder outreach is recruited to support the project team with above all the review of the project strategy. 3. An international S&L consultant will be recruited to review the project strategy and support the PM team with the strategic review of the project. 4. The revised project strategy and a clear set of goals, outcomes, outputs and activities to be presented to and discussed with the project stakeholders. 5. Prepare a justified extension request based on the strategic project review, implementation of the MTE recommendations and PMU strengthening. Initiate the procedure for Project extension up to May 2017.
[Added: 2014/05/12] [Last Updated: 2018/11/08]
Project team, NIM partner, UNDP 2015/04 Completed History
21. Recommendation: Enforcement of project team and project management Ideally the project team and project management would be enforced with additional project team members. A part time international CTA with an S & L background, a senior advisor to the project manager, a PR specialist and internal project analyst/consultant should be considered. Project management needs to be improved. Monthly M&E moments should be created and clear goals for the project team and project management should be set. The project team should also be steered towards focusing at clear deliverables and verification of whether actual results achieved are supporting achievement of the outcomes and the final Objective. The project team should be stimulated to manage by and focus on priorities. Considering the current state of the project, reviews should be carried out at least yearly and preferably twice a year either internally by UNDP or by an international CTA. The results achieved should be evaluated more regularly and more critically. Clear quarterly targets and deliverables must be set for the project team and project manager. It is essential to start with a good strategic overview and implementation plan that describes in detail all the steps needed for establishing an EE S&L scheme; a roadmap for an EE S&L scheme. The formulation of the outputs of the project design / logframe should be made formulated in a clear and unequivocal manner. Capacity building inside the project team should be increased and closer attention must be paid to sustainability of capacity built outside the team. (Section 1.03 Main conclusions, recommendations and lessons learned, (d) , (d) Main interventions: Extension of project duration, Substantive revision of project strategy, Enforcement of project team and project management p. 37-38)
Management Response: [Added: 2014/05/12]

Recommendation is accepted and will be implemented. See responses to recommendations #1, 9,10,11,12,13 and related key actions.

Key Actions:

Key Action Responsible DueDate Status Comments Documents
See key actions planned under recommendations #1, 9,10,11,12,13
[Added: 2014/05/12] [Last Updated: 2018/11/08]
Project team, NIM partner, UNDP 2015/12 Completed A number of specific strategic and management actions taken, including strengthening the project team with new staff, engaging international consultants for the strategic project review and replacement of the project manager History

Latest Evaluations

Contact us

1 UN Plaza
DC1-20th Floor
New York, NY 10017
Tel. +1 646 781 4200
Fax. +1 646 781 4213
erc.support@undp.org