PIMS 1273 CC FP: Improving the Energy Efficiency of Municipal Heating and Hot Water Supply

Report Cover Image
Evaluation Plan:
2005-2009, Armenia
Evaluation Type:
Project
Planned End Date:
06/2008
Completion Date:
09/2012
Status:
Completed
Management Response:
Yes
Evaluation Budget(US $):
2,950,000

Share

Document Type Language Size Status Downloads
Download document Mid-term_evaluation-Armenia-EE.pdf report Posted 615
Download document Final-Evaluation_Armenia_Heating-Project.pdf report English 1442.84 KB Posted 1028
Download document TE-Management-Response_Heating-Project.pdf report English 90.54 KB Posted 463
Download document Lessons-Learned-Report_Heating-Project-1273.pdf summary English 1392.60 KB Posted 2379
Title PIMS 1273 CC FP: Improving the Energy Efficiency of Municipal Heating and Hot Water Supply
Atlas Project Number: 00035799
Evaluation Plan: 2005-2009, Armenia
Evaluation Type: Project
Status: Completed
Completion Date: 09/2012
Planned End Date: 06/2008
Management Response: Yes
Focus Area:
  • 1. Crisis Prevention & Recovery
  • 2. Environment & Sustainable Development
Corporate Outcome and Output (UNDP Strategic Plan 2014-2017)
  • 1. Development plans and programmes integrate environmentally sustainable solutions in a manner that promotes poverty reduction, MDG achievement and low-emission climate-resilient development
  • 2. Local and national authorities have the capacities to access and integrate multiple sources of public and private environmental financing in support of sustainable human development, including gender equality and poverty reduction
  • 3. National and local governments and communities have the capacities to adapt to climate change and make inclusive and sustainable environment & energy decisions benefitting in particular under-served populations
Evaluation Budget(US $): 2,950,000
Source of Funding: GEF
Joint Programme: No
Joint Evaluation: No
Evaluation Team members:
Name Title Email Nationality
Jiri Zeman Team Leader jirkazeman@seznam.cz CZECH REPUBLIC
Jiri Zeman
Jiri Zeman
GEF Evaluation: Yes
Evaluation Type:
Focal Area: Climate Change
Project Type: FSP
GEF Phase: GEF-4
PIMS Number: 1273
Key Stakeholders:
Countries: ARMENIA
Lessons
Findings
Recommendations
1 It is recommended that the main directions of the project should be maintained with a number of minor reorientations on activities and approaches.
2 Since the policy and legislative frameworks are so crucial to overall project success, and still, as at the time of project design, represent the main barriers to be addressed, ongoing efforts should be made to move the policy agenda forward. The 'heat law' is of primary importance in this regard, and should receive the majority of effort, with the activities on other legislative issues being addressed as and when opportunities arise. Thus, it is recommended that policy work around condominium associations be given a low priority at the present time (with the project team perhaps merely monitoring conditions) unless circumstances change and opportunities arise. The work on forest management standards should be entirely dropped.
3 The advisory centre has not worked (partly due to UNDP bureaucracy which destroyed necessary continuity and stability, as well a lack of policies which could incentives stakeholders to make use of these services). It is recommended that, in the absence of legislative development, the contractor be used as needed as a project expert on retainer (for arranging and carrying out surveys for example), and that the advisory centre idea be revived once the legal frameworks develop sufficiently.
4 Support to development of heating strategies should continue to attempt to engage with municipalities when opportunities arise. With successful development of a heat law including municipal obligations the project can then reengage rapidly with municipalities and give a higher priority to capacity building.
5 Delivery of successful pilot projects (in particularly opportunities like Avan and Davidashen) should be given a high priority. This is likely to include supporting the Yerevan municipality to reinvigorate or tender the Davidashen concession. Concrete success in on the ground projects will enhance the policy work and give a stimulus to both municipalities and the population.
6 Given the importance of the Ministries of Energy and Finance & Economy, stronger engagement with these stakeholders could be valuable. In any case the project team should try to forge alliances with all government stakeholders to explain the arguments for a concerted effort to address heating and hot water supply - from fuel security, economic, health, safely, and environmental points.
7 The sustainability strategy needs urgent attention: how will the project ensure that 1) there is a government champion to take forward the issues when the project finishes, 2) the project benefits continue to be realized and grow after the end of the project. While appropriate legislation and institutional structures will largely address this, attention should be given to ensuring that services such as 'advisory support' and technical support to municipalities and private investors (and potentially to condominium associations) have a suitable 'home'.
8 Efforts should be made to reduce the bureaucratic burdens imposed by UNDP, and to develop procurement processes which can allow for long-term hiring of project staff, international experts and local experts / organizations where necessary since current processes are de-motivating staff, making difficulties in retaining qualified personnel, wasting substantial time, and disrupting the project strategy/vision. Potentially some high level discussion is needed, since most UNDP country offices manage to reduce these barriers. Tendering and re-tendering services such as the Advisory Centre is simply unacceptable and is stifling the project and wasting huge amounts of money and human resources.
9 The project is at a critical stage where if policy barriers are removed through the project efforts the market could take off in a substantial way. It would be a great pity if the project came to an end in the middle of these developments and the opportunity was lost. Since sufficient budget clearly exists, the project duration should thus be extended for at least an additional two years, and budgets planned accordingly.
10 Projects need to be able to adapt to changing market conditions rapidly in new markets such as in Armenia. With district heating collapsing during project preparation and a huge shift in heating practices (from approximately 56% wood and 7% gas in 2001 to 10% wood and 53% natural gas in 2006-7) during the project, a high degree of flexibility and agility is necessary. The value of revisiting the project logic and questioning assumptions and approaches is of great benefit.
11 Monitoring frameworks should be kept simple, with a minimum number of good quality indicators (specifying Quantity, Quality and Timeframe) and targeting desired impacts. Overcomplicated monitoring frameworks will not be used. Similarly changing indicators annually confuses evaluation processes, and does not support the logical integrity of the project.
1. Recommendation: It is recommended that the main directions of the project should be maintained with a number of minor reorientations on activities and approaches.
Management Response: [Added: 2009/10/07]

The recommendations will be taken into consideration.

Key Actions:

Key Action Responsible DueDate Status Comments Documents
Project Logical Framework (LogFrame) will be revised to incorporate suggested changes in the project strategy at the outputs/activities level.
[Added: 2009/10/07] [Last Updated: 2013/06/25]
UNDP Environmental Governance Portfolio, Project 2012/12 Completed Project was extended till December 2012.
2. Recommendation: Since the policy and legislative frameworks are so crucial to overall project success, and still, as at the time of project design, represent the main barriers to be addressed, ongoing efforts should be made to move the policy agenda forward. The 'heat law' is of primary importance in this regard, and should receive the majority of effort, with the activities on other legislative issues being addressed as and when opportunities arise. Thus, it is recommended that policy work around condominium associations be given a low priority at the present time (with the project team perhaps merely monitoring conditions) unless circumstances change and opportunities arise. The work on forest management standards should be entirely dropped.
Management Response: [Added: 2009/10/07]

The recommendations will be taken into consideration.

Key Actions:

Key Action Responsible DueDate Status Comments Documents
2.1. Ability of the project to have new Heat Law developed and adopted by the end of the project must be realistically assessed and consequently reflected in the revised LogFrame.
[Added: 2009/10/07] [Last Updated: 2013/06/25]
Ministry of Energy, UNDP Environmental Governance Portfolio, Project, GEF consultant 2009/06 Completed
2.2. Further actions of the project in regard to developed regulatory package for introduction and enforcement of energy passports must be redefined and accordingly reflected in the logframe considering the identified barriers and benefits.
[Added: 2009/10/07] [Last Updated: 2013/06/25]
Ministry of Energy, UNDP Environmental Governance Portfolio, Project, GEF consultant 2012/12 Completed Project was extended till December 2012.
2.3 Project efforts for the legal empowerment of condominiums also must be discussed further considering the ability of the project to address existing "gap" in legal status and ownership rights of condominiums. It is worth checking and maybe building on UNDP's work with condominiums undertaken by Governance Practice. If the challenge is too big for project to address, it is advised to drop "policy" component and reduce budget accordingly.
[Added: 2009/10/07] [Last Updated: 2013/06/25]
Ministry of Energy, UNDP Environmental Governance Portfolio, Project, GEF consultant 2009/06 Completed
2.4 The forest management standards issues exclusion will be discussed with Executing Agency and will be considered correspondingly. In case if the clear linkage with the objectives of the project in its current development stage will not be obvious the task may be dropped. The decision will be endorsed by the Outcome board.
[Added: 2009/10/07] [Last Updated: 2013/06/25]
Ministry of Energy, UNDP Environmental Governance Portfolio, Project, GEF consultant 2009/06 Completed
3. Recommendation: The advisory centre has not worked (partly due to UNDP bureaucracy which destroyed necessary continuity and stability, as well a lack of policies which could incentives stakeholders to make use of these services). It is recommended that, in the absence of legislative development, the contractor be used as needed as a project expert on retainer (for arranging and carrying out surveys for example), and that the advisory centre idea be revived once the legal frameworks develop sufficiently.
Management Response: [Added: 2009/10/07]

The selected modality for contracting NGO for provision of Advisory Services has caused problems for long term contracting. In that context the UNDP management has conducted several rounds of discussions with project manager and concluded as most rational and advisable the modality of provision of Micro-Capital Grant to selected specialized non-governmental organization.

Key Actions:

Key Action Responsible DueDate Status Comments Documents
There is a need to specify and justify the focus of Advisory Center's work in the absence of Heat Law. This should include: target groups and incentive.
[Added: 2009/10/07] [Last Updated: 2013/06/25]
Ministry of Energy, UNDP Environmental Governance Portfolio, Project, GEF consultant 2009/06 Completed
4. Recommendation: Support to development of heating strategies should continue to attempt to engage with municipalities when opportunities arise. With successful development of a heat law including municipal obligations the project can then reengage rapidly with municipalities and give a higher priority to capacity building.
Management Response: [Added: 2009/10/07]

The recommendations will be taken into consideration in project work-plan.

Key Actions:

Key Action Responsible DueDate Status Comments Documents
The recommendation will be discussed and included in the AWp during Annual Review
[Added: 2009/10/07] [Last Updated: 2013/06/25]
Ministry of Energy, UNDP Environmental Governance Portfolio, Project, GEF consultant 2009/12 Completed
5. Recommendation: Delivery of successful pilot projects (in particularly opportunities like Avan and Davidashen) should be given a high priority. This is likely to include supporting the Yerevan municipality to reinvigorate or tender the Davidashen concession. Concrete success in on the ground projects will enhance the policy work and give a stimulus to both municipalities and the population.
Management Response: [Added: 2009/10/07]

The results will be communicated to wide audience.

Key Actions:

Key Action Responsible DueDate Status Comments Documents
The results of the2008-2009 heating season will be evaluated by the social survey (satisfaction level and good practice) and the results will be broadly communicated: policy makers (SC meeting), general public (TV programme, people surveyed). Project must define clear mechanisms to present concrete successes in pilot projects and use them to enhance the effectiveness of policy work at national level and help in lobbying for new heat law. First of all, this include better presentation of the results of pilot projects in terms of clients satisfaction (improved comfort, reduced energy bills), environmental impacts (less GHG emission and local pollution), in a simple form that is understood by non-specialist. This should be complemented by communication plan: how to make decision-makers and general audience aware of the benefits of DH and get their buy-in for national-level reform.
[Added: 2009/10/07] [Last Updated: 2013/06/25]
Ministry of Energy, UNDP Environmental Governance Portfolio, Project, GEF consultant 2009/06 Completed
6. Recommendation: Given the importance of the Ministries of Energy and Finance & Economy, stronger engagement with these stakeholders could be valuable. In any case the project team should try to forge alliances with all government stakeholders to explain the arguments for a concerted effort to address heating and hot water supply - from fuel security, economic, health, safely, and environmental points.
Management Response: [Added: 2009/10/07]

The recommendations will be taken in consideration. Close cooperation with Ministry of Energy is already undergoing and as the Deputy Ministers of Energy and Finance are members of the SC.

Key Actions:

7. Recommendation: The sustainability strategy needs urgent attention: how will the project ensure that 1) there is a government champion to take forward the issues when the project finishes, 2) the project benefits continue to be realized and grow after the end of the project. While appropriate legislation and institutional structures will largely address this, attention should be given to ensuring that services such as 'advisory support' and technical support to municipalities and private investors (and potentially to condominium associations) have a suitable 'home'.
Management Response: [Added: 2009/10/07]

The sustainability issue is highly prioratized and project efforts are directed to reach it.

Key Actions:

Key Action Responsible DueDate Status Comments Documents
The most sustainable solution can be reached in case the Heat Law is adopted. However the current policy of the Government mostly relies on attracting the private investments in that sector. The current developments can be considered as positive, however the role of Municipalities must be enforced for ensuring sustainability of services and their quality. The project will elaborate the critical shortcomings of the current legislation to maximally assist the private and specialized companies to ensure their sustainable operations after project closure. The lobbing work will be undertaken with the Ministry of Urban Development in promotion of the state support to the multi-apartment building management bodies.
[Added: 2009/10/07] [Last Updated: 2013/06/25]
Ministry of Energy, UNDP Environmental Governance Portfolio, Project, GEF consultant 2012/12 Completed
8. Recommendation: Efforts should be made to reduce the bureaucratic burdens imposed by UNDP, and to develop procurement processes which can allow for long-term hiring of project staff, international experts and local experts / organizations where necessary since current processes are de-motivating staff, making difficulties in retaining qualified personnel, wasting substantial time, and disrupting the project strategy/vision. Potentially some high level discussion is needed, since most UNDP country offices manage to reduce these barriers. Tendering and re-tendering services such as the Advisory Centre is simply unacceptable and is stifling the project and wasting huge amounts of money and human resources.
Management Response: [Added: 2009/10/07]

Procurment COP is developed to make the process easier and more understandable.

Key Actions:

Key Action Responsible DueDate Status Comments Documents
The current suggested modality for contracting experts using LTA contracts which have to be engaged in provision of assistance to project partly solve the issue of keeping the expertise and institutional knowledge. The SSA contracts of 7 experts will be transferred to LTA till end of 2009.
[Added: 2009/10/07]
UNDP Operations Unit, Project, Environmental Governance Portfolio 2009/06 Completed
9. Recommendation: The project is at a critical stage where if policy barriers are removed through the project efforts the market could take off in a substantial way. It would be a great pity if the project came to an end in the middle of these developments and the opportunity was lost. Since sufficient budget clearly exists, the project duration should thus be extended for at least an additional two years, and budgets planned accordingly.
Management Response: [Added: 2009/10/07]

The Steering Committee and UNDP Outcome board have considered justifications provided by national coordinator, project manager, as well as MTE report and recommended to extend the project deadline for additional 2 years.

Key Actions:

10. Recommendation: Projects need to be able to adapt to changing market conditions rapidly in new markets such as in Armenia. With district heating collapsing during project preparation and a huge shift in heating practices (from approximately 56% wood and 7% gas in 2001 to 10% wood and 53% natural gas in 2006-7) during the project, a high degree of flexibility and agility is necessary. The value of revisiting the project logic and questioning assumptions and approaches is of great benefit.
Management Response: [Added: 2009/10/07]

The recommendation will be taken into consideration

Key Actions:

Key Action Responsible DueDate Status Comments Documents
Project LogFrame and budget will be revisited to account for changes in project environment and provide for adjustment in project outputs/ activities/indicators. Revised LogFrame should be presented and approved by Outcome board.
[Added: 2009/10/07] [Last Updated: 2013/06/25]
UNDP Environmental Gov. Portfolio, Outcome Board, Steering Committee 2012/12 Completed Project was extended till December 2012.
11. Recommendation: Monitoring frameworks should be kept simple, with a minimum number of good quality indicators (specifying Quantity, Quality and Timeframe) and targeting desired impacts. Overcomplicated monitoring frameworks will not be used. Similarly changing indicators annually confuses evaluation processes, and does not support the logical integrity of the project.
Management Response: [Added: 2009/10/07]

To the extent possible the performance indicators envisaged in CPAP and UNDAF (under preparation for 2010-2014) will be included in the revised LogFrame. The GHG emission reduction monitoring will be conducted according to the GEF Guidelines.

Key Actions:

Latest Evaluations

Contact us

1 UN Plaza
DC1-20th Floor
New York, NY 10017
Tel. +1 646 781 4200
Fax. +1 646 781 4213
erc.support@undp.org