Evaluation Detail

Small Decentralised Renewable Energy Project (DREG) Project, Lebanon

Evaluation Plan:: 2017-2022, Lebanon
Evaluation Type:: Final Project
Planned End Date:: 11/2018
Completion Date:: 11/2018
Status:: Completed
Management Response:: No
Evaluation Budget(US $):: 20,000

Document	Type	Language	Size	Status	Downloads
4695_UNDP-GEF-Terminal-Evaluation-Terms-of-Reference-DREG 09.05.2018.docx	tor	English	67.33 KB	Posted	1241
TE_Report_DREG_14_Nov_2018.pdf	report	English	2716.75 KB	Posted	1417

Title

Small Decentralised Renewable Energy Project (DREG) Project, Lebanon

Atlas Project Number:

00073116

Evaluation Plan:

2017-2022, Lebanon

Evaluation Type:

Final Project

Status:

Completed

Completion Date:

11/2018

Planned End Date:

11/2018

Management Response:

Yes

Focus Area:

1. Environment & Sustainable Development
2. Others

Corporate Outcome and Output (UNDP Strategic Plan 2018-2021)

1. Output 1.5. Inclusive and sustainable solutions adopted to achieve increased energy efficiency and universal modern energy access (especially off-grid sources of renewable energy)

SDG Goal

Goal 13. Take urgent action to combat climate change and its impacts
Goal 7. Ensure access to affordable, reliable, sustainable and modern energy for all
Goal 9. Build resilient infrastructure, promote inclusive and sustainable industrialization and foster innovation

SDG Target

13.3 Improve education, awareness-raising and human and institutional capacity on climate change mitigation, adaptation, impact reduction and early warning
7.a By 2030, enhance international cooperation to facilitate access to clean energy research and technology, including renewable energy, energy efficiency and advanced and cleaner fossil-fuel technology, and promote investment in energy infrastructure and clean energy technology
9.4 By 2030, upgrade infrastructure and retrofit industries to make them sustainable, with increased resource-use efficiency and greater adoption of clean and environmentally sound technologies and industrial processes, with all countries taking action in accordance with their respective capabilities

Evaluation Budget(US $):

20,000

Source of Funding:

Project

Evaluation Expenditure(US $):

11,570

Joint Programme:

Joint Evaluation:

Evaluation Team members:

Name	Title	Email	Nationality
Dinesh Aggarwal	International Consultant	dinesh.aggarwal7@gmail.com	INDIA

GEF Evaluation:

Yes

GEF Project Title:	Decentralised Renewable Energy Project (DREG)
Evaluation Type:	Terminal Evaluation
Focal Area:	Climate Change
Project Type:	FSP
GEF Phase:	GEF-5
GEF Project ID:	4749
PIMS Number:	4695

Key Stakeholders:

Ministry of Energy and Water

Countries:

LEBANON

Lessons
Findings
1.	Adaptive management and Feedback from M&E used for adaptive management The main questions for terminal evaluation are; (please see B) Did the project undergo significant changes as a result of recommendations from the mid-term review? Or as a result of other review procedures? Explain the process and implications. If the changes were extensive, did they materially change the expected project outcomes? Were the project changes articulated in writing and then considered and approved by the project steering committee? Whether feedback from M&E activities was used for adaptive management? Whether changes were made to project implementation as a result of the MTR recommendations? Tag: Effectiveness Knowledge management Monitoring and Evaluation Ownership Project and Programme management
2.	5.1.1 Attainment of results– Outcome 1: Investments in decentralized renewable energy (RE) power generation increased The Outcome 1 of the project, was focused on supporting actual investments in the pilots for DREG and sustaining market growth. The pilot projects were aimed to demonstrate the application of different RE technologies (PV, wind, small hydro and/or biogas and a combination) for decentralized power generation. The first targeted investors included private businesses and industrial complexes, universities, tourist resorts, bakeries, etc., all of which depend on continuous and reliable electricity supply. Tag: Renewable energy Effectiveness Private Sector
3.	5.2 Relevance The main questions for terminal evaluation are; (please see Annex B) To what extent is the activity suited to local and national development priorities and organizational policies, including changes over time? To what extent is the project in line with UNDP Operational Programs or the strategic priorities under which the project has been funded? The DREG project and the activities planned within the project are highly relevant to the development needs of Lebanon. This is considering that the project addresses the issue of availability of sustainable energy to all at one end, while on the other hand, it addresses the issue of pressure on the economy due to the subsidies provided to the energy sector. The project is in line with the UNDP operational programs for Lebanon. This is explained further in the following paragraphs. Tag: Renewable energy Relevance Country Government UN Agencies Agenda 2030
4.	5.3 Effectiveness & Efficiency The main questions for terminal evaluation are; (please see Annex B) To what extent the objectives have been achieved? To what extent the results have been delivered with the least costly resources possible? What are the positive and negative, foreseen and unforeseen changes to and effects produced by a development intervention? Tag: Emission Reduction Renewable energy Effectiveness Efficiency
5.	5.4 Country ownership The main questions for terminal evaluation are; (please see Annex B) Was the project concept in line with development priorities and plans of Lebanon? Were the relevant country representatives from government and civil society involved in project implementation, including as part of the project steering committee? Was an inter-governmental committee given responsibility to liaise with the project team, recognizing that more than one ministry should be involved? Have the government(s), enacted legislation, and/or developed policies and regulations in line with the project’s objectives? Tag: Relevance Ownership Country Government
6.	5.5 Mainstreaming The main questions for terminal evaluation are; (please see Annex B) How is the project successfully mainstreaming other UNDP priorities, including poverty alleviation, improved governance, the prevention and recovery from natural disasters, and women's empowerment? Whether it is possible to identify and define positive or negative effects of the project on local populations (e.g. income generation/job creation, improved natural resource management arrangements with local groups, improvement in policy frameworks for resource allocation and distribution, regeneration of natural resources for long term sustainability). If the project objectives conform to agreed priorities in the UNDP country programme document (CPD) and country programme action plan (CPAP). Whether there is evidence that the project outcomes have contributed to better preparations to cope with disasters. Whether gender issues have been taken into account in project design and implementation and in what way has the project contributed to greater consideration of gender aspects, (i.e. project team composition, gender-related aspects of pollution impacts, stakeholder outreach to women’s groups, etc.) Tag: Renewable energy Sustainability Gender Mainstreaming Country Government
7.	5.6 Sustainability The main questions for terminal evaluation are; (please see Annex B) Are there financial risks that may jeopardize the sustainability of project outcomes? What is the likelihood of financial and economic resources not being available once GEF grant assistance ends? Are there social or political risks that may threaten the sustainability of project outcomes? What is the risk for instance that the level of stakeholder ownership (including ownership by governments and other key stakeholders) will be insufficient to allow for the project outcomes/benefits to be sustained? Do the various key stakeholders see that it is in their interest that project benefits continue to flow? Is there sufficient public/stakeholder awareness in support of the project’s long-term objectives? Do the legal frameworks, policies, and governance structures and processes within which the project operates pose risks that may jeopardize sustainability of project benefits Are requisite systems for accountability and transparency, and required technical knowhow, in place? • Are there ongoing activities that may pose an environmental threat to the sustainability of project outcomes? Tag: Renewable energy Sustainability Global Environment Facility fund Policies & Procedures
8.	5.7 Impact The main questions for terminal evaluation are; (please see Annex B) • Whether, the project has demonstrated verifiable improvements in ecological status? • Whether, the project has demonstrated verifiable reductions in stress on ecological systems through specified process indicators, that progress is being made towards achievement of stress reduction and/or ecological improvement? The most direct impact of the project, in terms of GEF objectives, is the reduction in GHG emissions. The outcomes of the DREG project would lead to GHG emission reductions from the power sector in Lebanon on a long term basis. This will have the environmental and ecological co-benefits in terms of reduction in the emissions of particulate matter; lead, mercury and other heavy metals; acid gases like NOx and SOx. Tag: Environmental impact assessment Impact
9.	4.2 Partnership arrangements The main questions for terminal evaluation are; (please see Annex B) Were there adequate provisions in the project design for consultation with stakeholder? Whether effective partnerships arrangements were established for implementation of the project with relevant stakeholders involved in the country/region, including the formation of a Project Board? The Ministry of Energy and Water/LCEC was one of the ‘implementing partners’ for the project. Tag: Partnership Country Government Private Sector
10.	4.4 Monitoring and evaluation: design at entry The main questions for terminal evaluation are; (please see Annex B) Is the M&E plan well conceived at the design stage? Is M&E plan articulated sufficient to monitor results and track progress toward achieving objectives? Was the M&E plan sufficiently budgeted and funded during project preparation and implementation? How effective are the monitoring indicators from the project document for measuring progress and performance? Tag: Effectiveness Monitoring and Evaluation Programme/Project Design Project and Programme management
11.	4.6 UNDP and Implementing Partner implementation / execution coordination, and operational issues The main questions for terminal evaluation are; (please see Annex B) Whether there was an appropriate focus on results? Was there adequate UNDP support to the Implementing Partner and project team? Quality and timeliness of technical support to the Executing Agency and project team Were the management inputs and processes, including budgeting and procurement adequate? Tag: Implementation Modality Knowledge management Project and Programme management Country Government
12.	Output 1.2 The project team has carried out the viability analysis of the DREG projects and considering the drop in the capital cost of RE technologies over a period of time, is of the view that the ongoing scheme of soft loans through NEEREA would be sufficient to attract private sector investment for RE in future. The current thinking within the national counterparts (government agencies) within Lebanon also favours private sector investment for RE. For example, MEW/LCEC has released Lebanon’s National Renewable Energy Action Plan (NREAP) which outlines a vision for a tangible RE target by the year 2030. The NREAP envisages utility-scale RE projects to be financed exclusively through private investments. Lebanon’s Intended Nationally Determined Contribution (INDC), submitted by the Government as part of its commitment under the Paris Agreement, stipulates a 15% RE target (power and heat demand) by 2030, which can reach 20% with proper support. The policy and financial instruments proposed to be used to facilitate private sector financing for RE are; Fostering financial sector reform towards green infrastructure investment Strengthening financial sector’s familiarity with renewable energy and project finance Concessional public loans to IPPs Tag: Renewable energy Country Government Private Sector
13.	5.1.2 Attainment of results – Outcome 2: An enforced supportive policy and regulatory environment for attracting investments for privately owned, grid-connected power generation by RE sources Outcome 2 of the project was to support the establishment of an enabling legal and regulatory framework to attract investment for privately-owned, grid-connected power generation by renewable energy sources. The project was to support the required background analysis, consultations, awareness-raising and capacity building of the key stakeholders to allow the drafting of the new regulations and facilitate their effective adoption and implementation. The DREG project was to build on and improve upon the analysis available due to the baseline projects for promotion of RE in Lebanon. Tag: Renewable energy Country Government Capacity Building Institutional Strengthening Private Sector
14.	In terms of the performance of the first indicator for Outcome 3, the project regularly produced the reports on an annual basis. Although, the contents were mainly for solar PV. This is largely due to the fact that the level of market development of solar PV in Lebanon is far ahead than the other RE technologies. The project introduced the required quality control measures for all the pilot projects. The beneficiaries showed satisfaction with the PV systems’ operation and performance. Tag: Emission Reduction Renewable energy Effectiveness
15.	5.1.5 Global environmental benefits The global environmental benefits of the project are the reduction in the emission of greenhouse gases (GHG) to help the global community address climate change. The project document stipulates the project objective as ‘Reducing greenhouse gas emissions by the removal of barriers to widespread application of decentralized renewable energy based powerThe target values for direct GHG emission reduction was 35,000 tons of CO2 equivalent over the lifetime of 20 years of the demonstration projects. Details of the projected GHG emissions and the corresponding set of assumptions were provided in Table 20. The projected GHG emission reductions as given in Table 20 are on the conservative side due to two reasons. Firstly, due to the fact that the emission factor considered is that for the grid, whereas in actual practise the pilot projects would be replacing the small diesel generator-based power generated by the beneficiaries and secondly due to use of a dynamic baseline emission factor, which reduces over a period of time. Indirect GHG emission reductions due to the DREG project will happen over a period of time as result of sustained market growth due to: enabling conditions created by the project for continued investments in small decentralized RE generation capacity. There will also be indirect GHG reduction impact due to utility-scale RE development. generation’. Tag: Emission Reduction Renewable energy Effectiveness
16.	4.5 Monitoring and evaluation: implementation The main questions for terminal evaluation are; (please see Annex B) Whether the logical framework was used during implementation as a management and M&E tool? What has been the level of compliance with the progress and financial reporting requirements/ schedule, including quality and timeliness of reports? What has been the effectiveness of the monitoring reports and evidence that these were discussed with stakeholders and project staff? What is the extent to which follow-up actions, and/ or adaptive management, were taken in response to monitoring reports (APR/PIRs)? Whether APR/PIR self-evaluation ratings were consistent with the MTR. If not, were these discrepancies identified by the project steering committee and addressed? Tag: Effectiveness Monitoring and Evaluation Project and Programme management

Recommendations
1	Recommendation 1: The project design as presented in the ‘Project Document’ did specify the expected set of Outputs for each of the Outcome of the project. However, the expected outputs did not find their required place in the log-frame of the project. Indicators were provided at the outcome level, whereas the work planning of the project was done at the Output level. The monitoring (PIRs) of the progress of the project was done as per the results framework of the project. As all the activities / Outputs did not got covered in the results framework, some of the important activities (as provided in the Outputs) gets missed out in the monitoring / PIRs. It is recommended that for the future project design, the Indicators in the results frame-work be fixed at both the Outcome level and the Output level
2	Recommendation 3: The project has not been able to support other DREG technologies (other than solar PV). As a result, it is a missed opportunity to showcase/promote different RE technologies. It is recommended that in case of involvement of multiple technologies/sectors, the project design should specify different technologies/sectors to be demonstrated (by pilots), and should have provisions for a different set of efforts which would be required to promote/demonstrate such technologies. Different types and levels of technical support are required for promotions/demonstration of different type of RE technologies. Any future project design for the promotion/demonstration of DREG should either be technology specific or should clearly state the technologies to be used for different pilot projects. Recommendation 4: The project design had provisions like technical support, grants and soft loans for supporting the implementation of the DREG pilot projects. The kind and extent of support was uniform all across the RE technologies. It is recommended that the project design should also have technology-specific provisions for supporting the kind of RE technology to be demonstrated by way of pilots. For example, for the technologies which are not presently demonstrated in the country, there can be a provision to have a study tour of the prospective beneficiaries to the countries where such technologies are already in use. In addition, for the pilot projects based on RE technologies where sufficient technical expertise may not be available within the country, it would help to take on board ‘International Technical Experts’. Recommendation 7: Whenever, an opportunity for a new RE project in Lebanon arises, the project design may support formulation of regulations and establishment of the electricity regulatory authority. This will not only help decentralised renewable energy generation, but will also help the establishment of Independent Power Producers (including those for RE).
3	Recommendation 5: There are some very good case studies from the DREG project to demonstrate financial feasibility of solar PV technology (particularly considering the reduction in the capital cost of solar PV). This may be used to achieve replication of the solar PV on a larger scale.
4	Recommendation 6: The project has prepared quality standards for a number of solar PV equipment which are already with the government for approval by way of a decree. Efforts may be continued to achieve this.
5	Evaluation Recommendation 8: Soft loans from the central bank are a very effective fiscal instrument for the promotion of RE technologies. However, it takes a considerable amount of time for approval of the soft loans, thereby delaying the projects. There is a need to optimise the process at the level of the central bank so that the overall time taken is reduced. The government counterpart may explore the possibilities to optimise the process at the level of the central bank
6	Recommendation 2: The three indicators for the project objective (GHG emission reduction, Capacity of RE, and RE generation) were very closely interrelated. Thus, the additional indicators did not serve any purpose. Considering that the objective of the project on the one hand was “Reducing greenhouse gas emissions” while on the other it was, “removal of barriers to widespread application of decentralized renewable energy based power generation”, an indicator which indicates the removal of barriers or widespread application of DREG would have been more appropriate (instead of capacity of RE). Having said that, it is appreciated that having an appropriate indicator to indicate removal of barrier or wide spreading of DREG in itself is a big challenge. It is recommended that to the extent possible, the indicators of the ‘Project Objective’ should be independent of each other.

Small Decentralised Renewable Energy Project (DREG) Project, Lebanon

Share

Evaluations by Unit

Latest Evaluations

Evaluation Snap Shot

Contact us