- Evaluation Plan:
- 2017-2022, Lebanon
- Evaluation Type:
- Outcome
- Planned End Date:
- 07/2019
- Completion Date:
- 05/2019
- Status:
- Completed
- Management Response:
- No
- Evaluation Budget(US $):
- 20,000
Energy and Environment Outcome Evaluation
Share
Document | Type | Language | Size | Status | Downloads |
---|---|---|---|---|---|
![]() |
tor | English | 526.63 KB | Posted | 1113 |
![]() |
report | English | 1057.95 KB | Posted | 1400 |
Title | Energy and Environment Outcome Evaluation | ||||||||
---|---|---|---|---|---|---|---|---|---|
Atlas Project Number: | 00098956 | ||||||||
Evaluation Plan: | 2017-2022, Lebanon | ||||||||
Evaluation Type: | Outcome | ||||||||
Status: | Completed | ||||||||
Completion Date: | 05/2019 | ||||||||
Planned End Date: | 07/2019 | ||||||||
Management Response: | Yes | ||||||||
Focus Area: |
|
||||||||
Corporate Outcome and Output (UNDP Strategic Plan 2018-2021) |
|
||||||||
SDG Goal |
|
||||||||
SDG Target |
|
||||||||
Evaluation Budget(US $): | 20,000 | ||||||||
Source of Funding: | CO | ||||||||
Evaluation Expenditure(US $): | 21,000 | ||||||||
Joint Programme: | No | ||||||||
Joint Evaluation: | No | ||||||||
Evaluation Team members: |
|
||||||||
GEF Evaluation: | No | ||||||||
Key Stakeholders: | Ministry of Environment, Ministry of Energy, CDR | ||||||||
Countries: | LEBANON |
Lessons | |
---|---|
Findings | |
1. | 3.1 The Energy and Environment Programme Tag: Energy Environment Policy Integration Programme/Project Design Results-Based Management |
2. | 3.2 Programme relevance Tag: Relevance Project and Programme management |
3. | 3.3 Programme effectiveness Tag: Energy Effectiveness Project and Programme management Data and Statistics |
4. | 3.4 Implementation efficiency Efficiency is a measure of the extent and the competence through which resources have been converted into results. Tag: Efficiency Communication UNDP Regional Bureaux |
5. | 3.4.2 Personnel, expertise and knowledge sharing The most valuable and tangible resources of the Programme are its personnel, those in the central office and those leading and managing UNDP projects and other initiatives. Also in this category are the implementation partners through whom UNDP achieves its results. The central office serves as the hub for the Programme. This is where the Programme is designed, managed, coordinated and monitored. There is a staff of four – the Programme Manager, a Programme Associate, a Programme Assistant, and a Programme Support JPO. This immediate team is supported by the Country Office administration and support units such as Human Resources, Financial, Legal, etc. Tag: Energy Efficiency Oversight Project and Programme management Country Government UNDP management UNDP Regional Bureaux Operational Services |
6. | 3.4.3 Financial resources Financial resources are among the most tangible (and measurable) inputs for UNDP work and expenditure is often used as a measure of delivery, overshadowing the more important results achieved. There is only a slight tendency for such an approach with the E&E Programme. Tag: Efficiency Resource mobilization Human and Financial resources Country Government UN Agencies |
7. | 3.4.4 Partnerships UNDP is an implementing agency but most often it needs executing partners to realize any results; it also relies on donor partners for funding. As with the rest of the UNDP family, the E&E Programme depends almost entirely on executing partners and funding partners to achieve its goals. Worthy of special mention is the partnership that the Programme is developing with the private sector and this is in accordance with the adopted UNDP strategy . Tag: Environment Policy Partnership Country Government Donor UNDP Regional Bureaux Coordination |
8. | 3.4.5 Monitoring and Evaluation The CPD discussed monitoring and evaluation for the Country Programme and proposed a number of monitoring tools. The evaluator invited the E&E Programme Team to assess the extent to which these tools have been used and how efficient they had been. The results are summarized below. Tag: Monitoring and Evaluation |
9. | 3.4.6 Risk management The following risks were identified for the whole of the CPD together with possible mitigation measures. The Programme Team was invited to comment on the extent, if any, that the risks did eventuate for the Programme, and if so, whether the mitigation measures worked, and what the impacts on the Programme were. From all indications, it is evident that the Team is recognizing and managing risks adequately. However, it would be an improvement if there was a discussion of risks and mitigation measures focussed on the Programme in a dedicated Programme Document. Tag: Efficiency Risk Management |
10. | 3.4.7 The gender dimension Gender equality, women’s empowerment and human rights are among the key priorities of UNDP at the corporate level. However, they are best displayed at the projects level which is the delivery mechanism for the Programme. Less than 30% of Project Managers said that their project made a contribution towards gender equality and women’s empowerment. The rest said either that it did not do so or that the question did not apply to their project. This is a disappointing result but not entirely surprising since it is known that while MDG targets in Lebanon have been met in the health and primary education sectors, they were not in poverty, gender equality and environmental sustainability. UNDP has completed a Gender Strategy 37 which found that women’s participation in programme design and implementation contributed to reducing gender inequality; breaking traditional gender roles; increased lobbying for women’s representation and participation in national governance, and a greater number of leadership opportunities for young women. However, the Strategy also found that the Country Office “does not yet make the direct link between gender and environment. This can be explained by the fact that projects are designed to target all Lebanese citizens without distinction. The efforts for gender-mainstreaming at this level do not go further than maintaining a balance between male and female staff, ensuring a lack of discrimination against women, and, in some cases, including a theoretical section on gender”. This shortfall is recognized by the Programme and it is useful to use a very recent project currently awaiting signature as an example of how GEEW is being addressed. This project, which deals with land degradation and land use, has an impressive 10-page Gender Analysis and Mainstreaming Plan in an annex in addition to extensive gender provisions which are integrated in the project document. The Plan identifies gender concerns and opportunities to ensure that the project considers women’s and men’s different vulnerabilities and needs as well as capacities and skills and achieves an equitable distribution of its benefits, resources, status and rights. The Plan also outlines ways for the project to influence transformative changes in the norms, cultural values and the roots of gender inequalities and discriminations. Furthermore the Plan is aligned to other strategies and processes of the project to ensure feasibility and is expected to be mainstreamed as much as Tag: Efficiency Gender Equality Gender Mainstreaming Women's Empowerment MDGs |
11. | 4.5 Sustainability of results The likelihood of sustainability for the benefits and outcomes of the projects was seen as very high by Project Managers. In fact, 15 (88%) said there will be sustainability, one said maybe and one said it was not guaranteed. The questionnaire also asked Project Managers if their project had a “champion” who would inherit the project benefits and outcomes. Two did not understand the question, however, the most common champions identified by the rest (76%) were the project stakeholders while a small number (23%) listed the executing partner as the likely champion. Tag: Energy Waste management Water resources Sustainability Human and Financial resources Project and Programme management Civil Societies and NGOs UNDP management UNDP Regional Bureaux Capacity Building |
12. | 3.1.2 The Programme objectives and targets The CPD narrative refers to the analysis carried out and the conclusion reached that what is required is a combination of a “short-term crisis response with long-term initiatives that fall under theumbrella of the new sustainable development agenda”. More specifically, for the environment and energy thematic area, the objective is seen as “to halt the environmental degradation that has been exacerbated by the crisis and negatively influenced Lebanon’s chances of securing long-term sustainable development.” However, the Programme also takes its cue from the UNSF 2017-2020 Core Priority 3: Lebanon reduces poverty and promotes sustainable development while addressing immediate needs in a human rights/ gender-sensitive manner. Under this priority area, the Programme is meant to contribute to Outcome 3.3 which strives for the broad objective of – Lebanon has adopted measures to improve environmental governance. Tag: Environment Policy Partnership Policies & Procedures Policy Advisory SDG Integration Civil Societies and NGOs |
13. | 3.1.3 The projects portfolio The portfolio comprises 36 projects. Of these, 19 projects were carried over from the previous CPD period with ten having since been closed and nine still continuing. Some of the projects that have been carried over have been through a number of phases according to the PMs’ Questionnaire (see Six new projects have been initiated in this CPD period and all are still under implementation. In addition, two projects are pending signatures and are expected to start implementation soon. There are also another nine projects in a hard pipeline and some of these may be initiated during this CPD period.
Tag: Programme/Project Design Project and Programme management |
14. | 3.1.4 Programme resources Tag: Human and Financial resources |
15. | 3.2.1 Relevance to the Government and people of Lebanon At the Programme level, the assessment is deeper than in the CCA and it arises from the Team’s deep knowledge of the situation and their analysis of the country priorities, their close contact with various arms of the government system, the reports from Project Managers, and discussions with other stakeholders such as key NGOs.
A full 70% of the projects are considered Very Relevant and 23% are Relevant. Only one project is seen as Moderately Relevant. The projects in the portfolio rightly take their cue from the situation existing at the time, and reflect known government strategies and plans as well as international obligations. Of the Project Managers that responded to the related question in the Questionnaire (see Annex 6b), four appeared confused by the question. A further four saw their project contributing to the NREAP – the National Renewable Energy Action Plan. Other instruments mentioned included strategies/plans on Solid Waste and Water, and the Lebanon Crisis Response Plan. A good measure of Lebanon’s long term priorities are the SDGs (Sustainable Development Goals) and Project Managers, in their responses to the Questionnaire, overwhelmingly said that their respective projects contributed to Lebanon’s progress towards the SDGs 16 . It was ascertained during consultations with MoE and MoEW as well as CDR senior officials and others, that the contribution made by the E&E Programme is essential for a number of key aspects of the Government’s work. It can be concluded that the Programme and its portfolio of projects reflect the situation on the ground, current or immediately past, and are highly relevant to the Government and people of Lebanon. Tag: Relevance Project and Programme management Country Government |
16. | 3.2.2 Relevance to UNDP •ENVIRONMENT: nature-based solutions for development
The above have created the framework within which the E&E Programme could be designed and operated. By focussing the Key Areas and Signature Solutions it is possible to lead to seven thematic areas of direct interest to the E&E Programme. These are listed in the Table below together with an indicative matching of the 17 projects initiated during this CPD period. In the above table, the first five thematic areas arise from strategic UNDP documents whereas the Biodiversity, the Ozone and the institutional capacity thematic areas do not. It would seem that seven projects are not directly relevant to the current UNDP strategic priorities, however, 10 projects Tag: Biodiversity Relevance Programme Synergy Project and Programme management Results-Based Management |
17. | 3.2.3 Relevance to the CPD and the UNSF Outcome 3.3 The Programme addresses CPD Priority Area 4 with its focus on improving environmental governance. More specifically, it addresses the two CPD Outputs with five Indicators between them. The following table assesses the fit and relevance of the 17 projects that have been initiated or are in pipeline as they were assigned by the Programme Team to one or more CPD Indicators. However, in assessing the relevance of the constituent projects to the Outputs and Indicators of the CPD, and through them to the three Indicators of the UNSF Outcome 3.3, the internal relevance and logic problems of the SRF became more noticeable. The table shows that some of the projects can be said to be relevant to the CPD Indicators and not to the UNSF, others are relevant to the UNSF and not to the CPD. Possible reasons for this discrepancy and the apparent problem with uniform relevance are discussed in section 4.2 below. It is interesting to observe that 10 Project Managers (59%) identified correctly the CPD Output that their project contributed to. However, three PMs did not respond to the question and four provided an uncertain answer. In conclusion it can be said that the Programme is highly relevant to the Government and people of Tag: Relevance Project and Programme management |
18. | 3.3.2 Effectiveness according to the Outcome indicators and targets The results recorded above are a mixed bunch. Contributions made by projects towards the CPD Targets do not always lead to similar gains towards the UNSF Targets and as discussed elsewhere and analysed in section 4.1 below, this is the result of the internal lack of logic and relevance between the CPD and the UNSF. Considering that none of the above projects were designed specifically to deliver the indicators and targets of the current UNSF and CPD, the results show a surprisingly reasonable fit. This is the result of the painstaking effort by the Team to select the broadest and most inclusive wording for the CPD Outputs and Targets to allow existing projects to be accommodated (rather than the other way round). Whether this is justified or has been successful is discussed in section 4.3 below. The overall level of effectiveness of closed and evaluated projects in the E&E Programme is Satisfactory at the project level, but somewhat less certain when assessed against the Targets of the CPD. However, if the Indicators from the UNSF are assumed as truly indicative of what the Outcome 3.3 aimed for, then the conclusion is that the Outcome is very likely to be achieved. Whether this assumption is correct, is also discussed in section 4.3 below. Tag: Effectiveness |
Recommendations | |
---|---|
1 | 1.1 UNDP to accept that the larger part of the portfolio’s projects is carried over from previous CPD and that these projects were fully justified and relevant when they were started. 1.2 UN system that in order to resolve the challenge facing programme teams as they try to balance known national needs with diluted UNSF Outcomes, the role of the UNSF should be to set the scene by outlining needs and priorities and focusing on guidance and direction for the various agencies 1.3 It is recommended to the E&E Programme Team that they prepare a Programme Document which should distinguish between projects that are being carried over and those newly initiated in response to new needs and priorities |
2 | 1.4 It is recommended to the E&E Programme Team that all projects must undergo a terminal evaluation before closure so as to assess the relevance of each project to the country needs and priorities as identified in the CPD and the Programme, assess the effective delivery of results compared with the targets sought, assess the efficiency through which the resources available have been converted into results, and assess the likelihood of sustainability of the project’s results |
3 | 1.5 It is recommended to UNDP that in an effort to increase the likelihood of sustainability of project results, projects should preferably fit within a broad and strategic context such as national plans and strategies. |
4 | 1.6 An ex-post assessment should be carried out for each project some time (3-5 years) after project closure to assess sustainability and the circumstances that might have contributed towards or against it. |
5 | 1.7 It is recommended to the E&E Programme Team to continue with and improve the practice of requiring the formulation and implementation of a meaningful gender strategy for each project thus contributing to the achievement of SDG-5 for Gender Equality. |
6 | 1.8 It is recommended to the E&E Programme Team that in view of its current success and the continuing needs, priority should be accorded to interventions that deal with renewable energy, management of wastewater and prevention of pollution, sustainable land management and rehabilitation of degraded land and forests |