Mid-Term Evaluation for the Energy, Environment & Natural Resource Management (EENRM) Cluster in Sierra Leone

Report Cover Image
Evaluation Plan:
2015-2019, Sierra Leone
Evaluation Type:
Outcome
Planned End Date:
05/2018
Completion Date:
09/2018
Status:
Completed
Management Response:
Yes
Evaluation Budget(US $):
16,462

Share

Document Type Language Size Status Downloads
Download document TOR EENRM_Outcome Evaluation 2017.pdf tor English 183.46 KB Posted 126
Download document EENRM_MT Evl_Report_Final.pdf report English 1407.98 KB Posted 98
Title Mid-Term Evaluation for the Energy, Environment & Natural Resource Management (EENRM) Cluster in Sierra Leone
Atlas Project Number: 00042426
Evaluation Plan: 2015-2019, Sierra Leone
Evaluation Type: Outcome
Status: Completed
Completion Date: 09/2018
Planned End Date: 05/2018
Management Response: Yes
Corporate Outcome and Output (UNDP Strategic Plan 2018-2021)
  • 1. Output 1.4.1 Solutions scaled up for sustainable management of natural resources, including sustainable commodities and green and inclusive value chains
  • 2. Output 2.1.1 Low emission and climate resilient objectives addressed in national, sub-national and sectoral development plans and policies to promote economic diversification and green growth
Evaluation Budget(US $): 16,462
Source of Funding: M&E Plan
Evaluation Expenditure(US $): 16,462
Joint Programme: No
Joint Evaluation: Yes
  • Joint with GoSL, Donors
Evaluation Team members:
Name Title Email Nationality
Alexandre Diouf alexdiouf@hotmail.com
Dunstan Spencer
GEF Evaluation: No
Key Stakeholders: EPA, DMD, GEF, EPA, NPAA, DMD, ONS, MLCPE, MMMR
Countries: SIERRA LEONE
Comments:

An outcome evalution in 2018 would enable the assessment of progress made towards achieving the targets as outlined in the CPD and provide constructive feedback to inform further interventions.

This Evaluation has been changed to a Mid-Term Evaluation since the project is still on-going.

 

 

Lessons
1.

Lesson 1: Importance of community mobilisation and buy-in 

Social mobilisation, although a gruelling process, has been instrumental in the smooth running of operations in in the woodlots project. A lot of time was spent in mobilising and informing residents about their roles in the project before actual implementation began. This is not a one-off activity as had been envisaged but is a continuous process throughout the project cycle. As such, when budgeting this should be considered. Second, it is important to fully include community members at all levels of project implementation. The full participation of public institutions in project implementation has allowed them to be fully aware of all the project components and performance. With beneficiary communities, however, limited participation in implementation has not allowed them to be fully aware ofall projects aspects.

Lesson 2: Importance of a good community structure, joint planning, and an effctive communication strategy 

A good community organisation structure is a major boost to successful community project implementation, especially when it puts community members at the core. Such a structure gives mandate and legitimacy to a group to represent residents. In the EENRM projects, such structures have not been put in place. This creates a problem when information is to be passed on to the community or formal negotiations are to be held regarding the implementation of the projects.

Lesson 3: Public and community contribution is key component in project sustainability 

Where communities are contributing labour and even monies to pay for initial or running costs, there is a greater sense of ownership and responsibility. The projects that are promoted by UNDP–SL EENRM request beneficiary communities to provide limited resources that can help in the sustainability. In contrast, the support to public institutions does not request government participation, which does not promote ownership and can weaken the sustainability of the projects. It is important to request government financial participation in all the projects that aim to support government actions or provide services.

Lesson 4: Streamlining gender issues in project design, implementation, and M&E Gender issues must be addressed in project planning, implementation, and monitoring.

To do this, EENRM should provide gender analysis training to its staff. This will provide the tools staff need to be able to analyse the effects of gender on project planning and implementation, and take mitigation measures to reduce or eliminate negative outcomes. Gender will be taken into account through the design of appropriate M&E tools that allow for sex-disaggregated data. EENRM and partners will then be able to monitor the effects of project activities on women’s workload to ensure that the project is reducing rather than adding to it. By increasing gender awareness, EENRM will be able to take mitigation measures using appropriate means such as discussions with implementing partners, community leaders, and IEC campaigns.

Lesson 5: The compliance assessment can be a starting point to build partners’ capacity and assess its importance UNDP has conducted a partner assessment for all the public institutions it is planning to work with. The assessment was part of the compliance assessment process that is required before funds can be transferred to them for project implementation. The assessment made clear that when strengths and weaknesses are known and expectations and visions are shared, both UNDP and partners can grow in the programme and become more effective. The option to let several directorate and ministries lead the implementation of some of the EENRM activities is a direct result of the partner capacity assessment that has been done.

Lesson 6: Comprehensive M&E systems are needed for programme implementation and performance measurement
During the MTE it became clear that EENRM projects do not have adequate M&E systems in place. The situation of projects that have received external funds was nonetheless better because they had an M&E section in their proposals. EENRM has invested little into putting in place adequate M&E systems for its projects. Given the multiplicity of the activities undertaken, an M&E system must be in place either for each project or at cluster level. Failure to have such a system will likely lead to a misinterpretation of the change brought about by the projects and result in inadequate use of resources.

Lesson 7: The need for careful selection of project performance indicators and targets during the design
Although some of the projects have log frames, they are struggling to use the performance indicators that were selected during the design. Not only are those indicators difficult to measure, e.g the level of green house gas emission, but there is also no methodology that was shared to teach project actors how to measure and report them. In most cases, the projects are using a new set of indicators that look easier to measure or they are only monitoring output delivery. Selecting an appropriate set of indicators that would truly measure project performance during implementation, and are measurable using planned project resources, is necessary.

Lesson 8: With the right system, projects can be appropriately implemented using government staff
The projects are using some government staff during implementation. This arrangement seems to be working very well across the board. We can thus report that the use of government staff in project implementation can help to increase project efficiency and sustainability since it reduces costs and helps to build local capacity.

Lesson 9: The extent to which UNDP EENRM projects streamline accountability principles, depends on the origin of their funding
Defining projects’ theory of change is important for accountability purposes and is a critical part of the monitoring and evaluation system. EENRM projects that are funded with internal resources never get a detailed description of their ToC, while those that have been funded with external funds always describe their theory of change and identify the outcomes they want to achieve. Pushing for more utilization of external funding will likely result in more accountability from EENRM and the ministry of mines has, for example, welcome the establishment of a library that facilitates access to mining information for all actors. EENRM has established that library within the ministry.


Findings
Recommendations
1

Irrespective of the source of the funding, UNDP–SL EENRM should ensure that all interventions are linked to a strategic direction and have a clear background analysis, including a omprehensive gender analysis. EENRM should also ensure that its projects are environmentally sound and put in place adequate measures for participation of marginalised/vulnerable people.
 

2

It is important for all the projects implemented with support from UNDP–SL EENRM that a comprehensive M&E system be put in place in order to follow project performance and allow adequate, timely decisions to be made. M&E systems start with the establishment of a good ToC and results framework for each project. The second step is to identify good performance indicators with associated baseline and target values.
 

3

The financial participation of either the government (when it comes to support provided to public institutions) or communities (when it comes to projects that are directly implemented at community level) should be integrated as a key strategy to ensure participants’ buy-in and project sustainability.
 

4

Pilot projects are meant to demonstrate a concept or build a body of evidence that can be used to advocate for more support or attract interest from external actors. EENRM has a number of those projects. It should rethink the strategies around them and develop an additional set of activities for their scale-up.
 

5

The option to subsidise the improved cookstoves in the current GEF project will certainly increase the number of households who will purchase them. After the project ends, the private sector actor will no longer be able to offer the stoves at the subsidised price; this may bring the situation back to what it was before the project. We recommend that EENRM work with financial service providers during implementation to put in place an adequate financial product that poor households will be able to use in order to continue to have access to those cookstoves.

6

The most important objective of a pilot project is not its implementation per se but the quantity and depth of information it generates. Therefore, it is important that all pilot projects build in a good knowledge management protocol attached to its M&E system. UNDP–SL EENRM should ensure that the results from the pilot projects are published and shared among project staff as well as partner communities and institutions to inform future programme decisions.

7

UNDP participation in good governance starts with the compliance assessment that has been done on the public partners. As the projects are developed and implemented, responsibilities should be given to the public partners based on their behaviour with the management of the resources and their adherence to the principles of good governance. That should be continued.

8

It is important for staff to keep in mind and be constantly reminded of the importance of doing a full gender analysis. Such an analysis helps to identify the roles and responsibilities of both men and women in the activities that are supported and detect any impact—positive or negative—that they may have on the established gender balances in the institutions and communities to be supported.

9

Discrimination against people based on their origin, politics, or religious background may sometimes exist in Sierra Leone. It is important that EENRM staff be sensitised to the possibility of discrimination, and make sure they do not participate in its perpetuation. The projects that are supported by UNDP–SL EENRM should be explicit regarding anti-discrimination and fully ensure the participation of people of all origins and political, ethnic, or religious affiliation. Like the gender analysis, a detailed analysis on the role of vulnerable people in target institutions/communities should be conducted. The potential impacts of the interventions should be identified prior to project implementation and monitored throughout.

10

UNDP–SL EENRM should lead the establishment of a platform that will include all the UN agencies, partner institutions, and relevant public directorates interested in the EENRM issues at country level. That platform could support a sector-wide diagnosis that will help to identify the most important activities and pull resources together to maximise their benefits.

11

The following thematic interventions should be explored for future interventions:
- Establish a database of all the EENRM-related policies and creation of an online “one-stop shop” for interested parties.
- Support the updating of the EENRM-related policies and advocate to include the thematic areas that are important to UNDP (UNDAF).
- Continue to strengthen the institutional and organisational capacity of the public institutions.
- As the work under policy development is particularly important, take the opportunity to integrate UNDP strategic options and priorities in government policies.
- Identify appropriate entry points and support the development of guidelines for private sector engagement in promotion of alternative/efficient sources of energy (cookstoves, solar, biogas, etc.)
- Implement support to activities specified in the Nationally Determined Contribution framework.
- Ensure effective implementation of the Climate Change Act and relevant regulations.
- Promote environmental communication and awareness-raising.
- Implement support to the Nationally Appropriate Mitigation Action Framework.
- Support climate mainstreaming at local councils and other MDAs.

1. Recommendation:

Irrespective of the source of the funding, UNDP–SL EENRM should ensure that all interventions are linked to a strategic direction and have a clear background analysis, including a omprehensive gender analysis. EENRM should also ensure that its projects are environmentally sound and put in place adequate measures for participation of marginalised/vulnerable people.
 

Management Response: [Added: 2018/09/11] [Last Updated: 2018/11/01]

The EENRM project and ONS will endeavour to integrate marginalsed and vulnerable people as target groups and include them in project implementation activities.

Key Actions:

Key Action Responsible DueDate Status Comments Documents
UNDP EENRM will conduct capacity building training/community stakeholders meetings with community volunteers ,flood victims,district disaster management committees on community early warning systems and disaster
[Added: 2018/11/01] [Last Updated: 2019/03/26]
EENRM/ DMD-ONS 2019/02 Completed This activity has been completed. For ease of reference, please refer to activity 5.1.7 in the Disaster Management Department-ONS report attached below History
2. Recommendation:

It is important for all the projects implemented with support from UNDP–SL EENRM that a comprehensive M&E system be put in place in order to follow project performance and allow adequate, timely decisions to be made. M&E systems start with the establishment of a good ToC and results framework for each project. The second step is to identify good performance indicators with associated baseline and target values.
 

Management Response: [Added: 2018/09/11] [Last Updated: 2018/11/01]

The EENRM programme is working closely with PMSU to ensure that new projects have an effective Monitoring and Evaluation Framework in the project and all adopt setting targets based on baseline values.

Key Actions:

Key Action Responsible DueDate Status Comments Documents
To create workflows that suit design of regular data collection by project staff.
[Added: 2018/11/01] [Last Updated: 2019/03/26]
Project Manager 2019/01 Completed The 2019 AWP has been well streamlined and now include performance indicators for each result area to ensure easy and systematic data collection process during implementation of 2019 planned activities. History
Staff undertake routine monitoring and a Mid-term Evaluation was conducted in August 2018. Additionally, the findings from the mid-term evaluation will be used to improve the quality of future similar projects.
[Added: 2018/11/01] [Last Updated: 2018/11/08]
Project Manager 2018/11 Completed Will ensure to take into consideration lessons learnt from the mid-term evaluation to strengthen the next phase of programming. History
3. Recommendation:

The financial participation of either the government (when it comes to support provided to public institutions) or communities (when it comes to projects that are directly implemented at community level) should be integrated as a key strategy to ensure participants’ buy-in and project sustainability.
 

Management Response: [Added: 2018/09/11] [Last Updated: 2018/11/01]

 Each project within the cluster will endeavour to include a clearly defined Transition and Sustainbility section to show Government or community involvement.

Key Actions:

Key Action Responsible DueDate Status Comments Documents
Review of on-going projects to ensure sustainability strategy is clearly enshrined
[Added: 2018/11/01] [Last Updated: 2018/12/06]
Project Manager 2018/12 Completed Gaps identified and addressed History
Follow up with respective public and private sectors on co-financing commitment.
[Added: 2018/11/01]
Programme Specialist 2019/01 Overdue-Initiated This to be escalated to Steering Committee/Board
4. Recommendation:

Pilot projects are meant to demonstrate a concept or build a body of evidence that can be used to advocate for more support or attract interest from external actors. EENRM has a number of those projects. It should rethink the strategies around them and develop an additional set of activities for their scale-up.
 

Management Response: [Added: 2018/09/11] [Last Updated: 2018/11/02]

Activities originally developed which were intended to contribute to the achievement of project results will be reviewed and updated. The Programme has optimized data collection with staff’s field visits to make the data collection as quick and easy as possible and in realtime for knowledge management and strategic alignment.

Key Actions:

Key Action Responsible DueDate Status Comments Documents
to review pilot project activites & update where necessary in order to strengthen expected outcomes and donor support.
[Added: 2018/11/02] [Last Updated: 2019/03/26]
Programme Specialist 2019/01 Canceled All project managers will be engaged to contribute to the process History
Developed a monitoring template
[Added: 2018/11/02] [Last Updated: 2018/11/08]
PMSU 2018/10 Completed PMSU have developed monitoring templates which project staff now used in carryout monitoring on their activities. History
5. Recommendation:

The option to subsidise the improved cookstoves in the current GEF project will certainly increase the number of households who will purchase them. After the project ends, the private sector actor will no longer be able to offer the stoves at the subsidised price; this may bring the situation back to what it was before the project. We recommend that EENRM work with financial service providers during implementation to put in place an adequate financial product that poor households will be able to use in order to continue to have access to those cookstoves.

Management Response: [Added: 2018/09/11] [Last Updated: 2018/11/03]

To hire the service of a Financial Engineering expert to design a  grant/rebate scheme model that ensure fianacial sustainability.

Key Actions:

Key Action Responsible DueDate Status Comments Documents
ToR advertised and the service of consultant for Financial Engineering Expert to be hired
[Added: 2018/11/03] [Last Updated: 2019/03/26]
Project Manager 2019/01 Completed The recruitment process for a consultant has been launched/advertised and the selection process is still underway History
To improved cook stoves GEF project based on Mid Term Review (MTR) outcome.
[Added: 2018/11/03] [Last Updated: 2019/03/26]
Project Manager 2018/12 Completed MTR of the Efficient Energy Production and Utilization of Charcoal project under which the cook stoves component fall has been conducted and completed. Recommendations and lessons learned from the MTR process have been well noted for improvement of the cook stove component of the project. History
6. Recommendation:

The most important objective of a pilot project is not its implementation per se but the quantity and depth of information it generates. Therefore, it is important that all pilot projects build in a good knowledge management protocol attached to its M&E system. UNDP–SL EENRM should ensure that the results from the pilot projects are published and shared among project staff as well as partner communities and institutions to inform future programme decisions.

Management Response: [Added: 2018/09/11] [Last Updated: 2018/11/03]

 Increased collaboration with the UNDP Communication Unit for information gathering and communication to ensure good management protocol of related results from the pilot projects.

Key Actions:

Key Action Responsible DueDate Status Comments Documents
Conduct joint monitoring visits of pilot projects
[Added: 2018/11/03] [Last Updated: 2019/03/26]
Programme Specialist 2018/12 Completed Project implementation is being monitored on a quarterly basis and is an ongoing process. This takes place jointly with partners. History
7. Recommendation:

UNDP participation in good governance starts with the compliance assessment that has been done on the public partners. As the projects are developed and implemented, responsibilities should be given to the public partners based on their behaviour with the management of the resources and their adherence to the principles of good governance. That should be continued.

Management Response: [Added: 2018/09/11] [Last Updated: 2018/11/03]

Building on good practices in our collaboration with public partners.

Key Actions:

Key Action Responsible DueDate Status Comments Documents
Conduct due diligence at all times, ensuring to assess, verify and enforce compliance based on signed agreement with public partners
[Added: 2018/11/03] [Last Updated: 2019/03/26]
Project Manager, PMSU 2019/01 Completed With regards project activities to be implemented by partners for 2019 annual work plan, due diligence was followed and ensured through LoAs which were signed by UNDP and partner institutions before going forward. History
8. Recommendation:

It is important for staff to keep in mind and be constantly reminded of the importance of doing a full gender analysis. Such an analysis helps to identify the roles and responsibilities of both men and women in the activities that are supported and detect any impact—positive or negative—that they may have on the established gender balances in the institutions and communities to be supported.

Management Response: [Added: 2018/09/11] [Last Updated: 2018/11/03]

Staff encouraged to take online trainings on Gender and Environment

Key Actions:

Key Action Responsible DueDate Status Comments Documents
Staff implementation reports to segregate gender participation and involvement
[Added: 2018/11/03] [Last Updated: 2019/03/26]
Project Manager 2018/12 Completed Annual progress reports submitted for 2018 included Gender analysis which was incorporated to determine the level of participation and involvement of both male and female into project activities. Management has made it clear to all programme staff to include gender in their reporting History
Men and women involved in every stage of the project
[Added: 2018/11/03] [Last Updated: 2018/11/08]
Project Manager 2018/11 Completed On going History
9. Recommendation:

Discrimination against people based on their origin, politics, or religious background may sometimes exist in Sierra Leone. It is important that EENRM staff be sensitised to the possibility of discrimination, and make sure they do not participate in its perpetuation. The projects that are supported by UNDP–SL EENRM should be explicit regarding anti-discrimination and fully ensure the participation of people of all origins and political, ethnic, or religious affiliation. Like the gender analysis, a detailed analysis on the role of vulnerable people in target institutions/communities should be conducted. The potential impacts of the interventions should be identified prior to project implementation and monitored throughout.

Management Response: [Added: 2018/09/11] [Last Updated: 2018/11/03]

The "leave no one behind" re-interated in all meetings

Key Actions:

Key Action Responsible DueDate Status Comments Documents
Staff take related certificate courses
[Added: 2018/11/03] [Last Updated: 2019/03/26]
Programme Specialist 2018/12 Completed This is ongoing. In the meantime, all staff in the CO have completed and received certificates for at least 3 mandatory courses. History
Work with vulnerable communities and targeting poor people and those at risk
[Added: 2018/11/03] [Last Updated: 2019/03/26]
Project Manager 2019/01 Completed The main focus of the work of the CO is to build resilience of vulnerable communities and poor people History
10. Recommendation:

UNDP–SL EENRM should lead the establishment of a platform that will include all the UN agencies, partner institutions, and relevant public directorates interested in the EENRM issues at country level. That platform could support a sector-wide diagnosis that will help to identify the most important activities and pull resources together to maximise their benefits.

Management Response: [Added: 2018/09/11] [Last Updated: 2018/11/03]

Ensured it was included as a major agenda item in UNCT meetings

Key Actions:

Key Action Responsible DueDate Status Comments Documents
Collaborate with other agencies on EENRM issues
[Added: 2018/11/03] [Last Updated: 2019/03/26]
Team Leader 2018/12 Completed The cluster works with partners. Partners implement substantial aspects of planned activities of the Cluster. The cluster has been renamed to "Sustainable growth" as part of the Change management processes. History
Invite others to participate in Inception workshops, assessments
[Added: 2018/11/03] [Last Updated: 2019/03/26]
Team Leader 2018/12 Completed It is a norm at UNDP to work with partners, hence partners are fully involved at all stages of UNDP's work, including all projects. History
11. Recommendation:

The following thematic interventions should be explored for future interventions:
- Establish a database of all the EENRM-related policies and creation of an online “one-stop shop” for interested parties.
- Support the updating of the EENRM-related policies and advocate to include the thematic areas that are important to UNDP (UNDAF).
- Continue to strengthen the institutional and organisational capacity of the public institutions.
- As the work under policy development is particularly important, take the opportunity to integrate UNDP strategic options and priorities in government policies.
- Identify appropriate entry points and support the development of guidelines for private sector engagement in promotion of alternative/efficient sources of energy (cookstoves, solar, biogas, etc.)
- Implement support to activities specified in the Nationally Determined Contribution framework.
- Ensure effective implementation of the Climate Change Act and relevant regulations.
- Promote environmental communication and awareness-raising.
- Implement support to the Nationally Appropriate Mitigation Action Framework.
- Support climate mainstreaming at local councils and other MDAs.

Management Response: [Added: 2018/09/11] [Last Updated: 2018/11/03]

Resource mobilizing on these themes.

Key Actions:

Key Action Responsible DueDate Status Comments Documents
Developed concept notes on related thematic areas to be submitted to appropriate donors
[Added: 2018/11/03] [Last Updated: 2018/11/08]
Team Leader 2019/02 Overdue-Initiated Submitted to Donor History

Latest Evaluations

Contact us

220 East 42nd Street
20th Floor
New York, NY 10017
Tel. +1 646 781 4200
Fax. +1 646 781 4213
erc.support@undp.org