Mid Term Evaluation for Improved Charcoal Production

Report Cover Image
Evaluation Plan:
2016-2020, Uganda
Evaluation Type:
Mid Term Project
Planned End Date:
08/2016
Completion Date:
08/2017
Status:
Completed
Management Response:
Yes
Evaluation Budget(US $):
25,000

Many of the expected objectives /results / products / outputs of the Green Charcoal Project have been achieved.  Others have not been achieved to the degree expected at mid – point.  Overall, several of the achievements have been positive when they materialized yet there are also several issues with these achieved outputs. National level products (database, survey, database, standards, research, etc.) have been accomplished, laying a baseline for the attainment of policies, norms, financial investments and  other  products/outcomes sought within the Project  and beyond.  The  implementation and the attainment of products at the district level has been done at a good pace of delivery, notwithstanding issues and challenges encountered at the local level. Capacity building processes have taken place, in particular at the pilot sites, and they have been positive in dissemination of information and in the generation of  capabilities,  in  particular when  they were  accompanied  by  demonstration.  Extensive awareness  raising  has  taken  place,  mainly  through  dissemination  of  information  through  media. However, several expected outputs and products are being achieved at a moderate level, yet several barriers, weaknesses, issues, and challenges have been identified which (if acted upon in the remaining implementation period) can positively impact upon the progress toward outcomes and in achieving the Project’s objective in a sustained and sustainable manner.

 

Share

Document Type Language Size Status Downloads
Download document TOR MTR international Consultant for green Charcoal project.doc tor English 108.50 KB Posted 245
Download document TOR MTR National Consultant for green Charcoal project (1).doc tor English 108.00 KB Posted 220
Download document E xecutive Summary.docx summary English 106.77 KB Posted 160
Download document Green Charcoal MTR Final Report.pdf report English 1593.04 KB Posted 552
Title Mid Term Evaluation for Improved Charcoal Production
Atlas Project Number: 00074620
Evaluation Plan: 2016-2020, Uganda
Evaluation Type: Mid Term Project
Status: Completed
Completion Date: 08/2017
Planned End Date: 08/2016
Management Response: Yes
Focus Area:
  • 1. Environment & Sustainable Development
  • 2. Others
Corporate Outcome and Output (UNDP Strategic Plan 2014-2017)
  • 1. Output 1.5. Inclusive and sustainable solutions adopted to achieve increased energy efficiency and universal modern energy access (especially off-grid sources of renewable energy)
SDG Goal
  • Goal 7. Ensure access to affordable, reliable, sustainable and modern energy for all
SDG Target
  • 7.1 By 2030, ensure universal access to affordable, reliable and modern energy services
  • 7.2 By 2030, increase substantially the share of renewable energy in the global energy mix
Evaluation Budget(US $): 25,000
Source of Funding: GEF
Evaluation Expenditure(US $): 40,000
Joint Programme: No
Joint Evaluation: No
Evaluation Team members:
Name Title Email Nationality
MARIA ONESTINI International Evaluator rponesti@criba.edu.ar
ROBERT NABANYUMYA National Evaluator nabanyumya@yahoo.com UGANDA
GEF Evaluation: Yes
GEF Project Title: Improved Charcoal Production Technologies and sustainable land management practices through an integrated approach
Evaluation Type: Mid-term Review
Focal Area: Biodiversity
Project Type: EA
GEF Phase: GEF-1
GEF Project ID: 4644
PIMS Number: PIMS-4493
Key Stakeholders: Ministry of Energy
Countries: UGANDA
Lessons
1.

 

 

At the project formulation level there was a strong stakeholder analysis.  This occurred not only identifying institutions or typologies of institutions to be involved in the Project but also going further by carrying out an analysis of anticipated roles and responsibilities in project implementation.

 


Tag: Climate change

2.

Stakeholders engagement in the Green Charcoal Project is varied and ample.  Stakeholders from different sorts of institutions participate in one way or another in project’s guidance (technical and screening committees) as well as in implementation.


Tag: Climate change

3.

The conceptual design of the Project is based on thorough background, base information, research findings as well as experiences from other projects and programs dealing with charcoal, sustainable forest management, and sustainable land management in Uganda.   A very complete conceptual design describes at an excellent level of detail the issue of unsustainable use and production of charcoal in Uganda, with data, information, and lessons from other relevant projects embedded into the project design. Furthermore, it identifies the barriers that the country needs to overcome in order to improve charcoal production technologies and establish sustainable land management practices.


Tag: Climate change

4.

The relatively high ownership of the Project expressed by most national and district level stakeholders, as well their expressed support of its long-term objectives, indicates that there are no major institutional framework and governance risks threats to sustainability from most stakeholders.

 


Tag: Climate change

5.

The project did not include in its design a thorough sustainability strategy.  An exit strategy that fully takes into account the sustained implementation of achieved products in the short and medium term is lacking.  In summary, sustainability probabilities are mixed, yet the Project is still on time to design and drive sustainability factors by supporting several aspects of the achievements made and by driving an exit strategy.


Tag: Climate change

6.

The Project has had a series of setbacks in implementation processes. The Project also sustained changes brought about by UNDP reorganization, organizational and implementing standstills, and problems with the successful completion or attainment of products.  All of this, in turn, generated some setbacks in implementation and programming, in obtaining products and, of course, in obtaining results and achievements.  Therefore, it is considered that a no cost extension request ought to be granted, if requested.

 


Tag: Climate change

7.

Annual reporting (PIRs) take place following planned timing and content. The PIRs convey what activities and  processes  have taken place or not within the implementation period they  refer and they are sufficiently analytical to be of use for further implementation and for adaptive management if the latter would occur. Quarterly reporting does take place broadly following what was planned at design.

 


Tag: Climate change

8.

The Project does have a concrete communication strategy, called “A Communication and Media Campaign Strategy for The National Biomass Energy Strategy (NBEST) And Green Charcoal Project Activities”.   In line with this strategy, awareness campaigns using live radio talk-shows, community meetings, multi-stakeholder dialogues are on-going in all four target districts.  There is no discernible application of the strategy at the national level.


Tag: Climate change

Findings
1.

Measure                 MTR Rating                                                                        Achievement Description

Progress Towards Results

Objective: “Secure multiple environmental benefits by addressing the twin challenges of unsustainable utilization of fuel wood (including charcoal) and poor land management practices common in Uganda’s woodland through technology transfer, enhancement of the national policy framework and the promotion of Sustainable Land Management (SLM) and Sustainable  Forest  Management (SFM) practices.”

 

Achievement Rating: Moderately Satisfactory MS

As a composite, there are some shortcomings in the achievement of objective. Although some outputs have been achieved in a moderately effective and a moderately efficient manner (particularly those directly involving pilot sites and work with districts and national level data/surveys/etc.), other outputs and outcomes that make up and articulate the objective have not been met at the expected mid-point level. Since the objective seeks effects, and the approach to implementation has mainly centred on products, there is a gap in achieving objective fully. No shortcomings in terms of relevance.

I.        Data   collection   and   improved coordination and enforcement of regulations governing the biomass energy sector, in particular those related to sustainable charcoal.

 

Achievement Rating: Moderately Satisfactory MS

Some shortcomings in the achievement of objectives in terms of effectiveness and efficiency.    Although data collection has been successfully attained, improved coordination and enforcement of regulations governing sustainable charcoal has not been attained as a result of the Project.   There is little emphasis on the latter.   No shortcomings in terms of relevance

II.      Dissemination     of     appropriate technologies for sustainable charcoal production in selected (4) charcoal-producing districts (Mubende, Kiboga, Nakaseke and Kiryandongo).

 

Achievement Rating: Moderately Satisfactory MS

Some shortcomings in the achievement of objectives in terms of effectiveness.  A few shortcomings in terms of efficiency. Dissemination of technologies for charcoal production has taken place, perhaps not at the pace expected, yet several aspects of technology transferred have been ignored such as business aspects in the whole charcoal production as well as other market-related issues and sustainability. No shortcomings in terms of relevance.

III.        Strengthening the capacity of key stakeholders in SFM and SLM best practices and establishment of sustainable woodlots.

 

Achievement Rating: Moderately Satisfactory MS

Although pilots have been implemented in SFM and SLM, they are not truly linked to best practices and establishment of sustainable woodlots. To date, the majority of SFM pilots have dealt almost exclusively with reforestation (i.e. planting of seedlings) and SLM pilots (although highly positive in and of themselves) do not link to sustainable woodlots nor to charcoal production. Therefore, taking into account effectiveness there are shortcomings in this criterion.  Delivery indicative that there are no major shortcomings in efficiency. Few shortcomings in relevance.

Project Implemen- tation               & Adaptive Management

Rating: Moderately Satisfactory MS

Some shortcomings in the achievement of objectives in terms of effectiveness. Few shortcoming in term of efficiency.  Roles of partners unclear to some degree.  No adaptive management implementation has taken place.

Sustainability

Rating: Moderately Likely ML

At the midpoint, and as a composite assessment, there are some risks that not all key outcomes will carry on after project closure.  However, some outputs/activities likely to carry on after closure.


Recommendations
1

Evaluation Recommendation 1. Streamline organizational issues to improve coordination and efficiency in implementation.

2

Evaluation Recommendation 2:  Strengthen business and private sector components of the whole value chain of charcoal within the Project outputs and processes

3

Evaluation Recommendation 3:  Strengthen private sector engagement in the project, generate and disseminate business plans of introduced technologies.

4

Evaluation Recommendation 4:  Monitor and follow up the implemented pilots regarding technical issues and other matters that have arisen after piloting interventions or the products achieved with the districts thus far.

5

Evaluation Recommendation 5: Strengthen the monitoring of implementation, technologies, outputs and achievements at the results, effects, and impact levels; making certain that monitoring fulfils quality assurance and is based on robust techniques and adequate methodologies.

6

Evaluation Recommendation 5: Strengthen project expected outcomes that deal with national existing and ongoing policy, as well as national regulatory and institutional work on sustainable charcoal and land tenure security, and dealing with the mobilization of investments in the charcoal field.


7

Evaluation Recommendation 7: Ensure that gender considerations are fully integrated in all aspects of the Project to enhance benefit sharing for women from project intervention, taking into account (among other issues) women’s participation in the charcoal chain, land ownership, income generation, and meaningful participation

8

Evaluation Recommendation 8: Strengthen technical capacities of the Project; by incorporating technical advisor and by making certain that the products, processes, monitoring, and outputs are of high technical quality and focusing on the overall project goal. 


 

9

Evaluation Recommendation 9:  Generate materials to be used for follow-up, up-scaling and replication. 

10

  Evaluation Recommendation 10:  Begin to generate an exit strategy soonest, including financial and policy/institutional aspects that need to be taken into account if achievements are to be sustained after project closure

11

Evaluation Recommendation or Issue 11:   Apply for a “no - cost extension”.

1. Recommendation:

Evaluation Recommendation 1. Streamline organizational issues to improve coordination and efficiency in implementation.

Management Response: [Added: 2017/08/14] [Last Updated: 2017/08/14]

This is agreeable and already prioritized

Key Actions:

Key Action Responsible DueDate Status Comments Documents
Streamline organizational issues to improve coordination and efficiency in implementation.
[Added: 2018/08/29]
UNDP and partners 2020/12 Initiated
2. Recommendation:

Evaluation Recommendation 2:  Strengthen business and private sector components of the whole value chain of charcoal within the Project outputs and processes

Management Response: [Added: 2017/08/14]

This is agreeable and already prioritized

Key Actions:

Key Action Responsible DueDate Status Comments Documents
Strengthen business and private sector components of the whole value chain of charcoal within the Project outputs and processes
[Added: 2018/08/29]
UNDP and partners 2020/12 Initiated
3. Recommendation:

Evaluation Recommendation 3:  Strengthen private sector engagement in the project, generate and disseminate business plans of introduced technologies.

Management Response: [Added: 2017/08/14]

This is agreeable and already prioritized

Key Actions:

Key Action Responsible DueDate Status Comments Documents
Strengthen private sector engagement in the project, generate and disseminate business plans of introduced technologies.
[Added: 2018/08/29]
UNDP and partners 2020/12 Initiated
4. Recommendation:

Evaluation Recommendation 4:  Monitor and follow up the implemented pilots regarding technical issues and other matters that have arisen after piloting interventions or the products achieved with the districts thus far.

Management Response: [Added: 2017/08/14]

This is being followed up as a priority

Key Actions:

Key Action Responsible DueDate Status Comments Documents
4.1 Follow up on maintenance of introduced technologies; functioning, operational/maintenance and adaptation issues.
[Added: 2017/08/14] [Last Updated: 2018/08/29]
PMU/MAAIF/MEMD/UNDP 2018/06 No Longer Applicable [Justification: project ended and issue being addressed in other projects]
On-going History
4.2 Establish a multi-stakeholder monitoring plan focusing on identifying and addressing any technical issues associated with the pilots. Document lessons learned on the piloted initiated
[Added: 2017/08/14] [Last Updated: 2018/08/29]
PMU/MEMD/MAAIF 2018/06 Completed Oo-going History
5. Recommendation:

Evaluation Recommendation 5: Strengthen the monitoring of implementation, technologies, outputs and achievements at the results, effects, and impact levels; making certain that monitoring fulfils quality assurance and is based on robust techniques and adequate methodologies.

Management Response: [Added: 2017/08/14]

Already Identified as a priority by the PMU

Key Actions:

Key Action Responsible DueDate Status Comments Documents
Strengthen the monitoring of implementation, technologies, outputs and achievements at the results, effects, and impact levels;
[Added: 2018/08/29]
UNDP and partners 2020/12 Initiated
6. Recommendation:

Evaluation Recommendation 5: Strengthen project expected outcomes that deal with national existing and ongoing policy, as well as national regulatory and institutional work on sustainable charcoal and land tenure security, and dealing with the mobilization of investments in the charcoal field.


Management Response: [Added: 2017/08/14]

Agreed

Key Actions:

Key Action Responsible DueDate Status Comments Documents
Strengthen project expected outcomes that deal with national existing and ongoing policy, as well as national regulatory and institutional work on sustainable charcoal
[Added: 2018/08/29]
UNDP and partners 2020/12 Initiated
7. Recommendation:

Evaluation Recommendation 7: Ensure that gender considerations are fully integrated in all aspects of the Project to enhance benefit sharing for women from project intervention, taking into account (among other issues) women’s participation in the charcoal chain, land ownership, income generation, and meaningful participation

Management Response: [Added: 2017/08/14]

Agreed

Key Actions:

Key Action Responsible DueDate Status Comments Documents
Integrate gender considerations in all aspects of the Project
[Added: 2018/08/29]
UNDP and partners 2020/12 Initiated
8. Recommendation:

Evaluation Recommendation 8: Strengthen technical capacities of the Project; by incorporating technical advisor and by making certain that the products, processes, monitoring, and outputs are of high technical quality and focusing on the overall project goal. 


 

Management Response: [Added: 2017/08/14]

Agreed

Key Actions:

Key Action Responsible DueDate Status Comments Documents
Strengthen technical capacities of the Project; by incorporating technical advisor and by making certain that the products, processes, monitoring, and outputs are of high technical quality
[Added: 2018/08/29]
UNDP and partners 2020/12 Initiated
9. Recommendation:

Evaluation Recommendation 9:  Generate materials to be used for follow-up, up-scaling and replication. 

Management Response: [Added: 2017/08/14]

Agreeable

Key Actions:

Key Action Responsible DueDate Status Comments Documents
Generate materials to be used for follow-up, up-scaling and replication
[Added: 2018/08/29]
UNDP and partners 2020/12 Initiated
10. Recommendation:

  Evaluation Recommendation 10:  Begin to generate an exit strategy soonest, including financial and policy/institutional aspects that need to be taken into account if achievements are to be sustained after project closure

Management Response: [Added: 2017/08/14]

Agreed

Key Actions:

Key Action Responsible DueDate Status Comments Documents
Begin to generate an exit strategy soonest, including financial and policy/institutional aspects
[Added: 2018/08/29]
UNDP and partners 2020/12 Initiated
11. Recommendation:

Evaluation Recommendation or Issue 11:   Apply for a “no - cost extension”.

Management Response: [Added: 2017/08/14]

Agreed

Key Actions:

Key Action Responsible DueDate Status Comments Documents
Apply for a “no - cost extension
[Added: 2018/08/29]
UNDP and partners 2018/12 Completed completed

Latest Evaluations

Contact us

1 UN Plaza
DC1-20th Floor
New York, NY 10017
Tel. +1 646 781 4200
Fax. +1 646 781 4213
erc.support@undp.org