End of Project Evaluation: Sustainable Rural Biomass Energy Project

Report Cover Image
Evaluation Plan:
2014-2018, Bhutan
Evaluation Type:
Final Project
Planned End Date:
09/2016
Completion Date:
09/2016
Status:
Completed
Management Response:
Yes
Evaluation Budget(US $):
30,000

Share

Document Type Language Size Status Downloads
Download document TE_SRBE Bhutan_Final.pdf report English 1919.05 KB Posted 572
Download document Signed Management Response for SRBE Terminal Evaluation 2016.pdf related-document English 1930.52 KB Posted 224
Download document ToR Evaluation_SRBE UNDP Bhutan.pdf tor English 333.51 KB Posted 250
Title End of Project Evaluation: Sustainable Rural Biomass Energy Project
Atlas Project Number: 00060755
Evaluation Plan: 2014-2018, Bhutan
Evaluation Type: Final Project
Status: Completed
Completion Date: 09/2016
Planned End Date: 09/2016
Management Response: Yes
Focus Area:
  • 1. Environment & Sustainable Development
  • 2. Others
Corporate Outcome and Output (UNDP Strategic Plan 2014-2017)
  • 1. Output 1.5. Inclusive and sustainable solutions adopted to achieve increased energy efficiency and universal modern energy access (especially off-grid sources of renewable energy)
Evaluation Budget(US $): 30,000
Source of Funding: GEF
Evaluation Expenditure(US $): 17,140
Joint Programme: No
Joint Evaluation: No
Evaluation Team members:
Name Title Email Nationality
Andreas Karner
Chhimi Dorji
GEF Evaluation: Yes
GEF Project Title:
Evaluation Type:
Focal Area: Climate Change
Project Type: FSP
GEF Phase: GEF-4
GEF Project ID:
PIMS Number: 4181
Key Stakeholders: MoEA
Countries: BHUTAN
Lessons
1.

- SRBE Project in line with governmental priorities. The project was proposed during the 10th FYP 2008-2013 and implemented between the 10th FYP and 11th FYP (2013-2018). The main thrust area for the energy sector in the 11th FYP is "Energy Security for Sustainable Development" The Sector Key Result Areas are "Energy Security Maintained" and "Meaningful and Purposeful Renewable Energy Promoted". The SRBE project is very much in line with this strategy of energy security enhancement, meaningful use of renewable energy, and also GHG emission reduction in line with clean energy choices.

The 12th FYP plan is currently under preparation. However, it was learnt from GNHC (the planning body) that the SRBE project and related projects are very much in line with the country’s philosophy of "Green Development" and "Environmental Protection" while also looking at socio-economic development.

- Information on fuel wood consumption change initiated by SRBE project. There were mixed information available about the actual fuel wood consumption across Bhutan. According to a Kuensel article of July 17, 201613, it mentions that fuelwood consumptions at household levels, both in urban and rural areas have decreased due to grid-electrification, LPG penetration after getting road connectivity and also availability of biogas plants. However, the overall fuelwood supplied by the Natural Resources Development Corporation (NRDCL) show its supply of firewood increased by 1,165 truckloads in the past six years. From 31,176 cubic metres equivalent to 3,896 truckloads in 2010, it increased to 35,826 cubic metres in 2012, and further up to 35,988 cubic metres in 2014. The increase in total fuel wood is attributed mostly to institutions and schools where they do mass cooking using fuel wood.

- Functionality and appropriateness of the Database website by DRE. The database had information of the project, list of Dzonkhangs with their respective stoves, pdf copies of all the documents published through the SRBE Project and few other applications. However, the database is limited in terms of mapping or analysis applications. There is no functionality of analysis or mapping. If only static reports and figures these were going to be uploaded it would not have been necessary to set up a database, and used only the Department/Ministry website with a link to the project. Plus, who and how will the database be maintained in future is not certain. There are also frequently reported cases of the system being off-line due to various security breaches.

13 Refer to article: http://www.kuenselonline.com/firewood-consumption-declines/ UNDP-GEF Project "Bhutan Sustainable Rural Biomass Energy" (PIMS# 4181) Terminal Evaluation Report

- Importance of local considerations when designing cook stoves. The initially provided cook stove design by Austrian experts was finally too costly, and thus the major challenge was to convince/attract it to beneficiaries due to their low affordability. A re-design of the cook stove provided through a local Bhutanese consultant resulted in a cost reduction by almost 50%; the new design used local materials in construction and helped to reduce the metallic components, which in turn had brought down the costs significantly. The cost effective solution helped the project to design the financial support mechanism, which allowed for supplying metallic parts cook stove from Project funds, while the beneficiary households were supposed to provide locally made mud bricks and their work force for free.

- The general experience in implementing pilot demonstration elsewhere has highlighted the importance of location of pilot demonstration site, which requires careful consideration with a preference to logistics and ease of access. This is based on the fact, and borne out of experience that a pilot faces many unforeseen challenges during its implementation and therefore easy access to its location makes the field monitoring easy and also helps in addressing the issues in a timely manner as they arise, a key to the success.

- The project implementation faced the challenges of time involved in travelling in the hilly terrain of the country, weather and remote location of villages. Project activities, choice of intervention locations, workplans, monitoring and budgets should always consider the challenges of working in remote rural areas that can be especially difficult to access during inclement weather.

- Adequate staffing of the partner agencies involved in the project implementation is important in a national level projects as the amount of coordination required is high. Frequent changes of staff in DAHE had an adverse effect on the project progress. Project Board must maintain an oversight on the staffing requirement since the project has tight time-line for completing all the activities within 3 years.

- Financial sustainability of cook stove programmes is a key for future replication. To achieving financial sustainability, a pure subsidy-based scheme is not mandatory for a successful BET program, as long as improved stoves are affordable to the rural poor. As the experience from SRBE shows, low-cost stoves can be achieved if they are built mostly from locally available materials. Using funds for capacity building, quality assurance during the construction and motivation of users rather than subsidizing stoves or their components would help develop a sustainable system whereby users would be willing to pay the full costs and purchase cook stoves for their benefits.

- Project’s interaction with financial sector stakeholders was poor and requires appropriate strategy while looking into replication. An in-depth interaction with the banks and other stakeholders in Bhutan’s financial sector was not carried out during project preparation. Without full information of the financial barriers the project document and the PPM has few outcomes to provide fiscal incentive and create market for BET. However, in the current situation of the financial market, these are unlikely to be achieved by EOP. The banks in Bhutan have imposed restriction on offering loans to the private sector under the directive of the RGoB to maintain the foreign currency exchange to contain the current account deficit. With the situation likely to continue, the project may not possibly be in a position to extend the fiscal incentives and push for any market linked mechanisms.

- Implementation support through local stakeholders has been ensured with future replication to be effectively designed. The project has made implementation arrangements with the help of another RGoB agency DAHE, which has helped in the implementing the key project activities through awareness creation and installations of improved stoves, as the NFEI had strong links with rural population. The implementation capabilities and the support required by DAHE to ensure smooth roll out of cook stoves in 16 districts, however, has been limited and DAHE had confessed that the SRBE project had a negative impact on their core mandate, which is to educate rural population to increase their adult literacy rate. The findings from the field visits of the TE team warrants a strengthening of the role and engagement of CBOs and NGOs in future replication of improved stoves and BET dissemination and monitoring of the use, functionality, spare parts provision and maintenance of the stoves, which was not mandated neither to NFEIs nor to the CBOs involved in the SRBE project so far. Therefore, effective functionality checking and monitoring the use were raised as necessary elements to understand the real benefits and issues in any future replication project.

- Linkages with other initiatives and programmes are key for maintaining exchange of experiences and know-how. Since there are similar activities ongoing in countries of the region e.g. Bangladesh, Nepal, or India addressing the issues of efficient fuelwood use, or avoidance of indoor air pollution, linking similar projects and initiatives should be considered. Also, with reference to international donor programs, NGOs’ activities or public-private partnerships such as the Global Alliance for Clean Cookstoves (GACC) shall be pursued, to enhance the knowledge about efficient BET at stakeholder levels, from the central government to district administration level, Community Based Organizations and private sector players.


Findings
1.

Project design and implementation approach, including the project results framework

The SRBE Project was designed to promote market-based mechanisms to create demand for efficient technologies using fuel wood and support from the government in the form of incentives and policy measures. Thus the objective of the project to reduce the pressure on local forest due to inefficient consumption of fuel-wood, reduce the rate of deforestation and improve the air indoor air quality and an overall reduction in the GHG emissions through a wide spread use of use biomass energy technologies should be met.

Based on the barriers identified and mentioned above (see chapter 3.2) and through its specific approach, the project design seeks to enhance the impact leading to the reduction of GHG emissions from the improved production and efficient use of biomass throughout the country which will be achieved through awareness creation, training, building the capacity of governmental and non-governmental organizations and private sector participation.

The project’s focus on introduction of improved biomass energy technologies (BET) and dissemination of improved cook stoves/heating stoves was rationalized by a number of factors, including: (i) energy supply situation and heavy dependence on fuel wood in spite of impressive electrification; (ii) growth in the energy demand and biomass supply potential; (iii) dependence on conventional cook stoves by a high percentage of rural population with low level of household income; (iv) air quality issues in the household leading to high prevalence of health issues (mainly respiratory disorder among women and children); and (v) reduced wastage of biomass residue in private industries especially sawmills, from alternate BET which convert the residue into feedstock or for use in space heating or other alternatives than fuel wood.

The project results framework for SRBE is included in Table 5. The project design was developed in 2010-11 and from the beginning has foreseen 3 components, which respond to the general barriers that SRBE was trying to overcome. It is, however, noticeable that the PMU used the same initial version of project logframe throughout the whole project period, although targets have been revised throughout project implementation.

Altogether, the project planning logframe has established overall 44 indicators to track and report progress under the three components, which seems too high. While it is positive that there is a rationale to the indicators provided in the logframe towards the achievement of an outcome, the number of indicators is excessive with most outputs burdened with more than one indicator. This fact has been already raised by the Mid-Term Review (MTR) and recommendation was placed to revise the project logframe. Moreover, there are few indicators that have become redundant due to reasons outside the control of the project, which could have been removed from the logframe.

A description of indicators that have been revised or have become redundant over the project period is provided below:

- The Project Objective "Removal of barriers to sustainable utilization of available biomass resources in the country and application of BET" has been revised as per the 2nd PSC meeting, the number of energy-efficient stoves by end of project was reduced from 20,000 to 13,522.

- "Number of enterprises supplying clean and efficient biomass energy systems and services by end of project" was reduced from 3 to 2 as per the mid-term of the project.

- "Fiscal incentives such as smart subsidies to enable market mechanisms introduced" from Output 2.2. Due to the financial restrictions imposed by the RGoB and absence of lending by the banks to the private sector, it became obvious that the project will be unlikely to influence markets in the remaining time and the financial resources available to it, which will not contribute to the sustainability of the project.

- "Implemented and operational BET Full Scale model on biomass gasification for electricity services and thermal applications" from Output 2.5. As the Project Board had suggested dropping the demonstration of this technology as it was economically unviable, Output 2.5 had to be revised and the corresponding indicator on biomass gasification be dropped.

- "Project developers and micro-entrepreneurs trained on different aspects of BETs" from Output 3.3. The project has faced severe challenge in getting the attention of private sector players and fabricators to respond to the tenders. It was suggested at the MTR stage that unless the market has a steady demand for improved cook stoves and other BET, it is unlikely the entrepreneurs will come forward and invest their resources – and therefore the indicator to be dropped.

With the exception of the oversights and defaults regarding the project logframe, the project scope, design and implementation approach otherwise, including the overall structure of the project results framework, can be considered as satisfactory for a full-size project addressing the critical elements of sustainable biomass energy policy making, awareness raising and capacity building, supporting the implementation of innovative practices and ascertaining market mechanisms for local BET to develop and evolve.


Recommendations
1

Appropriate MRV System.

2

Calculation of GHG emission reductions related to carbon capture from lower fuel wood use and afforestation/plantation programme.

3

Develop a “Lessons Learnt Report” for in-country dissemination.

4

Project to provide grounds for continued awareness programs on efficient stoves and their benefits.

5

Appropriate design facilities and national research for new BET

6

Indoor air pollution and other health issues being further considered in the design of stoves programmes.

7

Enhance the support of capacity-building and skills programmes for beneficiaries.

8

Insurance schemes to help provide more safety and security in project implementation

1. Recommendation:

Appropriate MRV System.

Management Response: [Added: 2017/01/09] [Last Updated: 2018/10/11]

The management has taken up the need to strengthen the current Bioenergy Database provided online on the bioenergy website with the Department of Renewable Energy. It was discussed to make the database more robust incorporating GHG emission and fuelwood consumption reduction calculations. The strengthening of the current data base, quality monitoring and reporting system online and detailed activity reporting system will be incorporated in the similar programmes in future.  

Key Actions:

Key Action Responsible DueDate Status Comments Documents
?Strengthening of the current data base, quality monitoring and reporting system online and detailed activity reporting system will be incorporated in the similar programmes in future.
[Added: 2018/10/11]
CCM&E, UNDP, DRE, 2016/12 Completed Presented on the 9th (Final) Project Board Meeting of the SRBE Project on 30 December 2016. History
2. Recommendation:

Calculation of GHG emission reductions related to carbon capture from lower fuel wood use and afforestation/plantation programme.

Management Response: [Added: 2017/01/09] [Last Updated: 2018/10/11]

The accounting of GHG emission reduction through afforestation or plantation programme would further strengthen the GHG mitigation potential of the SRBE project intervention. However, due to difficulty in quantifying the potential in the absence of the baseline GHG emission from the community forestry, this is a real challenge. Therefore, the accounting of GHG emission reductions from afforestation programme is being dropped in this project period. The baseline information will have to be strengthened and once available, the same would be accounted in the similar projects in future.

Related, through the UNDP Low Emission Capacity Building (LECB) project with the National Environment Commission, UNDP is supporting the development of a GHG inventory data management system for data management and archival purposes, which will help ensure data accuracy, consistency and management going forward. For estimation of emissions, Bhutan will follow the guidance and methodology of the IPCC and UNFCCC. Capacity building and awareness raising for the GHG inventory estimation will be conducted through the Third National Communications and Biennial Update Report projects set to commence in early 2017, led by the National Environment Commission

Key Actions:

Key Action Responsible DueDate Status Comments Documents
The baseline information and GHG calculation methodology on the afforestation will be explored with the Department of Forest and Park Services of the Ministry of Agriculture and Forests in the future projects.
[Added: 2018/10/11] [Last Updated: 2018/11/12]
UNDP Bhutan CCM 2016/12 Completed Presented on the final board meeting of the SRBE Project on 30 December 2016. History
3. Recommendation:

Develop a “Lessons Learnt Report” for in-country dissemination.

Management Response: [Added: 2018/10/11]

A lessons learnt report summarizing the projects key findings, experiences, achievement, knowledge management and the way forward for future replication shall be prepared in close collaboration with the project implementation unit of the Department of Renewable Energy. This recommendation was presented on the 9th Board meeting and was agreed on the preparation of the summary of the lessons learnt for future reference.

Key Actions:

Key Action Responsible DueDate Status Comments Documents
?Presented on the final board meeting of the SRBE Project on 30 December 2016.
[Added: 2018/10/11]
CCM&E, UNDP, DRE, SFED 2016/12 Completed
4. Recommendation:

Project to provide grounds for continued awareness programs on efficient stoves and their benefits.

Management Response: [Added: 2018/10/11]

The major component of the project is the awareness creation among different levels of stakeholders including the direct beneficiaries. Efforts have been made to disseminate correct information about the project and the intended benefits that can be reaped through using these improved stoves. However, as per the recommendation, a day workshop has not been conducted during the project period. Therefore, an awareness workshop with school children and target beneficiaries will be conducted, and such components built in to future similar programmes.

Key Actions:

Key Action Responsible DueDate Status Comments Documents
Presented on the final board meeting of the SRBE Project on 30 December 2016. Mentioned that the awareness creation is a continuous process and need to be continued even after the closure of the project.
[Added: 2018/10/11]
UNDP Bhutan CCM team 2016/12 Completed
5. Recommendation:

Appropriate design facilities and national research for new BET

Management Response: [Added: 2018/10/11]

The design and testing facility is essential for continuous improvement of the technology through conduct of research and demonstration of stove technologies. In addition, such facility would enable the quality control and monitoring of the field activities, to provide timely intervention. Further, standardization of the technology is important to maintain efficiency, effectiveness, safety and GHG mitigation on the different products being developed based on market needs. Therefore, the Nodal Agency, the Department of Renewable Energy, is requested to look into developing an appropriate research facility to cater the growing Biomass Energy Technologies demand in the market. 

Key Actions:

Key Action Responsible DueDate Status Comments Documents
Presented on the final board meeting of the SRBE Project on 30 December 2016.
[Added: 2018/10/11]
UNDP Bhutan CCM team 2016/12 Completed
6. Recommendation:

Indoor air pollution and other health issues being further considered in the design of stoves programmes.

Management Response: [Added: 2018/10/11]

Indoor air pollution is a major concern and one of the objective of this project is to reduce indoor air pollution and exposure of pollutants to human health. In this respect, the project undertook the design of the stoves to maximize removal of smoke from the kitchen. However, the project did not work with the Ministry of Health on the promotion of the stoves or the monitoring of their impact on exposure to indoor air pollution. Therefore, in the future similar projects, the Ministry of Health and the Department of Energy could jointly develop indoor air pollution standards and quality control measures.

Key Actions:

Key Action Responsible DueDate Status Comments Documents
- Presented on the final board meeting of the SRBE Project on 30 December 2016. - To be considered in future project design and implementation including the SAARC Development Fund regional project, once finalized.
[Added: 2018/10/11]
UNDP Bhutan CCM team 2017/06 Completed
7. Recommendation:

Enhance the support of capacity-building and skills programmes for beneficiaries.

Management Response: [Added: 2018/10/11]

The capacity building of the beneficiaries and trained technicians in the communities were enhanced. However, during this project, specific support to help women become entrepreneurs in the the area of stoves design/construction/promotion/supply – or in the development of related entrepreneurial activities feasible through access to the improved stoves – were not undertaken in a dedicated way. This will therefore be explored as part of project design and implementation for future projects, including the SDF Energy project once final agreement is in place.

Key Actions:

Key Action Responsible DueDate Status Comments Documents
Presented on the final board meeting of the SRBE Project on 30 December 2016.
[Added: 2018/10/11]
UNDP Bhutan CCM team 2017/06 Completed
8. Recommendation:

Insurance schemes to help provide more safety and security in project implementation

Management Response: [Added: 2018/10/11]

Due to the nature of the Sustainable Rural Biomass Project, it was not initially considered as a high-risk project. However, during implementation including in remote areas, staff and contractors working on the project were required to navigate difficult terrain and topological conditions, with demonstrated risks. Therefore, exploring options for insurance schemes from local insurance companies as part of future project structures would be sensible and is recommended by Management. It should be noted that RGoB officials are considered to be part of the Civil Service Welfare Scheme, therefore such coverage may not be necessary, depending on who is hired under the project. An additional consideration would be to seek confirmation of insurance schemes from companies (such as construction companies) whose services are procured through future UNDP projects where risks are identified during project development. Such schemes could be indicated as ‘desirable’ in procurement TORs. This would need to be addressed on a case-by-case basis.

Key Actions:

Key Action Responsible DueDate Status Comments Documents
Presented on the 8th board meeting of the SRBE Project on 22 March 2016.
[Added: 2018/10/11]
UNDP Bhutan CCM team 2016/12 Completed

Latest Evaluations

Contact us

1 UN Plaza
DC1-20th Floor
New York, NY 10017
Tel. +1 646 781 4200
Fax. +1 646 781 4213
erc.support@undp.org