End of Project Evaluation: Sustainable Rural Biomass Energy Project

Report Cover Image
Evaluation Plan:
2014-2018, Bhutan
Evaluation Type:
Final Project
Planned End Date:
09/2016
Completion Date:
09/2016
Status:
Completed
Management Response:
Yes
Evaluation Budget(US $):
30,000

Share

Document Type Language Size Status Downloads
Download document TE_SRBE Bhutan_Final.pdf report English 1919.05 KB Posted 1847
Download document Signed Management Response for SRBE Terminal Evaluation 2016.pdf related-document English 1930.52 KB Posted 1126
Download document ToR Evaluation_SRBE UNDP Bhutan.pdf tor English 333.51 KB Posted 1157
Title End of Project Evaluation: Sustainable Rural Biomass Energy Project
Atlas Project Number: 00060755
Evaluation Plan: 2014-2018, Bhutan
Evaluation Type: Final Project
Status: Completed
Completion Date: 09/2016
Planned End Date: 09/2016
Management Response: Yes
Focus Area:
  • 1. Environment & Sustainable Development
  • 2. Others
Corporate Outcome and Output (UNDP Strategic Plan 2018-2021)
  • 1. Output 1.5. Inclusive and sustainable solutions adopted to achieve increased energy efficiency and universal modern energy access (especially off-grid sources of renewable energy)
Evaluation Budget(US $): 30,000
Source of Funding: GEF
Evaluation Expenditure(US $): 17,140
Joint Programme: No
Joint Evaluation: No
Evaluation Team members:
Name Title Nationality
Andreas Karner
Chhimi Dorji
GEF Evaluation: Yes
GEF Project Title:
Evaluation Type:
Focal Area: Climate Change
Project Type: FSP
GEF Phase: GEF-4
GEF Project ID:
PIMS Number: 4181
Key Stakeholders: MoEA
Countries: BHUTAN
Lessons
1.

- SRBE Project in line with governmental priorities. The project was proposed during the 10th FYP 2008-2013 and implemented between the 10th FYP and 11th FYP (2013-2018). The main thrust area for the energy sector in the 11th FYP is "Energy Security for Sustainable Development" The Sector Key Result Areas are "Energy Security Maintained" and "Meaningful and Purposeful Renewable Energy Promoted". The SRBE project is very much in line with this strategy of energy security enhancement, meaningful use of renewable energy, and also GHG emission reduction in line with clean energy choices.

The 12th FYP plan is currently under preparation. However, it was learnt from GNHC (the planning body) that the SRBE project and related projects are very much in line with the country’s philosophy of "Green Development" and "Environmental Protection" while also looking at socio-economic development.

- Information on fuel wood consumption change initiated by SRBE project. There were mixed information available about the actual fuel wood consumption across Bhutan. According to a Kuensel article of July 17, 201613, it mentions that fuelwood consumptions at household levels, both in urban and rural areas have decreased due to grid-electrification, LPG penetration after getting road connectivity and also availability of biogas plants. However, the overall fuelwood supplied by the Natural Resources Development Corporation (NRDCL) show its supply of firewood increased by 1,165 truckloads in the past six years. From 31,176 cubic metres equivalent to 3,896 truckloads in 2010, it increased to 35,826 cubic metres in 2012, and further up to 35,988 cubic metres in 2014. The increase in total fuel wood is attributed mostly to institutions and schools where they do mass cooking using fuel wood.

- Functionality and appropriateness of the Database website by DRE. The database had information of the project, list of Dzonkhangs with their respective stoves, pdf copies of all the documents published through the SRBE Project and few other applications. However, the database is limited in terms of mapping or analysis applications. There is no functionality of analysis or mapping. If only static reports and figures these were going to be uploaded it would not have been necessary to set up a database, and used only the Department/Ministry website with a link to the project. Plus, who and how will the database be maintained in future is not certain. There are also frequently reported cases of the system being off-line due to various security breaches.

13 Refer to article: http://www.kuenselonline.com/firewood-consumption-declines/ UNDP-GEF Project "Bhutan Sustainable Rural Biomass Energy" (PIMS# 4181) Terminal Evaluation Report

- Importance of local considerations when designing cook stoves. The initially provided cook stove design by Austrian experts was finally too costly, and thus the major challenge was to convince/attract it to beneficiaries due to their low affordability. A re-design of the cook stove provided through a local Bhutanese consultant resulted in a cost reduction by almost 50%; the new design used local materials in construction and helped to reduce the metallic components, which in turn had brought down the costs significantly. The cost effective solution helped the project to design the financial support mechanism, which allowed for supplying metallic parts cook stove from Project funds, while the beneficiary households were supposed to provide locally made mud bricks and their work force for free.

- The general experience in implementing pilot demonstration elsewhere has highlighted the importance of location of pilot demonstration site, which requires careful consideration with a preference to logistics and ease of access. This is based on the fact, and borne out of experience that a pilot faces many unforeseen challenges during its implementation and therefore easy access to its location makes the field monitoring easy and also helps in addressing the issues in a timely manner as they arise, a key to the success.

- The project implementation faced the challenges of time involved in travelling in the hilly terrain of the country, weather and remote location of villages. Project activities, choice of intervention locations, workplans, monitoring and budgets should always consider the challenges of working in remote rural areas that can be especially difficult to access during inclement weather.

- Adequate staffing of the partner agencies involved in the project implementation is important in a national level projects as the amount of coordination required is high. Frequent changes of staff in DAHE had an adverse effect on the project progress. Project Board must maintain an oversight on the staffing requirement since the project has tight time-line for completing all the activities within 3 years.

- Financial sustainability of cook stove programmes is a key for future replication. To achieving financial sustainability, a pure subsidy-based scheme is not mandatory for a successful BET program, as long as improved stoves are affordable to the rural poor. As the experience from SRBE shows, low-cost stoves can be achieved if they are built mostly from locally available materials. Using funds for capacity building, quality assurance during the construction and motivation of users rather than subsidizing stoves or their components would help develop a sustainable system whereby users would be willing to pay the full costs and purchase cook stoves for their benefits.

- Project’s interaction with financial sector stakeholders was poor and requires appropriate strategy while looking into replication. An in-depth interaction with the banks and other stakeholders in Bhutan’s financial sector was not carried out during project preparation. Without full information of the financial barriers the project document and the PPM has few outcomes to provide fiscal incentive and create market for BET. However, in the current situation of the financial market, these are unlikely to be achieved by EOP. The banks in Bhutan have imposed restriction on offering loans to the private sector under the directive of the RGoB to maintain the foreign currency exchange to contain the current account deficit. With the situation likely to continue, the project may not possibly be in a position to extend the fiscal incentives and push for any market linked mechanisms.

- Implementation support through local stakeholders has been ensured with future replication to be effectively designed. The project has made implementation arrangements with the help of another RGoB agency DAHE, which has helped in the implementing the key project activities through awareness creation and installations of improved stoves, as the NFEI had strong links with rural population. The implementation capabilities and the support required by DAHE to ensure smooth roll out of cook stoves in 16 districts, however, has been limited and DAHE had confessed that the SRBE project had a negative impact on their core mandate, which is to educate rural population to increase their adult literacy rate. The findings from the field visits of the TE team warrants a strengthening of the role and engagement of CBOs and NGOs in future replication of improved stoves and BET dissemination and monitoring of the use, functionality, spare parts provision and maintenance of the stoves, which was not mandated neither to NFEIs nor to the CBOs involved in the SRBE project so far. Therefore, effective functionality checking and monitoring the use were raised as necessary elements to understand the real benefits and issues in any future replication project.

- Linkages with other initiatives and programmes are key for maintaining exchange of experiences and know-how. Since there are similar activities ongoing in countries of the region e.g. Bangladesh, Nepal, or India addressing the issues of efficient fuelwood use, or avoidance of indoor air pollution, linking similar projects and initiatives should be considered. Also, with reference to international donor programs, NGOs’ activities or public-private partnerships such as the Global Alliance for Clean Cookstoves (GACC) shall be pursued, to enhance the knowledge about efficient BET at stakeholder levels, from the central government to district administration level, Community Based Organizations and private sector players.


Findings
1.

4 Key Findings

4.1 Project Design / Formulation

4.1.1 Project design and implementation approach, including the project results framework

The SRBE Project was designed to promote market-based mechanisms to create demand for efficient technologies using fuel wood and support from the government in the form of incentives and policy measures. Thus the objective of the project to reduce the pressure on local forest due to inefficient consumption of fuel-wood, reduce the rate of deforestation and improve the air indoor air quality and an overall reduction in the GHG emissions through a wide spread use of use biomass energy technologies should be met. Based on the barriers identified and mentioned above (see chapter 3.2) and through its specific approach, the project design seeks to enhance the impact leading to the reduction of GHG emissions from the improved production and efficient use of biomass throughout the country which will be achieved through awareness creation, training, building the capacity of governmental and non-governmental organizations and private sector participation. 


Tag: Emission Reduction Energy Environment Policy Relevance Programme/Project Design Project and Programme management Results-Based Management Theory of Change

2.

4.1 Project Design / Formulation

4.1.1 Project design and implementation approach, including the project results framework

A description of indicators that have been revised or have become redundant over the project period is provided below: - The Project Objective “Removal of barriers to sustainable utilization of available biomass resources in the country and application of BET” has been revised as per the 2nd PSC meeting, the number of energyefficient stoves by end of project was reduced from 20,000 to 13,522.- “Number of enterprises supplying clean and efficient biomass energy systems and services by end of project” was reduced from 3 to 2 as per the mid-term of the project. - “Fiscal incentives such as smart subsidies to enable market mechanisms introduced” from Output 2.2. Due to the financial restrictions imposed by the RGoB and absence of lending by the banks to the private sector, it became obvious that the project will be unlikely to influence markets in the remaining time and the financial resources available to it, which will not contribute to the sustainability of the project. - “Implemented and operational BET Full Scale model on biomass gasification for electricity services and thermal applications” from Output 2.5. As the Project Board had suggested dropping the demonstration of this technology as it was economically unviable, Output 2.5 had to be revised and the corresponding indicator on biomass gasification be dropped. - “Project developers and micro-entrepreneurs trained on different aspects of BETs” from Output 3.3. The project has faced severe challenge in getting the attention of private sector players and fabricators to respond to the tenders. It was suggested at the MTR stage that unless the market has a steady demand for improved cook stoves and other BET, it is unlikely the entrepreneurs will come forward and invest their resources – and therefore the indicator to be dropped.


Tag: Relevance Programme/Project Design Results-Based Management

3.

4.1.2 Assumptions and Risks

The projects risk and the mitigation strategies to address those risks were summarized in the Project Document (Risks 1-8)

In addition to the above, the Project Results Framework is listing a number of assumptions for successfully reaching: The project objective: - Recognition of the government on importance of reducing GHG emission and continuing commitment towards it. - Recognition of demonstration value of improved stoves by end-users; thus, resulting in widespread replacement of older more polluting stoves. - Rural households adopt the new technology which comes at higher costs Outcome 1: -Government continues to see biomass as a priority Outcome 2: - Existing knowledge, experience, skills and sources are adequate to source and access technologies - Availability of credit facility from financial institutions - Viability of demonstrated new BETs are resulting in decisions to further replicate BETs Outcome 3:-Target groups are willing to participate and are receptive to awareness campaigns and capacity building activities - Continued interest of participants to receive trainings - Widespread interest among potential replicators of BETs to join site visits and continued interest and willingness of project owners to host site visits


Tag: Programme/Project Design Project and Programme management Results-Based Management Risk Management Private Sector

4.

4.1.3 Lessons from other relevant projects incorporated into project design

The Project Document does not include a specific chapter to highlight the lessons from other projects that have been incorporated into project design, but the Project Document refers in some places indirectly to lessons learned from other project activities that the SRBE project wanted to pursue: - In the description of the components (chapter 2.3.1 of Project Document) under component 2, the development of technical standards, certification and guidelines for the design and manufacturing of efficient cook stoves is referred to build upon “experiences and lessons learnt from an earlier attempt to establish a smokeless stoves programme, which had suffered from a lack of adequate ownership and M&E, as well as poor perception of its effectiveness and value”.- Furthermore, within the same chapter, reference is made to experiences from GEF-Small Grants Programme (SGP), which through UNDP-Bhutan has in the past supported activities related to sustainable energy production through efficient use of biomass resources, particularly targeting schools, religious institutions and health centres. However, due to a lack of policy and institutional support, this work has not expanded beyond the project sites. The project was supposed to “build on the experience and lessons learnt from the SGP program to upscale and mainstream BETs into the wider rural economy, through wider understanding and accelerated market growth”.


Tag: Knowledge management Programme/Project Design Project and Programme management Results-Based Management

5.

4.1.4 Planned stakeholder participation

During the PIF and PPG stages, stakeholders consisting of relevant agencies, non-government organizations (NGOs) and private sector groups who could participate in the formulation and contribute to the successful implementation of the Project have been identified. Several workshops and individual face- to-face consultations were conducted to assess their needs, clearly define their role/involvement both during the project preparation and project implementation and ascertain their commitment to the objectives of the Project. As some stakeholders are also beneficiaries of the Project outcomes, their participation and commitment were ensured from the beginning, which adds assurance to the success of the Project. A table showing the different stakeholder groups and their involvement in the Project as well as the benefits they were supposed to expect to receive where included in the Project Document. Furthermore, a detailed capacity assessment of the DRE, which was nominated to be the Lead Executing Agency of the Project was provided. One of the main stakeholders and beneficiary of this project are women and children. The project design has given sufficient emphasis for inclusion of women in certain outputs. 


Tag: Efficiency Partnership Project and Programme management Country Government Capacity Building Civil Societies and NGOs Private Sector Vulnerable

6.

4.1.5 Replication approach (continuation)

At the government level, the DRE was supposed to become the host of the “Knowledge and Learning Platform” which aimed to consolidate, preserve and ensure continuing use of information and knowledge that were obtained and accumulated during the capacity building and other activities in this Project. However, the Platform as such was not implemented, only individual training and capacity-building activities were conducted in the scope of the Project. Special provision of trainings to the Social Forestry Department and its Community Forest Management Groups in order to strengthen capacity in the field of sustainable tree plantation for energy purposes where such elements of the sustainability and replication strategy that the Project followed in its initial design and were implemented as planned.


Tag: Effectiveness Efficiency Communication Knowledge management Partnership Capacity Building Data and Statistics Civil Societies and NGOs Private Sector Vulnerable

7.

4.1.6 UNDP comparative advantage

The strength of UNDPs involvement to implement SRBE is its long-term involvement in providing technical assistance for renewable energy development to developing countries with a focus on poverty alleviation and energy security. With UNDP having implemented more than 2,000 rural energy development projects for more than 20 years in a number of developing countries, it has developed strength from an excellent track record of developing local capacity, effectively working with multiple stakeholders from public and private sectors, technical experts, civil society, and grassroots level organizations. In the context of rural development, UNDP has a multidimensional development perspective, and an ability to address cross-sectoral issues and inclusiveness in constituency building. Therefore, the Project uses the UNDP’s experiences and comparative advantage in promoting sustainability, inclusive growth and poverty reduction, by supporting policies, capacity-building and innovative actions with regard to resource efficiency, climate-change mitigation, and access to renewable energy.


Tag: Climate Change Adaptation Climate change governance Energy Environment Policy Programme Synergy Results-Based Management Strategic Positioning Capacity Building Technical Support Civil Societies and NGOs Private Sector

8.

4.1.7 Linkages between project and other interventions within the sector (continuation)

Otherwise, the Project Document doesn’t mention specific linkages with other previous project activities or interventions within the sector, mainly because they did not exist at a national level at that time. The only references made in the Project Document are:

- TERI4 conducted in 2005 an extensive survey in Bhutan to develop an Integrated Energy Management Masterplan (IEMMP) for the Royal Government of Bhutan. The survey covered a total of 5,396 households (about 5% of total) spread over urban and rural areas and covering all Dzongkhags in the country, to assess the energy consumption in the residential sector. It was the only detailed energy demand assessment available at the time when the project was designed. Nevertheless, it is assumed that the consumption trend identified at that time hasn’t changed much since then.


Tag: Forestry Emission Reduction Energy Effectiveness Programme Synergy

9.

4.1.7 Linkages between project and other interventions within the sector (Continuation)

- Otherwise, large-scale initiatives on the national level have not existed before SRBE. It was therefore concluded, that a widespread dissemination of the efficient cook stoves in the country wouldn’t have happened without the Project intervention.

- Regarding the use of sawdust from sawmills, there are no well-documented initiatives. The Natural Resource Development Corporation Ltd. (NRDCL) currently operates the briquetting plant installed at Ramtokto in peri-urban Thimphu. The machine has a capacity of 500 kg/hour. The machine was bought from India. Owing to the poor quality of the machinery, it is often prone to breakdown. It has been reported in the national newspaper that the demand for Briquette has actually increased. In 2015, the briquette factory in Ramtokto, Thimphu sold 270.91 tonnes of briquette. There was a slight drop in supply last year than 2014 when 316.15 tonnes were sold. However, the demand for briquette was higher in 2013. From the total 404.34 tonnes produced 400.41 tonnes were sold. This was 56.16 tonnes more than the sale in 2012


Tag: Emission Reduction Energy Environment Policy Effectiveness Efficiency Partnership Programme/Project Design Awareness raising Private Sector

10.

4.1.8 Management arrangements

The SRBE project is being implemented by UNDP and executed by the Department of Renewable Energy (Ministry of Economic Affairs) and Social Forestry and Extension Division (SFED) of Ministry of Agriculture and Forests under guidelines for nationally implemented modality (NIM). Under this arrangement, UNDP assumes the overall management of the project under the direction of the NPD from DRE. The day-to-day management of the project has been carried out by a Project Management Unit (PMU) under the overall guidance of the Project Board (PB) consisting of DRE, Social Forestry and Extension Division, Non- Formal Education Division, and UNDP. The PMU is established within the premises of DRE, MoEA and reports to the DRE, the Executing Agency and the PSC. The Project management structure is provided in the figure below.


Tag: Effectiveness Implementation Modality Oversight Project and Programme management

11.

4.2 Project Implementation

4.2.1 Adaptive management

SRBE experienced delay in its initiation phase from January 2013, and implementation was slow until September 2013. The project start-up coincided with the 2nd parliamentary elections, during which the Election Commission of Bhutan banned public meetings and gatherings all over the country. The project implementation was designed to start with workshops, awareness raising and training components, but since this was in conflict with the order of the Election Commission, the project had to wait for the elections to be over in July 2013. Thereafter, three months of peak monsoon period coupled with farming season delayed the field work, as beneficiaries could not participate in the consultation process. Monsoon season also restricts the movement of vehicles in the hills and to the villages, which are away from main road. During this period, the project carried out redesign of stoves. Due to long delays in getting the original project activities started, the PMU needed to adaptively change the planned activities from the beginning to achieve the objectives set by the Project. In this regard, much of what has finally been accomplished by the Project, with notable progress achieved in 2015 and 2016, has been a result of adaptive management, which has helped the Project to achieve progress. 


Tag: Challenges Efficiency Project and Programme management Quality Assurance Risk Management

12.

4.2.1 Adaptive management (continuation)

- Engagement of NGOs/CBOs and procurement of works, goods and consultancy services through tendering processes proved to be a lengthy process. The PMU was initially not allowed to directly engage NGOs and CBOs due to restrictions in the government procurement rules and regulations. The PMU was instructed to invite a tender for Zhemgang Dzongkhag, which was at that time the second pilot Dzongkhag after Trashigang selected for implementing pilot cook stoves. The tender for the Expression of Interest was invited but the response was very poor. BAoWE was the only NGO that participated in the bidding process for Zhemgang Dzongkhag. It was also evident that the costs of dissemination of cook stoves through NGOs/CBOs were significantly higher than the project’s initial estimate. In line with the project’s objective to engage CBOs/NGOs in the dissemination of cook stoves, contract agreements were finally signed with local NGOs – Bhutan Association of Women Entrepreneurs (BAoWE) and Tarayana Foundation, for the implementation of stoves in 4 districts in South and Central Bhutan (completed by July 2015).

- In other cases, the PMU had to resort to conducting multiple tendering processes due to non-responsive bids. The participation of private sector has been very poor, since the RGoB has frozen loans from financial institutions for business entities due to the shortage of Indian Rupee in the market. Remoteness of the project sites, illiterate end users, monsoon season in summer and snowy months in winter hindered the project progress in various ways.

- As a result of the delays encountered in the procurement of metallic parts and lessons learnt thereof, the PMU has initiated and completed bulk procurement of metallic parts for all 20 districts.


Tag: Effectiveness Integration Partnership Procurement Quality Assurance Results-Based Management Civil Societies and NGOs

13.

4.2.1 Adaptive management (continuation)

- Decision to engage DAHE to support project implementation in 16 districts. In 2013, Trashigang Dzongkhag was identified as the pilot site for the dissemination of cook stoves. The project signed a Memorandum of Understanding (MoU) with the Department of Adult and Higher Education (DAHE), Ministry of Education, to engage the Non-Formal Education (NFE) instructors in creating awareness and construction of cook stoves in Trashigang Dzongkhag. This option was chosen mainly due to the widespread presence of NFE instructors at the village level. In August 2013, the first batch of 46 (26 female and 20 male) NFE instructors were trained in Trashigang on the construction and maintenance of improved stoves. In continuation to Trashigang pilot project, DAHE also consented to implement cook stoves in 15 other districts. MoU to this effect was extended and by the time DAHE has supported the construction of around 8,900 improved cook stoves and fodder stoves


Tag: Challenges Effectiveness Efficiency Resource mobilization Local Governance Partnership Policies & Procedures Project and Programme management Country Government Institutional Strengthening National Institutions

14.

4.2.2 Partnership arrangements

During the initial period of SRBE from 2013 to 2014, a number of activities related to community awareness, mobilisation and livelihood improvement activities were implemented. Partnerships that were established and strengthened during this time and included: 

Civil Society Organizations /NGOs: The project has been successful in engaging and signing partnership agreements with the Bhutan Association of Women Entrepreneur (BAoWE) and Tarayana Foundation, both experienced NGOs working for the empowerment of women and improving lives of rural women and improving the lives of disadvantaged people living in abject poverty in rural Bhutan. It was a strategic partnership as one of the goals of this project is to mainstream gender and provide equal opportunities to men and women. While fulfilling the project mandates on the dissemination of fuel-efficient cook stoves, BAoWE and Tarayana Foundation have the necessary capacity/expertise to effectively mobilize and engage women in particular in the project sites. These two NGOs are supporting in implementing project activities in four remote districts.


Tag: Gender Equality Women's Empowerment Project and Programme management Civil Societies and NGOs Private Sector

15.

4.2.2 Partnership arrangements (continuation)

GEF Small Grants Program Useful lessons are shared between the project and GEF Small Grants Programme. The UNDP CO and project personnel are represented in the national sub-committee and national steering committee of the Small Grants Programme.


Tag: Effectiveness Global Environment Facility fund Communication Knowledge management Partnership Capacity Building Institutional Strengthening National Institutions Private Sector Vulnerable

16.

4.2.3 UNDP and Implementing Partner Performance

The project was implemented based on the UNDP National Implementation Modality (NIM). During the inception phase the Project Management Unit (PMU) was nominated based on the proposed organization structure foreseen in the Project Document. It consists of a project director, associate engineers and project assistant who manage in co-operation with UNDP’s CC portfolio manager the Project on a daily basis. The project received high level guidance and oversight from the Project Board (PB), which is chaired by the Secretary, Ministry of Economic Affairs, as the home ministry for the Lead Executing Agency, DRE. The PB is responsible for making management decisions on a consensus basis for the Project when guidance is required by the Project Manager, including approval of project revisions. Project assurance reviews are made by the PB at designated decision points during the running of a project, or as necessary when raised by the Project Manager. In this sense, the PB provided important input to the PMU in adaptive project management, and in general very positive feedback was provided throughout the evaluation mission regarding the level of cooperation between DRE and UNDP. 


Tag: Efficiency Implementation Modality Oversight Partnership Strategic Positioning Country Government

17.

4.2.4 Project finance

SRBE is financed with USD 1,703,000 through a GEF grant, USD 200,000 from UNDP (also grant), USD 300,00 from BTFEC and USD 30,000 from PEI, and RGOB . In addition to, the project has received co-financing contributions from different donors. Both, ADB and Swiss Development Cooperation/Helvetas have provided support for parallel ongoing program activities. ADB is for instance providing a grant and a loan for construction of 2,800 family biogas units under its “Rural Renewable Energy Development” project, and Helvetas is supporting 550 communityforestry activities under its “Participatory Forest Management” project. The Bhutan Trust Fund for Environmental Conservation (BTFEC) is providing USD 300,000 for distribution of approximately 1,500 cook stoves and 1,600 heating stoves, UNDP/UNEP Poverty - Environment Initiative (PEI) USD 50,000 for capacity building and training activities, and the remainder of USD 500,000 through the RGoB (support for baseline activities and involvement of national institutions – DRE, DAHE, SFD).For the establishment of the Briquette Plant in Thimphu by the Association of Bhutanese Wood Based Industries a private company called Bhutan Briquette Private Limited was set up. The SRBE project had contributed to only 50% of the cost for a set of machines. The rest 50% of equipment, construction of shed, land lease, transportation and other establishment costs are all borne by the private company. 


Tag: Efficiency Global Environment Facility fund Government Cost-sharing Operational Efficiency Donor

18.

4.2.5 Monitoring and evaluation

The design of the Project’s Monitoring and Evaluation (M&E) system at the entry has relied on the standard UNDP requirements, including annual Project Implementation Reviews (PIRs) and the project Mid-Term Evaluation completed on time. In addition, the progress of the project has been monitored on an ongoing basis by regular Project Board meetings. The PSC meetings appear as the main forum on which major Project decisions were made. Information from the PMU and stakeholders was provided for discussion at the PSC meetings. M&E consists basically of logging of indicators and outputs of the Project Results Framework (logframe). The indicators (43 in total) used in the logframe, however, are considered too numerous for the PMU to be all tracked, therefore the PMU concentrated on reviewing the project progress by its 15 outputs. Some of the indicators, as mentioned in chapter 4.1.1, were considered no longer relevant and could have been eliminated throughout the time, to reduce monitoring efforts and keeping the project focused on result oriented activities in the limited time available. 


Tag: Challenges Efficiency Monitoring and Evaluation Project and Programme management Quality Assurance Results-Based Management Risk Management Theory of Change

19.

4.3 Project Results

Assessment of SRBE achievements and shortcomings are provided in this section against the initial Project logframe (from ProDoc). Each outcome was evaluated against the individual criterion of:  Relevance – the extent to which the outcome is suited to local and national development priorities and organizational policies, including changes over time;  Effectiveness – the extent to which an objective was achieved or how likely it is to be achieved;  Efficiency – the extent to which results were delivered with the least costly resources possible.

The Project outcomes were rated in regard to each criteria above, based on the following scale:  6: Highly Satisfactory (HS): The project has no shortcomings in the achievement of its objectives;  5: Satisfactory (S): The project has minor shortcomings in the achievement of its objectives; 4: Moderately Satisfactory (MS): The project has moderate shortcomings in the achievement of its objectives; 3: Moderately Unsatisfactory (MU): The project has significant shortcomings in the achievement of its objectives;  2: Unsatisfactory (U) The project has major shortcomings in the achievement of its objectives;  1: Highly Unsatisfactory (HU): The project has severe shortcomings in the achievement of its objectives. The attainment of objectives and corresponding rating (including explanations to the ratings) are provided in the following section


Tag: Emission Reduction Energy Environment Policy Project and Programme management Inclusive economic growth

20.

4.3.1 Overall results (attainment of objectives) (continuation)

4.3.1.1 Global Environmental Benefits

Tables below summarize the GHG reduction estimates (using GEF guidelines) that were generated during SRBE (to its estimated terminal date of December 31, 2016). The Project has introduced efficient stoves in three categories, namely: cook stoves, fodder stoves and heating stoves. These stoves replace traditional less efficient stoves that used to be installed in many households in Bhutan. Due to the much higher efficiency of the new stoves, less fuel wood is being used up for the same cooking and heating benefits derived by the end-users of the stoves. The GHG annual emission reductions from efficient stoves are calculated as follows:GHG emission mitigation = amount of fuel wood saved annually by the efficient stove x the emission factor of fuel wood x the number of stoves introduced. The GHG mitigation calculations provided below are referring to CDM Methodology AMS-II.G “Energy efficiency measures in thermal applications of non-renewable biomass”  . Number of installed stoves were provided by DRE. 

Project Direct GHG Emission Mitigation At the end of the four-year period of the Project (extended by 1 year), the summary of the expected annual and cumulative fuel wood savings and GHG mitigation as a result of using the efficient stoves is shown below . The achievement of the direct cumulative GHG emission reduction of 173,711 tonnes CO2eq per year at EOP against the target of 196,500 tonnes per year at EOP is due to the lower amount of improved stoves being implemented through the project (14,179 versus 20,000 planned). 


Tag: Energy Environment Policy Project and Programme management Risk Management

21.

4.3.1 Overall results (attainment of objectives) (continuation)

4.3.1.1 Global Environmental Benefits (continuation)

Indirect GHG Emission Reductions – Top-down: In the top-down approach, it is assumed that 50% of the total households of the current 120,000 (i.e. 60,000) will benefit from improved cook stoves (BES 2015 type) after 10 years of the Project closure; whereas about 10,000 households will have a heating stove installed by that time. The corresponding GHG emission potential for the “market potential” is equivalent to about 2,516,300 tonnes CO2eq. The expected Causality Factor (CF) in line with the GEF Methodology is assumed to be 80% - since the GEF contribution has been a dominant intervention resulting to the introduction of the improved cook stove and heating technology, however, some of the fuelwood and GHG emission reductions are to be attributed to a baseline development.UNDP-GEF Project "Bhutan Sustainable Rural Biomass Energy" (PIMS# 4181) Terminal Evaluation Report 34 Total market potential of GHG emission savings: 2,516,300 tonnes CO2eq CF: 80% Indirect Top-down GHG emission reductions: 2,013,029 tonnes CO2eq


Tag: Forestry Emission Reduction Energy Environment Policy

22.

4.3.1.2 Overall Evaluation of the Project

In conclusion and by taking into account the observed shortcomings compared to the initial, and in some cases overly ambitious, goals, it is evident that the project has had a critical role in boosting the biomass energy technology development within the market conditions of Bhutan, which growth is likely to continue also after the project closure. With some critical issues, such as availability of a clear roadmap being in line with the RE Policy and to support BET development, availability of appropriate financing mechanisms to support private sector engagement, or continuous awareness and capacity building support, remaining at the project end the relevance of the Project remains at high level, but requires a clear commitment and strategy for follow-up respectively defining further replication activities. 

All in all, its results and contribution to the Project objective and its stated targets can be considered as satisfactory and have overall improved compared to the Mid-term Report.

Table 5 below summarizes the progress towards the end-of-project targets for the project objective and each outcome.


Tag: Emission Reduction Energy Policies & Procedures Advocacy Capacity Building Private Sector

23.

4.3.2 Relevance

The key criteria for assessing the project relevance have been defined in the UNDP guidance for terminal evaluations as follows: - the extent to which the activity is suited to local and national development priorities and organizational policies, including changes over time; - the extent to which the project is in line with the GEF Operational Programs or the strategic priorities under which the project was funded. Further it is noted that, retrospectively, the question of relevance often becomes a question as to whether the objectives of an intervention or its design are still appropriate given changed circumstances.

The project was approved for funding under the Climate Change Strategic Program 4: “Promoting sustainable energy production from biomass” of the Focal Area Strategies and Strategic Programming for GEF-4. As successful outcome for this strategic program “the adoption of modern and sustainable practices in biomass production, conversion and use as energy” with indicators such as “tons of CO2e avoided; the adoption of modern biomass conversion technologies, improved efficiency of biomass energy use, kWh of electricity and heat generated from biomass sources, and energy services produced on the basis of biomass” were listed, while also emphasizing the need to ensure “that biomass energy use is sustainable and does not, therefore, contribute to deforestation, reduced soil fertility, or increased GHG emissions beyond project boundaries.” The topic and the stated targets of the project are in accordance with this expected outcome and the principles outlined above have been fully respected in the project design.


Tag: Energy Environment Policy Coherence Relevance Policies & Procedures Country Government

24.

4.3.3 Country ownership

As already discussed in chapters 4.1.7 and 4.1.8, the project design is consistent with the national development plans (especially 10th and 11th Five-Year Plan) of the RGoB. The importance and benefits of the project and increased focus on energy efficiency for cooking and heating purposes within rural households, together with its benefits of improved indoor air quality and overall health were also unanimously emphasized in all stakeholder interviews conducted during the evaluation mission. As evidenced by the annual Project Implementation Reviews as well as by the minutes of the Project Board meetings, the country representatives both at the governmental level as well as CSOs/NGOs and few private sector entities have actively participated in the project implementation and decision making. The Project Board has been consulted on all important decisions and approval sought before the final decision. The composition of the Project Board can be considered as adequate by taking into account the scope of the project.

Overall, the main governmental stakeholders (most of them involved in the PB anyway) have expressed during individual meetings their full satisfaction and positive experiences made under the umbrella of the SRBE Project, and concluded at the presentation of the TE mission results that a continuation of national activities to promote the further distribution of ICS/HS and support mechanisms for engaging private sector in BET replication on the Bhutanese market shall be sought. 


Tag: Emission Reduction Energy Environment Policy Effectiveness Sustainability Joint UN Programme Ownership SDG Integration Civil Societies and NGOs Private Sector

25.

4.3.4 Mainstreaming (continuation)

The overall SRBE project focus is in line with the One Programme Monitoring and Evaluation (M&E) Framework: UNDAF Outcome 1: By 2018, sustainable and green economic growth that is equitable, inclusive, climate and disaster resilient and promotes poverty reduction, and employment opportunities particularly for vulnerable groups enhanced. “Output 1.1: Policies and studies for integrated natural resource management, climate change adaptation/ mitigation and poverty-environment nexus developed” (especially referring to the indicator “1.1.2 Number of households using fuel-efficient stoves for cooking/heating”), furthermore with “Output 1.2 National and local institutions and individuals are better prepared and able to respond to and reduce climate change induced and other disaster risks”, with “Output 1.5: Youth, women and other vulnerable groups have access to sustainable employment with a focus on cottage, small and micro enterprises in line with the RGOB’s commitment to a green economy” and with “Output 1.6: The rural poor and underemployed have access to alternative income generation opportunities.”

And also to some extent with other outcomes:  UNDAF Outcome 2: By 2018, increased and equitable access, utilization and quality of inclusive essential social services for all with a focus on sustaining the MDGs and addressing emerging challenges.  UNDAF Outcome 3: By 2018, communities and institutions strengthened at all levels to achieve enhanced gender equality, empowerment and protection of women and children. Gender mainstreaming—as a process and as a strategy—is used by the UN in Bhutan to promote gender equality and women’s empowerment across all outcome areas. The UN wants to ensure that gender mainstreaming is central to all its activities—from policy development, research, advocacy/dialogue, legislations, resource allocation, and planning, implementation and monitoring of programme, projects and activities


Tag: Effectiveness Gender Equality Gender Mainstreaming Women's Empowerment Jobs and Livelihoods Poverty Reduction Vulnerable Women and gilrs

26.

4.3.5 Sustainability

Sustainability is generally considered to be the likelihood of continued benefits after the project ends. Consequently, the assessment of sustainability considers the risks that are likely to affect the continuation of project outcomes. Considering sustainability within UNDP supported GEF financed projects, the GEF guidelines establish four areas for considering risks to sustainability, each of which should be separately evaluated and then rated as to the likelihood and extent that they will impede sustainability of the project outcomes. These risks include: 1) financial risks, 2) socio-economic risks, 3) institutional framework and governance risks; and 4) environmental risks.

The following rating is applied in evaluating the Project’s sustainability prospects:  4 = Likely (L): negligible risks to sustainability;  3 = Moderately Likely (ML): moderate risks to sustainability;  2 = Moderately Unlikely (MU): significant risks to sustainability; and 1 = Unlikely (U): severe risks to sustainability. The overall rating is equivalent to the lowest sustainability ranking score of the 4 dimensions. Referring to the dimensions of sustainability presented in the paragraphs below, the overall prospects of sustainability of the SRBE Project are considered to be moderately likely. 


Tag: Sustainability Resource mobilization Policies & Procedures Risk Management Financial Inclusion Inclusive economic growth Poverty Reduction Vulnerable

27.

4.3.5 Sustainability (continuation)

4.3.5.2 Socio-economic risks to sustainability

The Project was promoting the idea to make improved stoves available to rural population at a low cost. The stoves were delivered at a subsidy but villagers had to mobilize the remaining cost of the stoves and contribute their work force for free. Nevertheless, the rural poor with no or few means of earning cash would find it difficult to mobilize money to pay for the cost of the stoves. Although it was initially considered in the ProDoc to provide credit through the Bhutan Development Finance Corporation and micro-finance institutions being initiated through the RGoB’s support, the poor would still not be able to access credit because of the need for collateral as a pre-requisite for taking loans. In addition, most of the installations and capacity-building activities (in 16 districts) were conducted and supervised by NFED, DAHE, District Education Offices and the NFE Instructors. Their experience and involvement were all in all very positive and enriching, and it is also believed by NFED and DAHE were successful in achieving their targets. However, NFED/ DAHE considering the main mandate of NFED being focussed on vocational education and increasing adult literacy rate, further involvement of NFED/DAHE structures in the future is expected, if at all, at a much lower level. Their scale of involvement would need to be marginal, shorter time period or mostly focused on education and advocacy component, which provides a general risk on how to ensure on-site support and instructions to villagers on the implementation of efficient stoves.


Tag: Emission Reduction Effectiveness Efficiency Sustainability Risk Management Inclusive economic growth Poverty Reduction

28.

4.3.5 Sustainability (continuation)

4.3.5.4 Environmental risks to sustainability

There is no environmental risk to SRBE sustainability since the project is designed to reduce use of fuel wood in improved cook stoves which are more energy efficient and emit less compared to conventional three-stone cook stoves. This is consistent with RGoB’s strategy to limit the energy demand by adopting and using efficient technologies. In a future consideration of a governmental and/or private-sector supported initiative to distribute more efficient stoves and increasing the production of wood briquettes – thus removing the adverse environmental effects of sawmill disposal – the environmental risk associated to sustainability is expected to remain low. The sustainability of the Project in respect to environmental risks is therefore likely. 


Tag: Emission Reduction Environment Policy Risk Management

29.

4.3.6 Impact

The SRBE Project is providing significant impact on some of the main issues addressed in the design of this UNDP/GEF initiative: 

Reduction of fuelwood consumption for cooking and heating in rural households The SRBE Project has been very relevant for Bhutan since it is one of the countries with high per capita domestic fuel wood consumption (about 1.17 tonnes per person per year). With a total 70 percent of the population living in rural areas, fuel wood is still the main source of energy for cooking, heating and preparing fodder for animals. The inefficient fuel wood consumption has been contributing to high rate of deforestation and forest degradation, high levels of indoor air pollution and greenhouse gas (GHG) emissions. The Project is likely to achieve its revised reduction targets concerning fuelwood use and corresponding GHG emissions reduction. The impact of the activity on the overall project objective is therefore obvious, and potential to further expand the stoves in more households across the country will increase the impact. For the future programming of activities, two elements are suggested to be considered in regard to stove distribution and thus increasing their impact (refer also to recommendations provided at the end of this report): (1) Regardless of the technology of the stove, encouraging community participation in design, implementation, and monitoring of stoves will help increase ownership, ensuring sustainability. (2) The proper use and maintenance of stoves is important to maintaining the efficiency of the stoves. Thus, awareness programs need to be continued and enhanced in the future. 


Tag: Emission Reduction Natural Resouce management Impact Health Crises

30.

4.3.6 Impact (continuation)

Increasing awareness and education of rural villagers and potential users of BET applications in regard to their major benefits (fuelwood reduction, health impacts, gender-related roles and issues concerning cooking behaviour) Based on experiences and feedback received from rural villagers (selected statements from TE mission site visits, conducted between 23rd and 27th July, 20169 ), the impact of information and increased awareness is significant, because many observations made were very positive in respect to the use of improved cook stoves: 

o Households experienced better indoor air quality and consumed less fuelwood. o However, there were also households that mentioned the fuel wood demand to increase, but assumed by evaluation team mainly due to improper use of the stoves (e.g. doors opened). In several cases the fuel wood availability was not a concern for households. o Majority of households mentioned that they used the stoves basically every day for heating water and cooking fodder for cattle, also by using the 2-Pot Stoves. o Roughly one 1/3 of households mentioned they do not use the 2-Pot stove in the morning, as it is too slow to heat up while the three-stone fire is faster. o Households were generally satisfied with improved 2-Pot-Stoves. Even though they had to use their relative for construction and not the local technicians they were positive about the stove. o The general observation by households is that when they use small branches and twigs the fuel wood consumption in the ICS is higher, but when they use larger wood pieces or logs the efficiency is definitely higher in the ICS. o Another advantage of the ICS is that pots remain fairly clean compared to the traditional 3 stone stoves. o Stoves in most cases are used only occasionally for mass cooking and daily for cooking fodder.


Tag: Emission Reduction Impact Gender Equality Gender Mainstreaming Women's Empowerment Health Crises Knowledge management Awareness raising Capacity Building Vulnerable

31.

4.3.6 Impact (continuation)

Addressing the benefits of improved stoves and new emerging biomass energy technologies to mainly women involved in household kitchen daily routines In Bhutan, like in other South-Asian countries, women play a significant and dominant role within the household cooking sector. Generally, women do most of the cooking and, therefore, are disproportionately affected by household air pollution caused by the inefficient burning of solid biomass cooking fuels. They are also required to spend a significant amount of time and effort collecting the traditionally used biomass fuels, a physically draining task that can take up to 20 or more hours per week. The project has achieved significant participation of women in different activities, education and capacity building (NFE instructors and representatives from CBOs like Tarayana and BAoWE) and participation in the construction of improved stoves. Women were always given highest priority in this project and they played key role for successful implementation of stove project. The NFEIs and NGO technicians were provided with an opportunity to create awareness on improved stoves and render technical assistance to the end users. With the implementation of energy efficient stoves, the livelihood of the women is greatly enhanced, since they are less exposed to smoke and better sanitation are being maintained. Most importantly, women and children’s precious time is saved from collecting firewood, thereby ensuring their security and their time can now be used in other productive activities. Women are being empowered as they are engaged in creating awareness and decision making regarding BETs in rural gatherings and through such activities some village technicians are even aspiring to participate in local government elections. 


Tag: Emission Reduction Impact Gender Equality Women's Empowerment Vulnerable Women and gilrs Youth

Recommendations
1

5.2 Recommendations

With the GEF-funded SRBE project terminating on December 31, 2016, the following recommendations are provided in regard to corrective actions in the design of succeeding Project activities and proposals for future follow-up actions. Some of the recommendations are coming from the MTR being considered still valid (or not properly been addressed during the second part of project implementation), while others are added based on the final review and overall achievements of the Project made towards the termination date (in fact about 3 months remaining for last implementation activities).

Corrective actions for the design, implementation, monitoring and evaluation of the Project:

R1. Appropriate MRV System.

Appropriate MRV system to be included in all projects of such dimension. The Project shall be continuously monitoring its main indicators, in the case of SRBE referring to fuel consumption, heat generation and related GHG reduction data from the newly installed improved stoves. The developed Bioenergy Database provided on the project website (http://bioenergy.gov.bt/biomass/public/biomass/index/index) could have been extended in its functionality to link the number of installations with the indicators fuel consumption/reduction against baseline and corresponding GHG emissions. Efforts towards this direction were started during the final evaluation, but would have been worth starting already earlier. Additionally, the Project’s monitoring and continuous reporting of progress on the output/activity level (against planned versus achieved tasks provided in the logframe) shall be consistently reviewed and progress reported (in more detail). 

2

R2. Calculation of GHG emission reductions related to carbon capture from lower fuel wood use and afforestation/plantation programme.

Considering the combined effect of the community forestry for carbon sequestration and efficiency improvement, the overall post-project GHG emission reduction benefit from SRBE is envisaged to be better than originally estimated during the project design. Therefore, as mentioned already in MTR, the Project could have taken the opportunity to work with SFED and other stakeholders (e.g. NEC being responsible for providing the National Inventory and monitoring of the country’s GHG emissions) to factor in the contribution of the project in reducing greenhouse gas emissions from as a result of direct intervention made due to community forestry plantations. Studies have showed a high rate of CO2 absorption by the plants during the first 10 years. The use of standard methodologies of IPPC and UNFCCC to estimate GHG emission reductions from forestry activities (mainly afforestation/plantation programmes) is a recommendation for the stakeholders to further pursue, throughout the termination of the SRBE project or follow-up activities thereafter. Reference materials and calculations are provided within the IPCC’s ‘The Good Practice Guidance for Land-Use and Land Use Change and Forestry’10, and UNFCCC’s CDM ‘Methodology on Afforestation and reforestation of lands except wetlands’ (AR-ACM0003).

3

R3. Develop a “Lessons Learnt Report” for in-country dissemination.

As a follow-up or concluding activity, although not planned, it is recommended that the PMU is to compile and disseminate across the main project stakeholders “lessons learnt” from the Project to contribute to the project’s knowledge management, learning and information dissemination strategy. An analytical, thorough and, as required, also critical report summarizing experience and findings of the Project achievements and conclusions for future action would highly benefit the country activities and UNDP’s country engagement. As a part of that, an analysis of the efficient stoves implementation program, the briquetting plant’s operational achievements and results concerning capacity-building, training and awareness shall be considered in such compilation. Excerpts of such “Lessons learnt” review shall be considered for public dissemination. 

4

R4. Project to provide grounds for continued awareness programs on efficient stoves and their benefits.

While the project had a focus on awareness-raising and building capacity among villagers and household owners related to the benefits and use of improved cook stoves / heating stoves, the sustainability of the Project could be enhanced by foreseeing further (continuous) activities on a country-level to increase the awareness of the population and the user behaviour. The initially foreseen development of a “Knowledge and Learning Platform” should encompass also a public awareness strategy. PR elements to be foreseen are the use of different media, e.g. video program on national TV, radio, and social networks (WeChat/Facebook), mainly on following aspects: benefits of improved stoves, correct use of stoves, correct placement and installation of chimney, correct way of working with the regulating knobs and also on the benefits of keeping the stove door shut. An awareness program on TV/ Radio/ Social network such as WeChat and Facebook by DRE team would be good for the general users to understand the concepts and manage the stoves themselves. 

On another end, preparation and implementation of awareness education on benefits and use of ICS to schoolchildren in the project sites. This activity may be initiated as a workshop by the NFEIs in the areas to the local schools for a day. 

5

Proposals for future directions underlining main objectives

R5. Appropriate design facilities and national research for new BET

Likewise, the above mentioned limited testing methodologies available in the country, there are also limited appropriate design facilities and research centres conducting research and development of new cook stove designs. Mostly, these activities have been limited to DRE’s engagement and the availability of donor-led programs (e.g. UNDP/GEF, but previously Austrian government). Women’s involvement within design facilities and research institutions has generally been very limited. 

Regardless of the technology of the stove, encouraging community participation in design, implementation, and monitoring of stoves will help increase ownership, ensuring sustainability. Stove designs should be based on cooking needs. As the primary users of cook stoves, women have better knowledge about their needs and should be involved more systematically within the entire market system of ICS. Women’s perspectives could play a central role in product design, quality assurance, research, capacity building activities, and increasing access to finance. Thus, it is important to include household level research to assist with designs of improved stoves, making them more user friendly.

6

Proposals for future directions underlining main objectives

R6. Indoor air pollution and other health issues being further considered in the design of stoves programmes.

The ‘Multisectoral National Action Plan for the Prevention and Control of Non-Communicable Diseases’, which was approved by the RGoB in 201512 , has called for specific measures to promote health and reduce associated risks. Under action area 2.6., the Ministry of Health (MoH) being the leading agency together with DRE is to establish standards for indoor air quality promotion, monitoring, and identify communities with exposure to poor indoor air quality and educate communities. In order to link the issue of associated health risks with quality of improved cooking and heating stoves, testing of stoves in regard to efficiency gains (e.g. related to fuelwood use) but also indoor air pollution effects need to be performed and incorporated into national standards for improved stoves. 

7

Proposals for future directions underlining main objectives

R7. Enhance the support of capacity-building and skills programmes for beneficiaries.

The SRBE project has made big efforts to build the technical skills of rural villagers, also to mention especially the involvement of women, in the production of improved stoves, but it has not included or led to supporting women in establishing new ICS enterprises. To date, there has been a lack of long-term, women-focused training programs or incubation support available especially for women entrepreneurs, and provision of follow-up services and resources to encourage women to be a part of improved stoves and fuel value chains. In addition, women entrepreneurs have limited access to finance. Training and capacity building therefore remain essential components of any future successful BET programmes. Training can be provided to entrepreneurs, program staff, CBOs/NGOs involved in implementation, and end users (especially women) in technology, design, maintenance, and troubleshooting. In order to increase the dissemination of improved household energy technologies and acceptance by users, programmes must develop strategies to provide adequate user training and after-sales service. Such a user-training component should lay particular emphasis on women.

8

Proposals for future directions underlining main objectives

R8. Insurance schemes to help provide more safety and security in project implementation

The TE evaluators were informed that there were two cases of accidents and loss of a life directly related to the SRBE project. Without knowing the exact details of the cause and situations it is not possible for the TE team to comment on the situation. However, there are experiences from a LDCF-funded NAPA I Project implemented by UNDP and executed by the Department of Geology and Mines in the Lunana Lake mitigation. General Personal Accident (GPA) Insurance was purchased by the project for part-time workers and other officials going on the strenuous 9 days and 2-ways trek and working up at 4500 masl. While SRBE and other projects are not as risky and difficult, there are definitely risk factors involved. Thus, purchase of similar insurance schemes from the local insurance companies can be thought of and approved by the Project Boards/Steering Committees. However, it has to be noted that RGoB officials are considered to be part of the Civil Service Welfare Scheme and it may not be necessary to cover them under any insurance depending on the situation. 

9

R9. Quality Assurance Mechanism for improved stoves required.

While different organizations outside the country are able to carry out field testing of improved stoves, there is lack of a uniform national testing methodology. Actually, there is no institution responsible for testing stoves to determine if they actually perform as claimed by those promoting them. Claims of efficiency, pollution reduction, and durability are not actually tested by objective, outside groups. This is especially relevant for the newly introduced Bhutan EcoStove 2015 (BES 2015) and the Bhutan Multipurpose Stove 2015 (BMS-2015). In the long-run, DRE as the Renewable Energy and Energy Efficiency nodal agency could also initiate discussions on establishment of a code of conduct/standard for open or 3-stone fires across the country using fuel wood. This should be framed and implemented with relevant agencies such as Forestry Department, Local Governments and Ministry of Health. Once a national standard on limiting of open/ 3-stone fires are made then user would shift from the traditional open fires to improved stoves as fuel wood reduction is not their priority right now.

10

Proposals for future directions underlining main objectives

R11. Awareness and motivation about BET benefits to be effectively communicated and maintained.

Discussions with consumers typically indicated that they are not aware of the substantial benefits of improved stoves. They are often familiar with the fact that smoke removal leads to less burning of eyes and cleaner pots, but they rarely associate this with long-term health benefits. Bringing about behavioural change by increasing awareness of health issues and the benefits of improved cook stoves could help create a demand approach to improved stoves. Thus, the importance of raising awareness of long-term benefits, such as better health and avoided death, resulting from reduction in indoor air pollution due to the use of improved cook stoves should not be underestimated.

Villagers and rural households should be made aware of other benefits, such as time savings due to faster cooking, and the development of the rural economy and improvement in the living conditions of the villages. If people are aware of all benefits, the willingness-to-pay for and the acceptance level of improved cook stoves is likely to improve considerably. New stoves or products must be effective in removing indoor air pollution and reducing fuel consumption. In this regard, the monitoring of stoves in actual use by households for a significant period of time and having quality products available in the marketplace are essential before embarking on a publicity campaign. As mentioned a few paragraphs above, the possibilities to organise public information campaigns (e.g. TV, radio, and social media) are vast and should be considered in future programmes.

11

Proposals for future directions underlining main objectives

R12.Maintenance of stoves: there does not seem to be a common contact point for the beneficiaries to seek help or complain on technical issues or maintenance-related questions. It might be necessary for future stove suppliers to have a contact number of the concerned office so that people can ask for help. Similar to a help line, but not dedicated and specialized.

12

Proposals for future directions underlining main objectives

R12. Financial aspects: ensuring that the promotion of BET in Bhutan is not totally dependent on grant financing and financially sound business models are introduced.

Properly targeted subsidies are fundamental to the sustainability of cook stove programs. Subsidies often create a mind-set of subsidy dependence among households. Cash income obviously is scarce among many households using biomass fuels, and they are often willing to wait for subsidized stoves despite looking at the benefits they provide to them. As subsidized BET are yet limited in quantity, the spread of improved technology becomes self-limiting. In such a situation, subsidized supply of improved cook stoves results in a low market demand and could actually suppress entrepreneurship in the development of new markets. However, the continuation of national programmes with some limited level of subsidies provided could help to promote market development while taking into consideration the needs of various income groups. Therefore, in the case of Bhutan, it will be important in future to think about a proper subsidy scheme to promote improved cook stoves and combine market development with specific types of subsidies that would promote equity and access to private financing means (e.g. through specific schemes providing guarantees or collateral for private entrepreneurs, introducing micro-financing schemes for women and rural technicians, etc.). 

13

Proposals for future directions underlining main objectives

R13. Expedite the implementation of briquetting project with private sector saw mills to better understand the future demands of the market for briquettes.

Since the briquetting project has only lately picked up and expected the pilot plant to be operational towards the last 2-3 months of the project only, the benefits achieved on the local market need to be documented before EOP and a strategy be put in place on how private sector, which faces constraints on account of restrictions imposed by RGoB and the banks can be further supported and seek for business models which will be easy to replicate in the future. In addition, as expressed by the AWBI, it is also necessary for existing national environmental standards and procedures to be monitored in the saw mill industry to ensure environmentally unsafe dumping of wood waste.

1. Recommendation:

5.2 Recommendations

With the GEF-funded SRBE project terminating on December 31, 2016, the following recommendations are provided in regard to corrective actions in the design of succeeding Project activities and proposals for future follow-up actions. Some of the recommendations are coming from the MTR being considered still valid (or not properly been addressed during the second part of project implementation), while others are added based on the final review and overall achievements of the Project made towards the termination date (in fact about 3 months remaining for last implementation activities).

Corrective actions for the design, implementation, monitoring and evaluation of the Project:

R1. Appropriate MRV System.

Appropriate MRV system to be included in all projects of such dimension. The Project shall be continuously monitoring its main indicators, in the case of SRBE referring to fuel consumption, heat generation and related GHG reduction data from the newly installed improved stoves. The developed Bioenergy Database provided on the project website (http://bioenergy.gov.bt/biomass/public/biomass/index/index) could have been extended in its functionality to link the number of installations with the indicators fuel consumption/reduction against baseline and corresponding GHG emissions. Efforts towards this direction were started during the final evaluation, but would have been worth starting already earlier. Additionally, the Project’s monitoring and continuous reporting of progress on the output/activity level (against planned versus achieved tasks provided in the logframe) shall be consistently reviewed and progress reported (in more detail). 

Management Response: [Added: 2017/01/09] [Last Updated: 2021/02/01]

The management has taken up the need to strengthen the current Bioenergy Database provided online on the bioenergy website with the Department of Renewable Energy. It was discussed to make the database more robust incorporating GHG emission and fuelwood consumption reduction calculations. The strengthening of the current data base, quality monitoring and reporting system online and detailed activity reporting system will be incorporated in the similar programmes in future.  

Key Actions:

Key Action Responsible DueDate Status Comments Documents
?Strengthening of the current data base, quality monitoring and reporting system online and detailed activity reporting system will be incorporated in the similar programmes in future.
[Added: 2018/10/11]
CCM&E, UNDP, DRE, 2016/12 Completed Presented on the 9th (Final) Project Board Meeting of the SRBE Project on 30 December 2016. History
2. Recommendation:

R2. Calculation of GHG emission reductions related to carbon capture from lower fuel wood use and afforestation/plantation programme.

Considering the combined effect of the community forestry for carbon sequestration and efficiency improvement, the overall post-project GHG emission reduction benefit from SRBE is envisaged to be better than originally estimated during the project design. Therefore, as mentioned already in MTR, the Project could have taken the opportunity to work with SFED and other stakeholders (e.g. NEC being responsible for providing the National Inventory and monitoring of the country’s GHG emissions) to factor in the contribution of the project in reducing greenhouse gas emissions from as a result of direct intervention made due to community forestry plantations. Studies have showed a high rate of CO2 absorption by the plants during the first 10 years. The use of standard methodologies of IPPC and UNFCCC to estimate GHG emission reductions from forestry activities (mainly afforestation/plantation programmes) is a recommendation for the stakeholders to further pursue, throughout the termination of the SRBE project or follow-up activities thereafter. Reference materials and calculations are provided within the IPCC’s ‘The Good Practice Guidance for Land-Use and Land Use Change and Forestry’10, and UNFCCC’s CDM ‘Methodology on Afforestation and reforestation of lands except wetlands’ (AR-ACM0003).

Management Response: [Added: 2017/01/09] [Last Updated: 2021/02/01]

The accounting of GHG emission reduction through afforestation or plantation programme would further strengthen the GHG mitigation potential of the SRBE project intervention. However, due to difficulty in quantifying the potential in the absence of the baseline GHG emission from the community forestry, this is a real challenge. Therefore, the accounting of GHG emission reductions from afforestation programme is being dropped in this project period. The baseline information will have to be strengthened and once available, the same would be accounted in the similar projects in future.

Related, through the UNDP Low Emission Capacity Building (LECB) project with the National Environment Commission, UNDP is supporting the development of a GHG inventory data management system for data management and archival purposes, which will help ensure data accuracy, consistency and management going forward. For estimation of emissions, Bhutan will follow the guidance and methodology of the IPCC and UNFCCC. Capacity building and awareness raising for the GHG inventory estimation will be conducted through the Third National Communications and Biennial Update Report projects set to commence in early 2017, led by the National Environment Commission

Key Actions:

Key Action Responsible DueDate Status Comments Documents
The baseline information and GHG calculation methodology on the afforestation will be explored with the Department of Forest and Park Services of the Ministry of Agriculture and Forests in the future projects.
[Added: 2018/10/11] [Last Updated: 2018/11/12]
UNDP Bhutan CCM 2016/12 Completed Presented on the final board meeting of the SRBE Project on 30 December 2016. History
3. Recommendation:

R3. Develop a “Lessons Learnt Report” for in-country dissemination.

As a follow-up or concluding activity, although not planned, it is recommended that the PMU is to compile and disseminate across the main project stakeholders “lessons learnt” from the Project to contribute to the project’s knowledge management, learning and information dissemination strategy. An analytical, thorough and, as required, also critical report summarizing experience and findings of the Project achievements and conclusions for future action would highly benefit the country activities and UNDP’s country engagement. As a part of that, an analysis of the efficient stoves implementation program, the briquetting plant’s operational achievements and results concerning capacity-building, training and awareness shall be considered in such compilation. Excerpts of such “Lessons learnt” review shall be considered for public dissemination. 

Management Response: [Added: 2018/10/11] [Last Updated: 2021/02/01]

A lessons learnt report summarizing the projects key findings, experiences, achievement, knowledge management and the way forward for future replication shall be prepared in close collaboration with the project implementation unit of the Department of Renewable Energy. This recommendation was presented on the 9th Board meeting and was agreed on the preparation of the summary of the lessons learnt for future reference.

Key Actions:

Key Action Responsible DueDate Status Comments Documents
?Presented on the final board meeting of the SRBE Project on 30 December 2016.
[Added: 2018/10/11]
CCM&E, UNDP, DRE, SFED 2016/12 Completed
4. Recommendation:

R4. Project to provide grounds for continued awareness programs on efficient stoves and their benefits.

While the project had a focus on awareness-raising and building capacity among villagers and household owners related to the benefits and use of improved cook stoves / heating stoves, the sustainability of the Project could be enhanced by foreseeing further (continuous) activities on a country-level to increase the awareness of the population and the user behaviour. The initially foreseen development of a “Knowledge and Learning Platform” should encompass also a public awareness strategy. PR elements to be foreseen are the use of different media, e.g. video program on national TV, radio, and social networks (WeChat/Facebook), mainly on following aspects: benefits of improved stoves, correct use of stoves, correct placement and installation of chimney, correct way of working with the regulating knobs and also on the benefits of keeping the stove door shut. An awareness program on TV/ Radio/ Social network such as WeChat and Facebook by DRE team would be good for the general users to understand the concepts and manage the stoves themselves. 

On another end, preparation and implementation of awareness education on benefits and use of ICS to schoolchildren in the project sites. This activity may be initiated as a workshop by the NFEIs in the areas to the local schools for a day. 

Management Response: [Added: 2018/10/11] [Last Updated: 2021/02/01]

The major component of the project is the awareness creation among different levels of stakeholders including the direct beneficiaries. Efforts have been made to disseminate correct information about the project and the intended benefits that can be reaped through using these improved stoves. However, as per the recommendation, a day workshop has not been conducted during the project period. Therefore, an awareness workshop with school children and target beneficiaries will be conducted, and such components built in to future similar programmes.

Key Actions:

Key Action Responsible DueDate Status Comments Documents
Presented on the final board meeting of the SRBE Project on 30 December 2016. Mentioned that the awareness creation is a continuous process and need to be continued even after the closure of the project.
[Added: 2018/10/11]
UNDP Bhutan CCM team 2016/12 Completed
5. Recommendation:

Proposals for future directions underlining main objectives

R5. Appropriate design facilities and national research for new BET

Likewise, the above mentioned limited testing methodologies available in the country, there are also limited appropriate design facilities and research centres conducting research and development of new cook stove designs. Mostly, these activities have been limited to DRE’s engagement and the availability of donor-led programs (e.g. UNDP/GEF, but previously Austrian government). Women’s involvement within design facilities and research institutions has generally been very limited. 

Regardless of the technology of the stove, encouraging community participation in design, implementation, and monitoring of stoves will help increase ownership, ensuring sustainability. Stove designs should be based on cooking needs. As the primary users of cook stoves, women have better knowledge about their needs and should be involved more systematically within the entire market system of ICS. Women’s perspectives could play a central role in product design, quality assurance, research, capacity building activities, and increasing access to finance. Thus, it is important to include household level research to assist with designs of improved stoves, making them more user friendly.

Management Response: [Added: 2018/10/11] [Last Updated: 2021/02/01]

The design and testing facility is essential for continuous improvement of the technology through conduct of research and demonstration of stove technologies. In addition, such facility would enable the quality control and monitoring of the field activities, to provide timely intervention. Further, standardization of the technology is important to maintain efficiency, effectiveness, safety and GHG mitigation on the different products being developed based on market needs. Therefore, the Nodal Agency, the Department of Renewable Energy, is requested to look into developing an appropriate research facility to cater the growing Biomass Energy Technologies demand in the market. 

Key Actions:

Key Action Responsible DueDate Status Comments Documents
Presented on the final board meeting of the SRBE Project on 30 December 2016.
[Added: 2018/10/11]
UNDP Bhutan CCM team 2016/12 Completed
6. Recommendation:

Proposals for future directions underlining main objectives

R6. Indoor air pollution and other health issues being further considered in the design of stoves programmes.

The ‘Multisectoral National Action Plan for the Prevention and Control of Non-Communicable Diseases’, which was approved by the RGoB in 201512 , has called for specific measures to promote health and reduce associated risks. Under action area 2.6., the Ministry of Health (MoH) being the leading agency together with DRE is to establish standards for indoor air quality promotion, monitoring, and identify communities with exposure to poor indoor air quality and educate communities. In order to link the issue of associated health risks with quality of improved cooking and heating stoves, testing of stoves in regard to efficiency gains (e.g. related to fuelwood use) but also indoor air pollution effects need to be performed and incorporated into national standards for improved stoves. 

Management Response: [Added: 2018/10/11] [Last Updated: 2021/02/01]

Indoor air pollution is a major concern and one of the objective of this project is to reduce indoor air pollution and exposure of pollutants to human health. In this respect, the project undertook the design of the stoves to maximize removal of smoke from the kitchen. However, the project did not work with the Ministry of Health on the promotion of the stoves or the monitoring of their impact on exposure to indoor air pollution. Therefore, in the future similar projects, the Ministry of Health and the Department of Energy could jointly develop indoor air pollution standards and quality control measures.

Key Actions:

Key Action Responsible DueDate Status Comments Documents
- Presented on the final board meeting of the SRBE Project on 30 December 2016. - To be considered in future project design and implementation including the SAARC Development Fund regional project, once finalized.
[Added: 2018/10/11]
UNDP Bhutan CCM team 2017/06 Completed
7. Recommendation:

Proposals for future directions underlining main objectives

R7. Enhance the support of capacity-building and skills programmes for beneficiaries.

The SRBE project has made big efforts to build the technical skills of rural villagers, also to mention especially the involvement of women, in the production of improved stoves, but it has not included or led to supporting women in establishing new ICS enterprises. To date, there has been a lack of long-term, women-focused training programs or incubation support available especially for women entrepreneurs, and provision of follow-up services and resources to encourage women to be a part of improved stoves and fuel value chains. In addition, women entrepreneurs have limited access to finance. Training and capacity building therefore remain essential components of any future successful BET programmes. Training can be provided to entrepreneurs, program staff, CBOs/NGOs involved in implementation, and end users (especially women) in technology, design, maintenance, and troubleshooting. In order to increase the dissemination of improved household energy technologies and acceptance by users, programmes must develop strategies to provide adequate user training and after-sales service. Such a user-training component should lay particular emphasis on women.

Management Response: [Added: 2018/10/11] [Last Updated: 2021/02/01]

The capacity building of the beneficiaries and trained technicians in the communities were enhanced. However, during this project, specific support to help women become entrepreneurs in the the area of stoves design/construction/promotion/supply – or in the development of related entrepreneurial activities feasible through access to the improved stoves – were not undertaken in a dedicated way. This will therefore be explored as part of project design and implementation for future projects, including the SDF Energy project once final agreement is in place.

Key Actions:

Key Action Responsible DueDate Status Comments Documents
Presented on the final board meeting of the SRBE Project on 30 December 2016.
[Added: 2018/10/11]
UNDP Bhutan CCM team 2017/06 Completed
8. Recommendation:

Proposals for future directions underlining main objectives

R8. Insurance schemes to help provide more safety and security in project implementation

The TE evaluators were informed that there were two cases of accidents and loss of a life directly related to the SRBE project. Without knowing the exact details of the cause and situations it is not possible for the TE team to comment on the situation. However, there are experiences from a LDCF-funded NAPA I Project implemented by UNDP and executed by the Department of Geology and Mines in the Lunana Lake mitigation. General Personal Accident (GPA) Insurance was purchased by the project for part-time workers and other officials going on the strenuous 9 days and 2-ways trek and working up at 4500 masl. While SRBE and other projects are not as risky and difficult, there are definitely risk factors involved. Thus, purchase of similar insurance schemes from the local insurance companies can be thought of and approved by the Project Boards/Steering Committees. However, it has to be noted that RGoB officials are considered to be part of the Civil Service Welfare Scheme and it may not be necessary to cover them under any insurance depending on the situation. 

Management Response: [Added: 2018/10/11] [Last Updated: 2021/02/01]

Due to the nature of the Sustainable Rural Biomass Project, it was not initially considered as a high-risk project. However, during implementation including in remote areas, staff and contractors working on the project were required to navigate difficult terrain and topological conditions, with demonstrated risks. Therefore, exploring options for insurance schemes from local insurance companies as part of future project structures would be sensible and is recommended by Management. It should be noted that RGoB officials are considered to be part of the Civil Service Welfare Scheme, therefore such coverage may not be necessary, depending on who is hired under the project. An additional consideration would be to seek confirmation of insurance schemes from companies (such as construction companies) whose services are procured through future UNDP projects where risks are identified during project development. Such schemes could be indicated as ‘desirable’ in procurement TORs. This would need to be addressed on a case-by-case basis.

Key Actions:

Key Action Responsible DueDate Status Comments Documents
Presented on the 8th board meeting of the SRBE Project on 22 March 2016.
[Added: 2018/10/11]
UNDP Bhutan CCM team 2016/12 Completed
9. Recommendation:

R9. Quality Assurance Mechanism for improved stoves required.

While different organizations outside the country are able to carry out field testing of improved stoves, there is lack of a uniform national testing methodology. Actually, there is no institution responsible for testing stoves to determine if they actually perform as claimed by those promoting them. Claims of efficiency, pollution reduction, and durability are not actually tested by objective, outside groups. This is especially relevant for the newly introduced Bhutan EcoStove 2015 (BES 2015) and the Bhutan Multipurpose Stove 2015 (BMS-2015). In the long-run, DRE as the Renewable Energy and Energy Efficiency nodal agency could also initiate discussions on establishment of a code of conduct/standard for open or 3-stone fires across the country using fuel wood. This should be framed and implemented with relevant agencies such as Forestry Department, Local Governments and Ministry of Health. Once a national standard on limiting of open/ 3-stone fires are made then user would shift from the traditional open fires to improved stoves as fuel wood reduction is not their priority right now.

Management Response: [Added: 2021/01/27] [Last Updated: 2021/02/01]

Key Actions:

10. Recommendation:

Proposals for future directions underlining main objectives

R11. Awareness and motivation about BET benefits to be effectively communicated and maintained.

Discussions with consumers typically indicated that they are not aware of the substantial benefits of improved stoves. They are often familiar with the fact that smoke removal leads to less burning of eyes and cleaner pots, but they rarely associate this with long-term health benefits. Bringing about behavioural change by increasing awareness of health issues and the benefits of improved cook stoves could help create a demand approach to improved stoves. Thus, the importance of raising awareness of long-term benefits, such as better health and avoided death, resulting from reduction in indoor air pollution due to the use of improved cook stoves should not be underestimated.

Villagers and rural households should be made aware of other benefits, such as time savings due to faster cooking, and the development of the rural economy and improvement in the living conditions of the villages. If people are aware of all benefits, the willingness-to-pay for and the acceptance level of improved cook stoves is likely to improve considerably. New stoves or products must be effective in removing indoor air pollution and reducing fuel consumption. In this regard, the monitoring of stoves in actual use by households for a significant period of time and having quality products available in the marketplace are essential before embarking on a publicity campaign. As mentioned a few paragraphs above, the possibilities to organise public information campaigns (e.g. TV, radio, and social media) are vast and should be considered in future programmes.

Management Response: [Added: 2021/01/27] [Last Updated: 2021/02/01]

Key Actions:

11. Recommendation:

Proposals for future directions underlining main objectives

R12.Maintenance of stoves: there does not seem to be a common contact point for the beneficiaries to seek help or complain on technical issues or maintenance-related questions. It might be necessary for future stove suppliers to have a contact number of the concerned office so that people can ask for help. Similar to a help line, but not dedicated and specialized.

Management Response: [Added: 2021/01/27] [Last Updated: 2021/02/01]

Key Actions:

12. Recommendation:

Proposals for future directions underlining main objectives

R12. Financial aspects: ensuring that the promotion of BET in Bhutan is not totally dependent on grant financing and financially sound business models are introduced.

Properly targeted subsidies are fundamental to the sustainability of cook stove programs. Subsidies often create a mind-set of subsidy dependence among households. Cash income obviously is scarce among many households using biomass fuels, and they are often willing to wait for subsidized stoves despite looking at the benefits they provide to them. As subsidized BET are yet limited in quantity, the spread of improved technology becomes self-limiting. In such a situation, subsidized supply of improved cook stoves results in a low market demand and could actually suppress entrepreneurship in the development of new markets. However, the continuation of national programmes with some limited level of subsidies provided could help to promote market development while taking into consideration the needs of various income groups. Therefore, in the case of Bhutan, it will be important in future to think about a proper subsidy scheme to promote improved cook stoves and combine market development with specific types of subsidies that would promote equity and access to private financing means (e.g. through specific schemes providing guarantees or collateral for private entrepreneurs, introducing micro-financing schemes for women and rural technicians, etc.). 

Management Response: [Added: 2021/01/27] [Last Updated: 2021/02/01]

Key Actions:

13. Recommendation:

Proposals for future directions underlining main objectives

R13. Expedite the implementation of briquetting project with private sector saw mills to better understand the future demands of the market for briquettes.

Since the briquetting project has only lately picked up and expected the pilot plant to be operational towards the last 2-3 months of the project only, the benefits achieved on the local market need to be documented before EOP and a strategy be put in place on how private sector, which faces constraints on account of restrictions imposed by RGoB and the banks can be further supported and seek for business models which will be easy to replicate in the future. In addition, as expressed by the AWBI, it is also necessary for existing national environmental standards and procedures to be monitored in the saw mill industry to ensure environmentally unsafe dumping of wood waste.

Management Response: [Added: 2021/01/27] [Last Updated: 2021/02/01]

Key Actions:

Latest Evaluations

Contact us

1 UN Plaza
DC1-20th Floor
New York, NY 10017
Tel. +1 646 781 4200
Fax. +1 646 781 4213
erc.support@undp.org