- Evaluation Plan:
- 2019-2023, Bhutan
- Evaluation Type:
- Final Project
- Planned End Date:
- 10/2022
- Completion Date:
- 09/2022
- Status:
- Completed
- Management Response:
- No
- Evaluation Budget(US $):
- 20,000
Terminal Evaluation of Bhutan Sustainable Low Emission Urban Transport
| Document | Type | Language | Size | Status | Downloads |
|---|---|---|---|---|---|
 UNDP GEF EV TOR International Consultant April 2022.docx |
tor | English | 210.97 KB | Posted | 58 |
| UNDP GEF EV TOR National Consultant April 2022.docx |
tor | English | 210.31 KB | Posted | 55 |
 GEF_SLEUT_Project_TE_Report.pdf |
report | English | 5283.44 KB | Posted | 103 |
 Annex M GEF6 CCM Tracking Tool Bhutan_TE_23.08.22.xlsx |
related-document | English | 60.62 KB | Posted | 69 |
| Title | Terminal Evaluation of Bhutan Sustainable Low Emission Urban Transport | ||||||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| Atlas Project Number: | 00094488 | ||||||||||||||
| Evaluation Plan: | 2019-2023, Bhutan | ||||||||||||||
| Evaluation Type: | Final Project | ||||||||||||||
| Status: | Completed | ||||||||||||||
| Completion Date: | 09/2022 | ||||||||||||||
| Planned End Date: | 10/2022 | ||||||||||||||
| Management Response: | Yes | ||||||||||||||
| Focus Area: |
|
||||||||||||||
| Corporate Outcome and Output (UNDP Strategic Plan 2018-2021) |
|
||||||||||||||
| SDG Goal |
|
||||||||||||||
| SDG Target |
|
||||||||||||||
| Evaluation Budget(US $): | 20,000 | ||||||||||||||
| Source of Funding: | Project M&E Budget | ||||||||||||||
| Evaluation Expenditure(US $): | 23,000 | ||||||||||||||
| Joint Programme: | No | ||||||||||||||
| Joint Evaluation: | No | ||||||||||||||
| Evaluation Team members: |
|
||||||||||||||
| GEF Evaluation: | Yes
|
||||||||||||||
| Key Stakeholders: | Royal Government of Bhutan - GNHC, Ministry of Agriculture, Ministry of Economic Affairs, Ministry of Homw & Cultural Affairs, Ministry of Works & Human Settlement, MInistyr of Information & Communication, | ||||||||||||||
| Countries: | BHUTAN | ||||||||||||||
| Comments: | United Nations Sustainable Development Partnership Framework (UNSDPF) Outcome 4 - By 2023, Bhutan’s communities and its economy are more resilient to climate induced disasters and biodiversity loss as well as economic vulnerability |
| Lessons | |
|---|---|
| 1. | Significant lessons can be drawn from this project: The project was highly innovative through actions stipulated in its outcomes such as the introduction of new policy frameworks and incentives, strengthening national awareness and supporting the delivery of the ambitions on the ground. While these contributions have already been presented under effectiveness, other examples are summarised here. Design Phase The project has produced innovative financial instruments such as provision of 70% collateral free loan and extension of the vehicle loan repayment period from 5 to 7 years to enhanced the affordability of the EVs by reducing the upfront cost and the EMI burden. Due to high loan default in the transport sector, the financial institutes were not willing to give collateral-free loans to the EV Drivers. The government made special considerations by discussing with the RMA and making a special concession for the project. Most of the EV drivers who participated in the TE expressed that the EMI for 70% loan on their EV is already quite high as compared to the EMIs they were paying for the ICE vehicle. Therefore, without the increase in loan repayment period, the EV uptake would have hampered due to high EMI. With concerns around low repayment rates, the project also innovated through introduction of a mechanism of Group Guarantee Scheme working with the Bhutan Taxi Association. Such design are seen as an innovative ways of allowing beneficiaries to invest in EVs and overcome the barriers to entry all the while reducing the risks to the financial institutes. Implementation Phase As stated, COVID- 19 brought about challenges in the EV supply chain. The overall number and size of the EV orders were also not significant which only compounded the challenges faced by the Bhutanese dealers as it made it harder for manufacturers to prioritise in the face of global supply chain disruption. The Kolkata port in West Bengal has been customarily the port for transshipment of all goods including vehicles and heavy machineries imported into Bhutan. The officials from Bhutan’s Department of Revenue and Customs are permanently stationed there to facilitate the transshipment. However, during the heights of the pandemic, the already limited shipment line available with the feeder vessel in Kolkata port became more infrequent leading to significant delays in the of the EVs. Faced with such challenges, for the first time, through excellent action of PMU and a whole of government approach, imports of EVs were achieved through Chennai Port in Tamil Nadu, India. A key benefit of this route is that transshipments incur lower freight charges due to the Roll-On/Roll-off vessel service. Besides achieving faster delivery of the EVs, the new route also provided crucial cost saving in shipment which essentially helped the car dealers to absorb the losses due to increasing USD exchange rate which otherwise would have to be transferred to the beneficiaries. The amount of saving for the import of EV through Chennai port has been reported as anywhere from Nu. 100,000.00 (USD 1,282) to Nu. 80,000.00 (USD 1,025)[1] for each car. This was crucial considering the already high costs of the EVs to taxi owners.
The power of national ownership was also demonstrated in this project through addressing credit and cash constraints faced by the car dealers due to the slowing down of the economy. The NCGS was brought in as the government’s guarantee to provide 70% loan to the Dealers as they faced acute working capital shortage due to COVID-19. Other ecosystem actors also introduced new schemes to cater for the new business actors. This was the case of the Royal Insurance Corporation of Bhutan (RICB) which offered 50% discounts on insurance while some dealers offered to bear 50% of the comprehensive insurance payment of the car for the first year.
In order to strengthen the EV ecosystem, the project started working with an App developer to have real-time information on EV charging features such as location, operational status, etc. This app will help taxi drivers to plan their day and reduce time. The setting up EV desk at the Airport, and other border areas such as Paro, Phuentsholing, and Gelephug also shows that measures are being introduced to support development of the EV market. Access to free charging from the public charging stations have also helped to encourage EV uptake. These innovations are particularly important to boost uptake and market development considering the government’s commitment to move beyond consumer status of EVs to become a global player in the EV value chain. These efforts are already yielding fruit in some ways through the support received from the Government of Japan in strengthening the national government fleet and charging stations under the “Leveraging Nationally Determined Contributions (NDCs) to achieve net-zero emissions and climate-resilient development, in response to the climate emergency” project.
The dealers and the EV drivers have a social media group (technical group) where they can immediately refer any issues with the car to the Dealer and resolve it right away. In case their staff is not able to do it, they arrange a conference call with the manufacturer and get the issues resolved. The creation of an EV Facebook page and the use of WeChat/Whatsapp chat groups between PMU and beneficiaries created an atmosphere of collegiality, mutual learning, and self-help. Respondents stated that these approaches helped them to address their concerns in a timely manner while learning and sharing experiences with peers. PMU also uses a demonstration EV to show case the technology to interested individuals by providing test drives. This is crucial to overcome some of the challenges and barriers to adoption. The PMU in collaboration with the MoLHR co-opted the expertise of a Master Trainer from Thailand, who was working with the TTIs, to attend to the issues of charging station as the project could not bring their external experts due to Covid-19 restrictions. It was reported that the Master Trainer was able to resolve most of the issues that were plaguing the changing stations. Such instances emphasizes the agility of the project to adapt to the changing scenarios and challenges.
[1] Figure reported by the Car Dealer The ownership and leadership of local and national government authorities – a whole of government approach The project demonstrates the importance of local and national government/authorities buy in and ownership in the delivery of such high innovation, high risk and high value project. The RGoB was able to use all levers of government in developing the policy framework and facilitating the financial framework required for delivery. The strategic positioning of the PMU within the PMO sent a strong message about the commitment and ownership of government. The government was pragmatic and demonstrated strong willingness to engage with a wide range of stakeholders to ensure the success of the project. The importance of ownership and government commitment was demonstrated in overcoming the procurement and supply chain issues brought about by the COVID-19 pandemic working across government. As most respondents acknowledged, this project would have failed without this intervention. The decision by H.E Prime Minister to take part in this evaluation is further evidence of the high-level support for the success of this project which is fully aligned with the government’s vision and ambition for EV expansion in the country and for becoming an active player in the global EV supply chain. Local authorities were involved throughout which ensured that local charging stations and related infrastructure was installed. The continuous operation and security of the stations depends on strong local support from authorities.
The private sector can play a key role in achieving the country’s climate change goals The private sector injected the match funding while car dealers mobilised resources and engaged in CSR activities supporting the delivery of the project. Even though the loan repayment rate in the taxi sector is low, banks provided much needed support with the backing of government and national guarantee scheme. While the project can be seen as a pilot, the support of private sector provides the framework for scalability. There is demand for this project to be expanded to other target sectors such as government fleet and public transport amongst others. The GEF project provided the start-up capital for this initiative working together with government. In the long run, private sector can play a more substantial role and hence contribute towards the achievement of government development and climate goals. One of the crucial co-benefits from this action would be a reduction in the country’s trade deficit as Bhutan spent around Nu. 8.34 billion (USD 106.92 million) annually on the import of fossil fuels. Based on an analysis carried out by the evaluators, the savings in terms of fuel import by the current EV taxis up until June 2022 works out to over Nu. 9 million (USD 115,384.62). Alternatively, the amount spent for charging these EVs taxis for the same duration works out to Nu. 213,165 (USD 2,732.89) which translates to 98% cost saving in fuel[3].
Awareness raising and capacity strengthening crucial to strengthen the adoption of new technologies and innovations The project was able to bring innovation by working on the factors facilitating or constraining the adoption of EVs. The project carried out awareness raising and advocacy actions to enhance interest and adoption of EVs. A combination of approaches was adopted including the use of social media and messaging applications. By adopting a bottom-up approach, the project ensured that taxi drivers were regularly informed and updated on progress. The social media tools applied ensured that the project listened and responded in real time to concerns while providing necessary assurances to taxi drivers in case of problems. Developing the educational and cognitive environment of the project proved successful through training, training of trainers and work with TTIs to develop an EV curriculum.
Multipronged approach to delivery The project adopted an evidence-based approach to decision making. For instance, implementation of feasibility studies, production of technical specificities drawing on world class experiences and research to inform the design and implementation of charging stations, identification of EV specifications amongst others. Obviously as an innovative project, initial focus included working with international consultants, but as progress was achieved in project delivery, the development of national capacity was also prioritised. Adopting a multi-pronged approach to supporting the roll out of innovations enhanced adoption with consequent positive efforts for sustainability.
An ecosystem approach is required This project also highlights the need for an ecosystems approach to the promotion of EVs. The key is not only to focus on the importation of cars, but also strengthen the policy environment and local infrastructure for its expansion. Developing local capacity for operations and maintenance while building trust with the private sector and banks is critical. Further, it is also important to constantly ensure that the citizens understand the merit of the action to bring them along through adoption of EVs or green e-mobility. This obviously requires a long-term perspective as opposed to a projectized approach. The development of the EV Roadmap; revision of building codes to integrate home charging facilities; facilitating access to credit etc. are all aspects of the country’s institutional environment for effective EV deployment. Signaling and modelling also emerged as a key factor with the PM and UNDP staff using EVs themselves to promote the new technology. The issuance of directives regarding the renewable of government fleet with EVs is another relevant piece but has to be monitored to ensure compliance.
UNDP comparative advantage and convening power GEF catalytic funding has enabled the government to deliver on this project. The Project respondents were unanimous that the introduction of subsidies to taxi drivers was the key tipping factor considering the prices of EVs. Considering the sustainability of this project is moderately likely, the benefits of this funding are likely to be felt for decades. The effectiveness of this support required an organisation like UNDP with the global comparative advantage to deliver. It showed great ability to work with government and drew from its global network of experts and knowledge base to support the PMU and government. Its neutrality and convening partner ensured that it was listened to by government and other national stakeholders.
Awareness and Training on Inclusive Public Transport Service The inclusion of PWD in the project design and implementation could have led to a more inclusive development of public transport sector. However, the project design did not undertake any deliberate attempts to include the needs of the PWDs in the project design. However, based on the findings of the TE, at the time of finalization of this report, a one day DET was organized for the EV drivers by the PMU in coordination with the UNDP. Such initiatives highlight the adaptive and openness of the project to incorporate activities that will ultimately benefit the larger stakeholders. In order to ensure a more inclusive development of public services, the needs and challenges of PWDs should be incorporated in the future project design and implementation right from the planning stage. |
| Findings | |
| 1. | The main findings from this TE are presented below. Relevance: the objective and design of the project aligns strongly with the priorities of Bhutan. The nation has a high reliance on the importation of fossil fuel to drive its transport sector. The RGoB is committed to achieve carbon neutrality and to reduce importation of fossil fuel. The SLEUTS project supports the nation towards meeting these commitments. The relevance of the project is further strengthened by the fact that the country is powered by 100% clean renewable hydroelectricity which makes the transportation sector one of the low hanging fruits for cutting down its emission. Effectiveness: irrespective of the COVID-19 pandemic, the SLEUTS project has made significant progress towards the realization of its objective and outcomes. At TE, 75% of the outcome 1 indicator targets were achieved while the rate of achievement for those of outcomes 2 and 3 were at 66% each. The remaining targets are however on-track to be achieved by the end of the project period. The project intended to address three key challenges and barriers linked to the promotion of EVs and their uptake in Bhutan: the lack of an enabling policy and regulatory environment for EVs, misperceptions and low level of technical knowledge amongst market/sector stakeholders, and high upfront costs and limited infrastructure for deployment. The barrier pertaining to the lack of regulatory framework and enabling policies for LEVs in Bhutan was addressed by the project through the adoption of a national target for LEV, establishment of EV enabling policies such as zero import duty and sales tax on EVs and increase in the loan for the purchase of EVs from 30% to 70% and a suite of other policy instruments. Based on the stated expected results, the TE team concludes that a favourable policy and regulatory environment for LEVs has been established to address the barriers identified. The high-upfront cost barrier for LEVs adoption was addressed through the provision of subsidies for the purchase of EVs demonstrating that specific financial support interventions can address financial barriers to EV adoption. It is important to note that this was piloted amongst the qualifying taxi drivers and as such this barrier will remain beyond the EOP in the absence of a follow-on subsidy mechanism. The low level of stakeholders’ technical knowledge, misperceptions and negative attitudes against EV was addressed through capacity building initiatives delivered faced to face and online due to the Covid 19 pandemic reaching 924 of the target 1000 stakeholders. While respondents in the evaluation reported improved knowledge and awareness, a planned end of project perception survey is expected to provide evidence of the effective level of improvement in knowledge and awareness achieved compared with the baseline[1]. This survey will provide evidence on the level of removal of constraints and the outstanding issues that could be addressed beyond the initial project period. At the level of the specific objective, the Greenhouse gas emissions reductions target will not be achieved by the end of the project due to most EVs only delivered towards the end of 2021 and 2022 given the impact of the Covid 19 pandemic on global supply chains. The two other indicators linked to numbers of citizens using EVs for transport and the mobilization of financial resources for EV promotion were significantly overachieved (217% and 213% respectively). The overall achievement of the project specific objective is moderately satisfactory. Efficiency: the project was efficiently delivered in a satisfactory manner. The project management team was able to leverage additional resources from the public and private sector. In the wake of COVID-19, implementation of activities virtually and the one-year no-cost extension were introduced as adaptive management measures. Sustainability: the sustainability of the project is moderately likely. Financial, environmental, socio-economic and institutional framework and governance risks were identified to potentially hamper the sustainability of the project outcomes. Environmental and social safeguards: environmental and social issues of the project were identified through screening conducted during the design phase of the project. Mitigative measures were identified for each identified risk. A well-established AGM was part of the project and phone numbers of the PMU were widely circulated to project stakeholders who were able to channel their grievances to the PMU through telephonic conversations. Gender: gender mainstreaming was ensured in the project from the design phase. A gender action plan was elaborated to address the gender gaps identified. Project implementation and reporting paid particular attention to gender reflected by participation of women in activities and the reporting of gender disaggregated statistics. Disability: the project design and implementation did not have any deliberate attempts to include the needs of PWD. This is seen as a missed opportunity. Impact: while it takes time for impact to manifest, competitive fares offered by EV taxi drivers and higher earnings made by EV taxi drivers are some of the indicators of the project’s positive impact.
[1]Based on the project team/prodoc, activity 2.1.4, this will be an analogous awareness/perception survey conducted following the same survey techniques and methodology as the baseline survey. The purpose is to monitor, assess and report on the achieved results and their effectiveness in terms of changes in perception and attitudes as well as identify remaining capacity and knowledge gaps were additional awareness/capacity needs could be provided post project. |
| Recommendations | |
|---|---|
| 1 | Evaluation recommendation 1. Inadequate capacities in the country for operation and maintenance of EVs and associated infrastructures such as charging stations |
| 2 | Evaluation recommendation 2. Sustainability – clarifying institutional ownership and agency in charge after project ends |