Promoting Access to Clean Energy Services in St. Vincent and the Grenadines (PACES )

Report Cover Image
Evaluation Plan:
2017-2021, Barbados
Evaluation Type:
Final Project
Planned End Date:
12/2018
Completion Date:
04/2019
Status:
Completed
Management Response:
No
Evaluation Budget(US $):
30,000

Share

Document Type Language Size Status Downloads
Download document TTO PIMS 5146-TE-TOR-PACES Clean.pdf tor English 469.04 KB Posted 464
Download document Final Evaluation Report PACES vOH (clean) (005).pdf report English 1085.88 KB Posted 568
Title Promoting Access to Clean Energy Services in St. Vincent and the Grenadines (PACES )
Atlas Project Number: 00080915
Evaluation Plan: 2017-2021, Barbados
Evaluation Type: Final Project
Status: Completed
Completion Date: 04/2019
Planned End Date: 12/2018
Management Response: Yes
Focus Area:
  • 1. Environment & Sustainable Development
  • 2. Others
Corporate Outcome and Output (UNDP Strategic Plan 2014-2017)
  • 1. Output 1.4. Scaled up action on climate change adaptation and mitigation across sectors which is funded and implemented
SDG Target
  • 7.1 By 2030, ensure universal access to affordable, reliable and modern energy services
  • 7.2 By 2030, increase substantially the share of renewable energy in the global energy mix
  • 7.b By 2030, expand infrastructure and upgrade technology for supplying modern and sustainable energy services for all in developing countries, in particular least developed countries, small island developing States and landlocked developing countries, in accordance with their respective programmes of support
Evaluation Budget(US $): 30,000
Source of Funding: Project
Evaluation Expenditure(US $): 14,126
Joint Programme: No
Joint Evaluation: No
Evaluation Team members:
Name Title Email Nationality
Oscar Huertas Díaz Evaluator oscarhuertas77@gmail.com COLOMBIA
GEF Evaluation: Yes
GEF Project Title: Promoting Access to Clean Energy Services
Evaluation Type: Terminal Evaluation
Focal Area: Climate Change
Project Type: MSP
GEF Phase: GEF-5
GEF Project ID: 5297
PIMS Number: 5146
Key Stakeholders: GEF
Countries: BARBADOS
Lessons
1.

Every design must include from the beginning a clear theory of change that allows identifying the chain of results from the inputs, through the activities, products and expected results.


2.

Projects should have a holistic approach combining short-term solutions with long term impacts. Nevertheless, when aiming for institutional changes and policy making, the projects need to clearly define outputs and results that fall under its control.


3.

Regarding the gender and human rights approach and SDG contributions, the evaluation highlights that the project aimed at male-dominated sectors, and focused on changes at the institutional level, making it difficult to have a robust gender strategy. However, PACES made efforts to strengthen the integration of the gender perspective in its implementation, as it strived for the equal participation of men and women in project activities, training was also carried out for all people, documents and project reports were also prepared to respond to the role of women in the project and disaggregating data by gender. It is important to note that women are particularly affected by the lack of access to energy, especially in rural areas.


4.

For future NIM projects that require an implementing agency, it is important to analyze the administrative challenges and procurement delays, as well as mitigation measures and alternatives for an agile execution. Given the novelty of the RE topic in SVG it is recommended to set realistic timelines taking into account the scarcity of local suppliers. Together with other UNDP Cos in the  Caribbean, it would be beneficial to build a list of international suppliers, consultants, experts, etc.


Findings
1.

Project Design

Analysis of the logical framework (AML) and the results framework (logic and project strategy, indicators)

1.1. The evaluation has been able to find that the design of the project is robust by clearly defining the results, effects, and outputs, as well as measurement indicators, baselines, and targets. The design is based on thorough research, technical and scientific developments in the development of renewable energies.


Tag: Biodiversity Renewable energy Relevance Global Environment Facility fund Women's Empowerment Programme/Project Design Results-Based Management Capacity Building

2.

Assumptions and risks

1.7. The project did a risk analysis from the onset and established external and internal risks. The following are the external risks, but the evaluation found there were no mitigation measures:

•Continued or sustained levels of energy subsidies to fossil fuels and electricity prices. While the GoSVGi s trying to reduce these subsidies, political pressure may result in the GoSVG being unable to reduce subsidies to the extent that the economics of renewable energy projects may not be attractive; ·

•Inability to build the necessary institutional and local capacity during the Project period due to lack of qualified personnel; ·

•Failure to secure co-financing from potential project partners. This may result from the unforeseen diversion of government budgets and resources for issues with more pressing priorities such as disaster relief and large infrastructure projects that would generate significant economic benefits.


Tag: Energy Local Governance Human and Financial resources Programme/Project Design Results-Based Management Risk Management Country Government Capacity Building

3.

Lessons from other relevant projects (e.g., the same area of interest) incorporated into the project design

1.8. As mentioned above, the project design is based on previous research on the GHG emissions, REin the Caribbean, as well as a dependency on global oil price volatility. There is no clear evidence of past national experiences that fed into the PACES project design. Beyond rigorous studies from CARICOM and other key institutions, the PACES project took stock upon 2004, GEF supported the Caribbean Renewable Energy Development Programme (CREDP) Project that was aimed at dismantling identified barriers (in the areas of policy, capacity, information, awareness and finance) to the increased use of RE in the region. CREDP was implemented by UNDP and executed by the Energy Programme within the CARICOM Secretariat with co-financing from GIZ. GEF support for CREDP was concluded in 2009 with only GIZ support continuing until 2012. CREDP did strengthen capacity and raised awareness of RE issues, laying a useful foundation for further developments in RE and EE in CARICOM countries.


Tag: Energy Donor relations Programme/Project Design Project and Programme management Donor UNDP management Capacity Building Multilateral Partners

4.

Stakeholders participation

1.11. In general, the project managed to bring together Government, Private Enterprise and foreign aid.This combination of actors is a case study worthy of analysis because participation remained active during the implementation of the project, including organizations that traditionally did not use RE like VINLEC. During the project implementation, partition also has been one of the project landmarks; for example, for the revision of the electricity supply act, there have been different stakeholders’ consultations including technical and policy level representatives from the Attorney Generals Chambers, Ministries of Health, Customs, Energy Unit, VINLEC, NGOs, Private sector, Chamber Commerce, etc.


Tag: Partnership Country Government Civil Societies and NGOs Private Sector

5.

Replication approach

1.12. In the current circumstances, the possibility of replicating the experience of the project to other products, organizations or even countries, is highly possible. The project has made efforts to register and share lessons and knowledge with the material generated by the training, the reports, and assessments. PACES has been sure to incorporate in all its contractual arrangements a need for knowledge transfer and training to those in direct receipt of the service. All training participants are obliged to share the learnings with colleagues in a trainer-of-trainers approach to replicate the capacities.


Tag: Communication Knowledge management Capacity Building Data and Statistics

6.

The comparative advantage of UNDP

1.17. UNDP is an institution with experience and technical capacity in project management, and in initiatives that seek the conservation of the environment and biodiversity. The Project was implemented by The Governmentand was closely followed by the programme officer from Barbados CO,and had the administrative support of that office. Likewise, the knowledge of the GEF and its administrative processes was an added value for the execution of the project, operational management.


Tag: Global Environment Facility fund Monitoring and Evaluation Operational Efficiency Project and Programme management Coordination Technical Support

7.

Project Implementation

Adaptive management

1.20. The adaptive management corresponds to the level of flexibility that the project had to attend to the changing dynamics and the supervening needs. These are the adjustment mechanisms to improve the execution of the project, after an agreement between the UNDP, GEF and the institutionsof St. Vincent and the Grenadines.


Tag: Change Management Oversight Ownership Project and Programme management Country Government

8.

Monitoring and Evaluation: input and execution design.

1.24. The project had different tools for monitoring and evaluation: logical framework matrix with results indicators, annual project implementation reports (PIR), quarterly narrative reports, tracking tools (GEF), follow-up with the UNDP Atlas administrative tool, and follow-up meetings. The project document includes a mid-term evaluation, but still, there was no budget for such evaluation. This is a missed opportunity because mid-term evaluations are formative, and can provide insights, and recommendations to improve implementation if needed.

List of PACES indicators

Objective

Reduction in GHG emissions from fossil-fired power generation and fossil fuel consumption for road transport through the exploitation of SVG’s renewable energy resources for power generation

Cumulative direct and direct post-project CO2 emission reductions resulting from the RE technical assistance and investments by end-of-project (EOP), ktons CO2.

% share of RE in the power generation mix of SVG by EOP

Outcome 1

The Energy Unit with the support of VINLEC evolves into a facilitation center to support private sector RE investment development, enable regulators to determine fair, flexible tariff structures, bring confidence to private RE investors, and increase the number of approved RE projects

Number of on-grid RETs approved based on studies of improved RE policy and tariffs and RE grid integration

Number of RE development project proponents that were assisted by staff from the Energy Unit and VINLEC in the technical design of their projects


Tag: Renewable energy Monitoring and Evaluation Trade and Development Awareness raising Capacity Building Private Sector

9.

Project financing

1.35. According to the data obtained, of the total GEF budget, to 2017, $ 877,926.76 had been executed, corresponding to 89.62% of the total. By the end of 2018, it is planned to complete 100% between executed expenses and commitments made at the evaluation date.


Tag: Global Environment Facility fund Resource mobilization Human and Financial resources Procurement Programme/Project Design Country Government Private Sector

10.

Coordination of the implementation and execution of UNDP and the partner for implementation and operational issues

1.41. The initial modality of the project is National Implementation (NIM). Ministry of National Security was acting as an implementer, as per the NIM project management implementation guidelines agreed by UNDP and the Government of St. Vincent and the Grenadines. The Project is co-financed with funding from the GEF and UNDP acts as the GEF Executing Agency. Components 1, 2, and 3 of the Project were implemented by the Energy Unit of MoNS, who assumed the overall responsibility for the achievement of Project results as the Implementing Partner (GEF Local Executing Agency). The organization structure of PACES is depicted in figure 1.


Tag: Implementation Modality Country Government Coordination

11.

Project Results (effectiveness)

1.43. According to all information sources, the expected results and the goals were achieved despite slight delays of the project in the initial months and external contingencies that were resolved during implementation. Figure 2 below shows the scale of the general ratings assigned to the project of the annual implementation reports (PIR). As you can see, the project radically improved its ratings from moderately unsatisfactory, to satisfactory.


Tag: Renewable energy Effectiveness Knowledge management Policies & Procedures Awareness raising Capacity Building Technical Support Civil Societies and NGOs Private Sector

12.

Relevance

1.64. The Government of SVG ratified the UN Framework Convention on Climate Change (UNFCCC) on December 2, 1996, and the Kyoto Protocol on December 31, 2004. With this being done it indicates the country's commitment to climate change and the reduction of carbon emissions. The PACES Project has contributed significantly to the country's goals of carbon reduction through numerous installations and the adoption of Renewable Energy Technologies (RETs) as well as through the establishment of the necessary policy framework. Similar to the UNFCCCs main goal to prevent the rise of greenhouse gas concentrations, the PACES Project has an overall goal to reduce GHG emissions from fossil fuel-based power generation by exploiting RE resources for electricity generation in SVG. As with the UNFCCC and many other international climate change initiatives, the project has sort of fostering awareness on the issue of the need for reduction of GHG emissions and has provided suitable alternatives and solutions to doing such. The Project's reference to several sustainable development goals references its close relationship to the objectives of many climate change organizations.


Tag: Climate Change Adaptation Climate finance Emission Reduction Energy Relevance Gender Equality Ownership Theory of Change Capacity Building SDG Integration SDG monitoring and reporting Private Sector

13.

Efficiency

1.76. The evaluation rates the efficiency of the PACES project as highly satisfactory given the number of outputs delivered, the scope of the project, and the resources available (both human and financial). During the consultation phase, no complaints were received about the project execution and progress during implementation. The project had to extend its execution period due to implementation delays already mentioned. According to all interviewees, a key success factor for the implementation was the selection of the project team that was key to implement the project activities, supervise consultants, and do a follow-up.


Tag: Efficiency Communication Human and Financial resources Implementation Modality Policies & Procedures Procurement Project and Programme management

14.

Sustainability

1.81. The evaluation found that the sustainability of the project is a point of attention because it is moderately unlikely to guarantee financial resources. On the other hand, socio-politicians support is likely, as well as the institutional framework and governance(likely).


Tag: Energy Sustainability Human and Financial resources

15.

Impact

1.86. Given the moment in time this evaluation was conducted, the evaluation methodology implemented, and the time it takes to assess impact, this evaluation cannot make a full impact assessment.

1.87. The analysis of the impacts refers to the evaluation of the changes from the implementation of the project. In this regard, the evaluation has been able to conclude that impact is significant given that the project has generated impacts in two aspects: at the level of the institutional framework, and the level of demonstrative projects.

 


Tag: Impact Operational Efficiency Results-Based Management Private Sector

16.

COMPONENT 2: CLEAN ENERGY CAPACITY DEVELOPMENT.

1.53. The project contributed to institutional strengthening by warranting that five managers in VINLEC and Energy Unit were dedicated to promoting of RE investments, and also that nine technical staff members in VINLEC, the Energy Unit, and in the private sector were able to provide technical oversight on RE solutions. VINLEC has five male managers who provide technical oversight to RE projects, and the Energy Unit has four technical officers of which 3 are men, and 1 is a woman.

1.54. Regarding the number of tradespersons who have local certification to construct, assemble, operate and maintain RE technologies, two technicians received IMI certification on maintenance and repairs for Electric Vehicles. Both participants were men, 13 locals received NABCEP Associate Solar PV Installers Certification. This group comprised of 12 men and one woman and three individuals who completed the NABCEP Certification were supported to receive Residential and Commercial Photovoltaic Systems Certification at Solar Energy International. This included women and one woman.

1.55. The project also aimed at civil society by doing community outreach in different ways; launching the website for the Energy Unit (www.energyunit.gov.vc), visiting rural areas and thought them about energy efficiency and RE (4 VILLAGES), producing brochures on RE (disseminated during exhibitions), visited schools(5schools visited). The project also delivered a documentary showcasing the current state of the energy sector (Hydro stations, solar PV farms), and it is uploaded onto Youtube and Facebook.

1.56. There was also an Electric mobility conference –partnered with Gizand CARICOM (vehicle) that established the roadmap for EV developments. This was useful for sensitization since there were presenters and case studies from Barbadosand Grenada.


Tag: Energy Effectiveness Communication Oversight

17.

COMPONENT 3: CLEAN ENERGY RE-BASED ELECTRICITY GENERATION DEMONSTRATIONS:

1.57. Activities have been implemented to increase the number of RE projects that are financed through RE funds where VINLEC has involvement; in total there are 3 specific projects: the Mayreau Micro-grid –120 kW with 100 kW battery storage, the AIA solar PV –367.2 kW grid-connected (plus 155 kW financed by PACES which brings the total system size to 522.2 kW), and the Union Micro-grid –800 kW with 800 kW battery storage.

1.58. The Solar PV at Argyle airport and prison proved significant financial savings; increased resilience regarding climate Change, e.g., against storms, and capacity building in installation.

1.59. In Belle Isle, the project was seen as highly positive because from inception EU contacted the prison directives to discuss plans, going green, and saving costs (it is very expensive to run prisons). PACES is seen as a high need for Energy saving, lower costs of electricity and gas. The Biogas is not yet implemented but expected to generate a high impact. The Grid connected solution generates 6.000 EC / month in credit from a total bill of 9.000 EC bill/month.

1.60.The prison provided labor counterpart with seven inmates and a couple of officers who were trained on installing solar PV, and maintenance. As result, the interviewees expressed their satisfaction with the capacity built, the drive in the men, encouragement, and motivation. “When I leave from here I will motivate to do work in a company.”BelleIsle Inmate.

1.61. In the Argyle Airport, the installed system generates a credit of EC40.000 to EC60.000 each month out of a monthly expense of EC 100.000 in electricity. The AIA has established an energy committee to do a follow-up, there is now an energy policy in place to raise awareness from staff and travelers, and there are plans to get 2MW with grants and own revenues. E.g.,CDF is willing to do 400kW to reach 1MW with a grant.“In a couple of years, this airport will be an example of energy matters.”An electrical engineer from the airport

1.62. Although it is difficult to establish the level of attribution of PACES regarding privately-financed RE projects connected to VINLEC electricity grid by EO; according to records, SVG has so far 79 domestic installations, ten commercial installations, one industrial installation, and 3 VINLEC owned installations.

1.63. Regarding MW of RE on-grid projects installed by EOP, there is a total of 1.64MW (1643.65 kW) installed solar PV to date. The geothermal project which is expected to contribute over 10 MW to the national grid has experienced several delays and is not expected to be commissioned until 2021 based on the current schedule.


Tag: Energy Effectiveness Efficiency Human and Financial resources

18.

Monitoring and Evaluation: input and execution design.

Outcome 3

Renewable energy accounts for an increased share of SVG’s power generation mix

Number of RE projects that are financed through RE funds where VINLEC has involvement in operationalization by EOP

Number of privately-financed RE projects connected to VINLEC electricity grid by EOPMW of RE on-grid projects installed by EOP

MW capacity of RE generation projects (on-grid and off-grid) in planning and design stages by EOP

% reduction in electricity drawn from the grid for each household with on-grid rooftop solar-PV panels

1.25. The indicators of the logical framework were measurable and specific, with clear units of measurement, and targets. The indicators are related to the products and results of the project, and the total number of indicators was manageable. However, in some cases, the indicators didn’t include a baseline because data was not available since this is a recent topic in the country. Not having a baseline makes it impossible to assess evolution and change, but also, to set proper targets from the onset. This is a difficulty derived from the novelty of the topic and the lack of databases.

1.26. Examples of good SMART indicators are cumulative direct and direct post-project CO2 emission reductions resulting from the RE technical assistance and investments by end-of-project(EOP), ktons CO2, and, % share of RE in the power generation mix of SVG by EOP. Both indicators are SMART and useful to measure the project objective on Reduction in GHG emissions from fossil-fired power generation and fossil fuel consumption for road transport through the exploitation of SVG’s renewable energy resources for power generation.

1.27. At the outcome level (components) there are also some good indicators such as Number of on-grid RETs approved based on studies of improved RE policy and tariffs and RE grid integration (outcome 1), Number of tradespersons who have local certification to construct, assemble, operate and maintain RE technologies (outcome 2), Number of RE projects that are financed through RE funds where VINLEC has involvement in operationalization by EOP, Number of privately-financed RE projects connected to VINLEC electricity grid by EOP (outcome 3).

1.28. But also, there were some indicators that are not very useful to measure progress to results at the outcome level; for example, Outcome 1 has an indicator on Number of managers in VINLEC and Energy Unit dedicated to promoting of RE investments. This indicator duplicates the metrics on capacity building from Outcome 2, and it doesn’t measure a specific result, just an output which is staff promoting RE solutions; promotion of solution is not a result by itself as it is an activity or process.

1.29. The final evaluation highlights the good quality of the annual PIR project implementation reports because it contains qualitative information about the project and its progress, with assigned ratings, identified obstacles, risks, and adjustments.

1.30. The project had no monitoring and evaluation officer as it was foreseen, so the project team fulfilled these functions with support from UNDP.


Tag: Emission Reduction Renewable energy Monitoring and Evaluation Programme/Project Design

19.

Feedback of M&E activities used for adaptation management

1.31. The project indicators are adequate at the output level, and the monitoring scheme shed light on the execution of resources, performance, and progress in the implementation of the project during the execution.

1.32. According to the implementation reports, the project ranged from moderately unsatisfactory rating in 2016 to satisfactory ratings, and the recommendations resulted in actions and institutional agreements for the achievement of the expected results. Several of the recommendations or suggestions in these reports were addressed in subsequent periods for improvement.

1.33. According to the project team, the Project Implementation Report (PIR) was always used as a guide to inform the Project on how it has been progressing to date and to raise awareness of the outstanding targets that need to be achieved as well as to identify the gaps for reporting. One such example is that of the need for Gender reporting and inclusiveness in the project implementation. Given the project's unsatisfactory production of gender statistics and inclusiveness, the project strived to rectify that by including the country's National Women Committee in the consultation on the Electricity Supply Act, offered training to the lone female electrical inspectorate at the Ministry of Transport and Works as well as engaged with several females by inviting them to consultations, workshops etc. The Project has also made sure to include Gender in several of the consultation documents by involving women during the consultations, but also, by including data on men/women.

1.34. Another example is that the PIR reports expressed concern about the slow pace of the installations; as a result, the PMU worked assiduously to identify and implement the additional solar PV systems as well as to support the Energy Unit’s ability to support planning and design of RE systems. All these efforts resulted in a faster delivery rate.


Tag: Gender Equality Project and Programme management

Recommendations
1

For UNDP

For future NIM projects that require an implementing agency, it is important to analyze the administrative challenges and procurement delays, as well as mitigation measures and alternatives for an agile execution

 

2

For UNDP

All GEF interventions need to consider gender mainstreaming strategies from the onset, as well as a clear link to the achievement of the SDGs.

3

For SVG Government

As a successful experience, UNDP and the SGV Government can make an extra effort in dissemination and communication. The project should make a compilation of the lessons learned and good practices in the process, focusing on the active participation of the private sector, decision-makers, and the incentives to keep a productive collaboration. This information can be translated into a common language, identifying key messages and narratives to share with UNDP regional and country offices, development partners, UNDP website, email lists, media, and social networks. Likewise, this information can be used for the systematization of PACES experiences.

Latest Evaluations

Contact us

1 UN Plaza
DC1-20th Floor
New York, NY 10017
Tel. +1 646 781 4200
Fax. +1 646 781 4213
erc.support@undp.org