Terminal Evaluation of the project: Promotion and up-scaling of climate-resilient, resource efficient technologies in a Tropical Island Context (Resource Efficiency)

Report Cover Image
Evaluation Plan:
2017-2020, Seychelles
Evaluation Type:
Final Project
Planned End Date:
11/2019
Completion Date:
12/2019
Status:
Completed
Management Response:
Yes
Evaluation Budget(US $):
20,000

Share

Document Type Language Size Status Downloads
Download document UNDP-GEF-TE-TOR RE Updated 6.3.2019.doc tor English 284.00 KB Posted 138
Download document Executive Summary TE for RE Project.docx summary English 20.74 KB Posted 179
Download document Terminal Evaluation report - Final report (final version).pdf report English 1596.55 KB Posted 179
Title Terminal Evaluation of the project: Promotion and up-scaling of climate-resilient, resource efficient technologies in a Tropical Island Context (Resource Efficiency)
Atlas Project Number: 00079923
Evaluation Plan: 2017-2020, Seychelles
Evaluation Type: Final Project
Status: Completed
Completion Date: 12/2019
Planned End Date: 11/2019
Management Response: Yes
Focus Area:
  • 1. Environment & Sustainable Development
  • 2. Others
Corporate Outcome and Output (UNDP Strategic Plan 2014-2017)
  • 1. Output 1.4. Scaled up action on climate change adaptation and mitigation across sectors which is funded and implemented
SDG Target
  • 13.2 Integrate climate change measures into national policies, strategies and planning
  • 14.5 By 2020, conserve at least 10 per cent of coastal and marine areas, consistent with national and international law and based on the best available scientific information
Evaluation Budget(US $): 20,000
Source of Funding: GEF
Evaluation Expenditure(US $): 20,990
Joint Programme: No
Joint Evaluation: No
Evaluation Team members:
Name Title Email Nationality
Frank Klickenberg Consultant frank@klickenberg.com
GEF Evaluation: Yes
GEF Project Title: Terminal Evaluation of the project: Promotion and up-scaling of climate-resilient, resource efficient technologies in a Tropical Island Context (Resource Efficiency)
Evaluation Type: Terminal Evaluation
Focal Area: Climate Change
Project Type: MSP
GEF Phase: GEF-5
GEF Project ID: 5316
PIMS Number: 4913
Key Stakeholders: MEECC, Seychelles Energy Commission, Public Utilities Corporation, UNDP, PCU
Countries: SEYCHELLES
Lessons
Findings
1.

4. Findings

4.1. Project Design & Formulation (Genral)

From a strategic point of view, this was a useful project, focusing on a topic of great importance for the Seychelles. Appliances, and in particular the ones targeted in this project, contribute greatly to the overall energy demand of the Seychelles and improvements in energy efficiency offer the opportunity to reduce that energy demand in a costeffective manner. Being virtually totally dependent on imported fuel for its electricity generation, end-use electricity savings also generate important reductions in greenhouse gas emissions as well as in the cost of fuel imports. All stakeholders point out that more attention for energy efficiency was much needed and is beneficial for the country.
 


Tag: Energy Water resources Global Environment Facility fund Human and Financial resources Results-Based Management Theory of Change Inclusive economic growth Jobs and Livelihoods Trade and Development

2.

4. Findings

4.1. Project Design & Formulation (In details)

4.1.1. Analysis of LFA/Results Framework (Project logic /strategy; Indicators)

The Project results framework / Logical framework includes many standard elements, which are grouped in an unusual and somewhat inconsistent way. Although outputs are generally recognisable from standard approaches for improving appliance resource efficiency, items are not combined logically into coherent outcomes with a good timing of activities. In addition, the number of outcomes and outputs far exceeds what can be successfully implemented in a Medium-Sized project. Many of the indicators for the project are activity-based, and not reflective of measurable changes in the market the project wishes to change. Examples include the collection of baseline data; realisation of demo sites; and an assessment of the potential savings from absorption technology.

Several of the indicators that are based on changes in the market have targets that only track the completion of activities, not their impact in the market. Examples of this include the indicator “SEC efficiency and renewable energy unit operationalised with clear mandate / work plan and trained staff”, which the target reduces to “EE / RE unit fully operational by end of year ”; indicator “system for measuring energy and water savings from EE residential appliances operational”, reduced via the baseline status to “no system in place for monitoring SEEREP by PUC” (which is a small sub-set of all energy and water savings); and “No. of private sector importers, dealers and retailers of household electrical appliances with access to market information (on product sourcing, pricing, quality, etc.) and maintenance of RSE technologies” which is reduced via the target to “At least 20 private sector partners have received training and support by end of project”. The project lacks indicators for results at outcome-level (also because the logical framework lacks properly defined outcomes), and its overall goal is based on the impact of a single output.


Tag: Energy Water resources Effectiveness Results-Based Management Theory of Change Private Sector Capacity Building

3.

4.1. Project Design & Formulation (In details)

4.1.2. Assumptions and Risks

The project document includes a long and detailed risk assessment, however, some of the risks listed are not project specific.


Tag: Disaster Risk Reduction Vulnerable Energy Government Cost-sharing Risk Management Capacity Building

4.

4.1. Project Design & Formulation (In details)

4.1.3. Lessons from other relevant projects (e.g., same focal area) incorporated into project design

There is no record of the project using results of other projects for the formulation of its strategy and the project document makes no reference at all to other countries having implemented similar approaches and their experience. This is even more surprising given that, at the time this project was being developed, UNDP was also implementing appliance standards and labels projects in Ghana, Nigeria, South Africa and Kenya, as well as having implemented similar projects in the past in dozens of other countries. Given how much the project’s design could have benefitted from such experiences this lack of exchanging experiences has seriously hindered the project’s success. 


Tag: Civic Engagement Partnership Programme/Project Design Project and Programme management South-South Cooperation

5.

4.1. Project Design & Formulation (In details)

4.1.5. Replication approach

This project had not been designed for replication. No effort was made to set up activities in a structured way to let other countries benefit from what was (supposed to be) happening in the Seychelles. This needs to be seen against the backdrop of the Seychelles being a very small country, however, and a follower of policy developments rather than an initiator. It would have made perfect sense for the Seychelles to look carefully at the approaches taken by other countries, select the most suitable ones and align its policy with those, rather than design a project for replication elsewhere. Unfortunately, as discussed in section 4.1.3, that has not happened.


Tag: Sustainability Programme/Project Design South-South Cooperation

6.

4.1. Project Design & Formulation (In details)

4.1.6. UNDP comparative advantage

In theory, UNDP would be perfectly placed to design and implement a policy-focused project like this one in the Seychelles. It has the experience of similar projects in other countries as well as experience with project implementation in the Seychelles. Further, this project is a typical policy and institutional development project, at the heart of UNDP’s competence. UNDP was thus in an excellent position to develop and execute this project and link it to other international initiatives and expertise. Unfortunately, none of that has happened, neither during project design nor during its implementation, and the project has suffered from that lack of international exchange. As a result, UNDP’s competitive advantage in designing and implementing this project is questionable and, in this specific case, it is possible that other parties, e.g., NGOs used to working in a variety of countries within their specific area of expertise would have done a better job.


Tag: Water resources Global Environment Facility fund Programme Synergy Programme/Project Design Strategic Positioning

7.

4.1. Project Design & Formulation (In details)

4.1.8. Management arrangements

Neither the project’s PIF nor its CEO Endorsement request describe the institutional or management arrangements for the project, even though these are required items in the CEO Endorsement request. Both UNDP and the GEF should have noticed this discrepancy in their reviews of the project documents and requested inclusion of this important information. The UNDP project document, however, does include a description of management arrangements. The ProDoc describes that the project would be executed by the Department of Environment, whereas the CER lists the Seychelles Energy Commission as the executing partner. Even though the SEC resides under the Ministry, and it is listed as responsible for day-to-day oversight of the project, there still is a significant discrepancy between being the executing partner responsible for the project (as per CER) and being responsible for day-to-day oversight only under the management of the Ministry (ProDoc). Management arrangement in the ProDoc describe that a project manager would be hired as part of the Project Coordination Office which was set up by UNDP and the Government of the Seychelles to manage GEF projects. The project manager would then report to UNDP, the Seychelles Energy Commission and the Project Steering Committee. While there are no doubt good reasons for this, it does not reflect the intentions of national execution of projects, and in particular for a policy-focused project, where close integration of project activities with those of the executing agency are critical. Reporting of the project manager directly to UNDP and the steering committee, rather than to the executing agency, further reduces the mandate of the executing agency to direct the project in the way it sees as most suitable. The ProDoc failed to mention that the National Project Director (residing at the Ministry, not at the Seychelles Energy Commission) is also the country’s GEF operational focal point. Such a mix of roles is no doubt beneficial for an efficient communication with the GEF OFP, however, it also makes the GEF OFP’s oversight role of the GEF portfolio impossible, since one cannot oversee projects for which one has a direct responsibility. The ProDoc’s management arrangements further specify a project assurance role by the UNDP Seychelles environment team leader.

The country office, being a small office for a small country, does not have a dedicated environment team leader. Overall, the project design and formulation for this project was unsatisfactory. The failure to properly take into account the situation in the Seychelles and lessons from other economies, the inclusion of too many components for a Medium-Sized Project and the lack of outcome-oriented indicators or targets leave no other option than this lowest possible rating. The poor quality of the project design, and in particular the lack of SMART targets, also presents problems for the rest of the evaluation, which must compare results with targets. This terminal evaluation therefore first assesses which SMART targets and indicators could have been set for the various components of the project, had the design been based on a full assessment of the situation in the Seychelles in its international context, with an appropriate number of outcomes for a Medium-Sized project (in a small island setting) and aiming to maximise impacts in a country that had not yet implemented its first resource efficiency project. Such a reconstruction of targets and indicators implicitly carries risks, since it is done retroactively with hindsight. It also does not benefit from the full review of a project document by the Government, stakeholders and UNDP and the GEF secretariat. Nevertheless, it is the only option for a meaningful discussion of project results in the rest of this terminal evaluation report. For each originally envisioned outcome of the project, the originally defined target (reconstructed from output targets) is defined, followed by a discussion of the outcome in the context of the project’s setting and a target to be used for this terminal evaluation’s review of project results.


Tag: Global Environment Facility fund Monitoring and Evaluation Operational Efficiency Programme/Project Design Project and Programme management Results-Based Management Theory of Change

8.

4.2. Project Implementation (In details)

Implementation of this project was characterised by an understaffed project team and substantial delays in the implementation of key regulations, against the background of a too complex and overly ambitious project design. As a result, the project team focused on a subset of planned activities, however, without much formal planning about which parts of the project to prioritise. As a result, at the end of the project, there are many activities that have been started but not fully finished, demonstrated, for example, by the large number of reports and other deliverables still in draft or near-final stage.

4.2.1. Adaptive management (changes to the project design and project outputs during implementation)

The project worked with a project design that was not suitable for the situation in which the objectives had to be achieved, with a set of outcomes that was too large for the project, targets that were overly ambitious and planning of activities that did not match the time needed to properly develop elements of the approach. This creates a difficult setting, in which project management has no choice but to fall behind on a substantial part of the project. Under these circumstances, it is hard to decide when a project is so far behind its original plan that a restrategising is needed and the original design needs to be abandoned. Around the time of the Mid-Term Review, it was evident that the project was far behind on most of its outcomes and that it was unrealistic to expect that it would achieve its targets by the end of the project. The MTR recommended an extension of the duration of the project, however, incomprehensibly, no rethinking about the project’s goals or timing. Given that an MTR is the preferred moment to re-assess a project’s strategy, and this MTR recommended that the project struggles on, project management is hardly to blame for continuing with a flawed project design. In reality, project management did make changes to the project’s strategy, prioritising elements on which it could move ahead, in particular its VAT exemption mechanism and the education and communication strategy, and letting go of other parts, such as MEPS and labels. While understandable, it is still not a good course of action to make these changes without a careful rethinking of the consequences of such choices. Not pursuing MEPS and labels, for example, leaves the country at the end of the project without an infrastructure on which to continue and extend energy efficiency regulations in future, and communication, while leading to a very useful awareness raising, cannot be used to prepare the ground for regulations in this setting. Good adaptive management would have required that the project, under initiative and guidance of its overseers (the project steering committee and UNDP) would do a strategic rethinking of its approach, adapt its planning to the realities of the situation and work from a realistic plan in which elements of the project are aligned towards the overall goal of the project and working towards an end-point that allows for a sustainable continuation of efforts after the project. 

The project, in conclusion, has been adapting its approach to a challenging situation and, understandably, focusing on fewer elements of the project than designed. It failed to think through the linkages between elements and re-set its targets once it was clear that the original design was not achievable. Instead, implementation comes across as an almost ad-hoc process, in which the project acted when an opportunity presented itself which was in the wider remit of the project, without much focus on an integrated strategy and what would have been needed to reach a sustainable end point. While the flexibility of the project team is commendable, the lack of planning and management is not. Overall, implementation of adaptive management was weak and this harmed the project.


Tag: Resource mobilization Human and Financial resources Partnership Programme/Project Design Results-Based Management Country Government Coordination

9.

4.2. Project Implementation (In details)

4.2.3. Feedback from M&E activities used for adaptive management

The project document included a very standard M&E plan, focused more on delivering formally required documents than on tailored activities to establish what the impacts of the project were. As seems to have been the habit of UNDP at the time (seen in various projects from the era), any impact monitoring has been left as to be defined during implementation, also without an assigned budget. That approach seems not to have worked (not unique to this project, it also has not worked in other projects), as no impact monitoring was taken up during implementation and no budget assigned to it. Formally required documents were delivered, although the project did not finish its inception workshop with a finalised inception report, leaving it in draft state. Overall, there has been no discernible monitoring of project activities or results, other than through the annual PIR cycle. Even at the end of the project, the project team struggled to report on direct achievements of its activities and could only deliver an overview of, for example, use of its VAT exemption mechanism after repeated requests as part of the terminal evaluation process. It should also be noted that Project Implementation Reports include an estimation of the transformation of the market achieved by the project. PIRs present this as an information-based estimate, and this likely led reviewers to conclude that the project was on track to delivering its results. The PIRs correctly state that numbers presented are estimates in the absence of actual market monitoring, however, the presentation of numbers still suggest a level of accuracy that did not exist. This comes across as misleading and is not a good way to report about project impacts.


Tag: Energy Waste management Water resources Effectiveness Impact Resource mobilization Monitoring and Evaluation Oversight Project and Programme management Results-Based Management Theory of Change

10.

4.2. Project Implementation (In details)

4.2.4. Project Finance

Financial management of the project was fine as far as it concerns adequately recording spending of GEF- and UNDP provided funds, however, tracking of co-financing did not happen. There is also no evidence of tracking project spending per component and making sure that spending remains within agreed deviations of the GEF funding application. It has been clear for some time that the Government of the Seychelles has underdelivered on some of its co-financing commitments, in particular for project staff (project manager cost, and staff cost for the REEM unit), without a response by the project or UNDP. In fact, UNDP decided to cover the cost of the project manager from the GEF budget when the Government of the Seychelles declined to deliver on its commitment for this cost. This is not good practice for an implementing agency as it encourages the government to underdeliver on its commitments and may also lead to spending a higher share of the GEF budget on project management cost than was planned or is allowed under GEF rules. In the absence of detailed tracking of cost per category, however, the latter cannot be established with certainty. The project budget, as submitted to the GEF, included $210,000 in co-financing of project management cost; it is unclear how this budget was composed, other than a specified (and apparently not delivered) $80,000 contribution of the Ministry of Environment & Energy for project management. 


Tag: Efficiency Global Environment Facility fund Government Cost-sharing Human and Financial resources Operational Efficiency Oversight Results-Based Management

11.

4.2. Project Implementation (In details)

4.2.5. Monitoring and evaluation: design at entry and implementation (*)

The project’s M&E plan was seriously lacking in contents, at design. The plan included all the usual formal progress review documents, as well as the usual reference to PIRs which report on progress towards targets, however, all measurements of means of verification were left to be developed and specified by the project team during implementation. In the absence of a clear idea of what to monitor and without clear oversight, this was a rather minimal solution. As a result, the M&E framework was, in essence, designed to underperform. 


Tag: Energy Effectiveness Efficiency Communication Monitoring and Evaluation Oversight Programme/Project Design Results-Based Management

12.

4.2 Project Implementation (In general)

4.2.6. UNDP and Implementing Partner implementation / execution (*) coordination, and operational issues

The overview of the implementation of the project presented in this section points to several issues with the management of and reporting about the project, and several cases of the project not following the project document. Even where there are good reasons to deviate from the original project design, such as the overly complex and ambitious design for this project, good project management requires that decisions to not implement parts of the project are justified and documented, and that the consequences of these changes for other parts of the project are taken into account. Similarly, when Government co-financing for the project team (SEC RE&EE unit and project manager) were not delivered at the expected level, actions should have been taken to hold the Government to account and if needed halt or slow down the project until the Government would make available the necessary staff to implement the project. The goal of a project is, after all, not to spend its GEF budget but to achieve a transition in a country – and that is rather difficult when Government staff supposed to implement a project is not made available.


Tag: Efficiency Global Environment Facility fund Government Cost-sharing Implementation Modality Partnership Project and Programme management Coordination Operational Services

Recommendations
1

Terminal Evaluation recommendation 1. The Government of the Seychelles is recommended to continue, the development of framework legislation for energy efficiency and once a strategy has been agreed and a collaboration with an experienced partner established, to move speedily to develop and implement appliance MEPS and labels.  The VAT exemption mechanism, now used independently, can then be used to smoothen the introduction of MEPS. 

Recommendation to: MEECC & SEC.

2

Terminal Evaluation recommendation 2. Project designs need better reviews, including checks on internal consistency and whether baseline information is complete and has been adequately addressed in the project’s strategy. This should also include a check on the project’s strategy and whether this is aligned with the experience and capacity of a country’s government and market parties. 

Recommendation to: UNDP, MEECC and others 

3

Terminal Evaluation recommendation 3.  For the Seychelles, the policy approach would need to include a response to its situation as a small island nation, with limited government capacity, a small, relatively unorganised market and complicated trade relationships.  Support of international expert groups might be needed to develop a suitable approach for such situations.

Recommendation to: MEECC & SEC.

4

Terminal Evaluation recommendation 4. It might be useful to explore whether the Seychelles can develop a collaboration with an established standards and labels programme in a country it has trade relations with. That might make it easier to continue its approach without having to build up the extensive technical knowledge needed to do so independently.

Recommendation to: MEECC & SEC.

5

Terminal Evaluation recommendation 5: Results of the project in communication and through its VAT mechanism need to be measured, through household surveys or similar means and through completion of the VAT exemption database. 

Recommendation to: MEECC & SEC.

6

Terminal Evaluation recommendation 6. It would be useful to explore extending the “Switch to LED” exchange campaign, to more lamps and/or to other appliances also. Such approaches are beneficial in particular to low-income households, can offer specific national benefits when electricity tariffs are subsidised and may have a role in tariff restructuring.  

Recommendation to: MEECC, SEC and others

7

Terminal Evaluation recommendation 7. Future policy or regulatory projects for resource efficiency are not recommended at this point, given that several more years may be needed to reach the objectives of this project and the experience gained with that would be needed for new projects.

Recommendation to: MEECC, SEC and others.

8

Terminal Evaluation recommendation 8: Staff levels at the Renewable energy and energy management unit of the Seychelles Energy Commission urgently need to be brought up to planned levels, so that there is capacity to carry out the regulatory, communication and training activities planned, but not completed, under this project.

Recommendation to: SEC

9

5. Conclusions, Recommendations & Lessons learnt

5.2. Actions to follow up or reinforce initial benefits from the project (In general)

6) The Government of the Seychelles is recommended to continue, with appropriate urgency, the development of framework legislation for energy efficiency and, once a strategy has been agreed and – hopefully – a collaboration with an experienced partner established, move speedily to develop and implement appliance MEPS and labels. The VAT exemption mechanism, now used independently, can then be used to smoothen the introduction of MEPS. 

 7) There may also be scope to continue and extend the lamps for LED bulbs exchange campaign. A campaign like this essentially shifts the investment from the household to the utility, which would normally not be advisable, however, makes sense if households receive a subsidised energy tariff. A utility then has the choice to reduce electricity demand, and thus the amount of subsidy it needs to provide, for an investment – in this case LED light bulbs. A dedicated calculation is needed whether this approach might make economic sense, at the national level, for extension of the programme (to more light bulbs) and/or to other appliances (which could be exchanged for free or for a reduced price). Reducing household electricity consumption, and thus bills, might also be beneficial for a tariff revision, and reduced cost-efficient appliances could be made part of a larger tariff revision programme.

5.3. Proposals for future directions underlining main objectives

8) At this point in time, no proposals for future directions are suggested. The Government of the Seychelles will need its full attention for bringing what was started with this project to completion, a process that may easily take years and may provide challenges not yet foreseen. New policy- or regulatory-based initiatives around resource efficiency would be ill-advised until the ones started 5 years ago are completed and lessons from that trajectory can be learned. More efforts on communicating the benefits of resource efficiency might benefit the country, however, it is needed to first establish the impact of the communication activities of this project and, given experience so far, it is something the Government should be able to do on its own.

5.4. Best and worst practices in addressing issues relating to relevance, performance and success

Given that project outcomes have largely not yet been achieved and that more work is needed to bring core parts of the project to completion, no practices relating to relevance, performance and success can be discussed at this point.

1. Recommendation:

Terminal Evaluation recommendation 1. The Government of the Seychelles is recommended to continue, the development of framework legislation for energy efficiency and once a strategy has been agreed and a collaboration with an experienced partner established, to move speedily to develop and implement appliance MEPS and labels.  The VAT exemption mechanism, now used independently, can then be used to smoothen the introduction of MEPS. 

Recommendation to: MEECC & SEC.

Management Response: [Added: 2020/02/03] [Last Updated: 2020/11/08]

Management response: Management concurs with the recommendation. A consultancy for development of legislation for EE is currently on-going and final report to be submitted to SEC by latter half of 2019. SEC will then proceed with the necessary procedures for enactment. Subsidiary legislation such regulation for MEPS and labels will be developed in due course- but that will fall outside the scope and timeline of the project. It is hoped that the project created the necessary platform for Seychelles to realize its intended ambitions for a renewable energy future.

Key Actions:

Key Action Responsible DueDate Status Comments Documents
1.1 Consultant to finalized Energy Legislation report
[Added: 2020/02/03]
SEC, MEECC 2019/09 Overdue-Initiated At the time of management response it was expected that the Energy Legislation report would be finalized by September 2019
1.2 SEC to submit to Attorney General Office
[Added: 2020/02/03]
SEC 2019/12 Overdue-Not Initiated At the time of management response it was expected that the finalized Energy legislation would be submitted to the Attorney General's office for further action.
1.3 Attorney General Office develop the White paper/ Green paper legislation
[Added: 2020/02/03]
Attorney General's Office, MEECC, SEC 2020/04 Overdue-Not Initiated The AGs office will only publish the White paper once due processes are complete (Cabinet endorsement). It is unlikely that this will be published in 2020.
2. Recommendation:

Terminal Evaluation recommendation 2. Project designs need better reviews, including checks on internal consistency and whether baseline information is complete and has been adequately addressed in the project’s strategy. This should also include a check on the project’s strategy and whether this is aligned with the experience and capacity of a country’s government and market parties. 

Recommendation to: UNDP, MEECC and others 

Management Response: [Added: 2020/02/03] [Last Updated: 2020/11/08]

Management response:  Management does not fully concur with this recommendation. While UNDP can support with the reviewing project design and ensuring it passes the robustness checks,  the overall  scope of project design and logframe are often suited to reflect the requirements of national stakeholders within the scope of the donor’s requirements. Government is also encouraged to ensure that sufficient institutional capacity exists or will be built during project life cycle. This recommendation also needs to be brought to the attention of GEF, to ensure that project designs are fit-for-purpose within national contexts.

Key Actions:

Key Action Responsible DueDate Status Comments Documents
2.1 Flag inconsistencies in CC project design to GEF and limitations of Seychelles as a SIDS
[Added: 2020/02/03]
UNDP 2020/04 Overdue-Not Initiated UNDP will highlight the issues in project design with the relevant UNDP-GEF RTA. One issue that keeps repeating itself is the time lag between project design and project implementation - as contexts often change from the time the PIF is formulated to the time the budgets are released.
2.2 Ensure UNDPs new CPD cycle has a robust M&E framework to review progress of project outcomes
[Added: 2020/02/03]
UNDP 2020/12 Overdue-Not Initiated As UNDP gears itself for the next programming cycle, relevant recommendations will be taken on board to ensure that the M&E framework is better equipped to tackle inconsistencies in project design and support project implementation to achieve intended outcomes in an evolving context.
3. Recommendation:

Terminal Evaluation recommendation 3.  For the Seychelles, the policy approach would need to include a response to its situation as a small island nation, with limited government capacity, a small, relatively unorganised market and complicated trade relationships.  Support of international expert groups might be needed to develop a suitable approach for such situations.

Recommendation to: MEECC & SEC.

Management Response: [Added: 2020/02/03] [Last Updated: 2020/11/08]

Management response: Management is aware of the limitations of Seychelles capacity and is already working with international organisations such as IRENA/ SADC/ IOC /CCI with ongoing bilateral cooperation with other countries. More emphasis, will be placed on getting support for Energy Efficiency and Climate Change Mitigation related programmes in future discussions to continue the work started under the RE project.

Key Actions:

Key Action Responsible DueDate Status Comments Documents
3.1 Review existing partnerships and agreements with International Partners.
[Added: 2020/02/03]
SEC, MEECC 2019/12 Overdue-Not Initiated This will depend on the capacity and political will of the agency to move forward with a holistic approach in collaborating with international partners and donors
3.2 Engage with the department of Foreign Affairs to ensure that EE remains part of bilateral discussions with select countries
[Added: 2020/02/03]
SEC, MEECC, DFA 2019/12 Overdue-Not Initiated SEC needs to engage with the DFA either through MEECC or directly to ensure that EE and CCM are tabled in bilateral discussions to assist Seychelles achieve its INDCs
4. Recommendation:

Terminal Evaluation recommendation 4. It might be useful to explore whether the Seychelles can develop a collaboration with an established standards and labels programme in a country it has trade relations with. That might make it easier to continue its approach without having to build up the extensive technical knowledge needed to do so independently.

Recommendation to: MEECC & SEC.

Management Response: [Added: 2020/02/03] [Last Updated: 2020/11/08]

Management response: Management will look into the feasibility of this recommendation in line with recommendation 3,and whilst reviewing international partnerships and agreements.

Key Actions:

Key Action Responsible DueDate Status Comments Documents
4.1 Same as key action 3.1 and 3.2
[Added: 2020/02/03]
SEC 2020/12 Overdue-Initiated
5. Recommendation:

Terminal Evaluation recommendation 5: Results of the project in communication and through its VAT mechanism need to be measured, through household surveys or similar means and through completion of the VAT exemption database. 

Recommendation to: MEECC & SEC.

Management Response: [Added: 2020/02/03] [Last Updated: 2020/11/08]

Management response: While management concurs with the recommendation, no further communications with regards to the project will be ongoing. However, the product registry database (NOT VAT EXEMPTION DATABASE) is operational and is being managed by the staff of the SEC. An IT consultant will be supporting the work until end of 2019 supported by project funds.

Key Actions:

Key Action Responsible DueDate Status Comments Documents
5.1 IT consultant hired to maintain database
[Added: 2020/02/03]
SEC 2019/11 Completed An IT consultant recruited under the project was able to support the SEC until November 2019 to populate the database.
5.2 Reporting of key results from the database
[Added: 2020/02/03]
SEC, MEECC 2020/12 Overdue-Not Initiated This depends on the political will of the agencies to track and provide key results that may help in tracking and monitoring effectiveness of policies and contributions towards the SDGs.
6. Recommendation:

Terminal Evaluation recommendation 6. It would be useful to explore extending the “Switch to LED” exchange campaign, to more lamps and/or to other appliances also. Such approaches are beneficial in particular to low-income households, can offer specific national benefits when electricity tariffs are subsidised and may have a role in tariff restructuring.  

Recommendation to: MEECC, SEC and others

Management Response: [Added: 2020/02/03] [Last Updated: 2020/11/08]

Management response: Management concurs with this recommendation and SEC with project partners to explore ways to further improve on the “Switch to LED” exchange programme and consider other similar campaigns, depending on available funding.

Key Actions:

Key Action Responsible DueDate Status Comments Documents
6.1 Report on the uptake of the “Switch to LED”
[Added: 2020/02/03]
PUC, SEC 2019/12 Completed PUC was collecting data on households that had participated in the SWITCH to LED programme. This information is available on request from PUC.
6.2. Identify ways to improve on the “Switch to LED” exchange programme
[Added: 2020/02/03]
SEC, PUC, MEECC 2020/12 Overdue-Not Initiated It is expected that the SEC, PUC and partners will continue to support the drive up deman for Energy efficient appliances in Seychelles. This activity however fall outside the scope and remit of the project cycle.
6.3 Evaluate innovative financing opportunities to promote EE for low income households
[Added: 2020/02/03]
SEC, PUC and others 2020/12 Overdue-Not Initiated As part of the Governments strategy towards a Renewable Energy future, mechanisms to provide incentives for low income households are being discussed at a sector strategic level. However these decisions are government led and fall outside the scope of the project to know if and when such mechanisms may be rolled out.
7. Recommendation:

Terminal Evaluation recommendation 7. Future policy or regulatory projects for resource efficiency are not recommended at this point, given that several more years may be needed to reach the objectives of this project and the experience gained with that would be needed for new projects.

Recommendation to: MEECC, SEC and others.

Management Response: [Added: 2020/02/03] [Last Updated: 2020/11/08]

Management response: Management does not wholly concur with the recommendation particularly the SEC disagrees with the recommendation from the Terminal evaluator. SEC feels it is imperative to build upon the lessons learned from the UNDP-GEF RE project. The Government will address new opportunities to help ensure the sustainability of Energy Efficiency in the future. This may be in collaboration with other funding partners such as GCF.

Key Actions:

Key Action Responsible DueDate Status Comments Documents
7.1. SEC to identify next steps in addressing gaps and challenges
[Added: 2020/02/03]
SEC 2020/12 Overdue-Not Initiated
7.2 SEC to continue collaborating with partners for support
[Added: 2020/02/03]
SEC, MEECC, PUC 2020/12 Overdue-Not Initiated
8. Recommendation:

Terminal Evaluation recommendation 8: Staff levels at the Renewable energy and energy management unit of the Seychelles Energy Commission urgently need to be brought up to planned levels, so that there is capacity to carry out the regulatory, communication and training activities planned, but not completed, under this project.

Recommendation to: SEC

Management Response: [Added: 2020/02/03] [Last Updated: 2020/11/08]

Management response: Management agrees that the SEC needs additional staff to build institutional capacity and to execute planned activities relating to EE- but this remains outside the scope of the project. This would need further discussions between MEECC, SEC, the Ministry of Finance, Department of Public Administration and Public Utilities Corporation as the roles of SEC and PUC need to be re-defined and job descriptions clarified.

Key Actions:

Key Action Responsible DueDate Status Comments Documents
8.1 SEC to initiate discussions to increase its staffing capacity with relevant authorities.
[Added: 2020/02/03]
SEC 2020/12 Overdue-Not Initiated
9. Recommendation:

5. Conclusions, Recommendations & Lessons learnt

5.2. Actions to follow up or reinforce initial benefits from the project (In general)

6) The Government of the Seychelles is recommended to continue, with appropriate urgency, the development of framework legislation for energy efficiency and, once a strategy has been agreed and – hopefully – a collaboration with an experienced partner established, move speedily to develop and implement appliance MEPS and labels. The VAT exemption mechanism, now used independently, can then be used to smoothen the introduction of MEPS. 

 7) There may also be scope to continue and extend the lamps for LED bulbs exchange campaign. A campaign like this essentially shifts the investment from the household to the utility, which would normally not be advisable, however, makes sense if households receive a subsidised energy tariff. A utility then has the choice to reduce electricity demand, and thus the amount of subsidy it needs to provide, for an investment – in this case LED light bulbs. A dedicated calculation is needed whether this approach might make economic sense, at the national level, for extension of the programme (to more light bulbs) and/or to other appliances (which could be exchanged for free or for a reduced price). Reducing household electricity consumption, and thus bills, might also be beneficial for a tariff revision, and reduced cost-efficient appliances could be made part of a larger tariff revision programme.

5.3. Proposals for future directions underlining main objectives

8) At this point in time, no proposals for future directions are suggested. The Government of the Seychelles will need its full attention for bringing what was started with this project to completion, a process that may easily take years and may provide challenges not yet foreseen. New policy- or regulatory-based initiatives around resource efficiency would be ill-advised until the ones started 5 years ago are completed and lessons from that trajectory can be learned. More efforts on communicating the benefits of resource efficiency might benefit the country, however, it is needed to first establish the impact of the communication activities of this project and, given experience so far, it is something the Government should be able to do on its own.

5.4. Best and worst practices in addressing issues relating to relevance, performance and success

Given that project outcomes have largely not yet been achieved and that more work is needed to bring core parts of the project to completion, no practices relating to relevance, performance and success can be discussed at this point.

Management Response: [Added: 2020/11/03] [Last Updated: 2020/11/08]

Key Actions:

Latest Evaluations

Contact us

1 UN Plaza
DC1-20th Floor
New York, NY 10017
Tel. +1 646 781 4200
Fax. +1 646 781 4213
erc.support@undp.org