Improving the Performance and Reliability of RE Power System in Samoa (IMPRESS)_MTR

Report Cover Image
Evaluation Plan:
2018-2022, Samoa
Evaluation Type:
Mid Term Project
Planned End Date:
09/2020
Completion Date:
10/2020
Status:
Completed
Management Response:
Yes
Evaluation Budget(US $):
30,000

Share

Document Type Language Size Status Downloads
Download document UNDP-GEF-MTR-TOR_IMPRESS MTR - Final (v3) for UNDP roster.pdf tor English 578.93 KB Posted 184
Download document IMPRESS MTR Final Report.pdf report English 2311.41 KB Posted 259
Title Improving the Performance and Reliability of RE Power System in Samoa (IMPRESS)_MTR
Atlas Project Number: 00100814
Evaluation Plan: 2018-2022, Samoa
Evaluation Type: Mid Term Project
Status: Completed
Completion Date: 10/2020
Planned End Date: 09/2020
Management Response: Yes
UNDP Signature Solution:
  • 1. Resilience
Corporate Outcome and Output (UNDP Strategic Plan 2018-2021)
  • 1. Output 2.3.1 Data and risk-informed development policies, plans, systems and financing incorporate integrated and gender-responsive solutions to reduce disaster risks, enable climate change adaptation and mitigation, and prevent risk of conflict
SDG Goal
  • Goal 13. Take urgent action to combat climate change and its impacts
  • Goal 7. Ensure access to affordable, reliable, sustainable and modern energy for all
SDG Target
  • 13.2 Integrate climate change measures into national policies, strategies and planning
  • 7.1 By 2030, ensure universal access to affordable, reliable and modern energy services
Evaluation Budget(US $): 30,000
Source of Funding: GEF Trust Fund
Evaluation Expenditure(US $): 17,000
Joint Programme: No
Joint Evaluation: No
Evaluation Team members:
Name Title Email Nationality
Johannes Van Den Akker jhavdakker@hotmail.com
GEF Evaluation: Yes
GEF Project Title: Improving the Performance and Reliability of RE Power System in Samoa (IMPRESS)
Evaluation Type: Mid-term Review
Focal Area: Climate Change
Project Type: FSP
GEF Phase: GEF-6
GEF Project ID: 9251
PIMS Number: 5669
Key Stakeholders: Government of Samoa(GoS), Ministry of Finance (MoF), Ministry of Natural Resources & Environment (MNRE), NGOs, CSOs
Countries: SAMOA
Lessons
Findings
Recommendations
1

Re-draft the list of outputs and outcome indicators

The current framework of outcomes, outputs, and activities needs to be revised with baseline activities that are clearly separated from GEF-incremental and selecting technology-application-target group combinations to focus on within the overall GEF-approved framework.  Some outcome progress indicators need to be redefined as well.  The MTR Consultant has therefore proposed a revised list outputs and outcome indicators (see Box 3 and Box 22) with the project Components on the vertical axis (as in the project’s results framework) and major thematic areas on the horizontal axis, namely a) sustainable energy policy, b) utility-scale RE (bio-gasification), c) community-scale RE for PUE/USUE (biogas for on-site heat applications), d) energy efficiency and integration of RE in buildings (and other sectors).

The biogas activities need to be more clearly linked with PUE and SUE in these communities. The MTR Consultant hopes this brings more focus on certain technology-target group combinations. Examples are larger-scale bioenergy (biomass gasification), small-scale bioenergy (biogas application in rural communities linked with local development), and application of energy efficiency and renewable energy (e.g. rooftop PV, solar water heating) in buildings. Financial support schemes need to address the real needs of specific technology-application-beneficiary combinations and build on existing schemes. It is premature to consider a financial mechanism (with a bank) for the cluster formed by biogas, rural PUE and SUE. At this stage, these applications have not been (commercially) proven in this area and given the income levels of beneficiaries, grant support may be more appropriate. For EE and RE integration in buildings and (commercial) productive uses, a financial mechanism might be an option, provided these go hand-in-hand with favorable government regulations (e.g. net-metering).Re-draft the list of outputs and outcome indicators - Proposed a revised list outputs and outcome indicators (see Box 3 and Box 22) with the project Components on the vertical axis (as in the project’s results framework) and major thematic areas on the horizontal axis, namely a) sustainable energy policy, b) utility-scale RE (bio-gasification), c) community-scale RE for PUE/USUE (biogas for on-site heat applications), d) energy efficiency and integration of RE in buildings (and other sectors).

2

Due to the emergencies in Samoa (e.g., COVID-19), the evaluation of candidates for the post of CTA was delayed. The CTA was recently appointed[1] and will be instrumental in helping to (re-)define and implement activities in which the project has been lagging, notably in the area of community-scale biomass and related productive and social uses, and energy efficiency. In this respect, it is suggested that:

  • The CTA will audit the Project based on the originally planned outcomes/outputs and scheduled activities (as given in ProDoc and in the work plan mentioned in the latest QPR);
  • The Project, UNDP CO and UNDP RTA have a discussion based on 1) the CTA’s audit of originally planned outputs/activities, 2) results of a planned integrated study on RE and EE[2]; 3) the suggestions given by the MTR consultant for re-drafting the list of ‘outputs and outcome indicators’, presented in the Boxes 3 and 22;
  •  For the remaining implementation period with the IMPRESS budget remaining of about USD 2.5 million, the MTR recommends that the re-drafted outputs have activities that are well-described in a budgeted work plan be made for the remaining 2-year period of with a clear list of remaining activities and deliverables to be produced; and,
  • Last, but not least, the CTA could take a lead role not only in revising activities, but also in having responsibility for implementation with an agreed timeline

[1]     Fonoti Perelini

[2]     An integrated study is planned on a) business model and financial schemes for the sustainable supply of biomass resources (production, harvesting, processing and supply of biomass for power and non-power uses), b) cost-benefit of RE technologies, integration into the existing EPC grid, and role of decentralized RE power generation, c) energy efficiency. 

3

 When re-drafting the IMPRESS work plan for 2021-2022, the following recommendation can be taken into account:

a)   The role of biogas. The Piu project (installed before IMPRESS started) has not been functioning and reasons should be incorporated in the study together with recommendations on the way forward. One needs to look very critically at the role of community-level electrification and its competitiveness and need for IMPRESS support, in view of the fact that almost 100% of Samoa is supplied with electricity;

b)   Expanding biomass for heat applications (replacing fuels) to villages must be approached too with caution because of the lack of capacity in villages, sustainable supply of feedstock, economy of scale, and competitiveness concerns; PCU and UNDP agree with recommendation;

c)   Study and discuss issues and options in recent IPP developments (apart from Afolau, the solar IPPs) and lessons learnt for future IPP development. This may serve as a guideline for the planned grid-connected solar PV development, as well as RE additions beyond the period 2023-25; PCU and UNDP agree with recommendation;

d) Initiating studies and discussion on efficiency and fuel use in marine and land transportation and needs for financial and technical support; PCU and UNDP agree with recommendation; and,

e) The MTR feels that it is premature to consider a financial mechanism (with a bank) for the cluster formed by biogas, rural PUE and SUE. At this stage, energy applications in this area are far from being commercially viable and, given the income levels of beneficiaries, grant support is likely to be more appropriate. It is suggested that the above-mentioned ‘integrated study’ under CTA guidance explores other ‘energy and financing’ options, such as a financial mechanism for EE and RE integration in buildings and (commercial) productive uses in combination with the introduction of appropriate government regulations (e.g. on net-metering or fiscal incentives for ‘green’ investments as an expansion of regulation on EE Act or Energy Bill).

In this respect, the new CTA may want to add to the tasks a review of international experiences (in particular in the Pacific region, see on the design, establishment, and operation of sustainable energy financing schemes to promote RE and EE.  Taking this into account, the Outputs of Component 3 need to be revised and re-drafted. It should be noted IMPRESS should follow the guidelines of UNDP Programme and Operations Policies and Procedures (POPP) regarding the use of ‘micro-capital grants’ for credit and non-credit purposes and the regulations regarding ‘performance-based payments’ (PBP). At the time of IMPRESS formulation, this PBP was not yet been approved. The Project/CTA or a consultant should investigate the need and options for using PBP-type schemes in IMPRESS. Even if it is not possible for such a financial mechanism to become operational in the IMPRESS period, the Project can lay the foundation by means of concrete proposals for post-IMPRESS implementation. 

4

Make more ‘easy-to-read’ reporting. 

The results of the Project are reported on in the Quarterly Reports that, apart from the progress reporting, comes with Minutes of TAG and TWG meeting, annual/quarterly work plans, and technical reporting on subjects that are of interest in the particular quarter. However, the QPRs are not accessible to outsiders. Even if available on the web, the information on progress is be scattered over the multitude of QPRs and annexes.  It is suggested that drafting of the next PIR (for UNDP/GEF internal reporting) is accompanied with an ‘annual progress report’  for 1) internal readers which gives a summary of a) concise narrative of key results in the particular year and a summary on progress per component in general, b) overview of planned actions and priorities for the coming year(s), and 2) for external readers a summary with success stories and highlights as well as issues and lessons learnt. To avoid having to write different annual reports two times a year, it is proposed that the timeframe of the ‘annual report’ coincides with that of the PIR. Many UNDP/GEF projects produce such reports. For example, the Nepal UNDP/GEF RERL project has been drafting quite detailed annual reports (see http://www.aepc.gov.np/rerl/public/). 

5

Project transportation - The Project has indicated the need for a project vehicle. Having this type of mobility will be more important now the project will shift to small interventions at various sites (e.g., the biogas sites) and to do M&E.

6

Sustainable energy and bioenergy planning

Sustainable energy master plan (with bioenergy component) - Towards the end of IMPRESS, have a consultancy assignment (guided by the CTA and project management) to formulate a “renewable energy master plan, with a separate section on bioenergy”. The RE master plan would build on the results of the before-mentioned integrated study (see point 4) and the latest expansion plans that would boost electric energy production from RE to about 90%.  The bioenergy section should cover opportunities at least in the two main areas, a) larger-scale power generation by IPPs for the grids on Upolu and Savaii (gasification or larger-scale biogas for power generation), and b) small-scale biomass for heat applications in agro-business, tourism facilities and social services (biogas, other). The national bioenergy action plan should cover short, and medium-term with targets aligned with the current SESP 2017-2022, EPC’s Power Plans, and Samoa’s longer-term development goals and come with an operational plan indicating institutional responsibilities and budget.

1. Recommendation:

Re-draft the list of outputs and outcome indicators

The current framework of outcomes, outputs, and activities needs to be revised with baseline activities that are clearly separated from GEF-incremental and selecting technology-application-target group combinations to focus on within the overall GEF-approved framework.  Some outcome progress indicators need to be redefined as well.  The MTR Consultant has therefore proposed a revised list outputs and outcome indicators (see Box 3 and Box 22) with the project Components on the vertical axis (as in the project’s results framework) and major thematic areas on the horizontal axis, namely a) sustainable energy policy, b) utility-scale RE (bio-gasification), c) community-scale RE for PUE/USUE (biogas for on-site heat applications), d) energy efficiency and integration of RE in buildings (and other sectors).

The biogas activities need to be more clearly linked with PUE and SUE in these communities. The MTR Consultant hopes this brings more focus on certain technology-target group combinations. Examples are larger-scale bioenergy (biomass gasification), small-scale bioenergy (biogas application in rural communities linked with local development), and application of energy efficiency and renewable energy (e.g. rooftop PV, solar water heating) in buildings. Financial support schemes need to address the real needs of specific technology-application-beneficiary combinations and build on existing schemes. It is premature to consider a financial mechanism (with a bank) for the cluster formed by biogas, rural PUE and SUE. At this stage, these applications have not been (commercially) proven in this area and given the income levels of beneficiaries, grant support may be more appropriate. For EE and RE integration in buildings and (commercial) productive uses, a financial mechanism might be an option, provided these go hand-in-hand with favorable government regulations (e.g. net-metering).Re-draft the list of outputs and outcome indicators - Proposed a revised list outputs and outcome indicators (see Box 3 and Box 22) with the project Components on the vertical axis (as in the project’s results framework) and major thematic areas on the horizontal axis, namely a) sustainable energy policy, b) utility-scale RE (bio-gasification), c) community-scale RE for PUE/USUE (biogas for on-site heat applications), d) energy efficiency and integration of RE in buildings (and other sectors).

Management Response: [Added: 2021/02/03] [Last Updated: 2021/04/05]

 Refer to comments given below. No objections to this recommendations made by Consultant  Agree!  

Key Actions:

Key Action Responsible DueDate Status Comments Documents
1.1 Re-draft and include changes are recommended. PCU and MNRE to work closely on the revision. Recommendations to be used for monitoring and evaluation.
[Added: 2021/02/21]
UNDP, MNRE, PCU 2021/06 Overdue-Initiated Clarify which proposed changes will be considered. Per design, the indicators of the outputs can be used by the IP/PCU for M&E. Similarly, changes in the indicators of revised/new outputs can be used by the IP/PCU for M&E.
1.2 A consultation to be held with the Technical Working Groups to map out the project activities and discuss the revised list of outputs and outcome indicators in Box 3 and Box 22 of the Final MTR Report with the project Components and major thematic areas and to Revise MYWP.
[Added: 2021/02/21]
UNDP, MNRE, PCU 2021/06 Overdue-Initiated The delivery of the proposed revised outputs in each component must contribute to the achievement of the component outcome. The GEF budget for revised/new activities should be based on incremental cost principles
2. Recommendation:

Due to the emergencies in Samoa (e.g., COVID-19), the evaluation of candidates for the post of CTA was delayed. The CTA was recently appointed[1] and will be instrumental in helping to (re-)define and implement activities in which the project has been lagging, notably in the area of community-scale biomass and related productive and social uses, and energy efficiency. In this respect, it is suggested that:

  • The CTA will audit the Project based on the originally planned outcomes/outputs and scheduled activities (as given in ProDoc and in the work plan mentioned in the latest QPR);
  • The Project, UNDP CO and UNDP RTA have a discussion based on 1) the CTA’s audit of originally planned outputs/activities, 2) results of a planned integrated study on RE and EE[2]; 3) the suggestions given by the MTR consultant for re-drafting the list of ‘outputs and outcome indicators’, presented in the Boxes 3 and 22;
  •  For the remaining implementation period with the IMPRESS budget remaining of about USD 2.5 million, the MTR recommends that the re-drafted outputs have activities that are well-described in a budgeted work plan be made for the remaining 2-year period of with a clear list of remaining activities and deliverables to be produced; and,
  • Last, but not least, the CTA could take a lead role not only in revising activities, but also in having responsibility for implementation with an agreed timeline

[1]     Fonoti Perelini

[2]     An integrated study is planned on a) business model and financial schemes for the sustainable supply of biomass resources (production, harvesting, processing and supply of biomass for power and non-power uses), b) cost-benefit of RE technologies, integration into the existing EPC grid, and role of decentralized RE power generation, c) energy efficiency. 

Management Response: [Added: 2021/02/03] [Last Updated: 2021/04/05]

Partially Agreed. We do not agree with the first and the second bullet point for reasons listed in the comment box.  We strongly support the CTA responsibilities listed in the last bullet point

Key Actions:

Key Action Responsible DueDate Status Comments Documents
2.1 Develop workplan for the remaining time frame of the project including the remaining utilization of funds ( 2.5million USD
[Added: 2021/02/25]
MNRE, UNDP CO, UNDP RTA, PCU 2021/01 Completed The work plan should also include the scheduling of the baseline/co-financed activities and the corresponding co-financing utilization
2.2 – Re-confirm the responsibilities of CTA as per TOR /Contract signed
[Added: 2021/02/25]
MNRE, UNDP CO, UNDP RTA, PCU 2021/01 Completed MNRE and UNDP to revise the Work Plan of the CTA to include suitable activities
3. Recommendation:

 When re-drafting the IMPRESS work plan for 2021-2022, the following recommendation can be taken into account:

a)   The role of biogas. The Piu project (installed before IMPRESS started) has not been functioning and reasons should be incorporated in the study together with recommendations on the way forward. One needs to look very critically at the role of community-level electrification and its competitiveness and need for IMPRESS support, in view of the fact that almost 100% of Samoa is supplied with electricity;

b)   Expanding biomass for heat applications (replacing fuels) to villages must be approached too with caution because of the lack of capacity in villages, sustainable supply of feedstock, economy of scale, and competitiveness concerns; PCU and UNDP agree with recommendation;

c)   Study and discuss issues and options in recent IPP developments (apart from Afolau, the solar IPPs) and lessons learnt for future IPP development. This may serve as a guideline for the planned grid-connected solar PV development, as well as RE additions beyond the period 2023-25; PCU and UNDP agree with recommendation;

d) Initiating studies and discussion on efficiency and fuel use in marine and land transportation and needs for financial and technical support; PCU and UNDP agree with recommendation; and,

e) The MTR feels that it is premature to consider a financial mechanism (with a bank) for the cluster formed by biogas, rural PUE and SUE. At this stage, energy applications in this area are far from being commercially viable and, given the income levels of beneficiaries, grant support is likely to be more appropriate. It is suggested that the above-mentioned ‘integrated study’ under CTA guidance explores other ‘energy and financing’ options, such as a financial mechanism for EE and RE integration in buildings and (commercial) productive uses in combination with the introduction of appropriate government regulations (e.g. on net-metering or fiscal incentives for ‘green’ investments as an expansion of regulation on EE Act or Energy Bill).

In this respect, the new CTA may want to add to the tasks a review of international experiences (in particular in the Pacific region, see on the design, establishment, and operation of sustainable energy financing schemes to promote RE and EE.  Taking this into account, the Outputs of Component 3 need to be revised and re-drafted. It should be noted IMPRESS should follow the guidelines of UNDP Programme and Operations Policies and Procedures (POPP) regarding the use of ‘micro-capital grants’ for credit and non-credit purposes and the regulations regarding ‘performance-based payments’ (PBP). At the time of IMPRESS formulation, this PBP was not yet been approved. The Project/CTA or a consultant should investigate the need and options for using PBP-type schemes in IMPRESS. Even if it is not possible for such a financial mechanism to become operational in the IMPRESS period, the Project can lay the foundation by means of concrete proposals for post-IMPRESS implementation. 

Management Response: [Added: 2021/02/03] [Last Updated: 2021/02/25]

Partially agree! Refer to UNDP comments below. We support the CTA’s recommendation on the Piu Project. 

Key Actions:

Key Action Responsible DueDate Status Comments Documents
3.3 CTA to produce a report on the issues and options in recent IPP developments and include lessons learnt for future IPP developments. Report to serve as a guideline for the planned-grid connected solar PV and future developments.
[Added: 2021/02/21]
PCU, MNRE 2021/06 Overdue-Initiated This is a huge area that is beyond the scope of the CTAs contract. It is anticipate the EPC will hire a Consultant to carry out this assignment.
3.4 To initiate studies and discussion on efficiency and fuel use in marine and land transport and needs for financial and technical support
[Added: 2021/02/21]
PCU, MNRE 2021/06 Overdue-Initiated IMPRESS include EE, but not intended for EE in transport. The intention was more on the EE use of electricity, and EE in the operation of RE-based energy generation units. The study can take place given there is budget available.
3.1 CTA to prepare a full report on the Piu Biogas Project and include recommendations on a way forward
[Added: 2021/02/21]
CTA, PCU, MNRE 2020/12 Completed PCU to work on Piu as per advice from IMPRESS CTA. CTA provided full detailed report on way forward for the Piu Biogas Project
3.2 To undertake studies and consultations with the villages with regards to the use of biogas for heating to ensure there is feed stock and capacity to operate
[Added: 2021/02/21]
PCU, MNRE 2021/01 Completed B.E.S.T. has completed visit of the proposed sites with community consultations completed
3.5 Develop a proposal for donors for any remaining fund post August 2022 to be used as starting capital for “Micro –capital” grant scheme; so that the financial mechanism/scheme formulated under the IMPRESS project can be made operational with the local commercial banks.
[Added: 2021/02/21]
PCU, MNRE 2021/09 Not Initiated Financing scheme and business scheme is already in Place. IMPRESS to contribute to the financial mechanism.
4. Recommendation:

Make more ‘easy-to-read’ reporting. 

The results of the Project are reported on in the Quarterly Reports that, apart from the progress reporting, comes with Minutes of TAG and TWG meeting, annual/quarterly work plans, and technical reporting on subjects that are of interest in the particular quarter. However, the QPRs are not accessible to outsiders. Even if available on the web, the information on progress is be scattered over the multitude of QPRs and annexes.  It is suggested that drafting of the next PIR (for UNDP/GEF internal reporting) is accompanied with an ‘annual progress report’  for 1) internal readers which gives a summary of a) concise narrative of key results in the particular year and a summary on progress per component in general, b) overview of planned actions and priorities for the coming year(s), and 2) for external readers a summary with success stories and highlights as well as issues and lessons learnt. To avoid having to write different annual reports two times a year, it is proposed that the timeframe of the ‘annual report’ coincides with that of the PIR. Many UNDP/GEF projects produce such reports. For example, the Nepal UNDP/GEF RERL project has been drafting quite detailed annual reports (see http://www.aepc.gov.np/rerl/public/). 

Management Response: [Added: 2021/02/03] [Last Updated: 2021/02/21]

Agree with this recommendation

Key Actions:

Key Action Responsible DueDate Status Comments Documents
.1 PCU to make PIRs and Project Quarterly reports accessible to the public
[Added: 2021/02/21]
PCU 2021/01 Completed Partially agree with recommendation because the GEF reporting is PIR and PCU reports quarterly. Reports to be released upon request.
5. Recommendation:

Project transportation - The Project has indicated the need for a project vehicle. Having this type of mobility will be more important now the project will shift to small interventions at various sites (e.g., the biogas sites) and to do M&E.

Management Response: [Added: 2021/02/03] [Last Updated: 2021/02/21]

MNRE/IMPRESS PCU fully supports this recommendation so that IMPRESS PCU team can adequately carry out their M & E especially with all the community sites scattered around the islands.

Key Actions:

Key Action Responsible DueDate Status Comments Documents
UNDP to work closely with PCU to provide support in terms of transport and mobility during site visits, M&E, and school visitations
[Added: 2021/02/21]
UNDP 2020/12 Completed UNDP and PCU to revise travel budget to be used for car rentals during project visits.
6. Recommendation:

Sustainable energy and bioenergy planning

Sustainable energy master plan (with bioenergy component) - Towards the end of IMPRESS, have a consultancy assignment (guided by the CTA and project management) to formulate a “renewable energy master plan, with a separate section on bioenergy”. The RE master plan would build on the results of the before-mentioned integrated study (see point 4) and the latest expansion plans that would boost electric energy production from RE to about 90%.  The bioenergy section should cover opportunities at least in the two main areas, a) larger-scale power generation by IPPs for the grids on Upolu and Savaii (gasification or larger-scale biogas for power generation), and b) small-scale biomass for heat applications in agro-business, tourism facilities and social services (biogas, other). The national bioenergy action plan should cover short, and medium-term with targets aligned with the current SESP 2017-2022, EPC’s Power Plans, and Samoa’s longer-term development goals and come with an operational plan indicating institutional responsibilities and budget.

Management Response: [Added: 2021/02/03] [Last Updated: 2021/02/21]

MNRE fully supports the recommendation.

Key Actions:

Key Action Responsible DueDate Status Comments Documents
6.2 Recruit TA to develop the RE Policy first
[Added: 2021/02/21] [Last Updated: 2021/02/25]
MNRE, PCU 2021/06 Overdue-Initiated UNDP to provide assistance in the recruitment of the TA to develop the RE Policy first. History
6.1 Develop TOR for TA (local consultant) to develop this RE Master Plan
[Added: 2021/02/21]
MNRE, PCU 2021/03 Completed Note this is part of the scope and service of the CTA for IMPRESS. Note EPC takes the lead in developing RE Master Plan.

Latest Evaluations

Contact us

1 UN Plaza
DC1-20th Floor
New York, NY 10017
Tel. +1 646 781 4200
Fax. +1 646 781 4213
erc.support@undp.org