Action for the Development of Marshall Islands Renewable Energies (ADMIRE) Project

Report Cover Image
Evaluation Plan:
2013-2017, Fiji
Evaluation Type:
Final Project
Planned End Date:
05/2016
Completion Date:
11/2018
Status:
Completed
Management Response:
Yes
Evaluation Budget(US $):
30,000

Share

Document Type Language Size Status Downloads
Download document UNDP GEF Terminal Evaluation Terms of Reference Template 2012_ADMIRE_final_6 October2015.doc tor English 197.00 KB Posted 8
Download document TE RMI UNDP Final 12 May 2016 Revised 2.pdf report English 795.12 KB Posted 8
Title Action for the Development of Marshall Islands Renewable Energies (ADMIRE) Project
Atlas Project Number: 00112837
Evaluation Plan: 2013-2017, Fiji
Evaluation Type: Final Project
Status: Completed
Completion Date: 11/2018
Planned End Date: 05/2016
Management Response: Yes
Focus Area:
  • 1. Others
Corporate Outcome and Output (UNDP Strategic Plan 2014-2017)
  • 1. Output 1.4. Scaled up action on climate change adaptation and mitigation across sectors which is funded and implemented
Evaluation Budget(US $): 30,000
Source of Funding: GEF Trust Fund
Evaluation Expenditure(US $): 25,750
Joint Programme: No
Mandatory Evaluation: No
Joint Evaluation: No
Evaluation Team members:
Name Title Email Nationality
Amitra Rath Evaluator arath@policyresearch.ca
GEF Evaluation: Yes
GEF Project Title: Action for the Development of Marshall Islands Renewable Energies (ADMIRE) Project
Evaluation Type: Terminal Evaluation
Focal Area: Climate Change
Project Type: MSP
GEF Phase: GEF-4
GEF Project ID: 2568
PIMS Number: 3094
Key Stakeholders: Energy Department - Ministry of Public Utilities and Infrastructures
Countries: FIJI
Lessons
Findings
Recommendations
1

Issue (Sub-chapter 3.5 - Weaknesses of project design)

 Issue (Sub-chapter 3.2, paragraph 42):  Project Design. The project design appears to have focused too narrowly on its goals without considering the specific issues of RMI that would be most relevant to the project implementation. Its small size, the distribution of its land and people in many islands and atolls and its own acknowledged lack of “capacity”.

Paragraph 48: Exclusion of donors as stakeholders and partners.   The project document did not mention any donors as stakeholders and partners, though it had mentioned that an “EU funded solar home system project in the outer islands provided for the co-financed component”. And besides the EU, the RMI has another half dozen important and large donors involved in multiple activities in the RMI on CC related work, many with RE elements, that link with ADMIRE.

 

Paragraph 49: Exclusion of known challenges in RMI to PV systems.  The PRODOC also missed some of the known challenges in RMI to PV systems, noted in the 2004 PIREP project report and later again in the PIGGAREP report (referred to in the PRODOC), page 74, of poor electricity tariff collection; lack of understanding among users and of commercial orientation; poor enforcement of project rules, political pressure to subsidize O&M, both leading to lack of collections, which had been noted in RMI. It noted also that the RMI Energy office had 2 staff (page 126), which was double that noted in PIREP in 2004, but clearly insufficient for the ambitious goals.

 

Paragraph 52: Reporting nature, mélange of activities, absence of key data.  The design weaknesses stemmed from several factors. One is the patchwork nature of the reporting in the MSP PDF/PPG report, this has been mentioned at several places. It was unduly ambitious in its expected impacts. It mixed risks of country capacity issues, with RE barriers, and generic versus project specific risks. The mistaken notions of risk did not then allow for suitable risk mitigation strategy. The second stems from the mélange of activities and outcomes as listed in Table 3. The MTR for the project in 2012, also critiqued the project design, stating it had a “typical GEF format by having a capacity building, policy component, institutional component, financial and awareness component with various activities that relate to the two technologies (PV and oil from copra processing) scattered over various components”. Third, it was seen in during the evaluation, the absence of key data on energy use and more often, the lack of pulling together data that is available to create a coherent map of energy in RMI was noted in the design.

 

2

Issue (Sub-chapter 3.7– Implementation after the Mid-Term Review: 2012-2015):

 

Paragraph 64: Unrevised outcome indicators.  The MTR suggested revised outcome indicators but the evaluation did not find any revisions to the LFA and to the outcome indicators for ADMIRE as recommended by the MTR. In fact, the subsequent Annual Work Plans reviewed continued to list the same six outcomes, with 18 activities (not counting learning and M&E).

 

Paragraph 65: Lack of additional support by UNDP experts.   The MTR also recommended focusing the training programs for the design, operation and maintenance of stand-alone and grid-connected PV systems in the country; the use of copra as form of payment for electricity tariff for the outer islands households SHS; and user training and awareness campaigns in rural schools and health centers; to explore and conclude opportunity of cooperation (e.g., North REP MoU) with Tobolar and ADB on copra oil processing. Any evidence that the recommendation for additional support to be provided by UNDP experts, together with close monitoring and technical support by UNDP office, was fulfilled was not seen in the evaluation.

1. Recommendation:

Issue (Sub-chapter 3.5 - Weaknesses of project design)

 Issue (Sub-chapter 3.2, paragraph 42):  Project Design. The project design appears to have focused too narrowly on its goals without considering the specific issues of RMI that would be most relevant to the project implementation. Its small size, the distribution of its land and people in many islands and atolls and its own acknowledged lack of “capacity”.

Paragraph 48: Exclusion of donors as stakeholders and partners.   The project document did not mention any donors as stakeholders and partners, though it had mentioned that an “EU funded solar home system project in the outer islands provided for the co-financed component”. And besides the EU, the RMI has another half dozen important and large donors involved in multiple activities in the RMI on CC related work, many with RE elements, that link with ADMIRE.

 

Paragraph 49: Exclusion of known challenges in RMI to PV systems.  The PRODOC also missed some of the known challenges in RMI to PV systems, noted in the 2004 PIREP project report and later again in the PIGGAREP report (referred to in the PRODOC), page 74, of poor electricity tariff collection; lack of understanding among users and of commercial orientation; poor enforcement of project rules, political pressure to subsidize O&M, both leading to lack of collections, which had been noted in RMI. It noted also that the RMI Energy office had 2 staff (page 126), which was double that noted in PIREP in 2004, but clearly insufficient for the ambitious goals.

 

Paragraph 52: Reporting nature, mélange of activities, absence of key data.  The design weaknesses stemmed from several factors. One is the patchwork nature of the reporting in the MSP PDF/PPG report, this has been mentioned at several places. It was unduly ambitious in its expected impacts. It mixed risks of country capacity issues, with RE barriers, and generic versus project specific risks. The mistaken notions of risk did not then allow for suitable risk mitigation strategy. The second stems from the mélange of activities and outcomes as listed in Table 3. The MTR for the project in 2012, also critiqued the project design, stating it had a “typical GEF format by having a capacity building, policy component, institutional component, financial and awareness component with various activities that relate to the two technologies (PV and oil from copra processing) scattered over various components”. Third, it was seen in during the evaluation, the absence of key data on energy use and more often, the lack of pulling together data that is available to create a coherent map of energy in RMI was noted in the design.

 

Management Response: [Added: 2018/11/19]

Regarding the design of the project, lessons learned from the TE findings and recommendations can be taken into account for future CCM project development in the RMI. The TE pointed out the shortcomings in the Project Document, even though the evaluator has been advised that he should be evaluating the Project Brief not the Project Document. Since the Project Document, as it was the format at that time, is indeed some sort of an executive summary. The details are in the Project Brief. The evaluator instead focused on the Project Document that it does not provide detailed descriptions of what should be done, when in fact the details are in the Project Brief. The TE Report just mentioned the Project Brief 4 times (2 in the main report and 2 as footnotes).

 

In GEF-4 the GEF did not allow the use of GEF funds for hardware, just for soft interventions. Hence, the design covered mostly soft interventions such as capacity building (training courses, studies, researches), policy work, institutional capacity building, and technical assistance. The interventions involving the deployment of hardware are mainly funded through co-financing. Most of these are from the EU-funded RE-based energy generation (e.g., solar home systems). ADMIRE supplemented the EU-funded REP-5 project in RMI with soft measures and interventions.

Key Actions:

2. Recommendation:

Issue (Sub-chapter 3.7– Implementation after the Mid-Term Review: 2012-2015):

 

Paragraph 64: Unrevised outcome indicators.  The MTR suggested revised outcome indicators but the evaluation did not find any revisions to the LFA and to the outcome indicators for ADMIRE as recommended by the MTR. In fact, the subsequent Annual Work Plans reviewed continued to list the same six outcomes, with 18 activities (not counting learning and M&E).

 

Paragraph 65: Lack of additional support by UNDP experts.   The MTR also recommended focusing the training programs for the design, operation and maintenance of stand-alone and grid-connected PV systems in the country; the use of copra as form of payment for electricity tariff for the outer islands households SHS; and user training and awareness campaigns in rural schools and health centers; to explore and conclude opportunity of cooperation (e.g., North REP MoU) with Tobolar and ADB on copra oil processing. Any evidence that the recommendation for additional support to be provided by UNDP experts, together with close monitoring and technical support by UNDP office, was fulfilled was not seen in the evaluation.

Management Response: [Added: 2018/11/19]

Because of the delays in the project implementation (mainly due to the absence of project manager, which also led to the replacement of the original implementing partner by the MRD – Ministry of Resources & Development), the project implementation spilled over to the phase 5 of GEF. In GEF-5, it was acceptable with GEF to use the GEF funds for hardware. Because of that the ADMIRE project could include some hardware-based interventions, as also suggested in the project MTR. However, many of the hardware-based activities where not completed at the time of the TE. The TE should have considered the circumstances that constrained the design of the ADMIRE project design. It is easy to point out defects and shortcomings during the TE but most of the comments are retrospective, without giving due consideration of the circumstances that limited the project design to what it was. Another fault of course was the delayed retrofitting of the project activities after the MTR. Poor project management was the main factor as can be gleaned from the PIR reports.

Key Actions:

Latest Evaluations

Contact us

220 East 42nd Street
20th Floor
New York, NY 10017
Tel. +1 646 781 4200
Fax. +1 646 781 4213
erc.support@undp.org