Ten Island Challenge - Terminal Evaluation

Report Cover Image
Evaluation Plan:
2017-2021, Barbados
Evaluation Type:
Final Project
Planned End Date:
08/2019
Completion Date:
12/2019
Status:
Completed
Management Response:
No
Evaluation Budget(US $):
20,000

Evaluation Ratings:

 

 

 

1. Monitoring and Evaluation

Rating

2. IA & EA Execution

Rating

M&E design at entry

MU

Quality of UNDP Implementation - Implementing Agency

S

M&E Plan Implementation

MS

Quality of Execution - Executing Agency 

S

Overall quality of M&E

MU

Overall quality of Implementation / Execution

S

3. Assessment of Outcomes 

Rating

4. Sustainability

Rating

Relevance 

R

Financial resources

ML

Effectiveness

MS

Socio-political

ML

Efficiency 

S

Institutional framework and governance

ML

Overall Project Outcome Rating

MS

Environmental 

ML

 

 

Overall likelihood of sustainability

ML

Share

Title Ten Island Challenge - Terminal Evaluation
Atlas Project Number: 00089334
Evaluation Plan: 2017-2021, Barbados
Evaluation Type: Final Project
Status: Completed
Completion Date: 12/2019
Planned End Date: 08/2019
Management Response: Yes
UNDP Signature Solution:
  • 1. Energy
Corporate Outcome and Output (UNDP Strategic Plan 2018-2021)
  • 1. Output 1.5.1 Solutions adopted to achieve universal access to clean, affordable and sustainable energy
SDG Goal
  • Goal 7. Ensure access to affordable, reliable, sustainable and modern energy for all
SDG Target
  • 7.1 By 2030, ensure universal access to affordable, reliable and modern energy services
  • 7.2 By 2030, increase substantially the share of renewable energy in the global energy mix
  • 7.b By 2030, expand infrastructure and upgrade technology for supplying modern and sustainable energy services for all in developing countries, in particular least developed countries, small island developing States and landlocked developing countries, in accordance with their respective programmes of support
Evaluation Budget(US $): 20,000
Source of Funding: Project
Evaluation Expenditure(US $): 27,199
Joint Programme: No
Joint Evaluation: No
Evaluation Team members:
Name Title Email Nationality
GEF Evaluation: Yes
GEF Project Title: The Ten Island Challenge: De-risking the Transition of the Caribbean from Fossil Fuels to Renewables
Evaluation Type: Terminal Evaluation
Focal Area: Climate Change
Project Type: MSP
GEF Phase: GEF-5
GEF Project ID: 89334
PIMS Number: 5526
Key Stakeholders:
Countries: BARBADOS
Lessons
1.

There are a series of learned lessons that can be assimilated in the future for enhanced project planning and implementation.  These lessons are listed below, and they are linked to proactive recommendations in the next section of this report.

  • Design robustness, or lack of, has an indelible impact on implementation and monitoring.
  • If indicators are not robust, applicable and achievable, then implementation and monitoring could be negatively affected.
  • Throughout implementation, opportunities for adjustments are essential (adjustments to indicators for adaptive management, etc.) in order to properly reflect changes in exogenous conditions or to adjust for over ambitiousness.
  • When innovative implementing modalities are applied, these should be accompanied by proper architecture and guidelines on how this implementation would take place from design onward, how reporting is to take place (to whom, when, etc.) and how the decision-making process and assurance are carried out.
  • Gender equality promotion does not occur unless a specific gender approach that addresses fully the different needs of men or women is developed early on in design and in the implementation process.
  • The usefulness of an exit and sustainability strategy is related not only to content but also to when it is drawn and carried out. In order for an exit strategy to be appropriated by the relevant parties which will implement all or some of the strategy’s options, an exit/sustainability strategy needs to be drawn and carried out at a proper time and not at the very end of a project.

Findings
1.

Evaluation Ratings:

 

 

 

1. Monitoring and Evaluation

Rating

2. IA & EA Execution

Rating

M&E design at entry

MU

Quality of UNDP Implementation - Implementing Agency

S

M&E Plan Implementation

MS

Quality of Execution - Executing Agency 

S

Overall quality of M&E

MU

Overall quality of Implementation / Execution

S

3. Assessment of Outcomes 

Rating

4. Sustainability

Rating

Relevance 

R

Financial resources

ML

Effectiveness

MS

Socio-political

ML

Efficiency 

S

Institutional framework and governance

ML

Overall Project Outcome Rating

MS

Environmental 

ML

 

 

Overall likelihood of sustainability

ML

 


Recommendations
1

Design, especially of complex interventions, should contain in-depth knowledge of the areas and countries where interventions will take place, not only of the subject per se of the project being implemented, but also of the development context, political framework as well as assumptions and risks of the intervention.  There has to be extensive underpinning and analysis during design to harness strategic knowledge of the area(s)/country(ies) where a project would be implemented.  If the project is to be implemented by an organization from outside the country, then there have to be links to national/regional resource persons and expertise which can provide knowledge and information.

2

In order to enhance stakeholder engagement in implementation, and therefore impel improved results with a strong ownership potential, at design there should be a careful stakeholder analysis.  This analysis should not only include a list of potential stakeholders, yet it should contain an accurate typology, stakeholder mapping, and also indicate what are stakeholders’ concerns, incentives, goals and expectations vis-à-vis the anticipated results.

3

Outcome indicators should be robust, yet they should also be applicable and achievable within the context of implementation as well as realistic overall in terms of a project’s resources and implementation time.

4

There should be opportunities for the adjustment and reformulation of indicators and metrics package throughout the implementation process as a means for adaptive management.

  1. This should be communicated properly to the relevant partners, indicating how these changes are to be conveyed, what supporting data and information is needed for the changes, and what are the appropriate mechanisms. 
  2. This process should be communicated to relevant partners early enough in the implementation process.
  3. Project reformulations, changes, reforms and other such alterations need to be precise, and implemented as soon as early signs of failings manifest themselves.
  4. When these opportunities are not presented compellingly, UNDP should assert the need for this to take place in order to improve implementation.
5

Projects such as this one, with inherent complexities, pilot projects and interventions testing innovative implementation modalities should have a mid-term review, not only to give transparency to the achievements up to the midpoint assessment but also to act as a catalysing factor to adjust whatever needs to be adjusted at that point.

6

Projects that instrument innovative sorts of implementation modalities should have very clear guidelines from onset on all matters related to implementation.

  1. Much as more traditional implementation modalities have manuals and guidelines and procedures, these should also be present for non-governmental modalities and the same imbedded in design instruments. 
  2. Matters such as accountability, reporting, and oversight should be made specific and adapted to the modality.
  3. Particularly when implemented by entities outside a region or outside a country, staffing should include national project coordinators, in order to support the implementation as a whole in the particular country, attend to national implementation, and to anchor the project at the national level.
7

When training activities are designed, uptake and use (that is, monitoring of the application of knowledge acquired), as well as actual capacity built should be measured in order to understand the effects of these activities and to promote improvements and upscaling in the future. Indicators for increasing or building capacity should be robust measures of improved capacity, and at all possible, also capture use of conveyed capacity.

8

Multi country projects should have mechanisms for horizontal exchanges between and among the countries involved.  Not only to trade information but also to exchange lessons learned, problems and achievements between the different stakeholders from the different countries where a project is implemented.

9

 Risks within a project should not be underestimated, and a risk management framework should be drawn at design and reviewed continuously.  Once properly established, risks should be continuously monitored in order to promote whatever mitigation measures or adaptive management needs to be implemented.

10

Development projects such as this should have as its primary prospect to generate durable capacity at the national/regional levels. All efforts should be made to generate local capacity as well as introduce national issues in the resulting products. Although engaging with an entity from outside a country or region might be necessary to pilot interventions, follow up should be carried out as much as possible with national or regional institutions and/or expertise.

11

In order for projects to promote a gender equality approach, a strategy (that is an action plan based on gender analysis) should be set that addresses fully the different needs of men or women.  It should be comprehended that a “women in” approach (that is the participation of women in whatever fields or activities a project promotes) is not a comprehensive gender equality approach. Projects need to consider gender mainstreaming strategies from the onset of an intervention.

12

Exit and sustainability plans should be drawn earlier than at project finalization.  This is in order for these plans to be appropriated by the relevant parties which will implement all or some of the strategy’s options. An exit/sustainability strategy should also aim to make implementing partners and implementing agencies redundant, by accurately transferring capacity and the mechanisms that generate sustainability of achievements in the long term and not continue to be dependent upon them.

13

Projects should have a clear communication strategy. 

  1. A project’s communication strategy should be an ongoing process that generates buy-in, generates knowledge about the issues a project deals with as well as acknowledge its visibility. 
  2. A communication strategy needs to be accompanied by clear inputs where the different partners are identified (funders, implementing agency(ies), UN agencies involved). 
  3. A communication strategy should document and communicate issues, achievements, and challenges. 
  4. Also related to communication is the need to give proper visibility and transparency to all partners involved and what are their roles in a project.

Latest Evaluations

Contact us

1 UN Plaza
DC1-20th Floor
New York, NY 10017
Tel. +1 646 781 4200
Fax. +1 646 781 4213
erc.support@undp.org