Final Evaluation Report- "Towards Creating a Centre of Excellence for Sustainable Development of Small Island Developing States (SIDS) in Aruba

Report Cover Image
Evaluation Plan:
2017-2021, Trinidad and Tobago
Evaluation Type:
Final Project
Planned End Date:
06/2019
Completion Date:
02/2019
Status:
Completed
Management Response:
Yes
Evaluation Budget(US $):
15,000

Share

Document Type Language Size Status Downloads
Download document TOR consultancy for evaluation of COE Project.pdf tor English 599.68 KB Posted 64
Download document Final Evaluation COE SIDS Report adjusted figure numbers April 2019.pdf report English 481.90 KB Posted 75
Title Final Evaluation Report- "Towards Creating a Centre of Excellence for Sustainable Development of Small Island Developing States (SIDS) in Aruba
Atlas Project Number: cellence
Evaluation Plan: 2017-2021, Trinidad and Tobago
Evaluation Type: Final Project
Status: Completed
Completion Date: 02/2019
Planned End Date: 06/2019
Management Response: Yes
Corporate Outcome and Output (UNDP Strategic Plan 2018-2021)
  • 1. Output 1.4.1 Solutions scaled up for sustainable management of natural resources, including sustainable commodities and green and inclusive value chains
Evaluation Budget(US $): 15,000
Source of Funding: Project budget
Evaluation Expenditure(US $): 15,126
Joint Programme: No
Joint Evaluation: No
Evaluation Team members:
Name Title Email Nationality
Oscar Ernesto Huertas Diaz Mr. oscarhuertas77@gmail.com COLOMBIA
GEF Evaluation: No
Key Stakeholders: Government of Aruba; University of Aruba; The Netherlands Organization
Countries: TRINIDAD AND TOBAGO
Lessons
1.

Development of sensitization workshops was successful in this project because it allowed presenting the project, its scope, and expectations in front of the beneficiary countries and key stakeholders.


2.

The technical assistance that was made in countries sought solutions tailored to the needs, taking into account national priorities, the context, and needs. This is a pragmatic approach that proved to be successful beyond theoretical approaches.


3.

Development interventions in the SmallIsland Development States (SIDS) face many challenges, including the lack of capacity of some countries. In these countries the possibility to execute resources and implement activities efficiently is likely to be constrained by the limited availability of government staff, and the low capacities of the countries. All future interventions need to take these issues as challenges and establish a risk mitigation strategy.


4.

The designation of national focal points was considered a good practice for it allowed the project to move forward positively on logistical issues, empowerment, coordination and better communication with each of the targeted countries.


Findings
1.

The project objectives and expected achievements are relevant and well aligned with the development priorities and needs of the countries in all regions. These priorities have not changed significantly since the start of the initiative and continue to be centred on lessons exchange for sustainable development in targeted countries.


2.

The project design was ambitious as it aimed at sustainable development, covering different countries in different contexts. Future intervention should aim at more impactful initiatives in fewer countries, with the possibility to scale up organically.


3.

The approach to achieve the objectives did not have an explicit Theory of Change (ToC) with specific links between inputs, activities, outputs, and outcomes. There was a clear link between inputs, activities, and outputs, but not outcomes as the project design did not aim for outcomes.


4.

The project delivery rate was acceptable, but scale-up and replication of best practices need to be done in the transition plan to have the project become part of the knowledge base of the University of Aruba. 


5.

The internal project monitoring system involved twenty-two (22) project board meetings, but this was not always sufficient as it appeared to focus mainly on activities and expenditure levels. The tracking of progress based on delivery levels (i.e. rates of expenditure) can be insufficient, although a common practice of many donors, including the UNDP. 


6.

The primary outcome of the project was to raise awareness of sustainable practices, peer-to-peer learning, and knowledge management. The project made a difference regarding behaviour and consciousness of these concepts. Nevertheless, regarding how effective were the project activities in contributing to sustainable development strategies, the evaluation could not find substantial evidence on that regard.


7.

The project was successful in sharing case studies, providing technical assistance and training.


8.

As the primary project beneficiaries were institutions, it was difficult for the project design to include a human rights and gender strategy. Nevertheless, during the implementation, the project strived to include both men and women in all activities.


9.

Sustainability of the project is likely as the University of Aruba willabsorb key elements of the project, but explicit support of the new Aruban Government for the Centre of Excellence and its achievements would greatly benefit its sustainability.


Recommendations
1

Based on Conclusions 2, 3 and 9

For UNDP future projects should take special care with the design phase for greater relevance, the logic ofintervention and achievement of impacts. Projects must have a Theory of Change from the beginning, which identifies the chain of specific results, roles and responsibilities.

  • Future intervention designs need to identify the expected outputs and outcomes and to elaborate a Theory of Change that describes the path from inputs to results (outputs and outcomes). The Theory of Change should include assumptions and be linked with a risk log.
2

Based on Conclusion 2 and 4

For UNDP, and Government of Aruba: For the next phases, reduce the number of topics to choose from and have an agenda already from the onset (the supply):

  • Sustainable Tourism (high dependency SIDS)
  • Sustainable Energy
  • Water management
  • Coastal protections
  • Waste management
3

Based on Conclusions 3, 4 and 5

For UNDP: A success factor for projects is a monitoring and evaluation system, based on specific and verifiable result indicators, which allows greater control over processes and results.

  • All projects should go beyond the results framework and should elaborate a Monitoring and Evaluation system with a clear framework on progress and result indicators, sources of information and verification, roles and responsibilities (data upload and analysis), reporting procedures, etc.
  • Ideally, this monitoring and evaluation system should be online for all stakeholders to see the project's evolution and results in a dashboard.
  • All indicators need to have a baseline to analyze evolution and change.
4

Based on Conclusions 6 and 9

For UNDP: The design of projects for capacity building must be meticulous and must take into account the participants profile, topics, the methodology, the study load, the time available to achieve the desired results, and the evaluation. Future interventions need to take into account the fact that capacity building is a sustained in time process, that needs follow-up.

5

Based on Conclusion 7

For UNDP and Government of Aruba: Pilot projects need to have a replication strategy to share and scale-up lessons learned and best practicies amongst countries.

  • The online platform can move from an online information repository to a live community of practice.
  • This knowledge management strategy should include dedicated staff to do a follow-up, facilitate discussions, create and update expert directories worldwide (by topic), elaborate new case studies, etc. 
6

Based on Conclusion 9

For UNDP,  Government of Aruba and University of Aruba: The project needs a sustained in time intervention to be able to scale-up its benefits. Therefore, funding and technical support are vital.

  • The transition process from the project to the University of Aruba needs to start from presenting the Centre of Excellence results, a roadmap for sustainability (exit strategy) with a detailed timeline, specific roles, and a budget.
1. Recommendation:

Based on Conclusions 2, 3 and 9

For UNDP future projects should take special care with the design phase for greater relevance, the logic ofintervention and achievement of impacts. Projects must have a Theory of Change from the beginning, which identifies the chain of specific results, roles and responsibilities.

  • Future intervention designs need to identify the expected outputs and outcomes and to elaborate a Theory of Change that describes the path from inputs to results (outputs and outcomes). The Theory of Change should include assumptions and be linked with a risk log.
Management Response: [Added: 2019/06/24] [Last Updated: 2019/07/26]

Management notes the recommendation made by the evaluator regarding the development of a detailed theory of change (ToC) and linking this to the project results and risks of future projects.

Key Actions:

Key Action Responsible DueDate Status Comments Documents
The Country Office has noted this finding and has actioned this recommendation. The M&E capacity of the programme staff was enhanced through M&E training conducted in the period July-August of 2019. All future project documents will include a clearly defined Theory of Change (ToC) that will be linked to project results as well as the risks determined.
[Added: 2019/07/26] [Last Updated: 2020/02/07]
UNDP TTO Programme 2019/12 Completed The Country Office has noted this finding and has actioned this recommendation. The M&E capacity of the programme staff was enhanced through M&E training conducted in the period July-August of 2019. All future project documents will include a clearly defined Theory of Change (ToC) that will be linked to project results as well as the risks determined. History
2. Recommendation:

Based on Conclusion 2 and 4

For UNDP, and Government of Aruba: For the next phases, reduce the number of topics to choose from and have an agenda already from the onset (the supply):

  • Sustainable Tourism (high dependency SIDS)
  • Sustainable Energy
  • Water management
  • Coastal protections
  • Waste management
Management Response: [Added: 2019/06/24] [Last Updated: 2019/07/26]

The UNDP TTO does not agree with the above-mentioned recommendation. While at project design stage there was a clearly defined agenda for the missions, the adjustments to the objectives of the missions reflected the needs of the countries and allowed the missions to maximize relevance.

Key Actions:

Key Action Responsible DueDate Status Comments Documents
None required
[Added: 2019/07/26]
Not Applicable 2019/07 No Longer Applicable [Justification: The UNDP TTO does not agree with the above-mentioned recommendation. While at project design stage there was a clearly defined agenda for the missions, the adjustments to the objectives of the missions reflected the needs of the countries and allowed the missions to maximize relevance.]
The UNDP TTO does not agree with the above-mentioned recommendation. While at project design stage there was a clearly defined agenda for the missions, the adjustments to the objectives of the missions reflected the needs of the countries and allowed the missions to maximize relevance.
3. Recommendation:

Based on Conclusions 3, 4 and 5

For UNDP: A success factor for projects is a monitoring and evaluation system, based on specific and verifiable result indicators, which allows greater control over processes and results.

  • All projects should go beyond the results framework and should elaborate a Monitoring and Evaluation system with a clear framework on progress and result indicators, sources of information and verification, roles and responsibilities (data upload and analysis), reporting procedures, etc.
  • Ideally, this monitoring and evaluation system should be online for all stakeholders to see the project's evolution and results in a dashboard.
  • All indicators need to have a baseline to analyze evolution and change.
Management Response: [Added: 2019/06/24] [Last Updated: 2019/07/26]

For future project iterations the monitoring and evaluation systems will be developed specifically to measure those indicators and targets as determined in the results and resources framework (RRF).

Although the CO agrees with the development of an online monitoring and evaluation system this will be constrained by the availability of resources and is not essential to the successful implementation of the project and accomplishment of objectives.

Key Actions:

Key Action Responsible DueDate Status Comments Documents
The Programme Unit of the Country Office is currently undergoing training in monitoring and evaluation that will assist the unit in developing customized monitoring and evaluation plans for future projects that will specifically measure indicators and targets as identified in the projects results and resources framework.
[Added: 2019/07/26] [Last Updated: 2020/02/07]
UNDP TTO Programme Unit 2019/12 Completed Monitoring and evaluation training was conducted in the period July to August of 2019. This strengthened the M&E capacity of the programme staff. It is anticipated that with this improved understanding of M&E will allow the CO programme officers to develop customized M&E plans for their future programme initiatives. History
4. Recommendation:

Based on Conclusions 6 and 9

For UNDP: The design of projects for capacity building must be meticulous and must take into account the participants profile, topics, the methodology, the study load, the time available to achieve the desired results, and the evaluation. Future interventions need to take into account the fact that capacity building is a sustained in time process, that needs follow-up.

Management Response: [Added: 2019/06/24] [Last Updated: 2019/07/26]

While we agree in principle that cpacity building should ideally be sustained in time,  the design of this initiative was constained by certain key factors which precluded a medium to long-term time span for intervention. This included  a funding ceiling resulting in time limitations. However, in an attempt to ensure that sustainability of the outputs was addressed the transitioning of the knowledge base to a national research and learning institution was embedded in the project design.

 

Key Actions:

Key Action Responsible DueDate Status Comments Documents
None required
[Added: 2019/07/26]
Not Applicable 2019/07 No Longer Applicable [Justification: While we agree in principle that capacity building should ideally be sustained in time, the design of this initiative was constrained by certain key factors which precluded a medium to long-term time span for intervention. This included a funding ceiling resulting in time limitations. However, in an attempt to ensure that sustainability of the outputs was addressed the transitioning of the knowledge base to a national research and learning institution was embedded in the project design.]
5. Recommendation:

Based on Conclusion 7

For UNDP and Government of Aruba: Pilot projects need to have a replication strategy to share and scale-up lessons learned and best practicies amongst countries.

  • The online platform can move from an online information repository to a live community of practice.
  • This knowledge management strategy should include dedicated staff to do a follow-up, facilitate discussions, create and update expert directories worldwide (by topic), elaborate new case studies, etc. 
Management Response: [Added: 2019/06/24] [Last Updated: 2019/07/26]

The online platform has been transitioned to a national learning and research institution (University of Aruba) which is staffed with appropriate personnel who can be dedicated to maintaing an interactive knowledge management approach.

Key Actions:

Key Action Responsible DueDate Status Comments Documents
Not Applicable- As control of the online platform has transitioned to the University of Aruba the UNDP TTO cannot exert any influence as to how the platform is ultimately utilized.
[Added: 2019/07/26]
The University of Aruba 2019/07 No Longer Applicable [Justification: Control of the online platform has transitioned to the University of Aruba the UNDP TTO cannot exert any influence as to how the platform is ultimately utilized. ]
6. Recommendation:

Based on Conclusion 9

For UNDP,  Government of Aruba and University of Aruba: The project needs a sustained in time intervention to be able to scale-up its benefits. Therefore, funding and technical support are vital.

  • The transition process from the project to the University of Aruba needs to start from presenting the Centre of Excellence results, a roadmap for sustainability (exit strategy) with a detailed timeline, specific roles, and a budget.
Management Response: [Added: 2019/06/24] [Last Updated: 2019/07/26]

The UNDP CO agrees in principle that the project requires an additional intervention to ensure the sustainability of current results. However, this is largely determined by the availability of funding.

The UNDP CO agrees that an exit strategy is essential and wishes to note here that an exit strategy/transition plan was developed which addressed the specific aspects mentioned in the recommendation. This exit strategy/ transition plan was presented to the evaluator.

Key Actions:

Key Action Responsible DueDate Status Comments Documents
None required
[Added: 2019/07/26]
Not Applicable 2019/07 No Longer Applicable [Justification: The UNDP CO agrees in principle that the project requires an additional intervention to ensure the sustainability of current results. However, this is largely determined by the availability of funding. The UNDP CO agrees that an exit strategy is essential and wishes to note here that an exit strategy/transition plan was developed which addressed the specific aspects mentioned in the recommendation. This exit strategy/ transition plan was presented to the evaluator.]

Latest Evaluations

Contact us

1 UN Plaza
DC1-20th Floor
New York, NY 10017
Tel. +1 646 781 4200
Fax. +1 646 781 4213
erc.support@undp.org