Terminal Evaluation of the project "IW:LEARN Strenghtening IW Portfolio Delivery and Impact" PIMS 4219 FSP

Report Cover Image
Evaluation Plan:
2014-2017, RBEC
Evaluation Type:
Project
Planned End Date:
12/2014
Completion Date:
07/2014
Status:
Completed
Management Response:
Yes
Evaluation Budget(US $):
20,000

Share

Document Type Language Size Status Downloads
Download document Terminal-EvaluationReport-IWL-JUL311.docx report English 3504.80 KB Posted 833
Download document Terminal Evaluation Terms of Reference.docx tor English 27.71 KB Posted 531
Title Terminal Evaluation of the project "IW:LEARN Strenghtening IW Portfolio Delivery and Impact" PIMS 4219 FSP
Atlas Project Number: 00076562
Evaluation Plan: 2014-2017, RBEC
Evaluation Type: Project
Status: Completed
Completion Date: 07/2014
Planned End Date: 12/2014
Management Response: Yes
Focus Area:
  • 1. Others
Corporate Outcome and Output (UNDP Strategic Plan 2014-2017)
  • 1. Output 1.3. Solutions developed at national and sub-national levels for sustainable management of natural resources, ecosystem services, chemicals and waste
Evaluation Budget(US $): 20,000
Source of Funding:
Joint Programme: No
Joint Evaluation: No
Evaluation Team members:
Name Title Email Nationality
Anthony Jerome Hooten Evaluator, AJH Environmental Services ajh@environmentservices.com
GEF Evaluation: Yes
Evaluation Type:
Focal Area: International Waters
Project Type: EA
GEF Phase: GEF-1
PIMS Number: 4219
Key Stakeholders: UNDP, GEF, UNEP
Lessons
Findings
Recommendations
1 Clearly define IWL?s value added proposition and define its first, second and third-order priorities so that it has a clear delineation of its mission, core objectives, responsibilities and can master its core business as a first order priority. Once it achieves this, it will be in a stronger position to reach out and broaden its learning and knowledge exchange partnership beyond the GEF and its IW portfolio. IWL should not spread itself too thinly with diverse activities in its next project design.
2 Adequate resourcing, staffing and centralization of the IWL Project Coordination Unit In the future, it would be to IWL's benefit to have a single, co-located Project Coordinating Unit, where the Project Manager has all staff in the same office (at least initially) and especially with direct responsibility and control over IW:Learn's web presence, so this important function can be most efficiently managed given its crucial role in daily information sharing and communication for the growing IWL community.
3 IWL should always plan for multiple IWCs in IWL?s programming and project design, as more than one IWC may span the life on a given future project and should be always be appropriately planned for and resourced within each project.
4 In the future, contractual agreements between IAs and project execution should be carefully reviewed by the Steering Committee to ensure that there is adequate resourcing allocated to meet the administrative demand. Alternatively, the IWL PCU should engage a chief Operating Officer with the necessary skill set to provide all administrative functions, so that the project manager and the technical staff can be allowed to undertake their respective technical roles as knowledge managers.
5 Fiscal flexibility is needed within IWL?s program structure to better respond to unforeseen, emergent/evolving need as project implementation unfolds.
6 Re-examine and reprioritize elements of IWL?s Web presence ? both content and the technology options needed to present and manage it. Transitioning to the next IWL project presents the opportunity to perform a stock-taking of IWLEARN.NET. The PCU should have both authority and responsibility for engaging a highly qualified third party contractor to work with the current technical staff to help deliver a web presence that effectively manages IWL?s growing content and satisfies user needs.
7 Partner with existing networks (i.e. the EBM Tools Network (www.ebmtools.org) or with Open Channels (http://openchannels.org/) to take advantage of programs that already have expertise in the application and use of webinars to advance IW learning, including the need to focus on different learning styles, including adult learning.
8 Further develop the ?Impact Tracker? as a relational database application for Long Term monitoring of IWL progress. The current IWL3 PCU has created a flat file database (i.e. in spreadsheet format) to track IWL participants over time so that the longer term impact of IWL engagement might be better measured in the future. This is an excellent concept and something needed to improve metrics for IWL?s growing influence.
9 Content Visualization needs to have a specific focus and priority initially with clear terms of reference to produce specific outputs, and then build upon it. IWL needs to develop a specific terms of reference for visualization and start small to prove the concept specifically for IW and then broaden its visualization areas of concern. UNEP is embarking on an ambitious web and database initiative??UNEP Live??which is proposed to serve as a comprehensive information management system, and much of UNEP?s managed content, and that of its willing partners, is intended to eventually reside with this new and emerging platform . IWLearn should remain in contact with UNEP about the progress of the ?UNEP-Live? platform, especially the prospects that it could hold for data visualization. However, it is recommended that IWL maintain its web presence and knowledge base independently under the PCU and directly manage the mission-critical operations of its content. IWL should continue communication with UNEP about the progress of the ?UNEP-Live? platform and whether any future partnership directly with this platform could be mutually beneficial.
10 Targeted Messaging and Communication: IWLEARN?s future projects should consider a highly strategic approach to targeting specific messages to the right stakeholders. It is recommended that for future IWL projects that a communications firm with specific experience in targeting resource messages be contracted on a part-time basis to work with both the PCU and the Steering Committee. This is a cost-effective approach to consider in pushing key messages for IW to the right audience at the appropriate time.
1. Recommendation: Clearly define IWL?s value added proposition and define its first, second and third-order priorities so that it has a clear delineation of its mission, core objectives, responsibilities and can master its core business as a first order priority. Once it achieves this, it will be in a stronger position to reach out and broaden its learning and knowledge exchange partnership beyond the GEF and its IW portfolio. IWL should not spread itself too thinly with diverse activities in its next project design.
Management Response: [Added: 2014/12/05] [Last Updated: 2014/12/08]

Recommendation is relevant, and it will be taken into account when developing the next phase of the project.

Key Actions:

Key Action Responsible DueDate Status Comments Documents
1.1. The developers of the IW:LEARN fourth phase will be advised to look for a smaller number of activities and deepening of the intensity of the remaining ones.
[Added: 2014/12/05] [Last Updated: 2015/10/23]
UNDP HQ / PPG development team 2015/09 Completed Initial due date: March 2015
1.2 A knowledge management study conducted during the PPG of the next phase is set to examine priorities (as well as priority types) of interventions by the project to support portfolio experience sharing/replication
[Added: 2014/12/05] [Last Updated: 2015/10/23]
UNDP HQ / PPG development team 2015/09 Completed Initial due date: March 2015
2. Recommendation: Adequate resourcing, staffing and centralization of the IWL Project Coordination Unit In the future, it would be to IWL's benefit to have a single, co-located Project Coordinating Unit, where the Project Manager has all staff in the same office (at least initially) and especially with direct responsibility and control over IW:Learn's web presence, so this important function can be most efficiently managed given its crucial role in daily information sharing and communication for the growing IWL community.
Management Response: [Added: 2014/12/05] [Last Updated: 2014/12/08]

Recommendation is relevant, and it will be taken into account when developing the next phase of the project.

Key Actions:

Key Action Responsible DueDate Status Comments Documents
2.1. The developers of the next phase will be recommended to carefully appropriate budgeting for an expanded and co-located PCU
[Added: 2014/12/05] [Last Updated: 2015/10/23]
UNDP HQ / PPG development team 2015/09 Completed Initial due date: March 2015
2.2. The developers of the next phase will be recommend to look at ways to deepen the collaboration amongst the implementing agencies on the website and streamline new processes (as well as a revamp of the site).
[Added: 2014/12/05] [Last Updated: 2015/10/23]
UNDP HQ / PPG development team 2015/09 Completed Initial due date: March 2015
3. Recommendation: IWL should always plan for multiple IWCs in IWL?s programming and project design, as more than one IWC may span the life on a given future project and should be always be appropriately planned for and resourced within each project.
Management Response: [Added: 2014/12/05] [Last Updated: 2014/12/08]

Relevant, but the PIF for the next phase has been approved, contains funding for the IWC8 and 9 (no action needed).

Key Actions:

Key Action Responsible DueDate Status Comments Documents
3.1 The GEF approved PIF for the next phase contains funding for the IWC8 and 9 (no action needed)
[Added: 2014/12/05]
2014/12 Completed
4. Recommendation: In the future, contractual agreements between IAs and project execution should be carefully reviewed by the Steering Committee to ensure that there is adequate resourcing allocated to meet the administrative demand. Alternatively, the IWL PCU should engage a chief Operating Officer with the necessary skill set to provide all administrative functions, so that the project manager and the technical staff can be allowed to undertake their respective technical roles as knowledge managers.
Management Response: [Added: 2014/12/05] [Last Updated: 2014/12/08]

Recommendation is relevant, and it will be taken into account when developing the next phase of the project.

Key Actions:

Key Action Responsible DueDate Status Comments Documents
4.1 The planners of the next phase will be informed to consider executing arrangements and PCU staffing arrangements as per
[Added: 2014/12/05] [Last Updated: 2015/10/23]
UNDP HQ / PPG development team 2015/09 Completed Initial due date: March 2015
5. Recommendation: Fiscal flexibility is needed within IWL?s program structure to better respond to unforeseen, emergent/evolving need as project implementation unfolds.
Management Response: [Added: 2014/12/05] [Last Updated: 2014/12/08]

Recommendation is relevant, and it will be taken into account when developing the next phase of the project.

Key Actions:

Key Action Responsible DueDate Status Comments Documents
5.1 The developers of the next phase will be recommended to build greater buffer in certain activities (such as project twinning) to allow for adaptive management to emerging/shifting portfolio needs.
[Added: 2014/12/05] [Last Updated: 2015/10/23]
UNDP HQ / PPG development team 2015/09 Completed Initial due date: March 2015
6. Recommendation: Re-examine and reprioritize elements of IWL?s Web presence ? both content and the technology options needed to present and manage it. Transitioning to the next IWL project presents the opportunity to perform a stock-taking of IWLEARN.NET. The PCU should have both authority and responsibility for engaging a highly qualified third party contractor to work with the current technical staff to help deliver a web presence that effectively manages IWL?s growing content and satisfies user needs.
Management Response: [Added: 2014/12/05] [Last Updated: 2014/12/08]

Recommendation is relevant, and it will be taken into account when developing the next phase of the project.

Key Actions:

Key Action Responsible DueDate Status Comments Documents
6.1 The developers of the next phase will be furnished with the existing website recommendations made during the current phase.
[Added: 2014/12/05] [Last Updated: 2015/10/23]
UNDP HQ / PPG development team 2015/09 Completed Initial due date: March 2015
6.2 The developers of the next phase will be advised to consider a suite of options in terms of website restructuring and the study referenced in 1.2 will also examine (stock-take) iwlearn.net
[Added: 2014/12/05] [Last Updated: 2015/01/13]
UNDP HQ / PPG development team 2014/03 Completed
7. Recommendation: Partner with existing networks (i.e. the EBM Tools Network (www.ebmtools.org) or with Open Channels (http://openchannels.org/) to take advantage of programs that already have expertise in the application and use of webinars to advance IW learning, including the need to focus on different learning styles, including adult learning.
Management Response: [Added: 2014/12/05] [Last Updated: 2014/12/08]

Recommendation is relevant, and it will be taken into account when developing the next phase of the project.

Key Actions:

Key Action Responsible DueDate Status Comments Documents
7.1 The developers of the next phase will be supplied with these TE recommendations and advised to consider investigating new partnerships
[Added: 2014/12/05] [Last Updated: 2015/10/23]
UNDP HQ / PPG development team 2015/09 Completed Initial due date: March 2015
8. Recommendation: Further develop the ?Impact Tracker? as a relational database application for Long Term monitoring of IWL progress. The current IWL3 PCU has created a flat file database (i.e. in spreadsheet format) to track IWL participants over time so that the longer term impact of IWL engagement might be better measured in the future. This is an excellent concept and something needed to improve metrics for IWL?s growing influence.
Management Response: [Added: 2014/12/05] [Last Updated: 2014/12/09]

Recommendation is relevant, and it will be taken into account when developing the next phase of the project.

Key Actions:

Key Action Responsible DueDate Status Comments Documents
8.1 The next phase PCU will be furnished with the existing database and also advised to continue expanding it and populating with data sought from IWL beneficiaries.
[Added: 2014/12/05] [Last Updated: 2015/10/23]
UNDP HQ / PPG development team 2015/09 Completed Initial due date: March 2015
8.2 They will also be advised to consider new levels of analysis.
[Added: 2014/12/05] [Last Updated: 2015/10/23]
2015/09 Completed Initial due date: March 2015
9. Recommendation: Content Visualization needs to have a specific focus and priority initially with clear terms of reference to produce specific outputs, and then build upon it. IWL needs to develop a specific terms of reference for visualization and start small to prove the concept specifically for IW and then broaden its visualization areas of concern. UNEP is embarking on an ambitious web and database initiative??UNEP Live??which is proposed to serve as a comprehensive information management system, and much of UNEP?s managed content, and that of its willing partners, is intended to eventually reside with this new and emerging platform . IWLearn should remain in contact with UNEP about the progress of the ?UNEP-Live? platform, especially the prospects that it could hold for data visualization. However, it is recommended that IWL maintain its web presence and knowledge base independently under the PCU and directly manage the mission-critical operations of its content. IWL should continue communication with UNEP about the progress of the ?UNEP-Live? platform and whether any future partnership directly with this platform could be mutually beneficial.
Management Response: [Added: 2014/12/05] [Last Updated: 2014/12/09]

Recommendation is relevant, and it will be taken into account when developing the next phase of the project.

Key Actions:

Key Action Responsible DueDate Status Comments Documents
9.1 Priorities for visualization will be further investigated in the planned PPG study referenced in 1.2.
[Added: 2014/12/05] [Last Updated: 2015/10/23]
UNDP HQ / PPG development team 2015/09 Completed Initial due date: March 2015
10. Recommendation: Targeted Messaging and Communication: IWLEARN?s future projects should consider a highly strategic approach to targeting specific messages to the right stakeholders. It is recommended that for future IWL projects that a communications firm with specific experience in targeting resource messages be contracted on a part-time basis to work with both the PCU and the Steering Committee. This is a cost-effective approach to consider in pushing key messages for IW to the right audience at the appropriate time.
Management Response: [Added: 2014/12/05] [Last Updated: 2014/12/08]

Recommendation is relevant, and it will be taken into account when developing the next phase of the project.

Key Actions:

Key Action Responsible DueDate Status Comments Documents
10.1 The recommendation will be passed to the developer of the next phase, with a specific mention to consider hiring a communications individual in the PCU or a firm on retainer.
[Added: 2014/12/05] [Last Updated: 2015/10/23]
UNDP HQ / PPG development team 2015/09 Completed Initial due date: March 2015

Latest Evaluations

Contact us

220 East 42nd Street
20th Floor
New York, NY 10017
Tel. +1 646 781 4200
Fax. +1 646 781 4213
erc.support@undp.org