Terminal Evaluation West Pacific Oceanic Fisheries Management

Report Cover Image
Evaluation Plan:
2012-2018, Philippines
Evaluation Type:
Final Project
Planned End Date:
03/2013
Completion Date:
01/2013
Status:
Completed
Management Response:
Yes
Evaluation Budget(US $):
22,796
The main lessons learned that could be applicable to other projects are: ? A good project document, being the manifestation of good planning, can lay a solid foundation for the subsequent success of the project. ? A baseline study (in this case status reports on national tuna fisheries) can evolve into a very useful product which can be significant, durable, and effectively serve several purposes. ? For best results, the best must be hired: project managers should strive to obtain consultant input of the highest quality, rather than that which is just adequate. ? For hard-to-achieve outcomes, the best approach could be to attack the issue early in the life of the project and constantly focus attention on the issue through a variety of interventions over an extended period. ? Knowledge management is a specialty that requires certain skills and experience that not everybody has (just as, for example, fishery stock assessment) - and professional advice may be required for effective knowledge management in a complex multi-country project.

Share

Document Type Language Size Status Downloads
Download document WPEA Project Evaluation TOR_ final.docx tor English 57.11 KB Posted 729
Download document WPEA Terminal Evaluation FINAL ver Jan 17.pdf report English 1551.42 KB Posted 1206
Download document Brief_WPEA Terminal Evaluation.pdf summary English 159.15 KB Posted 762
Title Terminal Evaluation West Pacific Oceanic Fisheries Management
Atlas Project Number: 00071925
Evaluation Plan: 2012-2018, Philippines
Evaluation Type: Final Project
Status: Completed
Completion Date: 01/2013
Planned End Date: 03/2013
Management Response: Yes
Corporate Outcome and Output (UNDP Strategic Plan 2018-2021)
Evaluation Budget(US $): 22,796
Source of Funding: Project funds
Joint Programme: No
Joint Evaluation: No
Evaluation Team members:
Name Title Nationality
Robert Gillett
GEF Evaluation: Yes
Evaluation Type:
Focal Area: International Waters
Project Type: MSP
GEF Phase: GEF-3
PIMS Number: 4084
Key Stakeholders: DA-BFAR
Countries: PHILIPPINES
Lessons
1.

The main lessons learned during the present evaluation that could be applicable to other projects are: ?

A good project document, being the manifestation of good planning, can lay a solid foundation for the subsequent success of a project. ?

A baseline study (in this case status reports on national tuna fisheries) can evolve into a very useful product which can be significant, durable, and effectively serve several purposes. ? For best results, the best must be hired: project managers should strive to obtain consultant input of the highest quality, rather than that which is just adequate. ? Knowledge management is a specialty that requires certain skills and experience that not everybody has (just as, for example, fishery stock assessment) - and professional advice may be required for effective knowledge management in a complex multi-country project. ?

For hard-to-achieve outcomes, the best approach could be to attack the issue early in the life of the project and constantly focus attention on the issue through a variety of interventions over an extended period. ?

In a regional project there is considerable merit in accepting that the participating countries are at different levels of development and have different speeds of progressing - and working with that reality with regards to expectations and annual work plans. ?

In getting the spectrum of stakeholders (including commercial interests) involved in various project activities, a constituency can be created which is aware of the need to see that outcomes are sustained – and can do something about it. ?

In the fisheries sector the time required to change established practices, institutions, legislation, and international relations should not be under-estimated – especially in countries where fisheries are of great importance.


2.

The Manager of the WPEA Project probably learned more lessons related to the project than anybody. His views have considerable credibility and therefore deserve some exposure. Some of his lessons learned (as given in project documents and e-mail communication on the subject24) are: ?

Not pushing each country to produce notable results, but rather encouraging them to meet their objectives ?

Allowing sufficient allowances to the in-country project team ?

Making frequent visits to each country in order to facilitate project activities and offer encouragement. ?

Ensuring sufficient project management support for project coordination. ? Communicating continuously between the project manager and country contacts and making face-to-face checks on project progress. ?

Recognizing that nothing can be done at one shot in executing this project. Implementation of a project in developing countries involves gradual progress – revisiting the topic, updating and refining the outputs are essential.


Findings
1.

The project was very well-formulated, as evidenced by the quality of the project document. As expressed by one stakeholder in the region: “the right medicine at the right time”. It appears that these favorable project preparations could be largely attributed to an appropriate skill mixture in the design team, especially having individuals with GEF experience, regional knowledge, and a great amount of technical expertise in tuna monitoring and management. Several positive features of the project’s design are mentioned in Section 5. Key aspects appear to be the project’s close association to the WCPFC, substantial co-financing, project coordination at the national level by senior government officials, and especially having a significant capacity development element in each component of the project. The major weakness in project design concerns the knowledge management component. A greater articulation of this element in the project document could have improved the result associated with this output.


2.

Project implementation was reasonably smooth, with an absence of major difficulties, as evidenced by the discussions at the Project Steering Committee. Minor difficulties were expected, encountered, and effectively dealt with. The day to day management of the project was effective, with responsible factors being the persistence of the Project Manager, the administrative infrastructure of the WCPFC, and ready access to admin and technical support from very competent individuals. There were no full-time staff on the project. This did not appear to negatively affect project implementation, although it placed considerable demand on the Project Manager. Such thin staffing arrangements were made feasible largely by the knowledge and experience of the Project Technical Adviser. With very few exceptions, the indicators listed in the logframe were implemented and/or achieved. The observer work in Indonesia and (to a degree) establishment of national knowledge management systems appear to be the only obvious ones where progress was less than anticipated. One important aspect of problem implementation is the approach used to achieve some of the more difficult project outcomes. The WPEA Project’s manner of working in these situations involved starting the intervention early in the project’s life, and focusing attention on the task over an extended period, with constant reinforcement from workshops, visits of specialists, attention of project supervisors, etc. – rather than going for a simpler one-time intervention or rushed interventions towards the end of the project. One person observed that this WPEA Project approach is the opposite of “throwing dollars at hard problems”.


3.

The project’s objective is “to strengthen national capacities and international cooperation on priority transboundary concerns relating to the conservation and management of highly migratory fish stocks in the west Pacific Ocean and east Asia (Indonesia, Philippines and Vietnam)”. Overall, it is concluded that the WPEA Project has made substantial progress towards this objective. The national capacities of the three project countries and their associated international cooperation in the conservation and management of fisheries for highly migratory fish stocks are certainly far stronger now than when the project began.The greatest achievement of the project is the remarkable progress made in moving towards almost all of the established outcomes. This is especially true for: ?

Improved knowledge of oceanic fish stocks and related ecosystems ?

Reduced uncertainty in stock assessments ?

National capacities in oceanic fishery monitoring and assessment strengthened ?

Participant countries contributing to management of shared migratory stocks ?

National capacities in oceanic fisheries management strengthened In this evaluation, three criteria were used in the evaluation to assess level of achievement of project outcomes: effectiveness, efficiency, and relevance.

The evaluation concludes: ?

The project has been extremely effective at achieving the outcomes established in the project document, certainly surpassing what could have been expected prior to the start of the project. The only disappointment involved the knowledge management systems – but this was to support an outcome that was effectively achieved by other means. ?

With respect to efficiency of achieving the seven outcomes, the project was very cost effective. An important cost efficiency aspect of the WPEA Project that repeatedly emerged in the evaluation was taking advantage of experience of setting up similar monitoring and management programmes in the Pacific Islands area. ?

A notable feature about outcomes established for the WPEA Project is that they were and remain all highly relevant to the country priorities. To a degree, the high relevancy was due to the talent of the architects of the WPEA project document, not the least of which is balancing country priorities with funding institution requirements. Using relevance, effectiveness, and efficiency as criteria, each of the seven outcomes established for the WPEA Project was rated on a scale given in the evaluation’s terms of reference.

The results of this rating are: 1. Improved knowledge of oceanic fish stocks and related ecosystems: “highly satisfactory”. 2. Reduced uncertainty in stock assessments: “highly satisfactory”. 3. National capacities in oceanic fishery monitoring and assessment strengthened: “highly satisfactory”. 4. Participant countries contributing to management of shared migratory stocks: “highly satisfactory”. 5. National laws, policies and institutions strengthened to implement applicable global and regional instruments: “highly satisfactory” for the Philippines, and “satisfactory” for Indonesia and Vietnam. 6. Key stakeholders participating in the project: “highly satisfactory”. 7. National capacities in oceanic fisheries management strengthened: “highly satisfactory”. The evaluation also examined each outcome from the perspective of sustainability. For two of the project outcomes, the risk to their sustainability is currently low. For others, the risk to sustainability is likely to be low at the conclusion of a planned follow-up project. Project results that have implications for global environmental benefits were considered by the evaluation. Bringing the tuna fisheries in the three project countries into an improved international management regime covering the entire WCPO would be an achievement of global scale. Considerable progress has been made by the project in various aspects of this process, but the project has not set up a mechanism to formally monitor the evolving situation.


Recommendations
1 It is recommended that GEF and UNDP prepare awareness material on the need for, and value of, their various M&E requirements (such as that for a detailed M&E plan) that is suitable for ordinary stakeholders ? who otherwise may feel that requirements are excessive.
2 It is recommended that to achieve desired project outcomes, project managers should strive to obtain consultant input of the highest quality, rather than what is just adequate. In this regard, two factors may be more important to consultants than the financial compensation: (a) advance planning (i.e. identifying/notifying individuals long in advance of period required), and (b) to the extent possible, encouraging their involvement in aspects of project design or fine tuning ? as was done on the WPEA Project. By involving the range of stakeholders (including commercial interests) in various project activities, the WPEA project in effect established a constituency aware of the need to see that some key outcomes are sustained ? and evidently willing to push for such continuation. In many cases this was done at little or no cost to the project, simply by inviting the range of stakeholders to relevant meetings and workshops. It is recommended that this type of intervention (i.e. stakeholder-related mechanisms to reduce sustainability risks) be considered in the implementation of similar projects
3 It would seem logical that a follow-up project should be at least partly oriented to reinforcing those outcomes established for the WPEA Project where risks to sustainability are greatest. In this respect, there are five outcomes where the evaluation judged the risk to outcome sustainability as ?moderately likely at present, decreasing to moderately unlikely at the end of a follow-up project ? Improved knowledge of oceanic fish stocks and related ecosystems ? Reduced uncertainty in stock assessments ? National capacities in oceanic fishery monitoring and assessment strengthened ? National laws, policies and institutions strengthened to implement applicable global and regional instruments ? National capacities in oceanic fisheries management strengthened Of the activities associated with the above outcomes, it is recommended that the tuna monitoring (especially the port sampling and annual catch estimation) be considered as top priority. This is because the monitoring has implications for many outcomes ? as well as affecting stock assessment in the entire western and central Pacific Ocean and even global environment benefits. As part of any follow-up dealing with this subject, in addition to project support for actual tuna monitoring, there should be considerable attention paid to developing mechanisms to assure that the governments of the three project countries will pick up the monitoring after GEF support ends. Following from the above, it is recommended that a cost-benefit study of tuna monitoring in the three countries be carried out. This would portray the expense of items such as port sampling and an observer programme against the value of, for example, the trade with the EU, which could be lost without government monitoring efforts. There are numerous advantages of having the private sector involved in the study, not the least of which is so they are well aware of the cost consequences should their governments neglect monitoring responsibilities. Also with respect to tuna monitoring, consideration should be given to expanding the coverage. This includes the scale (i.e. including artisan tuna fishing), the species (i.e. including neritic tuna), and the geographic area (i.e. including archipelagic waters)
4 Knowledge management was not a strong point of the WPEA Project. It was largely limited to creation of national e-mail address lists and a listing of project reports with a few dozen of those placed on the WPEA portion of the WCPFC website. This hardly equates to what was stated in the GEF and UNDP project documents: ?Widely publicize project findings and results to raise awareness on importance of oceanic fisheries management and highlight new information? and ?Results from the project will be disseminated within and beyond the project intervention zone through a number of existing information sharing networks and forums.? More could and should be done in this area in a successor project It is recommended that, early in the life of a follow-up project, a consultant communications professional be employed to formulate a communications strategy to be closely followed by the project
1. Recommendation: It is recommended that GEF and UNDP prepare awareness material on the need for, and value of, their various M&E requirements (such as that for a detailed M&E plan) that is suitable for ordinary stakeholders ? who otherwise may feel that requirements are excessive.
Management Response: [Added: 2014/02/28]

We are aware that the perceptions of the stakeholders on various M&E requirements are quite burden and to some extent excessive. We agree with the recommendation, but should not be limited to awareness material, since all these requirements can be seen online but rather constant capacity building and IEC on these requirements to our stakeholders.

Key Actions:

Key Action Responsible DueDate Status Comments Documents
Improve IEC on M&E requirements of UNDP and GEF to project stakeholders
[Added: 2014/02/28] [Last Updated: 2017/06/27]
UNDP 2014/12 Completed to be initiated in successive WPEA project History
2. Recommendation: It is recommended that to achieve desired project outcomes, project managers should strive to obtain consultant input of the highest quality, rather than what is just adequate. In this regard, two factors may be more important to consultants than the financial compensation: (a) advance planning (i.e. identifying/notifying individuals long in advance of period required), and (b) to the extent possible, encouraging their involvement in aspects of project design or fine tuning ? as was done on the WPEA Project. By involving the range of stakeholders (including commercial interests) in various project activities, the WPEA project in effect established a constituency aware of the need to see that some key outcomes are sustained ? and evidently willing to push for such continuation. In many cases this was done at little or no cost to the project, simply by inviting the range of stakeholders to relevant meetings and workshops. It is recommended that this type of intervention (i.e. stakeholder-related mechanisms to reduce sustainability risks) be considered in the implementation of similar projects
Management Response: [Added: 2014/02/28]

UNDP fully agrees with the recommendation on widening the range of experts / stakeholders to be involved in future project implementation

Key Actions:

Key Action Responsible DueDate Status Comments Documents
Ensure that next phase project would involve a more wider range of stakeholders
[Added: 2014/02/28] [Last Updated: 2017/06/27]
UNDP / WCPFC 2014/12 Completed To be initiated in successive WPEA project History
3. Recommendation: It would seem logical that a follow-up project should be at least partly oriented to reinforcing those outcomes established for the WPEA Project where risks to sustainability are greatest. In this respect, there are five outcomes where the evaluation judged the risk to outcome sustainability as ?moderately likely at present, decreasing to moderately unlikely at the end of a follow-up project ? Improved knowledge of oceanic fish stocks and related ecosystems ? Reduced uncertainty in stock assessments ? National capacities in oceanic fishery monitoring and assessment strengthened ? National laws, policies and institutions strengthened to implement applicable global and regional instruments ? National capacities in oceanic fisheries management strengthened Of the activities associated with the above outcomes, it is recommended that the tuna monitoring (especially the port sampling and annual catch estimation) be considered as top priority. This is because the monitoring has implications for many outcomes ? as well as affecting stock assessment in the entire western and central Pacific Ocean and even global environment benefits. As part of any follow-up dealing with this subject, in addition to project support for actual tuna monitoring, there should be considerable attention paid to developing mechanisms to assure that the governments of the three project countries will pick up the monitoring after GEF support ends. Following from the above, it is recommended that a cost-benefit study of tuna monitoring in the three countries be carried out. This would portray the expense of items such as port sampling and an observer programme against the value of, for example, the trade with the EU, which could be lost without government monitoring efforts. There are numerous advantages of having the private sector involved in the study, not the least of which is so they are well aware of the cost consequences should their governments neglect monitoring responsibilities. Also with respect to tuna monitoring, consideration should be given to expanding the coverage. This includes the scale (i.e. including artisan tuna fishing), the species (i.e. including neritic tuna), and the geographic area (i.e. including archipelagic waters)
Management Response: [Added: 2014/02/28]

UNDP agrees and in the next phase WPEA OFM Project the sustainability and improvements in tuna monitoring be given due attention. Suggested study on cost-benefits and expansion of coverage in tuna monitoring would really depend on the countries involved but UNDP will consider bringing this as part of the next phase

Key Actions:

Key Action Responsible DueDate Status Comments Documents
Ensure sustainability of WPEA OFM Project outputs in next phase project
[Added: 2014/02/28] [Last Updated: 2017/06/27]
UNDP / IP / countries 2014/12 Completed Sustainability measures are included in new WPEA project History
4. Recommendation: Knowledge management was not a strong point of the WPEA Project. It was largely limited to creation of national e-mail address lists and a listing of project reports with a few dozen of those placed on the WPEA portion of the WCPFC website. This hardly equates to what was stated in the GEF and UNDP project documents: ?Widely publicize project findings and results to raise awareness on importance of oceanic fisheries management and highlight new information? and ?Results from the project will be disseminated within and beyond the project intervention zone through a number of existing information sharing networks and forums.? More could and should be done in this area in a successor project It is recommended that, early in the life of a follow-up project, a consultant communications professional be employed to formulate a communications strategy to be closely followed by the project
Management Response: [Added: 2014/02/28]

UNDP will accommodate the communication specialist or any mechanism to develop communication strategy of new WPEA OFM Project

Key Actions:

Key Action Responsible DueDate Status Comments Documents
Improvement in Knowledge management System
[Added: 2014/02/28] [Last Updated: 2017/06/27]
UNDP / IP / countries 2014/12 Completed to be ensured in successive WPEA project History
Include in project workplan hiring of communication specialist
[Added: 2014/02/28] [Last Updated: 2017/06/27]
UNDP / IP 2014/12 Completed Included in PIF for new WPEA OFM Project History

Latest Evaluations

Contact us

1 UN Plaza
DC1-20th Floor
New York, NY 10017
Tel. +1 646 781 4200
Fax. +1 646 781 4213
erc.support@undp.org