Project Terminal Evaluation -” Community-based Forest and Coastal Conservation and Resource Management”

Report Cover Image
Evaluation Plan:
2018-2022, Papua New Guinea
Evaluation Type:
Final Project
Planned End Date:
05/2020
Completion Date:
12/2019
Status:
Completed
Management Response:
Yes
Evaluation Budget(US $):
50,000

Share

Document Type Language Size Status Downloads
Download document TOR_Lead Consultant_UNDP_GEF4_Terminal-Evaluation.pdf tor English 418.50 KB Posted 354
Download document PIMS 3936_PNG_Terminal Evaluation report_final_23 April 2020.pdf report English 12438.01 KB Posted 387
Title Project Terminal Evaluation -” Community-based Forest and Coastal Conservation and Resource Management”
Atlas Project Number: 00062283
Evaluation Plan: 2018-2022, Papua New Guinea
Evaluation Type: Final Project
Status: Completed
Completion Date: 12/2019
Planned End Date: 05/2020
Management Response: Yes
UNDP Signature Solution:
  • 1. Resilience
Corporate Outcome and Output (UNDP Strategic Plan 2018-2021)
  • 1. Output 1.1.2 Marginalised groups, particularly the poor, women, people with disabilities and displaced are empowered to gain universal access to basic services and financial and non-financial assets to build productive capacities and benefit from sustainable livelihoods and jobs
  • 2. Output 1.4.1 Solutions scaled up for sustainable management of natural resources, including sustainable commodities and green and inclusive value chains
  • 3. Output 2.4.1 Gender-responsive legal and regulatory frameworks, policies and institutions strengthened, and solutions adopted, to address conservation, sustainable use and equitable benefit sharing of natural resources, in line with international conventions and national legislation
SDG Goal
  • Goal 14. Conserve and sustainably use the oceans, seas and marine resources for sustainable development
  • Goal 15. Protect, restore and promote sustainable use of terrestrial ecosystems, sustainably manage forests, combat desertification, and halt and reverse land degradation and halt biodiversity loss
SDG Target
  • 14.5 By 2020, conserve at least 10 per cent of coastal and marine areas, consistent with national and international law and based on the best available scientific information
  • 15.1 By 2020, ensure the conservation, restoration and sustainable use of terrestrial and inland freshwater ecosystems and their services, in particular forests, wetlands, mountains and drylands, in line with obligations under international agreements
  • 15.2 By 2020, promote the implementation of sustainable management of all types of forests, halt deforestation, restore degraded forests and substantially increase afforestation and reforestation globally
Evaluation Budget(US $): 50,000
Source of Funding: Global Environment Facility
Evaluation Expenditure(US $): 24,000
Joint Programme: No
Joint Evaluation: No
Evaluation Team members:
Name Title Email Nationality
John Carter International Consultant jcart@allstream.net
Katherine Yuave National Consultant kathyuave@gmail.com PAPUA NEW GUINEA
GEF Evaluation: Yes
GEF Project Title: Community-based Forest and Coastal Conservation and Resource Management in Papua New Guinea
Evaluation Type: Terminal Evaluation
Focal Area: Biodiversity
Project Type: FSP
GEF Phase: GEF-4
GEF Project ID: 3954
PIMS Number: 3936
Key Stakeholders: Conservation and Environment Protection Authority, East and West New Britain Provincial Administrations, NGOs and CBOc
Countries: PAPUA NEW GUINEA
Comments:

Terminal Evaluation of the project.

Lessons
Findings
1.

3.1 Project Design / Formulation

It is important to point out that the evaluators, in looking at the features of project design and formulation, have focused on the “workability” and relevance of project objectives, outcomes, targets, and indicators (the project concept and structure) at the time the project was designed (but with the benefit of hindsight). There is a conscious effort to avoid judgments about project implementation and the achievement of results (these are addressed in Sections 3.2 and 3.3). The purpose of this filtering is to avoid pre-empting the observations on actual project implementation modalities and achievement of results. Clearly, any design flaws or weaknesses could (or would) have implications for whether or not activities can be effectively implemented, and related results achieved (and the MTR picked these up and led to revisions in targets; discussed later).


Tag: Relevance Programme/Project Design

2.

3.1.1 Analysis of Results Framework

The Logical Framework Analysis (LFA) is a critical project planning and implementation faremwork, since it is intended to create a frame for all project activities (justifying their inclusion), and clarifying their trajectory in a coherent manner towards relevant objectives. Whatever remains as unclear or illogical in a project LFA then remains open to varying individual interpretations and associated lack of accountability.

In looking at the PRF, the most critical elements are the targets, since their achievement is the whole point of the project, and these would be expected to contribute to higher level outcomes and possibly lead to positive impacts (longer-term results). Therefore, each of the project objectives and outcomes and their planned targets (as defined in the PRF, shown in italics below) are examined below for workability and relevance. The text from the RRF is interpreted literally; where the meaning is unclear, this is pointed out and the possible interpretations are then examined


Tag: Effectiveness Relevance Project and Programme management

3.

Overall Project Objective

Develop effective natural resource management and financing systems for community conservation areas.

As an overall objective, this is clear and relevant, with a focus on target areas (community appropriate), and emphasis given to resource management and financing systems (key essentials for conservation). The meaning of “effective” is clarified in subsequent expected outcomes.


Tag: Natural Resouce management Effectiveness Financial Inclusion

4.

Targets:

A comprehensive and integrated policy and regulatory framework for CCAs is enacted by end of year 2; supported by a coordinated whole-of-Government decision-making mechanism operational by year 3.

This target reflects an appropriate understanding of the importance of policies and regulations to embed conservation principles in Government action and decision-making (creating direction, predictability, and accountability in the process – in theory). However, it seems (now) that the timeline is extremely ambitious (unrealistic), given the need for legal review and extensive consultations and having to address conflicting interests. A whole-of-Government decision-making mechanism (for PA selection, design, implementation, monitoring, and financing) would be elusive at the best of times, given the need for institutional re-structuring required for agency leadership (CEPA) in this whole process, this was implemented through GEF4


Tag: Natural Resouce management Protected Areas Site Conservation / Preservation Rule of law Country Government

5.

Landowner commitment sufficient to ensure effective management and conservation of CCAs as measured at end-project.

This target is obscure, without a clear meaning of “commitment” or “sufficient”. It defaults to some measure of effective management and conservation in the project CCAs and assumes that this effectiveness is due to landowner commitment. No doubt, landowner commitment is needed, but all the elements of this are unspecified (community endorsements, committees established, self-financing schemes, monitoring and enforcement, etc.).


Tag: Natural Resouce management Protected Areas Site Conservation / Preservation Effectiveness

6.

Outcome 1: National enabling environment for a community-based sustainable national system of protected areas (PAs) containing globally and nationally significant biodiversity.

1.1 Improved whole-of-Government systems and processes for making land-use decisions to avoid degradation and conversion of PAs. [ the indicator was revised after the MTR to be : Indicator 6: Legal status of CCAs and legal tools being applied to provide protection of CCAs]

In the final year of the project, no established CCA suffers any direct impact due to landuse/ conversion decisions, or indirect impact due to adjacent or upstream development activity34.

This is a totally appropriate target, which captures the essence of the whole project (CCAs established and functional). It does place emphasis on the Government process35, and assumes (it is supposed) that local communities fully respect Government, and local community decisions regarding land use, and do not encroach on their own CCAs. Of course, the whole land use planning element is complicated36 by the fact that PNG land is almost completely owned by customary landowners, and no detailed and firm land use plans (generated by Government) really have to resonate with local communities (whole-of-Government systems in place, or not). Further, there are many cases of bad (inappropriate) land use plans being made by local communities (handing over their land for palm oil development, or logging37, for example, for meager compensation38) and whether certified and environment friendly (or not), going back on such land conversions is extremely unlikely39.


Tag: Biodiversity Protected Areas Site Conservation / Preservation Effectiveness National Institutions

7.

1.3. Integrated policy framework to support mainstreaming of environment conservation issues within whole-of-Government and sectoral decision-making processes developed and being implemented.[the indicator was revised after the MTR to be Indicator 7:

Evidence or degree of mainstreaming of protected areas within different national policies and development strategies] By year 3, policy frameworks for (i) SEAs, (ii) Sustainable agriculture and (iii) PA Financing have been developed, endorsed by CEPA and submitted to the Government for adoption41.[

This is not so different from 1.1 above. These are all appropriate elements for policy frameworks, but forestry and fisheries are missing (addressed elsewhere?). SEAs and sustainable agriculture are slightly tangential to PA development but are assumed to be in the remit of CEPA (and workable, as they are only policy frameworks, which are not overhwleming).


Tag: Protected Areas Site Conservation / Preservation Effectiveness Rule of law National Institutions

8.

1.4. Integrated legal framework to ensure effective planning and regulation of development and conservation activities.

A single integrated Act providing for a statutory authority with increased scope for PA management including benefit sharing arrangements; integrated CEPA Act to reconcile inconsistencies in current body of law, and introduce reforms.

This is a totally appropriate target (if ambitious), creating the anchor for the PA system.


Tag: Protected Areas Site Conservation / Preservation Effectiveness Rule of law National Institutions

9.

1.5 Integrated policy framework to support sustainable financing of PAs developed and evidence of success through increased funds for PA establishment and management. [ the indicator was revised after the MTR : Indicator 11: Level of Government funding available for PA establishment, management.

By end-project, available funding meets minimum requirement for gazetted CAs, as measured by the PA Financing Scorecard42.

This is also a totally appropriate target, recognizing the critical need for sustainable financing of conservation areas.


Tag: Protected Areas Effectiveness Rule of law Financial Inclusion National Institutions

10.

1.6. Strengthened institutional and technical capacities in relevant Government agencies, linked to a framework of national core competencies to support effective conservation planning and service delivery in PAs.

By end-project, CEPA institutional and technical capacity scores are rated as ‘Sufficient’ or ‘Adequate’ across all key competencies; institutional scores for other relevant agencies (including local governments) show increases on average between project mid-term and end-project assessments.

Of course, Government capacity is a requisite for conservation planning and implementation. However, this target defaults to a “proxy” measure (a scoring system, as discussed peviously), rather than identifying core competencies and highlighting a coherent capacity-building programme.


Tag: Protected Areas Site Conservation / Preservation Effectiveness Relevance Rule of law Institutional Strengthening National Institutions

11.

Outcome 2: Community-managed Conservation Areas identified and established in the Owen Stanley Range and New Britain.

2.1 At least 1,000,000 hectares added to the national system of community-managed protected areas through the establishment of new financially and ecologically viable Conservation Areas and/or conversion of existing Wildlife Management Areas to Conservation Areas.[the indicator was revised after the MTR: Hectares of new Protected Areas established under the new community conservation area framework - By year 5 at least 500,000 hectares added in pilot sites

It is totally appropriate to set a numerical target, for sure, although the rationale for this number is not clear (it seems arbitrary). The target could have been expressed as a percentage of the existing total conservation area in PNG. Also, the distribution of CAs between the Owen Stanley Range and New Britain could be more clearly specified. The option to convert WMAs to CAs makes the whole project workable (building on existing initiatives, which is a safe default position, although possibly blurring the line between the features of existing protected areas and new features facilitated by the project) 43.


Tag: Protected Areas Site Conservation / Preservation Effectiveness

12.

Outcome 3: Conservation Area Management Planning and Partnership Agreements with Communities. 3

.1 Conservation Areas effectively managed according to the requirements of their respective Management Plans, with 20% increase in METT scores over the project lifetime. [The indicator was deleted after the MTR].

By end-project, METT scores for each CA increase by at least 20% over initial baseline44.

There is the same issue here, with this target, with defaulting to METT scores. It would be better to be explicit about the attributes of effective CA management as required from the community. METT scoring can be subjective and setting a 20% increase seems arbitrary. This target also assumes that Management Plans are already, or will be, in place. It seems that developing workable management plans should be a target in itself. As noted previously, some measure of conservation effectiveness (such as stable or increasing habitat areas) should be explicit here.


Tag: Site Conservation / Preservation Effectiveness

13.

3.2. Service delivery, community development and economic development outcomes as specified in the Partnership Agreement being achieved45.

Within 2 years of CA establishment or by end-project (whichever is sooner) CAMCs report satisfactory compliance with service delivery, community development and economic development outcomes as specified in the respective Partnership Agreements.[This indicator was revised after the MTR: Partnership agreements (bilateral, tripartite or more) that are intended to support, in a demonstrable way (i.e., through provision of finance, alternative livelihood solutions, etc), establishment, and management of PAs signed and demonstrably implemented as measured by: a) number of agreements and b) demonstrated outcome of agreement]


Tag: Protected Areas Site Conservation / Preservation Effectiveness Relevance Financial Inclusion

14.

Outcome 4: Capacity development and support for implementation of CA Management Plans.

4.1 Capacity development and support for Conservation Areas stakeholders to enhance project implementation and delivery of project outputs. [ indicator was revised after the MTR: Institutional and individual/ technical capacities of Provincial and local level governments to ensure effective delivery of key project outputs]

Provincial and local level government (LLG) institutional and technical capacities to support establishment and management of CAs increases by at least 20% two years after establishment of each CA. Overall institutional capacity increases to at least 56.4%, and individual capacity increases to 50%.

Capacity development, as a project component, is a usual “catch-all” for all other project activities required to deliver outcomes. With this target, there is the same issue as noted above, with a reliance on a scoring system over time, and the attributes of provincial and LLG capacity remain obscure. It would be better to clearly define their roles in CA development and management and set up capacity-building accordingly. Also, measuring individuals seems inappropriate for an outcome target (government staff may change over time).


Tag: Site Conservation / Preservation Effectiveness Capacity Building Institutional Strengthening National Institutions

15.

4.2. Capacity development plans for landowners delivering greater capacity and improved outcomes from project activities [Indicator was revised after the MTR: Capacity of landowners to manage conservation areas and associated livelihoods/ service delivery activities].

Landowner groups have sufficient capacity to implement livelihood and service delivery activities46.

Targeting the CA communities for capacity-building is appropriate, but there is an odd single focus on livelihoods and service delivery, ignoring the requirement of communities to undertake specific conservation activities. Further, landowner group capacity for implementing livelihood and service delivery activities (assuming they are agreed by the community and workable) is so dependent on external factors, as well as internal capacity, such that there are probably many risks and challenges inherent in this target.


Tag: Effectiveness Jobs and Livelihoods Capacity Building

16.

4.3. Linking of livelihood, health and population issues with CA resource management.

All communities/ landowner groups involved in functioning community conservation areas enjoy documented improvement in at least two social service areas47.

This seems completely unworkable, since the link between conservation and social service delivery is obscure at the best of times, and very dependent on community structure, proximity to services, and different levels of Government support (as suggested above). Further, the target is vague (no numerical targets), and there is no actual capacity-building noted, to support community quality of life. It would be difficult for the project to be accountable for this target.


Tag: Site Conservation / Preservation Effectiveness Jobs and Livelihoods

17.

4.4. Learned lessons from the conservation management systems developed under the project are incorporated into policy and regulations and help improve management of the national PA system48.

Project demonstrates tangible and quantifiable increase in systemic, institutional and technical capacities by end- project.

Yes, of course, it makes sense to capture the lessons learned from the project. But it is unlikely that, within the project period, these lessons would be feeding an ongoing process of updating and revising policies and regulations, when the main project effort (Component 1) is developing these things in the first place, over a 2-3 year period. It is more likely that the project would learn from its own operational challenges and adjust its approaches accordingly.


Tag: Protected Areas Site Conservation / Preservation Knowledge management

18.

3.1.2 Assumptions and Risks

Assumptions and risks (correctly understanding them) form the operational context for effectively planning and implementing projects. These are always identified and addressed during the design phase of projects, and should be examined regularly and adjusted (and responded to) throughout the lifetime of a project. The GEF4 assumptions and risks are examined here.


Tag: Site Conservation / Preservation Relevance Sustainability Resource mobilization Ownership Risk Management

19.

State of Papua New Guinea continues to support PAs by all means against biodiversity threats. This is a very vague statement (almost meaningless) and ignores the reality that PA integrity will almost always depend on full local community engagement50, and there is only so much that the Government can do at the local level (technical and financial support being critical there).

External threats and pressures (e.g. climate change impacts, encroachment) do not adversely affect the status of biodiversity resources within CCAs. Climate change impacts would be quite slow in manifestation, and actually cannot be mitigated at the local level (and adaptation in specific CAs would be confined to village infrastructure and services, not natural habitats). Encroachment is a much more likely and threatening risk (expanding populations, and incursions from outside the area, which are still occurring in many areas).

Benefits of alternative land uses (e.g. agriculture, mining) do not drastically increase after agreement to set up CCAs is achieved. Well, this risk (ongoing oil palm development, logging, mining, and building roads) will always be there, and can only be countered by strict zoning and enforcement, and more importantly, alternative livelihoods for communities that are more compelling and bring more revenue than land deals and jobs associated with agriculture (like oil palm), logging, and mining. Further, these alternative livelihoods need to be self-financing (not always dependent on projects and subsidies)


Tag: Climate Change Adaptation Biodiversity Protected Areas Site Conservation / Preservation Sustainability Risk Management

20.

Inclusion of references to the National PA system on paper translate into tangible policy and financial support on the ground. This will require political will and commitment, and public awareness and advocacy related to conservation principles. The assumption correctly forms a key set of actions within the project (policy consultations and development of legislation); however, it is known that political concerns about the financial implications of new policies can stifle policy implementation.


Tag: Protected Areas Site Conservation / Preservation Sustainability Risk Management

21.

Changes in external factors, e.g. fiscal position of Provincial Governments and LLGs, does not adversely affect service delivery. This is a risk correctly identified, and almost impossible to manage, except to have back-up plans for project operations that reduce engagement with Provincial Governments, Districts, and LLGs. However, the key to effective conservation at the local level is to have increasing technical and financial support from lower levels of government.

 


Tag: Sustainability Risk Management Country Government

22.

3.1.3 Lessons from Other Relevant Projects Incorporated into Project Design

The project was exemplary in using previous conservation experience (in PNG and elsewhere) to inform project design. In the ProDoc, there is a very detailed and accurate picture of all the barriers to effective conservation in PNG. There is reference to the conventional PA approach being inadequate and unrealistic for PNG’s needs, which is then further elaborated with specific examples52. Customary land ownership is noted as both the challenge and opportunity (the experience to date indicates WMAs may not be working). Principles and approaches from other areas are incorporated into the dialogue (on the problem to be addressed); for example the CAR (comprehensive, adequate, representative) criteria from Australia may have some guidance. There is reference to the REDD payment issues and controversies. The experience with the Programme of Work on Protected Areas (CSIRO conservation needs approach) regarding the need for collecting biodiversity information and setting up a comprehensive spatial system is documented. There is reference to the ICAD project, with an increased emphasis on moral incentives as a way to engage local communities in conservation. And, there is reference to Government-supported initiatives that work against conservation principles53.


Tag: Natural Resouce management Protected Areas Site Conservation / Preservation Sustainability Knowledge management Programme/Project Design Project and Programme management

23.

3.1.4 Planned Stakeholder Participation

In the original project design, there was adequate and due emphasis given to identification of project partners (at a generic level; all levels of government, local communities, private sector, NGOs, etc.) and a verification of project direction, objectives, roles and responsibilities in an Inception Workshop. Further, there were other modalities, such as frequent meetings/ discussions, a Tripartite Review, Project Board, and field monitoring that would allow engagement of stakeholders. These were all obvious and necessary mechanisms to encourage stakeholder participation (and common to GEF and development projects), although the specific stakeholders were not actually identified in the ProDoc. Further, there was no actual Inception Report to document stakeholder perceptions and create accountability for meeting various expectations of the project. Some “givens” would be the requirement to engage NGOs, to fill the breech left by inadequate government services at the local level, and the possibility of CSR-type support from the private sector (these were implicit in the project design). Actual stakeholder participation in the project is discussed in Section 3.2.2.


Tag: Effectiveness Sustainability Bilateral partners Country Government Civil Societies and NGOs Private Sector

24.

3.1.5 Replication Approach

The replication approach described in the ProDoc focuses on national policy (to anchor appropriate conservation approaches) and showing the effectiveness of PES schemes in selected areas. It is a very simple and optimistic approach (yet, these are still requisites for replication). While it is true that good examples in some areas might spur similar approaches elsewhere, this will only occur if the good examples are correctly analyzed and disseminated in a timely manner to the right stakeholders, within the project time period56. A detailed end-of-project review with all project participants was proposed by several people during the evaluation, but this would not lead to any clear replication strategy or accountability for promoting or implementing it, just the hope that it would be picked up by subsequent projects or other initiatives


Tag: Site Conservation / Preservation Integration Knowledge management

25.

3.1.6 UNDP Comparative Advantage

It was implicit (at the project design phase) that UNDP PNG’s previous project experience in PNG and linkages to the GEF mechanism, as well as knowledge of available and relevant technical expertise and associated procurement systems, were expected to provide a comparative advantage (relative to other entities) in delivery of the GEF4 project. It was also implicit that UNDP financial resources could be used to fill gaps, as needed, which might occur with government partners


Tag: Global Environment Facility fund Procurement Strategic Positioning UN Agencies

26.

3.1.7 Linkages Between the Project and Other Interventions Within the Sector

Two other interventions are noted in the ProDoc, including the Kokoda Initiative (involving some REDD approaches) and working with the oil palm industry on an Oil Palm Code of Practice, DEC (CEPA) being the intermediary for these two other initiatives. There is an indirect linkage mentioned with the Coral Triangle Initiative in Kimbe Bay, which assumes that any success with forest conservation in that area would bring benefits to water and habitat quality in Kimbe Bay. Note that while the latter does not require any institutional linkages or coordination, the former two (Kokoda and oil palm industry) would be expected to be labour-intensive and require detailed coordination with stakeholders elsewhere. These expected linkages are explored further in Sections 3.2 and 3.3.


Tag: Effectiveness Partnership Private Sector

27.

3.1.8 Management Arrangements

While UNDP submitted the GEF proposal and agreed to provide services such as procurement of staff and consultants, and make access to the global roster of consultants, the Executing Agency for the project is the Department of Environment and Conservation (now CEPA). The project was designed with a Project Management Unit embedded within CEPA, with a Project Advisory Board (with representatives from various government agencies, Provincial Government and LLGs, and NGOs, as well as resource sector entities) providing input to the Deputy Secretary of Sustainable Environment Programs. The National Project Director was identified as the Executive Manager of the Terrestrial Environments Division (identified elsewhere as Executive Manager of Conservation Planning, helped by a Program Coordinator).

 


Tag: Global Environment Facility fund Human and Financial resources Procurement Programme/Project Design Project and Programme management Country Government Civil Societies and NGOs

28.

3.2 Project Implementation

GEF terminal evaluation report formats are generally prescribed to capture design, implementation, and results observations in a set sequence. This works for the overall sequence of project design-toimplementation-to-results; however, within the section on project implementation, the usual sequence (as noted in the TE ToRs and the Inception Report) is not logical and potentially leads to disjointed and repetitive documentation of observations. Therefore, the observations on project implementation, in this report, go through the following sequence:

  •  actual UNDP and implementing partner management actions related to implementation, including management structure, planning, coordination/communication, and addressing operational issues;
  •  management of partnership arrangements (all the other stakeholders involved in project implementation);
  •  the use of monitoring and adaptive management to keep the project on course; and,
  •  financial management effectiveness.

Tag: Global Environment Facility fund Implementation Modality Monitoring and Evaluation Programme/Project Design Project and Programme management

29.

3.2.1 UNDP and Implementing Partner Implementation / Execution, Coordination, and Operational Issues

As noted above, management effectiveness depends on the nature of the project management structure, the clarity and detail in planning, effective coordination and communication, and an ability to address operational issues in near-real time. These elements are discussed here.


Tag: Effectiveness Efficiency Human and Financial resources Implementation Modality Project and Programme management Country Government Coordination

30.

The GEF4 project was to be advised by and accountable to a Project Advisory Board (PAB). This was to include representatives of appropriate stakeholders, but the Inception Workshop (for which there is no report) tinkered with representation, and NGOs were taken off and private sector representatives added (an NGO representative was put back on the PAB after the recommendation of the MTR, with some still bitter feelings about being “kicked off”). The PAB met infrequently up to 2015, then not at all in 2016, three times in 2017, once in 2018, and once in 201960. It is implicit in the PIRs that the PAB, when they did meet, examined and approved workplans, but there is no evident “minuting” of PAB observations and decisions, and it appears that the comfortable management structure developed between UNDP and CEPA was able to make project decisions, without any obvious impedance from either the UNDP Country Office, the PAB, or the Regional Office in Bangkok61. A Roundtable Group was to be established to give voice to the various partners and stakeholders (especially in New Briatin), but this has yet to be implemented (some issues with selection of members)62.


Tag: Global Environment Facility fund Oversight Project and Programme management Country Government Civil Societies and NGOs Private Sector

31.

The operational situation in Papua New Guinea presents a unique combination of challenges, all of which have been experienced by the GEF4 project. These include extremely difficult transportation logistics64, communication challenges, lack of convergence of the different levels of government on understanding of mandates and funding responsibilities, lack of adequate staff numbers in government agencies (and inadequate institutional and technical capacity, in some cases, especially at the Provincial level65), the same in NGOs and CBOs, and an ongoing culture of lack of documentation of discussions and decisions. In theory, all of these would suggest that manag ement and operational effort needs to be as close-to-theground as possible, where the evidence of the project outcomes and impacts must be evident (in the districts and at the WMAs…66 ). The project seems to have grappled adequately with these operational challenges, and in the last three years has shifted attention and effort to New Britain, where the project gains can be anchored, while national level policy initiatives languish somewhat67. In this scenario (groundlevel activity), however, there has been a reliance on intermediaries (NGOs and CBOs), who have had a varying level of project understanding and competencies, creating a mosaic of different actions in time and space (successes and challenges, discussed further in Sections 3.2.2 and 3.3.1). Especially in this operational environment, there is a reliance on the chain of project partners (project staff, national government, provincial and district governments, NGOs/CBOs, and local communities), from top to bottom, to get things started and finished, and all must have clarity of plans, schedules, and expected outcomes well understood, in order to deliver anything. If anyone stumbles within this chain or misunderstands the tasks (or is not accountable), then actions cannot start, or may be disrupted, or may not get finished. Management responses (if there is agreement on what to do) might take 2-3 months to be mobilized to address such chain-of-action issues68.


Tag: Challenges Effectiveness Operational Efficiency Project and Programme management Civil Societies and NGOs

32.

Local communities had their own management challenges, including setting up committees (determining composition), understanding the CCA process, working out relationships with intermediaries (if there were any), and fitting in project tasks with everything else going on in the community. Financial management and reporting were challenges in a few communities (Klampun and Tavalo have had direct funding, but delays in expenditure reporting on their side have caused some late disbursements on the project side).


Tag: Site Conservation / Preservation Effectiveness Relevance Human and Financial resources Project and Programme management

33.

3.2.2 Partnership Arrangements

The GEF4 project has involved a huge group of partners, from the national level of Government down to individual communities in New Britain. As such, there was more discussion of partners and stakeholders during evaluation meetings and consultations than any other evaluation topic. Almost everyone had an observation or opinion about a general category of partners (for example, the provincial government, or CEPA, or NGOs/CBOs, etc.) or specific individuals. The evaluators have noted all these observations, but have rolled the observations up to project management of partnerships, and have filtered out observations or comments about specific individuals.


Tag: Effectiveness Project and Programme management Bilateral partners Country Government Civil Societies and NGOs

34.

2015: Organization for Industrial Spiritual & Cultural Advancement (OISCA), Barefoot Community Services, Mahonia Na Dari, and Live and Learn PNG are listed as being associated mostly with existing WMAs in East and West New Britain (and a few new WMAs and a pilot area in the Whiteman Range – not mentioned again, as well as the Warangoi catchment area – which has not advanced79) without clarifying the actions/outputs, but inferring that these NGOs have been engaged for project delivery.


Tag: Effectiveness Project and Programme management Bilateral partners Country Government Civil Societies and NGOs

35.

2019: In 2019, there is first reference to being inclusive and participatory with partners, with regular partners’ coordination meetings87 (at least twice per year, with a focus on onsite implementation of Components 3 and 4). These meetings apparently focussed on planning and budgeting and occurred before PAB meetings (there were only two PAB meetings in the period covered by the 2019 PIR report).


Tag: Site Conservation / Preservation Effectiveness Jobs and Livelihoods Coordination

36.

Further, there was some provincial (East New Britain) cynicism about the utility of the PAB, and there were several communities and Government staff who noted that some of the NGO/CBO partners themselves needed capacity building. On the other hand, several project partners said that the Government-NGO linkage is better (more functional) than it has been in the past, and the project has contributed to this, as respective roles get more clearly defined and separated, and delivery gaps are being filled.

 


Tag: Effectiveness Capacity Building Civil Societies and NGOs

37.

3.2.3 Monitoring and Evaluation:Design at Entry, Implementation, and Response (Adaptive Management) The key to effectively delivering a project and managing associated risks is to be completely on top of project activities (aware of all actions by all partners on a daily/weekly basis). This is the monitoring function, which requires frequent dialogue with all project partners and site visits as frequently as possible. The Project Manager and delegates normally handle these tasks, but there is also an obligation on all project partners to report on developing issues and seek clarification when plans are unclear (monitoring and adaptive management are two-way streets).


Tag: Challenges Effectiveness Monitoring and Evaluation Project and Programme management Results-Based Management

38.

In theory, ongoing project monitoring and evaluation of the effectiveness of project actions and outputs should be described numerically (to the extent possible) and qualitatively in the annual Project Implementation Review (PIR) reports. PIRs were produced in about June of each year from 2015 to 2019. A careful reading of each report does allow the reader to get a fairly clear picture of project status versus the targets in the LFA (PRF), but there is a certain amount of forensic work required to capture the time sequence of project actions, since there is quite repetitive reporting of project actions from year-to-year, and after the MTR, project actions are reported as cumulative results (perhaps simply constrained by the table format). As a result, it is not clear what exactly was achieved in a given year, versus the accumulation of results over 2-3 years. This is always a risk when the previous year’s PIR is used as a template for the current year being reported103.

There has been a further issue in reporting results as “will” statements; clearly indicating that actions have not been implemented, but should be soon. This is called “aspirational” reporting, and is not something that should pervade a PIR report. Part of the problem is in the report format: there is no summary of the annual workplan for the year under review, and therefore no actual accountability and reporting on commitments made at the beginning of the year. Related to this is a lack of listing of all meetings, workshops, training events, etc. (these are not in PIR reports, except as brief narratives, or in any acknowledged annexes). There was a listing of training/capacity-building events in the MTR report (obtained from various files, it seems), but this list included a large number of events that did not seem to involve project partners and locations, so their direct linkages to the GEF4 project remain obscure104.


Tag: Challenges Effectiveness Monitoring and Evaluation Project and Programme management

39.

3.2.4 Project Financial Management

Project financial management refers to the timely and appropriate disbursement of, and accountability for, project funds. UNDP has maintained a budget/expenditure reporting system for the GEF4 project (the Combined Delivery Report by Activity in ATLAS, and the Project Resource Overview). In theory, these two systems track all expenditures reconciled to budgets as transactions are entered (near-real time). Individual project partners (NGOs, CBOs, District and Provincial Governments) had their own financial management systems, but with varying implications for the project. For a long time, CEPA, and district and provincial governments, only reported on their financial commitmentsto the project, not on the value of in-kind support (staff time, office space, travel, etc.).


Tag: Efficiency Global Environment Facility fund Human and Financial resources Monitoring and Evaluation Operational Efficiency Project and Programme management Bilateral partners Civil Societies and NGOs

40.

The table below shows the project expenditures for the year 2019. This provides a “snapshot” of both the expenditure weighting by component and also by expenditure type, which is instructive (note that previous years may have had different weightings).Up to 2016, there was a relatively even expenditure rate for Components 1-3 (ranging from 24 to 28% of total expenditures), but little in Component 4 and about 13% for project management (the latter being a quite typical and appropriate ratio for management versus actual delivery). In the intervening years (2017-2018) the ratios between components changed, and in 2019, it was evident that there was little effort in Components 1 and 3, some increased effort in Component 2 and most of the expenditures going to Component 4 (69%, assumed for capacity building, which had been lagging in previous years).


Tag: Efficiency Global Environment Facility fund Human and Financial resources Monitoring and Evaluation Project and Programme management

41.

3.3 Project Results

It is clear from Section 3.2 above that the GEF4 project has had some implementation challenges, and many of these are just reflective of the project operational context in PNG. Despite these challenges, the project has produced relevant and significant results, which are discussed here.

3.3.1 Overall Results (Attainment of Objectives)

At the end of the day, the project is accountable for progress made against each of the original project objectives that have stayed in the project workplan (after the MTR, some performance indicators, and therefore their associated actions, were dropped). The project results are assessed individually below (original targets are noted in italics; revised targets and indicators, after the MTR, are noted where relevant, and tabulated later in this report, in any case, following the latest PRF), according to the sequence of overall project objectives and component outcomes. Examples of project results (and constraints, as well) are provided to the extent possible, as evidence to support the evaluation observations106. The quality of the results (relevance, sustainability, and impact) is discussed in more detail in subsequent sections of this report.


Tag: Effectiveness Efficiency Implementation Modality Project and Programme management

42.

Targets (Objectives):

 Overall:

A comprehensive and integrated policy and regulatory framework for CCAs is enacted by end of year 2; supported by a coordinated whole-of-Government decision-making mechanism operational by year 3.

The 2014 PNG PA Policy provided the basis for the PA Bill. The Bill is an excellent document (having gone through a rigourous process in development) that sets the frame for development and management of CCAs in the future. Also, the policy implementation plan provides good guidance on implementation details in the PA Bill. These can both be considered comprehensive and integrated for the purpose of devolving conservation to local landowners, and represent significant results from the GEF4 project. The whole-of-government mechanisms are less clear107. The caveat with the Bill is that it has not been passed, and there is a risk of it languishing. In 2018, 2 million K was committed by the national government to conservation, which is certainly progressive (however, only 1 million K in 2019)108. This component (the policy and regulatory framework) is “on-the-cusp” and needs the highest level political support (so, a persistent push from CEPA and UNDP is needed in that regard).


Tag: Site Conservation / Preservation Rule of law Capacity Building Institutional Strengthening National Institutions

43.

Landowner commitment sufficient to ensure effective management and conservation of CCAs as measured at end-project.

Based on detailed conversations with WMA communities in West and East New Britain (five in total) land owner commitment appears to be strong (perhaps more so in East New Britain that West New Britain), and as long as alternative livelihoods can be developed in these communities in a continuous manner, these areas have a high chance of being properly conserved. Several WMA members (East New Britain) were very articulate about conservation attributes and goals and the associated alternative livelihoods but expressed a desire for further technical and financial support (self-financing still being a ways off). At the time of project closing, it seems that 17 WMAs or new proposed CCAs in New Britain are in the gazettal process, with letters of consent provided, and the basic boundary details and adequate management plans defined. CEPA is now required to follow through, quickly, with the gazettal process (within the options not included in the draft PA Bill) to keep local communities in New Britain on side.


Tag: Site Conservation / Preservation Effectiveness

44.

3.3.2 Relevance

 The GEF4 project has been totally relevant (in design and implementation) in all respects concerning habitat and biodiversity management, at various levels, as follows:

  •  absolutely relevant to the global conservation agenda (given the disproportionately high amount of global biodiversity in PNG);
  •  relevant in acknowledging and trying to address the peculiarities of the land ownership/ development/ conservation challenges in PNG;
  •  appropriate and relevant in its approach, with parallel prongs aimed at the national/regulatory requirements, the engagement of various levels of Government, and a specific focus on local communities, to develop evidence of effective conservation approaches on-the-ground; and,
  •  total relevance in the project’s acute understanding of the need for alternative livelihoods to complement conservation efforts, intended to draw pressure off adjacent community habitats and curtail excessive natural resource exploitation.

Tag: Protected Areas Site Conservation / Preservation Relevance Global Environment Facility fund Country Government Capacity Building National Institutions

45.

At the community level (New Britain)… Perhaps the greatest degree of relevance was evident and articulated at the local community level in New Britain. While there may have been some issues with how the project was delivered and supported at the level of WMAs and proposed CCAs in New Britain, there was a pervasive feeling (expressed to evaluators, by those who are still active with the project) that the GEF4 project was certainly relevant to their current needs (both in terms of conservation and in terms of developing alternative livelihoods). There are numerous examples, as follows:


Tag: Protected Areas Site Conservation / Preservation Relevance Ownership Country Government

46.

3.3.3 Effectiveness and Efficiency

Effectiveness refers to the degree to which project objectives (those that have been retained) have been achieved, and efficiency refers to the cost-effectivenes (least cost for delivery) of the project overall. The evaluators’ assessment of effectiveness is summarized in the table below125. Note that the assessment of effectiveness, for each project objective, is quite a subjective exercise, reflecting both the perceptions of the delivery of actions and outputs (present or not, or somewhere in between), and their respective contributions to objectives (the quality of those contributions). Effectiveness is expressed as estimated % achievement.


Tag: Effectiveness Efficiency Project and Programme management Country Government Capacity Building

47.

3.3.4 Country Ownership

Everything in the GEF4 project is embedded in the Government mandate, community experience, and the psyche of the individuals involved (given the focus on land, and its correct use, nothwithstanding what developers think). For a start, everyone knows what the problems are, why there are problems, and the challenges of doing anything effectively in remote areas. Almost all the project activities had outputs (in many cases, outcomes) that will sit within Government practice and within the local communities in the conservation areas. As noted before, some key activities (on the regulatory side) are “on-the-cusp”, about 80% there in terms of achievement. While there is an overly strong reliance on donor-funded projects, there are increasingly positive signs of conservation financing within the country, at different levels (ongoing financial commitments at the national, provincial, and district levels, as well as private sector and local community contributions; these investments would not be made if they were not intended to solve the problems that all of PNG owns).


Tag: Biodiversity Natural Resouce management Protected Areas Site Conservation / Preservation Sustainability Global Environment Facility fund Ownership Country Government

48.

3.3.5 Mainstreaming

Mainstreaming refers to the degree of convergence of the GEF4 project with other aspects of UNDP’s country programme in PNG, in particular poverty alleviation, improved governance, prevention and recovery from natural disasters, and women’s empowerment. This is different from country ownership (discussed above) which is concerned with the extent to which the project has been embedded within policy implementation and routine tasks of different levels of government, as well as sustainable financing of those.


Tag: Protected Areas Site Conservation / Preservation Sustainability Global Environment Facility fund Women's Empowerment Ownership Jobs and Livelihoods

49.

The project has only an indirect relationship with prevention and recovery from natural disasters. It is axiomatic that conservation of forests will help reduce the risk of flooding and landslides, and proper conservation of coastal areas (maintaining mangrove forests and the integrity of the reef) will help reduce the risk of erosion during extreme weather events and as sea level rises. These would be important spinoffs of any successful conservation in the areas supported by the project. Slightly more indirect is the link between community resilience, usually associated with improved economic conditions (and infrastructure and services developing accordingly) and ability to recover from natural disasters. The project has given due emphasis to developing the economic conditions of the communities in the project areas.


Tag: Disaster risk management Site Conservation / Preservation Sustainability Global Environment Facility fund Women's Empowerment

50.

3.3.6 Sustainability

Sustainability is the biggest challenge for all projects(GEF, and development projects in general). The GEF4 project has just ended; everyone who was consulted expressed concern about how to keep the various initiatives going, and all local communities who were consulted asked for more technical and financial support (this is universal to all projects… ). Future GEF support (mostly GEF6) will provide a “lifeline” for some project partners (not yet identified), but possibly with a lag, as that project gets going. However, note that new project funding is never supposed to be the sustainability solution for any project initiatives. Ultimately, all conservation initiatives will have to be supported through a combination of self-financing and merit-based Government grants (coming from the general budget, or biodiversity offsets), which are key principles that have been clarified and developed (if not yet implemented) with the help of the GEF4 project135.


Tag: Sustainability Global Environment Facility fund Resource mobilization Ownership

51.

The developing cocoa export market in East New Britain (supported by the project) warrants special attention. There is much hope placed in this initiative, and much has been invested already. Almost all communities that were consulted during the evaluation expressed interest in getting involved (mostly for the potential community income, which may have some positive spin-offs for conservation efforts). However, there will be ongoing issues related to economies of scale (too many small villages and small crops, far away from each other and transportation hubs). It does not appear that any detailed business modeling/planning has been done that recognizes where cocoa initiatives can be profitable (under different scenarios) and where they will not likely be viable140.


Tag: Sustainability Risk Management Trade and Development

52.

3.3.7 Impac

t At the end of the day (if not the project), effective community-based conservation must be obvious, visible, and measureable, demonstrating stable or improving integrity of habitats adjacent to WMA/CCA communities, possibly increases in conservation areas, evidence of stable or increasing biodiversity, and clear evidence of community services, infrastructure, and disposable incomes increasing. This is the expected impact of effective community-based conservation. The project just does not have the means to verify either the baseline (at the beginning of the project, or even mid-way) or the current status of these attributes in areas where the project has been engaged. There is only reliance on anecdotal information from the communities themselves. As such, it is not possible to attribute any such impacts to the GEF4 project. However, as noted previously, the GEF4 project is very close to contributing to such positive impacts – “on-the-cusp”.


53.

By end-project each established CCA has demonstrated access to all funding required for core management and conservation activities for at least two consecutive years.

This target is extremely ambitious and dependent on all other targets being met (and probably some from other initiatives, as well). Without a national system of revenue collection and distribution directly related to conservation, and a system for disbursements to “worthy” CCAs, this target would remain elusive. However, it does capture the essential attribute of CCA self-financing.


Tag: Natural Resouce management Protected Areas Site Conservation / Preservation Effectiveness Financial Inclusion

54.

1.2. National economic development plans and sectoral plans incorporate and provide support for the objective of developing a Sustainable National System of PAs. [ the indicator was revised after the MTR to be Indicator 7:

Evidence or degree of mainstreaming of protected areas within different national policies and development strategies]

By year 3, PNG’s Medium-Term Development Strategy and related planning documents explicitly recognize the development of a sustainable National PA System as a development priority, under the ESEG framework40.

This is an appropriate target; planning strategies and documents should certainly be reflecting and supporting the PA concept


Tag: Protected Areas Effectiveness Capacity Building Institutional Strengthening

55.

Overview of Project Structure: Overall, the project structure is appropriate for CA development in PNG, with thrusts at the various levels of Government, including the regulatory context, and an appropriate focus on the ground level (community engagement in conservation), with related capacity-building expected (it is clear and sensible). With the project, all the necessary parts are in play, although going in parallel, which is a challenge, when one component possibly depends on the outcomes of other components (and associated timelines are ambitious). Further, the proposed development of various community conservation areas would provide good opportunities to test the policy implications49 of new Government initiatives (whether national or provincial) in various community contexts, as long as these are properly analyzed and documented.


Tag: Site Conservation / Preservation Relevance Capacity Building Institutional Strengthening National Institutions

56.

Sufficient level of cooperation obtained from other relevant agencies. This risk (and mitigating it) is very much dependent on the perceived strength and leadership of CEPA, when facing forestry and extractive industry agencies.


Tag: Sustainability Risk Management Bilateral partners

57.

Obtaining community/ landowner support for establishment of CCAs does not take significantly longer than envisaged in the project strategy. It seems that this risk is actually quite low, since project interventions will be mostly focused on existing WMAs, which already reflect local community commitment to conservation areas, for the most part. However, these communities assume that funds will flow as a result of gazetting – a reward for their commitment to conservation. This aspect is critical and early precedents for financial support, as a result of gazetting, are required to avoid local community cynicism about the whole conservation process. Otherwise, the inducements from adjacent operations, such as oil palm (jobs, mostly) will continue to be attractive, if local conservation cannot be monetized in some manner that spreads benefits throughout the community and helps to pay for infrastructure and services that most communities desperately need. Ongoing population increases and associated pressure on limited services and adjacent land will continue to work against conservation principles.

 


Tag: Site Conservation / Preservation Sustainability Risk Management Strategic Positioning

58.

CAs are established at least 3 years before project end, to allow sufficient time to demonstrate management improvements. This is a significant risk, known right from the beginning. However, some of this risk is a bit “fuzzy”, since project interventions occur in existing WMAs (so, what exactly is meant by "established?". The real risk is that, despite establishing CAs in some fashion, management improvements may not be evident (management plans alone do not constitute effective management, although they are a starting point for that). For effective conservation management, all stakeholders will require the same vision and expectations, which takes a lot of time, communication, and documentation of all related consultations and decisions (for transparency and accountability). Related to this is the issue of not fully understanding the biodiversity attributes in each area that is identified for conservation. This can lead to different views of what is important globally and what is important to local communities. Further, management improvements need to include effective monitoring and enforcement of community rules (as well as any government regulations that may apply to the area).


Tag: Site Conservation / Preservation Sustainability Risk Management

59.

Government commitment to provide revenue support to CCAs is sustained. This risk (the assumption being that it is not a risk) is extremely likely, as noted above, especially if future donor support continues to come in and Government support via Trust Funds, offsets, annual grants, etc. continues to be debated.

Government does not make any direct and deliberate (as opposed to indirect and inadvertent) decisions to sanction development activities which degrade CCAs. This is a significant risk, with plenty of precedents (e.g., logging concessions in identified and gazetted WMAs). With three levels of Government involved in development decisions, and some communities willing to make deals with developers, this risk needs vigilance, gazetting of WMAs/CCAs, and community cohesion, with strong conservation principles and effective monitoring and enforcement.


Tag: Risk Management Country Government Sustainability

60.

Existing commitments to provide social service support from partners such as Steamships Ltd. and Digicel are maintained, and other partnerships can be established where needed. Well, this seems like a good idea, and some precedents are provided. It is not clear who would be responsible for brokering these CSR-type arrangements, but it will probably require working at the National and Provincial Government levels.


Tag: Sustainability Partnership Risk Management Social Protection

61.

No external risk factors identified. This relates to incorporation of lessons learned into policy and regulations on an ongoing basis. It seems that there is a significant risk here that all the collection of lessons and linkages between the project and various Government and community partners may be fragile and not driven by any one entity (especially after the project is finished).


Tag: Knowledge management Risk Management

62.

Project management to ensure commitment to participatory evaluation, and debrief to key stakeholders. This is a project requisite, in any case, and must be driven by the PMU. Project board meetings and frequent field visits to project sites would address this required commitment.

In addition to the observations noted above, a significant risk implicit in project design and implementation arrangements was the key role of CEPA (DEC at the time) in delivering the project (with key activities in all components), yet being institutionally “constrained” itself and undergoing restructuring. As reflected in many past experiences elsewhere (other projects in other countries in the region), this combination of lack of capacity and ongoing re-structuring can make the institution quite “fragile” and challenged with delivery of activities and outputs beyond the routine mandate.


Tag: Sustainability Programme Synergy Programme/Project Design Risk Management

63.

Despite these challenges, the self-assessment and reporting (in the latest PIR at least) indicate that the GEF4 management were happy enough with governance and project management, and attributed this to project staff on the ground, and monthly meetings between the project team and the National Project Director at CEPA (or her designate), facilitating increased coordination and communication. On the other hand, some local communities were frustrated with their own communication constraints, relying on intermittent mobile phone coverage and email. But, when the opportunity for engagement through either workshops or consultations presented itself (either in Kokopo or Port Moresby), local community members were most appreciative, and this provided a time for getting clarity on project actions and expectations. In this operational context, UNDP staff in New Britain indicated that a dogged determination to get things done (and patience) are requisites for working on the project69.


Tag: Site Conservation / Preservation Effectiveness Communication Operational Efficiency Project and Programme management Coordination Civil Societies and NGOs

64.

2016: There is reference to the project establishing a coordination mechanism in each province known as Friends of New Britain (a venue for providing updates on project implementation as well as discussions with CEPA and provincial administrations to support conservation initiatives on the ground82) – this is not mentioned again in project reporting, so its make-up and status remain unclear. In addition to the organizations mentioned in 2015 (without further elaboration of progress from 2015, or future actions), Forcert has been added, and the Bishop Museum has been engaged83. In 2016, there is first reference to engagement with Members of Parliament, and Provincial and Local Level Governments (good).


Tag: Protected Areas Site Conservation / Preservation Effectiveness Bilateral partners Jobs and Livelihoods Trade and Development Coordination Civil Societies and NGOs

65.

These observations reinforce observations of local communities in many development projects, where community dynamics and power plays, as projects and finances are injected into communities, get activated. Having said this, there has been a natural evoloution of WMA leadership that reflects the leadership abilities and degree of articulation of conservation issues and approaches (and English language capability) of specific individuals, and it would be expected that such individuals take on the GEF4 project responsibilities.


Tag: Site Conservation / Preservation Effectiveness Bilateral partners Country Government Jobs and Livelihoods Civil Societies and NGOs

66.

The project has given a lot of emphasis to the METT scoring system (as required for GEF projects), and in fact several project actions were focused on how to use the METT system. While the METT scores may have some long-term value in terms of measuring protected area effectiveness over time (especially if used in exactly the same way every 3 years, to allow legitimate time-series comparisons99), the METT scoring system was deemed to be unsuitable for the PNG context, and was revised twice to fit PNG circumstances100.


Tag: Challenges Effectiveness Monitoring and Evaluation Project and Programme management

67.

1,000,000 hectares protected by end of Project.

This target was reduced to 500,000 ha after the MTR. Apparently 672,000 ha of protected area is still in play with the project (Kokoda and New Britain), although the project effort in Kokoda is obscure to the evaluators. The 434,000 ha in New Britain include previous WMAs that have been supported by the project (getting them ready for gazetting; but not all, as yet, gazetted109). However, evidence on the ground indicates that conservation is mostly occurring, so technically these areas are being successfully protected (good) 110.


Tag: Protected Areas Site Conservation / Preservation Effectiveness Relevance

68.

CCAs show sustained improvement in METT scores over the duration of the project, beginning from respective year of CCA establishment.

According to reporting, the METT scores in the CCAs have shown a continuous increase since the beginning of the project. While there are issues with METT scores being a proxy for conservation management competence, as discussed previously (perhaps not all attributes fit the community situation), undertaking these in the same way over time is a valid time-series measure of conservation competence111. Certainly the WMAs that were visited in New Britain articulate strong conservation awareness and have taken steps with project support to develop their management plans and prepare for gazetting (good112). Some WMAs also undertook their own training programmes, using “in-house” capability, which was a form of capacity-building in its own right, and creating competencies that could be shared with other WMAs.


Tag: Site Conservation / Preservation Effectiveness Project and Programme management

69.

By end-project each established CCA has demonstrated access to all funding required for core management and conservation activities for at least two consecutive years.

This is not evident, due to the late initiation of project activities at the WMAs in New Britain. However, some WMAs have their own sources of funding (various, such as tourism, guest house receipts, cocoa – incipient, and some communities paying in directly according to conservation needs). There is great anticipation of passage of the PA Bill, gazetting of CCAs, and a national budget that supports new CCAs (there is significant pressure on CEPA to pull this off… ). Financial support to cocoa initiatives (solar dryers) is not directly related to conservation, but indirectly should help raise local community incomes to take pressure off surrounding habitats, and to eventually help self-finance conservation, such as ranger activities113.


Tag: Site Conservation / Preservation Effectiveness

70.

Within the WMAs themselves (possibly CCAs in the future), there is a need to clarify community expenses related to monitoring and enforcement and ensure that sanctions or penalties for conservation infractions go back into conservation management. These would help sustain the conservation effort in most communities, but almost no community could articulate how their penalty systems actually operated with good accounting and transparency.


Tag: Site Conservation / Preservation Sustainability Oversight

Recommendations
1

Pass the draft PA Bill – the Project did handover to CEPA, a Bill that was cleared by State Solicitor and the Legislative Council. As the Bill seems to be hung up on considerations of revenue collection and subsequent distribution (related to Trust Fund mechanisms), it is suggested to not include the details of revenue collection and transfer in the Bill, but to specify instead commitment to examine current precedents for collection and distribution of revenues, that might apply to conservation (for example, the -Road Transport Authority model advised by Department of Treasury during the PA Bill inter-government consultations). These can then be picked up and adapted, as agreed by all parties, in subsequent regulations and guidelines. Furthermore, CEPA can more actively engage with ministers, Members of Parliament, and even the Prime Minister, to encourage quick adoption of the PA Bill, before further time is lost, and cynicism sets in while UNDP supports CEPA in playing the lobbying role. The concerns of private sector lobbyists have been taken into consideration, but the Bill was formulated by the majority of the rural customary landowning communities during the Bill’s extensive consultation process with all the regions of PNG. The Bill has captured the aspirations of the majority rural population for the sake of effective conservation in PNG.

2

Prioritize quickly gazette the WMAs and proposed CCAs facilitated by the project. This gap can be taken by GEF6.

3

With the lack of sufficient technical and financial support, it is recommended to establish more cohesive and coordinated system to continue support the development of provincial institutions that are mandated for conservation. The provincial offices and their functions provide a critical link between national government agencies; districts and local communities, therefore, engaging provincial offices in all project activities that go to local communities, so that their bridging/facilitating role is enhanced. 

4

Developing wide-scale work by CEPA and NGOs and CBOs on conservation areas in the hinterlands (higher altitude forested areas) in West New Britain and East New Britain (Baining, Nakanai, Whiteman range). It is recommended that this effort is initiated with review of the recent biodiversity surveys and detailed analysis of recent GoogleEarth images. Ultimately, the large size and contiguity of forest habitats (evident in the hinterlands) will have a bigger biodiversity impact than the smaller WMAs and CCAs and should therefore be given more attention.

5

Support CEPA and provincial/district governments to develop ecotourism profiles and feasibility studies for each community site in New Britain, being realistic and specific about opportunities, and identifying the required infrastructure and services. 

6

Develop business models for each local community venture (within those communities who have been involved with the GEF4 project), to properly understand viability, timeframes, required investments, potential profits, and benefit sharing associated with such initiatives as cocoa exports, market gardening, ecotourism, etc. (there is scope here for being more expansive and creative about potential business opportunities in the communities, such as insect collection and sales). it is recommended to consider encouraging more WMA accountability for revenue flows related to such ventures (a little more transparent) while ensuring that there are no lapses with the current cocoa export initiative (maintain export flows to meet buyer expectations).

7

For any future involvement in conservation area, it is recommended to reduce the involvement of intermediaries for community conservation and alternative livelihood initiatives, since they increase the cost of project delivery (and it is not always clear that there is added value from them).

8

Communities are encouraged to develop more specific action plans for their conservation activities (monitoring and evaluation of their WMAs), since at the moment, the status of their protected areas is often unclear and based on subjective perceptions. As noted previously, a time-series of GoogleEarth or drone images of each conservation area (updated every two years) could be developed5. These can then be housed in the PA Registry at CEPA. Youth in these communities could be engaged in science projects related to conservation and biodiversity monitoring (youth in several communities that were consulted expressed keen interest in doing this). Related to this, if there are specific zones or land use plans in the community conservation management plan, there is a need to set capacity limits for all activities within those zones. This means limiting the number of people or number of activities in specific zones.

9

It is beneficial to consider volunteer-type people with little cash incentives from the project or CEPA to work in the communities at least 3-4 weeks at a time and 4-5 times per year, if more technical support to be provided to local communities (whether conservation-related or addressing alternative livelihood development), that will help creating traction and develop effective working relationships.

10

Encourage setting up exchanges between WMAs/CCAs in New Britain, to disseminate the lessons learned (good and bad experiences) to all communities interested in conservation and developing alternative livelihoods.

11

For future, it is recommended to accurately assess the progress in building capacity through tracking all training events as a separate file, with topics, name of trainer, names and gender of trainees, and their positions/affiliations at the time of training (for ease of tracking capacity-building programmes).

12

In all future project performance reporting, do not revert to ‘cumulative”’ to show previous achievements; retain a record of project achievements for each specific year (otherwise, future evaluations will have to revert to “forensic” review to determine project accountability for annual workplans, as these details will remain obscure). Also, be clear in reporting what has actually been done and achieved; avoid aspirational statements about results expected in the future.

13

Review gender action plan to clarify on the level of women engagement and their role in the project and ensure a project reporting system includes gender disaggregated data.

14

The METT scoring system for PA management effectiveness is a “given” with GEF conservation type projects and is intended to help track progress during a project and also supposedly to allow comparisons between projects and countries. The METT experience with the GEF4 project, however, was mixed7. On the one hand, the report on METT scores in 2017 is a very useful “round-up” of PA status throughout PNG, based on the perceptions of the communities associated with these protected areas. On the other hand, there were issues with how questions were perceived, as well as how the answers were obtained (in a workshop format, without clear on-the-ground verification8). A true measure of PA management effectiveness is the area and quality of habitats and biodiversity in specific areas (this information being obtained scientifically and objectively). While many of the discussion points in the METT system are relevant and interesting, the evaluators believe it is important to introduce actual evidence of PA management effectiveness into the METT observations and scores (for example, drone images, or time-series of GoogleEarth images). Further, the utility of the METT scores is greatly increased when each WMA/CCA, or institution, develops a specific action plan directly responding to the constraints and issues evident in the scores. In other words, the METT scores and apparent constraints are reviewed with the WMAs, and responsive action plans are developed.

15

It is recommended to undertake a feasibility study (one site, as a pilot) for mini or micro-hydropower (using a horizontal Straflo-type turbine in the river).
Also based on field observations, it is recommended to examine the feasibility and develop gravity feed water systems in all project communities where this seems practical (simple plastic pipes coming from the river, streams, or springs) and feeding a community water storage tank, and then individual household feeds.

Management Response Documents
1. Recommendation:

Pass the draft PA Bill – the Project did handover to CEPA, a Bill that was cleared by State Solicitor and the Legislative Council. As the Bill seems to be hung up on considerations of revenue collection and subsequent distribution (related to Trust Fund mechanisms), it is suggested to not include the details of revenue collection and transfer in the Bill, but to specify instead commitment to examine current precedents for collection and distribution of revenues, that might apply to conservation (for example, the -Road Transport Authority model advised by Department of Treasury during the PA Bill inter-government consultations). These can then be picked up and adapted, as agreed by all parties, in subsequent regulations and guidelines. Furthermore, CEPA can more actively engage with ministers, Members of Parliament, and even the Prime Minister, to encourage quick adoption of the PA Bill, before further time is lost, and cynicism sets in while UNDP supports CEPA in playing the lobbying role. The concerns of private sector lobbyists have been taken into consideration, but the Bill was formulated by the majority of the rural customary landowning communities during the Bill’s extensive consultation process with all the regions of PNG. The Bill has captured the aspirations of the majority rural population for the sake of effective conservation in PNG.

Management Response: [Added: 2020/04/24] [Last Updated: 2021/02/03]

The Bill was led by CEPA during extensive meaningful consultations over 18 months and supported by the GEF 4 Project. For sustainability purposes, CEPA has ownership of the Bill and is responsible for progressing the Bill to enactment. The Government shall lead now the process for endorsing the Bill.

Key Actions:

2. Recommendation:

Prioritize quickly gazette the WMAs and proposed CCAs facilitated by the project. This gap can be taken by GEF6.

Management Response: [Added: 2020/04/24] [Last Updated: 2021/02/03]

This is the area that CEPA has to lead on. One gazettal has been completed by CEPA and the East New Britain Provincial Administration. CEPA will continue to progress the gazettal of the other WMAs, in collaboration with their other Project Partners, where relevant, given that CEPA administers the relevant Conservation laws.

Key Actions:

3. Recommendation:

With the lack of sufficient technical and financial support, it is recommended to establish more cohesive and coordinated system to continue support the development of provincial institutions that are mandated for conservation. The provincial offices and their functions provide a critical link between national government agencies; districts and local communities, therefore, engaging provincial offices in all project activities that go to local communities, so that their bridging/facilitating role is enhanced. 

Management Response: [Added: 2020/04/24] [Last Updated: 2021/02/03]

Acknowledged, despite that no key actions are required at this stage as the project will be closed, however, UNDP will ensure its continuous support is provided to strengthen the relation between CEPA and its provincial counterparts through other programme and projects.

Key Actions:

4. Recommendation:

Developing wide-scale work by CEPA and NGOs and CBOs on conservation areas in the hinterlands (higher altitude forested areas) in West New Britain and East New Britain (Baining, Nakanai, Whiteman range). It is recommended that this effort is initiated with review of the recent biodiversity surveys and detailed analysis of recent GoogleEarth images. Ultimately, the large size and contiguity of forest habitats (evident in the hinterlands) will have a bigger biodiversity impact than the smaller WMAs and CCAs and should therefore be given more attention.

Management Response: [Added: 2020/04/24] [Last Updated: 2021/02/03]

Acknowledged, this work shall be performed now by CEPA, so far the later initiated the support to the Provincial Administration of West New Britain to support the gazettal application of Via River Catchment; exercised a follow-up on a separate application of the Nakanai Range under the UNESCO WHS tentative list and used drone to follow-up on the status of OSR with the landuse planning that incorporates conservation uses as a proposed WHS. Under GEF 5, CEPA’s capacity.

Key Actions:

5. Recommendation:

Support CEPA and provincial/district governments to develop ecotourism profiles and feasibility studies for each community site in New Britain, being realistic and specific about opportunities, and identifying the required infrastructure and services. 

Management Response: [Added: 2020/04/24] [Last Updated: 2021/02/03]

Partially agree; UNDP will commit to working through other GEF funded projects on some sites, committing to other sites is not possible to guarantee at present. Greater support might be provided through the established Biodiversity Fund, if it is capitalised and operational.

Key Actions:

Key Action Responsible DueDate Status Comments Documents
Scanning exercise of in West New Britain. Tourism is identified as a finance option for this area.
[Added: 2020/04/24]
GEF6 team 2020/01 Completed Action taken.
Conduct a financial and technical feasibility linked to the Kimbe Bay MPA.
[Added: 2020/04/24] [Last Updated: 2021/05/03]
GEF6 team 2021/04 Completed This activity is part of the national PA finance and investment plan and the final report was already completed. Attach is the final report in draft to be finalized by the end of this month. History
6. Recommendation:

Develop business models for each local community venture (within those communities who have been involved with the GEF4 project), to properly understand viability, timeframes, required investments, potential profits, and benefit sharing associated with such initiatives as cocoa exports, market gardening, ecotourism, etc. (there is scope here for being more expansive and creative about potential business opportunities in the communities, such as insect collection and sales). it is recommended to consider encouraging more WMA accountability for revenue flows related to such ventures (a little more transparent) while ensuring that there are no lapses with the current cocoa export initiative (maintain export flows to meet buyer expectations).

Management Response: [Added: 2020/04/24] [Last Updated: 2021/02/03]

Partially agree. There is an assumption here that there has not been any work done in terms of business modelling. The New Britain Provinces have had a long history in agribusiness and information is probably located under various public and private entities that could be subject of a separate study. The WMA comprise parts of the natural environment of the Provinces and managed locally and it should be stated here that the East New Britain Provincial Administration is probably the best functioning provincial governments in PNG, with good infrastructure and communications facilities, compared to other Provinces. Furthermore, where WMAs are concerned, that the CEPA can cultivate linkages with other national government entities, for technical support but only at the invitation of the respective Provincial administrations, for ownership and sustainability purposes.

Key Actions:

7. Recommendation:

For any future involvement in conservation area, it is recommended to reduce the involvement of intermediaries for community conservation and alternative livelihood initiatives, since they increase the cost of project delivery (and it is not always clear that there is added value from them).

Management Response: [Added: 2020/04/24] [Last Updated: 2021/02/03]

Acknowledged, the recommendation will be taken into consideration in any future involvement with community conservation. During the Inception Workshop of GEF6, CEPA and UNDP are committed to build more flexibility in partnering directly with Districts and community groups.

Key Actions:

Key Action Responsible DueDate Status Comments Documents
Based on GEF4 lessons, provide inputs to review arrangements and stakeholder engagement plan of GEF6.
[Added: 2020/04/24]
GEF team 6 2020/01 Completed Discussions made with ongoing GEF projects from 5-14 Feb 2020 as part of handover.
Share experience of grant making to communities with GEF6 to influence the design of the grant making process with communities under the Biodiversity Fund. it will be incorporated into the design of the biodiversity fund operational manual. Dec 2020.
[Added: 2020/04/24] [Last Updated: 2020/09/20]
GEF team 6) 2021/12 Not Initiated Establishment of Biodiversity fund operational & governance mechanism/structures is still a work in progress. History
8. Recommendation:

Communities are encouraged to develop more specific action plans for their conservation activities (monitoring and evaluation of their WMAs), since at the moment, the status of their protected areas is often unclear and based on subjective perceptions. As noted previously, a time-series of GoogleEarth or drone images of each conservation area (updated every two years) could be developed5. These can then be housed in the PA Registry at CEPA. Youth in these communities could be engaged in science projects related to conservation and biodiversity monitoring (youth in several communities that were consulted expressed keen interest in doing this). Related to this, if there are specific zones or land use plans in the community conservation management plan, there is a need to set capacity limits for all activities within those zones. This means limiting the number of people or number of activities in specific zones.

Management Response: [Added: 2020/04/24] [Last Updated: 2021/02/03]

Through the Project and GEF 5, CEPA has facilitated for drones to be purchased and also for drone training, in collaboration with the East and West New Britain Provincial Administrations to support landuse planning that incorporates protected areas. The respective Administrations have taken ownership of this process and as such there is continuity and sustainability of the drone initiative that the Project started.


East and West New Britain provincial administrations have committed to establish an information hub to house stitched images to monitor real-time forest and land cover and resource inventory as well as to reconfirm demarcated boundaries of a protected area. Drone images that will be hosted in CEPA’s PA Registry was reflected in the PA Registry manual as part of follow-up action.

Key Actions:

Key Action Responsible DueDate Status Comments Documents
Continuing GEF5 drone trainings to include GEF4 sites
[Added: 2020/04/24] [Last Updated: 2020/12/02]
GEF5 2020/11 Completed The Drone Trainings were completed in 2019 and included GEF 4 sites. Refer to the attached reports funded by the GEF 5 Project that included GEF4 site personnel and also a visit to Kokopo, a GEF 4 site. History
Facilitate sharing of stitched drone images between CEPA and provinces
[Added: 2020/04/24] [Last Updated: 2020/12/02]
GEF5 2020/11 Completed The drone images compiled and shared with the provinces. Urban plans were compiled using the drone images. For example, the Urban Plan that included the Warangoi catchment area was part of Baining region of the Pomio District, History
9. Recommendation:

It is beneficial to consider volunteer-type people with little cash incentives from the project or CEPA to work in the communities at least 3-4 weeks at a time and 4-5 times per year, if more technical support to be provided to local communities (whether conservation-related or addressing alternative livelihood development), that will help creating traction and develop effective working relationships.

Management Response: [Added: 2020/04/24] [Last Updated: 2021/02/03]

In partial agreement. The management will consider the recommendation when developing other programmes and projects. CEPA may have their own constraints on resources and competing priorities to take this recommendation forward and will need developing partners for this outreach but at the invitation of the communities and the respective provincial administrations.

Key Actions:

Key Action Responsible DueDate Status Comments Documents
Cost efficiency will be factored through the development of GEF 7 FOLUR, in collaboration with CEPA.
[Added: 2020/04/24] [Last Updated: 2020/07/02]
UNDP project manager for FOLUR project 2020/06 Completed The management response is self-explanatory in recognising that CEPA has its own resource constraints and the invitation has to emanate from the respective provincial administration in collaboration with CEPA. This is government business. In addition, the GEF 7 FOLUR is currently in design phase and CEPA is actively participating in this design phase. History
10. Recommendation:

Encourage setting up exchanges between WMAs/CCAs in New Britain, to disseminate the lessons learned (good and bad experiences) to all communities interested in conservation and developing alternative livelihoods.

Management Response: [Added: 2020/04/24] [Last Updated: 2021/02/03]

Acknowledged, CEPA will prepare their follow up action plan based on the evaluation report and will consider the recommendation as part of their plan.

Key Actions:

11. Recommendation:

For future, it is recommended to accurately assess the progress in building capacity through tracking all training events as a separate file, with topics, name of trainer, names and gender of trainees, and their positions/affiliations at the time of training (for ease of tracking capacity-building programmes).

Management Response: [Added: 2020/04/24] [Last Updated: 2021/02/03]

Agree. As the project has ended, the CO will ensure capturing this recommendation through other programme and projects.

Key Actions:

Key Action Responsible DueDate Status Comments Documents
Develop Training Tracking tools and share with other projects.
[Added: 2020/04/24] [Last Updated: 2021/04/08]
GEF6 team, CEPA, UNDP 2021/12 Not Initiated A training guide for PNG METT is under development including other training materials which I can share with you this year History
12. Recommendation:

In all future project performance reporting, do not revert to ‘cumulative”’ to show previous achievements; retain a record of project achievements for each specific year (otherwise, future evaluations will have to revert to “forensic” review to determine project accountability for annual workplans, as these details will remain obscure). Also, be clear in reporting what has actually been done and achieved; avoid aspirational statements about results expected in the future.

Management Response: [Added: 2020/04/24] [Last Updated: 2021/02/03]

Acknowledged, clear and proper reporting on indicators and results will be ensured.

Key Actions:

13. Recommendation:

Review gender action plan to clarify on the level of women engagement and their role in the project and ensure a project reporting system includes gender disaggregated data.

Management Response: [Added: 2020/04/24] [Last Updated: 2021/02/03]

Agree- Gender action plan will be developed for other GEF-funded projects.

Key Actions:

Key Action Responsible DueDate Status Comments Documents
The GEF 6 will develop a gender analysis and action plan.
[Added: 2020/04/24] [Last Updated: 2021/09/07]
GEF6 team 2021/08 Completed The Gender Analysis and Gender Action Plan were developed and currently in use by GEF 6 Project. History
14. Recommendation:

The METT scoring system for PA management effectiveness is a “given” with GEF conservation type projects and is intended to help track progress during a project and also supposedly to allow comparisons between projects and countries. The METT experience with the GEF4 project, however, was mixed7. On the one hand, the report on METT scores in 2017 is a very useful “round-up” of PA status throughout PNG, based on the perceptions of the communities associated with these protected areas. On the other hand, there were issues with how questions were perceived, as well as how the answers were obtained (in a workshop format, without clear on-the-ground verification8). A true measure of PA management effectiveness is the area and quality of habitats and biodiversity in specific areas (this information being obtained scientifically and objectively). While many of the discussion points in the METT system are relevant and interesting, the evaluators believe it is important to introduce actual evidence of PA management effectiveness into the METT observations and scores (for example, drone images, or time-series of GoogleEarth images). Further, the utility of the METT scores is greatly increased when each WMA/CCA, or institution, develops a specific action plan directly responding to the constraints and issues evident in the scores. In other words, the METT scores and apparent constraints are reviewed with the WMAs, and responsive action plans are developed.

Management Response: [Added: 2020/04/24] [Last Updated: 2021/02/03]

Agree- METT will be improved through other GEF funded projects.

Key Actions:

Key Action Responsible DueDate Status Comments Documents
Activity to re-examine PNG METT
[Added: 2020/04/24]
CEPA, GEF6 team 2021/09 Not Initiated Included in approved GEF6 2020 AWP
Revised METT for PNG
[Added: 2020/04/24]
CEPA, GEF6 team 2021/12 Not Initiated TOR to be advertised by GEF6 project in Q2.
15. Recommendation:

It is recommended to undertake a feasibility study (one site, as a pilot) for mini or micro-hydropower (using a horizontal Straflo-type turbine in the river).
Also based on field observations, it is recommended to examine the feasibility and develop gravity feed water systems in all project communities where this seems practical (simple plastic pipes coming from the river, streams, or springs) and feeding a community water storage tank, and then individual household feeds.

Management Response: [Added: 2020/04/24] [Last Updated: 2021/02/03]

This is to acknowledge the recommendation, as the Community-based Forest and Coastal Conservation and Resource Management project will be closed, the recommendation will be implemented through other projects as per the targeted sites approved in these projects.

Key Actions:

Key Action Responsible DueDate Status Comments Documents
Feasibility study will be conducted through RE project.
[Added: 2020/04/24] [Last Updated: 2021/04/08]
Renewable Energy project. 2021/03 No Longer Applicable [Justification: The scope of the RE project does not overlap with the beneficiaries of the GEF4 project.]
The Action is no longer applicable. History

Latest Evaluations

Contact us

1 UN Plaza
DC1-20th Floor
New York, NY 10017
Tel. +1 646 781 4200
Fax. +1 646 781 4213
erc.support@undp.org