Terminal Evaluation –UNDP Global Support to NBSAPs (PIMS 5283)

Report Cover Image

Share

Document Type Language Size Status Downloads
Download document TOR for PIMS 5283 & 3456.docx tor English 43.04 KB Posted 479
Download document TE Global Support to NBSAPs_FINAL_4MAR19.docx report English 5122.99 KB Posted 429
Title Terminal Evaluation –UNDP Global Support to NBSAPs (PIMS 5283)
Atlas Project Number:
Evaluation Plan: 2018-2021, Bureau for Policy and Programme Support
Evaluation Type: Final Project
Status: Completed
Completion Date: 12/2018
Planned End Date: 06/2018
Management Response: Yes
Corporate Outcome and Output (UNDP Strategic Plan 2018-2021)
  • 1. Output 1.4.1 Solutions scaled up for sustainable management of natural resources, including sustainable commodities and green and inclusive value chains
Evaluation Budget(US $): 11,500
Source of Funding: UNDP-GEF project resources
Evaluation Expenditure(US $): 11,500
Joint Programme: No
Joint Evaluation: Yes
  • Joint with Flanders
Evaluation Team members:
Name Title Email Nationality
Josh Brann International Consultant Brann.Evaluation@gmail.com
GEF Evaluation: Yes
GEF Project Title: Support to GEF Eligible Countries for achieving Aichi Biodiversity Target 17 through a globally guided NBSAPs update process.
Evaluation Type: Terminal Evaluation
Focal Area: Biodiversity
Project Type: MSP
GEF Phase: GEF-5
GEF Project ID: 5601
PIMS Number: 5283
Key Stakeholders:
Countries: GLOBAL
Lessons
Findings
1.

I. Relevance

  1. The “relevance” evaluation criteria mainly relates to the project development and approval phase. The NBSAP project is considered relevant (or “moderately satisfactory” in terms of the relevance criteria). The relevance of the project objective is satisfactory, as the project supports implementation of the CBD Strategic Plan for 2011-2020, including the Aichi Biodiversity Targets; in particular the project specifically supports Aichi Target 17. The project is also in-line with the GEF-5 strategic priorities for the biodiversity focal area.
  2. The relevance of project strategy and design is considered moderately satisfactory, as there are multiple aspects of the project strategy and design that could have been improved, though these may only be clear in hindsight. The overall timing of the project reduced its relevance, as it was approved after the majority of the enabling activities for countries to revise their NBSAPs, and a significant portion of the capacity development content was produced after many countries had completed their NBSAPs revisions. The joint implementation approach with UNDP and UN Environment has proven to be relevant in terms of enhancing the consistency of support to countries, although the specific structure and approach could have been strengthened.
  3. The NBSAP forum has yet to firmly establish its relevance as a demand-driven resource. The “peer” review mechanism has demonstrated to be highly relevant, although it functions differently than originally envisioned. The strategy of combining a global level capacity support program with individual country support for NBSAP revision is relevant, but may have been more effective with a stronger balance toward the individual country support. In addition, an important barrier was that the NBSAP Forum remained separate from the CBD website throughout the project, and the CBD website also contains guidance on NBSAPs. This contributed to some confusion among key stakeholders (e.g. NBSAP focal points) on where to access and absorb the most relevant information and guidance regarding NBSAP development and revision. A project strategy relying on internet-based services for dissemination of large amounts of information also remains an issue for many targeted stakeholders, particularly in LDCs and SIDS.

2.

A.      Relevance of the NBSAP Project Objective

i.Relevance to the CBD and other Multilateral Environmental Agreements

  1. The CBD is a key multilateral environmental agreement for which the GEF is the financial mechanism. The NBSAPs project supports the CBD’s Strategic Plan for 2011-2020, which includes the Aichi Biodiversity targets. The project particularly supports Aichi Target 17:
  • Target 17: By 2015 each Party has developed, adopted as a policy instrument, and has commenced implementing an effective, participatory and updated national biodiversity strategy and action plan.
  1. The project also broadly supports CBD objectives by supporting multiple Convention articles, such as Article 6 (General Measures for Conservation and Sustainable Use), 7 (Identification and Monitoring), 8 (In-situ Conservation), 10 (Sustainable Use of Components of Biological Diversity), 11 (Incentive Measures), 12 (Research and Training), 13 (Education and Awareness), 14 (Impact Assessment and Minimizing Adverse Impacts) and 17 (Exchange of Information).
  2. The NBSAP project also supports other multilateral environmental agreements relevant to biodiversity, such as the Convention on Migratory Species, the Convention on International Trade in Endangered Species, the World Heritage Convention, the Ramsar Convention. The project also supports the United Nations Framework Convention on Climate Change, through support to mainstream climate change considerations in NBSAPs. For example, the project carried out a Massive Open Online Course (MOOC) on “Resilience for Development – Part 2: Applying Resilience Thinking to National Biodiversity Plans” (January 23-February 12, 2018). The project, in partnership with the CITES Secretariat, and six other UN Agencies including UNDP and UN Environment, developed a six-lesson online course on Illegal Trade in Wildlife. The course is in the final stages of development and will be launched on World Wildlife Day, March 3, 2019. Due to the key role of NBSAPs for the conservation of migratory species, the CMS Secretariat website refers to the NBSAP Forum as the key website which provides support for action and implementation on NBSAPs through 2020[1].

 


[1] https://www.cms.int/en/activities/capacity-building/nbsaps


3.

A.      Relevance of the Project Approach: Project Strategy and Design

  1. There is no explicit theory-of-change provided in the NBSAP project document. At the time of the initiation of the NBSAP project, it was not a requirement to have a theory-of-change in GEF project documents. The implied and re-constructed project theory-of-change is far removed from the impact level. To compensate for this evaluation challenge, this evaluation is partially applying a theory-based evaluation approach, “which means examining the assumptions underlying the causal chain from inputs to outcomes and impact”.[1] The development and assessment of this theory-of-change facilitates the identification of key assumptions in the project’s logic chain – for example, that the primary critical barrier is a lack of instructive guidance.
  2. Overall, the project’s theory-of-change is considered valid. However, effectiveness depends on a number of critical assumptions. A weak area of the project’s strategy relates to the uptake and absorption of guidance documents and learning modules. Learning products on key topics related to NBSAPs have the potential to improve the quality of NBSAPs, if the targeted stakeholders who need them can absorb those learning products. This means that NBSAP focal points would need to be able to quickly and easily find the information they need, and have the time and initial baseline capacity to absorb this material by reading guidance documents, or participating in webinars or MOOCs. However, the project team made a concerted effort to point project stakeholders to useful technical documents or learning opportunities, or parts of them, and sections of the NBSAP Forum that could be most helpful to address their specific needs through email and Skype communication. These activities are described in 95 and 96. The NBSAPs project has had an impressive number of total participants in its learning activities, though anecdotal information indicates that frequently government bureaucrats (i.e. NBSAP focal points) have little extra time to absorb in-depth learning products. Considering the extent of guidance information available related to NBSAPs, and the number of people who could potentially use it, there is likely a greater need to facilitate better access to and use of the extensive amount of guidance information available to NBSAP focal points.
  3. The project worked to provide targeted and useful information as much as possible. The project carried out a targeted, direct outreach program for building capacities of NBSAP focal points. Beginning in 2014, each country completed a regular stocktake form, informing the global project team about their NBSAP development and revision progress, as well as about the specific types of resources or guidance which countries required. Additionally, the global team had at least three touch points per month with colleagues in each country, and responded to the needs that were stated, and the ways in which they wanted to learn. If countries expressed a need, the global team would direct the countries towards resources, guidance and experts available on the NBSAP Forum, per their requirement.
  4. One positive example of addressing country needs is the initiation of the demand-driven webinar program on ABTs each month, with a question and answer segment at the end of each session, conducted while keeping in view the different time zones, and in languages the parties requested. The project also disseminated newsletters to address knowledge gaps that countries identified and requested support in. The project also conducted NBSAP Forum user surveys seeking feedback on the Forum’s functionality and learning needs of the Forum members. FAQs were developed and shared with each of the NBSAP focal points on how to use and engage with the website. Webinars were conducted on a demand-driven basis around the needs of the NBSAP practitioners, and their attendance ability.
  5. Another positive aspect of the relevance of the project’s design relates to the attention and focus given to producing and disseminating information and materials in multiple UN languages. The language barrier was specifically targeted in the project Results Framework, with multiple results targets highlighting a focus on producing outputs in multiple languages.

 


[1] White, Howard. 2009. “Theory-based Impact Evaluation: Principles and Practice,” 3ie Working Paper 3, June 2009.

 


4.

I. Project Management and Cost-effectiveness (Efficiency)

A.Least-cost Approach: Efficiency of Strategy and Design

  1. The “efficiency” evaluation criterion is one of the areas that is being split into one rating for project development and approval, and one rating for project implementation. All issues related to efficiency are linked to issues with the project’s strategy and design, and overall timing of the project approval, as elaborated in Section IV.B above on relevance of the project strategy and design, and in later Section VI on project effectiveness. Project efficiency related to project development and approval aspects is rated moderately satisfactory. Even if perfectly implemented, a project cannot be cost-effective if issues related to its design and timing reduce its effectiveness.
  2. As part of the efficiency evaluation criteria, the terminal evaluation must consider if the project was the least-cost approach for achieving the objective, and if it was cost-effective in doing so. In Section IV.B above on relevance of the project strategy and design, and in later Section VI on project effectiveness, this evaluation highlights that the overall timing of the project did not allow it to be as effective as it could have been. Specifically, the project would have been more effective (and therefore more efficient) if it could have been timed so that all key NBSAP guidance was produced before countries began substantive work on revising their NBSAPs. 
  3. There are also elements of the project design that have yet to be demonstrated as cost-effective – namely the NBSAP Forum. The NBSAP Forum has clear value, and has the potential to be a useful dynamic resource, but it has not yet fulfilled this potential. If after the project it fades to another outdated, static global portal website, it will not have been a cost-effective use of resources. With its established audience, the 6NR project is also leveraging the NBSAP Forum. On the other hand, the use of internet-based technical support (e.g. through MOOCs, online distribution of technical guidance documents, and the NBSAP Forum) has allowed the project to reach a much larger global audience than it otherwise would have been able to do. There are a number of useful lessons related to the efficiency of the project design and timing, which are outlined in Section VIII.A of this report on lessons.

5.

A.Summary of Efficiency of Overall Implementation

  1. Project efficiency of implementation is rated satisfactory. The project has been implemented in a fully cost-effective manner. The project’s excellent communication with partners and stakeholders is a highlight, as is the project’s strong partnership approach. Project management costs are expected to be approximately 9.4% of GEF funding, in-line with GEF requirements for MSPs. Financial management procedures are in-line with international norms, and conform to UNDP and UN Environment policies and procedures. Project co-financing was fully delivered, with 124% of co-financing reported as of the terminal evaluation, and actual unreported co-financing is likely to be much higher. The project team is highly professional and has demonstrated excellent communication and coordination, and adhered to high standards in terms of planning, reporting, and financial management.

6.

A.UNDP and UN Environment Oversight

  1. UNDP and UN Environment are the GEF Agencies responsible for the project and carry general backstopping and oversight responsibilities. The agencies have fully and adequately supported the project during implementation, with no significant issues. Implementation by UNDP and UN Environment is considered satisfactory.

7.

A.Execution (Project Management)

  1. This was a direct implementation project, meaning that UNDP and UN Environment were also responsible for project management. In the case of UN Environment this was further delegated to UNEP-WCMC, while UNDP has an internal project team for project execution (project execution can also be considered “project management”). Project execution is considered highly satisfactory. The NBSAP project is characterized by highly professional and efficient project management, good financial planning, strong adaptive management, comprehensive reporting, transparent communication, and excellent engagement of partners.

8.

A.Partnership Approach and Stakeholder Participation

  1. The project’s overall partnership approach and stakeholder participation was strong, with excellent communication and cooperation, as attested broadly by stakeholders and participants interviewed for this evaluation. The UNDP and UN Environment partnership for the joint implementation of the NBSAP project has proven to be effective, with the UN agencies collectively supporting the development and revision of NBSAPs by providing technical capacity building support to 128 countries. Both UN agencies worked closely, and supported the delivery of various guidance documents, joint in-person workshops, online courses and over 75 webinars in various languages. UNDP and UN Environment teams created a congenial working environment of support, trust and knowledge exchange, which has been carried over to the Sixth National Report to CBD project. This is the first time that a robust partnership between UNDP, UN Environment and SCBD has been developed on supporting national biodiversity conservation planning.
  2. Leveraging the technical expertise from UNDP and UNEP-WCMC was also considered another strong part of the partnership. In addition, although the CBD Secretariat was not directly responsible for implementation, the Secretariat was regularly consulted and was engaged in the project implementation process. The project engaged “learning partners” who contributed to the delivery of various technical guidance, webinars, etc.; more than 40 different individual organizations, individuals or institutions contributed to the delivery of webinars within the scope of the project.
  3. The implementation approach of a joint project between UNDP and UN Environment faced some coordination challenges, but overall, by the end of the project, the partners felt that the approach had been positive. A single, joint, virtually staffed help facility with a 24-hour turnaround time on requests was created. Questions were circulated and answered by CBD, UNDP and UN Environment staff. Most of the technical reviews were also jointly developed. For example, the technical review of Papua New Guinea was conducted jointly by UNDP and UN Environment, with UNDP staff providing feedback per the technical review check list, and UNEP-WCMC providing spatial maps to ensure the delivery of a robust NBSAP to CBD.  

9.

A.Risk Assessment and Monitoring

  1. The NBSAP project document includes the project risk analysis (Table 2, p. 13 of the UNDP Prodoc). In the Prodoc three risks were identified. Risks were monitored during project implementation quarterly through UNDP’s Atlas risk log, and annually through the PIR. No critical risks were identified during the project’s implementation. Only three risks were identified at the Prodoc stage, which is a minimal number (most UNDP-GEF projects identify 4-6 risks at the Prodoc stage). This is not a typical project so the existence of fewer risks could have been possible, but in fact the risks identified did not provide sufficiently comprehensive risk analysis. The project did not have an inception report, in which risks are typically reviewed and updated. The number of risks identified does not alone serve as an evaluative point for assessment; however, many GEF projects have weak initial risk analysis where they fail to adequately identify many risks and adequately assess the ones they do identify. Therefore, considering the number and type of risks identified provides some basic insight on whether an appropriate risk analysis was conducted at the project development phase.
  2. The key risk faced by the project, relating to the uptake of technical support by the target countries, was included at the Prodoc phase: “Some governments may not subscribe to wide information- sharing platforms, and thereby engage ineffectively in accessing technical resources.” This risk was only rated as low, when the risk in fact appears to be higher.

10.

A.Flexibility and Adaptive Management

  1. Flexibility is one of the GEF’s ten operational principles, and all projects must be implemented in a flexible manner to maximize efficiency and effectiveness, and to ensure results-based, rather than output-based approach. Thus, during project implementation adaptive management must be employed to adjust to changing circumstances.
  2. The project was implemented in an adaptive and flexible manner, following a results-based approach. As one example, the project team adjusted the approach and modality for disseminating technical support based on initial feedback from the target audience, and based on assessment of the NBSAP forum website analytics showing the frequency of use of certain parts of the website. The project also made budget revisions throughout implementation, in accordance with UNDP, UN Environment and GEF standard procedures.
  3. There were some issues related to the project timing and design, and to address these the project partners took a flexible and adaptive management approach by effectively tailoring support which was suitable for global and individual country needs. For example, to encourage countries to participate in the voluntary NBSAP “peer” or “technical” review process, the NBSAP forum’s public peer review process was switched to a closed expert review process in most cases. Another example is that the project team adjusted the approach and modality for disseminating technical support based on initial feedback from the target audience, and based on assessment of the NBSAP forum website analytics showing the frequency of use of certain parts of the website. The project also made budget revisions throughout implementation, in accordance with UNDP, UN Environment and GEF standard procedures.
  4. Figure 2 below shows the breakdown of actual spending by year by component. Figure 3 shows the project planned and actual budget expenditure by year. Figure 4 below shows disbursement broken down between the two agencies, by year. When reviewing different aspects of the project financial management and delivery it is important to keep in mind that the project was planned for 30 months, which in the project document was foreseen as three consecutive calendar years. However, since the UNDP portion of the project began official implementation in July 16, 2014 and will not be completed until May 2018, it will have spanned five calendar years. Therefore, for example, in the project document there was no planned expenditure for 2017 or 2018. The UN Environment budget in the Prodoc is planned for the calendar years 2013, 2014, and 2015, but the project began implementation in 2014 and completed operations in 2016.

11.

A.Planned and Actual Co-financing

  1. The expected project co-financing was $2,206,620, with $1,000,000 expected from each of UNDP and UN Environment, and $206,620 (150,000 euros) from the Government of Flanders. The co-financing from the Government of Flanders was direct cash co-financing to be managed by UNDP, and has been included in the summary of project financial management, in Section V.G above. The additional $2,000,000 in co-financing from UNDP and UN Environment was cash. This is an expected co-financing ratio of 1.3 : 1. Table 6 below shows planned and actual co-financing. According to data provided by the project team, the project had received a total of approximately $2.49 million USD in co-financing as of April 30, 2018. This is 124% of the expected co-financing. The breakdown of co-financing is not tracked by project outcome; much of the co-financing has gone to support all aspects of the project.
  2. Other sources of likely in-kind co-financing have not been closely tracked or reported. For example, staff from the CBD Secretariat have worked closely with the project throughout its lifetime, in particular when there was a team within the Secretariat specifically supported by Japan that was tasked with providing support for the NBSAP process.

12.

A.Monitoring and Evaluation

  1. The NBSAP project M&E design generally meets UNDP and GEF minimum standards, but had shortcomings related to the design of the Strategic Results Framework, and is considered moderately unsatisfactory. M&E implementation is considered satisfactory, and therefore overall M&E is considered moderately satisfactory.

13.

I. Effectiveness and Results: Progress Toward the Objective and Outcomes

A.Effectiveness Issues Related to Project Approval Timing and Strategy

  1. For the Global NBSAP project, “effectiveness” is the most important evaluation criterion where it is necessary to distinguish between effectiveness issues related to the project development and approval vs. effectiveness of project implementation.
  2. Effectiveness related to project development and approval timing is rated moderately unsatisfactory. The timing of the project, and other factors, did not allow it to be as effective as it could have been. The project started after most countries had already received support from the GEF to revise their NBSAPs: A total of 122 countries had their NBSAP Enabling Activities approved prior to the Global NBSAP support project, and a large majority of these (110) were approved approximately two years prior (by August 2012) to the formal start-up of the Global NBSAP project in July 2014. As can be seen in Figure 5 below, the project only started operations at the point when a limited number of GEF-eligible countries began submitting their revised NBSAPs to the CBD Secretariat, but most had begun writing them. At the same time, the project came to an end when there were still 37 (29% of the supported 129) countries that had not submitted revised NBSAPs to the CBD Secretariat (some of which were due to political reasons), when these countries may have benefited from further support.

14.

A.Effectiveness of Implementation, and Overall Achievement of Objective and Outcomes

  1. Based on the assessment of progress toward targets in the project results framework, the NBSAP project has achieved the project objective and the two planned outcomes. Project results / achievement of overall outcomes is rated satisfactory, and effectiveness of project implementation is also considered satisfactory. The project exceeded 12 of 14 results indicator targets, fully met one additional target, and partially met one target. Key results achieved include:
  • 91 countries with revised NBSAPs submitted to the CBD (34 UNDP supported countries (76% of 45 total), and 57 UN Environment supported countries (69% of 83 total)
  • NBSAPs from 68 countries (53% of countries supported) have had technical / “peer” reviews
  • Among reviewed NBSAPs, 88% were assessed to have addressed Aichi Biodiversity Targets (ABT) 2, 3, 5, 11, 12, 15 and 20; 79% showed evidence of including diverse stakeholders in the revision process.
  • The NBSAP Forum was developed as a knowledge hub, with the following usage statistics (as of June 30, 2017)
    • 3,356 NBSAP Forum members speaking 149 Google recognized languages from every country in the world;
    • 27,794 users over the life of the project, and 16,000 unique NBSAP Forum users in the last 12 months
    • An average session duration of 4 minutes, with four pages viewed per session.
  • The project generated multiple forms of technical support, most of which are available in at least three UN languages:
    • 10 publications, 7 posters, 22 guidance documents, and 5 tools relevant for technical support for revising and updating NBSAPs
    • 19 eLearning online training courses, with 7,494 online learning course registrants
    • 44 webinars, with 3,298 live webinar participants (English, French, Spanish), and 6,520 recorded webinar participants (English, French, Spanish)
    • Four Massive Online Open Courses (MOOCs), with 2,425 participants (English, French, Spanish) in the first two MOOCs (“Protected Areas System Design and Management” and “Greening Consumption and Production”)
    • 184 best practices and case studies have been published on the NSBAP Forum
    • 3,458 subscribers to the NBSAP Forum Aichi Biodiversity Target newsletter and 6,097 subscribers to the NBSAP Forum learning newsletter 
  • Analyses of a user satisfaction survey suggested that 96% of NBSAP country teams were satisfied with the quality of technical support services received
  1. The project objective level results indicators are summarized in Table 7 below. Detailed assessment of achievement of the project results framework targets is included in Annex 10 to this report. In addition, detailed and specific information identifying many project results not covered in this section is available in the “Self-assessment” column of Annex 10, which includes the project results framework and the project’s reporting on indicators and targets from the 2017 PIR.

15.

i.Component 2: Direct technical support delivery [for NBSAP preparation and implementation] => Outcome 2: Targeted, timely and high quality technical support to countries enables the adoption of best practices, guidelines and other materials, and corroborate the long-term goal of developing the capacity of countries to carry out effective biodiversity planning.

  1. The second component of the project involved the direct technical support provided to countries for revising their NBSAPs to reflect the Aichi biodiversity targets. The total GEF funding for Outcome 2 was originally planned at $658,500 USD, which is 34.5% of the total GEF funding for the project. Actual expenditure as of March 31, 2018 was $672,102 USD. The component activities were organized around three outputs. Key results indicators for Component 2 are summarized in Table 9 below. The results of each output are summarized following the table.

16.

A.Impacts and Global Environmental Benefits

  1. For the GEF biodiversity focal area project impacts are defined as documented changes in environmental status of species, ecosystems or genetic biodiversity resources. Global Environmental Benefits have not been explicitly defined, but are generally considered to involve sustained impact level results of a certain scale or significance.
  2. The NBSAP project’s theory-of-change results-chain was far removed from the impact level; as discussed earlier in this report, the project’s theory-of-change is considered valid, and thus is expected to contribute to impacts in the long-term. In terms of actually achieving biodiversity impacts during the lifetime of the project, the project’s results are diffuse and too distant from the impact level to be able to draw out any specific project contributions to measurable impacts. The project’s Request for GEF MSP Approval document identifies the specific Global Environmental Benefits that the NBSAP project was expected to contribute to (p. 19 of the Request for MSP Approval). The results listed are higher-level outcomes, rather than impact-level results:
  • Successful mainstreaming of biodiversity into national development planning frameworks and sector planning processes.
  • Increased understanding about the role intact habitat and biodiversity plan to help humans adapt to climate change and advances in ecosystem service valuation provide an opportunity to incorporate this knowledge into the revision of NBSAPs.
  • At the level of individual NBSAPs, the project’s specific benefits will be: i) the valuing of ecosystem goods and services; ii) biodiversity mainstreaming; iii) the incorporation of challenges and opportunities linked to ecosystem-based adaptation and resilience; iv) the establishment of national Aichi-inspired targets and development of biodiversity indicators for monitoring implementation; v) the integration of spatial planning considerations; and vi) the inclusion of feasible NBSAP implementation plans, including and in particular resource mobilization plans for biodiversity.
  1. The project is not being assessed negatively due to not having direct impact-level results, as this was clearly not the strategy of the project. Considering the global scope of the project, and the many other national-level initiatives to conserve biodiversity, it would be extremely difficult to extract the project’s contribution to impact-level results. Theoretically the adoption of the NBSAPs as national policy documents (in many countries) will lead to impacts generated in the long-term, but these cannot be identified and assessed as part of this evaluation, which assesses impacts within the life of the project. Considering that the project strategy is not targeted at the impact level, it is not appropriate to try to identify impacts attributed to the project. Consequently, impact ratings for the project must be assessed as follows:
  • Environmental status improvement is assessed as negligible;
  • Environmental stress reduction is assessed as negligible; and
  • Progress toward stress/status change is assessed as negligible.

17.

I. Key GEF Performance Parameters

  1. Sustainability is one of the five main evaluation criteria, as well as being considered one of the GEF operational principles. Other GEF operational principles not otherwise addressed are discussed below, including the project’s catalytic role and stakeholder participation.
  2. UNDP-GEF project evaluations are also required to discuss the mainstreaming of UNDP program principles. This is covered in Annex 11 of this evaluation report.

A.Sustainability

  1. While a sustainability rating is provided here as required, sustainability is a temporal and dynamic state that is influenced by a broad range of constantly shifting factors. It should be kept in mind that the important aspect of sustainability of GEF projects is the sustainability of results, not necessarily the sustainability of activities that produced results. In the context of GEF projects there is no clearly defined timeframe for which results should be sustained, although it is implied that they should be sustained indefinitely. When evaluating sustainability, the greater the time horizon, the lower the degree of certainty possible.
  2. Based on GEF evaluation policies and procedures, the overall rating for sustainability cannot be higher than the lowest rating for any of the individual components. Therefore the overall sustainability rating for the NBSAP project is moderately likely. Providing a single rating for the sustainability of results for the NBSAP project is challenging because it relates to outcomes generated at the country level from the revised NBSAPs, as discussed in Section VI above on results and effectiveness.
  3. One aspect that will support sustainability generally is that both UNDP and UN Environment will be working to support countries with their 6th National Reports to the CBD, under a series of recently initiated projects. Many of the same staff members and implementation structures from the NBSAP project will be leveraged for the 6th National Reports projects.

18.

A.Catalytic Role: Replication and Up-scaling

  1. The overall strategy and outcomes of the project will inherently have a catalytic effect. The production of revised NBSAPs will have a catalytic effect in each of the participating countries – if they are actually implemented. In addition, the capacity development aspect of the project, through the trainings, webinars, NBSAP Forum, etc. will theoretically lead to improved quality of biodiversity conservation activities at the global level, but it is not feasible to assess the extent to which the project has contributed to this.

19.

A.Gender Equality and Mainstreaming

  1. Gender equality and mainstreaming was a strong point of the project, even though the project was designed prior to implementation of UNDP’s Gender Equality Strategy 2014-2017. As described in the PIR,

“The project team undertook a gender analysis at the policy and site level to understand the extent to which gender is addressed in NBSAPs, and more importantly, how can we systematically integrate gender as a core component in national biodiversity strategies and actions. We also sought to understand and develop pathways to overcome the difficulties to mainstream gender. To do so, we analyzed the level of commitment to gender mainstreaming as a tool to achieve biodiversity goals in 24 countries that were known to be gender responsive. We found that, most of the time, countries stated gender and women as a general idea in their NBSAP, without designing related concrete actions or comprehensive strategies, which often reflected a lack of understanding on how to mainstream gender systematically. As a second step, the project team analyzed small-scale, field-based Small Grants Programme and the Equator Initiative conservation projects that successfully mainstreamed gender at the local level. They identified the difficulties and barriers encountered during implementation of the projects and how they were overcome, as well as the possibilities of scaling-up such initiatives at the level of national biodiversity planning and policymaking. Using a key word search, we identified 30 Equator Initiative case studies that described projects that successfully mainstreamed gender and achieved conservation outcomes. Using these results, we then developed a common taxonomy to tag 78 NBSAP actions and 346 case study actions with the same vocabulary.”

  1. The project included specific outputs related to gender mainstreaming. For example, the project produced an e-Learning module on Gender Mainstreaming, with two lessons: “Lesson 1: Introducing Concepts: Biodiversity Conservation, Gender and Tools; Lesson 2: Mainstreaming Gender into NBSAP Implementation: Key Entry- Points.”
  2. One of the NBSAP Forum newsletters was dedicated to gender mainstreaming in biodiversity conservation planning processes. It shared several original best practices developed by the NBSAP project team, online courses and publications on gender mainstreaming.  Over 64 results comprising webinars, best practice, online learning, and resources show up on the NBSAP Forum on a search related to gender mainstreaming.
  3. One gap appears to be that gender mainstreaming is not sufficiently reflected in the NBSAP Peer Review protocol that was developed by the project; this could have been a key opportunity to collect additional insights and information on the mainstreaming of gender in NBSAPs. There also remains more work necessary to fully disseminate and distribute work on gender mainstreaming in NBSAPs. Another shortcoming was that the project lacked gender disaggregated indicators in the project results framework.

20.

i.Component 1: Global learning and technical content development [for enhancing the quality of NBSAPs] => Outcome 1: New and innovative knowledge management tools enhance global learning on biodiversity planning and support GEF-financed NBSAP development processes, so that NBSAPs become more relevant policy instruments, integrated into sectoral national plans strategies and policies, thereby making a significant contribution to achieving Aichi Target 17.

  1. The first component of the project is focused on developing guidance and knowledge products for capacity development supporting strengthening of NBSAPs. The total GEF funding planned for the component was $1,069,589 USD, which was 56.1% of the total GEF funding for the project. The actual expenditure as of March 31, 2018 was $1,064,685 USD. The component activities were organized around five outputs. Key results indicators for Component 1 are summarized in Table 8 below. The main activities and results of the five outputs are summarized following the table.
  1. Output 1.1 User-friendly, customizable tools and assessment methodologies, e-learning, voluntary templates and other guidance material, including for benchmarking the technical quality of NBSAP products before submission, are developed and widely applied in GEF-financed NBSAP development processes. They are primarily disseminated through the NBSAP Forum.
  2. The project produced a large number of tools, eLearning modules, and guidance documents. The full list of project technical assistance guidance outputs is attached as Annex 11 to this evaluation report. As previously highlighted, this included the following:
  • 10 publications, 7 posters, 22 guidance documents, and 5 tools relevant for technical support for revising and updating NBSAPs
  • 19 eLearning online training courses, with 7,494 online learning course registrants
  • 184 best practices and case studies have been published on the NSBAP Forum
  1. The project’s outputs were produced by many experts considered global leaders in their field, and are technically of high quality. All of these resources are a strong and highly valuable contribution to the knowledge base for global biodiversity conservation practice and policy. Given the valid theory-of-change for the NBSAP project, it is highly likely that these materials and activities have contributed to increased quality of revised NBSAPs. NBSAPs of many countries did benefit from these materials, as can be seen from the technically reviewed NBSAPs, which highlight that 88% of the NBSAPs have addressed ABT 2, 3, 5, 11, 12, 15 and 20; and 79% showed evidence of including diverse stakeholders in the revision process. Feedback from the technical reviews nudged countries towards better inclusion of a variety of national targets which correspond to various ABTs.
  2. Given the high value of these technical assistance outputs, it would be useful if these materials can continue to be proactively disseminated and absorbed by the global biodiversity conservation community of practice. UNDP has committed to maintaining the availability of these outputs online through the NBSAP Forum and other sources such as the conservation training website; however, it is unlikely that these products (especially items such as webinars and eLearning training courses) are likely to have much uptake if they are not proactively disseminated and promoted, which is not likely to happen once this project is completed. At the time of this evaluation, learning materials had been repurposed on several online learning portals, include the CBD online learning site, the UNDP Learning for Nature website, and by The Nature Conservancy, UNITAR and InforMEA. This issue is discussed further under later Section VII.A on sustainability.
  3. Output 1.2 Online spatial planning tools for key thematic areas and cross-cutting issues are made available to countries to facilitate biodiversity status assessments.
  4. The spatial data capacity development focus was addressed through a variety of activities and outputs. Multiple project participants highlighted the project’s work on capacity development for spatial data planning as one of the most valuable elements of the project. Spatial data planning support tools were included in the technical assistance outputs counted under Output 1.1 above (see full list in Annex 11). These included, for example, an eLearning training course titled “Incorporating and utilizing spatial data and mapping in NBSAPs” offered in 2015, and a technical guidance titled, “Mapping Biodiversity Priorities.” IBAT Country Profiles were also developed. According to the 2017 PIR, “UNDP/UNE completed 68 NBSAP peer reviews, of which 77% include some results of spatial planning analyses presenting biodiversity status and trends. However, a secondary analysis of Post-2010 National Biodiversity Strategies and Action Plans and 5th National Reports indicated that most spatial planning analyses is related to protected area networks only and is not sufficient for policy makers to take action and fully achieve the ABT.” Both UNDP and UN Environment are working to understand and continue addressing this gap. UNDP and UN Environment are developing a spatial data planning portal, leveraging spatial data layers from multiple global partners, including NASA, numerous universities, and initiatives such as Global Forest Watch.
  5. Output 1.3 The NBSAP Forum Web Portal is functional and well maintained: (i) fully operational by end 2013; (ii) further developed to fulfil evolving clients’ needs throughout the project’s duration; (iii) hosting and maintenance are taken over by CBD for sustainability.
  6. This output relates to the NBSAP Forum website, which was one of the major areas of focus and work of the project, both as resource itself, and as a means for disseminating specific individual tools, guidance documents, and other resources developed by the project. The NBSAP Forum was first online in the 4th quarter of 2013. The Forum website was redesigned and re-launched in early 2018. The NBSAP Forum was developed through a partnership of the main project partners (UNDP, UN Environment and CBD Secretariat), but other partners also contributed, including multiple international NGOs (e.g. BirdLife International, IUCN, WWF, The Nature Conservancy, etc.).
  7. Figure 9 below provides a summary of NBSAP Forum user analytics from July 1, 2014 to June 30, 2017.
  8. As of June 30, 2017 the NBSAP Forum had the following analytics user statistics:
  • 3,356 NBSAP Forum members speaking 149 Google recognized languages from every country in the world;
  • 209,719 NBSAP Forum page views over the life of the project, and 102,879 page views over the past year;
  • 27,794 users over the life of the project, and 16,000 unique NBSAP Forum users in the last 12 months, with 40% - 48% of visitors continuing to be being new throughout the project;
  • 58,433 NBSAP Forum sessions over the life of the project, and 33,000 unique NBSAP Forum sessions in the last 12 months; and
  • An average session duration of 4 minutes, with four pages viewed per session.

Qualitative data collected during the terminal evaluation regarding the NBSAP Forum was mixed (see Table 10 below) in terms of the current utility and sustainability of the NBSAP Forum. It remains an evolving resource, and its status and utility for the future remains difficult to say. There are a couple of key issues related to the NBSAP Forum; the first of these relates to the strategic approach of creating an online platform, while other issues relate to the implementation approach used for the NBSAP Forum.

Qualitative data collected during the terminal evaluation regarding the NBSAP Forum was mixed (see Table 10 below) in terms of the current utility and sustainability of the NBSAP Forum. It remains an evolving resource, and its status and utility for the future remains difficult to say. There are a couple of key issues related to the NBSAP Forum; the first of these relates to the strategic approach of creating an online platform, while other issues relate to the implementation approach used for the NBSAP Forum.

  1. number of NBSAP Forum users continues to grow, but it is not clear that the website will ultimately exist in the long-term (or even medium-term) as an active and dynamic portal for biodiversity conservation professionals. Project participants highlight the fact that although most countries have completed their revised NBSAPs, there remains the necessary focus on NBSAP implementation. However, as this phase of NBSAP revision wraps up, the frequency with which biodiversity professionals search online for “NBSAP” related topics will inevitably decline. “NBSAP implementation” is essentially just the broad field of biodiversity conservation. However, the CBD’s Sixth National Report is tied directly to NBSAP achievement, and the post-2020 agenda is also tied to this evaluation. Therefore, some sustained relevance is expected. To effectively engage a wide global audience on this wider agenda, the NBSAP Forum may need to be re-branded and re-marketed to its target global audience. This is an important issue, because there are so many online sources and “communities of practice” in the biodiversity conservation field, and practitioners have limited time and ability to engage through online resources. Many biodiversity communities of practice are captured or represented through IUCN, with its commissions (e.g. World Commission on Protected Areas), or through the CBD, or many other organizations and initiatives. One respondent pointed out that for the GEF biodiversity focal area there is no “IWLearn” or “CCLearn”. The NBSAP Forum could theoretically form a building block of a potential future “BDLearn”, but it definitely does not currently serve that function.
  2. This evaluation recommends that for the long-term the NBSAP Forum should be rebranded to emphasize its broader relevance to “NBSAP implementation” biodiversity conservation topics. It is also recommended that the NBSAP Forum be more coordinated and integrated with the CBD NBSAP webpage. This does not have to mean that the NBSAP Forum has to be presented as an integral part of the CBD website. This approach could emphasize the NBSAP Forum’s utility as a resource for guidance for National CBD Focal Points and biodiversity conservation practitioners. The functionality of the NBSAP Forum could also be expanded as a “portal of portals” that provide guidance on biodiversity conservation, such as the Panorama website, BES-NET, and many others. Managers of the NBSAP Forum website should also optimize online search criteria for Google, Bing and other search engines so that the NBSAP Forum appears highly ranked in search results when people are searching for relevant topics other than NBSAPs as well.
  3. The second set of key issues regarding the NBSAP Forum relate to the implementation approach for this activity. The NBSAP Forum was developed and deployed as a stand-alone web portal, not directly part of or linked to the CBD website, or other existing resources. It may be however noted, that the NBSAP Forum website was developed using the same software i.e. Drupal, to ensure a smooth integration within the CBD website. There were a variety of valid reasons that this approach was chosen, after extensive discussions between the key project partners – UNDP, UN Environment, and the CBD Secretariat. However, NBSAPs are specifically and directly part of countries’ responsibilities under the CBD, and the CBD website (specifically the NBSAP webpage of the website) is logically the starting point for those looking for information related to NBSAPs. The CBD webpage includes a small link to the NBSAP Forum, but there was not a well-developed connection between the two websites. There was also not good coordination in terms of the resources available online for developing NBSAPs; for example, the CBD NBSAP website includes links to information on the following topics:
  • What is an NBSAP?”
  • “COP Guidance on NBSAPs”
  • “Other Guidance on NBSAPs”
  • “NBSAP Capacity Building Modules”
  • “NBSAP Capacity Building Workshops”
  1. These are all topics covered in the NBSAP Forum resources, often in more comprehensive and updated approaches. The CBD NBSAP website includes links to guidance documents from various sources from 2012 and earlier, including as far back as 1993. Biodiversity professionals coming to the CBD NBSAP website may naturally draw on these guidance documents as “officially sanctioned” by the CBD, and look no further due to lack of time or interest. Although there is no reason the CBD NBSAP website and the NBSAP Forum can’t co-exist, they need to be well-coordinated to ensure that the target audience is quickly and easily directed to the most relevant resources on NBSAPs.
  2. Some individuals interviewed for this evaluation felt that the NBSAP Forum could have been more effective and would have better sustainability if it was well-integrated with the CBD website, including providing some of the functions of the CBD Clearing House Mechanism.
  3. Many respondents felt that the NBSAP Forum had been very useful as a node for distributing information, tools, guidelines etc. relating to NBSAP development. A review of the NBSAP Forum conducted during this evaluation found that the platform was not always easily and logically organized in terms of access to resources, and the search function often did not work effectively; this has improved significantly with the NBSAP Forum redesign, which was launched just prior to project completion (April 2018).
  4. A final issue is that, anecdotally, internet connectivity does appear to have been an issue for at least some countries to effectively use the NBSAP Forum. This is likely to have been more of an issue for LDCs and SIDS, which have less extensive and less reliable internet connectivity. Countries at the bottom of the 2018 Global Connectivity Index[1] (which only includes 79 countries) included Ethiopia, Bangladesh, Bolivia, Uganda, Paraguay and Algeria.

In terms of short-term operational sustainability, the project partners have already agreed to provide funding, operational support, and technical support for the ongoing existence of the NBSAP Forum through 2020.

  1. Output 1.4 A partnership framework for collaboration among all agencies and entities involved in NBSAP process emerges with a view to supporting client countries and developing best practices.
  2. This output primarily relates to the overall implementation and execution approach of the project. The project’s partnership was one of the strong aspects of the project, as discussed previously in Section V.D on Partnership Approach and Stakeholder Engagement. The effectiveness of the project’s implementation approach is also discussed in previous Section VI.ii on Effectiveness. The project’s partnership approach was strong in the sense of the cooperative approach, coordination, and communication between UNDP and UN Environment (including UNEP-WCMC).
  3. Output 1.5 Capacity to Incorporate Climate Change Adaptation and Resilience Planning into NBSAPs is strengthened through the NBSAP Forum
  4. The area of the project under Output 1.5, which related to climate change adaptation and resilience, was to be funded specifically from the $206,620 cash co-financing contribution from the Government of Flanders, Belgium. The project integrated climate change adaptation and resilience in a number of the project activities and outputs, but there were some outputs specifically addressing this topic. The project collaborated with SwedBio and other partners on activities relating to climate change adaptation and resilience.
  5. For example, the Stockholm Resilience Center’s SwedBIO program, UNDP and MELCA-Ethiopia jointly organized a Multi-actor Resilience Dialogue between 12-14 November 2015 in Addis Ababa, Ethiopia, in order to explore key resilience concepts, multiple approaches for assessing resilience, and to identify steps for integrating social-ecological resilience principles and resilience thinking into development and biodiversity planning frameworks. The dialogue offered an opportunity for policy makers, scientists and practitioners to analyze various approaches and provide recommendations concerning resilience thinking, assessments and mainstreaming by focusing on how resilience is understood and managed in a variety of contexts. The report on the workshop is available at https://swed.bio/wp-content/uploads/2016/11/Resilience-Dialogue-Report_2016.pdf.
  6. The project also partnered with the Stockholm Resilience Center of other guidance documents and e-Learning modules. For example, an online course was developed in 2017 on “Applying Resilience Thinking to Biodiversity and Development Plans” (https://www.conservationtraining.org/enrol/index.php?id=303). Another e-Learning course was developed on “Understanding Resilience Thinking” with six individual lessons.
  7. Workshops on resilience were also organized at the 13th CBD COP in Cancun, Mexico (December 2016) and the World Conservation Congress in Hawaii (September 2016).
  8. Another major output was a Massive Online Open Course (MOOC) on resilience: “Introduction to Resilience for Development: Understanding Resilience Thinking” was a two-part MOOC. Part 1 was a four-week, facilitated course that ran from 31 October through 27 November and was offered free of cost in English, French, and Spanish for policymakers and practitioners working in conservation and development. The MOOC introduced the concept of resilience and showed how participants can use resilience assessments to address pressing global challenges. Participants learned about the key steps for conducting resilience assessments, how to develop resilience assessments using existing frameworks, and how to use these results to develop effective sustainable development and conservation actions. Part 2 ran in the first quarter of 2018.

 


[1] https://www.huawei.com/minisite/gci/en/country-rankings.html.


21.

i.Financial Risks

  1. The NBSAP project has limited financial sustainability exposure, based on the overall design and strategy of the project. There are three main considerations for financial sustainability: a.) implementation of revised NBSAPs at the national level; b.) ongoing use and uptake of the tools and knowledge products produced by the project; and c.) the future status of the NBSAP forum website, which is one of the key outputs of the project, and one which is clearly intended to remain active after the life of the project.
  2. Regarding point a.), within the scope of the NBSAP project and this evaluation, it is only applicable and feasible to assume that broadly speaking there will be financial resources available (both from national budgets and donor funding) at the national level to support implementation of revised NBSAPs, although this will vary significantly from country to country.
  3. Regarding point b.), there are not significant financial requirements for the ongoing uptake and promotion of the project knowledge outputs; sustainability in this regard is more dependent on socio-political and institutional aspects. At the same time, the ongoing dissemination and uptake of these outputs would be more likely if there were dedicated human resources focused on this task – as there was during the project implementation. The knowledge products and outputs have been made available online (on the NBSAP Forum, and on the www.conservationtraining.org website) where they can be accessed by interested practitioners, but this will hardly have the same effect as when these products were being actively promoted. Given the high value of these technical assistance outputs, the NBSAP project partners are undertaking several steps to ensure that these materials are repurposed and widely circulated as part of the project’s sustainability plan. The NBSAP project has forged partnerships with platforms such as UNITAR and InforMEA to showcase the NBSAP project’s online learning products i.e. online courses and webinars on these respective platforms. Most of the material developed under the project, particularly aiming towards the achievement of ABTs is being repurposed for the 6NR project.
  4. Regarding point c.), for short-term future the financial sustainability of the NBSAP Forum appears to be secure. UNDP has made arrangements with UN Environment and the CBD Secretariat for web-hosting and internal financial commitments to support the NBSAP Forum at least through 2020.
  5. Based on the above considerations, on the whole, financial sustainability of the NBSAP project is considered likely.

22.

i.Socio-political Risks

  1. Socio-political risks to sustainability are perhaps the most significant, because a major assumption for the results of the project to be sustained and lead to biodiversity impacts is for the NBSAPs to actually be implemented at the national level. This requires national ownership of the NBSAP documents, and national adoption. There are widely varying levels of stakeholder ownership and national adoption for NBSAPs among the 128 countries that were supported under the project. Even if NBSAPs are not formally adopted at the national level, it is anticipated that they will, in most cases, have a positive influence on biodiversity conservation efforts in their respective countries. Sustainability in this regard is considered moderately likely.

23.

i.Institutional and Governance Risks

  1. Institutional and governance issues related to sustainability are not critical, and sustainability in this regard is considered likely. The major question is whether the responsible institutions in each country will have the institutional capacity to achieve national adoption of their revised NBSAPs, and then to actually implement revised NBSAPs over the next 5-10 years. The institutional capacity for government institutions responsible for biodiversity conservation varies widely between countries. There is also frequent staff turnover at the national level, which is a challenging issue for the sustainability of any global capacity development effort. Governance is not a critical issue beyond the institutional issues above, which are linked to the level of political will in any given country to support biodiversity conservation.

 


24.

ii.Environmental Risks

  1. Environmental risks to sustainability are also not critical, and this aspect of sustainability is considered likely. The nature of the majority, if not all, of the project results (outputs and outcomes) means they are not susceptible to environmental risks – i.e. knowledge products, capacity development activities, increased awareness and understanding, etc. The main objective of the project, the production of high quality revised NBSAPs, is not applicable for environmental risks.

25.

Heading 1

Heading 2

Heading 3

Body text

Bold text

Italic text

  1. Numbered list
  2. Numbered list
  3. Numbered list
  • Bullits
  • Bullits
  • Bullits

Quote text

Hyperlinks

Column header Column header
Cell Cell
Cell Cell

 

Anchor tag


Recommendations
1

The GEF, UNDP, UN Environment and CBD Secretariat should be planning immediately for what type of enabling activity support will be extended to countries immediately following the CBD COP in 2020, with the objective of being prepared to disburse resources as quickly as possible after the 2020 COP to support planning, implementation, monitoring and reporting for the post-2020 strategic plan. [GEF Secretariat, UNDP, UN Environment, CBD Secretariat]

2

The GEF, UNDP, UN Environment, and CBD Secretariat should not embark on a new round of enabling activity funding for another NBSAP updating and revision process in response to the CBD 2021-2030 strategic planning period. Support will be required over the 2020-2025 timeframe for implementation of the current NBSAPs, many of which go to 2025 or 2030. Funding under the GEF enabling activities may be allocated to support NBSAP implementation in GEF-eligible countries. [GEF Secretariat, UNDP, UN Environment, CBD Secretariat]

3

The CBD strategic planning process for the 2021-2030 period should reflect current levels of national progress toward the ABTs. The revised strategic plan  should focus on incentivizing further incremental progress by countries, recognizing that parties are really only beginning implementation of their NBSAPs that were revised to reflect the ABTs. [CBD Secretariat, CBD Conference of Parties]

4

Considering the previous three recommendations, GEF Enabling Activity support to countries should focus on institutional and systemic capacity development at the national level, rather than individual capacity development. There should be an analysis of what makes national institutions responsible for biodiversity conservation effective, and then efforts to replicate those good practices to other countries. There are some indications that countries producing well-developed NBSAPs are the ones who do not require GEF assistance, and the systemic and institutional good practices from these countries should be replicated. Similar analytical work should be done in relation to types of national consultation processes and types of stakeholder engagement that have proven effective – for example, the extent to which civil society or the private sector have been involved in the NBSAP development process. [GEF, UNDP, UN Environment]

5

It would be useful to provide intensive targeted additional support to the 20 GEF-eligible countries that still do not have updated NBSAPs (the majority of which are LDCs and SIDS). However, it would be prudent to structure any such support so that countries that do not have revised NBSAPs by 2020 can incorporate the post-2020 CBD strategic plan. [GEF, UNDP, UN Environment]

6

For the long-term, the NBSAP Forum should be rebranded to emphasize its broader relevance to “NBSAP implementation” and execution of the post-2020 agenda. It is also recommended that the NBSAP Forum be more coordinated and integrated with the CBD NBSAP webpage. [UNDP, UN Environment, CBD Secretariat]

7

Future GEF Enabling Activity support to parties should explore the potential benefits of leveraging regional organizations (e.g. SPREP, CARICOM, etc.) to help provide Enabling Activity support for countries, in order to potentially further enhance efficiency and sustainability. It is more effective, efficient and impactful when the implementing agencies work with each other, and in coordination with regional agencies to deliver technical support and guidance. [GEF, UNDP, UN Environment, CBD Secretariat ]

8

UNDP and UN Environment should conduct a willingness(/ability)-to-pay survey of previous users of eLearning products (webinars, MOOCs, etc.) to assess the potential and appropriateness of instituting a payment-based system as part of a longer-term solution to financially sustaining this type of capacity support program. [UNDP, UN Environment]

9

Through the engagement of the community of practice in the capacity development program, this project has generated a wealth of data on the status and trends of the current global state of biodiversity conservation planning. UNDP and UN Environment should produce a summary analysis of their user databases, trends in topic interest, and other key data to submit to the CBD as an input to the post-2020 CBD strategic planning process. [UNDP, UN Environment]

10

If it is not possible to sustain such a capacity development program in its current form, UNDP and UN Environment should conduct a systematic analysis of their other relevant ongoing initiatives and opportunities to continue leveraging and disseminating and promoting the large library of eLearning modules, guidelines, etc. This could include, for example, requiring that all GEF project managers (and project team members) working on PA projects have successfully completed the eLearning modules related to PAs, PA financing, etc. It could also include, for example, ensuring that UNDP and UN Environment efforts in global forums such as the CBD COP and World Conservation Congress continue to promote and advertise the use of these eLearning modules. [UNDP, UN Environment]

11

It is considered good practice for GEF projects to have at least one project-specific audit during their lifetime (particularly when it is indicated in the project M&E plan), as audits usually result in a strengthening of financial management procedures, and reduce risks related to financial management. This evaluation recommends that UNDP-GEF and UN Environment-GEF projects have at least one audit during their lifetime. [UNDP, UN Environment]

1. Recommendation:

The GEF, UNDP, UN Environment and CBD Secretariat should be planning immediately for what type of enabling activity support will be extended to countries immediately following the CBD COP in 2020, with the objective of being prepared to disburse resources as quickly as possible after the 2020 COP to support planning, implementation, monitoring and reporting for the post-2020 strategic plan. [GEF Secretariat, UNDP, UN Environment, CBD Secretariat]

Management Response: [Added: 2019/03/19]

UNDP agrees with this recommendation The GEF, UNDP, UNE and CBD Secretariat are in the process of implementing enabling activities related to the Sixth National Report (6NR). They are also in the early stages of developing a GEF medium-sized project proposal for work in 2019 and 2020, as well as a full-sized project proposal to support planning, implementation, monitoring and reporting for the post-2020 CBD Strategic Plan. Their objective is to be  prepared to quickly disburse resources, although each project must ideally also allow adequate time to prepare those resources.

Key Actions:

Key Action Responsible DueDate Status Comments Documents
1.2. Work with GEF Secretariat, UNDP, UNE, CBD Secretariat to prepare GEF 7 MSP for 2019-2020
[Added: 2019/03/19] [Last Updated: 2020/12/18]
UNDP, GEF 2021/07 Overdue-Initiated The CBD COP is delayed until the fall of 2021 and the consultations proceeding the COP are also delayed. Conversations among the partners are in progress with the GEF Secretariat and partners. UNDP plans to submit the proposal during the first quarter of 2021. History
1.1. Implement 6NR Global Support Project
[Added: 2019/03/19] [Last Updated: 2020/12/18]
UNDP Istanbul Regional Hub 2020/06 Completed Implementation of the 6NR project was completed in June 2020 and the project is closed. (Note it is not overdue – action was completed on time). Terminal evaluation can be accessed here - https://undpgefpims.org/attachment-revision-file?attachmentRevisionId=1765676 History
1.3. Work with GEF Secretariat, UNDP, UNE, CBD Secretariat to prepare a Full-sized project proposal to support planning, implementation, monitoring and reporting for the post-2020 Strategic Plan.
[Added: 2019/03/19] [Last Updated: 2020/12/18]
UNDP, GEF 2021/12 Not Initiated The CBD COP is delayed until the fall of 2021 and the consultations proceeding the COP are also delayed. Conversations among the partners are in progress with the GEF Secretariat and partners regarding the planning, implementation, monitoring and reporting for the post-2020 Strategic Plan. The timeline of this submission is pending the rescheduling the of CBD COP. This will likely be a series of MSPs rather than a full-sized proposal. History
2. Recommendation:

The GEF, UNDP, UN Environment, and CBD Secretariat should not embark on a new round of enabling activity funding for another NBSAP updating and revision process in response to the CBD 2021-2030 strategic planning period. Support will be required over the 2020-2025 timeframe for implementation of the current NBSAPs, many of which go to 2025 or 2030. Funding under the GEF enabling activities may be allocated to support NBSAP implementation in GEF-eligible countries. [GEF Secretariat, UNDP, UN Environment, CBD Secretariat]

Management Response: [Added: 2019/03/19]

UNDP feels this recommendation is outside the scope of this evaluation. This recommendation focuses on high-level strategic policy decisions that Parties to the CBD will be making regarding the 2021-2030 strategic planning period. Projects like these respond to decisions that Parties make, but do not create the decisions. For example, based on the new strategic plan, the COP will make recommendations related to the frequency with which NBSAPs are revised, and if a focus on implementing the current NBSAPs is also needed.  Global enabling activities projects provide technical support related to these decisions. Additionally, 50% of NBSAPs only go through 2020, and will therefore need to be updated. New targets will also be updated.

UNDP supports the second half of the statement that enabling activities funding from the GEF should be used to support Parties with NBSAP implementation. This is within the scope of what the implementing agencies (UNDP, UNENV) can do. There is also an advantage to keeping the GEF, UNDP, UNE, CBD Secretariat partnership in place to support Parties to better implement the CBD, as intuitional synergies, knowledge, relationships would all be lost if the partnership did not continue.

Key Actions:

Key Action Responsible DueDate Status Comments Documents
2.1. Work with GEF Secretariat, UNDP, UNE, CBD Secretariat to prepare GEF 7 MSP for 2019-2020
[Added: 2019/03/19] [Last Updated: 2020/12/18]
UNDP, GEF 2021/12 Initiated The CBD COP is delayed until the fall of 2021 and the consultations proceeding the COP are also delayed. Conversations among the partners are in progress with the GEF Secretariat and partners regarding the planning, implementation, monitoring and reporting for the post-2020 Strategic Plan. The timeline of this submission is pending the rescheduling the of CBD COP. This will likely be a series of MSPs rather than a full-sized proposal. History
3. Recommendation:

The CBD strategic planning process for the 2021-2030 period should reflect current levels of national progress toward the ABTs. The revised strategic plan  should focus on incentivizing further incremental progress by countries, recognizing that parties are really only beginning implementation of their NBSAPs that were revised to reflect the ABTs. [CBD Secretariat, CBD Conference of Parties]

Management Response: [Added: 2019/03/19]

UNDP feels this recommendation is outside the scope of this evaluation. This recommendation focuses on high-level strategic policy decisions that Parties to the CBD will be making regarding the 2021-2030 strategic planning period. Projects like these respond to decisions that Parties make, but do not create the decisions. However, UNDP, UNE and the CBD Secretariat are working together during the 6NR project to support Parties to more accurately assess their current levels of national progress towards achieving the ABTs and implementing their NBSAPs.

Key Actions:

Key Action Responsible DueDate Status Comments Documents
3.1 Continue implementing 6R project.
[Added: 2019/03/19] [Last Updated: 2020/12/18]
UNDP IRH 2020/06 Completed Implementation of the 6NR project was completed in June 2020 and the project is closed. (Note it is not overdue – action was completed on time). History
4. Recommendation:

Considering the previous three recommendations, GEF Enabling Activity support to countries should focus on institutional and systemic capacity development at the national level, rather than individual capacity development. There should be an analysis of what makes national institutions responsible for biodiversity conservation effective, and then efforts to replicate those good practices to other countries. There are some indications that countries producing well-developed NBSAPs are the ones who do not require GEF assistance, and the systemic and institutional good practices from these countries should be replicated. Similar analytical work should be done in relation to types of national consultation processes and types of stakeholder engagement that have proven effective – for example, the extent to which civil society or the private sector have been involved in the NBSAP development process. [GEF, UNDP, UN Environment]

Management Response: [Added: 2019/03/19]

UNDP feels it is important to continue to focus on institutional and systemic capacity development at the national level, as well as individual capacity development. An institution is made of individuals, and these types of projects continue to prove that building the capacity of the individuals within an organization improves the capacity of the organization. The locus of change is typically at the individual level. It is extremely challenging to change a government institution that you have no control over, and that is signed on to a voluntary agreement. However, as this project proves, reducing the barriers that individuals have to capacity building can change elements of an institutional culture over time.

 

The recommended institutional analyses of the components of an effective institution and stakeholder consultations could prove informative, and some best practices could be distilled. However, the best practices achieved by countries like Mexico and Costa Rica are likely not achieved in other countries due to insurmountable constraints. Because of these overriding factors, what works in one country, might not work in another. UNDP is working with the countries it is supporting in the 6NR project to conduct capacity needs assessment at the NBSAP target level. However, in most cases rudimentary data is being returned.

 

UNDP disagrees that that GEF-eligible countries should no longer receive GEF assistance. These countries typically use their funding according to the project documents and timelines, and submit the documents on time, but spend all of the allotted funding. These countries produce best practices that can be documented for other GEF-eligible countries, and replicated where capacity exists to  do so.

Key Actions:

Key Action Responsible DueDate Status Comments Documents
4.1 Best practices developed for NBSAP project and posted on NBSAP Forum
[Added: 2019/03/19]
UNDP, GEF 2018/12 Completed
4.2 Best practices developed for 6NR project and posted on NBSAP Forum
[Added: 2019/03/19] [Last Updated: 2020/02/03]
UNDP IRH 2019/11 Completed Completed June 2019 History
4.3 Capacity needs assessment developed with Parties based on the results of their 6NR (which includes an assessment of their NBSAP)
[Added: 2019/03/19] [Last Updated: 2020/12/18]
UNDP IRH 2020/12 Completed We completed a capacity needs assessment based on the results of the 6NR and NBSAPS and made policy recommendations accordingly. Documents provided. History
5. Recommendation:

It would be useful to provide intensive targeted additional support to the 20 GEF-eligible countries that still do not have updated NBSAPs (the majority of which are LDCs and SIDS). However, it would be prudent to structure any such support so that countries that do not have revised NBSAPs by 2020 can incorporate the post-2020 CBD strategic plan. [GEF, UNDP, UN Environment]

Management Response: [Added: 2019/03/19]

UNDP supports this recommendation. Through the 6NR project, it is providing targeted technical support to 8 countries that are in this situation:  1. Bahamas 2. Barbados 3. Bolivia 4. Haiti 5. Kazakhstan 6. PNG 7. Saint Lucia 8. Uzbekistan. Haiti does not have an NBSAP. The remaining 7 countries do not have a post-2010 NBSAP.  The project team also worked in the second half of 2018 to help countries such as Micronesia and the Cook Islands complete their post-2010 NBSAP and submit it to the CBD.

Key Actions:

Key Action Responsible DueDate Status Comments Documents
5.1. Support the 8 countries without a post-2020 NBSAP to complete their 6NR, while considering how that process can be a basis to build on for the preparation and implementation of the next strategic plan and future NBSAP development. During this process, the team is supporting these countries to understand what national activities are linked to what ABTs, and how they contributed still, even if not included in an NBSAP. The 6NR project is also helping these countries to highlight the gaps where data are missing and provide recommendations on how to ensure data is available when designing and implementing the future NBSAP. These countries are also being guided to focus on capacity needs that come up during the design and implementation of NBSAP now, and to consider how to retain a similar team to design and implement the NBSAP.
[Added: 2019/03/19] [Last Updated: 2020/02/03]
UNDP IRH 2019/11 Completed Completed November 2019 History
6. Recommendation:

For the long-term, the NBSAP Forum should be rebranded to emphasize its broader relevance to “NBSAP implementation” and execution of the post-2020 agenda. It is also recommended that the NBSAP Forum be more coordinated and integrated with the CBD NBSAP webpage. [UNDP, UN Environment, CBD Secretariat]

Management Response: [Added: 2019/03/19]

UNDP supports this assessment.

Key Actions:

Key Action Responsible DueDate Status Comments Documents
6.1. UNDP continues conversations with CBD about rebranding the NBSAP and using it for execution of the post-2020 agenda.
[Added: 2019/03/19] [Last Updated: 2020/12/18]
UNDP, GEF 2020/12 Completed Complete. The NBSAP will be rebranded for execution of the post-2002 agenda. History
7. Recommendation:

Future GEF Enabling Activity support to parties should explore the potential benefits of leveraging regional organizations (e.g. SPREP, CARICOM, etc.) to help provide Enabling Activity support for countries, in order to potentially further enhance efficiency and sustainability. It is more effective, efficient and impactful when the implementing agencies work with each other, and in coordination with regional agencies to deliver technical support and guidance. [GEF, UNDP, UN Environment, CBD Secretariat ]

Management Response: [Added: 2019/03/19]

UNDP supports this recommendation.

Key Actions:

Key Action Responsible DueDate Status Comments Documents
7.1 The 6NR project is working with regional organizations such as SPREP and BIOPAMA networks to influence efficiency and impact
[Added: 2019/03/19] [Last Updated: 2020/02/03]
UNDP IRH 2019/11 Completed Completed January 2019 History
7.2. Future EA projects will seek to incorporate the support of regional organizations during the project design phase.
[Added: 2019/03/19] [Last Updated: 2020/12/18]
UNDP, GEF 2019/12 Completed UNDP has developed and implemented a process that incorporates regional organizations during the project design phase. History
8. Recommendation:

UNDP and UN Environment should conduct a willingness(/ability)-to-pay survey of previous users of eLearning products (webinars, MOOCs, etc.) to assess the potential and appropriateness of instituting a payment-based system as part of a longer-term solution to financially sustaining this type of capacity support program. [UNDP, UN Environment]

Management Response: [Added: 2019/03/19]

UNDP supports this recommendation and is actively surveying it user base; however, UNDP also feels that it is important for its users to have access to free and widely available capacity building opportunities that they could not otherwise access. UNDP is currently includes a willingness(/ability)-to-pay survey question in its post-course surveys to assess the potential and appropriateness of instituting a payment-based system as part of a longer-term solution to financially sustaining this type of capacity support program.  This survey question should have reached a number of our previous participants and all new enrollees in our new courses.  We have not yet checked to see how many people have filled it out or analyzed the results. The survey base could be further expanded to reach the NBSAP membership base.

Key Actions:

Key Action Responsible DueDate Status Comments Documents
8.1. Analyze survey responses related to the willingness(/ability)-to-pay for learning opportunities.
[Added: 2019/03/19] [Last Updated: 2020/02/03]
UNOPS 2019/06 Completed Completed in February 2019 History
9. Recommendation:

Through the engagement of the community of practice in the capacity development program, this project has generated a wealth of data on the status and trends of the current global state of biodiversity conservation planning. UNDP and UN Environment should produce a summary analysis of their user databases, trends in topic interest, and other key data to submit to the CBD as an input to the post-2020 CBD strategic planning process. [UNDP, UN Environment]

Management Response: [Added: 2019/03/19]

UNDP supports this recommendation and is currently working on a related capacity needs assessment on the global state of biodiversity conservation planning. This key data will be submitted to the CBD.

Key Actions:

Key Action Responsible DueDate Status Comments Documents
9.1. Develop capacity needs assessment
[Added: 2019/03/19] [Last Updated: 2020/02/03]
UNDP IRH 2019/09 Completed Completed in September 2019 History
10. Recommendation:

If it is not possible to sustain such a capacity development program in its current form, UNDP and UN Environment should conduct a systematic analysis of their other relevant ongoing initiatives and opportunities to continue leveraging and disseminating and promoting the large library of eLearning modules, guidelines, etc. This could include, for example, requiring that all GEF project managers (and project team members) working on PA projects have successfully completed the eLearning modules related to PAs, PA financing, etc. It could also include, for example, ensuring that UNDP and UN Environment efforts in global forums such as the CBD COP and World Conservation Congress continue to promote and advertise the use of these eLearning modules. [UNDP, UN Environment]

Management Response: [Added: 2019/03/19]

UNDP supports this recommendation. UNDP has leveraged the resource that were created during this project to work with FAO and WWF to create private course rooms, and to share modules with UNITAR, INFORMEA and the CBD Secretariat. UNDP has started leading quarterly instructional design calls which bring awareness of our products to a wide range of UN agencies, World Bank, and NGOs.

Key Actions:

Key Action Responsible DueDate Status Comments Documents
10.1. Continue ongoing collaborations
[Added: 2019/03/19] [Last Updated: 2020/12/18]
UNDP, GEF 2019/12 Completed Collaborations from the NBSAP project have been placed within existing projects and they remain ongoing. History
11. Recommendation:

It is considered good practice for GEF projects to have at least one project-specific audit during their lifetime (particularly when it is indicated in the project M&E plan), as audits usually result in a strengthening of financial management procedures, and reduce risks related to financial management. This evaluation recommends that UNDP-GEF and UN Environment-GEF projects have at least one audit during their lifetime. [UNDP, UN Environment]

Management Response: [Added: 2019/03/19]

:  As per the UNDP audit policy cited below, audits for low risk DIM projects are not conducted per project but per unit and UNDP-GEF unit is annually audited.  These projects are rated as low risk, therefore individual project audit was not required.

 

Here is the supporting excerpt from the UNDP audit policy:

Audit Policies Applicable to Directly Implemented Projects (DIM)

The projects that are directly implemented by UNDP are covered during the periodic audits that are carried out by UNDP Office of Audit and Investigations (OAI) as well as the financial audits carried out by the UN Board of Auditors (UN BoA), UNDP external auditors. 

Notwithstanding, a separate audit of a directly implemented project may be required. Some DIM projects have a limited duration and/or could be large, complex or high risk, that they may not be adequately covered by only relying on the periodic audits performed by OAI of the audits performed by the UN BoA

In these instances, OAI commissions the audit of a DIM project to supplement its periodic audits of UNDP business units and improve the level of assurance that it provides to stakeholders.  The audit of DIM projects could be performed by either by an audit firm hired by OAI to carry out a financial audit on its behalf (a majority of cases) or by OAI where an audit of systems and controls is performed.

As in the earlier scenarios, the audit of DIM projects is risk-based where the risk assessment is performed by OAI using eight risk indicators that are assigned to each DIM project (four quantitative such as the volume of expenditure and four qualitative such as exceptional circumstances or political situations).  Each DIM project is given risk rank and the following thresholds are applied to select those DIM project that require an audit:

  • if the DIM project is rated high risk and its annual spend is ≥ $2,000,000;
  • if the DIM project is rated medium risk and its annual spend is ≥ $4,000,000; and
  • if the DIM project is rated low risk and its annual spend is ≥ $6,000,000

The follow-up on the audit issues and audit recommendations arising from the audit of DIM projects is performed by OAI as an integral part of its follow-up on all OAI-issued audit recommendations.”

Key Actions:

Key Action Responsible DueDate Status Comments Documents
11.1. No action needed
[Added: 2019/03/19]
N/A 2018/12 No Longer Applicable [Justification: N/A Audit was not required]

Latest Evaluations

Contact us

1 UN Plaza
DC1-20th Floor
New York, NY 10017
Tel. +1 646 781 4200
Fax. +1 646 781 4213
erc.support@undp.org