- Evaluation Plan:
- 2021-2025, Indonesia
- Evaluation Type:
- Final Project
- Planned End Date:
- 03/2022
- Completion Date:
- 02/2022
- Status:
- Completed
- Management Response:
- Yes
- Evaluation Budget(US $):
- 80,000
Terminal Evaluation Final Report: “Transforming Effectiveness of Biodiversity Conservation in Priority Sumatran Landscapes (Tiger Project)”
| Document | Type | Language | Size | Status | Downloads |
|---|---|---|---|---|---|
 UNDP-GEF_Tiger Project_Final_TE_Report_clean_v2.0_23Jan2022 (signed).pdf |
report | English | 4341.06 KB | Posted | 272 |
| TOR Terminal Evaluation GEF -TIGER Project 2021 (1).pdf |
tor | English | 1181.14 KB | Posted | 38 |
| Title | Terminal Evaluation Final Report: “Transforming Effectiveness of Biodiversity Conservation in Priority Sumatran Landscapes (Tiger Project)” | ||||||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| Atlas Project Number: | 00085001 | ||||||||||||||
| Evaluation Plan: | 2021-2025, Indonesia | ||||||||||||||
| Evaluation Type: | Final Project | ||||||||||||||
| Status: | Completed | ||||||||||||||
| Completion Date: | 02/2022 | ||||||||||||||
| Planned End Date: | 03/2022 | ||||||||||||||
| Management Response: | Yes | ||||||||||||||
| Focus Area: |
|
||||||||||||||
| Corporate Outcome and Output (UNDP Strategic Plan 2018-2021) |
|
||||||||||||||
| SDG Goal |
|
||||||||||||||
| SDG Target |
|
||||||||||||||
| Evaluation Budget(US $): | 80,000 | ||||||||||||||
| Source of Funding: | |||||||||||||||
| Evaluation Expenditure(US $): | 37,746 | ||||||||||||||
| Joint Programme: | No | ||||||||||||||
| Joint Evaluation: | No | ||||||||||||||
| Evaluation Team members: |
|
||||||||||||||
| GEF Evaluation: | Yes
|
||||||||||||||
| Key Stakeholders: | Department for Forest Protection and Nature Conservation (PHKA), Sumatran Tiger Conservation Forum (HarimauKita), Wildlife Conservation Society (WCS), Fauna & Flora International (FFI) and Zoological Society of London (ZSL) | ||||||||||||||
| Countries: | INDONESIA |
| Lessons | |
|---|---|
| 1. |
Tag: Sustainability |
| Findings | |
| 1. | · The Project has been highly relevant for Indonesia, was aligned with national policies, and has supported the implementation of both the National Tiger Recovery Plan (2010-2022) and global broad-based efforts within the World Bank’s parallel Global Tiger Recovery Program. It is widely recognized to have been a model flagship project to address pressing concerns for endangered species in Indonesia and the region. · The Project has led to valuable advancements in the understanding of planning and implementation of conservation measures for the protection of Sumatran tigers (Panthera tigris sumatrae) and their landscapes in Indonesia, especially in varied production landscapes. For all intents and purposes, the UNDP-GEF Tiger project has been transformational with very high replication potential going forward. The Project is highly appreciated and recognized by beneficiaries at all levels as being a game-changer and has likely triggered an inflection-point effect that will benefit other landscapes due to clear institutionalization of methodologies and state-of-the-art decision-support tools, on-the-ground results and conservation change agents / champions that have moved to other NPs and key posts within the Indonesian Government. · Based on the collective body of information and evidence reviewed as part of the TE, the Project has achieved the majority of its expected outputs and target indicators with some varied progress between landscapes. In some cases, according to the information received, targets have exceeded the goal established for the end of the project. · At the Objective-level, the increase in tiger density as the umbrella indicator of the Project’s performance has been achieved in one landscape (Gunung Leuser National Park). While other landscapes have a lower than desired end-of-project target for tiger density figure, consensus has been that these are still within a healthy range, indicative of high population dynamics and an overall stable trend. · While the Project has fallen short against metrics at the Objective-level, it is imperative to recognize the UNDP-GEF Tiger project leaves some important legacies, particularly in aspects that are outside its results framework and that, consequently, have not been measured. There are countless testimonies however, that account for the impact of the Project, such as the contributions to and lessons on the need for monitoring tiger species using multivariable techniques going forward. For this reason and considering the difficult nature and attribution of species indicators, the evaluation team considered that admirable progress had been made towards the Objective and assigned a rating of Satisfactory overall. · Of the Project’s three Outcomes and illustrated by the results against indicators in Section IV C, two Outcomes were fully achieved and the remaining Outcome partially achieved in the view of the TE consultant team. The Project delivered some very important results, especially under Outcomes 1 and 2, which achieved change that was substantial and of real significance likely to deliver global environmental benefits. · Under Outcome 1, there was a fundamental shift and accelerated capacity and capabilities built to protect tigers and wildlife (including tiger prey), which would not have been possible without the GEF project funding within the same timeframe. This was achieved through a multi-pronged intervention of direct threat reduction, stepped-up patrolling and availability of value-added tools. It resulted in an across-the-board increase in both capacity scores and management effectiveness, and also manifested itself through an encouraging reduction of forest degradation rates. This was the perhaps the most important aspect of the Project from a sustainability perspective. · Under Outcome 2, the Project strengthened Indonesia’s capacity to address wildlife crime through intersectoral collaboration and enhanced forensic capability. The Project institutionalized and supported community and stakeholder participation in PA management and innovative forest/biodiversity projects. Perhaps the most important game changer is the integration of data-driven decision making and a common tiger, prey and forest habitat monitoring system at all 5 targeted PAs. More profoundly, the experience from implementing the Project changed the government’s approach to community engagement, consultation and collaborative management. · For Outcome 3, hard results and contribution to the Development Objective are slightly more varied. Without question, the Project has generated a tremendous amount of information, due diligence and has led to a deeper exploration - and in one case operationalization - of innovative financial mechanisms purpose-built for the Sumatran context. These include Government Islamic Bonds (SBSN), Public Private Partnership (PPP) Schemes, Carbon Credits and Trading and Nature-Based Tourism amongst others, as well as business plans which begin to integrate them. These financing mechanisms are still in their infancy however, and must be tested and operationalized further, as well as go through multiple cycles of business planning for refinement. · Because many of the sustainable financing mechanisms are not yet in place, true sustainable alternative funding streams and diversification have not taken root. At the time of the TE, consultations with NP authorities noted that funding mechanisms were not sufficiently in place to fill the resourcing. |
| Recommendations | |
|---|---|
| 1 | Terminal Evaluation recommendation 1.
Reconvene the Sumatran Tiger expert group prior to the Project’s operational closure to facilitate discussions on establishing and operationalizing a standard approach to monitoring tigers using multivariable methods and techniques discussed in mid-2021, going forward.
Responsible Units: PMU, MoEF, HarimauKita |
| 2 | Terminal Evaluation recommendation 3.
The Project’s exit strategy (Exit Strategy Report 2021) should be augmented to articulate a knowledge management transition plan - at minimum in an Annex - to be developed in consultation with the PMU’s Communications Officer.
Projects must explain their Knowledge Management tools and plans to learn, process and capture knowledge, as well as disseminate it in an insightful and useful way.
Responsible Units: PMU, UNDP Exit Strategy Consultant |
| 3 | Terminal Evaluation recommendation 6.
The experiences and results of the UNDP-GEF Tiger project have led to numerous best practices that are relevant to and need to be shared with the region through technical and scientific cooperation.
There are strong opportunities for cross pollination with other countries in the region on aspects of the Project that have been exemplary, and which can be a model, such as real time data-driven decision making and the institutionalization of the METT and capacities built for SMART patrols. Conversely, the UNDP-GEF Tiger project can learn from the results of other GEF projects in Malaysia on the implementation and piloting of sustainable financing plans and mechanisms such as National Conservation Trust Funds and performance-based ecological fiscal transfer schemes.
Responsible Units: UNDP Indonesia Country Office and UNDP RTA |
| 4 | Terminal Evaluation recommendation 8.
Project should develop business continuity and management plans as part of the inception phase to minimize disruption, ensuring that roles and responsibilities during times of “project crisis” are understood and internalized.
Responsible Units: UNDP Indonesia Country Office, UNDP RTA, GEF |
| 5 | Terminal Evaluation recommendation 9.
Projects should strategically plan throughout the project cycle for eventual transition and sustainability of the results (focusing on handover of products and services and the approaches used); this should occur throughout project development, inception, implementation and project close.
Responsible Units: UNDP Indonesia Country Office, PMUs |
| 6 | Terminal Evaluation recommendation 10.
Projects should work closely with Project Boards during implementation to value-add from Board members’ expertise and roles. Project Boards should not function as a formal reporting body but ought to be actively engaged in de-risking and overcoming obstacles through the championing of causes and providing subject-matter expertise.
Responsible Units: UNDP Indonesia Country Office, PMUs and PBs |
| 7 | Terminal Evaluation recommendation 11.
Future projects should put in place processes and control mechanisms to transparently track actual co-financing contributions during project implementation as part of Annual Work Planning. Post-facto calculations at MTR and TE result in errors (as noted by the TE) and omissions of the true value generated by projects.
Responsible Units: UNDP Indonesia Country Office and PMUs |
| 8 | Terminal Evaluation recommendation 12.
Prioritize and focus efforts of future initiatives on the piloting / ground truthing sustainable financing mechanisms before expanding them. The Project has generated a tremendous amount of due diligence on the potential of innovative financing mechanisms but has been short on piloting and integrating these into business planning.
Responsible Units: UNDP Indonesia Country Office, UNDP RTA, MoEF and GEF |
| 9 | Terminal Evaluation recommendation 13.
From a landscape perspective, future projects should invest in connectivity of landscapes and improving co-management arrangements in buffer areas surrounding the targeted National Parks where enforcement measures and opportunities for illegal activity is porous.
Responsible Units: UNDP Indonesia Country Office, MoEF and BAPPENAS |
Management Response Documents
- Final_Management Response_210222.pdf Agreed management responses
1. Recommendation:
Terminal Evaluation recommendation 1.
Reconvene the Sumatran Tiger expert group prior to the Project’s operational closure to facilitate discussions on establishing and operationalizing a standard approach to monitoring tigers using multivariable methods and techniques discussed in mid-2021, going forward.
Robust assessments of the spatial distribution and population dynamics of threatened species, including tigers, are crucial for designing effective conservation policies. This is often impeded by methodological differences employed by researchers to collect and analyze data. The continuing development of improved capture–recapture modeling techniques used to measure and monitor apex predators has also limited robust temporal and cross-site analyses due to different methods employed.
Responsible Units: PMU, MoEF, HarimauKita
Management Response: [Added: 2022/02/21]
Management response: Partially accept
Sumatran tiger standard monitoring protocol (tiger population monitoring guidebook) has been established by the project, however there are several different methods to measure the final tiger density data from project’s landscapes. The project has reconvened project’s partners’ technical staff to recalculate tiger density data using the same covariates in every landscape to make sure that the results of final tiger density analysis are comparable. After the event, project also asked tiger experts to reanalyze final tiger density numbers produced by the technical staff and came into conclusion of tiger density method reported by the project in the final Project Implementation Report 2021.
The project has reconvened project’s partners’ technical expert group to recalculate tiger density data using the same covariates in every landscape to make sure that the results of final tiger density analysis are comparable. The standardized approach for tiger monitoring has already established by the expert group in the form of the “Guidance Book on Sumatran Tiger Population Monitoring”
Key Actions:
| Key Action | Responsible | DueDate | Status | Comments | Documents |
|---|---|---|---|---|---|
| The project will reconvene project’s partners’ technical expert group to generate the lessons learned from the implementation of tiger population monitoring guidebook, and compile the recommendation for the future similar projects.
[Added: 2022/02/21] |
PMU, MoEF, Project’s partners expert group | 2022/02 | Completed | The technical expert group meeting was held of the 10th of February 2022 |
2. Recommendation:
Terminal Evaluation recommendation 3.
The Project’s exit strategy (Exit Strategy Report 2021) should be augmented to articulate a knowledge management transition plan - at minimum in an Annex - to be developed in consultation with the PMU’s Communications Officer.
Projects must explain their Knowledge Management tools and plans to learn, process and capture knowledge, as well as disseminate it in an insightful and useful way.
Responsible Units: PMU, UNDP Exit Strategy Consultant
Management Response: [Added: 2022/02/21]
Management response: Partially accept
Project’s exit strategy has been discussed between project’s communication officer and exit strategy consultant. Following the end of project’s communication officer’s working period, all project’s knowledge management information and communication accesses have been transferred to Directorate of Biodiversity Conservation and UNDP to be managed by Ministry of Environment and Forestry communication staff. One of key actions by the project is to integrate project’s website and social media channels into Situation Room that will be managed by the Directorate.
Key Actions:
| Key Action | Responsible | DueDate | Status | Comments | Documents |
|---|---|---|---|---|---|
| To integrate project’s website and social media channels into Situation Room (http://sitroomkkh.ksdae.menlhk.go.id/) that will be managed by the Directorate of Biodiversity Conservation on Species and Genetic.
[Added: 2022/02/21] [Last Updated: 2022/03/17] |
PMU, MoEF | 2022/03 | Completed | The project’s partners’ technical expert group have convened on the 10 February 2022, and generated the review and lessons learned document on the Sumatran Tiger Implementation Monitoring. History |
3. Recommendation:
Terminal Evaluation recommendation 6.
The experiences and results of the UNDP-GEF Tiger project have led to numerous best practices that are relevant to and need to be shared with the region through technical and scientific cooperation.
There are strong opportunities for cross pollination with other countries in the region on aspects of the Project that have been exemplary, and which can be a model, such as real time data-driven decision making and the institutionalization of the METT and capacities built for SMART patrols. Conversely, the UNDP-GEF Tiger project can learn from the results of other GEF projects in Malaysia on the implementation and piloting of sustainable financing plans and mechanisms such as National Conservation Trust Funds and performance-based ecological fiscal transfer schemes.
Responsible Units: UNDP Indonesia Country Office and UNDP RTA
Management Response: [Added: 2022/02/21]
Management response: Fully accept
Tiger project will open discussion on lessons learned with other projects in neighboring countries especially in Malaysia through Global Wildlife Program channel
Key Actions:
| Key Action | Responsible | DueDate | Status | Comments | Documents |
|---|---|---|---|---|---|
| To open discussion on lessons learned with other projects in neighboring countries especially in Malaysia through Global Wildlife Program channel
[Added: 2022/02/21] |
UNDP Indonesia Country Office, IP and UNDP RTA | 2022/12 | Not Initiated |
4. Recommendation:
Terminal Evaluation recommendation 8.
Project should develop business continuity and management plans as part of the inception phase to minimize disruption, ensuring that roles and responsibilities during times of “project crisis” are understood and internalized.
Responsible Units: UNDP Indonesia Country Office, UNDP RTA, GEF
Management Response: [Added: 2022/02/21]
Management response: Fully accept
UNDP and Ministry of Environment and Forestry will develop business continuity and management plans as part of project’s inception phase to minimize disruption during project crisis. The document contains the risk management strategy on managing the high level of staff’s turn-over and unpredicted changes on the structure of PMU, and IP.
Key Actions:
| Key Action | Responsible | DueDate | Status | Comments | Documents |
|---|---|---|---|---|---|
| Discussion on developing business continuity and management plans as part of project’s inception phase to minimize disruption during project crisis.
[Added: 2022/02/21] [Last Updated: 2022/10/02] |
UNDP Indonesia Country Office, UNDP RTA, GEF | 2022/09 | Completed | On the 20th of September 2022, KKHSG and UNDP discussed the management plans to avoid the high level of staff’s turn-over. One of the plans is to add points of “award and sanction” into the Recruitment’s ToR and also into the contract. The final Recruitmen’s ToR will be attached to the SoP of the Conserve Project Link: https://docs.google.com/document/d/1oSp9s-A7bLniDk2R0WZ2p4zratE69gm5/edit?usp=sharing&ouid=103736161726217898417&rtpof=true&sd=true History |
5. Recommendation:
Terminal Evaluation recommendation 9.
Projects should strategically plan throughout the project cycle for eventual transition and sustainability of the results (focusing on handover of products and services and the approaches used); this should occur throughout project development, inception, implementation and project close.
Responsible Units: UNDP Indonesia Country Office, PMUs
Management Response: [Added: 2022/02/21]
Management response: Full accept
The future project will create strategic plans to hand over products and services using UNDP and MoEF hand over mechanisms.
Key Actions:
| Key Action | Responsible | DueDate | Status | Comments | Documents |
|---|---|---|---|---|---|
| Preparing strategic plan to hand over products and services for project closing such as BAST (Hand-over Notes of products and services)
[Added: 2022/02/21] |
UNDP Indonesia Country Office, PMUs | 2022/12 | Initiated | Will be implemented for Conserve Project |
6. Recommendation:
Terminal Evaluation recommendation 10.
Projects should work closely with Project Boards during implementation to value-add from Board members’ expertise and roles. Project Boards should not function as a formal reporting body but ought to be actively engaged in de-risking and overcoming obstacles through the championing of causes and providing subject-matter expertise.
Responsible Units: UNDP Indonesia Country Office, PMUs and PBs
Management Response: [Added: 2022/02/21]
Management response: Fully accept
Project board members are actively involved in the Project Board Meeting, monitoring visits and also involved in the exit strategy technical meeting. Future projects will work closely with Project Boards during implementation to value-add from Board members’ expertise and roles
Key Actions:
| Key Action | Responsible | DueDate | Status | Comments | Documents |
|---|---|---|---|---|---|
| Future projects (Conserve and in-Flores) will work closely with Project Boards during implementation to value-add from Board members’ expertise and roles
[Added: 2022/02/21] |
UNDP Indonesia Country Office, PMUs and PBs | 2022/12 | Not Initiated | Will be well-articulated in the Project Board Meetings. |
7. Recommendation:
Terminal Evaluation recommendation 11.
Future projects should put in place processes and control mechanisms to transparently track actual co-financing contributions during project implementation as part of Annual Work Planning. Post-facto calculations at MTR and TE result in errors (as noted by the TE) and omissions of the true value generated by projects.
Responsible Units: UNDP Indonesia Country Office and PMUs
Management Response: [Added: 2022/02/21]
Management response: Fully accept
Future projects will put in place processes and control mechanisms to transparently track actual co-financing contributions during project implementation as part of Annual Work Planning.
Key Actions:
| Key Action | Responsible | DueDate | Status | Comments | Documents |
|---|---|---|---|---|---|
| Putting in place processes and control mechanisms to transparently track actual co-financing contributions during project implementation as part of Annual Work Planning.
[Added: 2022/02/21] [Last Updated: 2022/10/02] |
UNDP Indonesia Country Office and PMUs | 2022/09 | Completed | Conserve project has learned from other ongoing projects how to develop a co-financing tracking system. We have one good lesson from the National ATSEA2 project. The tracking system will be embedded in the Conserve Project’s SoP. Link: https://docs.google.com/spreadsheets/d/1XxtU5_VVD15Mz03zcxg6PwAuVDqXo08J/edit?usp=sharing&ouid=103736161726217898417&rtpof=true&sd=true History |
8. Recommendation:
Terminal Evaluation recommendation 12.
Prioritize and focus efforts of future initiatives on the piloting / ground truthing sustainable financing mechanisms before expanding them. The Project has generated a tremendous amount of due diligence on the potential of innovative financing mechanisms but has been short on piloting and integrating these into business planning.
Responsible Units: UNDP Indonesia Country Office, UNDP RTA, MoEF and GEF
Management Response: [Added: 2022/02/21]
Management response: Fully accept
During the latest PBM of the project, MoEF, and UNDP CO recommended piloting / ground truthing sustainable financing mechanisms before expanding them. The project recommendations have been discussed internally in the CO and might have been integrated into the strategies by the future project teams
Key Actions:
| Key Action | Responsible | DueDate | Status | Comments | Documents |
|---|---|---|---|---|---|
| Prioritizing and focusing efforts of future initiatives on the piloting / ground truthing sustainable financing mechanisms before expanding them.
[Added: 2022/02/21] |
UNDP Indonesia Country Office, UNDP RTA, MoEF and GEF | 2022/12 | Initiated |
9. Recommendation:
Terminal Evaluation recommendation 13.
From a landscape perspective, future projects should invest in connectivity of landscapes and improving co-management arrangements in buffer areas surrounding the targeted National Parks where enforcement measures and opportunities for illegal activity is porous.
Responsible Units: UNDP Indonesia Country Office, MoEF and BAPPENAS
Management Response: [Added: 2022/02/21]
Management response: Fully accept
UNDP Indonesia CO and MoEF have been discussed the connectivity of landscapes and improving co-management arrangements in buffer areas surrounding the targeted National Parks several times during the development of Conserve project. It might have been integrated into the strategies by future projects
Key Actions:
| Key Action | Responsible | DueDate | Status | Comments | Documents |
|---|---|---|---|---|---|
| UNDP Indonesia CO and MoEF will discuss more to implement the connectivity of landscapes and improving co-management arrangements in buffer areas / OECM /KEE (Essential Ecosystem Area)
[Added: 2022/02/21] [Last Updated: 2022/10/02] |
UNDP Indonesia Country Office, MoEF | 2022/11 | Completed | UNDP Indonesia CO and MoEF have discussed the connectivity of landscapes and improving co-management arrangements in buffer areas surrounding the targeted National Parks several times during the development of Conserve project. The lessons learned from the Sumatran Tiger Project as the reference to the developments of Conserve’s Project Document of the connectivity of landscapes and improving co-management arrangements in buffer areas / OECM /KEE (Essential Ecosystem Area) along with stakeholders around the National Park such as Conserve’s project site in Seblat Area that directly bordered with Sumatran Tiger’s Project Site of Kerinci Seblat National Seblat. Link: https://docs.google.com/document/d/1hzIol-Vv_n00xwW3Rh9nJaAofeV-sxQa/edit?usp=sharing&ouid=103736161726217898417&rtpof=true&sd=true https://docs.google.com/document/d/1L_HAnaEpzOrhPkzwt81tGCSJaHZjDwWl/edit?usp=sharing&ouid=103736161726217898417&rtpof=true&sd=true History |