Terminal Evaluation: Sustainable Forestry and land Management in the Dry Dipterocarp Forest Ecosystems of Southern Lao PDR

Report Cover Image
Evaluation Plan:
2017-2021, Lao
Evaluation Type:
Final Project
Planned End Date:
03/2022
Completion Date:
03/2022
Status:
Completed
Management Response:
Yes
Evaluation Budget(US $):
45,000

Share

Document Type Language Size Status Downloads
Download document UNDP-GEF-TE-TOR-SAFE Ecosystems-National-09082021 - final.docx tor English 92.78 KB Posted 63
Download document UNDP-GEF-TE-TOR-SAFE Ecosystems-International-29092021 - final.docx tor English 96.26 KB Posted 69
Download document TE SAFEECO Report.pdf report English 3951.91 KB Posted 123
Title Terminal Evaluation: Sustainable Forestry and land Management in the Dry Dipterocarp Forest Ecosystems of Southern Lao PDR
Atlas Project Number: 00094709
Evaluation Plan: 2017-2021, Lao
Evaluation Type: Final Project
Status: Completed
Completion Date: 03/2022
Planned End Date: 03/2022
Management Response: Yes
Focus Area:
  • 1. Sustainable
  • 2. Others
Corporate Outcome and Output (UNDP Strategic Plan 2018-2021)
  • 1. Output 1.4.1 Solutions scaled up for sustainable management of natural resources, including sustainable commodities and green and inclusive value chains
SDG Goal
  • Goal 15. Protect, restore and promote sustainable use of terrestrial ecosystems, sustainably manage forests, combat desertification, and halt and reverse land degradation and halt biodiversity loss
SDG Target
  • 15.2 By 2020, promote the implementation of sustainable management of all types of forests, halt deforestation, restore degraded forests and substantially increase afforestation and reforestation globally
  • 15.9 By 2020, integrate ecosystem and biodiversity values into national and local planning, development processes, poverty reduction strategies and accounts
Evaluation Budget(US $): 45,000
Source of Funding: GEF
Evaluation Expenditure(US $): 33,700
Joint Programme: No
Joint Evaluation: No
Evaluation Team members:
Name Title Nationality
Carsten Germer
Sengphachanh Sonethavixay LAO PEOPLE'S DEMOCRATIC REPUBLIC
GEF Evaluation: Yes
GEF Project Title: Sustainable Forestry and land Management in the Dry Dipterocarp Forest Ecosystems of Southern Lao PDR
Evaluation Type: Terminal Evaluation
Focal Area: Multifocal Areas
Project Type: FSP
GEF Phase: GEF-6
GEF Project ID: 6940
PIMS Number: 5448
Key Stakeholders: Ministry of Natural Resources and Environment
Countries: LAO PEOPLE'S DEMOCRATIC REPUBLIC
Comments:

The CO request to change the completion date from October 2021 to March 2022. The SAFE ECOSYSTEM project will end in May 2022. Changing the completion date will allow the CO to adhered to the guidelines for conducting TEs of UNDP-supported, GEF-financed projects, which suggests that the TE should complete a few months before the project ends.  

Lessons
Findings
Recommendations
1

The project should thoroughly review the project costs and underlying practices engaged in by the project to evaluate if similar approaches are to be pursued in future projects. While this recommendation spring from the large DSA expenditures it is equally relevant to look at project costs with regard to project benefits and evaluate if such expenses are fully justified. 

2

Regarding the project purchase equipment etc. the project (i.e. UNDP) ensure, to the extent possible, via its handover protocols, that the entities which has received and used equipment under the project are also the ones who maintain said equipment following the operational and financial closure of the project.

3

For future projects it is recommended that the project risks, including the Social and Environmental risks, its issue and lessons learned logs as well as the projects indicators (Results Framework and GEF Core indicators) are not treated as static documents but is used in project’s active and regularly monitoring. This is to ensure that the project is on track etc. and is meeting its obligations. Early identification and reactions can save the projects for complications later on in the implementation process.

4

The project should use the remaining time of the project to actively build knowledge hub where knowledge products from the project can be place for longevity. As part of this the project should ensure that its capacity building work is captured and made accessible online, for instance via recording project presentations and trainings etc. In this regard, the project could establish its own platform, use an existing national platform or rely on UNDP portals such as Exposure and Panorama. Along the same lines knowledge management should become a central part of a future projects and as part of any contract or sub-contract related to capacity building that training videos, on-line courses and accessible training materials should be prepared and made available on a web-based media.

5

The project should prior to project operational closure develop its exit strategy outlining not only how the bearing elements of the project can be brought forward through new project interventions, but also how the set-ups and benefits within the project area can be maintained through ongoing and future provincial and district interventions.

6

For future projects data collection exercises such as Capacity Development Scorecard, and the METT score analysis should not done through desk surveys but via stakeholder workshops, as this would provide added benefits in terms of education and knowledge sharing, as well as create broader local ownership of the exercise.

7

Future project should undertake Knowledge, Attitude and Practices surveys to enable the project to capture behavioral change among the project beneficiaries.

8

The project should prior to project operational closure undertake and end-of-project Biodiversity Intactness Index survey. While such survey might not provide for any new information per se, due to the limited time between this and the previous survey, running the survey again would verify whether the index (and the exercise) can be seen as an efficient, valuable and practical tool for biodiversity monitoring and evaluation.

9

The project should take a close look at its combined village engagement “packaged” to not only look at what worked well and what did not. The project should look at which components in an active village engagement is needed to make it effective and then identify how such interventions could be anchored within different government entities (and their respective programs). This analytical work could provide a central guide for cross-sectoral cooperation at provincial and district

10

As a central component of the project’s community work are the community agreements it is recommended that the project review of its approach for community involvement and identify lessons learned and best practices. This review could benefit from not only looking at the SAFE Ecosystems project’s work but also from similar engagements undertaken by other development partners, as well as the Lao PDR Government. The review could result in common agreed approaches for simple and cost-effective engagements.

11

Future projects should keep track of co-financing commitments as well as new investments during the project period regularly. It would have been prudent if the project had collected this information on a regular basis for instance in connection with the annual PIR and as a minimum prior to the Mid-term Review and the Terminal Evaluation.

12

For future projects project safeguard and monitoring tools should be well drafted and based on local ownership and input. Project results and indicator targets should be agreed to with the stakeholders who ultimately will be responsible for implementing the projects and ensuring that these targets are reached. In other words, project targets should not only be targets of the project but targets of the individual involved entities engaged in the project. As part of this local formulation workshops, engaging key champion stakeholders could be used in the drafting process.

13

For future projects the project should immediately after the onboarding of the central project staff key technical personnel undertake a thorough review of the project documentation to identify potential inconsistencies in said documentation, which might impact the project implementation or the achievement of the project goals and targets, including GEF Core Indicator and Results Framework targets. Identified concerns should be raised through available UNDP channels in order to bring about acceptable changes addressing the identified concerns where possible.

1. Recommendation:

The project should thoroughly review the project costs and underlying practices engaged in by the project to evaluate if similar approaches are to be pursued in future projects. While this recommendation spring from the large DSA expenditures it is equally relevant to look at project costs with regard to project benefits and evaluate if such expenses are fully justified. 

Management Response: [Added: 2022/04/04]

Fully Accept

The project has already reviewed the cost of the various project intervention activities with the aim of ensuring efficiency and minimizing unnecessary costs; this has been particularly noted during project implementation period between the Mid-Term Review and the Terminal Evaluation. The same approach will be used in the remaining months of the project as will apply the same criteria in future projects.

Key Actions:

Key Action Responsible DueDate Status Comments Documents
1.1 Monitor costs of all project-related activities to ensure efficiency and justification has been undertaken for the remainder of the project resulting in less than 30 percent in reduction of costs.
[Added: 2022/04/04] [Last Updated: 2022/06/05]
UNDP & PMU 2022/06 Completed The project reviewed the implementation costs in all project activities aimed at minimizing unnecessary expenditure. History
2. Recommendation:

Regarding the project purchase equipment etc. the project (i.e. UNDP) ensure, to the extent possible, via its handover protocols, that the entities which has received and used equipment under the project are also the ones who maintain said equipment following the operational and financial closure of the project.

Management Response: [Added: 2022/04/04]

Fully Accept

UNDP and Project Management Unit (PMU) will conduct hand-over of all the project assets in accordance with the Assets Management Section of the NIM SOPs prior to project closure with a focus upon ensuring proper maintenance of the assets and/or equipment after project closure.

Key Actions:

Key Action Responsible DueDate Status Comments Documents
2.1 Consultation with Implementation Partner on handing over timelines of all assets under the project in accordance to Assets Management Section of the NIM SOPs. Handing over all project assets and/or equipment.
[Added: 2022/04/04] [Last Updated: 2022/06/05]
UNDP & PMU 2022/06 Completed Engagements were undertaken with relevant stakeholders and handing over completed as per NIM standard operating procedures History
3. Recommendation:

For future projects it is recommended that the project risks, including the Social and Environmental risks, its issue and lessons learned logs as well as the projects indicators (Results Framework and GEF Core indicators) are not treated as static documents but is used in project’s active and regularly monitoring. This is to ensure that the project is on track etc. and is meeting its obligations. Early identification and reactions can save the projects for complications later on in the implementation process.

Management Response: [Added: 2022/04/04]

Fully Accept

All stakeholders acknowledge the importance of undertaking detailed project risk analysis with regards to social and environmental safeguards during the project development stage. Furthermore, it is recognized that the project results framework is not static and should be updated regularly during as part of stakeholder engagement to ensure the targets set are achievable during project implementation.

Key Actions:

Key Action Responsible DueDate Status Comments Documents
3.1 Steps have already been established to ensure that risk assessment is undertaken at project development and quarterly review of actions to mitigate the risks in future projects.
[Added: 2022/04/04]
UNDP 2022/03 Completed The newly revised UNDP 2021 Social and Environmental Standards (SES) Policy has necessitated that risks are to be assessed during the project development phase and regularly monitored throughout project implementation. Additionally, UNDP regularly monitors all risks as per the set risk management procedures.
4. Recommendation:

The project should use the remaining time of the project to actively build knowledge hub where knowledge products from the project can be place for longevity. As part of this the project should ensure that its capacity building work is captured and made accessible online, for instance via recording project presentations and trainings etc. In this regard, the project could establish its own platform, use an existing national platform or rely on UNDP portals such as Exposure and Panorama. Along the same lines knowledge management should become a central part of a future projects and as part of any contract or sub-contract related to capacity building that training videos, on-line courses and accessible training materials should be prepared and made available on a web-based media.

Management Response: [Added: 2022/04/04]

Fully Accept

There are many successes and lessons learned from the project implementation that can be utilized in future programming in Lao PDR. The project has already developed knowledge materials to showcase approaches which worked well for potential upscaling and replication in similar context including video documentation from the beneficiaries’ experiences. As such, the project will explore suitable platforms such as Exposure and Panorama and share relevant knowledge products to a wide audience before the project concludes.

Key Actions:

Key Action Responsible DueDate Status Comments Documents
4.1 Explore and set up at least two suitable platforms to enable increased sharing of knowledge products.
[Added: 2022/04/04] [Last Updated: 2022/06/05]
PMU 2022/06 Completed The project developed knowledge products of various forms such as photo albums, good practice booklets, case-studies, and video documentary, lessons learned and success stories shared through project social media, UNDP-panorama and exposure platforms. History
5. Recommendation:

The project should prior to project operational closure develop its exit strategy outlining not only how the bearing elements of the project can be brought forward through new project interventions, but also how the set-ups and benefits within the project area can be maintained through ongoing and future provincial and district interventions.

Management Response: [Added: 2022/04/04]

Fully Accept

The project initiated an exit strategy engagement with relevant stakeholders prior to Terminal Evaluation with the focus on the sustainability of key project intervention areas. Further engagements will be undertaken to ensure provision of continuous support by the relevant stakeholders on the key successful project interventions areas and continued benefits to the communities.

Key Actions:

Key Action Responsible DueDate Status Comments Documents
5.1 Continue to conduct stakeholder consultations on the potential takeover/hand-over of various project-initiated interventions.
[Added: 2022/04/04] [Last Updated: 2022/06/05]
PMU 2022/06 Completed The exit strategy was discussed with relevant stakeholders prior to terminal evaluation and finalized in Quarter 1, 2022. History
5.2 Final project exit strategy to be completed and responsibilities of various actors clarified.
[Added: 2022/04/04] [Last Updated: 2022/06/05]
PMU 2022/06 Completed Exit strategy was completed with key actions for various stakeholders beyond project cycle highlighted. History
6. Recommendation:

For future projects data collection exercises such as Capacity Development Scorecard, and the METT score analysis should not done through desk surveys but via stakeholder workshops, as this would provide added benefits in terms of education and knowledge sharing, as well as create broader local ownership of the exercise.

Management Response: [Added: 2022/04/04]

Fully Accept

Participatory stakeholder engagement in data collection exercises plays a significant role in strengthening ownership and capacity of the involved actors. It also strengthens the sustainability of the interventions and therefore the recommendation will be implemented in future GEF projects.

Key Actions:

Key Action Responsible DueDate Status Comments Documents
6.1 Mechanisms have already been put in place to ensure participatory engagement in data collection will be implemented in all future GEF projects.
[Added: 2022/04/04]
UNDP 2022/03 Completed N/A
7. Recommendation:

Future project should undertake Knowledge, Attitude and Practices surveys to enable the project to capture behavioral change among the project beneficiaries.

Management Response: [Added: 2022/04/04]

Fully Accept

Follow-up studies and surveys play an important role determining the progress made in the various project interventions by measuring changes in behavior, as well as determine the satisfaction of the beneficiaries. For the purpose of improved learning and informed decision making, this recommendation will be factored into future projects.

Key Actions:

Key Action Responsible DueDate Status Comments Documents
7.1 Incorporation of KAP surveys and follow-up studies where necessary in project development in accordance with results-based management approach.
[Added: 2022/04/04]
UNDP 2022/03 Completed N/A
8. Recommendation:

The project should prior to project operational closure undertake and end-of-project Biodiversity Intactness Index survey. While such survey might not provide for any new information per se, due to the limited time between this and the previous survey, running the survey again would verify whether the index (and the exercise) can be seen as an efficient, valuable and practical tool for biodiversity monitoring and evaluation.

Management Response: [Added: 2022/04/04]

Partially Accept

In principle, the Project agrees that undertaking a second Biodiversity Intactness Index survey would verify if this index is in fact an efficient and practical tool to determine the change in the status of biodiversity intactness as compared to the period before the project intervention. However, the cost of the study, available expertise on the subject in Lao PDR and the time required to complete the study limits the ability to implement this recommendation. However, the project will explore other potential simplified biodiversity data collection methodologies which could be implemented sustainably.

Key Actions:

Key Action Responsible DueDate Status Comments Documents
8.1 Explore simplified biodiversity data collection method to be adopted for biodiversity monitoring.
[Added: 2022/04/04] [Last Updated: 2022/06/05]
PMU 2022/06 Completed Review of biodiversity data collection and potential biodiversity intactness analysis explored. Survey protocols for key species had been developed and review of the existing data drew conclusion that further review required expertise which was not available in country. History
9. Recommendation:

The project should take a close look at its combined village engagement “packaged” to not only look at what worked well and what did not. The project should look at which components in an active village engagement is needed to make it effective and then identify how such interventions could be anchored within different government entities (and their respective programs). This analytical work could provide a central guide for cross-sectoral cooperation at provincial and district

Management Response: [Added: 2022/04/04]

Fully Accept

The sustainability of initiated activities is one of project priorities in ensuring continuation of successful interventions. The project will review the various project activities and the potential continuation by respective Government departments in the area working on the same interventions. Some district authorities have shared their commitment to continue with some of the impactful project activities.

Key Actions:

Key Action Responsible DueDate Status Comments Documents
9.1 Consultations with the relevant Government departments and development partners on takeover of initiated activities.
[Added: 2022/04/04] [Last Updated: 2022/06/05]
PMU 2022/06 Completed Consultations with a relevant government department held and key action for follow up identified as part of project exit strategy. History
10. Recommendation:

As a central component of the project’s community work are the community agreements it is recommended that the project review of its approach for community involvement and identify lessons learned and best practices. This review could benefit from not only looking at the SAFE Ecosystems project’s work but also from similar engagements undertaken by other development partners, as well as the Lao PDR Government. The review could result in common agreed approaches for simple and cost-effective engagements.

Management Response: [Added: 2022/04/04] [Last Updated: 2022/04/04]

Fully Accept

Community conservation agreements were a new concept in the areas targeted by the project. A number of lessons were learned, and practices identified during the project implementation. The project will review alternative community agreement approaches to enable identification of cost-effective approaches in interventions undertaken during project implementation in comparison with approaches undertaken by other development partners. The project will document conservation agreement experiences and implementation practices for sharing with the relevant stakeholders.

Key Actions:

Key Action Responsible DueDate Status Comments Documents
10.1 Review of conservation agreements approaches.
[Added: 2022/04/04] [Last Updated: 2022/06/05]
PMU 2022/06 Completed Review of conservation agreement approaches in the region undertaken. History
10.2 Prepare a knowledge management document on conservation agreement approaches completed.
[Added: 2022/04/04] [Last Updated: 2022/06/05]
PMU 2022/06 Completed Knowledge products shared on UNDP-panorama platform History
11. Recommendation:

Future projects should keep track of co-financing commitments as well as new investments during the project period regularly. It would have been prudent if the project had collected this information on a regular basis for instance in connection with the annual PIR and as a minimum prior to the Mid-term Review and the Terminal Evaluation.

Management Response: [Added: 2022/04/04]

Fully Accept                         

In line with updated GEF rules on co-financing, UNDP recognizes that regular and consistent tracking of co-financing resources will need to be done annually. It will also be important to examine whether or not parallel co-financing projects identified during the ppg phase remain relevant to the goals and objectives of GEF project throughout the implementation period.

Key Actions:

Key Action Responsible DueDate Status Comments Documents
11.1 In line with GEF rules on implementation, establish an annual tracking system on project parallel co-financing as part of the annual project implementation report.
[Added: 2022/04/04]
UNDP 2022/03 Completed Steps taken to ensure that co-funding resources are annually tracked.
12. Recommendation:

For future projects project safeguard and monitoring tools should be well drafted and based on local ownership and input. Project results and indicator targets should be agreed to with the stakeholders who ultimately will be responsible for implementing the projects and ensuring that these targets are reached. In other words, project targets should not only be targets of the project but targets of the individual involved entities engaged in the project. As part of this local formulation workshops, engaging key champion stakeholders could be used in the drafting process.

Management Response: [Added: 2022/04/04]

Fully Accept

With the launch of social and environmental standard (SES) policy in early 2021, UNDP has made it mandatory that all projects incorporate identified environmental and social risks within project activities, as well as participation of   key stakeholders and beneficiaries during the project development stage. Furthermore, in future project development, detailed engagements with relevant stakeholders are to be undertaken to ensure that targets set are achievable and ownership project interventions.

Key Actions:

Key Action Responsible DueDate Status Comments Documents
12.1 Trainings have already been conducted on social and environmental standards and awareness enhanced on participatory stakeholders' engagement at the project development.
[Added: 2022/04/04]
UNDP 2022/03 Completed Training has been conducted and SES Support team established.
13. Recommendation:

For future projects the project should immediately after the onboarding of the central project staff key technical personnel undertake a thorough review of the project documentation to identify potential inconsistencies in said documentation, which might impact the project implementation or the achievement of the project goals and targets, including GEF Core Indicator and Results Framework targets. Identified concerns should be raised through available UNDP channels in order to bring about acceptable changes addressing the identified concerns where possible.

Management Response: [Added: 2022/04/04]

Fully Accept

The importance of key staff from project inception is widely acknowledged to ensure that documentation, technical support and engagement with the relevant stakeholders are undertaken appropriately. Furthermore, facilitate identification of areas of concern within the project document which needs to be timely addressed for effective project implementation. 

Key Actions:

Key Action Responsible DueDate Status Comments Documents
13.1 In future projects, ensure timely recruitment of key project staffs as well as develop more robust inception period to facilitate adequate review of the project results-based management framework.
[Added: 2022/04/04]
UNDP 2022/03 Completed Plans have already been put in place to ensure timely recruitment of staff

Latest Evaluations

Contact us

1 UN Plaza
DC1-20th Floor
New York, NY 10017
Tel. +1 646 781 4200
Fax. +1 646 781 4213
erc.support@undp.org