Terminal Evaluation of Sustainable Forest Management to Secure Multiple Benefits in High Conservation Value Forests

Report Cover Image
Evaluation Plan:
2018-2022, Pakistan
Evaluation Type:
Final Project
Planned End Date:
01/2022
Completion Date:
05/2022
Status:
Completed
Management Response:
Yes
Evaluation Budget(US $):
20,000

Share

Title Terminal Evaluation of Sustainable Forest Management to Secure Multiple Benefits in High Conservation Value Forests
Atlas Project Number: 00086910
Evaluation Plan: 2018-2022, Pakistan
Evaluation Type: Final Project
Status: Completed
Completion Date: 05/2022
Planned End Date: 01/2022
Management Response: Yes
Focus Area:
  • 1. Sustainable
  • 2. Others
Corporate Outcome and Output (UNDP Strategic Plan 2018-2021)
  • 1. Output 1.4.1 Solutions scaled up for sustainable management of natural resources, including sustainable commodities and green and inclusive value chains
  • 2. Output 2.4.1 Gender-responsive legal and regulatory frameworks, policies and institutions strengthened, and solutions adopted, to address conservation, sustainable use and equitable benefit sharing of natural resources, in line with international conventions and national legislation
  • 3. Output 3.3.1 Evidence-based assessment and planning tools and mechanisms applied to enable implementation of gender-sensitive and risk-informed prevention and preparedness to limit the impact of natural hazards and pandemics and promote peaceful, just and inclusive societies
SDG Goal
  • Goal 12. Ensure sustainable consumption and production patterns
  • Goal 13. Take urgent action to combat climate change and its impacts
  • Goal 15. Protect, restore and promote sustainable use of terrestrial ecosystems, sustainably manage forests, combat desertification, and halt and reverse land degradation and halt biodiversity loss
SDG Target
  • 12.4 By 2020, achieve the environmentally sound management of chemicals and all wastes throughout their life cycle, in accordance with agreed international frameworks, and significantly reduce their release to air, water and soil in order to minimize their adverse impacts on human health and the environment
  • 13.2 Integrate climate change measures into national policies, strategies and planning
  • 15.2 By 2020, promote the implementation of sustainable management of all types of forests, halt deforestation, restore degraded forests and substantially increase afforestation and reforestation globally
Evaluation Budget(US $): 20,000
Source of Funding: GEF
Evaluation Expenditure(US $): 42,142
Joint Programme: No
Joint Evaluation: No
Evaluation Team members:
Name Title Nationality
Camillo Ponziani
Amjad Virk PAKISTAN
GEF Evaluation: Yes
GEF Project Title: Sustainable Forest Management to Secure Multiple Benefits in High Conservation Value Forests
Evaluation Type: Terminal Evaluation
Focal Area: Multifocal Areas
Project Type: FSP
GEF Phase: GEF-5
GEF Project ID: 5660
PIMS Number: 4674
Key Stakeholders: GEF, Ministry of Climate Change, Provincial Forest Departments
Countries: PAKISTAN
Lessons
Findings
Recommendations
1

It is recommended to ensure that all technical reports produced to date be made available to their intended audience and “consumers” following the Project’s operational closure. Additionally, it would be more advisable to put the key project reports and plans on the websites of the Ministry of Climate Change and respective provincial Forest Departments

2

Without delay, any surplus or unspent GEF funds should be transferred back to the UNDP Pakistan Country Office to be returned to the donor. Based on a comprehensive review of the cumulative Combined Delivery Reports, the Project has only managed to spend 95% of the total GEF budget at operational closure. Therefore, there are approximately 5% of unspent financial resources at operational closure (US$ 402,941.90) need to be reclaimed by UNDP’s Bureau for Policy and Programme Support of the Global Policy Network - Nature, Climate and Energy - Vertical Fund (BPPS NCE-VF) Directorate. Per financial guidelines, a refund to the GEF shall be done via reporting on financially closed projects after the

3

In the absence of a formal exit strategy and final project terminal workshop, the IP and the provincial Forest Departments should develop a succinct roadmap on the way forward and convene a meeting with the Implementing Agency, Ministry of Climate Change and representatives from each Provincial Forest and Wildlife Departments to transition unfinished activities, unapproved deliverables and settle any liabilities for partially completed work.

The Project contributed to improving the enabling environment for SFM and invested in testing and demonstrating innovative approaches. As per the GEF definition of the catalytic role, most results of this project are now ready for replication and scaling-up. In order to facilitate and ensure the sustainability of these results, it is recommended to collate together a summary of all the products and services developed by the Project, and identify an owner and transition strategy (phasing down, phasing out and phasing over). The roadmap document should detail what remaining work needs to be done to complete unfinished deliverables, when, how and who, to facilitate the transfer of project achievements to relevant partners and stakeholders. It would also contribute to ensuring the long-term sustainability of the Project’s achievements

4

Develop operational plans for ecotourism activities, captive-breeding/rescue centers, wireless-based communication system, and physical works which require ongoing operational support, as well as facilitate the development of business plans for local livelihood activities, i.e., sustainable use of Non-Timber Forest Products (NTFPs).

As an input into the roadmap noted above, a series of operational / maintenance plans should be established for key ecotourism ventures (i.e., Moto Tunnel, Mlakandi, Munro Track, and Tourist Village/Visitor Centre at Kund Siran Forest Division); forest squads and equipment; road maintenance in Chakwal and Kallar Syedan; and, wireless-GPS communication systems, to name just a few. Investment business plans should be developed to support continuation and self-sufficiency of livelihood.

5

The horse stable established in Siren Landscape for the revival of century-old practice for patrolling high-altitude reserve forests will not be viable in the long run, given the lack of proper facilities, professional handlers, and veterinary services within the KP forest department. This facility should either be desisted or managed with proper resources and expertise. The horse stable was established without a feasibility study and operational plan by bringing horses from the down country, which were not climatized to the cold environment, resulting in the loss of two horses. Given the availability of alternate options for surveillance of mountain forest tracts (reserve forests), this activity should be discontinued and the remaining horses are transferred to the Forest School in Abbottabad (as proposed by the school already) or another suitable facility in KP

6

For Future GEF programming:- Consider the seasonality of activities in project design and account for the die-off of seedlings/saplings in budgets for restoration works.

7

The planning of and management of co-financing resources must undergo a fundamental rethink. The ongoing management of co-financing contributions as inputs to AWP was missing and the impact of the Project was not aligned to the total funding envelope as envisaged at the time of CEO endorsement

8

At inception, ensure a clear selection/nomination process and supporting criteria for participants to attend training sessions (domestic and international) are developed from the outset. These should be developed with key beneficiaries in mind, especially at the lower tier of the provincial line agencies. The training in the UNDP-GEF SFM project, especially international workshops and exposure visits, disproportionately benefited senior management, the project team, and those that might have already had a relatively high level of capacity. 

9

Establish partnerships with local and provincial academic institutions and grassroots NGOs towards the contribution to SFM and biodiversity conservation.

Involvement of community-level governance structures, research institutions, and academia in activities targeting participatory resource assessment and biodiversity conservation, especially at the operational level, is crucial. Therefore, it is important to establish some operational-level partnerships with the local and provincial research institutions and academic sector, and with CBO-Support Organizations so that the participatory biodiversity conservation and monitoring efforts can continue after the operational closure of the Project

10

UNDP Pakistan Country Office to offer initial and refresher Project Management and gender training for projects within its portfolio. Training has the potential to act as a bridge between the IA and Implementing Partners and provides PMUs with the key underpinnings and playbooks to manage projects well and according to best practice, especially to government personnel with limited project exposure. The UNDP Pakistan Country Offices should also offer ongoing guidance on how to apply a gender lens to GEN2 projects. This should be the norm as is the case with other UNDP Country Offices.

11

The PIR is a core reference document and should be the source of truth for any project's logical framework. Projects should internalize and start monitoring cumulative progress against modified indicators following MTRs. PIRs should explicitly delineate changes to the results hierarchy and targets once adopted by the Project Board

12

For multi-focal area projects like SFM requiring the involvement of local communities in the on-the-ground implementation of project activities targeting livelihood improvement, there should have been a full-fledged community mobilization component at least at the outcome level. Such a provision in the project design could help in organizing local communities, strengthening traditional governance structures, and creating financial mechanisms for sustaining these structures and activities undertaken with their involvement. Therefore, it would be advisable for such type of future projects should have a desirable level of community mobilization/organization component with a clear outcome and outputs, and corresponding indicators

13

Future projects should ensure appropriate mechanisms are in place to ensure ownership is transferred and concentrated at the provincial level and that benefits accrue at the local level, especially local communities within the landscapes per a project’s design.

14

Project of technical nature like SFM should be managed at the technical level by the personnel having expertise and technical knowledge of the subject both within the project team and among implementing/executing agencies. This would help in proper designing, implementing, overseeing, and producing desired results envisaged under the project design.

15

UNDP should ensure that the Terms of Reference for project audits are expanded to include an assessment of administrative and financial management practices by a project team and implementing/executing agencies. This would be closer to a performance audit and could help GEF agencies target problems at an early stage and apply corrective measures. Furthermore, and as an extra measure of due diligence, a new financial audit should be undertaken, or added to the scope of an existing one. The scope of work should include:

A determination of whether GEF resources were used to extend the operational closure date of the Project and how this is reflected in financial reporting, as it is not clear from the latest Combined Delivery Reports shared with the TE consultant team how this has been reported

16

In spite of not being mentioned in the Project Document as a Responsible Party, IUCN was allocated a budget of US$ 1,516,900.00 via a Letter of Agreement (LoA), corresponding to 18.2% of the total GEF grant. Ensure that any partner not mentioned explicitly in the Project Document is selected through competitive tendering and RFP process.

1. Recommendation:

It is recommended to ensure that all technical reports produced to date be made available to their intended audience and “consumers” following the Project’s operational closure. Additionally, it would be more advisable to put the key project reports and plans on the websites of the Ministry of Climate Change and respective provincial Forest Departments

Management Response: [Added: 2022/07/06]

Agreed with the recommendation. The Project has been a prolific report generator and has produced an impressive body of knowledge including numerous baseline studies, assessments, technical manuals, training reports, guidelines, management plans, etc. As the Project closed by mid-March 2022, all these products will be collated and made available to their intended target audience and made available at the websites of the concerned agencies and departments, including a full listing in the final project report. This recommendation coincides with the need for a knowledge management strategy going forward to identify the people, processes, and technology to keep these reports up-to-date if subsequent revisions are warranted.

Key Actions:

Key Action Responsible DueDate Status Comments Documents
1.1- UNDP will make sure that following the operational closure, MOCC will be directed to put the key project reports and plans on their websites and respective provincial Forest Departments. Also, the PMU will make sure that the knowledge management strategy is there for future projects
[Added: 2022/07/06] [Last Updated: 2022/10/31]
PMU, PMIU, Implementing Partner, Provincial Forestry Departments and UNDP Country Office 2022/12 Initiated History
2. Recommendation:

Without delay, any surplus or unspent GEF funds should be transferred back to the UNDP Pakistan Country Office to be returned to the donor. Based on a comprehensive review of the cumulative Combined Delivery Reports, the Project has only managed to spend 95% of the total GEF budget at operational closure. Therefore, there are approximately 5% of unspent financial resources at operational closure (US$ 402,941.90) need to be reclaimed by UNDP’s Bureau for Policy and Programme Support of the Global Policy Network - Nature, Climate and Energy - Vertical Fund (BPPS NCE-VF) Directorate. Per financial guidelines, a refund to the GEF shall be done via reporting on financially closed projects after the

Management Response: [Added: 2022/07/06]

Agreed with the recomendation. 

Key Actions:

Key Action Responsible DueDate Status Comments Documents
2.1- Per financial guidelines, a refund to the GEF will be done via reporting on financially closed projects after the Country Office financially closes the project in Atlas (till the end of Dec 2022- 12 months following operational closure).
[Added: 2022/07/06]
Ministry of Climate Change and UNDP Country Office 2022/11 Initiated
3. Recommendation:

In the absence of a formal exit strategy and final project terminal workshop, the IP and the provincial Forest Departments should develop a succinct roadmap on the way forward and convene a meeting with the Implementing Agency, Ministry of Climate Change and representatives from each Provincial Forest and Wildlife Departments to transition unfinished activities, unapproved deliverables and settle any liabilities for partially completed work.

The Project contributed to improving the enabling environment for SFM and invested in testing and demonstrating innovative approaches. As per the GEF definition of the catalytic role, most results of this project are now ready for replication and scaling-up. In order to facilitate and ensure the sustainability of these results, it is recommended to collate together a summary of all the products and services developed by the Project, and identify an owner and transition strategy (phasing down, phasing out and phasing over). The roadmap document should detail what remaining work needs to be done to complete unfinished deliverables, when, how and who, to facilitate the transfer of project achievements to relevant partners and stakeholders. It would also contribute to ensuring the long-term sustainability of the Project’s achievements

Management Response: [Added: 2022/07/06]

.Agreed with the recomendations

Key Actions:

Key Action Responsible DueDate Status Comments Documents
A formal meeting was held between the MOCC, forest department, and UNDP PMU staff, and also project board meeting was held to look after all the matters
[Added: 2022/07/06]
Implementing Partner (MoCC) and the respective Provincial Forestry Departments 2022/06 Completed attached are the meeting minutes
4. Recommendation:

Develop operational plans for ecotourism activities, captive-breeding/rescue centers, wireless-based communication system, and physical works which require ongoing operational support, as well as facilitate the development of business plans for local livelihood activities, i.e., sustainable use of Non-Timber Forest Products (NTFPs).

As an input into the roadmap noted above, a series of operational / maintenance plans should be established for key ecotourism ventures (i.e., Moto Tunnel, Mlakandi, Munro Track, and Tourist Village/Visitor Centre at Kund Siran Forest Division); forest squads and equipment; road maintenance in Chakwal and Kallar Syedan; and, wireless-GPS communication systems, to name just a few. Investment business plans should be developed to support continuation and self-sufficiency of livelihood.

Management Response: [Added: 2022/07/06]

AS the project is operationally closed so this action is not applicable. UNDP will make sure that a letter will be send to MOCC with all these recommendations enclosed,

Key Actions:

5. Recommendation:

The horse stable established in Siren Landscape for the revival of century-old practice for patrolling high-altitude reserve forests will not be viable in the long run, given the lack of proper facilities, professional handlers, and veterinary services within the KP forest department. This facility should either be desisted or managed with proper resources and expertise. The horse stable was established without a feasibility study and operational plan by bringing horses from the down country, which were not climatized to the cold environment, resulting in the loss of two horses. Given the availability of alternate options for surveillance of mountain forest tracts (reserve forests), this activity should be discontinued and the remaining horses are transferred to the Forest School in Abbottabad (as proposed by the school already) or another suitable facility in KP

Management Response: [Added: 2022/07/06]

agreed with the recommendation. As the project is operationally closed now so UNDP will take care of this aspect in future projects.  

Key Actions:

Key Action Responsible DueDate Status Comments Documents
6.1-The MOCC will be intimated through a formal letter to discontinue this activity and to make sure that the remaining horses are transferred to the Forest School in Abbottabad (as proposed by the school already) or another suitable facility in KP
[Added: 2022/07/06] [Last Updated: 2022/10/31]
IP and respective provincial Wildlife Departments 2022/12 Initiated UNDP will send letter to MOCC to discontinue this activity and to make sure that the remaining horses are transferred to the Forest School in Abbottabad (as proposed by the school already) History
6. Recommendation:

For Future GEF programming:- Consider the seasonality of activities in project design and account for the die-off of seedlings/saplings in budgets for restoration works.

Management Response: [Added: 2022/07/06]

Agreed and the recommendation will be used in design of future programming.

Key Actions:

7. Recommendation:

The planning of and management of co-financing resources must undergo a fundamental rethink. The ongoing management of co-financing contributions as inputs to AWP was missing and the impact of the Project was not aligned to the total funding envelope as envisaged at the time of CEO endorsement

Management Response: [Added: 2022/07/06]

Agreed with the recomendation. As the project is financially closed so the recommendation will be taken care of in any future GEF related projects.

Key Actions:

8. Recommendation:

At inception, ensure a clear selection/nomination process and supporting criteria for participants to attend training sessions (domestic and international) are developed from the outset. These should be developed with key beneficiaries in mind, especially at the lower tier of the provincial line agencies. The training in the UNDP-GEF SFM project, especially international workshops and exposure visits, disproportionately benefited senior management, the project team, and those that might have already had a relatively high level of capacity. 

Management Response: [Added: 2022/07/06]

agreed. future programming will make sure that the lower tier is also included in such trainings.

Key Actions:

9. Recommendation:

Establish partnerships with local and provincial academic institutions and grassroots NGOs towards the contribution to SFM and biodiversity conservation.

Involvement of community-level governance structures, research institutions, and academia in activities targeting participatory resource assessment and biodiversity conservation, especially at the operational level, is crucial. Therefore, it is important to establish some operational-level partnerships with the local and provincial research institutions and academic sector, and with CBO-Support Organizations so that the participatory biodiversity conservation and monitoring efforts can continue after the operational closure of the Project

Management Response: [Added: 2022/07/06]

agreed 

Key Actions:

Key Action Responsible DueDate Status Comments Documents
10.1- UNDP through MOCC will try to establish some operational-level partnerships with the local and provincial research institutions and academic sector, and with CBO-Support Organizations so that the participatory biodiversity conservation and monitoring efforts can continue after the operational closure of the Project
[Added: 2022/07/06] [Last Updated: 2022/10/31]
IP and UNDP-CO 2022/10 Completed UNDP has taken onboard some of the stakeholders for development of GEF-7 projects. The same partnerships will continue as and when required at provincial and community level History
10. Recommendation:

UNDP Pakistan Country Office to offer initial and refresher Project Management and gender training for projects within its portfolio. Training has the potential to act as a bridge between the IA and Implementing Partners and provides PMUs with the key underpinnings and playbooks to manage projects well and according to best practice, especially to government personnel with limited project exposure. The UNDP Pakistan Country Offices should also offer ongoing guidance on how to apply a gender lens to GEN2 projects. This should be the norm as is the case with other UNDP Country Offices.

Management Response: [Added: 2022/07/06] [Last Updated: 2022/07/06]

Agreed.

Key Actions:

Key Action Responsible DueDate Status Comments Documents
UNDP CO will make sure that all the upcoming projects offer initial and refresher Project Management and gender training for projects within its portfolio. the UNDP Pakistan CO will also offer ongoing guidance on how to apply a gender lens to GEN2 projects.
[Added: 2022/07/06] [Last Updated: 2022/10/31]
UNDP-CO, PMU 2022/10 Completed This is noted, UNDP will ensure relevant projects staff are imparted relevant trainings for the upcoming GEF7 projects History
11. Recommendation:

The PIR is a core reference document and should be the source of truth for any project's logical framework. Projects should internalize and start monitoring cumulative progress against modified indicators following MTRs. PIRs should explicitly delineate changes to the results hierarchy and targets once adopted by the Project Board

Management Response: [Added: 2022/07/06]

Agreed and the recommendation will be take care of while designing future projects as this project is now operatioanlly closed.

Key Actions:

12. Recommendation:

For multi-focal area projects like SFM requiring the involvement of local communities in the on-the-ground implementation of project activities targeting livelihood improvement, there should have been a full-fledged community mobilization component at least at the outcome level. Such a provision in the project design could help in organizing local communities, strengthening traditional governance structures, and creating financial mechanisms for sustaining these structures and activities undertaken with their involvement. Therefore, it would be advisable for such type of future projects should have a desirable level of community mobilization/organization component with a clear outcome and outputs, and corresponding indicators

Management Response: [Added: 2022/07/06]

AS the project is operationally closed so the recommendation will help UNDP in designing new projects.

Key Actions:

13. Recommendation:

Future projects should ensure appropriate mechanisms are in place to ensure ownership is transferred and concentrated at the provincial level and that benefits accrue at the local level, especially local communities within the landscapes per a project’s design.

Management Response: [Added: 2022/07/06]

Agreed. the future projects will  ensure appropriate mechanisms are in place to ensure ownership is transferred and concentrated at the provincial level and that benefits accrue at the local level, especially local communities within the landscapes per a project’s design.

Key Actions:

14. Recommendation:

Project of technical nature like SFM should be managed at the technical level by the personnel having expertise and technical knowledge of the subject both within the project team and among implementing/executing agencies. This would help in proper designing, implementing, overseeing, and producing desired results envisaged under the project design.

Management Response: [Added: 2022/07/06]

Future projects will consider this recommendation during team formulation.

Key Actions:

15. Recommendation:

UNDP should ensure that the Terms of Reference for project audits are expanded to include an assessment of administrative and financial management practices by a project team and implementing/executing agencies. This would be closer to a performance audit and could help GEF agencies target problems at an early stage and apply corrective measures. Furthermore, and as an extra measure of due diligence, a new financial audit should be undertaken, or added to the scope of an existing one. The scope of work should include:

A determination of whether GEF resources were used to extend the operational closure date of the Project and how this is reflected in financial reporting, as it is not clear from the latest Combined Delivery Reports shared with the TE consultant team how this has been reported

Management Response: [Added: 2022/07/06]

will be considered in future GEF projects

Key Actions:

16. Recommendation:

In spite of not being mentioned in the Project Document as a Responsible Party, IUCN was allocated a budget of US$ 1,516,900.00 via a Letter of Agreement (LoA), corresponding to 18.2% of the total GEF grant. Ensure that any partner not mentioned explicitly in the Project Document is selected through competitive tendering and RFP process.

Management Response: [Added: 2022/07/06]

Agreed with the recommendation and UNDP will make sure this in future projects

Key Actions:

Latest Evaluations

Contact us

1 UN Plaza
DC1-20th Floor
New York, NY 10017
Tel. +1 646 781 4200
Fax. +1 646 781 4213
erc.support@undp.org