Terminal evaluation of the Project "Conservation and sustainable use of Pamir Alay and Tien Shan ecosystems for snow leopard protection and sustainable community livelihoods"

Report Cover Image
Evaluation Plan:
2016-2022, Tajikistan
Evaluation Type:
Final Project
Planned End Date:
08/2022
Completion Date:
09/2022
Status:
Completed
Management Response:
Yes
Evaluation Budget(US $):
45,000

Share

Document Type Language Size Status Downloads
Download document ToR_SL project_Terminal Evaluation_final.pdf tor English 668.14 KB Posted 20
Download document PIMS5437_Taikistan Snow Leopard_FE Report_final.pdf report English 2612.18 KB Posted 11
Title Terminal evaluation of the Project "Conservation and sustainable use of Pamir Alay and Tien Shan ecosystems for snow leopard protection and sustainable community livelihoods"
Atlas Project Number: 00085264
Evaluation Plan: 2016-2022, Tajikistan
Evaluation Type: Final Project
Status: Completed
Completion Date: 09/2022
Planned End Date: 08/2022
Management Response: Yes
Focus Area:
  • 1. Environment
  • 2. Others
Corporate Outcome and Output (UNDP Strategic Plan 2018-2021)
  • 1. Output 2.4.1 Gender-responsive legal and regulatory frameworks, policies and institutions strengthened, and solutions adopted, to address conservation, sustainable use and equitable benefit sharing of natural resources, in line with international conventions and national legislation
SDG Goal
  • Goal 15. Protect, restore and promote sustainable use of terrestrial ecosystems, sustainably manage forests, combat desertification, and halt and reverse land degradation and halt biodiversity loss
SDG Target
  • 15.1 By 2020, ensure the conservation, restoration and sustainable use of terrestrial and inland freshwater ecosystems and their services, in particular forests, wetlands, mountains and drylands, in line with obligations under international agreements
  • 15.2 By 2020, promote the implementation of sustainable management of all types of forests, halt deforestation, restore degraded forests and substantially increase afforestation and reforestation globally
Evaluation Budget(US $): 45,000
Source of Funding: project budget
Evaluation Expenditure(US $): 15,300
Joint Programme: No
Joint Evaluation: No
Evaluation Team members:
Name Title Nationality
Azam Orifov Local Evaluation Consultant
GEF Evaluation: Yes
GEF Project Title: Conservation and sustainable use of Pamir Alay and Tien Shan ecosystems for snow leopard protection and sustainable community livelihoods
Evaluation Type: Terminal Evaluation
Focal Area: Biodiversity
Project Type: EA
GEF Phase: GEF-1
GEF Project ID:
PIMS Number: 5437
Key Stakeholders:
Countries: TAJIKISTAN
Lessons
Findings
Recommendations
1

1. For the UNDP CO and the NIM IP to support the project operational and financial closure with a comprehensive exit plan. As agreed during the TE meeting with the UNDP CO, make sure that all pending contract commitments are closed (including two incomplete construction/repair objects) before the project financial closure.

2

2. For the NIM IP, to prepare a sustainability plan and communicate its key actions and responsibilities to the principal stakeholders. Present both the exit plan and the sustainability plan at the project final Steering Committee meeting.

3

3. For the NIM IP as an institutional successor of the Project Team, to make the project’s main achievements visible, analyze the end-of-project impact for your stakeholders and for the general public, and present it through leaflets, articles, success stories. Collect and disseminate best practices and lessons learnt from the project interventions. In particular, document the lessons learnt from, and good practices in, the development and implementation of the smart patrol system to guide the future expansion of smart patrols to other PAs. Consider placing the “exit” products above at the NBBC’s website, making sure that the website has references to the knowledge products developed by the project.

4

4. For the NBBC to consider implementing the course of actions suggested by the TE in response to the failure with the project plans to re-classify the Sangvor refuge.

5

5. For the NBBC, to plan a dedicated meeting with the Forestry Agency to discuss the findings, results and recommendations of the Terminal Evaluation. Discuss the plans of this project that did not materialize, as listed in para 93 of the Adaptive Management section of the TE Report and could be brought forward within the context of the upcoming GEF 7 project. Promote a more prominent role for the Agency in the upcoming project, as a partner, decision-maker (in the SC), and a liaison to the project area-based forestry management units

6

6.  In order to further enhance the capacity for implementing future projects, the TE suggested that it might be more efficient for the project reporting processes to record the impact and effect of a concrete measure or intervention right when one sees it on the ground. The PMU is advised to create a reporting data base, an evidence base and the project knowledge base as the project progresses with the implementation and not only when it’s actually time to report. Special attention should be given to the evidence base for PIR reporting, including the co-financing evidence. The PMU might consider specific instruments developed during this TE, such as the SGP Results and Resources Table, the Co-financing Tracking Table, and the Beneficiary Tracking Tool, for future use and modification.

Also, the M&E for the future projects should be strengthened where it concerns project adaptive management: any adaptive management action should be justified, recorded, and discussed openly and transparently. This is particularly relevant to the project onset plans that do not tend to materialize.

7

7.  In order to further enhance the capacity for implementing future projects, the TE suggested that the ToRs for the project substantive activities directly contributing to the achievement of the Results Framework targets should include improved reporting requirements, so that the substantive reports from subcontracts backed up with figures, maps, etc. could be used as evidence of project achievements. These are to be attached to the annual PIRs and verified by the MTR and TE. The reports should include an executive summary with the description of the task, its scope, methodology and results, and quantitative and qualitative statements of impact. the ToRs should also include the indicators of performance for the output/activity to be contracted, in accordance with the Results Framework and Monitoring Plan for the project, which are annexed to the Project Document and amended as required in the course of project adaptive management. Relevant indicators and means of verification should be proposed by the ToRs developer and agreed to with the contractor as part of the contract conditions. If more than one report is commissioned to support the project reporting for a particular RF indicator, the Component Leader or CTA should prepare an evidence note summarizing all relevant reports and their findings, impact indicators and qualitative and quantitative evidence of impact and progress. UNDP and NBBC should consider an enhanced capacity building effort aimed at potential and actual project contractors, so that the contract requirements are met, the ToRs get fully implemented, and the contract reporting (including qualitative and quantitative evidence of progress against project performance indicators) is improved

8

8.  In order to further enhance the capacity for implementing future projects, the TE advises NBBC to invest in professional help with the format of contracts, agreements, cooperation LOAs, letters of intention, etc., entered into by NBBC in its capacity as the project Implementing Partner. It is understood that NBBC generally follows one particular contract form based on the LOA with UNDP on provision of project support services. This format a) is not particularly relevant for cooperation agreements where two parties cooperate in implementation of parallel activities and ensure synergies and co-financing, and b) does not particularly respond to the nature of most contractual works (not services) engaged by the project. The contract forms should be reasonable, responsive to the principal requirements of UNDP procurement and financial policies, and should not contain statements nor conditions that are not reflective of the nature of contracted works or services, or are alien to the national contractual regulations and practices. The contracts should be free from elements that might potentially create risks to the relationship with the project partners or reveal weaknesses for potential legal cases (such as intellectual property rights, penalty for delayed performance, etc.)

9

9.  In order to comply with the existing GEF guidelines, the PMU of the upcoming GEF 7 project should develop a process for yearly confirmation and verification of the project co-financing. Every year, as part of the annual reporting (PIR) exercise, the PMU should be collecting firm evidence to confirm the co-financing. Copies of evidence should be maintained by the UNDP CO for any audit purpose, as well as made available for verification by the independent project terminal evaluation. For the parallel co-financing from sectoral ministries, specific guidance with the relevance criteria should be developed by the UNDP CO in cooperation with the sectoral stakeholders. This will ensure reliability and consistency of reporting and evidence. The methodology for collecting information on co-financing which has materialized should be streamlined; the formats for reporting and evidence should be shared with the partners and supported by relevant capacity building exercise. The current version of the NBBC co-financing letter can be used as a model

10

10.  The IP together with the future GEF 7 project team might consider preparation of two budget revisions a year, one in March and the second one toward the year-end, to ensure that the ASL for January-February of the following year is approved beforehand and the project does not have to wait for it in the new year

11

11.  The future project(s) might consider engaging international consultants as an outside help when the team’s capacity is strained, be it a complicated case of an innovative patrolling system, the best practice for ecosystem restoration, or an exit strategy for the project that would be prepared in time, supported with comprehensive consultations with all relevant stakeholders, and would actually work

12

12.  A timely preparation of the exit strategy and a sustainability plan is vital for a successful project phase-out. The exit strategy supported by a costed exit plan, and a sustainability plan for the project should be embedded in the project M&E design

1. Recommendation:

1. For the UNDP CO and the NIM IP to support the project operational and financial closure with a comprehensive exit plan. As agreed during the TE meeting with the UNDP CO, make sure that all pending contract commitments are closed (including two incomplete construction/repair objects) before the project financial closure.

Management Response: [Added: 2022/10/19]

Accepted. UNDP supports this recommendation and fully acknowledges the importance of a comprehensive exit strategy for the project. UNDP CO will continue working and follow up with national implementing agency NBBC on the exit strategy and a detailed action plan.

In regard to pending contract commitments, key actions that should precede the project financial closure are listed as key actions for Recommendations 1-6 below and will be tracked by both the UNDP CO Management and the Head of the Project Implementing Partner.

Key Actions:

Key Action Responsible DueDate Status Comments Documents
1.1 Complete the Repair of the checkpoint in the Sangvor section of the Tajik National Park, which was damaged by a natural disaster (rockfall) in January 2022* 1.1.1. Closely monitor the completion of work and report on the readiness of the facility for inspection and acceptance of work
[Added: 2022/10/19]
NBBC 2022/11 Overdue-Initiated * Repair works from January to April 2022 were not possible due to the inaccessibility of the territory and high snow cover. The park informed the project about the consequences of the rockfall on April 15, 2022. The park asked for help in repairing the facility. The contract with the contractor for the repair of the facility was concluded on May 23, 2022; The work under the contract was to be completed by June 30, 2022. Unfortunately, the contractor did not meet the deadlines for the execution of work under the contract. History
1.2 Complete the construction of the checkpoint in the Lakhsh section of the Tajik National Park** 1.2.1. Closely monitor the completion of work and report on the readiness of the facility for inspection and acceptance of work
[Added: 2022/10/19]
NBBC 2022/11 Overdue-Initiated ** The contract for the construction of a checkpoint in the Lyakhsh National Park was concluded on September 1, 2021; The work under the contract was to be completed by June 30, 2022. Due to the difficult geographical conditions of the territory (the facility is located at an altitude of 3,000 thousand meters above sea level) and the associated difficulties in transporting building materials, as well as due to the prolonged winter-spring period with snow cover, the contractor did not manage to complete construction work within the original timeframe.
1.3. Elaborate the project exit strategy and sustainability action plan
[Added: 2022/10/19]
NBBC 2022/11 Overdue-Initiated
1.4. Convene the final Steering Committee meeting for the project. The meeting can be timed to coincide with the Snow Leopard Day (October 23) and used to continue the information sharing and awareness raising activities of the project.
[Added: 2022/10/19]
NBBC (organization, stakeholder consultations, presentations) 2022/11 Overdue-Initiated
2. Recommendation:

2. For the NIM IP, to prepare a sustainability plan and communicate its key actions and responsibilities to the principal stakeholders. Present both the exit plan and the sustainability plan at the project final Steering Committee meeting.

Management Response: [Added: 2022/10/19]

Accepted. UNDP will follow up with NBBC on the exit strategy and sustainability plan. The recommendation was also communicated to the Committee for Environmental Protection; and the Committee would welcome concrete actions aimed at ensuring the long-term sustainability of project results to be embedded into NBBC’s own workplans.

Key Actions:

Key Action Responsible DueDate Status Comments Documents
2.1 Actions will include maintenance of the SMART patrolling system, coordination of data gathering from the camera traps installed by the project and availability of this data for decision-making, necessary follow-up on the National Snow Leopard Action Plan and other actions supporting the relevant international commitments of the Government, etc.
[Added: 2022/10/19]
NBBC 2022/11 Overdue-Initiated
3. Recommendation:

3. For the NIM IP as an institutional successor of the Project Team, to make the project’s main achievements visible, analyze the end-of-project impact for your stakeholders and for the general public, and present it through leaflets, articles, success stories. Collect and disseminate best practices and lessons learnt from the project interventions. In particular, document the lessons learnt from, and good practices in, the development and implementation of the smart patrol system to guide the future expansion of smart patrols to other PAs. Consider placing the “exit” products above at the NBBC’s website, making sure that the website has references to the knowledge products developed by the project.

Management Response: [Added: 2022/10/19]

Accepted. The recommendation has already been partially implemented, as NBBC prepared a comprehensive summary of the project achievements (10 pages available in Tajik) at the request of the Committee for Environmental Protection. Based on this document, the final PIR for the project and the TE report, a succinct report of the project achievements, impacts, sustainability aspects and lessons learned will be prepared and presented as a principal hand-out material at the project final Steering Committee Meeting and the associated Show Leopard Day festivities, and placed at the websites of NBBC, UNDP and the Committee for Environmental Protection. NBBC will discuss the disclosure issues for the project key deliverables and, based on the discussions, consider placing the key project products on-line at the NBBC website.

Key Actions:

Key Action Responsible DueDate Status Comments Documents
3.1 Prepare a summary report of the project achievements, impacts, sustainability aspects and lessons learned
[Added: 2022/10/19]
NBBC 2022/11 Overdue-Initiated
3.2 Distribute the project final report, the summary of the project achievements, impacts, sustainability aspects and lessons learned (Action 1.1. above) at the project Steering Committee meeting. Report to the SC of the TE findings. Complete a news release on the project completion.
[Added: 2022/10/19]
NBBC 2022/11 Overdue-Initiated
3.3 Make the “exit” products above (incl. the TE report) publicly available through the websites of NBBC, UNDP and, where appropriate, the Committee for the Environmental Protection.
[Added: 2022/10/19]
NBBC, UNDP, Committee for Environmental Protection 2022/11 Overdue-Initiated
4. Recommendation:

4. For the NBBC to consider implementing the course of actions suggested by the TE in response to the failure with the project plans to re-classify the Sangvor refuge.

Management Response: [Added: 2022/10/19]

Accepted. NBBC as the project Implementing Partner confirms its commitment to support the necessary follow-up on the issue with the Sangvor refuge beyond the project timeframe, through the key actions proposed by the TE, as presented below in Key Actions.  

Key Actions:

Key Action Responsible DueDate Status Comments Documents
4.2 For the Committee for Environmental Protection to conclude the consultations with the Forestry Agency regarding the refuge as soon as possible, bearing in mind that the refuge formally “expires” in 2023. In particular, to check if the Forestry Agency would be willing to confirm either their concurrence with the transfer of the refuge into the National Park or keep the refuge and assume responsibilities and costs that are more adequate to the natural value and conservation objectives declared at the time of the Sangvor Refuge’s establishment
[Added: 2022/10/19]
Committee for Environmental Protection, NBBC 2023/12 Initiated
4.3 For the Committee for Environmental Protection and NBBC to engage relevant expertise to confirm that the “lifetime” re-instatement of the refuge without the change in the PA category is possible, in accordance with the Law on Protected Areas. In case the feasibility is confirmed, prepare relevant documentation for the Government.
[Added: 2022/10/19]
NBBC, Committee for Environmental Protection 2023/01 Initiated
4.4 Based on the results of the activities listed above, proceed further with the action plan suggested by the TE (incl. the update of the key justification documents required to support the comprehensive consultations on the Sangvor Refuge, including the cost-benefit assessment and the PA capacity needs assessment)
[Added: 2022/10/19]
NBBC, Committee for Environmental Protection 2022/11 Overdue-Initiated
4.5 For the Management of the Sangvor Refuge and the Sangvor Section of the Tajik National Park to consider a cooperation agreement for joint patrolling, control and monitoring
[Added: 2022/10/19]
NBBC (drafting of the agreement); Committee for Environmental Protection, PA Agency, Forestry Agency (consultations); Management of the Sangvor section of the NP and the Sangvor refuge (conclusion of the Agreement) 2022/12 Initiated
4.1 Share the TE findings and suggestions (para 127 pf the TE report) with the principal Governmental stakeholders, the Committee for the Environmental Protection and the Forestry Agency
[Added: 2022/10/19]
NBBC 2022/09 Completed
5. Recommendation:

5. For the NBBC, to plan a dedicated meeting with the Forestry Agency to discuss the findings, results and recommendations of the Terminal Evaluation. Discuss the plans of this project that did not materialize, as listed in para 93 of the Adaptive Management section of the TE Report and could be brought forward within the context of the upcoming GEF 7 project. Promote a more prominent role for the Agency in the upcoming project, as a partner, decision-maker (in the SC), and a liaison to the project area-based forestry management units

Management Response: [Added: 2022/10/19]

Accepted. NBBC will proceed with the consultations upon confirmation with the Committee for the Environmental Protection, as part of the GEF-7 project inception phase.  

Key Actions:

Key Action Responsible DueDate Status Comments Documents
5.1 Plan a dedicated meeting with the Forestry Agency to discuss the findings, results and recommendations of the Terminal Evaluation
[Added: 2022/10/19]
NBBC 2022/11 Overdue-Initiated
6. Recommendation:

6.  In order to further enhance the capacity for implementing future projects, the TE suggested that it might be more efficient for the project reporting processes to record the impact and effect of a concrete measure or intervention right when one sees it on the ground. The PMU is advised to create a reporting data base, an evidence base and the project knowledge base as the project progresses with the implementation and not only when it’s actually time to report. Special attention should be given to the evidence base for PIR reporting, including the co-financing evidence. The PMU might consider specific instruments developed during this TE, such as the SGP Results and Resources Table, the Co-financing Tracking Table, and the Beneficiary Tracking Tool, for future use and modification.

Also, the M&E for the future projects should be strengthened where it concerns project adaptive management: any adaptive management action should be justified, recorded, and discussed openly and transparently. This is particularly relevant to the project onset plans that do not tend to materialize.

Management Response: [Added: 2022/10/19]

Accepted and will be taken into account during the upcoming GEF-7 project implementation.  

Key Actions:

Key Action Responsible DueDate Status Comments Documents
6.1 Streamline the future project’s M&E and the specific reporting processes as advised
[Added: 2022/10/19]
The future PMU of the upcoming GEF-7 FSP, as well as to NBBC and the UNDP CO 2026/12 Initiated History
7. Recommendation:

7.  In order to further enhance the capacity for implementing future projects, the TE suggested that the ToRs for the project substantive activities directly contributing to the achievement of the Results Framework targets should include improved reporting requirements, so that the substantive reports from subcontracts backed up with figures, maps, etc. could be used as evidence of project achievements. These are to be attached to the annual PIRs and verified by the MTR and TE. The reports should include an executive summary with the description of the task, its scope, methodology and results, and quantitative and qualitative statements of impact. the ToRs should also include the indicators of performance for the output/activity to be contracted, in accordance with the Results Framework and Monitoring Plan for the project, which are annexed to the Project Document and amended as required in the course of project adaptive management. Relevant indicators and means of verification should be proposed by the ToRs developer and agreed to with the contractor as part of the contract conditions. If more than one report is commissioned to support the project reporting for a particular RF indicator, the Component Leader or CTA should prepare an evidence note summarizing all relevant reports and their findings, impact indicators and qualitative and quantitative evidence of impact and progress. UNDP and NBBC should consider an enhanced capacity building effort aimed at potential and actual project contractors, so that the contract requirements are met, the ToRs get fully implemented, and the contract reporting (including qualitative and quantitative evidence of progress against project performance indicators) is improved

Management Response: [Added: 2022/10/19]

Accepted and will be taken into account during the upcoming GEF-7 project implementation.  

Key Actions:

Key Action Responsible DueDate Status Comments Documents
7.1 Amend the future project’s reporting and other relevant arrangements as advised
[Added: 2022/10/19]
The future PMU of the upcoming GEF-7 FSP, as well as to NBBC and the UNDP CO 2026/12 Initiated History
8. Recommendation:

8.  In order to further enhance the capacity for implementing future projects, the TE advises NBBC to invest in professional help with the format of contracts, agreements, cooperation LOAs, letters of intention, etc., entered into by NBBC in its capacity as the project Implementing Partner. It is understood that NBBC generally follows one particular contract form based on the LOA with UNDP on provision of project support services. This format a) is not particularly relevant for cooperation agreements where two parties cooperate in implementation of parallel activities and ensure synergies and co-financing, and b) does not particularly respond to the nature of most contractual works (not services) engaged by the project. The contract forms should be reasonable, responsive to the principal requirements of UNDP procurement and financial policies, and should not contain statements nor conditions that are not reflective of the nature of contracted works or services, or are alien to the national contractual regulations and practices. The contracts should be free from elements that might potentially create risks to the relationship with the project partners or reveal weaknesses for potential legal cases (such as intellectual property rights, penalty for delayed performance, etc.)

Management Response: [Added: 2022/10/19]

Accepted and will be taken into account during the upcoming GEF-7 project implementation.  

Key Actions:

Key Action Responsible DueDate Status Comments Documents
8.1. Amend the format of contracts, agreements, cooperation LOAs, letters of intention, etc., as advised
[Added: 2022/10/19]
NBBC 2026/10 Initiated
9. Recommendation:

9.  In order to comply with the existing GEF guidelines, the PMU of the upcoming GEF 7 project should develop a process for yearly confirmation and verification of the project co-financing. Every year, as part of the annual reporting (PIR) exercise, the PMU should be collecting firm evidence to confirm the co-financing. Copies of evidence should be maintained by the UNDP CO for any audit purpose, as well as made available for verification by the independent project terminal evaluation. For the parallel co-financing from sectoral ministries, specific guidance with the relevance criteria should be developed by the UNDP CO in cooperation with the sectoral stakeholders. This will ensure reliability and consistency of reporting and evidence. The methodology for collecting information on co-financing which has materialized should be streamlined; the formats for reporting and evidence should be shared with the partners and supported by relevant capacity building exercise. The current version of the NBBC co-financing letter can be used as a model

Management Response: [Added: 2022/10/19]

Accepted and will be taken into account during the upcoming GEF-7 project implementation.  

Key Actions:

Key Action Responsible DueDate Status Comments Documents
9.1 Amend the future project’s reporting and other relevant arrangements as advised
[Added: 2022/10/19]
The future PMU of the upcoming GEF-7 FSP, as well as to NBBC and the UNDP CO 2026/10 Not Initiated
10. Recommendation:

10.  The IP together with the future GEF 7 project team might consider preparation of two budget revisions a year, one in March and the second one toward the year-end, to ensure that the ASL for January-February of the following year is approved beforehand and the project does not have to wait for it in the new year

Management Response: [Added: 2022/10/19]

Accepted and will be taken into account during the upcoming GEF-7 project implementation.  

Key Actions:

Key Action Responsible DueDate Status Comments Documents
10.1 Amend the future project’s financial management and other relevant processes as advised
[Added: 2022/10/19]
The future PMU of the upcoming GEF-7 FSP, as well as to NBBC and the UNDP CO 2023/12 Not Initiated
11. Recommendation:

11.  The future project(s) might consider engaging international consultants as an outside help when the team’s capacity is strained, be it a complicated case of an innovative patrolling system, the best practice for ecosystem restoration, or an exit strategy for the project that would be prepared in time, supported with comprehensive consultations with all relevant stakeholders, and would actually work

Management Response: [Added: 2022/10/19]

Accepted and will be taken into account during the upcoming GEF-7 project implementation.  

Key Actions:

Key Action Responsible DueDate Status Comments Documents
11.1 Take the recommendation into account during the future GEF 7 project implementation
[Added: 2022/10/19]
The future PMU of the upcoming GEF-7 FSP, as well as to NBBC and the UNDP CO 2023/12 Not Initiated
12. Recommendation:

12.  A timely preparation of the exit strategy and a sustainability plan is vital for a successful project phase-out. The exit strategy supported by a costed exit plan, and a sustainability plan for the project should be embedded in the project M&E design

Management Response: [Added: 2022/10/19]

Accepted and will be taken into account during the upcoming GEF-7 project implementation.  

Key Actions:

Key Action Responsible DueDate Status Comments Documents
12.1 Take the recommendation into account during the future GEF 7 project implementation
[Added: 2022/10/19]
The future PMU of the upcoming GEF-7 FSP, as well as to NBBC and the UNDP CO 2023/08 Not Initiated

Latest Evaluations

Contact us

1 UN Plaza
DC1-20th Floor
New York, NY 10017
Tel. +1 646 781 4200
Fax. +1 646 781 4213
erc.support@undp.org