Sustainable Land and Forest Management in Greater Caucasus

Report Cover Image
Evaluation Plan:
2016-2020, Azerbaijan
Evaluation Type:
Project
Planned End Date:
12/2018
Completion Date:
02/2019
Status:
Completed
Management Response:
Yes
Evaluation Budget(US $):
7,540

Share

Document Type Language Size Status Downloads
Download document SLFM AZEREPORT FINAL.pdf report English 557.78 KB Posted 39
Download document ToR.DOC tor English 189.00 KB Posted 20
Title Sustainable Land and Forest Management in Greater Caucasus
Atlas Project Number: 63140
Evaluation Plan: 2016-2020, Azerbaijan
Evaluation Type: Project
Status: Completed
Completion Date: 02/2019
Planned End Date: 12/2018
Management Response: Yes
Corporate Outcome and Output (UNDP Strategic Plan 2018-2021)
  • 1. Output 1.4.1 Solutions scaled up for sustainable management of natural resources, including sustainable commodities and green and inclusive value chains
Evaluation Budget(US $): 7,540
Source of Funding: GEF
Evaluation Expenditure(US $): 7,540
Joint Programme: No
Joint Evaluation: No
Evaluation Team members:
Name Title Email Nationality
.Charles Louis J. Vanpraet .Mr. .charles.vanpraet@gmail.com
GEF Evaluation: Yes
GEF Project Title: Sustainable Land and Forest Management in Greater Caucasus
Evaluation Type: Terminal Evaluation
Focal Area: Biodiversity
Project Type: FSP
GEF Phase: GEF-5
GEF Project ID: 4332
PIMS Number: 4418
Key Stakeholders: Ministry of Ecology and Natural Resources
Countries: AZERBAIJAN
Lessons
Findings
Recommendations
1

Evaluation Recommendation No. 1:  chronological sequence of the implementation of the outcomes in the logical framework could have been more “logical,” as outlined below:

  1. Definition of the methodologies suitable for the assessment of the rangeland and forest resources conditions and trends, with the involvement of the relevant national institution(s); reason:one has to identify (a) problem (s), and the causes.
  2. Carry out inventories and trend analyses of the land, range and forest resources as required in the logical framework, and identify concrete management issues.   If a problem has to be resolved, it has to be clearly identified, as well as the causes of the problem.
  3. Preparation of the management plans with the relevant institution(s) and strengthening the capacities where needed.
  4. Based on the results under Part 1, analyse the existing legislation, identify shortcomings and propose updating where needed.
  5. Analyses/assessment of the institutional capacities and introduction of adjustments where needed.
  6. PES and C sequestration related activities by the relevant institutions and strengthening of these institutions where necessary in order to ensure sustainability

 

2

Evaluation Recommendation No. 2:  The structure of the Project Board of the Project was not fully in line with the organizational structure prescribed in the Project Document. Either the PB was not set up because of the lack of buy-in, or vice versa, or the absence of the PB resulted in the lack of buy-in.

3

Evaluation Recommendation No. 3: High number of activities was outsourced to national and international consulting companies/consultants.  This in itself does not promote ownership of the project by the national stakeholders, learning by doing, on the job training, on the job capacity building, and not the least sustainability of the Project.

4

Evaluation Recommendation No. 4: Some Monitoring and Evaluation Indicators are unverifiable and not relevant . For example,  Indicators like “improved management” are meaningless.

5

Evaluation Recommendation No. 5: Forestry is a very different sector. A fairly general satellite imagery overview of the Country, as well as some literature, created the impression that the Azeri forests were rather well preserved. While the Mission did not have the opportunity to visit many forests  in the Country during the site visits, the consultant did not see any sign of overgrazing, overcutting, erosion (other than natural)  etc.;  without therefore saying that such spots exist nowhere.

6

Evaluation Recommendation No. 6: Better coordination, at the time of the formulation of the various activities (EU, GIZ, WWF/EU, FSC, etc.)  could have contributed to a better division of the workload, and enhanced effectiveness and efficiency.

7

Evaluation Recommendation No. 7: The involvement of a very high number of external consulting firms and individuals reduced the efficiency (and also sustainability) of the operations due to their high cost and the lack of proper integration of project activities into the relevant national entities.

8

Evaluation Recommendation No. 8:   Following an initial document review, a mission of 6 days was foreseen to cover all aspects of the evaluation.  Needless to say that to examine a project of such scope and complexity, and carry out various sites visits, 6 days (17 to 23 October 2018) is extremely short.

9

Evaluation Recommendation No. 9: In the view of the TE Mission, the main weakness in the entire (complex) project was the absence of a well-structured, goal oriented and moderated inception workshop, which should have brought together all project stakeholders in order to define and agree on the responsibilities of each one of them; define domains of jurisdiction, responsibilities, prepare a clear roadmap of the project, and writing down the decisions in a clear inception report, endorsed by all parties.

1. Recommendation:

Evaluation Recommendation No. 1:  chronological sequence of the implementation of the outcomes in the logical framework could have been more “logical,” as outlined below:

  1. Definition of the methodologies suitable for the assessment of the rangeland and forest resources conditions and trends, with the involvement of the relevant national institution(s); reason:one has to identify (a) problem (s), and the causes.
  2. Carry out inventories and trend analyses of the land, range and forest resources as required in the logical framework, and identify concrete management issues.   If a problem has to be resolved, it has to be clearly identified, as well as the causes of the problem.
  3. Preparation of the management plans with the relevant institution(s) and strengthening the capacities where needed.
  4. Based on the results under Part 1, analyse the existing legislation, identify shortcomings and propose updating where needed.
  5. Analyses/assessment of the institutional capacities and introduction of adjustments where needed.
  6. PES and C sequestration related activities by the relevant institutions and strengthening of these institutions where necessary in order to ensure sustainability

 

Management Response: [Added: 2019/02/05] [Last Updated: 2019/02/05]

Don't agree.

Comments:

The project implemented activities exactly as it is shown above in the TE Report.  First, the project developed pasture inventory methodology based on already proved methodology developed by the other international organizations in the same geographical territory, applied remote sensing approaches and identified problems and causes. The next step was the preparation of the MPs and their application.

In turn, pasture inventory methodology was developed in accordance with the existing monitoring methodologies for pasturelands, which were approved by the Ministry of Ecology and Natural Resources with the involvement of the representatives of the SOIL institute of the National Academy of Sciences, Pasture institute under the MoA, and local freelance consultants, including the Institute of Landscape Greifswald (Germany), which have experience in the field of pasture assessment and inventory since 2004. 

In relation to the Forest management, the methodology identification was discussed with the MENR, also with the Institute of Forestry under the MENR, and agrarian Academy of Azerbaijan with the involvement of local scientists. Then, the proposed methodologies, including FAO methodology, Corine methodology and applied previously in Azerbaijan Austrian methodology were presented to the attention of the forestry department of the MENR.

Before selecting the methodology, the project also conducted baseline assessment by involving all above listed stakeholders. The rapid assessment to identify the gaps, causes, problems, identification of the forest borders, remote sensing, tree species varieties, erosion etc. classification and potential areas for reforestation and afforestation activities were conducted.

Then, the selected methodology was used to carry out the inventory of the forest resources and based on these information, the multifunctional forest management plans were developed and applied.

Above we have listed the chronology of activities carried out by the project in a logical framework.

In addition, number of trainings on forest inventory, on management of the forests has been conducted during the project implementation period.

Project Team has presented all drafts of normative and legislative documents to the Center for Economic Reforms and Communication (CERC) in 2016, and they were used by CERC to prepare the Roadmap for Agricultural Development for Azerbaijan. These drafts were presented to the MENR as well, which, in turn, have reviewed them, and submitted to the CabMin for approval.

The project, taking into account the complexity of the PES approaches, recruited international and local experts to develop the road map on PES in 2016. In 2017, the project implemented demonstration pilot activities on PES with the involvement of different university students and academic staff.

Key Actions:

2. Recommendation:

Evaluation Recommendation No. 2:  The structure of the Project Board of the Project was not fully in line with the organizational structure prescribed in the Project Document. Either the PB was not set up because of the lack of buy-in, or vice versa, or the absence of the PB resulted in the lack of buy-in.

Management Response: [Added: 2019/02/05] [Last Updated: 2019/02/05]

Don't agree.

Comments: According to UNDP rules, each project should contain the PB or steering committee. Those above mentioned two terms are the same in terms of functions. As discussed with the TE during the mission, besides the MENR, the Project Team involved into discussions and project implementation the National Academy of Science, the Ministry of Agriculture, the local executive authorities, municipalities, Azerbaijan Agrarian University, Baku State University. In accordance with the UNDP rules, the Steering Committee included the representatives of UNDP and MENR. All recorded SC Meeting Minutes were presented to the Terminal Evaluator.

Key Actions:

3. Recommendation:

Evaluation Recommendation No. 3: High number of activities was outsourced to national and international consulting companies/consultants.  This in itself does not promote ownership of the project by the national stakeholders, learning by doing, on the job training, on the job capacity building, and not the least sustainability of the Project.

Management Response: [Added: 2019/02/05] [Last Updated: 2019/02/05]

Dont agree.

Comments:

Don’t agree.

Comments:

During the PPG stage it was revealed that there is a need to bring to Azerbaijan and utilize international expertise, and also combine it with the knowledge of local experts. Therefore, the project used their knowledge and services during the project implementation. So, not only international experts, but also many local stakeholders and experts were involved in the project implementation, which enhanced the chances of sustainability of project results. In addition, the project involved hundreds of students to conduct pasture and forest inventories and participate in many capacity building seminars to sustain the project results. Thus, exactly “learning by doing” approach was applied. The local experts and students were getting experience and expertise in the most advanced methods and methodologies.

The project cooperated with Academy of Science of Azerbaijan, particularly with the Institute of Soil and Erosion, Institute of Genetic Resources, Institute of Botany, Baku State University, and Agrarian University. In addition to this, the project very closely cooperated with the Ministry of  Agriculture, Ministry of Ecology and Natural resources, Local Government Bodies in the regions, etc.

Key Actions:

4. Recommendation:

Evaluation Recommendation No. 4: Some Monitoring and Evaluation Indicators are unverifiable and not relevant . For example,  Indicators like “improved management” are meaningless.

Management Response: [Added: 2019/02/05]

Partially agree

Key Actions:

Key Action Responsible DueDate Status Comments Documents
The Project Team agrees that some indicators could be better identified to make them measurable. As the SLFM project is approaching its end, this recommendation will be taken into account during the preparation and baseline assessments of future projects, especially, for relatively bigger projects.
[Added: 2019/02/05]
UNDP 2018/12 Completed Project team partially agrees with this observation and recommendation indicated in the rating table. All indicators are determined during the baseline assessment and are interlinked. In particular, the “improved management” indicator is related to increasing biomass and decreasing the size of degraded areas, particularly erosion areas, which can be calculated by using remote sensing methodology. However, project team agrees that some other indicators are not measurable. For example, it is very hard to measure the results of activities implemented in Alpine and Subalpine zones in two or three years. Long term approach is necessary to monitor the dynamic of changes of such parameters. History
5. Recommendation:

Evaluation Recommendation No. 5: Forestry is a very different sector. A fairly general satellite imagery overview of the Country, as well as some literature, created the impression that the Azeri forests were rather well preserved. While the Mission did not have the opportunity to visit many forests  in the Country during the site visits, the consultant did not see any sign of overgrazing, overcutting, erosion (other than natural)  etc.;  without therefore saying that such spots exist nowhere.

Management Response: [Added: 2019/02/05] [Last Updated: 2019/02/05]

Don’t agree.

Comments:

During the field trips several times we have seen flocks of sheep and some flock of cows in the forest area. These sheep and livestock eat young growth and seedlings. These cases were observed more and can be seen more often in spring and summer time. The problems of the illegal grazing within the forest areas mentioned in all reports of local/international experts as well as in the reports of the main beneficiary, the MENR.

The last inventory of the pasturelands in Azerbaijan was done in 1949-1951 years. So, no updated information is available since that time. The number of sheep has increased from 2.88 million to 8.2 million. (In comparison with 1982) As opposed, pasture areas (both winter and summer) has decreased.

In addition to this, google satellite imageries cannot be used as a technical data to define the forest condition. The TE refers in his remark to the existing literature, however, does not specify which one.

Key Actions:

6. Recommendation:

Evaluation Recommendation No. 6: Better coordination, at the time of the formulation of the various activities (EU, GIZ, WWF/EU, FSC, etc.)  could have contributed to a better division of the workload, and enhanced effectiveness and efficiency.

Management Response: [Added: 2019/02/05] [Last Updated: 2019/02/05]

Partially agree

Key Actions:

Key Action Responsible DueDate Status Comments Documents
UNDP will take into account this recommendation and will conduct regular meetings with the mentioned and other relevant international organizations when developing and implementing new projects in the future.
[Added: 2019/02/05]
UNDP management 2018/12 Completed UNDP and the project conducted coordination meetings with the GIZ, signed the MOU with them, and closely cooperated with them during the project implementation. WWF, GIZ were involved in the project implementation during the pasture and forest MPs preparation and implementation. This is also documented in the training reports of the project. The project also contributed to the development and formulation of the GEF funded FAO project on SFM. Even though, the SLFM Projects ends, MENR and UNDP as depositories of the SLFM Project Results and also, the TE Report will use this recommendation during preparation and implementation of similar projects in the future.
7. Recommendation:

Evaluation Recommendation No. 7: The involvement of a very high number of external consulting firms and individuals reduced the efficiency (and also sustainability) of the operations due to their high cost and the lack of proper integration of project activities into the relevant national entities.

Management Response: [Added: 2019/02/05] [Last Updated: 2019/02/05]

Don’t agree.

 

Comments:

The project applied the first time in Azerbaijan completely new approaches and methodologies such as remote sensing and satellite imageries for analysing the existing situations in the field. Unfortunately, none of national stakeholders at the time of application of those nanotechnologies has relevant technical, human and financial resources to carry out those proposed activities. Thus, the project used external services and in parallel to this conducted series of trainings for national stakeholders such as the Ministry of Ecology and Natural Resources, Baku State University, Azerbaijan Agrarian university, University of pasture and hay meadows, Institute of Botany of National Academy of Sciences, etc. to strengthen the capacity of national stakeholders to sustain the project results.

For instance, local company and international consultant were hired to support with the development of road map in PES sector, in parallel the project involved students to carry out and scale up the information collection for the database and systemize it with the support of the mentioned external services.

Second example: external services were used to apply supervised and unsupervised analysis of the satellite imageries based on remote sensing approaches and in parallel series of training conducted for the students and staff of national Academy of Sciences and Agrarian university of Azerbaijan, in particular its Soil Science department.

The Project Team didn’t neither consider, nor treat SLFM Project as a small project. There were a lot of different activities and works to be done and done within the project. Azerbaijan is not big and young country, sometimes it is difficult to find and involve multifunctional companies in environmental and agriculture sector with sufficient and relevant expertise.

Also, the Project Team involved many local consultants with diverse experience to ensure synergy between them. Naturally, many consultants got involved.

Key Actions:

8. Recommendation:

Evaluation Recommendation No. 8:   Following an initial document review, a mission of 6 days was foreseen to cover all aspects of the evaluation.  Needless to say that to examine a project of such scope and complexity, and carry out various sites visits, 6 days (17 to 23 October 2018) is extremely short.

Management Response: [Added: 2019/02/05] [Last Updated: 2019/02/05]

Don’t agree.

Comments:

The number of days for the mission was put in accordance with the standard GEF TE TOR. The TE consultant had the opportunity to review the Project Document before applying for the position, therefore, he may decide in advance how many days could be required for the TE and request additional number of days for the mission. Also, based on our previous experience, usually one-week mission was satisfactory for both MTR and TE missions.

Key Actions:

9. Recommendation:

Evaluation Recommendation No. 9: In the view of the TE Mission, the main weakness in the entire (complex) project was the absence of a well-structured, goal oriented and moderated inception workshop, which should have brought together all project stakeholders in order to define and agree on the responsibilities of each one of them; define domains of jurisdiction, responsibilities, prepare a clear roadmap of the project, and writing down the decisions in a clear inception report, endorsed by all parties.

Management Response: [Added: 2019/02/05] [Last Updated: 2019/02/05]

Partially  agree

Key Actions:

Key Action Responsible DueDate Status Comments Documents
This recommendation can be taken into account during the preparation of the new projects.
[Added: 2019/02/05]
UNDP 2018/12 Completed The project conducted the inception workshop with the participation of all major local and national level stakeholders. The inception workshop was conducted with the detailed discussions of each outcome. However, the Project Team agrees with the TE that the Inception workshop could be pre-ceded by more detailed discussions with the stakeholders, in the result of which the Inception workshop may have produced more thorough implementation plan. However, at the same time it is worth mentioning that these deficiencies were revealed already during MTR report. Thus, the MT Report has made recommendations, which were then, followed by the Project Team. Therefore, the Project Team was able to compensate for the missions of the Inception Report. However, the Project Team agrees that more attentive approach to the Inception workshop would have benefited the Project, therefore, both MENR and UNDP will take this recommendation into account when starting implementation of the new environmental projects. History

Latest Evaluations

Contact us

220 East 42nd Street
20th Floor
New York, NY 10017
Tel. +1 646 781 4200
Fax. +1 646 781 4213
erc.support@undp.org