Terminal evaluation: GEF funded project on land degradation

Report Cover Image
Evaluation Plan:
2016-2020, Uzbekistan
Evaluation Type:
Final Project
Planned End Date:
05/2019
Completion Date:
07/2019
Status:
Completed
Management Response:
Yes
Evaluation Budget(US $):
30,000

Share

Document Type Language Size Status Downloads
Download document Terminal Evaluation-Land project.pdf report English 1522.32 KB Posted 120
Download document Annex A - Terms of Reference.pdf tor English 904.25 KB Posted 73
Title Terminal evaluation: GEF funded project on land degradation
Atlas Project Number: 00075602
Evaluation Plan: 2016-2020, Uzbekistan
Evaluation Type: Final Project
Status: Completed
Completion Date: 07/2019
Planned End Date: 05/2019
Management Response: Yes
Corporate Outcome and Output (UNDP Strategic Plan 2018-2021)
  • 1. Output 1.4.1 Solutions scaled up for sustainable management of natural resources, including sustainable commodities and green and inclusive value chains
SDG Goal
  • Goal 15. Protect, restore and promote sustainable use of terrestrial ecosystems, sustainably manage forests, combat desertification, and halt and reverse land degradation and halt biodiversity loss
SDG Target
  • 15.3 By 2030, combat desertification, restore degraded land and soil, including land affected by desertification, drought and floods, and strive to achieve a land degradation-neutral world
Evaluation Budget(US $): 30,000
Source of Funding:
Evaluation Expenditure(US $): 18,940
Joint Programme: No
Joint Evaluation: No
Evaluation Team members:
Name Title Email Nationality
Dr. Max Kasparek Dr. Max Kasparek
GEF Evaluation: Yes
GEF Project Title: Reducing pressures on natural resources from competing land use in non-irrigated arid mountain, semi-desert and desert landscapes of Uzbekistan
Evaluation Type: Terminal Evaluation
Focal Area: Land Degradation
Project Type: FSP
GEF Phase: GEF-5
GEF Project ID: 4600
PIMS Number: 4649
Key Stakeholders: State Committee for Land Resources, Geodesy, Cartography and State Cadastre
Countries: UZBEKISTAN
Lessons
Findings
1.

3.1 Project Design / Formulation

3.1 Project Design / Formulation 3.1.1 Formulation of the Results Framework

Principal project formulation took place mainly in 2011–2012, i.e. some seven years prior to the TE. It is difficult to evaluate this process retrospectively, and the observations during the TE towards this end are based mainly on an analysis of the Project Document and only to lesser extent on interviews and other personal communication.

3.1.2 Analysis of the Project Results Framework

Does the project address a priority for the development of the Uzbekistan? The Republic of Uzbekiâ??stan is a dry country with a total area of approximately 44.5 million ha, comprised mainly of mounâ??tains (20%) and arid/ semiâ??arid areas (70%). Land degradation is widespread in the country, and causes significant economic costs at three levels: (i) at the field level, in terms of decline in productivâ??ity; (ii) at the national level, in terms of loss of productive capacity of the agricultural land and lower growth of the agricultural gross domestic product and export earnings; and (iii) at the global level, in terms of negative impact on carbon sequestration and climate change, loss of biodiversity, and polluâ??tion of transboundary water resource flows.


Tag: Forestry Natural Resouce management Challenges Effectiveness Relevance Integration Programme Synergy Programme/Project Design Project and Programme management Country Government UNDP Management

2.

3.2 Project Implementation

3.2.1 Adaptive Management

Flexibility is one of the GEF’s operational principles, and all projects must be implemented in a flexi-ble manner to maximize efficiency and effectiveness, and to ensure a results-based, rather than an output-based approach. Thus, during project implementation adaptive management must be em-ployed to adjust to changing circumstances. There are two critical points where the project design needs to be reviewed and where adaptive management can best be introduced: in the Inception Phase and after the Mid-term Review (MTR).

Inception Phase. The Inception Report 2014 was prepared by the Project Manager and the National Project Technical Coordinator together with a National Project Consultant. An Inception Workshop was held in August 2014 and was attended by 53 representatives of various stakeholders. Based on the discussions and recommendations coming out from the workshop and further stakeholder con-sultations, the project design was slightly adapted, e.g. the number of outputs was reduced from five to four (without reducing the scope of interventions) and the phrasing of several indicators was adapted to better comply with the original Russian version (but without changing target values). Also the risk matrix was modified to better reflect the actual situation at the onset of the project.

Altogether, the Project took the opportunity to adapt some aspects of the Project design to the new circumstances which have evolved since project preparation.

Mid-term Review. The Project, at the time of the MTR in December 2016, is being implemented ac-cording to the rules and regulations pertaining to project implementation and is progressing towards its overall objectives, although the MTR was also of the opinion that the Project was going to achieve most of end-of-project targets with significant shortcomings. Significant shortcomings were found in Outcome / Component 1, major shortcomings in Outcome / Component 2. According to the results of the MTR, implementation of some components is not leading to efficient and effective project implementation and adaptive management with some components requiring remedial action.

According to the MTR, the Project failed to develop in the Inception Phase a proper road map which shows how to achieve the project objective, and to identify a set of SMART indicators in order to better monitor the progress and delivery of the Project outputs. The indicators which do not appear very practical/feasible were not adapted in the Inception Phase. For example, the target for the number of hectares of degraded or improved rangeland, forest and rain-fed arable land in two target districts that are under improved management are difficult to verify as long as there is no definition what “degraded” and “improved” means. Following the MTR, the Project prepared a comprehensive Management Response to the MTR, but did not prepare a strategic road map how to achieve the project objective and did not adapt the indicators to make them “SMART”.

While the MTR Report mentions a few times the lack of a roadmap for achieving the Project objec-tive, it appears from the Management Response and the subsequent activities that the recommenda-tion was probably not clear enough. This may be related to the fact that the MTR was not very out-spoken what aspects of the Project need to be changed, and what aspects a roadmap should entail. The MTR for example criticized the very wide scope of project activities, geographically and techni-cally, and favoured a more focused approach, but this is not well reflected in the recommendations.

Other aspects. The Government of Uzbekistan allowed the US-Dollar to float freely starting from 2017, and as a consequence, the exchange rate changed from 2,203 UZS for one US-Dollar in 2010 to 8,485 UZS in April 2019 (at the time of the TE). This had drastic consequences on the project budget and with the need for some modifications. The change in the exchange rate finally also led to the decision to extend the project duration for more than one year beyond the initially foreseen project closure.

 


Tag: Communication Human and Financial resources Monitoring and Evaluation Oversight Project and Programme management Donor International Financial Institutions UNDP Regional Bureaux Data and Statistics

3.

3.3 Project Results

3.3.1 Attainment of Project Objectives

The goal of the Project is to reduce competing land use pressures on natural resources of arid land-scapes, and the objective names Integrated Land Use Planning (ILUP) as the methodology to be ap-plied for this purpose. Regarding improved pastures, the Project reports a cumulative progress of 44,600 hectares, which is well beyond the EoP target of 22,000 ha. Out of 44,600 hectares, pasture rotation practices were introduced at 30,000 hectares and 300 hectares of previously degraded pastures underwent major improvements. Until early 2018, a total of 1,399 hectares of improved rain-fed lands (vs. EoP target of 2,000 ha) were delivered by the project. By early 2018, the Project has succeeded in bringing 3,574 hectares of forestry land under “improved management”14 vs. an EoP target of 11,000 ha. Although the Project carried out additional planting activities in Karakul State Forestry and did additional measures in Zaamin Forestry Farm in 2018/2019, it is unlikely that the Project will achieve the forest-related targets in the remaining months till the end of the Project. These achievements of the Project have to be put in relation to the fact that forests issues are secondary to resolving agricultural land use practices, and the fact that the surface area of improved pastures exceeds the original target by more than two times. The overall results as measured by the project indicators and as reported by the Project itself can therefore be regarded as highly satisfactory.


Tag: Agriculture Forestry Livestock Rural development Ecosystem based adaption Natural Resouce management Water resources Gender Equality Inclusive economic growth Jobs and Livelihoods

4.

3.3.2 Relevance

The Project is consistent with international priorities such as GEF strategies and objectives. It is programmed under the Land Degradation Focal Area for GEF-5, specifically with Land Degradation Stra-tegic Objective 3 (LD-3): “Reduce pressures on natural resources from competing land uses in the wider landscape”. The Project is also consistent with UNDAF Outcome “Principles of sustainable de-velopment integrated into country policies and programs” and contributes to the UNDP Strategic Plan Environment and Sustainable Development “Strengthen national capacity to manage the environment in a sustainable manner while ensuring adequate protection of the poor”, and to the Out-come “Mainstreaming environment and energy”. The Project is highly relevant also for the Government of Uzbekistan as land degradation is a major concern for income of rural people and food security. The livelihood of about 2.5 million people depends on dryland rangelands.

The project is therefore considered highly relevant as it as it combines issues of global importance such as adaptation to climate change, conserving biodiversity and combating desertification with the livelihood of local communities. In this sense it is in line with international commitments made by the Government of Uzbekistan (international environmental conventions such as UNCCD and Convention on Biological Diversity) and also aims at the conservation and rehabilitation of dryland ecosystems for which Uzbekistan has a global responsibility.


Tag: Climate Change Adaptation Biodiversity Natural Resouce management Relevance Sustainability SDG Integration

5.

3.3.3 Effectiveness

The Project delivered most of the foreseen outputs and can be considered highly effective. Most of the targets of the project indicators were fulfilled or the products delivered even exceed the planned targets. The Project was successful especially in delivering the outputs related to rangelands and somewhat less successful in the sectoral fields of forest management and rain-fed agriculture. How-ever, as noted above, the TE in line with the MTR20 considers these two fields of minor importance in the overall picture.

According to the project indicators, the Project rehabilitated 70 per cent more rangeland than fore-seen (Ind. #1) and stopped degradation of rangeland or improved its vegetation cover on a larger surface area than originally planned (Ind. #2). Forest rehabilitation and rehabilitation of rain-fed land lag behind the targets and the Project will surely not be able to reach the goal by project closure. On the national level, the Project assisted in preparing a Draft Law “On pastures” which was reviewed and adopted in 19th Plenary Session of the Senate of Oliy Majlis of the Republic of Uzbekistan on 3 May 2019and signed by the President of the Republic of Uzbekistan on 21 May 2019. The results of the Project exceed in this respect the expectations.

The Project delivered more products than required by the targets of the indicators. Nevertheless a few targets have not been fully achieved (e.g. surface area of pastures classified as “degraded”), but this does not significantly influence the big picture.


Tag: Forestry Effectiveness Rule of law

6.

3.3.4 Efficiency (Cost-effectiveness)

The Project performed in a very efficient way insofar most project activities were conducted in a timely manner and the Project achieved most activities in line with the time schedule of the annual work plans, and usually selected the most cost-effective way in order to achieve the intended objective. The Project was managed by a small team; the amount of managerial input given is consid-ered appropriate. In periods of high workloads, the Project hired national short- and medium-term experts.


Tag: Fishery Livestock Rural development Efficiency Women's Empowerment Capacity Building Trade and Development

7.

3.3.5 Country-ownership

The country ownership for this project is very high. There is a broad consensus both on policy level and on the level of local stakeholders in the demonstration areas that the project is extremely useful and pursues objectives that are important for the development of the country. Various stakeholders expressed to the TE that rangeland management ranks now much higher on the government agenda than at the onset of the project. The project contributed a lot to raise the awareness of decision makers of the relevance of sustainable rangeland management.

While the ownership for the project is in particular high for improving the productivity of rangeland and making better use of this land, environmental concerns play little role in this context.

The Project office is located within the premises of the State Committee on Land Resources, Geode-sy, Cartography and State Cadastre (Goskomzemgeodezkadaster), the project executing agency, thus allowing close and permanent exchange between the executing agency and the Project, and enhancing the efforts of the Project to mainstream the results into regular government work.

The Government of Uzbekistan has made a very significant financial commitment (approx. US$6.7 million) towards the implementation of the project, being almost the threefold amount of the GEF contribution. This is a good indication of a high level of ownership by the government. However, it seems that the government contribution committed at the time of project formulation was badly inflated and the amount the government has actually contributed is only a fraction of it. There are no seconded experts working for the Project with a government salary. Goskomzemgeodezkadaster, the national executing agency, provided free office space within their premises. All government contributions on national and district level in the form of work time (salaries) e.g. for Project Steering Committee members, project meetings, contributions to planning etc. were punctual. It is hardly possible to estimate the actual amount of government co-financing, but it is believed that it is less than 5 per cent of the originally pledged amount.

One may now blame the government that it has not fulfilled the commitments towards the imple-mentation of this project. However, the situation is actually more complex and needs careful evaluation: It is a general feature observed in practically all GEF projects that GEF pushes a lot for identify-ing and leveraging co-financing sources on the one side, but has, on the other side, no system and no standards to monitor and to claim these contributions. So governments make significant commit-ments for increasing the chances to get a project proposal approved, well knowing that there are no binding obligations to deliver. It seems to be GEF policy not to insist on full transparency.

GEF also does not distinguish between baseline financing, co-financing and parallel financing, but puts together all of them under the name “co-financing”. Actually, all government contributions should be counted as baseline funding as long as the government does not provide special financial resources for the Project.


Tag: Global Environment Facility fund e-Governance Donor relations Oversight Country Government

8.

3.3.6 Mainstreaming Cross-cutting Issues

Poverty alleviation: In Uzbekistan, poverty is on the decline but still 12.4 per cent of the population live below the national poverty line25. Rural poverty continues to be significant and 75 per cent of those living in extreme poverty in Uzbekistan live in rural areas. Regional disparities are also considerable with the rural population living in drylands being particularly affected by poverty. The Project directly addresses poverty issues. Creating income for the rural population and help them make their living is at upmost project priority. The Project aims at increasing the economic benefits derived from the rangeland situated in the steppes and semi-deserts. From the 117 measures conducted by the Project, more than 90 per cent were addressing issues with a direct effect on income generation for the rural population.


Tag: Livestock Rural development Climate Change Adaptation Gender Equality Country Government Capacity Building

9.

3.3.7 Sustainability Financial sustainability.

Most of the measures under Outcome 1 (local measures) were supporting something to which the beneficiary had to make an own contribution (self-help approach). While the project contribution usually consisted of goods and technical advice, the own contribution usually consisted of work. This means that the beneficiary was in most cases not requested to provide financial resources. The Project thus covered the cash needed for the investment, while the beneficiary could make an in-kind contribution. As most of the investments will bring back financial return, the interest of the beneficiaries to continue is high. With a very few exceptions, sustainability seems to be granted.


Tag: Livestock Biodiversity Environment Policy Sustainability Resource mobilization Data and Statistics

10.

3.3.8 Impact

Impacts are understood as the broader changes that occur within the community, organization, soci-ety, or environment as a result of programme outcomes. The actual impact of a project is often diffi-cult to assess as long-term impacts can often be seen only after the end of the project. Measuring the impact of a project has something in common with measuring the sustainability: the probability that it will happen needs to be assessed rather than the impact and sustainability itself.

The project measure with the most important potential long-term impact is surely the “Law on Pas-tures”, which was prepared and promoted with the assistance of the Project. It is the first compre-hensive legal framework on rangelands and is highly relevant for the sustainable usage of pasture lands. The law has been designed as framework law which sets forth the essential items. Still a lot has to be done to further develop it into an instrument for sustainable development. It for example does not set upper limits for the livestock number per hectare, or it mentions pasture rotation with-out saying whether this will be a must for all pastures, or it mentions geobotanical survey without saying how the results will put into practical action for pasture management. All these will need by laws and other regulations, and these will happen after Project closure and therefore without the assistance of the Project. It is therefore beyond the control of the Project, whether the Law of Pas-tures will fully meet the expectations and this needs to be taken as project risk.


 


Tag: Natural Resouce management Challenges Effectiveness Impact Women's Empowerment Social Protection

Recommendations
1
  1. Put more effort into the development of sound project concepts.

The main shortcoming of the Project is rooted in a project concept that has not been foreseen a logical flow of activities from local small-scale and micro-measures that relieve the pressure on dryland to upscaled interventions with broad impact, although this is understood as the overarching goal. Project concepts and designs must be based on reasonable results chains and it must be clear for the user of the LogFrame which activity is carried out for what purpose and why it is supported by the Project. Stronger guidance and quality control by UNDP and GEF is required.

2

Only livelihood activities which are linked to the conservation target to be supported by environmental projects with the identified contribution of local communities towards environmental protection.

Even though the improvement of the environmental situation is usually not possible without improving the socio-economic situation of people, this does not mean that all socio-economic measures have a positive effect on the environment.

3

Implement follow-up measures to make the “Law of Pastures” fully operational.

The “Law of Pastures” is a success story of the Project, but it still needs considerable efforts and resources to become operational. Guidance needs to be given to decision-makers especially for an adequate treatment of environmental concerns. UNDP may offer the government technical assistance towards this end and may use for this purpose committed but still unused track funds.

4

Give more guidance as regards accounting of co-financing.

Assessing the level of co-financing is challenging as it is not included in project monitoring. It is particularly difficult to monitor in-kind contributions without guidance what falls under in-kind contribution. Without such guidance, equal monitoring is not possible.

5

Reconsider the rating scale of the criterion „relevance”.

Relevance” can now only be rated as “relevant” or “not relevant”, whereas a finer scale extending e.g. from “highly relevant” over “partly relevant” to “not relevant” would be more appropriate to mirror project reality including the fact that a project often consists of several aspects/components with different levels of relevance.

1. Recommendation:
  1. Put more effort into the development of sound project concepts.

The main shortcoming of the Project is rooted in a project concept that has not been foreseen a logical flow of activities from local small-scale and micro-measures that relieve the pressure on dryland to upscaled interventions with broad impact, although this is understood as the overarching goal. Project concepts and designs must be based on reasonable results chains and it must be clear for the user of the LogFrame which activity is carried out for what purpose and why it is supported by the Project. Stronger guidance and quality control by UNDP and GEF is required.

Management Response: [Added: 2020/01/06] [Last Updated: 2020/11/23]

Partially agree. The concept of the project is aimed at the effective application of integrated land use management planning at the district level, and the existing experience dissemination with best practices in sustainable land use. Therefore, the expected results of the project activities are considered jointly and include the dissemination of best practices in sustainable land management after testing in local areas.

Key Actions:

Key Action Responsible DueDate Status Comments Documents
1.1 In future programming, where possible, to ensure support of pilot and demonstration measures on local level by a replication strategy which indicates the chance of their wide application under the specific circumstances.
[Added: 2020/02/03]
SDC (Environment) 2021/12 Initiated History
1.2 Strengthening the quality assurance process for the project development stage and the SOP on project management.
[Added: 2020/02/03]
SDC ( Environment), RMU 2020/12 Not Initiated
2. Recommendation:

Only livelihood activities which are linked to the conservation target to be supported by environmental projects with the identified contribution of local communities towards environmental protection.

Even though the improvement of the environmental situation is usually not possible without improving the socio-economic situation of people, this does not mean that all socio-economic measures have a positive effect on the environment.

Management Response: [Added: 2020/02/03] [Last Updated: 2020/11/23]

Partially agree. Project activities were aimed at providing alternative income sources, without harming the environment and reducing the burden on the natural resources (forests, pastures, rainfed farming) use, which were repeatedly discussed by the beneficiaries at the local and national levels with the leading experts and scientists involvement to ensure their efficiency and sustainability, according to UNDP and GEF procedures.

Key Actions:

Key Action Responsible DueDate Status Comments Documents
2.1. To strengthen the effectiveness of alternative income-generating activities to alleviate a human threat to specified conservation targets within environmental projects.
[Added: 2020/02/03]
SDC (Environment) 2024/12 Initiated History
3. Recommendation:

Implement follow-up measures to make the “Law of Pastures” fully operational.

The “Law of Pastures” is a success story of the Project, but it still needs considerable efforts and resources to become operational. Guidance needs to be given to decision-makers especially for an adequate treatment of environmental concerns. UNDP may offer the government technical assistance towards this end and may use for this purpose committed but still unused track funds.

Management Response: [Added: 2020/02/03] [Last Updated: 2020/11/23]

Partially agree. As a next step, UNDP will provide guidance/assistance where possible on the practical implementation of the Law within its ongoing and future interventions with a special emphasis on environmental concerns.

Key Actions:

Key Action Responsible DueDate Status Comments Documents
3.1. Include specific activities in ongoing and future programming on the practical implementation of the Law on Pastures with a focus on environmental concerns where relevant.
[Added: 2020/02/03]
SDC, Relevant environmental projects 2022/12 Initiated History
4. Recommendation:

Give more guidance as regards accounting of co-financing.

Assessing the level of co-financing is challenging as it is not included in project monitoring. It is particularly difficult to monitor in-kind contributions without guidance what falls under in-kind contribution. Without such guidance, equal monitoring is not possible.

Management Response: [Added: 2020/02/03] [Last Updated: 2020/11/23]

Agree. In the future, UNDP will take into account and monitor the participation of each national and local partner in co-financing the project.

Key Actions:

Key Action Responsible DueDate Status Comments Documents
4.1. To develop a tool (to be included in the SOP) as a clearer mechanism for monitoring of in-kind contribution from the government.
[Added: 2020/02/03]
SDC, GGC, RMU 2020/12 Initiated
5. Recommendation:

Reconsider the rating scale of the criterion „relevance”.

Relevance” can now only be rated as “relevant” or “not relevant”, whereas a finer scale extending e.g. from “highly relevant” over “partly relevant” to “not relevant” would be more appropriate to mirror project reality including the fact that a project often consists of several aspects/components with different levels of relevance.

Management Response: [Added: 2020/02/03] [Last Updated: 2020/11/23]

Disagree. This recommendation is beyond of UNDP Uzbekistan mandate and is strictly outlined in the UNDP GEF system.

Key Actions:

Key Action Responsible DueDate Status Comments Documents
5.1 UNDP will only communicate the recommendation to GEF Focal Point in IRH.
[Added: 2020/02/03] [Last Updated: 2020/03/17]
SDC, RMU 2020/03 Completed The recommendation is communicated to the GEF focal point in IRH - Mr. Maxim Vergeicichik. History

Latest Evaluations

Contact us

1 UN Plaza
DC1-20th Floor
New York, NY 10017
Tel. +1 646 781 4200
Fax. +1 646 781 4213
erc.support@undp.org