Evaluation Detail

Mid-Term Evaluation of Building Resilient Communities in Uganda Project

Evaluation Plan:: 2021-2025, Uganda
Evaluation Type:: Mid Term Project
Planned End Date:: 08/2021
Completion Date:: 12/2021
Status:: Completed
Management Response:: No
Evaluation Budget(US $):: 50,000

Document	Type	Language	Size	Status	Downloads
MTE TOR for International Consultant.docx	tor	English	216.31 KB	Posted	471
MTE TOR for Local Consultant.docx	tor	English	210.55 KB	Posted	471
Executive Summary.docx	summary	English	26.03 KB	Posted	448
Independent Evaluation Report GCF Wetlands_Final 30 September 2021.docx	report	English	3604.18 KB	Posted	457

Title

Mid-Term Evaluation of Building Resilient Communities in Uganda Project

Atlas Project Number:

00104372

Evaluation Plan:

2021-2025, Uganda

Evaluation Type:

Mid Term Project

Status:

Completed

Completion Date:

12/2021

Planned End Date:

08/2021

Management Response:

Yes

Focus Area:

1. Poverty
2. Sustainable
3. Others

Corporate Outcome and Output (UNDP Strategic Plan 2018-2021)

1. Output 2.1.1 Low emission and climate resilient objectives addressed in national, sub-national and sectoral development plans and policies to promote economic diversification and green growth
2. Output 3.4.1 Innovative nature-based and gender-responsive solutions developed, financed and applied for sustainable recovery

SDG Goal

Goal 1. End poverty in all its forms everywhere
Goal 13. Take urgent action to combat climate change and its impacts
Goal 15. Protect, restore and promote sustainable use of terrestrial ecosystems, sustainably manage forests, combat desertification, and halt and reverse land degradation and halt biodiversity loss

SDG Target

1.4 By 2030, ensure that all men and women, in particular the poor and the vulnerable, have equal rights to economic resources, as well as access to basic services, ownership and control over land and other forms of property, inheritance, natural resources, appropriate new technology and financial services, including microfinance
13.1 Strengthen resilience and adaptive capacity to climate-related hazards and natural disasters in all countries
13.3 Improve education, awareness-raising and human and institutional capacity on climate change mitigation, adaptation, impact reduction and early warning

Evaluation Budget(US $):

50,000

Source of Funding:

GCF

Evaluation Expenditure(US $):

50,000

Joint Programme:

Joint Evaluation:

Evaluation Team members:

Name	Title	Nationality
Dr. John Wasige	National Consultant	UGANDA

GEF Evaluation:

Key Stakeholders:

Ministry of Water and Environment, ministries, departments and agencies

Countries:

UGANDA

Lessons

The project documented following lessons learned in 2017-2020 (APRs 2017-2020):

Procurement of contracts for the large studies and specialized technical equipment in Uganda is very slow due to the very low numbers of vendors who could match the specifications. So, in a country with a very limited specialized technical sector, it is advisable to leave more time and, in some cases, reserve a higher budget than expected in order to be able to procure items and skills required. Additionally adequate market surveys and pre-qualification process to screen and identify viable vendors should be implemented. Specialized requirement should be identified beforehand, procurement strategy developed and support from the regional and international UNDP Procurement Offices requested;
Involvement of government agencies as a Responsible Parties without set up coordination mechanism between them can lead to uncoordinated and ineffective delivery of the project outputs. So, relevant MoU s between the RPs should be signed at the project inception;
Without formal MoUs between MWE and UNMA and MWE and MAAIF the flow of funds from MWE to the RPs has been delayed. So, it is recommended that fund transfer to RPs and LDGs should be implemented directly from UNDP;
The relatively small number of fulltime staff in the PMU, from both UNDP as well as the GoU may result in individuals being overstretched and or unavailable when required. So, as a first priority, the necessary human re-sources should be put in place for a project management, with clear roles and responsibilities determined from the project start;
Delays in funds disbursement (from GCF to UNDP and subsequently to implementing partners) is a serious issue for the project implementation. To mitigate the impact there is a need to identify key project activities, prioritize their implementation by ensuring the early initiation of procurement processes and use of framework contracts as a strategy;
Community engagement in the project activities is challenging during elections due to political campaigns implemented in the districts. So, that period the project should focus on activities that does not require large community involvement;
The impacts of COVID-19 pandemic slowed down implementation of project activities. However, in some cases remote and safe community engagement strategies like local radio stations, ICT based communication systems, SMS and phased meetings can be applied;
High community expectations from the project require long time and more active work with communities to clarify and explain what actually the project can provide, and involve them in the process;
Disaster mitigation and response plans are key to handling any emergencies (like flooding) that would otherwise affect timely activity implementation.

The lessons generated by the project and reflected in the APRs are mainly managerial, but not technical. For example, it would be great to generate lessons from wetland restoration and sustainable management activities and climate-smart agriculture and alternative sources of income initiatives by the project. Surprisingly, no such technical lessons were reported in the APRs, however, they are clearly present in the Field Monitoring Reports and BTORs. The project lessons have not been shared yet with stakeholders, other projects and programs.

Some potential best practices that could be considered for sharing has been recorded by the IE team during the field visit:

• In Kabale community policing of wetland restoration and demarcation is more effective than by the district technical staff and MWE officers. This is because the community own and take up responsibility to protect wetland resources for the benefit of whole community;

• In districts where alternative livelihoods have been delivered in full and sufficiently to the wetland community, there is no wetland re-encroachment (e.g. in Nshenga village word II Mitooma town council Mitooma district and Mazuba sub county in Namutumba district)

Findings

1.1.IE Ratings & Achievement Summary Table

IE Criteria	IE Rating	Comments
Project Strategy	MS	The project is designed to address a set of specific climate and non-climate threats for Uganda wetlands and communities, with the most important threat being Conversion of wetlands to agriculture, settlements, and other developments, as the rate of wetland conversion in Uganda is very high. During the project development at least 277 stakeholders were consulted at national and local levels, plus the project was also based on the previous consultations completed in the frameworks of COBWEB and JICA projects in the same districts. The project wetlands/catchments restoration and beneficiaries targets look very ambitious in comparison with area of wetlands restored in Uganda in 2014-2018. There are no specific budgets developed for each project Activity and budget notes do not explain what Activities are associated with each budget line. The PRF Indicators are not completely SMART and redundant.
Relevance	HS	The project is highly relevant to Uganda’s national priorities in wetland conservation and adaptation to climate change, and fully consistent with GCF and UNDP priorities and the Ramsar Convention global goals. Project strategies are fully relevant to address climate and non-climate threats to Uganda’s wetlands and communities. However, in some cases, they do not fully address low monetary value of wetland raw materials for local communities compared to high value agricultural crops produced in the wetlands.
Effectiveness	MU	The project implementation effectiveness is not sufficient yet and full achievement of the project Outcomes[1] as stated in the PRF is unlikely if the project implementation does not change considerably. Thus, Outcome 1 is likely to be achieved by 70% for wetlands and 20% of catchments by 2025; Outcome 2 - by 10% for climate-resilient households and 0.5% for area of climate-smart agriculture; Outcome 3 - by 2%. Given the rate of wetland conversion in Uganda of 294-805 km²/year[2] (1994-2015) the project efforts are currently unlikely to influence the conversion rate significantly[3], without overcoming the project current implementation issues.
Efficiency	MU	The project Output delivery is only 33% of planned by the Mid-Term, So, all Outputs, except Output 3 (that is also behind the planned delivery value) are currently not on the target to be achieved. Actual expenses to deliver the project Outputs are significantly lower than the amount that was planned in AWPs 2017-2021 and consistent with the Output delivery: 36% - for Output 1, 27% - for Output 2, and 27% - for Output 3 If the project continues without significant management changes it is projected to be completed only in 2033.
Progress towards Results	MU	The project had relatively long preparation period: the years of 2017, 2018, and partially 2019 were mainly used for some preparatory activities to deliver the project Outputs. In 2020-2021 the project implementation was significantly slowed down by COVID-19 pandemic and related restriction. The key barriers for the project implementation have been identified as the following: (1) regular delays in release of the project funds; (2) lack of strong coordination between UNDP, MWE, MAAIF, and UNMA to deliver the project Outputs; (3) insufficient understanding of and trust to the project by local communities; (4) insufficient efforts for restoration of wetland catchments; (5) COVID-19 pandemic restrictions in 2020-2021; and (6) Extreme Climate Events (e.g., floods).
Output 1 Delivery	MU	Output 1 delivery is only 27% of planned by the Mid-Term
Output 2 Delivery	MU	Output 2 delivery is only 16% of planned by the Mid-Term
Output 3 Delivery	MU	Output 3 delivery is only 57% of planned by the Mid-Term
Project Implementation and Adaptive Management	MS	The project has a relatively strong project management arrangements with highly representative PSC (17 members), well-developed PMU (12 employees), and TWG (52 members), and three Responsible Parties – WMD/MWE, MAAIF, and UNMA. LDGs and some NGOs are involved in implementation of the project activities as partners. However, collaboration between the PMU, RPs and UNDP is not yet effective and that leads to regular prolonged approval of the project AWPs and disbursement of funds to the PMU and RPs for implementation. The project multi-year plan and AWPs are not detailed enough to allow effective implementation of project activities in the project districts and do not address all UNDP requirements. That results in long AWP approval process by UNDP and regular loss of Q1, Q2 (and sometimes Q4) for the project implementation. Total project expenses of GCF funds for 2017-2021(Q1&Q2) so far are only 31% of the amount planned for the same period in the AWPs. Difference between the date the funds are requested by the RPs and the date the funds are received from UNDP is sometimes as much as 55-77 days (2019 and 2020). Additionally, MAAIF and UNMA receive funding 3-4 months later than MWE, and that does not allow simultaneous delivery of Outputs 1-3 (2018, 2019, 2020). In 2020 the RPs received from UNDP only 50% of requested funding. Overall community engagement in the project activities is only ~25% of total planned community members involvement in 2017-2020.
Sustainability	ML	Financial sustainability is ML; Socio-economic sustainability – MU; Institutional and governance sustainability – L; and Environmental Sustainability – ML. There are some socio-economic risks for sustainability of Output 1 and 2: the probability is still high that local people come back to restored wetlands and convert them to agriculture again in absence of sufficient alternative livelihood options, strong law enforcement and direct compensation from the GoU.
Country Ownership	S	The ownership of the project is high at the level of national government (including the President), but the ownership decreases to LDG and community level, especially in Eastern Uganda due to lack of sustainbale and sufficient benefits. Involvement of LDGs, NGOs, and private sector in the project activities is not yet sufficient. Given regular delays in disbursement of the project funds and long (5-6 months) periods of the project inactivity, involvement of key partners, including local communiies, in the project activities is negatively affected.
Gender Equity	S	The project has a regularly updated Gender Mainstreaming Plan. In 2017-2018 total sums of women and men involved in different project activities are 16,216 (39%) and 25,886 (61%) respectively. However, in 2018-2020 percentage of women participated in the project activities increased from 36% to 39%. The project management bodies – PMU and PSC - are strongly dominated by males with percentage of females of 27% and 18% respectively. The project budget does not have allocation for specific gender mainstreaming activities.
Innovativeness	S	Climate Change Early Warning System that is under development under Output 3 is innovative for Uganda approach to generate weather forecasts and broadcast them among stakeholders through different communication channels. Additionally, new climate-smart agriculture and sustainable livelihood options provided by the project to local communities (e.g., irrigation farming, upland rice farming, fish ponds and bee hives) can be called innovative at local level.
Unexpected Results	MS	In 2017-2021 the project had a number of “positive” and “negative” unexpected results. Some “negative” results have never been reported by the PMU and were discovered by the IE team during the field trip.
Replication and Scalability	S	Lessons generated by the project are mainly managerial, but not technical. Nevertheless, some of the project successful practices were replicated by local communities without the project support. Additionally, one of the project best practices (Limoto wetland sustainable management) was selected by GoU to prioritize and scale up in its third National Development Plan. Potential applicability and scalability of the project lessons and best practices in Uganda and abroad are high, however, they need to be formulated with more details to ensure their successful application.

[1] Introduced by the IE team based on the GCF Impact, Outcome, and Output Indicators in the PRF.

[2] Government of Uganda 2016. Uganda Wetlands Atlas. Volume 2. Popular Version; OAG 2018. A Value For Money Audit Report on the Management of Wetlands in Uganda by the Wetlands Management Department (WMD) under the Ministry of Water and Environment (MWE)

[3] The IE team could not obtain estimates of wetlands area in the project regions in 2020: MWE has not provided any data for the IE mission in a response to UNDP request.

Recommendations

Evaluation Recommendation 1: The IE Team recommend making recommended edits to the project indicators and targets in the PRF in accordance with updated project ToC (Annex 2), if GCF approves the changes. Additionally, it is recommended to develop a set of indicators for each project Activity (2-4 indicators for each Activity) and define their end of the project values consistent with updated project Outputs values. In this way the PMU will have clear understanding how each project Activity contributes to the project Outputs.

Evaluation Recommendation 2.1: The PMU and UNDP should submit a request to the GCF for the project extension for 2 additional years (until 2027) without increasing the project budget due to COVID-19 pandemic and restrictions, and dramatic flooding in 2020-2021.

Evaluation Recommendation 2.2: The PMU, MWE, MAAIF, and UNMA should update the multi-year project plan until 2025 (or until 2027 if the project extension is granted) and downscale it to the project districts, which will strengthen achievement of coordinated project implementation by the districts sharing same wetlands.

Evaluation Recommendation 2.3: By 20th September 2021 and each following year UNDP CO should provide clear requirements/checklist to the PMU, MWE, MAAIF, and UNMA on annual planning and reporting based on analysis of previous issues in the APRs and AWPs.:

Evaluation Recommendation 2.4: By October 30 2021 and each following year the PMU should produce an AWP for next year based on UNDP CO requirements. Each AWP should be downscaled to the district level similar to the multi-year project plan, so that each district will have a plan for delivery of the project Activities and clear annual budget for the project implementation

Evaluation recommendation 2.5: By November 15 2021 and each following year the PMU and UNDP CO should have a technical in-person meeting (3-4 days) with all necessary staff present to review and if necessary correct the AWP and complete clearance process of AWP by UNDP CO, before the AWP is submitted to PSC for approval. By November 30 2021 and each following year the PMU should organize a PSC meeting and approve AWP cleared by UNDP CO. By December 5 2021 and each following year UNDP CO should submit the AWP to UNDP GSSU for review and clearance with expectation to receive funds form GSSU by January 15 2020 and each following year. By February 1 2022 and each following year UNDP CO should provide the funds to MWE, MAAIF, and UNMA or directly to the project partners for implementation of the Activities.

Evaluation Recommendation 2.6: To fast-track delivery of the project activities in 2021-2025 (or 2027 if the extension is approved) the PMU should fully involve working potential of NGOs and LDGs.

Evaluation Recommendation 2.7: For effective delivery and sustainability of the Output 1 the PMU, RPs, and partners should not only demarcate the wetlands restored by the project, but ideally all target wetlands in the project districts[1].

Recommendation 2.8. During delivery of the project Outputs the RPs – MWE, MAAIF, and UNMA should work as one team to ensure that wetland restoration and climate-smart livelihood activities are implemented simultaneously, and communities do not wait for a long time for alternative livelihood options after they have vacated the wetlands

Recommendation 2.9. For delivery of the Outcome 2 the IE team suggests a specific set of the following recommendations: prioritize quality over quantity of provided livelihood options to make sure they are sufficient, sustainable, and accompanied with enough capacity building; exercise a due diligence to select credible local private sector entities in the districts to supply inputs such as seeds, feeds, pesticides, fertilizers, beehives; multiply a few successful livelihood options and avoid unsuccessful ones; pay more attention for development of alternative livelihood options in Eastern Uganda where the main income of local communities is generated from rice farming in wetland; explore and use potential of existing vocational training centers in Uganda instead of establishment of new job training centers; develop partnerships with FAO, IUCN, World Vision and other organizations to fast track delivery of sustainable livelihood options to local communities.

Recommendation 3.1. Similarly to the project planning, project annual and quarterly reporting should be more detailed with explanation of what was achieved in each of the project districts and detailed description of the project expenses against each activity[1].

Recommendation 3.2. To effectively fast-track the project implementation it is recommended that PMU employees seconded from the GoU should work full time for the project in 2021-2025 (or 2027) and should be freed from other responsibilities at their agencies. If this is not possible, consider hiring additional staff to the PMU to coordinate and ensure delivery of Outputs 1-3 on full time basis.

Mid-Term Evaluation of Building Resilient Communities in Uganda Project

Share

1.1.IE Ratings & Achievement Summary Table

Evaluations by Unit

Latest Evaluations

Evaluation Snap Shot

Contact us