Annual Evaluation of Project: Sustainable Development and Integrated Water Management

Report Cover Image
Evaluation Plan:
2017-2021, Saudi Arabia
Evaluation Type:
Mid Term Project
Planned End Date:
10/2019
Completion Date:
11/2019
Status:
Completed
Management Response:
Yes
Evaluation Budget(US $):
40,000
  1. The objective of this program is to initiate a systematic process of capacity development to help in sustainable development of water resources and management of water-related affairs in the Kingdom to ensure permanent and sufficient supply. In order to make a transition to sound water management mode, it is important to strengthen the technical and organizational capacities of the MEWA to deal with the triple challenge of water exploitation and distribution (operational side), research for additional resources and cutting-edge technologies to satisfy the increasing demand (research side), and rigid control and administration of all water-related aspects (control side). Second, a sound information base covering data on groundwater availability, quality, withdrawal, and usage is about to be put in place
  2. The specific objectives of this evaluation are to assess progress towards achieving project targets, assess the project appropriateness of the project design and its relevance and effectiveness and effectiveness, efficiency and sustainability in achieving results. This evaluation focuses more on process and results of first 18 months implementation and draw on emerged issues and opportunities to enable adaptive management.
  3. Key achievements (among many others)
    • Significant contribution to enhance water management capacity at MEWA
    • Led the development of policy paper for the upcoming G20 summit in KSA titled “Fostering Sustainable and Resilient Water System Globally”.
    • Helped the ministry to report on SDG 6 (sustainable water)
    • Developed water supply-demand model for the first time in Saudi,
    • Established a groundwater model for KSA
  4. Evaluation findings
    • The project overall objectives and outputs are aligned to the MEWA needs and priorities, vision 2030 and UNDP priorities.
    • Weak design of the project document (results framework, governance structure and risk identification and management).
    • A moderately satisfactory progress in meeting its expected results. The review of the project achievements indicates generally a strong focus on activities as opposed to developmental results (i.e capacity building).
    • The current management arrangements are ineffective. These are largely the result of ambiguities in the original project design (the Project Document).
    • Limited project management capacities in the project.
    • The project is largely reliant on external funding sources to implement its activities and no clear resource mobilization plan outlining funding alternatives – should the NTF funding fails. 
    • Key project outputs are under-resourced, mainly the NCWRS and Water Extensional Education Center.
    • There are key concerns over the sustainability of the program results, especially in terms of skills and knowledge transfer from experts to the MEWA staff, and MEWA ability to run the business beyond the project.
  5. Key recommendations
    • Undertake a substantive review of the project document
    • Identify capacity needs and develop a capacity building plan
    • Re-structure the project management unit with an intent to enhance the project management capacity and streamline decision making process.
    • Re-activate the project board and project coordination group (currently called steering committee)
    • UNDP to support the project with communication expertise.
    • Upgrade the software and hardware assets to meet the modelling technical requirements
    • Assess the project financial needs and develop a resource mobilization plan
    • Review the project resourcing strategy and ToRs of all experts and address the deficit
    • Design fit-for-purpose reporting templates with a greater focus on the M&E framework – including the performance indicators.

Share

Title Annual Evaluation of Project: Sustainable Development and Integrated Water Management
Atlas Project Number: 00107640
Evaluation Plan: 2017-2021, Saudi Arabia
Evaluation Type: Mid Term Project
Status: Completed
Completion Date: 11/2019
Planned End Date: 10/2019
Management Response: Yes
UNDP Signature Solution:
  • 1. Resilience
Corporate Outcome and Output (UNDP Strategic Plan 2018-2021)
  • 1. Output 1.4.1 Solutions scaled up for sustainable management of natural resources, including sustainable commodities and green and inclusive value chains
SDG Goal
  • Goal 12. Ensure sustainable consumption and production patterns
SDG Target
  • 12.2 By 2030, achieve the sustainable management and efficient use of natural resources
Evaluation Budget(US $): 40,000
Source of Funding: Government Cost Sharing
Evaluation Expenditure(US $): 31,500
Joint Programme: No
Joint Evaluation: No
Evaluation Team members:
Name Title Email Nationality
GEF Evaluation: No
Key Stakeholders:
Countries: SAUDI ARABIA
Lessons
1.
  • Adaptive management and the project cycle should be clearly understood. Absence of adaptive and agile project management approach leads to further complicate the barriers and missing the change opportunities. The inception phase is the first opportunity to review the current status of the project and to determine whether circumstances have changed against the assumptions made in the Project Document. The inception phase should have the confidence and authority to significantly challenge the project’s design and not to accept it as a fait accompli. The role of an inception phase that could have been utilised to allow for validating the project design prior actual implementation starts. It would have helped addressing the deficiencies in the project design. Such a project needed a defined inception phase at start up to review the design elements, engage stakeholders. Any changes to the programme strategy, management arrangements, monitoring framework and participation of stakeholders should be documented in an inception report, which should be endorsed by the management committee overseeing the development of the programme/project.
  • Project governance and management arrangements are critical to the successful outcome of a project. Much of the confusion surrounding the management arrangements in the project appear to have stemmed from imprecise instructions in the Project Document. Deficiencies in project governance design lead to severe delivery risks. This was evidently the case with this project where governance was largely ineffective and highly influenced the decision making and day to day business.
  • Projects need to balance the focus on activities versus the developmental results. Generating high quality products and services may not be enough, unless achieving the ultimate intended outcome (i.e capacity building). In case of this project, there are significant achievement mainly studies, analyses, information, and training events. There are indispensable deliverables but they also remain as information products, unless stakeholders uptake these information, used it and build the capacity, the long-term impact of the project will be limited.

Findings
1.
  • The project overall objectives and outputs are aligned to the MEWA needs and priorities, vision 2030 and UNDP priorities.
  • Weak design of the project document (results framework, governance structure and risk identification and management).
  • A moderately satisfactory progress in meeting its expected results. The review of the project achievements indicates generally a strong focus on activities as opposed to developmental results (i.e capacity building).
  • The current management arrangements are ineffective. These are largely the result of ambiguities in the original project design (the Project Document).
  • Limited project management capacities in the project.
  • The project is largely reliant on external funding sources to implement its activities and no clear resource mobilization plan outlining funding alternatives – should the NTF funding fails. 
  • Key project outputs are under-resourced, mainly the NCWRS and Water Extensional Education Center.
  • There are key concerns over the sustainability of the program results, especially in terms of skills and knowledge transfer from experts to the MEWA staff, and MEWA ability to run the business beyond the project.

Recommendations
1

Undertake a substantive review of the project document

2

Identify capacity needs and develop a capacity building plan

3

Re-structure the project management unit with an intent to enhance the project management capacity and streamline decision making process

4
 

Re-activate the project board and project coordination group (currently called steering committee)

5

UNDP to support the project with communication expertise

6

Upgrade the software and hardware assets to meet the modelling technical requirements

7

Assess the project financial needs and develop a resource mobilization plan

8

Review the project resourcing strategy and ToRs of all experts and address the deficit

9

Design fit-for-purpose reporting templates with a greater focus on the M&E framework – including the performance indicators

Management Response Documents
1. Recommendation:

Undertake a substantive review of the project document

Management Response: [Added: 2019/11/20] [Last Updated: 2019/12/16]

Recommendation taken under advisement

Key Actions:

Key Action Responsible DueDate Status Comments Documents
Review each activity, activity group, and output. Change scope, modify it, and define new ones. All directors and output managers interviewed, everything rearranged, additional personnel needs noted and calculated.
[Added: 2019/12/15] [Last Updated: 2020/01/25]
Project Chief Technical Advisor 2019/12 Completed All actions completed, personnel and project needs all taken into consideration, awaiting final approval by National Project Coordinator and Project Board, due to meet 2 February History
2. Recommendation:

Identify capacity needs and develop a capacity building plan

Management Response: [Added: 2019/11/20] [Last Updated: 2019/12/16]

Recommendation taken under advisement

Key Actions:

Key Action Responsible DueDate Status Comments Documents
Parallely, low-level starter courses are developed
[Added: 2019/12/15] [Last Updated: 2020/06/11]
Consultancy Firm/Project CTA 2020/09 Initiated Assessment just finished, courses are being developed History
Will be done after Phase of Applus Project.
[Added: 2019/12/15] [Last Updated: 2020/06/11]
Project Manager 2020/06 Completed capacity assessment completed History
Define its exit strategy
[Added: 2019/12/15] [Last Updated: 2020/06/11]
CTA/PM 2020/09 Not Initiated New CTA just got recruited History
3. Recommendation:

Re-structure the project management unit with an intent to enhance the project management capacity and streamline decision making process

Management Response: [Added: 2019/11/20] [Last Updated: 2019/12/16]

Recommendation taken under advisement

Key Actions:

Key Action Responsible DueDate Status Comments Documents
Detailed review of positions described in contract. Detailed review of other UNDP Projects in Saudi and outside
[Added: 2019/12/15] [Last Updated: 2020/05/06]
CTA/PM 2020/05 Completed New structure approved, new recruitment launched History
4. Recommendation:
 

Re-activate the project board and project coordination group (currently called steering committee)

Management Response: [Added: 2019/11/20] [Last Updated: 2019/12/16]

Recommendation taken under advisement

Key Actions:

Key Action Responsible DueDate Status Comments Documents
Suggested Groups will be reactivated and meeting plan will be drafted
[Added: 2019/12/15] [Last Updated: 2020/02/04]
PM/CTA 2020/01 Completed A plan has been drafted, Project board re-activated and new decisions to meet all other recommendations are currently being undertaken History
5. Recommendation:

UNDP to support the project with communication expertise

Management Response: [Added: 2019/11/20] [Last Updated: 2019/12/15]

Management and UNDP will meet to discuss

Key Actions:

Key Action Responsible DueDate Status Comments Documents
Steering committee meeting in January will identify needs and assist project in recruitment during Q1 2020
[Added: 2019/12/16] [Last Updated: 2020/05/06]
Mayssam Tamim 2020/04 Completed Steering committee met, identified needs all addressed and new recruitment currently ongoing History
6. Recommendation:

Upgrade the software and hardware assets to meet the modelling technical requirements

Management Response: [Added: 2019/11/20] [Last Updated: 2019/12/16]

Recommendation taken under advisement

Key Actions:

Key Action Responsible DueDate Status Comments Documents
Agreement on new experts needed will be finalized and hardware required will be purchased
[Added: 2019/12/15] [Last Updated: 2020/06/11]
Project Manager 2020/05 Completed restructuring finalized and new experts are being recruited History
7. Recommendation:

Assess the project financial needs and develop a resource mobilization plan

Management Response: [Added: 2019/11/20] [Last Updated: 2019/12/16]

Recommendation taken under advisement

Key Actions:

Key Action Responsible DueDate Status Comments Documents
Additional personnel requirements identified Detailed budget review and changes for sustainability made
[Added: 2019/12/15] [Last Updated: 2020/06/11]
CTA/PM 2020/05 Completed restructuring finalized and new personnel is being recruited History
TOR for ”Fund Raiser” Expert ready, Plan to be finalized by expert
[Added: 2019/12/15] [Last Updated: 2020/06/11]
CTA/PM 2020/06 Completed New advisor to deputy minister hired to assist with fund raising for ministry and project History
8. Recommendation:

Review the project resourcing strategy and ToRs of all experts and address the deficit

Management Response: [Added: 2019/11/20] [Last Updated: 2019/12/16]

Recommendation taken under advisement

Key Actions:

Key Action Responsible DueDate Status Comments Documents
All TORs of experts will be reviewed and resources availability and planning will be done
[Added: 2019/12/16] [Last Updated: 2020/08/12]
CTA/PM 2020/11 Initiated CTA recruitment delayed due to COVID. Many TORs have been reviewed but more remain needed History
9. Recommendation:

Design fit-for-purpose reporting templates with a greater focus on the M&E framework – including the performance indicators

Management Response: [Added: 2019/11/20] [Last Updated: 2019/12/16]

Recommendation taken under advisement

Key Actions:

Key Action Responsible DueDate Status Comments Documents
KPIs being developed.
[Added: 2019/12/15] [Last Updated: 2020/08/12]
CTA 2020/10 Initiated CTA delayed recruitment due to COVID will cause KPIs delay History

Latest Evaluations

Contact us

1 UN Plaza
DC1-20th Floor
New York, NY 10017
Tel. +1 646 781 4200
Fax. +1 646 781 4213
erc.support@undp.org