Terminal Evaluation for Project SAU10/82003 Saudi Food and Drug Authority Phase II

Report Cover Image
Evaluation Plan:
2017-2021, Saudi Arabia
Evaluation Type:
Final Project
Planned End Date:
03/2019
Completion Date:
12/2018
Status:
Completed
Management Response:
Yes
Evaluation Budget(US $):
21,000

Share

Document Type Language Size Status Downloads
Download document 180402 TOR for the Terminal evaluation mission - AS.doc tor English 140.50 KB Posted 48
Download document 181127 Evaluation Report 2018 - AS.pdf report English 1352.53 KB Posted 52
Title Terminal Evaluation for Project SAU10/82003 Saudi Food and Drug Authority Phase II
Atlas Project Number: 65588
Evaluation Plan: 2017-2021, Saudi Arabia
Evaluation Type: Final Project
Status: Completed
Completion Date: 12/2018
Planned End Date: 03/2019
Management Response: Yes
Corporate Outcome and Output (UNDP Strategic Plan 2018-2021)
  • 1. Output 1.1.1 Capacities developed across the whole of government to integrate the 2030 Agenda, the Paris Agreement and other international agreements in development plans and budgets, and to analyse progress towards the SDGs, using innovative and data-driven solutions
Evaluation Budget(US $): 21,000
Source of Funding: Project Budget
Evaluation Expenditure(US $): 21,000
Joint Programme: No
Joint Evaluation: No
Evaluation Team members:
Name Title Email Nationality
Lajos Kovacs Independent Evaluator mail@lajoskovacs.com HUNGARY
GEF Evaluation: No
Key Stakeholders:
Countries: SAUDI ARABIA
Lessons
Findings
1.

The Project addressed challenges that Saudi Arabia faces as introduced in the strategy of Ministry of Health (MoH). The challenges for the health sector include the high costs of health services due to the accelerating development of medical technologies in hardware, equipment tools and advanced and expensive medical technologies, as well as the unremitting discoveries of new expensive drugs. The United Nations Development Program (UNDP) project covered and provided technical support to the components of Saudi Food and Drug Authority (SFDA) Second Strategic Plan. 

Regarding the project management, to ensure coordination of the Project’s activities and to increase coherence between the implementation of different components the UNDP coordinated the activities in close cooperation with SFDA management. While UNDP was responsible for such coordination, SFDA also cooperated actively in organizing the Project activities.

The objectives of the Project were broadly valid and appropriate (Relevance). They helped focus the resources on issues that generate the most added value for SFDA, while accommodating existing needs and fitting well with health policy in Saudi Arabia. The activities are in general consistent with the strategy of Ministry of Health and they corresponded to the needs.

The project, through its impact on SFDA’s operation according to its mandate and by strengthening its institutional, staff and systems capacities in all sectors, helped to ensure that only good quality and safe products are sold in the market and contributed towards the improvement of the supply chain management (Effectiveness). The Project also supported SFDA in the improvement of its business processes and systems in licensing, inspection and registration. Therefore, the Project improved the level of the quality of healthcare in Saudi Arabia.

The terminal evaluation of the UNDP Project was not supposed to be a financial audit, instead the evaluation focused on whether the planned project objectives were achieved and assessed the utilization of funds made available for the Project by reviewing the Annual Audit Reports of the Independent Auditor. In the opinion of the Auditor, the statements of project expenses were prepared in conformity with the approved project budgets and for the approved purposes of the Project, followed the relevant UNDP regulations and rules, policies and procedures, and were supported by properly approved vouchers and other supporting documents. The Project added significant value and was cost-effective, as the activities were well designed and outcome-focused (Efficiency).

The evaluation examined the extent to which the objectives of the Project have been achieved as intended, and whether the effects of the Project have been facilitated or constrained by any factors. The assessment proved the logical reasoning that links the needs, the objectives and a range of policy options to address the needs (Impact). The Project significantly contributed to the health status of the Saudi people. The risk of food, feed and medicinal products on the market, which are falsified, of bad quality, with no active ingredients at all, etc., which might be fatal, especially for children or severe sick people, was reduced. In this respect, all activities planned and achieved towards better quality control and inspections are useful for the health and safety of population. The Project directly affected the political and policy decision makers, the employees of SFDA, the clients and customers of SFDA. The Project indirectly affected the Saudi population, by ensuring access to better quality and controlled food and medicinal products.

On the basis of its impacts and outcomes the Project seems to be sustainable, considering the following aspects of project implementation:

•         The Project maintained strong ownership of SFDA in regulation regarding food, drug and medical devices products.

•         The Project was beneficiary-driven targeting capacity building. 

•         The Project enjoyed strong leadership at the technical and governance at the highest level.

•         The Project proved that there is a need for a legal framework for operationalization and implementation of regulatory function regarding food, drug and medical devices.

•         The Project strengthened coordination of financial / funding sources and encouraged efficient use of resources.

•         The Project improved multi-sector collaboration.

•         The Project had impact on SFDA, its clients and consumers.

•         The Project developed IT systems and data management.

•         The Project built individual IT capabilities.

•         The Project built institutional capacity and developed skills of staff.

The Project created internal coherence as the structure provided a framework to fund and achieve activities in well-defined areas and fostered synergies between these activities. The different sectors of SFDA should co-operate better than in the past. This will generate synergistic effects which could be further supported by the newly established Operations Sectors through its horizontal, cross-sectoral tasks and responsibilities. Synergistic effects can also be accomplished by using a common quality management system, purchasing laboratory consumables (reagents, chemicals, etc.) at institutional level instead of laboratory level due to increased buying power, a possibly reduced administrative overhead, easier and faster communication e.g. on scientific / analytical issues and support. The Project also strengthened the links between international partners and ensured the benchmarking of SFDA operation, therefore provided external coherence.

The Project targeted the following cross-cutting issues, in particular: gender equality, good governance (in its reform agenda and decentralisation of the healthcare system), and the rights of the vulnerable groups of population. Both the Saudi Vision 2030, and on its basis the National Transformation Program (NTP) 2020 intend to open up the country to more employment opportunities for women and to empower women. SFDA has recently also employed qualified female staff for both technical, scientific and management positions which was not the case in the past years. Also, NTP 2020 includes several initiatives which aims at increasing the effectiveness of health system through establishment of new centres for various domains of health care which can be seen as a reform agenda for decentralization of and strengthening the healthcare system. Once the healthcare system will be further built on the basis of principles of equity, accessibility and quality of health care services, vulnerable groups will have equal access to equal services. SFDA, through the outputs of the UNDP Project and by implementing its upcoming Third Strategic Plan prepared in accordance with the Saudi Vision 2030 and the National Transformation Program 2020, also contributes to the better performance of healthcare system.

 


Recommendations
1

1.    Maintain the focus on specific objectives and areas where the project provided added value

2

2.    Maintain and strengthen links between the new project and the national policies (e.g. NTP2020, MoH Health Strategy) to maximize impact

3

3.    Support the project management and coordination with UNDP

4

4. Maintain the monitoring of the progress of the new project according to the programmatic and action specific indicators set in the Logical Framework.

5

Design and propose a new project to further support SFDA implementing its Third Strategic Plan and building institutional and individual capabilities

6

6.    Maintain the implementation of the activities of the new project in accordance with the plan of action.

7

7. Maintain the monitoring of the progress of the new project according to the programmatic and action specific indicators set in the Logical Framework

1. Recommendation:

1.    Maintain the focus on specific objectives and areas where the project provided added value

Management Response: [Added: 2019/01/13]

The New Project Document, on the basis of the outputs of SFDA Strategic Plans and previous UNDP Projects, as well as considering the UNDP and Saudi strategy and policy documents the new project, among other specific objectives of that, will work towards further building capacities in all relevant areas as per SFDA mandate

Key Actions:

Key Action Responsible DueDate Status Comments Documents
• Review the key functional areas of SFDA addressed by the objectives; • Make decision on completing / continuing the activities as required according to the current state of affairs; • If further actions are needed, ensure that the activities shall be completed in accordance with the work plan of the new project.
[Added: 2019/01/13]
Project board 2019/07 Overdue-Not Initiated
2. Recommendation:

2.    Maintain and strengthen links between the new project and the national policies (e.g. NTP2020, MoH Health Strategy) to maximize impact

Management Response: [Added: 2019/01/13] [Last Updated: 2019/01/13]

SFDA can (pro)actively take part in the activities of the Gulf Health Council to achieve its main objectives, such as those addressing the utilization of international experiences and strengthening collaboration with Arab, regional and international organizations working in the health field; the implementation of GCC Central Drug Registration Program for pharmaceutical companies to provide safe, effective and high-quality medicine  through a unified procurement program for medicines and medical equipment and pricing of its products; and conducting joint health research among Member States.

Key Actions:

Key Action Responsible DueDate Status Comments Documents
• Review the proposed objectives of the new project whether they are aligned with the national strategy and policy; • Adapt / finetune the new project as required to maximise impact.
[Added: 2019/01/13]
UNDP and SFDA project coordinator 2019/07 Overdue-Not Initiated
3. Recommendation:

3.    Support the project management and coordination with UNDP

Management Response: [Added: 2019/01/13] [Last Updated: 2019/01/13]

The Project will be administered jointly by SFDA as Beneficiary and UNDP Country Office in Riyadh. A Chief Technical Advisor / Team Leader will be appointed to manage the project as well as oversee all technical project activities, monitor progress and report to UNDP via progress reports. The National Project Manager will be appointed to run the Project on a day-to-day basis within the constraints laid down by the Project Board. The Project Board is a group responsible for making on consensus basis management decisions for the Project when guidance is required, including recommendation for approval of project revisions. In addition, internal coordination and monitoring bodies will be established including the necessary working groups and the committees, as needed.

Key Actions:

Key Action Responsible DueDate Status Comments Documents
• Establish the Project Board and approve its operational protocol; • Appoint project management team (i.e. Chief Technical Advisor / Team Leader and Project Manager); • Create communication channels and lines between project partners and define and schedule project management meetings
[Added: 2019/01/13]
UNDP and Project coordinator 2019/07 Overdue-Not Initiated
4. Recommendation:

4. Maintain the monitoring of the progress of the new project according to the programmatic and action specific indicators set in the Logical Framework.

Management Response: [Added: 2019/01/13] [Last Updated: 2019/01/13]

The Project will be monitored in accordance with UNDP Programme and Operations Policies and Procedures (POPP) outlined in the UNDP User Guide.

The progress of the project will be monitored against the detailed work plan. 

 

Key Actions:

Key Action Responsible DueDate Status Comments Documents
Quality and timeliness of the deliverables; • Assessment of progress giving details of any divergences from the work plan; • Use of funds and resources compared to the human and financial resource plan; • The adherence to the profile requirements for staff and performance of experts; • Speed and agility of responding to requests; • Adherence to deadlines; • Quality of the staff follow-up and service/contract management.
[Added: 2019/01/13]
Project Manager and UNDP 2020/12 Not Initiated
Monitoring mechanisms which are foreseen for this project include: • Objectively Verifiable Indicators; • Monitoring through regular progress reports and Project Board meetings;
[Added: 2019/01/13]
Project Manager 2020/12 Not Initiated
5. Recommendation:

Design and propose a new project to further support SFDA implementing its Third Strategic Plan and building institutional and individual capabilities

Management Response: [Added: 2019/01/13] [Last Updated: 2019/01/13]

UNDP will work with SFDA to ensure new Project Document is aligned with Saudi Vision 2030, NTP 202 as well as SGDs in full accordanc ewiht SFDA mandate and strategy

Key Actions:

Key Action Responsible DueDate Status Comments Documents
Design a new project to further support SFDA implementing its Third Strategic Plan and building institutional and individual capabilities; • Prepare logical framework in accordance with the most effective and feasible manner of implementation of the theory of change; • Prepare a detailed work plan for the new project and define and schedule activities according to the expected results. The work plan should include the progress reports, too.
[Added: 2019/01/13]
SFDA/UNDP 2019/07 Overdue-Not Initiated
6. Recommendation:

6.    Maintain the implementation of the activities of the new project in accordance with the plan of action.

Management Response: [Added: 2019/01/13] [Last Updated: 2019/01/13]

The project shall be implemented in accordance with the work plan. In addition, the project management will ensure that the project progress is regularly monitored in accordance with the project monitoring plan.  

 

Key Actions:

Key Action Responsible DueDate Status Comments Documents
Prepare a detailed work plan for the new project and define and schedule activities according to the expected results. The work plan to include regular progress reporting. And Ensure continuous and timely monitoring during the course of the project.
[Added: 2019/01/13]
Project Manager 2020/12 Not Initiated
7. Recommendation:

7. Maintain the monitoring of the progress of the new project according to the programmatic and action specific indicators set in the Logical Framework

Management Response: [Added: 2019/01/13] [Last Updated: 2019/01/13]

The main objective of monitoring is to make sure that the right products are delivered at the right time and within the given budget, to ensure that planned quality is achieved, to correctly direct and carry out planned activities, and to properly direct and utilize resources.

The Project will be monitored in accordance with UNDP Programme and Operations Policies and Procedures (POPP) outlined in the UNDP User Guide.

The progress of the project will be monitored against the detailed work plan

   Quality and timeliness of the deliverables;
•         Assessment of progress giving details of any divergences from the work plan;
•         Use of funds and resources compared to the human and financial resource plan;
•         The adherence to the profile requirements for staff and performance of experts;
•         Speed and agility of responding to requests;
•         Adherence to deadlines;
•         Quality of the staff follow-up and service/contract management.
Monitoring mechanisms which are foreseen for this project include:
•         Objectively Verifiable Indicators;
•         Monitoring through regular progress reports and Project Board meetings;
•         User reviews – receive Client’s approval on all deliverables. Only approved deliverables can be used as input to a further stage in the project except the opposite is explicitly agreed.

 

   

Key Actions:

Key Action Responsible DueDate Status Comments Documents
• Prepare and approve project monitoring plan; • Ensure continuous and timely monitoring during the course of the project; • Require, discuss and approve monitoring reports regularly.
[Added: 2019/01/13]
UNDP/Project Manager 2020/12 Not Initiated

Latest Evaluations

Contact us

220 East 42nd Street
20th Floor
New York, NY 10017
Tel. +1 646 781 4200
Fax. +1 646 781 4213
erc.support@undp.org