Evaluation of Primary Health Care Initiative Project

Report Cover Image
Evaluation Plan:
2017-2021, Trinidad and Tobago
Evaluation Type:
Final Project
Planned End Date:
06/2017
Completion Date:
05/2017
Status:
Completed
Management Response:
Yes
Evaluation Budget(US $):
30,000

Share

Document Type Language Size Status Downloads
Download document Final Report Impact Evaluation of the Primary Health Care Initiative Project.pdf report English 1545.33 KB Posted 207
Title Evaluation of Primary Health Care Initiative Project
Atlas Project Number: 81450
Evaluation Plan: 2017-2021, Trinidad and Tobago
Evaluation Type: Final Project
Status: Completed
Completion Date: 05/2017
Planned End Date: 06/2017
Management Response: Yes
Focus Area:
  • 1. Poverty and MDG
  • 2. Others
Corporate Outcome and Output (UNDP Strategic Plan 2014-2017)
  • 1. Output 3.2. Functions, financing and capacity of sub-national level institutions enabled to deliver improved basic services and respond to priorities voiced by the public
Evaluation Budget(US $): 30,000
Source of Funding: Project budget
Evaluation Expenditure(US $): 30,000
Joint Programme: No
Mandatory Evaluation: No
Joint Evaluation: No
Evaluation Team members:
Name Title Email Nationality
Alexa Khan Ms. alexak@hotmail.com
GEF Evaluation: No
Key Stakeholders: Ministry of Health; Medical Board of Trinidad & Tobago; Regional Health Authorities
Countries: TRINIDAD AND TOBAGO
Comments:

Mid-term project evaluation

Lessons
1.

Key Lessons learned from the evaluation are as follows:

  1. Any future iteration of the project must be conceptualized to ensure that the project's design isbased on an assessment of the capacity gaps in Primary care, including the assessment of the specific areas experiencing effective service delivery.
  2. Data collection strategies to monitor the project's contribution to key results as per Government policy must be developed. An Issue Log, Risk Log and a Lessons Learned Log should be established and should be updated quarterly. A monitoring framework should also be established whereby project progress reports are submitted on a regular basis. To this end it is deemed necessary that a project manager should be recruited to ensure that project implementation and administration are efficiently conducted.
  3. More coordination among the stakeholder were required in odred to ensure efficient project  implementation. The lack of coordination between the Ministry of Health (MoH) and the Regional Health Authorities (RHAs) led to delays in the in the assigning of UNV doctors. Furthermore, there were no criteria for the identification and selection of these health centres. This resulted in UN Volunteers being assigned to facilities that would not necessarily be described as high priority. Improved coordination between stakeholders should also result in the reduction in the time taken to secure work permits and medical licences for the UNV doctors.
  4. Need for an orientation of the UNV doctors on various aspects of the health system in Trinidad and Tobago including policies, procedures and protocols is deemed essential for the proper acclimatization of the UNV doctors into the local health care system.

Findings
1.

The key findings as based on the evaluation report are as follows:

Finding 1: The project was relevant to the Government of the Republic of Trinidad and Tobago (GORTT) strategic objective of providing safe, quality health services that are patient centered.

Finding 2: Data supporting the extent of human resources shortages in the Health Sector was not included in the project rationale; as such it is difficult to quantify the extent to which there was a ‘shortage’ of Primary Care Physicians (PCPs). Despite this, informal feedback indicated that primary care service delivery was severely compromised, particularly to rural health centres, due to local doctors’ reluctance to provide services in these underserved areas.

Finding 3: Overall, the project’s design elements were insufficient to achieve stated project objectives related to more effective and efficient health services. Several of the underlying assumptions did not hold true and key elements were therefore not considered.

Finding 4: The overall project design and its related objectives may have underestimated the implications of structural dysfunctions in the health sector, including the under- performance related to Doctors simultaneously practicing in the public and private sectors.

Finding 5: The UNV Doctors have made a significant contribution to patient centric care in the primary health system. Informal and formal feedback supports the high regard for the bedside manner and communication skills of UNV Doctors.

Finding 6: UNV Doctors have contributed to filling the gap related to Health Centres that did not have daily availability of a General Practitioners and/or the required allocation of Doctors. These included Cedros, Moruga, Tabaquite and Indian Walk, among others.

Finding 7: Overall, the project delivered on its primary output regarding the selection and recruitment of 26 UNV Doctors. The evaluation suggested that improved geographical access was evident in rural Health Centres such as Cedros and Tabaquite due to the assignment of UNV Doctors. However, the extent to which these benefits are sustainable is questionable, given that the South West Regional Health Authority (SWRHA) continues to experience challenges related to allocating health sector personnel to these rural centres.


Recommendations
1

Recommendation 1: A comprehensive assessment of the Primary care health system including the critical gaps in capacity should be the basis of any further UNDP/UNV intervention.

2

Recommendation 2: Any future iteration of the UNDP/UN Volunteer programme in health, particularly one involving the assignment of Doctors and Nurses must take into account several critical success factors as well as recognize the limitations of interventions in the health system.

3

Recommendation 3: A more rigorous results matrix should be developed.

4

Recommendation 4: Need to ensure that key stakeholders, particularly the executing agency and its personnel understand the overall project design, how elements are linked and the required sequence.

5

Recommendation 5: Proposals such as this need to ensure that complementary interventions are critical to achieving stated programme outcome.

6

Recommendation 6: Clearly articulated roles and responsibilities need to be documented in any project agreement, including expectations of performance.

7

Recommendation 7: The Ministry of Health needs to conduct a performance assessment of the primary health care system to identify obstacles to improved waiting times and integrated health care, among other things.

8

Recommendation: 8: The UNDP/UNV needs to develop indicators related to the organization’s mandate including the promotion of volunteerism.

9

Recommendation 9: Project Governance arrangements must be closely monitored to ensure the adequacy of oversight. The establishment of a Project Board or Steering Committee is vital to more effective project management

10

Recommendation 10: The establishment of a National UN volunteer as project manager within the Ministry of Health may contribute to more effective and efficient project implementation

11

Recommendation 11: Monitoring and Evaluation requirements must be adhered to in any future programmes.

1. Recommendation:

Recommendation 1: A comprehensive assessment of the Primary care health system including the critical gaps in capacity should be the basis of any further UNDP/UNV intervention.

Management Response: [Added: 2017/10/10] [Last Updated: 2017/10/23]

In future phases of the Primary Health Care Initiative project gap analysis using secondary sources of information will be reviewed. This will be supported by interviews with senior management in the Ministry of Health of Trinidad and Tobago and the Regional Health Authorities (RHAs).

Key Actions:

Key Action Responsible DueDate Status Comments Documents
The existing project would be coming to an end in December of 2017. The country office is currently in negotiations with the Ministry of Health to develop a new initiative. The recommendations and management responses are being taken into consideration in the development of this new project. A gap analysis for this new project has been recommended.
[Added: 2017/10/23] [Last Updated: 2018/04/27]
UNDP Country Office in collaboration with the Ministry of Health 2017/12 Completed The UNDP CO and the Ministry of Health (MoH) have agreed upon a new phase for the Primary Health Care Initiative project. This phase seeks to achieve the following objectives: 1. Improvement of the governance capacity of the Boards of Regional Health Authorities (RHAs) to have a positive impact on healthcare service delivery. 2. Upgrading of the governance, compliance, advisory and communications skills of Corporate Secretaries of RHAs. 3. Enhancement of the leadership skills of Chief Executive Officers (CEOs) and Executive Management of RHAs to have an immediate impact on healthcare service delivery. 4. Strengthening of the capacity of RHAs and the MOH to implement the new procurement system. The decision to redirect the project was made with the agreement of senior management in the Ministry of Health of Trinidad and Tobago and the Regional Health Authorities (RHAs). History
2. Recommendation:

Recommendation 2: Any future iteration of the UNDP/UN Volunteer programme in health, particularly one involving the assignment of Doctors and Nurses must take into account several critical success factors as well as recognize the limitations of interventions in the health system.

Management Response: [Added: 2017/10/10] [Last Updated: 2017/10/23]

Management has determined that any future iteration of the projects need to take into account the following:
a) The need to involve registration or regulatory agencies during project development. Specifically, the Medical Board and the Nursing council. This will facilitate discussion of issues that may hinder implementation including licensing non-nationals to practice in the primary health system. If the issue is related to the supervision of non-nationals, a Senior Medical Professional could be assigned to perform this function.
b) The selection and recruitment process should include at a minimum, criteria related to the MBTT guidelines for full registration status;
c) Involvement of the RHAs at the project development stage and the identification of focal points to support project administration and performance monitoring;

Key Actions:

Key Action Responsible DueDate Status Comments Documents
The existing project would be coming to an end in December of 2017. The country office is currently in negotiations with the Ministry of Health to develop a new initiative. The recommendations and management responses are being taken into consideration in the development of this new project. Currently the UNDP/UNV CO has only engaged the Ministry of Health (MoH). However a request from UNDP/UNV to MoH asking for an introductory meeting with other relevant stakeholders has been made.
[Added: 2017/10/23] [Last Updated: 2018/04/27]
UNDP/UNV Country Office 2017/12 Completed The UNDP CO and the Ministry of Health (MoH) have agreed upon a new phase for the Primary Health Care Initiative project. This phase seeks to achieve the following objectives: 1. Improvement of the governance capacity of the Boards of Regional Health Authorities (RHAs) to have a positive impact on healthcare service delivery. 2. Upgrading of the governance, compliance, advisory and communications skills of Corporate Secretaries of RHAs. 3. Enhancement of the leadership skills of Chief Executive Officers (CEOs) and Executive Management of RHAs to have an immediate impact on healthcare service delivery. 4. Strengthening of the capacity of RHAs and the MOH to implement the new procurement system. The decision to redirect the project was made with the agreement of senior management in the Ministry of Health of Trinidad and Tobago and the Regional Health Authorities (RHAs). History
3. Recommendation:

Recommendation 3: A more rigorous results matrix should be developed.

Management Response: [Added: 2017/10/10] [Last Updated: 2017/10/23]

Management agrees with the above-mentioned recommendation and has decided that with any future iteration of the project a detailed logframe will be developed in collaboration with the counterpart. This will include baselines and realistic indicators and measurable targets.

Key Actions:

Key Action Responsible DueDate Status Comments Documents
The existing project would be coming to an end in December of 2017. The country office is currently in negotiations with the Ministry of Health to develop a new initiative. The recommendations and management responses are being taken into consideration in the development of this new project. UNDP/UNV to develop an improved logframe for the new project.
[Added: 2017/10/23] [Last Updated: 2018/04/27]
UNDP/UNV Country Office 2018/02 Completed The UNDP CO and the Ministry of Health (MoH) have agreed upon a new phase for the Primary Health Care Initiative project. This phase seeks to achieve the following objectives: 1. Improvement of the governance capacity of the Boards of Regional Health Authorities (RHAs) to have a positive impact on healthcare service delivery. 2. Upgrading of the governance, compliance, advisory and communications skills of Corporate Secretaries of RHAs. 3. Enhancement of the leadership skills of Chief Executive Officers (CEOs) and Executive Management of RHAs to have an immediate impact on healthcare service delivery. 4. Strengthening of the capacity of RHAs and the MOH to implement the new procurement system. A new and improved logframe was developed for this new phase of the project which has been agreed to by senior management of MoH. History
4. Recommendation:

Recommendation 4: Need to ensure that key stakeholders, particularly the executing agency and its personnel understand the overall project design, how elements are linked and the required sequence.

Management Response: [Added: 2017/10/10] [Last Updated: 2017/10/23]

UNDP/UNV to continue working with the MInstry of Health, while ensuring that such key stakeholders as the regulatory bodies and Regional Health Authorities are informed. UNDP/UNV would also seek to ensure that relevant project information is shared among all the key stakeholders as the project progresses.

Key Actions:

Key Action Responsible DueDate Status Comments Documents
The existing project would be coming to an end in December of 2017. The country office is currently in negotiations with the Ministry of Health to develop a new initiative. The recommendations and management responses are being taken into consideration in the development of this new project. Currently the UNDP/UNV CO has only engaged the Ministry of Health (MoH). However a request from UNDP/UNV to MoH asking for an introductory meeting with other relevant stakeholders has been made. UNDP will ensure that the stakeholders invited to the project board will be representative of all key stakeholders.
[Added: 2017/10/23] [Last Updated: 2018/04/27]
UNDP/UNV country office 2017/12 Completed The UNDP CO and the Ministry of Health (MoH) have agreed upon a new phase for the Primary Health Care Initiative project. This phase seeks to achieve the following objectives: 1. Improvement of the governance capacity of the Boards of Regional Health Authorities (RHAs) to have a positive impact on healthcare service delivery. 2. Upgrading of the governance, compliance, advisory and communications skills of Corporate Secretaries of RHAs. 3. Enhancement of the leadership skills of Chief Executive Officers (CEOs) and Executive Management of RHAs to have an immediate impact on healthcare service delivery. 4. Strengthening of the capacity of RHAs and the MOH to implement the new procurement system. The decision to redirect the project was made with the agreement of key stakeholders including senior management in the Ministry of Health of Trinidad and Tobago and the Regional Health Authorities (RHAs). History
5. Recommendation:

Recommendation 5: Proposals such as this need to ensure that complementary interventions are critical to achieving stated programme outcome.

Management Response: [Added: 2017/10/10] [Last Updated: 2017/10/24]

As stated in the management response to Recommendation 3, UNDP/UNV would ensure that key stakeholders are involved and/or consulted in the design phase of the project, specifically seeking clarification on how the intended intervention complements the outputs and outcomes of ongoing programmes/projects in the Ministry of Health. Therefore indicators and outcomes developed for the project would benefit the long term objectives of the Ministry beyond the project itself.

Key Actions:

Key Action Responsible DueDate Status Comments Documents
The existing project would be coming to an end in December of 2017. The country office is currently in negotiations with the Ministry of Health to develop a new initiative. The recommendations and management responses are being taken into consideration in the development of this new project. At the project design stage the engagement of key personnel at the Ministry of Health and other key stakeholders would be sought to identify connections between the proposed project and wider objectives of the Government.The engagement and the participation of these key stakeholders at the project board would be utilized to gain insight into the wider objectives of the Ministry.
[Added: 2017/10/23] [Last Updated: 2018/04/27]
UNDP/UNV country office in collaboration with all key stakeholders 2018/02 Completed The UNDP CO and the Ministry of Health (MoH) have agreed upon a new phase for the Primary Health Care Initiative project. This phase seeks to achieve the following objectives: 1. Improvement of the governance capacity of the Boards of Regional Health Authorities (RHAs) to have a positive impact on healthcare service delivery. 2. Upgrading of the governance, compliance, advisory and communications skills of Corporate Secretaries of RHAs. 3. Enhancement of the leadership skills of Chief Executive Officers (CEOs) and Executive Management of RHAs to have an immediate impact on healthcare service delivery. 4. Strengthening of the capacity of RHAs and the MOH to implement the new procurement system. To date three (3) project board meeting have been held at which consensus regarding the new direction of the project was reached. History
6. Recommendation:

Recommendation 6: Clearly articulated roles and responsibilities need to be documented in any project agreement, including expectations of performance.

Management Response: [Added: 2017/10/10] [Last Updated: 2017/10/24]

The UNDP/UNV country office does not agree with this recommendation as stated. The Primary Health Care Initiative project document did elaborate on the roles and responsibilities of each project partner. However, the UNDP/UNV country office is unable to enforce any systems of accountability within the Ministry of Health to ensure that these roles, which were agreed to, upon the signing of the project document, are carried out.

Key Actions:

7. Recommendation:

Recommendation 7: The Ministry of Health needs to conduct a performance assessment of the primary health care system to identify obstacles to improved waiting times and integrated health care, among other things.

Management Response: [Added: 2017/10/10] [Last Updated: 2017/10/24]

In future projects the UNDP/UNV country office would insist upon a needs assessment. This needs assessment will be used to develop baselines and indicators for future projects. Mid-term and final evaluations would also highlight the status of the respective indicators (e.g. reduced waiting times).

Key Actions:

Key Action Responsible DueDate Status Comments Documents
The existing project would be coming to an end in December of 2017. The country office is currently in negotiations with the Ministry of Health to develop a new initiative. The recommendations and management responses are being taken into consideration in the development of this new project. A needs assessment will be conducted by the UNDP/UNV country office in collaboration with the Ministry of Health and the Regional Health Authorities (RHAs).
[Added: 2017/10/23] [Last Updated: 2018/04/27]
UNDP/UNV country office 2018/02 Completed A formal needs assessment in the form of a desk review and consultations with key stakeholders were conducted to determine the needs on the Ministry of Health (MoH) and the Regional Health Authorities (RHAs). The objectives as determined were agreed upon by all key stakeholders. History
8. Recommendation:

Recommendation: 8: The UNDP/UNV needs to develop indicators related to the organization’s mandate including the promotion of volunteerism.

Management Response: [Added: 2017/10/10] [Last Updated: 2017/10/24]

The UNDP/UNV country office is not in agreement with this recommendation. Volunteers are encouraged to participate in voluntary activities. However, this is optional. As such the UNDP/UNV believe that this should be tracked via their individual performance assessment conducted by the volunteer and their respective supervisors and not be part of the project document.

Key Actions:

9. Recommendation:

Recommendation 9: Project Governance arrangements must be closely monitored to ensure the adequacy of oversight. The establishment of a Project Board or Steering Committee is vital to more effective project management

Management Response: [Added: 2017/10/10] [Last Updated: 2017/10/23]

The UNDP/UNV country office is in agreement with this recommendation. We would seek to ensure that in future projects working project boards with appropriate representation from key stakeholders are formed and that project board meetings are held regularly (i.e. at least once per quarter).

Key Actions:

Key Action Responsible DueDate Status Comments Documents
The existing project would be coming to an end in December of 2017. The country office is currently in negotiations with the Ministry of Health to develop a new initiative. The recommendations and management responses are being taken into consideration in the development of this new project. For this new project the UNDP/UNV country office would insist on the configuration and activation of the project board, through continued advocacy and information sharing on the benefits of the project board. Additionally the country office would liaise with key stakeholders to ensure that the project board meetings are held at least once per quarter.
[Added: 2017/10/23] [Last Updated: 2018/04/27]
UNDP/UNV country office 2018/04 Completed The UNDP CO and the Ministry of Health (MoH) have agreed upon a new phase for the Primary Health Care Initiative project. This phase seeks to achieve the following objectives: 1. Improvement of the governance capacity of the Boards of Regional Health Authorities (RHAs) to have a positive impact on healthcare service delivery. 2. Upgrading of the governance, compliance, advisory and communications skills of Corporate Secretaries of RHAs. 3. Enhancement of the leadership skills of Chief Executive Officers (CEOs) and Executive Management of RHAs to have an immediate impact on healthcare service delivery. 4. Strengthening of the capacity of RHAs and the MOH to implement the new procurement system. Due to the fact that a new phase of the project has been initiated the project board meets regularly to discuss its progress. To date three project board meetings have been held. History
10. Recommendation:

Recommendation 10: The establishment of a National UN volunteer as project manager within the Ministry of Health may contribute to more effective and efficient project implementation

Management Response: [Added: 2017/10/10] [Last Updated: 2017/10/24]

The UNDP/UNV country office is in full agreement with this recommendation and will incorporate a project manager for all large scale projects. In the event that the coountrepart does not agree to expend funds on hiring a project manager the recommendation to engage a national UN volunteer (NUNV) focal point will be recommendedin the design and development of the project. This NUNV, like the project manager would be placed in the office of the implementing partner.

Key Actions:

Key Action Responsible DueDate Status Comments Documents
The existing project would be coming to an end in December of 2017. The country office is currently in negotiations with the Ministry of Health to develop a new initiative. The recommendations and management responses are being taken into consideration in the development of this new project. In this new project a project manager will be recruited who will sit at the Ministry of Health. Four national UN volunteers will be recruited and one will sit at each of the 4 Regional Health Authorities as project support officers.
[Added: 2017/10/23] [Last Updated: 2018/09/12]
UNDP/UNV in collaboration with the Ministry of Health 2018/06 Completed A project manager was recruited in July of 2018 for the next phase of the Primary Health Care Initiative project. History
11. Recommendation:

Recommendation 11: Monitoring and Evaluation requirements must be adhered to in any future programmes.

Management Response: [Added: 2017/10/10] [Last Updated: 2017/10/24]

The UNDP/UNV country office agrees with this recommendation. Future projects would ensure that the monitoring and evaluation plan incorporated within the project document is expanded and more detailed, including but not limited to data collection methods, the responsibilities of the diffrent project partners, and identify the key objectives to be evaluated.

Key Actions:

Key Action Responsible DueDate Status Comments Documents
The existing project would be coming to an end in December of 2017. The country office is currently in negotiations with the Ministry of Health to develop a new initiative. The recommendations and management responses are being taken into consideration in the development of this new project. An expanded and detailed monitoring and evaluation plan would be developed for this new project.
[Added: 2017/10/23] [Last Updated: 2019/01/14]
UNDP/UNV country office 2019/03 Initiated The new phase of the project has been agreed upon. The new project results and resources framework (RRF) as well as a new and expanded monitoring and evaluation plan for the this phase of the project have been drafted and are awaiting ratification from the Project Board which is scheduled to take place prior to the end of the first quarter 2019. History

Latest Evaluations

Contact us

220 East 42nd Street
20th Floor
New York, NY 10017
Tel. +1 646 781 4200
Fax. +1 646 781 4213
erc.support@undp.org