Evaluation of Primary Health Care Initiative Project

Report Cover Image
Evaluation Plan:
2017-2021, Trinidad and Tobago
Evaluation Type:
Final Project
Planned End Date:
06/2017
Completion Date:
05/2017
Status:
Completed
Management Response:
Yes
Evaluation Budget(US $):
30,000

Share

Document Type Language Size Status Downloads
Download document Final Report Impact Evaluation of the Primary Health Care Initiative Project.pdf report English 1545.33 KB Posted 3004
Title Evaluation of Primary Health Care Initiative Project
Atlas Project Number: 00081450
Evaluation Plan: 2017-2021, Trinidad and Tobago
Evaluation Type: Final Project
Status: Completed
Completion Date: 05/2017
Planned End Date: 06/2017
Management Response: Yes
Focus Area:
  • 1. Poverty and MDG
  • 2. Others
Corporate Outcome and Output (UNDP Strategic Plan 2018-2021)
  • 1. Output 3.2. Functions, financing and capacity of sub-national level institutions enabled to deliver improved basic services and respond to priorities voiced by the public
SDG Goal
  • Goal 1. End poverty in all its forms everywhere
SDG Target
  • 1.3 Implement nationally appropriate social protection systems and measures for all, including floors, and by 2030 achieve substantial coverage of the poor and the vulnerable
Evaluation Budget(US $): 30,000
Source of Funding: Project budget
Evaluation Expenditure(US $): 30,000
Joint Programme: No
Joint Evaluation: No
Evaluation Team members:
Name Title Nationality
Alexa Khan Ms.
GEF Evaluation: No
Key Stakeholders: Ministry of Health; Medical Board of Trinidad & Tobago; Regional Health Authorities
Countries: TRINIDAD AND TOBAGO
Comments:

Mid-term project evaluation

Lessons
1.

Key Lessons learned from the evaluation are as follows:

  1. Any future iteration of the project must be conceptualized to ensure that the project's design isbased on an assessment of the capacity gaps in Primary care, including the assessment of the specific areas experiencing effective service delivery.
  2. Data collection strategies to monitor the project's contribution to key results as per Government policy must be developed. An Issue Log, Risk Log and a Lessons Learned Log should be established and should be updated quarterly. A monitoring framework should also be established whereby project progress reports are submitted on a regular basis. To this end it is deemed necessary that a project manager should be recruited to ensure that project implementation and administration are efficiently conducted.
  3. More coordination among the stakeholder were required in odred to ensure efficient project  implementation. The lack of coordination between the Ministry of Health (MoH) and the Regional Health Authorities (RHAs) led to delays in the in the assigning of UNV doctors. Furthermore, there were no criteria for the identification and selection of these health centres. This resulted in UN Volunteers being assigned to facilities that would not necessarily be described as high priority. Improved coordination between stakeholders should also result in the reduction in the time taken to secure work permits and medical licences for the UNV doctors.
  4. Need for an orientation of the UNV doctors on various aspects of the health system in Trinidad and Tobago including policies, procedures and protocols is deemed essential for the proper acclimatization of the UNV doctors into the local health care system.

Findings
1.

Finding 1: The project was relevant to the GORTT strategic objective of providing safe, quality health services that are patient centered.

4.1.2 The GORTT’s Medium Term Policy Framework (MTPF) 2011-2014 revealed that one of the key strategies identified to improve the public health system was the development of an integrated Primary, Secondary and Tertiary Health Care System. Specifically, the MTPF9 included the following specific measures: • Development of a National Health Services Delivery Plan that rationalizes primary, secondary and tertiary health care services across the various Regional Health Authorities; • Extend opening hours at community health centres; • Upgrade earmarked community health centres to provide diagnostic facilities, screening programmes and multidisciplinary teams to provide holistic care as a method for reducing reliance on the general hospitals. 4.1.3 The foregoing suggests that the project, as conceptualized was generally consistent with the GORTT’s objective regarding the provision of safe quality health services. That is, for example, the project anticipated that through the provision of additional PHPs 1, health centres in underserved areas would benefit from extended opening hours and/or the provision of additional services during the 8.00 am to 4.00 pm period. The potential for the latter was due to the fact that Doctors may not have been assigned to certain rural health centres on a daily basis, for example Blanchisseuse and Cedros


Tag: Relevance National Rural Access to Medicines Health Sector Country Government

2.

Finding 2: Data supporting the extent of human resources shortages in the Health Sector was not included in the project rationale; as such it is difficult to quantify the extent to which there was a ‘shortage’ of PCPs. Despite this, informal feedback indicated that primary care service delivery was severely compromised, particularly to rural health centres, due to local doctors’ reluctance to provide services in these underserved areas.

4.1.4 It is important to note that at the time the PHI project was being developed (2011-2012) there was a shortage of Primary Care Physicians in the heath sector11. However, by the time that the project was operationalized, some two years later12 the shortage of PCPs was less severe. Data from a Joint Select Committee of Parliament (2013) indicated that the main shortages were specialists -inclusive of Registrars and Senior Primary Care Physicians (PCP II) - which accounted for almost 66% of these vacancies. Further, as data included in Exhibit 4.1 below illustrates, there were only six (6) vacancies identified for the PCP I category. It is possible that by 2014, there may have been an increased shortage; however, data related to the actual number of vacancies was not identified in the project rationale.


Tag: Rural Urban Health Crises Health Sector Programme/Project Design Technical Support

3.

4.1.8 The foregoing suggests that there is high interest and available health human resources to satisfy the demand. This includes a significant number of graduates (estimated at 177 to 200 annually) from the School of Medicine each year in addition to returning scholars. 

4.1.9 The abovementioned indicates a number of issues including, (a) that there now appears to be an oversupply of Doctors in the system rather than a shortage. Several key stakeholders noted that there were a large number of young Doctors who were unemployed, some for 18 months after graduation and including returning scholars, who the GORTT has a legal obligation to place within 3 months; (b) despite this oversupply, service provision to the rural and some urban centres is still compromised by the local system of Public Sector Doctors simultaneously providing services via private practice. This latter issue is beyond the control of this project and reflects weak performance management of local doctors. That said, effective service delivery is almost impossible unless the public/private practice is addressed.


Tag: Partnership Programme/Project Design Project and Programme management Capacity Building

4.

Finding 3: The Mid-Term evaluation report of the previous iteration of the project could have been utilized to ensure a more robust design. Information from that report indicated key areas of concern including the need for a functional Project Steering Committee, implementation of a more robust results matrix with measurable indicators; need to streamline administrative systems regarding registration of UNV doctors, approval of Work Permits and proper monitoring of staff, among others. The project design attempted to address some of these issues, however there was limited success.

4.1.11 On the 13th June 2003, the UNDP and the GORTT signed an agreement for the UNDP to support Institutional Strengthening of the Ministry of Health, through the provision of UNV Doctors to ameliorate the estimated shortage of 200 Doctors in the public health system. The intended outcomes of the programme were (i) increased equity and efficiency of delivery of public health services, (ii) promotion of the UNV volunteer ethic toward ensuring an adequate supply of health care professionals and services to address public health issues and issues causing a deficit in health care professionals.15


Tag: Health Crises Monitoring and Evaluation Partnership Programme/Project Design Project and Programme management Technical Support

5.

4.1.12 The report also includes recommendations regarding UNDP/UNV and MoH management of the project including the need for data collection and analysis to inform overall project management. The Mid-Term Evaluation of the earlier UNDP/UNV programme identified several key issues and made recommendations aimed at strengthening the overall effectiveness of that programme. A final evaluation report was not identified. As such, it is unclear as to whether the MoH or the UNDP/UNV implemented any of the recommendations. 4.1.13 The PHI (2014) included several key elements included in the report’s recommendations including the establishment of a Project Board and a Project Manager. However, neither of these was implemented. The UNDP made several attempts16 to schedule Project Board Meetings; however, attendance by the MoH could not be secured. The Ministry of Health was not in favour of paying for an external project manager but preferred to assign this role to the Director, of the International Desk.


Tag: Health Sector Monitoring and Evaluation Partnership Programme/Project Design Project and Programme management

6.

Finding 4: Overall, the project’s design elements were insufficient to achieve stated project objectives related to more effective and efficient health services. Several of the underlying assumptions did not hold true and key elements were therefore not considered.

4.1.16 Exhibit 4.1 below illustrates the PHI anticipated Theory of Change (TOC), that is, an articulation of how the programme was intended to operate. The PHI anticipated that the project inputs including coordination and collaboration among the key stakeholder groups would contribute to the completion of key activities related to selection and recruitment, facilitation of licences to practice, work permit exemptions, identification of priority health centres and assignment of UNV Doctors. It was expected that UNV’s would be allocated to primary health centres servicing the most underserved populations, specifically in rural areas and would facilitate extended opening hours and the delivery of additional services. The project also expected that a full staff complement would reduce wait times and also reduce self-referrals to the Regional and General Hospitals.


Tag: National Rural Health Sector Programme/Project Design Theory of Change Operational Services Technical Support

7.

Finding 5: The project design may have underestimated the need to include an orientation period for UNV Doctors to support their acclimatization to the nuances of the T&T culture, policies, procedures and protocols of the local health system.

4.1.20 One of the key issues that emerged during discussions with key informants as well as UNV Doctors was the need for an orientation period by the Ministry of Health of 1-2 months to acclimatize UNV Doctors to all aspects of the health system in Trinidad including policies, procedures and protocols. In addition, UNV Doctors need to be acclimatized to Trinidad’s slang and culture. The UNV facilitates a preliminary session on culture and life in Trinidad and Tobago and the MoH was expected to provide an orientation for UNV Doctors, however, this latter arrangement was not realized. The MoH indicated that due to the arrival of individual Doctors, rather than a group, one-on-one orientation was not possible. This assertion is questionable, given that orientation could have been organized by assigning UNV Doctors to observe the practices, protocols and procedures at any number of health centres while they awaited their approvals to work. The following feedback is illustrative: “ I requested an orientation about the local regulations, medical malpractice laws and the like but I was told there was none, but that the office will arrange for one, but it was many months later, and wasn’t necessary anymore as we were already working and had gotten the hang of it. I took it upon myself, and the other doctors waiting for their papers to visit one of the health centers being served by an IUNV doctor to get an idea of the working situation. I do not know how the earlier IUNV doctors coped without an orientation19 ”


Tag: Local Governance Health Sector Policies & Procedures Programme/Project Design Technical Support

8.

Finding 6: Recent data from the Health Sector Planning Division as of 2016, suggests that there are is a large number of vacancies for Primary Health Physicians specifically at the South West RHA and District Health visitors across all RHAs. Any future intervention should focus on the DHVs and other ancillary staff, and secondarily Doctors, given the challenges related to the registration of UNV Doctors to date.

4.1.22 Exhibit 4.2 details approved posts and vacancies by Regional Health Authorities based on the data received from the RHAs up to 31st December 2016. Taking into account Acting positions, the largest number of vacancies (without bodies) appears to be for PCP I (3920) and PCP II (26) in the South West RHA. In addition, there appear to be significant vacancies for District Health Visitors across each of the RHAs, particularly North West RHA (32) and Eastern RHA (17).


Tag: Health Crises Health Sector Nutrition Operational Services

9.

Finding 7: The overall project design and its related objectives may have underestimated the implications of structural dysfunctions in the health sector, including the underperformance related to Doctors simultaneously practicing in the public and private sectors.

4.1.25 As discussed at Findings 2 and 4, the performance of interventions such as the PHI are likely to be compromised by structural dysfunctions within the Health system. These include: a) The general acceptance, historically, of Doctors practicing in the public and private sector, simultaneously. Doctors, as with all employees are recruited based on terms and conditions that include working an 8 hour work day. However, as there is no legislation or regulations specifically disallowing the practice, many local Doctors also conduct a private practice. This means that they are rarely present for more than 3 to 4 hours at health centres as they must also see patients at their private practice. As such, although they are remunerated for a full (8 hour) workday, the majority of Doctors are not available at the health centres for that period. This issue is a significant contributing variable to ineffective service delivery in the health system. Further, it is essentially a performance management issue. That is, Doctors, at all levels must be held accountable for poor performance, including penalties for absences from the workplace. Unless this problem is addressed, interventions such as the PHI will have limited success;


Tag: Challenges Health Sector Partnership Programme/Project Design Country Government Technical Support

10.

Finding 8: The UNV Doctors have made a significant contribution to patient centric care in the primary health system. Informal and formal feedback supports the high regard for the bedside manner and communication skills of UNV Doctors.

4.2.2 A significant result of the PHI was the general consensus that UNV Doctors provided excellent quality care, communicated effectively with patients, spent a great deal of time with patients and carefully explained diagnoses and treatment approaches. In particular, their tremendous work ethic was commented on by every key informant interviewed. Peers and colleagues noted the willingness of UNV Doctors to work in excess of the hours required when patients needed to be seen or paperwork to be completed. This informal feedback is supported by Performance appraisal reports, wherein the majority of UNV Doctors scored above average and outstanding.


Tag: Impact Health Crises Health Sector Human and Financial resources Technical Support

11.

Finding 9: The Un Volunteer Doctors assigned to the Queens Park Counseling Centres at Port of Spain and San Fernando General Hospitals, respectively, facilitated a more human rights based approach (HRBA) to treating with persons diagnosed and living with HIV/AIDS. Doctors assigned to the aforementioned centres, noted the stigma and discrimination displayed by the medical staff and suggested the need for more education and sensitivity training of staff at the centres.

4.2.7 It is widely acknowledged that HIV-related stigma and discrimination in the health-care sector impedes access to services and impairs the quality of health-care delivery for people living with HIV and other key populations. It also undermines efforts to achieve the highest attainable standard of health for everybody.25 As such, stigmatizing and discriminatory practices/behaviour to persons living with HIV and AIDS (PLWHAs) affects access to services through for example, the denial of health care and unjust barriers to service provision, inferior quality of care and a lack of respect, abuse and other forms of mistreatment. Although the project agreement did not specify any outcomes related to improving the service provision to PLWHAs, this outcome would have been implicit in the GORTT’s strategic objective related to effective and efficient delivery of quality health care. In this regard, the UNV Doctors have made a significant contribution given their compassion and interest in the wellbeing of PLWHAs. The following testimonials are enlightening:


Tag: Impact Human rights Access to Medicines Health Sector HIV / AIDS Operational Services Technical Support

12.

Finding 10: From the perspective of the UNV program, the project has also contributed to planting the seed of volunteerism. The UNV Doctors have made a significant impact on the degree of community outreach activities within their respective communities and support provided to key partner institutions.

4.2.9 The project’s results framework does not specifically mention the ‘promotion of volunteerism’ as an objective of either the project or a measure of project success. However, the project document details the benefits of the UNV approach: “Assignments are characterized by the spirit of solidarity, cultural sensitivity and capacity building embodied in a volunteer ethic…UNV assignments strategically aim to foster programmes of community outreach, promote the volunteer ethic and train counterparts to ensure UNV inputs are sustainable in the long term”26 . 4.2.10 Further, the UNV Description of Assignment includes as an expected result: “… a final statement of achievements towards volunteerism for development during the assignment, such as reporting on the number of volunteers mobilized, activities participated in and capacities developed27”. Each of these implies that the UNDP/UNV also assesses the promotion of volunteerism as a key objective of programme effectiveness.


Tag: Regional Health Sector Nutrition Partnership Country Government Capacity Building Youth

13.

Project to assist in the instructional support and approaches of people with Autism: Autism spectrum disorder (ASD) manifests in early childhood and is characterized by qualitative abnormalities in social interactions, markedly aberrant communication skills, and restricted repetitive behaviors, interests, and activities (RRBs). In Trinidad and Tobago, there are 500 families registered with the Autistic Society of Trinidad and Tobago (ASTT), a non-governmental organization, but extrapolating international data of 1 in 68 children, there would be a few thousand individuals on the autism spectrum. In Trinidad and Tobago, there are limited facilities that cater to the needs of those living with autism spectrum disorder. The Therapeutic and Life Skills Centre, located in Point Fortin, is a centre that offers targeted education and training for those living with autism spectrum disorder. The Centre was established in November 2003 and accepted its first student in January 2004. Since then it has expanded its enrollment to more than 15 students with ages ranging from 3 years to more than 20 years of age. All of the teachers receive training in autistic care through the Autistic Society. Consultation with the head of the school revealed that there were many needs, which included a new building for the school and a bus to aid in the transportation of students. They also needed equipment to enhance the learning environment which includes benches and chairs for the younger students, sensory educational toys and a laminator machine.


Tag: National HIV / AIDS Non-Communicable Diseases Reproductive Health Education Jobs and Livelihoods

14.

Finding 11: Mixed views were expressed about the extent to which the project contributed to some transfer of knowledge and skills and learning by the UNV doctors.

4.2.13 On the one hand, several Doctors acknowledged that their overall experience has been good and they have shared their knowledge and skills and learned from their Peers and Senior Doctors. Several UNV Doctors noted that the Trinidad experience was their first as a UNV and they had learned a great deal, especially in the Accident and Emergency (A&E) Department. Further, the prevalence of chronic non-communicable diseases in T&T constituted a good learning opportunity for those UNV Doctors who had not dealt with this prevalence. “My third assignment (2014) was as a UNV Medical Doctor in Trinidad & Tobago, working as a Primary Health Care provider in the remote Tabaquite Health Centre, catering to the poor rural community. This health centre was a Chronic Disease Clinic for diabetes, hypertension, asthma, and a Mother and Child health care centre. Thanks to these volunteering experiences, I was able to provide satisfactory and quality health care services to the needy, poor and sick, especially women and children in remote areas. Volunteering gave me the strength to accept the things that I could not change in different and difficult environments, and to share my skills with colleagues and unit staff so that we, as a team, could provide better and quality health care services to the less fortunate sections of local communities, hence marching towards fulfilling the SDGs. Through these volunteer assignments I am now thoroughly experienced with diseases like malaria, HIV/AIDS, tuberculosis, diabetes, hypertension, and asthma. Volunteering gave me international exposure, with travels to different countries where I met people from different ethnic and cultural backgrounds.”(UN Volunteer)


Tag: Health Sector Knowledge management Monitoring and Evaluation Capacity Building Technical Support

15.

Finding 12: A positive unintended outcome has been the growing awareness of patients regarding the type and standard of services they should expect from GPs at the Health centre.

4.2.16 Informal feedback from the UNV Doctors indicated patients were very satisfied with the quality and care of UNV Doctors: “If we are talking about the impact of UNV Doctors, especially in the periphery, I would say we have made a big impact in terms of quality of care, the time spent with the patient, the physical examination and diagnosis…for example sometimes clients ask “what are you doing, I did not complain about that” but we try to do a general check- up…the client is used to being asked only about his specific complaint” (UN Volunteer)


Tag: Health Sector Awareness raising Technical Support

16.

Finding 13: UNV Doctors have contributed to filling the gap related to Health Centres that did not have daily availability of a General Practitioners and/or the required allocation of Doctors. These included Cedros, Moruga, Tabaquite and Indian Walk, among others.

4.2.18 The project’s rationale was predicated on an assumption that the reason for vacancies in the primary health sector was a shortage of primary health physicians. As a result of these vacancies, several health centres, particularly those in the rural areas, did not have the services of a PCP I on a daily basis. Exhibit 4.3 details a small sample of ‘rural’ Health Centres. Only two (2) of the nine (9) centres identified reflected daily attendance of a PCP, that is, Guapo and Brothers Road Tabaquite.


Tag: National Rural Health Sector Operational Services

17.

Finding 14: The weak communication between the Ministry of Health and the Regional Health Authorities resulted in a limited understanding of the PHI’s objectives. In the case of the Eastern Regional Health Authority, the most consistent demand and placement of UNV Doctors was noted, however they were not allocated to health centres as intended.

4.2.21 The lack of coordination between the MoH and RHAs contributed to delays in assigning UNV Doctors to selected centres. The assignment of nine (9) Doctors to the Eastern Regional Health Authority, with the majority providing services at the Arima Health Facility, rather than rural health centres indicated some level of misunderstanding and/or miscommunication about the intent of the programme. When asked why the ERHA was the most responsive to accepting UNV Doctors, the response was that there was always a need for good physicians and the services of UNV Doctors was not financed by the RHA’s budget.


Tag: Impact Health Sector Communication Country Government

18.

Finding 15: Overall, the project delivered on its primary output regarding the selection and recruitment of 26 UNV Doctors. The evaluation suggested that improved geographical access was evident in rural Health Centres such as Cedros and Tabaquite due to the assignment of UNV Doctors. However, the extent to which these benefits are sustainable is questionable, given that the SWRHA continues to experience challenges related to allocating health sector personnel to these rural centres.

4.3.2 According to the project document, the primary output of the project was the recruitment of 30 UNV Doctors. The baseline indicator was “pre-selected primary health care facilities identified for weak health care delivery due to shortage of health care professionals”. The anticipated target was “fifteen percent (15%) increase in utilization of primary health care facilities with extended opening hours”. As noted earlier under the Project Design section of the report, no criteria was developed for selection of the targeted Health Centres. Further, no baseline data was collected to measure the increased utilization of PHC facilities with extended opening hours. As such, a definitive conclusion on the effectiveness of the intervention cannot be made. On a positive note, the project successfully selected and recruited twenty six (26) UNV Healthcare Professionals.


Tag: Effectiveness Rural Access to Medicines Health Sector Programme/Project Design Technical Support

19.

Finding 16: Despite the objective to ensure expanded opening hours, in successive iterations of the UNV intervention, centres were opened for additional hours only in instances where two doctors may have been assigned. Further, the lack of data regarding centres with extended opening hours, since the assignment of UNV Doctors, means that a definitive conclusion cannot be made.

4.3.7 No evidence was available to assess the extent to which the project contributed to extended opening hours at targeted health centres. In fact the MoH and/or the RHAs should be able to state whether the health centres that UNV Doctors were allocated to, were able to offer extended opening hours. However, as at this evaluation, the UNV FU stated that they were not aware of any of the centres offering extended opening hours.


Tag: Effectiveness Health Sector Project and Programme management Operational Services

20.

Finding 17: Challenges related to the overall weak management of the health system have contributed to frustration and disappointment of the UNV Doctors regarding the potential for the programme in Trinidad and Tobago. At the same time, the ability to work through these challenges was gratifying in a few instances.

4.3.8 Several UNV Doctors noted their complete disappointment with the functioning of the health system in Trinidad. Issues highlighted included: • Perception of competitors: UNV Doctors revealed that they were initially perceived as threats by the peers. This made for a less than cooperative working environment. “We keep saying we are not here to take your jobs, we only want to help” 32On a positive note, the situation improved over time and the majority of UNV Doctors noted that the support of colleagues and peers has facilitated a pleasant and collaborative working environment. In fact, when asked to identify a high point of their experience, many cited the relationships developed with peers and colleagues.


Tag: Challenges Health Sector Project and Programme management Service delivery Operational Services Technical Support

21.

Perception of dysfunctions in the Health system: UNV Doctors expressed dismay regarding the apparent limited emphasis of prioritizing patients for care:

Last year I had this patient with renal problems so he was referred to Mount Hope for an echocardiogram…but Mount Hope said he could not have the echo until 2017 and the patient was so ill…I could not stomach that…I really did not understand that because that was a really urgent case…we need to prioritize patients” (UN Volunteer) “I had a patient with rheumatoid arthritis and he was given an appointment in 2019…this is a person in pain…and the way they send them to the specialist clinic they just manage them and send them back to the centre…once these patients don’t improve they don’t have follow up care at the specialist clinics…and they come back to you and sometimes there is nothing you can do for them and you can only advise them to go back to the specialist clinic as there condition needs to be managed in that setting…”(UN Volunteer)


Tag: Effectiveness Rule of law Health Sector Communication Knowledge management Policies & Procedures Quality Assurance

22.

Finding 18: The project appears to have been implemented in keeping with the budget allocation, based on informal feedback. An implementation schedule was not developed; therefore it is difficult to assess the implementation efficiency of key activities. That said, both the recruitment and placement processes were delayed by issues that the UNV/UNDP administrators had little control over.

4.4.2 The evaluation did not review or assess the project’s financial data. Feedback from the UNV FU indicated that the project did not require any additional allocation and to date, all tranches have been received by the UNDP. The absence of an implementation schedule means that a definitive assessment of the extent to which project activities were completed according to specified timelines was not feasible. That said, the project experienced a series of delays due to factors outside of the control of the UNV FU or the UNDP. These are discussed under the project administration section of the report.


Tag: Challenges Efficiency Human and Financial resources Implementation Modality

23.

Finding 19: Based on a comparison of the compensation package with the Cuban Doctors, IUNV Doctors are more cost effective and present good value for money.

4.4.3 The UNV FU notes that the recruitment of an international UN volunteer Doctor costs the Ministry of Health approximately $32,500TTD per month whereby it costs approximately $45,000TTD per month to recruit a Cuban doctor, therefore saving the government approximately $150,000TTD per doctor per year. In addition to the financial savings, UNDP/UNV organizes all the logistical aspects of a volunteer’s placement, including finding appropriate housing, processing of health insurance, allowance payments, etc. The UNDP/UNV requires support from the International Desk of the Ministry of Health regarding issuance of Work Permit Exemptions liaising with the respective health authorities regarding placements of volunteers, provision of relevant paperwork to the MBTT and support related to establishing interview panels.


Tag: Efficiency Human and Financial resources Country Government Operational Services

24.

Finding 20: The project management arrangements detailed in the project agreement were not implemented as stated. As discussed under Finding 4, a Project Board was not established and a Project Manager was not contracted. These deficiencies compromised effective and efficient implementation of the project.

4.5.1 According to the project agreement, a Project Board, comprising representatives of both the UNDP and the MoH was responsible for: • Making decisions on major project changes • Provide approval to move to different phases of the project • Recommend to the Minister and Cabinet cessation of activities or injection of additional resources • Oversee project Implementation on a quarterly basis via report/updates from project Manager • Scheduling and convening of Project Board meetings on a quarterly basis


Tag: Health Sector Human and Financial resources Operational Efficiency Project and Programme management Country Government

25.

Finding 21: The performance of the MoH was not consistent with stakeholder expectations. Although the project agreement detailed the role of each partner in project implementation, greater specificity of tasks may have contributed to more effective implementation. That said, informal feedback indicated that the tasks to be undertaken by the International Cooperation Desk of the MoH were clearly communicated to staff of the Unit.

4.5.5 Based on the Project Agreement, the MoH was responsible for: I. Obtaining and allocating resources for the project in a timely manner; II. Certification of annual expenditure reports prepared by UNDP III. Participation in Project Board Meetings IV. Participation in monitoring and evaluation of the project V. Collaborating with UNDP in drafting ToRs VI. Take responsibility for administrative arrangements within Government departments to facilitate working arrangements for project personnel


Tag: Effectiveness Health Sector Human and Financial resources Partnership Country Government

26.

4.5.7 Feedback from both the MoH and the UNDP, indicated that none of the aforementioned tasks were consistently undertaken by the MoH. As a result, the UNDP assumed responsibility for liaising with partner institutions such as the MBTT, RHAs, Ministry of National Security (Immigration) and the Ministry of Foreign Affairs. Feedback from Senior Doctors indicated complete lack of awareness of the purpose of the project and who they were supposed to liaise with regarding management of the UNV Doctors. This confusion regarding coordination and management negatively impacted overall project administration and contributed delays and serious issues, such as the registration and placement of doctors. The following is instructive:


Tag: Policies & Procedures Project and Programme management Awareness raising Operational Services

27.

Finding 22: Reversion to the Medical Board of Trinidad and Tobago (MBTT) for granting of licences to practice has constrained project implementation since the expiration of the “Panel for the issue of Special Temporary Licences” in July 2015. This was an oversight that could have been mitigated if it was noted at project conception, through either an extension of the Panel’s life or discussion with the MBTT. The MBTT requirements for issuing Temporary Licences are clearly detailed on their website, and Temporary licences do not grant permission to practice in Primary HealthCare Centres.

4.5.9 At commencement of the project, the medical registration of UNV Doctors was processed by the Ministry of Health’s Panel for the Issue of Temporary Licences, established by Act No. 7 of 2009. The Panel was instituted “to issue special temporary licences in cases of a shortage of persons available to practice medicine in the public health sector, for the recognition of diplomas granted by certain institutions and for matters related thereto”.39 The Panel’s was empowered for six (6) years from the time of Assent to the Amendment, which was July, 2009. Therefore, the Panel’s life expired in July 2015. The MBTT was the only legally constituted body to grant medical licences from that time.


Tag: Health Sector Oversight Project and Programme management Country Government Operational Services

28.

Finding 23: The project’s implementation was also delayed by the lengthy review process of the Solicitor General’s Department, the delay in receipt of the first tranche of funds from the GORTT and absence of a system for allocation of UNV Doctors to priority health centres.

4.5.13 Informal feedback indicated that the project start was delayed by one and a half years due to the lengthy review process undertaken by the Office of the Solicitor General. The Solicitor 52Impact Evaluation of the Primary Health Care Initiative | 5/11/2017 General’s department is responsible for advising on the legal aspects of Trinidad and Tobago's international relations including relations with international organizations. As such, the project agreement would have had to be vetted by the Department.


Tag: Government Cost-sharing Health Sector Implementation Modality Project and Programme management

29.

Finding 24: Cohesiveness of UNDP/UNV team and the willingness of UNDP/UNV staff to go the extra mile supported implementation of the project. 4.5.16 Informal feedback indicated that the cohesiveness of the UNDP/UNV FU team contributed to facilitating implementation of key elements of the project.

The small team of four (4) persons made significant efforts to follow up administrative arrangements that according to the project document should have been undertaken by the MoH. These included the need to liaise with state institutions and partners to ensure that: • Doctor’s medical registration with the MBTT was completed, • Coordination with the RHAs to ensure the assignment of UNV’s in a timely manner; • Facilitation of the approval of work permit exemptions through the provision of paperwork to the MoH and follow up communication to jump start the process • Monitoring and reporting on the performance of UN Volunteers The intervention of the UNV FU ensured that delays outside of their control were minimized as much as feasible.


Tag: Implementation Modality Monitoring and Evaluation Policies & Procedures Programme Synergy

30.

Finding 25: UNDP/UNV arrangements and administration requires some review and strengthening. UNV Doctors feedback indicated dissatisfaction with the settling in and management process.

4.5.17 The majority of UNV Doctors indicated that the relationship with the UNDP/UNV FU staff was good; staff was approachable and generally tried to address concerns as they arose. That said, the UNV Doctors noted several areas requiring improvement including: • The assignment of a UNV FU staff member to meet UNV Doctors on arrival at the airport to clarify any issues that may arise with Immigration; • Contracting of a driver or taxi service to transport UNV Doctors on arrival in Trinidad; • Preparation of permits and licences prior to their arrival to facilitate the start of work; • Provision of options for accommodations at reasonable costs; • Improved support to obtain medical registration, particularly compensation for unplanned internal travel; • Establishment of an orientation period at the health facility to which they will be posted; • Formal communication to the RHAs regarding their assumption of duties and their field of practice; • Allocation of Doctors according to their areas of specialization to ensure optimization of Doctors skills; • The assignment of an officer capable of assisting with personal issues including health and related issues. Key informants also noted the need for the UNDP/UNV to provide support to UNV Doctors regarding identification of proper accommodation and bureaucratic details regarding school entrance for dependents, among other things. Informal feedback from the UNV FU indicated that they continue to make an effort to provide the necessary support to UN Volunteers. including available resources such as the UNDSS approved counselors.


Tag: Health Sector Human and Financial resources Operational Efficiency Policies & Procedures Operational Services

31.

Finding 26: Despite the efforts of the UNDP/UNV FU to assume responsibilities for managing all aspects of the programme, the lack of clearly articulated responsibilities for each partner agency meant that several issues fell through the cracks.

4.5.18 Key informants noted that despite the best efforts of all involved, the programmed needed to streamline reporting and management responsibilities. It is important to note here that the management of UNV Doctors was somewhat dual track. That is, the UNDP/ UNV FU was responsible for almost all of the HR responsibilities of UNV Doctors from contracting to payment to performance monitoring. The RHA’s were responsible for day to day supervision. However, there remain gaps in the management of the programme. These include:


Tag: Communication Human and Financial resources Monitoring and Evaluation Project and Programme management Operational Services Technical Support

32.

Finding 27: There was limited adherence to the project’s monitoring framework. As such, critical information was not fed back into the project to support decision making.

As indicated at paragraph 3.6, the project’s monitoring and evaluation arrangements included: • Quarterly quality assessments monitoring progress toward completion of key results; • An Issue log to facilitate the tracking and resolution of potential problems; • Updating of the Risk Log 55Impact Evaluation of the Primary Health Care Initiative | 5/11/2017 • Project Progress Reports (PPR) generated and submitted by the Project Manager • A Project Lessons Learned Log to support ongoing learning and adaptation within the organization • A Monitoring Schedule Plan updated to track key management activities • Completion of Annual Review Reports • Annual Project Review aimed at assessment of the project during the fourth quarter of the year and completion of the Annual Work Plan (AWP) for the following year


Tag: Monitoring and Evaluation Project and Programme management Risk Management

Recommendations
1

Recommendation 1: A comprehensive assessment of the Primary care health system including the critical gaps in capacity should be the basis of any further UNDP/UNV intervention.

Finding 2 detailed the lack of data supporting the assertion that there was a shortage of doctors in Trinidad and Tobago. The issue of human resources capacity constraints should be based on a comprehensive assessment of capacity gaps at all levels of the health system. Further, such an assessment should indicate the factors that contribute to the capacity gaps, that is, if the Tertiary systems is producing x number of graduates annually and there are vacancies in the Health Centres, why are these vacancies not being filled. Further, can the system absorb the numbers of graduates being produced? Such an assessment also needs to take into account the HR needs related to new facilities and policy interventions. For example, the National Policy on NCDs and its related Strategy are likely to require community based interventions aimed at prevention of lifestyle related diseases. The need for health educators is therefore expected. The demand for District Health Visitors will also increase, as well as the personnel mentioned at paragraph 4.2.21 of this report. It is anticipated that the 10 year Health Sector Manpower Plan will be completed by April 2017 and will inform any further interventions.

2

Recommendation 2: Any future iteration of the UNDP/UN Volunteer programme in health, particularly one involving the assignment of Doctors and Nurses must take into account several critical success factors as well as recognize the limitations of interventions in the health system.

Findings 2, 4 and 7 reflected the project’s theory of change; that is, the expectations of project implementation and performance. The evaluation noted several assumptions did not hold true during implementation. In this regard, any future iteration needs to take account of the following: a) The need to involve registration or regulatory agencies during project development. Specifically, the Medical Board and the Nursing council. This will facilitate discussion of issues that may hinder implementation including licensing non-nationals to practice in the primary health system. If the issue is related to the supervision of non-nationals, a Senior Medical Professional could be assigned to perform this function. b) The selection and recruitment process should include at a minimum, criteria related to the MBTT guidelines for full registration status; c) Criteria for the identification of health centres that urgently require the support of primary health professionals. d) The provision of diagnostic equipment in health centres as stated in the GORTT policy statements; e) Involvement of the RHAs at the project development stage and the identification of focal points to support project administration and performance monitoring;

3

Recommendation 3: A more rigorous results matrix should be developed.

Results frameworks need to be clearly articulated, including the definition of measurable outcome indicators including relevant indicators, baselines data and targets as well as a clear data collection, analysis and reporting plan. In addition, projects need to incorporate from the design phase systems- including personnel - for data collection and reporting at the output and outcome level. One of the major limitations of this project evaluation was the lack of measurable outcomes, but also the poor documentation related to monitoring and reviewing project performance. Finally, project such as this one, need to identify indicators and targets that are most closely aligned with the intervention logic and can be realistically achieved. For example, enhanced patient satisfaction, additional services provided and availability of a Primary Care Physician providing services during an 8 hour work day are likely to be measurable indicators of project effectiveness.

4

Recommendation 4: Need to ensure that key stakeholders, particularly the executing agency and its personnel understand the overall project design, how elements are linked and the required sequence.

5

Recommendation 5: Proposals such as this need to ensure that complementary interventions are critical to achieving stated programme outcome.

Finding 4 indicated the extent to which the project intervention, that is, the recruitment of 26 UNV health professionals was insufficient to achieve stated outcomes related to reduced waiting times, extended opening hours, expanded service delivery and reduced self-referrals. The project design for interventions such as this one must recognize that the intervention, by itself, is unlikely to achieve intermediate outcomes as indicated by the results framework. Outcomes most closely related to the intervention should therefore be selected.

6

Recommendation 6: Clearly articulated roles and responsibilities need to be documented in any project agreement, including expectations of performance.

Findings 17, 18 and 19 also indicated the need for clearly stated roles and responsibilities of all partner agencies. While the project document articulated the general role that each partner may have had in the project, many of the elements should have been elaborated. In particular, the performance expectations for partners outside of the UNDP/UNV should be detailed. Clearly articulated MoUs between the UNDP and partner agencies may need to be implemented to support effective commitment to project implementation. Systems for accountability of project partners are critical to successful project implementation.

7

Recommendation 7: The Ministry of Health needs to conduct a performance assessment of the primary health care system to identify obstacles to improved waiting times and integrated health care, among other things.

The evaluation findings generally determined that reduced waiting time required more than the addition of Doctors. Factors related to the appointment system, allocation of files, the number of patients for specific clinics, adequacy of support staff (RNs, Nursing Assistants) etc. all affect waiting time. The need to triage patients depending on the severity of ailments also affects waiting time. In the case of the latter, the importance of triage should be explained to patients in the context of managing expectations regarding treatment. The identification of outcomes such as those included in the project agreement, suggests the need to understand all of the underlying issues that impact delivery of quality health care so that the most appropriate interventions are selected. An intervention to address re-engineering of processes at the various health facilities may support improved effectiveness and efficiency. The extent to which current processes, guidelines and protocols are consistent with globally accepted good practices should be assessed.

8

Recommendation: 8: The UNDP/UNV needs to develop indicators related to the organization’s mandate including the promotion of volunteerism.

The UNV Volunteer Agreement clearly articulates the importance of promoting the ‘volunteer ethic’. However, the extent to which this objective has been achieved has been hindered by the lack of indicators and reporting systems regarding volunteer activities. This area needs to be strengthened in any future iteration of the programme.

9

Recommendation 9: Project Governance arrangements must be closely monitored to ensure the adequacy of oversight. The establishment of a Project Board or Steering Committee is vital to more effective project management

The need for a functioning Project Board is critical, to provide oversight and support the mitigation of any issues that arise during project implementation. In the case of this intervention, such a Board should have included representatives from the respective RHAs as well as representative of the UNV Doctors so that each of the key stakeholders would be apprised of any emerging issues.

10

Recommendation 10: The establishment of a National UN volunteer as project manager within the Ministry of Health may contribute to more effective and efficient project implementation

Findings 23 and 24 detailed some of the gaps that occurred in administration, despite the efforts of the UNDP/UNV FU. The recruitment of a UNV Project Manager who could work as the focal point within the MoH but also liaise closely with the UNV FU would ensure that all stakeholders have a clear liaison on project issues. A Project Manager may also be the most effective mechanism to ensure improved communication and coordination during project implementation.

11

Recommendation 11: Monitoring and Evaluation requirements must be adhered to in any future programmes.

Finding 24 indicates the deficiencies in documenting and monitoring the project’s performance. Evaluation depends on performance data/documentary evidence of all aspects of a project in order to support findings and conclusions. Even where a project may have contributed to key objectives, without any supporting information, evaluation of the intervention is challenging. This is therefore another area that requires the allocation of dedicated resources to ensure that data is collected systematically

1. Recommendation:

Recommendation 1: A comprehensive assessment of the Primary care health system including the critical gaps in capacity should be the basis of any further UNDP/UNV intervention.

Finding 2 detailed the lack of data supporting the assertion that there was a shortage of doctors in Trinidad and Tobago. The issue of human resources capacity constraints should be based on a comprehensive assessment of capacity gaps at all levels of the health system. Further, such an assessment should indicate the factors that contribute to the capacity gaps, that is, if the Tertiary systems is producing x number of graduates annually and there are vacancies in the Health Centres, why are these vacancies not being filled. Further, can the system absorb the numbers of graduates being produced? Such an assessment also needs to take into account the HR needs related to new facilities and policy interventions. For example, the National Policy on NCDs and its related Strategy are likely to require community based interventions aimed at prevention of lifestyle related diseases. The need for health educators is therefore expected. The demand for District Health Visitors will also increase, as well as the personnel mentioned at paragraph 4.2.21 of this report. It is anticipated that the 10 year Health Sector Manpower Plan will be completed by April 2017 and will inform any further interventions.

Management Response: [Added: 2017/10/10] [Last Updated: 2021/02/05]

In future phases of the Primary Health Care Initiative project gap analysis using secondary sources of information will be reviewed. This will be supported by interviews with senior management in the Ministry of Health of Trinidad and Tobago and the Regional Health Authorities (RHAs).

Key Actions:

Key Action Responsible DueDate Status Comments Documents
The existing project would be coming to an end in December of 2017. The country office is currently in negotiations with the Ministry of Health to develop a new initiative. The recommendations and management responses are being taken into consideration in the development of this new project. A gap analysis for this new project has been recommended.
[Added: 2017/10/23] [Last Updated: 2018/04/27]
UNDP Country Office in collaboration with the Ministry of Health 2017/12 Completed The UNDP CO and the Ministry of Health (MoH) have agreed upon a new phase for the Primary Health Care Initiative project. This phase seeks to achieve the following objectives: 1. Improvement of the governance capacity of the Boards of Regional Health Authorities (RHAs) to have a positive impact on healthcare service delivery. 2. Upgrading of the governance, compliance, advisory and communications skills of Corporate Secretaries of RHAs. 3. Enhancement of the leadership skills of Chief Executive Officers (CEOs) and Executive Management of RHAs to have an immediate impact on healthcare service delivery. 4. Strengthening of the capacity of RHAs and the MOH to implement the new procurement system. The decision to redirect the project was made with the agreement of senior management in the Ministry of Health of Trinidad and Tobago and the Regional Health Authorities (RHAs). History
2. Recommendation:

Recommendation 2: Any future iteration of the UNDP/UN Volunteer programme in health, particularly one involving the assignment of Doctors and Nurses must take into account several critical success factors as well as recognize the limitations of interventions in the health system.

Findings 2, 4 and 7 reflected the project’s theory of change; that is, the expectations of project implementation and performance. The evaluation noted several assumptions did not hold true during implementation. In this regard, any future iteration needs to take account of the following: a) The need to involve registration or regulatory agencies during project development. Specifically, the Medical Board and the Nursing council. This will facilitate discussion of issues that may hinder implementation including licensing non-nationals to practice in the primary health system. If the issue is related to the supervision of non-nationals, a Senior Medical Professional could be assigned to perform this function. b) The selection and recruitment process should include at a minimum, criteria related to the MBTT guidelines for full registration status; c) Criteria for the identification of health centres that urgently require the support of primary health professionals. d) The provision of diagnostic equipment in health centres as stated in the GORTT policy statements; e) Involvement of the RHAs at the project development stage and the identification of focal points to support project administration and performance monitoring;

Management Response: [Added: 2017/10/10] [Last Updated: 2021/02/05]

Management has determined that any future iteration of the projects need to take into account the following:
a) The need to involve registration or regulatory agencies during project development. Specifically, the Medical Board and the Nursing council. This will facilitate discussion of issues that may hinder implementation including licensing non-nationals to practice in the primary health system. If the issue is related to the supervision of non-nationals, a Senior Medical Professional could be assigned to perform this function.
b) The selection and recruitment process should include at a minimum, criteria related to the MBTT guidelines for full registration status;
c) Involvement of the RHAs at the project development stage and the identification of focal points to support project administration and performance monitoring;

Key Actions:

Key Action Responsible DueDate Status Comments Documents
The existing project would be coming to an end in December of 2017. The country office is currently in negotiations with the Ministry of Health to develop a new initiative. The recommendations and management responses are being taken into consideration in the development of this new project. Currently the UNDP/UNV CO has only engaged the Ministry of Health (MoH). However a request from UNDP/UNV to MoH asking for an introductory meeting with other relevant stakeholders has been made.
[Added: 2017/10/23] [Last Updated: 2018/04/27]
UNDP/UNV Country Office 2017/12 Completed The UNDP CO and the Ministry of Health (MoH) have agreed upon a new phase for the Primary Health Care Initiative project. This phase seeks to achieve the following objectives: 1. Improvement of the governance capacity of the Boards of Regional Health Authorities (RHAs) to have a positive impact on healthcare service delivery. 2. Upgrading of the governance, compliance, advisory and communications skills of Corporate Secretaries of RHAs. 3. Enhancement of the leadership skills of Chief Executive Officers (CEOs) and Executive Management of RHAs to have an immediate impact on healthcare service delivery. 4. Strengthening of the capacity of RHAs and the MOH to implement the new procurement system. The decision to redirect the project was made with the agreement of senior management in the Ministry of Health of Trinidad and Tobago and the Regional Health Authorities (RHAs). History
3. Recommendation:

Recommendation 3: A more rigorous results matrix should be developed.

Results frameworks need to be clearly articulated, including the definition of measurable outcome indicators including relevant indicators, baselines data and targets as well as a clear data collection, analysis and reporting plan. In addition, projects need to incorporate from the design phase systems- including personnel - for data collection and reporting at the output and outcome level. One of the major limitations of this project evaluation was the lack of measurable outcomes, but also the poor documentation related to monitoring and reviewing project performance. Finally, project such as this one, need to identify indicators and targets that are most closely aligned with the intervention logic and can be realistically achieved. For example, enhanced patient satisfaction, additional services provided and availability of a Primary Care Physician providing services during an 8 hour work day are likely to be measurable indicators of project effectiveness.

Management Response: [Added: 2017/10/10] [Last Updated: 2021/02/05]

Management agrees with the above-mentioned recommendation and has decided that with any future iteration of the project a detailed logframe will be developed in collaboration with the counterpart. This will include baselines and realistic indicators and measurable targets.

Key Actions:

Key Action Responsible DueDate Status Comments Documents
The existing project would be coming to an end in December of 2017. The country office is currently in negotiations with the Ministry of Health to develop a new initiative. The recommendations and management responses are being taken into consideration in the development of this new project. UNDP/UNV to develop an improved logframe for the new project.
[Added: 2017/10/23] [Last Updated: 2018/04/27]
UNDP/UNV Country Office 2018/02 Completed The UNDP CO and the Ministry of Health (MoH) have agreed upon a new phase for the Primary Health Care Initiative project. This phase seeks to achieve the following objectives: 1. Improvement of the governance capacity of the Boards of Regional Health Authorities (RHAs) to have a positive impact on healthcare service delivery. 2. Upgrading of the governance, compliance, advisory and communications skills of Corporate Secretaries of RHAs. 3. Enhancement of the leadership skills of Chief Executive Officers (CEOs) and Executive Management of RHAs to have an immediate impact on healthcare service delivery. 4. Strengthening of the capacity of RHAs and the MOH to implement the new procurement system. A new and improved logframe was developed for this new phase of the project which has been agreed to by senior management of MoH. History
4. Recommendation:

Recommendation 4: Need to ensure that key stakeholders, particularly the executing agency and its personnel understand the overall project design, how elements are linked and the required sequence.

Management Response: [Added: 2017/10/10] [Last Updated: 2021/02/05]

UNDP/UNV to continue working with the MInstry of Health, while ensuring that such key stakeholders as the regulatory bodies and Regional Health Authorities are informed. UNDP/UNV would also seek to ensure that relevant project information is shared among all the key stakeholders as the project progresses.

Key Actions:

Key Action Responsible DueDate Status Comments Documents
The existing project would be coming to an end in December of 2017. The country office is currently in negotiations with the Ministry of Health to develop a new initiative. The recommendations and management responses are being taken into consideration in the development of this new project. Currently the UNDP/UNV CO has only engaged the Ministry of Health (MoH). However a request from UNDP/UNV to MoH asking for an introductory meeting with other relevant stakeholders has been made. UNDP will ensure that the stakeholders invited to the project board will be representative of all key stakeholders.
[Added: 2017/10/23] [Last Updated: 2018/04/27]
UNDP/UNV country office 2017/12 Completed The UNDP CO and the Ministry of Health (MoH) have agreed upon a new phase for the Primary Health Care Initiative project. This phase seeks to achieve the following objectives: 1. Improvement of the governance capacity of the Boards of Regional Health Authorities (RHAs) to have a positive impact on healthcare service delivery. 2. Upgrading of the governance, compliance, advisory and communications skills of Corporate Secretaries of RHAs. 3. Enhancement of the leadership skills of Chief Executive Officers (CEOs) and Executive Management of RHAs to have an immediate impact on healthcare service delivery. 4. Strengthening of the capacity of RHAs and the MOH to implement the new procurement system. The decision to redirect the project was made with the agreement of key stakeholders including senior management in the Ministry of Health of Trinidad and Tobago and the Regional Health Authorities (RHAs). History
5. Recommendation:

Recommendation 5: Proposals such as this need to ensure that complementary interventions are critical to achieving stated programme outcome.

Finding 4 indicated the extent to which the project intervention, that is, the recruitment of 26 UNV health professionals was insufficient to achieve stated outcomes related to reduced waiting times, extended opening hours, expanded service delivery and reduced self-referrals. The project design for interventions such as this one must recognize that the intervention, by itself, is unlikely to achieve intermediate outcomes as indicated by the results framework. Outcomes most closely related to the intervention should therefore be selected.

Management Response: [Added: 2017/10/10] [Last Updated: 2021/02/05]

As stated in the management response to Recommendation 3, UNDP/UNV would ensure that key stakeholders are involved and/or consulted in the design phase of the project, specifically seeking clarification on how the intended intervention complements the outputs and outcomes of ongoing programmes/projects in the Ministry of Health. Therefore indicators and outcomes developed for the project would benefit the long term objectives of the Ministry beyond the project itself.

Key Actions:

Key Action Responsible DueDate Status Comments Documents
The existing project would be coming to an end in December of 2017. The country office is currently in negotiations with the Ministry of Health to develop a new initiative. The recommendations and management responses are being taken into consideration in the development of this new project. At the project design stage the engagement of key personnel at the Ministry of Health and other key stakeholders would be sought to identify connections between the proposed project and wider objectives of the Government.The engagement and the participation of these key stakeholders at the project board would be utilized to gain insight into the wider objectives of the Ministry.
[Added: 2017/10/23] [Last Updated: 2018/04/27]
UNDP/UNV country office in collaboration with all key stakeholders 2018/02 Completed The UNDP CO and the Ministry of Health (MoH) have agreed upon a new phase for the Primary Health Care Initiative project. This phase seeks to achieve the following objectives: 1. Improvement of the governance capacity of the Boards of Regional Health Authorities (RHAs) to have a positive impact on healthcare service delivery. 2. Upgrading of the governance, compliance, advisory and communications skills of Corporate Secretaries of RHAs. 3. Enhancement of the leadership skills of Chief Executive Officers (CEOs) and Executive Management of RHAs to have an immediate impact on healthcare service delivery. 4. Strengthening of the capacity of RHAs and the MOH to implement the new procurement system. To date three (3) project board meeting have been held at which consensus regarding the new direction of the project was reached. History
6. Recommendation:

Recommendation 6: Clearly articulated roles and responsibilities need to be documented in any project agreement, including expectations of performance.

Findings 17, 18 and 19 also indicated the need for clearly stated roles and responsibilities of all partner agencies. While the project document articulated the general role that each partner may have had in the project, many of the elements should have been elaborated. In particular, the performance expectations for partners outside of the UNDP/UNV should be detailed. Clearly articulated MoUs between the UNDP and partner agencies may need to be implemented to support effective commitment to project implementation. Systems for accountability of project partners are critical to successful project implementation.

Management Response: [Added: 2017/10/10] [Last Updated: 2021/02/05]

The UNDP/UNV country office does not agree with this recommendation as stated. The Primary Health Care Initiative project document did elaborate on the roles and responsibilities of each project partner. However, the UNDP/UNV country office is unable to enforce any systems of accountability within the Ministry of Health to ensure that these roles, which were agreed to, upon the signing of the project document, are carried out.

Key Actions:

7. Recommendation:

Recommendation 7: The Ministry of Health needs to conduct a performance assessment of the primary health care system to identify obstacles to improved waiting times and integrated health care, among other things.

The evaluation findings generally determined that reduced waiting time required more than the addition of Doctors. Factors related to the appointment system, allocation of files, the number of patients for specific clinics, adequacy of support staff (RNs, Nursing Assistants) etc. all affect waiting time. The need to triage patients depending on the severity of ailments also affects waiting time. In the case of the latter, the importance of triage should be explained to patients in the context of managing expectations regarding treatment. The identification of outcomes such as those included in the project agreement, suggests the need to understand all of the underlying issues that impact delivery of quality health care so that the most appropriate interventions are selected. An intervention to address re-engineering of processes at the various health facilities may support improved effectiveness and efficiency. The extent to which current processes, guidelines and protocols are consistent with globally accepted good practices should be assessed.

Management Response: [Added: 2017/10/10] [Last Updated: 2021/02/05]

In future projects the UNDP/UNV country office would insist upon a needs assessment. This needs assessment will be used to develop baselines and indicators for future projects. Mid-term and final evaluations would also highlight the status of the respective indicators (e.g. reduced waiting times).

Key Actions:

Key Action Responsible DueDate Status Comments Documents
The existing project would be coming to an end in December of 2017. The country office is currently in negotiations with the Ministry of Health to develop a new initiative. The recommendations and management responses are being taken into consideration in the development of this new project. A needs assessment will be conducted by the UNDP/UNV country office in collaboration with the Ministry of Health and the Regional Health Authorities (RHAs).
[Added: 2017/10/23] [Last Updated: 2018/04/27]
UNDP/UNV country office 2018/02 Completed A formal needs assessment in the form of a desk review and consultations with key stakeholders were conducted to determine the needs on the Ministry of Health (MoH) and the Regional Health Authorities (RHAs). The objectives as determined were agreed upon by all key stakeholders. History
8. Recommendation:

Recommendation: 8: The UNDP/UNV needs to develop indicators related to the organization’s mandate including the promotion of volunteerism.

The UNV Volunteer Agreement clearly articulates the importance of promoting the ‘volunteer ethic’. However, the extent to which this objective has been achieved has been hindered by the lack of indicators and reporting systems regarding volunteer activities. This area needs to be strengthened in any future iteration of the programme.

Management Response: [Added: 2017/10/10] [Last Updated: 2021/02/05]

The UNDP/UNV country office is not in agreement with this recommendation. Volunteers are encouraged to participate in voluntary activities. However, this is optional. As such the UNDP/UNV believe that this should be tracked via their individual performance assessment conducted by the volunteer and their respective supervisors and not be part of the project document.

Key Actions:

9. Recommendation:

Recommendation 9: Project Governance arrangements must be closely monitored to ensure the adequacy of oversight. The establishment of a Project Board or Steering Committee is vital to more effective project management

The need for a functioning Project Board is critical, to provide oversight and support the mitigation of any issues that arise during project implementation. In the case of this intervention, such a Board should have included representatives from the respective RHAs as well as representative of the UNV Doctors so that each of the key stakeholders would be apprised of any emerging issues.

Management Response: [Added: 2017/10/10] [Last Updated: 2021/02/05]

The UNDP/UNV country office is in agreement with this recommendation. We would seek to ensure that in future projects working project boards with appropriate representation from key stakeholders are formed and that project board meetings are held regularly (i.e. at least once per quarter).

Key Actions:

Key Action Responsible DueDate Status Comments Documents
The existing project would be coming to an end in December of 2017. The country office is currently in negotiations with the Ministry of Health to develop a new initiative. The recommendations and management responses are being taken into consideration in the development of this new project. For this new project the UNDP/UNV country office would insist on the configuration and activation of the project board, through continued advocacy and information sharing on the benefits of the project board. Additionally the country office would liaise with key stakeholders to ensure that the project board meetings are held at least once per quarter.
[Added: 2017/10/23] [Last Updated: 2018/04/27]
UNDP/UNV country office 2018/04 Completed The UNDP CO and the Ministry of Health (MoH) have agreed upon a new phase for the Primary Health Care Initiative project. This phase seeks to achieve the following objectives: 1. Improvement of the governance capacity of the Boards of Regional Health Authorities (RHAs) to have a positive impact on healthcare service delivery. 2. Upgrading of the governance, compliance, advisory and communications skills of Corporate Secretaries of RHAs. 3. Enhancement of the leadership skills of Chief Executive Officers (CEOs) and Executive Management of RHAs to have an immediate impact on healthcare service delivery. 4. Strengthening of the capacity of RHAs and the MOH to implement the new procurement system. Due to the fact that a new phase of the project has been initiated the project board meets regularly to discuss its progress. To date three project board meetings have been held. History
10. Recommendation:

Recommendation 10: The establishment of a National UN volunteer as project manager within the Ministry of Health may contribute to more effective and efficient project implementation

Findings 23 and 24 detailed some of the gaps that occurred in administration, despite the efforts of the UNDP/UNV FU. The recruitment of a UNV Project Manager who could work as the focal point within the MoH but also liaise closely with the UNV FU would ensure that all stakeholders have a clear liaison on project issues. A Project Manager may also be the most effective mechanism to ensure improved communication and coordination during project implementation.

Management Response: [Added: 2017/10/10] [Last Updated: 2021/02/05]

The UNDP/UNV country office is in full agreement with this recommendation and will incorporate a project manager for all large scale projects. In the event that the coountrepart does not agree to expend funds on hiring a project manager the recommendation to engage a national UN volunteer (NUNV) focal point will be recommendedin the design and development of the project. This NUNV, like the project manager would be placed in the office of the implementing partner.

Key Actions:

Key Action Responsible DueDate Status Comments Documents
The existing project would be coming to an end in December of 2017. The country office is currently in negotiations with the Ministry of Health to develop a new initiative. The recommendations and management responses are being taken into consideration in the development of this new project. In this new project a project manager will be recruited who will sit at the Ministry of Health. Four national UN volunteers will be recruited and one will sit at each of the 4 Regional Health Authorities as project support officers.
[Added: 2017/10/23] [Last Updated: 2018/09/12]
UNDP/UNV in collaboration with the Ministry of Health 2018/06 Completed A project manager was recruited in July of 2018 for the next phase of the Primary Health Care Initiative project. History
11. Recommendation:

Recommendation 11: Monitoring and Evaluation requirements must be adhered to in any future programmes.

Finding 24 indicates the deficiencies in documenting and monitoring the project’s performance. Evaluation depends on performance data/documentary evidence of all aspects of a project in order to support findings and conclusions. Even where a project may have contributed to key objectives, without any supporting information, evaluation of the intervention is challenging. This is therefore another area that requires the allocation of dedicated resources to ensure that data is collected systematically

Management Response: [Added: 2017/10/10] [Last Updated: 2021/02/05]

The UNDP/UNV country office agrees with this recommendation. Future projects would ensure that the monitoring and evaluation plan incorporated within the project document is expanded and more detailed, including but not limited to data collection methods, the responsibilities of the diffrent project partners, and identify the key objectives to be evaluated.

Key Actions:

Key Action Responsible DueDate Status Comments Documents
The existing project would be coming to an end in December of 2017. The country office is currently in negotiations with the Ministry of Health to develop a new initiative. The recommendations and management responses are being taken into consideration in the development of this new project. An expanded and detailed monitoring and evaluation plan would be developed for this new project.
[Added: 2017/10/23] [Last Updated: 2019/04/04]
UNDP/UNV country office 2019/03 Completed The new project results and resources framework (RRF) and the expanded monitoring and evaluation plan for the new phase of the project have been finalized and ratified by the Ministry of Health. History

Latest Evaluations

Contact us

1 UN Plaza
DC1-20th Floor
New York, NY 10017
Tel. +1 646 781 4200
Fax. +1 646 781 4213
erc.support@undp.org