Terminal Evaluation of the Government Financing Project on Bottom Up Budgeting

Report Cover Image
Evaluation Plan:
2019-2023, Philippines
Evaluation Type:
Final Project
Planned End Date:
03/2020
Completion Date:
03/2020
Status:
Completed
Management Response:
Yes
Evaluation Budget(US $):
50,000

Share

Title Terminal Evaluation of the Government Financing Project on Bottom Up Budgeting
Atlas Project Number: 00094900
Evaluation Plan: 2019-2023, Philippines
Evaluation Type: Final Project
Status: Completed
Completion Date: 03/2020
Planned End Date: 03/2020
Management Response: Yes
UNDP Signature Solution:
  • 1. Poverty
Corporate Outcome and Output (UNDP Strategic Plan 2018-2021)
  • 1. Output 1.1.2 Marginalised groups, particularly the poor, women, people with disabilities and displaced are empowered to gain universal access to basic services and financial and non-financial assets to build productive capacities and benefit from sustainable livelihoods and jobs
  • 2. Output 1.2.1 Capacities at national and sub-national levels strengthened to promote inclusive local economic development and deliver basic services including HIV and related services
SDG Goal
  • Goal 1. End poverty in all its forms everywhere
SDG Target
  • 1.4 By 2030, ensure that all men and women, in particular the poor and the vulnerable, have equal rights to economic resources, as well as access to basic services, ownership and control over land and other forms of property, inheritance, natural resources, appropriate new technology and financial services, including microfinance
Evaluation Budget(US $): 50,000
Source of Funding: Project budget
Evaluation Expenditure(US $): 13,903
Joint Programme: No
Joint Evaluation: No
Evaluation Team members:
Name Title Email Nationality
Roberto Maria Aqrquiza Mr obbyarquiza@yahoo.com
GEF Evaluation: No
Key Stakeholders: Department of Social Welfare and Development; National Economic and Development Authority; Department of Budget and Management
Countries: PHILIPPINES
Lessons
1.

From reports generated by the BUB Project and from this evaluation, the following lessons learned were culled from new knowledge gained from circumstances which may be applicable with similar projects in the future.

1. The conduct of due diligence plays a key role for Project design developers to consider in all circumstances. An excellent understanding and appreciation of the real situation and existing issues and concerns would certainly lead to the development of a responsive package of interventions that is most effective and efficient, including human resource requirements.

2. The recognition that project proposals emanating from local government units will always carry with it some degree of political influence. The institutionalization of processes in developing project proposals reduces this level of influence. These processes may include community consultations, use of cost-benefit analysis, among others in order to prioritize and identify projects that bring the most benefit the communities.

3. Standardization and consolidation of procurement cannot be implemented on a national scale as nuances such as non-standard goods and geographic distances and accessibility have to be considered. Thus, a delineation of goods and services that can be served through both centralized and decentralized procurement methods have to be identified at the onset. Moreover, benefits from cheaper prices must be balanced with delivery cost and turn-around times, storage fees. Corollary, the procurement system has to have flexibility features to make it responsive to certain nuances, without foregoing accountability and transparency. Support services on finance and administration also play a critical role for the success of an innovative procurement system

4. Consistent fostering of communication, coordination and transparency are all equally crucial in project implementation as partnerships will always be faced with issues on sharing and delineation of roles and responsibilities.

5. Partnerships in direct implementation of Projects can become a feasible and acceptable modality and can be piloted further not only at the national level but also at the sub-national level. 6. The early adaption of monitoring and evaluation activities, especially on cross-cutting issues, can ensure that sufficient data can be captured, processed, and added to the attainment of Project results.


Findings
1.

1. The activation of regional G-hubs, as planned, proved difficult thus, was not fully realized. Instead, 5 CSOs play a key role on still on-going activities in each of 5 regions on implementing quality assurance standards, document best practices, recommend improvements in implementation, identify capacity needs, recommend strategies to sustain infrastructure projects, and conduct customer satisfaction surveys. The extent of attainment of these outputs remain undetermined; and, the cessation of UNDP regional support teams in July 2019 adds to the difficulty. The remaining regions (i.e. Central Luzon, NCR, CALABARZON, NIR, Central Visayas,) will not have the benefit from this technical assistance.

2. Another 11 CSOs were directly engaged on capacity building of LGUs; for beneficiary groups on supplemental feeding program, on Auxiliary Services (hearing impairment, provision of prosthesis) in PSB, and in the implementation of SLPs (skills training and provision of starter kits); and, on the construction of a level 2 potable water system. All these 11 CSOs were able to cover all regions.

3. The completion rates for SLP, PSB, and KC has reached 92.77%, 86.07%, and 86.88%, respectively, as of 30 October 2019. The availability of funds and the on-going status of these 3 programs render highly probable that all can approach a 100% completion rate by Project end in February 2020.

4. The monitoring of Project progress was not entirely compliant to the design of the M & E system as regular quarterly and annual progress reports were delayed, and annual project reviews were not conducted. Had quarterly reports been regularly generated on a timely manner, Project Board and management could have immediately addressed issues and concerns notably on BUB Project design limitations. The Project Board met only six (6) times during over the more than 3-year Project period, i.e. May 2016 - October 2019, and not at least once each quarter or as necessary as suggested in the ProDoc. However, in its place, a total of eight (8) technical meetings were conducted from November 2017 to May 2018 to discuss several implementation concerns.

5. The initial identified risks during the Project design phase were found to be adequate and indeed materialized. The continuing expansion of risks indicated that Project management consistently monitored the emergence of new risks with management responses to the identified risks were generally found to be sufficient in mitigating the risks. Hence, during Project implementation, management of risks was more highly responsive than proactive.

6. The OAI audit findings on the BUB Project has in fact strengthened the capacity of UNDP to formulate and implement new Projects. UNDP Management has agreed, implemented and adapted these on: Insufficient due diligence and risk assessment activities; Ineffective procurement strategy; Direct contracting of suppliers without appropriate justification; and, Single sourcing of suppliers for micro-purchases.

7. The BUB Project is found to be generally responsive and aligned with UNDP’s Country Development Program, its Strategic Plan and UNDAF. Furthermore, the design was largely anchored on the prevailing condition of DSWD BUB’s low implementation accomplishment rates during the years 2013 and 2014. And, the Project continue working towards making contribution to the SDGs.

8. The concept of ‘Project support at no extra cost to DSWD’ hinged on UNDP ability to generate accumulated funds (i.e. savings) through VAT exemption and from soliciting competitive prices of quality goods and services through consolidated procurement; and using these, UNDP then funded the programmable cost of the Project’s planned activities and interventions. Actual proportional cost of activities and interventions reached 25.5% of total Project value, higher than the ProDoc estimate of 19.4% - or, a difference of PhP17.3 Million. This resulted from: 1] The ‘out-ofscope’ activities notably the review and completion of project documentation for almost all projects and, the unexpected number of sub-projects that reached 1,183 at Project start; and, 2] Project extension of another 27 months that required extending the tenure of the Project staff at the home office and the regional teams.

9. Sustainability of results include: signing of the Mutual Partnership Agreement (MPA) between LGU representatives and DSWD Regional office to carry out O & M activities to extend the economic life of a project; referral systems for advanced optical care with regional public health centers and/or hospitals, and in the LGU for the repair, maintenance, and replacement of hearing aid, awareness sessions with LGU staff personnel, and counseling with family members of beneficiaries on use and maintenance of the device. Capacity building activities provided for LGUs, CSOs, implementers and beneficiary-communities on nutrition and feeding, initiatives in community based rehabilitation (CBR) in the provision of assistive devices, knowledge and skills on enterprise development, operations and management, enhancing self-wage and employment opportunities, monitoring system for fisher folks. On capacitating partners in the national level, UNDP was able to provide trainings in Project Management, Chartered Institute of Purchasing and Supply, Knowledge Product development and orientation on SDGs. These are proofs that sustainability in SLP and PSB projects have been enhanced, and likely to be continued during the post-Project period.

10. For the implementing partners, UNDP emphasized its commitment to strengthening low absorptive capacities of national and sub-national agencies to benefit beneficiary-communities; and, its adoption of contracting modalities to work with CSOs to ensure better project implementation – despite the Project’s design limitations. For DSWD, the presence of functioning DSWD field offices and their long established coordination mechanisms with their counterpart LGU-P/C/MSWDOs and their previous experience in implementing BUB prior to the UNDP-DSWD BUB partnership provided valuable Project head starts on the finalization of project proposals, on the already established relationships with communities because of other previous social programs, e.g. 4Ps, feeding program, conduct of community consultations, local procurement, and on familiarity with specifications of assistive devices.

11. The Project was able to successfully provide gender disaggregated data on beneficiaries and further classified these as children, senior citizens, PWDs, beneficiaries and participants. Various capacity building activities were conducted on sustainable livelihood, orientation on gender sensitivity and the Magna Carta for Women, case management of violence against women and children (VAWC) and children in conflict with the law (CICL).


Recommendations
1

The Project developer relied heavily on information gathered at the national and less from the sub-national level.   Hence, the first month of project implementation should include extensive scoping and validation of the ProDoc, with emphasis on operability of the results and resources framework; the feasibility of implementing activities with utmost consideration on timeframe and adequacy of fund resources; availability and commitments of key stakeholders, and organizational structure and complementation, among all other strategies.  The initial scoping and validation of BUB Project could have concluded and determined that the original 18 months Project period indeed maybe too short a period of time to attain intended results. 

Hence, a Project Inception Report can then be presented as one important agenda item during the first Project Board meeting.  The BUB Project did not undertake scoping and validation thus, never had the opportunity early on to assess the overall Project design and discover its limitations.  

2

The DIM provides for fast-tracking of contract and procurement processes.  The readiness of about 30 project proposals at Project start could have been the opportunity to pilot test the procurement system in consideration of a heterogeneous array of goods and services.  Certainly, red flags on issues on delivery and storage, specifications, contract repackaging, changes in the requirements of LGUs, turn-around times, and non-responsiveness of infrastructure contract packages would have immediately emerged, and provided Project management early indications and time to develop a more responsive procurement plan.  The plan’s consolidation and procurement process should be made on a quarterly basis to accelerate projects implementation to shorten turn-around times, thus avert further delays.

3

Engaging the G-Hubs and CSOs require the conduct of a stakeholders’ analysis during the scoping and validation period, as they are to perform a critical role and wide ranging responsibilities in accelerating implementation.  The analysis can provide a planning tool for engaging the services of functioning and available stakeholders both at the national and sub-national levels.

4

The development of Project manuals is an important Project activity and output.  These are the Implementation Manual and the Construction, Supervision and Contract Management Manual.  Moreover, as part of the M & E Plan, a Project Monitoring-Data Collection manual at the beneficiary level could have been developed for the conduct of a prospective surveys on satisfaction, and on changes on the lives of beneficiaries of sustainable livelihood programs and of assistive devices.  The Project could have opted to nevertheless proceed with its development and publication as future reference materials by DSWD and other national agencies

5

The latest progress report cited the need to strengthen documentation of experiences, lessons learned, and mitigation measures during project implementation.  The regular generation of quarterly reports as one of the M & E outputs should be given equal importance on cross-cutting issues, i.e. gender equality, human rights, etc., updating of risks and assumptions, monitoring or Project progress towards attaining results.  All these are critical inputs for decision making.  In all future Programmes and Projects, especially using DIM, the critical importance of timely generation of requisite reports, e.g. regular quarterly and annual progress reports and project implementation reviews, can never be downplayed

6

Oversight functions are best effective whenever regular policy and operational reviews are conducted with and at the Board level.  To enhance oversight, one agenda item is the presentation by project management of the latest quarterly and/or annual reports, and an annual Project Board review.  This certainly will lead to the exercise of better adaptive management through the timely and immediate resolution of issues and concerns.  As mentioned in the ProDoc, regular feedback will enable Project management to continuously learn lessons and modify approaches and strategies, and overcome challenges and exploit opportunities

1. Recommendation:

The Project developer relied heavily on information gathered at the national and less from the sub-national level.   Hence, the first month of project implementation should include extensive scoping and validation of the ProDoc, with emphasis on operability of the results and resources framework; the feasibility of implementing activities with utmost consideration on timeframe and adequacy of fund resources; availability and commitments of key stakeholders, and organizational structure and complementation, among all other strategies.  The initial scoping and validation of BUB Project could have concluded and determined that the original 18 months Project period indeed maybe too short a period of time to attain intended results. 

Hence, a Project Inception Report can then be presented as one important agenda item during the first Project Board meeting.  The BUB Project did not undertake scoping and validation thus, never had the opportunity early on to assess the overall Project design and discover its limitations.  

Management Response: [Added: 2020/04/03]

UNDP acknowledges the recommendation of the evaluator on the terminal evaluation of the project for the need to conduct scoping and validation at the onset of the project. UNDP further notes how crucial it is to have a well-informed scoping and validation exercise. UNDP will reflect and take necessary action if this project is up-scaled ensuring the new project design is based on a thorough and rigorous scoping and validation exercise.

Key Actions:

Key Action Responsible DueDate Status Comments Documents
No Action is required since the project will be closed after this terminal evaluation
[Added: 2020/04/03]
Not Applicable 2020/03 No Longer Applicable [Justification: No Action is required since the project will be closed after this terminal evaluation. UNDP will reflect and take necessary action if this project is up-scaled ensuring the new project design is based on a thorough and rigorous scoping and validation exercise.]
2. Recommendation:

The DIM provides for fast-tracking of contract and procurement processes.  The readiness of about 30 project proposals at Project start could have been the opportunity to pilot test the procurement system in consideration of a heterogeneous array of goods and services.  Certainly, red flags on issues on delivery and storage, specifications, contract repackaging, changes in the requirements of LGUs, turn-around times, and non-responsiveness of infrastructure contract packages would have immediately emerged, and provided Project management early indications and time to develop a more responsive procurement plan.  The plan’s consolidation and procurement process should be made on a quarterly basis to accelerate projects implementation to shorten turn-around times, thus avert further delays.

Management Response: [Added: 2020/04/03]

UNDP acknowledges the recommendation of the evaluator on the terminal evaluation of the project highlighting importance of a consolidated and updated procurement plan. There was an initial procurement plan created but due to continuous change in requirements of the LGUs, the project team encountered difficulty in tracking and monitoring the plan.

UNDP will reflect and take necessary action if this project is up-scaled ensuring a thorough review of the Procurement related activities to be conducted as part of new project design

Key Actions:

Key Action Responsible DueDate Status Comments Documents
No Action is required because project will be closed after this terminal evaluation.
[Added: 2020/04/03]
Not Applicable 2020/03 No Longer Applicable [Justification: UNDP will reflect and take necessary action if this project is scaled-up ensuring a thorough review of the Procurement related activities to be conducted as part of new project design.]
3. Recommendation:

Engaging the G-Hubs and CSOs require the conduct of a stakeholders’ analysis during the scoping and validation period, as they are to perform a critical role and wide ranging responsibilities in accelerating implementation.  The analysis can provide a planning tool for engaging the services of functioning and available stakeholders both at the national and sub-national levels.

Management Response: [Added: 2020/04/03]

UNDP acknowledges the recommendation of the evaluator on the terminal evaluation of the project highlighting the need for a thorough stakeholders’ analysis at the project design stage. UNDP will ensure that future projects will carry out a stakeholders’ analysis prior to project implementation.

Key Actions:

Key Action Responsible DueDate Status Comments Documents
No Action is required because project will be closed after this terminal evaluation.
[Added: 2020/04/03]
Not Applicable 2020/03 No Longer Applicable [Justification: If this project is scaled-up, UNDP will ensure that it will carry out a stakeholders’ analysis as part of project development/design stage.]
4. Recommendation:

The development of Project manuals is an important Project activity and output.  These are the Implementation Manual and the Construction, Supervision and Contract Management Manual.  Moreover, as part of the M & E Plan, a Project Monitoring-Data Collection manual at the beneficiary level could have been developed for the conduct of a prospective surveys on satisfaction, and on changes on the lives of beneficiaries of sustainable livelihood programs and of assistive devices.  The Project could have opted to nevertheless proceed with its development and publication as future reference materials by DSWD and other national agencies

Management Response: [Added: 2020/04/03]

UNDP acknowledges the observation and recommendation of the evaluator. Development of a specific manual on project monitoring data and collection was not carried out but UNDP confirms that beneficiary data was gathered as part of the M&E Form periodically generated to capture available info from the LGUs.  Prospective surveys were also conducted. The progress of the project based on the monitoring data was also reported to the Project board through Annual Progress Reports. UNDP also tracked the quality of monitoring and evaluation data through its Project Quality Assurance checklist in 2019 and reported to the Board for considerations.

Key Actions:

Key Action Responsible DueDate Status Comments Documents
No Action is required because project will be closed after this terminal evaluation.
[Added: 2020/04/03]
Not Applicable 2020/03 No Longer Applicable [Justification: If the project is scaled-up, UNDP will ensure Project Monitoring-Data Collection manual at the beneficiary level will be developed ]
5. Recommendation:

The latest progress report cited the need to strengthen documentation of experiences, lessons learned, and mitigation measures during project implementation.  The regular generation of quarterly reports as one of the M & E outputs should be given equal importance on cross-cutting issues, i.e. gender equality, human rights, etc., updating of risks and assumptions, monitoring or Project progress towards attaining results.  All these are critical inputs for decision making.  In all future Programmes and Projects, especially using DIM, the critical importance of timely generation of requisite reports, e.g. regular quarterly and annual progress reports and project implementation reviews, can never be downplayed

Management Response: [Added: 2020/04/03]

UNDP agrees with the recommendation of the evaluator. There is a need to strengthen documentation of experiences and lessons learned. UNDP also acknowledges the need for timely development of progress reports to inform programmatic decisions that need to be taken.

Key Actions:

Key Action Responsible DueDate Status Comments Documents
Final Project Report highlighting documentation of good practices and lessons learned from the project is in the process of compilation.
[Added: 2020/04/03] [Last Updated: 2020/10/04]
Project Manager 2020/05 Completed The project's terminal evaluation has been completed, which informed the project final report. History
6. Recommendation:

Oversight functions are best effective whenever regular policy and operational reviews are conducted with and at the Board level.  To enhance oversight, one agenda item is the presentation by project management of the latest quarterly and/or annual reports, and an annual Project Board review.  This certainly will lead to the exercise of better adaptive management through the timely and immediate resolution of issues and concerns.  As mentioned in the ProDoc, regular feedback will enable Project management to continuously learn lessons and modify approaches and strategies, and overcome challenges and exploit opportunities

Management Response: [Added: 2020/04/03]

UNDP acknowledges the recommendation of the evaluator. Regular feedback is key in ensuring sound management decisions and informed key actions. The progress of the project based on the monitoring data was also reported to the Project board through Annual Progress Reports. UNDP also tracked the quality of monitoring and evaluation data through its Project Quality Assurance checklist in 2019 and reported to the Board for considerations.

Key Actions:

Key Action Responsible DueDate Status Comments Documents
No Action is required because project will be closed after this terminal evaluation
[Added: 2020/04/03]
Not Applicable 2020/03 No Longer Applicable [Justification: If this project is scaled-up, UNDP will further enhance oversight functions and report/seek guidance from Project board.]

Latest Evaluations

Contact us

1 UN Plaza
DC1-20th Floor
New York, NY 10017
Tel. +1 646 781 4200
Fax. +1 646 781 4213
erc.support@undp.org