- Evaluation Plan:
- 2019-2023, Philippines
- Evaluation Type:
- Mid Term Project
- Planned End Date:
- 08/2021
- Completion Date:
- 07/2021
- Status:
- Completed
- Management Response:
- Yes
- Evaluation Budget(US $):
- 23,000
DICT-UNDP Free Wi-fi for All Project Mid-Term Evaluation
| Document | Type | Language | Size | Status | Downloads |
|---|---|---|---|---|---|
 TOR Free Wifi for All Midterm Evaluation.pdf |
tor | English | 991.37 KB | Posted | 748 |
| FWFA Project Midterm Evaluation_Final Report.pdf |
report | English | 3886.77 KB | Posted | 1414 |
| Title | DICT-UNDP Free Wi-fi for All Project Mid-Term Evaluation | ||||||
|---|---|---|---|---|---|---|---|
| Atlas Project Number: | 00113425 | ||||||
| Evaluation Plan: | 2019-2023, Philippines | ||||||
| Evaluation Type: | Mid Term Project | ||||||
| Status: | Completed | ||||||
| Completion Date: | 07/2021 | ||||||
| Planned End Date: | 08/2021 | ||||||
| Management Response: | Yes | ||||||
| Focus Area: |
|
||||||
| Corporate Outcome and Output (UNDP Strategic Plan 2018-2021) |
|
||||||
| SDG Goal |
|
||||||
| SDG Target |
|
||||||
| Evaluation Budget(US $): | 23,000 | ||||||
| Source of Funding: | Government | ||||||
| Evaluation Expenditure(US $): | 15,000 | ||||||
| Joint Programme: | No | ||||||
| Joint Evaluation: | No | ||||||
| Evaluation Team members: |
|
||||||
| GEF Evaluation: | No | ||||||
| Key Stakeholders: | |||||||
| Countries: | PHILIPPINES |
| Lessons | |
|---|---|
| 1. | At its midpoint, the DICT-UNDP Free Wi-fi for All Project offers critical lessons. The ninth and final section of this Draft Report present these lessons learned by revisiting ICT4D, the Network Effect and the last mile hypothesis. Revisiting ICT4D and Open Access. The significance, necessity and urgency of universal Internet access is a forgone conclusion (Bamford et al., 2021). The question now is how to achieve it in the most effective and efficient manner. The FWFA Project teaches us that one of the most important functions of ICTs is that they enable communities to respond to national emergencies. Hence, open access to Wi-fi and Internet resources is critical during disruptive times. The Free Wi-fi For All Project should not be undermined nor diminished but rather strengthened and fully supported by both the national government and UNDP. Revisiting the Network Effect. An often-quoted maxim in the field of ICT4D is Metcalf’s Law: resources available to stand alone systems increase mathematically with every system unit added while resources available to networked systems grow exponentially with every network node added. The FWFA contractor’s circumstances may be likened to a standalone system whose resources no matter how ample in the beginning may prove to be inadequate in disruptive times. A decentralized network of service providers may prove to be more resilient and agile in the long run. Revisiting the Last Mile Hypothesis. In 2005, I completed a Sector Study for the Assessment of the Demand for Access to Information and Communications Technology in Rural and Remote Areas of Cambodia, Lao PDR, and Viet Nam. One of my key informants was Dr. Michael Calvano of the International Telecommunications Union Regional Office in Bangkok. A year before the interview, Calvano introduced the Last Mile Hypothesis to ICT4D literature. According to this hypothesis, the best strategy for telecommunications infrastructure to reach the last mile from the backbone to the periphery is for national governments to provide basic services, i.e., education, agricultural extension, rural health, etc. to rural and remote areas through telecommunications in progressively increasing intensities (Calvano as quoted by Flor, 2005). Government intervention pump primes information and communications technology utilization. Once the government initiates this service, the private sector will follow suit paving the way for users down to the last mile. This hypothesis has been validated by the DICT-UNDP Free Wi-fi for All Project. With an additional player, the local government unit. Through the Project, stakeholders observed that LGUs show interest and willingness to invest resources and engage in free Wi-fi service delivery despite (or possibly, because of) the pandemic. Their active participation should be encouraged in the future with additional funds allotted for non-technical interventions. |
| Findings | |
| 1. | Relevance: In general, the Project’s potential contributions to gender equality, women’s empowerment and to other marginalized sectors are recognized and appreciated by target beneficiaries and stakeholders. Furthermore, there is widespread agreement that each project component directly supports the attainment of project outputs and outcomes. Additionally, online survey respondents, FGD participants and key informants share the opinion that Project interventions are relevant to the needs of the stakeholders, particularly in the current emergency. Finally, it is perceived that the Project is aligned with the Philippine government’s initiative to strengthen connectivity and the DICT’s development objectives. |
| 2. | Effectiveness: The MTRE found that the effectiveness of the implementation strategy and operations were severely affected by a confluence of factors influenced primarily by the pandemic which caused: a lack of mobility; office lockdowns; travel restrictions; heightened dependence on and increased demand for internet services. Delays in deployment were compounded by perceptions of lack of decentralization, coordination and communication. Difficulties with the contractor and its relationship with the subcontractor exacerbated the situation. Although delays were experienced, project outputs are indeed being achieved. The sequential nature of project components also contributed to these delays, particularly in Output 2. Under the original timeframe projects objectives and outputs were already difficult to achieve within the given timeframe. It also had to adjust to the annual budgeting process and other financing realities. |
| 3. | Efficiency: As in the case of effectiveness, the efficiency of the implementation strategy and operations were severely affected by a confluence of factors: the pandemic that hampered mobility, caused lockdowns and travel restrictions; perceptions of lack of decentralization, coordination, and communication; and difficulties with the contractor. The unique nature of FWFA as a national government funded internationally executed development assistance project had bearings on efficiency. However, this project execution and implementation modality is here to stay and the only way to avoid similar difficulties in the future is for both parties, i.e., the national government and the international development agency to adopt more flexible arrangements particularly during national emergencies. |
| 4. | Sustainability: Project interventions can be made more sustainable in terms of project ownership, technological appropriateness and stakeholder capacities with increased involvement of local government units and the private sector. There is little financial risk that may jeopardize the sustainability of project outputs provided that local government units and the private sector are involved. With increased demand for Wi-fi services there is little institutional, social, political and environmental risks to the sustainability of project results. Exit strategies would be more meaningful and sustainable with the involvement of local service providers. |
| Recommendations | |
|---|---|
| 1 | 1. Realistic project timeframe. A more realistic project timeframe should be designed factoring in the disruptions with attendant adjustments in implementing government and agency financial arrangements. As to the specifications of these adjustments, it would depend on flexibilities that may be accommodated by the national government (e.g., in terms of the annual budget cycle) and by UNDP (e.g., in terms of international and national bidding). |
| 2 | 2. Adopt additional technical refinements. Since the beginning of the project in September 2018 there have been improvements in VSAT models and technologies. Without dropping the SLA modality, minimum service requirements should be increased. Technologies that maximize upload/download speeds must be considered including shifting from KU to KA band technologies as stated earlier. The progressively increasing appetite for bandwidth should be planned for. |
| 3 | 3. Accommodate greater participation. The active engagement of DICT provincial engineers, LGUs and the private sector in maintenance, value-added services and technological enhancements should be encouraged and may be provided for in the SLAs. |
| 4 | 4. Future-proofing. The project should be more anticipatory and consider incorporating adaptive management in plans and strategies that would future-proof Project interventions leading to greater sustainability. These include: technological and service requirements forecasting for Outputs 1 & 3; and linking up DevLIVE training to NEDA’s Ambisyon 2040 outcomes for Output 2. |
| 5 | 5. Strengthen Internal and External Communication. It would be worthwhile investing in internal and external communication as well as social preparation with a minor rationalization of budgets to tip slightly towards non-technical interventions. |
Management Response Documents
1. Recommendation:
1. Realistic project timeframe. A more realistic project timeframe should be designed factoring in the disruptions with attendant adjustments in implementing government and agency financial arrangements. As to the specifications of these adjustments, it would depend on flexibilities that may be accommodated by the national government (e.g., in terms of the annual budget cycle) and by UNDP (e.g., in terms of international and national bidding).
Management Response: [Added: 2022/03/10]
1. In the Project Board meeting (PBM) held last 22 September 2021, it was agreed that a new project will be co-designed using the uncommitted resources of the project. This new project will consider the realistic timeline and adjusted technical specifications as recommended by the TER.
Key Actions:
| Key Action | Responsible | DueDate | Status | Comments | Documents |
|---|---|---|---|---|---|
| 1.1 UNDP and DICT, together with the identified Technical Working Group (TWG) members during the PBM, Department of Health (DOH) and Department of the Interior and Local Government (DILG), to co-design the new project which will respond to the new priorities and context of the government.
[Added: 2022/03/10] |
Project Board, Institutions and Partnerships Team, Project Management Team | 2021/11 | Completed |
2. Recommendation:
2. Adopt additional technical refinements. Since the beginning of the project in September 2018 there have been improvements in VSAT models and technologies. Without dropping the SLA modality, minimum service requirements should be increased. Technologies that maximize upload/download speeds must be considered including shifting from KU to KA band technologies as stated earlier. The progressively increasing appetite for bandwidth should be planned for.
Management Response: [Added: 2022/03/10]
2. In the Project Board meeting (PBM) held last 22 September 2021, it was agreed that a new project will be co-designed using the uncommitted resources of the project. This new project will consider the realistic timeline and adjusted technical specifications as recommended by the TER.
Key Actions:
| Key Action | Responsible | DueDate | Status | Comments | Documents |
|---|---|---|---|---|---|
| 2.1 UNDP and DICT, together with DOH and DILG as the identified TWG members during the PBM, to co-design the new project which will respond to the new priorities and context of the government.
[Added: 2022/03/10] [Last Updated: 2022/11/29] |
Project Board Institutions and Partnerships Programme Team Project Management Team | 2022/11 | Completed | Please refer to the highlights of the 3rd project board management: https://undp.sharepoint.com/sites/PipolKonekPMO/Shared%20Documents/Forms/AllItems.aspx?id=%2Fsites%2FPipolKonekPMO%2FShared%20Documents%2FGeneral%2F%5F%5FFWFA%20Files%5F%5F%2F3%2E%20PROJECT%20DOCUMENTS%2FPROJECT%20BOARD%20MATTERS%2F2022%20March%20FWFA%20PBM%2FFWFA%203rd%20PBM%5FMinutes%20%28signed%29%2Epdf&parent=%2Fsites%2FPipolKonekPMO%2FShared%20Documents%2FGeneral%2F%5F%5FFWFA%20Files%5F%5F%2F3%2E%20PROJECT%20DOCUMENTS%2FPROJECT%20BOARD%20MATTERS%2F2022%20March%20FWFA%20PBM History |
3. Recommendation:
3. Accommodate greater participation. The active engagement of DICT provincial engineers, LGUs and the private sector in maintenance, value-added services and technological enhancements should be encouraged and may be provided for in the SLAs.
Management Response: [Added: 2022/03/10]
3. Partnership arrangements with DICT provincial engineers and LGUs will be strengthened, and private sector engagement will be mobilized in the new project design.
Key Actions:
| Key Action | Responsible | DueDate | Status | Comments | Documents |
|---|---|---|---|---|---|
| 3.1 UNDP and DICT, together with DOH and DILG as the identified TWG members during the PBM, to co-design the new project which will respond to the new priorities and context of the government. Design will include LGUs and private sector.
[Added: 2022/03/10] [Last Updated: 2022/11/29] |
Project Board Institutions and Partnerships Programme Team Project Management Team | 2022/11 | Completed | PB Advisors roles indicated in the amended Prodoc and their participation in project development and implemented indicated in the 3rd and 4th PBM highlights History |
4. Recommendation:
4. Future-proofing. The project should be more anticipatory and consider incorporating adaptive management in plans and strategies that would future-proof Project interventions leading to greater sustainability. These include: technological and service requirements forecasting for Outputs 1 & 3; and linking up DevLIVE training to NEDA’s Ambisyon 2040 outcomes for Output 2.
Management Response: [Added: 2022/03/10]
4. Risk management and sustainability planning are key components of the FWFA Project. Anticipatory and adaptive management planning and strategies will be embedded in the new project design. While the recommendation to link DevLIVE to Ambisyon 2040 outcomes is laudable, this is too Ambitious for the project to implement at this point. UNDP will continue to promote the use of DevLIVE to ensure citizen’s participation and feedback in large-scale government projects.
Key Actions:
| Key Action | Responsible | DueDate | Status | Comments | Documents |
|---|---|---|---|---|---|
| 4.1 UNDP and DICT, together with DOH and DILG as the identified TWG members during the PBM, to co-design
[Added: 2022/03/10] |
Project Board Institutions and Partnerships Programme Team Project Management Team | 2021/11 | Completed |
5. Recommendation:
5. Strengthen Internal and External Communication. It would be worthwhile investing in internal and external communication as well as social preparation with a minor rationalization of budgets to tip slightly towards non-technical interventions.
Management Response: [Added: 2022/03/10]
5. UNDP welcomes the recommendation to invest in communications and social preparation. Improvements in communications strategy and social preparation will be incorporated in the new project document.
Key Actions:
| Key Action | Responsible | DueDate | Status | Comments | Documents |
|---|---|---|---|---|---|
| 5.1 UNDP and DICT, together with DOH and DILG as the identified TWG members during the PBM, to co-design the new project which will respond to the new priorities and context of the government and will include improved communications and social preparation.
[Added: 2022/03/10] [Last Updated: 2022/11/29] |
Project Board Institutions and Partnerships Programme Team Project Management Team | 2022/11 | Completed | PB Advisors roles indicated in the amended Prodoc and their participation in project development and implemented indicated in the 3rd and 4th PBM highlights History |