Success message
error message
warn message
Mid Term Review of SSRI Project
Commissioning Unit: Timor-Leste
Evaluation Plan: 2015-2020
Evaluation Type: Project
Completion Date: 05/2016
Unit Responsible for providing Management Response: Timor-Leste
Documents Related to overall Management Response:
 
1. Recommendation:

Midterm Review recommendation 1.

Recommendation 4.1.1 – Undertake a detailed capacity assessment of the PDIM process to include technical and functional capacity, assessment of existing resourcing (manpower, financial resources), effectiveness of existing institutional arrangements (where this impacts capability/capacity), methods, standards and protocols used throughout the process.  Based on the outcome of the capacity assessment, develop an institutional capacity development and training plan for project-based capacity development and long-term capacity development.   This should include a capacity development plan for the long-term implementation of climate resilient small scale infrastructure projects via the PDID process.  

Management Response: [Added: 2018/04/24] [Last Updated: 2018/09/12]

This recommendation remains relevant for the project and the team agrees that a capacity assessment needs to be undertaken and hence the development of a capacity development plan.

The project would recruit consultants (national and international) to conduct an in-depth organizational and institutional capacity assessment of the three focused municipalities as well as the technical capacities and resources required for addressing and integrating climate variability risks and vulnerabilities in small scale rural infrastructure planning, design, implementation and operation and maintenance. Based on the assessments conducted, a comprehensive capacity development plan would be developed.

Key Actions:

2. Recommendation:

Midterm Review recommendation 2.

Recommendation 4.1.2 – Undertake a detailed review of the CVCA work and take steps to address the major technical shortcomings identified.  Key considerations should include:  data availability and data use (re-examine datasets for all hazard assessments, incorporate primary data on infrastructure, establish socio-economic data collection methods for use in risk and vulnerability assessment);  review and strengthen hazard assessment and mapping methodology for all hazards, review and strengthen risk assessment methodology (incorporation of socio-economic data, use of established risk and vulnerability methods that include damage and loss assessment, loss of livelihoods, infrastructure risk assessment).  The review should also address shortcomings in the treatment of gender.  

Management Response: [Added: 2018/04/24] [Last Updated: 2018/09/12]

The team partially agrees with this recommendation. While this could not be undertaken in its entirety, the project will pursue the implementation of activities to address some of the issue and concerns raised by the MTR in the review.

Ongoing validation of the information and data presented in the CVCA report and maps will be done during the primary data collection activities undertaken as part of the Climate Change Adaptation Planning for Rural Infrastructure Development (CCAPRID) activities by the project team which includes participation of government counterparts for existing rural infrastructure and how extreme climate events are affecting these infrastructures.

The project will use the CVCA maps to share information with vulnerable communities on how to plan and develop rural infrastructure. CCAPRID activities will also give special attention to focused groups and public awareness campaign for mainstreaming climate change into local development planning. Special attention is also given to women, youths and the elderly.

Key Actions:

3. Recommendation:

Recommendation 4.1.3 – Re-focus the project strategy to under component 2, to ensure greater impact of the project on the PDIM process.  This should include the following inputs to the PDIM project selection process:

  1. Use of the CVCA (once it is revised and strengthened as per Recommendation 4.1.2) in the project identification process to provide a more comprehensive, robust and evidence-based means of identifying projects at suco level
  2. Provide technical assistance to AP staff and engineers in prioritizing projects at this level and in undertaking appropriate level of feasibility studies on which to based prioritization
  3. Technical input to the Municipal level project prioritization and review.  Introduce climate risk criteria into the prioritization process, and include other methods of measure benefits of projects other than number of beneficiaries (e.g. environmental enhancement).
  4. Provide training on engineering feasibility studies to include technical feasibility, investment feasibility, socio-economic cost-benefit analysis, optioneering and options appraisal methods and outline environmental impact assessment, to strengthen the feasibility process, safeguard investments and optimize engineering solutions. 
  5. Provide technical assistance to introduce climate change considerations into design of infrastructure to ensure that they will accommodate likely changes of environmental variables (frequency and intensity of occurrence) expected with climate change. 
  6. Introduce detailed Environmental impact assessment (EIA) at the detailed design stage, in line with international good practice to ensure that the potential impacts of the project are identified based on the detailed design and that mitigation measures can be built into the design.
  7. Provide technical assistance to streamline the procurement process by pre-qualifying contractors for the different types of projects to be implemented. 
  8. Provide technical assistance to strengthen the monitoring capacity at AP level through the provision of appropriate engineering expertise during implementation
Management Response: [Added: 2018/04/24]

The team partially agrees with this recommendation. While the project team is in agreement with b, c, d, e, f, and these will be pursued. The recommendations as per (a) the CVCA as developed by the project would be used while simultaneously conducting on the ground verification, the project will provide capacity building activities using CVCA at suco level for identification, planning and prioritization of projects for the suco development plans.  (g) it is unnecessary and impractical to pre-qualify contractors for the different projects to be implemented given the limited number of contractors and also since contractors are already evaluated on past experience and performance before contracts are awarded. In respect of recommendation (h), while the AP staff would be engaged in the monitoring during the project implementation, there are no technical staff (engineer) at AP level.

(b) Technical assistance and capacity development would also be provided to administrative post staff and municipality engineers in conducting technical feasibility assessments for project prioritization. During the implementation support and capacity development would also be provided to municipality technical staff for monitoring and supervision of the implementation of the construction projects.

(f) EIA in conformance with MCIE Decree Law No. 05/2011 would be carried out for each of the physical infrastructure projects to be implemented and that potential impacts are identified early so that mitigation measures can be put in place to address negative impacts. The EIA reports would be prepared and submitted to the MCIE NDPCEI for approval prior to project implementation.

Key Actions:

4. Recommendation:

Recommendation 4.1.4 – Review method of assessing project benefits and implement cost-benefit analyses more closely aligned with international best practice, for the purpose of providing sound and robust information to decision makers, and for providing evidence for project replicability and scaling up.

Management Response: [Added: 2018/04/24]

The team agree with this recommendation. In particular the metrics used for computing the number of project beneficiaries (direct and indirect) would be reviewed. The project will review the method for assessing the project benefits (including social and economic) and the number of beneficiaries for the climate resilient small scale infrastructure works. 

Key Actions:

5. Recommendation:

Recommendation 4.1.5 - Document more closely, the soil and land management hectares being planted by first identifying on GIS maps the planned route for planting (using a Polygon from which area can be derived.  This should be part of the agreed contract terms) and then using GPS to document and verify what has been planted.

Management Response: [Added: 2018/04/24] [Last Updated: 2018/04/24]

The team fully agrees with this recommendation. The project would provide capacity development to grant recipients for selecting the intervention/targeted areas and on the use of GPS to document the coverage of watershed management and soil bioengineering activities carried out by grant recipients (local NGOs/CBOs) in the three selected municipalities. 

Key Actions:

6. Recommendation:

Recommendation 4.2.1 – Re-examine the $150,000 budget limit and strengthen the vetting of projects to ensure they are technically feasible within the budget.

Management Response: [Added: 2018/04/24]

The team partially agree with this recommendation made by the MTR. While the projects will be reviewed to endure technical and financial feasibility, the SSRI project would continue to implement physical infrastructure projects within the $ 150,000 USD margin (PDIM Category A projects) based on the budgetary allocation and the number of infrastructure and villages to be targeted. Where required, provision would be made for technically and financially feasible projects that may require additional financing above $150,000.  Additionally, the project will review and strengthen all aspects of the infrastructure projects cycle, in particular the processes of identification, planning, implementation and monitoring and supervision of the projects to ensure that the physical infrastructure projects implemented satisfy the expectations of stakeholders and meets the needs of its beneficiaries.

Key Actions:

7. Recommendation:

Recommendations 4.2.2 – In the case of Lacoliu Irrigation Scheme (Baucau Municipality), identify budget from within the project or elsewhere to correct the problem.   

Management Response: [Added: 2018/04/24]

The team fully agrees with this recommendation. The SSRI project will ensure that the irrigation project that was implemented in Lacoliu meets the needs and expectations of the beneficiaries. The length of the channel constructed as part of the irrigation scheme project will be extended. The project will conduct a technical review and stakeholders’ consultation to determine the appropriate intervention to address the concerns raised by members of the community. 

Key Actions:

8. Recommendation:

Recommendation 4.2.3 – Gender and vulnerability currently only focus on numbers of participants in workshops or training sessions.  It is recommended that specific vulnerability indicators are introduced (e.g. CVCA methods targeting gender specific and vulnerable groups, training material translated into as many local minority languages as possible or visual training/assessment tools, participatory assessment tools to illiterate people).  It is also recommended that gender indicators should include measures of gender mainstreaming improvement.

Management Response: [Added: 2018/04/24]

The team fully agrees with this recommendation. Gender issues, especially the specific access patterns to infrastructure and ecosystem services and roles and responsibilities in the use and maintenance of rural infrastructure, are of special importance to the project with regards to inclusiveness and sustainability. The project will hire consultants (gender specialists) to develop a specific Gender Action Plan for gender mainstreaming for the project. The CVCA maps and reports and training and public awareness materials will be translated and distributed.

Key Actions:

9. Recommendation:

Recommendation 4.2.4 – Undertake a full capacity assessment (technical and functional) and develop a capacity plan which will form the basis of training for the remainder of the project.   It is also recommended that capacity indicators should be upgraded to include actual measures of increased capacity.    

Management Response: [Added: 2018/04/24]

This recommendation which is also related to recommendation 1.0 remains relevant for the project and the team agrees that a capacity assessment needs to be undertaken and hence the development of a capacity development plan. Although capacity development activities were already undertaken by the project, these were not aligned with a developed capacity development plan with specific indicators for monitoring increased capacity.

Consultations with national and sub-national counterparts and stakeholders will be organized in the third quarter of 2016 to develop the TOR for the consultancy to undertake a detailed capacity assessment and develop the Capacity Development Plan. 

Key Actions:

10. Recommendation:

This recommendation which is also related to recommendation 1.0 remains relevant for the project and the team agrees that a capacity assessment needs to be undertaken and hence the development of a capacity development plan. Although capacity development activities were already undertaken by the project, these were not aligned with a developed capacity development plan with specific indicators for monitoring increased capacity.

Consultations with national and sub-national counterparts and stakeholders will be organized in the third quarter of 2016 to develop the TOR for the consultancy to undertake a detailed capacity assessment and develop the Capacity Development Plan. 

Management Response: [Added: 2018/04/24]

The team agrees with the recommendation that adequate risk management must be undertaken. The project will ensure that active risk management and monitoring in respect of the project achieving its objectives is pursued. Where there are potential risks that may have an impact on the project, active and timely involvement and collaboration with stakeholders including government counterparts and the project team to immediately address the risks will be pursued. The project will maintain a risk and issues log that will be updated, tracked and monitored regularly.

Key Actions:

11. Recommendation:

Recommendation 4.3.1 – Training for M&E to project staff as well as municipality staff.

Management Response: [Added: 2018/04/24]

The team fully agrees with this recommendation.

The project plans to recruit an international consultant to provide technical assistance to the project team and counterparts in reviewing and developing robust and responsive internal M&E processes and reporting and to strengthen and develop the capacity of the project team and national counterparts through relevant training in Results Based Management and Monitoring & Evaluation.

Key Actions:

12. Recommendation:

Recommendation 4.3.1 - Engage international expertise to enhance local capacity in key areas such as hazard and risk assessment and mapping, engineering design, contract management, cost-benefit analysis and investment planning.  

Management Response: [Added: 2018/04/24]

The team partially agrees with this recommendation. While undertaking the CVCA, consultants would have been recruited for the conduct of risk assessment and mapping. However, due to resource constraints this consultancy could not be further extended.

The project plans to recruit international consultant to conduct an institutional and organizational capacity assessment in particular at sub-national (municipality) levels. As part of the assessment the areas mentioned above would be assessed to determine the existing capacities and thereafter develop a detailed Capacity Development plan based on the capacity assessment conducted.  Capacity development in areas such as project management, contract management and engineering designs would be supported by the project.

Key Actions:

13. Recommendation:

Recommendation 4.4.1 - Introduce a financial analysis output, based on the vulnerability mapping, cost-benefit analysis (and scaled up to rest of TL), to help identify the financial commitment that government will need for long-term national SSRI funding.

Management Response: [Added: 2018/04/24]

The team partially agrees with this recommendation. The project will be reviewing the financial and economic aspects of the investment such as cost/benefit analysis, etc. for the physical infrastructure project implemented Of importance is the effect of the climate resilient features incorporated in SSRI designs on the projects functionality and service life. The technique of combining hard engineering with soft engineering solutions (bio-engineering) to construct climate resilient infrastructure is a demonstration of how relatively small additional investment on the soft-solutions such as bio-engineering to protect the physical infrastructure contributes to improving and extending the service life of the project. Therefore, the project will seek to analyze the impact of the additional bioengineering features and associated costs against its overall lifecycle cost (which include maintenance costs as well). This information would be shared with counterparts (both national and sub-national levels) which is expected to influence the planning and budgeting processes for implementation of climate resilient rural infrastructure in all municipalities.

Key Actions:

14. Recommendation:

Recommendation 4.4.2 – Develop the financing model for maintenance of infrastructure and roll out for all SSRI schemes already built.  Include monitoring mechanism to collect evidence base and calibrate financial model for long-term maintenance financing.

Management Response: [Added: 2018/04/24]

The team partially agrees with this recommendation. Projects identified and implemented by the SSRI project are selected from the Municipality Investment Plan. The budgetary allocations provided in the project’s budget is for the construction of the infrastructure. After the project would have been completed and handed over to the municipality, the maintenance would be based on the respective municipalities maintenance plan and budgetary allocation.

The project will work closely with government agencies and counterparts to determine the mechanism to be put in place for maintenance and management of the infrastructure projects delivered. While all projects completed would be handed over to the respective sectoral agencies in the municipalities for management, and maintenance operations, the project would also assess and document the maintenance requirements for each of the project implemented and provide this information to the respective municipalities and sectoral agencies.

Key Actions:

15. Recommendation:

Recommendation 4.5.1 – Use evidence gathered from the project to provide cost-benefit evidence of implementing climate resilient SSRI and for defining the capacity (and feasibility) of replicating and scaling up the project nationally.  This can be done with closer evidence gathering, and parameter/indicator measurement.   

Management Response: [Added: 2018/04/24]

The team agrees with this recommendation. The project will review the financial and economic aspects of the investment such as cost/benefit analysis, etc. The assessment will focus on the impact of the climate resilient features that were incorporated in the designs for the projects, particularly on the infrastructure functionality and service life. It will attempt to analyze the additional investment costs required for the bioengineering features and the relationship to the overall lifecycle cost (which include maintenance costs as well).

Key Actions:

16. Recommendation:

Recommendation 4.5.2 – Formulate an investment framework based on cost-benefit analysis and evidence base, with project figures for the rest of TL

Management Response: [Added: 2018/04/24]

The team partially agrees with this recommendation. The investment framework can be formulated based on the actual cost for implementation of each of the physical infrastructure project undertaken by the SSRI project with respect to the combination of hard and soft engineering elements to develop climate resilient infrastructure. However, considering the constraints of the project, it would be difficult to provide project figures for the rest of TL.

The project will review the financial and economic aspects of the investment such as cost/benefit analysis, etc. The assessment will focus on the impact of the climate resilient features that were incorporated in the designs for the projects, particularly on the infrastructure functionality and service life. It will attempt to analyze the additional investment costs required for the bioengineering features and the relationship to the overall lifecycle cost (which include maintenance costs as well).

Key Actions:

17. Recommendation:

Recommendation 4.6.1 – Set up a lesson-learned log to include lessons identified, lessons learned and lessons disseminated as well as detailed write up of each project implemented for the purpose of evidence-based advocacy.

Management Response: [Added: 2018/04/24]

The project team fully agrees with this recommendation. Therefore, the project will develop and maintain a lessons learned log which will include information such as the situation, recommendations and follow-up actions required in respect of project activities. Additionally, the detailed write-up for each of the climate resilient rural infrastructure implemented will be pursued which will also include the lessons-learned.

Key Actions:

18. Recommendation:

Recommendation 4.6.2 – Hire a communications specialist, develop a communication plan, develop articles, video blogs, short programs, and other media material (tailor communication material to audience and medium) to disseminate information.

Management Response: [Added: 2018/04/24]

The project team fully agrees with this recommendation. Therefore, the project will hire a communications consultant to develop a detailed communication strategy for the project, associated work plan to be implemented and publication materials.

Key Actions:

Latest Evaluations

Contact us

220 East 42nd Street
20th Floor
New York, NY 10017
Tel. +1 646 781 4200
Fax. +1 646 781 4213
erc.support@undp.org