Assessment of Development Results: Pacific Islands

Report Cover Image
Evaluation Plan:
2009-2013, Independent Evaluation Office
Evaluation Type:
ADR
Planned End Date:
01/2012
Completion Date:
01/2012
Status:
Completed
Management Response:
Yes
Evaluation Budget(US $):
150,000

Share

Document Type Language Size Status Downloads
Download document ADR_Pacific_Islands.pdf report English 1350.99 KB Posted 1333
Title Assessment of Development Results: Pacific Islands
Atlas Project Number:
Evaluation Plan: 2009-2013, Independent Evaluation Office
Evaluation Type: ADR
Status:Completed
Completion Date:01/2012
Planned End Date: 01/2012
Management Response:Yes
Corporate Outcomes (UNDP Strategic Plan 2008-20013)
Evaluation Budget(US $): 150,000
Source of Funding:
Joint Programme:No
Mandatory Evaluation:No
Joint Evaluation: No
Evaluation Team members:
Name Title Email Nationality
Team Leader
GEF Evaluation:No
Key Stakeholders:
Location of Evaluation:Country Level
Countries: FIJI SAMOA
Lessons
Findings
Recommendations
1 Recommendation 1: Programme focus The four outcome areas with gender equality as a cross-cutting theme continue to be most relevant for the PICs. Hence, emphasis for the next programme cycle should be continued and consolidated in those areas. Experiences on some of those areas have started generating nationally embedded endeavours. Policy analysis and programme intervention support in the areas of poverty, employment, sustainable livelihood, food security, governance (parliament, electoral assistance, civic education, and decentralization), private sector, environment and climate change, and crisis prevention should continue to receive priority attention.
2 Recommendation 2: Programme strategy UNDP?s emphasis of work at central and policy level should be balanced with opportunities for work at downstream and outreach level with CSOs and communities in view of good experiences of effectiveness and results observed during the current cycle. This is particularly suitable in smaller islands. Downstream work should be used to inform policy-making. A differentiated programme strategy and approach could be considered for smaller island countries due to their specific situation, high unit cost of delivery and inherent capacity constraints. The development needs and interventions should be assessed based on the nature of the country. For example, options could be pursued for fewer and more integrated projects to reduce management workload, special measures for meeting capacity gaps, and joint/shared programme frameworks with other agencies. A coherent strategy should be strengthened and implemented for mainstreaming of gender equality. It should include a shared gender analysis at the regional level and at the national level. The analyses should assess priorities and opportunities for promoting gender equality and/ or women?s empowerment that should inform UNDP strategy. The project formulations must include a gender analysis for use in project management. Programme staff should have access to support and resources in this regard. The capacity development intent and content of projects should be made explicit at formulation stage with a detailed capacity assessment and statement of a strategy for capacity development which should be monitored and accounted for in progress reports.
3 Recommendation 3: Project cycle management
UNDP should accord priority and adequate technical support to this aspect. Project formulation should be addressed in a technically competent fashion. A thorough appraisal of the government's priority, and the project?s embeddedness in institutional context and capacity, should be undertaken during formulation to include all aspects.

Country demand management for substantive and technical support: Introduce a regime of organized country demand management in programming with a tight management oversight to address issues emerging at country project level and time-bound response system. UNDP should intensify conducting regular project management monitoring of progress. More importantly, it should introduce technical monitoring through quality-assurance support of important products of the projects. The difference between the two types of monitoring should be understood clearly. Technical professionals' services should be drawn from the Pacific Centre, if available, or from outside if necessary for this purpose. Monitoring should identify areas or products which require higher level dialogue and engagement within the government and policy-makers. This continued engagement with professional inputs is essential to ensure effectiveness of project outputs.

Monitoring and evaluation: Introduce a more thorough and disciplined monitoring and evaluation system as part of wider management strategy. A system of holding agenda-based periodic tripartite review meetings could be introduced coinciding with monitoring visits to countries/projects. Monitoring of activity schedules, outputs, progress towards outcomes and project/programme finances should be carried out and recorded as part of an institutional system. This documented information is essential as a base for monitoring and evaluation. Project and outcome evaluations should be planned, monitored and carried out with due diligence with clear accountability assigned to programme staff and management.
4 Recommendation 4 : Efficiency Efficiency issues should be addressed on a number of fronts:

1. Choice of implementation mode should be guided by the country situation rather than the corporate prescription of UNDP. The feasibility and efficiency of working with NEX and DEX modality should be studied in each case to choose the appropriate modality. If required, a well-argued case for flexibility in small islands should be made by the MCO to UNDP Headquarters based on efficiency and results considerations. Reasons should be identified for the trend in delays in approvals. If some systemic and process prescriptions require more time, provide it in the planning phase and avoid unrealistic planning targets at the outset.

2. More flexible HR modalities or options for project-level recruitment should be introduced. Introduce retainer contracts, periodic technical support from institutions in the region, where recruitment of longer term technical personnel is proving difficult.

3. The issue of delays in fund transfers to projects should be addressed. The system of transfer should work with equal efficiency in all cases, unless there are explainable constraints. At the project level, appropriate training should be imparted in cash-flow planning and management.
5 Recommendation 5 Production of a periodic subregional Human Development Report should be considered to facilitate advocacy work on sensitive issues in the subregion and also to provide added support for promotion and compliance with UN values.
6 Recommendation 6 Connect, integrate, and infuse UNDP?s global knowledge and solution to Pacific projectlevel work. The Pacific Centre?s comparative advantage in terms of its current work, focus and proven knowledge management competence should be coordinated with the MCOs? country demand management system. UNDP?s comparative advantage as a repository of global knowledge and experience requires greater application at the programme and project levels. This would also enhance the quality of project-level development work. This requires systematic and intentionality in application.
7 Recommendation 7 Introduce an institutional oversight system which would enable the MCOs and the Pacific Centre to consolidate the organization?s strength to deliver better-quality development assistance. The performance of the current rules of engagement should be reviewed and applied with regular oversight by the senior management of the MCOs and the Pacific Centre. A dedicated participatory management deliberation between the MCOs, the Pacific Centre and Regional Bureau for Asia and the Pacific is recommended to seriously explore potential options and follow it up with bold decisions to implement all consequential changes such as integrated work plan, clear decision-making structure and accountability and financial management. If the distinctive UNDP aspect of global knowledge infusion in programmes is not made visible and useful, its position as a value-adding partner to the PICs may be undermined.
1. Recommendation: Recommendation 1: Programme focus The four outcome areas with gender equality as a cross-cutting theme continue to be most relevant for the PICs. Hence, emphasis for the next programme cycle should be continued and consolidated in those areas. Experiences on some of those areas have started generating nationally embedded endeavours. Policy analysis and programme intervention support in the areas of poverty, employment, sustainable livelihood, food security, governance (parliament, electoral assistance, civic education, and decentralization), private sector, environment and climate change, and crisis prevention should continue to receive priority attention.
Management Response:

UNDP responsible units agree with this recommendation and commit to continue and consolidate work in key outcome areas for the next programme cycle.

Key Actions:

Key Action Responsible DueDate Status Comments Documents
Under the auspices of new UNDAF, in 2013-17programme cycle UNDP will consolidate its programme focus in the areas of poverty reduction, governance,and climate change, environment and disaster risk management; and designate gender equality as both a cross-cutting theme and standalone outcome. This will be formalized in the new UNDP Sub-Regional Programme Document (SRPD) for Pacific Island Countries and the countries will be encouraged to focus on one or two outcomes from the range of choices available in SPRD. Fiji MCO, Samoa MCO, PC 2013/01 Completed UNDP Pacific Sub-Regional Project Document (SRPD) approved at UNDP Executive Board, First Regular Session 2013;
2. Recommendation: Recommendation 2: Programme strategy UNDP?s emphasis of work at central and policy level should be balanced with opportunities for work at downstream and outreach level with CSOs and communities in view of good experiences of effectiveness and results observed during the current cycle. This is particularly suitable in smaller islands. Downstream work should be used to inform policy-making. A differentiated programme strategy and approach could be considered for smaller island countries due to their specific situation, high unit cost of delivery and inherent capacity constraints. The development needs and interventions should be assessed based on the nature of the country. For example, options could be pursued for fewer and more integrated projects to reduce management workload, special measures for meeting capacity gaps, and joint/shared programme frameworks with other agencies. A coherent strategy should be strengthened and implemented for mainstreaming of gender equality. It should include a shared gender analysis at the regional level and at the national level. The analyses should assess priorities and opportunities for promoting gender equality and/ or women?s empowerment that should inform UNDP strategy. The project formulations must include a gender analysis for use in project management. Programme staff should have access to support and resources in this regard. The capacity development intent and content of projects should be made explicit at formulation stage with a detailed capacity assessment and statement of a strategy for capacity development which should be monitored and accounted for in progress reports.
Management Response:

UNDP responsible units agree with this recommendation and commit to balance upstream (policy) support with downstream (grassroots) interventions.

Key Actions:

Key Action Responsible DueDate Status Comments Documents
UNDP Pacific Sub-Regional Programme Document for 2013-17 will: encompass a balanced approach between policy (upstream) and community (downstream) levels of engagement; acknowledge the need for differentiated strategy in smaller island countries; and mainstream gender equality and include sex disaggregated data to results and resources framework. Fiji MCO, Samoa MCO, PC 2013/01 Completed SRPD approved at First Regular Session, 2013.
Country Programme Action Plans (CPAPs) for smaller island countries will take a differentiated approach. They will seek to respond to the most acute development constraints where UNDP has comparative advantage, focus on one or two outcomes from the range of choices available in SRPD and, to the extent possible, leverage cross-practice solutions in order to strengthen development impact and reduce transaction costs for national counterparts. Fiji MCO, Samoa MCO 2013/03 Completed For UNDP, the umbrella UNDAF Action Plan (UNDAP) with CRM replaced CPAPs and are basis for coherent collaboration and provide for greater flexibility of emerging realities as well as ensures UNDPs focus on areas of comparative advantage. UNDAF Action Plans with Country Results Matrices (CRM) sent for country endorsement in December 2012. Country Matrices focus on individual country priorities.
A UNDP Pacific Gender Strategy will be developed and reviewed every year, taking into account an existing PC strategy for Pacific regional programme and similar MCO frameworks, for stronger mainstreaming of gender equality. The strategy will build on SRPD?s emphasis on gender as a standalone outcome and cross-cutting theme and seek to respond to relevant priorities outlined in the Joint PIF-UNSG statement (2011), while also building capacity and empowering UNDP programme staff as gender champions. Fiji MCO, Samoa MCO, PC (lead) 2013/12 Overdue-Initiated i) Samoa MCO has already commenced the development of a UN Gender programme (May 2012) and secured donor resources for implementation. ii) ? Strategic Gender Mainstreaming in Fiji MCO programming include: 1)revision of gender equality strategy, 2)Development of gender checklist for projects; 3) Use of gender-sensitive M & E Frameworks; 4)Focus on transformational change; iii) UNCT Approach to Gender Joint Programming developed for closer collaboration with UN Women.
3. Recommendation: Recommendation 3: Project cycle management
UNDP should accord priority and adequate technical support to this aspect. Project formulation should be addressed in a technically competent fashion. A thorough appraisal of the government's priority, and the project?s embeddedness in institutional context and capacity, should be undertaken during formulation to include all aspects.

Country demand management for substantive and technical support: Introduce a regime of organized country demand management in programming with a tight management oversight to address issues emerging at country project level and time-bound response system. UNDP should intensify conducting regular project management monitoring of progress. More importantly, it should introduce technical monitoring through quality-assurance support of important products of the projects. The difference between the two types of monitoring should be understood clearly. Technical professionals' services should be drawn from the Pacific Centre, if available, or from outside if necessary for this purpose. Monitoring should identify areas or products which require higher level dialogue and engagement within the government and policy-makers. This continued engagement with professional inputs is essential to ensure effectiveness of project outputs.

Monitoring and evaluation: Introduce a more thorough and disciplined monitoring and evaluation system as part of wider management strategy. A system of holding agenda-based periodic tripartite review meetings could be introduced coinciding with monitoring visits to countries/projects. Monitoring of activity schedules, outputs, progress towards outcomes and project/programme finances should be carried out and recorded as part of an institutional system. This documented information is essential as a base for monitoring and evaluation. Project and outcome evaluations should be planned, monitored and carried out with due diligence with clear accountability assigned to programme staff and management.
Management Response:

UNDP responsible units note this recommendation and commit to strengthen technical aspects of project formulation.

Key Actions:

Key Action Responsible DueDate Status Comments Documents
A regime of organized country demand management will be introduced through CPAPs. Each CPAP will be based on government priorities and embedded in respective institutional contexts. Technically sound formulation of CPAP outcomes and outputs/projects, as well as quality assurance of results, will be backstopped by Pacific Centre. National ownership of results will be reinforced, where applicable, through annual Joint CPAP/AWP reviews with governments. Fiji MCO, Samoa MCO No due date No deadline established i)Final approval of CRMs (for Fiji, Vanuatu, Solomon Islands, Tonga, Kiribati, Tuvalu, Palau, FSM, Marshalls, Nauru) undertaken by UN Quality Assurance team comprising of country focal points from UN agencies, with technical support of PC and submitted for country endorsement in December 2012. ii) CRMs were discussed at UN Joint Annual Reviews with Governments in Q4 2012 for endorsement.
Technically sound formulation of CPAP outcomes and outputs, as well as technically sound formulation of projects with explicit capacity development intent, will be introduced with support from the PC and the Asia Pacific Regional Centre (APRC) along with their technical backstopping of implementation and quality assurance of results and systematic application of global and regional knowledge products for project-level work. Fiji MCO, Samoa MCO, PC, APRC No due date No deadline established i) Fiji Office tightened up on observation of rigorous Programme/Project Appraisal process with particular focus on performance measures
In addition to regular project monitoring (including tri-partite reviews and site visits) and Joint CPAP/AWP reviews, the monitoring and evaluation systems will be further strengthened through strategic annual review of activities by the Pacific Programme Board. The Board will be formed on the basis of the Pacific Centre?s Steering Committee and include governments, regional organizations and non-state actors. All No due date No deadline established The Pacific Programme Board included in its agenda for December 2012 a review of the establishment of the SRPD document for UNDP covering 14 countries. The Board was also briefed on how in future, the Board would review progress on SRPD and on the Pacific component of the APRC document as these will happen in conjunction with each other.
In case of Fiji MCO a regime of organized country demand management will also encompass review of the organization of work flows and backstopping arrangements between the MCO, the Sub-office in Solomon Islands and the strengthened Joint and Field Presences in other Pacific island countries. Fiji MCO No due date No deadline established i) Realignment of programming units and development of standard office procedures for greater coherency amongst joint field presence units; inclusive processes strengthened to work jointly on corporate reporting with Solomon Islands; ii) Rigorous Programme/Project Appraisal process with particular focus on performance measures re-established according to realigned unit roles/responsibilities, with development of Standard Operating Procedures for improved programming process coordination in March Programme retreat.
4. Recommendation: Recommendation 4 : Efficiency Efficiency issues should be addressed on a number of fronts:

1. Choice of implementation mode should be guided by the country situation rather than the corporate prescription of UNDP. The feasibility and efficiency of working with NEX and DEX modality should be studied in each case to choose the appropriate modality. If required, a well-argued case for flexibility in small islands should be made by the MCO to UNDP Headquarters based on efficiency and results considerations. Reasons should be identified for the trend in delays in approvals. If some systemic and process prescriptions require more time, provide it in the planning phase and avoid unrealistic planning targets at the outset.

2. More flexible HR modalities or options for project-level recruitment should be introduced. Introduce retainer contracts, periodic technical support from institutions in the region, where recruitment of longer term technical personnel is proving difficult.

3. The issue of delays in fund transfers to projects should be addressed. The system of transfer should work with equal efficiency in all cases, unless there are explainable constraints. At the project level, appropriate training should be imparted in cash-flow planning and management.
Management Response:

UNDP responsible units agree with this recommendation and commit to pursue differentiated implementation modalities, also in view of recommendation 2b, within the framework of UNDP corporate standards.

Key Actions:

Key Action Responsible DueDate Status Comments Documents
As part of the Agenda of Organizational Change (AOC), UNDP will consider flexible modalities for programme implementation and human resources management (including project-level recruitment). These flexible modalities will recognize unique development challenges of small island developing states. Fiji MCO, Samoa MCO, PC, RBAP 2013/06 Overdue-Initiated Ongoing -Nationally executed projects with Country Office support on mutually agreed to service lines is included as part of project formulation/inception processes (Fiji Office)
Meanwhile, in light of SIDS constraints, UNDP will consider flexibility for approval of direct implementation (DIM) of programmes and projects, as well as national implementation with Country office support (NIM/COS) including direct payments to beneficiaries/vendors. Fiji MCO, Samoa MCO, PC, RBAP No due date No deadline established The newly formulated Tonga Governance Project was approved for DIM modality (Qtr 1 2013) (Fiji Office).
With the introduction of new CPAPs, UNDP will organize in-country training of government counterparts for more effective and efficient project management, and will review the relevant work flows to identify where implementation efficiencies could be improved. At the same time, Joint and Field presences will be progressively empowered as an integral part of continuous work flows with Fiji MCO in order to facilitate smooth service provision on the ground. Fiji MCO 2013/06 Overdue-Not Initiated Ongoing
With the introduction of new CPAPs, UNDP will continue ongoing training of all Project Coordinators to monitor project performance and delivery of results, while also providing ongoing training on Results-Based Management and project management. Samoa MCO 2013/01 Overdue-Not Initiated Ongoing
5. Recommendation: Recommendation 5 Production of a periodic subregional Human Development Report should be considered to facilitate advocacy work on sensitive issues in the subregion and also to provide added support for promotion and compliance with UN values.
Management Response:

UNDP responsible units agree with this recommendation and commit to consider production of sub-regional and, based on country demand, national human development reports.

Key Actions:

Key Action Responsible DueDate Status Comments Documents
UNDP will consider and submit for discussion with the Pacific Programme Board in October 2012 the various ways and modalities ? including human development reports ? to facilitate policy work on sensitive issues in the sub-region. Additional opportunities will be considered at the country level, such as using existing pro-poor policy options in Samoa as a launching pad for dialogue with the government and other stakeholders. Fiji MCO, Samoa MCO, PC (lead) 2013/12 Overdue-Initiated The Pacific Programme Board Meeting in Dec 2012 discussed modalities for programming and potential areas for programming based on critical areas raised by country reps at the meeting.
6. Recommendation: Recommendation 6 Connect, integrate, and infuse UNDP?s global knowledge and solution to Pacific projectlevel work. The Pacific Centre?s comparative advantage in terms of its current work, focus and proven knowledge management competence should be coordinated with the MCOs? country demand management system. UNDP?s comparative advantage as a repository of global knowledge and experience requires greater application at the programme and project levels. This would also enhance the quality of project-level development work. This requires systematic and intentionality in application.
Management Response:

UNDP responsible units acknowledge the importance of this recommendation and will endeavour to strengthen the organization?s role in knowledge management and development solutions exchange.

Key Actions:

Key Action Responsible DueDate Status Comments Documents
UNDP will strengthen its leadership role in knowledge management in the sub-region. Existing Solutions Exchange community of practice on climate change and development effectiveness will be further institutionalized (and sustained before the current pilot stage expires). UNDP will endeavor to build on that by exploring/creating other communities of practice. Fiji MCO, Samoa MCO, PC No due date No deadline established Communities of practice established on climate change and development effectiveness through Pacific Solutions Exchange
UNDP will further strengthen its role in knowledge management through continued active participation in government-led donor coordination mechanisms, such as quarterly donor meetings in Samoa or annual donor roundtables in Cook Islands. Samoa MCO No due date No deadline established Ongoing
7. Recommendation: Recommendation 7 Introduce an institutional oversight system which would enable the MCOs and the Pacific Centre to consolidate the organization?s strength to deliver better-quality development assistance. The performance of the current rules of engagement should be reviewed and applied with regular oversight by the senior management of the MCOs and the Pacific Centre. A dedicated participatory management deliberation between the MCOs, the Pacific Centre and Regional Bureau for Asia and the Pacific is recommended to seriously explore potential options and follow it up with bold decisions to implement all consequential changes such as integrated work plan, clear decision-making structure and accountability and financial management. If the distinctive UNDP aspect of global knowledge infusion in programmes is not made visible and useful, its position as a value-adding partner to the PICs may be undermined.
Management Response:

UNDP responsible units agree with this recommendation and commit to strengthen integrated institutional oversight systems in the sub-region.

Key Actions:

Key Action Responsible DueDate Status Comments Documents
UNDP will review performance of the current rules of engagement and explore consequential changes required to strengthen effective coordination of its activities across several units, such as integrated work planning, clear decision-making structure and accountability. This will apply equally to the relationship between the MCOs and the Pacific Centre as well between the MCOs and APRC Bangkok. Fiji MCO, Samoa MCO, PC, RBAP, APRC 2013/12 Overdue-Initiated Joint processes of programming reviewed at recent programming March 2013 Fiji MCO Programming retreat to be followed up with PC.
Pacific Programme Board (to be established on the basis of current PC Steering Committee) will be progressively strengthened as the central element of the integrated institutional oversight system. The Board will provide quality assurance of key outputs and plans of UNDP and steer coordination of activities across several UNDP units in the Pacific. Fiji MCO, Samoa MCO, PC (lead) No due date No deadline established The Pacific Programme Board, in its December 2012 meeting, was briefed on how in future, the Board would review progress on SRPD and on the Pacific component of the APRC document as these will happen in conjunction with each other.

Latest Evaluations

Contact us

220 East 42nd Street
20th Floor
New York, NY 10017
Tel. +1 646 781 4200
Fax. +1 646 781 4213
erc.support@undp.org