UNDAF (One Plan) Evaluation, 2012-2016

Report Cover Image
Evaluation Plan:
2012-2016, Viet Nam
Evaluation Type:
UNDAF
Planned End Date:
03/2016
Completion Date:
04/2016
Status:
Completed
Management Response:
Yes
Evaluation Budget(US $):
103,000

Share

Document Type Language Size Status Downloads
Download document Independent Review of the One Plan_Final Report.pdf report English 1292.54 KB Posted 261
Download document Independent Review of the One Plan_Annexes 1-11.pdf related-document English 1133.45 KB Posted 255
Download document Terms of Reference for One Plan Evaluation.pdf tor English 618.67 KB Posted 215
Title UNDAF (One Plan) Evaluation, 2012-2016
Atlas Project Number:
Evaluation Plan: 2012-2016, Viet Nam
Evaluation Type: UNDAF
Status: Completed
Completion Date: 04/2016
Planned End Date: 03/2016
Management Response: Yes
Focus Area:
  • 1. Others
Corporate Outcome and Output (UNDP Strategic Plan 2014-2017)
  • 1. Output 1.1. National and sub-national systems and institutions enabled to achieve structural transformation of productive capacities that are sustainable and employment - and livelihoods- intensive
  • 2. Output 1.4. Scaled up action on climate change adaptation and mitigation across sectors which is funded and implemented
  • 3. Output 2.1. Parliaments, constitution making bodies and electoral institutions enabled to perform core functions for improved accountability, participation and representation, including for peaceful transitions
  • 4. Output 2.2. Institutions and systems enabled to address awareness, prevention and enforcement of anti-corruption measures across sectors and stakeholders
  • 5. Output 2.3 Capacities of human rights institutions strengthened
  • 6. Output 2.4. Frameworks and dialogue processes engaged for effective and transparent engagement of civil society in national development
  • 7. Output 3.2. Functions, financing and capacity of sub-national level institutions enabled to deliver improved basic services and respond to priorities voiced by the public
  • 8. Output 5.2. Effective institutional, legislative and policy frameworks in place to enhance the implementation of disaster and climate risk management measures at national and sub-national levels
  • 9. Output 5.4. Preparedness systems in place to effectively address the consequences of and response to natural hazards (e.g. geo-physical and climate related) and man-made crisis at all levels of government and community
  • 10. Output 7.1. Global consensus on completion of MDGs and the post 2015 agenda informed by contributions from UNDP
  • 11. Output 7.2. Global and national data collection, measurement and analytical systems in place to monitor progress on the post 2015 agenda and sustainable development goals
  • 12. Output 7.5 South-South and Triangular cooperation partnerships established and/or strengthened for development solutions
  • 13. Output 3.4. Functions, financing and capacity of rule of law institutions enabled, including to improve access to justice and redress
Evaluation Budget(US $): 103,000
Source of Funding: OP/UNDAF Innovation Facility Funds
Evaluation Expenditure(US $): 120,000
Joint Programme: No
Joint Evaluation: Yes
  • Joint with All UN Agencies and Implementing partners
Evaluation Team members:
Name Title Email Nationality
GEF Evaluation: No
Key Stakeholders: UNDP/UNFPA/ UNICEF....
Countries: VIET NAM
Lessons
Findings
Recommendations
1

Develop a theory of change and identify medium term indicators that allow the assessment of progress towards outcomes, with insights as to the quality of results, underlying factors and likelihood of sustainability

This will require moving away from assumptions of how activities/outputs will translate into outcomes (e.g. training without analysis as to the type, use and quality) and a more detailed reflection not only of what it does best but also of what is the most effective and efficient way to do it, given its particular strengths and those of other actors present in the country (e.g. what type of training is the UN uniquely positioned to deliver, what is the value added of the UN’s on-the-ground interventions vis-à-vis other actors in the country). Many of the smaller activities observed could be undertaken by other actors, freeing limited resources to focus on actions it is uniquely positioned to undertake (…) The team recommends the new One Plan develops a theory that explains how the activities to be undertaken will lead to the desired high-level outcomes. This theory of change will help in the process of identifying medium-term progress indicators that will help management assess the validity of the original theory.

2

Identify an inclusive governance structure able to provide guidance and quality assurance

There is a need to identify an adequate fit-for-purpose and participatory governance structure that is able to provide oversight, guidance and quality assurance, as well as a means for coordination and substantive exchange with key stakeholders, mainly government and donors, to better reflect the tripartite nature of the DaO initiative. This structure will require a clearer accountability as well as incentive towards the One Plan.

3

Strengthen internal coordination mechanisms as a means to increase coherence and effectiveness

The UN should continue to strengthen the role of the JPGs as an effective mechanism to promote DaO, adequately incentivize and recognize staff coordination efforts and seek ways to help benefits from JPGs trickle up into high-level management. JPGs have the potential to be the drivers of policy coherence and prioritization as well as knowledge exchange between agencies. Some specific ways this could be achieved include:

• Use JPGs to drive policy-level dialogue and promote conceptual coherence (in partnership with other
development partners, etc.) and derive joint delivery to JPG Working Groups as needed.
• Use JPGS as a mechanism to identify short- and medium-term key priorities and means to assess progress. The team is of the opinion that development of a joint position paper with key strategic areas (as per the socialization paper) to help strengthen the UN’s position in Viet Nam both programmatically and when leveraging resources.

• Strengthen the accountability of the JPGs towards the UNCT and emphasize the importance of the role of the JPG convener. The team concludes it is crucial for the work at the JPG level to be an official part of staff responsibilities, and as such this should be adequately reflected in performance assessments; this would apply to both junior and senior staff (i.e. conveners, co-conveners).

• Ensure participation of well-informed and empowered staff in JPG meetings. The team acknowledges the potential value of having relatively junior staff attend JPG meetings as a means to increase their knowledge on certain subjects. Still, agency participation should be adequate to the task at hand. For example, discussions to agree key priority areas of focus or on common agreed definitions would benefit from more experienced staff to ensure the agency’s position is fully reflected and considered, otherwise the outcome document might lack relevance for that particular agency.

4

Continue to strengthen the role of the UN as a bridge between government and other actors, in particular civil society

The UN should use its competitive advantage as a trusted convener to promote substantive participation of relevant actors in relevant discussions, with particular focus on civil society and the private sector. The team observed that this is currently done, but, taking into account perceptions of the stakeholders’ interviewed, the UN in Viet Nam would benefit from doing this in a more structured manner. The team acknowledges it is not possible for these groups to be part of the governance structure, but perhaps this could be incorporated as a cross-cutting theme, which is monitored through regular reporting mechanisms and appraisal forms, helping ensure it becomes a more intrinsic part of the UN’s work in Viet Nam. Both the One Plan and the individual agencies would benefit from better and more strategic sharing of agencies’ existing networks between the agencies.

5

Strengthen monitoring and reporting (accountability) of the One Plan

Many of the limitations of this review stem from an inadequate monitoring, reporting and evaluation set up. It will also require identification of better indicators, including qualitative indicators and narratives that are able to capture quality and underlying factors. The team acknowledges the progress already achieved, but believes to fully achieve this the UNCT in Viet Nam will need to secure additional resources and either increase capacity at central level for the RCO to be able to lead this effort, or ensure more substantive support (through allocation of skilled staff and staff time) from the agencies to the One Plan and under the guidance of the RCO (more in line with the structure of the communications team). In addition to this the team recommends the UN in Viet Nam seeks ways to promote better alignment between agency level efforts and the One Plan, for example using the same indicators for both agency and One Plan level reporting. This would help avoid duplication of work and strengthen the quality of the M&E of the One Plan, providing better evaluative evidence for the mid-term and final evaluation of the One Plan to build on. A more strategic use of the IMEP, insuring agency level evaluations area shared and as much as possible aligned to feed into the One Plan evaluation time-table. These changes will increase the UN’s ability to report to the government and donors, as well as provide the basis for more evidence-based decision-making in the management of the One Plan. The team believes sharing of more detailed and up-to-date relevant information, both substantive and financial, would be a strategic tool in the process of securing additional resources from donor.

6

Strengthen the financial structure of the One Plan

The UN needs to identify clear incentives for the UN agencies and donors to prioritize the One Plan budget over bilateral funding, or mechanisms for bilateral funding to be accountable to the One Plan. In order to address concerns raised by the donors these would need to include:

• More comprehensive reporting to allow donors to understand both use of resources (results/impact) and what other resources their support was able to leverage (from other donors, RR and OR). (as per recommendation 5)

• Increased participation of donors to thematic and strategic One Plan dialogue in the areas of investment (as per recommendation 3)

• Promote structural mechanisms for documentation, dissemination and sharing between JPGs of good practices and lessons learned between JPGs (as per recommendation 2)

To address concerns around transparency raised and in line with the DaO principles, the One Plan design and reporting should incorporate all in-country financials, including agency level spending. Regular (quarterly) updates on spending etc should be made available to the RCO for regular management meetings. Updates on spending at country level (One Plan and agency) should be made available to donors and government. Much of this is already recommended in the 2014 Guide to the Common Budgetary Framework, which the UNCT is reportedly working towards implementing

7

Increased and more targeted focus on learning and on knowledge sharing

Overall, the team concludes the new One Plan would significantly benefit from a more reflective and learning-oriented approach, which will help identify not just progress of activities but also underlying factors for success. The One Plan M&E system needs to incorporate learning in a more systematic manner to allow for analysis of challenges and lessons learned, and to ensure these findings feed back into the management decision-making process in a timely manner. Analysis of the different types of interventions should be undertaken to help focus resources and efforts (e.g. investment in high-level policy-making vs. on-the-ground delivery; effectiveness of the different types of capacity building methodologies (investment vs return); underlying factors for success of successful vs not successful policy advice). This analysis should take into consideration the UN’s comparative advantages as well as those of other actors in the country. In addition, the new One Plan would benefit for a stronger knowledge management system between agencies and JPGs in particular of lessons learned, at both country level but also more broadly.

1. Recommendation:

Develop a theory of change and identify medium term indicators that allow the assessment of progress towards outcomes, with insights as to the quality of results, underlying factors and likelihood of sustainability

This will require moving away from assumptions of how activities/outputs will translate into outcomes (e.g. training without analysis as to the type, use and quality) and a more detailed reflection not only of what it does best but also of what is the most effective and efficient way to do it, given its particular strengths and those of other actors present in the country (e.g. what type of training is the UN uniquely positioned to deliver, what is the value added of the UN’s on-the-ground interventions vis-à-vis other actors in the country). Many of the smaller activities observed could be undertaken by other actors, freeing limited resources to focus on actions it is uniquely positioned to undertake (…) The team recommends the new One Plan develops a theory that explains how the activities to be undertaken will lead to the desired high-level outcomes. This theory of change will help in the process of identifying medium-term progress indicators that will help management assess the validity of the original theory.

Management Response: [Added: 2016/10/19] [Last Updated: 2016/10/19]

Agree

Key Actions:

Key Action Responsible DueDate Status Comments Documents
1.1 Identify and endorse strategic interventions in contribution to each of the proposed nine outcomes based on the Common Country Assessment and the UN’s comparative advantages
[Added: 2016/10/19] [Last Updated: 2017/08/09]
UNCT and GoV to endorse 2016/12 Completed History
1.2 Develop guidance for Results Groups annual/biennial work plans in such a way that focuses on limited number of key joint priorities, establishing Theories of Change (or other frameworks that focus on strategy and direction) to link outputs to outcomes, as well as medium-term measures to assess progress, transformation and contribution towards OSP outcomes
[Added: 2016/10/19] [Last Updated: 2017/08/09]
UN Results-based Management Working Group 2017/03 Completed See the attached Guidance History
1.3 Integrate in Results Groups’ annual/biennial planning regular assessments to provide information on quality of results, underlying success factors and likelihood of sustainability
[Added: 2016/10/19] [Last Updated: 2017/08/09]
Results Groups 2017/06 Completed The TOR for Results Groups already include assessments/evaluations. See examples attached. History
2. Recommendation:

Identify an inclusive governance structure able to provide guidance and quality assurance

There is a need to identify an adequate fit-for-purpose and participatory governance structure that is able to provide oversight, guidance and quality assurance, as well as a means for coordination and substantive exchange with key stakeholders, mainly government and donors, to better reflect the tripartite nature of the DaO initiative. This structure will require a clearer accountability as well as incentive towards the One Plan.

Management Response: [Added: 2016/10/19] [Last Updated: 2016/10/19]

Agree

Key Actions:

Key Action Responsible DueDate Status Comments Documents
2.1 For the period 2017-2021, joint UN-Government DaO Steering Committee is the highest decision-making authority on Delivering as One Initiative, with civil society and private sector representatives participating as observers and use an “action plan” to monitor its performance
[Added: 2016/10/19] [Last Updated: 2017/10/22]
GoV and RC/UNCT 2017/12 Completed It has been decided by the UNCT that the DaO Steering Committee is the highest decision-making authority on Delivering as One Initiative. See the attached OSP Coordination Architecture diagram. History
2.2 For the period 2017-2021, conduct annual review of results with participation of all stakeholders (in line with established OSP consultative processes) and facilitate dialogue around the value of DaO in middle-income country context
[Added: 2016/10/19] [Last Updated: 2017/08/09]
GoV and RC/UNCT 2017/12 Completed See the attached TOR for OSP Annual Review History
3. Recommendation:

Strengthen internal coordination mechanisms as a means to increase coherence and effectiveness

The UN should continue to strengthen the role of the JPGs as an effective mechanism to promote DaO, adequately incentivize and recognize staff coordination efforts and seek ways to help benefits from JPGs trickle up into high-level management. JPGs have the potential to be the drivers of policy coherence and prioritization as well as knowledge exchange between agencies. Some specific ways this could be achieved include:

• Use JPGs to drive policy-level dialogue and promote conceptual coherence (in partnership with other
development partners, etc.) and derive joint delivery to JPG Working Groups as needed.
• Use JPGS as a mechanism to identify short- and medium-term key priorities and means to assess progress. The team is of the opinion that development of a joint position paper with key strategic areas (as per the socialization paper) to help strengthen the UN’s position in Viet Nam both programmatically and when leveraging resources.

• Strengthen the accountability of the JPGs towards the UNCT and emphasize the importance of the role of the JPG convener. The team concludes it is crucial for the work at the JPG level to be an official part of staff responsibilities, and as such this should be adequately reflected in performance assessments; this would apply to both junior and senior staff (i.e. conveners, co-conveners).

• Ensure participation of well-informed and empowered staff in JPG meetings. The team acknowledges the potential value of having relatively junior staff attend JPG meetings as a means to increase their knowledge on certain subjects. Still, agency participation should be adequate to the task at hand. For example, discussions to agree key priority areas of focus or on common agreed definitions would benefit from more experienced staff to ensure the agency’s position is fully reflected and considered, otherwise the outcome document might lack relevance for that particular agency.

Management Response: [Added: 2016/10/19]

Agree

Key Actions:

Key Action Responsible DueDate Status Comments Documents
3.1 Define a UN coordination architecture in support of the One Strategic Plan’s implementation based on lessons from the One Plan 2012-2016
[Added: 2016/10/19] [Last Updated: 2017/08/09]
RC/UNCT 2016/10 Completed History
3.2 For the period 2017-2021, institutionalize the practice of Results Groups’ Conveners reporting to UNCT to assess changes in the programme environment, learning from experience, and propose to the GoV adjustments in the results matrix as necessary
[Added: 2016/10/19] [Last Updated: 2017/04/10]
RC/UNCT 2017/04 Completed History
3.3. Ensure that staff participation in Results Groups’ work is reflected, to the extent possible, in the respective Performance Assessments
[Added: 2016/10/19] [Last Updated: 2017/04/10]
RC/UNCT 2017/04 Completed History
4. Recommendation:

Continue to strengthen the role of the UN as a bridge between government and other actors, in particular civil society

The UN should use its competitive advantage as a trusted convener to promote substantive participation of relevant actors in relevant discussions, with particular focus on civil society and the private sector. The team observed that this is currently done, but, taking into account perceptions of the stakeholders’ interviewed, the UN in Viet Nam would benefit from doing this in a more structured manner. The team acknowledges it is not possible for these groups to be part of the governance structure, but perhaps this could be incorporated as a cross-cutting theme, which is monitored through regular reporting mechanisms and appraisal forms, helping ensure it becomes a more intrinsic part of the UN’s work in Viet Nam. Both the One Plan and the individual agencies would benefit from better and more strategic sharing of agencies’ existing networks between the agencies.

Management Response: [Added: 2016/10/19]

Agree

Key Actions:

Key Action Responsible DueDate Status Comments Documents
4.1 Include ‘Participation’ as a cross-cutting issue in the One Strategic Plan
[Added: 2016/10/19] [Last Updated: 2017/08/09]
GoV and RC/UNCT 2016/12 Completed History
5. Recommendation:

Strengthen monitoring and reporting (accountability) of the One Plan

Many of the limitations of this review stem from an inadequate monitoring, reporting and evaluation set up. It will also require identification of better indicators, including qualitative indicators and narratives that are able to capture quality and underlying factors. The team acknowledges the progress already achieved, but believes to fully achieve this the UNCT in Viet Nam will need to secure additional resources and either increase capacity at central level for the RCO to be able to lead this effort, or ensure more substantive support (through allocation of skilled staff and staff time) from the agencies to the One Plan and under the guidance of the RCO (more in line with the structure of the communications team). In addition to this the team recommends the UN in Viet Nam seeks ways to promote better alignment between agency level efforts and the One Plan, for example using the same indicators for both agency and One Plan level reporting. This would help avoid duplication of work and strengthen the quality of the M&E of the One Plan, providing better evaluative evidence for the mid-term and final evaluation of the One Plan to build on. A more strategic use of the IMEP, insuring agency level evaluations area shared and as much as possible aligned to feed into the One Plan evaluation time-table. These changes will increase the UN’s ability to report to the government and donors, as well as provide the basis for more evidence-based decision-making in the management of the One Plan. The team believes sharing of more detailed and up-to-date relevant information, both substantive and financial, would be a strategic tool in the process of securing additional resources from donor.

Management Response: [Added: 2016/10/19]

Agree

Key Actions:

Key Action Responsible DueDate Status Comments Documents
5.1 For the period 2017-2021, strengthen tracking and reporting of results by aligning outcome indicators in the One Strategic Plan to global SDG indicators and National Plan of Action for SDGs implementation
[Added: 2016/10/19] [Last Updated: 2017/10/22]
GoV and RC/UNCT 2016/12 Completed History
5.2 For the period 2017-2021, strengthen tracking and reporting of output level results (UN’s contribution) aligning OSP output level indicators to those used by UN Agencies’ Country Programmes
[Added: 2016/10/19] [Last Updated: 2017/04/10]
GoV and RC/UNCT 2017/04 Completed History
5.3 For the period 2017-2021, revise the format, roles/responsibilities and use of the Evaluation Plan, simplify content, ensure that reports are collected and stored as they become available and report on the status to the UNCT
[Added: 2016/10/19] [Last Updated: 2017/04/10]
UN Results-Based Management Working Group 2017/04 Completed History
5.4 Ensure that annual work-plans of RBM WG are fully supported with required human and financial resources, that staff support to joint RBM initiatives (such as data entry to the One Plan Database among others) is reflected in the respective Performance Assessments and that UNCT oversees the work of the RBM WG
[Added: 2016/10/19] [Last Updated: 2016/12/19]
RC/UNCT 2016/12 Completed History
6. Recommendation:

Strengthen the financial structure of the One Plan

The UN needs to identify clear incentives for the UN agencies and donors to prioritize the One Plan budget over bilateral funding, or mechanisms for bilateral funding to be accountable to the One Plan. In order to address concerns raised by the donors these would need to include:

• More comprehensive reporting to allow donors to understand both use of resources (results/impact) and what other resources their support was able to leverage (from other donors, RR and OR). (as per recommendation 5)

• Increased participation of donors to thematic and strategic One Plan dialogue in the areas of investment (as per recommendation 3)

• Promote structural mechanisms for documentation, dissemination and sharing between JPGs of good practices and lessons learned between JPGs (as per recommendation 2)

To address concerns around transparency raised and in line with the DaO principles, the One Plan design and reporting should incorporate all in-country financials, including agency level spending. Regular (quarterly) updates on spending etc should be made available to the RCO for regular management meetings. Updates on spending at country level (One Plan and agency) should be made available to donors and government. Much of this is already recommended in the 2014 Guide to the Common Budgetary Framework, which the UNCT is reportedly working towards implementing

Management Response: [Added: 2016/10/19]

Agree

Please note that this is contingent on UN HQ level harmonization of budget cycles. Specialized Agencies operate on a biennium budget cycle and it is difficult to project budget or develop a budgetary framework by Outcome/Output on an annual basis. With this in mind, this practice has been initiated in the current cycle and will be enhanced by including budgets in Results Groups Annual/Biennial workplans in the 2017-2021 cycle.

Key Actions:

Key Action Responsible DueDate Status Comments Documents
6.1 Review the UNCT approved Annual Report Concept Note to strengthen annual reporting incorporating lessons learned and through better link to OSP indicators and available financial information
[Added: 2016/10/19] [Last Updated: 2016/12/19]
UN Results-based Management Working Group 2016/12 Completed History
6.2 For the period 2017-2021, Results Groups will develop an Annual/Biennial Common Budgetary Framework (including funds secured and to be mobilized) and use it to monitor progress.
[Added: 2016/10/19] [Last Updated: 2017/10/22]
UN Results-based Management Working Group 2017/04 Completed The Annual Common Budgetary Framework has been developed for each Results Group (see the attached Note on Common Budgetary Framework) History
7. Recommendation:

Increased and more targeted focus on learning and on knowledge sharing

Overall, the team concludes the new One Plan would significantly benefit from a more reflective and learning-oriented approach, which will help identify not just progress of activities but also underlying factors for success. The One Plan M&E system needs to incorporate learning in a more systematic manner to allow for analysis of challenges and lessons learned, and to ensure these findings feed back into the management decision-making process in a timely manner. Analysis of the different types of interventions should be undertaken to help focus resources and efforts (e.g. investment in high-level policy-making vs. on-the-ground delivery; effectiveness of the different types of capacity building methodologies (investment vs return); underlying factors for success of successful vs not successful policy advice). This analysis should take into consideration the UN’s comparative advantages as well as those of other actors in the country. In addition, the new One Plan would benefit for a stronger knowledge management system between agencies and JPGs in particular of lessons learned, at both country level but also more broadly.

Management Response: [Added: 2016/10/19]

Agree

Key Actions:

Key Action Responsible DueDate Status Comments Documents
7.1 Revise M&E tools and guidelines for UN in Viet Nam to ensure that learning is incorporated in a more systematic manner
[Added: 2016/10/19] [Last Updated: 2016/12/19]
RC/UNCT 2016/12 Completed History

Latest Evaluations

Contact us

1 UN Plaza
DC1-20th Floor
New York, NY 10017
Tel. +1 646 781 4200
Fax. +1 646 781 4213
erc.support@undp.org