Evaluation of UNDP Strategic Plan 2008-2013

Report Cover Image
Evaluation Plan:
2009-2013, Independent Evaluation Office
Evaluation Type:
Thematic
Planned End Date:
01/2013
Completion Date:
04/2013
Status:
Completed
Management Response:
Yes
Evaluation Budget(US $):
120,000

Share

Document Type Language Size Status Downloads
Download document UNDP_SP_evaluation_2008-2013.pdf report English 1043.19 KB Posted 8644
Title Evaluation of UNDP Strategic Plan 2008-2013
Atlas Project Number:
Evaluation Plan: 2009-2013, Independent Evaluation Office
Evaluation Type: Thematic
Status: Completed
Completion Date: 04/2013
Planned End Date: 01/2013
Management Response: Yes
Corporate Outcome and Output (UNDP Strategic Plan 2014-2017)
Evaluation Budget(US $): 120,000
Source of Funding:
Joint Programme: No
Joint Evaluation: No
Evaluation Team members:
Name Title Email Nationality
Paul Balogun
GEF Evaluation: No
Key Stakeholders:
Lessons
Findings
1.

CHAPTER 3. FINDINGS ON THE PERFORMANCE OF UNDP

This chapter provides findings in relation to the question: How has UNDP performed during the period of the Strategic Plan relative to the strategic intent as expressed in Strategic Plan goals. It makes the assessment using three evaluation criteria: effectiveness, efficiency and sustainability (each of which is explained at the start of the relevant section). It draws on a meta-synthesis of independent evaluations but also incorporates analysis from the ERBM system and other related documentation including the reviews by donor partners.

3.1 EFFECTIVENESS

Although assessing performance across the organization is challenging it is nonetheless possible to identify some broad findings related to effectiveness and the factors affecting it. Evidence on performance largely comes from analysis and synthesis of evaluations that in turn base their assessments on contribution to national outcomes. Delivering the intended change in an outcome ultimately depends upon the actions of a wide range of stakeholders. Given the nature of what UNDP provides, successful delivery of UNDP results is likely to reveal constraints elsewhere as much as within the organization. It should also be noted that implementation of the Strategic Plan coincided with a severe financial and economic crisis that had a significant development impact on the world and in many programme countries.

Finding 1. In terms of effectiveness of contributions at the country level, UNDP is likely to have made, or to make, a significant contribution to the intended outcomes in most of its country programmes and across all four focus areas.


Tag: Effectiveness Business Model Country Support Platform MDGs Monitoring and Evaluation Policies & Procedures Results-Based Management

2.

Finding 2. A major reason for non-delivery of planned outputs is under-resourcing of projects due to the earmarked nature of funds. 


Tag: Efficiency Resource mobilization Country Support Platform Results-Based Management Theory of Change UNDP Regional Bureaux

3.

Finding 3. To increase effectiveness, learning needs to be strengthened at the country level and across the organization.


Tag: Effectiveness Country Support Platform Integration Knowledge management Monitoring and Evaluation UNDP Regional Bureaux Poverty Reduction South-South Cooperation

4.

3.2 EFFICIENCY

As part of its efforts to enhance organizational efficiency UNDP monitors the management efficiency ratio (total management expenditure divided by total management and programme expenditure). Since the start of the Strategic Plan the organization has been consistently above target, as illustrated in Table 6. While it may be interesting to monitor this from a management perspective, from a development one, it can mask differences in the business models used by different units.

The following analysis therefore takes a different approach and distinguishes between two types of efficiency: First is administrative efficiency, based on the following sub-criteria: (a) timeliness in executing programmes within deadlines and budgets and in the time required to perform certain operations and transactions; (b) use of human resources (e.g. time spent in administrative vs. substantive activities); and (c) use of management information systems. Second is programmatic efficiency, based on the extent to which UNDP has prioritized and concentrated its programme in a set of core activities rather than a fragmented set of programmes/projects that strain its capacity to manage efficiently. However, efficiency is treated inconsistently in evaluations, both in terms of depth of analysis and in use of a common understanding of efficiency. As a result, the evidence is only sufficient for flagging major areas of concern, not for making definitive judgements about where the problems lie across a sample of country programmes.

Finding 4. Even though measuring UNDP efficiency is difficult, the existing evidence is that strengthening the efficiency of projects and programmes is a major challenge.


Tag: Challenges Efficiency Country Support Platform Monitoring and Evaluation Operational Efficiency Ownership Policies & Procedures Bilateral partners UNDP Regional Bureaux

5.

Finding 5. In terms of programming efficiency, most of the country programme portfolios are not prioritized and not focused on a set of core activities.


Tag: Efficiency Country Support Platform Monitoring and Evaluation Policies & Procedures Programme Synergy Results-Based Management Theory of Change Data and Statistics

6.

3.3 SUSTAINABILITY

Sustainability of the benefits derived from UNDP’s contributions is extremely difficult to measure. When looking at the activities conducted since 2008, it is too early to make a judgement on the sustainability of many. The assessment therefore focused on the likelihood of sustainability, specifically, whether UNDP projects implemented strategies to manage possible threats to the sustainability of results. The findings below may well refer to the period before the start of the Strategic Plan, but even though the problem of sustainability has been reported for some time there is no evidence of a concerted effort to address the fundamental issues that surround it in the ADRs examined. This is notwithstanding efforts at UNDP headquarters to address the issue of sustainability of results through monitoring sustainability and creating necessary incentives.

Finding 6. Programmes are seldom designed with sustainability in mind. The UNDP funding model also adds to the sustainability challenge.


Tag: Efficiency Sustainability Business Model Country Support Platform Implementation Modality Ownership Partnership Policies & Procedures Programme/Project Design Bilateral partners Capacity Building

7.

CHAPTER 4. FINDINGS ON USE OF THE STRATEGIC PLAN AS A MANAGEMENT TOOL

This chapter examines the key roles the Strategic Plan plays in strengthening management for more effective contribution to development results. Although the Strategic Plan did not detail how it was to be used for this purpose, it did identify roles for the Strategic Plan in three areas: - To provide direction to the organization. - To communicate the areas in which it has comparative strengths.  - As the basis of the accountability framework. In addition the role of the Strategic Plan as the keystone of UNDP’s corporate planning system is examined.

4.1 DEVELOPING AND IMPLEMENTING A STRATEGIC PLANNING SYSTEM

The Strategic Plan contains all the elements expected in a stand-alone strategic planning document, although it should be assumed that strategic planning would be implemented through overall corporate level planning systems. The effectiveness of the Strategic Plan therefore depends upon whether these systems are in place, how effective they are and how the Strategic Plan interacts with them.

Finding 7. UNDP has established, and is in the process of strengthening, a comprehensive strategic planning system with the Strategic Plan at its core.


Tag: Effectiveness Change Management Country Support Platform Knowledge management Monitoring and Evaluation Operational Efficiency Policies & Procedures Results-Based Management UNDP Regional Bureaux

8.

4.2 PROVIDING DIRECTION TO THE ORGANIZATION

Providing direction to the organization is seen as the core role of the Strategic Plan. This section will examine how effective UNDP has been in this respect. Four specific elements of the Strategic Plan are examined: (a) focus areas; (b) core strategies; (c) mainstreaming efforts; and (d) areas where UNDP would not normally engage. Factors influencing the above areas are also examined.

Finding 8. UNDP has been increasingly successful in aligning its work to the four broad focus areas.


Tag: Coherence Effectiveness Policies & Procedures Bilateral partners

9.

Finding 9. UNDP has put in place systems to actively manage greater implementation of some, but not all, of the core strategies identified in the Strategic Plan.


Tag: Effectiveness Global Fund Human and Financial resources Ownership Partnership Policies & Procedures Programme/Project Design Project and Programme management Results-Based Management Civil Societies and NGOs Country Government Capacity Building

10.

Finding 10. UNDP success with mainstreaming gender equality and South-South cooperation has been mixed.


Tag: Effectiveness Efficiency Gender Equality Gender Mainstreaming Gender Parity Women's Empowerment Integration Knowledge management Operational Efficiency Policies & Procedures Results-Based Management Bilateral partners UNDP Regional Bureaux South-South Cooperation Data and Statistics

11.

Finding 11. UNDP country offices often do things the Strategic Plan states they should not normally do; specifically, undertaking downstream activities with no linkages to policy reform.


Tag: Coherence Effectiveness Local Governance Country Support Platform Human and Financial resources Integration Knowledge management Programme/Project Design Results-Based Management Capacity Building South-South Cooperation

12.

Finding 12. The trade-off between corporate priorities and the demands of programme partners and competing programming frameworks have added challenges to UNDP’s use of the Strategic Plan in providing direction.


Tag: Challenges Resource mobilization Country Support Platform Oversight Policies & Procedures Strategic Positioning UN Country Team UNDP Regional Bureaux

13.

4.3 COMMUNICATING UNDP’S FOCUS AND STRENGTHS

The Strategic Plan notes that it should play a role in communicating the focus areas and areas of UNDP’s comparative strength. Its ability to do so can also be included among the factors influencing senior management’s ability to provide direction to the organization.

Finding 13. The Strategic Plan was important for communicating internally between UNDP management and the Executive Board but was less effective in communicating across all of UNDP’s partners.


Tag: Communication Knowledge management Partnership UNDP Management

14.

4.4 PROMOTING GREATER ACCOUNTABILITY

Implementation of the Strategic Plan has also coincided with increased demand for accountability by taxpayers in countries providing resources to UNDP and then in turn by the development agencies of these countries. This demand has been captured in the idea of value for money and the need for greater clarity on reporting results.

Finding 14. Accountability through monitoring and reporting of performance under the Strategic Plan has improved over time.


Tag: Effectiveness Change Management Knowledge management Monitoring and Evaluation Policies & Procedures Results-Based Management Data and Statistics

15.

Finding 15. The system of decentralized evaluation is not working well, and while steps have been taken to improve overall utilization of evaluation, it is not as high as it could be.


Tag: Effectiveness Knowledge management Monitoring and Evaluation Policies & Procedures Quality Assurance UNDP Management

Recommendations
1

Recommendation 1. The new Strategic Plan needs to be clear about the direction it wants UNDP to take, and UNDP management needs to ensure that adequate tools are put in place to support and monitor implementation of the strategies and priorities contained in the plan.

A key conclusion was that efforts to implement priorities and approaches were mixed and not carefully monitored from the start of the Plan. Even though the role of the Plan is to provide direction, the role of UNDP management is to operationalize and implement this direction through ensuring clarity of concepts and approaches, developing tools such as guidance, and monitoring its implementation. Yet, the examples of capacity development and gender mainstreaming suggest that guidance and monitoring are not enough to ensure implementation of the approaches identified in the Strategic Plan. Even combined with strong leadership the central actions are necessary but not sufficient to guarantee implementation. 

2

Recommendation 2. The new Strategic Plan needs to explicitly address the trade-offs that occur as a result of the UNDP business model.

UNDP’s comparative strengths are not just in parts of its focus areas but, inter alia, in its longterm commitment and in general its position as a trusted partner. These are elements of an underlying ‘theory’ of how UNDP works that is operationalized through a business model. The trade-offs faced by UNDP are inherent in any business model and need to be made transparent, with decisions made by the Executive Board as to UNDP’s position. Specifically, agreement must be reached on the trade-offs in two areas: first, between national ownership and organizational priorities with clear organization, approaches, strategies and focus combined with guidance for programme units when faced with a potential trade-off; and second, between addressing long-term capacity development needs and short-term results, which requires a review of the incentives faced by programme units when reporting on results and the investments in processes of innovation, learning and adaptation that are required if long-term capacity development is to be effectively supported. 

3

Recommendation 3. The new Strategic Plan should emphasize the priority of support at the country level and explicitly recognize that no matter how good the work is at the centre, it is at the country level where the difference is being made.

Effective support to strengthening country offices must be the priority of UNDP central and regional support. Taking into account context variability, the generation of coherent policy support needs to be anchored at the regional level. For its central research function, in the spirit of One UN and in order to prevent duplication of work, UNDP should work more closely with the United Nations Department of Social and Economic Affairs and the United Nations University. Moreover, to overcome the tendency to work in silos, sustainable human development as a unifying principle should be explicitly at the centre of country-level programming. Practical support and guidance need to be provided to ensure that this becomes a reality.

4

Recommendation 4. When assessing performance, the new Strategic Plan needs to take the country programme as the unit of analysis. This will allow a more realistic and meaningful assessment, taking into account the country-level context.

In complex development situations and areas of intervention, the role of context is vital in determining performance. The contexts and areas in which UNDP works can only be fully internalized if performance assessment is made at the country level (i.e. the country programme). Taking the country level as the unit of analysis for performance in terms of contribution to outcomes would still require country office reporting on approaches and outputs for aggregation at the corporate level. Moreover, a focus on the country programme would allow the monitoring and evaluation systems to be vertically integrated so that selected independent evaluations could validate (or not) the performance assessment made through the monitoring system.

1. Recommendation:

Recommendation 1. The new Strategic Plan needs to be clear about the direction it wants UNDP to take, and UNDP management needs to ensure that adequate tools are put in place to support and monitor implementation of the strategies and priorities contained in the plan.

A key conclusion was that efforts to implement priorities and approaches were mixed and not carefully monitored from the start of the Plan. Even though the role of the Plan is to provide direction, the role of UNDP management is to operationalize and implement this direction through ensuring clarity of concepts and approaches, developing tools such as guidance, and monitoring its implementation. Yet, the examples of capacity development and gender mainstreaming suggest that guidance and monitoring are not enough to ensure implementation of the approaches identified in the Strategic Plan. Even combined with strong leadership the central actions are necessary but not sufficient to guarantee implementation. 

Management Response: [Added: 2013/05/20] [Last Updated: 2020/07/04]

UNDP agrees with this recommendation. The new strategic plan, 2014-2017, will be clearer about the future direction of UNDP. The organization recognizes the need to be more systematic in ensuring that adequate incentives and capacities are in place so that its priorities and approaches are more effectively reflected and implemented in all its programmes. As noted in the evaluation, UNDP made great efforts in gender and capacity development and, having learned from those efforts, is in a stronger position to support the use of its other approaches and priorities systematically over the coming years. UNDP is intent on ensuring a smoother, more managed transition to the new strategic plan, which includes developing stronger incentives to support the deployment of its approaches.

Key Actions:

Key Action Responsible DueDate Status Comments Documents
1.1 The new strategic plan provides greater clarity and focus for the future work of UNDP.
[Added: 2013/09/25]
Executive Office 2013/09 Completed The new strategic plan was approved by the Executive Board in September 2013.
1.2 UNDP develops adequate tools which enable the approaches outlined in the plan to be used fully in its programmes, and which support and monitor implementation of the strategic plan.
[Added: 2013/09/25] [Last Updated: 2014/02/27]
Bureau for Development Policy (BDP), Executive Office 2014/01 Completed UNDP has developed the SP IRRF, approved by the EB in September 2013, and accompanying tools including theories of change, indicator methodological notes, and guides to alignment, linking and baseline and target setting. UNDP has revised its CPD template (approved by the EB in January 2014) and related quality assurance processes and tools, and has updated its corporate scorecard to incorporate clear monitoring and incentive measures.
1.3 UNDP provides targeted support to country offices in order to support their capacity to transition to the new strategic plan, including through COSI.
[Added: 2013/09/25] [Last Updated: 2018/01/26]
Regional bureaux, BDP, Bureau for Crisis Prevention and Recovery (BCPR), Executive Office 2014/02 Completed RBx each have regional roadmaps and have included on average 5% of regional programmes for support to M&E/RBM; regional workshops and targeted CO support for SP alignment/ transition is being rolled out with BDP/BCPR support, including through ?socializing? theories of change and indicator methodological notes 2018 update: COSI benchmarks are analyzed based on ROAR data and reported on to the Executive Board. UNDP had a partnership with UNV, with an Evaluaiton Specialist Volunteer deployed to RBAS region during 2017. COSI benchmarks was one of the tasks the UNV followed-up on. UNDP and the IEO conducted training workshops for all five regions during 2016 and 2017, COSI benchmarks was among the items discussed. History
2. Recommendation:

Recommendation 2. The new Strategic Plan needs to explicitly address the trade-offs that occur as a result of the UNDP business model.

UNDP’s comparative strengths are not just in parts of its focus areas but, inter alia, in its longterm commitment and in general its position as a trusted partner. These are elements of an underlying ‘theory’ of how UNDP works that is operationalized through a business model. The trade-offs faced by UNDP are inherent in any business model and need to be made transparent, with decisions made by the Executive Board as to UNDP’s position. Specifically, agreement must be reached on the trade-offs in two areas: first, between national ownership and organizational priorities with clear organization, approaches, strategies and focus combined with guidance for programme units when faced with a potential trade-off; and second, between addressing long-term capacity development needs and short-term results, which requires a review of the incentives faced by programme units when reporting on results and the investments in processes of innovation, learning and adaptation that are required if long-term capacity development is to be effectively supported. 

Management Response: [Added: 2013/05/20] [Last Updated: 2020/07/04]

UNDP agrees with this recommendation. UNDP organizational results are defined at an organizational level and determined at the national level based on national demand for UNDP engagement in specific areas. Improved business analytics and pipeline management will further improve the ability of UNDP to ensure that there is a convergence between its priorities and demand for its support in key areas. Learning lessons from implementation of the current strategic plan, UNDP is intent on ensuring a smoother, more managed transition to the new strategic plan, including more clear guidance to its programmes on managing trade-offs that emerge from the approaches and priorities laid out in the strategic plan. Issues related to the UNDP funding model and associated trade-offs will continue to feature in performance monitoring and reporting to the Executive Board. UNDP will also use the new integrated results and resources framework to underpin a more robust reflection of the relationship between results and resources.

Key Actions:

Key Action Responsible DueDate Status Comments Documents
2.1 Reporting under the new strategic plan provides more robust analysis of the trade-offs that occur as a result of the UNDP business model, including the relationship between national demand and organizational priorities; between transformational and short-term results; and between funding sources and the pursuit of durability in the long term.
[Added: 2013/09/25] [Last Updated: 2018/01/26]
Executive Office 2014/02 Completed New SP/IRRF approved by EB in September 2013. Reporting will occur each year at the annual session of the board. The previous SP Annual Reports of the Administrator have seen a stronger focus on national demand and organizational priorities and the attempt to analyse the results-resources relation. Action completed, with activities in this area continuing under the recommendations 14.2 and 16.2 in the evaluation of the SP 2014-2017. History
2.2 Stronger UNDP performance management and business analytics enable more accurate forecasting of business model implications for achieving results under the new strategic plan.
[Added: 2013/09/25] [Last Updated: 2018/01/26]
Executive Office 2014/12 Completed This has been work in process throughout the implementation of the previous SP. Completed, with activities continuing in the SP 2018-2021, based on SP 2014-2017 evaluation recommendations. History
3. Recommendation:

Recommendation 3. The new Strategic Plan should emphasize the priority of support at the country level and explicitly recognize that no matter how good the work is at the centre, it is at the country level where the difference is being made.

Effective support to strengthening country offices must be the priority of UNDP central and regional support. Taking into account context variability, the generation of coherent policy support needs to be anchored at the regional level. For its central research function, in the spirit of One UN and in order to prevent duplication of work, UNDP should work more closely with the United Nations Department of Social and Economic Affairs and the United Nations University. Moreover, to overcome the tendency to work in silos, sustainable human development as a unifying principle should be explicitly at the centre of country-level programming. Practical support and guidance need to be provided to ensure that this becomes a reality.

Management Response: [Added: 2013/05/20] [Last Updated: 2020/07/04]

UNDP acts at the global, regional and country levels, and prioritizes support at the country level where development progress affects people?s lives. UNDP recognizes the importance of the global and regional policy functions inter alia to support country offices and to benefit all partners through shared knowledge and learning. UNDP accepts that the agenda for organizational change now needs to focus on country offices. The 2013 annual business plan includes important measures to capitalize on more than three years of intensive effort to improve the organization?s results-based management culture, and to strengthen country office performance and learning through greater focus, strategic priority-setting, ongoing efforts such as COSI and refining UNDP business models.

Key Actions:

Key Action Responsible DueDate Status Comments Documents
3.1 The new strategic plan clearly emphasizes that support for the country level is the priority of UNDP, and that it is at the country level that UNDP makes the greatest difference.
[Added: 2013/09/25]
Executive Office 2013/09 Completed The new strategic plan was approved by the Executive Board in September 2013.
3.2 See key action 1.3 above.
[Added: 2013/09/30] [Last Updated: 2018/01/26]
Regional bureaux, BDP, BCPR, Executive Office 2013/09 Completed 2018 update: COSI benchmarks are analyzed based on ROAR data and reported on to the Executive Board. UNDP had a partnership with UNV, with an Evaluaiton Specialist Volunteer deployed to RBAS region during 2017. COSI benchmarks was one of the tasks the UNV followed-up on. UNDP and the IEO conducted training workshops for all five regions during 2016 and 2017, COSI benchmarks was among the items discussed. History
3.3 UNDP?s future global programme leverages the global policy function to optimize the policy mix in order to service countries and to promote knowledge sharing and South-South exchange.
[Added: 2013/09/30] [Last Updated: 2014/02/27]
BDP, BCPR, Executive Office, regional bureaux 2014/01 Completed UNDP global programme 2014-2017 was approved by the EB in January 2014
4. Recommendation:

Recommendation 4. When assessing performance, the new Strategic Plan needs to take the country programme as the unit of analysis. This will allow a more realistic and meaningful assessment, taking into account the country-level context.

In complex development situations and areas of intervention, the role of context is vital in determining performance. The contexts and areas in which UNDP works can only be fully internalized if performance assessment is made at the country level (i.e. the country programme). Taking the country level as the unit of analysis for performance in terms of contribution to outcomes would still require country office reporting on approaches and outputs for aggregation at the corporate level. Moreover, a focus on the country programme would allow the monitoring and evaluation systems to be vertically integrated so that selected independent evaluations could validate (or not) the performance assessment made through the monitoring system.

Management Response: [Added: 2013/05/20] [Last Updated: 2020/07/04]

UNDP understands the recommendation, and agrees that there is the need to establish clearer performance criteria as part of a stronger performance monitoring system. UNDP recognizes the central role of the country programme as the main ?unit of analysis? in managing the organization?s performance; UNDP currently monitors performance against country programme outcome indicators and country management indicators as an integral part of its performance monitoring and reporting architecture. The challenge of aggregating data to analyse organizational performance, while understanding the unique country context within which results are generated, is one that is faced by all large development actors. UNDP has made notable progress in its use of performance data since 2011, and continues to build its own capacity to produce high quality, contextualized and nuanced performance metrics that go beyond compliance-oriented reporting to provide a meaningful picture of what is working and what is not.

Key Actions:

Key Action Responsible DueDate Status Comments Documents
4.1 The new strategic plan has a results framework with SMART indicators at every result level that can be incorporated into country-level programming, including new country programme documents and results and resources frameworks.
[Added: 2013/09/30] [Last Updated: 2018/01/26]
Executive Office, BDP, regional bureaux 2014/02 Completed The new strategic plan/ Integrated results and resources framework was approved by the Executive Board in September 2013 and will be fully populated with baselines and targets by June 2014. New CPD format approved by the EB in January 2014. Has been long implemented with the SP 2014-2017 IRRF. It can be demonstrated that several CPD have adopted the SP IRRF indicators. History
4.2 UNDP annual reports ensure that analysis of UNDP performance is founded on data and analysis of country level performance, and is reflective of country context.
[Added: 2013/09/30] [Last Updated: 2018/01/26]
Executive Office, regional bureaux 2013/09 Completed Aanalysis of country level performance drawn from the country offices’ result oriented annual reports informed UNDP annual reports of the administrator History
4.3 UNDP develops a stronger and more integrated performance monitoring system that provides a stronger evidence base for better evaluations.
[Added: 2013/09/30] [Last Updated: 2018/01/26]
Executive Office, Bureau of Management 2014/10 Completed This has been work in process throughout the implementation of the previous SP. Completed, with activities continuing in the SP 2018-2021, based on SP 2014-2017 evaluation recommendations. History

Latest Evaluations

Contact us

1 UN Plaza
DC1-20th Floor
New York, NY 10017
Tel. +1 646 781 4200
Fax. +1 646 781 4213
erc.support@undp.org