Evaluation of Unted Nations Development Assistance Framework (2010 - 2014)

Report Cover Image
Evaluation Plan:
2010-2014, Bosnia and Hercegovina
Evaluation Type:
UNDAF
Planned End Date:
01/2014
Completion Date:
06/2013
Status:
Completed
Management Response:
Yes
Evaluation Budget(US $):
13,314

Share

Document Type Language Size Status Downloads
Download document UNDAF and UNDP CPAP 2010-2014 Evaluation ToR - FINAL.pdf tor English 342.20 KB Posted 286
Download document UNDAF BiH 2010-2014 Evaluation Report - Final.pdf report English 3761.31 KB Posted 775
Download document Annex 12 UNDAF Financial Monitoring Matrix format (2010 - 2014) (31-5-13).xlsx related-document English 87.86 KB Posted 313
Title Evaluation of Unted Nations Development Assistance Framework (2010 - 2014)
Atlas Project Number:
Evaluation Plan: 2010-2014, Bosnia and Hercegovina
Evaluation Type: UNDAF
Status: Completed
Completion Date: 06/2013
Planned End Date: 01/2014
Management Response: Yes
Corporate Outcome and Output (UNDP Strategic Plan 2014-2017)
Evaluation Budget(US $): 13,314
Source of Funding: RCO budget
Joint Programme: No
Joint Evaluation: No
Evaluation Team members:
Name Title Email Nationality
Michael Askwith External Evaluator
GEF Evaluation: No
Key Stakeholders: State and Entity Governments
Countries: BOSNIA AND HERCEGOVINA
Comments:

Final evaluation

Lessons
Findings
Recommendations
1 Summary observation on Design: The UNDAF was quite well designed in terms of measurable outputs and indicators. But the wording of outcome and output statements was on the whole excessively long and imprecise, presenting particular challenges for the design of agency support and monitoring. Furthermore themes of these outcomes and outputs were not always clear due to the multiplicity of goals in each one, resulting in the need to prepare short summary thematic statements for each one, for ease of reference (See Annex 4). A clear thematic summary should have been introduced at the design stage, for the sake of clarity. The Evaluation found that many of the individual output lines were not defined in clear thematic or sectoral terms with the result that it was difficult for agencies to place their support under a relevant line, and which in any case may have differed from those used in agency CPs. The lack of clear sectoral lines such as for the social sectors (health, education, social protection) or for the productive sectors (agriculture, rural development, industry, services, etc.) meant that the entry points for the relevant agencies was confusing. Finally, it was felt that certain substantive areas were misplaced (e.g. water with biodiversity (3.2.3) and HIV/AIDs under 4. Human security.
2 Summary observation on Implementation: Despite the questionable design of the Results Matrix, the main thematic thrusts of the UNDAF in the form of outcomes and outputs were largely supported by UN agencies. The up-dating of the performance rating exercise carried out for the 2010-2011 PR (see Annex 13) led to an assessment that by the end of 2012 40% of indicator targets had been achieved, 30% were on track, 16% needed further clarification, 10% had not been achieved and 4% were no longer relevant. After three years of support and two more still to go, such results are commendable, with a good probability that during the remaining period of the UNDAF a high proportion of the targets will be achieved.
3 Summary observation on Management: The management arrangements foreseen in the UNDAF document, particularly the creation of an UNDAF Steering Committee, made up of members of the UNCT and of Working Groups, were disbanded by the UNCT as not providing sufficient value added vis-à-vis the work required. Instead, ad hoc issue-specific technical working groups were established. It is felt that this withdrawal of a formal oversight and management structure deprived the UNDAF of a necessary tool to facilitate its management and monitoring.
4 Summary observation on Monitoring: Notwithstanding the positive step of preparing the 2010/2011 Progress Report and Annex, and 2012 Annual Report, the presentation of the narrative information received from agencies was not linked to outputs, nor was evidence shown of its impact on the outcome, thus making monitoring of the results extremely challenging.
5 Summary of Recommendations - Design: Use of the 'Light' version of the UNDAF Guidelines, while retaining the levels of Outcome and Output in the Results Matrix, but with fewer indicators; The UNDAF would be made up of coordinated support to selected thematic and sub-thematic areas, organized according to a common model and format in order to promote joint programming in all areas, with full complementarity and coordination between UN agencies and national partners; A regrouping of output lines in order to bring together UN system support according to agreed criteria (preferably along sectoral and cross-cutting lines); Due account should be taken of sectoral and other classifications used by government and other donor partners, so as facilitate coordination.
6 Summary of Recommendations - Implementation: This would require the use of appropriate tools, and could include: (i) Use of thematic joint programming documents, which would be prepared according to a common yet flexible, format. Such JPDs would present a situation analysis of the thematic area being addressed, a summary of national policy frameworks designed to address the relevant issues; a summary of past UN and other cooperation in this area; a description of future needs in terms of UN system support and financial resources to be mobilized, and management arrangements for implementation. (ii) Thematic/JPD annual work plans covering the entire theme, to be supported by agencies using mutually agreed modalities (joint project, separate projects and funding, etc.)
7 Summary of Recommendations - Management: Three layers of oversight and management should be established; (i) The restoration of the UNDAF Steering Committee, made up of UNCT members; (ii) The restoration of Outcome area groups but with a suggested change of name and concept to 'Strategic Working Groups' (SWG), headed by a head of agency, and responsible for several TWGs; (iii) The institutionalization of Thematic Results Groups (TRG) responsible for the design, facilitation and monitoring of joint UN system support to each thematic area. (v) UN Communications, so as to provide material which could be used for wider information services.
8 Summary of Recommendations - Monitoring: UNDAF monitoring would take place on thematic lines, managed by the TWGs, which would complete reports based on the annual work plan. The format of these reports would be designed to serve a number of different levels of user, namely: (i) Project management, so as to be able to take corrective action in a timely way, as necessary; (ii) UN agencies, so that they can take account of the results achieved with their support; (iii) UNCT, in its capacity of an UNDAF Steering Committee, and the Heads of Agency chairs of Strategic Objectives Groups, in order to inform the UNCT and SWG chairs on the results achieved in each thematic area; (iv) RCO Secretariat, so as to develop an UNDAF data base of (a) Substantive results by UNDAF outcomes, outputs, indicators and targets; (b) Agency support to each outcome and output by project; (c) Financial information of resources targeted, mobilized and delivered; This information should be provided by results groups and agencies, with the help of agency M & E and finance officers.
1. Recommendation: Summary observation on Design: The UNDAF was quite well designed in terms of measurable outputs and indicators. But the wording of outcome and output statements was on the whole excessively long and imprecise, presenting particular challenges for the design of agency support and monitoring. Furthermore themes of these outcomes and outputs were not always clear due to the multiplicity of goals in each one, resulting in the need to prepare short summary thematic statements for each one, for ease of reference (See Annex 4). A clear thematic summary should have been introduced at the design stage, for the sake of clarity. The Evaluation found that many of the individual output lines were not defined in clear thematic or sectoral terms with the result that it was difficult for agencies to place their support under a relevant line, and which in any case may have differed from those used in agency CPs. The lack of clear sectoral lines such as for the social sectors (health, education, social protection) or for the productive sectors (agriculture, rural development, industry, services, etc.) meant that the entry points for the relevant agencies was confusing. Finally, it was felt that certain substantive areas were misplaced (e.g. water with biodiversity (3.2.3) and HIV/AIDs under 4. Human security.
Management Response: [Added: 2014/12/22] [Last Updated: 2014/12/22]

UNCT with the new UNDAF/One Programme 2015-2019 fully addressed this recommendation. Outcome definitions are much tighter and achievable. Health, education and social protection are under Social Inclusion development pillar, agriculture, rural development, services are under Sustainable and Equitable Development and Employment pillar.

Key Actions:

Key Action Responsible DueDate Status Comments Documents
RBM refreshment course for the UNCT staff engaged in UNDAF development.
[Added: 2014/12/22]
SPR Facilitator/RC Office 2013/10 Completed
Formation of the M&E Result Group.
[Added: 2014/12/22]
UNCT 2014/10 Completed
RBM and M&E QA by the RC M&E Specialist.
[Added: 2014/12/22]
RC Office 2014/12 Completed
RBM and M&E QA by the members of the M&E Result Group.
[Added: 2014/12/22]
M&E Group. 2014/12 Completed
2. Recommendation: Summary observation on Implementation: Despite the questionable design of the Results Matrix, the main thematic thrusts of the UNDAF in the form of outcomes and outputs were largely supported by UN agencies. The up-dating of the performance rating exercise carried out for the 2010-2011 PR (see Annex 13) led to an assessment that by the end of 2012 40% of indicator targets had been achieved, 30% were on track, 16% needed further clarification, 10% had not been achieved and 4% were no longer relevant. After three years of support and two more still to go, such results are commendable, with a good probability that during the remaining period of the UNDAF a high proportion of the targets will be achieved.
Management Response: [Added: 2014/12/22] [Last Updated: 2014/12/22]

Full 5-year review of the UNDAF 2010-2014 will be conducted in the first half of 2015. In financial terms, UNCT over delivered in comparison to commitment made in the UNDAF and it is anticipated that the delivery and implementation is in line with commitments made.

Key Actions:

Key Action Responsible DueDate Status Comments Documents
UNDAF 2010-2014 5 year review
[Added: 2014/12/22] [Last Updated: 2016/12/20]
UNCT/RC Office 2015/04 Completed History
3. Recommendation: Summary observation on Management: The management arrangements foreseen in the UNDAF document, particularly the creation of an UNDAF Steering Committee, made up of members of the UNCT and of Working Groups, were disbanded by the UNCT as not providing sufficient value added vis-à-vis the work required. Instead, ad hoc issue-specific technical working groups were established. It is felt that this withdrawal of a formal oversight and management structure deprived the UNDAF of a necessary tool to facilitate its management and monitoring.
Management Response: [Added: 2014/12/22] [Last Updated: 2014/12/22]

The UNCT for the new UNDAF/One Programme cycle has established four programmatic Result Groups (given that the programme has 4 development pillars) and three operational/supporting Results Groups, M&E Group, Communications Group and OMT. Responsibility for monitoring and reporting on progress towards achieving UNDAF outcomes and Joint Work Plans? outputs sits with the 4 Result Groups.

Key Actions:

Key Action Responsible DueDate Status Comments Documents
Formation of Result Groups.
[Added: 2014/12/22]
UNCT 2014/10 Completed
4. Recommendation: Summary observation on Monitoring: Notwithstanding the positive step of preparing the 2010/2011 Progress Report and Annex, and 2012 Annual Report, the presentation of the narrative information received from agencies was not linked to outputs, nor was evidence shown of its impact on the outcome, thus making monitoring of the results extremely challenging.
Management Response: [Added: 2014/12/22] [Last Updated: 2014/12/22]

The five year review will bear this recommendation in mind. The new UNDAF/One Programme documents are tightly aligned and the UNCT is pending from the HQ to receive adequate UN Country Results format for annual reporting.

Key Actions:

Key Action Responsible DueDate Status Comments Documents
New approach with the five-year UNDAF review.
[Added: 2014/12/22] [Last Updated: 2016/12/20]
UNCT/RC Office 2015/04 Completed History
5. Recommendation: Summary of Recommendations - Design: Use of the 'Light' version of the UNDAF Guidelines, while retaining the levels of Outcome and Output in the Results Matrix, but with fewer indicators; The UNDAF would be made up of coordinated support to selected thematic and sub-thematic areas, organized according to a common model and format in order to promote joint programming in all areas, with full complementarity and coordination between UN agencies and national partners; A regrouping of output lines in order to bring together UN system support according to agreed criteria (preferably along sectoral and cross-cutting lines); Due account should be taken of sectoral and other classifications used by government and other donor partners, so as facilitate coordination.
Management Response: [Added: 2014/12/22] [Last Updated: 2014/12/22]

The CO used the UNDAF light option when creating the new One Programme. UNDAF is elaborated at the Outcome level and JWPs have output and activity level. The UNCT is striving to reduce number of outputs and indicators to a manageable level, still fine-tuning the final version of the JWPs.

Key Actions:

Key Action Responsible DueDate Status Comments Documents
Fully following One Programme SoPs and committing to a manageable, simplified matrix with fewer outputs and indicators.
[Added: 2014/12/22]
UNCT/RC Office 2014/12 Completed
6. Recommendation: Summary of Recommendations - Implementation: This would require the use of appropriate tools, and could include: (i) Use of thematic joint programming documents, which would be prepared according to a common yet flexible, format. Such JPDs would present a situation analysis of the thematic area being addressed, a summary of national policy frameworks designed to address the relevant issues; a summary of past UN and other cooperation in this area; a description of future needs in terms of UN system support and financial resources to be mobilized, and management arrangements for implementation. (ii) Thematic/JPD annual work plans covering the entire theme, to be supported by agencies using mutually agreed modalities (joint project, separate projects and funding, etc.)
Management Response: [Added: 2014/12/22] [Last Updated: 2014/12/22]

This recommendation is in line with the new SoPs and it is taken fully on board.

Key Actions:

Key Action Responsible DueDate Status Comments Documents
Application of SoPs, adaptation of provided templates.
[Added: 2014/12/22]
UNCT/RC Office 2014/10 Completed
7. Recommendation: Summary of Recommendations - Management: Three layers of oversight and management should be established; (i) The restoration of the UNDAF Steering Committee, made up of UNCT members; (ii) The restoration of Outcome area groups but with a suggested change of name and concept to 'Strategic Working Groups' (SWG), headed by a head of agency, and responsible for several TWGs; (iii) The institutionalization of Thematic Results Groups (TRG) responsible for the design, facilitation and monitoring of joint UN system support to each thematic area. (v) UN Communications, so as to provide material which could be used for wider information services.
Management Response: [Added: 2014/12/22] [Last Updated: 2014/12/22]

This recommendation has been partly taken on board. UNCT formed 7 official Result Groups (4 programmatic and 3 operational/supportive) chaired at the HoA level. Given that each development pillar is complex, each Results Group formed 'sub-teams' which focused on one thematic issue or a specific outcome. These groups are not fixed, nor formal, they exist on an 'as needed' basis.

Key Actions:

Key Action Responsible DueDate Status Comments Documents
Formation of Result Groups.
[Added: 2014/12/22]
UNCT 2014/10 Completed
8. Recommendation: Summary of Recommendations - Monitoring: UNDAF monitoring would take place on thematic lines, managed by the TWGs, which would complete reports based on the annual work plan. The format of these reports would be designed to serve a number of different levels of user, namely: (i) Project management, so as to be able to take corrective action in a timely way, as necessary; (ii) UN agencies, so that they can take account of the results achieved with their support; (iii) UNCT, in its capacity of an UNDAF Steering Committee, and the Heads of Agency chairs of Strategic Objectives Groups, in order to inform the UNCT and SWG chairs on the results achieved in each thematic area; (iv) RCO Secretariat, so as to develop an UNDAF data base of (a) Substantive results by UNDAF outcomes, outputs, indicators and targets; (b) Agency support to each outcome and output by project; (c) Financial information of resources targeted, mobilized and delivered; This information should be provided by results groups and agencies, with the help of agency M & E and finance officers.
Management Response: [Added: 2014/12/22] [Last Updated: 2014/12/22]

This recommendation is taken on board, with the exception of the UNDAF database (yet to be discussed within the M&E Results Group) and the role of the RCO as Secretariat. The Secretariats are being provided to the Results Groups by the Lead Agencies that chair them. The RCO is responsible for overall quality assurance, technical support, coordination and compilation of the work of the Results Groups.

Key Actions:

Key Action Responsible DueDate Status Comments Documents
Formation of Result Groups.
[Added: 2014/12/22]
UNCT 2014/10 Completed

Latest Evaluations

Contact us

220 East 42nd Street
20th Floor
New York, NY 10017
Tel. +1 646 781 4200
Fax. +1 646 781 4213
erc.support@undp.org