Review of UNV Facility for Evaluation

Report Cover Image
Evaluation Plan:
2009-2011, UNV
Evaluation Type:
Thematic
Planned End Date:
12/2009
Completion Date:
07/2010
Status:
Completed
Management Response:
Yes
Evaluation Budget(US $):
30,000

Share

Document Type Language Size Status Downloads
Download document Review of UNV Facility for Evaluation - Final doc.pdf report English 926.67 KB Posted 1651
Download document TOR FACE Review - FINAL Revised Dates 30Sep09.docx tor English 380.62 KB Posted 827
Title Review of UNV Facility for Evaluation
Atlas Project Number:
Evaluation Plan: 2009-2011, UNV
Evaluation Type: Thematic
Status: Completed
Completion Date: 07/2010
Planned End Date: 12/2009
Management Response: Yes
Focus Area:
  • 1. Cross-cutting Development Issue
Corporate Outcome and Output (UNDP Strategic Plan 2014-2017)
  • 1. UNDP meets aid effectiveness standards
Evaluation Budget(US $): 30,000
Source of Funding:
Joint Programme: No
Joint Evaluation: No
Evaluation Team members:
Name Title Email Nationality
Team Leader
GEF Evaluation: No
Key Stakeholders:
Lessons
Findings
Recommendations
1 Assuming that the second Evaluation Specialist position is unfrozen and filled, the UNV EC should open a third Evaluation Specialist position, funded using FACE resources, as initially agreed in late 2008. The Head of the EU should then move to fill this position.
2 Free up capacity within the EU by moving more tasks to other units or sections in UNV as soon as possible. The most obvious candidate would be the analysis of the VRS data, whichclearly should be a responsibility of programme management. The revision of the VRS questionnaires to ensure that they provide the information needed for measuring UNV performance at corporate and country levels, reporting against the Corporate Plan, and corporate reporting in general are three other candidates and should be responsibilities of the new ?strategy and performance? section.
3 UNV implement a number of SVF projects as true pilots that aim to verify the programme logic for why UNV interventions and support will make a contribution. This would require designing the projects to define (using the methodology to assess the contribution of volunteering to development) and test the programme logic, which would mean identifying suitable possible projects at or before the concept note stage, gaining agreement of partners that the project include a pilot component, the use of counterfactuals, and investment in an evaluation process that might last for several years.
4 It is recommended that the EC consider using a formative evaluation to clarify where the opportunities for adding value to the volunteerism agenda lie for UNV, before a decision is taken and clearly communicated on what the corporate view is on where UNV will seek to add value.
5 EU should continue to prioritise support to enhancing learning from the project evaluations and should seek to ensure that key lessons, based on analysis, are concisely synthesised.
6 In future, the EU?s provisional work plan should be developed as part of the organisation?s formal planning and budgeting systems. The FACE Project Board should also be used as a forum to enhance discussion on what the work programme and priorities for the EU should be. The rationale for the final prioritisation of the work plan should also be communicated back to wider management within the organisation
7 A project document for a new phase of FACE should be drafted and approved.
8 UNV has never systematically and rigorously evaluated its core business processes ? identification of, recruitment, placement and support for, IUNVs. Evaluation of these basic processes should be considered by the EC.
1. Recommendation: Assuming that the second Evaluation Specialist position is unfrozen and filled, the UNV EC should open a third Evaluation Specialist position, funded using FACE resources, as initially agreed in late 2008. The Head of the EU should then move to fill this position.
Management Response: [Added: 2010/07/08]

As a result of the Changing Together (CT) process, tasks that are not strictly related to evaluation have moved to other units and freed up time in the Evaluation Unit team to implement more evaluations. It was therefore decided that there will be no third Evaluation Specialist post. Recruitment for the second Evaluation Specialist and Chief of Unit is underway.

Key Actions:

Key Action Responsible DueDate Status Comments Documents
Finalize recruitment for vacant posts in the Evaluation Unit Formalize changes in responsibility for the following tasks: - preparation of analysis reports and maintenance of VRS (new lead unit: QAU) -preparation of UNV biennial report to the Executive Board (new lead unit: OEC)
[Added: 2010/07/08]
HR, CMT 2010/06 Completed
2. Recommendation: Free up capacity within the EU by moving more tasks to other units or sections in UNV as soon as possible. The most obvious candidate would be the analysis of the VRS data, whichclearly should be a responsibility of programme management. The revision of the VRS questionnaires to ensure that they provide the information needed for measuring UNV performance at corporate and country levels, reporting against the Corporate Plan, and corporate reporting in general are three other candidates and should be responsibilities of the new ?strategy and performance? section.
Management Response: [Added: 2010/07/08]

The CT process results have freed up capacity within the unit by moving the following tasks to other units: management of VRS and preparation of analysis reports, corporate reporting (preparation of EB Report). The EU will still have an advisory role in a future revision of the VRS, but this activity will be led by the Quality Assurance Unit.

Key Actions:

Key Action Responsible DueDate Status Comments Documents
Revise TORs for all UNV staff members, including responsibilities, according to the agreements reached as a result of CT.
[Added: 2010/07/08]
CMT 2010/01 Completed
3. Recommendation: UNV implement a number of SVF projects as true pilots that aim to verify the programme logic for why UNV interventions and support will make a contribution. This would require designing the projects to define (using the methodology to assess the contribution of volunteering to development) and test the programme logic, which would mean identifying suitable possible projects at or before the concept note stage, gaining agreement of partners that the project include a pilot component, the use of counterfactuals, and investment in an evaluation process that might last for several years.
Management Response: [Added: 2010/07/08]

SVF is already used for pilot initiatives in UNV. The planning of projects (and subsequent evaluations) with control and treatment groups, as suggested by the consultant?s recommendation, is not applicable to the reality of UNV. On the other hand, assessing the added value of volunteerism is a very important issue for UNV, and efforts are being made in this direction. The finalization, dissemination and use of the ?methodology to assess the contribution of volunteering to development?, as well as other on-going initiatives such as assessing volunteer contribution in CBA (community-based adaptation to climate change), are significant steps towards measuring the contribution of volunteering.

Key Actions:

Key Action Responsible DueDate Status Comments Documents
Finalize and publish the handbook methodology to assess the contribution of volunteering to development
[Added: 2010/07/08] [Last Updated: 2016/12/30]
EU, CMT to approve 2010/09 Completed The Handbook for Assessing the Contribution of Volunteering to Development was completed and published. Following publication, trainings and workshops on the V-methodology were conducted. The handbook is also publicly available. History
Disseminate the handbook methodology to assess the contribution of volunteering to development
[Added: 2010/07/08]
EU No due date No deadline established
Ensure the use of the handbook and/or other applicable methodologies to assess the contribution of volunteering to development in new and existing UNV projects
[Added: 2010/07/08]
EU VPMG No due date No deadline established
4. Recommendation: It is recommended that the EC consider using a formative evaluation to clarify where the opportunities for adding value to the volunteerism agenda lie for UNV, before a decision is taken and clearly communicated on what the corporate view is on where UNV will seek to add value.
Management Response: [Added: 2010/07/08]

The themes and areas for evaluations to be carried out by UNV over the next two years will be discussed as part of the preparation of the UNV evaluation plan. On the other hand, it is important to consider that other initiatives (not only evaluations) currently taking place in UNV can generate and add information on the added value of volunteerism, which can be used in decision-making. Examples of other initiatives generating background studies and information are the development of the UNV programme strategy, and the preparations for IYV+10 (including a major report on the State of the World Volunteerism)

Key Actions:

Key Action Responsible DueDate Status Comments Documents
Prepare the UNV evaluation plan, with evaluations to be carried out over the next two years (see recommendation 6 below)
[Added: 2010/07/08] [Last Updated: 2010/08/17]
EU CMT to approve 2010/06 Completed The CMT had an initial discussion on what thematic/ corporate evaluations UNV should focus on for the next year. In addition to the already initiated evaluation in the area of volunteerism and peacekeeping, the CMT decided that UNV will conduct a second thematic evaluation during the next year, focusing on IVD. Preparations for this evaluation will start as soon as the second evaluation specialist is recruited. In the evaluation plan, project evaluations foreseen for the next two years will also be included
5. Recommendation: EU should continue to prioritise support to enhancing learning from the project evaluations and should seek to ensure that key lessons, based on analysis, are concisely synthesised.
Management Response: [Added: 2010/07/08]

The EU will continue commissioning the preparation of synthesis reports and discussion on lessons within UNV.

Key Actions:

Key Action Responsible DueDate Status Comments Documents
Finalize the 2000 ? 2009 synthesis report and carry out learning meeting with UNV staff
[Added: 2010/07/08]
EU 2010/06 Completed The report has been completed, presented to and discussed with UNV HQ. The final version will be shared within the organization through the knowledge platform.
6. Recommendation: In future, the EU?s provisional work plan should be developed as part of the organisation?s formal planning and budgeting systems. The FACE Project Board should also be used as a forum to enhance discussion on what the work programme and priorities for the EU should be. The rationale for the final prioritisation of the work plan should also be communicated back to wider management within the organisation
Management Response: [Added: 2010/07/08]

The EU workplan will be prepared in accordance with the UNV corporate planning exercises. The workplan will focus the unit?s activities, considering the key priorities identified by UNV for the next biennium, and financial allocation will be clearly indicated for the unit?s activities. Subsequently, an evaluation plan will be prepared, including the evaluations to be carried out over the next two years. It is also important to highlight that the management responsibility for setting directions to the evaluation unit lies with the CMT, instead of a project board. The project board for the future FACE project will have a more technical advisory role.

Key Actions:

Key Action Responsible DueDate Status Comments Documents
Finalize the EU workplan (including budget), in accordance with UNV?s key priorities
[Added: 2010/07/08]
EU CMT to approve 2010/04 Completed
Prepare the UNV evaluation plan, with evaluations to be carried out over the next two years
[Added: 2010/07/08] [Last Updated: 2010/08/17]
EU CMT to approve 2010/06 Completed
7. Recommendation: A project document for a new phase of FACE should be drafted and approved.
Management Response: [Added: 2010/07/08]

A new project document will be prepared.

Key Actions:

Key Action Responsible DueDate Status Comments Documents
Prepare the new FACE project document and submit to PAC
[Added: 2010/07/08] [Last Updated: 2016/12/30]
EU PAC to approve 2010/10 Completed Currently a FACE project does not exist and there is not FACE funding. In addition, the evaluation function in UNV has undergone several changes since this evaluation. Currently the function has been re-established and a plan for increasing evaluation capacity in the organization has been developed. The recommendations from this evaluation will be considered as part of the way forward for evaluation in UNV. History
8. Recommendation: UNV has never systematically and rigorously evaluated its core business processes ? identification of, recruitment, placement and support for, IUNVs. Evaluation of these basic processes should be considered by the EC.
Management Response: [Added: 2010/07/08]

While no formal evaluation has taken place, elements of this recommendation have been addressed by a full-fledged business process review in 2009, as part of the Changing Together process. As a result, significant changes have already taken place, such as the creation of a specialized volunteer recruitment unit. Other aspects of UNV?s business processes will also be addressed by the UNV audit in 2010. Once the audit is completed, the CMT will review whether more still needs to be done to assess UNV?s core business. If this is the case, an evaluation will be included in a future UNV evaluation plan.

Key Actions:

Key Action Responsible DueDate Status Comments Documents
Formalize changes in UNV?s structure with a view to improving its business processes.
[Added: 2010/07/08]
CMT 2010/02 Completed

Latest Evaluations

Contact us

220 East 42nd Street
20th Floor
New York, NY 10017
Tel. +1 646 781 4200
Fax. +1 646 781 4213
erc.support@undp.org