UNDAF final Evaluation

Report Cover Image
Evaluation Plan:
2012-2017, Cape Verde
Evaluation Type:
UNDAF
Planned End Date:
03/2017
Completion Date:
12/2016
Status:
Completed
Management Response:
No
Evaluation Budget(US $):
30,000

Share

Document Type Language Size Status Downloads
Download document ToRs Cabo Verde UNDAF Evaluation.pdf tor English 368.76 KB Posted 286
Download document Cabo Verde UNDAF Evaluation Report _Final Version_12 Dec 2016.pdf report English 3127.94 KB Posted 860
Title UNDAF final Evaluation
Atlas Project Number:
Evaluation Plan: 2012-2017, Cape Verde
Evaluation Type: UNDAF
Status: Completed
Completion Date: 12/2016
Planned End Date: 03/2017
Management Response: Yes
Corporate Outcome and Output (UNDP Strategic Plan 2014-2017)
Evaluation Budget(US $): 30,000
Source of Funding: UNRC
Evaluation Expenditure(US $): 26,000
Joint Programme: No
Joint Evaluation: No
Evaluation Team members:
Name Title Email Nationality
Craig Naumann Independent Evaluation/Research Expert ccnaumann@yahoo.com
GEF Evaluation: No
Key Stakeholders:
Countries: CAPE VERDE
Lessons
1.

The UNDAP, initially conceived as overall programme coordination tool, gradually withered away into the background after the design of the UNDAF. Actual yearly planning was carried out through the joint annual work plans which turned out to be a, if not the major tool for programme coordination.


2.

The UNDAF as such, and in particular its set of output and outcome indicators, was virtually shelved and forgotten during the actual implementation of the One Programme. This holds important lessons for the design of M&E mechanisms, tools and indicators. A lesson to be learned here is that there is not much added value and, to the contrary, a real risk of wasting lots of resources by designing detailed sets of high-level performance indicators at the programme impact/outcome level without a full understanding of what type of data sources exist, and what kind of data will shortly become available.


3.

There were a number of joint programmes but mostly, they were confined to results chains within the same pillar.


4.

While there was evidence for partial thematic programme design transcending traditional sector logic, most programmes and projects under the One Programme were conceived according to a silo approach. The one case where programmes were truly cross-sectorial, namely the first and second outcomes under Pillar 2 (“mini-UNDAF”), was somewhat ill-conceived, in that it was ultimately too broad and all-encompassing, preventing genuine synergies to kick in.


5.

Initial agency planning reflected in the architecture of the One Programme’s results framework shows traces of single agency planning logic being “front-loaded” in the UNDAF planning process, rather than the SOPs logic of joint planning in response to jointly identified comprehensive country needs from which joint programmes are derived. In particular, this concerns pillars 2 and 3.


6.

A more inclusive annual work plan (AWP) process is needed to ensure that all non-resident agencies feel that their respective mandate and related activities are fully reflected in a specific year's UNDAF Workplan.


Findings
1.

The three JO agencies dominate the UNDAF in terms of financial and operational presence: together, they account for 45% of allocated budget resources from 2012-2016. The Joint Office (JO) contributes to 59% (20/34) of outputs with involvement of one or several JO member agencies across all UNDAF outcomes; namely, 1/3 outcomes under Pillar 1, 2/3 outcomes under Pillar 2, 3/3 outcomes under Pillar 3, and 3/3 outcomes under Pillar 4, which makes for an overall presence in 9/12 outcomes across the entire UNDAF.


2.

While the output level indicators had baseline and final target values, the outcome and impact indicator level had neither of those. Another major problem here was that there was no existing data (source) for most of the indicators. Reasons included that the indicators were not SMART, no investment into requisite data collection processes had occurred, household survey data that could inform some of the indicators is only becoming available in the near future towards the end of the calendar year etc.


3.

In terms of the UNDAF One Programme’s overall relevance, the UNDAF provided support to all of the five strategic DECRP pillars.


4.

The cross-cutting programming principles of human rights and gender equality are reflected in the design of UNDAF outputs and activities, namely under UNDAF pillars 1, 2 and 3.


5.

Whereas the principle of environmental sustainability was captured under an UNDAF pillar in its own right (Pillar 4), systemic sustainability aspects were in particular addressed by related capacity building reflected in particular under the second and third outcomes of each pillar, which concentrate on institutional capacities and operational aspects while the first outcome has a focus on normative, upstream-level policy related work.


6.

In terms of general effectiveness, the analysis of performance indicators at the level of outputs shows good progress as far as advancement against initially fixed output targets is concerned. At the level of output measures, overall, final targets have already been at least met, if not surpassed, for 25% of related indicators (or 21/86 indicators that could be informed from among the total of 92). Regarding early indications for, or evidence of potential early impact, there are already some solid indications for arguably game changing interventions directly attributable to UNDAF interventions.


7.

In terms of general efficiency, the One Programme so far managed to mobilize 92% or USD76,056,910 of the initially planned USD82,445,090 for the 2012-2016 period.


8.

Furthermore, the recent expansion of the pillar working group structure to also include GoCV representatives from line ministries as chairs/co-chairs to mirror the UN internal co-leads increases the likelihood of generating future gains in terms of impact, effectiveness, efficiency, coherence and alignment vis-à-vis GoCV activities, in general.


9.

The UNCT successfully managed to leverage partnerships (EU, bilateral donors incl. Luxemburg etc.) including mobilizing financial and technical resources through South-South partnerships (Uruguay, Guinea-Bissau, Brazil, TimorLeste and African Portuguese-speaking countries such as São Tome e Principe).—While internal levels of coherence v are quite robust across the various domains of programming, managerial and operational coherence, there is still some room for improvement in terms of internal cross-agency information sharing and transparent communication.


10.

There are positive examples of DaO-induced operational practices such as the reality of the One House, joint LTAs, steps towards harmonizing per diem rates for national consultants etc. The CCPD for the three JO agencies was derived from and is thus intrinsically aligned with the UNDAF/UNDAP, which qualifies as best practice.


Recommendations
1

Organize a national SDG prioritization exercise to identify those SDGs that the national efforts including the respective UNDAF support should focus on. Thereafter, by focussing on identified priorities, use the logic of the SDG design to frame innovative, thematically integrated results groups that are aligned with the global level results structure.

2

Consider focusing on not more than three main programming platforms so as to avoid strategic overstretch that would risk dissipating and thus not reaching, required critical mass for UNDAF interventions. Depending on the final articulation of the next NDS/DECRP, this could e.g. consist of programmatic blocks such as: 1) “turquoise economy” (blue-green) focusing on job creation in sustainable tourism and related industries combined with a strong integrated environmental theme of protecting natural resources, mitigating climate change risks through adaptive agri-, silvi- and aquacultural practices and fisheries, building disaster resilient livelihoods and promoting innovative, creative culture (lead agencies: UNDP, FAO, UNIDO, UNESCO); 2) “public social service delivery and good governance” essentially merging major components of pillar 2 with pillar 3 (lead agencies: UNDP, WHO, UNICEF, UNFPA, UN Women).

3

Consider replacing UNDAF/UNDAP/yearly Joint AWPs with only an UNDAF light without results framework, or an UNDAP+ (logframe incl. indicators with light narrative) plus biennial work plans to reduce workload and enhance coherence of activities under work plans.

4

Consider introducing customization of planning/plans, at the level of Municipalities and/or different island(s), also considering to lump entities with similar profiles and challenges together into clusters across municipality/island borders, so as to enhance the quality of UNDAF services.

5

The UN (and the Government) to immediately start mobilizing technical expertise through the current UNDAF in order to conceive a long-term vision of at least 15 years, to align with the 2030 agenda. Expanding the general planning horizon beyond a five-year time line would not entail elaborating detailed action plans beyond the standard four to five-year cycle. Rather, it would plot a strategic course against which the upcoming NDS/PRSP cycles would be planned and the respective UNDAFs would have to align with. Such systematic mid- to long-term strategic planning of consecutive PRSPs and respectively aligned UNDAF cycles to jointly pursue the SDGs should prove beneficial in terms of plotting the strategic course to achieve the SDGs within the next 15 years including the UNDAF response as per alignment with the NDS.

6

Regional/HQ level: Suggest cross-agency integration of regional or global plans/frameworks to enable country level programmes to be conceptualized and implemented in line with a (n even) more coherent, inter-agency approach. A prime candidate in the West African context would be the nexus of environmental vulnerability, livelihoods and economic foundations including the following strands of challenges and related activities: security/anti-crime & anti-terrorism (trafficking of drugs and/or people)/DRR/monitoring and enforcing the regime of protected areas for wildlife/habitat protection (no-go areas, no poaching, no wood-cutting etc.)/life-saving coast guards). Short-listing criteria and candidates for such programmes to take place through regional consultations which could be facilitated by the Regional UNDG.

7

As default programme design and delivery model and unless good reasons stand against it, opt for joint programming/programme approach building on the platform of the UNDAF’s joint programming process, to provide depth and enhance inter-agency and inter-stakeholder coordination to One Programme activities, thus strenghtening internal cohesion at the level of implementation and managerial oversight.

8

Looking forward, there are indications based on statements put forward by the new government, that the upcoming DECRP will focus, among other issues, on economic growth and job creation for the youth as well as fighting crime, trafficking and drug abuse. The next UNDAF should consider devising an integrated programme addressing economic (re)orientation and education as a preventative crime fighting measure as well as (ideally regionally integrated) anti-crime and drug trafficking initiative. This would be an opportunity to combine reactive elements of the current UNDAF’s Pillar 2 with a proactive, prevention-driven logic where the UN could provide “soft” components including strategic upstream and capacity building support. UNODC’s on-going “National Integrated Programme in the fight against drugs and crime” which is derived from UNODC’s Regional Integrated Programme for West and Central Africa, which in turn is aligned with UNODC’s Global programming, would serve as a stepping stone for integrated multi-agency programming beyond the relatively timid examples of currently existing levels of UN agency involvement complementary to UNODC, under Pillar 2

9

Avoid “fuzzy” catch-all programme components such as the current UNDAF’s Pillar 2 design (“UNDAF within the UNDAF”). Instead, strive to find a balance between size and relevance of genuine results groups that will transcend the traditional silo or cluster approach reflected in the UNDAF design by following a cross-cluster oriented programming logic.

10

Introduce basket funds (potentially directly tied to SDGs/NDP-PRSP pillars as M(P)TFs, if GoCV/MoF agree) linked to strategic initiatives, allowing for controlled cherry-picking.

11

Align UN programmes to the next DECR’s pillar logic to a greater extent to mainstream UNDAF fund-raising purposes in the spirit of aid effectiveness (funding according to, or in the general spirit of budget funding).

12

Consider upgrading programme results group leadership/supervision role to Agency Chief level to enforce accountability vis-à-vis set performance goals/indicators.

13

Lessons to be learned from the JO’s experience in terms of reducing transaction costs through shared crossagency systems and standards (mostly by opting for one of the three corporate standards rather than inventing new integrated systems used to jointly manage operational processes for all three JO agencies) should be looked into beyond JO boundaries. The upcoming BOS will be crucial in this respect to map out remaining constraints and identify opportunities and need for urgent action.

14

Esp. if recommendation no. I is implemented, if and where applicable, use SDG indicators as high-level UNDAF indicators at impact/outcome level. Respective baseline data, targets, means of verification etc. could then be derived from and aligned with SDG tracking efforts at national level.

15

Rigorously apply the RBM logic from the outset, by aligning the UNDAF cycle’s design of results/performance indicators with available national data sources as means of verification. Where related data is not yet being generated, this might warrant specific technical support to be mobilized through the current UNDAF following the example of imminent UNDAF support to the Praia Group of governance-related statistics. Don’t forget also setting performance targets for higher-level indicators and consider linking results indicators to (yearly) milestones that would then serve as tangible targets at the level of planning and implementing the respective result group work plans.

16

In order to strengthen qualitative monitoring, social media, big data mining and other innovative approaches should be considered. This could be a new area of collaboration between INE and the UN, not least because UNDAF/One Programme M&E framework (indicators, data sources) need to be aligned with INE indicators at impact/outcome level (domesticated SDG indicators at impact level, part of outcome indicators using household surveys and official INE-sanctioned administrative routine data collection).

17

Introduce an UNDAF-specific cross-agency financial tracking process connected to related M&E data, to track financial allocations and actual expenditures against activities. At the aggregate level, this would enable results group and UNCT management to monitor and report against One Programme/UNDAF level beyond mere pillar level financial tracking. Such a system would be regularly populated with agency data, ast the results group and UNDAF level. This might require revising the ToR of the results groups.

18

Strengthen UNDAF M&E capacity at the UNCT level by creating at least one dedicated senior level national slot or, alternatively, at least one mid-level international position, to be added to the current RCO organigramme. Related mechanisms and process to be approved by the UNCT.

19

M&E is not to be treated as an afterthought but must be plugged into a foremost position within the planning chain including linking a data-driven situation and needs assessment to results-based planning, including establishing SMART indicators, baselines, targets and results statements; and subsequent evidence and results-based management processes. Add the tasks of regular tracking and reporting of UNDAF output and, to the extent possible, outcome and impact indicators to those of the Results Groups responsible for managing the implementation and M&E of respective UNDAF pillars, as per the DaO SOPs.

20

Introduce a regular anonymous DaO staff survey process to take the pulse of the corporate climate, to serve as internal monitoring mechanism in terms of internal communication, information sharing etc. This needn’t be a work-intensive major exercise and could be carried out only once or twice within an UNDAF cycle. To the contrary, it would ideally be conceived as an ultra-light, live on-line system with a limited number of key coherence and coordination-related indicators such as perceived quality of information sharing/transparency among agencies etc. Along the same vein, agency management might want to consider participating in the 180 (or 360) degree-type internal peer review process organized on a yearly basis by UNDG-DOCO; or else, introduce its own peer-review type mechanism which might well be conceived as an “oral”, interactive powwow-type exercise. Similarly, a “DaO-townhall” event including all staff categories might be an event to be introduced as a yearly or biannual exercise allowing to keep tabs on the reality of DaO among staff.

Latest Evaluations

Contact us

1 UN Plaza
DC1-20th Floor
New York, NY 10017
Tel. +1 646 781 4200
Fax. +1 646 781 4213
erc.support@undp.org