Country Programme Document 2012-2016 Final Evaluation

Report Cover Image
Evaluation Plan:
2012-2016, Mozambique
Evaluation Type:
Outcome
Planned End Date:
01/2017
Completion Date:
12/2016
Status:
Completed
Management Response:
Yes
Evaluation Budget(US $):
30,000

Share

Document Type Language Size Status Downloads
Download document UNDP CPD Final Evaluation - ToRs Signed.pdf tor English 7333.71 KB Posted 222
Download document Country Programme Document 2012-2016 - Final Evaluation Report.pdf report English 1588.78 KB Posted 308
Title Country Programme Document 2012-2016 Final Evaluation
Atlas Project Number: 84563,63219,63225,64436
Evaluation Plan: 2012-2016, Mozambique
Evaluation Type: Outcome
Status: Completed
Completion Date: 12/2016
Planned End Date: 01/2017
Management Response: Yes
Focus Area:
  • 1. Others
Corporate Outcome and Output (UNDP Strategic Plan 2014-2017)
  • 1. Output 5.2. Effective institutional, legislative and policy frameworks in place to enhance the implementation of disaster and climate risk management measures at national and sub-national levels
  • 2. Output 6.2. National and local authorities /institutions enabled to lead the community engagement, planning, coordination, delivery and monitoring of early recovery efforts
  • 3. Output 7.2. Global and national data collection, measurement and analytical systems in place to monitor progress on the post 2015 agenda and sustainable development goals
  • 4. Output 7.3. National development plans to address poverty and inequality are sustainable and risk resilient
SDG Goal
  • Goal 1. End poverty in all its forms everywhere
  • Goal 11. Make cities and human settlements inclusive, safe, resilient and sustainable
  • Goal 13. Take urgent action to combat climate change and its impacts
  • Goal 15. Protect, restore and promote sustainable use of terrestrial ecosystems, sustainably manage forests, combat desertification, and halt and reverse land degradation and halt biodiversity loss
  • Goal 17. Strengthen the means of implementation and revitalize the Global Partnership for Sustainable Development
SDG Target
  • 1.2 By 2030, reduce at least by half the proportion of men, women and children of all ages living in poverty in all its dimensions according to national definitions
  • 1.5 By 2030, build the resilience of the poor and those in vulnerable situations and reduce their exposure and vulnerability to climate-related extreme events and other economic, social and environmental shocks and disasters
  • 11.5 By 2030, significantly reduce the number of deaths and the number of people affected and substantially decrease the direct economic losses relative to global gross domestic product caused by disasters, including water-related disasters, with a focus on protecting the poor and people in vulnerable situations
  • 13.1 Strengthen resilience and adaptive capacity to climate-related hazards and natural disasters in all countries
  • 13.2 Integrate climate change measures into national policies, strategies and planning
  • 15.2 By 2020, promote the implementation of sustainable management of all types of forests, halt deforestation, restore degraded forests and substantially increase afforestation and reforestation globally
  • 17.14 Enhance policy coherence for sustainable development
  • 17.18 By 2020, enhance capacity-building support to developing countries, including for least developed countries and small island developing States, to increase significantly the availability of high-quality, timely and reliable data disaggregated by income, gender, age, race, ethnicity, migratory status, disability, geographic location and other characteristics relevant in national contexts
Evaluation Budget(US $): 30,000
Source of Funding: Trac 1
Evaluation Expenditure(US $): 32,000
Joint Programme: No
Joint Evaluation: No
Evaluation Team members:
Name Title Email Nationality
GEF Evaluation: No
Key Stakeholders: Government
Countries: MOZAMBIQUE
Lessons
1.

During the programme implementation there were critical activities not carried out due to weakness at the implementation units. It is also critical to ensure that necessary corrective measures are taken into consideration and timely implemented. The poor coordination at UN level also delayed timely reaction and corrective measures to address the quality of the consulting services.


2.

Monitoring and Evaluation: With many indicators not being informed properly, the M&E framework serves mostly as only a proxy to the performance of the UNDP CP. There should be a closer relationship between the theory of change, outcome indicators and outputs indicators. One weakness is the lack of data to verify certain indicators as documented. In these cases, it is preferable to only include indicators that are measurable based on the data available.


3.

Sustainability: As part of the programme implementation, UNDP has recruited a few international advisers who have been working with high level staff from Implementing Partners on the job training and capacity building to ensure sustainable capacities are built within the institutions during the programme cycle, as a basis for sustainability of efforts. In addition, the training of trainers approach used to improve the skills of members of parliament, police, and electoral staff will continue to benefit the institutions after the end of the projects. Both of these approaches have a potential multiplier effect as those who were initially trained will be able to replicate the train among their colleagues what may ensure durability of the results. Lastly, there is an effort from UNDP to promote strong national ownership at all levels and the involvement of stakeholders and beneficiaries.


Findings
1.

Relevance: The UNDP CP was formulated according to UNDAF (2012-2016) which is in turn aligned to PARP (2011-2014). Both documents are aligned to MDGs. The theory of change adopted for each outcome is appropriate to base the interventions.


2.

Coordination for CP implementation: Coordination mechanisms and integration among different stakeholders is still a problem, there are good coordination bodies at macro level , but that is not enough. During the programme implementation there were critical activities not carried out due to weakness at the implementation units.


3.

Monitoring and Evaluation: There is an apparent disconnection between outcomes and their indicators what creates a challenge to assess progress at outcome level and weakens accountability. Results-based management requires the identification of critical assumptions about the programme environment and risk assessments, clearly defined accountabilities and indicators for results, and performance monitoring and reporting.


4.

Equity: Geographical targeting has been used by UNDP team to focus the interventions in those areas were the citizens are the worst off. Overall, CPD design and implementation took into account equity considerations.


5.

With regard to effectiveness, in general, the implementation of UNDP CP was satisfactory as most activities were successfully implemented and outputs achieved as well as there was progress at the outcome level.


6.

The assessment of the efficiency indicates that most activities were implemented in a cost effective manner.


7.

In regard to sustainability, most of the initiatives implemented under UNDP CPD are likely to be sustainable.


Recommendations
1

Recommendation 1: The coming UNDP CPD should articulate a clear theory of change for the next country program and define its role within the change process
Articulating a theory of change for each programmatic area would allow the programme to precisely define causal links between what program implementers will do and the impacts the program will have, including the interim results (outputs and outcomes) that would help the program measure progress. A well-defined theory of change and simplified framework of results would help program managers decide what projects or components to pursue and where to invest scarce resources, and also how to assess progress and identify needed changes. It would also provide UNDP and its stakeholders with greater clarity about its direction in Mozambique.

2

Recommendation 2: UNDP should strengthen the use of effective RBM and M&E systems to monitor and manage the CP
Results need to be attributable to UNDP to ensure accountability and show progress. This is the reason why UNDP CPD should include a robust set of measurable results and for which UNDP can be held accountable. It is also important to set clearly how results will be monitored during the implementation and the tracking system that will be used for outputs and outcomes.

3

Recommendation 3: There is a need to move away from project activities to programme focus, this would allow to have a holistic and focused implementation.
UNDP has several projects in Mozambique which could be implemented as a programme per thematic area what would maximize complementarities and synergies and reduce potential duplicative activities.

4

Recommendation 4: Speeding up fund disbursements and planning needs to focus on available funds.
Several implementing partners emphasized that the UNDP needs to find more innovative ways of improving and speeding up funds disbursements to enable timely implementation of the activities planned. Timely disbursement of funds will ensure effective and efficient implementation of programmes and initiatives.

5

Recommendation 5: Improve coordination mechanisms.
Coordination mechanisms and integration among different stakeholders is still a problem; there are good coordination bodies at macro level, but that is not enough. Views at the implementation level are very much compartmented and short sighted. Workshops to formulate, launch, evaluate and monitor are important to reach the needed leadership, ownership and responsibility with clear targets and indicators adjusted to local conditions and specificities.

6

Recommendation 6: More attention to operation details.
In the formulation of the new programme, more attention should be given to the operational detail and to the institutional analysis that includes the individuals and the organization at different levels of implementation. This helps to define how operations will occur in practice to implement the programme, monitoring plans, determine the capacity needs, how the risks will be mitigated and approaches to ensure sustainability of the programme achievements.

7

Recommendation 7: Maximization of UNDP comparative advantages
Financial resources are always scarce and never enough, on the other hand, there are many needs and requests. UNDP´s main role, focus, knowhow and key advantage is not scaling up processes but pioneering, development of procedures and methodologies, and building capacity within national institutions.

8

Recommendation 8: Geographical targeting as way to focus on the most disadvantaged groups
UNDPs definition of provinces of intervention may need a revision. There is a need to be more flexible in terms of targeted provinces.

9

Recommendation 9: Inclusive market and finance for employment and income opportunities for vulnerable groups and sustainable poverty redution.
UNDP could assist Mozambique in the development a framework for private sector development to address the weaknesses of MSMEs. Alternatively UNDP could raise this need among other UN specialized agencies to identify the agency that could lead this intervention.

10

Recommendation 10: Provision of coordinated, equitable and integrated services at decentralized level
Despite the progress made through the development of key policy documents, the complexity of the decentralized process needs flexibility to adapt the instruments to a changing context and UNDP needs to be prepared to provide the assistance the government may require for an effective decentralization.

1. Recommendation:

Recommendation 1: The coming UNDP CPD should articulate a clear theory of change for the next country program and define its role within the change process
Articulating a theory of change for each programmatic area would allow the programme to precisely define causal links between what program implementers will do and the impacts the program will have, including the interim results (outputs and outcomes) that would help the program measure progress. A well-defined theory of change and simplified framework of results would help program managers decide what projects or components to pursue and where to invest scarce resources, and also how to assess progress and identify needed changes. It would also provide UNDP and its stakeholders with greater clarity about its direction in Mozambique.

Management Response: [Added: 2017/05/23] [Last Updated: 2017/05/23]

UNDP Mozambique will ensure a Theory of Change approach is used for the next cycle Country Programme in line with UNDP corporate requirements.

Key Actions:

Key Action Responsible DueDate Status Comments Documents
Use a Theory of Change approach in the formulation of the Country Programme 2017-2020.
[Added: 2017/05/23]
All Programme Unit. 2016/09 Completed Following the newly adopted corporate guidelines, the Country Programme Document 2017-2020 was formulated using a Theory of Change approach. The CPD was approved by the UNDP Executive Board in September 2016.
2. Recommendation:

Recommendation 2: UNDP should strengthen the use of effective RBM and M&E systems to monitor and manage the CP
Results need to be attributable to UNDP to ensure accountability and show progress. This is the reason why UNDP CPD should include a robust set of measurable results and for which UNDP can be held accountable. It is also important to set clearly how results will be monitored during the implementation and the tracking system that will be used for outputs and outcomes.

Management Response: [Added: 2017/05/23]

UNDP Mozambique will work to ensure a stronger, coherent and effective RBM and M&E system is defined and used for the next Country Programme using as a basisi the UNDP Strategic Plan 2014-2017 results framewrok.

Key Actions:

Key Action Responsible DueDate Status Comments Documents
Strengthen the Results Framework of the next CPD through alignment with UNDP Strategic Plan 2014-2017 Results Framework.
[Added: 2017/05/23]
M&E Specialist with support from RSCA RBM colleagues. 2016/09 Completed The CPD 2017-2020 and related Evaluation Plan and Results Framework were approved by UNDP Executive Board in September 2016. The CPD 2017-2020 RF is closely aligned with the UNDP Strategy Plan RF.
3. Recommendation:

Recommendation 3: There is a need to move away from project activities to programme focus, this would allow to have a holistic and focused implementation.
UNDP has several projects in Mozambique which could be implemented as a programme per thematic area what would maximize complementarities and synergies and reduce potential duplicative activities.

Management Response: [Added: 2017/05/23]

UNDP Mozambique will seek to promote a holistic and focused approach for its next Country Programme by reducing the fragmentation of interventions and improving synergies between the projects, including through the adoption of an issues-based approach. 

Key Actions:

Key Action Responsible DueDate Status Comments Documents
Ensure the new CPD 2017-2020 uses an holistic and issues-based approach to promote synergies and avoid fragmented interventions.
[Added: 2017/05/23]
Programme Unit 2016/09 Completed The CPD 2017-2020 was formulated using a strong and coherent issues-based approach. The CO will now develop projects to operationalize the CPD using the issues-based approach as a basis aiming at a reduced number of projects to avoid fragmentation.
4. Recommendation:

Recommendation 4: Speeding up fund disbursements and planning needs to focus on available funds.
Several implementing partners emphasized that the UNDP needs to find more innovative ways of improving and speeding up funds disbursements to enable timely implementation of the activities planned. Timely disbursement of funds will ensure effective and efficient implementation of programmes and initiatives.

Management Response: [Added: 2017/05/23] [Last Updated: 2017/05/23]

UNDP Mozambique will make greater efforts to ensure AWPs are signed within the first 2 months of the year to allow for early disbursement of funds to IPs and look for more effective ways to ensure financial reporting and requests of advances are processed in a timely manner.

Key Actions:

Key Action Responsible DueDate Status Comments Documents
The CO will ensure close monitoring of AWPs preparation during the last and initial months of the year to ensure all AWPs are signed by the February and funds disbursement and project implementation are on track.
[Added: 2017/05/24] [Last Updated: 2018/01/12]
Programme Unit 2018/03 Completed The CO is already discussing the 2018 AWPs with the IPs and expects to have them signed by mid-February. History
5. Recommendation:

Recommendation 5: Improve coordination mechanisms.
Coordination mechanisms and integration among different stakeholders is still a problem; there are good coordination bodies at macro level, but that is not enough. Views at the implementation level are very much compartmented and short sighted. Workshops to formulate, launch, evaluate and monitor are important to reach the needed leadership, ownership and responsibility with clear targets and indicators adjusted to local conditions and specificities.

Management Response: [Added: 2017/05/23]

UNDP Mozambique will assess ways to improve coordination at all levels: internally, with UN Agencies, with IPs and within portfolios.

Key Actions:

Key Action Responsible DueDate Status Comments Documents
The CO will carry out regular internal meetings to improve coordination on programmatic issues.
[Added: 2017/05/24] [Last Updated: 2018/01/12]
Country Director 2017/12 Completed The CO held several Unit and Programme meetings to discuss coordination issues. History
The CO will promote external meetings with UN Agencies, IPs and other stakeholders to improve coordination of programmtic activities.
[Added: 2017/05/24] [Last Updated: 2018/01/12]
Programme Unit 2018/06 Completed This is an ongoing action. The CO is systematically involved in UN Outcome Groups to discuss how to improve joint and coordinated activities. History
6. Recommendation:

Recommendation 6: More attention to operation details.
In the formulation of the new programme, more attention should be given to the operational detail and to the institutional analysis that includes the individuals and the organization at different levels of implementation. This helps to define how operations will occur in practice to implement the programme, monitoring plans, determine the capacity needs, how the risks will be mitigated and approaches to ensure sustainability of the programme achievements.

Management Response: [Added: 2017/05/23]

UNDP Mozambique will ensure that the new Country Programme takes into account aspects related to risks and institutional capacities to promote smooth implementation and sustainability of the interventions.

Key Actions:

Key Action Responsible DueDate Status Comments Documents
Include strong analysis of risks and institutional capacities in the new CPD 2017-2020.
[Added: 2017/05/23]
Programme Unit 2016/09 Completed The CPD 2017-2020 was approved in September 2016 and includes the required considerations in terms of risk analysis and mitigation as well as institutional capacities using HACT framework.
7. Recommendation:

Recommendation 7: Maximization of UNDP comparative advantages
Financial resources are always scarce and never enough, on the other hand, there are many needs and requests. UNDP´s main role, focus, knowhow and key advantage is not scaling up processes but pioneering, development of procedures and methodologies, and building capacity within national institutions.

Management Response: [Added: 2017/05/23] [Last Updated: 2017/05/23]

UNDP Mozambique will ensure that the new Country Programme builds on its comparative advantages recognized at country level.

Key Actions:

Key Action Responsible DueDate Status Comments Documents
Ensure that the new CPD takes advantage of UNDP strengths and comparative advantages.
[Added: 2017/05/23]
Programme Unit 2016/09 Completed The new CPD 2017-2020 was approved in September 2016 and build on UNDP comparative advantages: a unique body of knowledge, a multi-sectoral approach, and an impartial, honest-brokering role. In addition it takes into account the lessons learned from previous interventions and seeks to replicate successful approaches: embedding national experts in key ministries and provincial governments.
8. Recommendation:

Recommendation 8: Geographical targeting as way to focus on the most disadvantaged groups
UNDPs definition of provinces of intervention may need a revision. There is a need to be more flexible in terms of targeted provinces.

Management Response: [Added: 2017/05/23]

UNDP Mozambique will ensure the new Country Programme will be flexible enough to allow targeting geographical areas beyond the 3 priority provinces.

Key Actions:

Key Action Responsible DueDate Status Comments Documents
Promote flexibility in the selection of the geographical target of the next CPD interventions.
[Added: 2017/05/23]
Programme Unit 2016/09 Completed The new CPD focuses on the 3 priority provinces but includes statements showing that flexibility is allowed whenever applicable.
9. Recommendation:

Recommendation 9: Inclusive market and finance for employment and income opportunities for vulnerable groups and sustainable poverty redution.
UNDP could assist Mozambique in the development a framework for private sector development to address the weaknesses of MSMEs. Alternatively UNDP could raise this need among other UN specialized agencies to identify the agency that could lead this intervention.

Management Response: [Added: 2017/05/23]

UNDP Mozambique will ensure its active participation in the area of private sector development and coordinate with other UN Agencies working in this area.

Key Actions:

Key Action Responsible DueDate Status Comments Documents
Include private sector development as an intervention are for the next programme cycle.
[Added: 2017/05/23]
Economic and Policy Analysis Team (EPAT) 2017/05 Completed The EPAT started working on interventions under the CPD Pillar related to Sustainable and Inclusive Economic Transformation to promote private sector development. A prodoc has been drafted and will be approved in the upcoming months.
10. Recommendation:

Recommendation 10: Provision of coordinated, equitable and integrated services at decentralized level
Despite the progress made through the development of key policy documents, the complexity of the decentralized process needs flexibility to adapt the instruments to a changing context and UNDP needs to be prepared to provide the assistance the government may require for an effective decentralization.

Management Response: [Added: 2017/05/23] [Last Updated: 2017/05/23]

UNDP Mozambique will ensure the new Country Programme takes into account this recommendation and foresees interventions in the decentralization and local governance areas.

Key Actions:

Key Action Responsible DueDate Status Comments Documents
The CPD 20170-2020 will include interventions under the local governance and decentralization area.
[Added: 2017/05/23]
Governance Unit 2016/09 Completed The CPD 2017-2020 was approved in September 2016 and includes interventions in the decentralization and local governance area, more specifically, under the Governance Outcome.

Latest Evaluations

Contact us

1 UN Plaza
DC1-20th Floor
New York, NY 10017
Tel. +1 646 781 4200
Fax. +1 646 781 4213
erc.support@undp.org