One Programme Comprehensive Evaluation (End of Cycle)

Report Cover Image
Evaluation Plan:
2014-2018, Bhutan
Evaluation Type:
UNDAF
Planned End Date:
11/2018
Completion Date:
02/2018
Status:
Completed
Management Response:
Yes
Evaluation Budget(US $):
50,000

Share

Document Type Language Size Status Downloads
Download document TOR One Programme Evaluation (Team Leader) 21032017.pdf tor English 591.72 KB Posted 302
Download document Bhutan One Programme Evaluation Final 16022018.pdf report English 2559.07 KB Posted 37
Download document Management Response_UNDAF End of Cycle Evaluation.doc related-document English 114.50 KB Posted 31
Title One Programme Comprehensive Evaluation (End of Cycle)
Atlas Project Number: 00032102
Evaluation Plan: 2014-2018, Bhutan
Evaluation Type: UNDAF
Status: Completed
Completion Date: 02/2018
Planned End Date: 11/2018
Management Response: Yes
Corporate Outcome and Output (UNDP Strategic Plan 2014-2017)
Evaluation Budget(US $): 50,000
Source of Funding: Project
Evaluation Expenditure(US $): 44,331
Joint Programme: No
Mandatory Evaluation: No
Joint Evaluation: Yes
  • Joint with GNHC, all UN agencies and IPs
Evaluation Team members:
Name Title Email Nationality
Roger Maconick Intl. Consultant rmaconick@gmail.com
Dil Maya Local Consultant dilbanos@yahoo.com
GEF Evaluation: No
Key Stakeholders: GNHC and UN agencies
Countries: BHUTAN
Comments:

All Outcomes.

Lessons
1.

This evaluation identified the following lessons learned:

1. UNDAF Programming: The One Programme had budget constraints especially in Outcomes 3 and 4 with ambitious goals covering all the NKRAs of the RGoB. With the changing development finance landscape, budget cuts have become prominent especially among some agencies with delivery of less than USD 2 million per annum. Such budget cuts and an overly scattered focus (raised by the RGoB counterparts on a number of occasions) may contribute to low level of achievement. Efforts need to be made to have a prioritized and focused approach in the next UNDAF. While the One Programme at strategic outcome level, is adaptive and flexible to emerging needs and priories, there needs to be a well-documented rationale for results accountability and a means to monitor progress against agreed changes.

2. Planning, Monitoring and Evaluation: The planning, monitoring and evaluation of the One Programme was a challenge as the One Programme lacked a Theory of Change (ToC) and an M&E mechanism to monitor and report on UN contributions to the 11th FYP as well as ensure that the support is adequately reflected in the RGoB planning and monitoring system. The next UNDAF should have:

a) PME Group: The PME group was not active to ensure quality planning, monitoring and evaluation of the One Progamme. Well-resourced planning with dedicated capacity to the PME is recommended (bearing in mind the success achieved by extra capacity given to the OMT).

b) Theory of Change with simple Results and Resources Framework using common SDGs indicators with the RGoB.

c) Harmonized Approach for Reporting: UN agencies and implementing partners bore the burden of increased transaction costs related to multiple reporting. A harmonized approach for reporting among UN agencies should be explored. This approach will provide important opportunities for increasing efficiencies.

3. Coordination: There is room for improvement in coordination within UN agencies in Bhutan, and with the RGoB, CSOs, and non-resident agencies (NRA) and ESCAP of the UN.

a) NRA/ESCAP: The UNCT is not fully aware of the NRAs/ESCAP work in Bhutan. This needs to be addressed to avoid potential overlaps and duplication.

b) RGoB: The basic lesson in coordination with the RGoB is the UNCT should prioritise and work collectively considering the transaction costs that individual agency planning, monitoring and evaluation processes impose on the RGoB. At the same time, weak leadership and ownership of the RGoB and UN Co-chairs to annual planning, reviewing and reporting process was issue. The limited involvement of co-chairs in coordinating mid & end year reviews and CPB (the planning, reviewing and reporting) often resulted in less upstream policy discussion and strategic planning. Leadership and ownership at all levels both CPB and Outcome Group level of RGoB and UN should be enhanced.

i) CPB: The next UNDAF should review the CPB membership. More relevant Implementing partners need to be included as CPB members. CPB membership should be limited to those ministries which work directly with the UN – more CSOs should be included. 

ii) Outcome Group: The next UNDAF requires stronger management arrangements (eg. appointing RGoB Facilitators in addition to UN Facilitators). Enhanced engagement by Outcome Groups (especially Co-chairs) is required to reduce transaction costs.

4. Resource Mobilization: The UNCT needs to look beyond traditional source of financing as the phasing out of Bhutan’s development partners and the improvement in Bhutan’s microeconomic indicators takes effect. In the immediate years, Bhutan is likely to transition from LDC status which will also impact the UNs ability to undertake successful resource mobilization. The current under-resourcing of Outcomes 3 and 4 is a hard-earned lesson.

a) Joint Resource Mobilization: Joint programme initiatives can lead to enhanced resource mobilization opportunities and increased UNCT ownership of joint results. Going forward, actions to streamline joint resource mobilization should be considered-actioned.

b) One Fund: One fund can act as a catalyst for larger investments. The UNCT members should better utilize the One Fund including using it as a conduit for agency resources.

5. Communications: Stakeholders recognize agency specific results and achievements, however, there were low levels of recognition for the achievements of the One Programme. This is because there was less awareness and sensitization of the One Programme among stakeholders. There is space for improvement in joint communication including the establishment of a dedicated UN communications team.

a) Communications Group: The Joint Communication group was not active. A well-resourced plan with dedicated capacity for communications is important in order to achieve advocacy-communication results.

6. Common Business Operations: Common Business Operations has been relatively successful with a strong leadership of the OMT chair (UNDP) and strong support from the Common Service Officer. The following issues need to be addressed during the next UNDAF:

a) There was a delay in finalising and endorsing the BOS. Although the BOS was only signed in June, the finalization of the BOS took more than a year. This experience demonstrates that the next BOS should be formulated before the start of the UNSDPF. 

b) The Operations costs and budgets have not been integrated in the overall medium-term Common Budgetary Framework. This can be addressed by partially incorporating the operations costs and budgets, especially common services costs and budgets into the medium-term CBF.


Findings
1.

In the backdrop of the evaluation findings, the following pointers conclude the findings of the One Programme (2014–2018) support to the RGoB in its 11th Five Year implementation.

Relevance: There is alignment with the 11th Five Year Plan as the One Programme was formulated in consultation with the RGoB counterparts during its formulation. The One Programme covers all the NKRAs, but the scattered focus of the One Programme with the resource mobilization gaps especially in Outcomes 3 and 4 may cause risks of low achievement in the next two years although at this stage good progress was observed in both outcomes.

The One Programme has been flexible in responding to new issues including those on the 2030 Agenda for Sustainable Development and Sustainable Development Goals (SDG) through joint programmes/programming. However, there was a challenge in accommodating the priority shift under Outcome 4 as there was no system to properly monitor the progress of the outcome.

Effectiveness: As this evaluation is based on data/information available as of December 2016, there are still two years remaining before the full achievement of the One Programme can be determined. So far, progress has been seen across the outcomes as follows:

Outcome 1: The UN was effective in contributing to Bhutan’s sustainable and ‘green’ economic growth in terms of being more equitable, inclusive and resilient to climate change and natural disasters. The UN was effective in enhancing capacity for integrated natural resource management, climate change adaptation and mitigation, and poverty-environment mainstreaming. The UN was effective in strengthening institutional and coordination capacity for climate change adaptation and disaster risk management at both national and local levels supporting climate change preparedness and adaptation practices, including through strengthened early warning systems and response mechanisms. Although the UN provided effective policy interventions for food and nutrition security, the UN has been less effective in enhancing poverty reduction by significantly addressing employment opportunities for vulnerable groups, particularly in urban settings where unemployment is growing as a result of fast urbanization.

Outcome 2: The UN was effective in strengthening the education systems capacity, enhancing capacity to effectively coordinate, plan, implement and monitor the provision of quality and inclusive education services, improving preparedness and response plans for disaster and emergencies, outbreaks and health security threats, enhancing the management of school feeding and promoting water, sanitation and hygiene (WASH) in schools, and youth friendly services. The UN was effective in providing quality maternal, newborn and child health care, nutrition, reproductive health and sexually transmitted infection (STI) / human immunodeficiency virus (HIV) services and improving access to medical supplies and health services by strengthening the national public health procurement system and the legal and policy environment. The UN was effective in promoting WASH in schools, monastic schools and nunneries, enhancing knowledge and skills to adopt behaviours and practices for improved health, nutrition and well-being, access to adolescent, sexual and reproductive health services and young people.

Outcome 3: The UN contributed to strengthening laws and policies to advance the rights and protection of women and children, sensitizing parliament and building the capacity of the Royal Audit Authority for auditing from a gender perspective, gender mainstreaming in key ministries, autonomous bodies and non-governmental organizations with resourced gender mainstreaming strategies, increasing awareness of and positive attitudes towards preventing and eliminating gender-based violence, and enhancing knowledge, skills, resources and mechanisms to prevent and respond to violence against children. However, the UN has been less successful in mobilizing sufficient resources and having an impact on women’s participation in the political process and in decision-making positions in the civil service system. The UN’s ability to sustainably prevent and eliminate (or significantly reduce) gender-based violence was very limited.

Outcome 4: The UN was effective in contributing to the consolidation of Bhutan’s parliamentary democracy in the context of the 2008 Constitution, helping to strengthen the capacity of Parliament, the Election Commission, the Royal Audit Authority, the ACC and the Office of the Attorney General during their inception years. The UN partnered with CSOs as effective channels to promote democratic principles and to respond to the needs of vulnerable groups and bring about positive development results at the local level. The UN was effective in strengthening national and local institutions for integrated planning, monitoring of national five-year plans, and evidence-based decision making. The UN was effective in contributing to the preparation of the groundwork that will help coordinate the justice sector and promote citizens’ access to legal aid. However, the UN has been less effective in mobilizing sufficient resources.

Challenges were observed especially in the monitoring and evaluation (M&E) of the One Programme. The joint programmes/programming have become more active since 2016 because of the change in the nature of proposals being submitted which involve cross-cutting issues like gender, disability and environment sustainability. These types of cross-cutting issues brought more than one agency to work together leading to effective joint programmes/programming (i.e. SDGs, Data and Support to the 12th FYP Formulation, Promoting Gender Equality and Women’s Empowerment, Disability, and Emergency Preparedness and Response).

Efficiency: The UNCT Bhutan achieved 13 out of 15 core elements under the five pillars of DaO – Core Element 13 ‘Operations costs and budgets integrated in the overall medium-term Common Budgetary Framework’ and Core Element 14 ‘Joint Communication Strategy’ were the two exceptions. Due to active engagement by the OMT, common business operations efficiency was improved as was reported in the 2016 Annual Result Report: 19% (USD 65,820) from the total planned budget of USD 342,155 was saved by using cost-efficient approaches. In terms of programme efficiency, co-ordination was a challenge and some stakeholders, especially the RGoB and development partners, commented that high transaction costs due to multiple meetings with different UN agencies often occurred. A coordinated approach for meetings and consultation is essential for improving programme efficiency.

Sustainability: A number of capacity enhancement programmes/projects were carried out during the One Programme as well as policy development to establish an enabling environment of the implementing agencies including the RGoB and CSOs. This will help sustain the impact of the current One Programme.


Recommendations
1

One Programme Programming needs to be integrated and focused, but flexible

2

Establish a strong Planning, Monitoring and Evaluation Mechanism.

3

Establish a stronger Coordination Mechanism.

4

Active Joint Resource Mobilization

 

 

5

Active Communicating as One.

6

Advancing Delivering as One further for efficiency gains.

1. Recommendation:

One Programme Programming needs to be integrated and focused, but flexible

Management Response: [Added: 2018/11/09]

Agreed.

Areas of programme support will be prioritized to avoid a scattered focii and in order to bring innovations/best-case practices from the international arena and contextualize them to the needs of Bhutan. Outcome will be limited to three or four outcomes with a maximum of three joint programmes. Budget projections for the next UNDAF to make more accurate and explore and access alternative sources of financing and use limited resources more efficiently by leveraging and prioritizing investments for sustained development. Continue support for upstream policy work using regional and global expertise to ensure sustainability of the programme and enhance capacity of the RGoB and CSOs to collect, generate, analyse and translate relevant and reliable strategic data and information to achieve the Agenda 2030 Agenda of Leaving No One Behind

Key Actions:

Key Action Responsible DueDate Status Comments Documents
1. Make three or four outcomes with maximum of three Joint Programme.
[Added: 2018/11/09]
UN Agencies, GNHC, NSB 2019/03 Overdue-Not Initiated History
2. Enhance capacity of the RGoB and CSOs to collect, generate, analyse and translate relevant and reliable strategic data and information to achieve the Agenda 2030 Agenda of Leaving No One Behind.
[Added: 2018/11/09]
UN Agencies, GNHC, NSB 2019/03 Overdue-Not Initiated History
2. Recommendation:

Establish a strong Planning, Monitoring and Evaluation Mechanism.

Management Response: [Added: 2018/11/09]

The capacity of UN Planning, Monitoring and Evaluation (PME) group will be strengthened and one RCO staff will be assigned to provide dedicated support to the PME Group. A GNHC and UN PME task team (e.g. joint monitoring etc) will be formed and develop a clear ToC for the UNSDPF and a strong Results and Resource Framework (RRF) will be developed using common SDGs indicators with the RGoB and having less but well-defined indicators. The number of indicators will be reduced to a ‘manageable’ number with clear definitions for each indicator. Allocate specific responsibilities to UN agencies and/or implementing partners to monitor progress. The PME group will prepare a five-year M&E plan with improved monitoring tools, clear ToR for the PME, plus annual plan/schedule for the Mid Year Review and End Year Review. A UN and RGoB joint evaluation involving all UN agencies will be planned with NRAs/ESCAP to be included in AWP preparation, review and reporting processes aligning to AWP process with the RGoB planning process (APA). Explore the harmonizing of reporting methodologies among UN agencies. This approach will provide important opportunities for increasing efficiencies and reduced transaction costs.

Key Actions:

Key Action Responsible DueDate Status Comments Documents
1. Strengthen capacity of UN Planning, Monitoring and Evaluation (PME) group and assign one RCO staff to provide dedicated support to the PME Group.
[Added: 2018/11/09]
RCO, UNCT, UN PME Group 2019/01 Overdue-Not Initiated History
2. RCO will lead the PME group to prepare a five-year M&E plan with improved monitoring tools, clear ToR for the PME, plus annual plan/schedule for the Mid-Year Review and End Year Review.
[Added: 2018/11/09]
RCO, UN PME 2019/03 Overdue-Not Initiated
3. A UN and RGoB joint evaluation involving all UN agencies will be planned with NRAs/ESCAP to be included in AWP preparation, review and reporting processes aligning to AWP process with the RGoB planning process (APA).
[Added: 2018/11/09]
UN, RCO, RGoB 2019/06 Not Initiated
3. Recommendation:

Establish a stronger Coordination Mechanism.

Management Response: [Added: 2018/11/09]

One programme group will actively engage NRAs/ESCAP in the formulation, implementation, monitoring and evaluation of the next UNSPDF. The Country Programme Board membership will be revised and select only relevant IPs for the CPB membership with additional members from CSOs, and where relevant academia, and the private sector. RGoB co-facilitators will be appointed under each outcome and coordinate among UN agencies will combine meetings and consultations with the RGoB so as to reduce the transaction burden on the RGoB. CSOs will be supported in their capacity enhancement to play major role in the areas of Leaving No One Behind and to strengthen their sustainability in the medium–longer term. Promote active information sharing through partnership mapping to avoid duplications and overlapping by focusing on collaboration in the priority areas of: i) SDG Data and ii) Disaster Preparedness.

Key Actions:

Key Action Responsible DueDate Status Comments Documents
1. Country Programme Board membership will be revised and select only relevant IPs for the CPB membership with additional members from CSOs, and where relevant academia and the private sector.
[Added: 2018/11/09]
RCO, Outcome Group 2019/06 Not Initiated
2. CSOs will be supported in their capacity enhancement to play major role in the areas of Leaving No One Behind and to strengthen their sustainability in the medium–longer term.
[Added: 2018/11/09]
UNCT, RCO, UN PME. 2019/06 Not Initiated
4. Recommendation:

Active Joint Resource Mobilization

 

 

Management Response: [Added: 2018/11/09]

The UN Resident Coordinator (UNRC) and UNCT will lead joint resource mobilization actions (with a focus on joint programmes) and report quarterly on such actions through the monthly UNCT meetings. A Joint Resource Mobilization Strategy will be prepared with actions based on the analysis of the development financing landscape in Bhutan and innovative financing opportunities (i.e. Social Impact Investment, Green Financing, etc). As part of this strategy, ways to optimize the One Fund shall be explored.

Key Actions:

Key Action Responsible DueDate Status Comments Documents
1. A Joint Resource Mobilization Strategy will be prepared with actions based on the analysis of the development financing landscape in Bhutan and innovative financing opportunities.
[Added: 2018/11/09]
UNCT, RCO 2019/12 Not Initiated
5. Recommendation:

Active Communicating as One.

Management Response: [Added: 2018/11/09]

A dedicated communications officer from the RCO will assist joint communications/advocacy and act as secretariat of a Communications Results Group. With RCO’s lead, will formulate an annual Joint Communications Strategy workplan that is well resourced.

Key Actions:

Key Action Responsible DueDate Status Comments Documents
1. Formulate an annual Joint Communications Strategy workplan that is well resourced.
[Added: 2018/11/09]
RCO, UNCT 2019/12 Not Initiated
6. Recommendation:

Advancing Delivering as One further for efficiency gains.

Management Response: [Added: 2018/11/09]

The next Business Operations Strategy for UN agencies in Bhutan should be prepared and signed in the middle of 2018 with a start date of 1 January 2019. Common service costs and budgets will be included in the middle-term Common Budgetary Framework for the next UNSPDF.

Key Actions:

Key Action Responsible DueDate Status Comments Documents
1. Business Operations Strategy for UN agencies in Bhutan should be prepared and signed in the middle of 2018 with a start date of 1 January 2019.
[Added: 2018/11/09]
OMT, UNCT 2019/12 Not Initiated

Latest Evaluations

Contact us

220 East 42nd Street
20th Floor
New York, NY 10017
Tel. +1 646 781 4200
Fax. +1 646 781 4213
erc.support@undp.org