Independent Country Programme Evaluation: Indonesia

Report Cover Image
Evaluation Plan:
2018-2021, Independent Evaluation Office
Evaluation Type:
ICPE/ADR
Planned End Date:
12/2019
Completion Date:
12/2019
Status:
Completed
Management Response:
No
Evaluation Budget(US $):
50,000

Share

Document Type Language Size Status Downloads
Download document ICPE Indonesia ToR.pdf tor English 286.53 KB Posted 21
Download document ICPE Indonesia Annex.pdf related-document English 854.08 KB Posted 38
Download document ICPE Indonesia EvalBrief.pdf summary English 209.89 KB Posted 40
Download document ICPE Indonesia full report.pdf report English 5246.87 KB Posted 95
Title Independent Country Programme Evaluation: Indonesia
Atlas Project Number:
Evaluation Plan: 2018-2021, Independent Evaluation Office
Evaluation Type: ICPE/ADR
Status: Completed
Completion Date: 12/2019
Planned End Date: 12/2019
Management Response: Yes
Corporate Outcome and Output (UNDP Strategic Plan 2018-2021)
  • 1. Output 1.1.1 Capacities developed across the whole of government to integrate the 2030 Agenda, the Paris Agreement and other international agreements in development plans and budgets, and to analyse progress towards the SDGs, using innovative and data-driven solutions
  • 2. Output 1.2.1 Capacities at national and sub-national levels strengthened to promote inclusive local economic development and deliver basic services including HIV and related services
  • 3. Output 2.1.1 Low emission and climate resilient objectives addressed in national, sub-national and sectoral development plans and policies to promote economic diversification and green growth
  • 4. Output 2.3.1 Data and risk-informed development policies, plans, systems and financing incorporate integrated and gender-responsive solutions to reduce disaster risks, enable climate change adaptation and mitigation, and prevent risk of conflict
  • 5. Output 3.1.1 Core government functions and inclusive basic services4 restored post-crisis for stabilisation, durable solutions to displacement and return to sustainable development pathways within the framework of national policies and priorities
  • 6. Output 3.2.2 National and local systems enabled and communities empowered to ensure the restoration of justice institutions, redress mechanisms and community security
Evaluation Budget(US $): 50,000
Source of Funding:
Evaluation Expenditure(US $): 50,000
Joint Programme: No
Joint Evaluation: No
Evaluation Team members:
Name Title Email Nationality
Alan Fox Chief of Section
Landry Fanou Research and Evaluation Consultant
David Slattery Evaluation Advisor
Chasan Ascholani Evaluation Consultant
GEF Evaluation: No
Key Stakeholders:
Countries: INDONESIA
Lessons
Findings
1.

2.1 Sustainable employment and income generation (outcome 1) 

RELATED CPD OUTCOMES

Outcome 1 goal: By 2020, more vulnerable, low-income and food-insecure people have an adequate standard of living and equitable access to decent work, sustainable livelihoods, economic development and income-earning opportunities.

This is the outcome area with the smallest financial footprint. It represents 3 percent of the CPD’s indicative budget and 1.5 percent of the total expenditure from 2016 to October 2019.

Output 1.1. Sustainable employment and income generation Local governments and communities have enhanced capacity to design and implement sustainable livelihood programmes.

Finding 1. UNDP plays a strategic, important and valued role as chair of the Papua Platform, which sits under the Papua Desk in the National Planning Agency (BAPPENAS) and is expected to facilitate coordination and communication among development actors and programmes in Papua and West Papua.


Tag: Forestry Vulnerable Natural Resouce management Effectiveness Efficiency Sustainability Resource mobilization Local Governance Communication Donor relations Partnership Strategic Positioning Country Government Inclusive economic growth Jobs and Livelihoods Poverty Reduction Coordination SDG Integration

2.

2.2 Equitable access to quality basic social services and social protection (outcome 2)

RELATED CPD OUTCOMES Outcome 2 goal: By 2020, the poor and most vulnerable have better and more equitable access to quality basic social services, including health and education, and to comprehensive social protection and better access to water supply and sanitation.

This outcome constitutes 1.4 percent of the CPD’s indicative budget and nearly 12 percent of total expenditures from 2016 to October 2019.

Output 2.1. Mainstreaming Sustainable Development Goals Subnational authorities have improved financing, implementation/monitoring of Millennium Development Goals/Sustainable Development Goals acceleration programmes and delivery of basic services.

Finding 2. Working through a number of initiatives, UNDP has successfully promoted recognition of the importance of the SDGs as a universal objective, and one through which policy development and debate in Indonesia can be usefully framed. Support has been provided to the SDG Secretariat in BAPPENAS towards the development and launching of a National Action Plan and SDG Roadmap, and development of an SDG Monitoring and Evaluation Framework and Communication Strategy.


Tag: Vulnerable Drinking water supply Effectiveness Efficiency Local Governance Health Sector Operational Efficiency Results-Based Management Theory of Change Education Poverty Reduction Social Protection SDG accelerators SDG Integration

3.

Output 2.2. Health governance Policy/legal frameworks strengthened, and systems established nationally and in targeted regions, to improve access to social services and medicines by excluded groups.

Finding 3. UNDP has provided effective ongoing support for Indonesia’s utilization of Global Fund grants, contributing to improved health outcomes in areas covered by the grants.


Tag: Effectiveness Impact Global Fund Local Governance Access to Medicines Health Sector HIV / AIDS Non-Communicable Diseases Procurement Project and Programme management Bilateral partners Poverty Reduction Social Protection Technical Support

4.

Output 2.3. Subnational service delivery Institutional/financing capacity of subnational-level institutions enhanced to deliver improved basic services and respond to priorities voiced by the public.

Finding 4. UNDP has supported subnational service delivery, but on a limited scale (three provinces), including through a small number of micro-projects with subnational authorities. There are insufficient resources to provide substantive long-term support for building the service delivery capacity of subnational authorities and the potential scalability of existing work appears limited. Related to this, there is a lack of evidence of impact of subnational support on the quality of service delivery in areas supported.


Tag: Efficiency Sustainability Local Governance Service delivery

5.

Output 2.3. Subnational service delivery Institutional/financing capacity of subnational-level institutions enhanced to deliver improved basic services and respond to priorities voiced by the public. 

Finding 5. Not directly related to the objective of improved subnational service delivery but attributed to this output in UNDP’s systems, UNDP has supported innovation in the use of big data and financing, as administrative agent for the United Nations Secretary-General’s flagship innovation initiative on big data, Global Pulse. Work with partners to explore big data applications has produced some promising results, but stronger monitoring and evaluation data is required before a reliable judgement can be made about impact and value for money. A lab to promote exploration of innovative financing also has some potential, although it is early days. Both initiatives suffer from a lack of secure, long-term financing.


Tag: Disaster Risk Reduction Vulnerable Efficiency Sustainability Private Sector Financing Local Governance Innovation Monitoring and Evaluation Partnership Service delivery Bilateral partners Country Government International Financial Institutions UN Country Team Operational Services Data and Statistics SDG Integration

6.

2.3 Sustainable natural resource management and increased resilience (outcome 3)

RELATED CPD OUTCOMES Outcome 3 goal: By 2020, Indonesia is sustainably managing its natural resources, on land and at sea, with an increased resilience to the effects of climate change, disasters and other shocks.

As noted in Chapter 1, outcome 3 represents almost 90 percent of the CPD’s indicative budget and a little over 65 percent of the total expenditure from 2016 to October 2019. However, when compared to major environmental and climate change-related financial inflows in the country, UNDP’s environment-related expenditure is relatively small. The organization’s spending on the environment in Indonesia represents a fraction of overall GEF funding channelled to the country, which includes 128 projects, just over $1 billion in GEF grants, and $6.1 billion in additional cofinancing. UNDP’s environmental programmes are a little over a quarter the size of the GCF’s $200 million climate and energy engagement in Indonesia. UNDP has not been successful in mobilizing GCF resources for Indonesia.

Output 3.1. Sustainable palm oil Policy/institutional framework and extension services strengthened at national/subnational level for sustainable production of commodities by small landholders including palm oil.

Finding 6. UNDP has contributed to upstream-level initiatives to address gaps in policy and effective collaboration between the Government, civil society and the private sector for sustainable palm oil. UNDP had ambitious goals in promoting smallholder compliance with Indonesia’s Sustainable Palm Oil Standard. Inability to mobilize resources for this purpose has limited the scale of this work and the achievement of these goals. Given the limited prospects for scaling up existing interventions, UNDP’s continued investment in this area should be contingent on success in mobilizing significant additional resources.

 


Tag: Climate Change Adaptation Climate change governance Disaster Risk Reduction Natural Resouce management Global Climate Fund Global Environment Facility fund Resource mobilization Partnership Civil Societies and NGOs Country Government Private Sector Resilience Advocacy

7.

Output 3.2. Promoting green economic models National/subnational government capacities enhanced to adopt green economy/low carbon models and approaches and to protect forests.

Finding 7. UNDP has successfully promoted the adoption of green economy models in targeted provinces. Other non-targeted provinces are in the process of replicating these green economy models, and the low carbon development concept was included in the Indonesia Mid-term Development Plan 2020-2024.


Tag: Forestry Emission Reduction Green Economy Local Governance Jobs and Livelihoods Poverty Reduction

8.

Finding 8. UNDP has been an important source of support to the establishment of the Indonesia REDD+ infrastructure and capacity. Significant achievements during this planning cycle are the establishment of the MRV system, the national registry system, and the financing system which together with past achievements support Indonesia’s REDD+ readiness for implementation.


Tag: Forestry Climate Change Adaptation Climate change governance Disaster Risk Reduction Natural Resouce management Protected Areas Country Government

9.

Output 3.3. Biodiversity protection National/local governments have improved policies, systems, and partnerships with non-state actors to protect biodiversity and endangered species.

Finding 9. UNDP made valuable contributions to the development and implementation of institutional frameworks and systems for the sustainable management and conservation of biodiversity and endangered species. Key areas of achievement are peatland restoration, the Sulawesi protected area system and the protection of the Sumatran tiger. Delays in the adoption of key regulations and guidelines developed, and difficulties in the establishment of sustainable financial mechanisms risk limiting the impact and sustainability of these interventions.


Tag: Agriculture Forestry Biodiversity Energy Natural Resouce management Protected Areas Effectiveness Efficiency Sustainability Global Environment Facility fund Government Cost-sharing Gender Equality Gender Mainstreaming Women's Empowerment Local Governance Knowledge management Monitoring and Evaluation Partnership Policies & Procedures Civil Societies and NGOs Country Government Capacity Building Advocacy Technical Support

10.

Output 3.5. Hazardous waste management systems strengthened to properly manage, dispose and phase out hazardous chemicals.

Finding 10. UNDP has supported important strides in establishing a policy framework and enforcement strategy for the control of polybromodiphenyl ethers (PBDEs) and unintentional persistent organic pollutants (UPOPs) in the plastic manufacturing and recycling sectors. Key challenges that need to be addressed to realize intended environmental benefits include lack of disposal infrastructure, the cost inefficiency of proposed technologies, and delay in the adoption of key regulations.


Tag: Emission Reduction Waste management Sustainability Policies & Procedures Risk Management Private Sector

11.

Outputs 3.6 and 3.7. Clean energy Inclusive and sustainable solutions adopted to increase energy efficiency and universal modern energy access. (3.6) Policies and regulations issued/adjusted and systems established to increase private sector investments in clean energy. (3.7)

Finding 11. UNDP made significant attempts to put in place incentives to attract private sector investment in renewable energy. The lack of an effective financial mechanism and the lack of inclusion of small producers of renewable energy have constrained the outcomes achieved from UNDP’s contributions to date. There has been no major initiative focused on promoting energy efficiency.


Tag: Climate Change Adaptation Climate change governance Energy Waste management Private Sector Financing Partnership Private Sector

12.

Output 3.8. Climate change adaptation and disaster risk reduction

Policy and technical guidance are in place for integrating climate change adaptation (CCA) and disaster risk reduction (DRR) into spatial and local development planning.

Finding 12. UNDP has successfully supported the integration of CCA into development planning, albeit at a small scale and with limited evidence of replication of this work beyond project boundaries. Given this limited result, and continuing resource constraints, integration of CCA and DRR into spatial planning remains a challenge.


Tag: Climate Change Adaptation Climate change governance Disaster Risk Reduction Effectiveness Efficiency Sustainability Resource mobilization Gender Equality Gender Mainstreaming Gender Parity Women's Empowerment Local Governance Policies & Procedures Country Government Capacity Building Jobs and Livelihoods

13.

Output 3.9. Early recovery

Recovery preparedness is strengthened particularly in the areas of methodology, financing schemes and institutional arrangements.

Finding 13. UNDP has helped improve the planning and management capacities in disaster risk recovery in targeted areas.


Tag: Disaster Recovery Disaster Risk assessments Disaster Risk Reduction Natural Disaster Knowledge management Policies & Procedures Civil Societies and NGOs Country Government Advocacy Technical Support

14.

Outputs 3.4 and 3.10. Land and forestry management

Solutions developed for sustainable and conflictsensitive management of land and natural resources. (3.4)

Improved local forest management capacity through establishment of conservation forest management units and legal auditing system to monitor and identify violations in issuance of forest licenses. (3.10)

Finding 14. UNDP during this cycle has not made significant interventions towards improving government capacity in addressing recurring conflicts in land tenure and natural resource management (output 3.4.), nor for helping to establish and operationalize conservation forest management units and a legal auditing system to monitor and identify violations in the issuance of forest licenses. (Output 3.10).


Tag: Forestry Natural Resouce management Protected Areas Country Government Capacity Building

15.

2.4 Enhanced access to justice and more responsive, inclusive and accountable public institutions (outcome 4)

RELATED CPD OUTCOMES Outcome 4 goal: By 2020, disadvantaged populations benefit from enhanced access to justice and more responsive, inclusive and accountable public institutions that enjoy public trust.

The outcome encompasses nearly 6.5 percent of the CPD’s indicative budget and 22 percent of total expenditures from 2016 to October 2019.

Output 4.1. Access to justice

Capacity of targeted justice service providers increased to deliver quality legal services.

Finding 15. UNDP has made a substantial contribution to improving the capacity of the Supreme Court. In particular, by establishing improved case management, human resources management, and complaints management systems, UNDP helped to reduce the backlog of outstanding cases, improve transparency over case records, and reduce opportunities for corruption. It is likely, though this cannot be verified, that this has improved the quality of legal services, compared to if no external assistance had been provided.


Tag: Effectiveness Impact Sustainability Anti-corruption Civic Engagement e-Governance Human rights Justice system Rule of law Monitoring and Evaluation Multilateral Partners Policies & Procedures Capacity Building

16.

Output 4.2. Democratic governance

Supported subnational governments are better able to address democratic deficits and engage with non-state actors to improve performance.

Finding 16. UNDP’s support in promoting improved democratic governance has produced good results in the past, but is currently very limited. UNDP has recently commenced supporting the ongoing expansion of Indonesia’s national complaint handling system with funding from South Korea. It is too early to assess the results of this work. 


Tag: Local Governance Bilateral partners Data and Statistics

17.

Output 4.3. Preventing violent extremism

National policy frameworks and institutional mechanisms enhanced for peaceful management of conflicts.

Finding 17. UNDP has made an important contribution to informing the national debate and response to the growing phenomenon of intolerant, radical, and violent extremist interpretation of religious teachings in Indonesia. 


Tag: Human rights Justice system Rule of law Knowledge management Multilateral Partners Partnership Bilateral partners Country Government Conflict resolution Crisis prevention Extremism Advocacy Policy Advisory

18.

Output 4.4. South-South and triangular cooperation

South-South and triangular cooperation partnerships established and/or strengthened for development solutions.

Finding 18. UNDP has provided valuable contributions to the formulation of Indonesia’s international development cooperation, and establishment of a new stand-alone Agency for International Development (Indonesian AID) in 2019, which will provide an important platform for Indonesia’s foreign policy aspirations. UNDP has actively supported and facilitated Indonesia’s engagement in South-South and triangular cooperation.


Tag: Government Cost-sharing Donor relations Partnership Bilateral partners South-South Cooperation Technical Support

19.

2.5 Gender equality

UNDP uses a gender marker system to enable it to monitor the focus of programmes on, and their expected contribution to, gender equality and the empowerment of women. This system provides a basis to distinguish between outputs that: are not expected to contribute noticeably to gender equality (GEN0); make a limited contribution to gender equality (GEN1); promote gender equality in a significant and consistent way (GEN2); or have gender equality as a primary objective (GEN3).

Country programme expenditures that the country office attributed to GEN2 or GEN3 amounted to 55 percent of programme expenditure in 2016–2018, which is double the long-term (2011–2018) average (27 percent). This suggests significantly greater attention to gender equality in programming.


Tag: Agriculture Adaptation Fund Biodiversity Energy Environment Policy Effectiveness Impact Gender Equality Gender Mainstreaming Gender Parity Women's Empowerment Local Governance Monitoring and Evaluation Results-Based Management Extremism Capacity Building Inclusive economic growth Jobs and Livelihoods

20.

Joint programming

Among the UN country team in Indonesia, weak overall joint programming has been highlighted as an issue in the past. This is underscored by the following statement of the UN SDG Group meeting summary of 18 February 2019: “The UNCT [UN country team] recognized that while the UN Development System in Indonesia is very actively supporting the 2030 Agenda and the SDGs, the support remains largely agency-based with limited linkages established between the various activities of the UN system. This makes it difficult for the UNCT itself as well as for the Government and other partners to understand the scope, strength and complementarity of the UN support to the SDGs.”


Tag: Disaster Risk Reduction Green Climate Coherence Gender Equality Women's Empowerment Human rights Health Sector HIV / AIDS Integration Joint UN Programme Partnership Programme Synergy Country Government UN Agencies UN Country Team Extremism Security Education Jobs and Livelihoods Social Protection Youth Advocacy Agenda 2030 SDG Integration

21.

2.6 Country programme content

UNDP’s country programme establishes the context for UNDP’s work in Indonesia, but reads as highly aspirational about the work that UNDP was expected to undertake and the results it was expected to achieve. This reflects a lack of flexible resources, combined with a lack of predictability of donor interests, and where or how they will align with UNDP’s comparative advantage, and the Government’s policies. It also reflects a lack of realism about how hard it is to achieve significant policy and institutional reforms in a country as large and complex as Indonesia.

 


Tag: Coherence Global Environment Facility fund Donor relations Policies & Procedures Results-Based Management

22.

2.7 Results-based management

The design of the results and resources framework does not provide a good framework for communicating what UNDP is trying to achieve in Indonesia, and how it is progressing towards these objectives. Targeted outcomes, derived from UNPDF 2016–2020, provide an organizing framework for the United Nations support for Indonesia. However, associated indicators are too high level to provide any practical or meaningful insight into UNDP’s impact. If UNDP were to be assessed in terms of its influence over established outcome indicators, the organization contributes modestly to agreed country programme outcomes. UNDP has high or moderate influence over just two of the 21 outcome indicators established to assess UNDP’s contributions. To cite one of many possible examples, at the outcome level, the level of funding and engagement is insufficient to have a significant bearing on poverty rates and the Gini index for Papua and West Papua. It is problematic, therefore, for the country office to continue to source and set out, as it does, values for these indicators into UNDP’s results measurement system.


Tag: Quality Assurance Results-Based Management Theory of Change Poverty Reduction

23.

2.8 Audit and evaluation

There is room for improvement in the country office’s utilization of evaluations. The Indonesia country office has 24 evaluations in its evaluation work plan for the programme period 2016–2020. Of these, five cover projects outside of the climate change and environmental management outcome (outcome 3). Three years into the CPD period, nine evaluations have been completed. Only one covers work outside of the environment and climate change pillar. The IEO has assessed the quality of five of the nine completed evaluations, of which two were found to be unsatisfactory (e.g. Support for the Peat Restoration Authority, and Safer Communities through Disaster Risk Reduction phase two).


Tag: Resource mobilization Monitoring and Evaluation Procurement Risk Management

Recommendations
1

For the next cycle, UNDP should revise and consolidate its country programme in Indonesia, promoting a narrower set of outcomes and outputs that show promise for achieving scale and impact.

To maximize the potential impacts of programme interventions, especially, but not exclusively in the case of pilots or geographically delimited interventions, UNDP should endeavour to increase the scale of proven interventions and avoid engaging in small pilots of untested interventions that have limited prospects of influencing Indonesian Government policies or programmes.

UNDP should attempt to build on its role as a convener among development partners in Indonesia, strengthening the work it is currently doing in Papua and West Papua, and on countering violent extremism. There is also good potential for UNDP to expand its support to Indonesia’s foreign aid aspirations, including as a facilitator of South-South cooperation. Given the limited prospects for scaling up existing interventions relating to sustainable palm oil production, UNDP should consider getting out of this business.
 

2

The sustainable development portfolio, including disaster risk reduction, anchors the UNDP country programme in Indonesia, yet UNDP manages just a small fraction of environment and climate change financial inflows into the country. There is significant potential for growth in this area and UNDP should develop a strategy to expand its expertise and services.

In particular, UNDP should seek to capitalize on its strong position as a provider of environmental financial services, building on its successful advisory role in the issuance of Indonesia’s first-ever sovereign green bond.

3

In articulating its objectives, the next country programme should strive for more modest output descriptions that accurately reflect the substance and scope of anticipated work.

The design of the results and resources framework should be clearer on what UNDP is trying to achieve in Indonesia, and how it is progressing on these objectives. Outcome indicators are too high level to provide any practical or meaningful insight into UNDP’s impact. Output descriptions and associated output indicators should better reflect what UNDP can influence, taking into account resources and government access.
 

4

The country office should build on its strong efforts to mainstream gender equality and the empowerment of women across its programme.

While there is good evidence that the country office has a strong focus on gender mainstreaming, there are still gaps in attention to gender equality in both designs and reporting. In the next cycle, UNDP should give more consistent attention to the gender equality dimensions of its work in programme designs, regular performance reporting, and evaluations; and increase the accuracy of the gender marker rating so it better reflects likely contributions to improving gender equality. Monitoring and evaluation of gender-related outcomes should be enhanced with a view to obtaining deeper insights into the impacts of the country office’s commendable gender mainstreaming efforts.
 

Latest Evaluations

Contact us

1 UN Plaza
DC1-20th Floor
New York, NY 10017
Tel. +1 646 781 4200
Fax. +1 646 781 4213
erc.support@undp.org