Evaluation of Outcome 5 of the Country Programme Document 2015-2019: By 2019, legal and strategic frameworks are enhanced and operationalized to ensure sustainable management of natural, cultural and energy resources

Report Cover Image
Evaluation Plan:
2015-2019, Bosnia and Hercegovina
Evaluation Type:
Outcome
Planned End Date:
11/2018
Completion Date:
11/2018
Status:
Completed
Management Response:
Yes
Evaluation Budget(US $):
17,000

Share

Document Type Language Size Status Downloads
Download document ToR International Consultant for Evaluation of Outcome 5.pdf tor English 450.77 KB Posted 60
Download document BiH Outcome Evaluation Report.pdf report English 1070.23 KB Posted 40
Title Evaluation of Outcome 5 of the Country Programme Document 2015-2019: By 2019, legal and strategic frameworks are enhanced and operationalized to ensure sustainable management of natural, cultural and energy resources
Atlas Project Number: 00050813
Evaluation Plan: 2015-2019, Bosnia and Hercegovina
Evaluation Type: Outcome
Status: Completed
Completion Date: 11/2018
Planned End Date: 11/2018
Management Response: Yes
Corporate Outcome and Output (UNDP Strategic Plan 2018-2021)
  • 1. Output 1.4.1 Solutions scaled up for sustainable management of natural resources, including sustainable commodities and green and inclusive value chains
  • 2. Output 2.1.1 Low emission and climate resilient objectives addressed in national, sub-national and sectoral development plans and policies to promote economic diversification and green growth
Evaluation Budget(US $): 17,000
Source of Funding: UNDP core programme and project resources
Evaluation Expenditure(US $): 15,376
Joint Programme: Yes
Mandatory Evaluation: No
Joint Evaluation: No
Evaluation Team members:
Name Title Email Nationality
Elinor Bajraktari International Evaluation Consultant elinorbajraktari@gmail.com BOSNIA AND HERCEGOVINA
GEF Evaluation: No
Key Stakeholders: BiH Ministry of Foreign Trade and Economic Relations, BiH Ministry of Civil Affairs, FBIH and RS Ministry of Agriculture, Water and Forestry, Ministry of Environment and Tourism, environmental institutes, cantonal governments, other UN Agencies
Countries: BOSNIA AND HERCEGOVINA
Lessons
Findings
Recommendations
1

Positioning and Resource Mobilization: As far as resource mobilization is concerned, the main recommendation for the E&E sector is to explore more systematically the significant opportunities that have opened up with the EU after the adoption of the environmental and energy strategies. The CO could undertake a more systematic assessment and identification of opportunities by researching how the EU is currently detailing their priorities into specific programmes and actions on the basis of the strategic and approximation plans of the state and the entities.  In this context, the CO could launch a more organized process of exploration concerning all sectors and involving higher-level UNDP structures in Brussels and New York.

2

Programme Integration at the Sub-national Level:  At the sub-national level, the CO should strengthen collaboration between sectors by establishing integrated frameworks for project planning and implementation. RRD activities, especially the component on strategic planning at the local level, provide the E&E sector with a platform on which to embed environmental and energy efficiency initiatives. The CO could explore the feasibility of integrated work plans elaborated at the regional/local level and matched with the CO’s plan at the national level. An example of this would be the use of UNDP’s local presence (i.e. ILDP project) as vehicles for the implementation of UNDP projects in the respective areas.Such an area-based approach will enable UNDP to weave more effectively cross-cutting issues (such as energy efficiency, citizen engagement, transparency and accountability, gender equality) into other thematic activities (i.e. community development, service delivery, etc.). Stronger synergies may also be forged with international organizations at the sub-national level, which may also provide increased funding opportunities. UNDP can also support local authorities to facilitate more effectively donor coordination at the sub-national level. A crucial step in achieving a higher level of programme integration and consolidation could be the development of a clearer strategy for how UNDP should structure itself and operate at the sub-national level.

 

3

Policy Implementation: UNDP should further strengthen its focus on policy implementation, by thinking beyond just the passing of laws and strategies, and considering measures that consolidate organizational structures that will implement those laws and strategies. This includes actions like the creation of organizational structures, staffing organizations and allocating funding for their operations, training management and staff to implement policies, etc. The sector team has already been doing a lot of this, but the point here is to promote a mentality shift in the programme and in the government away from “form” (how a piece of law looks like) to functionality (how a law is implemented and what effects it produces).  From this perspective, it is important that the team consider how the capability of government organizations is built and changes. For this, the CO should develop RBM systems that track implementation parameters linked to functionality and outcomes rather than form and inputs/outputs and assess more rigorously the sustainability of achievements. Project documents should contain clear criteria related to performance based on a strategy for achieving and demonstrating results. Achieving this focus on functionality and outcomes is difficult when considering the short timeframes of UNDP projects, but it is not impossible. What is important is the mentality shift which implies that UNDP staff start designing and implementing projects with these implementation considerations in mind.

4

Co-financing: The sector team has done a good job when it comes to co-financing, but for projects that involve infrastructure investments it is essential to keep pushing for stronger competitive/market mechanisms to ensure the sustainability and scale of initiatives. Overall, the recommendation here is to stay on the same path and not backtrack, because market-based solutions to infrastructure problems are extremely important for their sustainability. Instead of providing grants, UNDP should keep strengthening incentives that promote access to international financial institutions and banks for finance. One opportunity UNDP could explore further is engaging with some of the lending activities of International Financial Institutions (IFIs) at the sub-national level and helping with the monitoring of loan operations through its partnerships with local governments and communities. Ensuring good governance and transparency at the local level are areas of work where UNDP has a comparative advantage, so partnerships with IFIs will be a win-win situation. This is area the CO could look into with the objective of coming up with a clear strategy and plan of action – and not only in the E&E area, but across the programme.

5

Awareness Raising:

In the area of awareness raising and information sharing, the CO should consider more strategically and systematically its approaches, methodologies and results. This is an area where there have been significant shifts in research and practice recently and it is time for UNDP to upgrade its approach. First, the CP should recognize the information sharing and awareness raising are done for a simple reason – to change people’s behavior. So, when designing  information campaigns and events, it is important to ask what behavior and whose behavior the programme or project is seeking to change. This requires a lot of careful thinking about the type of behavior the programme/project seeks to promote and the agents whose behavior it wants to change. As a next step, it is also important to understand what type of information and what channel of information has the potential to change the identified behavior in the target group. The way the information is packaged matters a lot, but who carries the information and how that person is perceived by the target group matters even more. In this sense, it is important to understand whose opinion matters for the target group and how that opinion can be constructed and used to influence behavior. It is also important to recognize that individuals operate in a social environment and that human behavior is largely influenced by social norms set by the community in which an individual embedded. So, to change an individual’s behavior, it is important to understand the prevailing social norms in his/her community and the factors that shape those social norms. This is something that the CO could examine a bit more closely in the context of the development of the new CPD and new projects.

 

6

Result-Based Management at the Sectoral Level: In preparation for the development of the new CPD, the Sector Team and CO management may consider some of the measures proposed below which are geared towards strengthening the RBM system at the sectoral (programme) level:

  • For the upcoming CPD, the sector should develop a stronger results framework based on SMART indicators, baselines and targets. The fact that a number of large projects have just started provides good predictability for the types of activities that the CO will be pursuing in this area in the new programme, which makes the development of a sound RRF easier.
  • The sector should also ensure that programme baselines, indicators and targets are harmonized and aligned with those of individual projects. Also, data collection approaches, means of verification and risks and assumptions should be harmonized between the programme and project levels.
  • The sector will also benefit from the development of a Theory of Change that connects all the specific pieces (projects) together. This is not just a theoretical exercise, but has practical value in that it will provide the team with insights into how these individual projects could be tied more effectively together.
  •  Also, the quality of project evaluations is something that could be improved. These are things that the CO could address in the formulation of the next CPD. Also, the CO should develop minimum quality criteria for project evaluations and should establish a tracking system to closely monitor their quality.
1. Recommendation:

Positioning and Resource Mobilization: As far as resource mobilization is concerned, the main recommendation for the E&E sector is to explore more systematically the significant opportunities that have opened up with the EU after the adoption of the environmental and energy strategies. The CO could undertake a more systematic assessment and identification of opportunities by researching how the EU is currently detailing their priorities into specific programmes and actions on the basis of the strategic and approximation plans of the state and the entities.  In this context, the CO could launch a more organized process of exploration concerning all sectors and involving higher-level UNDP structures in Brussels and New York.

Management Response: [Added: 2018/12/26] [Last Updated: 2018/12/26]

The recommendation is valid and accepted.

Key Actions:

Key Action Responsible DueDate Status Comments Documents
The E&E Sector will in the next years target EU funds for implementation of environmental and energy based projects. This will be done by tracking BiH’s need articulated via SPDs and Ads and positioning UNDP CO BIH as one of the key and most reliable partners in this sector.
[Added: 2018/12/26]
UNDP, Energy and Environment Sector, CO senior management 2020/12 Initiated
The E&E Sector will start to build its capacities in order to be able to gather financial resources (obtain resource mobilization) in the area of biodiversity and chemicals (including medical waste).
[Added: 2018/12/26]
UNDP, Energy and Environment Sector and CO senior management. 2020/12 Initiated
Although the E&E Sector already highlights SDGs in all of its project Documents which were developed after the initiation of SDGs, we will do our very best to even more highlight it and to provide internal E&E Sector capacities (human and financial) for the coordination and monitoring of SDGs in BiH (among both – government and UN/UNDP internally).
[Added: 2018/12/26]
UNDP, Energy and Environment Sector 2020/12 Initiated
2. Recommendation:

Programme Integration at the Sub-national Level:  At the sub-national level, the CO should strengthen collaboration between sectors by establishing integrated frameworks for project planning and implementation. RRD activities, especially the component on strategic planning at the local level, provide the E&E sector with a platform on which to embed environmental and energy efficiency initiatives. The CO could explore the feasibility of integrated work plans elaborated at the regional/local level and matched with the CO’s plan at the national level. An example of this would be the use of UNDP’s local presence (i.e. ILDP project) as vehicles for the implementation of UNDP projects in the respective areas.Such an area-based approach will enable UNDP to weave more effectively cross-cutting issues (such as energy efficiency, citizen engagement, transparency and accountability, gender equality) into other thematic activities (i.e. community development, service delivery, etc.). Stronger synergies may also be forged with international organizations at the sub-national level, which may also provide increased funding opportunities. UNDP can also support local authorities to facilitate more effectively donor coordination at the sub-national level. A crucial step in achieving a higher level of programme integration and consolidation could be the development of a clearer strategy for how UNDP should structure itself and operate at the sub-national level.

 

Management Response: [Added: 2018/12/26] [Last Updated: 2018/12/26]

The recommendation is valid and accepted.

Key Actions:

Key Action Responsible DueDate Status Comments Documents
The E&E Sector will cooperate and with our RRD sector in order to strengthen collaboration between sectors and thus enable integrated frameworks for project planning and implementation.
[Added: 2018/12/26]
UNDP, Energy and Environment Sector 2020/12 Initiated History
Although the E&E Sector already has very strong synergies and coordination with other donors and international community in BiH on sub-national level (EE, RES, DRR, Water mngt etc.) we will further seek for opportunities to strengthen those and to, were possible, formalize via MoUs and/or exchange of letters.
[Added: 2018/12/26]
UNDP, Energy and Environment Sector 2020/12 Initiated
3. Recommendation:

Policy Implementation: UNDP should further strengthen its focus on policy implementation, by thinking beyond just the passing of laws and strategies, and considering measures that consolidate organizational structures that will implement those laws and strategies. This includes actions like the creation of organizational structures, staffing organizations and allocating funding for their operations, training management and staff to implement policies, etc. The sector team has already been doing a lot of this, but the point here is to promote a mentality shift in the programme and in the government away from “form” (how a piece of law looks like) to functionality (how a law is implemented and what effects it produces).  From this perspective, it is important that the team consider how the capability of government organizations is built and changes. For this, the CO should develop RBM systems that track implementation parameters linked to functionality and outcomes rather than form and inputs/outputs and assess more rigorously the sustainability of achievements. Project documents should contain clear criteria related to performance based on a strategy for achieving and demonstrating results. Achieving this focus on functionality and outcomes is difficult when considering the short timeframes of UNDP projects, but it is not impossible. What is important is the mentality shift which implies that UNDP staff start designing and implementing projects with these implementation considerations in mind.

Management Response: [Added: 2018/12/26] [Last Updated: 2018/12/26]

The recommendation is valid and accepted

Key Actions:

Key Action Responsible DueDate Status Comments Documents
The E&E Sector will analyze the creation of organizational structures, staffing organizations and allocating funding for governmental operations, training management and staff to implement policies developed and adopted with the support of UNDP E&E projects. Based on that analysis all new UNDP E&E projects and initiatives (project documents) will include a RBM systems that tracks implementation parameters linked to functionality and outcomes rather than only form and inputs/outputs.
[Added: 2018/12/26]
UNDP, Energy and Environment Sector 2019/12 Initiated
4. Recommendation:

Co-financing: The sector team has done a good job when it comes to co-financing, but for projects that involve infrastructure investments it is essential to keep pushing for stronger competitive/market mechanisms to ensure the sustainability and scale of initiatives. Overall, the recommendation here is to stay on the same path and not backtrack, because market-based solutions to infrastructure problems are extremely important for their sustainability. Instead of providing grants, UNDP should keep strengthening incentives that promote access to international financial institutions and banks for finance. One opportunity UNDP could explore further is engaging with some of the lending activities of International Financial Institutions (IFIs) at the sub-national level and helping with the monitoring of loan operations through its partnerships with local governments and communities. Ensuring good governance and transparency at the local level are areas of work where UNDP has a comparative advantage, so partnerships with IFIs will be a win-win situation. This is area the CO could look into with the objective of coming up with a clear strategy and plan of action – and not only in the E&E area, but across the programme.

Management Response: [Added: 2018/12/26] [Last Updated: 2018/12/26]

The recommendation is valid and accepted.

Key Actions:

Key Action Responsible DueDate Status Comments Documents
The E&E Sector will keep strengthening incentives that promote access to international financial institutions and banks for finance in BiH. This will be done by promoting UNDP’s comparative and competitive advantage in the area of good governance and transparency, as well as project implementation and monitoring. The E&E Sector will seek for cooperation with EIB, KfW, EBRD, EIB, WB and other IFIs based in BiH.
[Added: 2018/12/26]
UNDP, Energy and Environment Sector 2020/12 Initiated
5. Recommendation:

Awareness Raising:

In the area of awareness raising and information sharing, the CO should consider more strategically and systematically its approaches, methodologies and results. This is an area where there have been significant shifts in research and practice recently and it is time for UNDP to upgrade its approach. First, the CP should recognize the information sharing and awareness raising are done for a simple reason – to change people’s behavior. So, when designing  information campaigns and events, it is important to ask what behavior and whose behavior the programme or project is seeking to change. This requires a lot of careful thinking about the type of behavior the programme/project seeks to promote and the agents whose behavior it wants to change. As a next step, it is also important to understand what type of information and what channel of information has the potential to change the identified behavior in the target group. The way the information is packaged matters a lot, but who carries the information and how that person is perceived by the target group matters even more. In this sense, it is important to understand whose opinion matters for the target group and how that opinion can be constructed and used to influence behavior. It is also important to recognize that individuals operate in a social environment and that human behavior is largely influenced by social norms set by the community in which an individual embedded. So, to change an individual’s behavior, it is important to understand the prevailing social norms in his/her community and the factors that shape those social norms. This is something that the CO could examine a bit more closely in the context of the development of the new CPD and new projects.

 

Management Response: [Added: 2018/12/26] [Last Updated: 2018/12/26]

The recommendation is valid and accepted.

Key Actions:

Key Action Responsible DueDate Status Comments Documents
For all new projects and initiatives, and for on-going projects were possible, the E&E Sector will start designing information campaigns and events in such a way that the outcome should lead towards change of people’s behavior of a certain project related target group.
[Added: 2018/12/26]
UNDP, Energy and Environment Sector 2019/12 Initiated
6. Recommendation:

Result-Based Management at the Sectoral Level: In preparation for the development of the new CPD, the Sector Team and CO management may consider some of the measures proposed below which are geared towards strengthening the RBM system at the sectoral (programme) level:

  • For the upcoming CPD, the sector should develop a stronger results framework based on SMART indicators, baselines and targets. The fact that a number of large projects have just started provides good predictability for the types of activities that the CO will be pursuing in this area in the new programme, which makes the development of a sound RRF easier.
  • The sector should also ensure that programme baselines, indicators and targets are harmonized and aligned with those of individual projects. Also, data collection approaches, means of verification and risks and assumptions should be harmonized between the programme and project levels.
  • The sector will also benefit from the development of a Theory of Change that connects all the specific pieces (projects) together. This is not just a theoretical exercise, but has practical value in that it will provide the team with insights into how these individual projects could be tied more effectively together.
  •  Also, the quality of project evaluations is something that could be improved. These are things that the CO could address in the formulation of the next CPD. Also, the CO should develop minimum quality criteria for project evaluations and should establish a tracking system to closely monitor their quality.
Management Response: [Added: 2018/12/26] [Last Updated: 2018/12/26]

The recommendation is valid and accepted

Key Actions:

Key Action Responsible DueDate Status Comments Documents
The E&E Sector will organize an internal brainstorming in order to develop a Theory of Change (ToC) that connects all the specific projects.
[Added: 2018/12/26]
UNDP, Energy and Environment Sector (programme and projects) 2019/12 Initiated
The E&E Sector will, in coordination with other sector and CO senior management, develop a stronger results framework based on SMART indicators, baselines and targets ensuring that programme baselines, indicators and targets are harmonized and aligned with those of individual projects and projects in other sectors (where synergies are to be created).
[Added: 2018/12/26]
UNDP, Energy and Environment Sector (programme and projects) 2019/12 Initiated
The E&E Sector will in close cooperation with UNDP’s M&E staff develop minimum quality criteria for project evaluations and will also establish its internal (sectoral) tracking system to closely monitor their quality and thus improve the quality of project evaluations.
[Added: 2018/12/26]
UNDP, Environment Sector (programme and projects) and UNDP M&E. 2019/12 Initiated

Latest Evaluations

Contact us

220 East 42nd Street
20th Floor
New York, NY 10017
Tel. +1 646 781 4200
Fax. +1 646 781 4213
erc.support@undp.org