Governance Outcome Evaluation

Report Cover Image
Evaluation Plan:
2006-2010, Indonesia
Evaluation Type:
Outcome
Planned End Date:
01/2007
Completion Date:
02/2007
Status:
Completed
Management Response:
Yes
Evaluation Budget(US $):
15,000

Share

Document Type Language Size Status Downloads
Download document TOR_GU_outcome_eval_indonesia.doc tor Posted 966
Download document Outcome Evaluation Governance_01_05_ Final.doc report English Posted 1312
Title Governance Outcome Evaluation
Atlas Project Number:
Evaluation Plan: 2006-2010, Indonesia
Evaluation Type: Outcome
Status: Completed
Completion Date: 02/2007
Planned End Date: 01/2007
Management Response: Yes
Focus Area:
  • 1. Democratic Governance
Corporate Outcome and Output (UNDP Strategic Plan 2018-2021)
  • 1. Strategic framework in support of good governance practices at the sub-national level, together with a commitment for coordinated oversight mechanisms, is agreed upon by donors and national and sub-national stakeholders.
Evaluation Budget(US $): 15,000
Source of Funding:
Joint Programme: No
Joint Evaluation: No
Evaluation Team members:
Name Title Nationality
Tim Lankaster and Medelina Hendityo Team Leader
GEF Evaluation: No
Key Stakeholders:
Countries: INDONESIA
Comments: Start Jan 11th 2007
Lessons
Findings
Recommendations
1 1). If the success of UNDP interventions is to be judged against specified outcome goals, these goals need to be specified more carefully in future. 2) Ownership of projects by the executing agency and implementing partner, and by the government more generally, is crucial for their success. Although local ownership is one of UNDP's mantras, the experience of the past few years shows that this was sometimes lacking. This was apparent with the NLC project and with the parliamentary projects 3) Where full ownership is not evident at the planning stage, UNDP should be cautious before proceeding (even though the project may be addressing a key "governance deficit"); and where ownership appears lacking during project implementation, UNDP should take this up with the relevant agency without delay. In extreme cases, UNDP should be ready to put the project on hold or cancel it. 4) Local ownership will only happen if there are excellent relations and a sense of mutual confidence between UNDP and the relevant agencies at the design and implementation stages. This requires a great deal of effort on both sides at both operational and more senior level 5) More attention needs to be given to the issue of institutionalization of project gains. In several of the projects we examined, this was a serious issue. There are no easy solutions, particularly given the pressure on budgets. In some cases, the project timetable was too short; in others, the intensity of the intervention may have been inadequate. In the case of the main electoral support project in 2004, the intervention was extremely intensive - indeed, so intensive that when the project staff disbanded, they took a lot of the knowledge and experience away with them. In general, more consideration needs to be given at the planning stage - and with a greater sense of realism - as to how the activity will be carried forward once the project is finished and how the project accomplishments will be embedded. 6) A related issue is the question of exit and possible follow-on projects. With several projects, it has been suggested that they came to a premature end when their continuation - or follow-on - could have led to further significant gains, Project stakeholders, including project staff, of course often have a vested interest in continuation. Nonetheless, we got the sense that UNDP had not given adequate consideration, and at an earlier enough stage, to exit or follow-on. 7) UNDP needs to recruit and retain high quality staff for governance work, which is operationally and intellectually demanding. Retention is important to ensure there is institutional memory and that there are staff who understand UNDP's quite complex operating procedures. There is competition for good local staff amongst the donors and UNDP cannot match private sector salaries. There have also been problems in recruiting international staff from within UNDP's network. So there are no easy solutions; but it has to be noted that high turnover of staff, as well as gaps in staffing, in the Governance Unit have reduced its effectiveness. A prime example is the time it took to put together the cluster of projects under the heading, "Support for Deepening Democracy in Indonesia". These projects appear to be well judged in design, but the hiatus in project activity after 2004 has meant some loss of momentum and loss of previously achieved benefits. 8) UNDP should consider investing more in analytical work so as to build its credibility with the government and donors on governance issues, and so as to inform its choice and design of governance projects. 9) While there have been some good evaluation and other reports on the earlier projects, it seems to us that monitoring and evaluation have been somewhat haphazard; and some of the problems that arose during implementation could have been dealt with if there had been better monitoring. There needs to be a specific monitoring and evaluation plan built into the programme, with clear responsibility ascribed (as appropriate) to the executing agent, the Governance Unit and Planning, Monitoring and Evaluation Unit. 10).Given the relative lack of own financial resources, UNDP will have to put a great deal of effort into engaging with other donors and persuading them that UNDP governance projects represent a sensible object for their funding. With the improvement in the public finances, UNDP should also give attention to securing larger financial contributions to its projects from national and local governments. 11) Given the continuing weaknesses in governance as outlined in paragraph 33, and subject to resource availability, the Governance Unit should give particular attention to how - within the political constraints that currently exist - it might provide further assistance in the areas of judicial reform and anti-corruption (especially at provincial and local level).
2 Ownership of projects by the executing agency and implementing partner, and by the government more generally, is crucial for their success. Although local ownership is one of UNDP's mantras, the experience of the past few years shows that this was sometimes lacking. This was apparent with the NLC project and with the parliamentary projects
3 Where full ownership is not evident at the planning stage, UNDP should be cautious before proceeding (even though the project may be addressing a key "governance deficit"); and where ownership appears lacking during project implementation, UNDP should take this up with the relevant agency without delay. In extreme cases, UNDP should be ready to put the project on hold or cancel it.
1. Recommendation: 1). If the success of UNDP interventions is to be judged against specified outcome goals, these goals need to be specified more carefully in future. 2) Ownership of projects by the executing agency and implementing partner, and by the government more generally, is crucial for their success. Although local ownership is one of UNDP's mantras, the experience of the past few years shows that this was sometimes lacking. This was apparent with the NLC project and with the parliamentary projects 3) Where full ownership is not evident at the planning stage, UNDP should be cautious before proceeding (even though the project may be addressing a key "governance deficit"); and where ownership appears lacking during project implementation, UNDP should take this up with the relevant agency without delay. In extreme cases, UNDP should be ready to put the project on hold or cancel it. 4) Local ownership will only happen if there are excellent relations and a sense of mutual confidence between UNDP and the relevant agencies at the design and implementation stages. This requires a great deal of effort on both sides at both operational and more senior level 5) More attention needs to be given to the issue of institutionalization of project gains. In several of the projects we examined, this was a serious issue. There are no easy solutions, particularly given the pressure on budgets. In some cases, the project timetable was too short; in others, the intensity of the intervention may have been inadequate. In the case of the main electoral support project in 2004, the intervention was extremely intensive - indeed, so intensive that when the project staff disbanded, they took a lot of the knowledge and experience away with them. In general, more consideration needs to be given at the planning stage - and with a greater sense of realism - as to how the activity will be carried forward once the project is finished and how the project accomplishments will be embedded. 6) A related issue is the question of exit and possible follow-on projects. With several projects, it has been suggested that they came to a premature end when their continuation - or follow-on - could have led to further significant gains, Project stakeholders, including project staff, of course often have a vested interest in continuation. Nonetheless, we got the sense that UNDP had not given adequate consideration, and at an earlier enough stage, to exit or follow-on. 7) UNDP needs to recruit and retain high quality staff for governance work, which is operationally and intellectually demanding. Retention is important to ensure there is institutional memory and that there are staff who understand UNDP's quite complex operating procedures. There is competition for good local staff amongst the donors and UNDP cannot match private sector salaries. There have also been problems in recruiting international staff from within UNDP's network. So there are no easy solutions; but it has to be noted that high turnover of staff, as well as gaps in staffing, in the Governance Unit have reduced its effectiveness. A prime example is the time it took to put together the cluster of projects under the heading, "Support for Deepening Democracy in Indonesia". These projects appear to be well judged in design, but the hiatus in project activity after 2004 has meant some loss of momentum and loss of previously achieved benefits. 8) UNDP should consider investing more in analytical work so as to build its credibility with the government and donors on governance issues, and so as to inform its choice and design of governance projects. 9) While there have been some good evaluation and other reports on the earlier projects, it seems to us that monitoring and evaluation have been somewhat haphazard; and some of the problems that arose during implementation could have been dealt with if there had been better monitoring. There needs to be a specific monitoring and evaluation plan built into the programme, with clear responsibility ascribed (as appropriate) to the executing agent, the Governance Unit and Planning, Monitoring and Evaluation Unit. 10).Given the relative lack of own financial resources, UNDP will have to put a great deal of effort into engaging with other donors and persuading them that UNDP governance projects represent a sensible object for their funding. With the improvement in the public finances, UNDP should also give attention to securing larger financial contributions to its projects from national and local governments. 11) Given the continuing weaknesses in governance as outlined in paragraph 33, and subject to resource availability, the Governance Unit should give particular attention to how - within the political constraints that currently exist - it might provide further assistance in the areas of judicial reform and anti-corruption (especially at provincial and local level).
Management Response: [Added: 2007/09/27]

1) As commended by the evaluator, a draft Governance Strategy Paper 2006 - 2010 has been developed for clearer focus of UNDP governance support for the country based on good analysis of the country's need. We will finalize the strategy paper and further refine the outcome goals and annual targets to reflect specific intervention UNDP will pursue. Also the Unit will hire a M&E specialist to train the staff in formulating better result based projects / programme framework 2) Based on the lessons learned from projects implemented in 2001 - 2001 period, new projects and programmes have been formulated with full ownership of government counterparts - both in overall coordinating and specific implementing roles through various ways - involving them in planning, monitoring and evaluations (CPAP focus group discussion). NPDs role has been strengthened by clearer definition of roles and responsibilities. 3) In addition to more consultative process for project formulation, coordination wi

Key Actions:

Key Action Responsible DueDate Status Comments Documents
Finalization of Governance Strategy Paper 2006 - 2010 Hiring a M&E specialist for improved RBM
[Added: 2007/09/27]
Governance 2007/12 Overdue-Initiated -
Strengthen the involvement of Bappenas as coordinating agency in addressing non-performing project and other serious implementation issues to ensure national ownership in solving problems. Establishing PEG per project. Engagement in the discussion of PP2 / 2006 operationalization and developing PMIG including details on collective decision making process
[Added: 2007/09/27]
Unit Head 2007/12 Overdue-Initiated Bappenas already involved in addressing overdue audit findings and assigning new NPDs. Discussion with implementing partners (NPD) on the establishment of PEG is on-going. Zero draft of PMIG is available for further discussion and refinement
Involvement of government stakeholders in formulation, planning (planning workshops) and monitoring and evaluation (CPAP review process)
[Added: 2007/09/27]
Unit Head 2007/09 Completed Good level of ownership from NPD for most projects
2. Recommendation: Ownership of projects by the executing agency and implementing partner, and by the government more generally, is crucial for their success. Although local ownership is one of UNDP's mantras, the experience of the past few years shows that this was sometimes lacking. This was apparent with the NLC project and with the parliamentary projects
Management Response: [Added: 2007/10/09]

Based on the lessons learned from projects implemented in 2001 - 2001 period, new projects and programmes have been formulated with full ownership of government counterparts - both in overall coordinating and specific implementing roles through various ways - involving them in planning, monitoring and evaluations (CPAP focus group discussion). NPDs role has been strengthened by clearer definition of roles and responsibilities.

Key Actions:

Key Action Responsible DueDate Status Comments Documents
Involvement of government stakeholders in formulation, planning (planning workshops) and monitoring and evaluation (CPAP review process) Briefings to NPDs on their roles and responsibilities in project implementation. ToR developed and shared with NPDs. New project formulation are being done in close consultation with Bappenas and future implementing partners Once new project director appointed, comprehensive briefings will be made
[Added: 2007/10/09]
GU 2007/10 Completed -
3. Recommendation: Where full ownership is not evident at the planning stage, UNDP should be cautious before proceeding (even though the project may be addressing a key "governance deficit"); and where ownership appears lacking during project implementation, UNDP should take this up with the relevant agency without delay. In extreme cases, UNDP should be ready to put the project on hold or cancel it.
Management Response: [Added: 2007/10/09]

In addition to more consultative process for project formulation, coordination with Bappenas strengthened to address the non-performing projects due to lack of ownership (and possibly for other issues). Each project will establish a Project Executive Group to make management decision (including termination, revision and continuation of projects) in collective manner (government, UNDP and other stakeholders) and the role of PEG to be fully understood by partners. Mechanisms to address non-performing projects to be further refined in the new Project Management and Implementation Guideline according to the new Government Regulation No. 2 / 2006 to which CO will contribute inputs.

Key Actions:

Key Action Responsible DueDate Status Comments Documents
Strengthen the involvement of Bappenas as coordinating agency in addressing non-performing project and other serious implementation issues to ensure national ownership in solving problems.
[Added: 2007/10/09]
Unit Heads and Programme Managers in coordination with PMEU 2007/11 Overdue-Initiated Bappenas already involved in addressing overdue audit findings and assigning new NPDs.
Establishing PEG per project.
[Added: 2007/10/09]
Programme Managers: GU 2008/04 Overdue-Initiated Discussion with implementing partners (NPD) on the establishment of PEG is on-going.
Engagement in the discussion of PP2 / 2006 operationalization and developing PMIG including details on collective decision making process
[Added: 2007/10/09]
PMEU / PCBM Manager as GU focal point 2007/12 Overdue-Initiated Draft of PMIG is available for further discussion and refinement

Latest Evaluations

Contact us

1 UN Plaza
DC1-20th Floor
New York, NY 10017
Tel. +1 646 781 4200
Fax. +1 646 781 4213
erc.support@undp.org