External Review and Mid-Term Evaluation of the Project "Afghanistan Access to Justice (AA2J)

Report Cover Image
Evaluation Plan:
2015-2019, Afghanistan
Evaluation Type:
Mid Term Project
Planned End Date:
06/2018
Completion Date:
06/2018
Status:
Completed
Management Response:
Yes
Evaluation Budget(US $):
60,000

Share

Document Type Language Size Status Downloads
Download document Final-MTR-Report-AA2Jproject2018.doc report English 506.50 KB Posted 82
Download document TOR AA2J MTR.pdf tor English 600.03 KB Posted 45
Title External Review and Mid-Term Evaluation of the Project "Afghanistan Access to Justice (AA2J)
Atlas Project Number: 00094291
Evaluation Plan: 2015-2019, Afghanistan
Evaluation Type: Mid Term Project
Status: Completed
Completion Date: 06/2018
Planned End Date: 06/2018
Management Response: Yes
Corporate Outcome and Output (UNDP Strategic Plan 2018-2021)
  • 1. Output 2.2.2 Constitution-making, electoral and parliamentary processes and institutions strengthened to promote inclusion, transparency and accountability
  • 2. Output 2.2.3 Capacities, functions and financing of rule of law and national human rights institutions and systems strengthened to expand access to justice and combat discrimination, with a focus on women and other marginalised groups
Evaluation Budget(US $): 60,000
Source of Funding: Donor funded + Project Resources
Evaluation Expenditure(US $): 35,000
Joint Programme: No
Mandatory Evaluation: No
Joint Evaluation: No
Evaluation Team members:
Name Title Email Nationality
Olivera Puric Team Leader
GEF Evaluation: No
Key Stakeholders:
Countries: AFGHANISTAN
Comments:

Included in AWP 2018. External review was requested by the donor (Swiss) and will be conducted as a joint exercise to also incorporate some additional elements required for mid-term evaluation as planned in Prodoc.

 

Lessons
Findings
Recommendations
1

Evaluation Recommendation 1. Capacity Development of National Partners

  1. Enhance cooperation and coordination with other human rights units within different line ministries.
  2. Ensure implementation of the LAGF Reform Roadmap
  3. Increase public consultations, in particular at the provincial level while ensuring systematic follow-up on recommendations provided during the consultations.
2

Evaluation Recommendation 2. Developing Demand Side Programming

2.1 The project should support the strengthening and enhancing of links among all actors in the justice chain at the provincial level, initially piloting this approach in a selected number of provinces with a view to rolling out the approach in all the provinces where the law clinics are operating

2.2  Law Clinics should coordinate even more closely with LAGF to ensure a comprehensive approach in referring cases but also in outreach and identifying flaws and gaps in access to justice- both formal and informal.

3

Evaluation Recommendation 3. Strengthening internal and external coordination

3.1 The project should continue to meet on a bi-weekly basis and to proactively share information and search for substantive linkages between the project activities

3.2 In particular with regards to gender justice the project should initiate consultations and dialogue with other UN Agencies and development partners

3.3 The project should actively participate in and contribute to the development of the LOTFA ToR to ensure that access to justice is well positioned in the future rule of law programming framework.

3.4 UNDP should address the issue of coordination of legal aid provision in a more holistic and systematic way, through the consideration and inclusion of all relevant legal aid service providers (institutional indigent defense providers, NGOs, private advocates, etc.).

4

Evaluation Recommendation 4. Transparency and Outreach

4.1 With national partners, conduct outreach and communicate progress towards results systematically with all national partners; provide follow-up on partners concerns and feedback.

4.2 With external partners proactively explore opportunities for joint approaches for access to justice common policy messaging and programming and potential programming.

5

Evaluation Recommendation 5. Geographical Focus

5.1 Identify criteria for the selection of provinces for future programming, keeping in mind where the law and order and anti-corruption projects are already operating in order to identify provinces where an integrated response to justice delivery can be piloted.

5.2 Start developing a pilot rollout plan again in coordination with the law and order and corruption projects.

6

Evaluation Recommendation 6. Sustainability and Developing an Exit Strategy

6.1 Engage in dialogue with the relevant government representatives in order to achieve full government ownership for the project results.

6.2 Open/continue discussions with national partners about absorbing project activities into the national budget, keeping in mind that justice reform is a long-term process.

7

Evaluation Recommendation 7. Staffing Arrangements

7.1 During the preparation of the next phase of the AA2J project in the next 10 months of the existing project implementation, UNDP needs to carefully consider project staff alignment with project needs as a priority.

7.2 All project positions should be reopened for the new phase and ToRs should be developed in advance according to the project focus.

7.3 The project management structure should be revisited. As a matter of priority UNDP should address the isolation of individual staff by project output to enhance the impact of support provided to cross-cutting elements, such as gender mainstreaming and the project management arrangements, where there is currently a mix of project management functions within some project positions, vis-à-vis cross-cutting substantive technical requirements.

7.4 UNDP should deploy AA2J coordinators in each region of intervention and explore the possibility of taking the lead of the UNAMA rule of law regional presence or merging it with AA2J.

8

Evaluation Recommendation 8. Relationship between the Project and the UNDP Programme Office and Senior Management

8.1 Having regular dialogue and information exchange between the project and programme office, seeking solutions in a proactive manner.

8.2 Engage programme office early on in the process of solving issues with national partners.

8.3 Mutual engagement between senior management and the project and programme office to ensure positioning and visibility of the AA2J project, particularly within the LOTFA framework.

9

Evaluation Recommendation 9. Development of the next phase of the AA2J

9.1 A joint visioning retreat for AA2J project staff and staff from the Rule of Law and Human Rights & Security Unit, as well as with SDC and national partners is held at the beginning of this process in order to identify a common platform – mission/vision and strong, realistic, politically informed theory of change

9.2 An institutional and context analysis (PEA) is undertaken of the justice sector in Afghanistan to give a clear understanding of the constraints, power dynamics and incentive structures and to ensure the development of a realistic theory of change.

9.3 The project should engage external support to support these processes as required and to develop the project document framework.

10

Evaluation Recommendation 10. Strengthening linkages with traditional justice mechanisms

10.1 The project should launch a consultative process, in particular at the provincial level, with regards to the development of the procedural regulation accompanying the Conciliation Law.

10.2 The project should engage in dialogue with civil society organisations and justice institutions that are positioning against the adoption of the draft Conciliation law.

10.3 A monitoring framework should be developed by the project in cooperation with the Law Clinics, with a view to enabling law students to monitor practices and proceedings with the traditional dispute resolution mechanisms, providing a full analytical picture of the non-compliance with the national normative framework. A similar monitoring framework should be developed for monitoring of EVAW trials in courts to allow for deeper analysis and understanding of potential human rights violations as well as the verdicts delivered in EVAW cases. In the next project phase, the design of a broader, more effective monitoring system must be considered.

10.4 UNDP should continue to reflect on how to strengthen linkages between formal justice institutions and traditional justice mechanisms and assess how other AA2J implementing partners (besides the legal clinic) could possible play a role in this regard. As an example, lawyers from the LAGF could assume a similar role.

11

Evaluation Recommendation 11. Evidence Based Programming

11.1 Invest in M&E project staff (consider hiring national M&E expert). Increase capacities of national project staff to conduct regular implementation monitoring, allowing for the (if hired) national M&E expert and international M&E expert to focus on results monitoring providing a comprehensive dataset that can feed into future programming.

11.2 Existing M&E mechanisms (LAGF monitoring agents, law clinics) should be fully utilised to have greater impact.

12

Evaluation Recommendation 12. Narrowing the Scope of the Project and Fundraising

12.1 Explore with potential donors the interest in supporting a more realistic access to justice programmatic framework.

12.2 Seek to narrow down the focus of the future project/phase, ensuring that gender and informal justice are crosscutting all/both components and are fully embedded in the project design.

13

Evaluation Recommendation 13. Partnership Strategy

13.1 The project needs to (re)initiate dialogue with the current and potential project partners, in particular, SC, MoWA and AGO.

13.2 The project should conduct a potential partners mapping and analysis focused on the relative power and interest of each partner in contributing to the project. This should include civil society, the private sector and academia and can be conducted as part of the ICA/PEA recommended above.

13.3 The project should discuss with each partner the required partnership format and if MoUs or LoAs or similar are sought by the partners, UNDP should facilitate this.

14

Evaluation Recommendation 14. Theory of Change

14.1 A realistic, politically informed, attainable theory of chain should be developed by the project and programme teams, in line with national priorities at a visioning workshop as detailed above.

14.2 The project should conduct a political economy analysis/institutional and context analysis (ICA) using UNDP’s ICA methodology, engaging external assistance if required.

1. Recommendation:

Evaluation Recommendation 1. Capacity Development of National Partners

  1. Enhance cooperation and coordination with other human rights units within different line ministries.
  2. Ensure implementation of the LAGF Reform Roadmap
  3. Increase public consultations, in particular at the provincial level while ensuring systematic follow-up on recommendations provided during the consultations.
Management Response: [Added: 2018/07/21]

Management Response:

UNDP agreed with this recommendation.

Key Actions:

Key Action Responsible DueDate Status Comments Documents
Reaching out to other Human Rights units with different line ministries
[Added: 2018/07/21]
AA2J Project team 2018/12 Initiated
Produce the monitoring brief of the LAGF Reform roadmap
[Added: 2018/07/21]
AA2J Project team 2019/03 Initiated
Organise at least 1 provincial public consultation
[Added: 2018/07/21]
AA2J Project team 2019/03 Initiated
2. Recommendation:

Evaluation Recommendation 2. Developing Demand Side Programming

2.1 The project should support the strengthening and enhancing of links among all actors in the justice chain at the provincial level, initially piloting this approach in a selected number of provinces with a view to rolling out the approach in all the provinces where the law clinics are operating

2.2  Law Clinics should coordinate even more closely with LAGF to ensure a comprehensive approach in referring cases but also in outreach and identifying flaws and gaps in access to justice- both formal and informal.

Management Response: [Added: 2018/07/21]

UNDP agreed with this recommendation.

Key Actions:

Key Action Responsible DueDate Status Comments Documents
Organise at least 1 provincial public consultation with all stakeholders
[Added: 2018/07/21]
AA2J Project team 2019/03 Initiated
3. Recommendation:

Evaluation Recommendation 3. Strengthening internal and external coordination

3.1 The project should continue to meet on a bi-weekly basis and to proactively share information and search for substantive linkages between the project activities

3.2 In particular with regards to gender justice the project should initiate consultations and dialogue with other UN Agencies and development partners

3.3 The project should actively participate in and contribute to the development of the LOTFA ToR to ensure that access to justice is well positioned in the future rule of law programming framework.

3.4 UNDP should address the issue of coordination of legal aid provision in a more holistic and systematic way, through the consideration and inclusion of all relevant legal aid service providers (institutional indigent defense providers, NGOs, private advocates, etc.).

Management Response: [Added: 2018/07/21]

UNDP agreed with this recommendation and will start to strengthen the coordination in Q3 of 2018.

Key Actions:

Key Action Responsible DueDate Status Comments Documents
Organise frequent meetings between; 1) Project members (bi-weekly) 2) Project and programme teams (monthly)
[Added: 2018/07/21] [Last Updated: 2018/09/11]
AA2J project team 2018/08 Completed The project has already started organizing regular bi-weekly meetings with project members and its team.
Provide inputs to the development of LOTFA’s ToR and its process to develop their next phase.
[Added: 2018/07/31] [Last Updated: 2018/09/11]
AA2J project and programme teams 2018/08 Completed AA2J has provided inputs to the development of LOTFA’s ToR. The draft ToR is developed, negotiations are on-going with donors and the government.
4. Recommendation:

Evaluation Recommendation 4. Transparency and Outreach

4.1 With national partners, conduct outreach and communicate progress towards results systematically with all national partners; provide follow-up on partners concerns and feedback.

4.2 With external partners proactively explore opportunities for joint approaches for access to justice common policy messaging and programming and potential programming.

Management Response: [Added: 2018/07/21]

UNDP agreed with this recommendation

Key Actions:

Key Action Responsible DueDate Status Comments Documents
Include outreach and communication activities to the project’s workplan
[Added: 2018/07/31] [Last Updated: 2018/09/11]
AA2J project and programme teams 2018/12 Overdue-Initiated The project will focus on this action item, starting third quarter onward.
Include partnership building activities to the project’s workplan
[Added: 2018/07/31] [Last Updated: 2018/09/11]
AA2J project and programme teams 2018/07 Completed This key action has been addressed in the evolved project structure in response to the Mid-term review recommendation.
5. Recommendation:

Evaluation Recommendation 5. Geographical Focus

5.1 Identify criteria for the selection of provinces for future programming, keeping in mind where the law and order and anti-corruption projects are already operating in order to identify provinces where an integrated response to justice delivery can be piloted.

5.2 Start developing a pilot rollout plan again in coordination with the law and order and corruption projects.

Management Response: [Added: 2018/07/21]

UNDP agreed with this recommendation and will incorporate them at the design process of the new phase.

Key Actions:

Key Action Responsible DueDate Status Comments Documents
Conduct a survey/ assessment or relevant institutional and context analysis to build strategic foundation of the 2nd phase
[Added: 2018/07/31] [Last Updated: 2018/12/06]
AA2J programme team 2019/03 Initiated Waiting for the consultant to be on board in mid-Dec 2018.
Organize at least 1 provincial public consultation with all stakeholders
[Added: 2018/07/31]
AA2J Project team 2019/03 Initiated
6. Recommendation:

Evaluation Recommendation 6. Sustainability and Developing an Exit Strategy

6.1 Engage in dialogue with the relevant government representatives in order to achieve full government ownership for the project results.

6.2 Open/continue discussions with national partners about absorbing project activities into the national budget, keeping in mind that justice reform is a long-term process.

Management Response: [Added: 2018/07/21]

UNDP agreed with this recommendation and in Q3 of 2018 will start the consultation to develop an Exit Strategy alongside the development of the new phase.

Key Actions:

Key Action Responsible DueDate Status Comments Documents
Conduct a survey/ assessment or relevant institutional and context analysis to build strategic foundation of the 2nd phase
[Added: 2018/07/31] [Last Updated: 2018/12/06]
AA2J programme team 2019/03 Initiated Waiting for the consultation to be on board in mid-Dec 2018
7. Recommendation:

Evaluation Recommendation 7. Staffing Arrangements

7.1 During the preparation of the next phase of the AA2J project in the next 10 months of the existing project implementation, UNDP needs to carefully consider project staff alignment with project needs as a priority.

7.2 All project positions should be reopened for the new phase and ToRs should be developed in advance according to the project focus.

7.3 The project management structure should be revisited. As a matter of priority UNDP should address the isolation of individual staff by project output to enhance the impact of support provided to cross-cutting elements, such as gender mainstreaming and the project management arrangements, where there is currently a mix of project management functions within some project positions, vis-à-vis cross-cutting substantive technical requirements.

7.4 UNDP should deploy AA2J coordinators in each region of intervention and explore the possibility of taking the lead of the UNAMA rule of law regional presence or merging it with AA2J.

Management Response: [Added: 2018/07/21]

UNDP agreed with this recommendation and proposed a re-alignment of project staff to respond to the revised and downsized LogFrame.

Key Actions:

Key Action Responsible DueDate Status Comments Documents
Based on the revised LogFrame and scope of the project, develop a new organogram for the project team
[Added: 2018/07/31] [Last Updated: 2018/09/11]
AA2J Programme team 2018/07 Completed Completed as part of the project adjustments to address the evaluation recommendations.
For the 2nd phase, all project positions will be re-opened with updated ToRs for public competition.
[Added: 2018/07/31]
AA2J Programme team 2019/03 Initiated
8. Recommendation:

Evaluation Recommendation 8. Relationship between the Project and the UNDP Programme Office and Senior Management

8.1 Having regular dialogue and information exchange between the project and programme office, seeking solutions in a proactive manner.

8.2 Engage programme office early on in the process of solving issues with national partners.

8.3 Mutual engagement between senior management and the project and programme office to ensure positioning and visibility of the AA2J project, particularly within the LOTFA framework.

Management Response: [Added: 2018/07/21]

UNDP agreed with this recommendation and will start to strengthen relationships between all internal parties in Q3 of 2018.

Key Actions:

Key Action Responsible DueDate Status Comments Documents
Organize a monthly joint meeting between the project team and the programme office to update progress and concerns.
[Added: 2018/07/31] [Last Updated: 2018/09/11]
Project and programme team 2018/12 Overdue-Initiated The project is conducting ad-hoc meetings with the programme office and going forward, it is planned to make it more systematic and organized.
Frequently engage the senior management on AA2J issues through weekly Heads of unit meeting.
[Added: 2018/07/31] [Last Updated: 2018/09/11]
Head of ROLHS and programme team 2018/08 Completed The project team is engaging the senior management on ad-hoc manner and going forward, it will be more structured and organized.
9. Recommendation:

Evaluation Recommendation 9. Development of the next phase of the AA2J

9.1 A joint visioning retreat for AA2J project staff and staff from the Rule of Law and Human Rights & Security Unit, as well as with SDC and national partners is held at the beginning of this process in order to identify a common platform – mission/vision and strong, realistic, politically informed theory of change

9.2 An institutional and context analysis (PEA) is undertaken of the justice sector in Afghanistan to give a clear understanding of the constraints, power dynamics and incentive structures and to ensure the development of a realistic theory of change.

9.3 The project should engage external support to support these processes as required and to develop the project document framework.

Management Response: [Added: 2018/07/21]

UNDP agreed with this recommendation and will incorporate them at the design process of the new phase.

Key Actions:

Key Action Responsible DueDate Status Comments Documents
Conduct a survey/ assessment or relevant institutional and context analysis to build strategic foundation of the 2nd phase
[Added: 2018/07/31] [Last Updated: 2018/12/06]
AA2J programme team 2019/03 Initiated Waiting for the consultation to be on board in mid-Dec 2018
Organize project design consultations with all stakeholders including but not limited to donors, government partners, CSOs, AA2J project staff and ROLHS staff
[Added: 2018/07/31]
Head of ROLHS and programme team 2018/12 Initiated
10. Recommendation:

Evaluation Recommendation 10. Strengthening linkages with traditional justice mechanisms

10.1 The project should launch a consultative process, in particular at the provincial level, with regards to the development of the procedural regulation accompanying the Conciliation Law.

10.2 The project should engage in dialogue with civil society organisations and justice institutions that are positioning against the adoption of the draft Conciliation law.

10.3 A monitoring framework should be developed by the project in cooperation with the Law Clinics, with a view to enabling law students to monitor practices and proceedings with the traditional dispute resolution mechanisms, providing a full analytical picture of the non-compliance with the national normative framework. A similar monitoring framework should be developed for monitoring of EVAW trials in courts to allow for deeper analysis and understanding of potential human rights violations as well as the verdicts delivered in EVAW cases. In the next project phase, the design of a broader, more effective monitoring system must be considered.

10.4 UNDP should continue to reflect on how to strengthen linkages between formal justice institutions and traditional justice mechanisms and assess how other AA2J implementing partners (besides the legal clinic) could possible play a role in this regard. As an example, lawyers from the LAGF could assume a similar role.

Management Response: [Added: 2018/07/21]

UNDP agreed with this recommendation and will incorporate them at the design process of the new phase.

Key Actions:

Key Action Responsible DueDate Status Comments Documents
Organize a provincial-level consultation as part of the 2nd phase project document development
[Added: 2018/07/31]
AA2J Project team 2018/12 Initiated
Develop M&E frame work as part of Project Document package
[Added: 2018/07/31]
AA2J programme team 2019/01 Initiated
11. Recommendation:

Evaluation Recommendation 11. Evidence Based Programming

11.1 Invest in M&E project staff (consider hiring national M&E expert). Increase capacities of national project staff to conduct regular implementation monitoring, allowing for the (if hired) national M&E expert and international M&E expert to focus on results monitoring providing a comprehensive dataset that can feed into future programming.

11.2 Existing M&E mechanisms (LAGF monitoring agents, law clinics) should be fully utilised to have greater impact.

Management Response: [Added: 2018/07/21]

UNDP agreed with this recommendation and will incorporate them at the design process of the new phase.

Key Actions:

Key Action Responsible DueDate Status Comments Documents
Design a new project’s organogram to effectively deliver the 2nd phase project document.
[Added: 2018/07/31]
AA2J programme team 2019/01 Initiated
12. Recommendation:

Evaluation Recommendation 12. Narrowing the Scope of the Project and Fundraising

12.1 Explore with potential donors the interest in supporting a more realistic access to justice programmatic framework.

12.2 Seek to narrow down the focus of the future project/phase, ensuring that gender and informal justice are crosscutting all/both components and are fully embedded in the project design.

Management Response: [Added: 2018/07/21]

UNDP agreed with this recommendation and will incorporate them at the design process of the new phase.

Key Actions:

Key Action Responsible DueDate Status Comments Documents
Organize project design consultations with all stakeholders including but not limited to donors, government partners, CSOs, AA2J project staff and ROLHS staff
[Added: 2018/07/31]
Head of ROLHS and programme team 2018/12 Not Initiated
13. Recommendation:

Evaluation Recommendation 13. Partnership Strategy

13.1 The project needs to (re)initiate dialogue with the current and potential project partners, in particular, SC, MoWA and AGO.

13.2 The project should conduct a potential partners mapping and analysis focused on the relative power and interest of each partner in contributing to the project. This should include civil society, the private sector and academia and can be conducted as part of the ICA/PEA recommended above.

13.3 The project should discuss with each partner the required partnership format and if MoUs or LoAs or similar are sought by the partners, UNDP should facilitate this.

Management Response: [Added: 2018/07/21]

UNDP agreed with this recommendation and will incorporate them at the design process of the new phase.

Key Actions:

Key Action Responsible DueDate Status Comments Documents
Organize project design consultations with all stakeholders including but not limited to donors, government partners, CSOs, AA2J project staff and ROLHS staff
[Added: 2018/07/31]
Head of ROLHS and programme team 2018/12 Initiated
Conduct a survey/ assessment or relevant institutional and context analysis to build strategic foundation of the 2nd phase
[Added: 2018/07/31] [Last Updated: 2018/12/06]
AA2J programme team 2019/03 Initiated Waiting for the consultation to be on board in mid-Dec 2018
Develop a guideline for project’s partnership agreement with different partners
[Added: 2018/07/31]
AA2J Programme team 2018/12 Initiated
14. Recommendation:

Evaluation Recommendation 14. Theory of Change

14.1 A realistic, politically informed, attainable theory of chain should be developed by the project and programme teams, in line with national priorities at a visioning workshop as detailed above.

14.2 The project should conduct a political economy analysis/institutional and context analysis (ICA) using UNDP’s ICA methodology, engaging external assistance if required.

Management Response: [Added: 2018/07/21]

UNDP agreed with this recommendation and will incorporate them at the design process of the new phase.

Key Actions:

Key Action Responsible DueDate Status Comments Documents
Develop strategic ToC based on the results of the survey/ assessment or relevant strategic institutional and context analysis
[Added: 2018/07/31]
AA2J Programme team 2018/12 Initiated

Latest Evaluations

Contact us

220 East 42nd Street
20th Floor
New York, NY 10017
Tel. +1 646 781 4200
Fax. +1 646 781 4213
erc.support@undp.org