- Evaluation Plan:
- 2018-2022, Pakistan
- Evaluation Type:
- Mid Term Project
- Planned End Date:
- 11/2021
- Completion Date:
- 11/2021
- Status:
- Completed
- Management Response:
- Yes
- Evaluation Budget(US $):
- 40,000
Mid-Term Evaluation of Decentralisation and Local Governance Project
| Document | Type | Language | Size | Status | Downloads |
|---|---|---|---|---|---|
 DHL-Evaluation QA Checklist- Final Draft[1].docx.pdf |
related-document | English | 664.87 KB | Posted | 555 |
| TORs - DHL Mid Term Evaluation 08.02.21.pdf |
tor | English | 262.14 KB | Posted | 628 |
 Final-UNDP DHL Draft Evaluation Draft Report final 26-11-2021 DGU 16.11.21 final.docx |
report | English | 718.82 KB | Posted | 509 |
| DHL-RBAP Evaluation QA Checklist- Final Draft.docx.pdf |
related-document | English | 638.60 KB | Posted | 490 |
| Title | Mid-Term Evaluation of Decentralisation and Local Governance Project | ||||||||
|---|---|---|---|---|---|---|---|---|---|
| Atlas Project Number: | 00070684 | ||||||||
| Evaluation Plan: | 2018-2022, Pakistan | ||||||||
| Evaluation Type: | Mid Term Project | ||||||||
| Status: | Completed | ||||||||
| Completion Date: | 11/2021 | ||||||||
| Planned End Date: | 11/2021 | ||||||||
| Management Response: | Yes | ||||||||
| Focus Area: |
|
||||||||
| Corporate Outcome and Output (UNDP Strategic Plan 2018-2021) |
|
||||||||
| SDG Goal |
|
||||||||
| SDG Target |
|
||||||||
| Evaluation Budget(US $): | 40,000 | ||||||||
| Source of Funding: | Donor | ||||||||
| Evaluation Expenditure(US $): | 46,585 | ||||||||
| Joint Programme: | No | ||||||||
| Joint Evaluation: | No | ||||||||
| Evaluation Team members: |
|
||||||||
| GEF Evaluation: | No | ||||||||
| Key Stakeholders: | UNDP and Provincial Law departments | ||||||||
| Countries: | PAKISTAN |
| Lessons | |
|---|---|
| Findings |
| Recommendations | |
|---|---|
| 1 | Consider financial viability while awarding contracts or providing advance payment to the implementing partners- All responsible parties do not have enough financial resources to implement a project on their own. Since UNDP has a delivery-based payment system on completion of deliverables, some payment issues may arise due to unexpected circumstances, such as COVID 19, where payments may get late. This, in turn, may create problems for implementation partners to retain some of their human resources dedicated to the project, and timely implementation of the project may also get affected. Hence, some advance payments may be transferred for the uninterrupted implementation of the projects. Alternatively, the financial viability of the implementation partners may also be considered at the time of awarding the project
|
| 2 | Collect more quantitative indicators for HRIMS with built-in disaggregation and action-oriented data-driven human rights policy analysis . While the current emphasis is on collecting qualitative data for HRIMS, the database should incorporate more quantitative data, which may also allow for gender (including transgender), age, child labor, religious and ethnic minorities, disability, and even district level disaggregation. Qualitative variables also have statistical problems. Qualitative variables are often presented in numerical form, but they do not possess the essential characteristics of statistical data: reliability (different people will come up with consistent results) and validity (being based on identifiable criteria that measure what they are intended to measure). Indicators, particularly quantitative ones, have long played important roles in the analysis of development policy. Quantitative variables can show the status of a particular human rights situation, reveal whether a situation is getting better or worse owing to a policy change, and guide the formation of better policy. Moreover, quantitative variables can also persuade the unconvinced where opinion alone cannot. In addition, they can also motivate policy change by revealing the ill effects of current practices, hold the state accountable for its policies, help to guide and improve policy, and would help monitor the implementation of various commitments. By including quantitative or statistical data into HRIMS, the database would have powerful statistical information related to various broad aspects of human rights, which in turn may also be analyzed further for evidence-based policymaking related to human rights. The data bank can contain all types of data, including quantitative data on women, health, education (out-of-school children, dropouts). And the data may also be analyzed for all kinds of policy analysis, including impact evaluation and understanding reasons or drivers of a particular human rights-based phenomena. Services of an expert applied statistician may be hired to understand how quantitative data may be incorporated in HRIMS and how HRIMS data may be used to disaggregate and analyze human rights-related data for policymaking, monitoring, evaluation, reporting, and |
| 3 | Facilitate learning trips on HRIMS to KP and Sindh for implementation of HRIMS in Balochistan and regular HRIMS coordination and implementation meetings According to an official from Social Welfare Department in Balochistan, DHL should facilitate visits to KP to learn more about HRIMS, as KP has been a pioneer in the establishment of KPVP, which was the predecessor to HRIMS at the national level and in various provinces. Similarly, learning from Sindh province on implementation of HRIMS would be very useful, too, as Sindh has a dedicated department to HRIMS, unlike Balochistan. Moreover, the consultative process with the provincial and federal government is an ongoing process; however, these coordination and implementation meetings are not held regularly, and it has been suggested by Social Welfare Department that HRIMS coordination and implementation meetings should be held regularly where UNDP should lead. |
| 4 | Conduct regular awareness campaigns and sensitization sessions for government line departments on reporting human rights and training on formats to report on human rights. Sensitize and create awareness on child sexual abuse as part of the Human Rights Component in Balochistan and other provinces It is important for UNDP DHL to conduct awareness campaigns and sensitize government line departments on the importance of reporting on human rights. Moreover, it is also recommended for DHL to train MoHR to translate the reporting requirements on human rights to align with the mandate of other line departments. Line departments mostly do not know how to report in line with the human rights reporting format. Hence, it would be advisable for DHL to train government line departments and MoHR on human rights reporting formats and how to extract data from various studies such as labor surveys to report in line with human rights reporting formats. One important area identified by Social Welfare Department officials in Balochistan for DHL UNDP to work was the issue of child sexual abuse, which has sharply risen in Balochistan in recent years. This issue could be a priority area for Human Rights Component in Balochistan and even other provinces. Under such a project, schools could be used to sensitize and create awareness on child sexual abuse through teachers and awareness as part of the curriculu |
| 5 | Since human rights may involve many diverse line departments, the key issue and challenge sometimes are drawing boundaries into the mandate of each line department. If one department takes action, such as on missing teachers from government schools or missing doctors from government clinics or hospitals, then another line department may feel being transgressed. Enforcement is also a key challenge in such cases. Hence, it is important to enforce and provide clarifications on human rights protection law where responsibilities and job description for each line department is clear. There is a limitation on behalf of MoHR taking action where the action is required to resolve a human rights issue |
| 6 | Engage non-traditional elements such as religious scholars to help with sensitization and advocacy of transgender rights and sensitization of various Rule of Law institutions on transgender rights
|
| 7 | Train targeted officials inline departments with less chance of interdepartmental or inter-ministry transfers
|
| 8 | Continue in-person psychosocial support counseling after completion of the short-term program. Both women beneficiaries who were interviewed believed that psycho-social in-person counseling should be continued and not abandoned after six months when program intervention comes to its end |
| 9 | Study and understand drivers of violent extremism in diverse environments for future Community Stabilization programming
|
| 10 | Extend the duration of Community Stabilization interventions, especially vocational training and in-person psychosocial support |
| 11 | 11-Restrict implementing partners to provide quality toolkits to program beneficiaries after vocational training Problems have been reported with the provision of toolkits to secure a job or gain financial independence for the beneficiaries under the community stabilization component. The female beneficiary from Multan and her other beneficiary colleagues were provided laptops as toolkits to work from home; however, most of these laptops were not in good working conditions. The beneficiaries complained about their toolkits, but the implementing partner had already finished their implementation of the short-term program and had left (beneficiary from Multan). Similarly, the beneficiary from Karachi was provided with a beautician toolkit which she found to be substandard (beneficiary from Karachi). Hence, toolkits need improvement, and DHL should consistently monitor the provision of toolkits so that implementing partners don’t supply substandard toolkits. |
| 12 | 12-Conduct a rigorous quantitative study using variation in the data on why women don’t approach Gender Desks, issues faced by them while approaching Gender Desks, and their satisfaction level with Gender Desk services. Gender Desks are one of the most important projects being implemented by both the Social Inclusion and Protection Component and the Community Stabilization Component of the DHL Project. While the Gender Desk provides free-of-charge services and facilitates access for women in remote areas through a mobile unit, women in remote areas who are the most in need of its services might not report their grievances due to numerous behavioral bottlenecks, including lack of awareness about the Desk and the full range of its services, lack of knowledge on how to access the Desk, lack of awareness on how and/or when to report a grievance, whether it is free of charge, misperceptions about the effectiveness of the Desk, social stigma and sense of shame around reporting of culturally sensitive topics, and forgetting to submit grievances when under high cognitive load that hinders decision-making. A rigorous quantitative study using variation in the data on why women don’t approach Gender Desks, issues faced by them while approaching Gender Desks, and their satisfaction level with Gender Desk services need to be conducted. This information will help the DHL unit to improve its Gender Desk intervention by improving its access and services and also help an information campaign to increase Gender Desk usage by women in remote areas |
| 13 | 13-Study and understand drivers of violent extremism in diverse environments for Conduct rigorous impact and process evaluations on various interventions in order to provide evidence of an impact or not and how an intervention or program impact may be maximized. In 2018, a rigorous impact evaluation was carried out by the DHL unit on Community Stabilization in Swat Valley. The study also elucidated drivers of conflict and violent extremism in Swat. Carrying out a rigorous impact assessment and studying drivers of violent extremism after the first pilot in Swat allowed the Community Stabilization Component to become a full-fledged PVE model. Moreover, it also allowed UNDP Pakistan to prove the impact of the intervention and to collect first-hand data related to PVE Pakistan – in a country where quantitative research on the subject is scarce. However, a rigorous impact evaluation and contextual drivers of violent extremism in various areas under the program are required, which have not been undertaken after the Swat project. The consolidated data collected by UNDP at the completion of all the implementation phases will be an invaluable asset for evidence-based data-driven policymaking and programme design.
|
| 14 | 14- Conduct regular monitoring spot checks for increased transparency and accountability and review meetings with implementing partners to understand implementing issues, gaps, and problems.
|
| 15 | 15-Diversify future funding opportunities for project sustainability A challenge for some DHL sub-components or projects may be funding in the future. Instead of relying on a single donor, the project needs to diversify and also look towards other donors such as the EU and other diverse donor pools. A similar challenge may also be faced with Gender Desks in the merged areas or newly merged districts in KP, where women record their issues, problems, and grievances. There needs to be a mechanism to sustain Gender Desks once funding runs out. In this case, DHL can enhance the capacity of the Social Welfare Department enough to sustain Gender Desks once funding for gender desks runs ou |
| 16 | 16-Make various DHL projects sustainable through a partnership with the government based on a similar model as of HRIMS- ust like HRIMS, where the government (MoHR) has taken over the ownership, and hence it has become sustainable, various other components of DHL should strive to become sustainable as well |
| 17 | 17- Improve methodology for Programme Quality Control Assessments-Very few quality control assessments have been conducted for the DHL Project. These assessments have also used a single rater/ratings to assess the quality of various attributes. Nevertheless, such single-person ratings could be subjective and lower in reliability (different ratings from different people may be inconsistent). Hence, it would be better methodologically to have more than one rater for ratings of an attribute and inter-rater reliability be calculated using such statistical methods as intraclass correlations for continuous variables and Kappa Cohen statistics for categorical variables. And then, the average of the ratings may be calculated for any particular attribute if the intraclass correlation or reliability is found as adequate or high. This strategy would give much more credible and reliable quality control ratings of various attributes of any intervention. |
Management Response Documents
1. Recommendation:
Consider financial viability while awarding contracts or providing advance payment to the implementing partners- All responsible parties do not have enough financial resources to implement a project on their own. Since UNDP has a delivery-based payment system on completion of deliverables, some payment issues may arise due to unexpected circumstances, such as COVID 19, where payments may get late. This, in turn, may create problems for implementation partners to retain some of their human resources dedicated to the project, and timely implementation of the project may also get affected. Hence, some advance payments may be transferred for the uninterrupted implementation of the projects. Alternatively, the financial viability of the implementation partners may also be considered at the time of awarding the project
Management Response: [Added: 2021/12/30]
UNDP is partially agreed with the recommendation as for funds disbursement, we already have policies in place and we need to follow the process which has been decided after the micro capacity assessment fo concerned partner.
Key Actions:
| Key Action | Responsible | DueDate | Status | Comments | Documents |
|---|---|---|---|---|---|
| 1.1:- Tranches released to the partners (be it for a procurement contract or Agreement under HACT, are based on the financial proposal and the ICE submitted by IP and RP. The project can budget as per the nature of activities for HACT and in the procurement plan. The unit will provide inputs as per policy.
[Added: 2021/12/30] |
UNDP-PMU (DGU) | 2022/12 | Initiated |
2. Recommendation:
Collect more quantitative indicators for HRIMS with built-in disaggregation and action-oriented data-driven human rights policy analysis . While the current emphasis is on collecting qualitative data for HRIMS, the database should incorporate more quantitative data, which may also allow for gender (including transgender), age, child labor, religious and ethnic minorities, disability, and even district level disaggregation.
Qualitative variables also have statistical problems. Qualitative variables are often presented in numerical form, but they do not possess the essential characteristics of statistical data: reliability (different people will come up with consistent results) and validity (being based on identifiable criteria that measure what they are intended to measure).
Indicators, particularly quantitative ones, have long played important roles in the analysis of development policy. Quantitative variables can show the status of a particular human rights situation, reveal whether a situation is getting better or worse owing to a policy change, and guide the formation of better policy.
Moreover, quantitative variables can also persuade the unconvinced where opinion alone cannot. In addition, they can also motivate policy change by revealing the ill effects of current practices, hold the state accountable for its policies, help to guide and improve policy, and would help monitor the implementation of various commitments.
By including quantitative or statistical data into HRIMS, the database would have powerful statistical information related to various broad aspects of human rights, which in turn may also be analyzed further for evidence-based policymaking related to human rights. The data bank can contain all types of data, including quantitative data on women, health, education (out-of-school children, dropouts). And the data may also be analyzed for all kinds of policy analysis, including impact evaluation and understanding reasons or drivers of a particular human rights-based phenomena. Services of an expert applied statistician may be hired to understand how quantitative data may be incorporated in HRIMS and how HRIMS data may be used to disaggregate and analyze human rights-related data for policymaking, monitoring, evaluation, reporting, and
Management Response: [Added: 2021/12/30]
agree with the recommendation
Key Actions:
| Key Action | Responsible | DueDate | Status | Comments | Documents |
|---|---|---|---|---|---|
| 2.1:- The project will conduct a review of the HRIMS indicators in 2022 and include (if agreed with relevant government partners) to include more quantitative indicators, in line with OHCHR Guidelines
[Added: 2021/12/30] |
PMU-DGU | 2022/12 | Initiated |
3. Recommendation:
Facilitate learning trips on HRIMS to KP and Sindh for implementation of HRIMS in Balochistan and regular HRIMS coordination and implementation meetings According to an official from Social Welfare Department in Balochistan, DHL should facilitate visits to KP to learn more about HRIMS, as KP has been a pioneer in the establishment of KPVP, which was the predecessor to HRIMS at the national level and in various provinces. Similarly, learning from Sindh province on implementation of HRIMS would be very useful, too, as Sindh has a dedicated department to HRIMS, unlike Balochistan. Moreover, the consultative process with the provincial and federal government is an ongoing process; however, these coordination and implementation meetings are not held regularly, and it has been suggested by Social Welfare Department that HRIMS coordination and implementation meetings should be held regularly where UNDP should lead.
Management Response: [Added: 2021/12/30] [Last Updated: 2021/12/30]
Agreed with the Recomendation
Key Actions:
| Key Action | Responsible | DueDate | Status | Comments | Documents |
|---|---|---|---|---|---|
| 3.1- Since the GMFA funded Human Rights Programme has concluded, the Project will conduct such visits to exchange best practices if new funding for HRIMS and/or Human Rights becomes available. Additionally, UNDP will convey the same to federal MoHR and Provincial Counterparts to engage more with KP to learn from their experiences
[Added: 2021/12/30] |
PMU- DGU | 2022/12 | Not Initiated |
4. Recommendation:
Conduct regular awareness campaigns and sensitization sessions for government line departments on reporting human rights and training on formats to report on human rights. Sensitize and create awareness on child sexual abuse as part of the Human Rights Component in Balochistan and other provinces
It is important for UNDP DHL to conduct awareness campaigns and sensitize government line departments on the importance of reporting on human rights. Moreover, it is also recommended for DHL to train MoHR to translate the reporting requirements on human rights to align with the mandate of other line departments. Line departments mostly do not know how to report in line with the human rights reporting format. Hence, it would be advisable for DHL to train government line departments and MoHR on human rights reporting formats and how to extract data from various studies such as labor surveys to report in line with human rights reporting formats. One important area identified by Social Welfare Department officials in Balochistan for DHL UNDP to work was the issue of child sexual abuse, which has sharply risen in Balochistan in recent years. This issue could be a priority area for Human Rights Component in Balochistan and even other provinces. Under such a project, schools could be used to sensitize and create awareness on child sexual abuse through teachers and awareness as part of the curriculu
Management Response: [Added: 2021/12/30] [Last Updated: 2021/12/30]
Agreed with the recommendation
Key Actions:
| Key Action | Responsible | DueDate | Status | Comments | Documents |
|---|---|---|---|---|---|
| 4.1- The project will include this in its activities planned for 2022 with the Government of Balochistan, under the “Women’s Access to Justice in Balochistan” (WAJIB) Programme
[Added: 2021/12/30] |
PMU- DHL project | 2022/07 | Initiated |
5. Recommendation:
Since human rights may involve many diverse line departments, the key issue and challenge sometimes are drawing boundaries into the mandate of each line department. If one department takes action, such as on missing teachers from government schools or missing doctors from government clinics or hospitals, then another line department may feel being transgressed. Enforcement is also a key challenge in such cases. Hence, it is important to enforce and provide clarifications on human rights protection law where responsibilities and job description for each line department is clear. There is a limitation on behalf of MoHR taking action where the action is required to resolve a human rights issue
Management Response: [Added: 2021/12/30] [Last Updated: 2021/12/30]
Agreed with the Recomendation
Key Actions:
| Key Action | Responsible | DueDate | Status | Comments | Documents |
|---|---|---|---|---|---|
| 5.1-The project will work closely with the National Commission for Human Rights (NCHR) in 2022; their work plan will be updated and relevant training will be organized (dependant on available funds).
[Added: 2021/12/30] |
PMU- Project | 2022/12 | Initiated |
6. Recommendation:
Engage non-traditional elements such as religious scholars to help with sensitization and advocacy of transgender rights and sensitization of various Rule of Law institutions on transgender rights
As remarked by one of the transgender beneficiaries, it is also important to engage various non-traditional elements such as religious scholars to create evidence if they can help with the advocacy of transgender rights. Similarly, it would also be important to sensitize the various other rules of law institutions such as “Judiciary” and “Prosecution” besides Police
Management Response: [Added: 2021/12/30] [Last Updated: 2021/12/30]
Agreed with the recommendation
Key Actions:
| Key Action | Responsible | DueDate | Status | Comments | Documents |
|---|---|---|---|---|---|
| 6.1-The Project will increase engagement with non-traditional actors in both its WAJIB as well as Community Stabilization Project in Balochistan and the NMAs.
[Added: 2021/12/30] |
DGU-PMU | 2022/05 | Not Initiated |
7. Recommendation:
Train targeted officials inline departments with less chance of interdepartmental or inter-ministry transfers
To counter the effects of frequent transfers within government line departments, UNDP conducted repeat training and capacity building to reach a wider range of officials at different levels in each department/ institution, specifically targeting posts that were less likely to be transferred. This helped UNDP to build a base of capacitated individuals, thus accounting for risks of staff transfers (DRKP). Hence, such training targeted at an individual with less likelihood of transfers would help to account for problems related to officials’ frequent transfers and save UNDP time and resources on multiple future training of officials with fresh line department or ministry posting.
Management Response: [Added: 2021/12/30] [Last Updated: 2021/12/30]
Agreed with the recomendation.
Key Actions:
| Key Action | Responsible | DueDate | Status | Comments | Documents |
|---|---|---|---|---|---|
| 7.1-Contingent on donor funding and government counterparts appetite, in 2022, and beyond, UNDP will prepare a targeted capacity development program for relevant departments more permanent members of the federal and provincial government.
[Added: 2021/12/30] |
DGU-PMU | 2022/12 | Initiated |
8. Recommendation:
Continue in-person psychosocial support counseling after completion of the short-term program. Both women beneficiaries who were interviewed believed that psycho-social in-person counseling should be continued and not abandoned after six months when program intervention comes to its end
Management Response: [Added: 2021/12/30] [Last Updated: 2021/12/30]
Agreed with the recommendation.
Key Actions:
| Key Action | Responsible | DueDate | Status | Comments | Documents |
|---|---|---|---|---|---|
| 8.1 These in-person training will be included in the Community Stabilization Project 2022-2023, Balochistan.
[Added: 2021/12/30] |
PMU-DPU | 2023/12 | Initiated |
9. Recommendation:
Study and understand drivers of violent extremism in diverse environments for future Community Stabilization programming
Although considering the diversity in the areas under Community Stabilization programming, the DHL team of UNDP Pakistan adapted the evolving model to each of those environments, it was a challenge to apply the similar definition to various areas with different push and pull factors and identify the right target beneficiaries for the program intervention. A study in drivers of violent extremism with different push and pull factors would have helped identify the right beneficiaries and strategies to lower violent extremism in diverse areas. Nevertheless, to learn about the push and pull factors that drive VE in a particular area, a study is recommended in each new program area to identify target beneficiaries and devise the right interventions based on evidence-based data-driven research.
Moreover, there is a general absence of baseline data to anchor Project rationales and approaches. Nevertheless, UNDP, through its Community Stabilization Project, is working on setting up a consolidated and well-ordered, accessible data set, which is sufficiently anonymous and meets UNDP’s standards of protecting beneficiary identities and privacy.
Management Response: [Added: 2021/12/30] [Last Updated: 2021/12/30]
Agreed with the recomendation
Key Actions:
| Key Action | Responsible | DueDate | Status | Comments | Documents |
|---|---|---|---|---|---|
| 9.1- A detailed study on drivers of violent extremism in the target location (Balochistan) will be included in the implementation strategy for 2022
[Added: 2021/12/30] |
DPU- DGU | 2022/07 | Initiated |
10. Recommendation:
Extend the duration of Community Stabilization interventions, especially vocational training and in-person psychosocial support
An important issue identified by the Social Welfare Department in Swat and also the beneficiaries of the Community Stabilization program was the short-term duration of the projects on average. When rapport building and intervention starts with the beneficiaries, soon it is time to end the project, which lasts for a few months. Almost half the time is over for the project when the project actually starts getting implemented. Furthermore, according to women beneficiaries interviewed from Multan and Karachi, the duration of the vocational skill-building courses was very short. The beneficiary from Multan was trained for three months in the basic IT course, which she believes should have been either of one-year duration or at least of six months duration. She believes that she was not able to learn much in just three months and moreover, it was very difficult to get a job in the field of IT with just a three-month basic course. Moreover, while psycho-social support has been highly appreciated, beneficiaries believe that psycho-social support should be permanent and in person.
Management Response: [Added: 2021/12/30] [Last Updated: 2021/12/30]
Agreed with the recomendation
Key Actions:
| Key Action | Responsible | DueDate | Status | Comments | Documents |
|---|---|---|---|---|---|
| 10.1- This is dependant on donor funding cycles – however, UNDP will submit multi-year proposals to donors for future programmes of a similar nature
[Added: 2021/12/30] |
PMU-DPU | 2022/12 | Initiated |
11. Recommendation:
11-Restrict implementing partners to provide quality toolkits to program beneficiaries after vocational training Problems have been reported with the provision of toolkits to secure a job or gain financial independence for the beneficiaries under the community stabilization component. The female beneficiary from Multan and her other beneficiary colleagues were provided laptops as toolkits to work from home; however, most of these laptops were not in good working conditions. The beneficiaries complained about their toolkits, but the implementing partner had already finished their implementation of the short-term program and had left (beneficiary from Multan). Similarly, the beneficiary from Karachi was provided with a beautician toolkit which she found to be substandard (beneficiary from Karachi). Hence, toolkits need improvement, and DHL should consistently monitor the provision of toolkits so that implementing partners don’t supply substandard toolkits.
Management Response: [Added: 2021/12/30]
Agreed with the recomendation
Key Actions:
| Key Action | Responsible | DueDate | Status | Comments | Documents |
|---|---|---|---|---|---|
| 11.1-The project will conduct a quality check of all toolkits being provided and a mechanism will be developed to seek beneficiaries feedback on a sample basis to beneficiaries in 2022 before they are handed over.
[Added: 2021/12/30] |
DGU- DHL project | 2022/12 | Not Initiated |
12. Recommendation:
12-Conduct a rigorous quantitative study using variation in the data on why women don’t approach Gender Desks, issues faced by them while approaching Gender Desks, and their satisfaction level with Gender Desk services. Gender Desks are one of the most important projects being implemented by both the Social Inclusion and Protection Component and the Community Stabilization Component of the DHL Project. While the Gender Desk provides free-of-charge services and facilitates access for women in remote areas through a mobile unit, women in remote areas who are the most in need of its services might not report their grievances due to numerous behavioral bottlenecks, including lack of awareness about the Desk and the full range of its services, lack of knowledge on how to access the Desk, lack of awareness on how and/or when to report a grievance, whether it is free of charge, misperceptions about the effectiveness of the Desk, social stigma and sense of shame around reporting of culturally sensitive topics, and forgetting to submit grievances when under high cognitive load that hinders decision-making. A rigorous quantitative study using variation in the data on why women don’t approach Gender Desks, issues faced by them while approaching Gender Desks, and their satisfaction level with Gender Desk services need to be conducted. This information will help the DHL unit to improve its Gender Desk intervention by improving its access and services and also help an information campaign to increase Gender Desk usage by women in remote areas
Management Response: [Added: 2021/12/30]
Agreed with the recomendation
Key Actions:
| Key Action | Responsible | DueDate | Status | Comments | Documents |
|---|---|---|---|---|---|
| 12.1- This will be included in the new project document and activities will be included as part of Phase 2 of the WAJIB Project.
[Added: 2021/12/30] |
PMU-DGU | 2022/12 | Not Initiated |
13. Recommendation:
13-Study and understand drivers of violent extremism in diverse environments for Conduct rigorous impact and process evaluations on various interventions in order to provide evidence of an impact or not and how an intervention or program impact may be maximized. In 2018, a rigorous impact evaluation was carried out by the DHL unit on Community Stabilization in Swat Valley. The study also elucidated drivers of conflict and violent extremism in Swat. Carrying out a rigorous impact assessment and studying drivers of violent extremism after the first pilot in Swat allowed the Community Stabilization Component to become a full-fledged PVE model. Moreover, it also allowed UNDP Pakistan to prove the impact of the intervention and to collect first-hand data related to PVE Pakistan – in a country where quantitative research on the subject is scarce. However, a rigorous impact evaluation and contextual drivers of violent extremism in various areas under the program are required, which have not been undertaken after the Swat project. The consolidated data collected by UNDP at the completion of all the implementation phases will be an invaluable asset for evidence-based data-driven policymaking and programme design.
Since 2019, such evaluation and studies should have also been conducted in Karachi and Multan with proper baselines and end lines, providing evidence to what worked and what did not and how such interventions may be made even more productive through process evaluation in addition to impact evaluation. Still, various end-line evaluations may be conducted for recently concluded interventions or programs or the programs starting in the future. Such evaluations may also be extended to interventions under Human Rights and Social Inclusion and Protection components. Gender Desks is an important intervention that may be evaluated in the future.
Management Response: [Added: 2021/12/30]
Agreed with the recommendation.
Key Actions:
| Key Action | Responsible | DueDate | Status | Comments | Documents |
|---|---|---|---|---|---|
| 13.1- These components, and all other project components, will be evaluated in the UNDP internal end-term evaluation. If donor funding is available, this activity will also be included in next year’s AWP.
[Added: 2021/12/30] |
DGU- DHL project | 2022/12 | Not Initiated |
14. Recommendation:
14- Conduct regular monitoring spot checks for increased transparency and accountability and review meetings with implementing partners to understand implementing issues, gaps, and problems.
For Community Stabilization and other programs, regular spot checks at the field level are recommended periodically. This could also include observing and monitoring training programs, meetings with community organizations such as ISGs, etc., and also conducting regular review meetings with implementing partners for financial spot checks. Regular feedback from implementing partners would also be of great help in order to understand implementing issues, gaps, and problems.
Management Response: [Added: 2021/12/30]
Agreed with the recommendation
Key Actions:
| Key Action | Responsible | DueDate | Status | Comments | Documents |
|---|---|---|---|---|---|
| 14.1-The project evaluation covered a large period where activities were impacted by COVID-19. As a result field visits for monitoring and spot checks were not possible. These will be continued, in line with WHO and the Government of Pakistan Covid-19 SOPs.
[Added: 2021/12/30] [Last Updated: 2022/04/03] |
DGU | 2022/03 | Completed | The project and program team are conducted field visits while following the SOPs so the key action has been met. History |
15. Recommendation:
15-Diversify future funding opportunities for project sustainability A challenge for some DHL sub-components or projects may be funding in the future. Instead of relying on a single donor, the project needs to diversify and also look towards other donors such as the EU and other diverse donor pools. A similar challenge may also be faced with Gender Desks in the merged areas or newly merged districts in KP, where women record their issues, problems, and grievances. There needs to be a mechanism to sustain Gender Desks once funding runs out. In this case, DHL can enhance the capacity of the Social Welfare Department enough to sustain Gender Desks once funding for gender desks runs ou
Management Response: [Added: 2021/12/30]
Agreed with the recomendation
Key Actions:
| Key Action | Responsible | DueDate | Status | Comments | Documents |
|---|---|---|---|---|---|
| 15.1-The Unit will support the DHL Project in drafting and submitting a wide-range of concept notes and project proposals to a more diverse donor pool
[Added: 2021/12/30] |
DGU-PMU | 2022/06 | Initiated | History |
16. Recommendation:
16-Make various DHL projects sustainable through a partnership with the government based on a similar model as of HRIMS- ust like HRIMS, where the government (MoHR) has taken over the ownership, and hence it has become sustainable, various other components of DHL should strive to become sustainable as well
Management Response: [Added: 2021/12/30]
Agreed
Key Actions:
| Key Action | Responsible | DueDate | Status | Comments | Documents |
|---|---|---|---|---|---|
| The project will include sustainability plans in all its components to ensure increased government ownership of its interventions
[Added: 2021/12/30] |
DGU | 2022/06 | Initiated | History |
17. Recommendation:
17- Improve methodology for Programme Quality Control Assessments-Very few quality control assessments have been conducted for the DHL Project. These assessments have also used a single rater/ratings to assess the quality of various attributes. Nevertheless, such single-person ratings could be subjective and lower in reliability (different ratings from different people may be inconsistent). Hence, it would be better methodologically to have more than one rater for ratings of an attribute and inter-rater reliability be calculated using such statistical methods as intraclass correlations for continuous variables and Kappa Cohen statistics for categorical variables. And then, the average of the ratings may be calculated for any particular attribute if the intraclass correlation or reliability is found as adequate or high. This strategy would give much more credible and reliable quality control ratings of various attributes of any intervention.
Management Response: [Added: 2021/12/30]
Agreed with the recommendation.
Key Actions:
| Key Action | Responsible | DueDate | Status | Comments | Documents |
|---|---|---|---|---|---|
| 17.1- The Unit will conduct quality assurance exercises in line with its regular policies; some were halted due to COVID-19
[Added: 2021/12/30] [Last Updated: 2022/04/03] |
DGU | 2022/05 | Initiated | History |