Evaluation Detail

Final Evaluation of the project "LOTFA Support to Payroll Management (SPM) and MoIA Police Development (MPD)

Evaluation Plan:: 2015-2020, Afghanistan
Evaluation Type:: Final Project
Planned End Date:: 03/2018
Completion Date:: 03/2018
Status:: Completed
Management Response:: Yes
Evaluation Budget(US $):: 70,000

Document	Type	Language	Size	Status	Downloads
ToR International Consultant.pdf	tor	English	486.28 KB	Posted	372
LOTFA SPM Project Final Evaluation Report - March 2018.pdf	report	English	2613.55 KB	Posted	1245

Title

Final Evaluation of the project "LOTFA Support to Payroll Management (SPM) and MoIA Police Development (MPD)

Atlas Project Number:

89137

Evaluation Plan:

2015-2020, Afghanistan

Evaluation Type:

Final Project

Status:

Completed

Completion Date:

03/2018

Planned End Date:

03/2018

Management Response:

Yes

UNDP Signature Solution:

1. Governance

Corporate Outcome and Output (UNDP Strategic Plan 2018-2021)

1. Output 2.2.3 Capacities, functions and financing of rule of law and national human rights institutions and systems strengthened to expand access to justice and combat discrimination, with a focus on women and other marginalised groups

SDG Goal

Goal 16. Promote peaceful and inclusive societies for sustainable development, provide access to justice for all and build effective, accountable and inclusive institutions at all levels

SDG Target

16.3 Promote the rule of law at the national and international levels and ensure equal access to justice for all

Evaluation Budget(US $):

70,000

Source of Funding:

Project resources

Evaluation Expenditure(US $):

70,000

Joint Programme:

Joint Evaluation:

Evaluation Team members:

Name	Title	Email	Nationality
Richard H. Langan	International Consultant
Tooryalay Nasary	Nationall Consultant		AFGHANISTAN

GEF Evaluation:

Key Stakeholders:

Ministry of Interior Affairs (MoIA)

Countries:

AFGHANISTAN

Lessons
Findings
1.	Assessment of Impact, Effectiveness and Efficiencies across the SPM Log-Frames and Outputs during mid-2015 to end-2017 The Evaluation’s review of documentation and interviews with stakeholders reveals that LOTFA-SPM has largely met its in-house technical objectives (i.e. WEPS development; systems management; oversight and data integrity). This can be said notwithstanding on-going issues that are largely outside the control of the SPM Project (i.e. power grid outages at the provincial level; stand-alone data bases outside of the WEPS eco-system that remain non-integrated and other technical difficulties in the payroll process). Tag: Relevance e-Governance Rule of law Technology Capacity Building Institutional Strengthening National Institutions
2.	MOIA Policy development and implementation Supporting legislative and regulatory amendment and alignment of EPS with national laws The Evaluation was able to document much work on the part of the LOTFA-SPM (and MPD) projects regarding support to the MOIA Legal Officer and legislative reform. UNDP facilitated several policy studies of ANP entitlements and pay scales that included legal analysis. In addition, the approved Payroll Management Procedures Manual contains a detailed overview of the laws and regulations pertaining to each step of the payroll process for ANP. Tag: Relevance e-Governance Rule of law Technology Capacity Building Institutional Strengthening National Institutions
3.	On this basis, upon completion of the legal review in November 2015, the LOTFA SPM Project lead a revision of pay entitlements in EPS and participate in a working group formed by the MOIA under the leadership of the DG Policy Research for this purpose. This directly led to the MOIA’s updating of its legislative policies and regulatory frameworks surrounding MOIA payroll management in 2016. Tag: Relevance Human rights Rule of law Project and Programme management
4.	Review of finance and human resources related to payroll and the “Manual on Human Resources Policies and Procedures” Reform and development of policies for finance and human resources are stipulated by the “Donor Conditions” for MOIA. In mid-2015, LOTFA-SPM undertook a review of GIROA/MOIA human resources and finance regulations that resulted in a set of findings and recommendations to the LOTFA-SPM Pay and Budget sub-TWG. The SPM Project also coordinated with the MPD Project to recommend reforms to MOIA of the budget, payroll and personal management regulations and processes. Tag: Relevance Human and Financial resources Policies & Procedures
5.	Strengthening Oversight and accountability Responding to requests from LOTFA’s Donors and in furtherance of the Donor Conditions, LOTFA-SPM put in place significant measures during 2015-2017 to strengthen oversight and accountability.48 These included establishment of the LOTFA Oversight Committee and commissioning an independent assessment of the LOTFA Risk Framework; the implementation of Thematic Working Groups (TWG); the establishment of the LOTFA Fund Fiduciary and Financial Management Unit (FFFMU)(housed at the UNDP CO); establishment of the LOTFA-SPM Project Quality and Assurance Team; establishing the LOTFA “Pay and Compensation Board”; and engaging the “Monitoring Agent” (MA). Tag: Effectiveness Relevance Oversight Donor
6.	The Evaluation reviewed key documentation available to it pertaining to the LOTFA Pay and Compensation Board. It is clear that overall the Pay and Compensation Board has lent due process in considering petitions from the ANP for pay and incentive increase. It has established a well-defined process for ANP to submit petitions to the MOIA Department of Finance, which are in turn taken to the Pay and Compensation Board and also presented in SPM TWG and ultimately the LOTFA Board.49 ANP commanders and MOIA are now aware that they must respect the process of the Pay Board when making requests for raises and incentives. An important LOTFA-SPM output related to the “Pay Board” was the “Pay Structure and Sustainability Strategy” carried out between October and December 2016. The Strategy looked at the MOIA pay structure and sustainability, while providing key findings and recommendations.50 The Strategy’s findings were presented to the LOTFA Board on 18 December 2016. The Pay Board also agreed with the findings and proposed conditions in their early 2017 meetings. Tag: Effectiveness Relevance Human and Financial resources Country Government National Institutions
7.	It appears to this Evaluation that the above described pay raise and incentives approved by the LOTFA Board in December 2017 may not have been entirely consistent with the recommendations of the 2016 “Pay Structure and Sustainability Strategy”. The Strategy recommends that, “the alignment of ANA and ANP should continue to match for baseline pay, hazard pay and cadre based allowances in the short and medium term”; and discusses the issue of ANP longevity and temporary pay.52 The Pay Board appears, however, to have used ANA as the sole guide to increasing ANP incentives, whereas the Strategy clearly states that the bonuses and incentives of the ANA are to be but one of many criteria considered.53 In this respect, the Pay Board’s decisional process concerning ANP incentives may not have been as robust or multi-factored as it should have been. If HDIP is to be considered among the ANP “incentives and bonuses”—and not a special category of its own—then the Pay Board should have taken more criteria into account other than ANA bonuses and incentives. The Strategy does allow for the Pay Board to modify the options for reform set forth in the Strategy, based upon further internal discussions.54 The Pay Board’s decision to combine temporary pay with base pay is, in the opinion of the Evaluation, likely within the liberal interpretation of Article 14 and 16 of the Police Inherent Law afforded by the Strategy and consistent with it. Tag: Effectiveness Relevance Human and Financial resources Project and Programme management Donor
8.	The above scenario underscores the importance of UNDP’s continued work to reform the Police Inherent Law and MOIA’s related policies for the longer-term. Further legal reform and integration of incentives and bonuses with the MOIA’s budget and planning process will ensure that in the future the reward process is formalized and predictable as GIROA is able to assume a higher percentage of budget resources for ANP and GDCPD salaries.58 Furthermore, as noted in the “Pay Structure and Sustainability Strategy”,” ANP and ANA are two separate and distinct organizations. Aligning ANP to ANA—and especially the HDIP increases—while immediately necessary may carry a risk that ANP continues to remain more of a “paramilitary” force, than a “civilian” police force. This Evaluation has highlighted the continuing need to build ANP civilian policing capacity. Operationally, the de facto security situation in Afghanistan continues to demand that ANP and its special operations units participate in and support joint operations with ANA and NATO forces. The extent to which ANA incentives should “drive” decisions on ANP incentives in the future is, however, an open question. The Evaluation met with several LOTFA donors while in Kabul who offered the opinion that the Pay and Compensation Board could be additionally strengthened to include more technical expertise. Some donors opined that donor membership in the Pay and Compensation Board should be increased. Tag: Sustainability Rule of law Strategic Positioning Donor
9.	The Monitoring Agent and SPM Quality and Assurance Team The Monitoring Agent (MA) (Moore Stephens) has been a dynamic addition to the LOTFA-SPM Project. Since 2015, LOTFA-SPM has supported the MA to be active in all 34 processes. This was coupled with the creation of a LOTFA-SPM Quality and Assurance Team in 2015 to oversee that MA’s work. The Evaluation reviewed all MA reports for 2016; as well as the SPM-TWG meetings and SPM quarterly and annual reports. The SPM Project leadership noted to the Evaluation that the MA experienced delay initially in 2015 with regard to reporting, but caught up with most of the late-2015 reports by end-2016. The SPM project has moved to improve the MA reporting and the MA was issuing its reports on time, but only Q1 and Q2 2017 MA reports were available to the Evaluation Team. Tag: Effectiveness Relevance Monitoring and Evaluation Project and Programme management Quality Assurance
10.	The verification tasks to be performed by the MA fall in two categories namely (i) First round: Expenditure Verification (EV) and (i) Second round: Headcount Verification (or “Physical Verification”- PV). Additionally, the Pandora Data Software procured by the SPM Project assisted the MA to identify overpayment of reservists and identification of other issues. The MA utilized EPS/WEPS and payroll data interrogation to generate findings. The Evaluation was able to verify that the MA was fulfilling its ToR via a review of MA reports and interviews with stakeholders. Types of material issues identified by the MA during 2015-2016 included, but were not limited to, the following: Tag: Effectiveness Relevance Monitoring and Evaluation Quality Assurance
11.	For the time period, July to December 2015, the MA visited 33 provinces every month (excepting Nuristan, which could not be visited due to security reasons). On the basis of the MA’s findings during this time period, the deductions of ineligible expenses made by UNDP from LOTFA advances for the time period Jan 2016-March 2017 was USD 2.2m As of Q1 and Q2 2017 the MA was testing an average of 76% of the Expenditures recorded in AFMIS in 30 provinces for ANP and GDPDC in compliance with the MA TOR under its new contract issued in 2017. According to SPM Project reports, the donors acknowledged that by 2016 there had been a visible improvement in addressing MA findings by MOIA over prior years. Tag: Effectiveness Relevance Monitoring and Evaluation Project and Programme management Quality Assurance
12.	It is to be noted that LOTFA’s policy is to pay ANP according to their “current position” (i.e. actual job function), rather than their “rank”, whereas MOIA stipulates that the officers should be paid by rank, not their current position. In most instances, ANP officers are carrying out functions and occupy current positions below their official rank. This is reported to have created tensions between MOIA and LOTFA-SPM. Salary arrears are another area that SPM has encountered (i.e. an ANP officer was not paid, because they were waiting for an ID to be issued). In these instances, SPM has occasionally overridden the system and approved such payments on grounds of fairness to the ANP officer concerned. Tag: Effectiveness e-Governance Rule of law Country Government Technology National Institutions
13.	SPM significantly improved the accuracy of the payroll process and closed the gap on opportunities to commit fraud, via the development and launch of the Digital M16 form. It is the view of the SPM project that the digital M16 has greatly improved the payroll process and closed the gap for corruption. The digital M16 eliminates the need for manual entry of data into the M16 form from the M41 form. This has had the effect of reducing discrepancies between WEPS and AFMIS payments and improving transparency and accuracy. The “leakage” in the system is improving as a result. Tag: Effectiveness Efficiency e-Governance Operational Efficiency Project and Programme management Technology National Institutions
14.	SPM support to strengthening the Payroll Unit of MOIA It is important not to lose sight of the fact that it is the MOIA that “creates” the payroll utilizing the WEPS platform at both the central and provincial levels. LOTFA closes each pay cycle, keeps the back-end of the WEPS system up and running, reconciles discrepancies based upon the MA reports and generates WEPS reports. Yet, it is the MOIA staff who enter data on the front line based upon time and attendance sheets and the Tashkill. Thus, SPM Project support to the Payroll Unit of the MOIA is at the heart of SPM’s capacity building. This has taken the form of providing MOIA staff with WEPS trainings and conducting a Micro-Capacity Assessment (MCA), development of the Payroll Management Procedures Manual and Payroll Capacity Building Plan and “twinning” of MOIA staff. MA reports also made concrete recommendations to MOIA for improving systems and procedures. Tag: Effectiveness Technology Capacity Building Institutional Strengthening National Institutions
15.	SPM Micro Capacity Assessment (MCA) and capacity assessment tools The LOTFA donors selected the management-consulting firm Grant Thornton to conduct a MCA in 2015 that utilized a questionnaire and monitoring framework that revealed a number of issues at MOIA. In addition, SPM staff worked with Grant Thornton beginning in Q4 2015 to develop capacity assessment tools for the MOIA payroll unit. An additional MCA was undertaken in 2016/17 that more closely evaluated MOIA, MOF, UNDP and CSTC-A progress towards meeting the Donor Conditions. The Evaluation reviewed the 2015 and 2016/17 MCAs. Tag: Effectiveness Monitoring and Evaluation Project and Programme management Donor
16.	As noted elsewhere in this Evaluation report, the Evaluation is also particularly concerned with MOIA Donor Condition #4, given the continuing problems surrounding ANP IDs in the field.A full list of the Donor Conditions appears in the Annex of this Evaluation report. The MCA reports 2016-2017 assess MOIA’s progress towards meeting MOIA condition #10 as being met; and conditions #11 and #12 being “on track”, but, again, significant Donors continue to question whether MCA’s characterization is overly generous. According to the SPM Project, as of end-2017, the MOIA Finance Unit still produces inaccurate budget figures and an inaccurate Tashkill. SPM does not deem the capacity of MOIA Finance officers sufficient to enable them to accept advanced training on finance and payroll. The officers are under MOIA and not SPM control. Furthermore, the MOIA lacked a Director of Finance for more than a year. Other leadership posts within MOIA change frequently and titles of staff often do not match their backgrounds.60 CSTC-A also expressed to the Evaluation that it has continuing strong concerns as of mid-2017 regarding MOIA progress towards MOIA Donor Conditions #10, #11 and #12. As noted in the June 2017 MCA report, CSTC-A itself still has two significant Donor conditions that are not yet fulfilled (i.e. both relating to the development of the APPS database). Tag: Relevance Sustainability Country Government Donor Institutional Strengthening National Institutions
17.	Yet, notwithstanding the June 2017 report of MCA, this Evaluation remains concerned about UNDP Donor Condition C-3 (i.e. capacity building of MOIA payroll management). As noted elsewhere in this report, this Evaluation cannot certify that UNDP has as yet built sufficient capacity for MOIA to fully operate its own Payroll Unit. As of end-2017 MOIA still lacks sufficient capacity for payroll to permit a “transfer” of the SPM Project’s payroll functions to MOIA. This appears to have been the determination of the December 2017 LOTFA Board that voted for a 2018 AWP based upon “no transfer”. Thus, it is the finding of this Evaluation that Donor Condition #3 for UNDP remains on-track, but as yet unmet if “transfer” is the operative criteria of the Donor Conditions. Only by taking a more narrow interpretation, focused upon only the goals of the AWP would it be possible to state that UNDP Donor Condition C-3 has been met. Yet, this is also subject to interpretation given the fact that the SPM supported “twinnings” that have occurred to date have been rather limited in scope. Tag: Donor Capacity Building
18.	Development of the Payroll Management Procedures Manual and Trainings on its Implementation SPM developed the Payroll Management Procedures Manual (PMPM) in part as an attempt to lend structure to the MOIA Finance Department and encourage it take the lead on training. The PMPM addresses the capacity gaps identified in the MCA and documents payroll procedures and operations for MOIA in a single comprehensive document. The PMPM was endorsed by the MOIA in early-2017 and approved by the MOIA and MOF in August 2017. Tag: Effectiveness Relevance Policies & Procedures Country Government Capacity Building
19.	Effectiveness of project communication The Evaluation received divergent opinion from various stakeholders interviewed as the overall level of communication between the SPM Project and their MOIA counterparts regarding payroll issues, policy, capacity building and systems development. SPM Project staff characterizes the overall level of communication as being frequent and satisfactory, citing both daily communication via email and regularized meetings (i.e. TWG meetings; Pay and Compensation Board, etc.) and reporting to MOIA (i.e. WEPS reports and other statistics) and other reports such as the MA reports that are circulated to MOIA officers and leadership. MOIA reported that staff works closely with SPM-WEPS on a day-to-day basis both at the central level and in provinces. Tag: Effectiveness Communication Project and Programme management
20.	Capacity building for payroll management WEPS is utilized countrywide by the MOIA at both central and provincial levels. Currently, the SPM Project-WEPS Team maintains the “back-end” of the system with a staff of 12 that includes IT, programming, database management and encryption; while MOIA officers at the central and provincial levels enter WEPS data on the “front end” of the system. Meanwhile, the SPM-Project Capacity Building Team has the function of training MOIA staff on how to use the system. [See the LOTFA Project Document (2016) and SPM-Project staff organogram for a complete overview of staffing in LOTFA-SPM]. The SPM Project had numerous outputs during July-2015 to end-2017 that were designed to build the capacity of MOIA personnel (primarily in MOIA’s Payroll, Human Resources, Finance and Budget directorates). These outputs included developing SOPs and strategic planning documents (i.e. the Payroll Unit Plan); monitoring and assessing MOIA’s capacities (i.e. Micro-Capacity Assessments - also discussed above); and trainings on WEPS and payroll management (i.e. of either a short-term duration or longer-term practical trainings designed as “twinning” of MOIA employees by SPM Project staff at MOIA Kabul). Tag: Effectiveness Regional e-Governance Technology Capacity Building Institutional Strengthening National Institutions
21.	SPM supported modification of ANP attendance book with gender-disaggregated data. The SPM project states that the time and attendance sheets developed for the ANP and disaggregated for gender have improved statistics on the ANP and female police officers. SPM providing capacity building trainings on the new attendance forms printed with SPM support. This was deemed to make an impact upon the accuracy of ANP reporting. If attendance is not accurately reported then WEPS will be calculating payments against “junk”. Quality assurance continues to be a major issue for the SPM Project and should continue to be strengthened. Tag: Effectiveness Relevance Gender Equality e-Governance Technology Capacity Building Institutional Strengthening National Institutions
22.	An SOP for AHRIMS was developed by the SPM Project in 2016. According to the report of the MCA, as of 30 June 2017, 89% (approximately 131,000) of the total ANP Tashkill of 148,000 ANP were registered in AHRIMS. CSTC-A was in the process at end-2017 of finalizing its Personal Asset Inventory (PAI). The SPM Project also supported implementation of the new PERSTAT template (2017) and WEPS-PERSTAT reconciliation. The SPM Project developed a new “PERSTAT template” for purposes of assisting in the data reconciliation effort. The PERSTAT template was only being used by 15 PPHQs as of end-June 2017. Usage had expanded to and estimated 50% of PPHQs by end-September 2017. Going forward, it will be required for SPM to continue working with MOIA to ensure that all PPHQs are utilizing the new PERSTAT template. As of end-2017, the MOIA still lacks capacity to fully handle its human resources databases. MOIA informed the Evaluation, however, that LOTFA-SPM had provided significant support for the development of SOPs, based upon extensive consultations with MOIA and field missions. The MOIA also accorded a high level of satisfaction with SPM workshops and trainings surrounding the development of the SOPs. Forthcoming workforce analysis being conducted by MOIA in conjunction with CSTC-A should lend more insight in 2018. Tag: Effectiveness Policies & Procedures Technology Capacity Building Institutional Strengthening National Institutions
23.	Drafting of the Payroll Unit Plan (PUP and Payroll Capacity Building Plan (PCBP) The SPM Project developed a draft Payroll Unit Plan (PUP) containing a Payroll Capacity Building Plan (PCBP) in late-2015 to mid-2016. The draft was presented to donors in the SPM-TWG and, thereafter, approved by MOIA and the LOTFA Board in May 2016. The PUP articulates the key payroll functions, legal requirements, processes and staffing of the MOIA payroll process. The PCBP sets forth the steps required to build the MOIA Payroll Unit capacity to be able to accept a transfer of the SPM Project’s payroll functions pursuant to the letters of H.E. President Ghani and in compliance with the Donor Conditions. It reflects the “gap analysis” and recommendations of the MCA of December 2015. Tag: Effectiveness Human and Financial resources Monitoring and Evaluation Technology Capacity Building National Institutions
24.	II. Trainings Capacity Development Support and Trainings for MOIA WEPS teams (ANP/GDPCD); PHQ and PFOs The LOTFA SPM Project has continued to experience challenges with training for the MOIA teams. There have been frequent turnovers of staff within MOIA—especially DG Finance. The high rates of turn-over reduce the impact of SPM trainings. Yet, the overall impression of the Donors and MOIA staff is that the trainings provided by SPM have increased the capacity of MOIA staff for payroll. Tag: Effectiveness e-Governance Technology Capacity Building Institutional Strengthening National Institutions
25.	“Twinned” MOIA staff at LOTFA-SPM The LOTFA-SPM Project has trained on a “twinning” basis, 4 MOIA staff in 2015 and a total of 14 MOIA staff in 2016-2017. At end-2017, six of the “twinned” staff had desks inside the SPM capacity building office and worked there on a day-to-day basis. The remainder of the “twinned” staff still occupied their own desks within their respective MOIA departments. Tag: Effectiveness e-Governance Technology Capacity Building Institutional Strengthening National Institutions
26.	The Evaluation questioned the SPM Project as to why SPM “twinning” has to date only occurred on the SPM capacity building/finance and human resources side of the SPM Project. The SPM Project confirmed that many of the “twinned” MOIA staff possess academic credentials similar to the SPM national project staff, but that none of the “twinned” MOIA or MOIA departmental staff possess adequate computer IT qualifications to enable them to absorb IT training. Thus, the SPM WEPS team has not yet trained any MOIA “twinned” or MOIA departmental staff on the “back end” of the WEPS system. Tag: Effectiveness e-Governance Human and Financial resources Technology Capacity Building Institutional Strengthening National Institutions
27.	Overall, it appears that monitoring of LOTFA-SPM’s “twinning’s” and trainings could be strengthened and made more detailed. SPM has made presentations to the SPM-TWG periodically and stated figures for trainings, but it is not immediately evident to this Evaluation that trainings have taken place with either the intensity or specificity required to enable MOIA staff to accept a transition of all payroll functions according to the Payroll Unit Plan approved in mid-2016. While it would be optimal if all existing SPM national project staff were to agree to convert to MOIA contracts at the time of transition, this cannot be taken for granted. It is also in the interest of the MOIA and in-line with the NIM that MOIA take ownership over this issue and that some amount of HR redundancy be built-in to a future MOIA Payroll Unit. Professional Certification Programmes for MOIA staff (American University of Afghanistan) As of end-2017 the Professional Certification Programmes for MOIA staff at the American University of Afghanistan continued to be delayed. SPM WEPS team noted to the Evaluation that the planned two-year master courses for selected MOIA staff to take place at the American University of Afghanistan could be configured to offer an avenue for MOIA staff to receive advanced IT qualifications. No such IT training, has as yet been included in the curricula. Tag: Effectiveness Human and Financial resources Monitoring and Evaluation Technology Capacity Building Institutional Strengthening
28.	Systems integration and interface SPM Project WEPS Team Support to the MOIA As of 2016, with SPM Project support the MOIA’s stand-alone EPS had been fully migrated into WEPS. By March 2016 the SPM Project had achieved 100% WEPS coverage in all 34 provinces for both ANP and GDPDC payroll stations. SPM efforts to improve the overall functionality of WEPS have included developing an online ticketing system; implementation of the digital M16, WEPS data storage and management and the use of Geographical Information Systems (GIS). The MA and the SPM Quality Assurance Team have visited the provincial level ANP, PHQ and PFOs regularly Tag: Effectiveness e-Governance Technology Capacity Building Institutional Strengthening National Institutions
29.	Combined Percentages of ANP and GDPDC Salaries Paid Through Electronic Funds Transfer (EFT) and Mobile Money (MM) as opposed to the Trusted Agent (TA) modality It appears that the SPM Project successfully supported the MOIA to reach its targets for Electronic Funds Transfer (EFT) and satisfy the Donor Condition. The percentage of ANP paid by EFT as of end-Q3 2017 was approximately 88%. The percentage of GDPDC pad by EFT as of end-Q3 2017 was 100% (note: To put GDPDC in perspective, there are only 6055 prison officers total in Afghanistan; which is about the number of ANP contained in a single province). The percentage of ANP/GDPDC personnel paid by EFT has expanded over time as shown in the following table: Tag: Efficiency e-Governance Donor Technology Capacity Building Institutional Strengthening National Institutions
30.	SPM support to data cleansing between WEPS and AHRIMS and ID “cleansing” in support of CSTC-A led APPS development The LOTFA SPM Project, with the assistance of the MOIA “twinned” staff has made progress between 2015 and 2017 with its data cleansing efforts to ensure that the WEPS database legacy data is reliable when transferred to CSTC-A for migration to the APPS database. One challenge encountered by the SPM Project was the fact that MOIA for many months refused to grant SPM access to the “back end” of the AHRIMS database. AHRIMS contains the MOIA’s human resources data for ANP/GDPDC, which is generated from PFOs/PHROs, based upon IDs. Meanwhile, the WEPS system makes payments on the basis of an individual ANP/GDPDC officer/patrolmen being able to prove identity with an ID. Thus, the ID is the common field that the two separate databases have in common. Reconciling and “cleansing” the data is a time-consuming task, made more difficult by the fact that MOIA had not granted WEPS full access to AHRIMS until end-2016/early-2017. (See, SPM-TWG reports where this was a frequent concern of the donors). Tag: Effectiveness Sustainability e-Governance Technology Capacity Building Institutional Strengthening
31.	Despite repeated requests from the SPM WEPS team, AHRIMS continued to refuse to form a data reconciliation interface with LOTFA. AHRIMS is in essence “out dated” and most all stakeholders agree that AHRIMS should be completely replaced by “APPS” database in 2018-19. The SPM WEPS Team states that as of end-2017 over 90% of the IDs are cleansed in preparation for APPS. But SPM notes that this has been very difficult to keep up with due to rapid changes and turn-over in the Tashkill due to high numbers of ANP desertion, attrition and numbers of ANP killed in action. One issue of concern to the Evaluation team is that the CSTC-A development of APPS currently does not cover the GDPCD. UNDP senior management has raised this concern with CSTC-A. MOIA has found it difficult to keep up with the frequency of changes in the Tashkill and feels that APPS will be an improvement. The Tashkill is changing each year at a rate that was formerly seen in every 3 to 4 year span on time. Tag: Effectiveness Sustainability e-Governance Technology Capacity Building Institutional Strengthening National Institutions
32.	Volume of WEPS Help Desk Calls and Categories The Help Desk has formed a crucial part of WEPS system management and overall communication between the WEPS Team, MOIA, ANP, PHQ and PFOs. The Help Desk sits with the WEPS database team. The SPM Project WEPS Help Desk appears to be operating with a high-degree of efficiency and has responded to many complaints from provinces and questions relating to payroll, reconciliation and security. In 2016, the WEPS Team developed an online “ticketing” system for Help Desk complaints. Tag: Sustainability e-Governance Technology National Institutions
33.	Systems infrastructure Development Disaster Resilience and Recovery (DRR) Plan The development of the MOIA DRR Plan for WEPS and payroll has been delayed and “off track” during the entire 2015-2017 programming period, principally due to MOIA’s refusal to allocate space for DRR servers. The SPM-Project pursued this issue diligently and repeatedly made proposals to MOIA based upon detailed assessments of possible locations to house the back-up DRR servers at MOIA facilities. The SPM-Project continued to report on progress on DRR at TWG meetings. Yet the MOIA kept rejecting the DRR proposals. Finally, near the end of 2016, MOIA approved and confirmed that the Training and Education General Command was a suitable site to house back-up servers for WEPS DRR and staff. Tag: Disaster risk management Sustainability e-Governance Technology Capacity Building Institutional Strengthening National Institutions
34.	Internet Connectivity Issues ICT connectivity and infrastructure is the “backbone” of the WEPS payroll system at both the national and sub-national levels. ICT infrastructure also forms a necessary part of any DRR plan. Achieving full Internet connectivity for both ANP and GDPDC, however, has been a challenge for LOTFA SPM. Tag: Sustainability Communication Technology
35.	The SPM Project and WEPS consultants developed a plan in 2016 for a more permanent solution that proposes to connect all GDPDC offices to the MOIA intranet via microwave bridges (i.e., wireless point-to-point connections). Yet, this process has been slow. The main cause of the delay is that in 20 provinces, GDPDC payroll stations cannot be directly connected to the LAN (Local Area Network) of PPHQs due to lack of “line of sight”. LOTFA-SPM raised this issue with the MOIA with a request that MOIA permit AWCC (MOIA’s communications provider) to permit the SPM Project contractor to install microwave wireless bridges on AWCC’s towers. MOIA refused this request, however, citing the contractual limitations of MOIA’s contract with AWCC. Thereafter, the SPM contractor proposed in mid-2017 that a VSAT (Very Small Aperture Terminal)—a satellite communications system—be installed as an alternative in the 20 provinces. This was deemed to be not a cost-effective solution. As of end-2017, discussions were still on-going between the SPM Project, MOIA and the contractor in an attempt to find a low-cost solution. SPM supported installation of fibre optics at the MOIA in Kabul procurement process stalled in 2015. Yet, SPM Project was successful in procuring ICT equipment for the MOI-ICT department in Kabul. Tag: Sustainability Communication Technology
36.	Enhanced Network Monitoring for Selected Regions (Solar Winds) In order to monitor performance and availability of WEPS in five regions in Afghanistan (Kandahar, Herat, Paktiya, Balkh and Kabul), LOTFA-SPM procured and deployed a network monitoring software solution in 2016 called “Solarwinds Network Performance Management”. This has enabled the SPM WEPS team to monitor issues related to WEPS and connectivity and to pro-actively respond to issues. Data gathered from the software platform logs indicated that in 2016 Kandahar, Balkh and Paktiya were the most problematic provinces due to poor Internet connectivity. In a further effort to prevent network downtime, SPM conducted specialized training on Solarwinds software in 2017. Additionally, the SPM Project supported the procurement of power back-up equipment at regional data centres. This effort proved successful and has greatly delays in payroll caused by connectivity issues. No payroll stations encountered delays due to network connectivity in Q1 through Q3 of 2017. Tag: Monitoring and Evaluation Results-Based Management Technology
37.	Cyber-security MOIA conducts periodic cyber-security assessments via contractors funded by CSTC-A and CSTC-A monitors the cyber-security aspects of the MOIA systems. This is not a function of LOTFA-SPM, although the SPM Project has taken measures such as encryption, user passwords, etc. to protect data and access. The Evaluation suggests that going forward, LOTFA-SPM Project could report to the LOTFA board on the cyber-security measures and policies surrounding the payroll process and implemented by MOIA and MOF. This could include updates and analysis on how MOIA and MOF cyber security measures and policies protect the security of the payroll process and data; including reporting on MOF’s “vetting” of GIROA’s “up stream” counter-parts such as “Da Afghanistan Bank” and commercial correspondent banks through which payments are distributed. Again, while this is currently not a function of the LOTFA-SPM project, cyber-security is a significant issue. Tag: Security Technology
38.	LOTFA Funds transfer to MOF for police pay LOTFA Commitment Letters (2015, 2016, 2017) (total amounts) It is the finding of the Evaluation that resources have been supplied to LOTFA-SPM in a predictable and regular manner. Overall the Donors have fulfilled their commitments to LOTFA in a timely manner, proactively and transfer money on time. In general, 95% of Donor contributions to LOTFA go to fund the ANP payroll under the LOTFA-SPM Project, with the other 5% going towards the LOTFA-MPD Project’s capacity building outputs and activities. Tag: Effectiveness Sustainability e-Governance Donor Technology
39.	Total LOTFA NIM Advances Paid to MOF vs. total NIM expenditures (liquidation) recorded in AFMIS Budget Delivery Rates The total transferred LOTFA funds to the Afghan police in 2015 were USD 451,355,568. The total annual expenditure was USD 442,476,891. This translates in a delivery rate of 98.26%. For 2016 annual cycle; the results provide an overall annual delivery rate of 98.3%. Tag: Efficiency Human and Financial resources Operational Efficiency
40.	Incentives paid to ANP In regard to incentives the SPM WEPS team developed a standard template with dropdown menus to select the applicable incentive. Incentive payments delivered to the AFN classified by type, during the period August-December of 2015, were USD 263,256,829 in total. USD 8,397,525 were paid for Cadre, USD 23,694,250 for Medical, USD 17,283,507 for Engineering and USD 213,881,547 for Special Operations positions Tag: Efficiency Human and Financial resources Operational Efficiency
41.	Monitoring Agent Expenditure Verification Exercise and Second-round physical verification As discussed elsewhere in this report, through the expenditure verification exercise, the MA has identified ineligible ANP/CPD salary payments amounting to USD 715,633.18 during January–July 2016. Of this amount, deductions totaling USD93,998.48 have been made while clarifications are still pending for the remaining amount of USD 621,634.69 [Note: It is beyond the scope of this Evaluation report to include all statistics contained in the MA reports. See the ANNEX to this report for some additional statistics]. Tag: Efficiency Human and Financial resources Operational Efficiency
42.	% of Payroll Stations able to Process Salary Payments On-time The percentage of financial reports finalized within 7 days of the close the month was 92% for 2015. The percentage of financial reports finalized on time was 90% for 2016. For Q1 2017, the percentage of payroll stations able to process monthly salary within 15 days of the close of the month was 99%. Most delays were caused by security concerns and New Kabul Bank Internet connectivity problems. For Q2 2017, the percentage was 99%. For Q3 2017, the percentage was 99%. Thus, the average actual payroll rates were near to 100% during all of 2017. Tag: Efficiency Human and Financial resources Operational Efficiency
43.	FFFMU Oversight The FFFMU has fulfilled its intended role to manage the funds and enforce a level of compliance in the payments system. FFFMU has reviewed MOIA expenditures for compliance with the agreed object codes fields in the annual LOTFA commitment letters in a timely manner. FFFMU has proven its importance to compliance by questioning MOIA charges that are outside the scope of the negotiated commitment letter and also deducts from the tranche released to MOIA based upon MA reports. In addition, the FFFMU has regularly attended the TWG meetings and receives donor reports. Tag: Efficiency Human and Financial resources Operational Efficiency Oversight
44.	SPM reporting to Donors In addition, UNDP and the SPM Project have regularly reported to the Donors and the LOTFA-Board, both informally and formally on a bilaterally and according to the LOTFA-SPM project document and governance structure frequently during the time period July-2015 to end-2017. This has occurred in the form of SPM Project quarterly and annual reports and in SPM-TWG, Combined-TWG and in LOTFA-Board Meetings. LOTFA Governance and reporting are further discussed under Output 6 infra. Tag: Sustainability Oversight Project and Programme management
45.	Effective and efficient implementation of the SPM Project Since mid-2015, LOTFA has instituted significant changes in terms of oversight, governance, procurement and reporting. The Evaluation interviewed a number of stakeholders to probe issues of SPM-Project Governance; implementation; lines of reporting between UNDP and MOIA; LOTFA-SPM Governance and Oversight and partnerships with implementing partners. The overall impression of Stakeholders was that the LOTFA-SPM Project Document and AWPs were the product of an inclusive and consultative process with the MOIA and UNDP. The Project document(s) (2016 and 2017 revision/extension) clearly articulate the respective roles of the MOIA, the UNDP Country Office, SPM Project, LOTFA Board and SPM-TWG and three sub-TWGs. Furthermore, the Project documents were informed by the findings of a 2014 DIM audit; a due diligence audit conducted by the United Kingdom and the UNDP Country Office Support Services to LOTFA audit. In sum, it is the finding of the Evaluation that LOTFA-SPM’s various governance and oversight mechanisms have fulfilled their targets and objects at end-2017. Tag: Effectiveness Efficiency Implementation Modality Country Government
46.	UNDP CO Programme Unit and LOTFA-SPM Project The UNDP CO, UNDP Country Director and the UNDP Rule of Law and Human Security Unit have been highly effective in both representing UNDP as an organization and in interfacing regularly with the LOTFA-Board and MOIA. UNDP organized numerous meetings with donors, the MOIA/ANP and the LOTFA-SPM Project during mid-July to end-2017 that were held in addition to the SPM-TWG meetings and LOTFA-Board meetings. This included discussion of the results of the Micro-Capacity Assessments supported by the project. UNDP senior management and the UNDP Rule of Law and Human Security Unit have been instrumental in assisting LOTFA-SPM to formulate its AWP and M&E frameworks. Additional bi-lateral meetings were held by UNDP/SPM-Project with Donors throughout 2016-17 to discuss the draft MOIA Payroll Unit Plan and the overall payroll transition agenda. UNDP and the SPM Project also met frequently with the MOIA DM for Policy and Strategy and the GD for Finance and Budget in preparing the Payroll Unit Plan. Tag: Sustainability Human and Financial resources Programme Synergy UNDP Independent Offices
47.	As noted elsewhere in this report, the Evaluation found that levels of communication between MOIA staff and SPM Project staff could be strengthened. Most of the MOIA DG interviewed by the Evaluation stated that UNDP and the SPM Project staff communicated expectations clearly and met with them frequently. However, some MOIA senior officers interviewed by the Evaluation complained that they felt excluded from SPM-TWG meetings and the decision process. The Evaluation heard such testimony frequently enough during its time in Kabul that it is cause for concern. Tag: Sustainability Communication Human and Financial resources
48.	LOTFA Board (Steering Committee) The Evaluation attended and observed the December 2017 meeting of the LOTFA Board, as well as reviewed notes and minutes from previous LOTFA Board meetings. The LOTFA Board appears to have worked well during the time period 2015 to 2017 as a vehicle for discussing and approving issues at the highest level of the MOIA. Its meetings are high profile and attended by all Donors and stakeholders, including UNDP, MOIA staff, ANP senior commanders, CSTC-A and other technical experts and consultants. The LOTFA-Board met twice yearly in July and December of 2015, 2016 and 2017. The Board has closely monitored progress against the “Donor Conditions” and approved an extension of the SPM Project in December 2016 for an additional year. Most recently, at its December 2017 meeting the LOTFA Board approved across the board pay increases for the ANP and incentives in order to equalize ANP with ANA (MOD) pay and incentives. This amounted to an across the board increase of ANP salaries of 5% at a total annual cost of $50 million per year. At its December 2017 meeting, the LOTFA Board also approved recommendations of the Pay and Compensation Board for changes to the approved incentives for ANP that included approval of incentives for medical; hazardous duty; danger pay and pay for ANP special forces.64 Tag: Sustainability Oversight Project and Programme management Donor
49.	SPM-TWG and sub-working groups The Evaluation reviewed a sample of combined SPM-TWG meeting minutes and materials held in years 2016 and 2017. It appears that the SPM-TWG and its three sub-TWG have fulfilled their Terms of Reference. [See LOTFA Project Document 2015 and LOTFA SPM Project Document 2016]. The representative sample of TWG meetings reviewed by the Evaluation shows that the Donors actively engaged in Q&A with LOTFA-SPM team leaders and staff making presentations and reports at the TWG meetings. In the period July-2015 to December 2015, 19 out of a total of 24 possible TWG meetings were held (79%) and during the year 2016 a total of 44 TWGs (both sub-TWG and combined SPM and MPD TWG) were held. During 2017, SPM held 7 TWG meetings from March -Dec 2017; and in Jan- Feb 2017, SPM had 3 sub-TWGs, and 1 combined TWG (for SPM and MPD projects). Three SPM TWG meetings were cancelled for the months of June, September and December 2017 due to lack of quorum /security reasons].67 Tag: Monitoring and Evaluation Oversight Project and Programme management
50.	MOIA General Directorate for International Cooperation (GDIC) The Evaluation was not able to interview the MOIA GDIC while in Kabul. Thus, it is difficult to assess the extent to which the MOIA General Directorate for International Cooperation (GDIC) has taken on increasing responsibilities for management and administration of LOTFA governance. The Evaluation’s findings that i) there was no MOIA request made upon the SPM Project to train staff for specific posts; ii) the fact that most developments and outputs during 2015 to 2017 appear to be either SPM or CSTC-A-led; and iii) the lack of sufficient capacity at MOIA as of end-2017 to warrant a partial transition; as well as iv) slow response on the part of MOIA for crucial elements of the payroll process (i.e. DRR), lead the Evaluation to the conclusion that MOIA GDIC could have been more proactive and taken on more duties. This said, however, MOIA staff were at all SPM-TWG meetings, accompanied SPM Project staff to the provinces to conduct systems inspections, verification and trainings and actively participated in the drafting of such deliverables as the “Payroll Unit Plan” and “Payroll Management Procedures Manual”. In addition, MOIA senior officers and staff attended all LOTFA Board meetings. These factors point to the GDIC’s involvement. Tag: Sustainability Bilateral partners Country Government National Institutions
51.	Implementation of Audit and Monitoring Recommendations Throughout 2015 and 2016 the SPM Project worked to implement the recommendations from the 2015 LOTFA DIM audit administered by the Office of Audit and Investigations (OAI), the 2015 NIM audit and its 14 recommendations; and the UK Due Diligence assessment. UNDP and the Country office also supported and facilitated the Micro-Capacity Assessments in 2016. UNDP, SPM Project and the Donors have also relied upon the findings of the Monitoring Agent (MA) consistently throughout 2016 and 2017. The findings of the MA are described in more detail in above sections of this report. As of end-2017, MOIA had completed its first-ever audit of its payroll using MOIA civilian staff and according to an OIG-approved checklist and criteria. Although in draft form and not made available to the Evaluation team, the audit is the first time that MOIA has actually identified specific individuals responsible for misreporting and committing fraud. It remains to be seen what action MOIA will take based upon the audit findings once finalized. [Note: The LOTFA-MPD Project has had outputs for OIG during 2015-2017]. Tag: Implementation Modality Monitoring and Evaluation Oversight
52.	Risk Management The Evaluation finds that the above LOTFA-SPM governance and oversight mechanisms have served to effectively manage risk during the period mid-2015 to end-2017. Analysis and discussion of risk was on the agenda of nearly every TWG meeting held in 2016-2017. SPM Project management utilized the TWG meetings to bring timely information of perceived risks to the attention of GIROA and the Donors (i.e. TWG meeting of 31 July 2017, wherein the SPM Project Team presented the risks of not continuing with the Disaster Recovery Plan (DRP), following a formal communication from MOIA’s ICT Department suggesting DRP was no longer a necessity). This proved invaluable during the life of the project and in the case of DRP (DRR) it enabled the Donors and SPM to convince MOIA to reverse its decision to put the DRP on hold and to resume negotiations for a site to house WEPS DRR. The SPM AWP, Risk Log and Monitoring Frameworks also helped SPM to manage risk. Operationally, the SPM Project can be seen effectively managing risk on a variety of levels. This is seen in the SPM Project Team’s response to the destruction of WEPS servers in Kunduz when anti-government forces seized control. As stated above, in 2016, UNDP commissioned an independent assessment of the LOTFA Risk framework, as part of the design and development of new post-2017 trust fund arrangements. Tag: Sustainability Risk Management Country Government Donor
53.	Gender The LOTFA SPM and MPD projects have closely coordinated in the area of gender. LOTFA-SPM as a project has had gender-specific outputs geared to improve the visibility of female ANP officers, their inclusion in the MOIA payroll and their capacity to participation in MOIA payroll data entry and management. Women—whether female ANP officers and patrolmen or the families of ANP officers killed or wounded in action or the communities served and protected by ANP –are either direct or indirect beneficiaries of SPM. Tag: Sustainability Gender Equality e-Governance Human and Financial resources Technology Capacity Building Institutional Strengthening National Institutions
54.	Such data has also enabled MOIA to track the distribution of female police officers across Afghanistan’s provinces and would not have been possible without the LOTFA-SPM intervention. The percentage of ANP and GDPDC paid by EFT has also been disaggregated by gender. The Evaluation finds that going forward, more LOTFA-SPM presentations and reports could provide gender-disaggregated statistics. SPM-TWG meeting materials should continue to provide such statistics. Tag: Sustainability Gender Equality e-Governance Human and Financial resources Technology Capacity Building Institutional Strengthening National Institutions
55.	National Ownership and the UNDP National Implementation Modality (NIM) The Assessment of the Evaluation is that in many ways the LOTFA-SPM Project has acted more like a DIM project than a NIM project. The LOTFA SPM project actually operates on a technical level as DIM. The Evaluation assesses that “national ownership” of the activities remains moderate as of end-2017. The Evaluation fully realizes, however, that the SPM Project has been significantly challenged in institutionalizing knowledge and trainings due to high turnover of staff and leadership at the MOIA. SPM appears to have been effective at training MOIA at the central level and provincial level on how to enter data into the “front end” of the WEPS system. It appears that MOIA the level of enthusiasm for such trainings at MOIA continues to be strong. Tag: Sustainability Implementation Modality Ownership Country Government Capacity Building Institutional Strengthening
56.	A review of the TWG meetings in 2016 and 2017 shows that staff from the President’s office have attended meetings, but not always. H.E. President Ghani’s appointed a special representative to liaise regularly with the SPM Project, TWG and LOTFA Project Board; however, stakeholders informed the Evaluation that this individual had been out of the country for much of 2017. Thus, LOTFA SPM-Project’s links to the President’s office could be strengthened. The Evaluation is making the recommendation that a “task force” of the LOTFA Board or a new sub-TWG be created for purposes of reporting on the transition both to the LOTFA Board and to the President’s office. The appointment of new Minister of the MOIA brings with it hope that there will be a renewed sense of urgency on the part of GIROA towards taking ownership of the payroll process, implementing necessary reforms and hiring new staff in accordance with the Donor Conditions and the Payroll Unit Plan. Uncertainty surrounding upcoming elections and spectre of further turn-over in MOIA leadership and staffing depending on election result will necessitate that the sustainability of LOTFA and national ownership be re-evaluated post-election. Tag: Sustainability Ownership Country Government
57.	The Donor Conditions and Prospects for Transition of LOTFA Payroll Functions to MOIA and Exit Strategy The scope of future support, including an exit strategy for LOTFA-SPM will depend upon whether or not the Donors and LOTFA Board approve a partial “transition” of LOTFA-SPM payroll functions to the MOIA in mid-2018 or whether the project continues on a “no transition” basis. The Donors have stipulated that the transition of LOTFA-SPM payroll management to the MOIA as “conditions-based”, not “time-based”, by way of contrast to H.E. President Ghani who has continued to set time-based deadlines. The basic idea behind the Donor Conditions is to ensure that sufficient capacity is built and key deliverables in place in advance of a transition of LOTFA SPM payroll functions to the fully functional MOIA Payroll Unit. As such, the Donor Conditions have served both as a catalyst for development and as a benchmark. Tag: Sustainability Strategic Positioning Donor National Institutions
58.	Given the fact that IDs are at the root of the payroll process, the Evaluation is suggesting that the LOTFA Board consider amending the Donor Conditions to add a new condition about the MOIA’s capacity to issue ID cards. The would entail a requirement that the MOIA strengthen its capacities for issuing biometric IDs, including timeliness of issuance and replacement; upgrading ID processing facilities in Kabul and creation of “mobile ID units” to service the ID needs of ANP at the provincial level. It is not clear to the Evaluation that the introduction of APPS will, in and of itself, solve the MOIA’s ID issuance and tracking problems. APPS will contain verified biometric data, but the current ID situation appears to be capable of repetition; unless MOIA improves its ID issuance capacity and procedures Tag: Sustainability Country Government Donor Capacity Building Institutional Strengthening
59.	It is an open question how many of the existing SPM Project staff would remain at the MOIA Payroll Unit if forced to go on an MOIA contract. While replacing current SPM staff could complicate the transition, it is certainly not an insurmountable problem for GIROA or the donors. The current SPM Project staff, including the WEPS Team, is highly qualified and capable, but they certainly are not indispensable and there are other Afghan nationals who possess similar qualifications and academic degrees. Tag: Sustainability Human and Financial resources Bilateral partners Country Government Capacity Building Institutional Strengthening National Institutions
60.	Sustainability As of end-2017, it does not appear to the Evaluation that results achieved by LOTFA to date are sustainable without continued international assistance and reinforced capacity building. While the MOIA exhibits increased public financial management capabilities as a result of the SPM support to ANP, this has not yet been institutionalized or reached a critical mass to be self-sustaining or permit even a partial transfer. Tag: Sustainability Country Government Capacity Building Institutional Strengthening
61.	A positive factor is that LOTFA-SPM has laid the foundation for future sustainability of its intervention (i.e. in WEPS system operability and design; legislative reform; the development of the Payroll Unit Plan and other SOPs and the introduction of oversight and monitoring tools). The question is the extent to which the MOIA has the political will to institutionalize such tools and LOTFA’s compliance and oversight mechanisms within the MOIA. In the opinion of the Evaluation this had yet to occur. This must be re-assessed by LOTFA in 2018 and 2019. Tag: Sustainability Human and Financial resources Oversight Programme/Project Design Risk Management Country Government National Institutions
62.	LOTFA Oversight Committee and LOTFA Risk Framework The LOTFA Oversight sub-committee held its first meetings in Q1 2016, wherein the LOTFA-SPM ToR and AWP for 2016 were reviewed and approved. In 2016, UNDP commissioned an independent assessment of the LOTFA Risk framework, as part of the design and development of new post-2017 trust fund arrangements. The revised framework was presented to donors within the LOTFA Oversight sub-Committee in May 2016, and was deemed sufficient by the LOTFA Oversight sub-Committee members. In December 2016, UNDP initiated the assessment to review the LOTFA risk plan, and strengthen this where needed. Tag: Effectiveness Relevance Oversight Risk Management
63.	Fund Fiduciary and Financial Management Unit (FFFMU) The UNDP Country Office established in 2015 a dedicated unit, the Fund Fiduciary Financial Management Unit (FFFMU), in order to provide financial oversight, reconciliation and donor reporting, as part of the trust fund arrangement for LOTFA. Established according to industry standards to manage large multi-donor contributions, the FFFMU oversees all financial and fiduciary aspects of monthly National Implementation Modality (NIM) advances for the monthly police payroll. The Evaluation’s assessment of FFFMU appears under Output 6 infra. Tag: Effectiveness Relevance Human and Financial resources Oversight
64.	The LOTFA Pay and Compensation Board In 2015 UNDP, MOIA and the LOTFA donors created the Pay and Compensation Board (“Pay Board”), and agreed on its ToR in a TWG meeting of 16 November 2015. The Pay Board is part of LOTFA’s governance structure, established to consider and approve or deny ANP proposals for incentives and entitlements and was fully operational by end-2016. It undertakes a bi-annual review of MOIA pay scales and incentive rates to ensure that the pay structure is fair and supports the sustainability of the ANP. The LOTFA Donors nominated Germany, United States, and CSTC-A to represent them on the Pay Board. The Board is Co-Chaired by the MOIA Deputy Minister (DM) for Support and the LOTFA-SPM Project Manager with additional representatives from MOIA and MOF. Tag: Sustainability Country Government Donor

Recommendations
1	The Evaluation recommends that a transfer of LOTFA SPM’s payroll functions to the MOIA not take place until such time as all donor conditions are fully met; the MOIA Payroll Unit is fully staffed and trained and the APPS data base is fully functional and deployed in all 34 provinces.
2	The Evaluation recommends that the LOTFA Board consider revising the MOIA Donor Condition B4 to add a requirement that the MOIA strengthen its capacities for issuing biometric IDs, including timeliness of issuance and replacement; upgrading ID processing facilities in Kabul and creation of “mobile ID units” to service the ID needs of ANP at the provincial level. “Clean” and current IDs are at the root of the payroll process. It is not apparent that the introduction of APPS alone will solve all of MOIA’s ID issues.
3	The Evaluation recommends that UNDP and the SPM Project strengthen its links to the President’s office, for purposes of reporting on the readiness of MOIA for a “transition” both to the LOTFA Board and to the President’s office.
4	The Evaluation respectfully recommends that the LOTFA Donors revisit the conditions proposed by the “Pay Structure and Sustainability Strategy”, and conduct a study to see if the pay increase and incentives approved by the LOTFA Board at its December 2017 meeting have sufficiently addressed the issues that were raised/used as the justification.
5	The Evaluation recommends that UNDP and the SPM Project review the format and content of reporting at TWG meetings so as to include an “Executive Summary” of each TWG meeting, and its presentations and minutes with key “action points” identified. TWGs should include tracking of legislative updates in their reports. The reform of the Police Inherent Law is a topic that should receive regular reporting within SPM-TWG meetings.
6	The Evaluation recommends that UNDP and the SPM Project monitor more extensively against CPD Outcome 2 going forward. UNDP might also consider at an appropriate time adding more detailed indicators in LOTFA projects for female ANP/GDPDC officers; women and children of surviving ANP killed in action and/or other indicators for most vulnerable groups. The “One UN” Results Matrix (from which the CPD derives) and/or future SPM AWP might be appropriate places for UNDP to consider placing such indicators.
7	The Evaluation recommends that UNDP and the SPM Project explore synergies with UNDP’s Afghanistan Access to Justice Project (AA2J)—particularly to enable the SPM Project to benefit from data from the AA2J Legal Aid Grant Facility (which offers legal aid to vulnerable groups across 8 of Afghanistan’s provinces) in order to better assess the link between payroll management for ANP and citizen’s experiences and perceptions of ANP and security.
8	The Evaluation recommends that UNDP and the LOTFA SPM Project continue to advocate with MOIA/ANP that more female officers be assigned to units having payroll functions.
9	The Evaluation recommends that MOIA and the SPM Project immediately prepare an inventory of training needs of all female ANP/GDPCD personnel having responsibility for the payroll process and develop a dedicated training plan and schedule for these female personnel. If ANP has not assigned any female offices to units or WEPS data entry and has not presented female officers for training, this fact should be clearly documented for each training; including a statement from MOIA as to why ANP could not supply female officers for such training.
10	The Evaluation recommends that the SPM Project implement a training plan for the Payroll Procedures Manual in all 34 provinces in 2018.
11	The Evaluation recommends that UNDP and the LOTFA Board review the SPM Project’s “twinning” of MOIA staff, including how individual MOIA staff have been tasked by the SPM Project; a review of each MOIA twinned staff member’s academic credentials and professional background; and the extent to which SPM Project “twinning” has aligned with the approved MOIA Payroll Unit Plan and Payroll Unit Capacity Building Plan. UNDP should ensure that the SPM Project “twinning” is sufficiently robust as to ensure a meaningful opportunity for MOIA to assume the non-fiduciary aspects of the payroll process. The twining programme going forward should take into account also the change over from WEPS to APPS that is scheduled to take place in 2018. The LOTFA Board, UNDP and the SPM Project should continue to advocate with MOIA for MOIA to hire IT qualified staff and refer them to the SPM Project for training and “twinning”.
12	The Evaluation recommends that the SPM Project incorporate enhanced “anti-corruption” outputs and activities (and/or indicators) in its programming and that the MOIA take all necessary steps –per MOIA Donor Condition B12—to ensure that the MA findings pointing to possible fraud or corruption within the MOIA or ANP—specifically related to and involving the payroll process—are investigated by the Inspector General Office (IGO) and violators referred to the Attorney General of Afghanistan for prosecution. Any SPM outputs in this regard should be closely coordinated with MPD Project outputs and activities for the IGO to bolster MOIA internal audit capacity and capability and the capacity of the anti-corruption cell within the IGO.
13	The Evaluation recommends that the MOIA OIG and MA more closely cooperate and coordinate and that the LOTFA Board expand the scope of the MA’s work including possibly extending the duration of the MA’s time spent in each province.
14	The Evaluation recommends that UNDP and the LOTFA Board advocate with CSTC-A to accelerate the piloting of the APPS database to achieve its full deployment by mid-2018; including conducting required APPS orientation and training sessions for MOIA staff at central and provincial levels.
15	The Evaluation recommends that UNDP and the LOTFA Board advocate strongly with GIROA/MOIA that a comprehensive forensic audit of the MOIA (ANP) payroll system be completed within 6 months following full operability and deployment of the APPS system, but in any event, that such audit be completed by no later than 01 April 2019. It is envisioned that such an audit would be more comprehensive and wide-ranging than the typical “sampling” conducted by the MA in its monthly reports. This is viewed as necessary to establish a new “baseline” to inform LOTFA and GIROA decision-making and future SPM programming at the point in time at which the APPS system is made fully operational.
16	The Evaluation recommends that the SPM Project probe the reasons underlying ANP willingness (or lack thereof) to use MM platforms. It is recommended that, if necessary, the SPM Project institute a new round of trainings for Provincial ANP Commanders; PPQH; PFOs and PHROs with the objective of expanding the percentage of ANP paid through the Mobile Money mechanism and to become familiar with the PERSTAT template.
17	The Evaluation recommends that the MOIA facilitate continued access to the AHRIMS database to enable SPM Project staff to crosscheck AHRIMS records against data in WEPS to enhance the integrity of WEPS legacy data transferred to CSTC-A in connection with the development of APPS.
18	The Evaluation recommends that MOIA continue to cooperate with the SPM Project to implement the DRR system for WEPS. MOIA should give utmost priority to the DRR system establishment.
19	The Evaluation recommends that going forward, LOTFA-SPM Project begin to report to the LOTFA board on the cyber-security measures and policies surrounding the payroll process and implemented by MOIA and MOF surrounding the payroll process. This could include updates and analysis on how MOIA and MOF cyber security measures and policies protect the security of the payroll process and data; including reporting on MOF’s “vetting” of GIROA’s “up stream” counter-parts such as Da Afghanistan Bank and commercial correspondent banks through which payments are distributed. While this is currently not a function of the LOTFA-SPM project, cyber-security is a significant issue.
20	The Evaluation recommends that UNDP and the SPM Project assess the manner and quality of engagement of SPM staff with MOIA senior “uniformed” officers and ensure that relations with MOIA uniformed staff are conducted with respect and courtesy.
21	The Evaluation recommends that the SPM Project educate ANP and their family members about the pay process; pay slips and entitlements via radio and TV broadcasts in addition to distributing printed materials.

1. Recommendation:

The Evaluation recommends that a transfer of LOTFA SPM’s payroll functions to the MOIA not take place until such time as all donor conditions are fully met; the MOIA Payroll Unit is fully staffed and trained and the APPS data base is fully functional and deployed in all 34 provinces.

Management Response: [Added: 2018/10/07] [Last Updated: 2021/01/13]

The decision to transfer LOTFA SPM’s payroll functions rests with the LOTFA project board. Achieving the conditions of the Donors Conditions Monitoring framework for the transfer of payroll functions to the MOIA would be the basis of a decision to transfer to payroll functions to the MOIA.The role of UNDP would be to advise the project board on the fulfillment of these conditions.

Fulfillment of the donor conditions will mean that MOIA has sufficient organizational and human capacity to successfully carry out the payroll functions [Donor Condition 11 and 12) and the APPS system is fully functional and deployed in all provinces.

Key Actions:

Key Action	Responsible	DueDate	Status	Comments	Documents
Carry out Risk and Accountability Assessment of WEPS-APPS transfer [Added: 2018/10/07]	ROLHSU	2018/06	Completed

2. Recommendation:

The Evaluation recommends that the LOTFA Board consider revising the MOIA Donor Condition B4 to add a requirement that the MOIA strengthen its capacities for issuing biometric IDs, including timeliness of issuance and replacement; upgrading ID processing facilities in Kabul and creation of “mobile ID units” to service the ID needs of ANP at the provincial level. “Clean” and current IDs are at the root of the payroll process. It is not apparent that the introduction of APPS alone will solve all of MOIA’s ID issues.

Management Response: [Added: 2018/10/07] [Last Updated: 2021/01/13]

The LOTFA Donors’ decision to pay only the PAI (Personnel Asset Inventory) verified ANPs since April 2018 has resolved this issue. The PAI is a MOIA led process supported by CSTC-A which requires the ANP personnel to biometrically register, complete 20 mandatory personal information fields, and obtain an ANP identification card (ID). The enforcement of LOTFA donors’ restriction of salary payment to only police that have been registered through PAI has yielded positive results. As of 30 August 2018, only 8,657 police officer were not registered and salary payment for those stopped pending their registration through PAI.

Key Actions:

Key Action	Responsible	DueDate	Status	Comments	Documents
Communicate the recommendation to the donors in LOTFA donors and UNDP monthly meetings. [Added: 2018/10/07]	ROLHSU	2018/06	Completed

3. Recommendation:

The Evaluation recommends that UNDP and the SPM Project strengthen its links to the President’s office, for purposes of reporting on the readiness of MOIA for a “transition” both to the LOTFA Board and to the President’s office.

Management Response: [Added: 2018/10/07] [Last Updated: 2021/01/13]

UNDP enjoyed cordial relationships with the Presidential Advisor, Mr. Malikzai, before he left his office. UNDP agrees to this recommendation and seeks ways to strengthen working relations with the President’s office on LOTFA and in particular, payroll transition issues.

Key Actions:

Key Action	Responsible	DueDate	Status	Comments	Documents
The UNDP leadership will consult the MOIA Minister to identify a point of contact in the President’s office. [Added: 2018/10/07] [Last Updated: 2019/07/14]	ROLHSU	2019/06	Completed	This action completed

4. Recommendation:

The Evaluation respectfully recommends that the LOTFA Donors revisit the conditions proposed by the “Pay Structure and Sustainability Strategy”, and conduct a study to see if the pay increase and incentives approved by the LOTFA Board at its December 2017 meeting have sufficiently addressed the issues that were raised/used as the justification.

Management Response: [Added: 2018/10/07] [Last Updated: 2021/01/13]

UNDP will communicate this recommendation to the LOTFA donors to revisit the conditions proposed by the “Pay Structure and Sustainability Strategy” and conduct a study to ascertain whether the pay increase and incentives approved by the LOTFA Board at its December 2017 meeting have sufficiently addressed the issues that were raised/used as the justification.

The pay award approved by The LOTFA donors at the December 2017 project board meeting had no link with the pay structure and sustainability study. UNDP has shared the pay structure report with LOTFA donors and made presentations on the report to them prior to their approval of the pay awards in December 2017.

Key Actions:

5. Recommendation:

The Evaluation recommends that UNDP and the SPM Project review the format and content of reporting at TWG meetings so as to include an “Executive Summary” of each TWG meeting, and its presentations and minutes with key “action points” identified. TWGs should include tracking of legislative updates in their reports. The reform of the Police Inherent Law is a topic that should receive regular reporting within SPM-TWG meetings.

Management Response: [Added: 2018/10/07] [Last Updated: 2021/01/13]

The Technical Working Group (TWG) meeting minutes structure was revised in 2016 to reflect ‘action points’ and ‘who should do what and by when’, in addition, the minutes of the previous meeting are reviewed by members before proceeding with other agenda items. The recommendation on improving the contents of the presentations/reports is well noted and the project will make all efforts to capture relevant topical issues where possible. TWGs are not designed for discussing legislative issues but will endeavor to capture such matters in the meeting minutes should such topics be discussed at any TWG meeting

Key Actions:

Key Action	Responsible	DueDate	Status	Comments	Documents
SPM project to undertake holistic review of the TWG presentations and ensure that these capture relevant topical issues and reflect results achieved [Added: 2018/10/07]	SPM project management	2018/07	Completed	The TWG minutes are reflecting action points and these are reviewed at subsequent meetings

6. Recommendation:

The Evaluation recommends that UNDP and the SPM Project monitor more extensively against CPD Outcome 2 going forward. UNDP might also consider at an appropriate time adding more detailed indicators in LOTFA projects for female ANP/GDPDC officers; women and children of surviving ANP killed in action and/or other indicators for most vulnerable groups. The “One UN” Results Matrix (from which the CPD derives) and/or future SPM AWP might be appropriate places for UNDP to consider placing such indicators.

Management Response: [Added: 2018/10/07] [Last Updated: 2021/01/13]

UNDP would consider adding more detailed indicators in LOTFA projects for female ANP/GDPDP officers; women and children of surviving ANP killed in action and/or other indicators for most vulnerable groups when designing new security sector projects. UNDP would also establish a proper Monitoring and Evaluation Framework to assess the impact of UNDP projects on key security and rule of law indicators.

Key Actions:

Key Action	Responsible	DueDate	Status	Comments	Documents
Develop Monitoring and Evaluation System for LOTFA moving forward. [Added: 2018/10/07] [Last Updated: 2019/07/14]	ROLHSU	2019/06	Completed	Work on the strategy completed

7. Recommendation:

The Evaluation recommends that UNDP and the SPM Project explore synergies with UNDP’s Afghanistan Access to Justice Project (AA2J)—particularly to enable the SPM Project to benefit from data from the AA2J Legal Aid Grant Facility (which offers legal aid to vulnerable groups across 8 of Afghanistan’s provinces) in order to better assess the link between payroll management for ANP and citizen’s experiences and perceptions of ANP and security.

Management Response: [Added: 2018/10/07] [Last Updated: 2021/01/13]

The revised LOTFA Terms of Reference, currently under review, would strengthen the linkages between different UNDP Rule of Law projects and enable cross-project information sharing for management decisions including the assessment of ANP performance on the rule of law and security.

Key Actions:

Key Action	Responsible	DueDate	Status	Comments	Documents
Implement the LOTFA M&E System; conduct the baseline assessment surveys and LOTFA impact evaluation surveys. [Added: 2018/10/07] [Last Updated: 2021/01/11]	ROLHU	2020/12	Completed	This activity is completed

8. Recommendation:

The Evaluation recommends that UNDP and the LOTFA SPM Project continue to advocate with MOIA/ANP that more female officers be assigned to units having payroll functions.

Management Response: [Added: 2018/10/07] [Last Updated: 2021/01/13]

SPM will advocate with the MOIA General Directorates of Personnel, and Finance and Budget to assign more women into payroll functions- human resources and finance. SPM will also collaborate with the General Directorate of Gender to join the campaign for inclusion of more women in payroll functions. Lastly, SPM will also coordinate with CSTC-A which pays female incentives to advocate for inclusion of women in payroll functions, and the same women could be supporting payroll processing in APPS later.

Key Actions:

Key Action	Responsible	DueDate	Status	Comments	Documents
Liaison with MOIA GDOP, CSTC-A, MOIA Gender dept. [Added: 2018/10/07] [Last Updated: 2021/01/14]	SPM project management	2021/12	Initiated	The first phase of APPS implementation commenced in August 2020, whereas Time and Attendance generated in APPS started to inform payroll in WEPS. The SPM project monitors the changed roles of the project in the new business processes driven by CSTC-A and identifies entry points to promote female participation in police payroll management.

9. Recommendation:

The Evaluation recommends that MOIA and the SPM Project immediately prepare an inventory of training needs of all female ANP/GDPCD personnel having responsibility for the payroll process and develop a dedicated training plan and schedule for these female personnel. If ANP has not assigned any female offices to units or WEPS data entry and has not presented female officers for training, this fact should be clearly documented for each training; including a statement from MOIA as to why ANP could not supply female officers for such training.

Management Response: [Added: 2018/10/07] [Last Updated: 2021/01/13]

The SPM project has always encouraged the MOIA to have female police in payroll management, but this effort has not yet been successful. The project is always willing to provide training on payroll management to female police who would be deployed to payroll management. The GDPDC has 3 female police in payroll management and over the years SPM has included those in the training programs and they have benefitted from several of our trainings. Our training reports always indicate the fact that no female staff were trained.

Key Actions:

Key Action	Responsible	DueDate	Status	Comments	Documents
Project to present the case for more female police to be involved in payroll management at subsequent TWG meetings where IP and donors meet. [Added: 2018/10/07] [Last Updated: 2021/01/14]	SPM Project Management	2021/12	Initiated	The training activities on various aspects of payroll management were suspended due to COVID-19 lockdown restrictions in 2020.

10. Recommendation:

The Evaluation recommends that the SPM Project implement a training plan for the Payroll Procedures Manual in all 34 provinces in 2018.

Management Response: [Added: 2018/10/07] [Last Updated: 2021/01/13]

The SPM project has already prepared a comprehensive training plan which covers all manuals recently produced for MOIA—i.e. payroll procedures manual; accounting manual; and internal controls policy manual. The training programs are being delivered to all 34 provinces and the manuals will be distributed to each and every payroll station in all 34 provinces.

Key Actions:

Key Action	Responsible	DueDate	Status	Comments	Documents
Roll-out training to all provincial finance officers; human resource officers and staff of the general directorate of finance ad budget [Added: 2018/10/07] [Last Updated: 2018/12/09]	SPM project Capacity Building team	2018/12	Completed	Training shall be delivered in collaboration with MOIA’s general directorate of finance and budget and the policy directorate. The Internal control policy is not approved and the training is also pending.

11. Recommendation:

The Evaluation recommends that UNDP and the LOTFA Board review the SPM Project’s “twinning” of MOIA staff, including how individual MOIA staff have been tasked by the SPM Project; a review of each MOIA twinned staff member’s academic credentials and professional background; and the extent to which SPM Project “twinning” has aligned with the approved MOIA Payroll Unit Plan and Payroll Unit Capacity Building Plan. UNDP should ensure that the SPM Project “twinning” is sufficiently robust as to ensure a meaningful opportunity for MOIA to assume the non-fiduciary aspects of the payroll process. The twining programme going forward should take into account also the change over from WEPS to APPS that is scheduled to take place in 2018. The LOTFA Board, UNDP and the SPM Project should continue to advocate with MOIA for MOIA to hire IT qualified staff and refer them to the SPM Project for training and “twinning”.

Management Response: [Added: 2018/10/07] [Last Updated: 2021/01/13]

The SPM initially requested 6 MOIA staff –2 Finance staff; 2 HR staff; and 2 IT staff –to be co-located with the SPM staff. The MOIA provided only 4 staff—2 HR staff and 2 Finance staff. Later, 2 of the HR staff got withdrawn. Due to lack of technically competent staff in MOIA they could not sponsor any IT-qualified staff to co-locate with the WEPS team. The SPM Project has had a good working relationship with the MOIA staff, working with them on daily basis. he SPM project is working with the MOIA finance and budget directorate to ensure that the APPS system achieves full operational capability and all the relevant staff can receive appropriate training in time for the change over from WEPS to APPS later in the year-2018. It is important to note that the payroll unit plan is not the basis for the ‘twinning ‘plan. Rather the payroll unit plan provides for recruitment and on-the-job training of the relevant MOIA staff for payroll management. 4 of the 5 positions recommended for recruitment have been filled and the recruited staff are part of the twinning.

Key Actions:

12. Recommendation:

The Evaluation recommends that the SPM Project incorporate enhanced “anti-corruption” outputs and activities (and/or indicators) in its programming and that the MOIA take all necessary steps –per MOIA Donor Condition B12—to ensure that the MA findings pointing to possible fraud or corruption within the MOIA or ANP—specifically related to and involving the payroll process—are investigated by the Inspector General Office (IGO) and violators referred to the Attorney General of Afghanistan for prosecution. Any SPM outputs in this regard should be closely coordinated with MPD Project outputs and activities for the IGO to bolster MOIA internal audit capacity and capability and the capacity of the anti-corruption cell within the IGO.

Management Response: [Added: 2018/10/07] [Last Updated: 2021/01/13]

There is already an ongoing collaboration between the SPM project, the OIG and the MPD project [*Note, the MPD Project closed on 30 June 2018 and will be replaced by 3 projects; currently, the Transitional Support to the MOIA Project (TSM) is continuing support to the OIG] on corruption related findings on the payroll. SPM has already conducted some training for the civilian staff of the OIG on payroll business processes and key control/risk points. These auditors are now engaged in routine payroll auditing and often share findings with SPM. SPM has supported the MPD project in developing audit tools for the OIG and other capacity building programs. With support from MPD, an audit committee has been formed with H.E the Minister of Interior Affairs as chair and the IG as secretary. CSTC-A, and MOIA head of intelligence are members, Head of legal and UNDP/ LOTFA in advisory capacity. The purpose of this committee is to review findings of the OIG and support implementation of recommendations, including reference of criminal findings for to the Attorney General for prosecution.

SPM will continue to support the TSM Project in developing audit tools and building the capacity of the OIG to perform audit to international standards level. Furthermore, SPM will provide information to the OIG where corruption cases are suspected such as cases of M16s generated outside WEPS, or multiple payments to individuals for days exceeding 360 in a 12-month period.

Key Actions:

Key Action	Responsible	DueDate	Status	Comments	Documents
SPM shall continue to share MA’s monthly reports with OIG [Added: 2018/10/07] [Last Updated: 2018/12/09]	SPM Quality Assurance team	2018/12	Completed	Monthly MA reports will be shared with OIG- It is an ongoing activity, whereas the report is being submitted on monthly basis.

13. Recommendation:

The Evaluation recommends that the MOIA OIG and MA more closely cooperate and coordinate and that the LOTFA Board expand the scope of the MA’s work including possibly extending the duration of the MA’s time spent in each province.

Management Response: [Added: 2018/10/07] [Last Updated: 2021/01/13]

The current SPM project coordination scope of work is limited to sharing the final reports of the MA findings with the OIG office. The MA reports are shared with the office of the inspector general (OIG) and key findings of the report related to the OIG office are highlight for OIG follow up. UNDP would present this recommendation to SPM Technical Working Group to assess the scope of MA’s work including their duration of work in each province.

Key Actions:

Key Action	Responsible	DueDate	Status	Comments	Documents
Present the recommendation to SPM TWG to assess the MA’s scope of work and duration in each province [Added: 2018/10/07] [Last Updated: 2021/01/14]	SPM Project Management team.	2021/12	Not Initiated	n 2020, the Time and Attendance generated in APPS started to inform payroll generation in WEPS. As per preliminary discussion with MOIA, CSTC-A and LOTFA donors, the transition from WEPS to APPS may happen in the first half of 2021. The suggestion is to extend due date with 'Not Initiated' status until December 2021.

14. Recommendation:

The Evaluation recommends that UNDP and the LOTFA Board advocate with CSTC-A to accelerate the piloting of the APPS database to achieve its full deployment by mid-2018; including conducting required APPS orientation and training sessions for MOIA staff at central and provincial levels.

Management Response: [Added: 2018/10/07] [Last Updated: 2021/01/13]

CSTC-A cancelled the pilot testing of the APPS and plans to launch APPS in its full operational capability by the end of November 2018. UNDP has persuaded CSTC-A through its WEPS-APPS technical committee to provide training on APPS to SPM teams, which has thus far resulted in an orientation training on APPS. CSTC-A has informed LOTFA donors and UNDP that training of MOIA staff is ongoing.

Key Actions:

Key Action	Responsible	DueDate	Status	Comments	Documents
Establish and work through a technical committee with CSTC-A to proactively work on Fiduciary Risk and Management related to WEPS-APPS transfer including training of MOIA and UNPD staff. [Added: 2018/10/07]	ROLHSU	2018/04	Completed	The committee was established in April 2018 and has been working on WEPS-APPS transfer related issues.

15. Recommendation:

The Evaluation recommends that UNDP and the LOTFA Board advocate strongly with GIROA/MOIA that a comprehensive forensic audit of the MOIA (ANP) payroll system be completed within 6 months following full operability and deployment of the APPS system, but in any event, that such audit be completed by no later than 01 April 2019. It is envisioned that such an audit would be more comprehensive and wide-ranging than the typical “sampling” conducted by the MA in its monthly reports. This is viewed as necessary to establish a new “baseline” to inform LOTFA and GIROA decision-making and future SPM programming at the point in time at which the APPS system is made fully operational.

Management Response: [Added: 2018/10/07] [Last Updated: 2021/01/13]

This recommendation is well noted. UNDP will advocate with LOTFA project board to carry out a comprehensive forensic audit of MOIA payroll system after the transfer WEPS to APPS is completed. However, UNDP’s engagement would depend on how different the role of UNDP /SPM would be after the transfer of payroll processing from WEPS to APPS.

Key Actions:

Key Action	Responsible	DueDate	Status	Comments	Documents
Present the recommendation to SPM TWG to recommend to LOTFA project board to carry out a forensic audit of MOIA payroll system. [Added: 2018/10/07] [Last Updated: 2021/05/06]	ROLHSU	2021/04	Completed	this activity has become redundant as WEPS has now been replaced by APPS for the Afghan National Police Payroll processing.

16. Recommendation:

The Evaluation recommends that the SPM Project probe the reasons underlying ANP willingness (or lack thereof) to use MM platforms. It is recommended that, if necessary, the SPM Project institute a new round of trainings for Provincial ANP Commanders; PPQH; PFOs and PHROs with the objective of expanding the percentage of ANP paid through the Mobile Money mechanism and to become familiar with the PERSTAT template.

Management Response: [Added: 2018/10/07] [Last Updated: 2021/01/13]

Although the use of MM has not expanded to the extent expected, there has been expansion in EFT, which coupled with the transfer of ABP (which operate in the remote parts of the country, without banking facilities), has led to a decline in the use of the TA to 7% from 9%. The main challenges affecting the expansion of MM require a political solution by the MOIA leadership. Training on the use of PERSTAT forms were completed and there are no problems.

Key Actions:

17. Recommendation:

The Evaluation recommends that the MOIA facilitate continued access to the AHRIMS database to enable SPM Project staff to crosscheck AHRIMS records against data in WEPS to enhance the integrity of WEPS legacy data transferred to CSTC-A in connection with the development of APPS.

Management Response: [Added: 2018/10/07] [Last Updated: 2021/01/13]

Given the coming of APPS in November 2018 and the implementation of the Personnel Assets Inventory (PAI), the need to push for access to AHRIMS records has become redundant. The PAI enforcement is actually proving even more effective than having records of the AHRIMS system—which would not prove the existence of an ANP more than PAI does.

Key Actions:

Key Action	Responsible	DueDate	Status	Comments	Documents
SPM Staff able to access HR information from APPS [Added: 2018/10/07] [Last Updated: 2019/07/14]	ROLHSU	2019/06	Completed	his activity achieved by the time the database for APPS is reconciled against WEPS data.

18. Recommendation:

The Evaluation recommends that MOIA continue to cooperate with the SPM Project to implement the DRR system for WEPS. MOIA should give utmost priority to the DRR system establishment.

Management Response: [Added: 2018/10/07] [Last Updated: 2021/01/13]

The MOIA has closely cooperated with the SPM Project for this activity. SPM has now awarded the contract to implement the DRR which will be located at the MOIA Training General Command. SPM project manager has held a meeting with the Commander of the training general command and he has promised maximum cooperation and support to the process. There are no further issues related to this recommendation.

Key Actions:

19. Recommendation:

The Evaluation recommends that going forward, LOTFA-SPM Project begin to report to the LOTFA board on the cyber-security measures and policies surrounding the payroll process and implemented by MOIA and MOF surrounding the payroll process. This could include updates and analysis on how MOIA and MOF cyber security measures and policies protect the security of the payroll process and data; including reporting on MOF’s “vetting” of GIROA’s “up stream” counter-parts such as Da Afghanistan Bank and commercial correspondent banks through which payments are distributed. While this is currently not a function of the LOTFA-SPM project, cyber-security is a significant issue.

Management Response: [Added: 2018/10/07] [Last Updated: 2021/01/13]

We appreciate the importance of cyber security and the potential impacts it could pose to the payroll system. However, given the efforts of the SPM project in terms of implementing the DRR, coupled with the numerous security and application controls implemented by SPM project, we believe there is no current need for the project to be involved in cyber security. Notwithstanding the project shall remain vigilant and carefully monitor IT security issues.

Key Actions:

20. Recommendation:

The Evaluation recommends that UNDP and the SPM Project assess the manner and quality of engagement of SPM staff with MOIA senior “uniformed” officers and ensure that relations with MOIA uniformed staff are conducted with respect and courtesy.

Management Response: [Added: 2018/10/07] [Last Updated: 2021/01/13]

The SPM project notes this recommendation. It is important to note that since the submission of the draft evaluation report, SPM has been reviewing the basis of this allegation from the MOIA and found that it emanated from the established UNDP security protocols to enter SPM offices —having to be subjected to 100% check and not allowed to enter with weapons, etc. That is a standard UNDP security protocols which the project cannot compromise. Notwithstanding, the SPM project manager has met with all project staff and brought out this matter and advised everyone to show maximum respect to all MOIA staff regardless of uniformed or civilian.

Key Actions:

21. Recommendation:

The Evaluation recommends that the SPM Project educate ANP and their family members about the pay process; pay slips and entitlements via radio and TV broadcasts in addition to distributing printed materials.

Management Response: [Added: 2018/10/07] [Last Updated: 2021/01/13]

As highlighted during the evaluation, SPM attempted to carry out radio talks in the past but the quotations provided by the vendors were too expensive and therefore we had to abandon the activity. Notwithstanding, the SPM project has made some interventions including printed communication materials; notes produced into pocket-size note booklets which contain valuable information on salary and other related benefits and how Police can reach out to the SPM team in case of any discrepancies. This will be distributed across all 34 provinces.

Key Actions:

Key Action	Responsible	DueDate	Status	Comments	Documents
SPM Capacity Building team to distribute the sensitization materials across all 34 provinces, in coordination with MOIA logistics department and GDOP [Added: 2018/10/07] [Last Updated: 2018/12/09]	SPM Capacity Building team	2018/12	Completed	40,000 booklets have been printed and distributed to all 34 provinces.

Final Evaluation of the project "LOTFA Support to Payroll Management (SPM) and MoIA Police Development (MPD)

Share

Management Response: [Added: 2018/10/07] [Last Updated: 2021/01/13]

Key Actions:

Management Response: [Added: 2018/10/07] [Last Updated: 2021/01/13]

Key Actions:

Management Response: [Added: 2018/10/07] [Last Updated: 2021/01/13]

Key Actions:

Management Response: [Added: 2018/10/07] [Last Updated: 2021/01/13]

Key Actions:

Management Response: [Added: 2018/10/07] [Last Updated: 2021/01/13]

Key Actions:

Management Response: [Added: 2018/10/07] [Last Updated: 2021/01/13]

Key Actions:

Management Response: [Added: 2018/10/07] [Last Updated: 2021/01/13]

Key Actions:

Management Response: [Added: 2018/10/07] [Last Updated: 2021/01/13]

Key Actions:

Management Response: [Added: 2018/10/07] [Last Updated: 2021/01/13]

Key Actions:

Management Response: [Added: 2018/10/07] [Last Updated: 2021/01/13]

Key Actions:

Management Response: [Added: 2018/10/07] [Last Updated: 2021/01/13]

Key Actions:

Management Response: [Added: 2018/10/07] [Last Updated: 2021/01/13]

Key Actions:

Management Response: [Added: 2018/10/07] [Last Updated: 2021/01/13]

Key Actions:

Management Response: [Added: 2018/10/07] [Last Updated: 2021/01/13]

Key Actions:

Management Response: [Added: 2018/10/07] [Last Updated: 2021/01/13]

Key Actions:

Management Response: [Added: 2018/10/07] [Last Updated: 2021/01/13]

Key Actions:

Management Response: [Added: 2018/10/07] [Last Updated: 2021/01/13]

Key Actions:

Management Response: [Added: 2018/10/07] [Last Updated: 2021/01/13]

Key Actions:

Management Response: [Added: 2018/10/07] [Last Updated: 2021/01/13]

Key Actions:

Management Response: [Added: 2018/10/07] [Last Updated: 2021/01/13]

Key Actions:

Management Response: [Added: 2018/10/07] [Last Updated: 2021/01/13]

Key Actions:

Evaluations by Unit

Latest Evaluations

Evaluation Snap Shot

Contact us