- Evaluation Plan:
- 2016-2020, Turkey
- Evaluation Type:
- Final Project
- Planned End Date:
- 12/2020
- Completion Date:
- 12/2020
- Status:
- Completed
- Management Response:
- Yes
- Evaluation Budget(US $):
- 13,650
Final Evaluation for "Border Surveillance Capacity Between Turkey and the EU Phase II Project"
| Document | Type | Language | Size | Status | Downloads |
|---|---|---|---|---|---|
 Greek Border Phase II Terms of Reference_FINAL.pdf |
tor | English | 264.41 KB | Posted | 966 |
| Greek Border Phase II Final Evaluation Report_2020.12.29.pdf |
report | English | 936.68 KB | Posted | 964 |
| Title | Final Evaluation for "Border Surveillance Capacity Between Turkey and the EU Phase II Project" | ||||||
|---|---|---|---|---|---|---|---|
| Atlas Project Number: | 96326 | ||||||
| Evaluation Plan: | 2016-2020, Turkey | ||||||
| Evaluation Type: | Final Project | ||||||
| Status: | Completed | ||||||
| Completion Date: | 12/2020 | ||||||
| Planned End Date: | 12/2020 | ||||||
| Management Response: | Yes | ||||||
| Focus Area: |
|
||||||
| Corporate Outcome and Output (UNDP Strategic Plan 2018-2021) |
|
||||||
| Evaluation Budget(US $): | 13,650 | ||||||
| Source of Funding: | Project budget | ||||||
| Evaluation Expenditure(US $): | 10,000 | ||||||
| Joint Programme: | No | ||||||
| Joint Evaluation: | No | ||||||
| Evaluation Team members: |
|
||||||
| GEF Evaluation: | No | ||||||
| Key Stakeholders: | |||||||
| Countries: | TURKEY |
| Lessons | |
|---|---|
| Findings |
| Recommendations | |
|---|---|
| 1 | UNDP should build on its work and continue to support constructive, capacity building trainings that supports the development of effective and efficient IBM. Important in this work, is evaluating the outcomes of the distance learning packages. |
| 2 | Future project designs should include an evaluator or implementation scientist to facilitate the participatory development of a ToC (the recommendation to include an evaluator in the Project design was also made in the Phase 1 evaluation) This will also help the team check its assumptions about the underlying theory of how the Action will “work” to create change and enable the development of realistic, effective and relevant indicators that can be monitored. A ToC could also help develop deeper insights into how the Action can contribute to gender equality, human rights and higher-level objectives. |
| 3 | Future projects should consider scaling up the distance learning (based on a needs assessment) as well as blended trainings and introduce simulator and other modern training technologies for law enforcement agencies. This will also support inter-agency learning and further increase efficiency of resources. |
| 4 | Current learning objectives as expressed in the Training Strategy document (also refer to Annex 6) are relatively vague and do not clearly spell out knowledge and cognitive dimensions of learning making evaluation and measurement against learning objectives problematic (this may be addressed in individual modules). Its recommended to be in line with best practice (e.g., remember, understand, apply, analyse and create). |
| 5 | In designing future border capacity building initiatives, UNDP needs to engage constructively with gender mainstreaming and work with a gender specialist to undertake a Gender analysis and develop SMART gender sensitive indicators that can be monitored. Proposed actions in the Project Gender Profile should be considered in any future phase. |
| 6 | Providing independent evaluators access to trainees and migrants apprehended at the borders is not realistic due to national security concerns. UNDP could however, support LFC develop its own specific indicators to gauge behavioural change in trainees and report on results periodically. |
| 7 | UNDP should leverage the work undertaken in Phases 1 and 2 to develop Standard Operating Procedures to embed change into practice and engage in deeper policy discussion to achieve EU goals, norms and standards in IBM. This will also assist in contributing to broader security governance objectives. |
| 8 | In a third Phase, appointment of more female officers for LFC operational needs could be included, possibly starting at the western borders with results disseminated to other borders. |
Management Response Documents
- Management Response Greek Border Phase II_2021.01.26_FINAL.pdf Managemen Response Greek Border Phase II
1. Recommendation:
UNDP should build on its work and continue to support constructive, capacity building trainings that supports the development of effective and efficient IBM. Important in this work, is evaluating the outcomes of the distance learning packages.
Management Response: [Added: 2021/01/21] [Last Updated: 2021/01/22]
Recommendation is accepted and below key actions are proposed as a response.
Key Actions:
| Key Action | Responsible | DueDate | Status | Comments | Documents |
|---|---|---|---|---|---|
| UNDP has proposed “Border Phase III Training Project” to EU to be funded under IPA III in order to support constructive, capacity building training that supports the development of effective and efficient IBM. Proposed project aims to scale up the distance learning as well as to use blended trainings. Tactical and operational training of border professionals is proposed to be organized in the training center to be equipped with high technology training tools (such as simulators, virtual reality and artificial intelligence).
[Added: 2021/01/22] |
UNDP, Ministry of Interior | 2020/05 | Completed | Selection of pipeline projects are ongoing for IPA III. | |
| The outcomes of the distance learning packages were evaluated by an external independent statistician which resulted in objective and plausible evaluation of trainings. This evaluation will give EDOK a clear direction for the revisions and update of the curriculum prepared. In addition, it will enable clear directions for distance learning training in practice and elaboration of training curricula.
[Added: 2021/01/22] [Last Updated: 2021/03/31] |
UNDP | 2021/03 | Completed | Key action is completed. Statistical evaluation was officially shared with Project Beneficiaries with the Distance Learning Training Report on 1 March 2021 History |
2. Recommendation:
Future project designs should include an evaluator or implementation scientist to facilitate the participatory development of a ToC (the recommendation to include an evaluator in the Project design was also made in the Phase 1 evaluation) This will also help the team check its assumptions about the underlying theory of how the Action will “work” to create change and enable the development of realistic, effective and relevant indicators that can be monitored. A ToC could also help develop deeper insights into how the Action can contribute to gender equality, human rights and higher-level objectives.
Management Response: [Added: 2021/01/21] [Last Updated: 2021/01/25]
Recommendation is accepted and below key actions are proposed as a response.
Key Actions:
| Key Action | Responsible | DueDate | Status | Comments | Documents |
|---|---|---|---|---|---|
| In a subsequent phase, the newly appointed M&E Analyst will work closely with the Project team to develop a sound Theory of Change for the project and to come up with a realistic and measurable logical framework that differentiates across outcome and output levels and that allows for measuring institutional capacity increases at higher levels.
[Added: 2021/01/25] [Last Updated: 2022/06/02] |
M&E Analyst IDG Portfolio Manager, Project team | 2022/06 | Completed | M&E Analyst has prepared a A FRAMEWORK FOR IMPLEMENTING PRE-POST TESTS FOR PROFESSIONAL GROUPS (I.E STAFF OF MINISTRIES, LAWYERS, TEACHERS) to measure capacity increases. History |
3. Recommendation:
Future projects should consider scaling up the distance learning (based on a needs assessment) as well as blended trainings and introduce simulator and other modern training technologies for law enforcement agencies. This will also support inter-agency learning and further increase efficiency of resources.
Management Response: [Added: 2021/01/21] [Last Updated: 2021/01/25]
Recommendation is accepted and below key actions are proposed as a response.
Key Actions:
| Key Action | Responsible | DueDate | Status | Comments | Documents |
|---|---|---|---|---|---|
| UNDP has proposed “Border Phase III Training Project” to EU to be funded under IPA III. Proposed project aims to scale up the distance learning as well as to use blended trainings. Tactical and operational training of border professionals is proposed to be organized in the training center to be equipped with high technology training tools (such as simulators, virtual reality and artificial intelligence). This model can be extended to other law enforcement agencies in Turkey and EU if funded. This training concept is based on practical continuous-in service method where daily proceedings evaluated by trainees themselves through peer review concept and moderated by instructors. This is in line with one of the most conducive learning environments, particularly useful in Law Enforcement practice, where participants can indicate, evaluate and correct potential gaps in daily LFC proceedings.
[Added: 2021/01/22] |
UNDP, Ministry of Inrerior | 2020/05 | Completed | Selection of pipeline projects are ongoing for IPA III. |
4. Recommendation:
Current learning objectives as expressed in the Training Strategy document (also refer to Annex 6) are relatively vague and do not clearly spell out knowledge and cognitive dimensions of learning making evaluation and measurement against learning objectives problematic (this may be addressed in individual modules). Its recommended to be in line with best practice (e.g., remember, understand, apply, analyse and create).
Management Response: [Added: 2021/01/21] [Last Updated: 2021/01/22]
This recommendation is rejected. In each individual module (IBM, Migrants’ Rights and COVID-19) learning objectives were clearly expressed by using “ADDIE” method, as expressed in the Needs Assessment Report and Training Strategy. ADDIE stands for Analysis, Design, Development, Implementation and Evaluation to guide the training programs. This model is selected with the aim of formulating high-quality distance learning training for the border professionals and following steps were followed in the design of the distance learning trainings. Distance Learning Center of LFC has incorporated ADDIE method in implementing a full-fledged distance learning training and this approach has been adopted by EDOK from then on. As a positive influence of the project, EDOK has started developing materials abiding by ADDIE technique/approach.
Key Actions:
5. Recommendation:
In designing future border capacity building initiatives, UNDP needs to engage constructively with gender mainstreaming and work with a gender specialist to undertake a Gender analysis and develop SMART gender sensitive indicators that can be monitored. Proposed actions in the Project Gender Profile should be considered in any future phase.
Management Response: [Added: 2021/01/21] [Last Updated: 2021/01/25]
Recommendation is accepted and below key actions are proposed as a response.
Key Actions:
| Key Action | Responsible | DueDate | Status | Comments | Documents |
|---|---|---|---|---|---|
| For the future border capacity building initiatives, constructive engagement of gender specialist will be ensured. Engagement of female border professionals, possibly starting at the western borders will be advocated and training components will be designed with gender equality perspective.
[Added: 2021/01/25] [Last Updated: 2022/06/02] |
UNDP (Programme) | 2022/06 | Completed | Another border management project De-Mining Phase III adopted this recommendation and completed the following activities: Right from the design stage, the intervention has deployed a gender-sensitive approach. The main targets identified to that end have been to strengthen the gender equality perspectives of the couterparts and service providers and to design and conduct a women-inclusive practice in the field. UNDP has started delivering two sets of common understanding sessions to the key TURMAC staff on the concept of gender equality and the gender-sensitive field work. Special modules on gender-sensitive field practice was planned to be included in the trainer’s training programmes targeting the CSO representatives, designated to deliver MRE trainings in the field. CSO trainer teams are planned to be gender-balanced in order to facilitate project outreach to women within the MRE training programmes and to showcase the gender parity principle invested in the project. In the villages, separate trainings will be organized for women and men to ensure full and active participation of women to the programmes and women-only trainings will be conducted by female trainers and in the places considered most convenient by the female participants, including homes and common community spaces. History |
6. Recommendation:
Providing independent evaluators access to trainees and migrants apprehended at the borders is not realistic due to national security concerns. UNDP could however, support LFC develop its own specific indicators to gauge behavioural change in trainees and report on results periodically.
Management Response: [Added: 2021/01/21] [Last Updated: 2021/01/25]
UNDP already provided comprehensive support to LFC command in Greek Border Phase I in measuring behavioural change in trainees. This type of work could not be repeated in Phase II because it requires field study which could not be repeated due to COVID-19 measures, however the beneficiaries observed the organization and conduct of this activity during Phase-I and they can further develop internal procedures to gauge behavioural change in trainees and report on results periodically. Hence this recommendation has already been addressed by UNDP.
Key Actions:
| Key Action | Responsible | DueDate | Status | Comments | Documents |
|---|---|---|---|---|---|
| Precondition for annual regular evaluation of training and its impact on the LFC proceedings, at the green borders, is to establish regular continuous in-service training. Once training is implemented in the LFC, behavioural changes of officers operating at the green borders, can be assessed. This assessment should correlate training participation (regular/irregular) with the quality of daily operational IBM and HR proceedings. UNDP can provide impartial expertise and develop methodology for future in-service continuous LFC training and its impact on quality of IBM and HR daily proceedings at the green borders. Hence, regular evaluation based on scientific plausible conclusions can increase quality of LFC operational, tactical and managerial proceedings, which can be measured. Under Border Phase I Project, UNDP had shared its know-how with LFC on how to collect data to observe and evaluate the behavioural change in trainees with the field study mission conducted. Through the interviews, data about the perception of the training participants on the relevance, effectiveness and efficiency of the trainings was collected and analysed. Effectiveness of the basic level trainings were measured and the benefit in the daily work of trainees was assessed. Detailed study methodology and set of questions was designed in agreement with the project beneficiaries and a report which summarized the remarks of the trainees was prepared. The outcome of the field study also guided the parties during design and conduct of trainings under Phase II Project.
[Added: 2021/01/25] |
UNDP | 2020/12 | Completed | UNDP has already addressed this recommendation. History |
7. Recommendation:
UNDP should leverage the work undertaken in Phases 1 and 2 to develop Standard Operating Procedures to embed change into practice and engage in deeper policy discussion to achieve EU goals, norms and standards in IBM. This will also assist in contributing to broader security governance objectives.
Management Response: [Added: 2021/01/21] [Last Updated: 2021/01/25]
In the Consultative Meetings organized under Phase 1 and 2 Projects, need for reviewing LFC legislation and operational proceedings was indicated. Nevertheless, for plausible conclusions, in-depth needs assessment of basic border training curriculum, legislation and current SOPs would be requested. Accordingly, consent for in-depth evaluation would be needed from the MoI and LFC. In a subsequent phase, SOP in the form of Handbook can be developed and serve for managerial, tactical and operational needs of LFC. The content from the SOP would be reflected in the basic and continuous in-service training curriculum. SOP would be drafted on an easy but comprehensible manner, which can be understandable for every LFC officer. It would offer clearer step-by-step guidance and lead LFC officers through different daily IBM procedures. This SOP would consider basic Law Enforcement principles; necessity and proportionality and assist LFC officers to increase quality of performance in terms of legality, humanity and their self-confidence. Due to sensitive and confidential nature of internal SOP’s, this extensive work can be done under the flagship of IBM Action Plan which is expected to be conducted by EU. Besides national partners (NACORAC) must agree on receiving external assistance to develop SOPs. This recommendation is therefore partially accepted since it requires preconditions to be accepted by national partners:
Key Actions:
| Key Action | Responsible | DueDate | Status | Comments | Documents |
|---|---|---|---|---|---|
| Upon consent of the national partners, as a part of the IBM action Plan or NACORAC request, UNDP is ready to provide assistance in developing SOPs to embed change into practice and engage in deeper policy discussion to achieve EU goals, norms and standards in IBM. UNDP Country Office would follow the possible project opportunities for providing assistance in developing SOPs and to enhance the partnership with the stakeholders.
[Added: 2021/01/25] [Last Updated: 2022/06/02] |
UNDP (Programmes) | 2022/06 | No Longer Applicable |
[Justification: No new opportunity on IBM has been identified in the last two years. Therefore this recommendation is no longer applicable.] History |
8. Recommendation:
In a third Phase, appointment of more female officers for LFC operational needs could be included, possibly starting at the western borders with results disseminated to other borders.
Management Response: [Added: 2021/01/21] [Last Updated: 2021/01/25]
Recommendation is partially accepted since UNDP does not have full leverage over LFC on appointing female border professionals. Below key actions are proposed as a response.
Key Actions:
| Key Action | Responsible | DueDate | Status | Comments | Documents |
|---|---|---|---|---|---|
| Appointment of more female officers for LFC operational needs possibly starting at the western borders with results disseminated to other borders will be advocated in the Final Report.
[Added: 2021/01/25] [Last Updated: 2021/06/17] |
UNDP (Programmes) | 2021/06 | Completed | The Final Report mentions the necessity of appointment of more female officers on page 18/50/54 and 60. The report is attached for your review. History | |
| For the future border capacity building initiatives, constructive engagement of gender specialist will be ensured. Engagement of female border professionals, possibly starting at the western borders will be advocated and training components will be designed with gender equality perspective.
[Added: 2021/01/25] [Last Updated: 2022/06/02] |
UNDP (Programmes) | 2022/06 | Completed | The following measures for taken in another border management project (de-mining Phase III): Right from the design stage, the intervention has deployed a gender-sensitive approach. The main targets identified to that end have been to strengthen the gender equality perspectives of the couterparts and service providers and to design and conduct a women-inclusive practice in the field. UNDP has started delivering two sets of common understanding sessions to the key TURMAC staff on the concept of gender equality and the gender-sensitive field work. Special modules on gender-sensitive field practice was planned to be included in the trainer’s training programmes targeting the CSO representatives, designated to deliver MRE trainings in the field. CSO trainer teams are planned to be gender-balanced in order to facilitate project outreach to women within the MRE training programmes and to showcase the gender parity principle invested in the project. In the villages, separate trainings will be organized for women and men to ensure full and active participation of women to the programmes and women-only trainings will be conducted by female trainers and in the places considered most convenient by the female participants, including homes and common community spaces. History |