Increasing Border Surveillance Capacity of Borders between Turkey and Greece

Report Cover Image
Evaluation Plan:
2016-2020, Turkey
Evaluation Type:
Final Project
Planned End Date:
02/2019
Completion Date:
02/2019
Status:
Completed
Management Response:
Yes
Evaluation Budget(US $):
7,300

Share

Document Type Language Size Status Downloads
Download document TOR Greek Border I.pdf tor English 697.82 KB Posted 1084
Download document Evaluation Report of Greek Border.pdf report English 1574.56 KB Posted 1219
Title Increasing Border Surveillance Capacity of Borders between Turkey and Greece
Atlas Project Number: 00097937
Evaluation Plan: 2016-2020, Turkey
Evaluation Type: Final Project
Status: Completed
Completion Date: 02/2019
Planned End Date: 02/2019
Management Response: Yes
Focus Area:
  • 1. Others
Corporate Outcome and Output (UNDP Strategic Plan 2018-2021)
  • 1. Output 3.2.1 National capacities strengthened for reintegration, reconciliation, peaceful management of conflict and prevention of violent extremism in response to national policies and priorities
SDG Goal
  • Goal 16. Promote peaceful and inclusive societies for sustainable development, provide access to justice for all and build effective, accountable and inclusive institutions at all levels
SDG Target
  • 16.6 Develop effective, accountable and transparent institutions at all levels
Evaluation Budget(US $): 7,300
Source of Funding: Project budget
Evaluation Expenditure(US $): 7,300
Joint Programme: No
Joint Evaluation: No
Evaluation Team members:
Name Title Nationality
Mr. Kjell Elefalk Evaluator
GEF Evaluation: No
Key Stakeholders:
Countries: TURKEY
Lessons
Findings
1.

7. Findings

The report is short and concise on findings because the opinions of virtually all concerned are relatively unanimous. This state sets out the assessment of key results that underpin the findings.


Tag: Effectiveness Relevance Migration Security Capacity Building

2.

In order to provide an appropriate clarity and overview in the analysis, the findings will be divided into the following parameters and subject areas below;

  •  Strategic Positioning, Concept and Design of the project

Strategic positioning is concerned with the way in which a project differentiates itself in a valuable way and delivers value to specific beneficiaries as MOI/LFC. Concepts are solutions to unsolved problems or new solutions to problems that are solved in a poor manner. A new concept can be a product, a service, or a combination of products and services. Design is a concept that describes the design, appearance or creation process of an object that is used from the idea stage to the finished product. Design is applied for the development of goods, services, processes and environments.


Tag: Relevance Programme/Project Design Strategic Positioning

3.
  •  Monitoring, Evaluation and Risk Management

Monitoring is the act of continuously checking on the progress being made in an activity against set results and targets in order to determine whether activity implementation is on course.

Systems for monitoring should be demand-driven and based on analyses of the actual needs of support for decision-making and control. That in turn is dependent on the situation, the character of the activity, etc.

Evaluation is a time-bound exercise that attempts to assess systematically and objectively the impact, relevance, performance and success of ongoing and completed programmes, projects and management initiatives.


Tag: Effectiveness Monitoring and Evaluation Project and Programme management Risk Management

4.
  •  Rights based approach and Gender mainstreaming:

Rights based approach is an approach to development promoted to achieve a positive transformation of power relations among the various actors. There are two stakeholder groups in rights-based development—the rights holders who do not experience full rights and the institutions obligated to fulfil the holders' rights.


Tag: Relevance Gender Equality Human rights Security

5.
  •  Relevance of the project:

Is the intervention consistent with the needs and priorities of its target group and the policies of the partner country and donor agencies?

There are no indications that the project has in any way deviated from the beneficiaries' goals, the donor's goals or the main task of increasing the LFC's IBM capabilities. There is reason to believe that the possibilities of introducing new technology have increased and that HR capacity has been strengthened. The foundation is laid in the way the beneficiaries and donors imagined.


Tag: Relevance Human rights Donor

6.
  • Effectiveness of the project;

Has the intervention achieved its objectives or will it do so in the future?

It is always complicated to assess goal fulfilment. The most appropriate answer for assessing the objectives is that the goals serve as the exclusive source of standards and criteria on this issue.

There is a consensus in the interviews among those concerned that the project met in a good way the beneficiaries’ objective, the donor's goals and the main goal of increasing the LFC's IBM capabilities.

There are also clear indications, in addition to phase 2, that the beneficiaries will, over time, continue to carry out improvements by IBM, though it would be better with long-term support from the donors.

In view of the conducted interviews and the questionnaire responses received, in accordance with the ToR grading scale, the goal fulfilment can only be assessed as "Highly satisfactory".


Tag: Effectiveness Capacity Building Institutional Strengthening National Institutions

7.
  •  Efficiency of the project:

Can the costs of the intervention be justified by the results?

The issue of cost efficiency is very difficult in general in development projects, and even more so when many benefits can lie in the future as in the case of IBM. Once this has been established, there is reason to recall the very positive opinions that the stakeholders of various kinds have given. So, no reason to say this project has become too expensive in relation to the outcome.


Tag: Efficiency Operational Efficiency Project and Programme management

8.
  •  Impact of the project:

What are the overall effects of the intervention, intended and unintended, long term and short term, positive and negative?

There is a consensus among all stakeholders that significant knowledge of the LFC to the necessity to realize its responsibilities to adopt modern technological tools for IBM. Also increased human resource capacity having the know-how on border management procedures and fundamental rights on migrants, international protection and combating human trafficking.

It was difficult to perceive among the stakeholders that the project had some unintentional or negative effects that were very clear now or would become apparent in the future. This ambiguity is also congruent with the very high CSI grades with few criticisms received by the project.


Tag: Impact Human rights Human and Financial resources Project and Programme management Migration Security Refugee Vulnerable

9.
  •  Partnership and coordination:

To what extent did the targeted population make the programme their own, taking an active role in it?

How effective has the UNDP project been in coordinating the communication among project partners and what have contributed to the effectiveness/ineffectiveness? The beneficiaries have unanimously stated that it has taken full responsibility for the project, since they always considered the project to be very important for developing IBM. This has in all ways been verified in interview form, questionnaire form and in the documents of the other stakeholders. In addition, the project's coordination work has received approved reviews from the stakeholders.


Tag: Effectiveness Communication Bilateral partners Coordination

10.
  •  Sustainability;

Will the benefits produced by the intervention be maintained after the cessation of external support?

The representatives of LFC and MoI believe that IBM and the facts about the borders of Turkey are such that the core business of the project will continue even if the donors do not continue. Phase 2 of the project is also fully prepared.


Tag: Sustainability Rule of law Security

11.
  •  Specific desk study of the evaluation of the effectiveness of training The scope of the pre and post training evaluation is to measure effectiveness and sufficiency of training provided to Land Forces Command responsible for “green” border surveillance between border crossing points at Turkey-Greece border.

Desk studies of the evaluation of the effectiveness of training was conducted by Data Scientist Mr.Oguzhan Akyildirim using average comparison before and after training. Score per participant for pre-training and post-training were calculated by summing up the accurate answers.


Tag: Effectiveness Migration Security Capacity Building Institutional Strengthening National Institutions

12.

8. Survey - Customer Satisfaction Index Study

Customer Satisfaction Index (CSI) is a system to collect, analyze and disseminate data about image, preferences and perceived quality as well as loyalty of customers, employees and other stakeholders to commercial entities, NGOs, project performance, governmental bodies and police organizations. The CSI approach focuses on analysis derived from structural model elaboration and thorough empirical studies in order to estimate numerical relationships.


Tag: Relevance Monitoring and Evaluation Project and Programme management Civil Societies and NGOs

13.

To divide the groups into more segments is not meaningful, but specific needs is met in faceto-face interviews and in desk studies.

The respondents indicate their satisfaction on the scale 1 to 10. The 1-10 scale was for ease of interpretation transformed into a 1-100 scale. The higher the score, the better judgment the project has received from its stakeholders. Over 75 are considered very good. Lower than 60 is considered a weak rating, which is not the project is close by.


Tag: Effectiveness Relevance Migration Security Private Sector

14.

8.1 CSI - Analytical definitions;

Customer Satisfaction Index (CSI): The overall Customer Satisfaction Index/the tangible Quality Index possible to compare with other businesses and project.

Image: The Respondent´s supposed existing picture of the PROJECT in the environment as assumed general view of the PROJECT in general. (Not the respondent’s own assessment of the PROJECT’s image)

Expectations: The Respondent’s own expectations reciprocally on the processing of the actual core business and the initial personally service delivered by the PROJECT.
 


Tag: Effectiveness Impact Monitoring and Evaluation

15.

8.2 Estimation of CSI results - Analysis of Data

1. Stakeholders total

The analysis of the response from all the stakeholders showed the following;

  •  The overall Quality Index (83) is very high and would be considered as excellent and at required level for international appreciated CSI reviews. This is an assessment from the individuals who work for the stakeholders. Despite this complexed relationship, their assessment must be of great validity with the coherence between the survey response, face-to-face interviews and desk studies. Very high CSI values like 83 in this study do not automatically mean that the project performed extremely well. However, it can be stated that those representatives of the stakeholders who have participated closely in the work are very satisfied with the project's results.

Tag: Effectiveness Impact Bilateral partners

16.

2. Beneficiaries

The analysis of the response from the beneficiaries showed the following;

  •  The overall Quality Index (82) is very high and would be considered as excellent and at required high level for international commendable CSI reviews. This is an assessment from the individuals who work for the beneficiaries. Despite this complexed relationship, their assessment must be of great validity with the coherence between the survey response, face-to-face interviews and desk studies. Very high CSI values like 82 do not automatically mean that the project worked extremely well. However, it can be stated that those who have followed closely the work of the project are very satisfied with the project's results.

Tag: Effectiveness Impact Refugee Vulnerable

17.

3. Other stakeholder individuals

The analysis of the response from the other stakeholder individuals showed the following;

  •  The overall Quality Index (84) is very high and would be considered as excellent and at required high level for international praiseworthy CSI reviews. However, this is an assessment from the committed individuals in the project environment. Despite this relationship, their assessment must be of great validity with the coherence between the survey response, face-to-face interviews and desk studies. Very high values like 84 need not mean that the project work is extremely good, but certainly that the project worked more than adequate and generated useful results.

Tag: Effectiveness Impact Bilateral partners

18.

4. Project staff - self-assessment

The analysis of the response from the project staff showed the following;

  •  The overall Quality Index (82) is very high and would be considered as excellent and at required laudable level for international CSI reviews. However, this is a selfassessment from the relevant project employees who have an unintended interest in putting high scores (risk for bias). However, the bias and the results can be assessed with a comparison of the other categories' values. These three groups reported 83, 84 and 82. That might indicate that the staff is as close as the stakeholders to fairness when it comes to objectivity.

Tag: Efficiency Impact Human and Financial resources Project and Programme management

Recommendations
1

There is reason to believe that the possibilities of introducing new technology have increased and HR capacity has been strengthened. The foundation is laid in the way the beneficiaries and donors imagined. It would be very unfortunate, and waste of funds, if enough resources were not decided to utilize for phase 2 and the future work.

2

Many good assessments of the project's work have a good basis in that it is supported from the very beginning of beneficiaries, that the goals were clear and that there were two components – both unpretentious and doable, enabled rather qualitative than quantitative delivery of the outcomes. There is every reason to emphasize that the structure of the future work should be similar.

3

An important lesson that has been developed in the course of the Phase I is that the next phase should be characterized by distance (practical) training to optimize the lessons of the first phase.

4

When it comes to evaluating the next phase and future work, it should be emphasized that at the start of the project, the evaluation should be prepared by an evaluation expert with an independent position.

5

There are observed weaknesses of gender mainstreaming in executing of the project. The main reason is the lack of female officers and soldiers. Institutional measures need to be broadened in order to better anchor gender mainstreaming in the long term.

6

The gender targets need to be reflected in formal and informal monitoring and evaluation (M&E), at both institutional and intervention level in the next phase

1. Recommendation:

There is reason to believe that the possibilities of introducing new technology have increased and HR capacity has been strengthened. The foundation is laid in the way the beneficiaries and donors imagined. It would be very unfortunate, and waste of funds, if enough resources were not decided to utilize for phase 2 and the future work.

Management Response: [Added: 2019/06/27] [Last Updated: 2021/02/04]

UNDP had already suggested to include the use of the output of the project, namely Feasibility Report, for the use of high technology training tools, at hand to result in establishment of Distance Learning Technology as a part of Phase II. In this respect Distance Learning Infrastructure will be enhanced in Land Forces Training and Doctrines Command, who has a high level of ownership as also mentioned in the Feasibility Report of the Simulation Based Training Center Moreover, UNDP prepared a concept note for a further possible pipeline project to include more high technology environment to be set up in Land Forces Training and Doctrines Command. This possible pipeline project has been shared with the Main Beneficiary. UNDP will follow up this and further possibilities.

Key Actions:

Key Action Responsible DueDate Status Comments Documents
Component II of IBM Phase II Project on Institutional Capacity Building that includes enhancement of Distance Learning Infrastructure Follow-up of possible pipeline project/s with EU for the use of high technology training tools.
[Added: 2019/06/27] [Last Updated: 2020/08/21]
IDG Portfolio 2020/08 Completed As recommended, Component II of the Phase II Project was designed to enhance distance learning infrastructure of Land Forces Command - Training and Doctrines Command and the project started in December 2018. So far, majority of the activities are completed. Distance learning training modules are prepared and organization of distance learning pilot trainings will follow. The project will be completed in December 2020. History
Component II of IBM Phase II Project on Institutional Capacity Building that includes enhancement of Distance Learning Infrastructure Follow-up of possible pipeline project/s with EU for the use of high technology training tools.
[Added: 2019/06/27] [Last Updated: 2020/08/21]
IDG Portfolio 2020/08 Completed As recommended, Component II of the Phase II Project was designed to enhance distance learning infrastructure of Land Forces Command - Training and Doctrines Command and the project started in December 2018. So far, majority of the activities are completed. Distance learning training modules have been finalized and organization of distance learning pilot trainings will follow. The project will be completed in December 2020. History
2. Recommendation:

Many good assessments of the project's work have a good basis in that it is supported from the very beginning of beneficiaries, that the goals were clear and that there were two components – both unpretentious and doable, enabled rather qualitative than quantitative delivery of the outcomes. There is every reason to emphasize that the structure of the future work should be similar.

Management Response: [Added: 2019/06/27] [Last Updated: 2021/02/04]

In terms of quality, UNDP has produced a sustainable and replicable module that will be utilized in the second Phase. However, there was also a good balance between qualitative and quantitative targets/outcomes in the Project. In terms of quantitative results, the Project surpassed its own targets such as the number of trainees, number of study visits. In the second phase, this balance between qualitative and quantitative targets will continue. While Phase II has targets related to numbers, it also has qualitative targets such as integration of learning materials in Land Force’s curriculum.

Key Actions:

Key Action Responsible DueDate Status Comments Documents
Close cooperation with Beneficiaries both in planning and implementation of activities. Integration of Lessons Learned from Phase I into Phase II Continuation of the delivery of trainings with minor revisions Sustainable use of Distance Learning Materials as part of the education system of Land Forces
[Added: 2019/06/27] [Last Updated: 2020/08/21]
IDG Portfolio 2020/08 Completed As referred in the Project Document, Phase II Project was developed in the light of lessons learned from the Phase I Project in terms of capacity building of border professionals. As such, most of the activities build on and/or complement the activities that have been realized within Phase-I and increase the impact. efficient design of the activities. The piloting of distance learning trainings in Phase II is found to be crucial and innovative to tackle challenges identified in Greek Border Phase I. Based on the lessons learned from Phase – I Project, Phase II Project was initiated in December 2018 and will be completed in December 2020. History
3. Recommendation:

An important lesson that has been developed in the course of the Phase I is that the next phase should be characterized by distance (practical) training to optimize the lessons of the first phase.

Management Response: [Added: 2019/06/27] [Last Updated: 2021/02/04]

A Feasibility Report has been produced in Phase I of the Project. This report summarizes the necessity for face to face, distance and blended learning besides the use of high technology training tools depending on the peculiarities of the training such as the profile of the trainees and the availability of facilities. In this respect, for being consistent with the Feasibility Report, face to face training method will remain in the curriculum with minor revisions to the training material. In addition, as already argued in the Recommendation I, the distance learning component is added to Phase II with a modified material.

Key Actions:

Key Action Responsible DueDate Status Comments Documents
Lessons learned to be reviewed. Curriculum to be reviewed. Training materials to be revised and finalized upon review before and after trainings.
[Added: 2019/06/27] [Last Updated: 2020/08/21]
IDG Portfolio 2020/08 Completed As referred in the Project Document, the curriculum of face to face training modules (Component 1) were prepared and agreed on the basis of findings, conclusions and recommendations from the Phase-I Project, suggestions from candidate and selected trainers, academicians and the project beneficiaries. Changes in the legislation, practices or organizational structure of the subject institutions were reflected. The training modules were revised and finalized after delivery of trainings (which were delivered in November -December 2019) based on the feedbacks received from the trainees, entry/ exit test results which was analysed by a statistician and last review by the trainers. The curriculum of distance learning training modules (Component 2) were prepared on IBM and Migrants rights based on the Distance Learning Training Needs Assessment Report and Training Strategy. History
4. Recommendation:

When it comes to evaluating the next phase and future work, it should be emphasized that at the start of the project, the evaluation should be prepared by an evaluation expert with an independent position.

Management Response: [Added: 2019/06/27] [Last Updated: 2021/02/04]

Under Phase II, another evaluation is going to be conducted at the end of the project which is a common practice in EU Projects. The proposed evaluation suggestions are already included in the activities of the Phase II Project including an evaluation to be conducted at the end of the project. Additionally, all UNDP evaluations are conducted according to UNDP Evaluation Guidelines under the supervision of M&E Analyst which guarantees an independent review of projects.

Key Actions:

Key Action Responsible DueDate Status Comments Documents
ToR of the Phase II independent evaluation
[Added: 2019/06/27] [Last Updated: 2020/08/21]
IDG Portfolio 2020/08 Completed In August 2020, ToR of the Phase II independent evaluation was finalized under the supervision of M&E Analyst and norms and standards set out in the UNDP Evaluation Guidelines were duly considered in the design. ToR is now under the review of Evaluation Reference Group and the vacancy will be announced by September 2020. Following the procurement process, independent evaluation mission will be initiated by mid-October 2020 and completed by December 2020 in accordance with the project action plan. History
5. Recommendation:

There are observed weaknesses of gender mainstreaming in executing of the project. The main reason is the lack of female officers and soldiers. Institutional measures need to be broadened in order to better anchor gender mainstreaming in the long term.

Management Response: [Added: 2019/06/27] [Last Updated: 2021/02/04]

Due to very difficult geographical and security conditions, the number of female staff for IBM remains limited. However, in the future, if the border service is civilianized and the security besides geographical conditions are improved then there can be a visible increase in the number of female border professionals. UNDP has and will report the lack of female professionals in progress reports to highlight this issue to be ready for a change if the conditions improve.

Key Actions:

Key Action Responsible DueDate Status Comments Documents
Notification of the lack of female border professionals in the Progress Reports.
[Added: 2019/06/27] [Last Updated: 2020/08/21]
IDG Portfolio 2020/01 Completed Even though there is no specific gender related activity foreseen in the Project Document, gender dimension was extended in all project activities; particularly in the training programs. In order to ensure gender lenses in the design and conduct of face to face and distance learning training programs, core team of LFC EDOK was established with 2/4 women officers. Lack of female border professionals and its root causes are recognized by all parties. UNDP stated its willingness to cooperate on this matter through bilateral meetings. Both Phase-I and Phase II projects contributed to awareness raising in management staff and among border professionals in terms of gender equality and the importance of assigning women border professionals. However, frequent rotation of lead managers in the project beneficiaries create a challenge in accomplishing this action. In the Progress Report of Phase II project, it was stated that best practices will be highlighted and gender related questions will be designed and directed to border guards in distance learning pilot training evaluation forms to understand their awareness and needs with respect to gender issues during their border duty. Lack of female professionals and possible action areas for a change will be highlighted once again in the Final Report which is due by 06/2021. History
6. Recommendation:

The gender targets need to be reflected in formal and informal monitoring and evaluation (M&E), at both institutional and intervention level in the next phase

Management Response: [Added: 2019/06/27] [Last Updated: 2021/02/04]

The bulk of Phase II activities are trainings to Land Forces Professionals. In line with gender mainstreaming standards, the trainings of Phase II are gender sensitive such as in training the professionals on body search etc..

Key Actions:

Key Action Responsible DueDate Status Comments Documents
Enhancing gender sensitive aspects of training modules
[Added: 2019/06/27] [Last Updated: 2020/08/21]
IDG Portfolio 2020/08 Completed Gender equality is considered in training material content of face to face and distance learning trainings. Gender sensitive approach and a non-discriminative language is used in all the visual and printed training materials. Best practices are highlighted, and gender related questions will be designed and directed to border guards in distance learning pilot training evaluation forms to understand their awareness and needs with respect to gender issues during their border duty. Recruitment of female border officers for land border operations needs would improve the quality of LFC operational procedures such as apprehension, body search of women immigrants, etc. History

Latest Evaluations

Contact us

1 UN Plaza
DC1-20th Floor
New York, NY 10017
Tel. +1 646 781 4200
Fax. +1 646 781 4213
erc.support@undp.org