- Evaluation Plan:
- 2016-2020, Turkey
- Evaluation Type:
- Final Project
- Planned End Date:
- 03/2018
- Completion Date:
- 04/2018
- Status:
- Completed
- Management Response:
- Yes
- Evaluation Budget(US $):
- 15,000
Final Evaluation for Clearence of Landmines in the Eastern Border Regions of Turkey (Demining Project - Phase I)
| Document | Type | Language | Size | Status | Downloads |
|---|---|---|---|---|---|
 ToR_Archibald Philip Law.pdf |
tor | English | 258.58 KB | Posted | 70 |
| Final Report for Demining.pdf |
report | English | 1392.74 KB | Posted | 63 |
| Title | Final Evaluation for Clearence of Landmines in the Eastern Border Regions of Turkey (Demining Project - Phase I) | ||||||||
|---|---|---|---|---|---|---|---|---|---|
| Atlas Project Number: | 00086968 | ||||||||
| Evaluation Plan: | 2016-2020, Turkey | ||||||||
| Evaluation Type: | Final Project | ||||||||
| Status: | Completed | ||||||||
| Completion Date: | 04/2018 | ||||||||
| Planned End Date: | 03/2018 | ||||||||
| Management Response: | Yes | ||||||||
| Focus Area: |
|
||||||||
| Corporate Outcome and Output (UNDP Strategic Plan 2014-2017) |
|
||||||||
| Evaluation Budget(US $): | 15,000 | ||||||||
| Source of Funding: | Project Budget | ||||||||
| Evaluation Expenditure(US $): | 14,216 | ||||||||
| Joint Programme: | No | ||||||||
| Mandatory Evaluation: | No | ||||||||
| Joint Evaluation: | No | ||||||||
| Evaluation Team members: |
|
||||||||
| GEF Evaluation: | No | ||||||||
| Key Stakeholders: | EC | ||||||||
| Countries: | TURKEY |
| Lessons | |
|---|---|
| Findings |
| Recommendations | |
|---|---|
| 1 | Strategic Positioning, Concept and DesignThere is a tension between the desire of Turkey to quickly clear mined areas to enhance its border surveillance capability and UNDP’s mine clearance contracting that conforms to national standards which are based on the International Mine Action Standards. These tensions were not resolved before the project was designed, an RFP developed and mine clearance assets contracted. This led to challenges for all partners. To the credit of all involved in the project there was a re-set of the project into two phases and realistic and achievable objectives, outputs, activities and inputs were established for 2017. More ambitious targets were then established for 2018 and it will be a challenge to achieve these targets. Most partners understand that this project will potentially run for 10-12 years although this must be determined by non-technical survey options which will verify the correct planning period. If this phased approach is adopted it needs to be implemented in such a way that it is consistent with the National Mine Action Strategy which has been approved by the Ministry of National Defence. This strategy focuses on three strategic objectives:
This phased approach would lead to blocks of three-year planning exercises which should be based on the principles of human rights and protection. This multi-year project should support Turkey to meet its obligations to the APMBC, strengthen its border management capability and be integrated with Turkey’s commitment to the Sustainable Development Goals (SDGs). This approach is similar to the methodology that the Government of the United Kingdom is using to clear the minefields on the Falkland Islands and some of the lessons from the Falklands could be used in this project. This approach will build more confidence with stakeholders such as the donor community and the Government of Turkey and will presumably lead to tranches of funds being programmed in response to the developments on the ground. |
| 2 | ImplementationThe implementation of the project has been generally successful and many activities have been achieved as explained in this report. This is a credit to the leadership and management of the project. Future planning for the project in 2019 and beyond needs to focus UNDP’s work more on building an effective capacity development programme which involves UNDP technical staff. It is suggested that UNDP consider the following approach: 1.1 Advocate for the establishment of a NMAA to have overall responsibility for mine action in Turkey.There is a need for a ministerial level awareness and understanding of Turkey’s international obligations in the field of mine action which can only be achieved through the creation of the Authority.The NMAA will create organisational clarity for the TURMAC on its operaitonal roles and responsibilities and will help to create a more effective TURMAC and a more effective national mine action programme. 1.2 Continue and increase the capacity development of the TURMAC and enhance the ability of TURMAC to coordinate all mine action operations in Turkey whether they be conducted by the Turkish Land Forces, the Gendamarie, commercial operators or NGOs. UNDP is uniquely positioned to provide this support to the Government of Turkey and this is a low cost and potentially high return activitity for UNDP to engage in. This would require UNDP to build a Mine Action Team that provides advisory services across the core activities of a Mine Action Centre.This support can be used to provide guidance to Turkey as it prepares its extension request on its article five obligations under the APMBC which is a major piece of work and a priority for the future of mine action in Turkey. 1.3 Work with TURMAC to build its capacity to conduct non-technical survey activities which will be a longer term initiative.This will have resource implications and would require an increased investment from UNDP to hire a technical advisor for survey activities and the recruitment and training of TURMAC staff who have the appropriate competencies and experience to quickly understand their responsibilities. 1.4 Continue the current UNDP project into phase three with a new RFP based on surveyed minefields that should be conducted by non-technical survey experts.The results of the survey would form the basis for the development of a work plan for phase three which could be planned over three years.The nature of the workplan would then determine what assets are required and tendered for through the RFP. UNDP could be responsible for the contract management and TURMAC would be responsible for the operational management of the project with capacity development support from UNDP. This would require survey activities to begin early in 2018 to enable the development of a project document and the identification of a donor to partner on this project from the beginning of 2019. |
| 3 | Partnership and CoordinationUNDP has been a catalyst for collaboration and partnership and has successfully mobilised national actors and international donors around the demining project. A number of those interviewed for this review have commented favourably on UNDP’s coordination capabilities and this respect for UNDP is evident with Steering Committee members. In addition, UNDP’s support for TURMAC has provided a focus for coordination efforts in the field of mine action in Turkey. UNDP was instrumental in working with Turkish Land Forces and the European Union to develop an assessment of the requirements of this project. Whilst this assessment was insufficient, UNDP successfully deployed the clearance and quality management components and the capacity development services which have delivered many of the projects successes. UNDP also worked successfully with the European Union and TURMAC to successfully reshape the project into two phases of activities. This was a major reason for the success of the project throughout 2017. The emergence of TURMAC is also a major success for UNDP and indicative of how UNDP has supported the Government and the EU to integrate TURMAC and capacity development support into this project. This success is visible with the development of the national standards, the introduction of IMSMA and the mentoring and support for TURMAC to build an understanding of humanitarian mine action. UNDP also developed the land release methodology “the Operational Demining Process” and was then able to guide and support TURMAC and other members of the Steering Committee to build their understanding of this methodology. This is one of the major successes of this project. This review is recommending that UNDP increase its support for TURMAC as the organisation’s development will be one of the legacies of the project. The capacity development of national institutions is one of UNDP’s core activities and UNDP is encouraged to work with TURMAC and partners to accelerate its capacity development efforts based on the recent capacity assessment. These efforts should focus on building the institutional capability of TURMAC to manage operations/quality management and information management which are the core of a mine action centre’s responsibilities. It would be useful for the TURMAC operations, quality management and information management staff to be re-located to the operational sites as a capacity development initiative. Whilst there has been substantial progress in building an increased capability in these three core areas, future improvements can only really occur with the benefit of coaching and mentoring from experienced professionals. It is recommended that the TURMAC staff work with the UNDP Chief of Operations to build their understanding of operations work through on the job training. Similarly, there should be an assessment of whether the TURMAC quality management staff can be attached to the RPS quality management staff to accelerate their learning. |
| 4 | Monitoring, Evaluation and Risk ManagementThe project has successfully captured data and monitored the clearance of minefields in metres cleared and the number of mines cleared. It has also been proven to be nimble in making some major decisions to build lessons learned into the project such as the development of the Operational Demining Process. However, there is more work required to build the outcomes of success into project design and planning. This could include UNDP clearing land for broader national priorities such as passageways through the wall, the construction of towers and fencing construction, and enabling patrolling of the areas around the wall. Such an approach would be easier if the work planning process involved the clearance capability being appropriated to the priorities of the border surveillance project as opposed to clearing the minefields which is currently the operating model. Such a planning process would make it easier for the TURMAC to record outcome level data in IMSMA and would lead to a shift in the focus of mine action in Turkey. There was no evidence of an evaluation plan being built into the project. It would be useful for this to happen in advance of the project’s completion in 2018. Such a plan would use the monitoring data from the project to work out the big questions that need to be answered, the best way for them to be answered, and determine who can answer them. This can feed into the methodology and programme for the next review in 2018 which should build on the methodology and programme used for this review. This will help ensure that all monitoring and information management is being directed to the review and will provide a more accurate assessment of the project’s success and lessons learned. There could be additional work in using the risk management framework to assess and mitigate the risks that face the project. At present risk management is largely practiced through the clearance contract although this should be a theme that integrates all of the project activities including capacity development. The UNDP TAT could consider updating the risk management on a monthly basis and then having the items which are high risk and high impact on the agenda of regular meetings so that they can be closely monitored. |
| 5 | Rights Based Approach and Gender MainstreamingThere was not a rights-based approach integrated in the project and this is an area which requires further thought from the Steering Committee and the Project staff. There are opportunities for rights based approaches to be used in the border management strategies particularly in Turkey’s response to the challenges posed by issues such as mass migration and people trafficking. The demining project and the border management project presents UNDP with an opportunity to work with the Government of Turkey and Land Forces to formulate approaches that promote the protection of people’s rights, particularly those who are fleeing persecution, conflict and the survivors of trafficking. A rights-based approach would consider how the voices, agency and immediate needs of those arriving at the border are integrated into Turkey’s immigration policy and operational response. This should include a focus on the access to food, water, shelter and medical services on arrival at the Border. This would potentially be an opportunity for UNDP to work more on gender mainstreaming activities such as ensuring that the survivors of sexual and gender based violence receive access to services, that women can access sexual and reproductive health services and that women’s rights are protected. Such an initiative would enable UNDP to respond to some gaps in how the project involves women in its work. The consultant did not witness any gender mainstreaming activities in the project nor was the issue of gender and mine action raised in any of the interviews with stakeholders. Some possible steps that could be taken in future is requiring contractors to employ and train women as deminers and support staff and there should be a target established, for example 50/50, in hiring women for demining work. All evidence from many different countries indicate that women are highly capable deminers and the project needs to prioritise and act on this. UNDP could support TURMAC to work with stakeholders and develop gender in mine action guidelines based on the international guidelines which would apply to TURMAC, UNDP mine action and all mine action operators. |
1. Recommendation:
Strategic Positioning, Concept and Design
There is a tension between the desire of Turkey to quickly clear mined areas to enhance its border surveillance capability and UNDP’s mine clearance contracting that conforms to national standards which are based on the International Mine Action Standards. These tensions were not resolved before the project was designed, an RFP developed and mine clearance assets contracted. This led to challenges for all partners. To the credit of all involved in the project there was a re-set of the project into two phases and realistic and achievable objectives, outputs, activities and inputs were established for 2017. More ambitious targets were then established for 2018 and it will be a challenge to achieve these targets.
Most partners understand that this project will potentially run for 10-12 years although this must be determined by non-technical survey options which will verify the correct planning period.
If this phased approach is adopted it needs to be implemented in such a way that it is consistent with the National Mine Action Strategy which has been approved by the Ministry of National Defence. This strategy focuses on three strategic objectives:
- To perform mine action in accordance with pre-determined priorities which include creating policies on the surveillance of mines/explosives, clearance activities, mine risk training, aids for mine victims, destruction of mines kept in stocks and to plan, manage and perform these activities in accordance with the pre-determined priorities.
- To enhance Turkey’s border security and establish professional border management which will prevent illegal migration, smuggling and terrorist acts on the border between Turkey and Syria.
- To develop a sustainable national capacity by building the staff capability and institutional capacity to plan, manage, coordinate and implement the responsibilities of Turkey to the APMBC.
This phased approach would lead to blocks of three-year planning exercises which should be based on the principles of human rights and protection. This multi-year project should support Turkey to meet its obligations to the APMBC, strengthen its border management capability and be integrated with Turkey’s commitment to the Sustainable Development Goals (SDGs).
This approach is similar to the methodology that the Government of the United Kingdom is using to clear the minefields on the Falkland Islands and some of the lessons from the Falklands could be used in this project. This approach will build more confidence with stakeholders such as the donor community and the Government of Turkey and will presumably lead to tranches of funds being programmed in response to the developments on the ground.
Management Response: [Added: 2018/04/20]
UNDP Turkey affirms its commitment to mine action in Turkey. UNDP Turkey will work with the National Authorities to develop a phased and priority based clearance methodology that utilizes non-technical and technical survey to ensure realistic objectives over a specific period of activity. This planning will also have flexibility to utilizes clearance data to alter planning as necessary to meet the priorities of the Government.
Key Actions:
| Key Action | Responsible | DueDate | Status | Comments | Documents |
|---|---|---|---|---|---|
| As part of the sustainability of mine action in Turkey a more holistic project to support the Government of Turkey to meet its anti-personnel landmine treaty is being developed and will include supporting a phased approach to clearances.
[Added: 2018/04/24] |
UNDP CO, Programme and Project Staff | 2018/12 | Overdue-Initiated |
2. Recommendation:
Implementation
The implementation of the project has been generally successful and many activities have been achieved as explained in this report. This is a credit to the leadership and management of the project.
Future planning for the project in 2019 and beyond needs to focus UNDP’s work more on building an effective capacity development programme which involves UNDP technical staff. It is suggested that UNDP consider the following approach:
1.1 Advocate for the establishment of a NMAA to have overall responsibility for mine action in Turkey.There is a need for a ministerial level awareness and understanding of Turkey’s international obligations in the field of mine action which can only be achieved through the creation of the Authority.The NMAA will create organisational clarity for the TURMAC on its operaitonal roles and responsibilities and will help to create a more effective TURMAC and a more effective national mine action programme.
1.2 Continue and increase the capacity development of the TURMAC and enhance the ability of TURMAC to coordinate all mine action operations in Turkey whether they be conducted by the Turkish Land Forces, the Gendamarie, commercial operators or NGOs.
UNDP is uniquely positioned to provide this support to the Government of Turkey and this is a low cost and potentially high return activitity for UNDP to engage in. This would require UNDP to build a Mine Action Team that provides advisory services across the core activities of a Mine Action Centre.This support can be used to provide guidance to Turkey as it prepares its extension request on its article five obligations under the APMBC which is a major piece of work and a priority for the future of mine action in Turkey.
1.3 Work with TURMAC to build its capacity to conduct non-technical survey activities which will be a longer term initiative.This will have resource implications and would require an increased investment from UNDP to hire a technical advisor for survey activities and the recruitment and training of TURMAC staff who have the appropriate competencies and experience to quickly understand their responsibilities.
1.4 Continue the current UNDP project into phase three with a new RFP based on surveyed minefields that should be conducted by non-technical survey experts.The results of the survey would form the basis for the development of a work plan for phase three which could be planned over three years.The nature of the workplan would then determine what assets are required and tendered for through the RFP.
UNDP could be responsible for the contract management and TURMAC would be responsible for the operational management of the project with capacity development support from UNDP. This would require survey activities to begin early in 2018 to enable the development of a project document and the identification of a donor to partner on this project from the beginning of 2019.
Management Response: [Added: 2018/04/20]
UNDP Turkey agrees with the recommendation and will advocate for the establishment and of NMAA and will work with the Government of Turkey to incorporate capacity building of a NMAA in future mine action programming.
UNDP Turkey will work with the TURMAC to develop a support to mine action programme that is both more holistic in nature and one that contains the capacity development necessary to enable the TURMAC to fully manage a national mine programme.
As part of the development of strategic planning listed above, UNDP Turkey will work with the TURMAC to develop a high level of non-technical and technical survey within the current national structure.
Key Actions:
| Key Action | Responsible | DueDate | Status | Comments | Documents |
|---|---|---|---|---|---|
| UNDP will continue to work with the Government of Turkey and TURMAC to advocate for a National Mine Action Authority to oversee all pillars of mine action. Future programming will be resourced to support NMAA’s role in sector coordination.
[Added: 2018/04/24] |
UNDP CO, Programme and Project Staff | 2018/12 | Overdue-Initiated |
3. Recommendation:
Partnership and Coordination
UNDP has been a catalyst for collaboration and partnership and has successfully mobilised national actors and international donors around the demining project. A number of those interviewed for this review have commented favourably on UNDP’s coordination capabilities and this respect for UNDP is evident with Steering Committee members. In addition, UNDP’s support for TURMAC has provided a focus for coordination efforts in the field of mine action in Turkey.
UNDP was instrumental in working with Turkish Land Forces and the European Union to develop an assessment of the requirements of this project. Whilst this assessment was insufficient, UNDP successfully deployed the clearance and quality management components and the capacity development services which have delivered many of the projects successes. UNDP also worked successfully with the European Union and TURMAC to successfully reshape the project into two phases of activities. This was a major reason for the success of the project throughout 2017.
The emergence of TURMAC is also a major success for UNDP and indicative of how UNDP has supported the Government and the EU to integrate TURMAC and capacity development support into this project. This success is visible with the development of the national standards, the introduction of IMSMA and the mentoring and support for TURMAC to build an understanding of humanitarian mine action. UNDP also developed the land release methodology “the Operational Demining Process” and was then able to guide and support TURMAC and other members of the Steering Committee to build their understanding of this methodology. This is one of the major successes of this project.
This review is recommending that UNDP increase its support for TURMAC as the organisation’s development will be one of the legacies of the project. The capacity development of national institutions is one of UNDP’s core activities and UNDP is encouraged to work with TURMAC and partners to accelerate its capacity development efforts based on the recent capacity assessment. These efforts should focus on building the institutional capability of TURMAC to manage operations/quality management and information management which are the core of a mine action centre’s responsibilities.
It would be useful for the TURMAC operations, quality management and information management staff to be re-located to the operational sites as a capacity development initiative. Whilst there has been substantial progress in building an increased capability in these three core areas, future improvements can only really occur with the benefit of coaching and mentoring from experienced professionals.
It is recommended that the TURMAC staff work with the UNDP Chief of Operations to build their understanding of operations work through on the job training. Similarly, there should be an assessment of whether the TURMAC quality management staff can be attached to the RPS quality management staff to accelerate their learning.
Management Response: [Added: 2018/04/20]
UNDP Turkey agrees with the recommendation and will continue to work with the TURMAC to develop the capacities of national mine action institutions as part of future programme planning. This will include, but will not be limited to, support to the TURMAC in quality management, information management operations management of field operations.
Key Actions:
| Key Action | Responsible | DueDate | Status | Comments | Documents |
|---|---|---|---|---|---|
| UNDP has prepared for the TURMAC staff to work with the Chief of Operations for Phase II. In addition, UNDP will work with the TURMAC to ensure future programming will include the capacity development necessary allow for TURMAC to manage/support the entire sector.
[Added: 2018/04/24] |
UNDP CO, Programme and Project Staff | 2018/12 | Overdue-Initiated | As the TURMAC continues to develop, addition resources will be required to ensure that internal government mechanism have the technical and resource capacity to fully manage all mine action activities. |
4. Recommendation:
Monitoring, Evaluation and Risk Management
The project has successfully captured data and monitored the clearance of minefields in metres cleared and the number of mines cleared. It has also been proven to be nimble in making some major decisions to build lessons learned into the project such as the development of the Operational Demining Process. However, there is more work required to build the outcomes of success into project design and planning. This could include UNDP clearing land for broader national priorities such as passageways through the wall, the construction of towers and fencing construction, and enabling patrolling of the areas around the wall.
Such an approach would be easier if the work planning process involved the clearance capability being appropriated to the priorities of the border surveillance project as opposed to clearing the minefields which is currently the operating model. Such a planning process would make it easier for the TURMAC to record outcome level data in IMSMA and would lead to a shift in the focus of mine action in Turkey.
There was no evidence of an evaluation plan being built into the project. It would be useful for this to happen in advance of the project’s completion in 2018. Such a plan would use the monitoring data from the project to work out the big questions that need to be answered, the best way for them to be answered, and determine who can answer them. This can feed into the methodology and programme for the next review in 2018 which should build on the methodology and programme used for this review. This will help ensure that all monitoring and information management is being directed to the review and will provide a more accurate assessment of the project’s success and lessons learned.
There could be additional work in using the risk management framework to assess and mitigate the risks that face the project. At present risk management is largely practiced through the clearance contract although this should be a theme that integrates all of the project activities including capacity development. The UNDP TAT could consider updating the risk management on a monthly basis and then having the items which are high risk and high impact on the agenda of regular meetings so that they can be closely monitored.
Management Response: [Added: 2018/04/20]
UNDP Turkey agrees with the recommendation that future programming will be outcome based with an appropriate level of reporting.
Key Actions:
| Key Action | Responsible | DueDate | Status | Comments | Documents |
|---|---|---|---|---|---|
| UNDP will work with the TURMAC, CFCU and Donors to ensure that future programming is developed to enable outcome based reporting.
[Added: 2018/04/24] |
UNDP CO, Programme and Project Staff | 2018/12 | Overdue-Initiated | Much of the reporting in the project was based on activities, future programming will be designed to be outcome based. |
5. Recommendation:
Rights Based Approach and Gender Mainstreaming
There was not a rights-based approach integrated in the project and this is an area which requires further thought from the Steering Committee and the Project staff. There are opportunities for rights based approaches to be used in the border management strategies particularly in Turkey’s response to the challenges posed by issues such as mass migration and people trafficking. The demining project and the border management project presents UNDP with an opportunity to work with the Government of Turkey and Land Forces to formulate approaches that promote the protection of people’s rights, particularly those who are fleeing persecution, conflict and the survivors of trafficking.
A rights-based approach would consider how the voices, agency and immediate needs of those arriving at the border are integrated into Turkey’s immigration policy and operational response. This should include a focus on the access to food, water, shelter and medical services on arrival at the Border. This would potentially be an opportunity for UNDP to work more on gender mainstreaming activities such as ensuring that the survivors of sexual and gender based violence receive access to services, that women can access sexual and reproductive health services and that women’s rights are protected. Such an initiative would enable UNDP to respond to some gaps in how the project involves women in its work.
The consultant did not witness any gender mainstreaming activities in the project nor was the issue of gender and mine action raised in any of the interviews with stakeholders. Some possible steps that could be taken in future is requiring contractors to employ and train women as deminers and support staff and there should be a target established, for example 50/50, in hiring women for demining work. All evidence from many different countries indicate that women are highly capable deminers and the project needs to prioritise and act on this. UNDP could support TURMAC to work with stakeholders and develop gender in mine action guidelines based on the international guidelines which would apply to TURMAC, UNDP mine action and all mine action operators.
Management Response: [Added: 2018/04/20]
UNDP Turkey fully agrees with the recommendation and will ensure that a rights-based approach and a higher level of gender mainstreaming is included as part of future support to mine action programming. UNDP Turkey will also maintain its role of initiator and nurture South-South and triangular collaboration, including with a wide range of development practitioners. This will focus on the creation of development benefits for affected women, men and communities. When possible, UNDP Turkey will support strengthening of civil society organizations that will enable a greater participation in the development, planning and delivery of mine action interventions.
Key Actions:
| Key Action | Responsible | DueDate | Status | Comments | Documents |
|---|---|---|---|---|---|
| UNDP is working with TURMAC to ensure that future programming has both a rights-based approach and more detailed gender mainstreaming component during project design. UNDP is working with TURMAC to enable south-south/triangular cooperation for Phase II and future programming with other regional affected states and international mine action intuitions.
[Added: 2018/04/24] |
UNDP CO, Programme and Project Staff | 2018/12 | Overdue-Initiated |