Clearing for Result III: Mine Action for Human Development (mid-term)

Report Cover Image
Evaluation Plan:
2016-2018, Cambodia
Evaluation Type:
Mid Term Project
Planned End Date:
02/2018
Completion Date:
02/2018
Status:
Completed
Management Response:
Yes
Evaluation Budget(US $):
30,000

Share

Document Type Language Size Status Downloads
Download document MAfHD_MTR_TOR_International consultant.pdf tor English 2966.20 KB Posted 97
Download document MAfHD_MTR_Final Report_20180205.pdf report English 1320.83 KB Posted 172
Title Clearing for Result III: Mine Action for Human Development (mid-term)
Atlas Project Number: 00090541
Evaluation Plan: 2016-2018, Cambodia
Evaluation Type: Mid Term Project
Status: Completed
Completion Date: 02/2018
Planned End Date: 02/2018
Management Response: Yes
Focus Area:
  • 1. Crisis Prevention & Recovery
  • 2. Others
Corporate Outcome and Output (UNDP Strategic Plan 2014-2017)
  • 1. Output 1.1. National and sub-national systems and institutions enabled to achieve structural transformation of productive capacities that are sustainable and employment - and livelihoods- intensive
Evaluation Budget(US $): 30,000
Source of Funding:
Evaluation Expenditure(US $): 21,814
Joint Programme: No
Joint Evaluation: No
Evaluation Team members:
Name Title Email Nationality
Denika Balcklock Evaluation Team Leader djbkarim@gmail.com
Chey Tech National Evaluation Consultant cheytech2012@gmail.com
GEF Evaluation: No
Key Stakeholders:
Countries: CAMBODIA
Lessons
1.

One of the challenges faced by the project was the shortfall in anticipated funds, as the project was designed and budgeted with the expectation that the RGC would commit to cost-sharing or co-financing some of the activities. However, the project document was signed before any commitments were agreed to, and it was difficult and finally impossible to get the Ministry of Economy and Finance to agree to cost-sharing in any form. The project has managed to undertake budget revisions to cover the short-falls to a large extent, but there will still be a gap in financing by the end of the project in 2019. Thus, one of the most important management lessons to take away from this project is that cost-sharing or co-financing arrangements with government should be agreed – in writing – prior to the signing of a project document. Moreover, it would be valuable to explore different modes of government cost-sharing, such as ‘in-kind’ contributions like the communications and basic office administration costs incurred through the implementation of project activities. An impact of the lack of formal cost-sharing, co-financing or ‘in-kind’ contributions is the fact that many CMAA staff view the products or mechanisms developed via CfRIII activities to be a ‘UNDP thing.’ The project team and CMAA senior management need to put concerted effort into ensuring CMAA takes full ownership of the project and its results, regardless of financing issues, for the long-term impact of the change effected by the project to eventuate.


2.

Another issue which was raised, particularly at the sub-national level, was the question of monetary compensation for implementing project activities. There would appear to be a fairly consistent misunderstanding – at the national level as well, but to a lesser degree – that the project is a UNDP project, rather than a government project. In fact, the project is implemented within the government workplan, consistent with UNDP’s National Implementation Modality (NIM), so any activities carried out by government employees (or contractors, as is the case in many instances within CMAA), are not stand-along project activities. The consultants routinely heard that staff implement this project ‘without compensation.’ It is therefore important for CMAA and UNDP to ensure that all staff (CMAA and MAPUs) understand that CfRIII is a government project, with technical and financial support from UNDP, to dispel misconceptions of ‘extra work’ or the need for ‘compensation.’


3.

From a technical perspective, there was one theme that was raised a number of times in terms of training. National and sub-national officials routinely noted the need for extra training due to the rotation of staff or the fact that training was not attended by the most appropriate individual (for example, MAPU staff would attend trainings in Phnom Penh, but because they were not the most appropriate staff member for such training, they did not have the capacity to transfer knowledge to other MAPU staff upon their return to their office). As a result, much of the knowledge that is imparted to training participants is lost – either due to a change in staff or an inability to apply the knowledge used. This is a classic example of capacity building activities focusing too heavily on the individual rather than on the institution or system, where enhanced capacities or tools are more sustainable in the longer term. The next phase of the project should be able to rectify this through the development of a Capacity Development Plan (through the CDNA process), but this is an important lesson to take away for CMAA as it anticipates working closely with ARMAC in the future for improved South-South Cooperation.


4.

Finally, this project is in its third phase, and with demining planned for completion by 2025, UNDP needs an exit strategy from the sector. In particular, UNDP (and its partners) need to assess how to transition away from a purely capacity development and demining approach to the sector, to focus more concretely on supporting the RGC to link the results of demining (land release) to more concerted poverty reduction and community development efforts, in particular supporting provincial and district governments to diversify local economies to support more robust community development and sustainable poverty reduction, in line with the SDG tag line ‘leave no one behind.’


Findings
1.

As the project proceeds through its third phase, the objectives and outputs continue to be highly relevant to the mine action sector - CfRIII’s approach to helping CMAA transition from a purely humanitarian objective to a more sustainable development-oriented focus is timely and valued. Its activities to develop a socially and environmentally-sensitive Performance Monitoring System (PMS), in line with recommendations from the CfRII Final Evaluation and the 2016 Environment and Social Impact Assessment (ESIA), help to broaden the understanding of what communities need once mines are cleared and land is released.


2.

Significant progress has been made against the project outputs. While a strong project management team has been critical to progress through 2017, other factors which have played an important role seeing the project through a relatively tough first half of Phase 3 is the long-standing relationship between UNDP and CMAA through the first two phases of the project, as well as the excellent cooperation between UNDP, DFAT and SDC to leverage political and diplomatic pressure to resolve procurement issues which were outside of the capacity of the project management team to resolve on their own. However, with a capacity building approach that primarily targets the individual, the constant rotation of government staff has held back to progress in capacity building which CfR (through all of its phases) could have had if a more comprehensive capacity development approach was applied.


3.

The results of CfRIII have been few in terms of overall numbers, but extremely important in terms of strategic change – including pushing NMAS finalization, expediting the development of the PMS to lay the ground work to link land release with poverty reduction, and piloting the ‘Safe Village’ strategy, of which many sub-national stakeholders are requesting to be scale-up. Each of these results is likely to lead to a more coordinated, human development-focused and efficient mine action sector over the next two years.


4.

CfRIII is one of the most efficient projects in terms of demining, getting the best value for money in terms of $/m2. This is largely due to the fact that operators do not charge CfRIII for the cost of new equipment, keeping costs to, on average, less than $0.20/m2. With the inclusion of non-technical surveys, this will increase the overall cost-efficiency of land release and will be complemented by decrease operational and transport costs if/when the ‘Safe Village’ strategy is rolled out to a larger number of communities.


5.

The results of the project to date are very likely to be sustainable considering that they have focused on policy and strategy development (NMAS, ‘Safe Village’), and mainstreaming tools and new practices (PMS) into the day-to-day work of CMAA. However, strategies and tools only go so far, and must be both owned and implemented by CMAA to effect any change.


6.

The change effected by the project has focused on laying the groundwork for a more cost-efficient mine action sector, and mainstreaming gender and poverty reduction themes into the sector through a more robust PMS and the piloting of the ‘Safe Village’ strategy. When assessing the short-term impact of the project, it’s contribution to setting the stage for more development-oriented policies and strategies in the sector has been essential and has helped to tip the balance of mine action in Cambodia away from a focus on short-term clearance results to longer-term sustainable development objectives. 


Recommendations
1
  1. The project should lead a Capacity Development Needs Assessment (CDNA) of CMAA and MAPUs (as well as related agencies anticipated to pick up residual MA work after 2025) using UNDP’s global Capacity Development Framework.  This would address a number of issues. First, it would solve the problem of current capacity building activities being too focused on the individual, creating gaps in capacity and workflow as government staff are routinely reassigned to new duties, departments or ministries. Secondly, it would address coordination of capacity building activities amongst partners in the sector, with a Capacity Development Plan serving as the main coordination tool for such activities. Finally, it would address capacity building issues for CMAA, MAPUs and other sectors as noted in the NMAS 2018-2025. Moreover, the last full assessment of CMAA was undertaken in 2014 and as such, the institution is well over-due for a follow-up.
2

2. CMAA, with CfRIII support, should develop a ‘Safe Village’ policy and necessary implementing regulations to scale-up the ‘Safe Village’ strategy being piloted by CfRIII in three provinces. As noted above in the findings section, the ‘Safe Village’ strategy has found much favour at the sub-national level and provincial leaders are keen to champion such an approach, as it will allow more comprehensive development planning to take place, and thereby reduce the costs for both government and donors in project implementation. It was noted that guidelines on what constitutes ‘safe’ would be needed (for example, shallow clearance, deep clearance and under water clearance), and a formal certification process to declare a village ‘safe’ be developed. The policy and implementing regulations should tangibly link land release with community development, encouraging donors to plan for activities in ‘Safe Villages’, most appropriately through the TWG-Mine Action (see below). The new PMS will be an excellent tool to support these links, as the data being collected will be more socially and environmentally-conscientious, and demonstrate a commitment to community needs and well-being, rather than simply a focus on the total area of land released.

3

3.  CMAA should prioritize the activities of the TWG-Mine Action to improve development effectiveness in the mine action sector. Using the NMAS 2018-2025, the proposed Capacity Development Plan and data collected through a more robust PMS, CMAA will have the tools it needs to effectively coordinate actors and activities in the sector, improve development effectiveness, and advocate for more, or at the least more strategic, resources for mine action and resulting community development needs. Engaging an advocacy and communications specialist to support the development of a workplan in this regard would be a worthwhile investment. CMAA should also explore options of setting up a Mine Action Trust Fund to pool financing for the sector to improve coordination, transparency and efficiency in land release, particularly as funding for the sector decreases. Such a fund would be administered through the Ministry of Economy and Finance, with the support of a Secretariat of seconded CMAA staff.

4

The project should develop a formal partnership strategy to improve efficiency in capacity development within the mine action sector and within CMAA specifically. The results of the CDNA and development of a Capacity Development Plan can serve as the basis for this, and should be guided by the coordination efforts of the TWG-Mine Action so as not to undermine their leadership in coordinating the sector.

5

5.  CfRIII, together with CMAA, needs to develop an action plan to support the work of ARMAC, which would leverage CMAA and other local stakeholders’ knowledge and experience to make ARMAC a centre of excellence, and facilitate Cambodia as a leading contributor to South-South Cooperation in the Mine Action Sector. Some initial ideas include surveying sector counterparts in the Southeast Asia region to identify particular needs or knowledge gaps, and designing training programmes or similar to which CMAA could lend its particular knowledge and expertise as trainers or facilitators.

6

6.  The issues related to the inconsistent information on the Planning and Prioritization (P&P) process in Battambang province.  It is recommended that CMAA undertake spot checks of the planning process to investigate what is actually happening on the ground, and who is participating in the process. The overall impression of the consultants was that irregularities in the P&P process were not related to capacity or knowledge gaps, but rather to attitudes within the MAPU office. While recommending staffing changes are not within the scope of this MTR, the consultants propose peer-to-peer learning, potentially seconding the Chief of MAPU from Banteay Meanchey for three to six months, to problem solve within the unit, and get the P&P process back on track.

7

7.  The consultants were privy to a number of comments suggesting an over-familiarity between operators and the Quality Monitoring teams which oversee their work. While there was no suggestion of wrong-doing, such as inaccurate reporting of non-compliance issues, it is important that Quality Monitoring teams retain a certain degree of impartiality towards operator activities. Moreover, ensuring that operators are abiding by mine action safety standards is of increasing importance as climate change impacts (such as flooding and landslides) can shift mines around and Quality Monitoring teams need to ensure that operators are taking these risks into account. This is particularly important in the case of CfRIII, in which the Quality Monitoring teams financially supported by the project oversee the operators undertaking land release activities funded by the project. It was recommended by another stakeholder that CMAA should institute a policy whereby Quality Monitoring teams are rotated between provinces (all teams, not just those financially supported by the project) on a regular basis to reinforce the integrity of their work. The MTR consultants fully agree with this recommendation.

8

8.  The MTR consultants, both results-based management specialists, noted that support for CMAA senior management to participant in international forums and meetings has been budgeted under output 2 of the project (support to monitoring and data collection). We were unclear as to the rationale for the placement of this activity under this output. However, with the implementation of the NMAS 2018-2025, and the recommendation for CfRIII to place some priority on developing a relationship with ARMAC to facilitate CMAA’s South-South Cooperation as part of NMAS, the consultants recommend that all activities related to CMAA participation in international fora be transitioned to output 1 of the project (policy and strategy), so that the results of such activities can make a strategic contribution to the achievement of this output and to the intended outcome of the project.

1. Recommendation:
  1. The project should lead a Capacity Development Needs Assessment (CDNA) of CMAA and MAPUs (as well as related agencies anticipated to pick up residual MA work after 2025) using UNDP’s global Capacity Development Framework.  This would address a number of issues. First, it would solve the problem of current capacity building activities being too focused on the individual, creating gaps in capacity and workflow as government staff are routinely reassigned to new duties, departments or ministries. Secondly, it would address coordination of capacity building activities amongst partners in the sector, with a Capacity Development Plan serving as the main coordination tool for such activities. Finally, it would address capacity building issues for CMAA, MAPUs and other sectors as noted in the NMAS 2018-2025. Moreover, the last full assessment of CMAA was undertaken in 2014 and as such, the institution is well over-due for a follow-up.
Management Response: [Added: 2018/03/01] [Last Updated: 2018/04/05]

The Project agrees with the MTR’s recommendation.

Key Actions:

Key Action Responsible DueDate Status Comments Documents
Engage external consultants to conduct the Capacity Development Needs Assessment and develop the capacity develop plan.
[Added: 2018/04/05] [Last Updated: 2019/03/14]
CMAA with support from CfRIII team 2020/02 Initiated Due date moved to February 2020 as recommended by Regional Bx due to pressure of ROAR preparation in December 2019. History
2. Recommendation:

2. CMAA, with CfRIII support, should develop a ‘Safe Village’ policy and necessary implementing regulations to scale-up the ‘Safe Village’ strategy being piloted by CfRIII in three provinces. As noted above in the findings section, the ‘Safe Village’ strategy has found much favour at the sub-national level and provincial leaders are keen to champion such an approach, as it will allow more comprehensive development planning to take place, and thereby reduce the costs for both government and donors in project implementation. It was noted that guidelines on what constitutes ‘safe’ would be needed (for example, shallow clearance, deep clearance and under water clearance), and a formal certification process to declare a village ‘safe’ be developed. The policy and implementing regulations should tangibly link land release with community development, encouraging donors to plan for activities in ‘Safe Villages’, most appropriately through the TWG-Mine Action (see below). The new PMS will be an excellent tool to support these links, as the data being collected will be more socially and environmentally-conscientious, and demonstrate a commitment to community needs and well-being, rather than simply a focus on the total area of land released.

Management Response: [Added: 2018/03/01] [Last Updated: 2018/04/05]

The Project agrees with the MTR’s recommendation.

The current CfRIII approach is to achieve impact-free, and not mine free, in target villages. In this case, the term “safe village” will refer to impact-free village.

Key Actions:

Key Action Responsible DueDate Status Comments Documents
- Develop and implement the impact free village policy. - Modify planning and prioritisation (P&P) guidelines to implement the impact-fee village concept.
[Added: 2018/04/05] [Last Updated: 2019/01/20]
CMAA with support from CfRIII team 2019/07 Initiated Mine-free village strategy is drafted, disseminated to operators for comments. CMAA plans to organize Clearance and Survey TRG meeting to discuss the policy and aim to have it officially finalized by 30 June 2019. As such, the initial deadline is extended to 30 June 2019. History
3. Recommendation:

3.  CMAA should prioritize the activities of the TWG-Mine Action to improve development effectiveness in the mine action sector. Using the NMAS 2018-2025, the proposed Capacity Development Plan and data collected through a more robust PMS, CMAA will have the tools it needs to effectively coordinate actors and activities in the sector, improve development effectiveness, and advocate for more, or at the least more strategic, resources for mine action and resulting community development needs. Engaging an advocacy and communications specialist to support the development of a workplan in this regard would be a worthwhile investment. CMAA should also explore options of setting up a Mine Action Trust Fund to pool financing for the sector to improve coordination, transparency and efficiency in land release, particularly as funding for the sector decreases. Such a fund would be administered through the Ministry of Economy and Finance, with the support of a Secretariat of seconded CMAA staff.

Management Response: [Added: 2018/03/01] [Last Updated: 2018/04/05]

The Project agrees with the MTR’s recommendation.

To ensure better coordination of the MA sector, CMAA will revise the Term of reference (ToR) and membership of the TWG-MA, and regularly hold the meetings of this group.

Establishment of MA Trust Fund is an interesting concept, CMAA will explore an interest from stakeholders on building this Fund.

A communications volunteer has already joined the project team.

Key Actions:

Key Action Responsible DueDate Status Comments Documents
Review the TOR and membership of the TWG-MA.
[Added: 2018/04/05] [Last Updated: 2018/10/28]
CMAA with support from CfRIII team 2018/10 Completed TOR of TWG-MA was developed. History
Convene TWG-MA meetings twice per year as stated in the NMAS 2018-2025.
[Added: 2018/04/05] [Last Updated: 2019/03/07]
CMAA with support from CfRIII team 2019/01 Completed TOR of TWG-MA was developed. The TWG-MA met on 29 March 2018. The TWG-MA met only once in 2018. MoEF raised the idea of forming a Trust Fund for mine action at the TWG-MA. Yet, no interest from stakeholders observed. History
Explore interest of stakeholders on establishment of the Trust Fund.
[Added: 2018/04/05] [Last Updated: 2018/10/28]
CMAA with support from CfRIII team 2018/10 Completed MoEF raised the idea of forming a Trust Fund for mine action at the TWG-MA. Yet, no interest from stakeholders observed History
4. Recommendation:

The project should develop a formal partnership strategy to improve efficiency in capacity development within the mine action sector and within CMAA specifically. The results of the CDNA and development of a Capacity Development Plan can serve as the basis for this, and should be guided by the coordination efforts of the TWG-Mine Action so as not to undermine their leadership in coordinating the sector.

Management Response: [Added: 2018/03/01] [Last Updated: 2018/04/05]

The Project agrees with the MTR’s recommendation.

 

The current mine action coordination mechanism at sub-national level, the MAPU, is where the MA uses to exchange information about contamination and released land and the development opportunity. This mechanism allows MA to leverage resources made available by others within and outside of the sector to maximize the benefit of the mine/ERW affected communities.

 

To systematically guide this effort, CMAA will develop a partnership strategy referring to the capacity development plan and the guidance from the TWG-MA (as it is also part of the partnership strategy).

Key Actions:

Key Action Responsible DueDate Status Comments Documents
Develop the partnership strategy
[Added: 2018/04/05] [Last Updated: 2019/01/14]
CMAA with support from CfRIII team and communication volunteer. 2019/06 Initiated Partnership strategy (with DFID CD project or other stakeholders) will be developed to support the implementation of CMAA’s CD activity History
5. Recommendation:

5.  CfRIII, together with CMAA, needs to develop an action plan to support the work of ARMAC, which would leverage CMAA and other local stakeholders’ knowledge and experience to make ARMAC a centre of excellence, and facilitate Cambodia as a leading contributor to South-South Cooperation in the Mine Action Sector. Some initial ideas include surveying sector counterparts in the Southeast Asia region to identify particular needs or knowledge gaps, and designing training programmes or similar to which CMAA could lend its particular knowledge and expertise as trainers or facilitators.

Management Response: [Added: 2018/03/01] [Last Updated: 2018/04/05]

The Project agrees with the MTR’s recommendation.

Action plan to support and engage with ARMAC is included in the CfRIII 2018 work plan. 

Key Actions:

Key Action Responsible DueDate Status Comments Documents
Upon approval by all ASEAN member states, the CMAA and CfRIII will provide technical advisory support to ARMAC in areas agreed upon with ARMAC.
[Added: 2018/04/05] [Last Updated: 2019/03/28]
CMAA with support from CfRIII team 2019/03 Completed A memorandum of understanding between CMAA and ARMAC was signed on 18 November 2018. The MOU outlines potential areas of cooperation and assistance for both organisations. CfRIII assisted CMAA in reviewing the MOU. History
Provide ARMAC with networking opportunities. This also includes support to south-south cooperation initiatives of the project.
[Added: 2018/04/05] [Last Updated: 2019/03/28]
CMAA with support from CfRIII team 2019/03 Completed CfR supported the ARMAC Regional Workshop in November 2018 that brought together various regional organisations, therefore allowing networks to be established. Also in November, CfR ensured that ARMAC was introduced to various mine action stakeholders at the Meeting of States Parties to the Mine Ban Treaty. History
6. Recommendation:

6.  The issues related to the inconsistent information on the Planning and Prioritization (P&P) process in Battambang province.  It is recommended that CMAA undertake spot checks of the planning process to investigate what is actually happening on the ground, and who is participating in the process. The overall impression of the consultants was that irregularities in the P&P process were not related to capacity or knowledge gaps, but rather to attitudes within the MAPU office. While recommending staffing changes are not within the scope of this MTR, the consultants propose peer-to-peer learning, potentially seconding the Chief of MAPU from Banteay Meanchey for three to six months, to problem solve within the unit, and get the P&P process back on track.

Management Response: [Added: 2018/03/01] [Last Updated: 2018/04/05]

The Project agrees with the MTR’s recommendation.

The revised P&P guideline specifies how villages are prioritized for annual mine clearance plan. CMAA will ensure a consistent application of the P&P guidelines.

Key Actions:

Key Action Responsible DueDate Status Comments Documents
CMAA to conduct a spot check to MAPU in Battambang province to monitor their performance on selection of prioritized villages for annual mine clearance plan.
[Added: 2018/04/05]
CMAA 2018/03 Completed Soon after the MTR report, a spot check to MAPU in BTB was conducted by CMAA. The chief of MAPU in BTB committed to improve their performance, and the chief of PMAC will review the MAPU chief’s performance over the next three months.
7. Recommendation:

7.  The consultants were privy to a number of comments suggesting an over-familiarity between operators and the Quality Monitoring teams which oversee their work. While there was no suggestion of wrong-doing, such as inaccurate reporting of non-compliance issues, it is important that Quality Monitoring teams retain a certain degree of impartiality towards operator activities. Moreover, ensuring that operators are abiding by mine action safety standards is of increasing importance as climate change impacts (such as flooding and landslides) can shift mines around and Quality Monitoring teams need to ensure that operators are taking these risks into account. This is particularly important in the case of CfRIII, in which the Quality Monitoring teams financially supported by the project oversee the operators undertaking land release activities funded by the project. It was recommended by another stakeholder that CMAA should institute a policy whereby Quality Monitoring teams are rotated between provinces (all teams, not just those financially supported by the project) on a regular basis to reinforce the integrity of their work. The MTR consultants fully agree with this recommendation.

Management Response: [Added: 2018/04/05]

The Project agrees with the MTR’s recommendation.

However, it is to acknowledge that because of the years-long working experience of the QM team and operator staff, it is very much likely that those people happened to know each other, though a QM team rotation is applied.

Key Actions:

Key Action Responsible DueDate Status Comments Documents
Revise internal CMAA/QM team regulation and include a requirement of team rotation.
[Added: 2018/04/05] [Last Updated: 2018/08/09]
CMAA 2018/06 Completed CMAA revised its QM team internal regulation and include the requirement of team rotation to strengthen the monitoring quality of the team. History
8. Recommendation:

8.  The MTR consultants, both results-based management specialists, noted that support for CMAA senior management to participant in international forums and meetings has been budgeted under output 2 of the project (support to monitoring and data collection). We were unclear as to the rationale for the placement of this activity under this output. However, with the implementation of the NMAS 2018-2025, and the recommendation for CfRIII to place some priority on developing a relationship with ARMAC to facilitate CMAA’s South-South Cooperation as part of NMAS, the consultants recommend that all activities related to CMAA participation in international fora be transitioned to output 1 of the project (policy and strategy), so that the results of such activities can make a strategic contribution to the achievement of this output and to the intended outcome of the project.

Management Response: [Added: 2018/04/05]

The Project agrees with the MTR recommendation.

Key Actions:

Key Action Responsible DueDate Status Comments Documents
Move the budget line under KD2 to KD1 and seek approval from the Project Board for this revision.
[Added: 2018/04/05] [Last Updated: 2018/08/09]
CMAA with support from CfRIII team 2018/04 Completed Budget revised and approved by Project Board History

Latest Evaluations

Contact us

220 East 42nd Street
20th Floor
New York, NY 10017
Tel. +1 646 781 4200
Fax. +1 646 781 4213
erc.support@undp.org