Mid-term evaluation for Comprehensive Environmentally Sound Management of PCBs in Montenegro

Report Cover Image
Evaluation Plan:
2017-2021, Montenegro
Evaluation Type:
Mid Term Project
Planned End Date:
12/2019
Completion Date:
12/2019
Status:
Completed
Management Response:
Yes
Evaluation Budget(US $):
20,000

Share

Document Type Language Size Status Downloads
Download document 5562 MTR_Report_MNE.pdf report English 1299.62 KB Posted 41
Download document UNDP-GEF-MTR-TOR-PCB Montenegro International.docx tor English 100.72 KB Posted 17
Download document UNDP-GEF-MTR-TOR-PCB Montenegro Local.docx tor English 101.13 KB Posted 13
Title Mid-term evaluation for Comprehensive Environmentally Sound Management of PCBs in Montenegro
Atlas Project Number: 00088794
Evaluation Plan: 2017-2021, Montenegro
Evaluation Type: Mid Term Project
Status: Completed
Completion Date: 12/2019
Planned End Date: 12/2019
Management Response: Yes
Focus Area:
  • 1. Environment & Sustainable Development
  • 2. Others
Corporate Outcome and Output (UNDP Strategic Plan 2014-2017)
  • 1. Output 1.3. Solutions developed at national and sub-national levels for sustainable management of natural resources, ecosystem services, chemicals and waste
SDG Goal
  • Goal 9. Build resilient infrastructure, promote inclusive and sustainable industrialization and foster innovation
SDG Target
  • 9.4 By 2030, upgrade infrastructure and retrofit industries to make them sustainable, with increased resource-use efficiency and greater adoption of clean and environmentally sound technologies and industrial processes, with all countries taking action in accordance with their respective capabilities
Evaluation Budget(US $): 20,000
Source of Funding: GEF
Evaluation Expenditure(US $): 20,000
Joint Programme: No
Joint Evaluation: No
Evaluation Team members:
Name Title Email Nationality
Dalibor Kysela Mr dkysela@gmx.at CZECH REPUBLIC
Sne┼żana Marstijepovic Ms snezana.marstijepovic@gmail.com
GEF Evaluation: Yes
GEF Project Title: Comprehensive Environmentally Sound Management of PCBs in Montenegro
Evaluation Type: Mid-term Review
Focal Area: Persistent Organic Pollutants
Project Type: FSP
GEF Phase: GEF-6
GEF Project ID: 9045
PIMS Number: 5562
Key Stakeholders: Government, Institutions
Countries: MONTENEGRO
Lessons
1.

On progress towards the project objective and outcomes

The technical and economic assessment of cost-effectiveness of various technology options will be a complex exercise and will require considerable time for completion. In case the dehalogenation technology is identified as the preferable option, there will be additional sizeable time period required to obtain all necessary legal permits for operation and complete procurement, delivery and commissioning of equipment.  

There is a risk that the protracted deficiency of the PPP legislation could negatively affect the environmentally sound PCB management beyond the duration of the current project. The Project Steering Committee should be considered as an interim body for coordination and oversight of PCB management in Montenegro until an effective PPP mechanism will have been created and institutionalized to assume this responsibility.

The information in the 2019 NIP update on amounts of PCB waste and PCB contaminated equipment in use in Montenegro suggests there is still a sizeable number of electrical equipment in the country that is potentially contaminated with PCBs where the level of PCB content has not yet been ascertained, and it is expected that throughout implementation of this project all PCB contaminated equipment and waste will be identified and the national PCB inventory will be adequately updated.

A thorough assessment of available options for handling the special transformers at Uniprom-KAP will be necessary to take into account advantages and disadvantages of total replacement of the special PCB transformers as well as alternatives to the total replacement in order to optimize the associated costs.

Although the Montenegrin legislation is well advanced and generally compliant with the international regulations (the Stockholm Convention and the EU directive on management of PCBs) and substantive progress has been achieved in updating the inventory of PCB waste, effectiveness of enforcement of the legislation is critical for the success of management and progress on PCB phase-out in Montenegro.

Postponement of the legal obligation for PCB phase-out could diminish the commitment to early action as the PCB holders may decide to push back their plans for PCB phase-out well beyond the completion date of the current project. Therefore, this motion raises concerns as to whether the project will be able to provide assistance in phasing-out the planned amounts of PCB equipment and waste during the implementation period of the project that will end in 2021.

Insufficient communication with wider circle of stakeholders could limit the general support for the intervention, especially in cases when advocacy or policy change are needed to increase the level of priority given to the PCB-related issued by the authorities.

One of the reasons that PCBs are not immediately perceived as a hazard by the common public is low level of involvement of institutions of higher education and NGOs in the national PCB debate. Consequently, the issue of PCBs is very often given a low priority by the public at large.

Although the 2014-2019 NIP called for establishment of a system for collecting data on use of PCBs in the industry of plastics, coatings, paints and varnishes, as well as paints in construction, no activities in this regard have been conducted to date. Montenegro has sufficient capacities for sampling and data analysis of caulking and paints used in building construction. This could become a foundation for eventual preparation of future activities on measuring PCB exposure levels for compliance with health-based exposure limits.


2.

On project implementation and reporting

Inconsistencies in the formulation of achievement target indicators in the project results framework obstruct monitoring and evaluation of the project performance.

Low amounts of in-kind co-financing reported by CEDIS and Uniprom-KAP and absence of in-kind co-financing data from MoSDT indicate that not all in-kind co-financing has been properly calculated and reported to PMU. This deficiency will hinder rigorous assessment of the parallel financing at the terminal evaluation.

Critical risk management is a standard part of the annual PIRs and periodic re-assessment of a risk management plan by both PMU and RTA is fundamental to the project’s implementation and success. Labelling a risk as critical provides an important alert to the project implementation that facilitates development of timely and effective risk mitigation measures.

 


Findings
1.

Concise Finding 1: The analysis of the technical and economic feasibility of disposal of low-concentration PCB waste has not been completed at the mid-term stage of the project as was planned.

Conclusion 1: The technical and economic assessment of cost-effectiveness of various technology options will be a complex exercise and will require considerable time for completion. In case the dehalogenation technology is identified as the preferable option, there will be additional sizeable time period required to obtain all necessary legal permits for operation and complete procurement, delivery and commissioning of equipment.   


2.

Concise Finding 2: The work on establishment of a public-private partnership that is proposed in the project to oversee the PCB management in the country in the medium to long term has not started yet due to the existing legislative gap, namely absence of the law on PPPs. Due to complexity of public discussion and subsequent parliamentary approval processes, it is difficult to predict whether the new law will have been enacted within the implementation period of the current project and enable thus creation of a PPP according to the planned work under this project sub-component in line with the new law.

Conclusion 2:   There is a risk that the protracted deficiency of the PPP legislation could negatively affect the environmentally sound PCB management beyond the duration of the current project. The Project Steering Committee should be considered as an interim body for coordination and oversight of PCB management in Montenegro until an effective PPP mechanism will have been created and institutionalized to assume this responsibility.


3.

Concise Finding 3:  The formulation of the project was based mainly on a “soft” data that was obtained through a preliminary inventory of potentially PCB-contaminated equipment carried out in 2013 by the Administration for Inspection. At the preparatory phase, capacitors in possession of EPCG had been identified as a potential source of PCBs but these have not yet been sampled and analysed under the project. Moreover, the 2019 NIP update contains information on sources of PCB-contaminated equipment and PCB waste throughout the country, such as the transformers in possession of the Railway Infrastructure of Montenegro, Adriatic Shipyard “Bijela”, Coal Mine Pljevlja, as well as the so-called transformers from “unknown” owners.

Conclusion 3: The updated information on transformers and capacitors suggests there is still a sizeable number of electrical equipment in the country that is potentially contaminated with PCBs where the level of PCB content has not yet been ascertained.


4.

Concise Finding 4: There has been a number of special transformers at Uniprom-KAP that require special attention since a total replacement of all these transformers is considered too expensive.

Conclusion 4: A thorough assessment of available options for handling the special transformers at Uniprom-KAP will be necessary to take into account advantages and disadvantages of total replacement of the special PCB transformers as well as alternatives to the total replacement in order to optimize the associated costs.


5.

Concise Finding 5: One of the leading premises for the PCB project formulation was the low effectiveness of enforcement of the existing legislation on PCBs in Montenegro. The project so far has provided only limited assistance to the environmental inspection authorities to fulfil their duties. The Administration for Environmental Inspections plans to recruit a number of new inspectors and it is therefore necessary to train the inspectors specifically on enforcement of the requirements related to the PCB management plans and maintenance of PCB “logbooks”.

Conclusion 5: Although the Montenegrin legislation is well advanced and generally compliant with the international regulations (the Stockholm Convention and the EU directive on management of PCBs) and substantive progress has been achieved in updating the inventory of PCB waste, effectiveness of enforcement of the legislation is critical for the success of management and progress on PCB phase-out in Montenegro.


6.

Concise finding 6: Through the revision of the Law on Waste Management (on-going at the MTR stage), the Government has signalled an intention to postpone the legal obligation for phasing-out the use of PCB-containing equipment by 5 years from the current deadline of 2020 to a new deadline of 2025.

Conclusion 6: Postponement of the legal obligation for PCB phase-out could diminish the commitment to early action as the PCB holders may decide to push back their plans for PCB phase-out well beyond the completion date of the current project. Therefore, this motion raises concerns as to whether the project will be able to provide assistance in phasing-out the planned amounts of PCB equipment and waste during the implementation period of the project that will end in 2021.


7.

Concise Finding 7: Despite strong linkages with MoSDT and the two principal PCB holders, there is lack of connections to supporting stakeholders that are indirectly affected by the project activities.

Conclusion 7: Insufficient connections with wider circle of stakeholders could limit the general support for the intervention, especially in cases when advocacy or policy change are needed to increase the level of priority given to the PCB-related issued by the authorities.


8.

 Concise Finding 8: There is a low level of awareness of the PCB issue at academic institutions and civic organizations.

Conclusion 8: One of the reasons that PCBs are not immediately perceived as a hazard by the common public is low level of involvement of institutions of higher education and NGOs in the national PCB debate. Consequently, the issue of PCBs is very often given a low priority by the authorities.


9.

Concise Finding 9: Although the 2014-2019 NIP called for establishment of a system for collecting data on use of PCBs in the industry of plastics, coatings, paints and varnishes, as well as paints in construction, no activities in this regard have been conducted to date.

Conclusion 9: Montenegro has sufficient capacities for sampling and data analysis of caulking and paints used in building construction. This could become a foundation for eventual preparation of future activities on measuring PCB exposure levels for compliance with health-based exposure limits.


10.

Concise Finding 10: The results framework in the Project Document contains several inconsistencies related to objective and output indicators and the indicator target values for measurement of the project performance.

Conclusion 10: Inconsistencies in the project results framework obstruct monitoring and evaluation of the project performance.


11.

Concise Finding 11: The Project Document identifies difficulties related to the accounting of the in-kind support as a risk for the periodic assessment of co-financing and further stipulates that the above risk will be mitigated by establishing clear accounting mechanisms and rules for the in-kind co-financing at project inception. However, no such rules have been established.

Conclusion 11: Low amounts of in-kind co-financing reported by CEDIS and Uniprom-KAP and absence of in-kind co-financing data from MoSDT indicate that not all in-kind co-financing has been properly calculated and reported to PMU. This deficiency will hinder rigorous assessment of the parallel financing at the terminal evaluation.


12.

Concise Finding 12: The project document contains a risk matrix composed of the risk description and type, assessment of risk impacts and probability, related mitigation measures, as well as owners of each identified risk. The evaluators found the initial identification of risks and mitigation measures reasonable and sufficiently detailed. However, the periodic risk reassessment did not identify few critical risks that became apparent in the course of the project implementation.

Conclusion 12: Critical risk management is a standard part of the annual PIRs and periodic re-assessment of a risk management plan by both PMU and RTA is fundamental to the project’s implementation and success. Labelling a risk as critical provides an important alert to the project implementation that facilitates development of timely and effective risk mitigation measures.


Recommendations
1

Recommendation 1: PMU should initiate the analysis of the technical and economic feasibility of disposal of low-concentration PCB waste as a matter of the highest priority and investigate the legislative requirements and timelines necessary for securing relevant permits for different technology options.

2

Recommendation 2: PPMU in cooperation with PSC should develop a road map for continued coordination of PCB management in the country, including consideration of temporary institutionalization of PSC beyond the project completion date.

3

Recommendation 3:  PMU in cooperation with CEDIS, and other owners of the recently identified potentially PCB-contaminated equipment and CETI should initiate sampling and analysis of this equipment, including capacitors owned by CEDIS and transformers owned by other entities including the so called “unknown owners” in order to establish the amounts of PCB-contaminated equipment and waste for disposal or decontamination.

4

Recommendation 4: PMU should solicit necessary external expertise for assessment of available technological and financial options in order to determine feasible alternatives for decontamination or disposal of the special transformers at Uniprom-KAP.

5

Recommendation 5:  PMU should ensure provision of international expertise in enforcement of PCB legislation for hands-on training of the national environmental inspectors.

6

Recommendation 6: PMU in collaboration with MoSDT should consider elaboration of a proposal for legal and financial incentives to encourage the PCB holders to take early actions for phase out of the in-service electrical equipment well in advance of the 2025 deadline. In addition, the PMU in collaboration with CEDIS and Uniprom-KAP should consider introducing presentation of maintenance plans of online PCB equipment at PSC meetings and discuss timelines for replacement and disposal of online PCB equipment well before the project ends in 2021.

7

Recommendation 7: PMU in cooperation with MoSDT should ensure cooperation with the on-going research project on health impact of PCBs that is being implemented by the Montenegrin Institute for Public Health. The cooperation should focus on monitoring PCB health impacts for workers with electrical equipment and communities living in the neighbourhood of the temporary PCB storage facilities.

8

Recommendation 8: PMU in cooperation with the main PCB holders should consider practical involvement of students of higher education in activities on PCB management, for example through participation of students in the preparation of the next export shipment of PCB waste.

9

Recommendation 9: PMU in cooperation with MoSTD and other relevant governmental agencies should consider pilot testing on sampling and analysis of PCBs in caulk, glazing and painting materials in older buildings.

10

Recommendation 10: PMU should consider a revision of the project results framework to ensure its consistency and full compliance with the principles of the results-based management.

11

Recommendation 11: PMU in cooperation with MoSDT and the two major PCB holders should develop and agree clear rules for accounting of the in-kind contributions to the project.

12

Recommendation 12: PMU should conduct a thorough reassessment of the project risks after the MTR stage and ensure that critical risks are properly identified and addressed in the Critical Risk Management section of the annual PIRs together with the corresponding assessment from the side of RTA.

1. Recommendation:

Recommendation 1: PMU should initiate the analysis of the technical and economic feasibility of disposal of low-concentration PCB waste as a matter of the highest priority and investigate the legislative requirements and timelines necessary for securing relevant permits for different technology options.

Management Response: [Added: 2019/12/31] [Last Updated: 2020/01/03]

Havin in mind that work on development of PCB inventory is in the final stage we have enough information on PCB quantities present in the country. Following the recommendation, the PMU start with this activity. 

Key Actions:

Key Action Responsible DueDate Status Comments Documents
Analysis of the technical and economic feasibility of disposal of low-concentration PCB
[Added: 2019/12/31]
PMU 2020/03 Initiated
2. Recommendation:

Recommendation 2: PPMU in cooperation with PSC should develop a road map for continued coordination of PCB management in the country, including consideration of temporary institutionalization of PSC beyond the project completion date.

Management Response: [Added: 2019/12/31] [Last Updated: 2019/12/31]

The PMU will initiate discussion on this with the PSC and propose development of PPP model with the business plan until 2025, in order to have clear strategy for the establishment of such entity. Since the new Law on PPP is still pending adoption, we will guide our work with the final draft of this Law.

Key Actions:

Key Action Responsible DueDate Status Comments Documents
Discuss this on PSC meeting
[Added: 2019/12/31]
PMU 2020/01 Overdue-Initiated
Engage consultant to work on the PPP model and business plan
[Added: 2019/12/31]
PMU 2020/05 Initiated
Implement proposed and accepted model of PPP
[Added: 2019/12/31]
PMU, PSC 2021/12 Not Initiated
3. Recommendation:

Recommendation 3:  PMU in cooperation with CEDIS, and other owners of the recently identified potentially PCB-contaminated equipment and CETI should initiate sampling and analysis of this equipment, including capacitors owned by CEDIS and transformers owned by other entities including the so called “unknown owners” in order to establish the amounts of PCB-contaminated equipment and waste for disposal or decontamination.

Management Response: [Added: 2019/12/31] [Last Updated: 2019/12/31]

PMU initiated discussion with CEDIS and they agreed to collect and provide us with data on this equipment. After that on PSC meeting, we will decide on how to proceed since during the sampling of the oil equipment can be damaged and we need to know who will be responsible to resolve the issue if occur.

 

Key Actions:

Key Action Responsible DueDate Status Comments Documents
CEDIS to collect data on this equipment
[Added: 2019/12/31]
PMU 2020/01 Overdue-Initiated
Discuss on PSC meeting on how to proceed with this
[Added: 2019/12/31]
PMU, PSC 2020/03 Initiated History
Execute the decision of PSC
[Added: 2019/12/31]
PMU 2020/05 Not Initiated
4. Recommendation:

Recommendation 4: PMU should solicit necessary external expertise for assessment of available technological and financial options in order to determine feasible alternatives for decontamination or disposal of the special transformers at Uniprom-KAP.

Management Response: [Added: 2019/12/31] [Last Updated: 2019/12/31]

Project already has international expert on contract but if project partners express their interest that someone other than this expert do the assessment, we will solicit other expertise. This will also be discussed at the PSC meeting.

Key Actions:

Key Action Responsible DueDate Status Comments Documents
Prepare ToR for the service
[Added: 2020/01/03]
PMU 2020/02 Not Initiated
Select expert
[Added: 2020/01/03]
PMU 2020/03 Not Initiated
Development of the assessment
[Added: 2020/01/03]
PMU 2020/05 Not Initiated
Actions to be taken based on the recommendations from the Assessment
[Added: 2020/01/03]
PMU 2021/12 Not Initiated
5. Recommendation:

Recommendation 5:  PMU should ensure provision of international expertise in enforcement of PCB legislation for hands-on training of the national environmental inspectors.

Management Response: [Added: 2019/12/31] [Last Updated: 2020/01/03]

PMU initiated discussion with the national environmental inspection and will intensify the work on building their expertise.

Key Actions:

Key Action Responsible DueDate Status Comments Documents
Prepare plan for the trainings
[Added: 2020/01/03]
PMU 2021/01 Initiated
Implementation of the plan
[Added: 2020/01/03]
PMU 2021/12 Not Initiated
6. Recommendation:

Recommendation 6: PMU in collaboration with MoSDT should consider elaboration of a proposal for legal and financial incentives to encourage the PCB holders to take early actions for phase out of the in-service electrical equipment well in advance of the 2025 deadline. In addition, the PMU in collaboration with CEDIS and Uniprom-KAP should consider introducing presentation of maintenance plans of online PCB equipment at PSC meetings and discuss timelines for replacement and disposal of online PCB equipment well before the project ends in 2021.

Management Response: [Added: 2019/12/31] [Last Updated: 2020/01/03]

The recommendation on incentives will be further investigated and discussed vit the PSC and the MoSDT.

The part about presentation of maintenance plans on PSC meetings will be implemented on the meetings that will be organized at the beginning of 2020.

Key Actions:

Key Action Responsible DueDate Status Comments Documents
Discuss this recommendation on PSC meeting
[Added: 2020/01/03]
PM, PSC 2020/06 Initiated History
7. Recommendation:

Recommendation 7: PMU in cooperation with MoSDT should ensure cooperation with the on-going research project on health impact of PCBs that is being implemented by the Montenegrin Institute for Public Health. The cooperation should focus on monitoring PCB health impacts for workers with electrical equipment and communities living in the neighbourhood of the temporary PCB storage facilities.

Management Response: [Added: 2019/12/31] [Last Updated: 2020/01/03]

CO already started with the formalization of the cooperation with the Institute for Public Health (IPH). This recommendation will be fully implemented.

Key Actions:

Key Action Responsible DueDate Status Comments Documents
Prepare and sign cooperation agreement with IPH
[Added: 2020/01/03]
PMU 2020/02 Initiated
Monitoring of PCB impact on health
[Added: 2020/01/03]
PM, IPH 2020/06 Initiated
8. Recommendation:

Recommendation 8: PMU in cooperation with the main PCB holders should consider practical involvement of students of higher education in activities on PCB management, for example through participation of students in the preparation of the next export shipment of PCB waste.

Management Response: [Added: 2019/12/31] [Last Updated: 2020/01/03]

Ensure participation of students

Key Actions:

Key Action Responsible DueDate Status Comments Documents
Ensure participation of students in next export of PCB waste
[Added: 2020/01/03]
PMU 2021/12 Not Initiated
9. Recommendation:

Recommendation 9: PMU in cooperation with MoSTD and other relevant governmental agencies should consider pilot testing on sampling and analysis of PCBs in caulk, glazing and painting materials in older buildings.

Management Response: [Added: 2019/12/31] [Last Updated: 2020/01/03]

This activity is not part of the PCB project that we were approved to implement. We will investigate the willingness of the Government to do some investigation to determine is there a PCB in caulk, glazing and painting materials in older buildings. This could be considered as an idea for another chemicals project.

This activity needs to be approved by the donor in order to be financed from already approved financial resources.

Key Actions:

Key Action Responsible DueDate Status Comments Documents
Discuss the issue on the PSC and agree on how to proceed with it.
[Added: 2020/01/03]
PM, PSC 2020/02 Not Initiated
10. Recommendation:

Recommendation 10: PMU should consider a revision of the project results framework to ensure its consistency and full compliance with the principles of the results-based management.

Management Response: [Added: 2019/12/31] [Last Updated: 2020/01/03]

The project was designed based on a number of years of implementation of similar programmes. We will review results framework again and decide on the PSC the proposed revision.

Key Actions:

Key Action Responsible DueDate Status Comments Documents
Review the results framework and discuss to need to change it
[Added: 2020/01/03]
PMU, PSC 2020/06 Not Initiated
11. Recommendation:

Recommendation 11: PMU in cooperation with MoSDT and the two major PCB holders should develop and agree clear rules for accounting of the in-kind contributions to the project.

Management Response: [Added: 2019/12/31] [Last Updated: 2020/01/03]

This recommendation will be implemented, and all in-kind contribution will be recorded.

Key Actions:

Key Action Responsible DueDate Status Comments Documents
Develop clear rules for accounting the in-kind contributions
[Added: 2020/01/03]
PMU 2020/02 Initiated
Collect data on in-kind co-financing from the partners
[Added: 2020/01/03]
PMU 2021/12 Not Initiated
12. Recommendation:

Recommendation 12: PMU should conduct a thorough reassessment of the project risks after the MTR stage and ensure that critical risks are properly identified and addressed in the Critical Risk Management section of the annual PIRs together with the corresponding assessment from the side of RTA.

Management Response: [Added: 2019/12/31] [Last Updated: 2020/01/03]

The PMU will do the reassessment of the project risks and continue to follow it on a quarterly basis. Every risk that is considered to be critical will be reported in the annual PIR.

Key Actions:

Key Action Responsible DueDate Status Comments Documents
Reassessment of project risks
[Added: 2020/01/03]
PMU 2020/02 Initiated
Quarterly follow up of the risks
[Added: 2020/01/03]
PMU 2021/12 Initiated

Latest Evaluations

Contact us

1 UN Plaza
DC1-20th Floor
New York, NY 10017
Tel. +1 646 781 4200
Fax. +1 646 781 4213
erc.support@undp.org