Mid-term evaluation for Comprehensive Environmentally Sound Management of PCBs in Montenegro

Report Cover Image
Evaluation Plan:
2017-2021, Montenegro
Evaluation Type:
Mid Term Project
Planned End Date:
12/2019
Completion Date:
12/2019
Status:
Completed
Management Response:
Yes
Evaluation Budget(US $):
20,000

Share

Document Type Language Size Status Downloads
Download document 5562 MTR_Report_MNE.pdf report English 1299.62 KB Posted 425
Download document UNDP-GEF-MTR-TOR-PCB Montenegro International.docx tor English 100.72 KB Posted 200
Download document UNDP-GEF-MTR-TOR-PCB Montenegro Local.docx tor English 101.13 KB Posted 188
Title Mid-term evaluation for Comprehensive Environmentally Sound Management of PCBs in Montenegro
Atlas Project Number: 00088794
Evaluation Plan: 2017-2021, Montenegro
Evaluation Type: Mid Term Project
Status: Completed
Completion Date: 12/2019
Planned End Date: 12/2019
Management Response: Yes
Focus Area:
  • 1. Environment & Sustainable Development
  • 2. Others
Corporate Outcome and Output (UNDP Strategic Plan 2014-2017)
  • 1. Output 1.3. Solutions developed at national and sub-national levels for sustainable management of natural resources, ecosystem services, chemicals and waste
SDG Goal
  • Goal 9. Build resilient infrastructure, promote inclusive and sustainable industrialization and foster innovation
SDG Target
  • 9.4 By 2030, upgrade infrastructure and retrofit industries to make them sustainable, with increased resource-use efficiency and greater adoption of clean and environmentally sound technologies and industrial processes, with all countries taking action in accordance with their respective capabilities
Evaluation Budget(US $): 20,000
Source of Funding: GEF
Evaluation Expenditure(US $): 20,000
Joint Programme: No
Joint Evaluation: No
Evaluation Team members:
Name Title Email Nationality
Dalibor Kysela Mr dkysela@gmx.at CZECH REPUBLIC
Sne┼żana Marstijepovic Ms snezana.marstijepovic@gmail.com
GEF Evaluation: Yes
GEF Project Title: Comprehensive Environmentally Sound Management of PCBs in Montenegro
Evaluation Type: Mid-term Review
Focal Area: Persistent Organic Pollutants
Project Type: FSP
GEF Phase: GEF-6
GEF Project ID: 9045
PIMS Number: 5562
Key Stakeholders: Government, Institutions
Countries: MONTENEGRO
Lessons
1.

On progress towards the project objective and outcomes

The technical and economic assessment of cost-effectiveness of various technology options will be a complex exercise and will require considerable time for completion. In case the dehalogenation technology is identified as the preferable option, there will be additional sizeable time period required to obtain all necessary legal permits for operation and complete procurement, delivery and commissioning of equipment.  

There is a risk that the protracted deficiency of the PPP legislation could negatively affect the environmentally sound PCB management beyond the duration of the current project. The Project Steering Committee should be considered as an interim body for coordination and oversight of PCB management in Montenegro until an effective PPP mechanism will have been created and institutionalized to assume this responsibility.

The information in the 2019 NIP update on amounts of PCB waste and PCB contaminated equipment in use in Montenegro suggests there is still a sizeable number of electrical equipment in the country that is potentially contaminated with PCBs where the level of PCB content has not yet been ascertained, and it is expected that throughout implementation of this project all PCB contaminated equipment and waste will be identified and the national PCB inventory will be adequately updated.

A thorough assessment of available options for handling the special transformers at Uniprom-KAP will be necessary to take into account advantages and disadvantages of total replacement of the special PCB transformers as well as alternatives to the total replacement in order to optimize the associated costs.

Although the Montenegrin legislation is well advanced and generally compliant with the international regulations (the Stockholm Convention and the EU directive on management of PCBs) and substantive progress has been achieved in updating the inventory of PCB waste, effectiveness of enforcement of the legislation is critical for the success of management and progress on PCB phase-out in Montenegro.

Postponement of the legal obligation for PCB phase-out could diminish the commitment to early action as the PCB holders may decide to push back their plans for PCB phase-out well beyond the completion date of the current project. Therefore, this motion raises concerns as to whether the project will be able to provide assistance in phasing-out the planned amounts of PCB equipment and waste during the implementation period of the project that will end in 2021.

Insufficient communication with wider circle of stakeholders could limit the general support for the intervention, especially in cases when advocacy or policy change are needed to increase the level of priority given to the PCB-related issued by the authorities.

One of the reasons that PCBs are not immediately perceived as a hazard by the common public is low level of involvement of institutions of higher education and NGOs in the national PCB debate. Consequently, the issue of PCBs is very often given a low priority by the public at large.

Although the 2014-2019 NIP called for establishment of a system for collecting data on use of PCBs in the industry of plastics, coatings, paints and varnishes, as well as paints in construction, no activities in this regard have been conducted to date. Montenegro has sufficient capacities for sampling and data analysis of caulking and paints used in building construction. This could become a foundation for eventual preparation of future activities on measuring PCB exposure levels for compliance with health-based exposure limits.


2.

On project implementation and reporting

Inconsistencies in the formulation of achievement target indicators in the project results framework obstruct monitoring and evaluation of the project performance.

Low amounts of in-kind co-financing reported by CEDIS and Uniprom-KAP and absence of in-kind co-financing data from MoSDT indicate that not all in-kind co-financing has been properly calculated and reported to PMU. This deficiency will hinder rigorous assessment of the parallel financing at the terminal evaluation.

Critical risk management is a standard part of the annual PIRs and periodic re-assessment of a risk management plan by both PMU and RTA is fundamental to the project’s implementation and success. Labelling a risk as critical provides an important alert to the project implementation that facilitates development of timely and effective risk mitigation measures.

 


Findings
1.

Project Design

The Montenegro PCB project is aligned with The National Strategy for Sustainable Development (NSSD) until 2030 that represents a strategic framework for the transposition of the UN sustainable development goals (SDGs) and its indicators into the national context. The Strategic Goal 4.3 of the related Action Plan calls for improvements in the waste management through circular economy approaches and the Strategic Goal 5.2 aims at strengthening environmental management by improving implementation of environmental protection instruments. The project is also linked with the National Chemical Management Strategy 2019-2022, and the related Action Plan 2019-2022 for the purpose of ensuring adequate management of chemicals, production or import until disposal. The overall strategic goal to be achieved by the Strategy is: “…. to establish a chemical management system that provides a high level of protection to human health and the environment, as well as improving free exchanges with the EU countries and other countries while fostering the competitiveness of the Montenegrin economy through introduction of safer chemicals and technological processes.” The project addresses PCB waste from several sources that are listed in the National Plan on Waste Management 2015-2020 that includes, inter alia, quantities of waste equipment containing PCBs and export data on PCB waste (types, quantities and origin of waste exported).


Tag: Waste management Global Environment Facility fund National Donor relations Partnership Programme/Project Design Agenda 2030

2.

Results Framework/Logframe

The evaluators performed critical analysis of the project results framework in order to establish whether it has the necessary elements and whether it enables measurement of success and progress to success. The formulation of the Montenegro PCB project started with preparation and approval of the Project Identification Form (PIF) in June 2015 that outlined the objective, outcomes and outputs. This document also served as a basis for formulation of objectives and outputs of the Project Document, approved in October 2016. Both documents have been prepared through extensive consultations with relevant governmental and industrial stakeholders. The PIF contains results framework composed of three substantive components, eight outcomes and twenty outputs, the logframe in the Project Document is composed of three components/outcomes and 8 outputs. It appears that there was a mismatch in PIF where the outputs were in fact activities. Therefore, the results framework in PD provides a better description of the project results hierarchy and structure. The three project components address the three most important areas for intervention for a PCB project, namely i) technical and institutional capacity building for PCB management, ii) establishment of an ESM system of PCBs, and iii) practical implementation of the ESM system.


Tag: Waste management Gender Mainstreaming Rule of law Awareness raising Capacity Building Institutional Strengthening Technical Support

3.

Progress Towards Results

Progress towards outcomes analysis

The information presented in this section has been sourced from the annual Project Implementation Reports (PIR) for 2018 and 2019, supplemented with information collected during the MTR mission to Montenegro.

Output 1.1: A training workshop was organized on 8-9 October 2017 with participation of 18 trainees (7 women, 11 men) from the governmental environmental protection and inspection authorities, accredited laboratories for sampling and analysis and the Ministry of Sustainable Development and Tourism (Directorate for Environmental Protection, Directorate for Waste and Communal Affairs). The training covered several phases of a PCB management cycle – identification, sampling and analysis of potentially contaminated equipment, safe temporary storage, transportation and final disposal of PCBs. A set of guidance documents on Environmentally Sound Management of PCBs was developed with incorporated requirements from the Stockholm and Basel conventions, EU regulations on POPs and PCBs management, and international guiding elements on Best Available Technology (BAT) and Best Environmental Practices (BEP) for PCB treatment and disposal. The set consists of the following 7 parts: 1. PCBs - Chemical properties, application and impact on human health and environment 2. Identification of PCBs 3. Use and maintenance of PCB-containing equipment, gradual shutdown, decommissioning and replacement 4. Packing and storage of PCB-containing waste 5. Transport of PCB waste 6. Emergency and security procedures 7. Disposal and decontamination of PCB-containing equipment and waste


Tag: Waste management Effectiveness Partnership Capacity Building Technical Support

4.

Output 1.2: The project has supported development of the following legislative acts: • Rulebook on methods for testing hazardous waste (adopted on 7 June 2018); • Rulebook on handling the waste and equipment containing PCBs (adopted on 17 October 2018); • Decree on the parameters and conditions for waste storage (draft in the process towards adoption); The first rulebook was developed to help with the national PCB inventory process through introducing the use of the fast screening method for testing of PCB oils8 . A set of checklists was developed for the Environmental Inspection to be used during site inspections for PCB-containing equipment and waste. The checklists will standardize the control function of environmental inspectors related to identification, operation, stocktaking, phasing out, packing, transportation, disposal of the PCB and will also ensure sustainable transfer of knowledge to new inspectors in case of personnel rotation.


Tag: Waste management Effectiveness Rule of law

5.

Summary Assessment of Outcome 1:

The set of guidance documents on ESM and related training workshops have substantially increased the national capacities of the environmental inspectors as well as operators from the two main PCB holders. The checklists developed for the environmental inspectors contributed to standardization of execution of the environmental inspections related to PCBs through on-the-job training. The project was instrumental in supporting development of new legislative rulebooks. Adoption of the two new rulebooks finally triggered the implementation of PCB inventory (see achievements under Outcome 2 below). The study tour to the FYR Macedonia group had discussions on the company’s motive to participate in that similar project; the activities covered by the company regarding the treatment of the PCB-containing transformers; how did the project contribute towards enhancing the company's knowledge and capacities for sound PCB management; how did the project achievements enable the company to comply with the national regulation; how does the noncombustion disposal option suit the company’s needs and the experience of the company related to the cooperation with the PCB treatment facility installed at company Rade Koncar; are there any tests on the PCB presence performed on the decontaminated transformers returned back in operation.


Tag: Waste management Effectiveness National Rule of law Partnership Project and Programme management

6.

Output 2.1: There was a delay in the sampling of the electrical equipment due to the fact that the national legislation did not recognize the possibility to do fast-screening tests on PCB presence but allowed only for an expensive laboratory instrumental analysis by gas chromatography. In order to reduce the cost of the PCB analyses required for the inventory, the project assisted in the development of the Rulebook on Methods for Testing Hazardous Waste that introduced the use of the Clor-n-Oil for fast screening of the presence of PCBs. Adoption of the Rulebook in June 2018 prompted CEDIS to start with the sampling and testing process of their power equipment in addition to the work that has been conducted through the project. The national inventory of PCB equipment and waste has been entrusted to the Ecotoxicological Research Center (Centar za ekotoksikologiško ispitivanje – CETI). The latter organization holds accreditation according to ISO/IEC 170259 for sampling and analysis of PCBs in transformer oil and various environmental matrices. The contract with CETI envisages analysis of 4,000 samples and implies the use of fast screening tests as well as the use of a gas chromatographic (GC) method for confirmation of PCB contamination levels (contamination above 50 ppm is classified as PCB waste) and determination of PCB concentration in transformer oil and other matrices.


Tag: Waste management Rule of law Technology Technical Support

7.

Output 2.2: Due to the incomplete data on PCB presence in Montenegro, the work on the National PCB Management plan has been postponed until the full data from the inventory process becomes available. Upon initial consultations with the relevant authorities, CEDIS electric power distribution company has commenced the work on a draft PCB management plan. Further progress depends on the inventory of PCB equipment which has not been completed. Uniprom-KAP already has a draft PCB management plan but has also been waiting for the completion of their PCB inventory to update the plan. Output 2.3: The Law on Public Private Partnership (PPP) in Montenegro has not been enacted yet and currently is in the process of public hearings. Once approved, the Law will establish a clear institutional framework for developing, procuring and implementing PPPs in Montenegro. Because of the uncertainty regarding the actual approval of the Law, the project team should consider and commence interim activities for achievement of the end-of-project targets.


Tag: Waste management Effectiveness National Rule of law Partnership

8.

Summary Assessment of Outcome 2:

After the initial lag period caused by the legislative obstacles hindering use of the fast screening methods for testing of PCB contamination, the sampling and analyses had gathered a staggering pace in the period from April 2018 to August 2019. After 18 months the total number of samples analysed has already exceeded the original project target of 3,000 samples. This is a remarkable achievement given the fact that although the project developed procedures for sampling both online (active) and offline (disconnected) equipment and provided related training for the operators, the internal CEDIS procedures do not allow sampling of online transformers. Therefore, the sampling occurred at a rate determined by CEDIS yearly maintenance plans that allow for equipment to be temporarily disconnected and samples collected. The fact that mid-way through the project the number of analyzed transformer oil samples has already outstripped the original final target is an achievement worth mentioning. The reason is that a majority of samples were taken from electrical equipment still in use and that sampling of the on-grid equipment is a complex task requiring a significant coordination effort (such as coordination with maintenance schedule of the equipment).


Tag: Waste management Effectiveness Partnership Technology Policy Advisory

9.

Output 3:1: For a new PCB storage at CEDIS, the project funded preparation of the technical documentation principal design according to the requirements of national and international standards. The documentation was prepared by the architectural designer ‘Arhiplan CG’ doo Podgorica. and independent technical revision of the same documentation was performed by another company specialized in technical supervision ‘Urbi.Pro’ doo Podgorica. Moreover, PSC approved to cover about 30% of the construction costs of the storage from the project budget. UNDP and CEDIS could not sign a contract for the construction of the storage on the principle of co-financing since in such case the entire investment (including the 70% share by CEDIS) would be considered by the CEDIS's financial department as a donation, and due to the possibility of CEDIS to receive a partial investment refund from the National Energy Regulatory Agency In order to mitigate the situation, the last PSC meeting in July 2019 advised to break the construction into components and use the contribution from the project for one of the components, namely procurement of horizontal mechanization. The selection of a concrete component is pending on progress in the construction and dynamics of other components of the project, namely eventual placement of a mobile PCB dehalogenation unit. Until commissioning of the new storage, CEDIS continues to rent a storage space at Hemosan located in the port of Bar.


Tag: Energy Effectiveness National Partnership Technical Support

10.

Output 3.2: Implementation of this output is pending on completion of the national inventory of PCBs. Based on the completed inventory, the quantities of equipment with low to medium level of PCB contamination (up to 2,000 ppm) will be established that can be treated by the dehalogenation technology11. Once the quantities of low PCB-contaminated equipment are known, the Project Document envisages commissioning analysis of technical and economic feasibility of the following options:

• Procurement of the dehalogenation technology (stationary or mobile) • Dehalogenation of low-contaminated electric equipment either through shipment to established facilities abroad or treatment in the country with a rented dehalogenation equipment; • Drainage of the PCB-contaminated oil and shipment abroad for final disposal


Tag: Waste management Effectiveness National Technology

11.

Output 3.3: Tender for disposal of concentrated PCB waste was closed in June 2018 and the contract was awarded to the Greek company ‘Polyeco S.A.’ with experience from similar operations in the CEE/NIS region (Georgia, Kosovo) and worldwide. For all activities on packing/repacking (handling, oil pumping, draining of power equipment) of the PCB waste, Polyeco sub-contracted the local waste management company Hemosan. The preparatory work was conducted in the period November - December 2018. During the packaging of PCB waste, all safety measures were followed in line with established international benchmarks, as follows: - the needed UN-approved packages, labels, PPE, tools and equipment were delivered on site and the personnel responsible for the packing and draining activities was trained by Polyeco; - the emergency equipment was readily available on site (absorbents, solvents, fire extinguishers); - the site was divided into different zones, in line with PCB management guidelines established by the Basel convention on transboundary movement of hazardous wastes, separated by red and white plastic bands: contaminated zone (i.e. packing area, where packing/repacking and draining works take place); buffer zone (temporary storage of the packed drums and containers awaiting loading into the truck/sea container) and clean zone (storage of personal protection equipment (PPE), drum handling equipment and resting place and area where the trucks/sea containers are to be loaded with the packed PCB waste); - all workers involved in the packing activities were provided with the personal protection equipment (PPE), namely overalls, masks, gloves, safety glasses. 


Tag: Waste management Local Governance Technology Technical Support

12.

Summary assessment of Outcome 3: The above summary of findings shows a mixed picture of progress under this project component. The most visible achievement has been recorded for Output 3.3 where the first shipment of PCB waste constitutes about 35% of the end-of-project disposal target for the PCB equipment waste. Limited progress has been achieved under Output 3.1 on the technical design documents for construction of a new PCB storage for CEDIS and for upgrade of the existing PCB storage at Uniprom-KAP, respectively. The work on the technical and economic assessment of options for addressing the low-concentrated PCB waste under Output 3.2 has not started as it has reportedly been pending on completion of the PCB inventory. Based on the above findings, the progress towards achievement of the end-of-project targets for Outcome 3 is rated Moderately Satisfactory (MS).


Tag: Waste management Effectiveness Public administration reform

13.

Summary assessment of progress towards the Project Objective: The Outcome ratings above are based on the premise that the project has to be completed within the officially approved implementation period, i.e. by the end of 2021. Hence the rating scores are given on the expectation whether the outcomes will or will not achieve their respective endof-project targets by the end of the approved project period. The GEF guidelines for mid-term reviews require the evaluators to provide only one overall rating for each outcome and the overall objective. Rating at the level of outputs is indicated by the colour shading of the last column in Tables 4 – 8 hence no text ratings are given at the level of outputs. From the above listed achievements under the three project substantive components it is obvious that the project has made remarkable progress towards establishing of foundations for environmentally sustainable management of PCBs in Montenegro by contributing to removal of several barriers to effective implementation of the country’s obligations under the Stockholm Convention that had been identified at the PIF/PPG stage.

At the time of MTR, the flagship deliverable of the project has been the advanced national inventory of PCB-containing electrical equipment in the format of an on-line database platform. The PCB registry helps to substantially reduce the information gap on the extent of PCB presence in the country through provision of exact information in terms of quantities and location of PCB-containing equipment. Using the already established strong national capacity for PCB analysis, the inventory also provides accurate information on the extent and level of PCB contamination of the electrical equipment. It is also worth noting that the first national registry of PCB-containing equipment, also serves as a springboard for elaboration of a National Plan for PCB Management and corporate PCB management plans to be prepared by the two principal PCB owners in the country. Collectively, the PCB management plans will ensure gradual phase-out, decommissioning and disposal of PCB-contaminated electrical equipment in line with the Stockholm and Basel Conventions.


Tag: Effectiveness National Rule of law Partnership Project and Programme management Jobs and Livelihoods Technology Capacity Building

14.

Issues to be addressed by the end of the project

The formulation of the project was based mainly on a “soft” data that was obtained through a preliminary inventory of potentially PCB-contaminated equipment carried out in 2013 by the Administration for Inspection. This survey was not exhaustive and for most of the equipment listed in the inventory traceability was not ensured and did not follow with rigorous verification of the surveyed information. Based on these rough estimates, the project was formulated with the premise that at least 700 tonnes of PCB-contaminated electrical equipment and transformer oil as well as 200 tonnes of PCB-contaminated soil would be found through the national PCB stock taking. The progress with the inventory work of the PCB-contaminated electrical equipment and PCB waste suggests that the above expected PCB waste quantities in possession of the two principal PCB holders were overestimated.

The 2019 NIP update does not contain revised information on PCB and PCB-contaminated equipment throughout the country. The first and updated NIPs includes transformers and capacitors from “unknown owners”, i.e. electrical equipment that used to be possessed by companies that had filed for bankruptcy and their assets had been frozen under the bankruptcy administration procedures (e.g. Complex of the former plant “Radoje Dakic” in Podgorica or Electrical Industry "Obod" in Cetinje,). CEDIS reported that due to some legal issues they are currently not authorized to sample and test the equipment under administration for PCB contents, however they are preparing a list of all transformers and capacitors from “unknown owners” and they will submit it to the PCB project team by the end of this year. 


Tag: Waste management Challenges National Health Sector Partnership Programme/Project Design Project and Programme management Technology Urbanization

15.

This intervention is expected to start with development of a business plan with sustainability considerations in support of the PPP approach. The plan should be verified and amended based on the experience gathered after one initial year of the project’s activities. Due to the complexity of the public discussion and subsequent parliamentary approval processes, it is difficult to predict whether the new law will be enacted within the implementation period of the PCB project and give thus the opportunity to conduct the planned work on this project sub-component in line with the new law. As discussed under Institutional sustainability, the protracted deficiency of the PPP legislation could be detrimental to the project’s institutional and financial sustainability. The innovative feature of this intervention combined with the protracted absence of PPP legislation and receding available time for the project implementation collectively create a serious apprehension related to attainability of this goal and suggest a need to explore alternative approaches.


Tag: Waste management Business Model Implementation Modality Partnership Sustainability

16.

The evaluators have no doubts that the technical and economic appraisal of different scenarios for destruction and/or final disposal of low-concentrated PCB waste envisaged within the Montenegro PCB project will be a complex exercise that will take into account multiple parameters, namely the country needs, nature and quantity of the PCB waste streams, prevailing economic and market conditions, as well as availability of the PCB dehalogenation technology and other PCB ultimate disposal options in the neighboring countries (Serbia and the North Macedonia). The reason for concerns is the delayed start of this sub-component and the time remaining for its implementation in the remaining period until the end of the project. According to reports from the PCB management project in Northern Macedonia, the entire process of disposal of PCB waste by the dehalogenation technology including preparation of the ToR, selection of the technology provider, procurement and installation of the equipment and finally treatment of the transformer oil took 22-24 months.


Tag: Waste management Challenges National Monitoring and Evaluation Procurement Project and Programme management Technology

17.

Project Implementation and Adaptive Management Arrangements

This section of the MTR report provides assessment of the seven components of the project implementation and adaptive management, namely management arrangements, work planning, finance and co-finance, project-level monitoring and evaluation, management of risks, stakeholder engagement, as well as reporting and communications.

Management arrangements

The PCB project is being implemented by the UNDP Country Office (CO) in Montenegro using the Direct Implementation Modality (DIM). The decision to use the DIM approach was taken in line with the 2006 SBAA, namely to the fact that the capacity of the Government to formulate and implement development interventions in the area of hazardous waste management had still been developing. Therefore, DIM was considered optimal for meticulous implementation of the project combined with elements of systematic national capacity development of relevant stakeholders. UNDP in Montenegro appears to have all the requisite capacities needed for DIM, namely a thorough understanding of the country’s needs and capabilities; a strong reputation with national and international partners, as well as proven capacities in logistics, recruitment, procurement, financial management, and reporting.


Tag: Effectiveness Change Management Communication Human and Financial resources Implementation Modality Monitoring and Evaluation Risk Management Institutional Strengthening

18.

Work planning

In consultation with the relevant stakeholders, PMU prepares results-based Annual Work Plans (AWPs) with the planned activities and related indicative timeframe under each project output for the coming year. AWPs are presented to PSC meetings for discussion and approval. The evaluators reviewed Annual Work Plans (AWPs) for the years 2017, 2018 and 2019 and found them realistic with sufficiently detailed narrative description of planned interventions. However, none of the three AWPs contained information on financial inputs earmarked for each of the planned activities. This practice appears to deviate from the standard UNDP AWP template. Although the PSC minutes proved that some financial inputs were discussed at PSC meetings, this discussion usually occurred only for financial inputs and activities that were of interest to one or more PSC members. Systematic inclusion of allocated financial inputs in line with the standard UNDP AWP format would give PSC members better insight into the project implementation and increase thus the transparency of the annual work planning. The MTR team rates the project work planning Satisfactory (S).


Tag: Implementation Modality Programme/Project Design

19.

Monitoring and evaluation

The Project Document states that the project performance monitoring and evaluation will be conducted at several levels in line with the UNDP Programme and Operations Policies and Procedures (POPP) and the UNDP Evaluation Policy. The monitoring is provided in the first instance by PMU and in the second instance by the bi-annual PSC meetings. However, the evaluators noted that there was no systematic compilation of progress data on the output and outcome indicators as agreed in the project results framework. In addition, the Project Manager and the UNDP-GEF Regional Technical Advisor compile on annual basis the GEF Project Implementation Reports (PIRs) that cover the reporting period from July (previous year) to June (current year) for each year of project implementation. Two PIRs have been provided so far, covering the periods July 2017-June 2018 and July 2018-June 2019, respectively. The evaluators found both PIRs in line with the standard GEF PIR format and with one exception (see Risk management below) to contain adequate level of details in narrative descriptions of achievements during the reporting period as well as justified ratings of progress in project implementation and of overall progress towards the project development objective.


Tag: Waste management Monitoring and Evaluation Programme/Project Design Project and Programme management

20.

Identification and management of risks

The project document contains a risk matrix composed of the risk description and type, assessment of risk impacts and probability, related mitigation measures, as well as owners of each identified risk. The last column of the matrix contains assessment of the risk status that is required to be periodically re-assessed. The evaluators consider the initial identification of risks and mitigation measures reasonable and sufficiently detailed as the matrix contains a variety of project risk types such as strategic, organizational, financial, environmental and social risks. No risk was identified in relation to the accomplishment of the co-financing that had been pledged at the project inception phase. However, as to the reporting of risks, a periodic re-assessment of the identified risks is recorded in the reports in the UNDP Atlas that are prepared by PMU. However, there were no risks reported under the paragraph Critical Risk Management in the annual project implementation reports (PIRs) prepared jointly by PMU and RTA. Critical risk management is a standard part


Tag: Waste management Implementation Modality Risk Management Technology

21.

Finance and co-finance

The tables below provide a summary of resources allocation for the project and of level of disbursement of the GEF grant funds as well as the estimated actual amount of co-finance up to MTR. The financial data in Table 8 show that as of 31 August 2019 the total disbursement of GEF grant at the MTR stage stands at US$ 1,009,428.05 that gives the rate of implementation of the GEF grant 28.43%. As the project has already entered the second half of the implementation period, the outstanding balance of US$ 2,540,571.95 represents a substantial budget available for the remaining 28 months of the project implementation period. The rates of implementation for the individual project components reflect the achieved progress towards the end-of-project targets described above (see Tables 4-7 and related text). The relatively high implementation rate for the PCB inventory (Component 2, 72.65%) reflects the fact that the number of the samples analysed for PCB contents has exceeded the end-of-project target and that the work on completion of the PCB inventory is in the final stages. The relatively low rate of implementation for ESM of PCBs (Component 3, 22.16%) exhibits the reality that the project has addressed only about 35% of the planned amount of PCB waste (248 tonnes out of the planned 700 tonnes) and that the assistance to the construction/upgrade of the two PCB storage facilities has barely commenced. The low implementation rate for the capacity strengthening (Component 1, 35.65%) suggests that the mid-term targets for this component could have been underestimated. Overall, the financial data from Table 14 clearly highlight the need to significantly increase the rate of implementation for the remaining period of the project.


Tag: Waste management Efficiency Resource mobilization Human and Financial resources Implementation Modality

22.

Stakeholder engagement

The original Project Document presents a stakeholder analysis through a table including the stakeholders’ names and their respective roles. However, this list is rather generic and does not comprehend the differing positions of the identified stakeholders, namely the distinction between core (involved) and supporting or peripheral stakeholders. MoSDT, CEDIS and Uniprom.KAP as the primary (core) stakeholders had been involved during the design phase of the project through baseline surveys and consultation workshops/meetings. The extensive core stakeholder engagement has been continued during the project implementation to date, mainly throughout the biannual PSC meetings. The initial intent to involve also supporting stakeholders through their membership in PSC proved to be hindering the effectiveness of PSC functionality. Therefore, at the 5th PSC meeting in July 2019 the PSC membership had been reduced to include only representatives of the core stakeholders. From the project inception, there have been no links to peripheral stakeholders (such as NGOs, academia, the public at large) with the exception of an initial invitation of the NGO Ozon to participate on PSC that was declined. The decision to reduce the PSC membership in reality means that the project has no links to supporting stakeholders (such as other ministries and governmental agencies).


Tag: Waste management Implementation Modality Multilateral Partners Partnership Awareness raising Technical Support

23.

Reporting and communication

Reporting during project implementation helps to identify potential issues that may endanger the project’s capacity to achieve its development objectives. Reporting also helps to make informed decisions, provides valuable information for project evaluation, and provides lessons to be learnt for future projects. Effective and timely communication between the PMU and the core stakeholders is a key element in that respect. In addition to the review of project-related reports, the MTR team obtained through interviews information about numerous meetings PMU held with the project core stakeholders. The evaluators also reviewed the project web page that has been operational since 2018. It appears that reporting and communication within the core stakeholder group has been extensive, particularly in the first year of the implementation when some of the core stakeholders did not show full commitment to the project interventions. However, there is a room for improvement in communication outside the group of core stakeholders. Lack of effective communication with secondary stakeholder groups could result in relatively low level of public awareness and understanding of the PCB management-related issues as well as of health and environmental impacts of PCBs. This could be detrimental to the social and environmental impact assessments in case the project will engage in the establishment of a PCB dehalogenation facility or rental of PCB dehalogenation services for decontamination of low-contaminated PCB oil and for PCB-polluted site assessments.


Tag: Waste management Change Management Communication Project and Programme management

24.

Mainstreaming

During the implementation so far, the project team made a concerted effort for ensuring and recording women's involvement in the project, namely participation of women in the project trainings. The statistics about the participants in the training component at the MTR stage show a very good gender balance of the trainees. More interventions on mainstreaming will be possible after the completion of the Study on the Gender Dimension on POPs issue in Montenegro. Preparatory work for contracting a national consultant that just started at the time of MTR and will be completed in the 1st quarter of 2020.


Tag: Gender Mainstreaming National

25.

Sustainability

The sustainability is defined as continuation of benefits from an intervention after the development assistance has been completed. The important aspect here is the sustainability of results, not necessarily sustainability of the activities that had produced the results. The assessment of sustainability requires evaluation of risks that may affect the continuation of the project results. In general, the activities supported by the project have the potential to ensure long-term sustainability but with serious challenges described in the text below.

Financial sustainability

Both CEDIS and Uniprom-KAP have made sizeable co-financing commitments to the project and a sizeable portion of the commitments has been realized until the MTR stage (see Tables 9 and 10 above). While the CEDIS continued commitment due to the majority ownership by the Government appears to be secured, the situation could be different for Uniprom-KAP. Due to not fully resolved ownership issues of the company, the scope of the World Bank project on site remediation has been recently downsized and instead of financing remediation works for two selected hazardous waste sites, the remaining activities to be financed from the loan will be the preparation of design and bidding documents and technical assistance. The reduction of financial support from the World Bank loan for the site remediation coupled with the current difficult situation on the aluminium markets (expensive production inputs, primarily the electricity and raw materials, in combination of difficulties related to low trading prices of aluminium product outputs) could under certain conditions limit the company’s co-financing commitments to the PCB project. Financial sustainability of the project is rated Moderately Satisfactory (MS).


Tag: Challenges Sustainability Health Sector Human and Financial resources Risk Management Sustainability

26.

Institutional framework and governance sustainability

The work under the project is aligned with the key governmental regulatory agencies and the two major PCB holders in the country. Training provided to a number of inspectors and operators from the leading national institutions and the private sector has strengthened the already existing institutional base in the country. This together with the PCB inventory constitutes a robust foundation for good governance of the PCB management in medium to long term. However, the real institutional sustainability will be achieved only through establishment of an innovative public-private partnership for the management of PCB-contaminated equipment and waste in support of the national effort to introduce ESM of PCBs in the country (to be established under Output 2.3.). This is expected to put in place various financial mechanisms to ensure continuous decontamination of the in-service PCB-containing equipment and disposal of the highly contaminated PCB waste in line with the obligations of the Stockholm Convention. Unfortunately, the work on this output has not started yet due to absence of the law on PPPs (as described above under the Progress towards outcomes analysis). Further protraction of this legal deficiency could endanger the establishment of PPP mechanisms for management of PCBs beyond the project duration. Institutional and governance sustainability of the project is rated Moderately Satisfactory (MS).


Tag: Waste management Sustainability Rule of law Partnership Institutional Strengthening

27.

Environmental sustainability

It is critical for environmental sustainability that in the remaining period of implementation the project makes a concentrated effort on disposal and decontamination of as much as possible of PCB waste. The first batch of PCB waste sent for final disposal abroad was a massive step towards environmental sustainability since the 248 tonnes of PCB-equipment, oil and soil waste has been restricted from entering the environment. Further steps towards environmental sustainability include preparation of corporate PCB management plans and securing temporary storage facilities to safeguard PCB stockpiles before disposal, as well as adoption and enforcement of all regulatory measures developed for ESM of PCBs. Through the revision of the Law on Waste Management (on-going at the MTR stage), the Government has signalled an intention to postpone the legal obligation for phasing-out the use of PCB-containing equipment by 5 years from the current deadline of 2020 to a new deadline of 2025. Postponement of the legal obligation for PCB phase-out could diminish the commitment to early action as the PCB holders may decide to push back their plans for PCB phase-out well beyond the completion date of the current project. Therefore, this motion raises concerns as to whether the project will be able to provide assistance in phasing-out the planned amounts of PCB equipment and waste during the implementation period of the project that will end in 2021. Postponed phase-out of the PCB equipment in service could have negative environmental effects in case of leakages and/or more severe accidents related to operation and maintenance of electrical equipment. Environmental sustainability of the project is rated Moderately Satisfactory (MS).


Tag: Environment Policy Waste management Sustainability Rule of law Implementation Modality

Recommendations
1

Recommendation 1: PMU should initiate the analysis of the technical and economic feasibility of disposal of low-concentration PCB waste as a matter of the highest priority and investigate the legislative requirements and timelines necessary for securing relevant permits for different technology options.

2

Recommendation 2: PPMU in cooperation with PSC should develop a road map for continued coordination of PCB management in the country, including consideration of temporary institutionalization of PSC beyond the project completion date.

3

Recommendation 3:  PMU in cooperation with CEDIS, and other owners of the recently identified potentially PCB-contaminated equipment and CETI should initiate sampling and analysis of this equipment, including capacitors owned by CEDIS and transformers owned by other entities including the so called “unknown owners” in order to establish the amounts of PCB-contaminated equipment and waste for disposal or decontamination.

4

Recommendation 4: PMU should solicit necessary external expertise for assessment of available technological and financial options in order to determine feasible alternatives for decontamination or disposal of the special transformers at Uniprom-KAP.

5

Recommendation 5:  PMU should ensure provision of international expertise in enforcement of PCB legislation for hands-on training of the national environmental inspectors.

6

Recommendation 6: PMU in collaboration with MoSDT should consider elaboration of a proposal for legal and financial incentives to encourage the PCB holders to take early actions for phase out of the in-service electrical equipment well in advance of the 2025 deadline. In addition, the PMU in collaboration with CEDIS and Uniprom-KAP should consider introducing presentation of maintenance plans of online PCB equipment at PSC meetings and discuss timelines for replacement and disposal of online PCB equipment well before the project ends in 2021.

7

Recommendation 7: PMU in cooperation with MoSDT should ensure cooperation with the on-going research project on health impact of PCBs that is being implemented by the Montenegrin Institute for Public Health. The cooperation should focus on monitoring PCB health impacts for workers with electrical equipment and communities living in the neighbourhood of the temporary PCB storage facilities.

8

Recommendation 8: PMU in cooperation with the main PCB holders should consider practical involvement of students of higher education in activities on PCB management, for example through participation of students in the preparation of the next export shipment of PCB waste.

9

Recommendation 9: PMU in cooperation with MoSTD and other relevant governmental agencies should consider pilot testing on sampling and analysis of PCBs in caulk, glazing and painting materials in older buildings.

10

Recommendation 10: PMU should consider a revision of the project results framework to ensure its consistency and full compliance with the principles of the results-based management.

11

Recommendation 11: PMU in cooperation with MoSDT and the two major PCB holders should develop and agree clear rules for accounting of the in-kind contributions to the project.

12

Recommendation 12: PMU should conduct a thorough reassessment of the project risks after the MTR stage and ensure that critical risks are properly identified and addressed in the Critical Risk Management section of the annual PIRs together with the corresponding assessment from the side of RTA.

1. Recommendation:

Recommendation 1: PMU should initiate the analysis of the technical and economic feasibility of disposal of low-concentration PCB waste as a matter of the highest priority and investigate the legislative requirements and timelines necessary for securing relevant permits for different technology options.

Management Response: [Added: 2019/12/31] [Last Updated: 2021/01/27]

Havin in mind that work on development of PCB inventory is in the final stage we have enough information on PCB quantities present in the country. Following the recommendation, the PMU start with this activity. 

Key Actions:

Key Action Responsible DueDate Status Comments Documents
Analysis of the technical and economic feasibility of disposal of low-concentration PCB
[Added: 2019/12/31] [Last Updated: 2020/12/24]
PMU 2020/12 Completed Analysis of the technical and economic feasibility of disposal of low-concentration PCB is finalized. History
2. Recommendation:

Recommendation 2: PPMU in cooperation with PSC should develop a road map for continued coordination of PCB management in the country, including consideration of temporary institutionalization of PSC beyond the project completion date.

Management Response: [Added: 2019/12/31] [Last Updated: 2021/01/27]

The PMU will initiate discussion on this with the PSC and propose development of PPP model with the business plan until 2025, in order to have clear strategy for the establishment of such entity. Since the new Law on PPP is still pending adoption, we will guide our work with the final draft of this Law.

Key Actions:

Key Action Responsible DueDate Status Comments Documents
Discuss this on PSC meeting
[Added: 2019/12/31] [Last Updated: 2020/12/24]
PMU 2020/12 Completed This activity is approved by PSC in activity plan for 2020. History
Engage consultant to work on the PPP model and business plan
[Added: 2019/12/31] [Last Updated: 2020/12/24]
PMU 2020/12 Completed The consultant was engaged and the work on PPP model has been finalized. History
Implement proposed and accepted model of PPP
[Added: 2019/12/31]
PMU, PSC 2021/12 Not Initiated
3. Recommendation:

Recommendation 3:  PMU in cooperation with CEDIS, and other owners of the recently identified potentially PCB-contaminated equipment and CETI should initiate sampling and analysis of this equipment, including capacitors owned by CEDIS and transformers owned by other entities including the so called “unknown owners” in order to establish the amounts of PCB-contaminated equipment and waste for disposal or decontamination.

Management Response: [Added: 2019/12/31] [Last Updated: 2021/01/27]

PMU initiated discussion with CEDIS and they agreed to collect and provide us with data on this equipment. After that on PSC meeting, we will decide on how to proceed since during the sampling of the oil equipment can be damaged and we need to know who will be responsible to resolve the issue if occur.

 

Key Actions:

Key Action Responsible DueDate Status Comments Documents
Discuss on PSC meeting on how to proceed with this
[Added: 2019/12/31] [Last Updated: 2020/12/24]
PMU, PSC 2021/06 Initiated PSC still did not decide hot to approach this issue and resolve it. History
CEDIS to collect data on this equipment
[Added: 2019/12/31] [Last Updated: 2020/12/24]
PMU 2020/12 Completed Data are collected. History
Execute the decision of PSC
[Added: 2019/12/31] [Last Updated: 2020/12/24]
PMU 2021/12 Not Initiated After the final decision on this the PMU will execute it. History
4. Recommendation:

Recommendation 4: PMU should solicit necessary external expertise for assessment of available technological and financial options in order to determine feasible alternatives for decontamination or disposal of the special transformers at Uniprom-KAP.

Management Response: [Added: 2019/12/31] [Last Updated: 2021/01/27]

Project already has international expert on contract but if project partners express their interest that someone other than this expert do the assessment, we will solicit other expertise. This will also be discussed at the PSC meeting.

Key Actions:

Key Action Responsible DueDate Status Comments Documents
Prepare ToR for the service
[Added: 2020/01/03] [Last Updated: 2020/12/24]
PMU 2020/12 Completed All project partners and PSC are satisfied with the expert support provided by the international expert engaged by the project and there is no need to solicit the services of additional one for the same scope of work. History
Select expert
[Added: 2020/01/03] [Last Updated: 2020/12/24]
PMU 2020/12 Completed Same as the previous key action: All project partners and PSC are satisfied with the expert support provided by the international expert engaged by the project and there is no need to solicit the services of additional one for the same scope of work. History
Development of the assessment
[Added: 2020/01/03] [Last Updated: 2020/12/24]
PMU 2020/12 Completed The assessment has been developed and the project will follow the recommendations from it. History
Actions to be taken based on the recommendations from the Assessment
[Added: 2020/01/03]
PMU 2021/12 Not Initiated
5. Recommendation:

Recommendation 5:  PMU should ensure provision of international expertise in enforcement of PCB legislation for hands-on training of the national environmental inspectors.

Management Response: [Added: 2019/12/31] [Last Updated: 2021/01/27]

PMU initiated discussion with the national environmental inspection and will intensify the work on building their expertise.

Key Actions:

Key Action Responsible DueDate Status Comments Documents
Prepare plan for the trainings
[Added: 2020/01/03] [Last Updated: 2021/05/12]
PMU 2021/12 Initiated This activity has been initiated, as the project team started the negotiations/communication with partners. Due to Covid-19 pandemic this activity is pending as the authorities introduced a number of measures in order to protect health of the citizens and hands-on trainings are not possible to be organized. History
Implementation of the plan
[Added: 2020/01/03]
PMU 2021/12 Not Initiated
6. Recommendation:

Recommendation 6: PMU in collaboration with MoSDT should consider elaboration of a proposal for legal and financial incentives to encourage the PCB holders to take early actions for phase out of the in-service electrical equipment well in advance of the 2025 deadline. In addition, the PMU in collaboration with CEDIS and Uniprom-KAP should consider introducing presentation of maintenance plans of online PCB equipment at PSC meetings and discuss timelines for replacement and disposal of online PCB equipment well before the project ends in 2021.

Management Response: [Added: 2019/12/31] [Last Updated: 2021/01/27]

The recommendation on incentives will be further investigated and discussed vit the PSC and the MoSDT.

The part about presentation of maintenance plans on PSC meetings will be implemented on the meetings that will be organized at the beginning of 2020.

Key Actions:

Key Action Responsible DueDate Status Comments Documents
Discuss this recommendation on PSC meeting
[Added: 2020/01/03] [Last Updated: 2020/12/24]
PM, PSC 2021/12 Not Initiated History
7. Recommendation:

Recommendation 7: PMU in cooperation with MoSDT should ensure cooperation with the on-going research project on health impact of PCBs that is being implemented by the Montenegrin Institute for Public Health. The cooperation should focus on monitoring PCB health impacts for workers with electrical equipment and communities living in the neighbourhood of the temporary PCB storage facilities.

Management Response: [Added: 2019/12/31] [Last Updated: 2021/01/27]

CO already started with the formalization of the cooperation with the Institute for Public Health (IPH). This recommendation will be fully implemented.

Key Actions:

Key Action Responsible DueDate Status Comments Documents
Monitoring of PCB impact on health
[Added: 2020/01/03] [Last Updated: 2020/12/24]
PM, IPH 2021/12 Initiated Due to COVID-19 outbreak this activity is put on hold. History
Prepare and sign cooperation agreement with IPH
[Added: 2020/01/03] [Last Updated: 2020/12/24]
PMU 2020/12 Completed The cooperation agreement with IPH has been signed. History
8. Recommendation:

Recommendation 8: PMU in cooperation with the main PCB holders should consider practical involvement of students of higher education in activities on PCB management, for example through participation of students in the preparation of the next export shipment of PCB waste.

Management Response: [Added: 2019/12/31] [Last Updated: 2021/01/27]

Ensure participation of students

Key Actions:

Key Action Responsible DueDate Status Comments Documents
Ensure participation of students in next export of PCB waste
[Added: 2020/01/03]
PMU 2021/12 Not Initiated
9. Recommendation:

Recommendation 9: PMU in cooperation with MoSTD and other relevant governmental agencies should consider pilot testing on sampling and analysis of PCBs in caulk, glazing and painting materials in older buildings.

Management Response: [Added: 2019/12/31] [Last Updated: 2021/01/27]

This activity is not part of the PCB project that we were approved to implement. We will investigate the willingness of the Government to do some investigation to determine is there a PCB in caulk, glazing and painting materials in older buildings. This could be considered as an idea for another chemicals project.

This activity needs to be approved by the donor in order to be financed from already approved financial resources.

Key Actions:

Key Action Responsible DueDate Status Comments Documents
Discuss the issue on the PSC and agree on how to proceed with it.
[Added: 2020/01/03] [Last Updated: 2020/12/24]
PM, PSC 2021/07 Initiated History
10. Recommendation:

Recommendation 10: PMU should consider a revision of the project results framework to ensure its consistency and full compliance with the principles of the results-based management.

Management Response: [Added: 2019/12/31] [Last Updated: 2021/01/27]

The project was designed based on a number of years of implementation of similar programmes. We will review results framework again and decide on the PSC the proposed revision.

Key Actions:

Key Action Responsible DueDate Status Comments Documents
Review the results framework and discuss to need to change it
[Added: 2020/01/03] [Last Updated: 2020/12/24]
PMU, PSC 2021/12 Initiated History
11. Recommendation:

Recommendation 11: PMU in cooperation with MoSDT and the two major PCB holders should develop and agree clear rules for accounting of the in-kind contributions to the project.

Management Response: [Added: 2019/12/31] [Last Updated: 2021/01/27]

This recommendation will be implemented, and all in-kind contribution will be recorded.

Key Actions:

Key Action Responsible DueDate Status Comments Documents
Develop clear rules for accounting the in-kind contributions
[Added: 2020/01/03] [Last Updated: 2020/12/24]
PMU 2020/12 Completed PMU will provide guidelines for the in-kind contributions accounting to project partners. History
Collect data on in-kind co-financing from the partners
[Added: 2020/01/03]
PMU 2021/12 Not Initiated
12. Recommendation:

Recommendation 12: PMU should conduct a thorough reassessment of the project risks after the MTR stage and ensure that critical risks are properly identified and addressed in the Critical Risk Management section of the annual PIRs together with the corresponding assessment from the side of RTA.

Management Response: [Added: 2019/12/31] [Last Updated: 2021/01/27]

The PMU will do the reassessment of the project risks and continue to follow it on a quarterly basis. Every risk that is considered to be critical will be reported in the annual PIR.

Key Actions:

Key Action Responsible DueDate Status Comments Documents
Quarterly follow up of the risks
[Added: 2020/01/03]
PMU 2021/12 Initiated
Reassessment of project risks
[Added: 2020/01/03] [Last Updated: 2020/12/24]
PMU 2020/12 Completed Risks are reassessed. History

Latest Evaluations

Contact us

1 UN Plaza
DC1-20th Floor
New York, NY 10017
Tel. +1 646 781 4200
Fax. +1 646 781 4213
erc.support@undp.org