- Evaluation Plan:
- 2018-2021, Bureau for Policy and Programme Support
- Evaluation Type:
- Mid Term Thematic
- Planned End Date:
- 12/2018
- Completion Date:
- 02/2019
- Status:
- Completed
- Management Response:
- Yes
- Evaluation Budget(US $):
- 75,000
Evaluation of Stage I HCFC Phaseout Management Plans (HPMPs)
| Document | Type | Language | Size | Status | Downloads |
|---|---|---|---|---|---|
 TOR _Stage I HPMP Evaluation Expert_Final.doc |
tor | English | 138.50 KB | Posted | 959 |
| TOR _Stage I HPMP Evaluation Resource.docx |
tor | English | 70.94 KB | Posted | 977 |
 Stage I-HPMPs-Evaluation-FINAL.pdf |
report | English | 1285.99 KB | Posted | 1741 |
| Title | Evaluation of Stage I HCFC Phaseout Management Plans (HPMPs) | ||||||||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| Atlas Project Number: | 00069439 | ||||||||||||||||
| Evaluation Plan: | 2018-2021, Bureau for Policy and Programme Support | ||||||||||||||||
| Evaluation Type: | Mid Term Thematic | ||||||||||||||||
| Status: | Completed | ||||||||||||||||
| Completion Date: | 02/2019 | ||||||||||||||||
| Planned End Date: | 12/2018 | ||||||||||||||||
| Management Response: | Yes | ||||||||||||||||
| Focus Area: |
|
||||||||||||||||
| Corporate Outcome and Output (UNDP Strategic Plan 2018-2021) |
|
||||||||||||||||
| SDG Goal |
|
||||||||||||||||
| SDG Target |
|
||||||||||||||||
| Evaluation Budget(US $): | 75,000 | ||||||||||||||||
| Source of Funding: | |||||||||||||||||
| Evaluation Expenditure(US $): | 64,000 | ||||||||||||||||
| Joint Programme: | No | ||||||||||||||||
| Joint Evaluation: | No | ||||||||||||||||
| Evaluation Team members: |
|
||||||||||||||||
| GEF Evaluation: | No | ||||||||||||||||
| Key Stakeholders: | Key government officials working on ozone issues, UNDP, MPU/Chemicals, Executive Committee of the Multilateral Fund for the Implementation Committee | ||||||||||||||||
| Countries: | GLOBAL |
| Lessons | |
|---|---|
| Findings | |
| 1. | The implementation of HPMPs is part of key instruments instituted under the Montreal Protocol to phase-out ozone-depleting substances (ODS). The main objective of HPMPs is to phase-out HCFCs, which have been transitional substances to replace CFCs. It could be seen as the second phase of the MP in eliminating ODS. The first phase was the elimination of CFCs, the second phase is the phasing-out of HCFCs through the implementation of HPMPs and then the third phase will be the phasing out of the HFCs, which is the objective of the Kigali agreement11. HPMPs are, therefore, a relevant instrument to replace HCFCs with alternatives technologies that are not damaging to the ozone layer and are clearly contributing to the objective of the MP. Tag: Climate change governance Environment Policy Relevance |
| 2. | UNDP is one of the most active agencies supporting countries in their foam and refrigerant transition to climate-friendly technologies as well as helping countries promote energy efficiency in the foam, refrigeration and A/C sectors. It supports Article 5 countries to eliminate ODS. It is one of four agencies to implement HPMPs; UNEP, UNIDO and the World Bank are the other three institutions. In addition to be an implementing agency for the MLF financing the implementation of the MP, UNDP is also an implementing agency for the Global Environment Facility (GEF), which also funds programmes to eliminate ODS in countries with economies in transition. Tag: Climate change governance Environment Policy Green Economy Relevance Global Environment Facility fund UN Agencies UNDP Management UNDP management Agenda 2030 |
| 3. | Participating countries are all Parties to the Montreal Protocol; they ratified the Protocol and as such they are obligated to comply with the obligations set by the Treaty. The Montreal Protocol has set binding progressive phase-out obligations for developed and developing countries for all the major ozone depleting substances, including chlorofluorocarbons (CFCs), halons and less damaging transitional chemicals such as hydrochlorofluorocarbons (HCFCs). It targets 96 ozone depleting chemicals in thousands of applications across more than 240 industrial sectors. In 2016 (Kigali agreement) the MP also became responsible for setting binding progressive phase down obligations for the 18 main hydrofluorocarbons (HFCs). Tag: Climate change governance Environment Policy Country Government Policy Advisory Technical Support Agenda 2030 SDG Integration |
| 4. | One characteristic of these HPMPs is the strong focus on the private sector, recognizing that small and medium-sized enterprises are key stakeholders, which need to be involved in phasing out HCFCs. Based on the experience with activities to phase-out CFCs, it was found that end-users had difficulties in gaining access to appropriate alternative technologies and information on the availability assistance in the field. For instance, in Georgia, it was found that this difficulty to access alternative technologies, limited the introduction of innovative technologies into the country and diminished the competitiveness of the private sector. Compounded with the lack of skilled technicians, it restricted the capacity to implement new alternative technologies. Tag: Relevance Partnership Private Sector Technical Support |
| 5. | The main purpose of HPMPs is to develop country-based strategies to allow Article 5 countries to meet the reduction levels in HCFC consumption as agreed by the Parties to the Montreal Protocol in decision XIX/6. In agreeing to the accelerated phase-out schedule, the Parties to the MP were encouraged to promote the selection of alternatives to HCFCs that minimize environmental impacts, in particular impacts on climate, as well as meeting other health, safety and economic considerations. At its 54th meeting, the ExCom adopted comprehensive guidelines for preparing HPMPs, followed by a guide developed by the MLF Secretariat to assist and facilitate the process of preparing stage I HPMPs. Tag: Climate change governance Environment Policy Coherence Relevance Programme/Project Design Country Government Donor Private Sector |
| 6. | Each HPMP submission was approved by the ExCom with a set of baseline and scheduled targets including the freeze consumption and production at baseline level by 2013, the reduction by 10% by 2015 and for those HPMPs with a longer timeframe, a reduction by 35% by 2020. It represented a total expected of HCFCs to be phased-out of 2,744 ODP tonnes per year by 2015. Tag: Climate change governance Climate finance Emission Reduction Environment Policy Effectiveness Agenda 2030 |
| 7. | In the meantime, despite that countries met their 2015 targets, many activities under these HPMPs did not take place as planned. Among HPMPs that had only the 2013 and 2015 targets to achieve and to be completed by end of 2017 (19 HPMPs), only 10 PCRs were available at the time of this evaluation; though other projects may already be financially closed. The limited information on the status of each HPMP at a particular point in time rendered the analysis difficult. However, these delays in implementing HPMP activities were also observed within the tranche requests reviewed for this evaluation. These delays included investment activities and capacity development activities, despite that not much information has been available to assess the timing of their implementation.
Tag: Climate finance Challenges Monitoring and Evaluation Procurement Project and Programme management |
| 8. | However, these delays in implementing investment and non-investment activities, coupled with the reducing supply of HCFCs may result in enterprises moving straight to HFC alternatives. More and more reports in this area highlight market trends moving toward the use of HFCs, especially in the RAC sector where the use of R410a is increasing. A recent analysis of surveys of ODS alternatives commissioned by the ExCom15, reveals that the use of R410a has been increasing by 40% (compounded annual growth rate) during the period 2012-2015, which means that the consumption of R410a is doubling every other year. This creates significant risks as countries will meet their HCFC targets but their HFC consumptions will increase. Tag: Challenges Effectiveness Monitoring and Evaluation Private Sector |
| 9. | The Evaluation Team also reviewed how gender considerations were mainstreamed in the portfolio of HPMPs implemented by UNDP. It found that gender has not been considered in the full project cycle for HPMPs from the formulation stage, monitoring and reporting progress to completion reports. No information on gender was found in all project documents reviewed for this evaluation. A mid-term evaluation for the Stage I HPMP in Jamaica concluded that “there is no indication that gender was a consideration in the design and/or execution of the Project”. Tag: Gender Equality Gender Mainstreaming Gender Parity Monitoring and Evaluation Programme/Project Design |
| 10. | Among the portfolio of HPMPs reviewed, technology conversions happened mostly in medium and large countries; LVCs are mostly dependent on imports. Almost half HPMPs (41%) reviewed for this evaluation intervene in the PU foam sector (see Table 6 in Annex 9 for alternative technologies selected under HPMPs implemented by UNDP). This is a sector where it exists well proven alternatives that medium and large companies took advantage of, such as replacing HCFCs with cyclopentane and more broadly hydrocarbons (n-pantene, etc.). As several interviewees stated, this is a “low hanging fruit” for Stage I HPMPs and a relatively easy way to meet the reduction target of 2015 with a few conversion projects. Tag: Emission Reduction Effectiveness Innovation Private Sector Technical Support |
| 11. | The review conducted for this evaluation indicates three main types of risks related to the implementation of HPMPs: (a) the risks that countries become in non-compliance; (b) the risks of accidents and negative effect on health associated with alternative technologies to HCFCs such as the use of flammable and toxic refrigerants; and (c) the management risks related to the implementation of HPMPs. Tag: Effectiveness Health Sector Implementation Modality Monitoring and Evaluation Project and Programme management Risk Management |
| 12. | The financing of HPMPs by MLF grants is done according to well established guidelines to assess the cost-effectiveness of these projects. At the 16th meeting of the ExCom19, sector and sub-sector cost-effectiveness threshold values were adopted to be applied to projects. In November 2007, a paper was prepared by the MLF Secretariat on options for assessing and defining eligible incremental costs for HCFC consumption and production phase-out activities20, which was presented at the 53rd ExCom meeting. Further analyses took place including a document summarizing the revised analysis of relevant cost considerations surrounding the financing of HCFC phase-out21, which was presented at the 55th ExCom meeting in July 2008. Tag: Effectiveness Efficiency Global Environment Facility fund Multi Donor Trust Funds Operational Efficiency |
| 13. | Regarding co-financing, the MLF guide to prepare HPMPs provide some guidelines to countries in this area; however, it is not a requirement when submitting a HPMP for funding. These guidelines include Decision 54/39(h), which states that “countries and agencies were encouraged to explore potential financial incentives and opportunities for additional resources to maximize the environmental benefits from HPMPs”. It also includes the ExCom Decision 74/17(b), which encourages LVC countries to consider a report on resource mobilization for climate benefits produced by UNEP24 when seeking additional resources for climate co-benefits during the implementation of stage I and future stages of HMPMs. Tag: Climate finance Government Cost-sharing Resource mobilization Anti-corruption Monitoring and Evaluation |
| 14. | UNDP established a dedicated Montreal Protocol Unit (MPU) based in New York in 1991 to spearhead and coordinate its efforts to support Article 5 developing countries as one of the implementing agencies of the MP-MLF. This unit is the focal point for UNDP programme related to the Montreal Protocol and as such is responsible for strategic planning, policy, programme and financial oversight for matters related to the Protocol. The MPU also reports UNDP activities to the MP-MLF Secretariat and the ExCom. Tag: Implementation Modality Oversight Project and Programme management UNDP Management UNDP management |
| 15. | The review conducted for this evaluation revealed that programmes and projects funded by the MLF undergo intense scrutiny by the MLF Secretariat and the ExCom, including a yearly assessment of the performance of each implementing agency of the Montreal Protocol. According to the Decision 41/93 of the ExCom, the performance of each implementing agency is assessed yearly through eight weighted performance indicators in three areas: approval, implementation, and administration. For each of these indicators yearly targets are established and submitted to the ExCom. The list of indicators is presented below:
Tag: Efficiency Monitoring and Evaluation Project and Programme management Results-Based Management |
| 16. | Interviews conducted during this evaluation confirm the efficiency of this relatively small implementation team; particularly when considering the volume of projects and financing to manage and administer. The few stakeholders interviewed in the context of this evaluation, stated their appreciation for the support they get from UNDP during the formulation of these HPMPs but also through implementation and particularly the support to formulate the tranche requests.
Tag: Efficiency Communication Human and Financial resources Implementation Modality Operational Efficiency Oversight Project and Programme management Country Government Donor UNDP Management UNDP management |
| 17. | As discussed in section 5.1, HPMPs are performance-based agreement between each Article 5 country and the MLF Executive Committee, whereby agreed-upon funding tranches are released when conditions related to ODS phaseout and disbursements are met. Each HPMP is the object of a standard agreement including key targets, roles and responsibilities of each Party (country, implementing agency and MLF), requested funding and the relevant list of ExCom decisions to consider for the implementation of these HPMP. These agreements are the central piece to establish partnerships between the MLF Secretariat, the implementing agency: UNDP and the country-based NOUs. They set the target of ODS to be phased-out, the conditions to finance these plans (Funding Approval Schedule), and various operational matters such as the need to monitor and report the progress made, the conditions to reallocate funds and the funding terms and conditions in case of non-compliance with the established targets. In the meantime, these agreements can also be amended to reflect any changes occurred after the formulation of a HPMP. It includes a possible one-time revision of targets in the case where the baseline consumption for compliance was not fully established during the formulation of the HPMP. It was also noted that any change in alternative technology, which was proposed in the project document, must be the object of a request for change with the identification of the associated incremental costs and must be approved by the ExCom. Tag: Efficiency Harmonization Partnership Project and Programme management Country Government Donor UNDP Management UNDP management |
| 18. | The implementation of HPMPs does not include any guidelines requiring the utilization of local capacities. Often, these projects need international expertise as alternative technologies are not well developed at each state of phasing-out ODS. Most HPMPs have replaced HCFCs by HFCs. When formulating these HPMPs, some of them mentioned options such as methyl formate, HFOs, etc. but they were not really considered to be implemented, particularly at the outset of these HPMPs, which was during the period 2011-2012. At that time, these alternative technologies were not well tested. It is only now that alternatives to HCFCs are emerging as viable alternative technologies, particularly alternatives with low GWP. Tag: Efficiency Human and Financial resources Implementation Modality Operational Efficiency Technical Support |
| 19. | As discussed under the sub-question 2.1, all 47 Article 5 countries complied with their respective 2015 targets. Together, as of 2015, the 49 Stage I HPMPs contributed to the elimination of 8,062 ODP tonnes per year and more is expected for HPMPs that have a longer timeframe, including the 2020 targets. As discussed in Section 5.1, HPMPs are part of an overall strategy of the Montreal Protocol to phase-out ozone-depleting substances (ODS). As shown on the diagram, HPMPs represent the second phase in eliminating ODS, following the first phase, which focused on the phase-out of CFCs. Tag: Climate change governance Emission Reduction Effectiveness Impact Programme/Project Design |
| 20. | The Evaluation Team found that not much information is collected on the positive and negative impacts of these HPMPs on the local environment. The entire process to formulate, implement, monitor, verify and report is much focused on the technical aspects of phasing-out ODS – HCFCs in the case of Stage I HPMPs. The entire body of knowledge accumulated through the implementation of HPMPs is much focused on explaining and documenting how to reach the targets set to eliminate HCFCs. It consists mostly in detailing how these plans will replace existing technologies that contain damaging ODS with not damaging alternative technologies. Verification reports are also a process to confirm that national targets were achieved as planned. Tag: Impact Local Governance Monitoring and Evaluation Technical Support |
| 21. | In the meantime, interviews conducted for this evaluation reveal that the efficient processes have contributed to the development of national capacities. As discussed under the sub-question 3.2, NOUs are strongly supported by UNDP, particularly by the regional teams. They feel part of an informal network, benefiting from the various exchanges. Through communications with UNDP and the MLF Secretariat, NOUs constantly learn new skills and knowledge, including on new tested alternative technologies. Tag: Impact Innovation Knowledge management Country Government Donor UNDP Management UNDP management Capacity Building |
| 22. | Additionally, these HPMPs have a strong interaction with the private sector, which is the main driver in the production and consumption of HCFCs. HPMPs implemented by UNDP are mostly considered as investment projects that is they support the cost of retrofitting or replacing existing technologies with not damaging technologies. Through the process, the targeted enterprises do also acquire new skills and knowledge and the review of this projects indicate that they should contribute to raising the productivity of these enterprises and consequently their competitiveness. Tag: Private Sector Capacity Building Jobs and Livelihoods Technology Trade and Development |
| 23. | Before discussing specific questions about sustainability of HPMPs, the review of documents conducted for this evaluation revealed that the concept of sustainability is not much developed. The guides provided by the MLF Secretariat to help the formulation of HPMPs barely mentioned sustainability. No particular guidelines are given on sustainability and no section is required in the project document when submitting a request for funding a HPMP or a funding tranche request. The only reference to sustainability in these guidelines for developing a HPMP is to demonstrate the long-term sustainability of training programmes. It was also noted that in the few project completion reports reviewed, the sustainability of HPMP activities are not discussed. Tag: Sustainability Monitoring and Evaluation |
| 24. | The adoption of alternative technologies is also often accompanied by clear economic benefits for these enterprises, which will contribute much toward sustaining these achievements. For instance, a mid-term evaluation of the HPMP project in Jamaica found that a company that changed its process to manufacture foam for roof insulation using HCFC-141b with HFCs would not revert to its old practices due – as the enterprise said - mostly for economic reasons. The new process is cheaper and increased the profit margin for the enterprise. Tag: Emission Reduction Sustainability Sustainability Private Sector Jobs and Livelihoods Technology Trade and Development |
| 25. | Similar to the financial sustainability discussed above, there are no real organizational arrangements and continuation of activities issues linked to the implementation of HPMPs. These projects are mostly about eliminating the use of HCFCs. They consist mostly in phasing-out the use of HCFCs in the manufacturing and servicing sectors. As such, once HCFCs have been phased-out, the expected environmental impacts are achieved. Tag: Emission Reduction Sustainability Programme/Project Design Sustainability |
| 26. | It is expected that Article 5 countries have established HCFC control measures (licensing and quota system) through legislation and regulations at the time of their HPMP submissions or their submissions may not be approved. If this condition is not met, countries are encouraged to defer their HPMP submissions. Therefore, a HPMP submission should include a description of ODS legislation and regulations in place, including the operational licensing and quota systems to import/export HCFCs and, if applicable the registration system of importers and exporters. It should also include policies related to the phase-out of HCFCs such as bans on import of HCFCs, HCFC equipment, and other government initiatives in response to the accelerated phase-out of HCFCs. Countries are also encouraged to develop and adopt regulations, codes of practices, and standards for alternatives to HCFCs - which are often classified with some level of flammability and/or toxicity - before they are introduced in the country. Tag: Emission Reduction Policies & Procedures Country Government Policy Advisory |
| 27. | As per the MLF guide to prepare Stage I HPMPs, the policy component of HPMPs is to focus on capacity development for enforcement personnel. However, when formulating the overarching strategy to phase-out HCFCs, the guide includes the possibility of strengthening policy instruments that may be needed to reduce the supply and/or demand of HCFCs, such as import quotas, price controls, ban on imported HCFC-based equipment, ban on imported HCFC 141b pre-blended polyols, restrictions on high GWP non-HCFC alternatives. Tag: Climate change governance Emission Reduction Sustainability Parliament Rule of law Policies & Procedures Country Government Capacity Building Policy Advisory |
| 28. | Considering the discussion above, institutional and individual capacities are assumed to be adequate once these HPMPs are completed. Developing capacities of individuals is a focus of these plans. Under the policy component of HPMPs the focus is to train enforcement personnel. Additionally, in the private sector, people involved with HCFCs are also trained on alternative technologies particularly with the introduction of more flammable and toxic alternative technologies to HCFCs. Finally, as discussed at the beginning of this section, the guidelines to formulate a Stage I HPMP ask for demonstrating the long-term sustainability of training programmes. Tag: Sustainability Monitoring and Evaluation Sustainability Country Government Private Sector Capacity Building Jobs and Livelihoods Technology |
| 29. | As discussed under the sub-question 4.2, not much information is collected on the positive and negative effects following the implementation of HPMPs, including the social and political impacts. No issues in these areas have been found during this evaluation. On the social side, it can be expected that some actions to eliminate/replace HCFCs may have impacted employment; however, no information was found to support this type of impact. Tag: Sustainability Civic Engagement Parliament Monitoring and Evaluation Country Government |
| 30. | The review of the HPMP process with its guidelines indicates that it is a good process to support countries to eliminate their consumption and production of HCFCs. It was confirmed by few stakeholders interviewed for this evaluation, saying that the procedures, though exhaustive, are clear and accompanied by helpful guidelines. As discussed in section 5.2, HPMPs have successfully contributed to the elimination of HCFCs. Based on the experience accumulated since the decision of the ExCom to adopt a staged approach for the implementation of HCFC phase-out management plans (HPMPs) (Decision 54/39), a model to implement such programme has emerged. Tag: Sustainability Implementation Modality Policies & Procedures Programme/Project Design Sustainability Country Government Private Sector |
| Recommendations | |
|---|---|
| 1 | Recommendation 1: It is recommended to conduct a study of MPU operations to identify bottlenecks and potential cost-saving measures. Issue to Address
|
| 2 | Recommendation 2: It is recommended that UNDP strengthens its knowledge sharing/exchange including best practices as well as tracking prices, costs and technologies. Issue to Address
|
| 3 | Recommendation 3: It is recommended to better monitor other benefits from HPMPs such as impact on productivity on enterprises, competitiveness, employment, health, etc. Issue to Address There is no information on capacities built, nor on productivity gains with new technologies in the private sector, nor on employment, etc. It is recommended to monitor these potential other benefits, including an assessment of these benefits at the end of these HPMPs and this information being reported in project completion reports. |
| 4 | Recommendation 4: It is recommended to conduct stronger assessments/evaluations at the end of HPMPs to better capture achievements at the country level, including best practices and lessons learned. Issue to Address |
| 5 | Recommendation 5: As HCFCs are being phased-out, it is recommended to track the HFCs market and use in countries to monitor the market movements of HFCs. Issue to Address |
Management Response Documents
1. Recommendation:
Recommendation 1: It is recommended to conduct a study of MPU operations to identify bottlenecks and potential cost-saving measures.
Issue to Address
Based on the experience accumulated through the implementation of these HPMPs, a model to implement such programme has emerged with established guidelines and procedures of the MLF and of UNDP. This model could certainly be replicated for the next phases in eliminating HCFCs but also HFCs under the Kigali agreement. It has been tested and over time, procedures and guidelines were improved. However, despite recognizing that the overall process is good, these guidelines and procedures have tended to become more time consuming. There is a perception that the process is being more and more centrally micro-managed, and pressure is increasing on the UNDP technical teams to deliver these projects on time and on budget but with the same level of resources.
It is recommended to conduct a review of guidelines but particularly of procedures to identify where bottlenecks exist and where potential cost-saving measures could be implemented to streamline processes to formulate, implement, monitor, verify and report on these projects. The review should include the identification of all steps needed to formulate these projects, procedures, templates needed and also the type of system to manage/administer the information related to the implementation of these projects. With a full web-based system, staff should be able to easily access all information on projects and get the status of any project “at their fingertips” such as the GEF website project database providing access by project and by country.
Management Response: [Added: 2020/11/30] [Last Updated: 2020/12/31]
Management Response: MPU is considering a number of procedures to streamline its procedures, including reviewing procurement and other administrative processes. One of the key agenda items during the MPU retreat in 2019 is the identification of significant bottlenecks to project design, development and implementation and the streamlining of operations. At the 2019 MPU retreat, measures that could be implemented to improve and streamline existing processes to formulate, implement, monitor, verify and report on MPU projects will be considered.
Key Actions:
| Key Action | Responsible | DueDate | Status | Comments | Documents |
|---|---|---|---|---|---|
| 1.1. This recommendation is acknowledged and during the 2019 and 2020 MPU retreat, a number of measures to streamline MPU procedures, including reviewing SOP, procurement and other administrative processes and related costs will be proposed and considered.
[Added: 2020/12/31] |
MPU Director and HQ team | 2019/07 | Completed | The SOP was reviewed and discussed in the team retreat in 2019. The retreat in 2020 focused on the bottlenecks of delivery. The possibility of centralized procurement for similar tools and equipment had been discussed with the procurement team. MPU continues to improve the management of the portfolio for an efficient and effective process. The performance of UNDP for the management of the MLF projects assessed by the MLF Secretariat was the highest among all implementing agencies with score of 90 in 2017, 95 in 2018 and 94 in 2019. | |
| 1.3. Enhance the standardization in terms of Project Document development, budgeting, TOR of individual contractors, as well as the progress report
[Added: 2020/12/31] [Last Updated: 2021/10/01] |
MPU Director and HQ team | 2021/07 | Completed | -MPU has developed comprehensive communication documents of UNDP Experiences on the implementation the Montreal Protocol and a BPPS offer on Montreal Protocol, Chemical and Waste Management to guide activities in country offices. -A standard training PPT on the Kigali Amendment was developed and shared with regional teams for training to country offices and partners. -The summaries of the Ex.Com meeting were shared with MPU team, NCE Director and BPPS Director. -MPU regional teams communicated on daily basis with the country offices to support the implementation and reporting. - The team has updated the template of project document for MP projects and has been used in 5 new stage-II HPMPs in LAC region in 2021. The template was also recommended for the stage-II HPMP projects in other regions that are in the process of development of Pro.Doc. History | |
| 1.2. Regarding the recommendation on the web-based system, it has already been decided that Montreal Protocol projects will be included into the GEF project database (PIMS+). It should be mentioned that a MPU project database already exists but it doesn’t contain the full functionalities, particularly when it comes to reporting. This will be further explored.
[Added: 2020/12/31] [Last Updated: 2022/04/14] |
MPU Director and HQ team | 2022/03 | Completed | The design of the MLF module in PIMS+ has concluded. History | |
| 1.4. Strengthen communication and training of UNDP Country Offices
[Added: 2020/12/31] |
MPU Director and Regional team | 2019/08 | Completed | -MPU has developed comprehensive communication documents of UNDP Experiences on the implementation the Montreal Protocol and a BPPS offer on Montreal Protocol, Chemical and Waste management to guide the activities in the country offices. -A standard training PPT on the Kigali Amendment was developed and shared with the regional teams for the training to country offices and partners. -The summaries of the Ex.Com meeting were shared with MPU team, NCE director and BPPS director. -MPU regional teams communicated on daily basis with the country offices to support the implementation and reporting. |
2. Recommendation:
Recommendation 2: It is recommended that UNDP strengthens its knowledge sharing/exchange including best practices as well as tracking prices, costs and technologies.
Issue to Address
Phasing-out HCFCs means retrofitting and/or replacing the use of HCFCs by alternative technologies that are not damaging to the ozone layer. As of the time when these HPMPs started to be formulated, alternative technologies were mostly limited to HFCs. Over time, other alternative technologies with lower GWP have been identified but they are also often classified with some level of flammability and/or toxicity requiring additional security measures. However, the time it takes for this information to “trickle” down to each country can be long and uneven. Currently, the main mechanisms for transferring this knowledge is mostly through exchanges with international experts, exchanges at the various meetings led by the MP, ExCom and the implementing agencies as well as informal networks among people involved in eliminating HCFCs in countries, implementing agencies and MLF Secretariat.
It is recommended that UNDP develops a platform for knowledge exchange and sharing best practices as well as tracking prices, costs and available technologies, including regionally-based and/or country-based information.
Management Response: [Added: 2020/11/30] [Last Updated: 2020/12/31]
Management Response: Under the Montreal Protocol, there are dedicated mechanisms for sharing information and knowledge generated from the projects funded by the Multilateral Fund (MLF). For example, a dedicated Compliance Assistance Programme (CAP), hosted by UNEP and funded by MLF) delivers the information clearinghouse mandate for all MLF Implementing Agencies. UNDP actively participates and contributes its knowledge, information and best practices through CAP and the regional networks. In terms of in-house knowledge and information sharing, UNDP MPU will continue to do so through the following mechanisms: a dedicated Intranet website, blogs and other communication tools, periodic team face-to-face and online meetings, meetings with recipient countries, and publications devoted to the results of successful projects (for example, https://www.undp.org/content/undp/en/home/library.html?start=0&sort=date&view=cards&tag=topics:sustainable-development/ozone-layer-protection). For tracking prices, costs and available technologies, UNDP MPU will continue utilizing the existing UNDP MPU project database to store the project completion reports with the information about project document and results and may consider upgrading to PIMS+ as referenced earlier. MPU organized a number of side events in 2018 to disseminate project results, best practices and knowledge gained from MLF projects and will continue to organize a number of side events in 2019.
Key Actions:
| Key Action | Responsible | DueDate | Status | Comments | Documents |
|---|---|---|---|---|---|
| 2.1. Including MPU projects into PIMS+ will facilitate improved knowledge management and sharing.
[Added: 2020/12/31] [Last Updated: 2022/04/14] |
MPU Director and HQ team | 2022/03 | Completed | The design of the MLF module in PIMS+ has concluded. History | |
| 2.2. Continue utilizing existing MPU tools to share information, knowledge and best practices
a. Project managers will include information about prices, costs and available technologies in project completion reports, as much as possible (considering the sensitivities and commercial confidentiality) and applicable (some projects don’t address technology choices but only address capacity building).
b. Continue to collect and compile information of project results, best practices, and innovative technologies through MPU tools, publications, blogs and other key communication tools.
c. Facilitate South-South cooperation for knowledge exchange and best practices scaling up.
[Added: 2020/12/31] |
MPU Director, HQ and Regional teams | 2020/03 | Completed | MPU has paid great attention to the quality of the completion reports which contain a large amount of detailed information. The great achievements in 2018-2020 in this regard are the accomplishments of a dozen of demonstration projects of the MLF that UNDP implemented. The reports and fact sheets were posted in the MLF portal for widespread dissemination of knowledge and information generated by the innovative projects supported by UNDP. In addition, MPU submitted the completion report of the first investment project of MLF for the implementation of the Kigali Amendment in 2020. We made great efforts to contain the required and additional information in relation to the capital and operation costs of the conversion, climate benefits including direct and indirect emissions. We are very proud that UNDP is the first agency to submit the completion report to the MLF on the investment projects for the Kigali Amendment. History | |
| 2.3. As in 2018, MPU is organizing a number of workshops throughout 2019 to share information and knowledge gained through its projects.
a. MPU is organizing workshop on HFC alternatives in New York City in May 2019. The workshop will bring together participants from Article 5 countries and experts to discuss challenges, opportunities and solutions, and to identify short-term priority activities and long-term strategies to effectively implement the Kigali Amendment and improve energy efficiency. This workshop will facilitate the dissemination of knowledge gained through the ODS alternatives and HFC surveys implemented under UNDP MPU projects.
b. MPU is organizing a side event around the Open-Ended Working Group (OEWG) in July 2019 to introduce best practices in the servicing sector.
c. In 2020, MPU will organize a side event to introduce best practices in the foam sector with a developing country partner.
d. MPU will continue to organize site events in the occasion of the Meeting of the Parties, UNEP network meetings and organize workshops for in depth-knowledge exchanges with stake holders and private sectors.
[Added: 2020/12/31] |
MPU Director and HQ/Regional teams | 2019/07 | Completed | A series of event/workshop/webinar have been organized by MPU to facilitate the knowledge sharing and promote the best practices: 1.UNDP hosted the workshop “Towards the effective implementation of the Kigali Amendment”, in May 2019 in New York. 2. National Cooling Plans Linking to Cooling to energy efficiency interventions, a side event co-organized with K-CEP at the 41st OEWG of the Montreal Protocol in Bangkok in July 2019. 3. Sound ODS/HFC waste management and disposal - Colombia experience co-organized by UNDP and GIZ during the 31st Meeting of Parties in Room in November 2019. 4. In 2020, MPU adapted quickly in response to the covid-19 pandemic, and successfully organized 30 webinars for the knowledge exchange on innovative low GWP cooling technologies, best practices in the servicing sector, policies and data reporting on HFCs, etc. These webinars were well received by our partners and stakeholders in the client countries. More than 1500 people including 35% women participated in UNDP webinars. | |
| 2.4. Support the Multilateral Fund Secretariat in the development and finalization of factsheets on the HCFC demonstration projects, which will be disseminated to a wide general audience.
[Added: 2020/12/31] |
MPU Director, HQ and Regional teams | 2019/12 | Completed | Factsheets for 14 demonstration projects implemented by UNDP were finalized and posted in the MLF portal. |
3. Recommendation:
Recommendation 3: It is recommended to better monitor other benefits from HPMPs such as impact on productivity on enterprises, competitiveness, employment, health, etc.
Issue to Address
Not much information is collected on the positive and negative impacts of these HPMPs on the local environment. The entire process to formulate, implement, monitor, verify and report on Stage I HPMPs is mostly focused on the technical aspects of phasing-out HCFCs. The entire body of knowledge accumulated through the implementation of HPMPs is much focused on documenting progress made towards the set targets to eliminate HCFCs, including details on how these plans will replace existing technologies that contain damaging HCFCs with not damaging alternative technologies.
There is no information on capacities built, nor on productivity gains with new technologies in the private sector, nor on employment, etc. It is recommended to monitor these potential other benefits, including an assessment of these benefits at the end of these HPMPs and this information being reported in project completion reports.
Management Response: [Added: 2020/11/30] [Last Updated: 2020/12/31]
Management Response: We agree that the developmental impacts arising from the implementation of the HCFC phaseout need to be better measured and captured. One key goal of MPU in 2019 is to strengthen the impact and delivery of the MLF programme as well as knowledge sharing and replication of best practices. UNDP is in the process of defining corporate impact indicators and these indicators will be applied to the MLF portfolio as well. MPU will explore mechanisms/processes to include these impact indicators in project documents. Gender is already an issue that MPU has increasingly been engaged on. The MLF currently does not have a policy on gender and does not fund gender activities, so the MPU ozone programme does not take gender into account as much as it could. In order to strengthen the gender dimension of our ozone projects, MPU has been actively engaged on the development of a MLF policy on gender and recruited a gender expert to develop a gender analysis and gender action plan (GAP) for the Montreal Protocol projects in Peru, China, and Nigeria. The gender analysis documents baseline information on what is currently being done to advance gender equality in Montreal Protocol projects as well as what the main challenges and opportunities are. It also identifies entry points for future gender mainstreaming in Montreal Protocol projects. The GAP proposes suggested results and indicators that can be used for the formulation of project documents and/or further GAPs and the financing of gender outputs and activities.
Key Actions:
| Key Action | Responsible | DueDate | Status | Comments | Documents |
|---|---|---|---|---|---|
| 3.1 Apply the corporate impact indicators defined by UNDP to MLF projects and establish a process/mechanism for reporting annually on these indicators (similar to what is already done for the GEF Chemicals programme).
[Added: 2020/12/31] [Last Updated: 2022/01/31] |
MPU Director and HQ team | 2021/07 | Completed | Indicators and targets in relation to the MPU business such as numbers of completion of projects, delivery, completion status of the preparation projects, ratification status of the country for the Kigali amendment through enabling activities, completion status of the national cooling plans, HCFC consumption yearly data were included in the corporate result reporting framework. MPU has submitted the results for all projects in past three years. History | |
| 3.2 Apply UNDP gender policy on all MLF projects
[Added: 2020/12/31] [Last Updated: 2022/01/31] |
MPU Director and HQ Team | 2021/10 | Completed | The Executive Committee of the Multilateral Fund adopted the gender mainstreaming policy in 2019. MPU has been working with the partners to add gender related activities in new proposals that had been submitted to the MLF since 2020. Two regional training workshops on gender were organized in 2021. A gender consultant was hired for this purpose. History |
4. Recommendation:
Recommendation 4: It is recommended to conduct stronger assessments/evaluations at the end of HPMPs to better capture achievements at the country level, including best practices and lessons learned.
Issue to Address
Information on achievements of HPMPs can mostly be found in Project Completion Reports (PCRs) and to some extent in verification reports. Verification reports focus mostly on verifying the data provided by the country on the elimination of HCFCs. PCRs are completed on the basis of a template provided by the MLF Secretariat. They contain 8 sections but are mostly descriptive in nature and are completed by the implementation units in the respective countries.
It is recommended to conduct external project assessments/evaluations near the end of these plans to provide an external review of the performance of these HPMPs. It should include assessments of other potential benefits of these HPMPs and of the technologies involved in these projects, particularly focusing on what worked and what did not work and also recommendations for the way forward based on the respective experiences in these countries.
Management Response: [Added: 2020/11/30] [Last Updated: 2020/12/31]
Management Response: We believe that this is a good recommendation by the evaluators but will need to be discussed further with the MLF Secretariat and/or Executive Committee. The Executive Committee of the Multilateral Fund approves yearly work programmes for its Senior Monitoring and Evaluation officer (SMEO), who is based at the Secretariat in Montreal and who carries out a series of sector-wide desk reviews and field evaluations. These reports usually cover more than one implementing agency, although each agency’s work is described in detail. The reports are then submitted to the Executive Committee for further comments, lessons learned, and recommendations, which are then fed back to each implementing agency. This is why UNDP does not carry out evaluations of its Montreal Protocol programmes. All evaluation reports commend the work of the implementing agencies, including UNDP, and contain a series of lessons learned and findings, which are taken into account in future programming. In order to conduct HPMP evaluations at the national level, their corresponding budgets would need to be approved by the MLF Executive Committee, which may prove to be difficult since this would not be a part of its policy. Although final project evaluations are not currently a requirement of the MLF, if there are any remaining funds, MPU regional officers will recommend that UNDP country offices and government counterparts conduct evaluations for key, larger projects.
Key Actions:
| Key Action | Responsible | DueDate | Status | Comments | Documents |
|---|---|---|---|---|---|
| 4.1 UNDP Country Offices and government counterparts will be encouraged to undertake national level evaluations of their HPMPs if and when there are some funds remaining. This has already been occurring in some countries (i.e. Jamaica).
[Added: 2020/12/31] |
MPU Regional Teams | 2019/12 | Completed | Independent evaluation at the project level were advised by MPU to the country offices and partners. Some country offices have done it although it is not a requirement by the MLF. Since the Montreal Protocol is an international treaty with binding obligations on the control of ODS, UNDP conducted independent verifications of the consumptions for all countries that UNDP is the lead agency. MPU has been coordinating with the senior evaluation officer of the MLF in the evaluations required by the Ex.Com. | |
| 4.2. The cost of evaluation, knowledge dissemination and sharing activities will be included in the development of new projects.
[Added: 2020/12/31] |
MPU Regional Teams | 2019/12 | Completed | The cost was included in the work plan in which the evaluation activity was suggested. |
5. Recommendation:
Recommendation 5: As HCFCs are being phased-out, it is recommended to track the HFCs market and use in countries to monitor the market movements of HFCs.
Issue to Address
The impact of delays in implementing investment and non-investment activities, reduction of supply of HCFCs and the apparent lack of alternative technologies may result in enterprises moving straight to HFC alternatives. More and more reports in this area highlight market trends moving toward the use of HFCs. A recent analysis of surveys of ODS alternatives commissioned by the ExCom, reveals that the use of R410a in the RAC sector has been increasing by a compounded rate of 40% per year during the period 2012-2015, which means that the consumption of R410a is doubling every other year. This creates significant risks as countries will meet their HCFC targets but their HFC consumptions may significantly increase.
It is recommended to monitor the use of HFCs at the country level but also the HFCs market and highlight any trends, which would indicate a surge of HFCs use. The reduction of HCFCs should not result in higher HFCs consumption.
Management Response: [Added: 2020/11/30] [Last Updated: 2020/12/31]
Management Response: The HFC use and market is already closely monitored in the countries where we have Stage I and Stage II HPMPs as well as the 16 countries where we have received funding from the MLF for HFC enabling activities. MPU will be supporting these countries to undertake a range of enabling activities to help their national ozone units fulfil their initial obligations with regards to the phase-down of HFCs, in line with the Kigali Amendment, and will include country-specific activities aimed at initiating support on institutional arrangements, review of licensing systems, data reporting on HFC consumption and production, and developing national strategies. In addition, MPU has been assisting another seven countries to develop stand-alone investment projects to phase down the use of HFCs through replacement with suitable lower GWP technologies. In order to disseminate this information to countries, MPU has already been organizing a number of workshops on the information requirements arising from Kigali and available, cost-effective and safe alternative technologies throughout 2018.
Key Actions:
| Key Action | Responsible | DueDate | Status | Comments | Documents |
|---|---|---|---|---|---|
| 5.1 Continue monitoring the use of HFCs in countries through implementation of Stage II HPMPs
[Added: 2020/12/31] |
MPU Regional Teams | 2019/12 | Completed | MPU is supporting 16 countries on the enabling activities (EA) for the Kigali Amendment. Data of HFC import and consumption were collected by the National Ozone Unit in the implementation of the EA projects. | |
| 5.2. MPU has already conducted ODS alternative surveys in 11 countries. MPU will be analyzing the results of these ODS alternative surveys and prepare a consolidated project completion report, which will provide more information on trends on HFC use in countries.
[Added: 2020/12/31] |
MPU Director and Panama regional team | 2018/12 | Completed | UNDP have submitted all reports. The data collected by the surveys have been included in the overall analysis of the results of the surveys of ODS alternatives (document 80/54) did by the MLF Secretariat. | |
| 5.3. MPU is currently implementing HFC enabling activities in 16 countries, where we are helping countries to establish licensing systems and supporting countries to build capacity on data reporting. Data reporting mechanisms will be set up as part of the HFC enabling activity.
[Added: 2020/12/31] |
MPU Regional Teams | 2019/12 | Completed | 10 countries with EA project supported by UNDP have established the licensing system of HFCs. Countries ratified the Kigali Amendment reported HFCs data in their Country Program report and A7 report in 2020. | |
| 5.4. MPU will continue supporting A5 countries on their HFCs data reporting mechanisms as part of the HFC enabling activity.
[Added: 2020/12/31] |
MPU Regional Teams | 2020/07 | Completed | In 2019 and 2020, MPU organized several sessions in the workshops and webinars for strengthening the capacities of NOUs on the HFC data reporting. | |
| 5.5. MPU is organizing a workshop on HFC alternatives in New York City in May 2019. The workshop will bring together participants from Article 5 countries and experts to discuss challenges, opportunities and solutions, and to identify short-term priority activities and long-term strategy to effectively implement the Kigali Amendment and improve energy efficiency. This workshop will facilitate the dissemination of knowledge gained through the ODS alternatives and HFC surveys.
[Added: 2020/12/31] |
MPU Director and HQ team | 2019/05 | Completed | On May 2019, UNDP hosted the workshop “Towards the effective implementation of the Kigali Amendment”, in New York - USA. | |
| 5.6. MPU will continue to support developing countries to access the low GWP alternatives and narrow the gap at supply chain. A “green supply chain workshop for cooling without warming” has been organized in Jakarta, Indonesia in 2018 with participants from 18 countries.
[Added: 2020/12/31] |
MPU HQ/ Regional Teams | 2019/12 | Completed | UNDP organized the workshop of “Green Supply Chain Workshop for Cooling Without Warming” (8 to 10 October 2018 in Jakarta, Indonesia). The workshop brought together Ozone Units, Experts, Private Sector and Associations to narrow the Supply Chain gap of green Cooling Solutions. MPU has been helped a number of countries to successfully address the supply chain issues related to the new alternatives including Indonesia, Lebanon, Malaysia, Trinidad & Tabago, Egypt, and Cuba. | |
| 5.7. MPU is scaling up the incentive programs in the HPMPs to help countries access the low GWP and efficient cooling technologies.
[Added: 2020/12/31] |
MPU Regional Teams | 2020/07 | Completed | MPU has been supported all our client countries in the introduction and scaling up of the low GWP and efficient cooling technologies. UNDP conducted 13 ender-user incentive schemes (out of total 28 similar activities supported by MLF) to support the developing countries to access the new technologies in Bhutan, Brunei Darussalam, Cambodian, Cuba, Fiji, Ghana, Malaysia, Moldova, Maldives, Nepal and Sri Lanka. |