BRA 16/G31 Fourth National Communication UNFCCC

Report Cover Image
Evaluation Plan:
2017-2021, Brazil
Evaluation Type:
Mid Term Project
Planned End Date:
07/2019
Completion Date:
07/2019
Status:
Completed
Management Response:
Yes
Evaluation Budget(US $):
25,000

Share

Document Type Language Size Status Downloads
Download document BRA16G31_MTR_TOR_DEZ2018.docx tor English 75.11 KB Posted 36
Download document DuBois-Deliverable_3-GEF-MTR BRA16 G31_finalreport.docx report English 570.84 KB Posted 2
Title BRA 16/G31 Fourth National Communication UNFCCC
Atlas Project Number: 85388
Evaluation Plan: 2017-2021, Brazil
Evaluation Type: Mid Term Project
Status: Completed
Completion Date: 07/2019
Planned End Date: 07/2019
Management Response: Yes
Corporate Outcome and Output (UNDP Strategic Plan 2018-2021)
  • 1. Output 2.3.1 Data and risk-informed development policies, plans, systems and financing incorporate integrated and gender-responsive solutions to reduce disaster risks, enable climate change adaptation and mitigation, and prevent risk of conflict
Evaluation Budget(US $): 25,000
Source of Funding: GEF
Evaluation Expenditure(US $): 20,000
Joint Programme: No
Joint Evaluation: Yes
  • Joint with Donors
  • Joint with Ministry of Science, Technology and Innovation (MCTIC)
Evaluation Team members:
Name Title Email Nationality
Random Dubois Msc
GEF Evaluation: Yes
GEF Project Title: Fourth National Communication and Biennial Update Reports to the United Nations Framework Convention on Climate Change (UNFCCC)
Evaluation Type: Mid-term Review
Focal Area: Biodiversity
Project Type: EA
GEF Phase: GEF-5
GEF Project ID: 5378
PIMS Number: 5187
Key Stakeholders: Ministry of Science, Technology and Innovation (MCTIC)
Countries: BRAZIL
Lessons
Findings
Recommendations
1

Given that the Project is at the midpoint of implementation and progress seems to be proceeding satisfactorily with no major issues found it is not recommended to revise the Logframe (e.g., replacing existing indicators with SMART indicators and estimating new, quantifiable targets) at this time.  However, in looking ahead to the Terminal Evaluation (TE) there are several issues that should be addressed at this time.  These are:

- project extension.  The existing PMU strategy for the remaining half of the Project calls for submission of an approved NC in December 2020, 4 months later than is called for in the PRODOC and additional 12 months for outreach, communication and information dissemination activities following the NC submission for a total of 16 months.  The Mission believes that a no-cost extension is warranted;
- elimination of a component activity.  Under Component/Outcome 3, the PRODOC calls for the establishment of a network of low-cost data collection devices for the assessment of the human perception of climate variability.  This appears to be an “orphan” activity and should be dropped (or modified) and the indicator/target eliminated (or revised); and
- revision of indicator (or revision of means of monitoring/reporting).  Under Component/Outcome 4 it is not clear how the preparation and distribution of knowledge products as reported in the PIR 1 and partial PIR 2 will lead to increasing level of institutional capacity in Brazil for education, training and public awareness related to climate change (the respective Results Framework/Logframe indicator). Either the indicator should be changed or provide a different means of monitoring and reporting progress against it.

It is further recommended that these changes and more generally, the main results and recommendations resulting from the PMU strategic planning workshop held in February 2019 should be put into a proposal including budget and calendar to cover the remainder period of project implementation (including the extension) to review and discuss with UNDP.

2

The PMU/UNDP should avail of the opportunity to bring the lessons learned derived from the 4NC into the project design of the 5NC formerly and work for a more robust and clearer PRODOC and a Logframe that can be used as a useful M&E and reporting tool.  The recommended “no cost” extension above would provide an ideal opportunity to do this in parallel to the other proposed activities following the submission of the 4NC.xtension) to review and discuss with UNDP.

3


The 5NC process provides the opportunity to continue to build on the strength of previous NCs.  It is suggested that the PMU, building on many of its own initiatives, should consider supporting: (i) greater inter-ministerial outreach; (ii) broadening the circle of traditional partner institutions; (iii) increasing public awareness and communication; (iv) incorporating State and municipal governments into the process; (v) putting greater emphasis on broadening and diversifying the message of the NC to reach “the people;” and (vi) developing a communication strategy and plan to be incorporated into project design of the 5NC.

4

Simple process indicators should be developed and included in the 5NC results framework to facilitate tracking and evaluating project progress perhaps using the LOAs as a basis to identify indicators and targets per component/sub-component.

5


For the 5NC it is recommended that an inception workshop, scheduled for the first trimester following project initiation, be built into project design with accompanying indicator and budget

6


In the coming 5NC, dependent on staff changeover in the PMU and broadening and inclusion of new partners, UNDP may want to consider supporting additional workshops early during the implementation of the project in particular in explaining administrative, budgeting and reporting requirements

7


The Mission recommends that the PMU consider the increased use of strategic workshops (e.g., similar to the February 2019 workshop) perhaps on an annual basis starting during the PRODOC design process of 5NC and followed in anticipation of AWP preparation.

8

The Mission recommends that the Tripartite Committee TPC should meet on a “when and as needed” basis.

9

Though not a GEF requirement for EA projects, the PMU and UNDP may want to consider identifying and tracking sources and amounts of co-financing to have on hand at the time of the Terminal Evaluation (TE); always useful to report as an indicator of government and public support for the NC process.

10

Brazil’s cumulative and evolutionary experience over 4 NCs has led to the development of relative sophisticated and effective institutional arrangements and processes, particularly in the preparation of the GHG Inventory.  This approach appears to be both highly valuable and likely to be unique.  It is suggested external resources should be sough, perhaps with the assistance of UNDP/GEF/UNFCCC to support greater efforts to disseminate these experiences (e.g., an international workshop attended by other countries to share experiences and compare lessons learned).

1. Recommendation:

Given that the Project is at the midpoint of implementation and progress seems to be proceeding satisfactorily with no major issues found it is not recommended to revise the Logframe (e.g., replacing existing indicators with SMART indicators and estimating new, quantifiable targets) at this time.  However, in looking ahead to the Terminal Evaluation (TE) there are several issues that should be addressed at this time.  These are:

- project extension.  The existing PMU strategy for the remaining half of the Project calls for submission of an approved NC in December 2020, 4 months later than is called for in the PRODOC and additional 12 months for outreach, communication and information dissemination activities following the NC submission for a total of 16 months.  The Mission believes that a no-cost extension is warranted;
- elimination of a component activity.  Under Component/Outcome 3, the PRODOC calls for the establishment of a network of low-cost data collection devices for the assessment of the human perception of climate variability.  This appears to be an “orphan” activity and should be dropped (or modified) and the indicator/target eliminated (or revised); and
- revision of indicator (or revision of means of monitoring/reporting).  Under Component/Outcome 4 it is not clear how the preparation and distribution of knowledge products as reported in the PIR 1 and partial PIR 2 will lead to increasing level of institutional capacity in Brazil for education, training and public awareness related to climate change (the respective Results Framework/Logframe indicator). Either the indicator should be changed or provide a different means of monitoring and reporting progress against it.

It is further recommended that these changes and more generally, the main results and recommendations resulting from the PMU strategic planning workshop held in February 2019 should be put into a proposal including budget and calendar to cover the remainder period of project implementation (including the extension) to review and discuss with UNDP.

Management Response: [Added: 2019/07/26]

In general, there is consensus between the evaluator, the project team and UNDP on the proposed recommendations. Since the implementation of the MTR mission in April 2019, some initiatives have been taken to find solutions for some of the indicated points, such as promoting contact with a new potential partner to develop activities related to the evaluation of human perception about vulnerability to change of climate; as well as some meetings were held between the PMU/MCTIC and UNDP to deal with the execution of the budget and planned activities until the end of the execution of the project considering its extension. Thus, we consider that the recommendation to drop or modify the Output 3.4 of the PRODOC at that time would not be appropriate, since there is a possibility of establishing a partnership that meets this demand. In addition, the recommendation for sharing the updated strategic work plan to the review and discussion with UNDP is ongoing.

It is important to note that faced with the new national political scenario and its respective priorities, it is possible that the process of approval of the 4NC document to be submitted to the UNFCCC will extend beyond the planned period. Thus, the extension of the project is absolutely necessary to ensure that all stages of project work and completion are completed in a safe and viable way.

Key Actions:

Key Action Responsible DueDate Status Comments Documents
1.1 Require the project extension (plus 16 months with no-cost). 1.2 Project extension proposal approval. 1.3 Discuss a revision proposal for indicator (or revision of means of monitoring/reporting) under Outcome 4. 1.4 Consolidate proposal including budget and calendar to cover the remainder period of project implementation (including the period of extension) to review and discuss with UNDP.
[Added: 2019/07/26]
PMU/MCTIC and UNDP 2020/08 Not Initiated
2. Recommendation:

The PMU/UNDP should avail of the opportunity to bring the lessons learned derived from the 4NC into the project design of the 5NC formerly and work for a more robust and clearer PRODOC and a Logframe that can be used as a useful M&E and reporting tool.  The recommended “no cost” extension above would provide an ideal opportunity to do this in parallel to the other proposed activities following the submission of the 4NC.xtension) to review and discuss with UNDP.

Management Response: [Added: 2019/07/26] [Last Updated: 2019/07/26]

The PMU/MCTIC fully agrees with this recommendation, including intending to contract specialized consulting for the development of PRODOC of 5NC, in order to ensure improvement in the project indicators.

Key Actions:

Key Action Responsible DueDate Status Comments Documents
2.1 Conduct a final evaluation workshop on project technical activities (focus on Outcomes 1 and 3) to identify lessons learned and opportunities for improvement. 2.2 Hire specialized consultants to develop the 5NC project document. 2.3 Conduct a final evaluation workshop on project to identify lessons learned and opportunities for improvement.
[Added: 2019/07/26]
PMU/MCTIC 2021/05 Not Initiated Activity already planned.
3. Recommendation:


The 5NC process provides the opportunity to continue to build on the strength of previous NCs.  It is suggested that the PMU, building on many of its own initiatives, should consider supporting: (i) greater inter-ministerial outreach; (ii) broadening the circle of traditional partner institutions; (iii) increasing public awareness and communication; (iv) incorporating State and municipal governments into the process; (v) putting greater emphasis on broadening and diversifying the message of the NC to reach “the people;” and (vi) developing a communication strategy and plan to be incorporated into project design of the 5NC.

Management Response: [Added: 2019/07/26]

The recommendation is aligned with expectations for the future project and, as far as possible, will be taken into account during the preparation of the 5NC project document.

Key Actions:

Key Action Responsible DueDate Status Comments Documents
3.1 Consider the recommendation at the time of the elaboration of PRODOC of 5NC.
[Added: 2019/07/26]
PMU/MCTIC 2021/06 Not Initiated The consultant to be hired will take this recommendation into consideration under guidance of PMU/MCTIC
4. Recommendation:

Simple process indicators should be developed and included in the 5NC results framework to facilitate tracking and evaluating project progress perhaps using the LOAs as a basis to identify indicators and targets per component/sub-component.

Management Response: [Added: 2019/07/26]

The recommendation is aligned with expectations for the future project and, as far as possible, will be taken into account during the preparation of the 5NC project document

Key Actions:

Key Action Responsible DueDate Status Comments Documents
4.1 Consider the recommendation at the time of the elaboration of PRODOC of 5NC
[Added: 2019/07/26]
PMU/MCTIC 2021/06 Not Initiated The consultant to be hired will take this recommendation into consideration under guidance of PMU/MCTIC
5. Recommendation:


For the 5NC it is recommended that an inception workshop, scheduled for the first trimester following project initiation, be built into project design with accompanying indicator and budget

Management Response: [Added: 2019/07/26]

The recommendation is aligned with expectations for the future project and, as far as possible, will be taken into account during the preparation of the 5NC project document. In addition, it intends to take advantage of the final period of execution of the 4NC project to ensure the best possible transition to the next project, in order to facilitate the realization of the inception workshop, and other important initial steps, in the beginning of 5NC.

Key Actions:

Key Action Responsible DueDate Status Comments Documents
5.1 Consider the recommendation at the time of the elaboration of PRODOC of 5NC and in the transition period among the projects.
[Added: 2019/07/26]
PMU/MCTIC 2021/06 Not Initiated This recommendation will be included into Project design
6. Recommendation:


In the coming 5NC, dependent on staff changeover in the PMU and broadening and inclusion of new partners, UNDP may want to consider supporting additional workshops early during the implementation of the project in particular in explaining administrative, budgeting and reporting requirements

Management Response: [Added: 2019/07/26]

The PMU/ MCTIC and UNDP recognize the importance of this recommendation and will take it into account in the process of implementing and during the 5NC project execution.

Key Actions:

Key Action Responsible DueDate Status Comments Documents
6.1 Take into account the recommendation during the implementation process of the 5NC to plan and organize workshops and training for capacity building in administrative, budgeting and reporting requirements.
[Added: 2019/07/26]
UNDP 2022/06 Not Initiated This recommendation will be implemented during project execution.
7. Recommendation:


The Mission recommends that the PMU consider the increased use of strategic workshops (e.g., similar to the February 2019 workshop) perhaps on an annual basis starting during the PRODOC design process of 5NC and followed in anticipation of AWP preparation.

Management Response: [Added: 2019/07/26]

Strategic planning meetings/workshops of the project team, such as the one held in February 2019, as well as monitoring meetings of the project activities with the partners are held periodically since 2017. The recommendation is aligned with expectations for the future project and, as far as possible, will be taken into account during the preparation of the 5NC project document.

Key Actions:

Key Action Responsible DueDate Status Comments Documents
7.1 Consider the recommendation at the time of the elaboration of PRODOC of 5NC.
[Added: 2019/07/26]
PMU/MCTIC 2021/06 Not Initiated UNDP will try to change the standard text that goes into the PRODOC to reflect the reality applied to the projects.
8. Recommendation:

The Mission recommends that the Tripartite Committee TPC should meet on a “when and as needed” basis.

Management Response: [Added: 2019/07/26]

In relation to the experience gained during the execution of the 4NC project, the recommendation reflects our perception that the frequency of meetings foreseen in PRODOC does not meet the real need. In this way, anticipating tripartite meetings only "when and as needed" is aligned with expectations for the future project and, as far as possible, will be taken into account during the preparation of the 5NC project document.

Key Actions:

Key Action Responsible DueDate Status Comments Documents
8.1 Consider the recommendation at the time of the elaboration of PRODOC of 5NC.
[Added: 2019/07/26]
PMU/MCTIC 2021/06 Not Initiated UNDP will try to change the standard text that goes into the PRODOC to reflect the reality applied to the projects.
9. Recommendation:

Though not a GEF requirement for EA projects, the PMU and UNDP may want to consider identifying and tracking sources and amounts of co-financing to have on hand at the time of the Terminal Evaluation (TE); always useful to report as an indicator of government and public support for the NC process.

Management Response: [Added: 2019/07/26]

This recommendation is very interesting and will be implemented as far as possible. This opportunity may indicate that the amount was even greater than initially estimated through the co–financing letters from the partner institutions.

Key Actions:

Key Action Responsible DueDate Status Comments Documents
9.1 To draw up the amounts related to the co-financing of the partner public institutions for the execution of the project
[Added: 2019/07/26]
PMU/MCTIC 2021/06 Not Initiated Even though this might be considered a good practice, there is a political element on the co-financing for EAs that shall not be ignored given non-Annex 1 positioning at UNFCCC.
10. Recommendation:

Brazil’s cumulative and evolutionary experience over 4 NCs has led to the development of relative sophisticated and effective institutional arrangements and processes, particularly in the preparation of the GHG Inventory.  This approach appears to be both highly valuable and likely to be unique.  It is suggested external resources should be sough, perhaps with the assistance of UNDP/GEF/UNFCCC to support greater efforts to disseminate these experiences (e.g., an international workshop attended by other countries to share experiences and compare lessons learned).

Management Response: [Added: 2019/07/26]

The recommendation is aligned with the work plan activities for this year and it will be put in place. The PMU plans discussions with other actors to share our experiences, as well as identifying ways of improving, including identifying other national and international events beyond CoP-UNFCCC.

Key Actions:

Key Action Responsible DueDate Status Comments Documents
10.1 Organize events to share our experiences (lessons learned and challenges) within the project. 10.2 Organize events to disseminate the results achieved through project execution.
[Added: 2019/07/26]
PMU/MCTIC 2020/06 Initiated The PMU/MCTIC recently consulted the national focal point with the UNFCCC for guidance on how to promote a side event at the next CoP. MCTIC and UNDP can look for alternatives to disseminate Project’s results.

Latest Evaluations

Contact us

220 East 42nd Street
20th Floor
New York, NY 10017
Tel. +1 646 781 4200
Fax. +1 646 781 4213
erc.support@undp.org