Fortalecimiento del proceso de preparacion sobre REDD+ en Mexico y fomento a la cooperacion Sur-Sur (EvF)

Report Cover Image
Evaluation Plan:
2014-2019, Mexico
Evaluation Type:
Final Project
Planned End Date:
01/2016
Completion Date:
06/2016
Status:
Completed
Management Response:
Yes
Evaluation Budget(US $):
30,000

Share

Document Type Language Size Status Downloads
Download document Draft ToRs Terminal EvaluationREDD plus.docx tor English 595.48 KB Posted 246
Download document Evaluation UNDP PROJECT 00079208 Mexico.pdf report English 1909.01 KB Posted 439
Title Fortalecimiento del proceso de preparacion sobre REDD+ en Mexico y fomento a la cooperacion Sur-Sur (EvF)
Atlas Project Number: 79208
Evaluation Plan: 2014-2019, Mexico
Evaluation Type: Final Project
Status: Completed
Completion Date: 06/2016
Planned End Date: 01/2016
Management Response: Yes
Focus Area:
  • 1. Others
Corporate Outcome and Output (UNDP Strategic Plan 2014-2017)
  • 1. Output 1.5. Inclusive and sustainable solutions adopted to achieve increased energy efficiency and universal modern energy access (especially off-grid sources of renewable energy)
Evaluation Budget(US $): 30,000
Source of Funding:
Joint Programme: No
Mandatory Evaluation: No
Joint Evaluation: No
Evaluation Team members:
Name Title Email Nationality
Maria Onestini Ms rponesti@criba.edu.ar ARGENTINA
GEF Evaluation: No
Key Stakeholders:
Countries: MEXICO
Lessons
Findings
Recommendations
1

The value of a robust design cannot be underestimated. The design of any project, especially one as substantial as this one, should encompass a series of aspects that cannot be left to be defined at the implementation stage. Matters such as a results based log frame, indicators of achievement, roles of agencies and institutions involved need to be clearly defined at the design level before implementation begins. 

2
Results-based projects (ie those that seek results, effects and impacts, not just products) need to include this goal in the design and implementation processes. The effects and results have to be fully operational throughout the implementation process, and it is not desirable to expect them to be generated automatically from the products and processes
3

Design and inception should also clearly delineate what the roles of each agency and institutional stakeholder should be, taking into account agencies value added and national capacities and needs, as well as what each institution’s and agencies’ objectives are within a project.

4

Sustainability of results should begin to be drawn at the very design stages, with a clear exit strategy, an institutionalization framework, and a genuine financial plan to support continuity of achievements and processes.

5

Design should be realistic also as to the time needed to implement complex monitoring projects, also including realistic estimations of startup, as well as accurate estimates regarding setting up and running all technical aspects of a project.

6

The articulation and linking of information systems with public policy in natural resource management and sustainable development should also be clearly delineated from the inception and not be left only as enunciations for these aspects are to be pursued.

7

Projects should implement knowledge management processes and systematic mapping of its achievements (expected and unexpected) throughout their life span, not only to generate visibility for its outputs, outcomes, and results but also to record institutional learning and become an institutional memory with the aim of sustainability of results. This knowledge management processes and systematic mapping is also a useful tool follow up and to reinforce initial benefits from a project on the national and international scales.

8

An exit strategy should begin to be designed from the very first stages of implementation, and implemented as much as possible from the very first stages of achievements. Although sustainability in terms of institutionalization is not the ultimate responsibility of a project, given that it is the country through its institutions that creates the conditions for sustainability and/or institutionalization, an exit strategy designed early in the process of implementation can have more chances of sustaining project gains after a project concludes, in particular when referring to complex projects such as the one evaluated here and where government is an implementing partner.

9

The internal monitoring of a project cannot be carried out sporadically throughout the intervention, and external evaluation tools should be implemented when necessary, such as mid-term evaluations, which should not be used only for the Report, but also to make required adjustments in a timely manner.

10

In a project of this technical complexity should consider since the beginning the work of a Specialist in Monitoring and Evaluation within its organization chart, who should be in charge of the functions of planning, monitoring, evaluation, reporting and support the processes of capacity building and knowledge management.

11

Cooperation (be it South – South / triangular or any mix thereof) should also be clearly defined by all parties involved, including cooperation recipients. What is meant by cooperation should also be well-defined, especially when working with countries with varying needs and capacities.

12

The utility, use, and function of such a complex and sophisticated system of monitoring, such as the monitoring system here developed should be clear and assimilated throughout the design and implementation process. The multiple goals and varied uses of should be fostered by working with diverse actors in order to not only monitor forest resources but also impel the use of monitoring data and findings for the generation of sustainable integrated public policy in national resource use and management by national and subnational institutions.

1. Recommendation:

The value of a robust design cannot be underestimated. The design of any project, especially one as substantial as this one, should encompass a series of aspects that cannot be left to be defined at the implementation stage. Matters such as a results based log frame, indicators of achievement, roles of agencies and institutions involved need to be clearly defined at the design level before implementation begins. 

Management Response: [Added: 2016/12/01]

These design issues were addressed at the project implementation stage. Even when they should have been addressed at the design stage, actions taken at the implementation stage helped to correct significant design errors and improved project management. 

Key Actions:

Key Action Responsible DueDate Status Comments Documents
1. Redefinition of the Project Results Framework 2. Redesign of the Project Monitoring Framework (Construction of a Result Indicator System) 3. Even when the roles of the actors were considered during the design stage (PRODOC), some specific functions and responsibilities were established during the implementation stage of the project: (i) In the collaboration agreements; (Ii) In the planning of specific products that required inter-institutional coordination (e.g BUR); (Iii) In the South-South Cooperation Strategy, etc.
[Added: 2016/12/01]
Country Office & Project No due date Completed actions have been completed
2. Recommendation:
Results-based projects (ie those that seek results, effects and impacts, not just products) need to include this goal in the design and implementation processes. The effects and results have to be fully operational throughout the implementation process, and it is not desirable to expect them to be generated automatically from the products and processes
Management Response: [Added: 2016/12/01]

Even when the original design of the project had weaknesses to include the Results- Based Management approach in the design tools, adaptive management that was carried out during the last months of 2014 reached to include this approach in the Results Framework (MRP in Spanish) and in the Project Monitoring Framework (MMP), and in planning instruments such as the Annual Work Programs (PAT).

Regarding implementation stage, the project was guided by the operationalization of effects and results. This could be appreciated in Outcomes 1 and 2, which were developed within a framework of synergy and institutional operation, responding to national and regional needs.

In the case of Outcome 1, it is important to clear up that the MRV System is generated from the compliance of its products, processes and subsystems, for this reason, the MMP for this outcome was built beneath this logic.

 

 

Key Actions:

Key Action Responsible DueDate Status Comments Documents
1. Redefinition of MRP and MMP, applying more coherence in terms of the results chain. 2. Development of PAT 2015 (including PAT Aug-Dec 2015) through the Results-Based Management approach. 3. In Outcome 2, a South-South Cooperation Strategy was developed which integrated the Results-Based Management approach. This action was undertaken in the early stage of implementation, which allowed to make operative the component 2 in a logic of obtaining results and effects
[Added: 2016/12/01]
CO and project 2015/02 Completed completed
3. Recommendation:

Design and inception should also clearly delineate what the roles of each agency and institutional stakeholder should be, taking into account agencies value added and national capacities and needs, as well as what each institution’s and agencies’ objectives are within a project.

Management Response: [Added: 2016/12/02]

At the design stage, PRODOC established in a general way, the responsibilities of the actors, agencies and institutions involved. The specific functions and objectives were set up during the implementation phase, in the collaboration agreements and the institutional arrangements resulted from the participation platforms created by the project or in which the project participated.

Key Actions:

Key Action Responsible DueDate Status Comments Documents
1. Development of collaboration agreements with different institutions and agencies (specific functions were established in these agreements) 2. Signature of Inter-agency agreement UNDP-FAO for Project Implementation
[Added: 2016/12/02]
Country Office, Project & CONAFOR No due date Completed The action complited
4. Recommendation:

Sustainability of results should begin to be drawn at the very design stages, with a clear exit strategy, an institutionalization framework, and a genuine financial plan to support continuity of achievements and processes.

Management Response: [Added: 2016/12/02]

Despite an exit strategy was not considered at the design or start-up stage, the definition of the governance structure of the project (designating the General Coordinator of Planning and Information of CONAFOR as National Project Director and ensuring the alignment of the Project with institutional priorities) eased the process of institutional operation of the project outcomes.

At the implementation stage, a Proposal of Institutional operation of the MRV System was designed, which included a strategy to integrate this system within the institutional structures, in addition to other key actions to make official other important results, such as the Virtual Center of Excellence in Forest Monitoring.

Key Actions:

Key Action Responsible DueDate Status Comments Documents
1. Appointment of the General Coordinator of Planning and Information of CONAFOR as the National Project Director. 2. Design and delivery of a proposal of institutional operation of the MRV system to the Government of Norway.
[Added: 2016/12/02]
Country Office, Project & CONAFOR 2015/05 Completed Completed
5. Recommendation:

Design should be realistic also as to the time needed to implement complex monitoring projects, also including realistic estimations of startup, as well as accurate estimates regarding setting up and running all technical aspects of a project.

Management Response: [Added: 2016/12/02]

The design did not consider the required period to achieve the expected results and products. However, the following column lists some actions that improved the planning of activities and mitigated this design flaw

Key Actions:

Key Action Responsible DueDate Status Comments Documents
1. Once the MMP was able to calculate the overall progress of the project, it was possible to objectively make a strict prioritization of activities, applying a discrimination of those that were not essential for the achievement of the expected products. 2. Extension of the project staff, depending on the identification of needs and / or specific areas. This is clear for Outcome 2 (for example, the M & E specialist had functions of this component) and for Outcome 1. 3. Hiring key consultancies to achieve specific products: some specific technical products and sub-products of the MRV System were completed through specialized consultancies.
[Added: 2016/12/02]
Project 2015/01 Completed Completed all actions
6. Recommendation:

The articulation and linking of information systems with public policy in natural resource management and sustainable development should also be clearly delineated from the inception and not be left only as enunciations for these aspects are to be pursued.

Management Response: [Added: 2016/12/02]

The redefinition of the Project Results Framework enclosed the scope in this regard. In particular, the content of Intermediate Outcome 3.1 was clearly defined. Identification of national policy linkages with REDD + monitoring efforts.

On the other hand, as a result of the governance structure of the project, the institutional arrangements and the technical nature of the intermediate results of Outcome 1, it was possible to had key contributions during the implementation stage of the project for the design of public policies (INDC's, IRE, Forests and Climate Change Project, etc.) and that the potential of the MRV System will be used more and with greater clarity.

Key Actions:

Key Action Responsible DueDate Status Comments Documents
1. Preparation of the document: "Links of the MRV System with the management of public policies relevant to REDD +", which has served as an input to date to publicize the potential of the MRV System, especially in CONAFOR. 2. Contribution of basic information for the design and management of public policies and programs in the forestry sector (INDC's, IRE, Forest and Climate Change Project, SIGECO Platform, etc.). 3. In the Proposal of the institutional operation of the MRV System and the new MRV Implementation Project (to begin in the following months with FAO), the articulation and linkage with the management of CONAFOR's public policies and programs has been considered, through the Management of the National System of Forest Monitoring.
[Added: 2016/12/02]
Project & CONAFOR 2016/12 Completed Actions Completed
7. Recommendation:

Projects should implement knowledge management processes and systematic mapping of its achievements (expected and unexpected) throughout their life span, not only to generate visibility for its outputs, outcomes, and results but also to record institutional learning and become an institutional memory with the aim of sustainability of results. This knowledge management processes and systematic mapping is also a useful tool follow up and to reinforce initial benefits from a project on the national and international scales.

Management Response: [Added: 2016/12/02]

i) The first focused only on the information management level, through the creation of the Project website (www.mrv.mx), framed in a communication strategy and systematization of project products and activities.
ii) The establishment of a Virtual Center of Excellence in Forest Monitoring, whereby knowledge management is the central focus, considering information, network, learning and change management.
Regarding the achievements mapping (expected and unexpected), although some of them were systematized in institutional reports of CONAFOR, it was in the Project Final Report that a greater collecting and systematization effort was made.

Key Actions:

Key Action Responsible DueDate Status Comments Documents
1. Development and maintenance of the Project website (www.mrv.mx) for the management of technical information (documents, videos, tutorials, etc.) 2. Establishment of the Virtual Center of Excellence in Forest Monitoring of Mesoamerica (CEVMF). This is one of the main results of the project, and currently is in systematization process of the information produced by the project, including the objects of knowledge and learning. 3. Results mapping (expected and unexpected) in the Project Final Report
[Added: 2016/12/02]
Country Office, Project & CONAFOR 2016/03 Completed Actions Completed
8. Recommendation:

An exit strategy should begin to be designed from the very first stages of implementation, and implemented as much as possible from the very first stages of achievements. Although sustainability in terms of institutionalization is not the ultimate responsibility of a project, given that it is the country through its institutions that creates the conditions for sustainability and/or institutionalization, an exit strategy designed early in the process of implementation can have more chances of sustaining project gains after a project concludes, in particular when referring to complex projects such as the one evaluated here and where government is an implementing partner.

Management Response: [Added: 2016/12/02]

Even when the Exit Strategy or the Institutional Operation Proposal was designed in the final stage of implementation, the main actions to claim the long-term sustainability of the project results are associated to this strategy.
On the other hand, as mentioned above, the governance structure of the project (aligned with institutional priorities) and the joint work of its technical specialists with CONAFOR staff and other institutions (INECC, CONABIO and INEGI) eased some institutional operation processes.

Key Actions:

Key Action Responsible DueDate Status Comments Documents
1. Design and delivery of a proposal for the institutional operation of the MRV system to the Government of Norway, 2. Establishment of the Management of the National Forest Monitoring System within the General Coordination of Planning and Information of CONAFOR, with specific responsibilities to ensure MRV System operation. 3. Creation of a Technical Unit Specialized in MRV, to preserve the technical capacities created by the Project.
[Added: 2016/12/02]
Country Office, Project & CONAFOR 2016/01 Completed Actions Completed
9. Recommendation:

The internal monitoring of a project cannot be carried out sporadically throughout the intervention, and external evaluation tools should be implemented when necessary, such as mid-term evaluations, which should not be used only for the Report, but also to make required adjustments in a timely manner.

Management Response: [Added: 2016/12/02]

This is a weakness of the implementation stage of the project. However, following the redesign of the Project Monitoring Framework, monitoring progress has been systematic on a monthly basis.
On the other hand, even without a mid-term evaluation (which is absolutely necessary for further initiatives), the redefinition of the Results Framework and the measurement of progress with the Indicators System served to redirect and adjust activities in the Final stage of the project and to reach expected results.

Key Actions:

Key Action Responsible DueDate Status Comments Documents
1. Systematic monitoring of project progress and early warning alerts (red lights). 2. Redefinition of the results framework and systematic monitoring made it possible to prioritize the activities and focus the project's efforts and resources on the key products (some of them were just starting to be developed). This action made it possible to improve planning at the final stage of the project and achieve the expected results.
[Added: 2016/12/02]
M&E Specialist and Project Coordinator 2016/03 Completed Actions completed
10. Recommendation:

In a project of this technical complexity should consider since the beginning the work of a Specialist in Monitoring and Evaluation within its organization chart, who should be in charge of the functions of planning, monitoring, evaluation, reporting and support the processes of capacity building and knowledge management.

Management Response: [Added: 2016/12/02]

An M & E Specialist was hired, who was in charge of the monitoring, evaluation and reporting functions. In addition, he supported planning, capacity building and knowledge management activities.

Key Actions:

Key Action Responsible DueDate Status Comments Documents
In 2014 the Project Board recommended to hire a Specialist in Monitoring and Evaluation (M & E) because at the beginning of the project there was not an internal specific area in charge of M & E activities. On August 1, 2014, the Specialist was hired, some he was responsible for determining if the results effectively assist the realization of the objectives and the results framework of the project.
[Added: 2016/12/02]
Country Office & Project 2014/08 Completed Completed
11. Recommendation:

Cooperation (be it South – South / triangular or any mix thereof) should also be clearly defined by all parties involved, including cooperation recipients. What is meant by cooperation should also be well-defined, especially when working with countries with varying needs and capacities.

Management Response: [Added: 2016/12/02]

The concept of South-South Cooperation implemented in the project is used by UNDP-Mexico, which is based on the definition of the High-Level Committee for South-South Cooperation of the United Nations.

Key Actions:

Key Action Responsible DueDate Status Comments Documents
1. The project received the advice of the South-South Cooperation area of UNDP-Mexico for the implementation of the South-South Cooperation Strategy and the creation of the Virtual Center of Virtual Excellence in Forest Monitoring. 2. In the design of the South-South Cooperation Strategy, the project defined a set of guiding principles of this strategy, such as compatibility with recipient countries' needs, quality and neutrality, among other principles. 3. As part of the South-South Cooperation Strategy, a diagnostic of the State of National Forest Monitoring Systems in Mesoamerica was made. A mapping of needs (demand) of EMSA countries was also done, which is part of the Long-term Work Plan, agreed by all countries.
[Added: 2016/12/02]
Country Office & Project 2015/01 Completed Actions Completed
12. Recommendation:

The utility, use, and function of such a complex and sophisticated system of monitoring, such as the monitoring system here developed should be clear and assimilated throughout the design and implementation process. The multiple goals and varied uses of should be fostered by working with diverse actors in order to not only monitor forest resources but also impel the use of monitoring data and findings for the generation of sustainable integrated public policy in national resource use and management by national and subnational institutions.

Management Response: [Added: 2016/12/02]

The utility and functions of the MRV System became clear in the final stage of project implementation, particularly when its sub-systems or tools were completed. Joint collaboration  with the Forestry and Geomatics Inventory Management (now National Forestry Monitoring System Management), helped to clarified the potential of the MRV System However, it was not until the design of the Technical Unit Specialized in MRV that a part of its potentialities were detailed in terms of the expected products of this Unit.

Key Actions:

Key Action Responsible DueDate Status Comments Documents
1. During the last Project Planning meeting, joint actions with CONAFOR were identified in order to ensure the administrative and technical closure of the project and also potential products of the MRV System were identified in 2016 through the Specialized Technical Unit In Forest Monitoring. These potential products are considered within the framework of a National Forest Monitoring System, which involves multiple actors and technical areas of CONAFOR. 2. Development of the document: "Ties of the MRV System with the management of relevant public policies to REDD +", which has served as an input to make known the potential of the MRV System, especially in CONAFOR.
[Added: 2016/12/02]
Project & CONAFOR 2015/12 Completed Completed

Latest Evaluations

Contact us

220 East 42nd Street
20th Floor
New York, NY 10017
Tel. +1 646 781 4200
Fax. +1 646 781 4213
erc.support@undp.org