Final Evaluation of Project "Colombia’s Third National Communication on Climate change to the UNFCCC"

Report Cover Image
Evaluation Plan:
2015-2019, Colombia
Evaluation Type:
Final Project
Planned End Date:
08/2017
Completion Date:
09/2017
Status:
Completed
Management Response:
Yes
Evaluation Budget(US $):
25,000

Share

Document Type Language Size Status Downloads
Download document TERMINOS DE REFERENCIA Y CG DEL PNUD PARA IC.PDF tor Spanish 231.39 KB Posted 85
Download document Informe_TNC_final_inglés_oct2017.docx report English 1058.27 KB Posted 378
Download document Informe_TNC_final_sp_oct2017.docx report Spanish 3409.00 KB Posted 104
Title Final Evaluation of Project "Colombia’s Third National Communication on Climate change to the UNFCCC"
Atlas Project Number: 00073934
Evaluation Plan: 2015-2019, Colombia
Evaluation Type: Final Project
Status: Completed
Completion Date: 09/2017
Planned End Date: 08/2017
Management Response: Yes
Focus Area:
  • 1. Environment & Sustainable Development
  • 2. Others
Corporate Outcome and Output (UNDP Strategic Plan 2014-2017)
  • 1. Output 1.4. Scaled up action on climate change adaptation and mitigation across sectors which is funded and implemented
Evaluation Budget(US $): 25,000
Source of Funding: GEF
Evaluation Expenditure(US $): 23,688
Joint Programme: No
Joint Evaluation: No
Evaluation Team members:
Name Title Email Nationality
Jorge Armando Leiva Valenzuela jleiva@greenlaneconsultores.com
Jorge Armando Leiva Valenzuela jleiva@greenlaneconsultores.com
GEF Evaluation: Yes
GEF Project Title: Colombia’s Third National Communication on Climate change to the UNFCCC
Evaluation Type: Terminal Evaluation
Focal Area: Climate Change
Project Type: FSP
GEF Phase: GEF-5
GEF Project ID: 4619
PIMS Number: 4676
Key Stakeholders:
Countries: COLOMBIA
Lessons
1.

Lack of proper indicators and objectives made difficult the evaluation of the project, since they were based on collecting information and elaboration of products, masking in some way the results attained for institutional strengthening, awareness and impact.

From the international point of view, participation of experts from IPCC, FAO, etc., provided strength to the methodology developed by the project, but an additional effort will have to be made to reach a minimum level of agreements with relevant private and public sectors (energy, water, mining), concerning the type of information that is to be produced, indicators and joint actions to deal with climate change, to avoid cast doubt upon project results in the future.

Although the project achieved participation of stakeholders from private and public sectors, it did not have enough involvement of actors such as NGOs and regional and local authorities, thus it is still necessary to validate project results among these actors.

Documentation that systematize discussions made at groups’ project worktables was missing, therefore, it does not know the subjects where actors shown agreements and discords, and the ways by which these differences were overcame during the implementation of different project components. Moreover, no information is available on actual commitments that each actor assumed during project implementation, thus blurring the scope, attainments and progress made by these working groups.

The midterm evaluation was made late (final report delivered by end of January 2017), this situation deducted impact from this activity, and there is no evidence on how its recommendations and conclusions were addressed.

50% of the country is “sea”, but it is a sector that has few investments for monitoring specific marine climate related data, and this can affect the quality of simulation for scenarios from national communications, therefore, an effort should be made to increase this type of monitoring stations.


Findings
1.

It was noted that during project elaboration, stakeholder participation was improved when compared with the former two national communications, which allowed to incorporate lessons learnt from these communications and from other climate change related projects as well, with the result that this project incorporates methodological and stakeholder participation innovations that former communications did not have.

Thus, the project is and is still relevant for the country and its authorities, and complies with GEF relevance criterium, in the sense of that project results are included in country’s development programs, on UNDP’s country program and UNDAF’s assistance framework.

Regarding project indicators and logical framework matrix, it was noted that these are a mere listing of desired products with no targets for midterm review, thus concluding that indicators do not satisfy the SMART criterium. This situation -also noted during the midterm review- makes difficult to evaluate results and, therefore, it covers-up project impacts.

For the implementation stage, a good participation of actors distributed in different technical worktables was noted and, besides, a communicational strategy based on showing partial progress for each project product was promoted, thus maintaining in such a way, the actors’ interest in country’s climate change issue. Despite of these advances, there was no important involvement of NGO, neither regional nor local environmental authorities, therefore, more efforts in this sense will be needed in the future. 

The M&E system was only regular. Annual Work Plans (AWP) were excel sheets that only included activities, tentative deadlines for implementation and allocated budget, but these were not supported by a strategic document showing the reasons for such activities, its prioritization and expected results, thus making these AWP a compilation of actions only. On the other hand, the project executive committee did not make activities of strategic follow-up, but it was an instance for reporting project activities only. Besides, when decisions were made, no follow-up for responsibilities, deadlines nor progress reports for activities’ results were included.

The midterm review (MTR) was made late (mission on September 2016 and final report on January 2017), therefore its recommendations did not have the expected impact.

Regarding financing performance, this was efficient and effective, considering that products and activities generated by the project were beyond the expectations appeared on the project document. However, it was noted that 30% of disbursements recorded in the UNDP’s ATLAS management system did not contain beneficiary names of awarded contracts (project team stated that these were their salaries). 

At the time of the final evaluation (July 2017), 85% of GEF resources were spent (US$ 1.7 million), leaving a remaining close to US$ 376,000. In cash co-financing from government was near US$ 260,000, thus being in compliance with what was committed, whereas in-kind contribution was US$ 1.96 million, exceeding in a 143% of what was stipulated on the project document.

Despite project was delayed and had to be extended by an additional year, at the moment of the final evaluation the products obtained are of very good quality, exceeding prodoc expectations in the sense that methodological developments have been added for calculation of GHG inventory and analysis for vulnerability scenarios. These developments include new indicators for vulnerability based on risk analysis, uncertainty calculations for parameters and results from modelling. Land scale could also be reduced for vulnerability analysis and GHG inventory, reaching a level of detail for regions and municipalities (scale 1:100,000). The above was complemented by a strong awareness and communication campaign towards public and specific stakeholders, as well as project dissemination made jointly with COLCIENCIAS, amongst students from 1,000 schools around the country.

Regarding sustainability of results, no significative problems of institutionality nor country’s political instability were noted, but reductions of nearly 60% in budgets for the environmental sector were noted in 2017, situation that could prevent continuity of project activities, at least those depending from state financing. Therefore, the main challenge in the short and mid-term (3-4 years)  is the continuity of the work made specifically in the generation of information and improvements of the methods for calculation of GHG inventory and vulnerability, since the institutionality of IDEAM has not been able to incorporate the project’ technical team due to lack of resources, therefore this kind


Recommendations
1

Los futuros proyectos de comunicación nacional debieran contener indicadores y objetivos consistentes con los efectos que se desean lograr y no ser expresados como productos, ya que esto limita los alcances de los proyectos al momento de evaluar sus resultados. Los resultados debieran estar expresados en lenguaje de cambio y los indicadores debieran ser una medida de resultados que se desean lograr. También es fundamental que los proyectos de este tipo incluyan metas para la evaluación de mitad de período.

2

Sería adecuado ir identificando las lecciones aprendidas a medida que se van produciendo las situaciones claves en la implementación del proyecto, como señal de buena práctica

3

Los POA deberían ser documentos estratégicos que respalden las razones por la cuales se realizan las actividades y como se priorizan éstas dentro del contexto de la estrategia o ajustes a los proyectos. Las planillas Excel son útiles, pero solo son parte de la programación de un proyecto

4

Se sugiere que cuando se conformen grupos de trabajo con varios actores, se reporten los acuerdos, desacuerdos y compromisos que se logren al interior de este tipo de instancias, de manera de conservar la “memoria institucional” de los proyectos

5

Se sugiere que, en el futuro, se realicen las evaluaciones de medio término lo más ajustadas posible a la mitad de período, porque de otro modo, las evaluaciones no reflejan la realidad de los proyectos de ese período y las recomendaciones son difíciles de implementar en corto tiempo

6

Se sugiere que las transacciones registradas en el sistema ATLAS contengan todos los datos de los beneficiarios de los contratos realizados, incluyendo al personal del proyecto, con el objetivo de mantener la transparencia de los procesos.

7

Para futuros proyectos de comunicaciones nacionales, se sugiere que en el diseño se incluya una componente de replicación de resultados y la elaboración de una estrategia de salida que permita visualizar como, donde y cuando se podrían utilizar los modelos y resultados de estos proyectos en la planificación territorial de las regiones y autoridades locales, al tiempo de proponer mecanismos de financiamiento para este tipo de actividades

1. Recommendation:

Los futuros proyectos de comunicación nacional debieran contener indicadores y objetivos consistentes con los efectos que se desean lograr y no ser expresados como productos, ya que esto limita los alcances de los proyectos al momento de evaluar sus resultados. Los resultados debieran estar expresados en lenguaje de cambio y los indicadores debieran ser una medida de resultados que se desean lograr. También es fundamental que los proyectos de este tipo incluyan metas para la evaluación de mitad de período.

Management Response: [Added: 2018/01/17]

Procesamiento de Management response e inclusión en la sección de lecciones aprendidas en el informe de cierre

Key Actions:

Key Action Responsible DueDate Status Comments Documents
• Entre de documentos e informes técnicos para el cierre del proyecto
[Added: 2018/01/17]
IDEAM- PNUD 2017/11 Completed
2. Recommendation:

Sería adecuado ir identificando las lecciones aprendidas a medida que se van produciendo las situaciones claves en la implementación del proyecto, como señal de buena práctica

Management Response: [Added: 2018/01/17]

Procesamiento de Management response e inclusión en la sección de lecciones aprendidas en el informe de cierre

Key Actions:

Key Action Responsible DueDate Status Comments Documents
• Entre de documentos e informes técnicos para el cierre del proyecto
[Added: 2018/01/17]
IDEAM- PNUD 2017/11 Completed
3. Recommendation:

Los POA deberían ser documentos estratégicos que respalden las razones por la cuales se realizan las actividades y como se priorizan éstas dentro del contexto de la estrategia o ajustes a los proyectos. Las planillas Excel son útiles, pero solo son parte de la programación de un proyecto

Management Response: [Added: 2018/01/17]

Procesamiento de Management response e inclusión en la sección de lecciones aprendidas en el informe de cierre

Key Actions:

Key Action Responsible DueDate Status Comments Documents
• Entre de documentos e informes técnicos para el cierre del proyecto
[Added: 2018/01/17]
IDEAM-PNUD 2017/11 Completed
4. Recommendation:

Se sugiere que cuando se conformen grupos de trabajo con varios actores, se reporten los acuerdos, desacuerdos y compromisos que se logren al interior de este tipo de instancias, de manera de conservar la “memoria institucional” de los proyectos

Management Response: [Added: 2018/01/17]

El IDEAM ha hecho una amplia difusión de los resultados de TCN en territorio

Key Actions:

Key Action Responsible DueDate Status Comments Documents
• Realización de talleres por cuenta del IDEAM
[Added: 2018/01/17]
IDEAM 2017/11 Completed
5. Recommendation:

Se sugiere que, en el futuro, se realicen las evaluaciones de medio término lo más ajustadas posible a la mitad de período, porque de otro modo, las evaluaciones no reflejan la realidad de los proyectos de ese período y las recomendaciones son difíciles de implementar en corto tiempo

Management Response: [Added: 2018/01/17]

El acceso a consultores expertos disponibles en Latinoamérica con experiencia en proyectos GEF es limitada. Se preverá convocatorias más amplias en tiempo para este proceso

Key Actions:

Key Action Responsible DueDate Status Comments Documents
Observación incluida en lecciones aprendidas
[Added: 2018/01/17]
IDEAM- PNUD 2017/11 Completed
6. Recommendation:

Se sugiere que las transacciones registradas en el sistema ATLAS contengan todos los datos de los beneficiarios de los contratos realizados, incluyendo al personal del proyecto, con el objetivo de mantener la transparencia de los procesos.

Management Response: [Added: 2018/01/17]

Procesamiento de Management response e inclusión en la sección de lecciones aprendidas en el informe de cierre

Key Actions:

Key Action Responsible DueDate Status Comments Documents
• Entre de documentos e informes técnicos para el cierre del proyecto
[Added: 2018/01/17]
IDEAM-PNUD 2017/11 Completed
7. Recommendation:

Para futuros proyectos de comunicaciones nacionales, se sugiere que en el diseño se incluya una componente de replicación de resultados y la elaboración de una estrategia de salida que permita visualizar como, donde y cuando se podrían utilizar los modelos y resultados de estos proyectos en la planificación territorial de las regiones y autoridades locales, al tiempo de proponer mecanismos de financiamiento para este tipo de actividades

Management Response: [Added: 2018/01/17]

El IDEAM ha hecho una amplia difusión de los resultados de TCN en territorio

Key Actions:

Key Action Responsible DueDate Status Comments Documents
• Realización de talleres por cuenta del IDEAM
[Added: 2018/01/17]
IDEAM 2017/11 Completed Esta es una actividad realizada por el IDEAM luego del cierre del proyecto

Latest Evaluations

Contact us

220 East 42nd Street
20th Floor
New York, NY 10017
Tel. +1 646 781 4200
Fax. +1 646 781 4213
erc.support@undp.org