Mid-Term Review: Achieving Low Carbon Growth in Cities through Sustainable Management in Thailand

Report Cover Image
Evaluation Plan:
2017-2021, Thailand
Evaluation Type:
Mid Term Project
Planned End Date:
03/2020
Completion Date:
04/2020
Status:
Completed
Management Response:
Yes
Evaluation Budget(US $):
42,000

This evaluation has been rescheduled to be Jan-Mar 2020 due to delays of implementation. The work schedule is now available and the solicitation of the consultants process is in progress.

Share

Document Type Language Size Status Downloads
Download document UNDP-GEF Midterm Review ToR International_13 Nov 19.docx tor English 117.20 KB Posted 47
Download document UNDP-GEF Midterm Review ToR national_19 Nov 19_ss.docx tor English 83.67 KB Posted 49
Download document LCC-MTR-Final Report_200423.pdf report English 2072.17 KB Posted 12
Title Mid-Term Review: Achieving Low Carbon Growth in Cities through Sustainable Management in Thailand
Atlas Project Number: 00086188
Evaluation Plan: 2017-2021, Thailand
Evaluation Type: Mid Term Project
Status: Completed
Completion Date: 04/2020
Planned End Date: 03/2020
Management Response: Yes
UNDP Signature Solution:
  • 1. Energy
Corporate Outcome and Output (UNDP Strategic Plan 2018-2021)
  • 1. Output 2.1.1 Low emission and climate resilient objectives addressed in national, sub-national and sectoral development plans and policies to promote economic diversification and green growth
SDG Goal
  • Goal 13. Take urgent action to combat climate change and its impacts
  • Goal 17. Strengthen the means of implementation and revitalize the Global Partnership for Sustainable Development
SDG Target
  • 13.2 Integrate climate change measures into national policies, strategies and planning
  • 17.9 Enhance international support for implementing effective and targeted capacity-building in developing countries to support national plans to implement all the Sustainable Development Goals, including through North-South, South-South and triangular cooperation
Evaluation Budget(US $): 42,000
Source of Funding: GEF
Evaluation Expenditure(US $): 42,000
Joint Programme: No
Joint Evaluation: No
Evaluation Team members:
Name Title Email Nationality
GEF Evaluation: Yes
GEF Project Title: Achieving Low Carbon Growth in Cities through Sustainable Management in Thailand
Evaluation Type: Mid-term Review
Focal Area: Climate Change
Project Type: FSP
GEF Phase: GEF-5
GEF Project ID:
PIMS Number: 4778
Key Stakeholders:
Countries: THAILAND
Comments:

Due to implementation delays caused by the government's changed procurement procedures, the work on developing carbon emission baseline for selected low carbon pilots, city's carbon footprints as well as the monitoring and verification system was delayed by almost one year in the beginning. Thereby, making it difficult for the MTR to undertake necessary assessments of the projects if the MTR were to be conducted within the original schedule. This rescheduling has been consulted with the implementing partner and concurred by the RTA.

Lessons
Findings
Recommendations
1

Mid-term Review Rating: Objective Achievement Rating:  Moderately Unsatisfactory (MU). 33,195.72 tCO2e as of 2019 vs targets: 177,708/182,000 (update) tCO2eq completed by 18.24%, far below the target.

  • KK: 23,923.37 tCO2e vs target: 100,500 tCO2e; completed by 23.80% 
  • NR: 7,705.47 tCO2e vs target: 10,000 tCO2e; completed by 77.06%
  • CM: 946.47 tCO2e vs target: 70,000 tCO2e; completed by 1.35%
  • SM: 620.41 tCO2e vs target: 1,500 tCO2e; completed by 41.36%

Cumulative direct GHG emission reductions is a good indicator. In practice, (i) some components might not generate cumulative direct GHG emission reductions. (ii) Other components might have already achieved cumulative direct GHG emission reductions without the Project.

177,708 tCO2 eq. was considered as an unrealistic target for goal as the realized cumulative direct GHG emission reductions resulting from the technical assistance and investments up to the end of 2019 and by the end-of-project were significantly lower than this figure.

The actual amount of fuel saving was not available and Annual amount of waste gainfully used was 244,043.36 tonnes as of Dec 2019; completed by 63%.

2

A good PLF [Project Log Frame] design always results in a good implementation, which in turn results in good project outcomes. The Project should update the LF by taking into account the chain between activities, outputs and outcome and also the chain between the results, targets and indicators as these two logical chains provide a powerful instrument for managing and monitoring the project implementation. More importantly, any updates on outcomes, outputs and activities should take into account the indicators and targets at the same time, vice versa.

3

The Project should use the consistent methodology to estimate the GHG emission reduction for both target and performance evaluation purposes. The Project should also use the traditional approach based on some assumptions to verify and estimate the actual GHG emission reduction for GEF. In the same time, the Project should also utilize ERM service to re-estimate the goal of GHG emission reduction and distribute total amount of emission reduction among subprojects in four municipalities based on the activities undertaken and the total target.

4

The Project should update the first two indicators and targets of the objective. The Project can delete the existing first two indicators and targets as the two indicators do not provide consistent measurement. Alternatively, the Project can re-design the indicators of the objective by three sectors: waste management, transport and energy. In addition, the targets of objective by sector should be consistent with the target of goal.

5

The Project should work on the gap after the end of MTR so as to enhance the effectiveness of the Project. In particular, the Project should give high priority to the key outputs that have been delivered but have not yet been implemented in order to enhance the impact of the subprojects, particularly

  • Electronic Common Ticket for all urban transit, Chiang Mai;
  • Real time tracking system via on-line application, Chiang Mai
  • Organic Waste Management Improvement for Samui Organics Recycling Bophut Station, Samui
  • Organic Waste Management Improvement for Baan Ya Suan Pu Station, Samui
  • Energy saving for the household sector, Nakhon Ratchasima
  • Low emissions building for the department stores and malls, Nakhon Ratchasima
  • Damage cost study from traffic congestion in the municipality area, Nakhon Ratchasima

Light Rail Transit (LRT), Khon Kean

6

The Project should move the first general indicator and target of Outcome 2.1 to Outcome 1.2 as total amount of new investment leveraged through local plans of participating cities for low carbon projects by EOP is closely associated with the investment in the subprojects in 4 municipalities under Outcome 1.2. Further, the Project should re-design the indicators and target for Output 1.2 2 through Output 1.2.5 (subprojects in 4 municipalities) as the related activities have been updated.

7

The Project should update output 2.1.1 and associated indicator and target as the activity has been updated.  As the first general indicator and target has been moved to Outcome 1.2, the remaining activities under Outcome 2.1 can be re-grouped into a component called “Capacity Building”. In addition, the Project should update the name Outcome 2.1 and related indicators and target.

8

The implementation of the Project after the MTR should take into account the AWP so as to ensure that the level of resourcing and implementation timeframe are better aligned with the objectives and scope of the Project. Also, the Project should install a Project Management System by incorporating project accounting, procurement, asset management, and grant disbursement. The system should be connected to local municipality and UNDP to avoid delay in reconciliation and approval.

9

The Project should change the methodology for co-financing from an official co-financing letters provided to PMU to an innovative leveraged investment approach, where the actually realized co-financing contributions against the sizable commitment should be provided before the actual disbursement of GEF funds.

10

In addition to the target of GHG emission reduction for the Project and the subprojects in 4 municipalities, the Project should design a set of indicator and target for 6 activities in Samui, and 4 activities in Chang Mai, Khon Kaen Nakhon Ratchasima respectively. For 4 activities in Chiang Mai as an example, no. of passengers in the public (integrated) transport system might be a good indicator to avoid small incremental low carbon benefits during the project life. Other indicators can be designed for the Light Rail Transit (LRT) in Khon Kaen to avoid no benefits from emission reduction before the end of the Project.

1. Recommendation:

Mid-term Review Rating: Objective Achievement Rating:  Moderately Unsatisfactory (MU). 33,195.72 tCO2e as of 2019 vs targets: 177,708/182,000 (update) tCO2eq completed by 18.24%, far below the target.

  • KK: 23,923.37 tCO2e vs target: 100,500 tCO2e; completed by 23.80% 
  • NR: 7,705.47 tCO2e vs target: 10,000 tCO2e; completed by 77.06%
  • CM: 946.47 tCO2e vs target: 70,000 tCO2e; completed by 1.35%
  • SM: 620.41 tCO2e vs target: 1,500 tCO2e; completed by 41.36%

Cumulative direct GHG emission reductions is a good indicator. In practice, (i) some components might not generate cumulative direct GHG emission reductions. (ii) Other components might have already achieved cumulative direct GHG emission reductions without the Project.

177,708 tCO2 eq. was considered as an unrealistic target for goal as the realized cumulative direct GHG emission reductions resulting from the technical assistance and investments up to the end of 2019 and by the end-of-project were significantly lower than this figure.

The actual amount of fuel saving was not available and Annual amount of waste gainfully used was 244,043.36 tonnes as of Dec 2019; completed by 63%.

Management Response: [Added: 2020/05/29]

Partially agree. The assessment of the project achievement at MTR must be done against the mid-term target (24,529 tCO2e of which 33,195 tCO2e has been achieved as of Dec 2019), not the end of project (EOP) target (177,708 tCO2e). As such, it is unreasonable to compare the achievement of Year II against the EOP target. While it is agreed that the EOP target would be challenging to meet given that the anticipated major low carbon projects would need more time to materialize due to technical and political issues (e.g. land expropriation). There are some ideas discussed among the cities to address this shortfall. However, these were included in the report. In addition, a specific recommendation is missing from this report, i.e., how the project should mitigate this potential shortfall.

Key Actions:

Key Action Responsible DueDate Status Comments Documents
Project board meeting to discuss mitigation measures to address the shortfall target
[Added: 2020/05/29]
Project manager / TGO 2020/12 Initiated
2. Recommendation:

A good PLF [Project Log Frame] design always results in a good implementation, which in turn results in good project outcomes. The Project should update the LF by taking into account the chain between activities, outputs and outcome and also the chain between the results, targets and indicators as these two logical chains provide a powerful instrument for managing and monitoring the project implementation. More importantly, any updates on outcomes, outputs and activities should take into account the indicators and targets at the same time, vice versa.

Management Response: [Added: 2020/05/29]

Disagree. It is not clear in the report why the LF needs to be updated. The Result Framework was logical, was developed based on conditions and critical assumptions during the project development (PPG) stage; and was approved by the GEF Council. Any changes to the Results Framework (specifically the goal, objective and outcomes) need re-approval from the GEF Council and the Thai Cabinet, which is a time-consuming process and not recommended at this point of the project timeline, if it does not involve drastic change in the risk analysis. Therefore, changes at the outcome level are not necessary unless it can be proven that there are substantive changes in the magnitude of the barriers and the conditions and assumptions described in the project document.

Key Actions:

3. Recommendation:

The Project should use the consistent methodology to estimate the GHG emission reduction for both target and performance evaluation purposes. The Project should also use the traditional approach based on some assumptions to verify and estimate the actual GHG emission reduction for GEF. In the same time, the Project should also utilize ERM service to re-estimate the goal of GHG emission reduction and distribute total amount of emission reduction among subprojects in four municipalities based on the activities undertaken and the total target.

Management Response: [Added: 2020/05/29]

Agree.  There are two sets of methodologies that were used for GHG emissions reduction calculation. One is the methodology used in the project design, which gives an estimate based on various assumptions. The other methodology is the one developed during project implementation, which is more conservative, based on internationally accepted procedures, and is applicable to the current project environment. These two methodologies provide different results in terms of magnitude and direction. The recommendation to re-examine the methodology and GHG reduction target is valid. The project has decided to use the original methodology that was used during the project design, so that the comparison of the actual GHG ER that will be achieved by the project can be logically compared to the set EOP target value.

Key Actions:

Key Action Responsible DueDate Status Comments Documents
Use the methodology developed during the project design to reporting results. However, for accuracy purpose, continue to use ERM methodology to track progress of pilots and convert the figures to the original methodology used by the Project Document.
[Added: 2020/05/29]
Project manager and TGO 2020/05 Completed
4. Recommendation:

The Project should update the first two indicators and targets of the objective. The Project can delete the existing first two indicators and targets as the two indicators do not provide consistent measurement. Alternatively, the Project can re-design the indicators of the objective by three sectors: waste management, transport and energy. In addition, the targets of objective by sector should be consistent with the target of goal.

Management Response: [Added: 2020/05/29]

Disagree. Cumulative direct fuel savings and annual amount of waste gainfully used are key indicators of the project objective. These indicators are linked to the project goal particularly in terms of the associated reduction of future GHG emissions from cities. The way the project was designed considered the consistency of the link the project objective and the project goal. Achieving the project objective, means the project is contributing to the achievement of the project goal. There is no apparent reason as to why these indicators should be revised. However, it is agreed their targets should be revised as per Recommendation 2.

Key Actions:

5. Recommendation:

The Project should work on the gap after the end of MTR so as to enhance the effectiveness of the Project. In particular, the Project should give high priority to the key outputs that have been delivered but have not yet been implemented in order to enhance the impact of the subprojects, particularly

  • Electronic Common Ticket for all urban transit, Chiang Mai;
  • Real time tracking system via on-line application, Chiang Mai
  • Organic Waste Management Improvement for Samui Organics Recycling Bophut Station, Samui
  • Organic Waste Management Improvement for Baan Ya Suan Pu Station, Samui
  • Energy saving for the household sector, Nakhon Ratchasima
  • Low emissions building for the department stores and malls, Nakhon Ratchasima
  • Damage cost study from traffic congestion in the municipality area, Nakhon Ratchasima

Light Rail Transit (LRT), Khon Kean

Management Response: [Added: 2020/05/29]

Agree. The implementing partner, the cities, and stakeholders must maintain the momentum built in the implementation of low carbon technology demos to ensure that GHG emission reduction will be realized. However, some of the above activities related to the transport sector in Chiang Mai and Khon Kaen are facing great challenges regarding existing laws and regulations, which are beyond the cities’ authority/jurisdiction. However, it does not mean the proposed demonstrations have to be canceled.

The following activities are already completed:

  • Low carbon home handbook as a part of energy saving for the household sector – This is provided to the municipality for distribution.
  • Damage cost study from traffic congestion in the municipality area – This is in Nakhon Ratchasima and has been presented to the Mayor. It is well accepted and will be used as inputs for city planning.

 

The implementation of the rest of the demos are ongoing. Results are expected by the end of the project.

Key Actions:

Key Action Responsible DueDate Status Comments Documents
Continue implementation of the activities. AWP 2020 needs to be monitored.
[Added: 2020/05/29]
Project manager / TGO 2020/12 Initiated
6. Recommendation:

The Project should move the first general indicator and target of Outcome 2.1 to Outcome 1.2 as total amount of new investment leveraged through local plans of participating cities for low carbon projects by EOP is closely associated with the investment in the subprojects in 4 municipalities under Outcome 1.2. Further, the Project should re-design the indicators and target for Output 1.2 2 through Output 1.2.5 (subprojects in 4 municipalities) as the related activities have been updated.

Management Response: [Added: 2020/05/29]

Disagree. There is no need to replace the indicators of Outputs 1.2.2 to 1.2.5 because these were not changed. Even though some of the activities to deliver these outputs were changed these activities are still contributing to the delivery of these outputs. Updated indicators and associated targets are already captured by the MRV system.

Key Actions:

7. Recommendation:

The Project should update output 2.1.1 and associated indicator and target as the activity has been updated.  As the first general indicator and target has been moved to Outcome 1.2, the remaining activities under Outcome 2.1 can be re-grouped into a component called “Capacity Building”. In addition, the Project should update the name Outcome 2.1 and related indicators and target.

Management Response: [Added: 2020/05/29]

Disagree. There is no need to change/update the outputs as recommended. Despite the Thai Voluntary Emissions Reduction Scheme being in place before the project started, there are other analyses of financial incentive schemes that were carried out and the results of these analyses were shared with respective authorities at the four cities. Further, through other UNDP Country Office’s projects, an analysis of public policies to stimulate private investments on renewable energy (i.e., a low carbon solution) was completed in 2017 and a similar analysis for energy efficiency is currently ongoing. These knowledge products can be shared with city leaderships for use in making their decisions to further promote low carbon city activities/investments.

Key Actions:

8. Recommendation:

The implementation of the Project after the MTR should take into account the AWP so as to ensure that the level of resourcing and implementation timeframe are better aligned with the objectives and scope of the Project. Also, the Project should install a Project Management System by incorporating project accounting, procurement, asset management, and grant disbursement. The system should be connected to local municipality and UNDP to avoid delay in reconciliation and approval.

Management Response: [Added: 2020/05/29]

Partially Agree. The project (UNDP/TGO) will closely plan and monitor the remaining activities in alignment with remaining budget and timeframe to ensure successful achievement of the required project goal and objective. The project has already developed the SOP for operations/implementation which is being followed by TGO/UNDP. For project accounting, procurement, asset management, and grant disbursement, there are mechanisms already in place both at UNDP and TGO (e.g. financial audit, procurement plan, asset inventory) to plan and support these tasks. It is not clear what this part of the recommendation specifically addresses. Investing in a new Project Management System at this point of the project is deemed unnecessary and costly.

Key Actions:

Key Action Responsible DueDate Status Comments Documents
Continue closely plan the remaining AWPs and monitor implementation in alignment with the remaining budget and timeframe.
[Added: 2020/05/29]
Project Manager / TGO 2020/12 Initiated
9. Recommendation:

The Project should change the methodology for co-financing from an official co-financing letters provided to PMU to an innovative leveraged investment approach, where the actually realized co-financing contributions against the sizable commitment should be provided before the actual disbursement of GEF funds.

Management Response: [Added: 2020/05/29]

Disagree. While a need to see tangible investments being made prior to actual disbursement of GEF funds is practical, it is difficult to do since some investments were expected to be triggered by successful implementation of the project-supported demos. This will continue to be a challenge. However, during the implementation period, all partners with committed co-financing are required by GEF procedure to provide updates to their actual co-financing amounts. This is considered a means to mitigate a risk of the committed co-financing funds not being fully realized by the end of the project.

Key Actions:

10. Recommendation:

In addition to the target of GHG emission reduction for the Project and the subprojects in 4 municipalities, the Project should design a set of indicator and target for 6 activities in Samui, and 4 activities in Chang Mai, Khon Kaen Nakhon Ratchasima respectively. For 4 activities in Chiang Mai as an example, no. of passengers in the public (integrated) transport system might be a good indicator to avoid small incremental low carbon benefits during the project life. Other indicators can be designed for the Light Rail Transit (LRT) in Khon Kaen to avoid no benefits from emission reduction before the end of the Project.

Management Response: [Added: 2020/05/29]

Partially Agree. While the good intention of this recommendation is understood, highlighting milestones achievements of individual demos/activities would not attribute to the total amount of GHG emissions reduction (tCO2e) as expected at the project goal level. In fact, to mitigate the target shortfall risk for GHG emission reduction amount, the project has started to investigate the impacts of other related activities, such as the ‘Say No to Plastic Bag’ campaign, which is widely implemented in the four cities, as a means to count towards achieving the cumulative GHG reduction target. Instead, achievements from individual low carbon technology application demos should be presented as co-benefits/contributions towards other SDGs.

Key Actions:

Key Action Responsible DueDate Status Comments Documents
Continue to explore low carbon attributions from alternative activities/campaigns in cities
[Added: 2020/05/29]
Project manager / TGO 2021/04 Initiated

Latest Evaluations

Contact us

1 UN Plaza
DC1-20th Floor
New York, NY 10017
Tel. +1 646 781 4200
Fax. +1 646 781 4213
erc.support@undp.org