- Evaluation Plan:
- 2016-2020, Turkey
- Evaluation Type:
- Outcome
- Planned End Date:
- 01/2019
- Completion Date:
- 02/2019
- Status:
- Completed
- Management Response:
- Yes
- Evaluation Budget(US $):
- 25,000
Outcome Evaluation of Climate Change and Environment (CCE) Portfolio
| Document | Type | Language | Size | Status | Downloads |
|---|---|---|---|---|---|
 TOR for OutcomeEvalution of CCE Portfolio.pdf |
tor | English | 919.32 KB | Posted | 44 |
| CCE Portfolio Outcome Evaluation 2016-2020 - Final.pdf |
report | English | 1989.99 KB | Posted | 66 |
| Title | Outcome Evaluation of Climate Change and Environment (CCE) Portfolio | ||||||||
|---|---|---|---|---|---|---|---|---|---|
| Atlas Project Number: | 00061845,00056563,00082077,00081796,00096470 | ||||||||
| Evaluation Plan: | 2016-2020, Turkey | ||||||||
| Evaluation Type: | Outcome | ||||||||
| Status: | Completed | ||||||||
| Completion Date: | 02/2019 | ||||||||
| Planned End Date: | 01/2019 | ||||||||
| Management Response: | Yes | ||||||||
| Corporate Outcome and Output (UNDP Strategic Plan 2018-2021) |
|
||||||||
| Evaluation Budget(US $): | 25,000 | ||||||||
| Source of Funding: | Budgets of projects under the CCE Portfolio | ||||||||
| Evaluation Expenditure(US $): | 20,000 | ||||||||
| Joint Programme: | No | ||||||||
| Joint Evaluation: | No | ||||||||
| Evaluation Team members: |
|
||||||||
| GEF Evaluation: | No | ||||||||
| Key Stakeholders: | |||||||||
| Countries: | TURKEY |
| Lessons | |
|---|---|
| Findings |
| Recommendations | |
|---|---|
| 1 | When technically possible, update the design and RF if more than two years have elapsed after preparation before the project becomes effective |
| 2 | Introduce detailed costings and prepare joint budgets to be implemented as parallel co financing.
|
| 3 | Explore ways and means of doing joint co-financing as opposed to parallel co-financing.
|
| 4 | Introduce some economic feasibility analysis in the designs, and link it with the Results Framework. |
| 5 | Consider making the main report part of the Project Documents smaller in size, while expanding their annexes. |
| 6 | Try to stay away from quantitative targets unless well justified and warranted. |
| 7 | Always bear in mind that UNDP’s key role is to trigger change. |
| 8 | Utilize the Theory of Change intensively and do not take chances on the Result Framework Preparation |
| 9 | Continue with the efforts to mainstream gender equality across the entire portfolio, and start firmly with the designs. |
| 10 | Incorporate the concept of resilience in the CCE strategy and action by ensuring that relevant project designs include multi-hazard risk assessments, taking into account seismic and climatic risks and others if deemed necessary and providing estimates of the potential impact on project staff, partners, beneficiaries and project activities |
| 11 | Embedded project staff at implementing partners is key to sustained partner cooperation. Try to spread this practice with all FSPs. |
| 12 | Detailed multi-year joint (integrated) budgets can enhance ownership and increase accountability with implementing partners. Give it a try despite difficulties |
| 13 | Try to closely monitor co-financing to the extent politically and technically feasible. |
| 14 | Remain alert and evaluation-ready at all times. Do not wait for 5 years till the next OE |
| 15 | Establish a central system (or re-model the existing system) with satellite sub-systems based at large projects with one PMU per cluster (or focus area) if and when possible.
|
| 16 | Do system automation, and produce reports on demand by database queries. |
| 17 | Prepare semi-annual summaries for the portfolio, and annual impact reports using the incoming reports to the system, mainly the PIRs. |
| 18 | Disclose M&E Reports on the WEB |
| 19 | Adopt co-financing ratio as a distinct performance indicator and monitor it closely. |
| 20 | Use co-financing amounts and ratios as indicator for efficiency and sustainability. |
| 21 | Help establish a program/portfolio M&E based on a re-configured Portfolio Results Framework and GEF principles using external consultants |
| 22 | Bring gender within the purview of M&E and insist on disaggregated data where disaggregated data makes sense. |
1. Recommendation:
When technically possible, update the design and RF if more than two years have elapsed after preparation before the project becomes effective
Management Response: [Added: 2019/02/18]
The design of the projects are revised and updated during the PPG phases of the projects. If the donor is different than GEF project inception phases are going to use for the necessary revisions and updates.
Key Actions:
| Key Action | Responsible | DueDate | Status | Comments | Documents |
|---|---|---|---|---|---|
| The design and the RF will be updated if more than two years have elapsed before the project officially starts.
[Added: 2019/04/02] |
Portfolio Manager and related Cluster Leads | 2020/12 | Initiated |
2. Recommendation:
Introduce detailed costings and prepare joint budgets to be implemented as parallel co financing.
Management Response: [Added: 2019/02/18]
Key Actions:
| Key Action | Responsible | DueDate | Status | Comments | Documents |
|---|---|---|---|---|---|
| Preparing joint budgets requires corporate decision. If any new rules and regulations will be introduced at the POPP regarding this matter it will be adopted directly to the project management.
[Added: 2019/04/02] |
Portfolio Manager and related Cluster Leads. | 2020/12 | Initiated |
3. Recommendation:
Explore ways and means of doing joint co-financing as opposed to parallel co-financing.
Management Response: [Added: 2019/02/18]
Key Actions:
| Key Action | Responsible | DueDate | Status | Comments | Documents |
|---|---|---|---|---|---|
| Preparing joint budgets requires corporate decision. If any new rules and regulations will be introduced at the POPP regarding this matter it will be adopted directly to the project management.
[Added: 2019/04/02] |
Portfolio Manager and related Cluster Leads. | 2021/12 | Initiated |
4. Recommendation:
Introduce some economic feasibility analysis in the designs, and link it with the Results Framework.
Management Response: [Added: 2019/02/18]
Key Actions:
| Key Action | Responsible | DueDate | Status | Comments | Documents |
|---|---|---|---|---|---|
| Economic feasibility analysis to be introduced in the designs of the projects. Also, the analysis will be linked to the Results Framework.
[Added: 2019/04/02] |
Portfolio Manager and related Cluster Leads. | 2020/12 | Initiated |
5. Recommendation:
Consider making the main report part of the Project Documents smaller in size, while expanding their annexes.
Management Response: [Added: 2019/02/18]
Key Actions:
| Key Action | Responsible | DueDate | Status | Comments | Documents |
|---|---|---|---|---|---|
| Further clarifications and new format for Project Documents will be explored.
[Added: 2019/04/02] |
Portfolio Manager and related UNDP staff. | 2020/12 | Initiated |
6. Recommendation:
Try to stay away from quantitative targets unless well justified and warranted.
Management Response: [Added: 2019/02/18]
Key Actions:
| Key Action | Responsible | DueDate | Status | Comments | Documents |
|---|---|---|---|---|---|
| Both well justified quantitative and qualitative targets will be justified and warranted through a set of SMART indicators during the design of the projects in line with the M&E system of the portfolio.
[Added: 2019/04/02] |
Portfolio Manager, related Cluster Leads and M&E Advisor. | 2020/12 | Initiated |
7. Recommendation:
Always bear in mind that UNDP’s key role is to trigger change.
Management Response: [Added: 2019/02/18]
Key Actions:
| Key Action | Responsible | DueDate | Status | Comments | Documents |
|---|---|---|---|---|---|
| Following the building and adoption of a CCE Portfolio Level Results Framework, relevant indicators to track these changes would be developed and existing ones sharpened.
[Added: 2019/04/02] |
Portfolio Manager, related Cluster Leads and M&E Advisor. | 2020/12 | Initiated |
8. Recommendation:
Utilize the Theory of Change intensively and do not take chances on the Result Framework Preparation
Management Response: [Added: 2019/02/18]
Key Actions:
| Key Action | Responsible | DueDate | Status | Comments | Documents |
|---|---|---|---|---|---|
| Results Framework specific for the CCE Portfolio will be prepared collaboratively in line with the CPD and SDG targets of the portfolio considering to utilize the theory of change.
[Added: 2019/04/02] |
Portfolio Manager, related Cluster Leads and M&E Advisor. | 2020/12 | Initiated |
9. Recommendation:
Continue with the efforts to mainstream gender equality across the entire portfolio, and start firmly with the designs.
Management Response: [Added: 2019/02/18]
Key Actions:
| Key Action | Responsible | DueDate | Status | Comments | Documents |
|---|---|---|---|---|---|
| Human capacity of projects and portfolio for gender mainstreaming will be enriched via hiring experts for relevant projects. In 2018, a need analysis finalized and hiring relevant experts will be in place in 2019.
[Added: 2019/04/02] |
Portfolio Manager | 2019/12 | Initiated |
10. Recommendation:
Incorporate the concept of resilience in the CCE strategy and action by ensuring that relevant project designs include multi-hazard risk assessments, taking into account seismic and climatic risks and others if deemed necessary and providing estimates of the potential impact on project staff, partners, beneficiaries and project activities
Management Response: [Added: 2019/02/18]
Key Actions:
| Key Action | Responsible | DueDate | Status | Comments | Documents |
|---|---|---|---|---|---|
| Once the strategy is decided on how to incorporate the concept of resilience in the CCE portfolio, multi-hazard risk assessments, taking into account seismic and climatic risks and others will be included to the relevant project designs.
[Added: 2019/04/02] |
Portfolio Manager and related Cluster Leads. | 2020/12 | Initiated |
11. Recommendation:
Embedded project staff at implementing partners is key to sustained partner cooperation. Try to spread this practice with all FSPs.
Management Response: [Added: 2019/02/18]
Key Actions:
| Key Action | Responsible | DueDate | Status | Comments | Documents |
|---|---|---|---|---|---|
| Embedding dedicated project staff at implementing partners will tried to be spread as a common practice for the projects under the portfolio.
[Added: 2019/04/02] |
Portfolio Manager | 2020/12 | Initiated |
12. Recommendation:
Detailed multi-year joint (integrated) budgets can enhance ownership and increase accountability with implementing partners. Give it a try despite difficulties
Management Response: [Added: 2019/02/18]
Key Actions:
| Key Action | Responsible | DueDate | Status | Comments | Documents |
|---|---|---|---|---|---|
| Preparing joint budgets requires corporate decision. If any new rules and regulations will be introduced at the POPP regarding this matter it will be adopted directly to the project management.
[Added: 2019/04/02] |
Portfolio Manager | 2020/12 | Initiated |
13. Recommendation:
Try to closely monitor co-financing to the extent politically and technically feasible.
Management Response: [Added: 2019/02/18]
Key Actions:
| Key Action | Responsible | DueDate | Status | Comments | Documents |
|---|---|---|---|---|---|
| During the preparation of project progress reports, mid-term and terminal evaluation reports a co-finance table will be prepared within consultation to the partners of the project as a monitoring activity
[Added: 2019/04/02] |
Portfolio Manager, Cluster Leads and M&E Advisor | 2020/12 | Initiated |
14. Recommendation:
Remain alert and evaluation-ready at all times. Do not wait for 5 years till the next OE
Management Response: [Added: 2019/02/18]
Key Actions:
| Key Action | Responsible | DueDate | Status | Comments | Documents |
|---|---|---|---|---|---|
| Within the scope of the M&E strategy of the portfolio it is already decided to conduct the outcome evaluations every 3 years.
[Added: 2019/04/02] |
Portfolio Manager and M&E Advisor | 2020/12 | Initiated |
15. Recommendation:
Establish a central system (or re-model the existing system) with satellite sub-systems based at large projects with one PMU per cluster (or focus area) if and when possible.
Management Response: [Added: 2019/02/18]
Key Actions:
| Key Action | Responsible | DueDate | Status | Comments | Documents |
|---|---|---|---|---|---|
| For every cluster and the project under the portfolio the responsibility unit/staff for the M&E activities were introduced in line with the M&E strategy of the portfolio to increase the efficiency of the M&E activities.
[Added: 2019/04/02] |
M&E Advisor | 2020/12 | Initiated |
16. Recommendation:
Do system automation, and produce reports on demand by database queries.
Management Response: [Added: 2019/02/18]
Key Actions:
| Key Action | Responsible | DueDate | Status | Comments | Documents |
|---|---|---|---|---|---|
| More efficient and automated report formats will be explored and new methodologies will be introduced in order to increase effectiveness and efficiency of the portfolio.
[Added: 2019/04/02] |
M&E Advisor | 2020/12 | Initiated |
17. Recommendation:
Prepare semi-annual summaries for the portfolio, and annual impact reports using the incoming reports to the system, mainly the PIRs.
Management Response: [Added: 2019/02/18]
Key Actions:
| Key Action | Responsible | DueDate | Status | Comments | Documents |
|---|---|---|---|---|---|
| All these reporting tools will be used to prepare annual summary for the portfolio and also impact assessment will be conducted to the portfolio.
[Added: 2019/04/02] |
M&E Advisor | 2020/12 | Initiated |
18. Recommendation:
Disclose M&E Reports on the WEB
Management Response: [Added: 2019/02/18]
Key Actions:
| Key Action | Responsible | DueDate | Status | Comments | Documents |
|---|---|---|---|---|---|
| Further ways will be explored to disclose the M&E reports on the website of UNDP. In addition, a process for dissemination of M&E reports and gathering suggestions from key stakeholders will be explored.
[Added: 2019/04/02] |
Portfolio Manager and M&E Advisor | 2020/12 | Initiated |
19. Recommendation:
Adopt co-financing ratio as a distinct performance indicator and monitor it closely.
Management Response: [Added: 2019/02/18]
Key Actions:
| Key Action | Responsible | DueDate | Status | Comments | Documents |
|---|---|---|---|---|---|
| Co-finance tables will be prepared within consultation to the partners of the projects as a monitoring activity and tools will be developed to monitor co-financing.
[Added: 2019/04/02] |
Portfolio Manager and M&E Advisor | 2020/12 | Initiated |
20. Recommendation:
Use co-financing amounts and ratios as indicator for efficiency and sustainability.
Management Response: [Added: 2019/02/18]
Key Actions:
| Key Action | Responsible | DueDate | Status | Comments | Documents |
|---|---|---|---|---|---|
| Co-finance tables will be prepared within consultation to the partners of the projects as a monitoring activity and tools will be developed to monitor co-financing.
[Added: 2019/04/02] |
Portfolio Manager and M&E Advisor | 2020/12 | Initiated |
21. Recommendation:
Help establish a program/portfolio M&E based on a re-configured Portfolio Results Framework and GEF principles using external consultants
Management Response: [Added: 2019/02/18]
Key Actions:
| Key Action | Responsible | DueDate | Status | Comments | Documents |
|---|---|---|---|---|---|
| Results Framework specific for the CCE Portfolio will be prepared collaboratively in line with the CPD and SDG targets of the portfolio.
[Added: 2019/04/02] |
Portfolio Manager | 2020/12 | Initiated |
22. Recommendation:
Bring gender within the purview of M&E and insist on disaggregated data where disaggregated data makes sense.
Management Response: [Added: 2019/02/18]
Key Actions:
| Key Action | Responsible | DueDate | Status | Comments | Documents |
|---|---|---|---|---|---|
| Sex-disaggregated data to be collected and analysis of male/female (M/F) differences to be included in all reports and assessments to clarify differences between M/F vulnerability will be done. In addition, in availability of sex- disaggregated data will be identified.
[Added: 2019/04/02] |
Portfolio Manager, Gender Advisor and M&E Advisor | 2020/12 | Initiated |