- Evaluation Plan:
- 2018-2022, Sri Lanka
- Evaluation Type:
- Final Project
- Planned End Date:
- 04/2021
- Completion Date:
- 04/2021
- Status:
- Completed
- Management Response:
- Yes
- Evaluation Budget(US $):
- 20,000
Citra Social Innovation Lab Evaluation Report
| Document | Type | Language | Size | Status | Downloads |
|---|---|---|---|---|---|
 Citra evaluation TOR.pdf |
tor | English | 1599.91 KB | Posted | 787 |
| Final Evaluation Report - Citra Social Innovation Lab Evaluation -.pdf |
report | English | 1503.50 KB | Posted | 838 |
| Title | Citra Social Innovation Lab Evaluation Report | ||||||
|---|---|---|---|---|---|---|---|
| Atlas Project Number: | 00093957 | ||||||
| Evaluation Plan: | 2018-2022, Sri Lanka | ||||||
| Evaluation Type: | Final Project | ||||||
| Status: | Completed | ||||||
| Completion Date: | 04/2021 | ||||||
| Planned End Date: | 04/2021 | ||||||
| Management Response: | Yes | ||||||
| Focus Area: |
|
||||||
| Corporate Outcome and Output (UNDP Strategic Plan 2018-2021) |
|
||||||
| SDG Goal |
|
||||||
| SDG Target |
|
||||||
| Evaluation Budget(US $): | 20,000 | ||||||
| Source of Funding: | UNDP | ||||||
| Evaluation Expenditure(US $): | 20,000 | ||||||
| Joint Programme: | No | ||||||
| Joint Evaluation: | No | ||||||
| Evaluation Team members: |
|
||||||
| GEF Evaluation: | No | ||||||
| Key Stakeholders: | Information and Communication Technology Agency, SLIDA, SLTDA, Department of motor traffic, Sri Lanka Educational Affairs Presidential Task Force, Ministry of Public Services. | ||||||
| Countries: | SRI LANKA | ||||||
| Comments: | The Citra Evaluation was conducted to evaluate the impact and the progress of its first phase and to set improved KPIs for its second phase. |
| Lessons | |
|---|---|
| Findings |
| Recommendations | |
|---|---|
| 1 | Consider developing a formal triage approach for prospective work, with value and impact KPI thresholds to help guide project selection to maximize resource utilization (Citra Process, Innovation Capability & SEET Recommendation) |
| 2 | Consider using the current Citra 2.0 opportunity to design and implement the overall approach to Impact Realization and value KPIs for Citra Lab 2.0. Use the current Department of Motor Transport engagement as an opportunity to support this design, and test against prior projects, such as Parliamentary Visits. (Citra 2.0, Outcomes, Dept Motor Transport, Parliamentary Visits) |
| 3 | Consider defining and agreeing on quantifiable Key Performance Indicators (KPIs) with partners during project identification, to support benchmarking, ongoing quantification of economic and social value during and sustainability post project. (Innovation Capability & Outcomes & SEER Recommendation) |
| 4 | Consider integrating the Handover phase design into the Identification phase of the Citra Process to ensure a clear path to disengagement is defined, preserving value, continuity and sustainability for long-term engagement value. (Citra Process) |
| 5 | Considering consistently benchmark the pre-project state for new projects and embedding measurement into the project process for ongoing validation that projects are attaining the required KPI levels (Citra Process & Innovation Capability & SEER). |
| 6 | Within the Citra Process, consider developing and implementing a formal post-project evaluation framework to validate lasting value and success based on defined KPIs. (Citra Process)
|
| 7 | Consider reviewing the relationship between Board and Citra Lab to ensure the Board is providing strategic steering of Citra 2.0 goals with metrics and framework for goal and value tracking (Governance). |
| 8 | Consider introducing a formal process for Citra Lab to report project economic and social values based on KPIs to the Advisory Board and, in aggregate, publicly. (Citra Process) |
| 9 | Consider keeping HackaDev closely aligned and partially integrated into Citra Lab, leveraging the alumni and ambassador network to add value on projects. (HackaDev & Relationships) |
| 10 | Consider developing a fellowship program to bring outstanding young innovators into Citra Lab to work on government projects, funded by project partners. Reference 'Code for movement’ as an example (HackaDev & Relationships). |
| 11 | Consider establishing a permanent challenge infrastructure for Sri Lanka similar to the US’s Challenge.gov. Source initial funding from organisations supporting citizen-centric platforms and charge cost recovery fees to agencies using the platform to defray costs |
| 12 | Consider implementing measurable and reportable KPIs for all NextGenGov participant projects to support Citra Lab to evidence value delivery and programme investment. |
| 13 | Consider providing structured participant support during project implementation and provide additional participant certification on project completion. |
| 14 | Consider extending the program with short and cheap innovation courses (online and face-to-face) alongside the premium 5-day residential program to scale impact to more public servants. While preferable to collaborate on design with SLIDA or other government bodies, Citra Lab has the credentials to start quickly now and collaborate later |
| 15 | Consider co-founding an annual Innovation Award for the Sri Lankan public sector with an appropriate government entity. |
| 16 | Develop current ‘How To’ guide into more comprehensive patterns and a playbook for innovation within a Sri Lankan public sector context, using Citra Lab project expertise as examples. Consider selling/distributing globally as a support tool for other nations. |
| 17 | Develop and maintain a central power map of Citra Lab connections, networks and relationships to support ongoing management of relations in the advent of staff turnover and accelerate identification of partner opportunities. (Relationships) |
1. Recommendation:
Consider developing a formal triage approach for prospective work, with value and impact KPI thresholds to help guide project selection to maximize resource utilization (Citra Process, Innovation Capability & SEET Recommendation)
Management Response: [Added: 2021/11/23]
Fully Agree - Develop and incorporate a formal triage approach to be included in the proposed formal post-project evaluation framework,
Key Actions:
| Key Action | Responsible | DueDate | Status | Comments | Documents |
|---|---|---|---|---|---|
| 1. Develop and implement a formal triage framework drawing from previous experiences and lessons learned in the first project cycle
[Added: 2021/11/23] [Last Updated: 2022/03/31] |
Lab Team | 2022/10 | Initiated | Initial steps has been taken by Citra by conducting workshops and training to develop a formal triage framework. more lessons to be considered before drawing the framework as of the quick changes in the current context. History |
2. Recommendation:
Consider using the current Citra 2.0 opportunity to design and implement the overall approach to Impact Realization and value KPIs for Citra Lab 2.0. Use the current Department of Motor Transport engagement as an opportunity to support this design, and test against prior projects, such as Parliamentary Visits. (Citra 2.0, Outcomes, Dept Motor Transport, Parliamentary Visits)
Management Response: [Added: 2021/11/23]
Fully Agree - Develop and incorporate quantifiable KPIs relating to individual interventions. This can be done in consultation with partners and UNDP M&E focal points.
Key Actions:
| Key Action | Responsible | DueDate | Status | Comments | Documents |
|---|---|---|---|---|---|
| 1, Building on ongoing discussions with RRMT and other project partners, define and agree on quantifiable KPI’s at the design stage of each intervention.
[Added: 2021/11/23] [Last Updated: 2022/06/28] |
Lab Team | 2022/11 | Initiated | Currently Citra has held discussions with RRMT on target setting, further project-level target setting for the new interventions is still underway. History | |
| 2. For existing interventions, develop and agree on quantifiable KPI’s with relevant project partners, RRMT and any other relevant focal points.
[Added: 2021/11/23] [Last Updated: 2022/06/28] |
Lab Team | 2022/11 | Initiated | Currently, Citra has held discussions with existing partners through LPAC for target setting, and further project-level discussions with partners are underway. History |
3. Recommendation:
Consider defining and agreeing on quantifiable Key Performance Indicators (KPIs) with partners during project identification, to support benchmarking, ongoing quantification of economic and social value during and sustainability post project. (Innovation Capability & Outcomes & SEER Recommendation)
Management Response: [Added: 2021/11/23]
Fully agree - Quantifiable KPIs to be designed at the project identifications stage, this can be done in consultation with partners and UNDP M&E focal points.
Key Actions:
| Key Action | Responsible | DueDate | Status | Comments | Documents |
|---|---|---|---|---|---|
| 1. Building on initial discussions with RRMT to develop appropriate KPIs, discussions will also be held with project partners at the design phase of each intervention to define and agree on quantifiable KPIs.
[Added: 2021/11/23] [Last Updated: 2022/03/29] |
Lab Team | 2021/11 | Completed | Citra prodoc. RRF was reviewed with RRMT, and appropriate KPIs were finalized, including concluded several project targets at LPAC History | |
| 2. Develop and incorporate necessary tools for data collection tools to support monitoring and evaluation processes (this could include surveys and feedback forms).
[Added: 2021/11/23] [Last Updated: 2022/06/28] |
Lab Team | 2022/05 | Completed | Perception survey form is already developed by Citra, the master form is contextualized for the project level while conducting perception. History |
4. Recommendation:
Consider integrating the Handover phase design into the Identification phase of the Citra Process to ensure a clear path to disengagement is defined, preserving value, continuity and sustainability for long-term engagement value. (Citra Process)
Management Response: [Added: 2021/11/23]
Fully Agree -
The current Lab Process will be re-designed to reflect the integration of the handover phase at the inception or identification stage of the process
Key Actions:
| Key Action | Responsible | DueDate | Status | Comments | Documents |
|---|---|---|---|---|---|
| 1. Evaluate and revise the current Lab process.
[Added: 2021/11/23] [Last Updated: 2022/05/03] |
Lab Team | 2022/03 | Completed | Citra's lab process has been revised based on evaluated recommendations, several areas has been incorporated along with the newly developed Project design and targets, the actions of Citra has been planned under the three categories: 1. At the issue identification phase we work to also identify possible partners who we could handover the project to 2. At the engagement phase we reach out to them, establish that the project will be handed over to them, and thereafter engage them in all co-design processes 3. Handover phase where the project is finally given over to the pre-identified partner as agreed History |
5. Recommendation:
Considering consistently benchmark the pre-project state for new projects and embedding measurement into the project process for ongoing validation that projects are attaining the required KPI levels (Citra Process & Innovation Capability & SEER).
Management Response: [Added: 2021/11/23]
Fully Agree - Incorporate baselines and establish these where necessary in order to fully evaluate progress on new initiatives against established KPIs
Key Actions:
| Key Action | Responsible | DueDate | Status | Comments | Documents |
|---|---|---|---|---|---|
| 1. Incorporate the inclusion of baselines and/or the establishment of a baseline as required in developing the existing Lab Process, particularly with reference to the development of a formal evaluation methodology.
[Added: 2021/11/23] [Last Updated: 2022/06/09] |
Lab Team | 2022/07 | Initiated | pre-discussion in progress within team History | |
| Incorporate a regular review mechanism to evaluate progress and to support data collection and regular reporting to support the validation process (this could include surveys, feedback forms etc)
[Added: 2021/11/23] [Last Updated: 2022/06/09] |
Lab Team | 2022/12 | Initiated | History |
6. Recommendation:
Within the Citra Process, consider developing and implementing a formal post-project evaluation framework to validate lasting value and success based on defined KPIs. (Citra Process)
Management Response: [Added: 2021/11/23]
Fully Agree - The Lab team will work with relevant M&E focal points to develop and implement a project evaluation framework.
Key Actions:
| Key Action | Responsible | DueDate | Status | Comments | Documents |
|---|---|---|---|---|---|
| 1. Design and Implement a formal evaluation methodology for all interventions undertaken by the Lab. This could include the development of an evaluation form to be shared with partners for feedback soon after completion/handover.
[Added: 2021/11/23] |
Lab Team | 2022/08 | Not Initiated |
7. Recommendation:
Consider reviewing the relationship between Board and Citra Lab to ensure the Board is providing strategic steering of Citra 2.0 goals with metrics and framework for goal and value tracking (Governance).
Management Response: [Added: 2021/11/23]
Fully agree - The role of the advisory board for Citra 2.0 was clearly articulated in the project document.
Key Actions:
| Key Action | Responsible | DueDate | Status | Comments | Documents |
|---|---|---|---|---|---|
| 1, Communicate the roles and responsibilty to the advisory board through LPAC and agree.
[Added: 2021/11/23] |
Lab Team | 2021/07 | Completed |
8. Recommendation:
Consider introducing a formal process for Citra Lab to report project economic and social values based on KPIs to the Advisory Board and, in aggregate, publicly. (Citra Process)
Management Response: [Added: 2021/11/23]
Fully agree - This will clearly be stipulated as part of the agenda for the annual advisory board meeting.
Key Actions:
| Key Action | Responsible | DueDate | Status | Comments | Documents |
|---|---|---|---|---|---|
| 1. Ensure the inclusion of a progress review to be included as a key component of the annual meeting of the Advisory Board
[Added: 2021/11/23] [Last Updated: 2022/06/28] |
Citra Team | 2022/10 | Not Initiated | The Advisory board meeting is yet to be held. History |
9. Recommendation:
Consider keeping HackaDev closely aligned and partially integrated into Citra Lab, leveraging the alumni and ambassador network to add value on projects. (HackaDev & Relationships)
Management Response: [Added: 2021/11/23]
Fully Agree - Hackadev and all related components was incorporated into Citra 2.0 project document.
Key Actions:
| Key Action | Responsible | DueDate | Status | Comments | Documents |
|---|---|---|---|---|---|
| Include components supporting Hackadev program into the Citra prodoc RRF
[Added: 2021/11/23] |
Citra Lab | 2021/07 | Completed | Hackadev component included as youth and entrepreneur skill development |
10. Recommendation:
Consider developing a fellowship program to bring outstanding young innovators into Citra Lab to work on government projects, funded by project partners. Reference 'Code for movement’ as an example (HackaDev & Relationships).
Management Response: [Added: 2021/11/23]
Fully Agree, this can be explored through the already established HackaDev Alumni Network and HackaDev Ambassador Programme which has already identified outstanding young innovators
Key Actions:
| Key Action | Responsible | DueDate | Status | Comments | Documents |
|---|---|---|---|---|---|
| 1. When lab cycles are run, opportunities could be explored more explicitly on how these young innovators can play a direct role within, wherever applicable.
[Added: 2021/11/23] |
Citra Lab | 2023/01 | Not Initiated |
11. Recommendation:
Consider establishing a permanent challenge infrastructure for Sri Lanka similar to the US’s Challenge.gov. Source initial funding from organisations supporting citizen-centric platforms and charge cost recovery fees to agencies using the platform to defray costs
Management Response: [Added: 2021/11/23]
Fully Agree - The Lab Team will engage with key partners across various sectors including the government and private sector to establish a permanent challenge infrastructure for Sri Lanka. Discussions will build on and leverage existing partnerships for the current Social Innovation Challenge.
It should be noted that the establishment and launch of this challenge mechanism is envisioned to take place towards the middle/end of this project cycle and so is not reflected in the key actions which instead only reference the process of establishment.
Key Actions:
| Key Action | Responsible | DueDate | Status | Comments | Documents |
|---|---|---|---|---|---|
| 1. Stakeholder consultations for the development of a permanent challenge infrastructure
[Added: 2021/11/23] |
Lab Team | 2022/08 | Not Initiated | ||
| 2. The Hackadev platform to be expanded and established as a long term innovation challenge platform of UNDP.
[Added: 2021/11/23] |
Lab Team | 2023/01 | Not Initiated |
12. Recommendation:
Consider implementing measurable and reportable KPIs for all NextGenGov participant projects to support Citra Lab to evidence value delivery and programme investment.
Management Response: [Added: 2021/11/23]
Fully Agree - Measurable KPI’s to be developed in consultation with RRMT and any other key partners and incorporated.
Key Actions:
| Key Action | Responsible | DueDate | Status | Comments | Documents |
|---|---|---|---|---|---|
| 1. Develop measurable and reportable KPIs for NextGenGov participant projects
[Added: 2021/11/23] |
Citra Lab | 2022/08 | Not Initiated |
13. Recommendation:
Consider providing structured participant support during project implementation and provide additional participant certification on project completion.
Management Response: [Added: 2021/11/23]
Fully Agree – The Lab Team will review and revise current processes and mechanisms of supporting and recognising NGG fellows.
Key Actions:
| Key Action | Responsible | DueDate | Status | Comments | Documents |
|---|---|---|---|---|---|
| 1. Review and revise the mentorship component of the NGG fellowship programme
[Added: 2021/11/23] [Last Updated: 2022/06/09] |
Citra Lab | 2022/12 | Initiated | Work is in progress for next year plan on Next Gen plan on mentorship History | |
| 2, Review and revise current NGG Fellow recognition practices
[Added: 2021/11/23] [Last Updated: 2022/06/09] |
Citra Lab | 2022/12 | Initiated | Current recognition practices are being re-evaluated in accordance with a refreshed session plan and programme content History |
14. Recommendation:
Consider extending the program with short and cheap innovation courses (online and face-to-face) alongside the premium 5-day residential program to scale impact to more public servants. While preferable to collaborate on design with SLIDA or other government bodies, Citra Lab has the credentials to start quickly now and collaborate later
Management Response: [Added: 2021/11/23]
Partially Agree - The Lab team will engage with key partners including SLIDA to develop a series of shorter innovation courses. This would build on the workshops conducted thus far, as well as the 4 day trainings conducted for Class 3 officials at SLIDA. It should be noted that given ongoing Covid 19 related restrictions as well as this recommendation being a long term goal/intervention full completion may not be possible within the stipulated 18 months.
Key Actions:
| Key Action | Responsible | DueDate | Status | Comments | Documents |
|---|---|---|---|---|---|
| 1. Develop relevant curricula for shorter training courses for inductees and other officials in their training schedule.
[Added: 2021/11/23] |
Citra Lab | 2022/07 | Not Initiated | ||
| 2, Short training courses on innovation (virtual or on-site) to be introduced to SLIDA.
[Added: 2021/11/23] |
Citra Lab | 2023/01 | Not Initiated |
15. Recommendation:
Consider co-founding an annual Innovation Award for the Sri Lankan public sector with an appropriate government entity.
Management Response: [Added: 2021/11/23]
Fully Agree – Explore the possibility of the establishment of such an annual Innovation Award for the Sri Lankan public sector with relevant government partners
Key Actions:
| Key Action | Responsible | DueDate | Status | Comments | Documents |
|---|---|---|---|---|---|
| 1. Hold consultations with relevant government partners such as SLIDA and/or the Ministry of Public Administration regarding the establishment of an annual Innovation Award for the Sri Lankan public sector
[Added: 2021/11/23] |
Citra Lab | 2023/01 | Not Initiated |
16. Recommendation:
Develop current ‘How To’ guide into more comprehensive patterns and a playbook for innovation within a Sri Lankan public sector context, using Citra Lab project expertise as examples. Consider selling/distributing globally as a support tool for other nations.
Management Response: [Added: 2021/11/23]
Partially Agree – Design and develop the existing ‘How To Guide’ or develop similar innovation playbooks that address the Sri Lankan context. These products will be freely available to innovation practitioners across the world.
Key Actions:
| Key Action | Responsible | DueDate | Status | Comments | Documents |
|---|---|---|---|---|---|
| 1. Revise and/or update the existing How to Guide
[Added: 2021/11/23] |
Citra Lab | 2022/12 | Not Initiated | ||
| 2. Design, develop and expand on the existing guide to create a localized innovation playbook
[Added: 2021/11/23] |
Citra Lab | 2023/01 | Not Initiated |
17. Recommendation:
Develop and maintain a central power map of Citra Lab connections, networks and relationships to support ongoing management of relations in the advent of staff turnover and accelerate identification of partner opportunities. (Relationships)
Management Response: [Added: 2021/11/23]
Fully Agree – Maintain and develop a comprehensive partner engagement list
Key Actions:
| Key Action | Responsible | DueDate | Status | Comments | Documents |
|---|---|---|---|---|---|
| 1. Design and implement a monitoring process/template to capture existing partnerships
[Added: 2021/11/23] |
Citra Lab | 2022/12 | Not Initiated |