- Evaluation Plan:
- 2017-2021, Bangladesh
- Evaluation Type:
- Final Project
- Planned End Date:
- 10/2020
- Completion Date:
- 10/2020
- Status:
- Completed
- Management Response:
- Yes
- Evaluation Budget(US $):
- 35,000
Final Evaluation of Partnership for a Tolerant, Inclusive Bangladesh (PTIB) project
| Document | Type | Language | Size | Status | Downloads |
|---|---|---|---|---|---|
 TOR_PTIB_Final Evaluation_International Consultant.pdf |
tor | English | 453.25 KB | Posted | 320 |
| TOR_PTIB_Final Evaluation_National Consultant.pdf |
tor | English | 454.82 KB | Posted | 317 |
| Evaluation Report_PTIB_Final Evaluation.pdf |
report | English | 4476.65 KB | Posted | 620 |
| Title | Final Evaluation of Partnership for a Tolerant, Inclusive Bangladesh (PTIB) project | ||||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| Atlas Project Number: | 86326 | ||||||||||||
| Evaluation Plan: | 2017-2021, Bangladesh | ||||||||||||
| Evaluation Type: | Final Project | ||||||||||||
| Status: | Completed | ||||||||||||
| Completion Date: | 10/2020 | ||||||||||||
| Planned End Date: | 10/2020 | ||||||||||||
| Management Response: | Yes | ||||||||||||
| UNDP Signature Solution: |
|
||||||||||||
| Corporate Outcome and Output (UNDP Strategic Plan 2018-2021) |
|
||||||||||||
| SDG Target |
|
||||||||||||
| Evaluation Budget(US $): | 35,000 | ||||||||||||
| Source of Funding: | Project budget | ||||||||||||
| Evaluation Expenditure(US $): | 21,008 | ||||||||||||
| Joint Programme: | No | ||||||||||||
| Joint Evaluation: | No | ||||||||||||
| Evaluation Team members: |
|
||||||||||||
| GEF Evaluation: | No | ||||||||||||
| Key Stakeholders: | |||||||||||||
| Countries: | BANGLADESH |
| Lessons | |
|---|---|
| 1. | Keeping messaging positive: When addressing divisive rhetoric, the project made a deliberate decision to keep UNDP messaging positive – to focus not on “countering” extremist or hateful narratives directly, but to promote alternative peaceful, positive narratives. Throughout the cycle, the project recognized that while deliberate incitement and hate speech did exist in Bangladesh, the best counter was not censorship but better arguments and better evidence. |
| 2. | Understanding PVE and prevention of hate speech as peacebuilding efforts: While UNDP internally applies the Prevention of Violent Extremism label to this work, in 2019 the project realized how adaptable this model is to address issues of hate speech being raised by the UN Secretary General. Though the means and manner of hate speech may be different, the dynamics and the implications are similar to extremist rhetoric. |
| 3. | Welcoming supportive government partners: The project has identified government counterparts who care about promoting tolerance and inclusivity and want to get it right. Building a collaborative and positive relationship with government partners has supported mutual trust to grow and influence positive changes. |
| 4. | Recognizing that long-term change requires mentorship: Throughout 2019, the project internalized its realization from the previous years that one-off events do not work well. Extended follow-up and technical support are needed to promote lasting change. This was particularly evident to the PTIB project when following-up with winners of the Digital Khichuri Challenges. The winners' selection was just the beginning of a journey, not the end – the real end was when the team had successfully launched their new platform. By expanding and institutionalizing the mentorship for DKC winners, the project maximized their success chances in 2019. This same insight also led the project to train dozens of organizations before submitting their final proposals for UNDP’s Diversity 4 Peace grants programme. The end proposals' quality and actionability were significantly improved by training interested applicants beforehand, even before the activities started and the funding was disbursed. |
| Findings | |
| 1. | PTIB was conceived in a fluid and uncertain country context. It was given the green light after a major terrorist event when much was unknown about the nature of violence and extremism in Bangladesh. For this reason, the project was conceived as a learning project with an emphasis on research as a first step better to understand the problem of violent extremism in Bangladesh. It was always expected to evolve with the growing knowledge base, which helps create and remain relevant. PTIB worked well because of the adaptative management style that learned from its successes and failures and was guided by its action-orientated research. As more was understood, PTIB’s activities evolved with a greater emphasis on supporting inclusion and tolerance and countering hate speech and disinformation. As it prepares to enter the second phase, this body of experience and knowledge can provide a strong foundation for future programming. |
| 2. | This evaluation judged PTIB to be a relevant project as its objectives, purpose, and outcomes were consistent with Bangladesh's needs and interests identified by the host government. They were also aligned with global strategies outlined by the UN Secretary General’s Plan of Action to Prevent Violent Extremism. As an adaptive project, PTIB remained relevant when significant and unexpected challenges faced by Bangladesh. After the Rohingya influx of 2017, PTIB created a new specialized research unit in Cox’s Bazar to improve the humanitarian response's ability to understand the nature of violence around the refugee camps. When the UN Secretary General announced a global plan of action on hate speech in 2019, PTIB had already identified this as a challenge for Bangladesh through its research and had programming activities compatible with the global agenda underway. In 2020, albeit the outbreak of the COVID-19 pandemic, PTIB was already active with online activities and social media monitoring. The project not only kept the momentum high by switching offline activities to online ones, but its analytical teams were well placed to provide unique reporting on the nature of disinformation. These programming pivots and its adaptability showed a key PTIB strength. Needless to say that PTIB started as a relevant project, but the project remains to be pertinent in a highly unpredictable context. |
| 3. | PTIB was adaptable, and understanding its experimental nature is key to measure its effectiveness. Each PTIB component, in its way, broke new ground. In the research field, BPO has created the first national database of violence. The SecDev social media monitoring has provided insights into online extremism that were not gathered in any systematic way before PTIB’s engagement. CARU in southeastern Bangladesh has allayed fears of radicalism, shown where the actual sources of violence are, and highlighted the deadly political economy of border drug smuggling in a way that was not previously understood. As a group, the research component could improve how they communicate their findings, build relationships with policymakers, and track those interactions. |
| 4. | In the citizen engagement component, PTIB developed new media, messengers, and messages. D4P has made inroads into new and not typically engaged communities. New activities, such as youth engagement, are being developed. The citizen engagement component has been effective in engaging different communities in new ways, especially youth. Collectively, these activities could improve by better explaining how they connect with PTIB’s origins in preventing violent extremism and as well create new measures to track attitudinal and behavioral change among their participants and beneficiaries. |
| 5. | PTIB has built new relations through its government engagement component. These include partnerships with the police, Ministry of Foreign Affairs, and ICT Division. These connections provided the project the space to do some cutting-edge work in a very constricted political environment. This permission is granted as the government agencies understood the work and saw its benefits. The limit of this engagement is that there are more powerful actors that impede progress on key milestones, such as the acceptance of a National CPVE strategy, which goes beyond the reach of PTIB’s influence. |
| 6. | PTIB has come up against the opposing forces of intolerance, hate speech, and disinformation in supporting partnerships for inclusion and tolerance. It has had to overcome significant cultural and political barriers that create biases against women, ethnic and religious minorities, and vulnerable groups. The marginalization of these groups has been a challenge for project activities. These biases are often entrenched and created by forces outside the control of the project. Male users dominate the Bangladeshi internet. Communities on the geographic periphery lack online access. The history of Bangladesh has given many in the country particular perspectives on the majority religion and language that are not always inclusive and tolerant. For each of these challenges, PTIB has had to create a specific programmatic response. Some tactics, such as those to increase participation of women or ethnic minorities, have worked; however, learning from these, PTIB now needs to move forward more strategically in the future phase. |
| 7. | Looking ahead, this evaluation identified several recommendations grouped around the themes of management, capacity building, communications, and leaving no one behind. PTIB has done much good work, but better monitoring, regular evaluation, action research, and learning from its activities would strengthen a future project. Such management changes could include a more explicit change theory to tell a more compelling story about the project. When working with local partners, regular assessment of their needs and progress is essential in terms of the capacity building since it takes some time for behavior change. Project external communication could be improved. PTIB has been a high operational tempo project. More effort could be made to explain its work to key stakeholders as the knowledge products do not always sell themselves. Finally, Bangladesh presents particular challenges for working with women, young people, and the vulnerable and marginalized groups. PTIB has experimented with these groups, and some of its most innovative work is with these constituencies. Moving from the ad hoc and tactical to a more strategic and persistent approach will be essential if PTIB’s next phase is to leave on no one behind truly. |
| Recommendations | |
|---|---|
| 1 | MANAGEMENT: Better Monitoring, Evaluation, Research, and Learning (MERL) PTIB has been and should continue to be a learning project. Global evidence shows that development programmes are more effective if they conduct applied analysis, learning, and rapid-cycle evaluation during project implementation. These processes reduce the risk of program failure by ensuring that activities can adjust and choose the most likely path to achieving the goals. In its next phase, PTIB should devote more effort to monitoring, evaluation, and research and learning (MERL) systems to improve its existing adaptative management culture and the project’s effectiveness. This could include the following sub-recommendations:
|
| 2 | CAPACITY BUILDING: Improved capacity and diversity of local partners The sustainability of the ideas and practices that PTIB has introduced depends on the capacities of partners and supported groups. Therefore local partners, implementers, and participants require training for an activity, then carry it out and receive feedback afterward. Moreover, they need expert coaching over a sustained period. PTIB could increase the diversity of local partners and plan to sustain support to them with an ongoing emphasis on improving research, organizing, and communication to sustain changes in attitudes and behavior among target groups. This could include the following sub-recommendations:
|
| 3 | COMMUNICATIONS: Better project communications and expanded outreach Communicating with its complete circle of stakeholders and a growing number of beneficiaries requires significant efforts by key project staff In its next phase, PTIB could improve its internal processes and practices to better explain to key stakeholders, especially the GoB, what the project is working on, and why and how it is to be done. This could include the following sub-recommendations:
|
| 4 | LEAVE NO ONE BEHIND (LNOB): Gender, youth, vulnerable, and marginalized groups There are significant al and political challenges to creating an inclusive project in Bangladesh that must be recognized and continuously addressed. PTIB acknowledges these barriers, and many of its component activities are specifically designed to address them. They will remain a challenge in the second phase of the project and will need to be more systematically addressed. Based on its experience in the first phase, PTIB could improve its strategy and conduct more thorough planning to addresses cultural biases to ensure its future programming is more inclusive. This could include the following sub-recommendations:
|
1. Recommendation:
MANAGEMENT: Better Monitoring, Evaluation, Research, and Learning (MERL)
PTIB has been and should continue to be a learning project. Global evidence shows that development programmes are more effective if they conduct applied analysis, learning, and rapid-cycle evaluation during project implementation. These processes reduce the risk of program failure by ensuring that activities can adjust and choose the most likely path to achieving the goals.
In its next phase, PTIB should devote more effort to monitoring, evaluation, and research and learning (MERL) systems to improve its existing adaptative management culture and the project’s effectiveness.
This could include the following sub-recommendations:
- Developing a new Theory of Change (ToC) that tells a compelling story of how PTIB is a learning project finding relevant ways to promote inclusivity and tolerance and adapt to changes in the context.
- The ToC should emphasize that PTIB’s role is to develop a greater understanding through research. PTIB then applies this knowledge to its activities to build a new generation of media, messengers, and messages that counter hate speech, intolerance, and disinformation. In this way, PTIB contributes to preventing extremism and violence in Bangladesh.
- Exploring how the knowledge products of PTIB’s research components cross-fertilize and inform the work of its citizen and government engagement components. Given the sensitivities and time involved in approving and distributing written documents, this may include using more discussion-based techniques to share knowledge more widely.
- Using various knowledge management tools, such as short case studies, action reports, and practice notes, to better document how PTIB has learned lessons and applied them to the project.
- Using the market and public opinion research tools, such as focus groups, to measure the effectiveness of messages developed for its citizen engagement component and to continually sharpen its communications and effectiveness with beneficiaries.
- Developing new measures to better engage its audience before and after activities, such as quick quizzes, to better measure and understand attitudinal shifts and the potential for behavioral change.
- Drawing on the growing data sets of PTIB’s research components, such as BPO and CARU, to conduct more empirical studies and secondary analysis to increase general understanding of the challenges of countering violence in Bangladesh and guiding future programming.
- Engaging a MERL specialist to work with PTIB to advise on how to improve its internal knowledge management and information sharing between sub-components.
Management Response: [Added: 2020/11/03]
Agreed. For the next phase of PTIB, the ToC will be rewritten, following the evaluation recommendations. PTIB will put more emphasis on producing knowledge products within the current phase. Especially, PTIB response to the COVID crisis provides good opportunities to explain how research components have cross-fertilized citizens and government engagement components. Knowledge products and research will draw on the project databases. Within the current phase, PTIB will start to measure its effectiveness of messaging and begin with a review and improvement of the existing M&E framework with innovative measures like quick quizzes and use of social media analysis software. For the next phase's design, PTIB will involve a MERL specialist to ensure MERL is integrated into the program design. The design will include a review of organizational development and will ensure MERL responsibilities are allocated effectively.
Key Actions:
| Key Action | Responsible | DueDate | Status | Comments | Documents |
|---|---|---|---|---|---|
| Drafting of a new prodoc, including improvement of ToC
[Added: 2020/11/03] [Last Updated: 2021/07/27] |
PTIB | 2021/09 | Initiated | The draft ProDoc was shared in the LPAC meeting held on 11 March 2021. The project team finalized edits to address the comments from LPAC members. The final draft has been submitted for ERD clearance on 20 May 2021. History | |
| Knowledge product on PTIB COVID response
[Added: 2020/11/03] [Last Updated: 2021/03/31] |
PTIB | 2021/03 | Completed | Knowledge product was developed. The consultant has submitted the final report in March 2021. History | |
| Study on trends and impact of COVID disinformation in Bangladesh
[Added: 2020/11/03] [Last Updated: 2021/03/03] |
PTIB | 2021/03 | Completed | UNDP has received and accepted the final report. History | |
| Procurement of Crowdtangle, analysis software of social media in support of M&E
[Added: 2020/11/03] |
PTIB | 2020/10 | Completed | PTIB has procured Crowdtangle to 4 PTIB staff as users. This software will support improved analysis of social media campaigns of PTIB and its partners. | |
| MERL integrated in the prodoc design
[Added: 2020/11/03] [Last Updated: 2021/03/03] |
PTIB | 2021/03 | Completed | MERL is properly integrated in the prodoc, reviewd and cleared by UNDP M&E lead. History |
2. Recommendation:
CAPACITY BUILDING: Improved capacity and diversity of local partners
The sustainability of the ideas and practices that PTIB has introduced depends on the capacities of partners and supported groups. Therefore local partners, implementers, and participants require training for an activity, then carry it out and receive feedback afterward. Moreover, they need expert coaching over a sustained period.
PTIB could increase the diversity of local partners and plan to sustain support to them with an ongoing emphasis on improving research, organizing, and communication to sustain changes in attitudes and behavior among target groups.
This could include the following sub-recommendations:
- Updating the 2019 BPO evaluation with another periodic external review to measure progress and identify new needs of capacity development after a year of more intensive support from the data scientist in residence.
- Continuing support to BPO to improve its research capabilities and written analytical products and gather more information about how its data is being used and who is using it.
- Extending the focus on working with non-traditional partners among marginalized and vulnerable communities, especially those without reliable internet access and distant from Dhaka.
- Understanding that all capacity building requires long-term support, technical advice, mentoring, and coaching and planning in the next phase for sustained interventions of these types.
Management Response: [Added: 2020/11/03]
Agreed. The next phase will be designed to ensure capacity building and measuring its impact will become more prominent. Long-term capacity development support will be provided. PTIB plans an external BPO review to measure progress from the 2019 review and inform BPO support in the next phase. The Diversity for Peace initiative will be expanded in the next phase with priority on non-traditional partners with limited access to the internet.
Key Actions:
| Key Action | Responsible | DueDate | Status | Comments | Documents |
|---|---|---|---|---|---|
| Drafting of prodoc, including long-term capacity building efforts
[Added: 2020/11/03] [Last Updated: 2021/03/03] |
PTIB | 2021/03 | Completed | The draft prodoc included the long term capacity building of local partners. History | |
| BPO review
[Added: 2020/11/03] [Last Updated: 2021/07/27] |
PTIB | 2021/09 | Not Initiated | History |
3. Recommendation:
COMMUNICATIONS: Better project communications and expanded outreach
Communicating with its complete circle of stakeholders and a growing number of beneficiaries requires significant efforts by key project staff
In its next phase, PTIB could improve its internal processes and practices to better explain to key stakeholders, especially the GoB, what the project is working on, and why and how it is to be done.
This could include the following sub-recommendations:
- Finding new and regular channels, such as small group events, as venues to discuss project’s more sensitive work on violence and social media monitoring to help key stakeholders, especially decision makers, explore how this research could be better used for policymaking and planning.
- Making social media monitoring information and analysis more widely available to a larger group of stakeholders through workshops, webinars, and other such discussion-based interactions. PTIB has used such techniques but could do this more regularly and build on its newly created youth panel model.
- Better documenting and analyzing the interactions its research components have with government and international officials to show how these interactions' seniority or frequency create relationships that can contribute to policy processes.
- Initiating discussions with the Advisory Board about the shape of its Phase Two and how it would build on those activities in Phase One that have been effective or having potential.
- Engaging partners in the Digital Peace Movement to discuss what a new phase of their work might look like, including how to keep up momentum and networks if there were a funding gap or disruption in funding.
Management Response: [Added: 2020/11/03] [Last Updated: 2020/11/03]
Agreed. PTIB, during the remaining of its current phase, will organize different events to present and discuss findings of monitoring and research to key stakeholders, including policy makers. PTIB will expand its youth empowerment initiative and will form a second youth platform. In addition to the orientation training and quarterly discussions on VE online narratives, PTIB will inform the platforms on broader issues related to hate speech and disinformation. PTIB has surveyed the PTIB COVID brief recipients on their feedback on the information provided to explain how the information has been used, among survey participants are government and international officials. The Project Advisory Board and participants of the digital peace movement will be consulted during the prodoc design.
Key Actions:
| Key Action | Responsible | DueDate | Status | Comments | Documents |
|---|---|---|---|---|---|
| Presentation of knowledge product on PTIB COVID response
[Added: 2020/11/03] [Last Updated: 2021/05/28] |
PTIB | 2021/06 | Completed | Instead of presentation meeting, the report has been shared widely. History | |
| Presentation of trends and impact of COVID disinformation
[Added: 2020/11/03] [Last Updated: 2021/03/31] |
PTIB | 2021/03 | Completed | The study was presented in a UNDP weninar held on 3rd March 2021. History | |
| Presentation of VE online messaging in 2020
[Added: 2020/11/03] [Last Updated: 2021/03/31] |
PTIB | 2021/03 | Completed | Presented in a webinar organized by UNDP on 3rd March 2021. History | |
| User survey among recipients of PTIB COVID briefs
[Added: 2020/11/03] |
PTIB | 2020/10 | Completed | PTIB has conducted a survey through survey monkey to seek feedback and have a better understanding about how information in the briefs are used. History | |
| Involve project advisory board and digital peace movement participants in consultations
[Added: 2020/11/03] [Last Updated: 2020/12/20] |
PTIB | 2020/12 | Completed | PTIB has consulted with project advisory board members and digital peace movement participants on the prodoc design. Consultations brought important recommendations to inform the activities and strategies of phase 2. History |
4. Recommendation:
LEAVE NO ONE BEHIND (LNOB): Gender, youth, vulnerable, and marginalized groups
There are significant al and political challenges to creating an inclusive project in Bangladesh that must be recognized and continuously addressed. PTIB acknowledges these barriers, and many of its component activities are specifically designed to address them. They will remain a challenge in the second phase of the project and will need to be more systematically addressed.
Based on its experience in the first phase, PTIB could improve its strategy and conduct more thorough planning to addresses cultural biases to ensure its future programming is more inclusive.
This could include the following sub-recommendations:
- Formalizing its gender strategy in future project documents and creating gender marker data to address and measure the response to the known gender biases against women in its project activities, including access to the internet and overnight mentoring boot camps.
- Including the challenges to the inclusion of women in project activities in the revised project’s Theory of Change for the next phase.
- Developing more specific strategies and plans about addressing gender, marginalized and vulnerable groups challenges, especially those living outside of Dhaka.
- Developing more advanced M&E tools to analyze and track women’s participation in PTIB activities.
- Discussing and strategizing with all partners how to counter known biases in using digital space, including gender and geography. Design additional activities and explore new themes to mitigate these entrenched biases.
- Commissioning more research on gender-specific issues to stimulate more discussion about women's role and maximize women’s participation, agency, and voice.
- Building on and documenting the experience of DKC to explore how to expand women’s participation through changing rules, evaluation criteria, and adapting mentorship programs.
- Adding a component to increase women's participation in digital literacy at the core of PTIB’s objectives.
Management Response: [Added: 2020/11/03]
Agreed. PTIB will include a gender strategy in the new prodoc. The challenges of women's inclusion will be well reflected in the theory of change and the M&E framework. In the next phase, PTIB will conduct a series of research on women's inclusion issues, including gender biases in the digital space. PTIB will equally include women's voices in the consultation while desiging the next phase of the project.
Key Actions:
| Key Action | Responsible | DueDate | Status | Comments | Documents |
|---|---|---|---|---|---|
| Gender and women issues well reflected in drafting of the new prodoc
[Added: 2020/11/03] [Last Updated: 2021/03/03] |
PTIB | 2021/03 | Completed | Gender and women issues are well reflected in draft prodoc. History | |
| Ensure equal women participation in consultations part of the prodoc design
[Added: 2020/11/03] [Last Updated: 2020/12/20] |
PTIB | 2020/12 | Completed | Prodoc consultation ensured equal participation of women. PTIB arranged separate group discussions for male and female of the same programme segment to ensure women's voice is not lost in male dominance. History |