Mid-term Evaluation of the Shaping Inclusive Finance Transformations Programme SAARC Region

Report Cover Image
Evaluation Plan:
2018-2021, UNCDF
Evaluation Type:
Others
Planned End Date:
06/2019
Completion Date:
08/2019
Status:
Completed
Management Response:
Yes
Evaluation Budget(US $):
48,370

The Shaping Inclusive Finance Transformations (SHIFT) in the South Asian Association for Regional Cooperation (SAARC) programme is a regional market development initiative implemented by the United Nations Capital Development Fund’s (UNCDF) Financial Inclusion Practice Area (FIPA). The programme seeks to stimulate investment, business innovations and regulatory reform to bring about lasting changes for low-income people. Specifically, it aims to expand economic participation of and opportunities for women and small and growing businesses to be active agents in the formal economy. By 2021, it aims to have enabled “at least one million low-income people, 65% of whom are women, and 30,000 small and growing businesses to access and use financial services to secure opportunities for employment, enterprise development, and increased sustainable consumption.The programme design anticipated that SHIFT in SAARC would address both demand and supply-side constraints in interlinked financial markets, working primarily through financial service providers (FSPs).

As a regional programme, SHIFT in SAARC aims to eventually take advantage of regional-level policy and cooperation processes, to leverage multi-country perspectives and achieve synergies in policy advocacy. At the time of this evaluation, the programme has only implemented activities in Bangladesh.

The SHIFT in SAARC programme has benefited from three sources of funding as of the mid-term evaluation; The European Union (EU), The Bill and Melinda Gates Foundation (BMGF), and internal UNCDF core funding. As described in the full report, the BMGF reduced its commitment during programme implementation. The total committed contribution to the programme was initially USD 9,284,227. At the end of 2018, the revised committed contribution from all three sources was USD 7,604,777, and actual expenses against that commitment amounted to USD 4,019,190.

Share

Title Mid-term Evaluation of the Shaping Inclusive Finance Transformations Programme SAARC Region
Atlas Project Number:
Evaluation Plan: 2018-2021, UNCDF
Evaluation Type: Others
Status: Completed
Completion Date: 08/2019
Planned End Date: 06/2019
Management Response: Yes
Corporate Outcome and Output (UNDP Strategic Plan 2014-2017)
Evaluation Budget(US $): 48,370
Source of Funding:
Evaluation Expenditure(US $): 48,370
Joint Programme: No
Joint Evaluation: No
Evaluation Team members:
Name Title Email Nationality
Liz McGuinness Team Leader lmg.consulting.dc@gmail.com
Marcella Willis Co-evaluator marcella.willis@gmail.com
Jason Wares Project Supervisor jason.wares@imcworldwide.com
Farah Hussain National Specialist farahiqbalhussain@gmail.com
GEF Evaluation: No
Key Stakeholders:
Countries: GLOBAL
Lessons
1.

 

1. Future Project Planning

For future multi-year market system development projects, the evaluation team recommends budgeting resources for a preparation phase that allows time to build relationships with the relevant stakeholders, negotiate and seek government approvals to launch the programme, fully recruit and train the programme team, finalize activity plans with stakeholders, and identify viable innovation grant recipients. In addition, the market analysis should be more robust and include a gender analysis. This will ensure that real economy sector work can support gender equality and women’s economic empowerment. Similarly, there should be an analysis of entry points to promote DFS to identify high-volume sectors that include priority segments, such as women.

2. Improve Design

The evaluation team found the programme, as described in documents and by staff, to be overcomplicated and confusing. The evaluation team recommends that UNCDF simplify and clarify their programming in Bangladesh. While the SHIFT in SAARC framework was very broad and flexible, it may be that UNCDF could have implemented the two projects as stand-alone activities, with each being simpler. Alternately, if the overarching goal is to increase access and use of DFS, SHIFT in SAARC should concentrate on activities centred on this goal.

3. Match the programme need with funding opportunity

The SHIFT in SAARC programme uses a market systems development approach and incorporates an appropriate level of flexibility. However, UNCDF’s funders did not fully support the level of adaptation that was/is required. For example, BMGF reduced funding when it was clear that MNOs were not included in the revised MFS Guidelines and the EU reportedly will not permit UNCDF to adjust the results framework or to drop certain activities. This constrains SHIFT in SAARC’s ability to achieve its objectives.

4. Risks

The programme design included a Risk Management Plan which identified potential strategic, political, operational and organizational risks to SHIFT in SAARC and associated mitigation measures. Several of these risks have occurred, and it is useful to go back and assess how well the anticipated mitigation measures worked.

Risk: The data that is collected and analysed is not properly understood and used correctly.

There is no indication that data has not been understood, some believe that policymakers do not take evidence into consideration. The description of this risk may need to be expanded and appropriate mitigation measures identified. For example, adapting SHIFT in SAARC to use a broader approach to policy advocacy.

Risk: The products and business models do not lead to impact beyond the programme.

This is a real risk for SHIFT in SAARC, and the proposed mitigation measures (monitoring to allow adjustments and supporting providers) will be insufficient if there is not enough time to disseminate the pilot results before programme funding ends. This risk should be expanded in the future to consider delays in the innovation models and how these can be mitigated.

Risk: Staff recruitment takes longer than anticipated.

Although good mitigation measures were identified and employed by the programme (e.g. support from the regional office, use of long-term consultants), the programme experienced operational delays associated with this risk. UNCDF needs to identify other measures that can expedite recruitment or ensure other global/regional staff are available for future programmes.

Risk: SHIFT in SAARC does not lead to impact beyond the programme timeline.

The mitigation measures included engaging strategic partners to ensure their contribution to the long-term sustainability of the programme. It is unclear to what extent the Programme is engaging the co-applicants to identify how they can contribute to long-term sustainability. While capacity building of these partners will have sustainable effects on those institutions, how will their activities under SHIFT in SAARC continue when funding ends?

 

 

 

 


Findings
1.

5.1 Criterion 1: Relevance and quality of design
The appropriateness of the programme’s objectives to the real problems, needs and priorities of its target groups/beneficiaries and the quality of programme design through which these objectives are to be reached.


The SHIFT in SAARC programme design is relevant to the Bangladeshi context due to its effective stakeholder consultation during design and adaptive approach to implementation. The programme design directly addresses the priorities of the Government of Bangladesh for digital financial inclusion and gender equality. It also contributes to UNDAF and several SDGs.

 


Tag: Relevance Gender Equality Programme/Project Design Country Government UN Agencies Inclusive economic growth Agenda 2030

2.

The programme design is aligned with internationally recognized best practices for market systems development for financial inclusion, despite a few notable exceptions. The timeframe and secured funding are insufficient for this approach. The overall objective - ‘to increase the percentage of adults using DFS’ - for the Bangladesh programme is not measurable.


Tag: Relevance Innovation Programme/Project Design Inclusive economic growth Trade and Development

3.

Question 1.1: How relevant are the SHIFT in SAARC programme approaches that are being implemented to the priorities and policies of the Government of Bangladesh including the regulatory/policymakers on financial inclusion?
The programme is relevant to the GoB’s policy priorities, particularly those related to digital financial inclusion for low-income people.


Tag: Relevance Rule of law Innovation Programme/Project Design Country Government Inclusive economic growth Poverty Alleviation

4.

SHIFT in SAARC addresses key priorities of the GoB, particularly the focus on achieving financial inclusion through digital pathways, as articulated in the Seventh Five Year Plan for FY2016-2020: Accelerating Growth, Empowering Citizens. The goals and objectives of SHIFT in SAARC specifically support several of the plan’s objectives, including financial access for low-income people through MFS and agent banking, and use of MFS for key payments (e.g. Government-to-People (G2P) and remittances). The Seventh Five Year Plan also emphasizes empowering women and reducing gender inequality.


Tag: Relevance Gender Equality Women's Empowerment Innovation Country Government Inclusive economic growth Poverty Alleviation

5.

The GoB is focused on achieving the SDGs.32 SHIFT in SAARC contributes to the SDGs, primarily SDG1 (No Poverty) and SDG17 (Partnerships for the Goals). SHIFT in SAARC also focuses on marginalized communities thus contributing to SDG 10 (Reduced Inequality). The work with Kallyani women and research related to women’s financial inclusion contributes to SDG5 (Gender Equality).

 


Tag: Relevance Gender Equality Partnership Country Government Inclusive economic growth Poverty Reduction Agenda 2030

6.

Question 1.2: As presently designed, how coherent is the SHIFT in SAARC programme design in view of its objectives, and how well is it designed with regard to eventual transition, expansion and replication of the programme approach by others?
The programme’s TOC is logical and coherent. However, there is incomplete evidence that the logic will work to achieve its objectives. The programme is only partly designed according to best practices for market systems development for financial inclusion, and its timeframe and secured funding are insufficient for its approach. The overall objective for the Bangladesh programme is not measurable.


Tag: Relevance Theory of Change Inclusive economic growth Trade and Development

7.

Implicit in the TOC reasoning is the assumption that actors are embedded in contexts, particularly institutional contexts, which will support the desired behaviour change. As noted earlier, the financial inclusion policy context of Bangladesh was very favourable when the programme was under design but by the time of implementation had become more restrictive. Consequently, it has been more difficult for the programme to achieve its policy and advocacy objectives.
 


Tag: Theory of Change Inclusive economic growth Policy Advisory

8.

Two aspects of the TOC merit mention. The evaluation team found the placement of communications activities within the Innovation pillar confusing since communications is a critical function that runs across all pillars. Also, the overarching objective for SHIFT in SAARC in Bangladesh - To increase the percentage of the adult population in Bangladesh actively using a variety of affordable DFS from 9% to 35% by 2019 - is not actually measurable. It is only possible to count the number of DFS accounts (not individuals) with available national-level data.


Tag: Communication Monitoring and Evaluation Results-Based Management Theory of Change

9.

Question 1.2.1: Is the programme design in line with internationally recognized best practices?
The programme design is aligned with internationally recognized best practices for market systems development for financial inclusion, with a few exceptions. For example, there is a need for a long-term perspective and for an improved accountability approach.

The SHIFT in SAARC design takes a market systems development (MSD) approach by focusing on whole market systems (financial services and FMCG retail) and facilitating changes across these markets. The programme design follows key MSD principles of the Consultative Group to Assist the Poor (CGAP) for financial inclusion. The programme aims for systemic change in attempting to influence inclusiveness in both market systems. SHIFT in SAARC plays a facilitative role in these market systems by building capacity and enabling market actors to perform their functions more effectively. The programme also allows for experimentation through incentivizes for innovative financial product pilots and services for the FMCG market. This experimentation will support learning and adaptation among actors in both market systems.


Tag: Relevance Change Management Programme/Project Design Inclusive economic growth Trade and Development

10.

Question 1.2.2: Are the programme approaches and activities, as designed, likely to achieve the objectives, based on experiences of similar programs and the context?
Based on experience with similar programmes and the political economy context in Bangladesh, it is not clear that SHIFT in SAARC will achieve its programme objectives with the existing funding and the associated timeframe.

A comparison of the SHIFT in SAARC programme design to other UNCDF MSD programs for financial inclusion reveals that there is little evidence to date that the policy and advocacy, data analysis, or capacity building pillars will result in the desired long-term outcomes. There is some limited evidence that technical assistance and grant support for FSPs increases the supply of DFS products.


Tag: Effectiveness Civic Engagement Programme/Project Design Capacity Building Inclusive economic growth Policy Advisory Technical Support Data and Statistics

11.

The SHIFT in SAARC programme follows the same MSD approach as SHIFT in ASEAN (SHIFT ASEAN) upon which it was modelled. SHIFT ASEAN has a similar TOC, goals and intervention approaches with a few notable exceptions. SHIFT ASEAN, which has been implemented at a regional level since 2014, focuses solely on improving the financial inclusion ecosystem to bring about the targeted impacts, and it supports financial services innovations through a Challenge Fund.


Tag: Innovation Programme/Project Design Theory of Change Inclusive economic growth

12.

UNCDF’s Mobile Money for the Poor (MM4P) programme uses an MSD approach to build the overall DFS ecosystem in selected countries, although it differs from the SHIFT design in several ways. Activities are organized into six workstreams related to elements of the DFS ecosystem.34 Activities centre on “ecosystem development,” consisting of the range of interdependent measures that bring stakeholders together to build an inclusive sector.35, 36 MM4P uses a three-phase approach for in-country programme development: 1) Buy-in: build relationships with all stakeholders; 2) Improve: support key stakeholders through smaller engagements to improve current business; and 3) Innovate: work with several stakeholders to design and launch innovative concepts and partnership models.
SHIFT in SAARC is not engaged in all the MM4P workstreams, although the framework is certainly flexible enough for it to do so. Additionally, SHIFT in SAARC has skipped the Improve phase, raising the question of whether the program would have benefited from supporting a small group of stakeholders in order to become familiar with them and their capabilities before moving on to the innovation grant activity.


Tag: Implementation Modality Innovation Programme Synergy Inclusive economic growth

13.

The MM4P programme was found to have “achieved greater traction among FSPs than in the policy/regulation and infrastructure space.” The MM4P programme has provided a significant amount of direct technical assistance and grants to FSPs which was found to result in greater investment in DFS. The DFS Working Group was also found to be an important contributor to DFS development. These finding suggests that the innovation grants, along with support for the DFCG, may lead to greater investment in DFS in Bangladesh. In comparison, MM4P’s work in policy and regulation was relatively limited, therefore the team could not draw conclusions about the ability of that workstream to develop a DFS eco-system.


Tag: Resource mobilization Innovation Technical Support

14.

Question 1.2.3: Is the programme design consistent between programme objectives and partners’ needs, strategy and capacity to implement?
The programme as designed is consistent between the programme objectives and partners’ needs and strategy. However, the design has not always been implemented as planned. See Section 5.3: Effectiveness of this report, which provides details on SHIFT in SAARC’s partners within the Bangladesh Bank.


Tag: Relevance Programme/Project Design

15.

Question 1.2.4: Are funding amounts and planned timeframe consistent with objectives and provide necessary flexibility?
The five-year timeframe of the SHIFT in SAARC framework is the minimum time required for an MSD programme’s success37. These types of programmes seek to change the behaviour and relationships of market actors, which takes time.

The context also affects the ability of SHIFT in SAARC to achieve its objectives within time and funding limits. As one interviewee reported, “In Bangladesh, the policy context is very critical, policymakers are not interested to welcome new innovation or new discussions at the policy level…two to three years is not sufficient to have impact at the policy level.”
SHIFT in SAARC’s funding to date has not been sufficiently flexible for an MSD programme. It has not always supported adaptation to changed conditions. For example, the BMGF reduced funding when it was clear that the outcome they desired for the MFS guidelines would not be achieved. Still SHIFT in SAARC had enough remaining funds to change the programme approach to one that leverages other actors (the DFCG) and projects (A2i) to influence policymakers.


Tag: Effectiveness Resource mobilization

16.

Questions 1.2.5: How well is the programme designed with regard to eventual transition, expansion and replication of the programme approach by others?
The SHIFT in SAARC design relies primarily on innovation grants and capacity building to ensure that the programme approach is transitioned, expanded and/or replicated by others.
Innovation grants are designed to test innovations and provide cases that can be taken up by the private sector. These business models are well placed to be scaled up and replicated if they are successful and profitable. However, it is too early to come to any conclusions.
UNCDF has developed a DFS training module and transferred it to the BBTA so that they can build the capacity of the entering class of central bankers on DFS each year. This initiative has the potential to ensure a continuous improvement in the Bank’s capacity in DFS.
The programme intended that capacity building activities for micro-merchants and Kallyani women entrepreneurs be expanded and replicated by co-applicants. Whether this will be possible depends on the specific partner’s core business interests and business model.


Tag: Sustainability Innovation Programme/Project Design Private Sector Capacity Building

17.

Question 1.3: To what extent is the SHIFT in SAARC programme design sufficiently taking into consideration the needs and interests of: demand-side customer segments (low-income groups, women and small and growing businesses); supply-side service providers; and meso-level organizations (e.g. BDS providers, associations)?
SHIFT in SAARC has considered the needs and interests of demand-side customers and the target merchants by undertaking field-level research with these target groups.
The MMLA report provided evidence that informed the design of the business training, mobile app and other activities for the demand-side customers, specifically the micro-merchants. The InfoLady programme supported by SHIFT in SAARC, considered the needs and requirements for rural women to earn an income, as well as the needs of female consumers. It is designed to allow women to work on their own schedules and sell valuable products and services to neighbouring women, who are not able to buy these items elsewhere.

The DFS Assessment of policymakers informed the capacity building programme and the policy and advocacy activities implemented with BMGF funding. SHIFT in SAARC co-funded the Bangladesh PoWER study and the Gender Centrality research, to better understand financial inclusion and economic challenges and opportunities faced by women in Bangladesh. There is not yet evidence that these have influenced programme activities or had an impact on the broader DFS ecosystem.


Tag: Relevance Women's Empowerment Programme/Project Design

18.

Question 1.4: To what extent is programme design sufficiently taking (into consideration) cross-cutting issues such as human rights, marginalized groups (including the disabled) and gender?
The programme design aligns with national and funder level gender standards
. It implicitly aligns with United Nations standards on mainstreaming gender. No gender analysis was included in the initial assessment.


Tag: Vulnerable Gender Equality Human rights Programme/Project Design Disabilities

19.

Question 1.4.1: Was a gender analysis included during the initial needs assessment?
No gender analysis was performed for Bangladesh before or during the design of the SHIFT in SAARC programme.38 As a result, the programme team was surprised when the MMLA revealed that only two percent of all micro-merchants in the country are women. Gender disaggregated secondary data was referred to during project design, but the reporting of this data was inadequate.
The evaluation team found that there is no requirement for using a gender analysis as an input to designing a programme framework. There are gender equality screening questions as part of the programme approval process, but it is not clear if these were applied to the programme, nor is it clear that they would have been effective in this case.39 Requiring a gender analysis, with mandatory primary research, would allow UNCDF to better align programme designs that specifically aim to advance gender equality, with the given context.


Tag: Gender Equality Programme/Project Design Data and Statistics

20.

Question 1.4.2: Does the project align with United Nations standards on mainstreaming strategy on gender into the design of SHIFT?
SHIFT in SAARC was designed with an implicit but not explicit alignment to UNCDF or United Nations gender equality guidelines. The design of the EU-funded project was required to follow EU gender guidelines and is aligned accordingly. The BMGF-funded project has no explicit gender alignment. In designing programmes, UNCDF design teams are expected to refer to broad United Nations principles about gender equality and the empowerment of women, but staff report that these high-level principles are not easily implementable. There is no specific gender analysis tool in place that a design team could use to better address gender equality objectives in programme design.


Tag: Gender Equality Gender Mainstreaming Programme/Project Design UN Agencies

21.

Question 1.4.3: Does the programme align with national gender-related goals?
On paper, the emphasis on gender aligns with national gender-related goals as articulated in the GoB’s Vision 2021 which targets empowerment and equal rights for women by 2021. It also aligns with the Sixth and Seventh Five Year Plans. Gender equality and empowerment is one of seven targets of the Sixth Five Year Plan, with goals to raise women’s participation in tertiary education and women’s literacy continued under the Seventh Five Year Plan.


Tag: Gender Equality Programme/Project Design Country Government

22.

5.2 Criterion 2: Efficiency
The extent to which the programme has delivered quality outputs that are appropriately managed and overseen.

UNCDF management and operations have delivered several activities and outputs over the past two years. They have developed and disseminated important and impactful research and evidence on key themes; launched six innovation grants; built the capacity of policymakers and regulators; built a coalition of public and private stakeholders in the DFS ecosystem; and built the capacity of implementing partners and low-income entrepreneurs. The evaluation’s interviews and focus groups revealed that programme beneficiaries strongly appreciated the trainings provided. The PSC members appreciate the programme and its leaders as well as SHIFT in SAARC’s taking up the micro-merchant segment, and linking the real economy sector, (i.e. the FMCG industry) to FSPs.


Tag: Efficiency Small Grants Programme Innovation Partnership Private Sector Capacity Building

23.

Question 2.1: How well has UNCDF management and operations (at both the programme and headquarters level) supported delivery of the programs to date?
The SHIFT in SAARC team has accomplished much in two years, but the team was understaffed and lacked certain requisite expertise. The programme met most of its output targets, even though the programme was challenged with serious delays in the first year of operations. While the team has caught up with implementation, delays in launching the innovation grants will negatively impact the achievement of programme objectives. Management is relying on informal methods to keep abreast of changes in the financial and real economy markets. Adaptive management of the program could be strengthened by using systematic enhanced monitoring methods too.


Tag: Challenges Efficiency Human and Financial resources Monitoring and Evaluation Operational Efficiency Project and Programme management

24.

The relatively small SHIFT in SAARC team in Bangladesh, with the assistance of the regional team in Bangkok, accomplished many activities in just two years. To keep the programme staff “nimble and focused on looking at trends, managing relationships and influencing people,” the programme is designed to “staff-up” as funding agreements are signed while using a pool of short- and long-term consultants and sharing resources with other UNCDF programmes.40 The programme also relies on delivering activities through co-applicants, partners and contractors/grantees. There are indications, however, that the number and mix of in-country staff were insufficient to meet the volume of work required. One position in Bangladesh remained vacant for almost one year as the process for replacing the incumbent was drawn out, and the Dhaka team lacks staff with programming expertise in Medium Small and Micro Enterprises (MSMEs) development. In late 2018, in response to delays and to ensure that planned activities under the MDDRM project are completed on time, three additional national consultants were hired.
As compared to plan, the actual composition of the regional and in-country teams reflects the priorities of funding received. The regional Women’s Economic Empowerment Analyst position went unfilled while an unplanned Innovation Manager position was filled. Importantly, there are no women on the technical teams of either the regional or national offices.


Tag: Challenges Efficiency Human and Financial resources

25.

In 2016 and 2017, spending against budget lagged somewhat for the BMGF project. However, in 2018, both projects under SHIFT in SAARC spent around 99 percent of their annual budget. By the end of 2018, programme funds utilized against budget was as follows: BMGF funding 95% delivered; EU funding 34% delivered; and UNCDF core funding 100% delivered. Through 2018, it appears that programme staff managed their activities well within the available financial resources, despite delays in programme implementation. However, analysis of spending versus budget suggests that SHIFT in SAARC will have to spend US$3,358,587 during the period January 2019 through January 2020 to use all available funds by the end of the EU project period. This does not seem realistic given that they have not spent more than $2,360,718 in any given year, so far. Even though the programme plans to spend US$1.5 million on innovation grants through the end of the project period, it will still be difficult to spend all remaining funds.


Tag: Efficiency Operational Efficiency Project and Programme management

26.

The evaluation team found that the SHIFT in SAARC Board meetings are well-documented with both extensive reporting from the programme and detailed minutes. The Board TOR states that “a representative of the SAARC Secretariat and/or SAARC member country will hold at least one seat on the Board” in order to represent the interests of programme beneficiaries, i.e. low-income people and MSME owners. The Board has not had a beneficiary representative present at any of the Board meetings.41


Tag: Efficiency Communication Operational Efficiency Project and Programme management

27.

The PSC has been active in guiding the programme, and it recommended an increase in the target outcome for the BMGF-funded project.42 Per the TOR, the PSC was to meet twice per year, but members agreed to meet every quarter. Records and interviews indicate that the committee has met less frequently, with meetings recorded on November 21, 2017, January 31, 2018, and August 12, 2018. Meetings are scheduled based on the needs of the MoF. Some PSC members suggested that the programme should find more ‘vibrant’ representatives, meet more often, and take members to the field to see SHIFT in SAARC in action.


Tag: Efficiency Communication Operational Efficiency Project and Programme management

28.

The SHIFT in SAARC team and stakeholders recognize that programme implementation was significantly delayed during the first year. The BMGF-funded project, signed in July 2016, did not produce a major output until May 2017. Stakeholders agree that the team and certain partners have made much progress in catching up. However, the delays will likely result in SHIFT in SAARC underachieving its objectives when current funding ends in January 2020.
Implementation delays reportedly were due to lags in obtaining government approval and in recruiting the team. These processes were known risks and could have been better mitigated. However, closer analysis suggests that implementation delays also could have been caused by management and external issues.

  • • The in-country team’s inexperience working with UNCDF and the United Nations system contributed to procurement difficulties and subsequent delays in contracting and operations. More effective support from the UNCDF regional office on administrative and operational functions could have helped to address this challenge.
  • • Implementation of the BMGF-funded project activities was slowed at the donor’s request while the donor reconsidered its approach.
  • • Weak local capacity on the part of the MMLA research firm and the co-applicants led to significant delays that had knock-on effects on subsequent activities of the EU-funded MDDRM project.43
  • • Staff were required to take on additional and unplanned activities when SHIFT in SAARC was unable to bring on two implementing partners under the BMGF-funded project.

As a result of the delays, it is unlikely there is sufficient time left in the final year of the programme to build on the results of the piloted innovations. This lack of time is unfortunate as several stakeholders emphasized the importance of having the demonstration effects of successful innovations in order to encourage the private sector to enter the market.


Tag: Project and Programme management Challenges Efficiency Human and Financial resources Operational Efficiency

29.

The programme has met or exceeded its targets in terms of the SHIFT in SAARC output indicators for the first three pillars. Given the delays with implementation, this indicates that UNCDF underestimated possible achievement of outcome targets. If UNCDF had not experienced delays, the programme could have achieved higher output numbers.
The programme found it difficult to meet the targets in Pillar Four activities due to delays in starting the innovation grants. Only one grant agreement had been signed by the end of 2018 as compared to the target of two. By the end of 2019, the programme aims to have signed all six grant agreements. Delays were in part due to holdups with the MMLA as noted above, and to the more time-consuming process required to design and procure six separate innovation grants rather than one challenge fund, as originally planned.


Tag: Efficiency Operational Efficiency

30.

Question 2.1.1: Is programme management monitoring changes to the financial and real market economies and political/economic environment and making course corrections as necessary?
Management’s ability to effectively change course depends on having accurate and timely information on financial services and real economy markets and the political economy. Management appears to focus on informal monitoring of market and political economy changes through dialogue with key partners. SHIFT in SAARC management is closely engaged with policymakers and in contact with private sector actors.
Despite the lack of a systematic enhanced monitoring system, the programme has adapted to changing circumstances based on field research and assessments. For example, the MDDRM project was adapted based on evidence that 1) there were few BDS providers serving micro-merchants and 2) there were few women micro-merchants. The programme would be well served in future by having a more systematic approach to tracking changes in the market system.


Tag: Efficiency Monitoring and Evaluation Operational Efficiency Project and Programme management

31.

Question 2.2: To what extent have programme deliverables (outputs) met the expectations of programme beneficiaries, board members and development partners?
The quality of programme outputs has met the expectations of end-user beneficiaries and the expectations of stakeholders and partners. However, PSC members and one funder are concerned with the timeliness of delivery. Both funders are dissatisfied with the quality and quantity of communications from the SHIFT in SAARC team.


Tag: Challenges Effectiveness Communication Project and Programme management

32.

As previously mentioned, the BMGF reduced the amount and length of time for funding SHIFT in SAARC in early 2018. The reasons given were: 1) the changing regulatory context implied that the objective of achieving a more competitive and open market for other entrants could not happen; 2) there were other agencies working in the DFS space that attracted BMGF funding, and; 3) delays in SHIFT in SAARC implementation meant that it would be impossible to achieve all the deliverables within the timeframe.44


Tag: Donor Challenges Human and Financial resources

33.

All funders were concerned with the lack of communication from SHIFT in SAARC management. Concerns include the lack of responses to funder inquiries and forgetting to credit funder support, e.g. for sponsoring a conference.45 Additionally, funders prefer more tailored programme updates as opposed to the general programme communications (i.e. newsletter) that they receive. Programme management appears aware of some, but not all, of these funder concerns and reportedly has improved their communication methods.


Tag: Challenges Communication Project and Programme management

34.

SHIFT in SAARC has built partnerships with BTCA and a2i around engagement on policy issues. A2i appreciates that SHIFT in SAARC brings technical expertise on financial inclusion as well as an international perspective. For its part, a2i provides SHIFT in SAARC with policy access as they have been active in Bangladesh since 2006 and currently sit in the ICT Ministry. The two organizations are working together on eKYC, DFS and financial literacy and have jointly hosted policy events, discussions and workshops. The engagement with a2i is relatively young but there are plans to engage more fully in the near future and begin to collaborate on advocacy for DFS interoperability.


Tag: Partnership Policy Advisory Technical Support

35.

Question 2.3: To what extent is the programme MRM generating sufficient evidence to inform accountability and contribute to programme improvements?
The MRM system has been a challenge and is not providing information that would contribute to programme improvement. The challenge is a result of weak demand from management for MRM evidence to support decision making, incomplete MRM data, and misaligned indicators. The MRM contains gender indicators and calls for gender-disaggregated data, but no gender-disaggregated data was available.


Tag: Challenges Gender Equality Gender Mainstreaming Monitoring and Evaluation Results-Based Management Data and Statistics

36.

The MRM challenge for SHIFT in SAARC is due also in part to personnel issues.46 The M&E and Data Analyst position was vacant for almost one year. External consultants have supported the programme by designing and assisting with the MRM, however, their efforts were not enough.47 At the time of the MTE inception report, the MRM system benefited from the hiring of a new M&E Officer. Bringing the system up to date has been difficult since the proper documentation is missing.48 At the time of the MTE field visit, the SharePoint site that the MRM will be housed in was still under construction.


Tag: Challenges Human and Financial resources Monitoring and Evaluation Data and Statistics

37.

The MRM system has indicators that measure programme outputs, programme outcomes, market outcomes, SHIFT in SAARC outcomes and impacts. At the higher levels of the results chain, this categorization of the indicators does not map directly to systemic changes (or changes in the inclusive finance system or the inclusive FMCG retail market systems). Changes to the financial and real economy markets are tracked instead by indicators such as “number of improved operational practices introduced” and “percent increase in the number of active service providers in the DFS eco-system.” Changes to the policy environment are tracked with indicators such as “number, types and specific features of new or improved regulations.” To date, many of these indicators have no associated data. For example, data for 2017 and 2018 is available only for one impact indicator, five market outcomes indicators and two programme outcomes indicators. This is indicative of the early stage of the programme. (See Annex 7: Country Report for more details.)


Tag: Challenges Monitoring and Evaluation Results-Based Management

38.

The programme reported that it captures market development changes, specifically new partnerships involving DFS. However, this information is not reported on the MRM, even though it is an indicator of systemic change. Instead, it is reported through a newsletter to stakeholders and through the Progress Narrative for 2017 to the BMGF.49 This is useful data that should be better captured by the MRM. The programme does not appear to use a framework such as Adopt, Adapt, Respond and Expand to assess what they are finding, and nor does it convert that information into management decisions.50


Tag: Challenges Monitoring and Evaluation Results-Based Management Data and Statistics

39.

While the MRM contains indicators for all levels of the TOC from outputs to impact, the programme has focused on data collection and reporting for output indicators. Little data is available for any outcome indicators and what is available comes primarily from satisfaction surveys from training participants.
In theory, MRM reports are generated bi-annually. In practice, no report was produced last July per the schedule. When asked whether and how MRM data informs management decisions, senior managers only offered vague responses. Management is more reliant on conversations and interactions with stakeholders to stay updated on changes that occur due to the programme. Nevertheless, the SHIFT in SAARC team reports that they have quarterly indicators that they use for planning purposes and that guide their planning at weekly meetings.51


Tag: Challenges Monitoring and Evaluation Project and Programme management Results-Based Management Data and Statistics

40.

Questions 2.3.1: Does the programme MRM have specific indicators and data to measure progress on human rights and gender equality? Are these being collected, analysed, utilized regularly?
Of the thirty-one SHIFT in SAARC indicators in the MRM, seven relate to women or gender. All gender indicators have zero (0) as a baseline data point. Many other indicators call for gender-disaggregated data, but no such data was found in the MRM. The programme also tracks gender-related results in a “gender tracker.” This spreadsheet tracks all SHIFT in SAARC publications, including event highlights, and documents whether the event/publication address gender issues and how it did so. This is a useful tracker with the results feeding into the MRM. However, the copy provided to the evaluation team was not up to date.


Tag: Challenges Gender Equality Monitoring and Evaluation Results-Based Management

41.

Question 2.4: How well are partner contributions/involvement in the programme working?
The partner contributions to the programme have been mixed. Dnet is now performing well and ahead of schedule, BDMS is catching up after a late start, while FBCCI is still underperforming. The organizational capacity assessment tool used by UNCDF over-estimated the capacity of these partners. The partners’ contributions are not likely to facilitate women’s financial inclusion or economic empowerment due to the selection of micro-merchants and the Fast-Moving Consumer Goods sector as the focal points for the MDDRM project.


Tag: Effectiveness Partnership

42.

SHIFT in SAARC has invested significant time and effort into building the capacity of the co-applicants, with mixed results. SHIFT in SAARC conducted a capacity assessment of each partner before contracting them, but this assessment over-estimated the true capacity of these partners. The programme provided training on M&E and project and materials design, and co-applicants received a master training on DFS and participated in many of the SHIFT in SAARC conferences and meetings. The programme also provided ongoing coaching to support quality implementation of activities. This support has been most effective in preparing Dnet to deliver TOTs and trainings to micro-merchants and Kallyani.


Tag: Monitoring and Evaluation Capacity Building

43.

Dnet has performed better than initial expectations. After a slight delay, Dnet officially began implementation in May 2017. The organization has met targets, milestones and timelines, and reported progress to SHIFT in SAARC in a timely fashion. Dnet reported that it caught up with the schedule by assigning sufficient staff and conducting several activities simultaneously. At the time of the evaluation, they were running ahead of schedule.


Tag: Effectiveness

44.

BDMS was significantly delayed in starting implementation due to an internal dispute that prevented it from signing the grant agreement with SHIFT in SAARC until July 2018. As a result, BDMS only started to deliver activities in the three to four months prior to the MTE. BDMS is reported to have good organizing skills and has been making efforts to catch up. Nevertheless, SHIFT has had to invest unexpected time and effort into building the internal capacity of BDMS.


Tag: Effectiveness

45.

FBCCI has the lowest capacity of the three partners in terms of understanding DFS or digital transactions related to the FMCG value chain. In 2018, FBCCI had 10 activities planned and delivered only two. The shortcoming was said to be due to their pre-occupation with election-related activities that year. At the time of the evaluation, SHIFT in SAARC had not received any final reports from FBCCI, and the organization was still not performing to expectations despite the significant amount of time and effort that UNCDF put into building their capacity on DFS, programme implementation and M&E.


Tag: Effectiveness

46.

iSocial is a social enterprise that was part of Dnet at the time of programme design but was spun-off in 2017 to become an independent legal entity. iSocial implements the InfoLady program, which uses a franchise model, with Kallyani women. Within the UNCDF programme, Dnet recruits and trains women for the InfoLady programme, after which they operate under iSocial. iSocial does not receive any direct funding or support from SHIFT in SAARC, but they are critical to the ultimate success of the Kallyani programming, and they benefit indirectly from UNCDF support.52


Tag: Women's Empowerment Partnership Capacity Building

47.

SHIFT in SAARC had also planned to partner with two other national institutions: BIBM and InM, which are considered influential by Bangladesh Bank and DFS providers alike. While unable to bring these institutions on board as sub-grantees due to lack of Board approval, UNCDF conducted joint, limited activities with them:

  • • BIBM is a national training, research, consultancy and education institute focused on the banking and finance sectors and collectively owned by all the banks in Bangladesh. They have worked with SHIFT in SAARC to co-host a conference. Originally, SHIFT in SAARC planned that BIBM would develop and then deliver the DFS training module for central bankers.
  • • The InM (previously the Institute for Microfinance), is a non-profit organization engaged in research and training for the microfinance sector. SHIFT in SAARC commissioned InM to produce a study on Interoperability.

Tag: Partnership Civil Societies and NGOs Private Sector

48.

Question 2.4.1: Are partner contributions sufficient and aligned to facilitate women’s financial inclusion and economic empowerment?
The implementation of SHIFT in SAARC in Bangladesh has not reflected the emphasis placed on gender in the SHIFT in SAARC programme framework. Part of the reason for this is that the BMGF-funded project had no explicit gender alignment, as it is primarily concerned with policy advocacy. The design of the EU-funded project followed EU gender guidelines. However, the project is locked into working with a segment of entrepreneurs in which women are underrepresented.

UNCDF is addressing women’s financial inclusion and economic empowerment through their support for the InfoLady programme and, to a lesser extent, the micro-merchants. But this is insufficient for reaching large numbers of women entrepreneurs. Dnet will train 1700 to 1750 micro-merchants and 225 to 240 women entrepreneurs. BDMS will train 800 to 900 micro-merchants, and reach 140 to 170 micro-merchants through informative training sessions and reach 80 to 100 members of women’s business groups with information sharing events. Women thus comprise 11 percent of the entrepreneurs targeted to receive training.
The amount of resources allocated to achieve gender-related objectives is low compared to the amount of emphasis the Programme Framework Document placed on gender-related objectives. This allocation reflects funder priorities. To have a significant impact on women’s economic empowerment with current funding, SHIFT in SAARC would need to have worked in economic sectors where women are more prevalent. There is no evidence that they attempted to take this step.


Tag: Gender Equality Gender Parity Women's Empowerment Capacity Building Inclusive economic growth

49.

5.3 Criterion 3: Effectiveness
Extent to which SHIFT deliverables are contributing (or not) to capacity development of partner organizations.

The extent to which SHIFT in SAARC is on track to contribute to the capacity development of partner organizations depends on the partner. At the government level, although satisfaction with capacity development and evidence and data activities is high, SHIFT in SAARC’s capacity building efforts have produced mixed results to date. The programme has clearly had a positive influence on the eKYC draft guidelines and to a lesser extent the NFIS. But there is less evidence and considerable disagreement among stakeholders about the extent to which SHIFT in SAARC’s efforts have contributed to promoting enabling financial inclusion policy and regulations within the Bangladesh Bank.


Tag: Effectiveness Partnership Inclusive economic growth Policy Advisory

50.

At the micro-level, Dnet reports there is high demand for the micro-merchant business training and that the merchants were very engaged during the trainings. BDMS reported similar findings. Additionally, BDMS reports that news of the trainings is spreading even beyond the four project districts by means of the business associations, leading to increased demand.
In FGDs and interviews, micro-merchants, Kallyani entrepreneurs and business association members, all trainees expressed satisfaction with the business trainings. Micro-merchants reported that they never had an opportunity before to learn about innovative technologies that could help their businesses. The association members said they had never before considered the bottlenecks facing micro-merchants. All training participants that were interviewed agreed that the trainings were relevant, of good quality and very effective, and that they would recommend the training to others.

At the meso-level, SHIFT in SAARC’s TOTs for business association representatives resulted in the trainees changing their business practices. It is too early to assess the impact of the TOT on the micro-merchants who are the ultimate target.


Tag: Effectiveness Partnership Capacity Building Inclusive economic growth

51.

At the financial system level, the Data and Evidence work of SHIFT in SAARC shows signs that it has positively affected FSPs by improving the use of evidence and increasing information exchange among DFS stakeholders. Both private sector and government stakeholders cited the research on micro-merchants and the work convening the DFCG as influencing FSP behaviour. FSPs reported that SHIFT in SAARC brought the issue of DFS to the forefront of their attention. Yet, there is no evidence that SHIFT in SAARC has helped FSPs yet develop inclusive financial services business models. The programme only recently launched the innovation grants, which are intended to incentivize the development of financial services innovations for micro-merchants. Thus, it is too early to assess the extent of the grants’ demonstration effects on the financial services sector.


Tag: Effectiveness

52.

Programme stakeholders disagreed about the extent to which the programme has built awareness of the opportunities and constraints for gender equality or awareness of the women’s market among policymakers as well as other stakeholders. The evaluation team found little effect of the programme’s work in this area. Several stakeholders recommended that SHIFT in SAARC do more substantive work on gender equality.


Tag: Effectiveness Women's Empowerment

53.

Question 3.1: To what extent are programme deliverables helping financial services partner organizations to develop and scale viable financial inclusion business models for men and women?
There is no evidence yet that SHIFT in SAARC has helped FSPs to develop financial inclusion business models. However, the programme has improved the use of evidence and information exchange among DFS stakeholders. It recently launched the innovation grants, which will incentivize the development of financial services innovations for micro-merchants. These steps are precursors to the development of financial inclusion business models.

The SHIFT in SAARC programme works to increase the supply of financial products and business models by improving the private sector’s use of evidence for decision making, increasing information exchange among private sector actors, and providing innovation incentives to FSPs.
SHIFT in SAARC has improved the use of evidence to some extent through the MMLA, which influenced FSPs by demonstrating that there are 1.6 million retailers with 60% of their inventory in FMCG products. This data revealed “a great opportunity for a backward value chain.” One stakeholder reported that the numbers in the report made it easier for a Singaporean social enterprise to make the final decision to enter the Bangladesh market for value chain digitization. There was no evidence of other data and analysis products, such as the POWER Study or the financial diaries, having yet had any effect on FSPs.
SHIFT in SAARC improved information exchange among DFS stakeholders by promoting participation in coalitions, such as the DFCG. This prompted one provider to begin to explore opportunities in digital savings for their microfinance clients.


Tag: Effectiveness Women's Empowerment Business Model Private Sector Inclusive economic growth

54.

Question 3.2: To what extent are programme deliverables helping meso-level stakeholders (e.g. BDS providers, associations) to deliver services to stated beneficiaries?
Meso-level stakeholders such as business association leaders and BDS providers have not yet delivered services to the end-user beneficiaries, the micro-merchants. These stakeholders are on track to do so, however, as some of them have recently completed business training and have begun to adopt changed business practices themselves.

To date, the SHIFT in SAARC programme has trained the trainers of two co-applicant organizations, Dnet and BDMS, to provide training to meso-level stakeholders such as business association representatives and BDS providers. These groups were trained to both understand and provide support to enhance the growth of micro-merchant businesses. At the time of the MTE, 17 BDS providers and 119 business association members had been trained by either Dnet or BDMS. FBCCI has provided training programs for business association leaders and has organized a session on DFS, focused on SMEs and awareness building for leaders.
An FGD with business association representatives who participated in business training provided by Dnet found that participants had also made changes to their own businesses. These representatives now use the Dnet mobile app to keep a record of all their credit sales; they have a better understanding of business management and are more organized with inventory and cash management. They self-report that they are attracting more customers as a result. There was no evidence, however, of them using their training yet to mentor other micro-merchants, which was the expected outcome of the training. This is likely because the trainees had only been trained in December 2018. However, the high level of engagement of these participants suggests that there may be lasting improvements in the business management behaviours of the business association representatives (See Annex 7: Country Report for more details).


Tag: Effectiveness Service delivery Capacity Building

55.

Question 3.3: To what extent are SHIFT in SAARC’s policy and advocacy activities at the national level on track to contribute to changes in the capacity of policymakers/regulators to develop and enact financial inclusion policies/regulations affecting low-income people and specifically women?
SHIFT in SAARC has delivered several outputs which aim to build policymakers’ and regulators’ capacity to enact financial inclusion policies and regulations for low-income people. There is some evidence that data and evidence activities, such as the DFS Regulatory Assessment, and capacity development activities, such as Knowledge Sharing events, Global Knowledge Exchange Forums, and the DFS Training module, have increased the knowledge and abilities of policymakers. To date, the evidence shows that new capacity has had little effect on the development of new policies or regulations, with the notable exception of the eKYC guidelines and to a lesser extent NFIS.


Tag: Effectiveness Women's Empowerment Rule of law Country Government Capacity Building Inclusive economic growth Policy Advisory

56.

Stakeholders are not in agreement as to why SHIFT in SAARC has not had more influence on the development of enabling financial inclusion policies and regulations. Although stakeholders also had widely varying opinions about the extent to which SHIFT in SAARC has built build awareness of the constraints and opportunities for gender equality, the evaluation team found little effect of the programme’s work in this area.


Tag: Effectiveness Gender Equality Inclusive economic growth Policy Advisory

57.

Key informants reported that under the programme’s leadership, the DFCG became a “very effective” and "interesting platform to express the needs and limitations of all stakeholders." SHIFT in SAARC brought the government into the DFCG coalition for the first time and was credited with getting the government to be more open with the private sector. For example, as a result of participation in the DFCG, the government publicly stated that interoperability was important and announced a schedule for addressing the issue. This statement was a first, according to a SHIFT in SAARC key informant. The DFCG has effectively promoted conversations among DFS stakeholders; it has raised awareness of DFS among different actors; created a positive environment for advocacy among the public and private stakeholders; and given the private sector access to lobby the government.


Tag: Effectiveness Country Government Private Sector Advocacy

58.

SHIFT in SAARC undertakes other policy and advocacy activities including direct consultations with the central bank. These consultations include commenting on draft policies and regulations. For example, SHIFT in SAARC commented on the draft NFIS and persuaded the Bangladesh Bank to include DFS in that strategy. The evaluation team, however, could not obtain a stakeholder observation on the effect of these less visible activities.
SHIFT in SAARC has supported its policy and advocacy efforts with research to provide data and evidence to regulators and policymakers. This effort has effectively improved information exchange and dialogue among regulators/policymakers and FSPs by raising awareness of DFS issues, providing evidence for the market, and promoting a dialogue between regulators/policymakers and the private sector, particularly FSPs. Stakeholders recognized the importance of evidence that the programme produced. One stakeholder that the evaluation team interviewed stated, “They conducted a DFS Regulatory Framework Assessment. They have identified the gap areas, (between) what service providers are saying, what regulators think should be improved. That was one of the important improvements…for the development of DFS in our country.”


Tag: Rule of law Country Government Advocacy Policy Advisory Data and Statistics

59.

UNCDF has raised stakeholders’ awareness on many DFS related topics including: financial inclusion, merchant payment, eCommerce, blockchain, interoperability and eKYC. Through their activities, SHIFT in SAARC started a conversation on DFS among the policymakers/regulators and market actors in Bangladesh, making DFS a "new buzzword." However, there was no agreement on whether the programme raised awareness on gender constraints and/or the gender gap in DFS.


Tag: Effectiveness Gender Equality Women's Empowerment Innovation Inclusive economic growth

60.

SHIFT in SAARC has conducted activities to directly increase the capacity of regulators and policymakers to support the development of enabling policies that can address the needs of low-income people. They have raised central bankers’ awareness of emerging technologies, including interoperability, blockchain and regulatory sandboxes. The programme has also built the capacity of policymakers and regulators through knowledge sharing events and global knowledge exchange forums. These types of activities were effective in influencing the eKYC policy development process and promoting regulatory issues such as regulatory sandboxes. One government stakeholder who had convinced executives of blockchain’s importance commented, “Before [SHIFT in SAARC], within Bangladesh Bank very, very few people were aware of the emerging technologies, like blockchain or IoT [internet of things. [Now] there is rising awareness among the mid-level and even the senior level staff. For example, [before] I could not convince senior management about blockchain. Two weeks after having training by UNCDF, our Finance Minister is now interested in application of blockchain.”


Tag: Innovation Country Government Capacity Building Inclusive economic growth Technology Trade and Development

61.

In the long term, SHIFT in SAARC’s DFS training module may have the most impact on policymakers’ and regulators’ capacity. This training module has the potential to increase DFS knowledge and capacity of the entire Bangladesh Bank staff over time. The programme provided a TOT for BBTA and Bangladesh Bank staff with the purpose of preparing BBTA to be able to provide DFS training as an ongoing activity. The programme will use the module to train the annual intake of Bangladesh Bank recruits each year while delivering parts of the module as standalone trainings


Tag: Effectiveness Country Government Private Sector Capacity Building

62.

SHIFT in SAARC has worked with a2i in promoting an enabling eKYC policy in Bangladesh. SHIFT in SAARC’s direct contribution includes producing the DFS regulatory assessment and a policy synthesis paper, providing support for policy dialogues, South-South exchange visits, and coalition building, and supporting regulators and policymakers to undertake best-practice consultative processes to develop holistic and inclusive regulatory framework and policy for low-income consumers.53 Two years after SHIFT in SAARC started operations, there is now a new eKYC national-level committee, draft guidelines (tailored to the local context), a draft circular and plans to begin an eKYC pilot. However, the evaluation team could not conclude that the SHIFT in SAARC programme was the main contributor to advancing the eKYC regulations, as a2i has taken the lead and has been a major influence. As described in Annex 9: The eKYC Case Study, the real impact of SHIFT in SAARC has most likely been to speed up the process.
Aside from the eKYC policy, there is considerable among stakeholders about the influence of SHIFT in SAARC on the DFS enabling environment in Bangladesh. One private sector stakeholder expressed the belief that the provision of information and evidence will impact policymakers: “I’ve seen people from different ministries taking part in [DFCG]. You are giving them information. We expect that this information will contribute.” However, others in the private sector disagree with this assessment.


Tag: Effectiveness Rule of law Country Government Private Sector Inclusive economic growth Advocacy Policy Advisory

63.

The results of SHIFT in SAARC to date call into question the programme’s TOC around influencing policy. Both funders and UNCDF believed that raising policymakers’ and regulators’ awareness and knowledge about DFS would change behaviour and lead to more inclusive DFS policies. However, it appears that the mechanisms of raising awareness and sharing knowledge, even when coupled with coalition building and advocacy, are insufficient in Bangladesh to change policy. For example, a KII respondent said, “There were some areas where we thought that greater awareness among policymakers, regulators, and FSPs on key issues around digital financial services could help unlock regulatory barriers…[What was observed was that] the report was done, the dissemination workshop was done but I have not seen evidence of influence or attitude change as a result of that work.” Government representatives (along with one or two private sector respondents) emphasized that it takes time in Bangladesh to influence policy and regulations. This suggests that it could well be too early to assess whether SHIFT in SAARC can achieve other policy changes.


Tag: Effectiveness Rule of law Programme/Project Design Theory of Change Donor Policy Advisory

64.

Only a few government stakeholders addressed SHIFT in SAARC’s influence on changing policy and they emphasized that it takes considerable time in Bangladesh to move policy. Government stakeholders were more likely to attribute raised awareness as the outcome of SHIFT in SAARC’s efforts. No private sector stakeholders identified any changes to policy or regulations due to the efforts of SHIFT in SAARC and about half stated there had been no effect. These stakeholders provided various explanations for why SHIFT in SAARC’s capacity building and advocacy efforts varied have not had more traction, including the following:


Tag: Effectiveness Country Government Capacity Building Advocacy Policy Advisory

65.

(1) SHIFT is perceived by some in the private sector as not strong enough. Its attempts to influence priority policies are constrained by lack of capacity and/or bandwidth. This was the reason given for missing the opportunity to lead the NFIS, although SHIFT did provide a supporting activities.


Tag: Effectiveness Private Sector

66.

(2) DFCG is not sufficiently action-oriented. UNCDF focused on building the DFCG when they realized that a broader-based coalition could have more influence with policymakers. Several stakeholders, even strong supporters of the DFCG, though, point out that it is primarily a networking and information platform rather than a decision-making body. Observers suggest that it needs to be more action-oriented. Respondents believe that this would require the participation of senior decision-makers from key organizations, who are not currently participating in DFCG events. As one DFCG participant noted, “And the engagement in DFCG – you’ll see that the top-notch decision-makers from the institutions are not participating. You need to engage the “high-ups” to influence policy.”


Tag: Effectiveness Rule of law Country Government Advocacy Policy Advisory

67.

(3) Power to change regulation is housed elsewhere. There is disagreement among stakeholders about where the power to change regulation lies. Programme documents called for UNCDF to “directly collaborate [within the central bank] with the Financial Inclusion Department (FID) and the Payment Systems Department (PSD).” Some believe the institutions that SHIFT has targeted with advocacy do not have power to effect DFS regulations. One private sector respondent commented, “It is not the central bank that takes decisions, it is the PM (prime minister), ICT advisor and certain groups that take decision. If you want to control this it needs to come from the top.” Additionally, the centre of power to effect policies and regulations varies depending on the topic. For example, another private sector stakeholder commented, “The next goal is fintech – the highest trade body [of e-commerce] is Bangladesh Association of Software and Info Services (BASIS). The current ICT minister used to be the BASIS president. So now [the ICT] Minister has the agenda of bringing this Digital Bangladesh to a better shape…BASIS [is] very influential.”


Tag: Challenges Effectiveness Advocacy

68.

Underachievement may also have resulted from not focusing on advocacy targets specified in the programme’s design. Though the programme design targeted FID and PSD, the GoB assigned SHIFT in SAARC to work with the Sustainable Finance Department (SFD) which is responsible for green finance, and FID, to a lesser extent. Consequently, SHIFT in SAARC’s relationship with the PSD is complicated and has likely resulted in missed opportunities. A central banker commented that UNCDF’s work is redundant to PSD’s work and that it would be better if the work of both groups was synchronized.


Tag: Effectiveness Programme/Project Design Advocacy

69.

Question 3.3.1: To what extent is the programme building awareness of constraints and opportunities for gender equality?
Stakeholders had widely varying perspectives on this, ranging from positive to negative on building awareness of gender constraints. Generally, the evaluation team observed little evidence of effects from SHIFT’s work in this area. The SHIFT team has not yet taken on board the POWER study recommendations in any activities. There was little evidence that stakeholders are aware of SHIFT’s publications and events on the topic of gender. Several stakeholders, from both private sector and government identified gender as an area where SHIFT needs to do more work in the future. Additionally, it was observed that it would be better to have gender equality permeate all of SHIFT’s programming rather than treat it as a peripheral activity. The one innovation grant targeted to women is the last innovation grant issued. The preceding five innovation grants do not have targets for women beneficiaries nor even requirements for sex-disaggregated data when reporting results.


Tag: Effectiveness Gender Equality Gender Mainstreaming Women's Empowerment Data and Statistics

70.

Question 3.3.2: To what extent is the programme building awareness and understanding of the women’s market?
To date, SHIFT has not effectively built awareness or understanding of the women’s market. The programme suffered a setback in this area when the MMLA revealed that women comprised only a small share of all micro-merchants in the country. SHIFT adapted the programme to include women merchants by supporting an expansion of the existing InfoLady or Kallyani programme. While this was a step in the right direction, this programme is much smaller than the micro-merchant programme, thus limiting its potential impact.


Tag: Effectiveness Women's Empowerment

71.

5.4 Criterion 4: Likely Impact
Likely programme impact at both beneficiary and market/policy system levels, supply side, and meso level.

SHIFT in SAARC has had a limited but positive impact on improving eKYC guidelines, and ultimately this will have an influence on the forthcoming eKYC policy. This improvement could positively impact the inclusive financial services system. The eKYC policy has the potential to accelerate the on-boarding of millions of RMG workers to digital payments.


Tag: Impact Innovation Inclusive economic growth Technology Trade and Development

72.

SHIFT in SAARC has undertaken several activities with the aim of developing the FMCG value chain to promote the use of DFS. The programme has developed the capacity of implementing partners, meso-level actors, and end-users, developed and disseminated evidence and data on micro-merchants as a viable market for financial services, and designed and solicited six innovation grants to promote new relationships and the use of DFS in this value chain. The evidence to date suggests only that the capacity building of micro-merchants and Kallyani women has led to changes in their business management practices, which they report has led to higher demand for goods, higher sales and better customer loyalty. Micro-merchants also believe that aggregate demand for retail will increase in markets where several micro-merchants have received business training.

SHIFT in SAARC’s impact on gender equality will likely be modest due to low outreach to women. The few hundred Kallyani women reached by the programme can potentially benefit through increased incomes, empowerment, self-confidence and agency. The programme may also have indirect effects on gender equality at the level of female consumers as a result of the products and services provided by the InfoLady programme.


Tag: Gender Equality Women's Empowerment Service delivery Capacity Building Trade and Development

73.

Question 4.1: (Supply-side) On the basis of FSP products launched to date, which are most likely to deliver impact at the level of programme beneficiaries, including particularly low-income women, and why?
To date, no FSP products have been launched as a result of SHIFT in SAARC. The programme plans to pilot various financial services with male and female micro-merchants through the innovation grants that the programme recently awarded. Interviews with micro-merchants in the field suggests that the cost of digital financial products will need to come down to attract large numbers of merchants and their customers.

At the time of the MTE, the programme had not contributed to the launch of any new FSP products. The Innovation Grants, which will pilot a variety of financial services with male and female micro-merchants were just launched at the time of the mid-term evaluation, therefore no products had been developed yet.


Tag: Impact Gender Equality Women's Empowerment Inclusive economic growth Technology

74.

Question 4.2: (Regulatory) On the basis of programme performance to date, what is the likely impact of the changes in policies and regulations at the level of financial markets and systems in Bangladesh?
As discussed, SHIFT in SAARC has contributed to the forthcoming eKYC policy, which has the potential to accelerate the on-boarding of millions of RMG workers to digital payments.

SHIFT’s contribution is significant as the eKYC policy when finalized, should make on-boarding new DFS customers, such as RMG workers, faster and easier. For example, driven by the BTCA, there are programmes in Bangladesh to mandate switching payments for 4.5 million RMG workers, approximately 70 percent of whom are women, from cash to digital payment. Currently, the process can take up to three to four months per factory to sign on all workers. For example, Sarathi signed 10,000 RMG workers to accounts last year. With eKYC, the process could take just minutes for one worker to be on-boarded rather than hours or days. As more RMG workers get paid digitally, they will be remitting money at less cost and more securely to family members in rural areas. The indirect impacts of SHIFT in SAARC’s work to advance eKYC guidelines could be very widespread if family members on the receiving end of remittances become encouraged to take up DFS.


Tag: Impact Advocacy Policy Advisory

75.

Question 4.2a What are the possible long-term effects on gender equality? Are the gender-related outcomes likely to be sustainable?
The only positive long-term effects on gender equality that are likely to occur will result from SHIFT in SAARC’s support for Kallyani women in the InfoLady programme.
The InfoLady programme demonstrates positive non-financial effects on women entrepreneurs. In an FGD, Kallyani women reported that they feel more socially connected and self-confident, and they believe they are empowered by this work. They are proud that they can help others, particularly other women.. Whether the women are earning very much from their work was unclear to the evaluation team, however, it is important to note that these women have worked in the InfoLady programme for less than one year.
The programme is likely having positive indirect impact on the women served by the Kallyani entrepreneurs. The impact of SHIFT in SAARC on women’s use of DFS is unclear, whereas the impact of business training on the women is likely to be sustainable. The only risk to the impact of the training is that it will depend on the InfoLady business model remaining viable for them.


Tag: Impact Gender Equality Inclusive economic growth

76.

Question 4.3 (Meso) [new] On the basis of programme performance to date, what is the likely impact of the changes in capacity and in relationships between market actors, at the level of the real market economy?
As discussed elsewhere in this report, SHIFT in SAARC’s work developing the FMCG value chain has developed the capacity of implementing partners, meso-level actors and end-users or micro-merchants. The evidence to date shows only that the capacity building of micro-merchants and Kallyani women led to changes in business management practices.

The focus of SHIFT in SAARC so far is on building the capacity of the implementing partners, collecting the foundational evidence to inform the project design, and developing the capacity of the micro-merchants, BDS providers and business associations. The innovation grants are intended to promote new relationships by encouraging collaborative partnerships between micro-merchants, distributors, wholesalers and regional FMCG companies, along with FSPs and technology companies, including Fintechs.


Tag: Impact Small Grants Programme Innovation Service delivery Private Sector Capacity Building

77.

The Data and Evidence work has raised the awareness of FSPs of the micro-merchant sector as a viable market for financial services. The DFS-related workshops and events have raised DFS awareness among companies high up the FMCG value chain. However, the evaluation team could not meet with a representative from a major supplier in the FMCG value chain due to scheduling conflicts, thus this perspective is missing from the evaluation. As a result, it is perhaps not surprising that the team found no evidence of changes in relationships between FMCG value chain market actors (i.e. micro-merchants, distributors, wholesalers, manufacturers and others) at this stage of the programme. It is also somewhat early to assess the impact of changes in micro-merchants’ or Kallyani women’s capacity. However, there are signs of positive impact of business training and of using the app.


Tag: Impact Innovation Capacity Building Technology Trade and Development Data and Statistics

78.

Micro-merchants reported several positive effects of the training on their business activities, including a higher demand for products, higher sales, and more customers. Furthermore, merchants reported that there should be a positive aggregate demand effect when several merchants within their bazaar (i.e. marketplace) receive the same training. Key informants, however, believe that further business training is needed to ensure sustainable behaviour change among the micro-merchants. (See Annex 7 for more information.) Users of the mobile app for micro-merchants also reported many benefits, such as better judgement about extending credit, less forgetfulness about debtors, and quicker collections. These benefits should improve cash flow. (See Annex 7)


Tag: Impact Capacity Building

79.

Dnet is working to mainstream the mobile app to FMCG providers, which could have a significant impact on the FMCG value chain throughout Bangladesh. At the same time, the innovation pilots (in combination with the app) could impact micro-merchants by saving them time and money, increasing the speed of stocking their shops, reducing debts, and increasing cash flow and improving sales, although it is too early to say (See Annex 7 for more information).


Tag: Innovation Inclusive economic growth Technology

80.

5.5 Criterion 5: Sustainability
Sustainability of programme results within the broader policy and market environments.

Sustainability relates to the ability of the financial services and real economy market systems to provide low-income consumers access to financial services and increased incomes beyond the end of the SHIFT in SAARC programme. The ability of the government to continue improving the enabling environment for inclusive DFS is an important underpinning of financial inclusion. The DFS training provided to BBTA shows great promise for sustainably building the capacity of central bankers in DFS and new financial technologies. The programme’s success in getting DFS included in the NFIS and its influence in advancing eKYC should have sustainable effects. The DFCG, which serves as a platform for public-private dialogue on inclusive DFS, could be sustainable if another organization takes over leadership of it. There is little evidence that SHIFT in SAARC has effectively changed the capacity of FSPs to reach the low-income market. There is also no evidence yet of sustained use of DFS by micro-merchants, and it is too early to assess whether the innovation grants will result in sustainable DFS use by consumers. The sustainability of impacts on beneficiaries’ incomes will depend on the lasting effects of business training for micro-merchants and Kallyani women. For the women, sustainability also depends on the continued viability of the InfoLady business model in the four districts. The most likely sustainable effects on gender equality will result from the Kallyani programme, which appears to be having positive impacts through effects on women’s confidence, self-esteem and agency.


Tag: Sustainability Small Grants Programme Innovation Partnership Sustainability Capacity Building Inclusive economic growth

81.

The sustainability of results in market systems development programs is often examined with reference to the Adopt, Adapt, Expand and Respond framework. In the Adopt stage, a market actor has 1) successfully adopted a behaviour or practice change to the ultimate benefit of the low-income consumer, women or small business, 2) recognized the value of continuing these changes irrespective of inputs from a programme like SHIFT in SAARC, and 3) made a plan to invest in keeping these changes. In the Adapt stage, the market actor has made investments that allow them to continue with the changed practice without programme support.54 The SHIFT in SAARC programme is primarily in the Adopt stage across its activities. In terms of the UNCDF Maturity Model, the programme is in the Innovation stage: Innovate and test new financing/business models, learn and create conditions for consolidation.


Tag: Sustainability Innovation Sustainability

82.

Question 5.1: To what extent are changes in the capacity of FSPs to reach low-income men and women likely to continue over time?
There is little evidence yet that SHIFT in SAARC has effectively changed the capacity of FSPs to reach the low-income market, and it is too early to say what changes at the FSP level might continue over time.

Before SHIFT in SAARC, FSPs had less awareness of DFS and little incentive to extend services to low-income men and women, particularly those considered “last mile.” The SHIFT Programme has built FSPs capacity in DFS and started to provide innovation grants to broaden the DFS market to reach low-income men and women. However, the programme’s short timeframe threatens to halt activities just as they are building momentum. The limited remaining time also means there will be little time for the market to learn from and react to the results of the innovation grants. The time remaining may not allow for FSPs who are participating in the innovation grants to continue operating any successful new business models without support from SHIFT in SAARC. Further, as a few private sector stakeholders pointed out, the financial services market needs to see business cases to convince them to start innovations. Innovation labs or challenges funds could support this effort. Business cases would promote sustainability among the FSPs.


Tag: Sustainability Small Grants Programme Innovation Sustainability

83.

Question 5.2: To what extent are changes in GoB priorities and performance introduced as a result of SHIFT – including in the way GoB implements gender-sensitive financial inclusion policies - likely to be sustainable over time?
One of the long-lasting impacts of SHIFT in SAARC is likely the DFS training, which the programme has handed over to BBTA so that the academy may train all newly-recruited Bangladesh Bank officials. There is potential for the central bank to continuously upgrade their knowledge of emerging issues in DFS.

The DFCG has effectively convened the public and private sectors in Bangladesh around emerging issues in DFS. For the DFCG to continue, another institution will have to take ownership of it. Even if UNCDF were to extend the SHIFT in SAARC programme, one stakeholder reminded the evaluation team that: “the DFCG should be handed over to someone else. Because a development partner’s responsibility is [only] to kick-off such initiatives.” During the next several months, a succession plan for the DFCG needs to be concluded to ensure a smooth transition to a new secretariat.
There has been no change to date in the way the GoB implements gender-sensitive financial inclusion policies.


Tag: Sustainability Gender Equality Women's Empowerment Rule of law Ownership Sustainability Country Government Private Sector Inclusive economic growth

84.

Question 5.2.1: To what extent has the programme contributed to the sustainable access and use of DFS by the target end-users, especially women?
Interviews with micro-merchants and the Kallyani women suggest that entrepreneurs’ use of MFS is related to cost and convenience, while customers’ use of MFS depends on awareness. The sustainable use of MFS by merchants will depend on the affordability of the channel. Dnet also reported that the micro-merchants are not ready to switch to digital finance yet although they are “pushing them step by step.” Until demand from customers increases significantly, there does not appear to be a strong rationale for merchants to deepen their use of MFS. The cost of DFS would need to come down for demand to increase. Interviews with merchants found that about two-thirds have used MFS services such as bKash since before SHIFT in SAARC started.
The DNET mobile app could be sustainable after the programme ends. The FMCG companies could promote or support the app in the future. For example, there is interest from Unilever in this, as well as in the toolkit that Dnet developed.


Tag: Sustainability Gender Equality Women's Empowerment Inclusive economic growth Poverty Reduction Trade and Development

85.

Question 5.2.2: What are the possible long-term effects on gender equality? Are the gender-related outcomes likely to be sustainable?
The most likely sustainable effects on gender equality will result from the Kallyani programme, which appears to have positive impacts through effects on women’s confidence, self-esteem and agency. The sustainability of impacts on women’s income depends on the sustainability of the InfoLady business model. (See Annex 7 for more information.)


Tag: Sustainability Gender Equality Women's Empowerment Sustainability Inclusive economic growth Poverty Alleviation

86.

6. FINDINGS, CONCLUSIONS AND RECOMMENDATIONS
6.1 Findings
6.1.1 Relevance and quality of design
The appropriateness of the programme’s objectives to the real problems, needs and priorities of its target groups/beneficiaries and the quality of programme design through which these objectives are to be reached.

The SHIFT in SAARC programme design is relevant to the Bangladeshi context due to its effective stakeholder consultation during design and adaptive approach to implementation. The programme design directly addresses the priorities of the Government of Bangladesh for digital financial inclusion and gender equality. It also contributes to UNDAF and several SDGs.


Tag: Relevance Programme/Project Design Country Government UN Agencies Agenda 2030

87.

The programme’s TOC is logical. It outlines four pillars of activities that address identified market system weaknesses that prevent financial inclusion. However, there is incomplete evidence that the logic will work to achieve objectives. Based on similar programmes’ experience in the context of Bangladesh, it is not clear that SHIFT in SAARC will achieve its programme objectives with the existing funding and the associated timeframe.
The programme design is aligned with internationally recognized best practices for market systems development for financial inclusion, despite a few notable exceptions. The timeframe and secured funding are insufficient for this approach. The overall objective - ‘to increase the percentage of adults using DFS’ - for the Bangladesh programme is not measurable.


Tag: Relevance Programme/Project Design Theory of Change

88.

SHIFT in SAARC has considered the needs and interests of demand-side customers and the target merchants by undertaking field-level research with these target groups. The programme as designed is consistent between the programme objectives and partners’ needs and strategy. The design relies primarily on innovation grants and capacity building to ensure that the approach can be transitioned, expanded and/or replicated by others.
The programme design aligns with national and funder-level gender standards. It implicitly aligns with United Nations standards on mainstreaming gender. Yet, the programme has also done less well in considering women’s financial inclusion and employment needs. By incorporating a robust gender analysis in the design phase, UNCDF could have developed stronger gender programming.


Tag: Relevance Gender Equality Programme/Project Design Private Sector

89.

6.1.2 Efficiency
The extent to which the programme has delivered quality outputs that are appropriately managed and overseen.

UNCDF management and operations have delivered several activities and outputs over the past two years. They have developed and disseminated important and impactful research and evidence on key themes; launched six innovation grants; built the capacity of policymakers and regulators; built a coalition of public and private stakeholders in the DFS ecosystem; and built the capacity of implementing partners and low-income entrepreneurs.
The evaluation’s interviews and focus groups revealed that programme beneficiaries strongly appreciated the trainings provided. The PSC members appreciate the programme and its leaders as well as SHIFT in SAARC’s taking up the micro-merchant segment, and linking the real economy sector, (i.e. the FMCG industry) to FSPs. Funders are concerned with the communications from management. Programme management is reported to have improved communications with funders.


Tag: Efficiency Small Grants Programme Communication Innovation Programme/Project Design Project and Programme management Donor

90.

SHIFT in SAARC managed their activities well in terms of keeping within the available budget. However, the programme may be challenged in spending down the committed funds balance within the remaining time period. Although the programme used a “staff-up” approach whereby staff were hired as funding was secured, the level and mix of in-country staff were insufficient for work required by the two projects.


Tag: Efficiency Human and Financial resources Operational Efficiency

91.

The programme has faced other challenges including falling short on the timing of output delivery and achievement of results, due to factors both within and outside of UNCDF’s control. The programme’s MRM system is another area of weakness, which is due at least in part to a long-term staff vacancy. The MRM system does not generate enough evidence to inform accountability and contribute to programme improvements, but it could make a more valuable contribution to programme management with certain modifications. However, SHIFT in SAARC has largely overcome the challenges that caused delays, and the programme was operating at a faster pace at the time of the evaluation.


Tag: Challenges Monitoring and Evaluation Results-Based Management

92.

6.1.3 Effectiveness
Extent to which SHIFT deliverables are contributing (or not) to capacity development of partner organizations.

The extent to which SHIFT in SAARC is on track to contribute to the capacity development of partner organizations depends on the partner. At the government level, although satisfaction with capacity development and evidence and data activities is high, SHIFT in SAARC’s capacity building efforts have produced mixed results to date. The programme has clearly had a positive influence on the eKYC draft guidelines and to a lesser extent the NFIS. But there is less evidence and considerable disagreement among stakeholders about the extent to which SHIFT in SAARC’s efforts have contributed to promoting enabling financial inclusion policy and regulations within the Bangladesh Bank.


Tag: Effectiveness Capacity Building Inclusive economic growth

93.

At the micro-level, Dnet reports there is high demand for the micro-merchant business training and that the merchants were very engaged during the trainings. BDMS reported similar findings. Additionally, BDMS reports that news of the trainings is spreading even beyond the four project districts by means of the business associations, leading to increased demand.
In FGDs and interviews, micro-merchants, Kallyani entrepreneurs and business association members, all trainees expressed satisfaction with the business trainings. Micro-merchants reported that they never had an opportunity before to learn about innovative technologies that could help their businesses. The association members said they had never before considered the bottlenecks facing micro-merchants. All training participants that were interviewed agreed that the trainings were relevant, of good quality and very effective, and that they would recommend the training to others.


94.

At the meso-level, SHIFT in SAARC’s TOTs for business association representatives resulted in the trainees changing their business practices. It is too early to assess the impact of the TOT on the micro-merchants who are the ultimate target.
At the financial system level, the Data and Evidence work of SHIFT in SAARC shows signs that it has positively affected FSPs by improving the use of evidence and increasing information exchange among DFS stakeholders. Both private sector and government stakeholders cited the research on micro-merchants and the work convening the DFCG as influencing FSP behaviour. FSPs reported that SHIFT in SAARC brought the issue of DFS to the forefront of their attention. Yet, there is no evidence that SHIFT in SAARC has helped FSPs yet develop inclusive financial services business models. The programme only recently launched the innovation grants, which are intended to incentivize the development of financial services innovations for micro-merchants. Thus, it is too early to assess the extent of the grants’ demonstration effects on the financial services sector.


Tag: Effectiveness Capacity Building

95.

Programme stakeholders disagreed about the extent to which the programme has built awareness of the opportunities and constraints for gender equality or awareness of the women’s market among policymakers as well as other stakeholders. The evaluation team found little effect of the programme’s work in this area. Several stakeholders recommended that SHIFT in SAARC do more substantive work on gender equality.


Tag: Effectiveness Gender Equality Women's Empowerment

96.

6.1.4 Likely Impact
Likely programme impact at both beneficiary and market/policy system levels, supply side, and meso level.

SHIFT in SAARC has had a limited but positive impact on improving eKYC guidelines, and ultimately this will have an influence on the forthcoming eKYC policy. This improvement could positively impact the inclusive financial services system. The eKYC policy has the potential to accelerate the on-boarding of millions of RMG workers to digital payments.

SHIFT in SAARC has undertaken several activities with the aim of developing the FMCG value chain to promote the use of DFS. The programme has developed the capacity of implementing partners, meso-level actors, and end-users, developed and disseminated evidence and data on micro-merchants as a viable market for financial services, and designed and solicited six innovation grants to promote new relationships and the use of DFS in this value chain. The evidence to date suggests only that the capacity building of micro-merchants and Kallyani women has led to changes in their business management practices, which they report has led to higher demand for goods, higher sales and better customer loyalty. Micro-merchants also believe that aggregate demand for retail will increase in markets where several micro-merchants have received business training.


Tag: Impact Capacity Building

97.

SHIFT in SAARC’s impact on gender equality will likely be modest due to low outreach to women. The few hundred Kallyani women reached by the programme can potentially benefit through increased incomes, empowerment, self-confidence and agency. The programme may also have indirect effects on gender equality at the level of female consumers as a result of the products and services provided by the InfoLady programme.


Tag: Impact Gender Equality Gender Parity Women's Empowerment

98.

6.1.5 Sustainability
Sustainability of programme results within the broader policy and market environments.

Sustainability relates to the ability of the financial services and real economy market systems to provide low-income consumers access to financial services and increased incomes beyond the end of the SHIFT in SAARC programme. The ability of the government to continue improving the enabling environment for inclusive DFS is an important underpinning of financial inclusion. The DFS training provided to BBTA shows great promise for sustainably building the capacity of central bankers in DFS and new financial technologies. The programme’s success in getting DFS included in the NFIS and its influence in advancing eKYC should have sustainable effects. The DFCG, which serves as a platform for public-private dialogue on inclusive DFS, could be sustainable if another organization takes over leadership of it. There is little evidence that SHIFT in SAARC has effectively changed the capacity of FSPs to reach the low-income market. There is also no evidence yet of sustained use of DFS by micro-merchants, and it is too early to assess whether the innovation grants will result in sustainable DFS use by consumers. The sustainability of impacts on beneficiaries’ incomes will depend on the lasting effects of business training for micro-merchants and Kallyani women. For the women, sustainability also depends on the continued viability of the InfoLady business model in the four districts. The most likely sustainable effects on gender equality will result from the Kallyani programme, which appears to be having positive impacts through effects on women’s confidence, self-esteem and agency.
The sustainability of results in market systems development programs is often examined with reference to the Adopt, Adapt, Expand and Respond framework. In the Adopt stage, a market actor has 1) successfully adopted a behaviour or practice change to the ultimate benefit of the low-income consumer, women or small business, 2) recognized the value of continuing these changes irrespective of inputs from a programme like SHIFT in SAARC, and 3) made a plan to invest in keeping these changes. In the Adapt stage, the market actor has made investments that allow them to continue with the changed practice without programme support.55 The SHIFT in SAARC programme is primarily in the Adopt stage across its activities. In terms of the UNCDF Maturity Model, the programme is in the Innovation stage: Innovate and test new financing/business models, learn and create conditions for consolidation.


Tag: Sustainability Rule of law Ownership Sustainability Inclusive economic growth Poverty Alleviation Trade and Development

99.

SHIFT in SAARC has considered the needs and interests of demand-side customers and the target merchants by undertaking field-level research with these target groups. The programme as designed is consistent between the programme objectives and partners’ needs and strategy. The design relies primarily on innovation grants and capacity building to ensure that the approach can be transitioned, expanded and/or replicated by others.
 


Tag: Relevance Programme/Project Design Capacity Building

100.

The programme design aligns with national and funder-level gender standards. It implicitly aligns with United Nations standards on mainstreaming gender. Yet, the programme has also done less well in considering women’s financial inclusion and employment needs. By incorporating a robust gender analysis in the design phase, UNCDF could have developed stronger gender programming.


Tag: Relevance Gender Equality Gender Mainstreaming Women's Empowerment Programme/Project Design Inclusive economic growth

101.

The programme’s TOC is logical. It outlines four pillars of activities that address identified market system weaknesses that prevent financial inclusion. However, there is incomplete evidence that the logic will work to achieve objectives. Based on similar programmes’ experience in the context of Bangladesh, it is not clear that SHIFT in SAARC will achieve its programme objectives with the existing funding and the associated timeframe.


Tag: Effectiveness Relevance Programme/Project Design Theory of Change

102.

The programme team used a consultative process during programme design in 2015 and early 2016, and an adaptive approach during programme implementation to ensure the relevance of the programme approach. UNCDF consulted closely with financial inclusion regulators and policymakers at the Bangladesh Bank and other government agencies during programme design and implementation. The programme has also adapted in response to government requests by addressing specific topics of interest to the government, e.g. Blockchain.


Tag: Relevance Implementation Modality Innovation Programme/Project Design Project and Programme management Country Government Policy Advisory

103.

The programme design team used two foundational studies, the DFS Regulatory Self-Assessment (DFS Assessment) and the MMLA to ensure that the programme was relevant to target beneficiaries. The studies described how to tailor the capacity building activities to the needs of the policymakers, regulators and merchants.


Tag: Relevance Programme/Project Design Capacity Building Data and Statistics

104.

The SHIFT in SAARC TOC articulates logical steps from programme activities to expected outcomes.33 Among other assertions, the TOC says that information and data will incentivize the private sector to invest in financial product innovations and business models for low-income markets. It also asserts that capacity building combined with information will cause policymakers and regulators to develop more enabling policies for DFS. It further asserts that capacity building of micro-merchants and women entrepreneurs will create demand for DFS and other innovations in the FMCG retail market.


Tag: Business Model Innovation Theory of Change Private Sector Capacity Building Inclusive economic growth Data and Statistics

105.

Despite the logic of the TOC, the descriptions of the programme provided by SHIFT in SAARC can be overly complicated and confusing. One funder reported that the description of the project objective is “confusing” and “not good.” Another funder stated, “I have never seen a coherent package of products in SHIFT.” The broad and flexible design of SHIFT in SAARC, which in some respects is a strength, allowed the implementation of both the BMGF-funded and the EU-funded projects simultaneously. This has forced the programme to over-emphasize links between the two market systems with an assumption that there would be synergies in implementation and outcomes, which have not been realized. Due to different approaches and objectives reflecting different donor priorities, this combination of projects may have led to confusion about programme purpose and methods and over-extended the SHIFT in SAARC team’s attention, rather than achieving greater impacts.
 


Tag: Challenges Relevance Implementation Modality Programme/Project Design

106.

MSD programs require a long-term perspective because the path to development outcomes is non-linear, unpredictable and takes time. Additionally, the approach to accountability in MSD programmes requires enhanced monitoring, effective risk management and evidence-based adaptation, as well as a conducive environment for engaging in experimentation.


Tag: Rule of law Monitoring and Evaluation Programme/Project Design Risk Management

107.

By the time of the SHIFT ASEAN mid-term evaluation in 2018, the programme had increased market actors’ awareness through capacity development and/or policy and advocacy activities. There was direct outreach to client level beneficiaries resulting from one Challenge Fund activity. It is not clear, based on these results, that the capacity development and policy and advocacy activities of SHIFT in SAARC are appropriate for achieving the stated objectives.


Tag: Capacity Building Policy Advisory

108.

Funders are concerned with the communications from management. Programme management is reported to have improved communications with funders. SHIFT in SAARC managed their activities well in terms of keeping within the available budget. However, the programme may be challenged in spending down the committed funds balance within the remaining time period. Although the programme used a “staff-up” approach whereby staff were hired as funding was secured, the level and mix of in-country staff were insufficient for work required by the two projects.


Tag: Communication Human and Financial resources Operational Efficiency Project and Programme management Donor

109.

The programme has faced other challenges including falling short on the timing of output delivery and achievement of results, due to factors both within and outside of UNCDF’s control. The programme’s MRM system is another area of weakness, which is due at least in part to a long-term staff vacancy. The MRM system does not generate enough evidence to inform accountability and contribute to programme improvements, but it could make a more valuable contribution to programme management with certain modifications. However, SHIFT in SAARC has largely overcome the challenges that caused delays, and the programme was operating at a faster pace at the time of the evaluation.


Tag: Challenges Efficiency Rule of law Human and Financial resources Operational Efficiency Project and Programme management

110.

PSC members appreciate the programme and its leadership. PSC members are supportive of the programme’s work in raising policymakers’ awareness around emerging financial services technologies, building the capacity of government policymakers and regulators, working with micro-merchants, and linking the private sector (i.e. the FMCG industry) to financial services.
However, some members are concerned with the pace of implementation. One stated, “we would not say that we are very excited or happy, but we are satisfied with the progress of the project (considering) the reality.” They recommended that SHIFT in SAARC place more emphasis on getting things done on time. The evaluation team found mixed reactions regarding the responsiveness of SHIFT in SAARC to PSC requests for information.


Tag: Challenges Efficiency Communication Project and Programme management

111.

The purpose of the policy and advocacy activities such as coalition building, networks, policy dialogue and coordination, was to promote an improved policy and regulatory environment for digital financial inclusion. SHIFT in SAARC has used its convening power in Bangladesh to strengthen coalitions and partnerships between the public and private sector actors to promote enabling DFS policy. It has convened organizations through events such as high-level roundtables and leadership of the DFCG. One stakeholder stated, “Some important topics in regulations [are] being taken up (e.g. Interoperability and eKyc]. I see these changes and there are not many (organizations) working on this.”


Tag: Effectiveness Innovation Partnership Inclusive economic growth Advocacy Coordination Policy Advisory

112.

While SHIFT in SAARC has had positive effects on building the awareness and capacity of policymakers, the evidence on whether those improvements have resulted in improved financial inclusion policies and regulations that can meet the needs of low-income consumers is for the moment scant. Of the priority policy agenda that SHIFT in SAARC identified, it has been most effective in promoting enabling financial inclusion policies through efforts to support eKYC policy, (see case study, Annex 9). Fewer than half of the government key informants mentioned that SHIFT in SAARC had influenced the eKYC policy while no private sector stakeholders identified this as an accomplishment. However, there is little evidence yet that SHIFT in SAARC’s efforts have influenced any other policies, aside from the NFIS as mentioned above.


Tag: Effectiveness Policy Advisory

Recommendations
1

Recommendation from the Evaluation or Issue-1: Innovate Faster

Start innovation actions faster. It is not possible to have an impact or achieve sustainability without the adoption and scale-up of successful innovation pilots. The pilots need to be done in a timely way to give the market evidence that will encourage them to invest. Along these lines, the expected demonstration effects of the pilots should be explicit in the Theory of Change (TOC) and the measurement and results management (MRM) system. Related to this, the Digital Finance Consultative Group (DFCG) needs to become more action-oriented. The DFCG is currently effective at achieving its main objective “to open up dialogue on the strengths, opportunities, challenges, gaps in the use of DFS for a more inclusive economy and sustainable economic growth in Bangladesh.” As a neutral organization, SHIFT in SAARC is well positioned to introduce a more purpose-oriented, participatory, action-oriented working group that can take on specific topics to promote real change in the sector. The programme could co-facilitate this group as the secretariat with a senior level leader from the Central Bank (i.e., a committed champion who will devote time), regular meetings, designated members, and specific topical sub-groups that engage on priority issues selected by the group. UNCDF has experience coordinating DFS working groups in other regions (e.g., West Africa) that have action-oriented agendas from which they could borrow.

2

Recommendation from the Evaluation or Issue-2: Create a more robust capacity building system 

SHIFT in SAARC is well positioned to play a lead capacity development role at the sector level. Though several training workshops have been conducted with various stakeholder groups, the evaluation team received feedback that the training was not sufficient for those trained and was not extended to other relevant segments. SHIFT in SAARC documentation also did not demonstrate a systematic and robust approach to measuring the results of the capacity building and training output. There remain many additional opportunities to increase DFS usage, but capability remains a critical barrier. Reinforced capacity development for key stakeholders, especially on gender issues, is relevant and could support long-term behaviour change while creating institutional change management with a range of FSPs

3

Recommendation from the Evaluation or Issue-3: Adapt programme management to time and resources 

Given SHIFT in SAARC’s focus on changes in market actor behaviour leading to economic opportunity for women and small businesses, realistic expectations for market systems change should be better aligned with project and funding length. The market system development approach requires time to develop the system and achieve changes. To achieve impact by the end of SHIFT, the evaluation team recommends a deep dive on key sectors where energies and investments are already focused rather than spreading resources across new topics and activities. While adapting to the changing context is essential to remain relevant, reacting without considering how a new activity fits within the strategy diminishes focus on the impact of key activities. Considering the time remaining, the evaluation team recommends not expanding to new segments, sectors or initiatives; instead, it would be useful to consolidate resources to go “deeper” on fewer activities. This approach would include focusing on key sectors already researched and creating better tailored knowledge management resources, distribution channels, and learnings highlighted from the activities already carried out. The programme should focus on quality over quantity. For the innovation grants, focusing on quality would mean focusing on a couple of well-designed projects with robust PBAs that could deliver some quick wins for evidence. It is inadvisable to start market research on new topics or organize large stakeholder roundtables/conferences to present findings.

4

Recommendation from the Evaluation or Issue-4: Review and Revise the MRM System   

The MRM system should be better aligned with the CGAP framework for measuring market development in inclusive finance to improve accountability. This involves identifying and measuring systemic change, as well as changes in the inclusive financial and real economy market systems, and distinguishing these changes from intermediate outcomes over which the programme has more control. A careful review of existing results framework indicators and targets is needed to ensure that indicators are in the right category. These changes would produce evidence that staff can learn from to better manage the programme.

The programme could also improve change monitoring in market systems to better support necessary course corrections by introducing new indicators that address the specific constraints SHIFT in SAARC is trying to alleviate. For example, most micro-merchants have unregistered businesses which prevents them from opening merchant DFS accounts with higher daily transaction amount limits that would allow them to do more business with DFS. As the programme is encouraging merchants to register their businesses, the number of merchants that register their businesses and take up merchant accounts would be a strong indicator of programme effectiveness and changing practices of a market actor.

The timing of data collection could be improved. The co-applicant’s data is reported to SHIFT in SAARC when they reach a milestone and need to apply for the next tranche of funding. It would benefit the monitoring function if this data were provided by co-applicants on a regular basis, say every three to six months, so that the SHIFT in SAARC team could obtain a snapshot of the whole programme at regular intervals and use the data to make management decisions.

The programme does not appear to have a plan to conduct follow-up monitoring visits three to six months after the MDDRM business trainings are finished, such as with micro-merchant or women entrepreneur trainees. One co-applicant recommended that SHIFT in SAARC provide resources to allow for ongoing monitoring of changes at the micro-merchant level, “so we can know the impact of the activities we have implemented and take corrective measures.”

Finally, with respect to the TOC and the structure of outputs, communications activities should be pulled out and treated as a cross-cutting theme like gender equality. This would ensure they receive the required attention and would increase TOC logic.

5

Recommendation from the Evaluation or Issue-5: Incorporate gender issues as a key strategic focus

UNCDF FIPA should require gender analysis for all proposed programmes. This analysis should involve secondary and primary research with the target market segments during the design phase to ensure that the data is current. In the words of one key informant “first, we need to know the problem" regarding women as entrepreneurs.

Within the country or regional office, SHIFT in SAARC or UNCDF should recruit a technically qualified gender champion at the senior management level in order to advance the women’s economic empowerment agenda. The champion should build the capacity of the programme team and focus on the innovation grants and DFCG to raise the awareness of women as a critical market segment. Specifically, the champion should:

  • Build the national team’s awareness of gender equality and women’s empowerment issues so they can ask the right questions and better engage with donors and partners on this issue. Headquarters should take the lead on this to ensure a common understanding of gender equality throughout the agency;
  • Ensure that the team understands the POWER findings, a study to which they contributed, and determine how they may implement the programme based on the study’s findings;
  • Work with innovation grant recipients to ensure female customers/entrepreneurs are included;
  • Create a DFCG working group on gender issues; and
  • Ensure that gender is incorporated into all programmes, activities and institutional structures.
1. Recommendation:

Recommendation from the Evaluation or Issue-1: Innovate Faster

Start innovation actions faster. It is not possible to have an impact or achieve sustainability without the adoption and scale-up of successful innovation pilots. The pilots need to be done in a timely way to give the market evidence that will encourage them to invest. Along these lines, the expected demonstration effects of the pilots should be explicit in the Theory of Change (TOC) and the measurement and results management (MRM) system. Related to this, the Digital Finance Consultative Group (DFCG) needs to become more action-oriented. The DFCG is currently effective at achieving its main objective “to open up dialogue on the strengths, opportunities, challenges, gaps in the use of DFS for a more inclusive economy and sustainable economic growth in Bangladesh.” As a neutral organization, SHIFT in SAARC is well positioned to introduce a more purpose-oriented, participatory, action-oriented working group that can take on specific topics to promote real change in the sector. The programme could co-facilitate this group as the secretariat with a senior level leader from the Central Bank (i.e., a committed champion who will devote time), regular meetings, designated members, and specific topical sub-groups that engage on priority issues selected by the group. UNCDF has experience coordinating DFS working groups in other regions (e.g., West Africa) that have action-oriented agendas from which they could borrow.

Management Response: [Added: 2020/01/30] [Last Updated: 2020/12/16]

Agreed

Key Actions:

Key Action Responsible DueDate Status Comments Documents
1.1 Fast-track implementation of the innovation grants: SHIFT SAARC has already operationalized 6 innovation pilots, and they are being implemented. The donor has agreed to the proposal of SHIFT SAARC to extend the project for 8 more months recently till Sep 2020. The extension is mainly aimed at capturing the learning from implementation and business innovation pilots and sharing the evidence with relevant stakeholders.
[Added: 2020/01/30]
SHIFT SAARC Program Manager 2020/12 Completed
1.2. Periodically update the Theory of Change and MRM to include demonstration effects and learnings from pilots.
[Added: 2020/01/30] [Last Updated: 2021/03/29]
SHIFT SAARC Programme Manager 2021/03 Completed Significant progress on this were made since 2020. MRM dashboards for each of the Business Innovation Pilots are in place. These dashboards are tools to reflect the demonstration effects and learnings from the pilots so that the project managers can make course corrections during implementation. These dashboards are updated periodically. TOCs of each business innovation pilot is contextualized to existing market realities in terms of indicators and trackers. History
1.3 SHIFT SAARC will attempt to transform DFCG into an action-oriented platform together with Gov through the UNDP a2i initiative, re-activate members participation and engagement with policymakers and regulators.
[Added: 2020/01/30] [Last Updated: 2021/03/29]
SHIFT SAARC Program Manager 2021/03 Completed DFCG was a part of the BMGF funded program, and we continued DFCG as of now as this platform has given UNCDF an edge in the digital financial sector for exerting influence and collectively gaining market insights. UNCDF has explored synergies with UNCDF a2i initiative and has connected the members of DFCG with the FinLab initiative. The FinLab is a joint initiative of UNDP, a2i and UNCDF whose overall objective is to encourage innovations in financial sector and improves the efficiency of the sector for achieving greater financial inclusion. FinLab will be formally launched in April 2021 and will be interacting and collaborating with DFCG members and related institutions. History
2. Recommendation:

Recommendation from the Evaluation or Issue-2: Create a more robust capacity building system 

SHIFT in SAARC is well positioned to play a lead capacity development role at the sector level. Though several training workshops have been conducted with various stakeholder groups, the evaluation team received feedback that the training was not sufficient for those trained and was not extended to other relevant segments. SHIFT in SAARC documentation also did not demonstrate a systematic and robust approach to measuring the results of the capacity building and training output. There remain many additional opportunities to increase DFS usage, but capability remains a critical barrier. Reinforced capacity development for key stakeholders, especially on gender issues, is relevant and could support long-term behaviour change while creating institutional change management with a range of FSPs

Management Response: [Added: 2020/01/30] [Last Updated: 2020/12/16]

Agreed

Key Actions:

Key Action Responsible DueDate Status Comments Documents
2.1 Capacity Development is a long term effort, and it requires prior planning and budget. SHIFT SAARC arranged different kinds of interventions, including the development of a DFS module that can be tailored to the needs of the regulators and handed it over to the training academy. Further efforts will be made to convince the Central Bank to organize regular courses based on that module. Key action: Support the Central Bank to organize regular courses based on the DFS module developed by SHIFT.
[Added: 2020/01/30] [Last Updated: 2020/09/29]
SHIFT SAARC Program Manager 2020/12 Completed UNCDF initiated discussions with the Bangladesh Bank to organize regular courses based on the DFS module developed. To that end, UNCDF organized a ToT for 30 central bank officials, led by international experts, to create a pool of master trainers on DFS. 4 volumes of training module were printed for the sector regulators and shared with Bangladesh Bank. History
2.2 Key Action: The programme will also work towards raising resources or collaborate with institutions having resources on stakeholder capacity building to draw up a comprehensive plan for capacity building of regulators and critical providers as well, especially addressing how decision-makers consult data and evidence and how gender issues and needs of the left-behind segments are viewed on a policy level. Capacity Development through data insights on micro-merchants, its customers, and policy aspects will be undertaken once the innovation pilots are completed.
[Added: 2020/01/30] [Last Updated: 2021/03/29]
SHIFT SAARC Programme Manager 2021/03 Completed Towards this end, keeping in line with UNCDF’s global Inclusive Digital Economy Strategy (IDES), the country strategy in Bangladesh prioritizes capacity building of stakeholders and key user segments in its plan. Currently, the team is in the process of partnering with a2i to build up FinLab which will be launched in April 2021 and will be a platform which will be leveraged, to jointly mobilize resources for building capacities of regulators, service providers and key user segments among other joint activities. UNCDF has also initiated financial institutions training along with the regulators and resources being co-funded by all the partners. History
3. Recommendation:

Recommendation from the Evaluation or Issue-3: Adapt programme management to time and resources 

Given SHIFT in SAARC’s focus on changes in market actor behaviour leading to economic opportunity for women and small businesses, realistic expectations for market systems change should be better aligned with project and funding length. The market system development approach requires time to develop the system and achieve changes. To achieve impact by the end of SHIFT, the evaluation team recommends a deep dive on key sectors where energies and investments are already focused rather than spreading resources across new topics and activities. While adapting to the changing context is essential to remain relevant, reacting without considering how a new activity fits within the strategy diminishes focus on the impact of key activities. Considering the time remaining, the evaluation team recommends not expanding to new segments, sectors or initiatives; instead, it would be useful to consolidate resources to go “deeper” on fewer activities. This approach would include focusing on key sectors already researched and creating better tailored knowledge management resources, distribution channels, and learnings highlighted from the activities already carried out. The programme should focus on quality over quantity. For the innovation grants, focusing on quality would mean focusing on a couple of well-designed projects with robust PBAs that could deliver some quick wins for evidence. It is inadvisable to start market research on new topics or organize large stakeholder roundtables/conferences to present findings.

Management Response: [Added: 2020/01/30] [Last Updated: 2020/12/16]

Agreed

Key Actions:

Key Action Responsible DueDate Status Comments Documents
In support of the recommendation to provide ‘deep dive support’ on crucial sectors rather than spreading across new topics and activities, the following key actions are proposed: 3.1 i) Use insights from the micro-merchant assessment for more tailored innovation pilots and for fine-tuning innovation models. ii) Commission additional research on digital transformation of supply chain operations and payments to better understand value chain actors and inform potential investors and market actors.
[Added: 2020/01/30] [Last Updated: 2021/03/29]
SHIFT SAARC Program Manager 2021/03 Completed UNCDF has onboarded six private sector-led business innovation pilots in Bangladesh during the MTE phase. Apart from these, 4 more business innovation models were implemented. Also in terms of fine tuning and upscaling 4 existing partners (out of the 6) were further supported. In addition to supporting the implementation of these pilots, the team sat with implementation partners at regular intervals. In these sessions, besides tracking project progress, the team also made sure that product features and project operations accommodate insights from UNCDF’s micro-merchant landscape assessment. Different private sector partners were also introduced to each other to collaborate and leverage on the respective business models. History
3.2 Realign focus on a few well-designed innovation pilots that could deliver quick wins for evidence. In support of this, SHIFT SAARC will closely monitor the pilots to ensure adherence to the principles of lean innovation and customer-centricity and for overall improvements.
[Added: 2020/01/30] [Last Updated: 2020/06/30]
SHIFT SAARC Program Manager 2020/06 Completed Prior to the COVID-19 outbreak in Bangladesh, the team had collected some user stories from micro-merchants onboarded by partners in the project districts. Using user segmentation techniques on baseline data and MIS data from the backend of partners’ digital platforms, multiple customer profiles of micro-merchants had been developed before going for interviews. These interviews have been conducted in light of the principles of lean innovation and customer centricity. The staff spent a number of days on the field visiting many micro-merchant shops at a stretch. Besides validating milestones reached as part of innovation pilots, the team aimed to learn firsthand from micro-merchants how joining these digital platforms made a difference for them. For instance, micro-merchant assessment had found that recovery of credit sales often become a growth constraint for micro-merchants. Although SureCash, one of our innovation partners, developed a feature of credit sales record in their TallyKhata app earlier, we learnt from a group of “early adopters” that a reminder feature would be useful as well. Following discussions and more inquiries, SureCash is on the way of releasing another version of the app. Besides a host of other useful features, this version will include customer reminder notification via WhatsApp and imo besides SMS. Following movement restrictions and rising health safety concerns due to COVID-19, the team started even more closely monitoring the situation in project districts by teaming up with all the partners. To date, about 580 micro-merchants have been surveyed over phone to understand how they are affected by the current pandemic and what support they require in the recovery process. As a follow-through, the team has convened all four partners of ongoing innovation contracts and Prime Bank Limited to quickly extend digital credit as working capital support to micro-merchants. To that end, the bank has categorically agreed to leverage digital business records of micro-merchants for alternative credit assessment and lending. UNCDF is actively facilitating the process between each partner and the bank so that the support can be extended as soon as possible. History
3.3 The programme will periodically document the knowledge management lessons of the innovations being piloted, thereby capturing learnings for wider market and stakeholder benefits
[Added: 2020/01/30] [Last Updated: 2021/04/20]
SHIFT SAARC Programme Manager 2021/03 Completed UNCDF has been actively documenting the lessons of the innovations that are being piloted in the form of case studies, social media posts, blog write-ups, consultative and sharing sessions etc. UNCDF has completed a series of case studies and blogs that will capture the learnings for wider market and stakeholders. Besides, a Micro-merchant Policy Brief has been developed in light of implementation and market insights from the business innovation pilots. The team has also developed a Bangla translation of UNCDF’s micro-merchant landscape assessment. History History
3.4 Through training, immersive workshops and Partner Review Meetings with the innovation partners, SHIFT SAARC has been making sure that they have a proper understanding of gender issues, challenges and needs of these overlooked segments and the techniques of capturing and building on implementation lessons. This will be further intensified during the remaining period of the project period.
[Added: 2020/01/30] [Last Updated: 2021/03/29]
SHIFT SAARC Program Manager 2021/03 Completed UNCDF has been closely monitoring the partner’s activities by periodically coordinating virtual meetings and communicating with them. The learnings (challenges faced due to COVID19 restrictions and opportunities) are being documented to understand the gender issues, challenges and the needs of the overlooked segments in this critical time. Based on the challenges and partner activities, UNCDF has been giving guidance to the partners on the aforementioned issues. UNCDF does believe that the current partners have been sensitized on the issues. Additionally, UNCDF has hired new pilots ensuring that the implementers are properly sensitized through onboarding sessions and meetings. Additionally, UNCDF in partnership with its partners has organized three webinars; one was organized with iSocial that focused solely on the gender issues and the other two were organized with SureCash and Ekshop and have discussed gender. History
4. Recommendation:

Recommendation from the Evaluation or Issue-4: Review and Revise the MRM System   

The MRM system should be better aligned with the CGAP framework for measuring market development in inclusive finance to improve accountability. This involves identifying and measuring systemic change, as well as changes in the inclusive financial and real economy market systems, and distinguishing these changes from intermediate outcomes over which the programme has more control. A careful review of existing results framework indicators and targets is needed to ensure that indicators are in the right category. These changes would produce evidence that staff can learn from to better manage the programme.

The programme could also improve change monitoring in market systems to better support necessary course corrections by introducing new indicators that address the specific constraints SHIFT in SAARC is trying to alleviate. For example, most micro-merchants have unregistered businesses which prevents them from opening merchant DFS accounts with higher daily transaction amount limits that would allow them to do more business with DFS. As the programme is encouraging merchants to register their businesses, the number of merchants that register their businesses and take up merchant accounts would be a strong indicator of programme effectiveness and changing practices of a market actor.

The timing of data collection could be improved. The co-applicant’s data is reported to SHIFT in SAARC when they reach a milestone and need to apply for the next tranche of funding. It would benefit the monitoring function if this data were provided by co-applicants on a regular basis, say every three to six months, so that the SHIFT in SAARC team could obtain a snapshot of the whole programme at regular intervals and use the data to make management decisions.

The programme does not appear to have a plan to conduct follow-up monitoring visits three to six months after the MDDRM business trainings are finished, such as with micro-merchant or women entrepreneur trainees. One co-applicant recommended that SHIFT in SAARC provide resources to allow for ongoing monitoring of changes at the micro-merchant level, “so we can know the impact of the activities we have implemented and take corrective measures.”

Finally, with respect to the TOC and the structure of outputs, communications activities should be pulled out and treated as a cross-cutting theme like gender equality. This would ensure they receive the required attention and would increase TOC logic.

Management Response: [Added: 2020/01/30] [Last Updated: 2020/12/16]

Agreed

Key Actions:

Key Action Responsible DueDate Status Comments Documents
4.1. The MRM system will be further developed and aligned with CGAP framework on measuring market system development for the poor.
[Added: 2020/01/30] [Last Updated: 2021/03/29]
PM 2021/03 Completed The MRM system of SHIFT SAARC has been thoroughly reviewed, and some areas identified areas where there could have been better alignment with the CGAP framework for measuring market development in inclusive finance to improve accountability. Given the MDDRM project is at its end, the team recognizes that the MRM system cannot be fully updated at this stage but the team have identified some indicators can be aligned with the CGAP framework. Based on further review and approval of this identification and in alignment with the outcomes of the dashboard exercise of innovation pilots, the MRM system has been updated. History
4.2 SHIFT SAARC will further strengthen its existing MIS which presently tracks both programme and partner level activities. Data from digital media platforms will be collected in near real-time basis.
[Added: 2020/01/30] [Last Updated: 2020/09/29]
PM 2020/09 Completed UNCDF is using a data-driven and real-time mechanism of managing program activities. The majority of the events are now being tracked via the Management System (MIS) platforms of the partners. These MIS dashboards are being updated at real-time. Project’s M&E team tracks the performance data at regular intervals and suggest course corrections in monthly coordination meetings. In the next quarters, MIS platforms of incoming implementers for new innovation projects will also be connected and tracked accordingly. History
4.3: SHIFT SAARC will step up follow-up visits to understand how and why training did or did not make any difference to the target segments.
[Added: 2020/01/30] [Last Updated: 2020/12/08]
PM 2020/12 Completed Following the outbreak of COVID-19 in Bangladesh, mobility and assembly of people across the country has been greatly limited and as such in person follow-up visits were not conducted. However, UNCDF team has periodically conducted virtual follow-ups with the partners in the form of meetings and telephonic verification of the trainings. These verifications have been extensive and comprehensive and UNCDF has ensured a statistically significant sample size when making the verifications. The learnings from the verifications and the overall learnings from the business utility of trainings have been documented and captured in blogs and internal reporting documents. The blog has been tailored in such a way that provides an insight on how the trainings are conducted, what worked best and what can be further improved. It will be published and circulated with wider audience. The team will conduct physical visits when things improve in Bangladesh. History
4.4 SHIFT SAARC will further enhance its ongoing communications where findings from research initiatives and anecdotal impact stories is transformed into targeted messaging for specific stakeholder groups and broadcasted via newsletters, info sheets and social media channels. Further strengthening of cross-sharing of updates and insights among partner networks will be undertaken
[Added: 2020/01/30] [Last Updated: 2020/09/29]
PM 2020/09 Completed Going beyond the formal activities related to communications, UNCDF has been continuously working on ramping up the project’s communications in very impactful ways. In the recent times, COVID – 19 impact on micro-merchants, anecdotal impact stories from the field as seen by the partners have been transformed into targeted messaging for specific stakeholder groups and broadcasted via social media channels, blogs, etc. Besides, a dedicated FB page for micro-merchants has been created from where regular targeted messaging is being done for micro-merchants and related actors in the value chain. More of these activities will be conducted in the next quarters. History
5. Recommendation:

Recommendation from the Evaluation or Issue-5: Incorporate gender issues as a key strategic focus

UNCDF FIPA should require gender analysis for all proposed programmes. This analysis should involve secondary and primary research with the target market segments during the design phase to ensure that the data is current. In the words of one key informant “first, we need to know the problem" regarding women as entrepreneurs.

Within the country or regional office, SHIFT in SAARC or UNCDF should recruit a technically qualified gender champion at the senior management level in order to advance the women’s economic empowerment agenda. The champion should build the capacity of the programme team and focus on the innovation grants and DFCG to raise the awareness of women as a critical market segment. Specifically, the champion should:

  • Build the national team’s awareness of gender equality and women’s empowerment issues so they can ask the right questions and better engage with donors and partners on this issue. Headquarters should take the lead on this to ensure a common understanding of gender equality throughout the agency;
  • Ensure that the team understands the POWER findings, a study to which they contributed, and determine how they may implement the programme based on the study’s findings;
  • Work with innovation grant recipients to ensure female customers/entrepreneurs are included;
  • Create a DFCG working group on gender issues; and
  • Ensure that gender is incorporated into all programmes, activities and institutional structures.
Management Response: [Added: 2020/01/30] [Last Updated: 2020/12/16]

Agreed

Key Actions:

Key Action Responsible DueDate Status Comments Documents
5.1 The programme will specifically identify issues related to female customers/entrepreneurs onboarding and benefits accrued under its innovation grant recipients project. Based on the issues identified, technical assistance will be provided to maximise on gender outcomes under the innovation pilots.
[Added: 2020/01/30] [Last Updated: 2021/03/29]
PM 2021/03 Completed UNCDF is actively looking into apparent lack of gender analysis at the design of programs and putting systems in place for filling any gaps in this regard. Issues related to female customers/entrepreneurs onboarding and benefits have been identified in the case studies. To that end, UNCDF conducted customer journey mapping of Kallyanis (women MMs onboarded by iSocial) through in-person interviews, observations and phone interviews. These insights have been showcased in a case study of the women MM model and featured in some blogs. History
5.2 SHIFT SAARC programme will support a workshop with its partners and national stakeholders on incorporating gender issues as a key strategic focus while working on digital financial services.
[Added: 2020/01/30] [Last Updated: 2020/12/08]
PM 2020/12 Completed UNCDF has planned on organizing a series of webinars, one of which is on incorporating gender issues as a key strategic focus while working on digital financial services. To that end, UNCDF organized a webinar with iSocial titled “Creating Livelihood Opportunities for Women at the Bottom of the Pyramid through Digital Means” on Monday, 2nd November, 2020. The objective of the webinar was to discuss the importance of digital and financial empowerment of women, while highlighting the findings from the UNCDF-backed business innovation project that iSocial has implemented as part of MDDRM supported by EU. History

Latest Evaluations

Contact us

1 UN Plaza
DC1-20th Floor
New York, NY 10017
Tel. +1 646 781 4200
Fax. +1 646 781 4213
erc.support@undp.org