Mid-Term Evaluation of Shaping Inclusive Finance Transformations (SHIFT) Programme in Asean

Report Cover Image
Evaluation Plan:
2016-2017, UNCDF
Evaluation Type:
Others
Planned End Date:
03/2018
Completion Date:
12/2018
Status:
Completed
Management Response:
Yes
Evaluation Budget(US $):
77,200

Share

Document Type Language Size Status Downloads
Download document TOR SHIFT.docx tor English 830.41 KB Posted 280
Download document SHIFT ASEAN Mid-term Evaluation_Appendix_Country Reports (1).pdf related-document English 1193.63 KB Posted 538
Download document SHIFT ASEAN Final Mid-term Evaluation Report_November 2018.pdf report English 1729.72 KB Posted 715
Title Mid-Term Evaluation of Shaping Inclusive Finance Transformations (SHIFT) Programme in Asean
Atlas Project Number:
Evaluation Plan: 2016-2017, UNCDF
Evaluation Type: Others
Status: Completed
Completion Date: 12/2018
Planned End Date: 03/2018
Management Response: Yes
Corporate Outcome and Output (UNDP Strategic Plan 2014-2017)
Evaluation Budget(US $): 77,200
Source of Funding:
Evaluation Expenditure(US $): 77,200
Joint Programme: No
Joint Evaluation: No
Evaluation Team members:
Name Title Email Nationality
Frances Sinha Team Leader INDIA
Shantanu Das Team Member INDIA
Sanjay Sinha Team Member INDIA
Abhirup Bhunia Team Member INDIA
Nitin Madan Team Member INDIA
GEF Evaluation: No
Key Stakeholders:
Countries: GLOBAL
Lessons
Findings
1.

1Evaluation Findings

1.1Relevance and Quality of Design

How relevant is the programme design to the contexts of intervention, and how has it evolved?

The programme design is relevant to the contexts of intervention, being grounded in effective consultation in (most) countries of the region and responding to ASEAN Economic Community and individual country interest in financial inclusion as a key to economic growth. In applying a market systems approach to address low levels of financial inclusion particularly in the LDCs of the region, there has been a significant element of flexibility and evolution in response to opportunities, although implementation has also been affected by funding constraints.


Tag: Relevance Programme/Project Design Inclusive economic growth

2.

Programme alignment with the country and ASEAN institutional and policy/regulatory context and evolving scenario; addresses identified needs and gaps in area of intervention in the countries of operation

The process of SHIFT design in 2013-14 and its evolution over the last few years since then, has been characterised by an effective consultative and adaptive approach, by which the programme team has responded well to stakeholder priorities in the ASEAN region. These priorities were articulated by the ASEAN Economic Community (AEC) in identifying financial inclusion as a policy objective for equitable economic growth across the region,10 and also reflected in the requests for technical partnerships from country regulators within the WC-FINC of ASEAN as well as from the Alliance for Financial Inclusion (at the regional and global level).


Tag: Relevance Partnership Programme/Project Design Project and Programme management Inclusive economic growth Inequalities Technical Support

3.

The programme design has directly addressed issues in the low levels of financial inclusion in CLMV, a need to strengthen regulatory and policy capacity, coordinate across financial markets, apply a gender lens). The design moved beyond the conventional approach of focusing support into MFIs, to address wider market systems issues around policy and capacity across a range of financial services, whilst responding to the emerging technological opportunities.


Tag: Gender Mainstreaming Programme/Project Design International Financial Institutions Inclusive economic growth Capacity Building Coordination Policy Advisory

4.

The Programme Document recognised the importance of building flexibility in applying a market development approach.13 SHIFT was appropriately designed to be implemented in a ‘modular approach’ for flexibility in implementation over time. A modular approach is evident from the varying stages of implementation of the different pillars - both regionally and across the four countries, with an initial focus on two countries (Cambodia and Myanmar). This has been in response to a variation in the opportunities as well as complementarity to country activities of UNCDF and other financial development players14 (noted further in the next subsection).

In the initial stage, programme interventions were determined by DFAT’s request to focus on the four least developed ASEAN countries (CLMV) and on three of the four pillars, particularly the Challenge Fund facility. Since 2017, with some internal changes within DFAT as well as some success in garnering alternative funding within the region, SHIFT has begun to engage more closely with partners in ASEAN countries other than CLMV, and to start activities under the learning & capacity building pillar. The latter is itself an adaptation of the original design which was to focus on e-learning.15

 


Tag: Partnership Programme/Project Design Trade and Development Capacity Building

5.

Convergence and complementarities with activities of other donors, governmental initiatives, existing schemes and projects

In designing many of its activities, SHIFT has converged well with global and regional programmes, including for example: CGAP’s work to define the key elements of measuring a market development approach for financial inclusion,17 and providing substantial technical inputs to AFI’s publications on gender, specifically building the technical guidelines to support the Denarau Action Plan under which members have all committed to women’s financial inclusion.18fice), providing direct technical back up to the World Bank initiative.


Tag: Gender Equality Women's Empowerment Harmonization Programme Synergy Programme/Project Design Inclusive economic growth

6.

Are each of the programme designs coherent and how well are they designed for eventual transition, expansion and replication?

SHIFT’s TOC delineates well the four pillars of activities which are interlinked for a market systems approach. Though the overarching TOC remains relevant to capturing the development pathways (from input to impact levels), it does not capture different pathways of activity in the countries of focus nor the (2-way) interlinkages between regional and country activities. Intended tracking of policy related processes at the regional and country level has not been applied.

19The programme has laid the groundwork for eventual transition, expansion and replication through a number of strategic partnerships serving different roles, the link with UNCDF, and at the country level (in Cambodia) putting mechanisms in place for the regulator to take forward the implementation of the NFIS. The design of the Challenge Fund anticipates the scope for expansion and replication of projects that are successful.


Tag: Sustainability Partnership Programme/Project Design Sustainability Theory of Change Trade and Development Policy Advisory National Institutions Regional Institutions

7.

Definition and clarity of the TOC to capture the developmental pathways at the input, output, outcome and impact levels. Is the TOC on track to capture programme objectives and results?

SHIFT’s TOC is built on the premise that influencing behaviour change of partners (both public and private sector) through data, knowledge, capacity building, incentivising investments, policy advocacy and high level political change can bring systemic market change, as the necessary precursor to achieving financial inclusion. The TOC logic is that increasing access and usage of financial products and services can only result from the increased number, diversity and affordability of such products and services that meet the needs of low-income consumers, micro-entrepreneurs, and small and medium business, especially women.


Tag: Resource mobilization Women's Empowerment Theory of Change Inclusive economic growth Inequalities Jobs and Livelihoods Trade and Development Capacity Building Policy Advisory Data and Statistics

8.

The roles and interlinkages of the different pillars have been well delineated: the fund facility was intended to support new business models to serve low income customers, inspiring replication (by other players – FSPs, potential investors) in the wider market, and drawing the attention of regulators; policy and advocacy activities were intended to promote and guide financial inclusion strategies at both regional and country levels; data and analysis was intended to support country level strategy development, to document the challenge fund findings, and to generate ‘big data; all these were intended to lead into learning and skills development activities - for both regulator and FSP staff - for improved strategic thinking and product development.


Tag: Business Model Sustainability Inclusive economic growth Advocacy Policy Advisory Data and Statistics

9.

Partnerships at the regional, national and local levels have clearly delineated roles; ownership of programme by implementing partners

Strategic partnerships have been a core feature of SHIFT’s design at all levels of intervention, from global to local. Roles and objectives are well set out in separate partner agreements and are discussed further under 4.2. The partners’ list (e.g. AFI, CGAP, ILO; see programme profile in Section 2) is an impressive roll call of experienced organisations who have clear delineated roles across the four pillars of SHIFT’s programme. This has helped in establishing a wide network, reduced costs (for example UN Pulse Lab work on data analytics and developing a new CF in Indonesia) and leveraged additional resources (for example, UNCDF Clean Start funding in CF Window 1, MAP and Visa’s albeit small funding in CF Window 3 Ownership at the country level in Cambodia is evident in the stake of the regulator (National Bank of Cambodia) supported by the MAP process.


Tag: Ownership Partnership

10.

4.1.1Does the programme design incorporate cross cutting issues of inclusion, gender and human rights?

Issues of inclusion appear central to SHIFT’s programme design and targeting. Gender is well incorporated as a cross-cutting issue, though there are measurement issues whilst non-financial aspects affecting women’s economic empowerment are not addressed. Issues of appropriately defining and measuring outreach to “poor” and vulnerable populations are still to be addressed (as part of Finscope surveys too). A strategy to include people with disabilities (targeted by the donor), has been missing although there could be opportunities for this.


Tag: Gender Equality Gender Mainstreaming Gender Parity Women's Empowerment Human rights Programme/Project Design Disabilities Inclusive economic growth Inequalities Poverty Reduction

11.

Issues of inclusion have been mostly well covered in terms of setting targets and reporting. End goals are articulated in terms of access to financial services for low income people, women, and including the “poor” and “vulnerable”. The ProDoc defines poor and “vulnerable” populations (as below or ‘just above’ the US$1.25 international poverty line); this is also referenced in partner agreements for the CF. However, this definition has not been thought through in terms of its applicability to CLMV. Surprisingly, it is also absent from Finscope surveys’ analysis. Service outreach to rural areas – where there is typically more poverty than in urban areas – is often targeted but not directly reported for SHIFT’s interventions. The SHIFT team has identified the need to look more closely at definition and effective measurement of low-income people.


Tag: Rural Urban Results-Based Management Inclusive economic growth Poverty Reduction Vulnerable

12.

People with disability represent a significant segment20 of ‘vulnerable’ populations. The programme management is yet to respond to the donor mandate21 and efforts to introduce mechanisms for inclusion of people with disabilities or ethnic minorities, as a part of the target market of ‘vulnerable populations’. The team is now looking at a strategy and mechanisms to do this – where it appears most relevant and feasible.22


Tag: Disabilities Vulnerable

13.

A focus on women and mainstreaming of gender has been reflected in SHIFT’s targets (65% of outreach to be women, 100% of SMEs to be led by or primarily employing or serving women’) along with reporting across all pillars; and are the focus of DFAT’s Gender Equality Fund investment through SHIFT being implemented in Myanmar since mid-2017.23 SHIFT has focused on reducing the gender gap as part of NFISs24, presenting sex-disaggregated data on financial services for product related decisions25 and advocacy on gender and sex-disaggregated data analytics at many forums, involving both private and public sector stakeholders. CF partners are required to target women as part of their business model, and to report on sex-disaggregated data on women as employees, agents or end clients. However, this is not always well reported and there are some conceptual issues as well as capacity gaps.26


Tag: Gender Mainstreaming Gender Parity Women's Empowerment Results-Based Management Data and Statistics

14.

These interventions reflect the framework of SHIFT’s national Gender Strategy documents, developed for each of country under CMLV. They are partially in line with the UNCDF Strategic Framework 2018- 202127, supporting the first element of expanding the supply of products/services responsive to the needs of women and girls; but not (yet) directly addressing the other elements: increasing usage of and control over benefits of financial services; improving enabling environment of policies and regulations affecting women’s access and agency in financial services; as well as reinforcing ecosystem support structures and networks to reach women and girls.


Tag: Gender Equality Service delivery Strategic Positioning Inclusive economic growth Women and gilrs

15.

SHIFT’s design aligns to two of the Sustainable Development Goals (SDG) - SDG 5 (achieving gender equality and economic empowerment of women) and SDG 1 (ending poverty). The programme plans to address poverty measurement in the future so as to be able to track and thereby try to ensure usage of services - and benefits - by the poor and low-income people.


Tag: Gender Equality Women's Empowerment Poverty Reduction Agenda 2030

16.

Efficiency

How well have the programme inputs (at both programme and HQ levels) been managed?

Given the resource constraints, the programme inputs have been managed well by co-funding of staff positions, leveraging other UNCDF programme resources and building partnerships within and outside UN organisations. Dependence on a single donor DFAT led initially to a focus on the Challenge Fund. In Window 1, UNCDF Clean Start provided the major fund commitment. With varying partnerships for investments, the administrative costs to SHIFT’s Challenge Fund committed ratio28 is around 0.97 – (or 0.73 based on SHIFT’s revised data). This is a high ratio compared to other Challenge funds. However, it should be noted that the Challenge fund has successfully leveraged an impressive $4.25 from the grantees for every dollar SHIFT spent (compared to 50% of funding in most enterprise challenge funds)29 and $1.1 from other UNCDF programmes.

The average unit cost of reaching a direct end client, based on current estimates for challenge fund outreach (and effective average usage for the investment with the largest outreach), works out to $17. We note that any calculation of genuine VfM needs to take account of usage – not just the fast growth of initial access to digital financial services.

Understaffing, especially related to policy advocacy, learning and capacity building has to some extent affected delivery and progress on these two pillars.

 


Tag: Efficiency Government Cost-sharing Human and Financial resources Operational Efficiency Partnership

17.

SHIFT is majorly funded by DFAT. The project remains underfunded resulting in understaffing of key positions, as well as some constraints in focus and activities across pillars and countries. Out of the total resources raised so far for implementation of the programme, 87% has come from DFAT. This influenced the priorities and spending pattern of the SHIFT’s intervention, particularly at the start.


Tag: Challenges Efficiency Human and Financial resources

18.

To respond to underfunding issues, SHIFT has adopted a mix of approaches to increase efficiency. This includes prioritisation of activities (e.g. dropping the original concept of the Learning and Development portal, focusing on two of the CLMV countries, focusing on CF), lean shared staffing (short term consultants (instead of dedicated staff) were hired along with cross-leveraging of staff from other UNCDF programmes).


Tag: Efficiency Human and Financial resources Operational Efficiency

19.

The Challenge Fund was the key mechanism for the programme to achieve the direct outreach targets. This is reflected in 44% of total spend reported for managing and funding the Challenge Fund. However, it should be noted that revised clarification shared by SHIFT team suggests that the proportion set aside for the Challenge Fund can be reduced, since it seems that some M&E, costs of consultants and workshops are included within the CF budget pillar but relate to other pillars.30


Tag: Efficiency Human and Financial resources Monitoring and Evaluation Operational Efficiency

20.

The Challenge Fund investment made so far is only 10% of the total programme spend. The fund facility has successfully managed to leverage 4.25 times of investment from grantees and 1.1 times from UNCDF’s Clean Start programme, thereby minimizing its own fund exposure.31 Consequently, and combined with the discipline of PBAs and fund tranches dependent on achieving agreed targets, SHIFT has been able to keep its financial risk at a low level (so far only losing $161,000 in closed window 1 investments).

Detailed analysis of challenge fund investments reveals that the average size of the challenge fund investments is $246,000 including the commitment of other donors, with SHIFT’s investment being $139,887. This is significantly lower than the other challenge fund investments where it is expected that the average donor investment should be $500,000-600,000 to make a significant dent on market systems and begin to show impacts on the ground.32 In a comparable economy, DFID’s Vietnam Business Challenge Fund investment shows that the average ticket size was around $300,000.

The administrative cost ratio33 of managing the challenge fund is 0.97 (or 0.73 if we consider SHIFT’s programme team revised expenses on CF cost estimate – to exclude certain costs which can be allocated to other pillars). In either case, this is a very high ratio (It compares with 0.37 in the Vietnamese Business Challenge Fund).34


Tag: Efficiency Operational Efficiency

21.

Based on the direct outreach figures of the Challenge Fund, which is 1.99 million people at the end of Q1 in 2018, the average unit cost for SHIFT to reach an end client averaged $3.07 (based on total spend). As seen in Table 9 (page 49), this CF outreach data is largely due to the very high outreach of one project, which far exceeded its target in terms of opening e-wallets, though active usage is much lower. Based on an average estimated usage of 17% in this project35, and an estimated usage data for other continuing projects, the unit cost works out to more than $17. This raises the question of what represents a genuine VfM. If other investments, particularly those involving digital technology and remittances roll out, the potential direct outreach will grow significantly, whilst the real value will lie in regular usage. Ultimately, the total expenses of the SHIFT programme are expected to contribute to indirect outreach and impacts – which should be accounted for as part of VfM, depending on the contribution that is estimated for SHIFT.


Tag: Efficiency Operational Efficiency

22.

Given the resource constraints, it is important formally to close all the suspended investments (e.g. AMRET is not closed yet), and unlock the resources committed in UNCDF books of accounts for use in other pillars and activities. Locked in resources inhibits efficient management of cash.


Tag: Efficiency Human and Financial resources Operational Efficiency

23.

VfM analysis related to other pillars is challenging since the benefits of data analysis and research products (CJAR, Youth paper etc.) and policy support (NFIS strategy) cannot be quantified or monetized at this stage. In case of Learning & Development though, the total spend is comparatively low (US$526,424) and only 8% of the total expenditure so far. As per the estimate provided by the SHIFT programme team, so far, 120 people have been trained at an average cost of $431 per participant. Given the capacity gaps and the need to reach scale to influence market development, the current SHIFT thinking of tie ups with academic institutions and developing a pool of master trainers is a more cost effective and a sustainable approach


Tag: Efficiency Operational Efficiency Capacity Building Policy Advisory Data and Statistics

24.

The overall staffing pattern is significantly less than the original design. The present resource deployment is a mix of part-time staff and consultants (deployed based on requirements), for SHIFT ASEAN with no fulltime staff for the Policy Advocacy or Learning Pillars which are being managed as shared responsibilities. SHIFT has hired consultants on gender and on resource mobilisation. This approach has resulted in a low 50% ratio of staffing/consultant to total costs. Nevertheless, programme effectiveness is likely to be enhanced if SHIFT is able to raise the resources to fill specialist positions.36


Tag: Efficiency Human and Financial resources Operational Efficiency

25.

How well are the processes and activities managed to ensure results?

The management of processes and activities under the different pillars has been variable with a number of issues (including timing, planning, ensuring relevance) in the management of the Challenge Fund first two windows, and in the first round of data training. Lessons have been documented and taken into account in current management and in the planning for further rounds.


Tag: Efficiency Project and Programme management Capacity Building Data and Statistics

26.

Quality of underlying processes and timelines for activities (eg. Selection of Challenge Fund partners, etc.)

The quality of processes and timeliness for activities has been variable, but these have evolved and been strengthened based on the learnings over the programme period.

Policy and advocacy/data work in Cambodia has followed the MAP process steps but over a lengthy time frame, which still continues. The FinScope survey was completed in 2015, but the consultations and approvals for drafting and Government acceptance of the Financial Inclusion Roadmap and Action Plan took another 2 years. The NBC Governor’s approval coincided with this mid-term evaluation (mid 2018). This length of time was in part due to a number of reviews and a reported need for data corrections by the NBC,37 indicating some inefficiency in the process


Tag: Efficiency Project and Programme management Advocacy Policy Advisory Data and Statistics

27.

The management procedures for the Challenge Funds have evolved over the subsequent rounds to ensure a more robust selection and agreement process.

The experience with the first Window led the team to aim for a better degree of balance between higher- risk and lower-risk investments, together with comprehensive guidelines and strengthened agreements to include specific conditions (legal, establishments of partnerships), to be met before funding would start.38 Window 2 selection has the geographical coverage of all CLMV and a good spread of technologies and approaches compared to Window 1. Nevertheless, there have been continuing issues related to timing and target setting which point to some inefficiency or lack of practical clarity in the planning and negotiation process (as well as a degree of optimism/ambition – for both SHIFT and the project organisations).39 Fund-related delay has compressed the time to achieve agreed milestones.40 Window 3 has seen a funding gap that has reduced some of the innovation component proposed, 41 and contractual arrangements that are delaying the start of the projects.

The IC’s assessment parameters were strengthened following Window 1, and its composition is adapted to the different themes. However, there is no feedback loop to the IC on the performance of different projects which can provide useful learnings for future rounds of project appraisal.42


Tag: Efficiency Project and Programme management

28.

Learning and capacity building was undertaken in Cambodia during 2017, through a few training programmes for FSPs and regulators. Whilst training served the purpose of initial exposure on data analytics and use of data, it is not by itself sufficient for building capacity – for which a more sustained and long-term support through regular trainings would be more effective, albeit at an additional cost. Reporting and direct feedback from participants indicates a need for better needs assessment and contextual understanding of different organisations so as to adjust the training content before it is delivered. 43 More attention is required on the needs and capacities of participants, so that they should be in a position to take the training forward. SHIFT is learning from this as it explores opportunities for data training in Myanmar. Here, FSPs need to understand the basic tools for data analysis before venturing into big data. SHIFT will be reorienting the focus of trainings to involve more basic functions in its modules.


Tag: Capacity Building Data and Statistics

29.

Robustness of the M&E systems to capture programme results and generate evidence to ensure accountability and improve the programme

In line with the CGAP guidelines on measuring market development, SHIFT’s MRM is relying on a “bottom-up” and “top-down” approach in adapting to the requirements of measuring diverse activities supporting the market system for financial inclusion. Accordingly, the MRM assesses performance on a range of activities and factors leading to outcomes at the level of the financial eco-system in turn leading to development outcomes for end-clients.

Bottom-up monitoring and partner feedback is well documented and quite robust, capturing process and risk issues, and going beyond access to usage of financial services. Quarterly reporting by project partners covers activities, challenges, and risks following a specified format as well as details of expenses, and – for challenge fund partners – details of (sex disaggregated) outreach achieved against the agreed milestones. Partner reports and data are verified through in-person visits which include visits to the field and interactions with staff/agents and end clients, with detailed Back to Office Reports (BTORs).

From the start, MRM has required partners to report on usage of services, not just access – which is an important requirement, given the well-known problem of dormant financial accounts, and indeed this emerged as an issue in the otherwise successful CF with LVPB.


Tag: Implementation Modality Monitoring and Evaluation Project and Programme management Inclusive economic growth

30.

There are some gaps – in partner and generally in financial reporting systems - in not being able to capture the end-client profile, whether low income/BoP or even rural. For example, LVPB, (which accounts for more than 98% of direct client outreach under SHIFT), is not reporting on the economic or demographic profile of its client base (because this is not part of the KYC requirements for e-wallets). Without this information, however, the programme lacks evidence that it is contributing to the financial inclusion of low-income populations. SHIFT is trying to address this by adding Poverty Probability Index (PPI)44 questions to a client survey (now being applied telephonically by LVPB). CF reporting by partners is expected to include annual client surveys, a client case study and progress on client protection issues. This is an ambitious requirement in which partners may struggle to do well.45 Depending on their level of research capability, partners are likely to require technical assistance, which SHIFT is planning to provide in the zfuture. Capturing development impact of access and usage – how clients use services and benefit from the results – is also to be planned.


Tag: Challenges Monitoring and Evaluation Demographic change Inclusive economic growth Poverty Reduction Technical Support

31.

Measurement of systemic change and ‘top down’ measurement of results, including an ‘Adopt-Adapt: Respond-Expand (AAER) model and contribution analysis of SHIFT’s role in such results is planned. This has been initiated for the two relatively successful CF projects so far (Greenlight and LVBP), and will be developed further for CF and other interventions as they reach a point of contributing to systemic change (i.e. expansion or replication). While SHIFT captures through its quarterly and annual review details of activities (i.e. number of workshops, events, participants, etc.), there is no measurement of how this has translated into policy results (new policies, regulations, etc.). The interlinkages and contributory effects of policy and advocacy activities at ASEAN level are not yet being tracked.


Tag: Monitoring and Evaluation Results-Based Management Advocacy Policy Advisory

32.

M&E systems ability to capture data on women – and how is the data used to mainstream and enhance focus on women for SHIFT’s programme benefits?

A focus on women monitored by sex-disaggregated data is a core theme and reporting requirement throughout SHIFT’s programme; however, programme reporting on women’s access cannot be fully accurate. There are information system challenges for FSPs in general, and for digital FSPs and MNOs in particular, in capturing changes in women’s agency (whether the woman herself is using a phone, whether she manages the financial account, whether she manages the business for which a financial service is provided). This is an issue that applies to the financial services sector in general.

For Window 1 (alternative energy with financial mechanisms), partners did not capture the gender of customers, but relied on an assumed percentage, ranging from 50% to 30% women, which may or may not hold in practice.46 Solar lighting is seen as an investment for the household, benefitting all in the family, women and men, girls and boys. Pushing for joint ownership may have been the appropriate option here – and follow up47 on who in the family takes responsibility for recharge or mostly benefits would provide the gender dimension.


Tag: Gender Equality Gender Mainstreaming Gender Parity Women's Empowerment Monitoring and Evaluation Data and Statistics

33.

In financial services, microcredit in general is traditionally measured in terms of outreach to women clients regardless of whether the loan is used for a woman’s business or for another business in the family. Banking and other services, including digital accounts and mobile based services, do not usually have sex disaggregated data on ownership. With the use of digital financial services for remittances, it will be equally problematic to report details whether of the sender or the receiver. This gap in gender reporting is a feature within existing banking systems and the Management Information Systems (MIS) for digital services and requires a technical solution.48 LVPB (CF Window 2) was asked by SHIFT to adjust their data collection system so as to comply with programme reporting standards – it is not clear how successful this is.


Tag: Challenges Gender Parity Women's Empowerment Monitoring and Evaluation Micro-credit

34.

Questions of women’s actual use of a service and women’s agency – as required by a women’s empowerment perspective – go well beyond quantitative access and usage reporting. SHIFT’s MRM aims to capture this through field level research. This will apply to future investments to determine whether or not women ‘lead the business’ for which credit is extended. Depending on country context, there may be a case for distinguishing businesses managed by women by themselves from those involving women and men in ‘joint’ business management within a family, though the woman may be the credit recipient.

SHIFT is encouraging CF partners to adapt their information systems (if this is technically and financially feasible), or MRM may directly apply different research tools to address these issues. Related research and an emphasis on sex-disaggregation of data will help to mainstream and enhance focus on women as users of services and business managers. Nevertheless, practical challenges in reporting have to be recognised.

SHIFT requires CF partners to report sex disaggregated data on their own staff at management and field levels, as well as agents. This is also a component under GEF work with FSPs in Myanmar. This will be an important gender aspect to consolidate and document across FSPs.


Tag: Challenges Gender Equality Gender Mainstreaming Gender Parity Women's Empowerment Monitoring and Evaluation Results-Based Management

35.

How are data collected at granular level aggregated and standardised

The MRM collects and consolidates quarterly reports from CF partners into the dashboard for each project, including qualitative feedback and quantitative data agreed as part of the PBA. The definition and data on access and usage is formulated differently for each type of financial service depending on the focus of the project (ranging from single time credit for an energy investment to training of agents for digital financial services). However, the simple aggregation of the numbers for reporting outreach results for the entire fund whilst useful for headline reporting, blurs these different elements of financial inclusion.


Tag: Monitoring and Evaluation Data and Statistics

36.

Degree of adaptation - Is internal decision-making based on evidence generated by data/MRM system

Through MRM, significant information is generated from partner reporting, other monitoring (such as by the Frankfurt School of Management on the Challenge Fund), and staff BTORs. MRM information along with feedback from partners and other stakeholders (e.g. trainees, workshop participants) is well documented and acted upon. There are a number of examples of MRM information and data being used as evidence in internal decision making for programme adaptation and improvement, particularly in the improved management of the Challenge Fund, and planning for future training.

The CF reporting has been consolidated 6-monthly50 and used for internal decision making to guide immediate follow up with CF partners and to adapt CF guidelines. For example, low levels of usage of the CF supported e-wallet reported by LVPB has led to specific attention to this issue. CF reports for Windows 1 and 2 have been further scrutinised through a qualitative analysis to draw out the patterns of experience and direct lessons across all the projects.51 The early experience has led to better understanding of process issues and improvements in the management.


Tag: Monitoring and Evaluation Partnership

37.

Governance – the SHIFT Board

The programme has a well-established governance and management structure. The Board of SHIFT has final responsibility for programme decisions and strategy. The Board keeps oversight of the programme and periodically reviews the progress of outstanding issues and approves the work plan and budgets and investments. The Board has a strong mix of engaged members, chaired by the Director 

of UNCDF’s FIPA, with direct donor (DFAT) representation and staff as observers. The members represent an impressive combination of ASEAN level financial sector direct experience and global engagement around financial inclusion from different perspectives, including market systems development, private sector investment and fund-raising strategies. Since 2017, there is (long overdue) representation from a regulator in the region (Cambodia).55

Board meetings are held twice or thrice a year, usually once in person with varying attendance. Minutes and ET interviews with a few of the Board members indicate informed and engaged review of issues with specific and clear action points to the SHIFT team, (all of whom attend the meetings). The agenda is usually packed within half or one day. Reporting to the board is quite ‘heavy’ – both in number and length of documents and complexity of some of the power- points. One Board member’s view of the reporting was that it is ‘heavy and dense with information buried in technical detail”.

 


Tag: Monitoring and Evaluation Oversight Project and Programme management Trade and Development

38.

Evidence of complementarity and convergence between the four pillars of SHIFT. What factors would ensure better alignment of the four pillars to the common goal of the SHIFT programme?

Evidence of complementarity between the four pillars is emerging in relation to specific themes – particularly women’s financial inclusion, remittances – with data & research providing an effective resource contributing to policy and to the CF and beginning to link in to learning & capacity building.

Data and research is an underlying hub which, since 2015 has been able to focus on the core themes in financial inclusion (demand and supply side surveys, women’s access to financial services, women’s role in MSMEs, remittances across ASEAN countries), and through different types of research, analysis and documentation has contributed across the other three pillars: to policy initiatives (at country level57 and regionally – through ASEAN and AFI) and the formulation of national gender strategies; data analysis has provided the basis for learning and capacity work (in Cambodia in 2016/17)58; and linked effectively to the CF through: a) case study analysis of both market and policy issues,59 b) systematic qualitative analysis of Windows 1 and 2, and c) using remittance research to showcase fintech partners in relevant forums attracting the interest of investors and regulators.60 Ongoing work is showcasing the innovation of successful CF investments and contributing to product development.61


Tag: Women's Empowerment Innovation Programme Synergy Inclusive economic growth Policy Advisory Data and Statistics

39.

In implementation, SHIFT activities are reaching a degree of convergence in one country, Cambodia, where the learning and capacity building pillar started in 2017. All activities are at an intermediate level and yet to show results in influencing the market systems effectively. In the other countries due to a combination of factors – UNCDF’s existing role in the MAP process, varying country receptiveness, existing donor focus and lack of additional funding until recently - the focus has been on the challenge fund. As CF Windows 2 and 3 get off the ground and demonstrate results, already happening in one country, Vietnam (LVPB), this is likely to provide a strong base for policy influence and replication, as well as linking in to data and learning.SHIFT has begun to present its different activities on each pillar in relation to the specific themes (women’s financial inclusion, remittances, clean energy), showing how each theme involves: engagement at the policy level (regional/ASEAN and country), training of women (as clients or agents), and related data and research.62 This seems to be an appropriate way of showing the inter-linkages between the four pillars and the inherent potential and scope for follow through on all types of activities e.g. the need to have a strategy to follow up on dissemination of research.


Tag: Women's Empowerment Implementation Modality Capacity Building Policy Advisory Data and Statistics

40.

Are progress and intermediate results meeting expectations of key stakeholders?63

SHIFT’s work so far is appreciated for technical support at the regional ASEAN level, and in beginning to bring together developed players for fin-tech (in Singapore and Malaysia) to link into the (rural) areas for remittances in the CLMV. In terms of policy, results are yet to emerge at the regional level, although SHIFT’s work on gender and remittances have created a momentum, which sustained engagement can translate into policy outcomes. The presence of full-time dedicated staff for the policy pillar in all four countries would enable more effective partnership building. Other outputs so far are below the expectations of key stakeholders – the Challenge Fund, the time involved in supporting the NFIS (in one country), and an apparent lack of progress on vulnerable populations that include disability and indigenous people. Inadequate funding, well below the amount planned, is seen as one reason for limited application of all the four pillars. CF partners report issues in managing the PBAs, but a number of partnerships have worked well and are now strengthening within the region.


Tag: Relevance Partnership Policy Advisory Technical Support

41.

Is progress in line with expectations of UNCDF board, DFAT, country government partner and management of SHIFT/FIPA?

Key stakeholders have realistic expectations and appreciation of the gradual process involved for SHIFT in building a market systems development approach, through implementation of the different pillars in different contexts within ASEAN, starting with a focus on two countries – Cambodia and Myanmar. Relatively slow progress is attributed to the fact that whilst SHIFT was able to mobilise some additional funding, a substantial funding gap remains.


Tag: Relevance Resource mobilization Country Government International Financial Institutions UN Agencies Trade and Development

42.

The Cambodia government partner has been very pleased with SHIFT support on the NFIS, and indeed since 2017 has been represented on the SHIFT board. Though expectations remain on the need for better contextualisation, and for future capacity building.

 


Tag: Effectiveness Country Government Capacity Building

43.

Degree to which partners are satisfied with management response to implementation bottlenecks

Feedback from CF grantees indicates a high degree of fit of projects with organisational priorities, effective communications with the SHIFT team and follow up support, but some issues in terms of realistic planning.65 In case of PBA milestones, approvals for extension or reduction in targets is reported to take at least 4-5 months, which affects implementation and reporting. In a (Window 2) project in Cambodia that is now suspended, SHIFT supported the grantee in making suggestions to adapt product terms and in facilitating linkages for digital finance, but appeared “totally inflexible” on the planned targets; (for SHIFT a proposed change in business model was non-negotiable).66

Within the management team, there is a sense that in the absence of adequate funding across the four verticals, the deliverables in terms of market systems development to date may have been limited. Nevertheless, it has been possible to build on various opportunities. And with new initiatives and partnerships that have taken shape in the past year, the programme is well placed to build on its experience and momentum so far, to take forward the work on identified themes (gender, fin-tech, remittances, big data).


Tag: Challenges Human and Financial resources Implementation Modality Partnership Project and Programme management

44.

Partnerships at the regional, national and local levels have clearly delineated roles; ownership of programme by implementing partners; level of partner engagement and contribution and how efficiently it is working

Partnerships have shown good engagement and contribution, with some variation depending on the focus of the partnership – and the bandwidth of the SHIFT team to pay attention to the partnership.67


Tag: Efficiency National Regional Operational Efficiency Ownership Partnership

45.

Effectiveness

Is SHIFT helping FSPs develop and scale viable financial inclusion business models for men and women?

The Challenge Fund investments have supported diverse ideas for innovation in products and service delivery. Lessons from difficulties in earlier windows are being incorporated into current Windows (for remittances and SMEs) and these appear to have considerable potential to develop and scale, though a longer time frame may be required to address the constraints. Services are expected to be used by both men and women, but sex disaggregated data may not be automatically/easily tracked in digital services. The business models to some extent include new employment opportunities for women.

 


Tag: Effectiveness Women's Empowerment Business Model Innovation Monitoring and Evaluation Service delivery Inclusive economic growth

46.

FSP capacity and motivation to understand customer needs, analyse data and design customised/tailored financial products and solutions which address existing gaps in the market

The Challenge Fund investments have supported diverse ideas for innovation in products and service delivery. Under the CF windows, a market orientation was largely lacking under Window 1 with inadequate assessment of market conditions, of competing models and suitability/affordability for the target markets, as well as the financing mechanism being introduced. This gap was one of the reasons for the high rate of failure. Following the experience with Window 1, SHIFT has increased consultation with key market providers on services development needs for different sectors, through specific and effective market research. Though doing market research well remains an issue.

Under Window 2, FSPs have shown stronger market-oriented motivation and awareness, in identifying gaps and understanding needs, and to design customised products. Though, not necessarily to good effect, even when international agencies support product design74 – which underlines the challenges involved in fully understanding market conditions, interpreting data and being able to design products appropriately in response. An FSP partnership with an on the ground research agency under Window 2 is seeking to apply data collected through a financial diary process for product design.75 The development of an app for gamification of financial literacy has been able to base the design on some limited market research of the target market.76


Tag: Challenges Programme/Project Design Trade and Development Data and Statistics

47.

How far is data helping make mid-course corrections to project design and implementation mechanism to make it more relevant to the local context?

Monitoring data of the CF projects is helping to make mid-course corrections to programme design and implementation mechanisms to make it more relevant to the local context. This is notable in the case of the fast growing LVPB project where the data showed after the first quarter (end 2016) usage of under 30% and lower. Another interesting example is the Family+ Savings and remittance product of Amret in Cambodia. Lack of progress in uptake of the product led to specific recommendations from the SHIFT monitoring team to adapt the product to market feedback.


Tag: Relevance Implementation Modality Monitoring and Evaluation Programme/Project Design Data and Statistics

48.

Degree of innovations in products/services/delivery channels to support deeper financial inclusion (access and usage)

The Challenge Fund aims to support private sector innovations to support deeper financial inclusion (access and usage), providing public capital to support private businesses to initiate activities and services with an expected social return (serving development goals). The fund has supported research- based product design, use of technology for service delivery and for financial literacy, together with new partnerships between FSPs and other players to support design and implementation, targeting low income and rural women as a key market and the involvement of women as agents or employees.77 (Table 6, next page).

The degree of innovation in Window 1 lay partly in the alternative energy technologies themselves (Clean Start’s focus), but also in the funding mechanisms (instalment payments linked to a PAYGO technology and an attempt to involve women as marketing agents (selling to women as intended users). For ongoing CF projects under Window 2 or about to start under Window 3, the Table below highlights the range of innovations planned. One project (LVPB), has already rapidly expanded and other banks are reported to be taking the cue.78 Three projects are work in progress and are likely to be successful (WAVE money, BRAC and AWBA/1 stop) albeit somewhat delayed. A key reason why LVPB achieved success in Round 2 was due to it being an established bank, with a substantial customer base across the country, which it could leverage in terms of reach with the digital app. Unlike, LVPB, most other CF grantees were new products / services / firms operating in less mature markets.


Tag: Innovation Service delivery Inclusive economic growth Inequalities Poverty Reduction Technology Women and gilrs

49.

The extent to which the CF investments will translate into ‘deeper’ financial inclusion is not clear – due to the lack of outreach profiling so far for poverty or rural outreach, and sometimes even for gender. Nevertheless, the organisations in Myanmar are committed to rural expansion, and BRAC works exclusively with women clients targeting low income households. The digital convenience store 1-stop stated that given the market opportunities they would eventually have moved to rural areas. The SHIFT grant combined with the partnership with AWBA microfinance has facilitated an immediate push to rural areas, though they anticipate lower levels of rural business compared to the major urban centres where they are currently focused.79 Businesses that the ET visited linked to Window 2 (in Myanmar) represented lower income/small businesses as well as higher level businesses.


Tag: Impact Rural Urban Monitoring and Evaluation Inclusive economic growth Inequalities Micro-credit Poverty Reduction Data and Statistics

50.

Degree of use of technology in FSPs to reach scale

The use of technology in partner FSPs is high and responding to the developing opportunities of FinTech. For example, the bank project (LVBP) represents the application of mobile technology; Wave money and 1-stop depend on mobile technology for expansion; in the BRAC project, mobile phones are being effectively used for quick data capture for the financial diaries.

Projects under Window 3 for remittances will if successful represent a significant formalisation of existing fund flows from migrants into the region, including into rural areas. These projects rely on banking technology with the introduction of mobile money and e-wallets. However, there is a question mark over the time required to overcome the physical, infrastructural and literacy constraints. 80


Tag: Rural Innovation Technology

51.

Are the capacity building initiatives, policy and advocacy activities resulting in changes within organisations and eventually in the market?

Capacity building and policy & advocacy activities are happening primarily in Cambodia and to a certain extent in Myanmar. These are at a preliminary stage, have led to increased awareness, but are yet to result in changes within organisations. SHIFT’s gender initiatives within WC-FINC and AFI are seen as having made a significant contribution to policy guidelines, to influence policy makers to have gender targets. At FSP level, mainstreaming of gender is yet to address a number of practical and normative issues – including how to design a genuinely woman centred product.


Tag: Impact Women's Empowerment Inclusive economic growth Advocacy Awareness raising Capacity Building Policy Advisory

52.

Changes in business environment and regulatory framework – new policies etc.

SHIFT has laid the groundwork (through MAP – Finscope, government engagement, CJAR, data training, WC-FINC, AFI advocacy work), for changes in the business environment and regulatory framework in Cambodia where SHIFT has worked directly with the regulator since 2015, helping in the development of the NFIS and Action plan.81 Final approval by the government is still to take place to lead into policy formulation and implementation.

ET interviews with the regulator and other stakeholders in the Cambodian government indicate a strong commitment to the relevance of financial inclusion as a key component of the financial sector development strategy (drawn up with the ADB), including client protection and financial education. Work on the NFIS so far has helped the regulator to emphasise consumer protection policies, to bring attention towards customer financial literacy programmes (already started through the school curriculum), and to strengthen the process of licencing for payments.


Tag: Impact Relevance Rule of law Education Inclusive economic growth Policy Advisory

53.

Capacity of regulator and policy actors to enable and foster deepening of markets for the poor

Our interviews in Cambodia indicate significant awareness and motivation for financial inclusion by the regulator, based on the consultative process for developing the NFIS. The country government partner – that is the NBC, together with the Ministry of Planning – appreciates the engagement and support from SHIFT, the fact that the financial sector strategy now has more than just a brief paragraph on financial inclusion, and that there has been significant value addition in building the regulator’s understanding of what financial inclusion entails. The Finscope demand side data supported by the deep dive data analysis conducted by SHIFT (data hub) – with a focus on the themes of gender differences, agriculture and clean energy – is recognised as an important tool for evidence based decision making and for tracking change over time. At the same time, there is a concern about the time involved in the strategic planning process and some issues on some of the technical content, the need for better contextualization and moving from diagnosis to specific policies and “practical concrete solutions”. The interviews, thus, suggest some hesitation in figuring out specific policies, and the need for direct guidance to do so.82

SHIFT is seen to have made a significant contribution in providing technical inputs for AFI members’ Denarau Plan of Action in 2016 and the subsequent gender guidelines.83 Within ASEAN, the gender gap in access to financial services, economic participation and institutional representation is less marked than in other regions. Nevertheless, going beyond these numbers to discuss issues of opportunity and agency for women, is getting traction.84 However, the ability to report with sex- disaggregation even on access numbers remains a challenge in the banking sector and particularly in tracking digital finance transactions.


Tag: Gender Equality Women's Empowerment Strategic Positioning Country Government Inclusive economic growth Awareness raising Data and Statistics National Institutions

54.

FSP reported strengthening in product development, outreach, monitoring, data analysis etc.

In Cambodia there have been trainings for data analysis using Finscope data and big/transactions data. Data training of staff of four MDIs (representing 50% of microfinance in Cambodia) took place in 2016- 17 with the purpose of building FSP staff capacity in data management, analysis and use. This has had some effect in terms of building motivation and understanding, with some improved capacity in applying basic Excel (dashboard) tools and thinking about data dashboards. However, it has not yet led to capacity improvement and application for outreach analysis or product development, which will require a more sustained and systematic support.85


Tag: Capacity Building Data and Statistics

55.

Extent of mainstreaming of gender within FSP institutional practices. Gender responsiveness of financial services/products i.e. designed for both men and women, catering to their varying needs

Mainstreaming of gender within FSP institutional practices is well integrated with CF requirements, reflecting the central theme of women’s financial inclusion along with reporting requirements for sex disaggregated data (on women’s access, and women agents). A general information system gap in the finance sector is that some FSPs (particularly digital FSPs), may only be able to report sex disaggregated data based on assumptions of the proportion of outreach – which means that gender is not really mainstreamed. Similarly, HR reporting systems are not usually geared to regular reporting of the gender profile within their staff. SHIFT is trying to address both these issues, for example by using the name of a user as a proxy, or asking an investee to add gender as a field in the software.


Tag: Gender Mainstreaming Gender Parity Inclusive economic growth Data and Statistics

56.

Even when FSPs have sex-disaggregated data they do not know how to apply this information. The learning and capacity building work around data in Cambodia started to address the understanding of the relevance of gender analysis for product development in 2017 but will need follow up and more specific guidelines to make a difference.86

These have emerged as some of the issues in the application of the institutional self-assessment gender tool under the Gender Equity Fund in Myanmar. The tool (adapted by the SHIFT team from existing examples87), and intended as a public good, was tested with 10 FSPs, 6 MFIs, 3 Fintechs, and 1 bank in early 2018. Initial feedback from the MFIs involved is moderately positive, as they wait to see whether there will be practical follow up to help in product design and marketing for women. The bank is enthusiastic – looking to blaze a trail of suitable products for women SMEs – though as yet not clear on what a ‘woman-centric product design’ will include and raising practical questions on how to define a woman owned/managed business, when businesses often involve both women and men in the family.88 The rising stars of mobile money in Myanmar (WAVE money – which is notwithstanding a CF grantee for the development of a financial literacy app targeting women - and Truemoney) prefer not to be distracted from their focus on commercial growth, and point to the fact that anyway women are active economic players and ‘likely to be part of their core business’.89


Tag: Gender Mainstreaming Gender Parity Innovation Monitoring and Evaluation Service delivery Inclusive economic growth Poverty Reduction

57.

The GEF CF project with BRAC will be a test case of designing a product for women clients (loans for SMEs) drawing on the detailed information collected through the financial diaries. WAVE money has recently completed the design of an an app with gamification around financial literacy which was tested out with women in Yangon (starting with garment workers, engaging more with housewives and students). The app was launched in June 2018, so take up has only just started with WAVE money aiming to report sex disaggregated use, since the app was designed to capture gender at the time users sign up.90

The upcoming Window 4 for product development for women managed MSMEs will provide a strong opportunity for design of products and services to address the needs of business women. The opportunity to include a focus on women’s needs as senders or recipients of remittances under Window 3 will need attention.


Tag: Global Environment Facility fund Women's Empowerment Innovation Service delivery Technology

58.

Extent of crowding in of investments due to SHIFT investments resulting in overall market development

It is premature to comment on this. So far, for two CF projects Green light (Window 1) and LBVP (Window 2) there is evidence of other investors taking up similar business models.


Tag: Impact Sustainability Business Model Trade and Development

59.

4.3.1Are investments improving markets for the poor?

At mid-term, SHIFT’s activities under different pillars are yet to come together to influence market systems and will need information on the socio-economic profile of the client base to be able to demonstrate that is it contributing to the financial inclusion of low income populations. The evidence of whether outreach is to the BoP requires definition and is about to be tested by SHIFT through a client survey now being introduced with the current successful CF investment.

 


Tag: Impact Inclusive economic growth Poverty Reduction

60.

Is the integrated market development approach resulting in new products, approaches and services [addressing the needs of the low income and vulnerable populations]?

At mid-term, SHIFT’s activities are yet to come together for tangible influence on market systems. At the level of policy and advocacy, SHIFT’s work in Cambodia to develop the NFIS has prepared the ground to address the scope for inclusion of different segments of the population.

New products, approaches and services are emerging under the Challenge Fund (Window 2), mainly in the area of digital services (wallet or mobile based payments, financial literacy app) with some developments in solar energy with paygo technology (in Myanmar). Further innovations are expected with projects to formalise remittance services into CLMV under Window 3 (supported by appropriate in- country regulation and licensing91); and to develop credit services for women MSMEs under the future Window 4.

While investments under CF have led to SHIFT meeting the client outreach targets for access, usage is still low, and most of the customers are users of wallets and e-payment applications, rather than loans/credit or savings products. In terms of employment effects of the program, the CF investments have led to employment that includes women as digital payment agents (LVPB, AWBA), or field researchers (BRAC/LIFT).


Tag: Impact Service delivery Micro-credit Technology Trade and Development Women and gilrs

61.

Market depth and breadth in intervention areas catering to the bottom of the pyramid client base. Extent to which new investments/products/services are targeting the BoP

So far, there has been no measurement of the economic level of the people being served, but SHIFT is now implementing a mechanism to do this with clients of LVPB (the largest, most successful CF project so far). However, the design and analysis need attention to be able to capture effectively whether indeed the BoP are being served.92 Similar approaches will be rolled out across investments as they reach momentum. On digital products & services, the debate is wide open as to whether low income, vulnerable communities (particularly women) do have access to the technology or the infrastructure, including the literacy level to make it work. The country gender strategies aim to address these issues. Nevertheless, given the infrastructure challenges, it seems likely that the e-remittance services being introduced will become useful over a 3-5 year (or even longer) period.93


Tag: Impact Gender Equality Poverty Reduction

62.

1.1Likely impacts

SHIFT has exceeded programme targets for direct outreach numbers through CF projects, reaching so far (2018, Q1) nearly 2 million clients, compared to the 2020 target of 1 million. However, 99% of these numbers is accounted for by just one CF project, in a country with relatively low SHIFT activity so far. The product - e-wallet along with financial education – reflects in a 38% proportion of women, below the 65% SHIFT target. (The proportion of women targeted for credit is more than 90%). More significantly however for performance, usage of the e-wallets is very variable and low. It is clear that outreach effects are particularly high in terms of access to digital services/e-wallets, and, as discussed earlier (in 4.2.2) significant differences in financial inclusion are masked by standardised aggregation of reporting across the different type of projects.


Tag: Impact Education Inclusive economic growth Technology

63.

The new projects to be funded under CF window 3 for remittances will involve digital payments and will therefore have potential for very high outreach. Increased usage of services is however a challenge which SHIFT is trying to address. The below target outreach to women reflects the barriers to women’s use of digital services. Reporting usage by women is also a challenge, though SHIFT is working to encourage DFS providers to add a field in their software for this, and to use names as a proxy.


Tag: Innovation Micro-credit Technology Women and gilrs

64.

It is too soon to speculate on indirect outreach. The target numbers (5 million) are based on assumptions that bear further examination. The proposed ‘top-down’ and ‘bottom-up’ approach to measuring this in future seems appropriate but will depend on reliable secondary information at the country level on financial inclusion trends.

At the market/policy systems level, SHIFT initiatives (at country and regional levels) have led to an increase in awareness, understanding and motivation, mainly in Cambodia, with work under way to support change in the capacities of FSPs and regulators in future in Myanmar and potentially in Vietnam.


Tag: Impact Implementation Modality

65.

There is considerable potential for income and livelihood changes from the use of new financial services if they are successfully developed and rolled out. The 4th CF window for MSME products for women, can be well placed to add this gender focus. To the extent that new services do specifically address women’s needs and target women effectively, as clients or as agents, in the process can address the barriers that women face (mobility, asset ownership etc.). This will represent a significant move to changing social norms. The likelihood of including vulnerable or marginalised populations is not yet being tracked – and will depend on such populations gaining access to digital assets and infrastructure.


Tag: Impact Jobs and Livelihoods Poverty Reduction Vulnerable Women and gilrs

66.

Regional collaboration to address common constraints and deepen markets is being strengthened through SHIFT’s role within WC-FINC and AFI. Linked to work under CF Window 3 and data work, SHIFT has been very successful in being able to position itself as a UN agency helping to bridge the gap between innovation in fintech to support financial inclusion in LDCs and rural areas. The collaborations and engagement across different banking systems are taking time to work through but will strengthen the ability to address common constraints. SHIFT is well placed to facilitate this, as a regional player, and to bring the lessons to the regional platforms – AFI and WC-FINC.


Tag: Impact Regional Innovation Partnership Inclusive economic growth Technology

67.

Ownership, access, use and regularity of use of financial services by low income and women clients

The programme has reached 2 million (Q1 2018) clients, over achieving the 1 million target by 2020 but below the 65% target for women, with 38% reported. It is to be noted that this proportion may not be fully accurate (for the reasons noted earlier) and indeed may be overstated.

Table 7 presents the reported data, based on which the following figure (Figure 5) shows the skew in the results to one country (Vietnam – a country with relatively low SHIFT activity) and, so far, one type of financial service (digital payments/wallets). This is down to the fast expansion of one project – LVPB - which already had a couple of hundred thousand e-wallet accounts at baseline before the CF project started, and easily exceeded the PBA target outreach within the first quarter of starting.


Tag: Impact Ownership Service delivery Women and gilrs

68.

However, performance of these services in terms of usage (SHIFT applies the sector definition – agreed by the GSMA, Groupe Speciale Mobile Association - of at least 1 transaction in the quarter) is low and variable. The highest reported usage for LVPB (based on cumulative accounts) was 29% (26% for women) in the last quarter of 2017, but since then has fallen to under 10%. For VfM analysis (4.2.1) we applied the average annual usage rate which during 2017 works out to 17%.

Comparing the target numbers for different financial products/mechanisms, it is clear that outreach effects are particularly high for digital services/e-wallets. The new projects to be funded under CF Window 3 for remittances will involve digital payments and will therefore have potential for very high outreach. Active usage of digital services is however likely to be a challenge.


Tag: Effectiveness Innovation Service delivery Technology

69.

The market systems approach anticipates not only direct impact but also indirect impact through replication of successful Challenge Fund projects as well as policy and capacity building effects. There is no indirect impact yet. But this seems an appropriate place to look at the assumptions behind the overall target which is used in SHIFT’s public documents: the 6 million target by 2020 which includes 5 million through indirect impact. This was estimated in the ProDoc at around half of anticipated growth in access to financial services across CLMV, Indonesia and the Philippines, drawing on baseline Findex data, assumptions for the likely achievement of MAP Roadmap targets by 2020 and population data for each country.94 The assumptions appear to relate not only to SHIFT’s work but also to UNCDF’s – and the major proportion of the 5 million (62%) is anticipated from Myanmar.95 The basis for assuming attribution to SHIFT of half the overall expected numbers in each country is unclear.

The suggested approach to measuring this in future – through ‘top down’ and ‘bottom up’ data analysis combined with contribution analysis – seems appropriate, and the best option.96 Effective measurement will depend on being able to track the replication effects of CF projects as well as following through on use of data and product development by FSPs associated with the learning activities. ‘Top-down’ analysis will require reliable secondary data for comparison of trends over time.


Tag: Impact Capacity Building Policy Advisory Data and Statistics

70.

How effective the policy has been to structure and design innovative financial services that would promote Markets for the Poor (M4P)

At midterm, SHIFT’s work and current planning has reached a stage where results at the market systems level can begin to emerge. The fund facility, data and learning interventions are beginning to show tangible implications for development of the ecosystem and policy, supported by constituency building particularly in Cambodia, and also in Myanmar, Vietnam and across ASEAN countries too, notably with the new remittance channels being supported under the Challenge Fund (Window 3).

Along with policy and regulatory reforms, new models or products developed by FSPs will need to achieve scale. Organisations need to develop their understanding of low-income customers’ needs in order to better target them and develop tailored products. This will be possible through continued support to FSPs on the data and learning fronts along with financial literacy of clients.


Tag: Effectiveness Rule of law Innovation Service delivery Policy Advisory

71.

In Cambodia, SHIFT’s support to NBC on developing the NFIS has helped to reinforce formulation of client protection guidelines, mainstreaming of financial literacy through the education department, reforms in payment gateway systems and permission to an FSP (AMK) for cross-border remittance services. The current thinking on deposit insurance may help to build trust and confidence of low income people in the formal financial systems. However, information systems to track outreach to the poor will also need to be put in place. In Myanmar, ongoing CF investments under the GEF, together with continuing policy engagement by UNCDF, and new data interventions with FSPs will also help to promote M4P. Digital payments under CF investments will potentially lead to qualifying credit scores, improving credit availability for micro-entrepreneurs.


Tag: Innovation Education Inclusive economic growth Jobs and Livelihoods Micro-credit Technology

72.

The mobilisation of cross-country players to support remittances into CLMV from other countries of ASEAN under CF Window 3 will also in future, potentially link economies across ASEAN and (if successful) make a contribution to policy changes (enabling cross-border solutions), and therefore a regional impact on markets for the poor. SHIFT’s engagement in ASEAN working groups (for financial inclusion and beginning this year for payments and settlements) is helping to reinforce this interlinked market focus.


Tag: Impact Resource mobilization Regional Trade and Development

73.

Level of change in capacities of FSPs, regulators and mainstreaming of gender in roles – measured through awareness, knowledge, skillsets etc.

At the market/policy systems level, SHIFT initiatives (at country and regional levels) have led to an increase of awareness, understanding and motivation, with work under way to support change in capacities of FSPs and regulators in future. The change has so far been particularly in Cambodia (through SHIFT’s direct work in the MAP process), indirectly (SHIFT providing some technical support to UNCDF) in Myanmar97 and to some extent in Vietnam through initiation of the MAP process. Policy work is seen to reinforce current initiatives by the regulator in Cambodia (client protection, financial education, licensing of payment service providers) and provide an evidence base for future policy making (through the MAP process, Finscope survey and initial data training). In Myanmar, SHIFT has the potential to influence evidence-based policy formulation and mainstreaming of gender through participation in the Working Group for Economic and Political Participation of Women, drawing on data pillar work; and through the GEF institutional assessment and capacity building work with FSPs. The next step to achieve changes in specific capacities is likely to require more targeted learning work – not only through training, but through a mentoring approach.

While stakeholders are acknowledging the importance of data-based decisions, and appreciate SHIFT work in this area, more systemic use and institutionalisation of these processes, will need continued and long- term support from SHIFT, through ongoing training interventions with FSPs and FIs.


Tag: Impact Global Environment Facility fund Gender Mainstreaming Gender Parity Rule of law Awareness raising Capacity Building Policy Advisory Data and Statistics

74.

Income and livelihood changes of clients due to access and use of financial services

There is considerable potential for income and livelihood changes from the use of new financial services if they are successfully developed and rolled out. The two GEF projects in Myanmar look promising for direct impacts – clients are expected to benefit from credit that is customised to their business needs (BRAC), and store keepers will have an additional source of revenue as agents for mobile payments (AWBA-1-stop). Digital financial services – already expanding with LVPB under Window 2 and planned for remittances under Window 3 – are expected to support more stable cash flows and savings for users (direct impacts), whilst also providing new employment opportunities for people who work as agents, (as an additional source of income to an existing business or as a main income source).

Credit focused products – as planned under Window 4 for MSMEs – will if successful support the growth of small businesses at different scales. These will be direct impacts from the successful Challenge Fund projects which may also lead to indirect wider impacts if replication takes place. Until client economic profiles are conclusively tracked, it is difficult to assess whether income and livelihood changes will accrue to the targeted low income or vulnerable populations.


Tag: Impact Global Environment Facility fund Jobs and Livelihoods Micro-credit Poverty Reduction

75.

Change in social well-being, norms and empowerment of women and vulnerable/marginalised populations

SHIFT has effectively promoted the involvement of women as field staff or as agents for digital services under different CF projects.98 These are often women, but, as noted,99 registering the woman as an agent does not automatically make her the one running the business. The same will apply for credit to MSMEs that may or may not be run by women.

To the extent that new services do specifically address women’s needs and target women effectively, as clients or as agents, as remittance senders or receivers, CF investments can address the barriers that women face (mobility, asset ownership etc.), representing a significant change in social norms. The 4th CF window for MSME products for women, is well placed to add this gender focus combined with an understanding of women’s role in different businesses. However, financial inclusion and change in social well-being of vulnerable or marginalised populations is contingent on such populations gaining access to and usage of digital assets and infrastructure.


Tag: Impact Women's Empowerment Inclusive economic growth Jobs and Livelihoods Poverty Reduction Technology Vulnerable

76.

Level of change in regional collaboration addressing common constraints and deepening markets

Level of change in regional collaboration addressing common constraints and deepening markets is expected to be visible with successful roll out and implementation of the CF Window 3 cross border remittances projects. Interest has been shown by regulators and financial institutions of Singapore and Thailand to work with FSPs in CLMV countries. The collaborations and engagement across the different banking systems are taking time to work through but will strengthen the ability to address common constraints. SHIFT is well placed to facilitate this, as a regional player, and to bring the lessons to the regional platforms – AFI and WC-FINC. The adoption of the Denarau Action Plan is also a successful demonstration of addressing the common gender constraints in these markets.


Tag: Regional Partnership

77.

Sustainability – To what extent are impacts at the policy, FSP and end client level likely to sustain over time?

Future developments and sustainability will depend on further mobilizing of funding and investment across the four pillars and countries of focus. Whilst it is too early to be able to identify increased institutional capacity or policies across various levels of government, private sector and regulators, we see a strong sense of ownership on the part of the regulators in Cambodia where SHIFT’s direct policy work has so far focused, as well as acknowledgement of the importance of good data (from the data training), but a perceived need for further funding and focused TA support – for effective capacity development. At end client level, successful challenge fund projects are likely to expand and be replicated, though the record of success is relatively low. SHIFT has done well to build constituencies at all levels, within the region, particularly in Cambodia and Myanmar, and now beginning in Vietnam, whilst supporting UNCDF work in Laos. Partnerships – continuing and new – at the regional & country level provide a good platform for continuing engagement influencing policies and building ownership for sustainability – at all levels.


Tag: Sustainability Resource mobilization Ownership Sustainability Policy Advisory National Institutions Private Sector

78.

Funding by country governments on future work on financial inclusion would represent ‘skin in the game’ to support the sustainability of implementation of NFISs. Thailand provides a good example of this where the National Bank is now funding the next Finscope Survey. In CLMV ongoing bilateral funding from the World Bank and the ADB for financial sector development has been the norm over a number of years. It may therefore require a shift in the mind-set of these governments to pay for the type of support that SHIFT can provide.


Tag: Government Cost-sharing Inclusive economic growth

79.

In Cambodia where SHIFT has been directly active in promoting the NFIS, the sense of ownership by the NBC is high. The regulator and government players are committed to taking forward key requirements for financial inclusion and are (at the time of this mid-term evaluation) about to move to the stage of translating MAP into policy formulation and monitoring. Nevertheless, they are looking for funding to do this.100 In Myanmar, where UNCDF is leading on MAP, the regulator had the same request.

Whilst Australia has made a substantial contribution, adding no cost extensions to allow time for results to take place, additional funding to the substantial contribution made from Australia is critical for the continuity and future success of SHIFT. Programme engagement with external agencies in the ASEAN region has strong potential for future sustainability, through the partnerships, the recognition and funding – albeit somewhat limited – from UN Pulse Lab Jakarta (linked to the data pillar, and a new challenge fund in Indonesia), and from MAS and Visa (linked to CF work on remittances).


Tag: Sustainability Ownership Project and Programme management Sustainability Donor Inclusive economic growth

80.

Demonstration of increased institutional capacity at various levels of governments to formulate policies would partly represent SHIFT’s sustainability. At the mid-term stage, given the level of development so far, it is too early to identify increased institutional capacity or policies across various levels of government, private sector and regulators. Interventions are yet to scale up, and there are capacity gaps in policy and data and expressed need from stakeholders for continued support.

 


Tag: Sustainability Rule of law Capacity Building National Institutions

81.

Self-reported ability of all relevant market ecosystem actors to sustain changes without UNCDF support

The sustainability of a successful Challenge Fund project may be about to happen with one Window 1 investment in Myanmar (Greenlight) but is too soon to see for Window 2. The potential for replication and sustainability will depend on effective practical application (product design, marketing and systems support), addressing infrastructural constraints and ownership by the FSPs.

SHIFT has built constituencies at all levels, within the region, and continuing engagement with different partners particularly at the regional level – the ILO, AFI and recent links with Visa, MAS, UN Pulse, UNDP and UNESCAP – provides a potential platform for future replication, linking country level data and activities to specific programmes (remittances and migration, youth employment and financial services).


Tag: Sustainability Regional Partnership Sustainability

82.

Government created institutions to carry forward SHIFT interventions of schemes modelled on SHIFT

Through the MAP process for NFIS, in Cambodia, SHIFT has supported the government to create internal mechanisms (an inter-ministerial working group), for approvals and buy-in. Whilst a secretariat within the NBC is likely to be set up later this year for further implementation. In Vietnam, for developing the NFIS, SHIFT is facilitating setting up of a similar mechanism for building consensus and ownership of the process. In Myanmar, such institutions have been created through the UNCDF country initiative and demonstrate the potential of the MAP process for in-country ownership – but also the risks and challenges depending on the country institutional context.103 Such risks will need to be addressed as work moves forward in Cambodia and Vietnam – and elsewhere in the region.


Tag: Sustainability Ownership Sustainability Country Government

83.

The development of SHIFT national country gender strategies (for each of CLMV) has followed a similar consultative and evolutionary process. SHIFT has been able to leverage several complementary programmes and partnerships at the regional, global and country levels and supplemented UNCDF FIPA in two countries (Myanmar and Laos).


Tag: Gender Equality Gender Mainstreaming Partnership Programme Synergy

84.

In terms of the validity of the instrument itself, the CF was an appropriate instrument to trigger private investment in high-risk markets; reaching untapped or hard-to-reach markets; and unlocking hidden innovation (e.g. Greenlight in Myanmar), which would otherwise not be pursued from a purely commercial perspective. However, success of CF is contingent on a range of factors, which are discussed in SHIFT’s context later in detail.


Tag: Innovation Private Sector

85.

Key public policy themes (women’s role in financial inclusion, remittances, MSME), were identified by SHIFT through consultation with country governments (mainly two countries – Myanmar and Laos where UNCDF was already active), financial institutions and related stakeholders. These themes were picked up in the successive windows of the Challenge Fund

The use of CF in CLMV is also found to be relevant. The CF as an instrument is appropriate in stimulating private-sector led innovation for addressing social problems and poverty alleviation in underdeveloped markets with a lack of risk capital, such as in CLMV.

 


Tag: Women's Empowerment Country Government International Financial Institutions Inclusive economic growth Jobs and Livelihoods Poverty Alleviation Trade and Development Private Sector

86.

SHIFT’s integration of women into all features of its programme followed a similar process, drawing on an initial ThinkShop in early 2015, attended by key stakeholders and representatives internationally (including UNWomen) and from the ASEAN region.11 The strategy12 reflected a grounded analysis of the realities of policy, culture, demography and the market economy affecting women’s access and agency, and underlines the scope for leveraging the drivers of change across SHIFT’s four pillars of engagement.


Tag: Gender Mainstreaming Demographic change Inclusive economic growth Inequalities Policy Advisory

87.

The overall programme was well designed to adopt a regional approach for policy, data and research dissemination and coordination so as to support linkages across the ASEAN region, and to create potential for south-south cooperation that could benefit the smallest financial markets.16


Tag: Programme/Project Design South-South Cooperation Data and Statistics

88.

Partnerships with various stakeholders of the region were important to build a wider network for leveraging of resources and tools. For example, the first round of the CF was initiated (very quickly) by linking up with UNCDF’s Clean Start programme which provided the major component of funding and managed the selection process.


Tag: Resource mobilization Partnership

89.

Policy and advocacy work at the country level follows the MAP approach developed by UNCDF in South Africa, drawing on the Finmark Trust and Cenfri for country wide demand surveys (the Finscope surveys) and supply side mapping. For this, SHIFT has focussed its work in Cambodia – a country where there was no previous UNCDF FIPA programme, whilst playing more of a supporting/following role in Myanmar where there was such a programme and similarly to a lesser extent given local constraints in Laos.

In CLMV, the international institutions - World Bank, the ADB, and the IMF – are already engaged in financial sector planning and development. SHIFT has targeted its activities to build on these initiatives, in Cambodia and in Vietnam (which does not have a UNCDF of


Tag: Harmonization Partnership Programme Synergy International Financial Institutions UN Agencies Advocacy Policy Advisory National Institutions

90.

The programme interventions are aligned with the ToC and the strategies adopted reinforce each other towards achieving the programme objectives and results.

The Evaluation Team notes that while the TOC does well to present what is a complex programme with a range of activities at different levels, there are a few issues in tracking programme results:

  • The (2-way) interlinkages between regional activities and country activities are not clearly delineated.
  • Outputs and outcomes are not clearly demarcated to track the pathways (from different activities) at country level.
  • Tracking of policy related processes is set out but has not been introduced so far.

Tag: Challenges Monitoring and Evaluation Theory of Change National Institutions Regional Institutions

91.

The differences emerging in CF projects so far relate to the following:

  1. Credit projects (such as instalment payments and PAYGO for clean energy under Window 1, enterprise lending about to start with BRAC under Window 2) report access in terms of being a borrower, and ‘usage’ in terms of being an outstanding borrower. However, for clean energy the issue is that once the credit is paid off, the energy customers are still benefiting from the service. So usage of the financial service will go down while client benefits should continue. In terms of the financial service, the relevant metrics are the access to credit and on-time payment of the instalment. And for business loans, whether a borrower borrows again

Tag: Energy Micro-credit

92.
  1. Digital financial services often involve agents who handle the transactions. Reported data on outreach has combined both agents and customers (for instance AWBA, Myanmar – this has now been revised to cover only the agents, for whom the question of ‘usage’ will need to be defined e.g. in terms of the number and volume of transactions for each agent; reporting for LVPB covered in the next note – also reports on agents and merchants along with customers)

Tag: Innovation Service delivery Technology Data and Statistics

93.
  1. E-wallets have emerged as the fastest growing project, with large numbers of e-wallets opened but with a very low and variable usage, adopting the definition for an active account as at least one transaction every quarter.49 Given the ease of opening an e-wallet, usage, not access, appears to be the significant measure for financial inclusion.

Tag: Innovation Inclusive economic growth Inequalities Technology

94.
  1. Financial education for product uptake The completed viewing of the app designed by WAVE money will be different from uptake of WAVE products. Both are relevant and important, and expected to be kept separate, once the project moves to be able to report on this.

Table 7 in section 4.4. (page 43) disaggregates the standardised data for CF projects which have results so far.


Tag: Education Inclusive economic growth

95.

Lessons from the data training in Cambodia (in 2017) are leading to a more adaptive approach in planning for simpler forms of analysis in Myanmar.

Policy and advocacy activities (at ASEAN level and at country level) and feedback on those appear to be less well documented or consolidated. For example, it is not clear what the programme response was at the time to repeated challenges52 mentioned in the NBC quarterly reports from 2016.53

Recognising the need to have a systematic feedback loop within the management team, periodic meetings on particular themes/issues have been introduced.54


Tag: Project and Programme management Advocacy Capacity Building Policy Advisory

96.

It is the Board that takes the final decision on CF grants based on presentation of the proposals, and the board minutes record the informed queries and conditions that the Board raises. However, members of both the Board and the IC commented on the lack of overlap between the two. Another gap for the Board is the lack of a member to represent the end-client perspective. This is a requirement in the TORs according to UN practice and has been raised in Board meetings.56


Tag: Challenges Oversight

97.

A major challenge still to be addressed, and this was raised by Cambodia stakeholders (Regulators and FSPs) is how data analysis can in practice directly inform regulation and strategic action at the policy level, or product and service development at FSP level. Whilst the theory of using data evidence sounds right, actual examples and practice still need to be worked out, documented and disseminated. It will be a significant achievement for SHIFT going forward if it is able to demonstrate this.


Tag: Challenges Policy Advisory Data and Statistics

98.

SHIFT’s role in cross country networks such as AFI and ASEAN’s WC-FINC, is seen as providing an important and stable basis for influence across the region and for leveraging the data and research work. Publications are seen as well received through different fora – though these are still to be taken forward into policy. Sustained consensus building at the country and regional levels is likely to lead to concrete policy influence.

Whilst the Challenge Fund has received most of the SHIFT funding, the results so far have fallen short of expectations, apart from one investment. Nevertheless, it is also recognised that lessons have been learned for subsequent rounds, and the board has been careful to ensure that lessons are followed through. The new Window 3 for remittances is seen as a considerable achievement, building on data and research work, drawing in stakeholders across the region, and seeking to leverage banking and financial technology for the benefit of rural, low income populations, including women.


Tag: Trade and Development Policy Advisory National Institutions Regional Institutions

99.

A specific area of shortfall noted is an apparent lack of attention to people with disability. For the key donor, people with disability and indigenous people represent an important category of the ‘vulnerable’ even though not specifically mentioned in SHIFT’s ProDoc. Whilst an approach to include such people is a specialist area, there may be ways that SHIFT can do so, via an FSP partnering with an NGO (as for example WAVE Money in Myanmar).64


Tag: Challenges Programme/Project Design Disabilities Civil Societies and NGOs Vulnerable

100.
  • The AFI partnership has worked well to link in to SHIFT gender expertise for the development of practical gender guidelines for country regulators in its global membership. There is scope for SHIFT to draw on its country experience within ASEAN to share that too with AFI membership.

Tag: Gender Mainstreaming Women's Empowerment Partnership

101.
  • The CGAP partnership was an important step in building an effective system for monitoring a market systems development approach in financial inclusion, contributing both to a strong system for SHIFT and to a practical testing and demonstration of the CGAP guidelines.68
  •  

Tag: Monitoring and Evaluation Partnership Trade and Development

102.
  • SHIFT has partnered with the ILO’s TRIANGLE in ASEAN projects69 to share and contribute to the analysis of Finscope data for publications on remittances, including one (forthcoming), which the ILO is aiming to take forward in its advocacy work in Laos.70 The ILO also provided some inputs to the guidelines for CF window 3 (remittances). The ILO would have been interested to know how their inputs were used and they could have been further involved in the CF selection process (as a subject specialist invitee on the IC). Given the ILO’s work with low income migrants in ASEAN, their insights into the experience of migrants (difficulties in accessing banks, low digital and financial literacy) and effective infrastructure in the region (remittance portal that includes a cost comparator of remittance products, migrant worker resource centres), this is an opportunity to link in beyond data sharing. This partnership needs more attention – albeit on both sides – to make it more efficient.7

Tag: Partnership UN Agencies Migration Data and Statistics

103.
  • SHIFT is open to engage with UNDP wherever possible. The UNDP coordinator indicated appreciation of the role of UNCDF in Myanmar financial inclusion, though there is no on-the-ground engagement. Apart from this, SHIFT’s partnership with the UNDP Youth Lab in Bangkok was based on data sharing for analysis of youth entrepreneurship and financial inclusion. It has led to a policy note for governments in the region, with SHIFT staff attending some of the conferences organised. This partnership too needs attention to explore the synergies with SHIFT’s own policy work and gender focus in CLMV.

Tag: Gender Equality Partnership UNDP Accelerator Lab UNDP Management UNDP management Policy Advisory

104.

Not surprisingly, given the financial constraints, the SHIFT team is devoting more energy to partnerships that involve a degree of funding support alongside technical/sector expertise. Recent partnerships have involved both – UNPulse (big data management, contribution to upcoming CF in Indonesia), and the Monetary Authority of Singapore and VISA (contributing to CF Window 3 for remittances).


Tag: Partnership Project and Programme management

105.

Nevertheless, programme interventions in the data sphere in Cambodia have led to important discussions on the usefulness of generating insights through data, including big data, and helping shape policies and decisions. Across the stakeholders, the acknowledgment of its importance, as well as the momentum generated, is an indication that the data work will be carried forward towards eventual ownership within the market actors (policymakers, and FSPs via a link now being established with the Cambodia Microfinance Association). Additional TA support is likely to be a requirement.101 This is a useful experience base as SHIFT extends the data work to support FSPs in Myanmar.102


Tag: Ownership Data and Statistics

Recommendations
1

/

2

Evaluation Recommendation or Issue 1: The programme team is currently working with consultants to assist in resource mobilisation. The focus here should be to raise resources for the non-challenge fund pillars, ideally to try to generate funding on long term commitment, alongside shorter commitments (such as MAS and VISA towards a particular challenge fund). The programme should also explore building synergies with ongoing donor-assisted programs on financial inclusion (such as World Bank, ADB, DFID). Additional funding commitment would enable SHIFT to strengthen staff resources – particularly for policy/advocacy, and learning - so as to have a more balanced approach across pillars. For specific activities – training, research, surveys - partner payment contributions should also be considered. This would serve to focus partner attention to optimise what they are paying for. 

3

Evaluation Recommendation or Issue 2: Limited funding commitments will require an adjustment in expectations with a more focused programmatic approach. Within continuing resource constraints, we recommend more focus on policy/advocacy and learning/capacity building (regulators, FSPs) with a continuing emphasis on gender and inclusion, identifying specific policy measures that support Markets for the Poor (M4P), and helping to create an appropriate environment for pro-poor private capital. Geographically, current activities should continue regionally and in Cambodia and Myanmar, with extension to Laos and Vietnam as opportunities arise.

4

Evaluation Recommendation or Issue 3: To support more sustainable funding, we recommend alternative types of funding under the CF. An important rationale for companies to participate in a challenge fund appears to be access to risk-willing capital, rather than the access to subsidized or ‘free’ money and hence repayment need not be an issue, particularly if the project is successful. Alternatives to funding grants are: conditional loans of interest, returnable capital, or a venture capital mechanism.

5

Evaluation Recommendation or Issue 4: To improve effectiveness of the Challenge Fund, it is important to 1) set realistic targets within a practical time frame. (A move away from grant funding is also likely to inject more realism into partner ambitions); 2) ensure investments in digital services build in core financial products (credit, savings, insurance). While digital payments (wallets, etc.) are a vehicle for formalisation and employment, it is the core products that will contribute to SHIFT’s goals of reduced vulnerability and poverty reduction; and 3) arrange for technical assistance – ideally on a cost sharing basis - to support partners to address some of the implementation bottlenecks (such as action-oriented market research – which is very difficult to do well) as well as robust reporting (see below).

6

Evaluation Recommendation or Issue 5: SHIFT’s present thinking under the learning and capacity building pillar, of building a pool of master trainers and tie-ups with institutes/networks (being planned with the Cambodia Microfinance Association) is in the right direction and should be explored across the focus countries. There are opportunities to develop synergies with respect to curriculum, sharing facilities and costs with other agencies (multilaterals, and others). For example, IMF training centres for capacity building of policy makers, possible tie-up with ILO on Save ASEAN remittance portal, migrant resource centres. This will help in building both capacity and sustainability and provide a link to capacity building of consumers and consumer financial literacy, including financial literacy training programmes for migrant workers.

7

Evaluation Recommendation or Issue 6: Disaggregate overall outreach/target reporting to different financial services to capture the different elements of financial inclusion (digital transactions, specific financial services – credit, savings, insurance; agent employment, financial education). For each type of service ‘usage’ needs to be differently and appropriately defined, and gender issues and targets are likely to vary. Gender related measurement needs programme follow up, to define women’s role in household enterprises, and particularly for digital services, to support relevant fields in the MIS and analysis. The CF projects are an excellent opportunity to do this and to engage specifically on defining the relevant parameters of products and services that have a ‘gender focus’.

8

Evaluation Recommendation or Issue 7: Update the definition of poor, vulnerable to reflect a benchmark appropriate to the focus countries. A useful guideline is to target the ‘poorer 40% of households’. The Poverty Probability Index (PPI http://www.povertyindex.org/ppi-country) is a recognised tool to measure this and the PPI country design documents include useful country benchmark data. SHIFT has done well to introduce this to LVBP (successful CF partner in Vietnam) but needs to follow up to ensure the index is applied correctly and reporting is robust. The PPI is out-of-date or not available for some countries. SHIFT – policy/advocacy(and UNCDF) are well placed to mobilise interest and policy support to update the PPI and extend its use (including as part of the Finscope surveys). MRM needs to work closely with its partners (with TA and review of the MIS – software) to measure outreach to the poor – and to rural areas

9

Evaluation Recommendation or Issue 8: CLMV, particularly Myanmar, has sizable ethnic minority populations. Between 4-6% of the population in CLMV are disabled. The issue of including people who are indigenous (ethnic minorities) or have a disability, deserves attention and the programme is well placed to explore opportunities through under the CF – through potential FSP partnerships with NGO programmes, and engaging with the regulator as part of NFIS.

10

Evaluation Recommendation or Issue 9: The Theory of Change does well to provide a framework for a market systems approach, focusing on expected systemic changes (intermediate outcomes) which lead to improved services used by increasing numbers of both men and women (long-term outcomes). Reporting needs to focus on usage, not access, and direct achievement in usage under the fund facility should be seen as short-term/intermediate outcomes, to be differentiated from longer term outcomes (from replication, and other activities). The long term targets of 5 million (in the ProDoc) need review – since the assumptions include contributions to financial inclusion in 6 countries. UNCDF/SHIFT should directly promote Finscope data as the more reliable data set to measure changes over time (compared to Findex).

11

Evaluation Recommendation or Issue 10: MSME development and financing is also important to address the programme objectives of increased income and employment which have not been adequately focused upon so far. While the fourth round of CF will be on MSME financing, however, challenge fund interventions may not be sufficient to bring systematic changes unless it is supported through policy level changes for incentivising banks and FSPs to provide micro enterprise level financing (like mandatory priority sector lending in India), for women headed firms.

1. Recommendation:

/

Management Response: [Added: 2018/12/14] [Last Updated: 2018/12/14]

Overall comments: Management would like to thank the evaluation team for the evidence, analysis and insights presented in the Mid Term Evaluation Report. SHIFT ASEAN found the recommendations provided useful to further build on the achievements of the programme to date, and to strengthen programme implementation moving forward.

One of the goals of the SHIFT ASEAN program is to influence a systemic and market system change, through its combined interventions. The evaluation acknowledges that this process is complex and takes time, and it has found that “SHIFT is yet to show results of its activities in terms of influencing market development”.  The program team assessment is that one can actually see initial signs of market influence, for example the programme’s digital financial service business model investment in Viet Nam is being used as a basis for the development of another bank’s digital financial service offerings. Also, through SHIFT’s ASEAN level policy engagements, peer-pressure mechanisms among regulators, have triggered faster decision making on financial inclusion roadmap approval and implementation processes. The program team recognizes this is an ambitious goal and is committed to undertake contribution analysis on such changes in the coming years of the program.

It should also be noted that sex-disaggregated data for DFS and all SHIFT ASEAN interventions is tracked; SHIFT ASEAN has worked with its project partners to develop systems, which collect sex-disaggregated data and also provide trainings on the importance and analysis of sex disaggregated data for product development. Further data collection methods can be further improved by the programme.

SHIFT ASEAN notes the evaluations assessment that the program design is relevant to the context of ASEAN countries and that SHIFT ASEAN’s policy efforts have been effective at the ASEAN level (through WC-FINC) and has brought together the developed players for fin-tech (in Singapore and Malaysia) and linked them with rural areas for remittances in CLMV.

Overall, UNCDF management  accepts the recommendations of the evaluation, with some caveats as indicated in the detailed response, and will take necessary measures to strengthen the SHIFT ASEAN programme.  The evaluation’s recommendations will also be applied as UNCDF designs the next phase of its programming in the ASEAN under its new integrated “Digital@UNCDF” strategy. Other evaluation exercises scheduled for SHIFT SAARC and the PFIP program in the Pacific will be used in a similar fashion to design successor programming in those regions under the new global Digital@UNCDF strategy. This more integrated programmatic approach will enable UNCDF to develop a more integrated and strategic approach to financial inclusion and digital finance across Asia, with strong cross learning and shared experiences between the three sub-regions.      

Key Actions:

2. Recommendation:

Evaluation Recommendation or Issue 1: The programme team is currently working with consultants to assist in resource mobilisation. The focus here should be to raise resources for the non-challenge fund pillars, ideally to try to generate funding on long term commitment, alongside shorter commitments (such as MAS and VISA towards a particular challenge fund). The programme should also explore building synergies with ongoing donor-assisted programs on financial inclusion (such as World Bank, ADB, DFID). Additional funding commitment would enable SHIFT to strengthen staff resources – particularly for policy/advocacy, and learning - so as to have a more balanced approach across pillars. For specific activities – training, research, surveys - partner payment contributions should also be considered. This would serve to focus partner attention to optimise what they are paying for. 

Management Response: [Added: 2018/12/14] [Last Updated: 2020/12/20]

Management Response: Agreed, with the objective to build the foundation of future programming in the ASEAN under the integrated Digital@UNCDF strategy. 

Key Actions:

Key Action Responsible DueDate Status Comments Documents
1.1. The evaluation recommends to focus on raising resources for non-challenge fund areas of work, looking at both long term commitments alongside shorter term commitments. The programme has already initiated mapping donor interests and developing concept notes for non-Challenge fund areas of work. The programme is in the process of finalizing hiring of a fundraising support consultant and has set fundraising targets as part of Staff performance assessments.
[Added: 2018/12/14] [Last Updated: 2020/01/15]
SHIFT Programme Manager 2019/12 Completed The fundraising consultant joined in mid-March and is currently further mapping funding opportunities for the innovation fund as well as other programme pillars. Additional funds have been raised for the data and learning hub of the SHIFT programme. These funds were sourced both from UNCDF flexible non-core funding, as well as from international donors such as FMO and DFID (the latter as a parallel contribution coming from UNCDF Myanmar programme). These funds will be used to take forward lean data surveys in partnership with 4-5 local FSPs in Myanmar, Cambodia, Vietnam and Lao PDR. The aim of the surveys is to supplement partner monitoring and cost-effectively measure SDG impacts for SHIFT Digital finance innovations. History
1.2 The evaluation also recommends that SHIFT ASEAN explore synergies with ongoing donor-assisted programmes. Discussions and coordination with other development agencies such as ADB in Cambodia and Vietnam, and World Bank in Vietnam, have been explored. SHIFT ASEAN will be scale-up in 2019 its discussions to find synergy areas with World Bank, ADB, DFID, IFC, USAID, BMGF, Canada, and DFAT country posts and share action points for implementation to the SHIFT ASEAN Board.
[Added: 2018/12/14] [Last Updated: 2020/01/15]
SHIFT Programme Manager 2019/12 Completed 1.2 The evaluation also recommends that SHIFT ASEAN explore synergies with ongoing donor-assisted programmes. Discussions and coordination with other development agencies such as ADB in Cambodia and Vietnam, and World Bank in Vietnam, have been explored. SHIFT ASEAN will scale-up in 2019 its discussions to find synergy areas with World Bank, ADB, DFID, IFC, USAID, BMGF, Canada, and DFAT country posts and share action points for implementation to the SHIFT ASEAN Board. 31/12/2019 SHIFT Programme Manager Completed SHIFT has explored synergies with ongoing donor assisted programmes, particularly in Myanmar. Currently, these initiatives have centered around FMO and DFID work in Myanmar, and the potential of their support for the development of gender smart financial products and data analytics. This partnership is currently being established and further implementation will be taken forward in the second half of 2019. Synergies with both VISA and Mastercard are also being explored within the framework of the data hub work. History
1.3 The evaluation suggests that for specific activities i.e. trainings, partner payments and contributions should be considered. This is currently being undertaken with cost sharing and in-kind support from partner institutions in providing venue space, staff time, but most importantly institutional commitments made to improve product development based on learnings from the trainings. With recent first level of trainings imparted on data analytics and value proposition for the same realized, the programme will explore alternate contribution arrangements in terms of funds, customer data analytics etc. further in 2019.
[Added: 2018/12/14] [Last Updated: 2020/06/30]
SHIFT Data and Learning Hub 2020/06 Completed 1.3 The evaluation suggests that for specific activities i.e. trainings, partner payments and contributions should be considered. This is currently being undertaken with cost sharing and in-kind support from partner institutions in providing venue space, staff time, but most importantly institutional commitments made to improve product development based on learnings from the trainings. With recent first level of trainings imparted on data analytics and value proposition for the same realized, the programme will explore alternate contribution arrangements in terms of funds, customer data analytics etc. further in 2019. 30/06/2020 SHIFT Data and Learning Hub Completed Using a cost sharing model, SHIFT has taken forward a variety of workshops including; - one on NFIS M&E which was co-financed by regional central banks and the MAP UNCDF programme; - product development and data analysis trainings taken forward with Myanmar FSP were co-financed by the local UNCDF initiative on Gender Transformative Finance; and - trainings taken forward with the State Bank Viet Nam (SBV) on data collection and analysis were co-financed by the SBV who provided in kind support and data. Overall, no specific model for this has been established, and the trainings as well as how they are co-financed is determined on a case by case basis. History
3. Recommendation:

Evaluation Recommendation or Issue 2: Limited funding commitments will require an adjustment in expectations with a more focused programmatic approach. Within continuing resource constraints, we recommend more focus on policy/advocacy and learning/capacity building (regulators, FSPs) with a continuing emphasis on gender and inclusion, identifying specific policy measures that support Markets for the Poor (M4P), and helping to create an appropriate environment for pro-poor private capital. Geographically, current activities should continue regionally and in Cambodia and Myanmar, with extension to Laos and Vietnam as opportunities arise.

Management Response: [Added: 2018/12/14] [Last Updated: 2020/12/20]

Management Response: Agreed

Key Actions:

Key Action Responsible DueDate Status Comments Documents
2.1. Due to the funding constraints it is agreed that the expectations of the programme should be considered both in terms of activities and targets. SHIFT ASEAN is working to update its programme targets based on funding and activity outreach scenarios. Also, a SHIFT ASEAN 2019 strategy focused on policy/advocacy, linked with regulator capacity building activities, has been developed in order to guide and focus the programme implementation around a specific and strategic set of activities for future work. Specifically, SHIFT will continue to engage with ASEAN working committees by providing technical assistance and advocating on specific issues at the ASEAN Working Committee for Financial Inclusion meetings. SHIFT will also undertake bi-lateral advocacy and support based on its bi-lateral agreements with Cambodia and Viet Nam, which will feed-into regional policy mechanisms/platforms. Apart from the existing commitments, partnerships with existing programmes, UN agencies are being leveraged to expand SHIFT ASEAN’s objectives in Lao and Vietnam. With engagement with ASEAN policy bodies and existing partnerships with regulators in CLMV, the work on policies/advocacy related to gender and inclusion will be further accelerated.
[Added: 2018/12/14] [Last Updated: 2020/01/15]
SHIFT Programme Manager 2019/12 Completed Recently a policy expert has been onboarded to take the established policy/advocacy strategy forward, this document has now been finalised. SHIFT has continued to engage with the WC-FINC, and the most recent meeting in February focused on the status of the committee action plans and deliverables and had a particular thematic focus on digital financial services and anti-money laundering and counter terrorism financing (AML/CFT). UNCDF contributions included the presentation of available resources and TA to ASEAN member states on digital finance; UNCDF’s global digital strategy and UNCDF’s FinTech investment. Based on this, the Bank of Indonesia requested insights from the FinTech Innovation investments to support their regulatory decision making and sandbox. WC-FINC members also requested that UNCDF presents the Findex dashboard that UNCDF has been developed for ASEAN countries. For the first time, UNCDF participated in the WC-PSS meeting presenting on UNCDF’s remittance research and key findings, as well as learnings and insights from UNCDF’s remittance investments. Furthermore, SHIFT has continued to engage CLMV regulators based on bi lateral agreements. This includes the National Bank of Cambodia with whom high level government approval of the financial inclusion roadmap is pending, and the State Bank of Vietnam with whom SHIFT is working to provide guidance on the financial inclusion survey, which will also feed into regional policy platforms. In both cases support was focused on highlighting women’s financial inclusion, amongst other topics. SHIFT has also begun to expand the scope of its work regarding target populations who lack access to finance, particularly persons with disabilities, targeting specific interventions and helping to train financial service providers to better meet their needs and close this access gap. History
4. Recommendation:

Evaluation Recommendation or Issue 3: To support more sustainable funding, we recommend alternative types of funding under the CF. An important rationale for companies to participate in a challenge fund appears to be access to risk-willing capital, rather than the access to subsidized or ‘free’ money and hence repayment need not be an issue, particularly if the project is successful. Alternatives to funding grants are: conditional loans of interest, returnable capital, or a venture capital mechanism.

Management Response: [Added: 2018/12/14] [Last Updated: 2020/12/20]

Management Response: Agreed 

Key Actions:

Key Action Responsible DueDate Status Comments Documents
3.1 UNCDF through its LDC Investment Platform has been implementing blended finance, debt and guarantee instruments. SHIFT ASEAN will connect its existing and future pipeline generated through Challenge fund and other direct partnership mechanism with the LDC Investment platform for accessing risk-willing capital. This will be dependent on the quality of the company’s business model and UNCDF fund availability. Also SHIFT ASEAN has recently implemented the Indonesia Challenge fund, where linkage with investors is one of the outputs. The programme will expand its collaboration with investor community to connect its private sector partner to alternate sources of capital and technical assistance. Specifically, SHIFT will publicize the business models through its communication channels, we will showcase the business models in relevant events and introduce the companies with impact investors through the organization of an impact investment workshop with the investor community, This will result in crowding-in of additional risk capital.
[Added: 2018/12/14] [Last Updated: 2020/06/30]
SHIFT Programme Manager, UNCDF Headquarters and SHIFT Board 2020/06 Completed With respect to the LDC Investment Platform, the SHIFT challenge fund will act as a pipeline for the platform along with all the other UNCDF programmes. The LDC platform is building an investment fund which is currently being raised with Bamboo. From the SHIFT programme, the TYM fund, INFOCORP and Zigway are being connected to the platform. The Indonesia innovation fund was launched at the end of 2018 with the selection of 6 business ideas targeting women’s MSME. The selected companies were offering technical assistance through partnerships established with ORACLE and Deloitte and during the application process they received mentoring from VISA. In an effort to link these business models with additional scale financing and venture capital, a few are selected to present their business models at conferences and meetings including the UNGA. Some were also presented to the Monetary Authority of Singapore and its pool of associated investors. Furthermore, selected business models from SHIFTs most recent innovation funding round, were invited to present their models at the Singapore Fintech Festival, as well as at the Cyberport Venture Capital forum in Hong Kong. The FinTech business models supported by SHIFT in Indonesia were also showcased during the 7th WC-FINC meeting. Finally, investment briefs have been completed for the latest innovation window winners, and further briefs are currently being completed for the other SHIFT investees. History
5. Recommendation:

Evaluation Recommendation or Issue 4: To improve effectiveness of the Challenge Fund, it is important to 1) set realistic targets within a practical time frame. (A move away from grant funding is also likely to inject more realism into partner ambitions); 2) ensure investments in digital services build in core financial products (credit, savings, insurance). While digital payments (wallets, etc.) are a vehicle for formalisation and employment, it is the core products that will contribute to SHIFT’s goals of reduced vulnerability and poverty reduction; and 3) arrange for technical assistance – ideally on a cost sharing basis - to support partners to address some of the implementation bottlenecks (such as action-oriented market research – which is very difficult to do well) as well as robust reporting (see below).

Management Response: [Added: 2018/12/14] [Last Updated: 2020/12/20]

Management Response: Agreed 

Key Actions:

Key Action Responsible DueDate Status Comments Documents
4.1 Targets are now being reviewed given different project timeframes and PBAs are being developed accordingly. This process is already being taken forward in SHIFT ASEAN’s most recent Indonesia Challenge Fund Window. As a result in December 2018 targets of the Indonesia Challenge Fund Window were reviewed and aligned with expected and market growth rates. Technical assistances partnerships were also developed and will linked with the specific business models in the first quarter of 2019. This same approach will be taken for the Asia-Pacific Women Enterprise Innovation Fund in 2019, to result in appropriate targets and technical assistance which will allow for sustainable business models.
[Added: 2018/12/14] [Last Updated: 2019/07/09]
Fund Facility Coordinator, Monitoring and Evaluation 2019/03 Completed No further progress, although the same balanced approach that was established with the Indonesia Innovation Fund Window will be followed for the PBA for the grantees of the Asia-Pacific Women Enterprise Innovation Fund. This will be done in June 2019. History
4.2 The future windows since the Mid-Term Evaluation was conducted has already focused on having business models that go beyond payment solutions. The recent Indonesia Challenge Fund Window has selected some business models that addresses the enterprise credit and capacity building issues. The upcoming Asia-Pacific Women Enterprise Innovation Fund will also take the non-digital payment solutions into consideration and prioritization based on specific country market ecosystems and needs. This will be integrated into the challenge fund term paper and will be launched in the first quarter of 2019. This will result in a wider range of business models and solutions being considered under the challenge fund round.
[Added: 2018/12/14] [Last Updated: 2019/07/09]
Fund Facility Coordinator 2019/06 Completed Although this action has been marked as completed, it is important to report that SHIFT has succeeded in moving beyond business models that just focus on payments. The Indonesia Innovation Fund Window grantees address a range of issues including improved credit scoring systems, digitalizing payments systems, SME inventory-based loans and financial literacy. With respect to the Asia-Pacific Women Enterprise Innovation Fund, shortlisted business models include developing the first digital lending platform of Myanmar, alternative collateral lending models, and peer to peer lending models amongst others. History
4.3 The programme has initiated technical assistance delivery under its Indonesia Challenge fund window and future windows. Also the programme has received approval from board on partner ranking selection method of contracting that will enable technical assistance components to be delivered much more effectively. The programme will integrate technical assistance in all its Challenge fund windows in 2019 and will develop at least 2 partnerships with relevant technical assistance providers by June 2019. SHIFT ASEAN has also created a partnership with global corporates with expertise in digital financial solutions, for their selected team members to act as technical assistance provider and mentors.
[Added: 2018/12/14] [Last Updated: 2019/10/18]
Fund Facility Coordinator 2019/09 Completed 4.3 The programme has initiated technical assistance delivery under its Indonesia Challenge fund window and future windows. Also the programme has received approval from board on partner ranking selection method of contracting that will enable technical assistance components to be delivered much more effectively. The programme will integrate technical assistance in all its Challenge fund windows in 2019 and will develop at least 2 partnerships with relevant technical assistance providers by June 2019. SHIFT ASEAN has also created a partnership with global corporates with expertise in digital financial solutions, for their selected team members to act as technical assistance provider and mentors. 30/09/2019 Fund Facility Coordinator Completed Technical assistance delivery to the Innovation Fund grantees has continued, and its success in terms of grantee satisfaction and resulting performance has meant that this model was expanded to the Asia-Pacific Women Enterprise Innovation Fund grantees which were announced at the end of June 2019. As such, broader portfolio wide Technical Assistance partnerships with VISA, FMO and Oracle are currently being established and are at different levels of development. Current technical assistance was focused on the Indonesia Innovation Fund Window, and has included receiving mentorship support from VISA for the development of the application pitches, as well as support and in kind access to ORACLE cloud based technical infrastructure support platforms, and management expertise from Deloitte. VISA has continued to partner with SHIFT in this and provided mentorship support for the applicants to the Asia-Pacific Women Enterprise Innovation Fund. History
6. Recommendation:

Evaluation Recommendation or Issue 5: SHIFT’s present thinking under the learning and capacity building pillar, of building a pool of master trainers and tie-ups with institutes/networks (being planned with the Cambodia Microfinance Association) is in the right direction and should be explored across the focus countries. There are opportunities to develop synergies with respect to curriculum, sharing facilities and costs with other agencies (multilaterals, and others). For example, IMF training centres for capacity building of policy makers, possible tie-up with ILO on Save ASEAN remittance portal, migrant resource centres. This will help in building both capacity and sustainability and provide a link to capacity building of consumers and consumer financial literacy, including financial literacy training programmes for migrant workers.

Management Response: [Added: 2018/12/14] [Last Updated: 2020/12/20]

Management Response: Agreed

Key Actions:

Key Action Responsible DueDate Status Comments Documents
5.1 In 2018, SHIFT ASEAN’s capacity building and technical assistance activities have effectively become a cross-cutting support mechanism to the entire SHIFT ASEAN programme. Training, technical assistance and learning related activities are now being mainstreamed across all SHIFT ASEAN workstreams i.e. data support trainings with both policy makers and FSPs, and targeted FSP technical assistance support through the Challenge Fund. The building out of SHIFT ASEAN’s capacity building activities will be continued throughout the remainder of the SHIFT ASEAN programme. While it took time to build up the content and approach to SHIFT ASEAN’s capacity building activities, they have been now been improved, refined and ready to be expanded. Training content has been revised to emphasize the link between data and product development, based on the customer journey. The training has also been made more practical and an improved pre-training survey will ensure a more tailored approach, which should create a more enabling environment, for the development of targeted financial products. Regarding partnerships, SHIFT ASEAN has established new partnerships that are aimed at strengthening the programme’s capacity building activities, specifically with the Cambodia Microfinance Association (CMA), similar partnerships will be explored in 2019. As suggested, the programme will explore partnerships with IMF, ILO and others on various aspects of capacity building on segmental and sectoral aspects.
[Added: 2018/12/14] [Last Updated: 2019/07/09]
SHIFT Programme Manager, and Data and Analytics Specialis 2019/06 Completed SHIFT training activities have continued to expand, effectively becoming a cross cutting support mechanism for the entire SHIFT ASEAN Programme. Mentorship training on pitch development has now become a standard feature of the Innovation fund application process, allowing grantees to develop their capacity in this and improving the interaction between SHIFT, the Investment Committee and the grantees, allowing for better insights into project ideas and improving the selection process overall. TA during the implementation phase of the Innovation fund, is also now being piloted with the Indonesia Innovation fund grantees with the aim of creating a standard TA programme, which will be complemented with tailor made interventions (i.e. what SHIFT is currently exploring with LVPB). This is aimed at increasing financial products usage. Based on the Customer Journey Action Research (CJAR) training taken forward in 2016, SHIFT has now developed a broader training programme based on the comments of participating trainees which has become less technical but has strengthened the pre training preparation with a survey, the practical scope of the training, as well as the link between data, product development and the customer journey. This has resulted in the creation of new training modules with facilitation guidelines that have been used in Myanmar and Cambodia to train FSPs. The training modules were designed with a training of trainers programme in mind, and currently partnerships for this are being developed. For example, the training in Cambodia was executed in partnership with the Cambodia Microfinance Association (CMA), with whom we are also discussing opportunities for partnership expansion. Training modules aimed at CLMV regulators have also been expanded, and recently SHIFT developed a secondment programme at UNCDF in Bangkok for SBV staff with the aim of building their capacity in financial inclusion strategy development. The same is being evaluated for NBC, albeit more focused on strategy implementation. History
7. Recommendation:

Evaluation Recommendation or Issue 6: Disaggregate overall outreach/target reporting to different financial services to capture the different elements of financial inclusion (digital transactions, specific financial services – credit, savings, insurance; agent employment, financial education). For each type of service ‘usage’ needs to be differently and appropriately defined, and gender issues and targets are likely to vary. Gender related measurement needs programme follow up, to define women’s role in household enterprises, and particularly for digital services, to support relevant fields in the MIS and analysis. The CF projects are an excellent opportunity to do this and to engage specifically on defining the relevant parameters of products and services that have a ‘gender focus’.

Management Response: [Added: 2018/12/14] [Last Updated: 2020/12/20]

Management Response: Agreed 

Key Actions:

Key Action Responsible DueDate Status Comments Documents
6.1 The evaluation suggests that usage should be defined and disaggregated in programme reporting. It should be noted that this already exists in the SHIFT ASEAN Monitoring and Results Measurement system, which contains dashboards for each investment or project under the programme, with defined specific indicators that are both quantitative and qualitative in nature. However often donors and stakeholders prefer to see overall high-level usage numbers, rather than separately defined usage figures across several business models. It is agreed that disaggregated usage reporting is preferred as it improves accuracy and is being done internally in SHIFT ASEAN’s systems already. To make disaggregated reporting and success stories more comparable across the portfolio the programme has developed an aggregation system that defines indicators, how they are measured and allows for automated aggregation and disaggregation by business model, product, client type, etc. This will allow for SHIFT ASEAN to more systematically disaggregate and aggregate across indicators as required by different stakeholders and for improved reporting of challenges and success stories.
[Added: 2018/12/14] [Last Updated: 2019/07/09]
SHIFT Programme Manager, Monitoring and Evaluation, and Fund Facility Coordinator 2019/03 Completed Although this activity has been completed, further improvements of this system are currently being implemented. Overall, access and usage are now being aggregated in a Power Bio dashboard that is consistently up to date and accessible for reference by the whole SHIFT team. This system allows the user to disaggregate figures by window and by grantee, providing up to date direct and indirect data on overall programme access and usage figures. What requires further work, is to build this system out to allow the user to further disaggregate the data by country, as well as by business model type. To this end, SHIFT has defined and categorized the portfolio grantees accordingly and the updated data dashboard should be available by the end of 2019. The definition of financial product usage has also been further refined according to business model, allowing SHIFT to report more accurate usage figures in the future. History
6.2 The evaluation also notes that gender related measurement in terms of women’s role and agency needs follow-up. This will be taken forward with project partners in defining the relevant parameters of product and service offerings. Lean data analysis and big data analysis can also support these efforts, as well as a case study approach. While, impact evaluation focused on women’s impact beyond financial access/usage, women’s empowerment and agency, is also desirable (however subject to additional funding)
[Added: 2018/12/14] [Last Updated: 2020/06/30]
SHIFT Programme Manager, Monitoring and Evaluation, Fund Facility Coordinator 2020/06 Completed SHIFT is currently implementing a series of Lean Data collection surveys with 3 ASEAN Innovation Fund partners (LVPB, BRAC and Thunes). This will allow SHIFT and its partners to supplement the monitoring of business model performance. The most advanced of these is with LVPB, with whom the survey has been launched. With BRAC the survey is in the process of being translated. Each partner will have its own tailored survey, which can include for example poverty outreach (% of poor, low income customers), % of women/ men with control over finance, customer satisfaction, productive and non-productive use of finance, enterprise growth, sector, job creation, MSME sizes and/or digital financial literacy. This will allow SHIFT to gain insights into women’s agency across the participating partners portfolio and will provide indicative insights overall. This will be combined with available company transactional data as well as market data, including the available FinScope data. History
8. Recommendation:

Evaluation Recommendation or Issue 7: Update the definition of poor, vulnerable to reflect a benchmark appropriate to the focus countries. A useful guideline is to target the ‘poorer 40% of households’. The Poverty Probability Index (PPI http://www.povertyindex.org/ppi-country) is a recognised tool to measure this and the PPI country design documents include useful country benchmark data. SHIFT has done well to introduce this to LVBP (successful CF partner in Vietnam) but needs to follow up to ensure the index is applied correctly and reporting is robust. The PPI is out-of-date or not available for some countries. SHIFT – policy/advocacy(and UNCDF) are well placed to mobilise interest and policy support to update the PPI and extend its use (including as part of the Finscope surveys). MRM needs to work closely with its partners (with TA and review of the MIS – software) to measure outreach to the poor – and to rural areas

Management Response: [Added: 2018/12/14] [Last Updated: 2020/12/20]

Management Response: Agreed 

Key Actions:

Key Action Responsible DueDate Status Comments Documents
7.1 The evaluation suggests a definition of poor and vulnerable that reflect a benchmark appropriate to the focus country. Initially SHIFT ASEAN defined low-income as less than $1.25/day, however it is recognized that how low-income is defined varies across country contexts and that the bottom 40 in many contexts are considered vulnerable and financially excluded or underserved. An improved benchmarking of this was initiated in 2018 with appropriate benchmarks being set for all CLMV countries. Also SHIFT ASEAN is undertaking GIS mapping of its agents and beneficiaries to better understand and benchmark socio-cultural traits i.e. proxy income and minority status based on their geolocation. A further review and benchmarking will be undertaken as recommended.
[Added: 2018/12/14] [Last Updated: 2020/12/04]
SHIFT Programme Manager and Monitoring and Evaluation 2020/10 Completed Completed for Myanmar and progressing for Cambodia, Lao PDR, Viet Nam and Indonesia. SHIFT is addressing this by using secondary data sources and establishing benchmarks for the definition of poverty in the SHIFT ASEAN final review report to be completed by year end. History
7.2 SHIFT is using the PPI index with its grantees to better understand the segments of the population that are being covered by the financial service offerings being used. Advocacy efforts to support the updating or implementation of the PPI index can also be considered under SHIFT’s work in 2019 and 2020.
[Added: 2018/12/14] [Last Updated: 2020/06/30]
SHIFT Programme Manager and Monitoring and Evaluation 2020/06 Completed To better understand the socio-economic profile of the SHIFT Programme end beneficiaries, SHIFT is implementing a series of Lean Data collection surveys with 3 ASEAN Innovation Fund partners (LVPB, BRAC and Thunes), allowing SHIFT to complement existing data on business model performance. Survey implementation is based on 3 business cases including e wallets, MSME loans and remittances. The surveys consist of a maximum of 20 questions, of which 10 will be based on the established poverty probability index (PPI) questionnaire which is country specific and estimates the likelihood of a respondent falling below the poverty line. The other 10 questions will be focused on gaining insights related to financial service usage. This will allow SHIFT to gain insights into the segments of the end beneficiaries being reached by the FSP and propose product adjustments where relevant and feasible. To date, the most advanced of these is with LVPB, with whom the survey has already been launched. The survey to be taken forward with BRAC clients is currently being revised and translated. History
9. Recommendation:

Evaluation Recommendation or Issue 8: CLMV, particularly Myanmar, has sizable ethnic minority populations. Between 4-6% of the population in CLMV are disabled. The issue of including people who are indigenous (ethnic minorities) or have a disability, deserves attention and the programme is well placed to explore opportunities through under the CF – through potential FSP partnerships with NGO programmes, and engaging with the regulator as part of NFIS.

Management Response: [Added: 2018/12/14] [Last Updated: 2020/12/20]

Management Response: Agree in principle and would require partnership efforts and related resources

Key Actions:

Key Action Responsible DueDate Status Comments Documents
8.1 Due to the SHIFT ASEAN programmes constrained resources, a focus and priority needs to be on improving, deepening and taking forward SHIFT’s current initiatives. However, SHIFT ASEAN does aim to attempt to mainstream disability and minority considerations into its practices, through an approach which is proportionate and relevant to SHIFT ASEAN. SHIFT ASEAN will engage on these issues with DFAT, and other relevant representatives of these vulnerable groups, in order to identify relevant experts and partnerships who can provide sensitization and support to SHIFT grantees and stakeholders, for their considerations in policies/regulations, product design/development (i.e. through workshops or technical support), planning, monitoring and evaluation opportunities.
[Added: 2018/12/14] [Last Updated: 2020/03/31]
SHIFT Programme Manager 2020/03 Completed Based on the insights and data generated by the financial diaries that were co-financed by UNCDF, BRAC is currently developing a loan product in Myanmar that is specifically aimed at people with disabilities, or people that are caring for people with disabilities. A research note has been prepared by UNCDF on how financial services could support low income individuals who have a disability and based on this workshops have been taken forward to collect insights on the financial needs of people with disabilities, as well as the needs of FSPs that want to target people with disabilities. To date the workshop in Myanmar was completed, and insights include that companies don’t realize the extent of disabilities there are (different kinds, different grades). This lack of knowledge is also coupled with a lack of funds to cater to this market. Further workshops are pending the solution of the COVID 19 crisis. In addition, disability has been incorporated into SHIFT ASEAN successor programme and will remain a programmatic priority for the next 5 years. History
10. Recommendation:

Evaluation Recommendation or Issue 9: The Theory of Change does well to provide a framework for a market systems approach, focusing on expected systemic changes (intermediate outcomes) which lead to improved services used by increasing numbers of both men and women (long-term outcomes). Reporting needs to focus on usage, not access, and direct achievement in usage under the fund facility should be seen as short-term/intermediate outcomes, to be differentiated from longer term outcomes (from replication, and other activities). The long term targets of 5 million (in the ProDoc) need review – since the assumptions include contributions to financial inclusion in 6 countries. UNCDF/SHIFT should directly promote Finscope data as the more reliable data set to measure changes over time (compared to Findex).

Management Response: [Added: 2018/12/14] [Last Updated: 2020/12/20]

Management Response: Agreed 

Key Actions:

Key Action Responsible DueDate Status Comments Documents
9.1 The theory of change is being updated based on the evaluation findings and also broken down by country to better measure systemic changes in the market. SHIFT ASEAN is working to update its programme targets based on funding and activity outreach scenarios.
[Added: 2018/12/14] [Last Updated: 2020/06/30]
SHIFT Programme Manager and Monitoring and Evaluation 2020/06 Completed 9.1 The theory of change is being updated based on the evaluation findings and also broken down by country to better measure systemic changes in the market. SHIFT ASEAN is working to update its programme targets based on funding and activity outreach scenarios. 30/06/2020 SHIFT Programme Manager and Monitoring and Evaluation Completed This work was launched with the development of the first country-based Theory of Change (TOC) with indicators in Cambodia. Based on this, additional ToCs were developed for the regional programme as well as each country within the scope of the programme including Lao PDR, Myanmar and Viet Nam. History
9.2 As noted in point 6, usage is currently defined across different products and services, however for external communication and often donors prefer to see one overall usage number, rather than separately defined usage figures. It is agreed that disaggregated usage reporting is preferred as it improves accuracy. SHIFT has developed an aggregation system that defines indicators, how they are measured and allows for automated aggregation and disaggregation by business model, product, client type, etc.
[Added: 2018/12/14] [Last Updated: 2019/07/09]
SHIFT Programme Manager, Monitoring and Evaluation, Fund Facility Coordinator and Donor 2019/03 Completed As previously indicated, SHIFT has worked towards establishing an effective system to manage, visualize and report access and usage figures. With a particular focus on usage, SHIFT has made these figures available using a Power BI dashboard that allows the user to disaggregate this data by gender, grantee and window. What is now required, is to further allow the disaggregation of this data by country and business model type. To this end, the business models have now been classified and complete disaggregation of usage data should be possible by the end of 2019. History
11. Recommendation:

Evaluation Recommendation or Issue 10: MSME development and financing is also important to address the programme objectives of increased income and employment which have not been adequately focused upon so far. While the fourth round of CF will be on MSME financing, however, challenge fund interventions may not be sufficient to bring systematic changes unless it is supported through policy level changes for incentivising banks and FSPs to provide micro enterprise level financing (like mandatory priority sector lending in India), for women headed firms.

Management Response: [Added: 2018/12/14] [Last Updated: 2020/12/20]

Management Response: Agreed

Key Actions:

Key Action Responsible DueDate Status Comments Documents
10.1 It is agreed that MSME development and financing challenges will not be addressed by challenge fund mechanisms alone. This is why every activity that SHIFT ASEAN undertakes looks at the underlying market constraints i.e. regulatory, supply, support services, etc. to understand the bottlenecks. In 2018 SHIFT ASEAN has engaged and advocated on barriers faced by women MSMEs with the ASEAN Coordination Committee on Micro, Small and Medium Enterprises (ACCMSME) and bi-laterally with FinTech Associations and Central Banks; these advocacy and policy support efforts will continue in 2019 and beyond. The findings of the SME financing study presently being undertaken in Cambodia in collaboration with UNESCAP, will also inform 2019 and 2020 activities around MSME financing and women enterprises.
[Added: 2018/12/14] [Last Updated: 2020/06/30]
SHIFT Programme Manager and Fund Facility Coordinator 2020/06 Completed Between March and April of 2019, SHIFT in partnership with UNESCAP, VISA, FMO and DFAT launched the Asia-Pacific Women Enterprise Innovation Fund. In total, 110 applications were submitted of which 103 were deemed eligible. Of these, 30 were shortlisted, and 19 were presented to the SHIFT board for funding. 10 were finally selected for funding. Business models include developing the first digital lending platform of Myanmar, alternative collateral lending models, and peer to peer lending models amongst others. SHIFT has also collaborated with partners (UNESCAP) in the development of a series of detailed national studies related to micro, small and medium economic segments. The first of these were taken forward in Cambodia and Nepal. SHIFT provided technical expertise to these studies which were summarized as recommendations and presented to a variety of government officials. SHIFT, in collaboration with UNCDF team in Myanmar, worked towards a synthesis of Myanmar gender demand side data to allow for better access to it and gain insights in women’s MSME amongst other. This will be complemented with the lean data surveys which are being planned for 2020 Furthermore, SHIFT has collaborated in many regional events aimed at (women’s) MSME financing including working with the Indonesian Fintech Association (AFTECH) to organize an event on innovative solutions to digitize MSME. This formed the basis of a long-term partnership that has been established with the AFTECH, and which was followed up by SHIFT participation in another MSME focused event. History

Latest Evaluations

Contact us

1 UN Plaza
DC1-20th Floor
New York, NY 10017
Tel. +1 646 781 4200
Fax. +1 646 781 4213
erc.support@undp.org