Mid-Term Evaluation of Shaping Inclusive Finance Transformations (SHIFT) Programme in Asean

Report Cover Image
Evaluation Plan:
2016-2017, UNCDF
Evaluation Type:
Others
Planned End Date:
03/2018
Completion Date:
12/2018
Status:
Completed
Management Response:
Yes
Evaluation Budget(US $):
77,200

Share

Document Type Language Size Status Downloads
Download document TOR SHIFT.docx tor English 830.41 KB Posted 82
Download document SHIFT ASEAN Mid-term Evaluation_Appendix_Country Reports (1).pdf related-document English 1193.63 KB Posted 128
Download document SHIFT ASEAN Final Mid-term Evaluation Report_November 2018.pdf report English 1729.72 KB Posted 200
Title Mid-Term Evaluation of Shaping Inclusive Finance Transformations (SHIFT) Programme in Asean
Atlas Project Number:
Evaluation Plan: 2016-2017, UNCDF
Evaluation Type: Others
Status: Completed
Completion Date: 12/2018
Planned End Date: 03/2018
Management Response: Yes
Corporate Outcome and Output (UNDP Strategic Plan 2014-2017)
Evaluation Budget(US $): 77,200
Source of Funding:
Evaluation Expenditure(US $): 77,200
Joint Programme: No
Joint Evaluation: No
Evaluation Team members:
Name Title Email Nationality
Frances Sinha Team Leader INDIA
Shantanu Das Team Member INDIA
Sanjay Sinha Team Member INDIA
Abhirup Bhunia Team Member INDIA
Nitin Madan Team Member INDIA
GEF Evaluation: No
Key Stakeholders:
Countries: GLOBAL
Lessons
Findings
Recommendations
1

/

2

Evaluation Recommendation or Issue 1: The programme team is currently working with consultants to assist in resource mobilisation. The focus here should be to raise resources for the non-challenge fund pillars, ideally to try to generate funding on long term commitment, alongside shorter commitments (such as MAS and VISA towards a particular challenge fund). The programme should also explore building synergies with ongoing donor-assisted programs on financial inclusion (such as World Bank, ADB, DFID). Additional funding commitment would enable SHIFT to strengthen staff resources – particularly for policy/advocacy, and learning - so as to have a more balanced approach across pillars. For specific activities – training, research, surveys - partner payment contributions should also be considered. This would serve to focus partner attention to optimise what they are paying for. 

3

Evaluation Recommendation or Issue 2: Limited funding commitments will require an adjustment in expectations with a more focused programmatic approach. Within continuing resource constraints, we recommend more focus on policy/advocacy and learning/capacity building (regulators, FSPs) with a continuing emphasis on gender and inclusion, identifying specific policy measures that support Markets for the Poor (M4P), and helping to create an appropriate environment for pro-poor private capital. Geographically, current activities should continue regionally and in Cambodia and Myanmar, with extension to Laos and Vietnam as opportunities arise.

4

Evaluation Recommendation or Issue 3: To support more sustainable funding, we recommend alternative types of funding under the CF. An important rationale for companies to participate in a challenge fund appears to be access to risk-willing capital, rather than the access to subsidized or ‘free’ money and hence repayment need not be an issue, particularly if the project is successful. Alternatives to funding grants are: conditional loans of interest, returnable capital, or a venture capital mechanism.

5

Evaluation Recommendation or Issue 4: To improve effectiveness of the Challenge Fund, it is important to 1) set realistic targets within a practical time frame. (A move away from grant funding is also likely to inject more realism into partner ambitions); 2) ensure investments in digital services build in core financial products (credit, savings, insurance). While digital payments (wallets, etc.) are a vehicle for formalisation and employment, it is the core products that will contribute to SHIFT’s goals of reduced vulnerability and poverty reduction; and 3) arrange for technical assistance – ideally on a cost sharing basis - to support partners to address some of the implementation bottlenecks (such as action-oriented market research – which is very difficult to do well) as well as robust reporting (see below).

6

Evaluation Recommendation or Issue 5: SHIFT’s present thinking under the learning and capacity building pillar, of building a pool of master trainers and tie-ups with institutes/networks (being planned with the Cambodia Microfinance Association) is in the right direction and should be explored across the focus countries. There are opportunities to develop synergies with respect to curriculum, sharing facilities and costs with other agencies (multilaterals, and others). For example, IMF training centres for capacity building of policy makers, possible tie-up with ILO on Save ASEAN remittance portal, migrant resource centres. This will help in building both capacity and sustainability and provide a link to capacity building of consumers and consumer financial literacy, including financial literacy training programmes for migrant workers.

7

Evaluation Recommendation or Issue 6: Disaggregate overall outreach/target reporting to different financial services to capture the different elements of financial inclusion (digital transactions, specific financial services – credit, savings, insurance; agent employment, financial education). For each type of service ‘usage’ needs to be differently and appropriately defined, and gender issues and targets are likely to vary. Gender related measurement needs programme follow up, to define women’s role in household enterprises, and particularly for digital services, to support relevant fields in the MIS and analysis. The CF projects are an excellent opportunity to do this and to engage specifically on defining the relevant parameters of products and services that have a ‘gender focus’.

8

Evaluation Recommendation or Issue 7: Update the definition of poor, vulnerable to reflect a benchmark appropriate to the focus countries. A useful guideline is to target the ‘poorer 40% of households’. The Poverty Probability Index (PPI http://www.povertyindex.org/ppi-country) is a recognised tool to measure this and the PPI country design documents include useful country benchmark data. SHIFT has done well to introduce this to LVBP (successful CF partner in Vietnam) but needs to follow up to ensure the index is applied correctly and reporting is robust. The PPI is out-of-date or not available for some countries. SHIFT – policy/advocacy(and UNCDF) are well placed to mobilise interest and policy support to update the PPI and extend its use (including as part of the Finscope surveys). MRM needs to work closely with its partners (with TA and review of the MIS – software) to measure outreach to the poor – and to rural areas

9

Evaluation Recommendation or Issue 8: CLMV, particularly Myanmar, has sizable ethnic minority populations. Between 4-6% of the population in CLMV are disabled. The issue of including people who are indigenous (ethnic minorities) or have a disability, deserves attention and the programme is well placed to explore opportunities through under the CF – through potential FSP partnerships with NGO programmes, and engaging with the regulator as part of NFIS.

10

Evaluation Recommendation or Issue 9: The Theory of Change does well to provide a framework for a market systems approach, focusing on expected systemic changes (intermediate outcomes) which lead to improved services used by increasing numbers of both men and women (long-term outcomes). Reporting needs to focus on usage, not access, and direct achievement in usage under the fund facility should be seen as short-term/intermediate outcomes, to be differentiated from longer term outcomes (from replication, and other activities). The long term targets of 5 million (in the ProDoc) need review – since the assumptions include contributions to financial inclusion in 6 countries. UNCDF/SHIFT should directly promote Finscope data as the more reliable data set to measure changes over time (compared to Findex).

11

Evaluation Recommendation or Issue 10: MSME development and financing is also important to address the programme objectives of increased income and employment which have not been adequately focused upon so far. While the fourth round of CF will be on MSME financing, however, challenge fund interventions may not be sufficient to bring systematic changes unless it is supported through policy level changes for incentivising banks and FSPs to provide micro enterprise level financing (like mandatory priority sector lending in India), for women headed firms.

1. Recommendation:

/

Management Response: [Added: 2018/12/14] [Last Updated: 2018/12/14]

Overall comments: Management would like to thank the evaluation team for the evidence, analysis and insights presented in the Mid Term Evaluation Report. SHIFT ASEAN found the recommendations provided useful to further build on the achievements of the programme to date, and to strengthen programme implementation moving forward.

One of the goals of the SHIFT ASEAN program is to influence a systemic and market system change, through its combined interventions. The evaluation acknowledges that this process is complex and takes time, and it has found that “SHIFT is yet to show results of its activities in terms of influencing market development”.  The program team assessment is that one can actually see initial signs of market influence, for example the programme’s digital financial service business model investment in Viet Nam is being used as a basis for the development of another bank’s digital financial service offerings. Also, through SHIFT’s ASEAN level policy engagements, peer-pressure mechanisms among regulators, have triggered faster decision making on financial inclusion roadmap approval and implementation processes. The program team recognizes this is an ambitious goal and is committed to undertake contribution analysis on such changes in the coming years of the program.

It should also be noted that sex-disaggregated data for DFS and all SHIFT ASEAN interventions is tracked; SHIFT ASEAN has worked with its project partners to develop systems, which collect sex-disaggregated data and also provide trainings on the importance and analysis of sex disaggregated data for product development. Further data collection methods can be further improved by the programme.

SHIFT ASEAN notes the evaluations assessment that the program design is relevant to the context of ASEAN countries and that SHIFT ASEAN’s policy efforts have been effective at the ASEAN level (through WC-FINC) and has brought together the developed players for fin-tech (in Singapore and Malaysia) and linked them with rural areas for remittances in CLMV.

Overall, UNCDF management  accepts the recommendations of the evaluation, with some caveats as indicated in the detailed response, and will take necessary measures to strengthen the SHIFT ASEAN programme.  The evaluation’s recommendations will also be applied as UNCDF designs the next phase of its programming in the ASEAN under its new integrated “Digital@UNCDF” strategy. Other evaluation exercises scheduled for SHIFT SAARC and the PFIP program in the Pacific will be used in a similar fashion to design successor programming in those regions under the new global Digital@UNCDF strategy. This more integrated programmatic approach will enable UNCDF to develop a more integrated and strategic approach to financial inclusion and digital finance across Asia, with strong cross learning and shared experiences between the three sub-regions.      

Key Actions:

2. Recommendation:

Evaluation Recommendation or Issue 1: The programme team is currently working with consultants to assist in resource mobilisation. The focus here should be to raise resources for the non-challenge fund pillars, ideally to try to generate funding on long term commitment, alongside shorter commitments (such as MAS and VISA towards a particular challenge fund). The programme should also explore building synergies with ongoing donor-assisted programs on financial inclusion (such as World Bank, ADB, DFID). Additional funding commitment would enable SHIFT to strengthen staff resources – particularly for policy/advocacy, and learning - so as to have a more balanced approach across pillars. For specific activities – training, research, surveys - partner payment contributions should also be considered. This would serve to focus partner attention to optimise what they are paying for. 

Management Response: [Added: 2018/12/14]

Management Response: Agreed, with the objective to build the foundation of future programming in the ASEAN under the integrated Digital@UNCDF strategy. 

Key Actions:

Key Action Responsible DueDate Status Comments Documents
1.1. The evaluation recommends to focus on raising resources for non-challenge fund areas of work, looking at both long term commitments alongside shorter term commitments. The programme has already initiated mapping donor interests and developing concept notes for non-Challenge fund areas of work. The programme is in the process of finalizing hiring of a fundraising support consultant and has set fundraising targets as part of Staff performance assessments.
[Added: 2018/12/14] [Last Updated: 2019/07/09]
SHIFT Programme Manager 2019/12 Initiated The fundraising consultant joined in mid-March and is currently further mapping funding opportunities for the innovation fund as well as other programme pillars. Additional funds have been raised for the data and learning hub of the SHIFT programme. These funds were sourced both from UNCDF flexible non-core funding, as well as from international donors such as FMO and DFID (the latter as a parallel contribution coming from UNCDF Myanmar programme). These funds will be used to take forward lean data surveys in partnership with 4-5 local FSPs in Myanmar, Cambodia, Vietnam and Lao PDR. The aim of the surveys is to supplement partner monitoring and cost-effectively measure SDG impacts for SHIFT Digital finance innovations. History
1.2 The evaluation also recommends that SHIFT ASEAN explore synergies with ongoing donor-assisted programmes. Discussions and coordination with other development agencies such as ADB in Cambodia and Vietnam, and World Bank in Vietnam, have been explored. SHIFT ASEAN will be scale-up in 2019 its discussions to find synergy areas with World Bank, ADB, DFID, IFC, USAID, BMGF, Canada, and DFAT country posts and share action points for implementation to the SHIFT ASEAN Board.
[Added: 2018/12/14] [Last Updated: 2019/08/14]
SHIFT Programme Manager 2019/12 Initiated SHIFT has explored synergies with ongoing donor assisted programmes, particularly in Myanmar. Currently, these initiatives have centered around FMO and DFID work in Myanmar, and the potential of their support for the development of gender smart financial products and data analytics. This partnership is currently being established and further implementation will be taken forward in the second half of 2019. Synergies with both VISA and Mastercard are also being explored within the framework of the data hub work. History
1.3 The evaluation suggests that for specific activities i.e. trainings, partner payments and contributions should be considered. This is currently being undertaken with cost sharing and in-kind support from partner institutions in providing venue space, staff time, but most importantly institutional commitments made to improve product development based on learnings from the trainings. With recent first level of trainings imparted on data analytics and value proposition for the same realized, the programme will explore alternate contribution arrangements in terms of funds, customer data analytics etc. further in 2019.
[Added: 2018/12/14] [Last Updated: 2019/07/09]
SHIFT Data and Learning Hub 2019/12 Initiated This is currently being explored. History
3. Recommendation:

Evaluation Recommendation or Issue 2: Limited funding commitments will require an adjustment in expectations with a more focused programmatic approach. Within continuing resource constraints, we recommend more focus on policy/advocacy and learning/capacity building (regulators, FSPs) with a continuing emphasis on gender and inclusion, identifying specific policy measures that support Markets for the Poor (M4P), and helping to create an appropriate environment for pro-poor private capital. Geographically, current activities should continue regionally and in Cambodia and Myanmar, with extension to Laos and Vietnam as opportunities arise.

Management Response: [Added: 2018/12/14]

Management Response: Agreed

Key Actions:

Key Action Responsible DueDate Status Comments Documents
2.1. Due to the funding constraints it is agreed that the expectations of the programme should be considered both in terms of activities and targets. SHIFT ASEAN is working to update its programme targets based on funding and activity outreach scenarios. Also, a SHIFT ASEAN 2019 strategy focused on policy/advocacy, linked with regulator capacity building activities, has been developed in order to guide and focus the programme implementation around a specific and strategic set of activities for future work. Specifically, SHIFT will continue to engage with ASEAN working committees by providing technical assistance and advocating on specific issues at the ASEAN Working Committee for Financial Inclusion meetings. SHIFT will also undertake bi-lateral advocacy and support based on its bi-lateral agreements with Cambodia and Viet Nam, which will feed-into regional policy mechanisms/platforms. Apart from the existing commitments, partnerships with existing programmes, UN agencies are being leveraged to expand SHIFT ASEAN’s objectives in Lao and Vietnam. With engagement with ASEAN policy bodies and existing partnerships with regulators in CLMV, the work on policies/advocacy related to gender and inclusion will be further accelerated.
[Added: 2018/12/14] [Last Updated: 2019/07/23]
SHIFT Programme Manager 2019/12 Initiated History
4. Recommendation:

Evaluation Recommendation or Issue 3: To support more sustainable funding, we recommend alternative types of funding under the CF. An important rationale for companies to participate in a challenge fund appears to be access to risk-willing capital, rather than the access to subsidized or ‘free’ money and hence repayment need not be an issue, particularly if the project is successful. Alternatives to funding grants are: conditional loans of interest, returnable capital, or a venture capital mechanism.

Management Response: [Added: 2018/12/14]

Management Response: Agreed 

Key Actions:

Key Action Responsible DueDate Status Comments Documents
3.1 UNCDF through its LDC Investment Platform has been implementing blended finance, debt and guarantee instruments. SHIFT ASEAN will connect its existing and future pipeline generated through Challenge fund and other direct partnership mechanism with the LDC Investment platform for accessing risk-willing capital. This will be dependent on the quality of the company’s business model and UNCDF fund availability. Also SHIFT ASEAN has recently implemented the Indonesia Challenge fund, where linkage with investors is one of the outputs. The programme will expand its collaboration with investor community to connect its private sector partner to alternate sources of capital and technical assistance. Specifically, SHIFT will publicize the business models through its communication channels, we will showcase the business models in relevant events and introduce the companies with impact investors through the organization of an impact investment workshop with the investor community, This will result in crowding-in of additional risk capital.
[Added: 2018/12/14] [Last Updated: 2019/07/09]
SHIFT Programme Manager, UNCDF Headquarters and SHIFT Board 2019/12 Initiated In partnership with the local business and investor community, technical assistance activities have been developed for the FinTech innovation fund grantees in Indonesia. This has included receiving mentorship support from VISA for the development of the application pitches, as well as support and in-kind access to ORACLE cloud based technical infrastructure support platforms, and management expertise from Deloitte. The success of these activities has meant that a similar training programme is being evaluated for the Asia-Pacific Women Enterprise Innovation Fund window, and currently partners are being identified for this. The FinTech business models supported by SHIFT in Indonesia were showcased during the 7th WC-FINC meeting. History
5. Recommendation:

Evaluation Recommendation or Issue 4: To improve effectiveness of the Challenge Fund, it is important to 1) set realistic targets within a practical time frame. (A move away from grant funding is also likely to inject more realism into partner ambitions); 2) ensure investments in digital services build in core financial products (credit, savings, insurance). While digital payments (wallets, etc.) are a vehicle for formalisation and employment, it is the core products that will contribute to SHIFT’s goals of reduced vulnerability and poverty reduction; and 3) arrange for technical assistance – ideally on a cost sharing basis - to support partners to address some of the implementation bottlenecks (such as action-oriented market research – which is very difficult to do well) as well as robust reporting (see below).

Management Response: [Added: 2018/12/14]

Management Response: Agreed 

Key Actions:

Key Action Responsible DueDate Status Comments Documents
4.1 Targets are now being reviewed given different project timeframes and PBAs are being developed accordingly. This process is already being taken forward in SHIFT ASEAN’s most recent Indonesia Challenge Fund Window. As a result in December 2018 targets of the Indonesia Challenge Fund Window were reviewed and aligned with expected and market growth rates. Technical assistances partnerships were also developed and will linked with the specific business models in the first quarter of 2019. This same approach will be taken for the Asia-Pacific Women Enterprise Innovation Fund in 2019, to result in appropriate targets and technical assistance which will allow for sustainable business models.
[Added: 2018/12/14] [Last Updated: 2019/07/09]
Fund Facility Coordinator, Monitoring and Evaluation 2019/03 Completed No further progress, although the same balanced approach that was established with the Indonesia Innovation Fund Window will be followed for the PBA for the grantees of the Asia-Pacific Women Enterprise Innovation Fund. This will be done in June 2019. History
4.2 The future windows since the Mid-Term Evaluation was conducted has already focused on having business models that go beyond payment solutions. The recent Indonesia Challenge Fund Window has selected some business models that addresses the enterprise credit and capacity building issues. The upcoming Asia-Pacific Women Enterprise Innovation Fund will also take the non-digital payment solutions into consideration and prioritization based on specific country market ecosystems and needs. This will be integrated into the challenge fund term paper and will be launched in the first quarter of 2019. This will result in a wider range of business models and solutions being considered under the challenge fund round.
[Added: 2018/12/14] [Last Updated: 2019/07/09]
Fund Facility Coordinator 2019/06 Completed Although this action has been marked as completed, it is important to report that SHIFT has succeeded in moving beyond business models that just focus on payments. The Indonesia Innovation Fund Window grantees address a range of issues including improved credit scoring systems, digitalizing payments systems, SME inventory-based loans and financial literacy. With respect to the Asia-Pacific Women Enterprise Innovation Fund, shortlisted business models include developing the first digital lending platform of Myanmar, alternative collateral lending models, and peer to peer lending models amongst others. History
4.3 The programme has initiated technical assistance delivery under its Indonesia Challenge fund window and future windows. Also the programme has received approval from board on partner ranking selection method of contracting that will enable technical assistance components to be delivered much more effectively. The programme will integrate technical assistance in all its Challenge fund windows in 2019 and will develop at least 2 partnerships with relevant technical assistance providers by June 2019. SHIFT ASEAN has also created a partnership with global corporates with expertise in digital financial solutions, for their selected team members to act as technical assistance provider and mentors.
[Added: 2018/12/14] [Last Updated: 2019/07/23]
Fund Facility Coordinator 2019/09 Initiated History
6. Recommendation:

Evaluation Recommendation or Issue 5: SHIFT’s present thinking under the learning and capacity building pillar, of building a pool of master trainers and tie-ups with institutes/networks (being planned with the Cambodia Microfinance Association) is in the right direction and should be explored across the focus countries. There are opportunities to develop synergies with respect to curriculum, sharing facilities and costs with other agencies (multilaterals, and others). For example, IMF training centres for capacity building of policy makers, possible tie-up with ILO on Save ASEAN remittance portal, migrant resource centres. This will help in building both capacity and sustainability and provide a link to capacity building of consumers and consumer financial literacy, including financial literacy training programmes for migrant workers.

Management Response: [Added: 2018/12/14]

Management Response: Agreed

Key Actions:

Key Action Responsible DueDate Status Comments Documents
5.1 In 2018, SHIFT ASEAN’s capacity building and technical assistance activities have effectively become a cross-cutting support mechanism to the entire SHIFT ASEAN programme. Training, technical assistance and learning related activities are now being mainstreamed across all SHIFT ASEAN workstreams i.e. data support trainings with both policy makers and FSPs, and targeted FSP technical assistance support through the Challenge Fund. The building out of SHIFT ASEAN’s capacity building activities will be continued throughout the remainder of the SHIFT ASEAN programme. While it took time to build up the content and approach to SHIFT ASEAN’s capacity building activities, they have been now been improved, refined and ready to be expanded. Training content has been revised to emphasize the link between data and product development, based on the customer journey. The training has also been made more practical and an improved pre-training survey will ensure a more tailored approach, which should create a more enabling environment, for the development of targeted financial products. Regarding partnerships, SHIFT ASEAN has established new partnerships that are aimed at strengthening the programme’s capacity building activities, specifically with the Cambodia Microfinance Association (CMA), similar partnerships will be explored in 2019. As suggested, the programme will explore partnerships with IMF, ILO and others on various aspects of capacity building on segmental and sectoral aspects.
[Added: 2018/12/14] [Last Updated: 2019/07/09]
SHIFT Programme Manager, and Data and Analytics Specialis 2019/06 Completed SHIFT training activities have continued to expand, effectively becoming a cross cutting support mechanism for the entire SHIFT ASEAN Programme. Mentorship training on pitch development has now become a standard feature of the Innovation fund application process, allowing grantees to develop their capacity in this and improving the interaction between SHIFT, the Investment Committee and the grantees, allowing for better insights into project ideas and improving the selection process overall. TA during the implementation phase of the Innovation fund, is also now being piloted with the Indonesia Innovation fund grantees with the aim of creating a standard TA programme, which will be complemented with tailor made interventions (i.e. what SHIFT is currently exploring with LVPB). This is aimed at increasing financial products usage. Based on the Customer Journey Action Research (CJAR) training taken forward in 2016, SHIFT has now developed a broader training programme based on the comments of participating trainees which has become less technical but has strengthened the pre training preparation with a survey, the practical scope of the training, as well as the link between data, product development and the customer journey. This has resulted in the creation of new training modules with facilitation guidelines that have been used in Myanmar and Cambodia to train FSPs. The training modules were designed with a training of trainers programme in mind, and currently partnerships for this are being developed. For example, the training in Cambodia was executed in partnership with the Cambodia Microfinance Association (CMA), with whom we are also discussing opportunities for partnership expansion. Training modules aimed at CLMV regulators have also been expanded, and recently SHIFT developed a secondment programme at UNCDF in Bangkok for SBV staff with the aim of building their capacity in financial inclusion strategy development. The same is being evaluated for NBC, albeit more focused on strategy implementation. History
7. Recommendation:

Evaluation Recommendation or Issue 6: Disaggregate overall outreach/target reporting to different financial services to capture the different elements of financial inclusion (digital transactions, specific financial services – credit, savings, insurance; agent employment, financial education). For each type of service ‘usage’ needs to be differently and appropriately defined, and gender issues and targets are likely to vary. Gender related measurement needs programme follow up, to define women’s role in household enterprises, and particularly for digital services, to support relevant fields in the MIS and analysis. The CF projects are an excellent opportunity to do this and to engage specifically on defining the relevant parameters of products and services that have a ‘gender focus’.

Management Response: [Added: 2018/12/14]

Management Response: Agreed 

Key Actions:

Key Action Responsible DueDate Status Comments Documents
6.1 The evaluation suggests that usage should be defined and disaggregated in programme reporting. It should be noted that this already exists in the SHIFT ASEAN Monitoring and Results Measurement system, which contains dashboards for each investment or project under the programme, with defined specific indicators that are both quantitative and qualitative in nature. However often donors and stakeholders prefer to see overall high-level usage numbers, rather than separately defined usage figures across several business models. It is agreed that disaggregated usage reporting is preferred as it improves accuracy and is being done internally in SHIFT ASEAN’s systems already. To make disaggregated reporting and success stories more comparable across the portfolio the programme has developed an aggregation system that defines indicators, how they are measured and allows for automated aggregation and disaggregation by business model, product, client type, etc. This will allow for SHIFT ASEAN to more systematically disaggregate and aggregate across indicators as required by different stakeholders and for improved reporting of challenges and success stories.
[Added: 2018/12/14] [Last Updated: 2019/07/09]
SHIFT Programme Manager, Monitoring and Evaluation, and Fund Facility Coordinator 2019/03 Completed Although this activity has been completed, further improvements of this system are currently being implemented. Overall, access and usage are now being aggregated in a Power Bio dashboard that is consistently up to date and accessible for reference by the whole SHIFT team. This system allows the user to disaggregate figures by window and by grantee, providing up to date direct and indirect data on overall programme access and usage figures. What requires further work, is to build this system out to allow the user to further disaggregate the data by country, as well as by business model type. To this end, SHIFT has defined and categorized the portfolio grantees accordingly and the updated data dashboard should be available by the end of 2019. The definition of financial product usage has also been further refined according to business model, allowing SHIFT to report more accurate usage figures in the future. History
6.2 The evaluation also notes that gender related measurement in terms of women’s role and agency needs follow-up. This will be taken forward with project partners in defining the relevant parameters of product and service offerings. Lean data analysis and big data analysis can also support these efforts, as well as a case study approach. While, impact evaluation focused on women’s impact beyond financial access/usage, women’s empowerment and agency, is also desirable (however subject to additional funding)
[Added: 2018/12/14] [Last Updated: 2019/07/09]
SHIFT Programme Manager, Monitoring and Evaluation, Fund Facility Coordinator 2019/12 Initiated SHIFT is currently preparing to implement a series of Lean Data collection surveys with 4-5 ASEAN Innovation Fund partners. This will allow SHIFT and its partners to supplement the monitoring of business model performance. Each partner will have its own tailored survey, which can include for example poverty outreach (% of poor, low income customers), % of women/ men with control over finance, customer satisfaction, productive and non-productive use of finance, enterprise growth, sector, job creation, MSME sizes and/or digital financial literacy. This will allow SHIFT to gain insights into women’s agency across the participating partners portfolio and will provide indicative insights overall. This will be combined with available company transactional data as well as market data, including the available FinScope data. History
8. Recommendation:

Evaluation Recommendation or Issue 7: Update the definition of poor, vulnerable to reflect a benchmark appropriate to the focus countries. A useful guideline is to target the ‘poorer 40% of households’. The Poverty Probability Index (PPI http://www.povertyindex.org/ppi-country) is a recognised tool to measure this and the PPI country design documents include useful country benchmark data. SHIFT has done well to introduce this to LVBP (successful CF partner in Vietnam) but needs to follow up to ensure the index is applied correctly and reporting is robust. The PPI is out-of-date or not available for some countries. SHIFT – policy/advocacy(and UNCDF) are well placed to mobilise interest and policy support to update the PPI and extend its use (including as part of the Finscope surveys). MRM needs to work closely with its partners (with TA and review of the MIS – software) to measure outreach to the poor – and to rural areas

Management Response: [Added: 2018/12/14]

Management Response: Agreed 

Key Actions:

Key Action Responsible DueDate Status Comments Documents
7.1 The evaluation suggests a definition of poor and vulnerable that reflect a benchmark appropriate to the focus country. Initially SHIFT ASEAN defined low-income as less than $1.25/day, however it is recognized that how low-income is defined varies across country contexts and that the bottom 40 in many contexts are considered vulnerable and financially excluded or underserved. An improved benchmarking of this was initiated in 2018 with appropriate benchmarks being set for all CLMV countries. Also SHIFT ASEAN is undertaking GIS mapping of its agents and beneficiaries to better understand and benchmark socio-cultural traits i.e. proxy income and minority status based on their geolocation. A further review and benchmarking will be undertaken as recommended.
[Added: 2018/12/14] [Last Updated: 2019/08/14]
SHIFT Programme Manager and Monitoring and Evaluation 2019/12 Initiated Completed for Myanmar and continuing work for Cambodia, Lao PDR, Viet Nam and Indonesia which is pending the implementation of lean data surveys. History
7.2 SHIFT is using the PPI index with its grantees to better understand the segments of the population that are being covered by the financial service offerings being used. Advocacy efforts to support the updating or implementation of the PPI index can also be considered under SHIFT’s work in 2019 and 2020.
[Added: 2018/12/14] [Last Updated: 2019/08/14]
SHIFT Programme Manager and Monitoring and Evaluation 2019/12 Initiated To better understand the socio-economic profile of the SHIFT Programme end beneficiaries, SHIFT is preparing to implement a series of Lean Data collection surveys with 4-5 ASEAN Innovation Fund partners, allowing SHIFT to complement existing data on business model performance. Survey implementation will be based on 3 business cases including e wallets, MSME loans and remittances. The surveys will consist of a maximum of 20 questions, of which 10 will be based on the established poverty probability index (PPI) questionnaire which is country specific and estimates the likelihood of a respondent falling below the poverty line. The other 10 questions will be focused on gaining insights related to financial service usage. This will allow SHIFT to gain insights into the segments of the end beneficiaries being reached by the FSP and propose product adjustments where relevant and feasible. To date, the survey is most advanced with LVPB, BRAC and DAWN for whom the surveys have been developed, and in one case are being revised by the partner. At least three surveys should be completed by the end of 2019. History
9. Recommendation:

Evaluation Recommendation or Issue 8: CLMV, particularly Myanmar, has sizable ethnic minority populations. Between 4-6% of the population in CLMV are disabled. The issue of including people who are indigenous (ethnic minorities) or have a disability, deserves attention and the programme is well placed to explore opportunities through under the CF – through potential FSP partnerships with NGO programmes, and engaging with the regulator as part of NFIS.

Management Response: [Added: 2018/12/14]

Management Response: Agree in principle and would require partnership efforts and related resources

Key Actions:

Key Action Responsible DueDate Status Comments Documents
8.1 Due to the SHIFT ASEAN programmes constrained resources, a focus and priority needs to be on improving, deepening and taking forward SHIFT’s current initiatives. However, SHIFT ASEAN does aim to attempt to mainstream disability and minority considerations into its practices, through an approach which is proportionate and relevant to SHIFT ASEAN. SHIFT ASEAN will engage on these issues with DFAT, and other relevant representatives of these vulnerable groups, in order to identify relevant experts and partnerships who can provide sensitization and support to SHIFT grantees and stakeholders, for their considerations in policies/regulations, product design/development (i.e. through workshops or technical support), planning, monitoring and evaluation opportunities.
[Added: 2018/12/14] [Last Updated: 2019/07/09]
SHIFT Programme Manager 2019/12 Initiated Based on the insights and data generated by the financial diaries that were co-financed by UNCDF, BRAC is currently developing a loan product in Myanmar that is specifically aimed at people with disabilities, or people that are caring for people with disabilities. History
10. Recommendation:

Evaluation Recommendation or Issue 9: The Theory of Change does well to provide a framework for a market systems approach, focusing on expected systemic changes (intermediate outcomes) which lead to improved services used by increasing numbers of both men and women (long-term outcomes). Reporting needs to focus on usage, not access, and direct achievement in usage under the fund facility should be seen as short-term/intermediate outcomes, to be differentiated from longer term outcomes (from replication, and other activities). The long term targets of 5 million (in the ProDoc) need review – since the assumptions include contributions to financial inclusion in 6 countries. UNCDF/SHIFT should directly promote Finscope data as the more reliable data set to measure changes over time (compared to Findex).

Management Response: [Added: 2018/12/14]

Management Response: Agreed 

Key Actions:

Key Action Responsible DueDate Status Comments Documents
9.1 The theory of change is being updated based on the evaluation findings and also broken down by country to better measure systemic changes in the market. SHIFT ASEAN is working to update its programme targets based on funding and activity outreach scenarios.
[Added: 2018/12/14] [Last Updated: 2019/07/23]
SHIFT Programme Manager and Monitoring and Evaluation 2019/09 Initiated History
9.2 As noted in point 6, usage is currently defined across different products and services, however for external communication and often donors prefer to see one overall usage number, rather than separately defined usage figures. It is agreed that disaggregated usage reporting is preferred as it improves accuracy. SHIFT has developed an aggregation system that defines indicators, how they are measured and allows for automated aggregation and disaggregation by business model, product, client type, etc.
[Added: 2018/12/14] [Last Updated: 2019/07/09]
SHIFT Programme Manager, Monitoring and Evaluation, Fund Facility Coordinator and Donor 2019/03 Completed As previously indicated, SHIFT has worked towards establishing an effective system to manage, visualize and report access and usage figures. With a particular focus on usage, SHIFT has made these figures available using a Power BI dashboard that allows the user to disaggregate this data by gender, grantee and window. What is now required, is to further allow the disaggregation of this data by country and business model type. To this end, the business models have now been classified and complete disaggregation of usage data should be possible by the end of 2019. History
11. Recommendation:

Evaluation Recommendation or Issue 10: MSME development and financing is also important to address the programme objectives of increased income and employment which have not been adequately focused upon so far. While the fourth round of CF will be on MSME financing, however, challenge fund interventions may not be sufficient to bring systematic changes unless it is supported through policy level changes for incentivising banks and FSPs to provide micro enterprise level financing (like mandatory priority sector lending in India), for women headed firms.

Management Response: [Added: 2018/12/14]

Management Response: Agreed

Key Actions:

Key Action Responsible DueDate Status Comments Documents
10.1 It is agreed that MSME development and financing challenges will not be addressed by challenge fund mechanisms alone. This is why every activity that SHIFT ASEAN undertakes looks at the underlying market constraints i.e. regulatory, supply, support services, etc. to understand the bottlenecks. In 2018 SHIFT ASEAN has engaged and advocated on barriers faced by women MSMEs with the ASEAN Coordination Committee on Micro, Small and Medium Enterprises (ACCMSME) and bi-laterally with FinTech Associations and Central Banks; these advocacy and policy support efforts will continue in 2019 and beyond. The findings of the SME financing study presently being undertaken in Cambodia in collaboration with UNESCAP, will also inform 2019 and 2020 activities around MSME financing and women enterprises.
[Added: 2018/12/14] [Last Updated: 2019/07/09]
SHIFT Programme Manager and Fund Facility Coordinator 2019/12 Initiated Between March and April of 2019, SHIFT in partnership with UNESCAP, VISA, FMO and DFAT launched the Asia-Pacific Women Enterprise Innovation Fund. In total, 110 applications were submitted of which 103 were deemed eligible. Of these, 30 were shortlisted, and 19 are currently being presented to the SHIFT board for funding. Business models include developing the first digital lending platform of Myanmar, alternative collateral lending models, and peer to peer lending models amongst others. History

Latest Evaluations

Contact us

220 East 42nd Street
20th Floor
New York, NY 10017
Tel. +1 646 781 4200
Fax. +1 646 781 4213
erc.support@undp.org