Mid-Term Evaluation of Mobile Money for the Poor (MM4P)

Report Cover Image
Evaluation Plan:
2016-2017, UNCDF
Evaluation Type:
Others
Planned End Date:
09/2017
Completion Date:
12/2018
Status:
Completed
Management Response:
Yes
Evaluation Budget(US $):
168,000

Share

Document Type Language Size Status Downloads
Download document MM4P ToR .docx tor English 473.75 KB Posted 69
Download document Genesis_UNCDF_MTE of MM4P_Final report - 29.01.2019.pdf report English 2441.77 KB Posted 208
Title Mid-Term Evaluation of Mobile Money for the Poor (MM4P)
Atlas Project Number:
Evaluation Plan: 2016-2017, UNCDF
Evaluation Type: Others
Status: Completed
Completion Date: 12/2018
Planned End Date: 09/2017
Management Response: Yes
Corporate Outcome and Output (UNDP Strategic Plan 2014-2017)
Evaluation Budget(US $): 168,000
Source of Funding:
Evaluation Expenditure(US $): 178,146
Joint Programme: No
Joint Evaluation: No
Evaluation Team members:
Name Title Email Nationality
Richard Ketley Team Leader
Mauro Mela Digital Financial Services specialist
Caitlin Smit Senior Evaluator
Comfort Phelane Digital Financial Services specialist
Mark Schoeman Digital Financial Services specialist
Ashleigh Flynn Evaluator
Thandiwe Ng’ombe Evaluator
Dikshya Dawadi Local Consultant (Nepal)
Suman Gelal DFS Expert: Nepal
Agnes Fall DFS Expert: Senegal SENEGAL
Rasmi Pillar DFS Expert: Uganda INDIA
GEF Evaluation: No
Key Stakeholders:
Countries: GLOBAL
Lessons
Findings
Recommendations
1
2

Evaluation Recommendation or Issue 1:

 

The programme TOC and country results chains should be updated to reflect what has been learnt in each market. MM4P has been operating in each market for some time and has generated important learning about the market context, the key barriers to DF market development, and programming opportunities that have arisen in each country. Since the programme’s TOC and country results chains were developed at inception when programming activities had not been rolled out, there is now an opportunity for these to be updated. These should also explicitly state assumptions, risks and possible external factors for each link in the ToC, to assist with future assessments of MM4P’s contribution to market development shifts. The learning generated from this mid-term review should provide some input to this process, such as the role of innovation as a potential workstream and the need to incorporate KM and ecosystem work as causal links in the chain, as well as better articulating gender outcomes at the customer outcome level. However, each country team should consider how the state of the market has evolved since inception, and where results chains need to be updated and adapted as a result.

3

Evaluation Recommendation or Issue 2:

 

MM4P will have to incorporate innovation into its work stream as DFS markets develop and new types of providers emerge. Innovation here relates to the development of new business models for financial services providers, such as fintechs, and other service providers where digital payments form part of the new business models. Examples of innovation already emerging from the programme include fintechs (agile tech-focused financial service providers that use technology to change or improve the way financial services are produced and consumed) and digital platforms and marketplaces in the sharing economy that make use of digital payment options (such as Tootle in Nepal). Once digital payment rails are in place and the focus shifts to developing use cases around these rails, supporting innovation becomes an important activity for DFS market development. MM4P is already thinking about how to incorporate this into its programme design from the outset, including how to encourage and engage with non-traditional financial service providers, such as fintechs, which should be continued and formally built into the programme design.

4

Evaluation Recommendation or Issue 3:

 

MM4P should articulate the impact of its programming beyond financial inclusion to account for shifting funding priorities. The funding of financial inclusion activities within the donor community has shifted from considerations of financial inclusion as the end objective, to a broader focus on financial inclusion as a means to other objectives. These include improvements in the livelihoods of specific customer segments – such as women, refugees, and the youth – and the support of particular sectors – such as energy, food security, and agriculture. MM4P should monitor these shifting funding priorities among key donors and articulate the impact its work has on these priority segments/sectors in order to attract funding going forward

5

Evaluation Recommendation or Issue 4:

Given funding restrictions and donor priorities that have resulted in limitations on regulatory/policy and infrastructure work in some countries, MM4P needs to have plans in place to work around these limitations. The table below (see final report for the table as it was too long to insert here) provides a few examples from the programme of where policy/regulatory impact was created despite the limitations, in countries where these limitations existed. These examples should be replicated across the programme where possible. Importantly, these plans need to be in place at the outset of programming and built in to the programme design. MM4P can also draw on its existing strengths at ecosystem-level, namely information provision and convening to focus stakeholders’ attention on these limitations, even where MM4P does not have funding or resources to directly address these limitations.

6

Evaluation Recommendation or Issue 5:

MM4P needs to develop careful sustainability plans for its ecosystem facilitation role and start implementing this plan now to provide enough time before programming winds down. MM4P is playing a critical convening function within the DFS ecosystem, for example with the formation of DFS working groups. However, MM4P is often the only stakeholder playing this convening role and so needs to develop mechanisms for stakeholders to continue collaborating once the programme ends. Taking a lesson from MM4P in Malawi, the formation of an industry association with funding contributed by members appears to be a sustainable approach to having this convening function continued. MM4P should look to set up similar industry associations in countries with DFS working groups, and in Nepal where a DFS working group has yet to be formed.

7

Evaluation Recommendation or Issue 6:

 

MM4P can better incorporate the mainstreaming of gender into its programming activities over the second half of the programme. To date, requiring providers to collect sex-disaggregated data has been a tick-box exercise rather than encouraging providers to make use of the data. Programming should therefore focus on building the business case for using sex-disaggregated data among providers as part of MM4P support. This could include capacity building/TA or research such as market sizing and opportunity analysis for female customers, encouraging providers to make use of their data to tap in to latent opportunities, or using the data to show providers where women are desirable clients to attract. In addition, as programming matures to specific DFS use cases, there is scope for the programme to focus on use cases and sectors that are known to have a livelihood impact on women in particular, such as digital education and healthcare payments. The work that has been completed already on Zambian mothers and female farmers in Nepal, as well as the DFS4Women partner even that led to Orange Liberia rolling out a pilot on onboarding female agents, is promising in this regard.

8

Evaluation Recommendation or Issue 7:

 

Knowledge management activities need to capture and disseminate lessons on project failures as well as project successes. Thus far, MM4P’s KM activities have largely focused on successful projects with limited coverage of the challenges and failures experienced by providers. This information is equally useful for stakeholders in the ecosystem to learn from the experience of other providers, and more focus should be placed on covering these lessons on failure. KM activities can also disseminate learnings on failures throughout a project’s lifecycle, rather than waiting for the end of a project to publish lessons[1].

 


[1] See http://www.cgap.org/blog/series/embracing-failure-ultimate-success-branchless-banking and https://www.raflearning.org/post/smallholder-finance-big-learnings-start-sharing-real-failures

9

Evaluation Recommendation or Issue 8:

Country teams will have to carefully consider which stakeholders to partner with as programming matures. Since MM4P has established its credibility in the markets where it operates, country teams are receiving more requests for support. This evaluation has found that MM4P support is most impactful when it is longer-term and where in-country DFS experts are embedded in the partner organisation.  While it was appropriate for the programme to experiment with different partner types at the beginning of the programme, this suggests that MM4P should now focus on fewer projects with larger budgets and longer ongoing technical support. This does not necessarily mean that MM4P should only partner with a particular partner type, or those that it has partnered with before. For example, working with fintechs may become a priority in the future as DFS market development continues. Given that fewer partners equate to higher risk, these partnerships must have a high focus on sustainability and demonstration of a viable business case as selection criteria.

10

Evaluation Recommendation or Issue 9:

 

UNCDF’s procurement and selection process for partners should be reviewed given the changing dynamics in the market. With the emergence of new types of financial service providers, such as fintechs which tend to be more innovative and fast and flexible, UNCDF will have to adapt its procurement process for partners to reflect these changing market dynamics. This should be achieved through a review of the current process against its ability to respond to the current and future needs of the DFS market.

11

Evaluation Recommendation or Issue 10:

 

MM4P country teams will have to assess the capacity of their technical staff given the importance of longer-term TA provided directly by DFS experts in-country. Most of the country teams are currently coping with the pipeline of projects but would require additional in-country staff should this pipeline increase further. In cases where current technical specialists’ contracts are ending, it is important for the programme to replace these with in-country specialists that can develop long term relationships with partners. In countries where a greater focus on policy/regulatory work is required, the programme may need to consider widening its skill set to include staff with experience in policy and regulatory issues. 

12

Evaluation Recommendation or Issue 11:

The programme should consider how to strengthen its pre-project feasibility assessment to improve the likelihood that investments will lead to sustainable customer outcomes. In several cases, projects were unable to be implemented as they were planned in the design phase. The table below provides a few examples from the programme where this was the case. MM4P undertakes a thorough assessment of prospective partners before providing support, including a peer review of the project document by a technical specialist in another country and review by the investment committee. However, these examples suggest that the programme could benefit from other tools or frameworks for assessing viability, particularly methods for assessing whether the economics of a project makes sense (sufficient customer numbers and appetite, adequate infrastructure coverage, right regulatory approval processes in place, etc.). Learning from the programme thus far suggests that the first project with a partner should be short, with a relatively short feasibility assessment, since the team learns most about a partner and their feasibility by working directly with them. These small projects can act as the feasibility assessment for larger projects going forward.

13

Evaluation Recommendation or Issue 12:

 

MM4P should not focus solely on its market development indicators, and should simplify its country and project RM frameworks by significantly reducing the number of indicators it tracks. Priority indicators should be decided on by identifying a) priority outcomes for donors, b) indicators that the MM4P team frequently uses to make strategic and operational decisions. The programme’s market development stages provide a helpful framework for understanding market dynamics to inform programming decisions, but do not adequately capture MM4P’s contribution to shifts along the spectrum.

1. Recommendation:
Management Response: [Added: 2019/02/05]

Overall comments:

In 2013, based on the initial results of the new UNCDF MM4P programme, UNCDF chose to implement a new approach, moving away from large scale grant making in favor of placing strong DF (digital finance) expertise at country level to properly support all stakeholders, grow the expertise locally and position UNCDF as neutral technical broker in each country.  It introduced a market development approach for the DF market and is now recognized as a unique player in supporting digital finance.  To keep the team focused on technical work, MM4P decided to consolidate important operational, communications and results measurement functions for a growing number of countries in a Project Management Office in Brussels to better build cross-country skills, learning & sharing and to ensure the best quality support were available to all countries to allow country teams to focus on technical support rather than programme management.  The team of more than 40 persons mixed international and local staff with highly qualified technical experts recruited from industry leaders.  MM4P considers this mid-term review as a very positive evaluation which highlights the value of the technical team on the ground, their approach to engaging partners through continuous interaction and relationship building as an effective approach to building markets.  The recommendations provided are in line with future orientation of the programme and UNCDF has already taken steps to implement many of them.

Key Actions:

2. Recommendation:

Evaluation Recommendation or Issue 1:

 

The programme TOC and country results chains should be updated to reflect what has been learnt in each market. MM4P has been operating in each market for some time and has generated important learning about the market context, the key barriers to DF market development, and programming opportunities that have arisen in each country. Since the programme’s TOC and country results chains were developed at inception when programming activities had not been rolled out, there is now an opportunity for these to be updated. These should also explicitly state assumptions, risks and possible external factors for each link in the ToC, to assist with future assessments of MM4P’s contribution to market development shifts. The learning generated from this mid-term review should provide some input to this process, such as the role of innovation as a potential workstream and the need to incorporate KM and ecosystem work as causal links in the chain, as well as better articulating gender outcomes at the customer outcome level. However, each country team should consider how the state of the market has evolved since inception, and where results chains need to be updated and adapted as a result.

Management Response: [Added: 2019/02/05]

Given the dynamic market and the lessons learned implementing a market development programme, the TOC has been an iterative process since its development in 2013.  Adapting the TOC has been a primary focus since late 2017 when MM4P started the process of developing a new strategy and framework to adapt its activities to quickly changing market dynamics around digital finance and digital economies. The strategy includes two new workstreams:

  • Inclusive Innovation: this workstream reflects the expanded scope of the program to contribute to the development of inclusive innovation ecosystems and accelerate the development and usage of digital services in key sectors such as energy, health, education and agriculture.
  • Empowered Customers: this workstream is included as a separate work stream to reinforce within our program the focus on the empowerment of key segments (youth, women, refugees, migrants, farmers, MSME’s).

 

As part of the new strategy, the programme Theory of Change and country results chains are updated to reflect the programme’s lessons learnt and include the inclusive innovation and customer empowerment workstreams and to show the specific outcomes related to focus segments including women. Also, MM4P is currently reviewing its iterative processes to more frequently update the Theory of Change and results chains based on results and learnings from programme activities.

Key Actions:

Key Action Responsible DueDate Status Comments Documents
1.1. Include workstream Inclusive Innovation in strategy
[Added: 2019/02/05]
MM4P 2018/12 Completed
1.2. Include workstream Empowered Customers in strategy
[Added: 2019/02/05]
MM4P 2018/12 Completed
1.3. Update programme Theory of Change
[Added: 2019/02/05] [Last Updated: 2019/03/29]
MM4P/ RM 2019/06 Completed Updated History
1.4. Update results chain
[Added: 2019/02/05] [Last Updated: 2019/03/29]
MM4P/ RM 2019/09 Initiated History
3. Recommendation:

Evaluation Recommendation or Issue 2:

 

MM4P will have to incorporate innovation into its work stream as DFS markets develop and new types of providers emerge. Innovation here relates to the development of new business models for financial services providers, such as fintechs, and other service providers where digital payments form part of the new business models. Examples of innovation already emerging from the programme include fintechs (agile tech-focused financial service providers that use technology to change or improve the way financial services are produced and consumed) and digital platforms and marketplaces in the sharing economy that make use of digital payment options (such as Tootle in Nepal). Once digital payment rails are in place and the focus shifts to developing use cases around these rails, supporting innovation becomes an important activity for DFS market development. MM4P is already thinking about how to incorporate this into its programme design from the outset, including how to encourage and engage with non-traditional financial service providers, such as fintechs, which should be continued and formally built into the programme design.

Management Response: [Added: 2019/02/05]

Management Response:

 

See response to Recommendation 1 - MM4P is in the process of developing a new strategy which includes a workstream Inclusive Innovation. As part of its annual planning exercise, MM4P has started to structure its activities according to the new programme design. The 2019 country workplans all include activities to deliver under workstream Inclusive Innovation.

Key Actions:

Key Action Responsible DueDate Status Comments Documents
2.1. Include workstream Inclusive Innovation in strategy
[Added: 2019/02/05]
MM4P 2018/12 Completed History
2.2. planning and budgeting under new program design for 2019 to include inclusive innovation workstream at country level
[Added: 2019/02/05]
MM4P 2018/12 Completed
4. Recommendation:

Evaluation Recommendation or Issue 3:

 

MM4P should articulate the impact of its programming beyond financial inclusion to account for shifting funding priorities. The funding of financial inclusion activities within the donor community has shifted from considerations of financial inclusion as the end objective, to a broader focus on financial inclusion as a means to other objectives. These include improvements in the livelihoods of specific customer segments – such as women, refugees, and the youth – and the support of particular sectors – such as energy, food security, and agriculture. MM4P should monitor these shifting funding priorities among key donors and articulate the impact its work has on these priority segments/sectors in order to attract funding going forward

Management Response: [Added: 2019/02/05]

Management Response

 

Linking digital finance to the real economy and greater development issues is important for several reasons, namely:  (i) one of the greatest hurdles to increasing customer use is the lack of good uses cases for low income families; (ii) actors in the real economy are more likely to drive these uses cases and invest in them than the financial sector itself; (iii) such a shift is consistent with the United Nation’s commitment that its resources be used to achieve the Sustainable Development Goals; and (iv) funding opportunities for UNCDF necessitate this shift as its traditional funders also align their support to the 2030 Agenda. 

MM4P is in the process of developing a new strategy that reflects the changes in market dynamics which increase the potential of leveraging technology and innovation in improving people’s livelihoods, beyond financial inclusion. The strategy accommodates different impact pathways, depending on the focus segment and sector. As part of the new strategy, a Theory of Change will be developed including each target segment (women, youth, refugees, farmers) and sector (finance, agriculture, energy, health, education), visualizing the potential benefits and customer impact of services leveraging technology for these segments/sectors.

Since its inception, MM4P has targeted specific population segments in some of its projects (e.g. youth agents in Senegal, needs and barriers of mothers in Zambia, women agents in Malawi, smallholder farmers in Uganda and Nepal). More recently MM4P has started implementing projects across various sectors, including agriculture, education, health, energy and transport).  These projects have allowed UNCDF to increase skills and knowledge of specific sectors and segments that are leveraged in the development of the new strategy. 

To ensure alignment to donor priorities, the new strategy is developed in consultation with key donors. MM4P has designed a donor mapping at country and global level, showing donor priorities across the different sectors and segments. The programme will continue to implement across its countries.

Key Actions:

Key Action Responsible DueDate Status Comments Documents
3.1 Document UNCDF projects around key sectors and segments to use with donors and proposals
[Added: 2019/02/05] [Last Updated: 2019/07/08]
MM4P/ KM/FIPA 2019/06 Completed Document “Topic 3 1 KM” is the list of our global publications. For each publication, it is mentioned the workstreams, the sectors and/or segments. This document is our database for knowledge management. It is used to select key references used with donors, proposals, etc. History
3.2. Implement donors mapping, including donor priorities - global and country level
[Added: 2019/02/05] [Last Updated: 2019/07/09]
MM4P 2019/06 Completed The donor mapping has been done for Benin, Senegal and Zambia (MCF countries). But on top of that we used same approach for an additional 10 countries. The approach has also been presented to the FIPA team meeting and will be used widely within FIPA this year (22 countries). History
3.3. Develop segment- and sector- Theory of Change
[Added: 2019/02/05] [Last Updated: 2019/07/08]
MM4P/RM 2019/06 Completed It has been agreed that a specific Theory of Change will not be done for each segment and sector. The global ToC has been currently used for other segments without issues. We will keep it simple and use one ToC as of today. But we will make sure to adapt the RMF with the right indicators for segments and sectors. The last version of the Theory of Change is now widely used in all FIPA countries (after the Team Meeting).0 Again, the impact is well beyond the 3 MCF countries. It is FIPA wide. History
5. Recommendation:

Evaluation Recommendation or Issue 4:

Given funding restrictions and donor priorities that have resulted in limitations on regulatory/policy and infrastructure work in some countries, MM4P needs to have plans in place to work around these limitations. The table below (see final report for the table as it was too long to insert here) provides a few examples from the programme of where policy/regulatory impact was created despite the limitations, in countries where these limitations existed. These examples should be replicated across the programme where possible. Importantly, these plans need to be in place at the outset of programming and built in to the programme design. MM4P can also draw on its existing strengths at ecosystem-level, namely information provision and convening to focus stakeholders’ attention on these limitations, even where MM4P does not have funding or resources to directly address these limitations.

Management Response: [Added: 2019/02/05]

MM4P always recognized and stressed the importance of the policy and regulatory environment and infrastructure as key building blocks of the development of a digital finance ecosystem. MM4P will continue to engage with regulators as explained in the examples mentioned in the mid-term review. Infrastructure is a key component of the new strategy, as part of the workstream that aims to establish Open Digital Payment Ecosystem. Focus areas related to infrastructure include enhancing network and data coverage, phone ownership and appropriate financial infrastructure accessible to all. 

 

Bill and Melinda Gates Foundation recently committed US$ 5.9 million for activities specifically focused on Policy & Regulation in Africa which MM4P will implement in the coming 4 years. MM4P will continue to fundraise for activities specifically aimed on improving the regulatory environment and addressing infrastructural gaps.

 

Country strategies will be drafted during Q1 & Q2 2019 based on the new strategy, including country-specific objectives and activities in policy, regulation and infrastructure.

Key Actions:

Key Action Responsible DueDate Status Comments Documents
4.1. Develop country strategies, including objectives and activities related to policy, regulation and infrastructure
[Added: 2019/02/05] [Last Updated: 2019/07/09]
MM4P and all FIPA countries adopting new strategy as of 2019 2019/06 Completed Since the beginning of the year, MM4P adopted the new strategic framework for their strategies and country plans. It includes the workstreams “Enabling Policy and Regulation” and “Open Digital Payment Ecosystem” highlighting key activities on policy, regulation and infrastructure. The country strategies template is now being used for the 22 priority countries of FIPA. The lessons learned from MM4P are now used for all FIPA. This template has been used during the FIPA Team Building and Sharing meeting in June 19. History History
6. Recommendation:

Evaluation Recommendation or Issue 5:

MM4P needs to develop careful sustainability plans for its ecosystem facilitation role and start implementing this plan now to provide enough time before programming winds down. MM4P is playing a critical convening function within the DFS ecosystem, for example with the formation of DFS working groups. However, MM4P is often the only stakeholder playing this convening role and so needs to develop mechanisms for stakeholders to continue collaborating once the programme ends. Taking a lesson from MM4P in Malawi, the formation of an industry association with funding contributed by members appears to be a sustainable approach to having this convening function continued. MM4P should look to set up similar industry associations in countries with DFS working groups, and in Nepal where a DFS working group has yet to be formed.

Management Response: [Added: 2019/02/05]

MM4P aims at sustainability of its engagement in the market, including the ecosystem activities. In 2019, MM4P will engage in specific activities to ensure the sustainability of its facilitation role in the countries where the programme is closing. Note that the facilitation role is mostly aimed at improved public-private dialogue and ensuring greater consultation around policies and regulation.  As noted by the evaluators, UNCDF plays this unique role in many countries and this should be a “selling point” to funders for UNCDF’s continued presence in countries where sustaining such engagement is unlikely without outside funding and expertise.

Key Actions:

Key Action Responsible DueDate Status Comments Documents
5.1. Include activities to ensure sustainability of MM4P facilitation role after closing of a country programme in 2019 country workplans
[Added: 2019/02/05] [Last Updated: 2019/07/08]
MM4P 2019/06 Completed As of today, the only country that will be closed end of this year is Benin. For Senegal and Benin, we are continuing the implementation with new funders. Other countries (Zambia, Malawi, Uganda, Nepal, Malaysia, China, Senegal) are not closing. The country plan for Benin includes specific activities regarding sustainability. History
5.2. Include long-term plan for sustainability of facilitation role in country strategies
[Added: 2019/02/05]
MM4P and all FIPA countries adopting new strategy as of 2019 2019/12 Not Initiated
7. Recommendation:

Evaluation Recommendation or Issue 6:

 

MM4P can better incorporate the mainstreaming of gender into its programming activities over the second half of the programme. To date, requiring providers to collect sex-disaggregated data has been a tick-box exercise rather than encouraging providers to make use of the data. Programming should therefore focus on building the business case for using sex-disaggregated data among providers as part of MM4P support. This could include capacity building/TA or research such as market sizing and opportunity analysis for female customers, encouraging providers to make use of their data to tap in to latent opportunities, or using the data to show providers where women are desirable clients to attract. In addition, as programming matures to specific DFS use cases, there is scope for the programme to focus on use cases and sectors that are known to have a livelihood impact on women in particular, such as digital education and healthcare payments. The work that has been completed already on Zambian mothers and female farmers in Nepal, as well as the DFS4Women partner even that led to Orange Liberia rolling out a pilot on onboarding female agents, is promising in this regard.

Management Response: [Added: 2019/02/05]

Recognizing that women represent a substantial development and business opportunity for providers, supporting activities that address the unique barriers and needs of women has been one of the priority areas of MM4P since inception. The programme has implemented several projects specifically targeting women, conducted focused research on the barriers and needs of women, included sex-disaggregated indicators in reporting templates developed for and adopted by regulators for market monitoring, and organized a global partner event with the theme DFS4Women. Market research is disaggregated by sex, and MM4P requests sex-disaggregated data from all partners as part of their reporting requirements.

 

Generally, stakeholders’ interest in better reaching women with appropriate DFS remains low. This is also reflected in the limited number of stakeholders that monitor sex-disaggregated KPIs, and the challenges that persist in partner reporting on sex-disaggregated data. The programme recognizes the need to improve its efforts in better reaching women. As part of the new strategy MM4P will identify how to better mainstream the gender perspective across programme activities, incorporating the programme lessons learnt so far. This also includes an approach on improving partners’ data collection, analysis and reporting of sex-disaggregated results. The strategy also specifically focuses on how programme activities can contribute to women empowerment, which will be incorporated and tailored to country strategies and activities.

It should be noted that MM4P experience shows that at earlier stages of DFS market development, when the commercial potential of DFS is not yet recognized by most providers, it is less likely stakeholders show interest in increasing their reach to women. Providers operating in more advanced DFS markets are more likely to prioritize better reaching women.

Also, the programme expects an increased interest in women from Fintech and start-up companies that are entering markets with specific products and services. These types of companies generally adopt a more customer centric approach. In its new strategy, MM4P will leverage and support these players in addressing the barriers and needs of women. 

Finally policy and regulatory pressure make a difference. Where private sector “buy-in” is lagging, UNCDF may need to focus on external pressure to get the basic prerequisite of data collection and reporting on gender on the industry’s agenda.   

Key Actions:

Key Action Responsible DueDate Status Comments Documents
6.2. Mainstream gender and design focus on women empowerment in global and country strategies
[Added: 2019/02/05] [Last Updated: 2019/07/09]
MM4P and all FIPA countries adopting new strategy as of 2019 2019/06 Completed MM4P and FIPA have updated the gender strategy as part of the new strategic framework. In the strategies for Benin, Senegal, Zambia, Nepal, there are specific activities regarding gender mainstreaming and women empowerment. The same framework is used within MM4P and also FIPA. History
8. Recommendation:

Evaluation Recommendation or Issue 7:

 

Knowledge management activities need to capture and disseminate lessons on project failures as well as project successes. Thus far, MM4P’s KM activities have largely focused on successful projects with limited coverage of the challenges and failures experienced by providers. This information is equally useful for stakeholders in the ecosystem to learn from the experience of other providers, and more focus should be placed on covering these lessons on failure. KM activities can also disseminate learnings on failures throughout a project’s lifecycle, rather than waiting for the end of a project to publish lessons[1].

 


[1] See http://www.cgap.org/blog/series/embracing-failure-ultimate-success-branchless-banking and https://www.raflearning.org/post/smallholder-finance-big-learnings-start-sharing-real-failures

Management Response: [Added: 2019/02/05]

 

It is important to differentiate projects that have failed (and were discontinued) from failures within projects that MM4P used as learnings to adapt the projects for better impact. For the latter, MM4P widely shared successes but also challenges faced in various projects.  This has been for example the case for the various agriculture value chains and all projects. But this can be improved in future publications.

 

For the former, we have some projects that have been discontinued for various reasons (partnership not found between partners, disengagement of the government, etc.).  Partners often do not want to publicly speak about these failures and this is a challenge for MM4P.  For these projects, MM4P will analyze best ways to disseminate lessons from these projects with the concerned partners.

Key Actions:

Key Action Responsible DueDate Status Comments Documents
7.1. Review KM/RM processes and templates to better capture lessons learnt on failures and challenges along the project life cycle.
[Added: 2019/02/05]
MM4P/KM RM 2019/12 Not Initiated
7.2. Include publications on challenges and failures in country KM workplans
[Added: 2019/02/05] [Last Updated: 2019/03/29]
MM4P/KM 2019/03 Completed Included History
9. Recommendation:

Evaluation Recommendation or Issue 8:

Country teams will have to carefully consider which stakeholders to partner with as programming matures. Since MM4P has established its credibility in the markets where it operates, country teams are receiving more requests for support. This evaluation has found that MM4P support is most impactful when it is longer-term and where in-country DFS experts are embedded in the partner organisation.  While it was appropriate for the programme to experiment with different partner types at the beginning of the programme, this suggests that MM4P should now focus on fewer projects with larger budgets and longer ongoing technical support. This does not necessarily mean that MM4P should only partner with a particular partner type, or those that it has partnered with before. For example, working with fintechs may become a priority in the future as DFS market development continues. Given that fewer partners equate to higher risk, these partnerships must have a high focus on sustainability and demonstration of a viable business case as selection criteria.

Management Response: [Added: 2019/02/05]

The maturity of the market and the depth of the relationship with a partner are critical factors when choosing partners.  MM4P engages with and supports stakeholders in the market through its full-time in-country presence of digital experts. In its engagement with market players, MM4P plays the role of a neutral broker. These are key components of the programme that allow MM4P to continue to identify trends and opportunities in the market rather than building exclusive relationships.

 

The selection of projects and partners are determined by the existing market constraints and the scope of the engagement and design of projects reflect the need of the partners and the market as a whole. MM4P’s experience shows that partnerships with private sector players, particularly in new markets, require a buy-in phase that is done through one or two short term engagements, to establish the relationship, trust and show how the programme can support the partner. MM4P recognizes that longer term involvement with partners is beneficial and is assessing the appropriate approach as part of the new strategy—but not all partners wish longer term engagements and not all remain good (or necessary) partners over time.  The number and type of partners MM4P supports depends on the market constraints that MM4P aims to unlock, and the funding available in each country.

As part of its new strategy to better leverage Fintech and start-up companies to unlock market development, MM4P plans to provide shorter-term support to a larger number of such players. Through these engagements, players with the most viable business plan and largest potential for contributing to market development will be identified for longer-term support. The programme recognizes that the commercial viability of the business models developed with MM4P support is key in the sustainability of programme activities. As part of the new strategy, due diligence processes are reviewed as well as the monitoring systems related to project sustainability.

Key Actions:

Key Action Responsible DueDate Status Comments Documents
8.1. Detailed analysis of key market constraints in each country strategy (2019-2024) and potential activities and support to partner
[Added: 2019/02/05] [Last Updated: 2019/07/08]
MM4P and all FIPA countries adopting new strategy as of 2019 2019/06 Completed A detailed analysis of key market constraints is a key pillar of the new strategic framework as we focus on market development for inclusive digital economies. Ecosystem activities and partners activities are planned based on the identified constraints. Our objective is to unlock these constraints to develop sustainable development of the market in the countries where we operate. It has been done for Benin, Senegal and Zambia. Again the framework is used for MM4P and beyond. All FIPA countries used the same framework and this is part of all country strategies. History
8.2. Review due diligence processes
[Added: 2019/02/05]
MM4P 2019/09 Not Initiated
8.3. Improve monitoring processes of project sustainability
[Added: 2019/02/05]
MM4P/RM 2019/09 Not Initiated
10. Recommendation:

Evaluation Recommendation or Issue 9:

 

UNCDF’s procurement and selection process for partners should be reviewed given the changing dynamics in the market. With the emergence of new types of financial service providers, such as fintechs which tend to be more innovative and fast and flexible, UNCDF will have to adapt its procurement process for partners to reflect these changing market dynamics. This should be achieved through a review of the current process against its ability to respond to the current and future needs of the DFS market.

Management Response: [Added: 2019/02/05]

We understand that the recommendation refers to how MM4P adapts to the market dynamics.  Please note that (1) procurement is the process of acquiring goods and services for the organization whilst partner selection is a part of the (2) programming process, which enables UNCDF to provide technical and financial assistance to partners. Within this context both the procurement processes and programming guidelines (known as UNCDF’s operations manual) are decided upon by UNCDF senior management, whereas partner selection is largely the pure view of the programme management.

 

Since its inception, the programme has continuously looked at innovative and flexible ways to engage with partners, to be as reactive as possible in fast and dynamic markets.  With an increased focus on innovation and the willingness to work with start-ups and entrepreneurs, this is becoming a key priority for MM4P.

 

Partner selection and assistance:

MM4P’s method of partner selection is somewhat different than other programmes. Given the early failures in using request for applications and proposals, MM4P uses and a series of engagements that include training, direct technical support, contracted technical support and grants to help identify the best partners and project going forward.  It is a process of winnowing.  This works particularly well in nascent markets or within nascent stakeholder groups (e.g. Fintechs). 

 

MM4P is currently testing a model where technical assistance support (mentoring) and financial support (small grants) for innovation journeys with start-up companies are managed by an external company. This approach proposed by MM4P also led to the modification of the operations manual. Based on the learnings of implementing the approach in Nepal, Malaysia and Uganda in 2019, the model may also be adopted in other countries.

Procurement processes:

If MM4P is keen to working together with its programme management unit to continuously improve these processes to respond to the market needs. For example, in order to facilitate processes the programme proposed changes to UNCF individual consultant policy that have led to better working conditions to retain DFS experts. MM4P has also always been at the forefront of UNCDF when using agile procurement instruments such as Log Term Agreements and rosters. It agrees that working with FinTechs requires a rapid response approach and will prioritize this issue in its work and with the UNCDF MSU. 

Key Actions:

Key Action Responsible DueDate Status Comments Documents
9.1 Test new section approach to support start-ups and entrepreneurs
[Added: 2019/02/05] [Last Updated: 2019/03/29]
MM4P 2019/12 Initiated History
9.2 Enhance procurement plan and resource tracking system for the programne,
[Added: 2019/02/05]
PMU/MSU with MM4P support 2019/12 Not Initiated
11. Recommendation:

Evaluation Recommendation or Issue 10:

 

MM4P country teams will have to assess the capacity of their technical staff given the importance of longer-term TA provided directly by DFS experts in-country. Most of the country teams are currently coping with the pipeline of projects but would require additional in-country staff should this pipeline increase further. In cases where current technical specialists’ contracts are ending, it is important for the programme to replace these with in-country specialists that can develop long term relationships with partners. In countries where a greater focus on policy/regulatory work is required, the programme may need to consider widening its skill set to include staff with experience in policy and regulatory issues. 

Management Response: [Added: 2019/02/05]

Since its inception, MM4P has focused on developing a team with strong technical expertise in DFS, which has proven to be key in the achievements of the program.  The team is composed of UNCDF staff and long-term consultants.  Long-term engagement of our team on the ground with stakeholders is a unique asset for UNCDF.  A key priority of MM4P management has been to grow the team and to offer them opportunities in various countries.

 

MM4P recognizes the importance of continuing its efforts in building a team with strong DFS expertise, but also to increase the mobility of resources across teams, particularly in the case of funding gaps. 

 

Several areas should be investigated by UNCDF:

  • Contracts allowing to keep consultants for longer periods
  • Investigate career growth paths within UNCDF that include long-term consultants
  • Putting operational procedures in place to allow staff and experts to do short-term assignments across UNCDF
  • Plan DFS demand centrally with UNCDF and make budget available to fill-in funding gaps for key resources within UNCDF

Key Actions:

Key Action Responsible DueDate Status Comments Documents
10.1 Include contracts’ renewals and DF experts procurement in countries workplans
[Added: 2019/02/05]
MM4P 2018/11 Completed
10.2 Investigate career growth paths within UNCDF
[Added: 2019/02/05] [Last Updated: 2019/07/08]
HR with MM4P support 2019/06 Completed We started investigation and it will be a ongoing process to see best ways to keep our team to deliver the strategy. History
10.3 Plan DFS demand centrally with UNDF and make budget available to fill-in funding gaps for key resources within UNCDF
[Added: 2019/02/05]
FIPA Management 2019/09 Not Initiated
12. Recommendation:

Evaluation Recommendation or Issue 11:

The programme should consider how to strengthen its pre-project feasibility assessment to improve the likelihood that investments will lead to sustainable customer outcomes. In several cases, projects were unable to be implemented as they were planned in the design phase. The table below provides a few examples from the programme where this was the case. MM4P undertakes a thorough assessment of prospective partners before providing support, including a peer review of the project document by a technical specialist in another country and review by the investment committee. However, these examples suggest that the programme could benefit from other tools or frameworks for assessing viability, particularly methods for assessing whether the economics of a project makes sense (sufficient customer numbers and appetite, adequate infrastructure coverage, right regulatory approval processes in place, etc.). Learning from the programme thus far suggests that the first project with a partner should be short, with a relatively short feasibility assessment, since the team learns most about a partner and their feasibility by working directly with them. These small projects can act as the feasibility assessment for larger projects going forward.

Management Response: [Added: 2019/02/05]

It must be considered that the programme operates within the framework of the Least Developed Countries (LDCs). Consequently, the risks are higher than in other markets and there is often a “chicken and egg paradigm” (e.g. customer demand is weak because of the lack of infrastructure which is weak because of the lack of perceived customer demand).  Lacking the resources to fully address all sides of supply and demand, the programme has pushed some projects that may not have the perfect market conditions to build awareness, interest and hopefully a more attractive market opportunity or pressure to change regulations.

 

In the context of the new strategy, processes will be reviewed, simplified and strengthened. The risks are frequently discussed and are in the Project Appraisal Documents, however they could be better acknowledged as part of the pre-project feasibility. 

Kindly note that MM4P is also working in new tools (Toolkit for managing partner steering committees (SCs) as well as streamlining its’ monitoring tools (project works books).

Key Actions:

Key Action Responsible DueDate Status Comments Documents
11.1 Review and strengthen project design phase including due diligence
[Added: 2019/02/05]
MM4P/PM 2019/09 Not Initiated
11.2 Review and strengthen project workbooks
[Added: 2019/02/05]
MM4P/ PM 2019/09 Not Initiated History
11.3. Launch and implement toolkit for managing SCs
[Added: 2019/02/05]
MM4P/PM 2019/09 Not Initiated
11.3. Launch and implement toolkit for managing SCs
[Added: 2019/02/05]
MM4P/ PM 2019/09 Not Initiated
13. Recommendation:

Evaluation Recommendation or Issue 12:

 

MM4P should not focus solely on its market development indicators, and should simplify its country and project RM frameworks by significantly reducing the number of indicators it tracks. Priority indicators should be decided on by identifying a) priority outcomes for donors, b) indicators that the MM4P team frequently uses to make strategic and operational decisions. The programme’s market development stages provide a helpful framework for understanding market dynamics to inform programming decisions, but do not adequately capture MM4P’s contribution to shifts along the spectrum.

Management Response: [Added: 2019/02/05]

The programme Results Measurement Framework (RMF) was developed by a DCED expert, in close consultation with the Mastercard Foundation, one of MM4P key donors. The market development measurement reflects the priority outcomes of this donor. In addition to the market development measurement (top-down measurement), the programme also measures the results of its projects (bottom-up measurement).

 

It must also be noted that after the first year of implementation of the RMF, the programme has focused on the eight market development indicators to measure the progress of the markets. This reduced the number of indicators tracked on country level. The project monitoring tools have also been simplified to ensure the ability of the partner to report on the requested indicators.

The programme acknowledges the need to better include the aggregation of direct programme results in its monitoring and reporting, in combination with market development measurement. MM4P also recognizes the need to more systematically assess and document the contribution of its activities on market development, particularly of its ecosystem activities, and better use these insights to inform programming.

In the context of the new strategy, MM4P is currently updating the Theory of Change, and will develop the RMF for the new strategy, including indicators and measurement plan. The RM strategy will also be revised, including all RM-related processes, tools and templates.

Key Actions:

Key Action Responsible DueDate Status Comments Documents
12.2. Develop programme and country RMFs for new strategy including ToC, indicators and measurement plans
[Added: 2019/02/05] [Last Updated: 2019/03/29]
MM4P/RM 2019/09 Initiated History
12.3. Revise RM strategy (processes, tools, templates)
[Added: 2019/02/05] [Last Updated: 2019/03/29]
MM4P/RM 2019/09 Initiated History
12.2. Develop tracking and measurement plan for MM4P ecosystem contribution
[Added: 2019/02/05]
MM4P/RM 2019/09 Not Initiated

Latest Evaluations

Contact us

220 East 42nd Street
20th Floor
New York, NY 10017
Tel. +1 646 781 4200
Fax. +1 646 781 4213
erc.support@undp.org