Success message
error message
warn message
Mid-term Evaluation of the Making Access Possible (MAP) programme
Commissioning Unit: UNCDF
Evaluation Plan: 2018-2021
Evaluation Type: Others
Completion Date: 03/2020
Unit Responsible for providing Management Response: UNCDF
Documents Related to overall Management Response:  
1. Recommendation:

Management welcomes the findings and recommendations from the Mid Term Evaluation Report and is committed to evolve the MAP program based on those recommendations. Management welcomes the evaluation’s assessment that MAP was a timely and significant initiative - at a time when there was strong international and regional endorsement of developing national financial inclusion strategies (NFIS) - highlighting UNCDF’s forward-looking approach to market development. The combination of the constituents of the MAP approach - a comprehensive diagnostic incorporating a robust evidence base (in-depth demand and supply side data) combined with broad based stakeholder engagement and consultation to build local relevance and buy-in, as well as government ownership - was new and distinct, continuing to be relevant today. Management also acknowledges that the environment in which the MAP program was developed has also considerably changed, with the acceleration of digital finance solutions. The evaluation is quite useful and important for UNCDF to examine our diagnostics within the context of our “Leaving No One Behind in the Digital Era” strategy as well as the forthcoming report of the Digital Financing of the SDGs Task Force. We’re at a very exciting and daunting moment as digital is transforming the financial sector as well as the data and diagnostic tools available. The evaluation provides the opportunity to be forward looking as we consider our wider vision of digital economies and the range of tools in the rapidly evolving data and diagnostics space. When MAP started, it filled an important gap that existed at the time. Now, as the space has moved, we need to be more agile, and also have more tools and options to respond quickly to the shift countries are managing at any given point in time.

Management Response: [Added: 2020/04/17] [Last Updated: 2020/04/27]

Key Actions:

2. Recommendation:

The new opportunities in West Africa and with the Myanmar refresh will be a good area to seek to apply the lessons from this evaluation. In Myanmar there is funding available, in West Africa look at linkages with BCEAO.  Specifically the programme needs to work on in-country communication/dissemination, stakeholder processes, supporting government capacities and monitoring.  And ensure there are the skills within UNCDF to do this. In Nepal too, a strong and interactive engagement with the World Bank to maximise the programmatic implications of the new household survey would be beneficial.  [FIPA management, MAP hub]

 
Management Response: [Added: 2020/04/17] [Last Updated: 2020/04/27]

UNCDF Team to develop a 2020 Annual Work Plan (AWP) to capture the opportunities noted above as well as next phase (MAP) Data and Diagnostic programming that incorporates the findings and recommendations from the MTE that is primarily forward looking. This pivot will create the ability for UNCDF’s work in data and diagnostics to be more agile, responsive with more options to respond to where countries are at any given point in time and how quickly shifts are happening. Next stage design will focus on helping LDC countries be able to understand the shifts taking place and respond quickly. The data and diagnostic work will be fully integrated within UNCDF’s “Leaving No One Behind in the Digital Era” strategy and structure.” These specific opportunities noted in the recommendation will be examined in that context.

 

Key Actions:

Key Action Responsible DueDate Status Comments Documents
1.1. MAP Programme Manager and FIPA Team to develop a 2020 AWP to address the opportunities noted above. As part of 2020 AWP, MAP team to review how use MAP funding to increase support (both on data / diagnostic and stakeholder engagement processes) in key LDC countries with less capacity e.g. DRC, Malawi, Madagascar, Benin, Togo, and provide a plan to the UNCDF to enable implementation.
[Added: 2020/04/27] [Last Updated: 2020/09/30]
MAP Programme Manager, FIPA 2020/09 Completed Complete History
1.2. MAP Programme Manager and FIPA Team to develop the successor arrangement / programmatic document for next phase of Data and Diagnostic work
[Added: 2020/04/27] [Last Updated: 2021/01/20]
MAP Programme Manager, FIPA 2020/12 Completed Complete History
1.3. FIPA Investment Committee reviews and endorses the new programmatic document
[Added: 2020/04/27] [Last Updated: 2021/01/20]
MAP Programme Manager, FIPA 2020/12 Completed Complete History
1.4. UNCDF PAC reviews and endorses the new programmatic document
[Added: 2020/04/27] [Last Updated: 2021/07/16]
MAP Programme Manager, FIPA 2021/06 Completed Revised MAP pro-doc presented to IC and approved for submission to PAC History
1.5. Define role of programme after roadmap adoption more clearly: 1.5.1. FIPA Team and MAP Programme Manager to review the range of tools deployed in other UNCDF programmes (DFS working groups (MM4P); National Financial Sector Strategy working groups (PFIP) as well as learnings from MAP evaluation and develop a range of options for LDCs for stakeholder engagement. 1.5.2. To capture opportunities mentioned above, strike (better) balance between diagnostic and implementation; the number of countries and in-country implementation activities; the sample size (representativeness) and the frequency of data collection and refreshes; the costs and time of data collection and the institutionalization and data usage capacity-building needs among LDC stakeholders, FIPA’s in-house coordination capacity and reliance on external stakeholder engagement.
[Added: 2020/04/27] [Last Updated: 2020/10/01]
FIPA, MAP Programme Manager 2020/09 Completed The prodoc for the MAP Successor Programme was developed. The MAP successor programme has been integrated into the UNCDF “Leaving No One behind” (LNOB) Strategy, under which all FIPA programming is being integrated, even as it continues to build on its past successes and experiences under MAP 2015 – 2020 to expand the frontiers of UNCDF in support of the objective of moving the dial towards the 2030 targets of the SDGs. History
1.6. Decide which tools to be deployed for MAP refresh diagnostics and follow up activities (including on the basis of the analysis mentioned under 1.5.1). Refresh diagnostics to also examine the challenges around implementation of national strategy and the inhibiting factors within Government which will inform the operational 1.6. Decide which tools to be deployed for MAP refresh diagnostics and follow up activities (including on the basis of the analysis mentioned under 1.5.1). Refresh diagnostics to also examine the challenges around implementation of national strategy and the inhibiting factors within Government which will inform the operational
[Added: 2020/04/27] [Last Updated: 2020/09/30]
MAP Programme Manager 2020/09 Completed Presentation History
1.7. FIPA to identify current skill set within FIPA and fully leverage FIPA’s digital policy accelerator and data and research teams to support policy implementation work at country level.
[Added: 2020/04/27] [Last Updated: 2020/10/01]
FIPA 2020/09 Completed Mapping of current skill set within FIPA on data / research as well as policy completed. This capacity will be leveraged to support the broader FIPA team in the field, in those areas. History
3. Recommendation:

Build transparency around costs within the programme – and monitor costs more closely, including parallel funding.  Technical partners should be required to share full cost details, even if work is not commissioned directly by UNCDF.  

 
Management Response: [Added: 2020/04/17] [Last Updated: 2020/04/27]

The Programme has noted the recommendation and has started gathering detailed costs and will incorporate this going forward.

 

Key Actions:

Key Action Responsible DueDate Status Comments Documents
2.1. Demand-side data collection methodology updated to include new technology, lowering costs further experiment with over-laying and linking demand and supply side data.
[Added: 2020/04/27] [Last Updated: 2021/01/20]
MAP Manager 2020/12 Completed Report completed History
2.2 Require technical partners to share full cost details, including parallel funding received from other development partners, even if work is not commissioned directly by UNCDF. Amend current agreements as needed.
[Added: 2020/04/27] [Last Updated: 2020/07/06]
MAP Manager 2020/06 Completed Both technical partners shared the information. History
4. Recommendation:

Commission a separate study to examine and try to benchmark the costs of national market surveys for financial (or other) services, taking into account appropriate sampling and quality of analysis

Management Response: [Added: 2020/04/17] [Last Updated: 2020/04/27]

FIPA will build transparency around costs not only within MAP, but also the range of other tools that FIPA is using for its diagnostic and data work.

Key Actions:

Key Action Responsible DueDate Status Comments Documents
3.1 Using in-house FIPA experience, engage with the World Bank, IFC and other development actors to compare costs of key surveys and make transparent to stakeholders multiple options (Findex, Intermedia, Finscope, national surveys), to help them think around core sampling design issues and costs, e.g. “sampling and representativeness vs the frequency of refreshes; costs and time of data collection; depth vs breadth of indicators; new vs readily available data , required data outputs within LDC stakeholders to ensure in-house usage of the data.
[Added: 2020/04/27] [Last Updated: 2021/01/20]
MAP Programme Manager, FIPA 2020/12 Completed Technical report completed History
3.2 Assess the needs and demand of institutionalization of national financial service market surveys within LDC stakeholders.
[Added: 2020/04/27] [Last Updated: 2021/01/20]
FIPA 2020/12 Completed Technical report completed History
5. Recommendation:

If there is a refresh or repeat, make the process leaner, less monopolistic in terms of technical support, have a more deliberate approach to identifying local consultants ensuring contextual knowledge of financial inclusion, be more adaptive and quicker in the delivery of reports.  As part of this, it should not be necessary to depend on a branded survey.  Ensure quality – but the demand survey does not have to be Finscope, at least outside SADC.  Retain the key elements of Finscope (essentially a robust sample focusing on people’s access, use and perceptions of financial services, allowing for key segmentation analysis), but:  make sure the segments are contextually the key ones, think of leaner periodic options – applying technology wherever feasible, align the indicators of a core questionnaire with updated international definitions of financial inclusion, drawing on relevant indiators from other survey processes, so as to enable cross-country comparison and eliminate current mis-matches in the key data sets.  Look at ways of building alternative capacities for this.  [FIPA management, more widely drawing  on skills of FIPA teams, including MAP hub]

Management Response: [Added: 2020/04/17] [Last Updated: 2020/04/27]

Given how fast the diagnostics and data space has moved and continues to move UNCDF will need to be more agile, and also have more options to respond to where countries are at any given point in time. Key is how quickly shifts are happening, countries need to be able to understand the shifts, and respond quickly. UNCDF’s focus depend on our ability to be agile and not have one instrument we’re locked into and not always responsive to what our countries need.

Key Actions:

Key Action Responsible DueDate Status Comments Documents
4.1 FIPA to complete the ‘diagnostics toolbox’ of the range of diagnostic tools leveraging digital that FIPA has deployed in parallel to MAP, while also doing a market scan of the range of emerging digital tools coming available. Develop a user friendly guide for LDC stakeholders to best match diagnostic instruments to the ‘problem they are trying to solve’ and available budgets.
[Added: 2020/04/27] [Last Updated: 2021/04/06]
FIPA, MAP Programme Manager 2020/12 Completed Diagnostic toolbox done, as well as data strategy report History
4.2 FIPA to develop a ‘stakeholder engagement toolbox’ based on the range of tools deployed in other UNCDF programmes (DFS working groups (MM4P); National Financial Sector Strategy working groups (PFIP) as well as learnings from MAP evaluation. Develop a user-friendly guide for LDC stakeholders to best match stakeholder processes to the ‘problem they are trying to solve’ and time/resources available.
[Added: 2020/04/27] [Last Updated: 2020/09/30]
FIPA, MAP Programme Manager 2020/09 Completed Complete History
4.3 Refresh evidence-base for existing countries: • MAP Refresh has updated the analytical lens to encompass the role of financial inclusion within the inclusive growth agenda; • Refreshes cost USD50,000 (not including FinScope) with the explicit mandate of understanding other data sources in country within national institutions; • Updated evidence-base with a shorter document
[Added: 2020/04/27]
MAP Manager 2020/03 Completed
6. Recommendation:

In the market analysis, build in a strong, clear analysis of gender disparities and the barriers to women’s access and use, so as to develop more empowering strategies to enable  women’s access and use of financial services particularly with new developments in digital finance where the gender gap is wide and increasing. Relatedly, ensure gender disaggregated data across all parameters (market segments) on the demand side and look at opportunities for disaggregated data on the supply side.  [FIPA, PMU, future technical partners]

 
Management Response: [Added: 2020/04/17] [Last Updated: 2020/04/27]

We accept that there were often a lack of recommended actions in the road map (or consequent) NFIS related to gender which figures in the evaluation findings. Going forward, UNCDF’s Digital Strategy “Leaving No One Behind in the Digital Era” integrates a strong focus on women’s economic empowerment as a key customer segment; within the ‘empowered customers’ workstream; as well as the need for sex disaggregated data within the policy workstream. A gender perspective is also included in most of the segmented analysis of the FinScope surveys and usually in overall roadmap targets. Some of the more recent roadmaps in Asia have incorporated a gender perspective to include policies for a gender sensitive strategy across financial services with an emphasis on gender disaggregated data.

Key Actions:

Key Action Responsible DueDate Status Comments Documents
5.1 In the ‘diagnostics toolbox’, ensure sex disaggregated data and gender analysis are clearly featured in the range of tools and their respective strengths.
[Added: 2020/04/27] [Last Updated: 2021/04/06]
MAP Manager 2020/12 Completed Diagnostic toolbox done In addition, additional insights note with gender focus produced History
5.2 Include gender lens in refreshes, as well as in updated refresh roadmaps. Within FIPA’s Policy Accelerator, consider capacity building for LDC regulators and private sector in gender smart policy making, product development and sex-disaggregated data analytics.
[Added: 2020/04/27] [Last Updated: 2021/04/06]
MAP Manager/FIPA 2020/12 Completed Gender lens included in updated MAP refresh methodology. Training and capacity building also undertaken in ASEAN (Jan 2020 workshop in Cambodia) and SADC (Jan 2020 Workshop Johannesburg). History
7. Recommendation:

The programme requires a more systematic accountability and monitoring linked to more attentive management/leadership within FIPA. [FIPA management]

Management Response: [Added: 2020/04/27] [Last Updated: 2020/04/27]

UNCDF will fully integrate MAP within the “Leaving No One Behind in the Digital Era” strategy and structure. To support this effort to integrate the work we do on data and diagnostics into our programmes, FIPA will create a management committee composed of the Regional Digital Hub Managers and FIPA Directorate to provide leadership and oversight and ensure integration of data and diagnostics work within regional strategies.

Key Actions:

Key Action Responsible DueDate Status Comments Documents
6.1 FIPA Management to establish management committee for MAP for 2020
[Added: 2020/04/27] [Last Updated: 2020/07/06]
FIPA Management 2020/06 Completed Completed History
6.2 Management arrangements to be included in the concept note for MAP
[Added: 2020/04/27] [Last Updated: 2020/09/30]
FIPA Management 2020/09 Completed Complete History
8. Recommendation:

Pay more attention to the structures and technical support necessary for effective stakeholder engagement. [FIPA management, PMU]

Management Response: [Added: 2020/04/27]

Related to Recommendation 4, given how fast the diagnostics and data space has moved and continues to move, UNCDF will need to be more agile and also have more options to respond to where countries are at any given point in time. Based on how quickly shifts are happening, countries need to be able to understand those shifts and respond quickly. UNCDF’s added value depends on our ability to be agile and not have one stakeholder engagement instrument we’re locked into, and always be responsive to what countries need. FIPA will review the PMU MAP management and delivery structure to ensure it allows implementation to be aligned with these rapidly evolving needs Current typical elements of MAP coordination have included: • Country coordinator and the ‘in-country champion’. • Host ministry, a National Steering Committee, with in-country secretariat evolving into a special group on implementation. • Defined research, implementation and M&E processes, including the roadmap and the action plan. • The identification and implementation of one or more priority projects through which implementation is achieved. Based on the best fit of ‘stakeholder engagement tools’ (see 7.1 below) discuss with host countries the best option to enhance and support UNCDF’s “Leaving No One Behind in the Digital Era” strategy for country Governments.

Key Actions:

Key Action Responsible DueDate Status Comments Documents
7.1 FIPA to develop a ‘stakeholder engagement toolbox’ based on the range of tools deployed in other UNCDF programmes (DFS working groups (MM4P); National Financial Sector Strategy working groups (PFIP) as well as learnings from MAP evaluation.
[Added: 2020/04/27] [Last Updated: 2020/09/30]
FIPA, MAP Programme Manager 2020/09 Completed Complete History
7.2. Develop in-house staffing (team) to support this wider stakeholder engagement including policy and data and research specialists to support evidence-based policy making.
[Added: 2020/04/27] [Last Updated: 2021/04/06]
FIPA Management 2020/12 Completed MAP annual workplan completed with inputs received from the regional digital managers. History
9. Recommendation:

Strategically, there is clear scope to link in key elements of the MAP approach within the new FIPA strategy whilst addressing the gaps and building on the lessons: i.e. having a holistic country level frame of reference, highlighting the utility of ‘good enough’ standardized, demand side data, engaging systematically with all stakeholders – government (different departments)/regulators, development partners and commercial sector from the start, and ensuring regular follow up (with technical assistance) to monitor policy actions, programme interventions and supply side trends. For demand side data on financial inclusion, the widely accepted Findex survey is not currently funded beyond its third round that was completed in 2017. This will leave a huge market gap on financial inclusion data at country level which will need to be addressed. Look at what is useful about Findex and see how the market demand survey can be adjusted to fill the gap that there will be without Findex. [FIPA management, PMU, future technical partners]

Management Response: [Added: 2020/04/27]

UNCDF will pro-actively link the MAP approach with related learnings / recommendations from the evaluation to its new strategy and in doing so seek to addressing existing gaps and build on lessons learnt. UNCDF will especially look at linking MAP approach to in-country dialogue platforms, demand and supply side data-driven diagnostics, and stakeholder engagement processes. In doing so, UNCDF will help governments harness the power of data and data analytics for evidence-based decision-making process, and to work collaboratively and strategically with other sectors to meet their citizens’ needs. Within this context, it will be important to further investigate the potential to share data initiatives and engage in data collaboratives with different donor partners on a more systematic level from a UNCDF wide perspective. In addition to the activities outlined in recommendation 1, the following actions are outlined below.

Key Actions:

Key Action Responsible DueDate Status Comments Documents
8.1 FIPA to engage with the World Bank and donor partners to determine potential areas to collaborate on surveys and data sharing.
[Added: 2020/04/27] [Last Updated: 2021/07/16]
FIPA, MAP Programme 2021/06 Completed MAP PMU has engaged with the World Bank Poverty unit and are in discussions to collaborate on the LSMS surveys and for UNCDF to include the financial inclusion module. Collaboration already underway in Nepal. History

Latest Evaluations

Contact us

1 UN Plaza
DC1-20th Floor
New York, NY 10017
Tel. +1 646 781 4200
Fax. +1 646 781 4213
erc.support@undp.org