Microlead Mid-term Evaluation

Report Cover Image
Evaluation Plan:
2011-2011, UNCDF
Evaluation Type:
Others
Planned End Date:
05/2012
Completion Date:
02/2013
Status:
Completed
Management Response:
Yes
Evaluation Budget(US $):
150,000

Share

Document Type Language Size Status Downloads
Download document Microlead_MIDTERM_0213_ENG.pdf report English 3490.17 KB Posted 617
Title Microlead Mid-term Evaluation
Atlas Project Number:
Evaluation Plan: 2011-2011, UNCDF
Evaluation Type: Others
Status:Completed
Completion Date:02/2013
Planned End Date: 05/2012
Management Response:Yes
Focus Area:
  • 1. Poverty and MDG
Corporate Outcomes (UNDP Strategic Plan 2008-20013)
  • 1. Capacities of national and local institutions enhanced to scale up proven MDG acceleration interventions and to plan, monitor, report and evaluate the MDG progress in the context of related national development priorities
  • 2. Inclusive growth and social equity promoted through pro-poor macroeconomic and fiscal policies that support income, employment and social protection of youth, women and vulnerable groups in a sustainable way
  • 3. Policies, strategies and partnerships established to enhance public-private sector collaboration and private sector and market development that benefit the poor and ensure that low-income households and small enterprises have access to a broad range of financial and legal services
Evaluation Budget(US $): 150,000
Source of Funding:
Joint Programme:No
Mandatory Evaluation:No
Joint Evaluation: No
Evaluation Team members:
Name Title Email Nationality
GEF Evaluation:No
Key Stakeholders:
Lessons
Findings
Recommendations
1 Relevance: - Ensure future investments have multiple, vested programmatic and expert input to design (e.g., CTAs and TSPs with accountability structures and or contractual responsibilities to support and or execute PBAs). - Carefully evaluate the relevance of greenfield investments, taking into account the relative advantages of different kinds of risk-taking related to business models. - Ensure MicroLead investment strategy is well aligned with FIPA mandate and intervention logic, as well as a projected evolution of the target inclusive financial sector development. - Develop a quantitative due diligence tool to systematically identify and measure all types of investment risk and FSP/sector additionality expectations.
2 Effectiveness: 1. Document the different impulses or influence behind business models at the investment structuring stage of investment and monitor through life of PBA. 2. Increase staff resources for monitoring investee progress beyond quarterly reports (e.g., catalytic capital, macro/meso influences, market leadership effects, etc.) 3. Consider consulting support (PBA required) to monitor and support greenfield investments. 4. Set clearly articulated, measurable and accountable financial sector development outcomes, including macro, meso and knowledge generation targets. 5. Require clear documentation of UNCDF?s intended/expected investment input value-added during due diligence, in investment decision reports, and in PBAs. 6. Have investees disclose funding from all sources (amount and purpose) at the time of investment as well as annually. 7. Produce case studies of MicroLead-CSP synergies leading to improved outcomes. 8. Produce case studies on MicroLead?s experiences with greenfield investments. 9. Establish key lessons learned and channel these lessons into programme and management guidance.
3 Efficiency: 1. Increase staffing through: a. Formalization of roles, responsibilities and accountability mechanisms of RTAs, CTAs and other, internal UNCDF stakeholders (e.g., job descriptions, staff incentives and staff appraisals); b. Increase programme staffing by two full-time-equivalent (programme staff and/or administrative assistant and knowledge management expert). 2. Establish discretionary budget for rapid response/ support for FSP experiencing difficulties. 3. Link financial and development outcomes to disbursement milestones. 4. Expand the standardized reporting template to incorporate more reporting on knowledge generation, client protection principles and other innovations. 5. Review the structure of the incentives in PBAs and consider adding/enforcing repayment of grants due to non-performance, and positive incentives for performance that, by the end of the PBA period, exceeds targets.
4 Impact: 1. Fund short term TA to boost savings innovation (for women/outreach impact) particularly in countries lacking CTAs or TSPs. 2. Set PBA soft targets as a means to: a. Encourage women in senior management and Boards of Directors positions; b. Compliance with national environmental legislation and regulation; c. Encouraging basic environmental loan screening; and d. Promotion of environmental products and services and synergy with CleanStart. 3. Consider a simple social and environmental monitoring system, or scorecard (based, for example, on a simplification of the Global Reporting Initiative format). 4. Publish short, actionable articles on impact enhancement products/services (improving GNI, contribute to women?s economic empowerment, biogas finance, etc.).
5 Sustainability: 1. Develop tools to better track, verify and report on performance in prioritized areas, including aggregated and disaggregated sustainability trend analysis for stronger monitoring, analysis and decision-making. 2. Develop measures to report on financial sustainability trends including a MicroLead/donor dependency ratio. 3. Review and reconsider MicroLead FSP crisis intervention policies. 4. Designate funding for crisis management. 5. Develop a clearer exit policy/strategy for MicroLead.
6 Management Value Added: 1. Add two full-time staff (management assistance and knowledge generation). 2. Articulate RTA and CTA roles and responsibilities related to MicroLead. 3. RTA and CTA MicroLead/GTI management training, including incentivization of, sensitization to and concrete guidance on potential synergy and value add to country and regional portfolios. 4. Develop governance consistent with Fund risk profile including proactive management intervention policy and independent voices (i.e., not related to UNCDF or main funders). 5. Provide proactive problem resolutions tools (e.g., discretionary S/LTTA funding, pre-negotiated PBA management intervention clauses, positive PBA incentives). 6. Operationalise budget and measure knowledge generation targets. 7. Define, track and report on non-financial impact targets (e.g., rural clients, rural branches, direct and indirect management gender and environmental decisions etc.). 8. Define and track specific non-FSP strategic Fund priorities (e.g., leadership positioning, knowledge generation, innovative products, CPP focus, etc.).
1. Recommendation: Relevance: - Ensure future investments have multiple, vested programmatic and expert input to design (e.g., CTAs and TSPs with accountability structures and or contractual responsibilities to support and or execute PBAs). - Carefully evaluate the relevance of greenfield investments, taking into account the relative advantages of different kinds of risk-taking related to business models. - Ensure MicroLead investment strategy is well aligned with FIPA mandate and intervention logic, as well as a projected evolution of the target inclusive financial sector development. - Develop a quantitative due diligence tool to systematically identify and measure all types of investment risk and FSP/sector additionality expectations.
Management Response:

Under the MicroLead Expansion (MLE) programme (the 'phase II' of MicroLead), RTAs and CTAs were consulted during programme design, application review, and investment decision making (CTAs were asked to confirm support for applications prior to IC). An external expert was recruited to provide expert input into the MLE. She participated in the ICs, is available for consultation, reviews donor reports, and participates in donor calls. Her expertise has proven highly valued by UNCDF as well as the donors (Gates Fdn and MCF). In the MLE application process, few greenfield applications were received. Of those received, the country context (need for greenfield) as well as applicant focus on BOP was taken into consideration, resulting in seven greenfield approvals (one in CAR, one in Nigeria, one in Burundi, and four new credit unions in Liberia). MLE investment strategy was aligned with FIPA?s mandate and the development of the sector via a RFA process which awarded innovations (in particular via technology) and a focus on women and rural markets, with financial education components. A new due diligence tool was developed for the MLE RFA process and site visits were conducted to all shortlisted applications.

Key Actions:

Key Action Responsible DueDate Status Comments Documents
1.1. Multiple, vested programmatic and expert input to design MicroLead Programme Advisor (MLPA) 2013/02 Completed External expert hired; RTA and CTA input/support for applications; Majority of grants to TSP/FSP with joint responsibility.
1.2. Evaluate relevance of greenfield investments MLPA 2013/02 Completed Done during 2012 MLE IC
1.3. MicroLead investment strategy well aligned with FIPA mandate and intervention logic, as well as projected evolution of target inclusive financial sector MLPA 2013/02 Completed Done during 2012 MLE IC; technology-enhanced applications in all MLE projects (agent or m-banking or both) plus financial education components, women and rural focus.
1.4. Develop a quantitative due diligence tool MLPA and external consultants 2013/02 Completed Done during 2012 MLE RFA process-new assessment tool developed, including spider graph, risk ranking and defined value addition.
2. Recommendation: Effectiveness: 1. Document the different impulses or influence behind business models at the investment structuring stage of investment and monitor through life of PBA. 2. Increase staff resources for monitoring investee progress beyond quarterly reports (e.g., catalytic capital, macro/meso influences, market leadership effects, etc.) 3. Consider consulting support (PBA required) to monitor and support greenfield investments. 4. Set clearly articulated, measurable and accountable financial sector development outcomes, including macro, meso and knowledge generation targets. 5. Require clear documentation of UNCDF?s intended/expected investment input value-added during due diligence, in investment decision reports, and in PBAs. 6. Have investees disclose funding from all sources (amount and purpose) at the time of investment as well as annually. 7. Produce case studies of MicroLead-CSP synergies leading to improved outcomes. 8. Produce case studies on MicroLead?s experiences with greenfield investments. 9. Establish key lessons learned and channel these lessons into programme and management guidance.
Management Response:

ML/MLE agrees that the programme was understaffed at the time of the evaluation and that KM was lacking. During the time of the MTE, recruitment was underway for a Programme Specialist (she was hired in April 2012). In agreement with the MLE donor, a second Programme Specialist was hired nine months earlier than contemplated (he was hired in April 2013). The two PS's cover Africa investments. Also, an amendment to the MLE programme document was executed which brought MLE to Myanmar where a Programme Specialist was hired July 2013. All MLE PBAs contain specific deliverables on KM. Value addition in each MLE project was presented at IC and noted in IC minutes. An annex was added to the standard PBA which requires the FSP to disclose funding sources on a quarterly basis. A consultant was hired in January 2013 to research and write four case studies on the ML programme. Based on programme progress, ML/MLE does not agree to produce a case study on ML-CSP synergies as the experience mentioned in the MTE (Rwanda) is specific to persons on the ground and other topics are of more interest to the IF community. ML has not documented 'impulses' or 'influence' behind business models as the recommendation is unclear and it is not apparent how to document an impulse. MLE has ?considered? consulting support for new greenfield investments but all greenfield investments are supported, via PBA, by parent organizations or technical NGOs so MLE does not agree to include additional support above executed grant amounts. ML/MLE does not agree to set macro and meso development outcomes as the programme is directed at the retail level. CSP are designed for intervention at all three levels, and GTI MM4P appears to also be focusing efforts at the macro level.

Key Actions:

Key Action Responsible DueDate Status Comments Documents
2.1. Increase staff resources for monitoring investee progress beyond quarterly reports MLPA 2013/07 Completed Programme Specialists hired in April 2012, April 2013 and July 2013
2.2. Set clearly articulated, measurable and accountable knowledge generation targets MLPA 2014/06 Completed KM targets included in all MLE PBAs
2.3. Require clear documentation of UNCDF?s intended/expected investment input value-added during due diligence, in investment decision reports, and in PBAs MLPA 2014/02 Completed Value add noted in IC presentation and IC minutes;
2.4. Have investees disclose funding from all sources (amount and purpose) at the time of investment as well as annually MLPA 2014/02 Completed Reporting requirement included in all MLE PBAs
2.5. Produce case studies on MicroLead?s experiences with greenfield investments. Establish key lessons learned and channel these lessons into programme and management guidance MLPA and external consultant 2014/06 Completed Case studies are underway, two of which pertain to greenfields. JANUARY 2015: Case studies published; lessons learned incorporated into MLE programme
3. Recommendation: Efficiency: 1. Increase staffing through: a. Formalization of roles, responsibilities and accountability mechanisms of RTAs, CTAs and other, internal UNCDF stakeholders (e.g., job descriptions, staff incentives and staff appraisals); b. Increase programme staffing by two full-time-equivalent (programme staff and/or administrative assistant and knowledge management expert). 2. Establish discretionary budget for rapid response/ support for FSP experiencing difficulties. 3. Link financial and development outcomes to disbursement milestones. 4. Expand the standardized reporting template to incorporate more reporting on knowledge generation, client protection principles and other innovations. 5. Review the structure of the incentives in PBAs and consider adding/enforcing repayment of grants due to non-performance, and positive incentives for performance that, by the end of the PBA period, exceeds targets.
Management Response:

Although roles and responsibilities for RTAs and CTAs vis-à-vis their support to ML/MLE are well defined in the Programme Documents, management acknowledges that UNCDF needs to make an effort in terms of accountability. Coordination between thematic initiatives and country programmes is an important issue that was discussed during the FIPA retreat in 2012 and 2013. Since then, some corrective measures have been put into place by FIPA management (e.g., more accountability of UNCDF CTAs regarding representation of thematic initiatives in their countries) and UNCDF will continue working actively on this agenda. Formalization of roles, responsibilities and accountability mechanisms of RTAs, CTAs and other, internal UNCDF stakeholders (e.g., job descriptions, staff incentives and staff appraisals) are the purview of FIPA management and are addressed in the portfolio review management response and thus will not be addressed here. MLE staffing was significantly increased from 2012-2013 (3 PS's and 1 KM Associate). Establishing a discretionary budget for rapid response/support for FSP experiencing difficulties is not possible based on current structure of PBA and donor requirement to spend down funds (e.g., cannot have fund set aside which, at the end of the grant, may not have been used). Positive financial incentives for performance that exceeds targets is not possible due to donor requirements to spend down funds; non-financial incentives will be provided via KM activities.

Key Actions:

Key Action Responsible DueDate Status Comments Documents
3.1. Increase programme staffing by two full-time-equivalent (programme staff and/or administrative assistant and knowledge management expert) MLPA 2013/07 Completed 3 Programme Specialists hired, and KM Associate hired (MLE pays 1/3 of costs of KM Associate)
3.2. Link financial and development outcomes to disbursement milestones MLPA 2014/02 Completed Included in MLE PBAs
3.3. Expand the standardized reporting template to incorporate more reporting on knowledge generation, client protection principles and other innovations MLPA 2014/02 Completed Included in MLE PBAs
3.4. Positive incentives for performance will be achieved via COP MLPA 2014/02 Completed Community of Practice of MLE grantees established; over the life of MLE, webinars, newsletters, etc will highlight those grantees with exceptional results/impact
4. Recommendation: Impact: 1. Fund short term TA to boost savings innovation (for women/outreach impact) particularly in countries lacking CTAs or TSPs. 2. Set PBA soft targets as a means to: a. Encourage women in senior management and Boards of Directors positions; b. Compliance with national environmental legislation and regulation; c. Encouraging basic environmental loan screening; and d. Promotion of environmental products and services and synergy with CleanStart. 3. Consider a simple social and environmental monitoring system, or scorecard (based, for example, on a simplification of the Global Reporting Initiative format). 4. Publish short, actionable articles on impact enhancement products/services (improving GNI, contribute to women?s economic empowerment, biogas finance, etc.).
Management Response:

Management agrees to set soft targets in PBAs regarding women and environment. Management does not agree to fund short term TA to boost savings innovation since TSPs are already engaged, nor does management agree to encourage basic environmental loan screening since ML is a programme on savings

Key Actions:

Key Action Responsible DueDate Status Comments Documents
4.1. Set PBA soft targets as a means to: a. Encourage women in senior management and Boards of Directors positions; b. Promotion of environmental products and services and synergy with CleanStart MLPA 2014/02 Completed Women targets for senior management and BoDs plus SEMS requirement included in MLE PBAs. Synergy with CleanStart will occur when projects are in the same countries
4.2. Consider a simple social and environmental monitoring system, or scorecard. MLPA 2014/02 Completed SEMS adoption by BoD required in all MLE PBAs
4.3. Publish short, actionable articles on impact enhancement products/services (improving GNI, contribute to women?s economic empowerment, biogas finance, etc.). MLPA and KM specialist 2014/12 Completed With the addition of the KM specialist, newsletter, webinars, etc. are under development. JANUARY 2015: Newsletters produced and webinars hosted during 2014
5. Recommendation: Sustainability: 1. Develop tools to better track, verify and report on performance in prioritized areas, including aggregated and disaggregated sustainability trend analysis for stronger monitoring, analysis and decision-making. 2. Develop measures to report on financial sustainability trends including a MicroLead/donor dependency ratio. 3. Review and reconsider MicroLead FSP crisis intervention policies. 4. Designate funding for crisis management. 5. Develop a clearer exit policy/strategy for MicroLead.
Management Response:

Management agrees to better track performance and has already implemented this with an in-depth portfolio analysis. Management does not agree with crisis intervention policies, designating crisis management funding or developing a clearer exit policy. As stated in the MTE, ML is a fund that invests in risky areas; it was never expected nor intended that all investments were ?successful?; management does not agree to implement funding for crisis management since grantees were chosen based on their ?market leader? characteristics which assumes these grantees are knowledgeable and able to handle crises. With regard to an exit policy, there is no expectation from ML grantees that there is further funding since all PBAs already clearly state this, and UNCDF exits an average of 30% of its FSP PBAs on an annual basis. Thus, management does not agree that a ML specific exit policy/strategy is needed.

Key Actions:

Key Action Responsible DueDate Status Comments Documents
5.1. Develop tools to better track, verify and report on performance in prioritized areas, including aggregated and disaggregated sustainability trend analysis for stronger monitoring, analysis and decision-making. MLPA 2013/04 Completed Detailed portfolio analysis completed in Q1 2013 based on 2012 yearend figures; analysis will be completed annually as per FIPA policy
5.2. Develop measures to report on financial sustainability trends including a MicroLead/donor dependency ratio. MLPA 2014/02 Completed All MLE PBAs contain reporting requirement of funding sources/future funding sources
6. Recommendation: Management Value Added: 1. Add two full-time staff (management assistance and knowledge generation). 2. Articulate RTA and CTA roles and responsibilities related to MicroLead. 3. RTA and CTA MicroLead/GTI management training, including incentivization of, sensitization to and concrete guidance on potential synergy and value add to country and regional portfolios. 4. Develop governance consistent with Fund risk profile including proactive management intervention policy and independent voices (i.e., not related to UNCDF or main funders). 5. Provide proactive problem resolutions tools (e.g., discretionary S/LTTA funding, pre-negotiated PBA management intervention clauses, positive PBA incentives). 6. Operationalise budget and measure knowledge generation targets. 7. Define, track and report on non-financial impact targets (e.g., rural clients, rural branches, direct and indirect management gender and environmental decisions etc.). 8. Define and track specific non-FSP strategic Fund priorities (e.g., leadership positioning, knowledge generation, innovative products, CPP focus, etc.).
Management Response:

Management agrees with parts of this recommendation (hire addition staff, include KM targets, define/track/report on non-financial targets and fund priorities). Management does not agree to other recommendations in this section, such as pre-negotiated PBA management intervention/proactive management intervention policy since UNCDF is not able to have direct intervention in its grantees. Although roles and responsibilities for RTAs and CTAs vis-à-vis their support to ML/MLE are well defined in the Programme Documents, management acknowledges that UNCDF needs to make an effort in terms of accountability. Coordination between thematic initiatives and country programmes is an important issue that was discussed during the FIPA retreat in 2012 and 2013. Since then, some corrective measures have been put into place by FIPA management (e.g., more accountability of UNCDF CTAs regarding representation of thematic initiatives in their countries) and UNCDF will continue working actively on this agenda. Formalization of roles, responsibilities and accountability mechanisms of RTAs, CTAs and other, internal UNCDF stakeholders (e.g., job descriptions, staff incentives and staff appraisals) are the purview of FIPA management and are addressed in the portfolio review management response and thus will not be addressed here.

Key Actions:

Key Action Responsible DueDate Status Comments Documents
6.1 Add two full-time staff (management assistance and knowledge generation). MLPA 2013/09 Completed 3 Programme Specialists hired in April 2012, April 2013 and July 2013; KM specialist hired in Sept 2013
6.2. RTA and CTA MicroLead/GTI management training, including incentivization of, sensitization to and concrete guidance on potential synergy and value add to country and regional portfolios. FIPA management team 2013/10 Completed Discussed during FIPA Technical Retreat
6.3. Develop governance consistent with Fund risk profile including independent voices (i.e., not related to UNCDF or main funders). MLPA 2013/12 Completed External expert hired, participated in MLE IC, continues to provide guidance and support; MLE PBAs
6.4. Provide positive PBA incentives. MLPA 2014/02 Completed See Key Action 3.4
6.5. Operationalise budget and measure knowledge generation targets. MLPA 2014/02 Completed Included in all MLE PBAs
6.6. Define, track and report on non-financial impact targets (e.g., rural clients, rural branches, direct and indirect management gender and environmental decisions etc.). MLPA 2013/12 Completed Included in all MLE PBAs
6.7. Define and track specific non-FSP strategic Fund priorities (e.g., leadership positioning, knowledge generation, innovative products, CPP focus, etc.). MLPA 2013/12 Completed Included in all MLE PBAs

Latest Evaluations

Contact us

220 East 42nd Street
20th Floor
New York, NY 10017
Tel. +1 646 781 4200
Fax. +1 646 781 4213
erc.support@undp.org