Joint Social Protection Programme Evaluation

Report Cover Image
Evaluation Plan:
2016-2021, Tanzania
Evaluation Type:
Final Project
Planned End Date:
03/2018
Completion Date:
07/2018
Status:
Completed
Management Response:
Yes
Evaluation Budget(US $):
40,000

Share

Document Type Language Size Status Downloads
Download document Terms of Reference - SP joint evaluation.pdf tor English 749.91 KB Posted 692
Download document SP JP Final Report.pdf report English 1486.54 KB Posted 368
Title Joint Social Protection Programme Evaluation
Atlas Project Number: 00085002
Evaluation Plan: 2016-2021, Tanzania
Evaluation Type: Final Project
Status: Completed
Completion Date: 07/2018
Planned End Date: 03/2018
Management Response: Yes
UNDP Signature Solution:
  • 1. Poverty
Corporate Outcome and Output (UNDP Strategic Plan 2018-2021)
  • 1. Output 2.1.2 Capacities developed for progressive expansion of inclusive social protection systems
SDG Goal
  • Goal 1. End poverty in all its forms everywhere
SDG Target
  • 1.3 Implement nationally appropriate social protection systems and measures for all, including floors, and by 2030 achieve substantial coverage of the poor and the vulnerable
Evaluation Budget(US $): 40,000
Source of Funding: SDG-F, UNDP
Evaluation Expenditure(US $): 16,380
Joint Programme: Yes
Joint Evaluation: Yes
  • Joint with UN Agencies
  • Joint with UNICEF, UNFPA, ILO
Evaluation Team members:
Name Title Email Nationality
Birgitte Woel KENYA
GEF Evaluation: No
Key Stakeholders: UNICEF, UNFPA, ILO, TASAF, Ministry of Finance, National Bureau of Statistics
Countries: TANZANIA (UNITED REPUBLIC OF )
Comments:

This is a joint programme terminal evaluation

Lessons
1.

- The strong emphasis on saving groups has proved very positive, as life opportunities have enhanced significantly and they are utilised. While being successful in introducing income generating activities, it is essential to introduce the next step in time, in this case market linkages, to avoid disappointments and subsequent loss of interest in taking charge of own life.

- The involvement of Tanzania Social Action Fund (TASAF) at all levels has worked well and will ensure sustainability. However, training staff long before the materials are ready may be ineffective and inefficient as the trainers were losing motivation at the time of the evaluation and may have forgotten some of the messages that they need to pass as trainers when the materials are ready. The ILO guide to LE had been tested and was ready for use. The remaining 200 youth had been trained after programme expiry showing that the ILO developed guide on Livelihood Enhancement (LE) made it possible to continue the training without Joint Programme (JP) support.

- At macro-level the continued support of the policy work despite a change from National Social Protection Framework to National Social Protection Policy was appreciated and will result in a far better document, as a policy will force budget allocation and thereby a stronger and more stable development of national social protection efforts. The response to the Government of Tanzania needs proved very relevant and effective in this situation.

- The provision of support for development of support systems such as the National Social Protection Framework and National Social Protection Policy, Poverty Monitoring System, nutrition materials, LE guideline, training of trainers was well received and early application of the support seems to have the potential to generate sustainable changes.

- The establishment of a joint programme goes beyond making a number of UN agencies deliver to the same ministerial response. In this case four agencies each deliver what was assumed relevant for the governmental needs and not what was realistic with the available resources. The agencies collaborated effectively as parallel entities, but did not make recorded efforts on involving in joint JP activities enhancing the ability to support TASAF through well-planned internal activities that could have generated a synergy effect making the agencies as group have a common, strong platform from which to engage in evidence-based, innovative activities, which could only happen because of the very composition of agencies.

- When looking at the multitude of complex deliverables over a three-year period, a feasibility study of JP man-power/effort requirements would have forced a choice, which could have resulted in fewer activities being fully completed. The incomplete deliverables were not solely due to a change in government priorities and late roll-out.


Findings
1.

- It is important that UN defines internal capacity before venturing into design and support of next TASAF phase to ensure definition of realistic achievements.

- Considering which donors may be relevant for more efforts at macro level management across capacity development management of Social Protection (SP) measures, PMS, Communication and Advocacy, Knowledge Management and other areas supported under TASAF III.

- It may as well be relevant to involve national and/or international research institutions with expertise in SP, technical training and other relevant areas, think tanks, relevant associations etc. To know whom to include there may need to conduct a needs assessment, which could provide exact ideas of relevant partners.

- The results matrix should have a clear line, or logic, from activity to output to outcome. The indicators should clearly reflect the desired outcome, as the indicator is the measure stick determining whether the outcome has been achieved. The indicator can therefore not concern latrines, when the outcome concerns establishment of TASAF management systems as in the latest results matrix as was the case in the evaluated results matrix. It is recommended to seek assistance for development of the next results matrix, as a correct matrix will both make the writing of the programme document and later monitoring and reporting be easy. It will further help having a common understanding of which support to provide/expect across all involved parties.

a) A stronger JP M&E system with relatively few key performance indicators enabling a joint learning from experience and subsequent engagement in such learning activities. Such planned learning would be an obvious JP activity, partly for JP internal learning – and when having developed a simple approach or other products, offer this approach as a support function to JP partners and UN agencies at large. There is little evidence-based recording of the innovative aspect of joint programmes. Focus is mainly on simplifying the administration.

- The National Social Protection Policy should be finalized and approved before the design of TASAF phase IV is completed with the aim to ensure targeted and coordinated support of the enforcement of the policy.

- Communication and Advocacy Strategy to be combine with a Knowledge Management System (KMS). The KMS could provide part of the information for general communication and advocacy activities. The KMS will provide details on who (donors, different levels of government, different categories of staff, newspapers, radio, TV etc.) shall know what, in which format (folder, report, one-page fact sheet, community information materials etc.)

- Capacity building in gender mainstreaming. Gender mainstreaming should be applied at macro, meso and micro levels of interventions. Not least at micro level it requires strong involvement of both sexes in taking responsibility for the Households and other family and community related developments.


2.

4.1 Level of Evaluation

Because of the short implementation time from May 2015-September 2017 and change of frame for the Joint Programme (JP) support this evaluation is not an outcome/impact evaluation, but does rather evaluate the process of inputs-outputs defining achievements where these were documentable. The roll-out ofa number of mainly meso-level activities awaited approval of the Social Protection Framework, while others like support to TASAF of integration of cross-cutting issues (gender, nutrition and youth) were implemented. It was, however, not approved. Instead the Government of Tanzania decided to merge the existing Social Security Policy with the proposed National Social Protection Framework and instead develop a Social Protection Policy.

This evaluation does only assess activities and results achieved by the JP within the programme period. The JP support is viewed as enabler for TASAF achievements. But this is not an evaluation of TASAF achievements per se.

The findings are organised so that they answer the questions raised in TOR (programme relevance)15and in the evaluation matrix16. The Evaluation Matrix questions are placed at the top of each chapter marked in blue. Since some of the questions overlap, some chapters answer more than one Evaluation Matrix question. Observations, which do not correspond with a specific Evaluation Matrix question, are added in the relevant chapter.


Tag: Strategic Positioning

3.

4.2 Programme Relevance

This analysis of the programme relevance will be divided into two: One analyzing the relevance of the support tothe Government of Tanzania (GoT), and the other analysing the relevance of the joint approach.

4.2.1Relevance of the joint programme content and structures

1)To what extent was the joint programme aligned with national development strategies and the UNDAP?

2)How has the joint programme contributed to solve the needs and problems identified in the design phase, in particular with reference to the baseline situation?

The programme was developed on request from TASAF and was a response to expressed needs. The JP was as a consequence relevant to GoT. When the Social Protection Framework was not approved, new needs arose to which the JP responded accordingly. In addition, the JP design was guided by a number of key issues that emanated from the World Bank/partners mission and the stakeholder consultations:

a)The PSSN programme design was a given and could not be changed. This meant that the joint programme could only focus on in complementarity and supporting the gaps of the existing structure, content and approach of the PSSN Programme;

b)The need to complete the NSPF was a priority for all stakeholders, including the TASAF Management Unit itself;

c)At design stage, TASAF III was supposed to reach about 275,000 households over five years with a resources envelope of $200 million. However, the scale-up of the PSSN now requires that TASAF reaches 920,000 households in 161 districts. The total resource requirement is estimated at about$400 million;


Tag: Relevance Joint UN Programme Programme/Project Design

4.

The JP support was provided as follows:

UNICEF supporting:

  • Training in sexual reproduction rights and Livelihood Enhancement (LE) to youth
  • Developing, testing and implementing a community toolkit, Stawisha Maisha, providing training of mothers in infant and young child nutrition during the days of time of cash transfer?
  • Development of TASAF Communication and Advocacy Strategy
  • Supporting development of Social Protection Framework, subsequently Social Protection Policy.

 


Tag: Communication Joint UN Programme UN Agencies Social Protection Capacity Building Youth

5.
  • Formulation of Social Protection Framework
  • Development of two statistical M&E packages?Development and integration of social protection indicators
  • Training of TASAF staff in M&E
  • Establishment of data capture centres in selected PAAs to enable PAAs to enter PSSN monitoring data electronically and thereby ensuring timely and quality monitoring and reporting
  • Development of social protection indicators and inclusion in NBS statistics
  • Support to conduct comprehensive gender assessment of the PSSN programme
  • Support to develop Gender action plan for TASAF and the PSSN programme based on the assessment
  • Support to develop gender mainstreaming training package

Tag: Gender Equality Joint UN Programme UN Agencies Social Protection Capacity Building Technical Support

6.

ILO supporting:

Establishment and implementation of sustainable livelihoods models among the youth from poorest households supported by TASAF through:

  • Support development of models for entrepreneurship development and business development services
  • Development of tailor-made training materials and guideline for LE activities
  • Conduct trainings among youth on entrepreneurship and business development and services and support them tapping of income generating opportunities through implementation of business plans
  • Development of sustainable support structures including Training of Trainers (ToT) for enhancing supportive supervision, coaching and hands-on support
  • Development of training materials and guideline for LE activities;
  • Support of youth in implementing their business plans; and?
  • Tapping into key sector plans to support Social Protection (SP) mainstreaming.

Tag: Joint UN Programme UN Agencies Social Protection Trade and Development Capacity Building Youth

7.

UNFPA supporting:?

  • Supported family planning outreach services with special focus on young people from beneficiaries households(working with Local Government (LG) staff, KIWOHEDE and Marie Stopes).
  • Supported Family Planning awareness creation and demand generation to empower and enable them to set their own priority.

Tag: Local Governance Youth

8.

The intention of the JP was to address multiple poverty factors parallel through:

  • Income opportunities
  • Better nutrition resulting in improved health and subsequently fewer visits to health clinics in combination with more energy to invest in income generating activities (IGAs) and schooling.
  • Family planning with the view to have fewer, healthier family members to cater for.

This parallel support addressing multiple vulnerabilities envisioned to increase resilience and implicitly protect families from shocks. Supporting the finalization of the Social Protection Framework the JP was in line with national policies and strategies. The non-approval by Cabinet and the change into support of development of a National Social Protection Policy did not change the fact that the JP support was fully in line with national priorities andstrategies including:

  • “National Strategy for Growth and Reduction of Poverty (NSGRP)” popularly known as MKUKUTA & MKUZA
  • The National Employment Policy (2007)
  • The National Food Security Policy and
  • The National Disaster Management Policy
  • Health Sector Strategic Plan III putting emphasis on the extension of healthcare to the poor and vulnerable, including supporting those with HIV/AIDS
  • The National Aging Policy (2003)
  • The Child Law Act (2009)?
  • The Persons with Disability Act (2009)?
  • The National Costed Plan of Action for the Most Vulnerable Children (2007-2010)
  • The National Education and Training Policy (1995) guarantees access to education and adult literacy for all citizens as a basic right.
  • Zanzibar Social Protection Policy (2013)
  • Umaskini Zanzibar (MKUZA I and II) (RGoZ, 2010). The MKUZA II facilitating the implementation of Vision 2020
  • Zanzibar Most Vulnerable Children Monitoring and Evaluation Plan (2015)

Tag: Relevance Nutrition Results-Based Management Education Social Protection

9.

4.2.2 Relevance of delivering as Joint Programme

3)To what extent was joint programming the best option to respond to development challenges described in the programme document?

4)To what extent have the implementing partners participating in the joint programme added value to solve the development challenges stated in the programme document?

11) To what extent were joint programme’s outputs and outcomes synergistic and coherent to achieve better results when compared to single-agency interventions? What efficiency gains/losses were there as a result?

With regard to programming the partners found that the joint initiative worked well. The planning allowing for specialised responses to the TASAF request enableda complementarity and coordination, which was continuously ensured through monthly coordination meetings between TASAF, the four JP agencies, World Bank and DFID all of whom support social protection.

After the programming the internal JP work as ONE was less. Each agency supported the agreed TASAF initiatives without assessing and/or defining potential synergy effects from one agency to another or from agency to TASAF, from agency to other implementing partners –and vice versa. The potential of “joint” was thus not fully exploitedj oint terms of e.g. joint research on synergy of merging exactly the JO scope of support as means to address SP or joint inspiration visits which would have illustrated the coherence of the supported activities. Synergy is the benefit that results when two or more agents work together to achieve something neither have achieved on its own.

Although UNICEF combined the LE with SRH and HIV/AIDS information it seems that having both UNICEF and ILO engaging in LE and with two different approaches seems inadequate in a JP, which should provide complementary expertise and not similar type of support. Despite the baseline emphasis on the need for quality educationto prevent intergenerational povertyt he JP did not deliverat all to this areaunder the JP, which could have been an obvious UNICEF contribution. The JP design and the baseline study highlighted needs for interventions did thus not tallyin an area of key importance for reduction of future poverty level.

 


Tag: Relevance Joint UN Programme Programme Synergy Country Government UN Agencies Capacity Building Coordination

10.

8) To what extent has the joint programme advance the national ownership processes and outcomes (the design and implementation of National Development Plans, Public Policies, UNDAF, etc.)

The JP has supported national development of SP relevant frameworks, plans, and policies firstly by supporting the development of the Social Protection Framework and subsequently by strongly supporting development and finalisation on the Social Protection Policy. The latter is spearheaded by UNICEF.

The Social Protection Policy will be informed by the revised Social Security Policy from 2003 and the Social Protection Framework. The entire work does thus make use of the best from existing national documents.

The National Social Protection Working Group(SPWG), chaired by MoFP was not active during the last programme year despite UN efforts to make it convene.

The intensive capacity building at both macro, meso and micro level all had the purpose of enabling MoFP, TASAF at central and devolved levels, relevant Local Government officers, village leaders, committees and beneficiaries to manage and implement SP activities across all levels.

The JP was managed and coordinated through use of existing structures. Overall guidance of the Joint Programme was provided by the already existing TASAF National Steering Committee (NSC). Members of the NSC were drawn from the public and private sectors and are appointed by the President of the United Republic of Tanzania.


Tag: Relevance Ownership Policies & Procedures Social Protection Capacity Building

11.

9) To what extent did the joint programme help to increase stakeholder/citizen dialogue and or engagement on development issues and policies?

The design was developed to support implementation of intervention by using the existing PSSN delivering structures –LGAs, ward, village and communities.The design approach built entirely on community engagement making communities decide whom among the villagers should be recipients of support activities. Village leaders and committee members have been trained as trainers supporting saving groups and conducting training in FP, HIV/AIDS and entrepreneurship. TASAF trainers experienced that communities have become very confident in handling own SP situation because of the full-scale involvement. The devolvement of decision-making reached from central to village level, which has made the villagers develop ownership of all cash plus efforts. This has resulted in that villagers do not wait for assistance to be brought, they take action when action is needed. The villagers make self-assessment after training to know the value, adjust the training to own conditions and explain to those who may not have fully understood.

There were problem in the beginning understanding the idea of the cash plus and JP, but intensive awareness and involvement have generated a full understanding, accept and engagement. The communities participated in monitoring and have participated in determining poverty indicators, which will be used for the national PMS.


Tag: Relevance HIV / AIDS Promotion of dialogue Social Protection Capacity Building

12.

10) To what extent was the joint programme’s management model (governance and decision-making structure, i.e. lead agency, Joint Programme Coordinator, Programme Management Committee and National Steering Committee, financial management and allocation of resources, i.e. one work plan, one budget) efficient in comparison to the development results attained?

12)What type of work methodologies, financial instruments, business practices did the implementing partners use to promote/improve efficiency

13)What type of (administrative, financial and managerial) obstacles did the joint programme face and to what extent have these affected its efficiency?

There was a routine monthly meeting between the four agencies chaired by One UN as part of coordination and reporting. There has been two occasion that the four agencies met jointly with their partners. However, as mentioned under programme design the JP could have worked more jointly with clear upfront definition of synergy effects and described best use of the competencies of each participating agency. In addition, there was no analysis of which competencies were required and which agency would best at delivering. However, there were JP joint initiatives including joint PSSN monitoring missions, joint biannual narrative and financial progress reporting, joint press/advocacy mission with the RC and Spanish Ambassador. The joint part overall concerned administrative matters and not making use of the scope of agency competences to develop new cross-agency approaches applying the wider scope of competences in a new and more effective manner.

The idea of having a JP was appreciated by TASAF as it meant having one system for financing, reporting, meetings etc. The JP management set-up was regarded as very effective by partners.


Tag: Relevance

13.

4.3 Effectiveness

This chapter does not present findings for question in the evaluation matrix although placed under this headline. Lessons learned are presented in a separate chapter

6)To what extent did the joint programme attain the development outputs and outcomes described in the programme document?.

The JP result matrix has no formulation of an overall objective. It consists of three outcomes as follows:

1)PSSN Programme implementation and delivery systems of the TASAF Management Unit, Regional Officers, District Councils and Communities strengthened

2)Social protection interventions are coordinated across sectors under National Social Protection Framework to address supply side issues

3)Sustainable livelihood and resilience mechanisms for the PSSN strengthened

The findings for each of the outcomes will be treated individually. The achievements given in the table refer to the achievements reported in the final SDGF report, October 2017 –at programme expiry.

The timeframe allocated for data collection did not allow for follow-up on earlier data and/orrevisit interviewees, conduct additional studies etc. The findings presented below is based on a mix of JP final reporting data, other data from the desk studies and data from the interviews.


Tag: Effectiveness

14.

4.3.1 Achievements under outcome

OUTPUT 1.2 Institutional capacities of the PSSN Programme implementation structures strengthened

There is no reporting against targets. Further, the results matrix holds no definition of which capacities and structures should be developed, which makes evaluation of deliverables against plans be difficult.The decentralisation in  LGAs and ROs is not reported on.

Part of the achievements reported in the final SDGF report, October 2017 apparently mix TASAF achievements with JP achievements as the reporting for example mentions number of people reached with cash transfer.

The Stawisha Maisha activity, addressing nutrition at the days of cash transfer, has continued since programme expiry. During the JP UNICEF supported development of a community engagement toolkit on nutrition, which, during the February evaluation, was in its final testing before being ready for national roll-out. District facilitators have been trained and Stawisha Maisha groups have been established in respectively Mbeya districts and in Zanzibar. A total of 100 sites in Mbeya and 20 sites in Zanzibar have established groups of 19-500 participants. The challenge has been to develop a nationally relevant material as the vast Mainland is very diverse, while Zanzibar has a different culture affecting the design of the materials.

The second training of the facilitators was still pending. The piloting was planned to be completed in quarter 2, 2018.

For UNICEF the involvement with TASAF has had positive spill-over effect to UNICEF activities in general through:

  • Supports to the nutrition-cash transfer linkage
  • Access to most vulnerable people for other UNICEF inputs as well through TASAF registration of vulnerable people
  • The opportunity to apply a multi-facetted approach combining LE + cash plus + sexual reproduction.

Tag: Nutrition Communication Cash Transfers Vulnerable

15.

OUTPUT 1.3 Robust PSSN Programme Monitoring and Evaluation and Management Information System (MIS) enhanced, including indicators on gender, climate change and other cross-cutting issues

The development of the M&E system was based on a gaps analysis concluding at that time that there was need for training in (i) construction of poverty indexes and poverty maps;

(ii) monitoring and evaluation frameworks for SP programmes and results-based monitoring, and (iii) Statistical packages.The MIS has faced some challenges among other because capacities are still inadequate.

There is need for provision of skills –not only training –in the following and an up-scaling to be meaningful:

  • Use of GIS
  • Design and use of graphics?
  • Development of simple, focused questionnaires?
  • Few districts connected. All need to be connected

The M&E department has realised that poverty analysis is critical, but that there is need to have something simple combined with longer-term with mentored hands-on practising before training others. That stage has not yet been reached. Thereby the target has not yet been achieved. The partner conclusion is that a real-time monitoring system with PSSN relevant data is a precondition for an effective and efficient implementation of the PSSN and later implementation of the Social Protection Policy.

 


Tag: Integration Monitoring and Evaluation Operational Efficiency Technology

16.

OUTPUT 1.4 Sector line ministry community extension workers capacity built to manage and implement Community Sessions

In relation to the cash plus LE groups there was late pairing of groups with extension service to ensure continued support. There is no availed reporting on this, as it took off around the time of programme expiry.The targets/deliverables are overall not in consistent with the output. However, UNICEF and UNFPA have delivered as planned within HIV/AIDS, Family Planning (FP) and Sexual Reproductive Health (SRH). Although made a target under output 1.4, it is reported on under output 3.2 in the final SDGF report.


Tag: HIV / AIDS Cash Transfers Capacity Building

17.

OUTPUT 1.5 NSPF and PSSN indicators incorporated into NBS national, thematic and panel survey instruments.

As mentioned above the M&E system at large was close to ready, but was experiencing various shortcomings although supported, one of them being development of simple, focused questionnaires for use at macro and meso level. As the questionnaire was not yet approved there has been no staff training and there has been no annual survey.


Tag: Operational Efficiency

18.

4.3.2 Achievements under outcome 2

OUTPUT 2.1National Social Protection Framework (NSPF) finalized and implemented

Since the GoT did not approve the Social Protection Framework and instead wanted to develop a Social Protection Policy all support changed. UNICEF is actively engaged in strengthening the Social Security Policy and merge it with the Social Protection Framework intentions.

A deliverable, which is not included in the framework, was under-way at the time of programme expiry. This concerns an institutional assessment of challenges, gaps and opportunities of implementing, coordinating and monitoring social protections. The assessment will identify institutional and budgetary bottlenecks and challenges that hinder optimum coordination, monitoring and implementation. The report is not yet made public. The outcomes are instead used for development of the National Social Protection Policy.

The targets were not reached and an unplanned activity had been added without any documentation of the why and when the decision was made.


Tag: Challenges Policies & Procedures Social Protection

19.

OUTPUT 2.2 A harmonised PSSN and NSPF coordination mechanisms established

The indicators and the PMS was not approved at the time of programme expiry. Since the PMS was not yet approved, there was no report on operational lessons.


Tag: Coordination

20.

4.3.3 Achievements under outcome 3

OUTPUT3.1 Pro-poor and child-sensitive social protection institutionalized at all levels to prevent inter-generational poverty

The development of the ZSPP Implementation Plan was a consultative process which involved range of participants from Government and Non-Government. The final version was validated in one day workshop chaired by the PS of MLEEYWC. Neither the ZSPP nor the Child Policy were ready, approved and rolled-out at the time of programme expiry.

 


Tag: Policies & Procedures Social Protection

21.

OUTPUT3.2 Mechanisms for strengthening medium to long term community and household resilience to risk and shocks developed

ILO supported training of 800 youth against the targeted 1,000 youth. The remaining 200 were trained after programme expiry. Some are still preparing their business, while quite a number have started and some have had to employ peers because the business has picked very well. This has enabled the involved youth to contribute towards their HH economy and engage in savings.

UNDP combined the first two activities and provided support in the development of TASAF LE framework. UNDP started piloting LE capacity development in 2017 in collaboration with TASAF in selected districts.

The adoption of LB approaches was dropped in the initial stages of the project

UNDP also provided support to integrate poverty-environment-gender linkages into the design of the LE Framework e.g. through learning visits to UNDP pilot projects, studies and technical assistance.

UNICEF supported development of materials on youth behaviour, which were not ready at the time of programme expiry.

The reporting referred to above is general reporting, which was done.

 


Tag: Effectiveness Capacity Building Youth

22.

OUTPUT3.3 Effective gender sensitive and sustainable livelihoods models tested and implemented

The achievement for output 3.3 are significant although mainly at output level. There are, however, clear indications that the deliverables are viable.

In Zanzibar a meeting groups of beneficiaries and local leaders showed that the JP LE complemented TASAF requirements for cash plus resulting in the women recipients starting saving groups and subsequently income generating activities(IGA) either individually or in groups. Some do both and have benefitted to an extent where they save from their IGAs and make further investments in e.g. additional agricultural productions.

In the Muslim communities, where women are not supposed to engage in manual labour in the field, the savings enable hiring of youth to do the manual work. This mean that agri-based production supporting the nutrition training and increasing income opportunities is now an IGA option. The training in agri-based LE had components of environmental measures to make sure that families can continue producing.


Tag: Effectiveness Gender Equality Cash Transfers Capacity Building

23.

4.4 Positive and negative effects of the JP intervention

Questions 16 and 17 will not be answered as it was agreed in the inception meeting to leave out evaluation of impact, as all documentable results are at output level.

14) To what extent and in what ways did the joint programme contribute to the SDGs?

The JP response delivered to eight out of the 17 SDGs plus tothe cross-cutting SDGgoal17: Partnerships to achieve the Goal.

On the side of negative effect it is noteworthy that when having a specific activity on gender mainstreaming the deliverables to SDG goal 5 and 10 were not amply gender balanced. It is recognised that the GoT has decided to pay the cash transfer to women as recipients on behalf of the HHs as GOT has experienced that these are better at making the payment be used optimum. Description of results in chapter 4.3 will confirm that the women have wholly utilised the cash opportunity. However, men have had difficulties understanding how they, as breadwinners, can be side-lined. The JP added to this by providing LE, sexual reproduction training and training in nutrition to women only. This has resulted in some unnecessary household challenges, which could have been addressed by involving men, for example in planning the development of the household. It has also made husbands and male youth not attend information meetings “since they are not paid”.This did not only refer to meetings held at payment days.

On the positive side the JP supported environmentally sound LE addressing women and youth with the aim to have more equality (goal 5 and 10), while securing nutritious food production (goal 1, 2 3 12 and 15), training women in composition of nutritious meals (goal 2 and 3) and providing training of women, men and youth in FP, SRH and HIV/AIDS(goal 3).


Tag: Effectiveness Gender Equality Cash Transfers Youth SDG monitoring and reporting

24.

15)To what extent and in what ways did the joint programme contribute to the targeted cross-cutting issues: gender mainstreaming and women’s empowerment, public private partnerships (PPPs) and sustainability at the local and national levels?

18)What unexpected/unintended effects did the joint programme have, if any?

On the negative side the inadequate gender mainstreaming is mentioned above.

The JP did not engage in public private partnerships due to delays in implementation and uncertainty with regard to need for support.

On the positive side women’s empowerment at the relatively few sites has had a very positive response, especially with regard to LE,and Stawisha Maisha groups are established although operations have not yet started.

With regard to FP, SRH and HIV/AIDS most inputs have been delivered, while the effects in terms of behaviour change cannot be assessed at this early stage. It was noted that the target group, being the poorest among the villagers, seemed more occupied with LE and to some extent nutrition than with FP, SRH and HIV/AIDS.

Sustainability is treated separately in the following chapter.

 


Tag: Effectiveness Women's Empowerment HIV / AIDS Nutrition Partnership

25.

4.5 Efficiency

With reference to chapter 2.2. the programme reported the utilisation of funds in the final report. From the SDGF funding all is either spent or committed, while the use of matching funds exceeded the budget.

Viewing this in the light ofd eliverables, which are mainly at output level, it must be concluded that efficiency has been low. However, with the PJ type of activities the most costly period would be the input-output period, while the move from output to outcome and impact for most of the delivered support is significantly less costly.

If looking purely at value for money it must concluded that the intended change in capacity, effectiveness of systems and structures under improvement and/or in people’s living conditions is yet to be seen both at outcome and impact level. This next level of delivery is the essence of development support. Therefore, if these many outputs are not lifted to the next level, this efficiency of this phase of support will be dissatisfactory.


Tag: Efficiency

26.

4.6 Sustainability

The sustainability analysis will cover the following evaluation matrix questions:

20) To what extent has the capacity of beneficiaries (institutional and/or individual) been strengthened such that they are resilient to external shocks and/or do not need support in the long term?

Since the outputs were delivered around the time if programme expiry the sustainability cannot be documented, but only be anticipated.

Starting at the micro-level effective and community managed systems are in place for all PSSN support activities. Cash plus beneficiary groups have been established all of which engage in savings for individual or group IGAs, for payment of school fees etc. Livelihood groups generating income at group level have been running for around one year, preliminary coordination of the extension support has been established and communities participate in all decision concerning cash plus and participate in the monitoring of the effect of cash plus. Systems are thus in place and seemingly operational.

At meso-level a national a wide range of outputs with sustainability potential have been delivered. This included: (i) PMS is under development; (ii)TASAF staff at central level have received training in M&E (insufficient)to make the M&E system be operational. When gaining new knowledge, unforeseen new needs for capacity are often revealed; (iii)staff has been supported in developing the details of the cash plus and the systems for community involvement; (iv)TASAF staff and local government staff from relevant line ministries have been trained and involved in development of materials for Stawisha Maisha; (v)guideline for establishment of LE, and (vi)family planning, HIV/AIDS and sexual reproductive healthis positively received and is regarded as an enabler –and precondition –for stable income and HH development.

 


Tag: Sustainability Knowledge management Capacity Building Cash Transfers

Recommendations
1

It is strongly recommended that UN defines internal capacity before venturing into design and support of next TASAF phase to ensure definition of realistic achievements.

2

Considering which donors may be relevant for more efforts at macro level management across capacity development management of SP measures, PMS, Communication and Advocacy, Knowledge Management and other areas supported under TASAF III.

3

It may as well be relevant to involve national and/or international research institutions with expertise in SP, technical training and other relevant areas, think tanks, relevant associations etc. To know whom to include there may need to conduct a needs assessment, which could provide exact ideas of relevant partners.

4

The results matrix should have a clear line, or logic, from activity to output to outcome. The indicators should clearly reflect the desired outcome, as the indicator is the measure stick determining whether the outcome has been achieved. The indicator can therefore not concern latrines, when the outcome concerns establishment of TASAF management systems as in the latest results matrix as was the case in the evaluated results matrix. It is recommended to seek assistance for development of the next results matrix, as a correct matrix will both make the writing of the programme document and later monitoring and reporting be easy. It will further help having a common understanding of which support to provide/expect across all involved parties.

5

A stronger JP M&E system with relatively few key performance indicators enabling a joint learning from experience and subsequent engagement in such learning activities. Such planned learning would be an obvious JP activity, partly for JP internal learning – and when having developed a simple approach or other products, offer this approach as a support function to JP partners and UN agencies at large. There is little evidence-based recording of the innovative aspect of joint programmes. Focus is mainly on simplifying the administration.

6

Finalization and implementation of the National Social Protection Policy. The policy should be finalized and approved before the design of TASAF phase IV is completed with the aim to ensure targeted and coordinated support of the enforcement of the policy.

7

Finalization, test and amendment of the PMS. The focus should be on simplicity in design to ensure usability across all levels of data collection and use. Development of a user-manual, which includes M&E responsibilities at each of the three levels of implementation. Finally, capacity development in poverty analytics.

8

Finalization and test of the Communication and Advocacy Strategy. It is recommended to combine it with a Knowledge Management System (KMS) as the intentions presented in the results matrix are more in line with KMS than with a Communication and Advocacy Strategy. The KMS could provide part of the information for general communication and advocacy activities. The KMS will provide details on who (donors, different levels of government, different categories of staff, newspapers, radio, TV etc.) shall know what, in which format (folder, report, one-page fact sheet, community information materials etc.).

9

Capacity building in gender mainstreaming. UN defines gender mainstreaming as: ““The process of assessing the implications for women and men of any planned action, including legislation, policies or programmes, in all areas and at all levels.”. What is essential is the consideration of implication for respectively women and men. Empowerment of the one sex without including the development of the other has resulted in gender imbalances over the decades of donor assistance – first with an unintended empowerment of male dominated areas primarily related to land (better production) and livestock (better producing and less sick animals), which later turned into a focus on women, which, in some areas more than others, resulted in negative male response to the HH development contrary to donor intentions. The emphasis on the mainstreaming where the needs, capacities and interests of both parties meet in united efforts to make their HHs, businesses or organizations develop is the essence of gender mainstreaming. Gender mainstreaming should be applied at macro, meso and micro levels of interventions. Not least at micro level it requires strong involvement of both sexes in taking responsibility for the HH and other family and community related developments. The ways and means will be locally different. Only involvement will decide the final approach and design.

10

Enhancing gender mainstreaming capacities by finalizing and rolling-out of the gender training package and supporting the integration of gender into all components of the PSSN through support to finalization and implementation of the gender action plan.

1. Recommendation:

It is strongly recommended that UN defines internal capacity before venturing into design and support of next TASAF phase to ensure definition of realistic achievements.

Management Response: [Added: 2018/07/06] [Last Updated: 2020/12/08]

Agree

Key Actions:

Key Action Responsible DueDate Status Comments Documents
With support from relevant agencies, the social protection outcome group lead (UNICEF) will take the recommendation forward.
[Added: 2018/07/06] [Last Updated: 2020/09/03]
UNICEF 2018/10 Completed done History
2. Recommendation:

Considering which donors may be relevant for more efforts at macro level management across capacity development management of SP measures, PMS, Communication and Advocacy, Knowledge Management and other areas supported under TASAF III.

Management Response: [Added: 2018/07/06] [Last Updated: 2020/12/08]

Agree

Key Actions:

Key Action Responsible DueDate Status Comments Documents
The social protection outcome group (through the outcome lead) will take the recommendation forward when developing any new social protection joint programme.
[Added: 2018/07/06] [Last Updated: 2020/09/03]
UNICEF, UNDP, ILO, UNFPA 2018/10 Completed done History
3. Recommendation:

It may as well be relevant to involve national and/or international research institutions with expertise in SP, technical training and other relevant areas, think tanks, relevant associations etc. To know whom to include there may need to conduct a needs assessment, which could provide exact ideas of relevant partners.

Management Response: [Added: 2018/07/06] [Last Updated: 2020/12/08]

Agree

Key Actions:

Key Action Responsible DueDate Status Comments Documents
The social protection outcome group (through the outcome lead) will take the recommendation forward when developing any new social protection joint programme
[Added: 2018/07/06] [Last Updated: 2020/09/03]
UNICEF, UNDP, ILO, UNFPA 2018/10 Completed done History
4. Recommendation:

The results matrix should have a clear line, or logic, from activity to output to outcome. The indicators should clearly reflect the desired outcome, as the indicator is the measure stick determining whether the outcome has been achieved. The indicator can therefore not concern latrines, when the outcome concerns establishment of TASAF management systems as in the latest results matrix as was the case in the evaluated results matrix. It is recommended to seek assistance for development of the next results matrix, as a correct matrix will both make the writing of the programme document and later monitoring and reporting be easy. It will further help having a common understanding of which support to provide/expect across all involved parties.

Management Response: [Added: 2018/07/06] [Last Updated: 2020/12/08]

Agree

Key Actions:

Key Action Responsible DueDate Status Comments Documents
The social protection outcome group will take the recommendation forward when developing any new social protection joint programme.
[Added: 2018/07/06] [Last Updated: 2020/09/03]
UNICEF, UNDP, ILO, UNFPA 2018/10 Completed done History
5. Recommendation:

A stronger JP M&E system with relatively few key performance indicators enabling a joint learning from experience and subsequent engagement in such learning activities. Such planned learning would be an obvious JP activity, partly for JP internal learning – and when having developed a simple approach or other products, offer this approach as a support function to JP partners and UN agencies at large. There is little evidence-based recording of the innovative aspect of joint programmes. Focus is mainly on simplifying the administration.

Management Response: [Added: 2018/07/06] [Last Updated: 2020/12/08]

Agree

Key Actions:

Key Action Responsible DueDate Status Comments Documents
The social protection outcome group will take the recommendation forward when developing any new social protection joint programme
[Added: 2018/07/06] [Last Updated: 2020/09/24]
UNICEF, UNDP, ILO, UNFPA 2018/10 Completed Done History
6. Recommendation:

Finalization and implementation of the National Social Protection Policy. The policy should be finalized and approved before the design of TASAF phase IV is completed with the aim to ensure targeted and coordinated support of the enforcement of the policy.

Management Response: [Added: 2018/07/06] [Last Updated: 2020/12/08]

Agree

Key Actions:

Key Action Responsible DueDate Status Comments Documents
The agencies - UNICEF, ILO and UNDP - involved will continue support the process
[Added: 2018/07/06] [Last Updated: 2020/09/03]
UNICEF, ILO, UNDP 2018/12 Completed done History
7. Recommendation:

Finalization, test and amendment of the PMS. The focus should be on simplicity in design to ensure usability across all levels of data collection and use. Development of a user-manual, which includes M&E responsibilities at each of the three levels of implementation. Finally, capacity development in poverty analytics.

Management Response: [Added: 2018/07/06] [Last Updated: 2020/12/08]

Agree

Key Actions:

Key Action Responsible DueDate Status Comments Documents
The recommendation will be proposed to be followed up by the UNDP Poverty Environment Initiative II project team
[Added: 2018/07/06] [Last Updated: 2020/09/21]
UNDP 2018/11 Completed done History
8. Recommendation:

Finalization and test of the Communication and Advocacy Strategy. It is recommended to combine it with a Knowledge Management System (KMS) as the intentions presented in the results matrix are more in line with KMS than with a Communication and Advocacy Strategy. The KMS could provide part of the information for general communication and advocacy activities. The KMS will provide details on who (donors, different levels of government, different categories of staff, newspapers, radio, TV etc.) shall know what, in which format (folder, report, one-page fact sheet, community information materials etc.).

Management Response: [Added: 2018/07/06] [Last Updated: 2020/12/08]

Agree

Key Actions:

Key Action Responsible DueDate Status Comments Documents
UNICEF will continue to support the finalization of the strategy
[Added: 2018/07/06] [Last Updated: 2020/09/21]
UNICEF 2018/10 Completed done History
9. Recommendation:

Capacity building in gender mainstreaming. UN defines gender mainstreaming as: ““The process of assessing the implications for women and men of any planned action, including legislation, policies or programmes, in all areas and at all levels.”. What is essential is the consideration of implication for respectively women and men. Empowerment of the one sex without including the development of the other has resulted in gender imbalances over the decades of donor assistance – first with an unintended empowerment of male dominated areas primarily related to land (better production) and livestock (better producing and less sick animals), which later turned into a focus on women, which, in some areas more than others, resulted in negative male response to the HH development contrary to donor intentions. The emphasis on the mainstreaming where the needs, capacities and interests of both parties meet in united efforts to make their HHs, businesses or organizations develop is the essence of gender mainstreaming. Gender mainstreaming should be applied at macro, meso and micro levels of interventions. Not least at micro level it requires strong involvement of both sexes in taking responsibility for the HH and other family and community related developments. The ways and means will be locally different. Only involvement will decide the final approach and design.

Management Response: [Added: 2018/07/06] [Last Updated: 2020/12/08]

Agree

Key Actions:

Key Action Responsible DueDate Status Comments Documents
UNDP will take the recommendation forward, in collaboration with TASAF.
[Added: 2018/07/06] [Last Updated: 2020/09/21]
UNDP and TASAF 2018/09 Completed done History
10. Recommendation:

Enhancing gender mainstreaming capacities by finalizing and rolling-out of the gender training package and supporting the integration of gender into all components of the PSSN through support to finalization and implementation of the gender action plan.

Management Response: [Added: 2018/07/06] [Last Updated: 2020/12/08]

Agree

Key Actions:

Key Action Responsible DueDate Status Comments Documents
UNDP will support the implementation of the TASAF gender mainstreaming action plan.
[Added: 2018/07/06] [Last Updated: 2020/09/03]
UNDP 2018/09 Completed done History

Latest Evaluations

Contact us

1 UN Plaza
DC1-20th Floor
New York, NY 10017
Tel. +1 646 781 4200
Fax. +1 646 781 4213
erc.support@undp.org