Joint Social Protection Programme Evaluation

Report Cover Image
Evaluation Plan:
2016-2021, Tanzania
Evaluation Type:
Final Project
Planned End Date:
03/2018
Completion Date:
07/2018
Status:
Completed
Management Response:
Yes
Evaluation Budget(US $):
40,000

Share

Document Type Language Size Status Downloads
Download document Terms of Reference - SP joint evaluation.pdf tor English 749.91 KB Posted 104
Download document SP JP Final Report.pdf report English 1486.54 KB Posted 84
Title Joint Social Protection Programme Evaluation
Atlas Project Number: 85002
Evaluation Plan: 2016-2021, Tanzania
Evaluation Type: Final Project
Status: Completed
Completion Date: 07/2018
Planned End Date: 03/2018
Management Response: Yes
Corporate Outcome and Output (UNDP Strategic Plan 2018-2021)
  • 1. Output 2.1.2 Capacities developed for progressive expansion of inclusive social protection systems
Evaluation Budget(US $): 40,000
Source of Funding: SDG-F, UNDP
Evaluation Expenditure(US $): 16,380
Joint Programme: Yes
Joint Evaluation: Yes
  • Joint with UN Agencies
  • Joint with UNICEF, UNFPA, ILO
Evaluation Team members:
Name Title Email Nationality
Birgitte Woel KENYA
GEF Evaluation: No
Key Stakeholders: UNICEF, UNFPA, ILO, TASAF, Ministry of Finance, National Bureau of Statistics
Countries: TANZANIA (UNITED REPUBLIC OF )
Comments:

This is a joint programme terminal evaluation

Lessons
1.

- The strong emphasis on saving groups has proved very positive, as life opportunities have enhanced significantly and they are utilised. While being successful in introducing income generating activities, it is essential to introduce the next step in time, in this case market linkages, to avoid disappointments and subsequent loss of interest in taking charge of own life.

- The involvement of Tanzania Social Action Fund (TASAF) at all levels has worked well and will ensure sustainability. However, training staff long before the materials are ready may be ineffective and inefficient as the trainers were losing motivation at the time of the evaluation and may have forgotten some of the messages that they need to pass as trainers when the materials are ready. The ILO guide to LE had been tested and was ready for use. The remaining 200 youth had been trained after programme expiry showing that the ILO developed guide on Livelihood Enhancement (LE) made it possible to continue the training without Joint Programme (JP) support.

- At macro-level the continued support of the policy work despite a change from National Social Protection Framework to National Social Protection Policy was appreciated and will result in a far better document, as a policy will force budget allocation and thereby a stronger and more stable development of national social protection efforts. The response to the Government of Tanzania needs proved very relevant and effective in this situation.

- The provision of support for development of support systems such as the National Social Protection Framework and National Social Protection Policy, Poverty Monitoring System, nutrition materials, LE guideline, training of trainers was well received and early application of the support seems to have the potential to generate sustainable changes.

- The establishment of a joint programme goes beyond making a number of UN agencies deliver to the same ministerial response. In this case four agencies each deliver what was assumed relevant for the governmental needs and not what was realistic with the available resources. The agencies collaborated effectively as parallel entities, but did not make recorded efforts on involving in joint JP activities enhancing the ability to support TASAF through well-planned internal activities that could have generated a synergy effect making the agencies as group have a common, strong platform from which to engage in evidence-based, innovative activities, which could only happen because of the very composition of agencies.

- When looking at the multitude of complex deliverables over a three-year period, a feasibility study of JP man-power/effort requirements would have forced a choice, which could have resulted in fewer activities being fully completed. The incomplete deliverables were not solely due to a change in government priorities and late roll-out.


Findings
1.

- It is important that UN defines internal capacity before venturing into design and support of next TASAF phase to ensure definition of realistic achievements.

- Considering which donors may be relevant for more efforts at macro level management across capacity development management of Social Protection (SP) measures, PMS, Communication and Advocacy, Knowledge Management and other areas supported under TASAF III.

- It may as well be relevant to involve national and/or international research institutions with expertise in SP, technical training and other relevant areas, think tanks, relevant associations etc. To know whom to include there may need to conduct a needs assessment, which could provide exact ideas of relevant partners.

- The results matrix should have a clear line, or logic, from activity to output to outcome. The indicators should clearly reflect the desired outcome, as the indicator is the measure stick determining whether the outcome has been achieved. The indicator can therefore not concern latrines, when the outcome concerns establishment of TASAF management systems as in the latest results matrix as was the case in the evaluated results matrix. It is recommended to seek assistance for development of the next results matrix, as a correct matrix will both make the writing of the programme document and later monitoring and reporting be easy. It will further help having a common understanding of which support to provide/expect across all involved parties.

a) A stronger JP M&E system with relatively few key performance indicators enabling a joint learning from experience and subsequent engagement in such learning activities. Such planned learning would be an obvious JP activity, partly for JP internal learning – and when having developed a simple approach or other products, offer this approach as a support function to JP partners and UN agencies at large. There is little evidence-based recording of the innovative aspect of joint programmes. Focus is mainly on simplifying the administration.

- The National Social Protection Policy should be finalized and approved before the design of TASAF phase IV is completed with the aim to ensure targeted and coordinated support of the enforcement of the policy.

- Communication and Advocacy Strategy to be combine with a Knowledge Management System (KMS). The KMS could provide part of the information for general communication and advocacy activities. The KMS will provide details on who (donors, different levels of government, different categories of staff, newspapers, radio, TV etc.) shall know what, in which format (folder, report, one-page fact sheet, community information materials etc.)

- Capacity building in gender mainstreaming. Gender mainstreaming should be applied at macro, meso and micro levels of interventions. Not least at micro level it requires strong involvement of both sexes in taking responsibility for the Households and other family and community related developments.


Recommendations
1

It is strongly recommended that UN defines internal capacity before venturing into design and support of next TASAF phase to ensure definition of realistic achievements.

2

Considering which donors may be relevant for more efforts at macro level management across capacity development management of SP measures, PMS, Communication and Advocacy, Knowledge Management and other areas supported under TASAF III.

3

It may as well be relevant to involve national and/or international research institutions with expertise in SP, technical training and other relevant areas, think tanks, relevant associations etc. To know whom to include there may need to conduct a needs assessment, which could provide exact ideas of relevant partners.

4

The results matrix should have a clear line, or logic, from activity to output to outcome. The indicators should clearly reflect the desired outcome, as the indicator is the measure stick determining whether the outcome has been achieved. The indicator can therefore not concern latrines, when the outcome concerns establishment of TASAF management systems as in the latest results matrix as was the case in the evaluated results matrix. It is recommended to seek assistance for development of the next results matrix, as a correct matrix will both make the writing of the programme document and later monitoring and reporting be easy. It will further help having a common understanding of which support to provide/expect across all involved parties.

5

A stronger JP M&E system with relatively few key performance indicators enabling a joint learning from experience and subsequent engagement in such learning activities. Such planned learning would be an obvious JP activity, partly for JP internal learning – and when having developed a simple approach or other products, offer this approach as a support function to JP partners and UN agencies at large. There is little evidence-based recording of the innovative aspect of joint programmes. Focus is mainly on simplifying the administration.

6

Finalization and implementation of the National Social Protection Policy. The policy should be finalized and approved before the design of TASAF phase IV is completed with the aim to ensure targeted and coordinated support of the enforcement of the policy.

7

Finalization, test and amendment of the PMS. The focus should be on simplicity in design to ensure usability across all levels of data collection and use. Development of a user-manual, which includes M&E responsibilities at each of the three levels of implementation. Finally, capacity development in poverty analytics.

8

Finalization and test of the Communication and Advocacy Strategy. It is recommended to combine it with a Knowledge Management System (KMS) as the intentions presented in the results matrix are more in line with KMS than with a Communication and Advocacy Strategy. The KMS could provide part of the information for general communication and advocacy activities. The KMS will provide details on who (donors, different levels of government, different categories of staff, newspapers, radio, TV etc.) shall know what, in which format (folder, report, one-page fact sheet, community information materials etc.).

9

Capacity building in gender mainstreaming. UN defines gender mainstreaming as: ““The process of assessing the implications for women and men of any planned action, including legislation, policies or programmes, in all areas and at all levels.”. What is essential is the consideration of implication for respectively women and men. Empowerment of the one sex without including the development of the other has resulted in gender imbalances over the decades of donor assistance – first with an unintended empowerment of male dominated areas primarily related to land (better production) and livestock (better producing and less sick animals), which later turned into a focus on women, which, in some areas more than others, resulted in negative male response to the HH development contrary to donor intentions. The emphasis on the mainstreaming where the needs, capacities and interests of both parties meet in united efforts to make their HHs, businesses or organizations develop is the essence of gender mainstreaming. Gender mainstreaming should be applied at macro, meso and micro levels of interventions. Not least at micro level it requires strong involvement of both sexes in taking responsibility for the HH and other family and community related developments. The ways and means will be locally different. Only involvement will decide the final approach and design.

10

Enhancing gender mainstreaming capacities by finalizing and rolling-out of the gender training package and supporting the integration of gender into all components of the PSSN through support to finalization and implementation of the gender action plan.

1. Recommendation:

It is strongly recommended that UN defines internal capacity before venturing into design and support of next TASAF phase to ensure definition of realistic achievements.

Management Response: [Added: 2018/07/06]

Agree

Key Actions:

Key Action Responsible DueDate Status Comments Documents
With support from relevant agencies, the social protection outcome group lead (UNICEF) will take the recommendation forward.
[Added: 2018/07/06]
UNICEF 2018/10 Overdue-Initiated Taken for future project design
2. Recommendation:

Considering which donors may be relevant for more efforts at macro level management across capacity development management of SP measures, PMS, Communication and Advocacy, Knowledge Management and other areas supported under TASAF III.

Management Response: [Added: 2018/07/06]

Agree

Key Actions:

Key Action Responsible DueDate Status Comments Documents
The social protection outcome group (through the outcome lead) will take the recommendation forward when developing any new social protection joint programme.
[Added: 2018/07/06]
UNICEF, UNDP, ILO, UNFPA 2018/10 Overdue-Initiated Taken for future project design
3. Recommendation:

It may as well be relevant to involve national and/or international research institutions with expertise in SP, technical training and other relevant areas, think tanks, relevant associations etc. To know whom to include there may need to conduct a needs assessment, which could provide exact ideas of relevant partners.

Management Response: [Added: 2018/07/06]

Agree

Key Actions:

Key Action Responsible DueDate Status Comments Documents
The social protection outcome group (through the outcome lead) will take the recommendation forward when developing any new social protection joint programme
[Added: 2018/07/06]
UNICEF, UNDP, ILO, UNFPA 2018/10 Overdue-Initiated Taken for future project design
4. Recommendation:

The results matrix should have a clear line, or logic, from activity to output to outcome. The indicators should clearly reflect the desired outcome, as the indicator is the measure stick determining whether the outcome has been achieved. The indicator can therefore not concern latrines, when the outcome concerns establishment of TASAF management systems as in the latest results matrix as was the case in the evaluated results matrix. It is recommended to seek assistance for development of the next results matrix, as a correct matrix will both make the writing of the programme document and later monitoring and reporting be easy. It will further help having a common understanding of which support to provide/expect across all involved parties.

Management Response: [Added: 2018/07/06]

Agree

Key Actions:

Key Action Responsible DueDate Status Comments Documents
The social protection outcome group will take the recommendation forward when developing any new social protection joint programme.
[Added: 2018/07/06]
UNICEF, UNDP, ILO, UNFPA 2018/10 Overdue-Initiated Taken for future project design
5. Recommendation:

A stronger JP M&E system with relatively few key performance indicators enabling a joint learning from experience and subsequent engagement in such learning activities. Such planned learning would be an obvious JP activity, partly for JP internal learning – and when having developed a simple approach or other products, offer this approach as a support function to JP partners and UN agencies at large. There is little evidence-based recording of the innovative aspect of joint programmes. Focus is mainly on simplifying the administration.

Management Response: [Added: 2018/07/06]

Agree

Key Actions:

Key Action Responsible DueDate Status Comments Documents
The social protection outcome group will take the recommendation forward when developing any new social protection joint programme
[Added: 2018/07/06]
UNICEF, UNDP, ILO, UNFPA 2018/10 Overdue-Initiated Taken for future project design
6. Recommendation:

Finalization and implementation of the National Social Protection Policy. The policy should be finalized and approved before the design of TASAF phase IV is completed with the aim to ensure targeted and coordinated support of the enforcement of the policy.

Management Response: [Added: 2018/07/06]

Agree

Key Actions:

Key Action Responsible DueDate Status Comments Documents
The agencies - UNICEF, ILO and UNDP - involved will continue support the process
[Added: 2018/07/06]
UNICEF, ILO, UNDP 2018/12 Overdue-Initiated Technical Support
7. Recommendation:

Finalization, test and amendment of the PMS. The focus should be on simplicity in design to ensure usability across all levels of data collection and use. Development of a user-manual, which includes M&E responsibilities at each of the three levels of implementation. Finally, capacity development in poverty analytics.

Management Response: [Added: 2018/07/06]

Agree

Key Actions:

Key Action Responsible DueDate Status Comments Documents
The recommendation will be proposed to be followed up by the UNDP Poverty Environment Initiative II project team
[Added: 2018/07/06]
UNDP 2018/11 Overdue-Initiated New Poverty Environment Initiative II project
8. Recommendation:

Finalization and test of the Communication and Advocacy Strategy. It is recommended to combine it with a Knowledge Management System (KMS) as the intentions presented in the results matrix are more in line with KMS than with a Communication and Advocacy Strategy. The KMS could provide part of the information for general communication and advocacy activities. The KMS will provide details on who (donors, different levels of government, different categories of staff, newspapers, radio, TV etc.) shall know what, in which format (folder, report, one-page fact sheet, community information materials etc.).

Management Response: [Added: 2018/07/06]

Agree

Key Actions:

Key Action Responsible DueDate Status Comments Documents
UNICEF will continue to support the finalization of the strategy
[Added: 2018/07/06]
UNICEF 2018/10 Overdue-Initiated Technical and financial support
9. Recommendation:

Capacity building in gender mainstreaming. UN defines gender mainstreaming as: ““The process of assessing the implications for women and men of any planned action, including legislation, policies or programmes, in all areas and at all levels.”. What is essential is the consideration of implication for respectively women and men. Empowerment of the one sex without including the development of the other has resulted in gender imbalances over the decades of donor assistance – first with an unintended empowerment of male dominated areas primarily related to land (better production) and livestock (better producing and less sick animals), which later turned into a focus on women, which, in some areas more than others, resulted in negative male response to the HH development contrary to donor intentions. The emphasis on the mainstreaming where the needs, capacities and interests of both parties meet in united efforts to make their HHs, businesses or organizations develop is the essence of gender mainstreaming. Gender mainstreaming should be applied at macro, meso and micro levels of interventions. Not least at micro level it requires strong involvement of both sexes in taking responsibility for the HH and other family and community related developments. The ways and means will be locally different. Only involvement will decide the final approach and design.

Management Response: [Added: 2018/07/06]

Agree

Key Actions:

Key Action Responsible DueDate Status Comments Documents
UNDP will take the recommendation forward, in collaboration with TASAF.
[Added: 2018/07/06]
UNDP and TASAF 2018/09 Overdue-Initiated Gender Action Plan to be finalized
10. Recommendation:

Enhancing gender mainstreaming capacities by finalizing and rolling-out of the gender training package and supporting the integration of gender into all components of the PSSN through support to finalization and implementation of the gender action plan.

Management Response: [Added: 2018/07/06]

Agree

Key Actions:

Key Action Responsible DueDate Status Comments Documents
UNDP will support the implementation of the TASAF gender mainstreaming action plan.
[Added: 2018/07/06]
UNDP 2018/09 Overdue-Not Initiated Activity to follow after the finalization of Gender Action Plan

Latest Evaluations

Contact us

220 East 42nd Street
20th Floor
New York, NY 10017
Tel. +1 646 781 4200
Fax. +1 646 781 4213
erc.support@undp.org