Evaluation of the Youth at risk project and some security related components

Report Cover Image
Evaluation Plan:
2011-2017, Somalia
Evaluation Type:
Final Project
Planned End Date:
12/2012
Completion Date:
12/2012
Status:
Completed
Management Response:
Yes
Evaluation Budget(US $):
25,000

Share

Document Type Language Size Status Downloads
Download document UNDP Somalia_ToR_Y4C Process Evaluation_v3.0_08.2012.pdf tor English 251.38 KB Posted 1143
Download document DFID funded - Youth at-Risk Final Evaluation Report 092012.pdf report English 2731.98 KB Posted 2085
Title Evaluation of the Youth at risk project and some security related components
Atlas Project Number:
Evaluation Plan: 2011-2017, Somalia
Evaluation Type: Final Project
Status: Completed
Completion Date: 12/2012
Planned End Date: 12/2012
Management Response: Yes
Focus Area:
  • 1. Crisis Prevention & Recovery
Corporate Outcome and Output (UNDP Strategic Plan 2018-2021)
  • 1. National and local institutions have the capacities to prevent, reduce and mitigate the impact of conflict
  • 2. Livelihoods and economic recovery generated, including infrastructure restoration, employment and sustainable income earning opportunities for crisis affected communities
Evaluation Budget(US $): 25,000
Source of Funding: DFID
Joint Programme: No
Joint Evaluation: Yes
  • Joint with ILO, UNICEF
Evaluation Team members:
Name Title Email Nationality
David Guillemois
Muktar Sheikh Mohamed
Mohamed Ibrahim Mohamed
GEF Evaluation: No
Key Stakeholders: Donors, UN agencies, civil society, public sector actors, private sector actors, beneficiaries
Countries: SOMALIA
Comments: Process evaluation
Lessons
Findings
Recommendations
1 The socio-economic rehabilitation and reintegration requires a more specific target caseload with comprehensive profiling complemented by reflexive and contextualised programming :
2 Inclusive and representative approach. The current caseload could be more representative of the communities (e.g. targets children from marginalized communities and children with disabilities, etc.). The programme must ensure that it is accessible to and tailored for a caseload representative of the region's diversity to account for age, gender, clan and socio-economic status.
3 The identification process needs to include all agencies and partners in the identification and selection process of the beneficiaries.
4 Communications and messaging throughout the programme must be consistent.
5 More strategic Programming is needed Programme components must be strategically organised and better tailors to meet the speicfic needs of the youth.
6 Create follow up mechanism for the youth who have passed through the project. The monitoring system should enable sufficient profiling information about each youth trained at the center to get regular feedback on their evolution.
7 Clarify the concept of the Women?s Civilian Protection Unit (WCPU). The approach of this specific unit has not been clearly defined and confusion was expressed on the role and the approach to the WCPU. Data has not been recorded accurately enough to estimate impacts of this unit at police intervention level or to help inform a view of the security issues.
8 More support for the reintegration processes is required. Reintegration of the youth within the community should be encouraged by providing incentive and motivation in order to support the community to facilitate reintegration activities and foster an enabling environment. The youth should continue community services to maintain the connection to serving the community. Reintegration should be supported through the district safety committees where the district safety plans exist. Children above the legal working age should also have access to economic reintegration support. This should be built into the project design and clearly communicated. It will require information sharing and training of implementing partners and other stakeholders with regard to the specificities of dealing with children (e.g. need to protect them from hazardous work as described below).
9 There is a need for better coordination and engagement with the Government. More engagement with the government counterparts would benefit with increased support, understanding and buy-in of the officials into the programme activities and programmes on the ground. There is a need to redefine the coordination aspects between the local authorities and the community based project approach
10 Services should be expanded through the expertise of other UN agencies. All Y4C agencies should explore other partnerships outside UNDP, UNICEF and ILO, such as FAO who have larger implementation capacity.
11 There is potential for Youth for Change linkages to the Peacebuilding Framework and the positioning of the Community Security Project Success of the Y@R project requires parallel progress on peace-building and therefore implementation should support good governance. These processes need to be better linked
12 Monitoring and Evaluation improvements are needed. More specifically: ? A review of the monitoring plan of the project and the related indicators and means of verification and monitoring tools should provide timely access to information. ? There is a need for more concrete, specific and contextualised objectives and detailed logical framework. ? The case management system should be designed to cover all components equally. The [Y@R] programme monitoring and evaluation measures should be aligned to include more joint field missions from the agency representatives. ? Due to the nature of the programme the components overlap with the justice and security sectors. The programme will benefit and have a more measureable impact with stronger linkages to the justice sector and to become better integrated into the police sector. ? The project will also benefit from a community feedback loop mechanism enabling a better appreciation of the project deviance and local perception.
13 Impact of Curricula/ Contextualised curricula: Although there have been extensive resources invested into the revisions of the Y4C curricula, there remains sensitive imagery that undermines the social rehabilitation module delivery.
1. Recommendation: The socio-economic rehabilitation and reintegration requires a more specific target caseload with comprehensive profiling complemented by reflexive and contextualised programming :
Management Response: [Added: 2013/12/16]

In the pilot phase, the join agencies chose to define the target group as a more broad cohort for those at risk or already associated with lifestyles of crime an violence. This was necessary to enable the agencies and stakeholders to learn more about the specific needs through the pilot phase. It was intended that from this, future programming would include more clear distinctions within that group and also allow the agencies to help support the needed structures. Since, the agencies have worked with relevant counterparts to formulate the necessary legal frameworks for engagement with the youth categories.

Key Actions:

Key Action Responsible DueDate Status Comments Documents
Clearly distinguish target groups programme is meant to engage: Youth for Change 2013 (1) Children and youth at risk of being recruited into armed groups; (2) Children and youth associated with conflict and armed groups; (3) Children and youth in conflict with the law; (4) Children and youth sentenced to prison/ in detention. Clearly define target districts for engagement: Hargeisa, Boroma, Burao, Las Anod in Somaliland; Bosasso, Garowe, Eyl, Gardo in Puntland; and 4 centres in Mogadishu to cover all 16 districts to include: Wadajir, Hamar Jajab (previously established), Karan and Hodan (newly established); and last one centre in Baidoa (100%). Develop Legal frameworks for engagement with each target group to include selection though the Parole/Probation scheme and Alternative Dispute Resolution in Somaliland and Puntland.
[Added: 2013/12/16] [Last Updated: 2016/11/22]
UNDP and UNICEF 2014/01 Completed January 2012 100% integrated into Phase 2 programming; 25% implemented min Youth for change January 2013 100% in integrated into larger scale Youth initiative programme document. History
2. Recommendation: Inclusive and representative approach. The current caseload could be more representative of the communities (e.g. targets children from marginalized communities and children with disabilities, etc.). The programme must ensure that it is accessible to and tailored for a caseload representative of the region's diversity to account for age, gender, clan and socio-economic status.
Management Response: [Added: 2013/12/16]

The youth at risk offered an innovative first in programmatic interventions developed in order to engage with the actors of crime and violence, therefore there is still much to learn. More specifically, due to the target group already requiring many special needs are met (traumatization, lack of education, etc), which the programming is becoming more and more responsive to, the more we understand our beneficiaries. Future programming will require that International expertise (e.g. Handi-cap international) are vetted in order to help integrate mainstreaming of these groups into the programming

Key Actions:

Key Action Responsible DueDate Status Comments Documents
Based on the above defined categories, formulate a clear strategy to engage with a wider and more representative caseload (to include a gender balance, marginalized youth and youth with disabilities).
[Added: 2013/12/16] [Last Updated: 2016/11/22]
UNDP, ILO and UNICEF 2014/01 Completed January 2012: Y4C Gender mainstreaming 100% integrated into programming through female inclusion and gender sensitization training January 2013: YI 100% inclusion for all groups integrated into larger scale Youth initiative programme document. History
3. Recommendation: The identification process needs to include all agencies and partners in the identification and selection process of the beneficiaries.
Management Response: [Added: 2013/12/16]

During the pilot phase, UNDP took the lead on selection of the above 18 years beneficiaries and UNICEF took the lead on the below 18 under 18 year old beneficiaries due to their respective expertise UNDP dealing with the actors of crime and violence and UNICEF child protection. Management must devise a strategy to ensure that the identification and registration process is jointly implemented.

Key Actions:

Key Action Responsible DueDate Status Comments Documents
Create sensitization materials on the identification processes and nature of the caseloads for all agencies. Devise clear and inclusive identification and registration strategy.
[Added: 2013/12/16] [Last Updated: 2016/11/22]
UNDP, ILO and UNICEF 2014/01 Completed January 2012: Y4C 25% integrated and implemented into programming January 2013: YI integrated into larger scale Youth initiative programme document. History
4. Recommendation: Communications and messaging throughout the programme must be consistent.
Management Response: [Added: 2013/12/16]

In the first phase, consistent messaging was attempted though outreach and sensitization campaigns, however lessons learnt showed that in reality information was not always communicated consistently ,leaving room for misinformation/ rumors to flourish. The agencies originally agreed that the UNDP would take the lead in organizing outreach and sensitization for the programme as a whole, being best placed with on-going activities and resources to reach out to the community level (i.e. the peace caravan). and Agree to communication strategy for intra agency and stakeholder communications.

Key Actions:

Key Action Responsible DueDate Status Comments Documents
(1) Create a joint communication strategy; to include (a) inter-agency and agency-implementing partner communications and coordination; (b) guidelines for communicating with the communities and beneficiaries through messaging, outreach and media; (c) visibility plans for UN agencies and for donor organizations with expected branding; and (d) the implementation plan and timeframes for the various communication activities. (2) Train all stakeholders on the strategy (3) Comprehensive Orientation Packages for Implementing Partners will be developed.
[Added: 2013/12/16] [Last Updated: 2016/11/22]
UNDP, ILO and UNICEF 2014/01 Completed January 2012: Y4C 100% communication strategy created and partners trained. January 2013: YI Integrated into larger scale Youth initiative programme document. History
5. Recommendation: More strategic Programming is needed Programme components must be strategically organised and better tailors to meet the speicfic needs of the youth.
Management Response: [Added: 2013/12/16]

The joint agencies approach in the pilot phase included an all for one approach with immersion of the beneficiaries into all of the programme components. This was understood to be a good way to diversify the week?s activities for youth who were not used to a classroom setting. However, the lessons learnt from this show that there are some components/ elements that are foundational to the others (e.g. literacy and numeracy training and social skills) which will make the youth more likely to succeed in others (e.g. business skills). Based on lessons learned from the management will has devised a clear strategy to establish foundational learning which will optimize and each component of the programme.

Key Actions:

Key Action Responsible DueDate Status Comments Documents
Components organised to include a first segment that will focus on the social rehabilitation and alongside literacy and numeracy training (4 months); the second segment will include economic reintegration packages.
[Added: 2013/12/16] [Last Updated: 2016/11/22]
UNDP, ILO and UNICEF 2014/01 Completed January 2012: Y4C Staged and staggered approach taken to programming (e.g. Lit/num and Social rehabilitation before economic reintegration). January 2013: YI More tailored approach integrated into larger scale Youth initiative programme document. History
6. Recommendation: Create follow up mechanism for the youth who have passed through the project. The monitoring system should enable sufficient profiling information about each youth trained at the center to get regular feedback on their evolution.
Management Response: [Added: 2013/12/16]

During the pilot phase, the joint agencies attempted to build a rigorous online case management system (the Youth-sms system) to profile and track beneficiary progress. This was however too ambitious for the capacities on the ground to maintain, and the system was abandoned and forced to rely on more archaic devices (printed registration sheets). The management has committed to developing a more simplified system that is manageable at the ground level. Ensure that the programme has the ability to track and trace youth through a rigorous case management system.

Key Actions:

Key Action Responsible DueDate Status Comments Documents
(1) Organise for Tracer study of youth; (2) Develop manageable simplified tools for profiling and tracking progress of individual beneficiaries. (3) Improved mentorship follow-up post programme.
[Added: 2013/12/16] [Last Updated: 2016/11/22]
UNDP, ILO and UNICEF 2014/01 Completed January 2012: Y4C 100% Tracer study conducted (avail January 2013); Community based mentors were integrated into programming. January 2013: YI Case management More tailored approach integrated into larger scale Youth initiative programme document. History
7. Recommendation: Clarify the concept of the Women?s Civilian Protection Unit (WCPU). The approach of this specific unit has not been clearly defined and confusion was expressed on the role and the approach to the WCPU. Data has not been recorded accurately enough to estimate impacts of this unit at police intervention level or to help inform a view of the security issues.
Management Response: [Added: 2013/12/16]

The concept behind the women?s civilian Protection unit was to empower women as agents for peace while reinforcing the youth programming through an added layer of community monitoring, etc. in Mogadishu. While the programme provided valuable services (monitoring, incident reporting, SGBV referrals, etc), their link to supporting the Y@R was not clear because the issues in Mogadishu are more complex. Management has committed to reflect on and revise concept of the civilian protection units and their and role and utility, especially in regards to the programme.

Key Actions:

Key Action Responsible DueDate Status Comments Documents
(1) Review the Concept of ?Civilian protection mechanisms; (2) Revise and customize for next programme cycle.
[Added: 2013/12/16] [Last Updated: 2016/11/22]
UNDP 2014/01 No Longer Applicable January 2012: Y4C The civilian protection mechanism was not integrated into the second phase of programming and so on. History
8. Recommendation: More support for the reintegration processes is required. Reintegration of the youth within the community should be encouraged by providing incentive and motivation in order to support the community to facilitate reintegration activities and foster an enabling environment. The youth should continue community services to maintain the connection to serving the community. Reintegration should be supported through the district safety committees where the district safety plans exist. Children above the legal working age should also have access to economic reintegration support. This should be built into the project design and clearly communicated. It will require information sharing and training of implementing partners and other stakeholders with regard to the specificities of dealing with children (e.g. need to protect them from hazardous work as described below).
Management Response: [Added: 2013/12/16]

A reintegration mechanism was included in the pilot phase through (1) utilizing the community as part of the identification and selection processes; (2) assigning community mentors to beneficiaries to provide personalized support; (3) including community volunteer activities (e.g. cleaning public spaces); and last; (4) the Cash for work programme was based on community infrastructure projects. These did seem to help repair the perception and relationship between the communities and the youth, however more work is needed in the details of what the ?reintegration? elements should consist of. Management has jointly reviewed the reintegration process and strengthen the role of the community in the rehabilitation and subsequent reintegration process. Children above the legal working will be provided with more comprehensive economic reintegration packages to include training and potential job placements.

Key Actions:

Key Action Responsible DueDate Status Comments Documents
(1) Liaise with the community members for possible entry points/ areas for inclusive involvement. (2) Strengthen role of the community within each component; (3) Develop social impact indicators for an in depth understanding of social reintegration; (4) UNICEF to address Economic reintegration support to below 18 years beneficiaries.
[Added: 2013/12/16] [Last Updated: 2016/11/22]
UNDP, ILO and UNICEF 2014/01 Completed January 2012: Y4C The role of the community was strengthened for implementation of Y4C January 2013: YI 100% EBD geared towards development of social impact indicators into programming. History
9. Recommendation: There is a need for better coordination and engagement with the Government. More engagement with the government counterparts would benefit with increased support, understanding and buy-in of the officials into the programme activities and programmes on the ground. There is a need to redefine the coordination aspects between the local authorities and the community based project approach
Management Response: [Added: 2013/12/16]

Although the ministries were fully informed and involved in the programme inception and design phase, the programme being somehow more complicated, representing work of various sectors requires a more cohesive approach to involving the government counterparts. The joint agencies as since strengthened the coordination mechanism at the national level getting full understanding and buy-in from all of the relevant ministers.

Key Actions:

Key Action Responsible DueDate Status Comments Documents
(1) Youth programming prioritised in national level policy documents. (2) Improve the coordination modalities on the boarder level. (3) Strengthen distance management by Re-framing coordination bodies to better fit the existing structures and develop efficient lines for feeding information.
[Added: 2013/12/16] [Last Updated: 2016/11/22]
UNDP, ILO and UNICEF No due date Completed January 2012: Y4C ? The Inter-ministerial Steering Committee (consists of Ministry of Planning, Ministry of Justice, Ministry of Interior, Ministry of Youth and Social Affairs) was established in Hargeisa and Garowe to support troubleshooting of the Y4C programme and ensure national ownership of all processes (100% operational). ? Monthly coordination meetings are held in the targeted areas and lead by the Governor/Mayor and includes the DSC, partner NGO, UNDP, Observatory staff (100% operational). ? Monthly coordination meetings are held in Nairobi among the heads of programme (UNDP, ILO, UNICEF) (100% operational). January 2013: YI Above structures are being revised and adapted for integration into 2014 programming. History
10. Recommendation: Services should be expanded through the expertise of other UN agencies. All Y4C agencies should explore other partnerships outside UNDP, UNICEF and ILO, such as FAO who have larger implementation capacity.
Management Response: [Added: 2013/12/16]

The implementation modality of the pilot phase as a joint initiative in itself has many challenges. Before more UN agencies can be included, the programme needs to raise the (1) understanding of the specific needs of the target groups; (2) build onto the knowledge of our partners on the ground and work to strengthen the existing systems.

Key Actions:

Key Action Responsible DueDate Status Comments Documents
(1) Identify gaps programming that may need more external support; (2) Take inventory of opportunities outside of implementing agencies; (3) Integrate other agencies comparative advantages into programming.
[Added: 2013/12/16] [Last Updated: 2016/11/22]
UNDP, ILO and UNICEF 2014/01 Completed January 2012: Y4C No other UN agencies were partnered to implement jointly, however others were called on for in specific supported arenas (e.g. BCPR in Gender mainstreaming in reintegration; UNIDIR for Evidence based programming) January 2013: The comparative advantage of who should be involved was completed and the next proposal for Japanese funds was developed accordingly. History
11. Recommendation: There is potential for Youth for Change linkages to the Peacebuilding Framework and the positioning of the Community Security Project Success of the Y@R project requires parallel progress on peace-building and therefore implementation should support good governance. These processes need to be better linked
Management Response: [Added: 2013/12/16]

The Y@R and Y4C were supported by the Ministry of Interior (MOI) and Community Security and Peacebuilding Unit and the MOJ in Somaliland; the Ministry of Security (MOS) and DDR and Peacebuilding Unit in Puntland; and the MOI in South Central Somalia by providing incentive and technical support in order to perform monitoring and coordination of the Youth for Change project. Youth were prioritized in Policy development over 2012, reinforcing structures through the Peacebuilding and Community Security (PB&CS) Infrastructure.

Key Actions:

Key Action Responsible DueDate Status Comments Documents
This framework was recently formalised under the Peacebuilding policies to include bodies at the Community, District, Regional and State levels (policies finalised and endorsed in both Puntland and Somaliland 100%).
[Added: 2013/12/16] [Last Updated: 2016/11/22]
UNDP 2014/01 Completed January 2012: Y4C The Y4C is aligned directly to the peacebuilding and youth policies of PL and SL. January 2013: YI These policies will be used to guide programming for 2014 and beyond. History
12. Recommendation: Monitoring and Evaluation improvements are needed. More specifically: ? A review of the monitoring plan of the project and the related indicators and means of verification and monitoring tools should provide timely access to information. ? There is a need for more concrete, specific and contextualised objectives and detailed logical framework. ? The case management system should be designed to cover all components equally. The [Y@R] programme monitoring and evaluation measures should be aligned to include more joint field missions from the agency representatives. ? Due to the nature of the programme the components overlap with the justice and security sectors. The programme will benefit and have a more measureable impact with stronger linkages to the justice sector and to become better integrated into the police sector. ? The project will also benefit from a community feedback loop mechanism enabling a better appreciation of the project deviance and local perception.
Management Response: [Added: 2013/12/16]

Management to reflect on issues of M&E and ways to strengthen the system in total.

Key Actions:

Key Action Responsible DueDate Status Comments Documents
(1) Y4C Resources and Results Framework (RRF) developed on behalf of all three agencies and submitted to the GOJ upon delivery of Phase 2 proposal; (2) Joint Agency missions and carried out. (3) The mind-set assessment to be carried out pre and post programme, (4) Carry out Tracer study of pilot project beneficiaries. (5) Detailed profiling by case managers. (6) New simplified tools for case management (7) The Case Managers will ensure that the profile is updated on a regular basis and contains exam updates.
[Added: 2013/12/16] [Last Updated: 2016/11/22]
UNDP, ILO and UNICEF 2014/01 Completed January 2012: Y4C Joint RRF developed and Secretariat services to coordinate these activities on behalf of all three agencies. January 2013: YI More rigorous M&E systems are being integrated into programming for 2014. History
13. Recommendation: Impact of Curricula/ Contextualised curricula: Although there have been extensive resources invested into the revisions of the Y4C curricula, there remains sensitive imagery that undermines the social rehabilitation module delivery.
Management Response: [Added: 2013/12/16]

All curricula will be revised for future programming

Key Actions:

Key Action Responsible DueDate Status Comments Documents
(1) It is advised that the curricula is once again revised in a consultative process using the youth as test subjects before the new version is finalised. (2) Curricula to be revised and rebranded as UNDP publications (rather than OCVP); (3) Linked to the Ministry of education to provide national qualifying courses for youth to obtain certificates in.
[Added: 2013/12/16] [Last Updated: 2016/11/22]
UNDP 2014/01 Completed January 2012: Y4C All Social rehabilitation modules revised under Version 2 for Y4C (published by OCVP). January 2013: YI New modules are being developed to expand of the social rehabilitation (e.g. youth empowerment) 10 % of revisions completed; 100% integrated into larger scale Youth initiative programme document and 2014 project planning History

Latest Evaluations

Contact us

1 UN Plaza
DC1-20th Floor
New York, NY 10017
Tel. +1 646 781 4200
Fax. +1 646 781 4213
erc.support@undp.org