Evaluation of The Joint Gender Programme

Report Cover Image
Evaluation Plan:
2015-2016, Botswana
Evaluation Type:
Project
Planned End Date:
12/2016
Completion Date:
12/2016
Status:
Completed
Management Response:
Yes
Evaluation Budget(US $):
30,000

Share

Document Type Language Size Status Downloads
Download document UN JGP EVALUATION - TOR final.pdf tor English 566.36 KB Posted 182
Download document FINAL JGP Evaluation Report - Feb 2017.pdf report English 1673.02 KB Posted 233
Title Evaluation of The Joint Gender Programme
Atlas Project Number:
Evaluation Plan: 2015-2016, Botswana
Evaluation Type: Project
Status: Completed
Completion Date: 12/2016
Planned End Date: 12/2016
Management Response: Yes
Focus Area:
  • 1. Others
Corporate Outcome and Output (UNDP Strategic Plan 2014-2017)
  • 1. Output 4.2. Measures in place and implemented across sectors to prevent and respond to Sexual and Gender Based Violence (SGBV)
  • 2. Output 4.3. Evidence-informed national strategies and partnerships to advance gender equality and women's empowerment
Evaluation Budget(US $): 30,000
Source of Funding: TRAC and Cost Shariing
Evaluation Expenditure(US $): 12,000
Joint Programme: No
Joint Evaluation: Yes
  • Joint with UN Agencies
  • Joint with UNFPA, UNICEF, UN Women
Evaluation Team members:
Name Title Email Nationality
Sennye Obuseng Evaluator sennye.obuseng@prexecutive.biz
GEF Evaluation: No
Key Stakeholders: Gender Affairs Department
Countries: BOTSWANA
Lessons
1.

a. Gender remains a priority development challenge for Botswana: The message from interviewees and documentary evidence is emphatic. Gender remains a priority development challenge for Botswana. Progress has been made in some areas, e.g. regulatory reforms and the representation of women in management positions, but gender inequality is still pervasive. It persists because it is a product of a culture of patriarchy and policies and laws with a history of discrimination against women. Despite progressive regulatory reforms, it will take time and effort to change the institutionalised culture of discrimination against women. The statistics on gender based violence also suggest it is a serious problem. So, an abiding lesson from the JGP is that the national need for action on gender is as strong as ever. 


2.

b) Effective programme design has a decisive influence on programme performance: This is also an important lesson from the GEJGP. Deficiencies in programme design have consequences for programme implementation and performance. In the specific case of the JGP, there are specific design issues that could have been handled better with potentially significant improvements in programme performance. They include:

Robust assessments: A good situational analysis and a good capacity assessment. The 2014 Guidance Note on Joint Programmes is especially explicit on the issue of capacity assessments:

In planning for a Joint Programme, the capacity and comparative advantages of the government, implementing partners and participating UN organizations to coordinate, manage and provide inputs (e.g., cash, supplies, in-kind or technical expertise) to support implementation and monitoring of the Joint Programme should be carefully considered. Where applicable, differences in methodology and approach – e.g. prioritization of areas and target population groups, methodology for community mobilization, modality of delivery of technical assistance – should be identified and resolved at the planning stage. For non-resident agencies, the Resident Coordinator should ensure their engagement in the process as needed in accord with their interests (p 9).

The scale of programme ambition should match the human, financial and time resources: The scale of the JGP’s ambition was way out of  sync with the limited resources – people, skills, money and time - available to the programme. This literally set the programme for under-achievement.

 

More thoughtful and purposeful selection of participating UN agencies and IPs: The JGP had more UN agencies and CSOs than it needed. The selection of PUNOs should be strategic. The Guidance Note on Joint Programmes suggests as follows:

The value-added contributions or comparative advantage of each agency should be considered by the UN Country Team, Resident Coordinator, national partner/s and donor/s. Participating UN Organizations (PUNOs) should be chosen only if they are essential for the successful implementation of the project and for producing the joint results and have the capacity for timely delivery of outputs of the Joint Programme. This may include adequate capacity of PUNOs to undertake results-based planning, budgeting, monitoring and evaluation (p 9).

This principle applies to the selection of IPs.

 

  • Clear definition of results is critical for programme performance: Closer attention should be given to the definition of results in accordance with RBM taxonomy. SMART results facilitate implementation performance and monitoring and evaluation.
  • Greater clarity regarding the capacities of partners and the division of roles is critical: A good capacity assessment should inform the ascription of roles on programmes and the mitigation of the capacity constraints of key role players. Thus, GeAD ought to be a major area of focus for programmes on gender, especially those sponsored by development agencies because aid effectiveness principles oblige them to strengthen the capacity of national institutions and work through them. It should also have been clear to all that the UN would not play the role of a donor on the JGP. It is also essential for a government IP such as GeAD to fully appreciate its role and obligations on programme. The UN and other stakeholders are also obliged to recognise and support this role by a government department.
  • Clearer focus on key areas of performance is critical for programme performance: A focus on results also requires a clear focus on key dimensions of programme performance at the design phase. Coherence, coordination, effectiveness, sustainability and mutual accountability do not just happen and cannot be assumed. They require deliberate planning. 

3.

c) Clear focus on results and reporting is essential: One of the critical weaknesses of the JGP was inadequate monitoring during its two years of implementation. This deprived the programme of essential feedback and learning and weakened accountability for results. Thus, a key lesson from the JGP is that monitoring should be systematically integrated into every stage of the programme as an exercise in verification, feedback, informing decision making and learning. This requires the engagement of M&E personnel from the design phase, through implementation to evaluation. Furthermore, it is essential that results are effectively communicated and reported as exercises in accountability, transparency, legitimisation and supporting the mobilisation of resources and partnerships. 


4.

d) Systematic approach to accountability enhances programme harmonisation and performance: As the GEJGP observes, “… accountability … should be integral to a joint gender programme …” (p 35). The UN has very robust mechanisms for accounting for resources. A similar level of focus on M&E and mutual accountability would strengthen programme quality and performance. Amongst the measures the GEJGP suggests are8:

 

  • The clarification and formalisation of the role of the Resident Coordinator: The RC is one
  • of the most critical partners for programme harmonisation and mutual accountability. Ideally, the RC should be more visible and engaged, with a clear focus on calling PUNOs to account.
  • Joint monitoring and reporting, with, a central focal point collating and presenting individual results: To a significant extent, the JGP met this imperative, with a dedicated gender specialist serving as the joint programme coordinator. Even so, this one resource was not sufficient to cover the entire breadth of the coordination demands, especially monitoring.
  • Feedback loops to inform programme decision making: This includes monitoring, reporting and active engagement, lateral and vertical, by senior people, notably the RC and the UNCT and their counterparts in government.
  • Investing in building “…demand for accountability by partner governments and CSOs”: This would include strengthening programme governance and strengthening duty bearers such as women’s organisations.
  • Shift in reporting towards results: For the most part, IP reporting on the JGP was relatively strong with regard to financial resources and activities. Accountability would be strengthened by a shift towards reporting on results – outputs and progress towards outcomes.

 

The GEJGP makes the important point that part of what weakens mutual accountability is “… lack of organizational incentives for coherence/harmonization, and clear disincentives”. An obvious obstruction to mutual accountability is the internal organisation of the UN, with all agencies reporting directly to their headquarters. Mutual accountability thus requires the RC to rely on soft assets to secure the commitment and accountability of PUNOs.


5.

e) Stronger RC and UNCT engagement raises programme efficiency and effectiveness: Consistent with the GEJGP, the feedback from stakeholders suggests that one of the key lessons to be learned from the JGP is that the active and visible engagement of the top leadership of the UN is critical to programme performance.


6.

f) A robust and systematic approach to advocacy and communication: A systematic approach to communication and advocacy is essential for programme performance and accountability. The JGP could have used its products and tools more effectively to drive advocacy and to inform government accounting for work on gender. A programme such as the JGP requires a communication and advocacy strategy to communicate results, expose innovative solutions and tools to a wider audience, facilitate resource mobilisation and partnership development, and generally influence behavioural change at the community and institutional levels. 


7.

g) CSOs are strategic delivery mechanism for services on gender: The CSOs partners on the JGP have proved to be both innovative and effective in delivering gender related services. They not only reach places and constituencies the government is not always able to reach but they often also bring innovative approaches and products to bear on their engagement with communities and rights holders. Even so, and relative to the magnitude of need, CSOs has a limited footprint. This presents both an opportunity and a challenge. The challenge is that CSOs have limited capacity, compounded by the relative lack of donor interest in Botswana on account of its Middle Income Status. The opportunity lies in the fact that CSOs provides a potentially potent and efficient mechanism through which the government and development partners can expand the reach of gender related services. There is need therefore, for the government and development partners, especially the UN, to reflect on where and how CSOs could be used to more effectively deliver gender related programmes and services. 


Findings
1.

a) JGP Results

The JGP achieved some consequential results over the two years it was implemented. Especially noteworthy are:

i) The integration of gender into Vision 2036 and NDP 11: This is a substantial achievement in the context of the JGP output “Government policies, planning and programming are gender mainstreamed”. Due to their positions at the top of development planning, Vision 2036 and NDP 11 are strong entry points for integrating gender into development processes.

ii) Development of the National Policy on Gender and Development (NPGAD): The JGP provided technical support to the development of the NPGAD and its related documents.

iii) Capacity Building: The programme provided capacity development services to several strategic institutions, including the National Gender Commission (orientation), the Gender

Affairs Department (GeAD) and several government ministries that received training on gender mainstreaming.

iv) Research and advocacy: The programme supported research on GBV and integrated HIV services. Notable results were also achieved on advocacy, especially through CSO partners. For instance, Men and Boys for Gender Equality used the media effectively to drive advocacy on male involvement in eradicating GBV and improving access to SRH services for men and boys. Gender Links’ “I-stories”, ‘which document the experiences of victims of GBV, are tangible and powerful advocacy products.

The JGP has also shown that UN agencies can work together in the context of a JGP and achieve results. The One UN voice objective was achieved. The agencies shared resources, especially knowledge and technical expertise on gender that were largely limited to no more than two agencies. They put together an effective coordination function, which though under capacitated, managed the programme quite well, supporting IPs and reporting on programme performance. They also developed an active Technical Working Group (TWG) on gender.

Overall, the JGP’s contributions fell short of the scale of ambition reflected in its key outputs. For instance, little progress was made in the key areas of surveillance and monitoring and evaluation, making national laws gender sensitive, and establishing national interest in research on GBV. Furthermore, whilst the independent actions of individual IPs resulted in significant contributions in their specific areas, their respective outputs were often not complementary enough to generate significant cumulative effects. For instance, the research work on GBV was an isolated activity with limited potential to generate national interest on gender. This suggests limited synergy and harmonisation amongst actual activity outputs despite the apparent synergy amongst planned outputs. The notable exception is the synergy CSO IPs achieved on advocacy, mostly leveraging the media platforms cultivated by MBGE

 


2.

b) Programme Design

The JGP was developed through a consultative process and was strongly aligned with national needs and priorities, the UNDAF and UN gender norms and standards. Even so, there is evidence that it had design problems that affected its performance. These are discussed in detail in section 2.2 of this report. In summary, they are:

(i) Under-investment in analysis, especially the systematic assessment of the capacities and readiness of GeAD as the fulcrum of the programme, and the readiness of participating UN agencies (PUNOs) to work together under a joint gender programme. In the final analysis, GeAD, as the national focal point on gender, was not able to provide the horizontal and vertical coordination necessary to achieve internal and external coherence.

(ii) Failure to reconcile the scale of the programme’s ambition with the constraint imposed by resources at its disposal, which inevitably set it up for low achievement. The substantive outputs the JGP targeted require considerably more time, money and human resources than the JGP had.

(iii) Lack of focus, which limited the programme’s capacity for impact. The JGP sought to do too much, too soon and with too little. Working on both gender mainstreaming and gender based violence was a big enough challenge on its own. Yet, the JGP sought to contribute to three fairly diverse UNDAF outcomes in these areas and within them 21 outputs, some of them quite broad. It also spread its limited resources across too many IPS. In the end, even the exceptional performance of individual IPs did not aggregate into achievement of substantive outputs.

(iv) Weak definition of results: The articulation of the JGP results did not quite follow the SMART Specific, Measurable, Achievable, Relevant and Time-bound - rule. Several of them ran afoul of one or more of the SMART requirements, especially specificity and achievability

 

(v) Inadequate profiling and mitigation of risks to the achievement of results. The design of the JGP simply did not factor in risks to the achievement of outputs. Consequently, even risks as basic as an inadequately capacitated GeAD or joint gender coordination function were not identified and mitigated.

As the report on the global evaluation joint gender programmes observes, “The design process - far more than the resulting artefact of the programme document - is the foundation of a programme’s ability to deliver its results”. The defects in the design of the JGP compromised its performance and prospects for realising the benefits of a joint gender programme. 


3.

c) Relevance

The JGP is an enormously relevant programme. In establishing the JGP, the UN was responding to government request for support, made at the ministerial level. The extent of the programme’s relevance is discussed in detail in Section 4.1 of this report, where it is pointed out that the programme’s relevance is strongly affirmed at five levels, namely: evident national need; national priorities as articulated in national plans, the national vision and national policies; the mandates of implementing partners; the UNDAF; and international norms and standards such as CEDAW, the MDGs and the SDGs. The joint programme approach also responds to the imperative for the UN to approach partners as one, reduce transaction costs and speak with one voice on gender. By most accounts, the JGP scores a perfect score on relevance. 


4.

d) Coherence Synergy and Efficiency

The JGP did achieve some measure of internal and external coherence. There is a logical connectedness between individual agency outputs, JGP outputs and JGP outcomes. These links are, however, often weakened by flaws in programme design. Especially problematic are the outcomes, which were borrowed verbatim from the UNDAF/UNPOP. These outcomes are too broad to provide the necessary specificity in relation to the outputs of the JGP. For instance, Outcome 1, “Effective and efficient service delivery for the fulfilment of human rights” is too broad to provide an efficient enough anchor for the JGP outputs. It lacks the specificity necessary to establish strong causality between the JGP outputs and the outcome they support. Yet the outputs themselves are generally mutually supportive.

 

The case for internal coherence and synergy is easier to make with regard to working arrangements. There is evidence that the limited expertise and experience on gender within the UN benefited all the participating UN organisations (PUNOs) and IPs and that individual agencies were often able to bring their respective strengths together for the benefit of the programme, especially funding and expertise. The UN was also able to build a strong and motivated Thematic Working Group on Gender that will serve programming on gender well in the future. NGOs were able to work together to expand the impact of their work, leveraging opportunities opened by others. For instance, several CSOs exploited the media platforms created by MBGE to advance their own advocacy.  It is also quite evident that the results of the JGP are perfectly aligned with readily identifiable gender needs and national priorities as articulated in key national policies and plans, in particular NPGAD, Vision 2016 and Vision 2036. They are also strongly aligned to the mandates of individual Participating United Nations Organisations (PUNOs), CSO IPs, the UNDAF and global gender norms and standards. In this regard, the programme did achieve an appreciable measure of external coherence. Weakness in external coordination did limit coherence in action. For instance, there is little evidence that the JGP was able to link-up with initiatives on gender by development partners such as the European Union and USAID.


5.

e) Accountability

By all accounts, the JGP excelled on accountability for resources. The UN has good systems for the management of, and accountability for, programme resources and these were strictly adhered to. However, the programme had serious challenges with regard to accountability for results. Monitoring, which is the principal method of ensuring accountability for results, was weak. Furthermore, there was no systematic approach to mutual accountability, with the result that PUNOs who failed to meet their obligations to the JGP were not held accountable. There is no evidence that the government and the UN held each other accountable for overall programme performance either. Whilst mechanisms such as the Component Coordination Group (CCG), and the Programme Steering Committee (PSC) could serve mutual accountability purposes, they focused on reports by IPs. Leadership engagement, that is the RC and the UNCT, is crucial for mutual accountability. And so is the engagement of the government. There is little evidence of such engagement by the parties to ensure that each performed its role.


6.

f) Sustainability

 

There is strong evidence that the results the JGP pursued shall be sustained beyond the life of the programme. First, each of the JGP components has a narrative that suggests three elements of the thinking on sustainability at the design stage, namely, integrating gender issues into laws and policies, changing attitudes at the “grass root” level and building the capacity of duty bearers. However, more supportive to the sustainability of JGP results are context issues. These include strong commit to gender by the government, CSOs all PUNOs.

The GoB has already made firm commitments to fund programmes on gender mainstreaming and GBV beyond the life of the programme. CSOs are also positioned to continue their work on gender mainstreaming and GBV beyond the current JGP cycle. To begin with, JGP funding constituted only a small proportion of programme/project funding of most CSO stakeholders, the only exception being Gender Perspectives. Second, CSOs are mobilising resources from development agencies such as the European Union. It helps that most donors recognise that gender is a powerful enabler of the achievement of results in other areas of development and are, as a consequence, committed to supporting work on gender. 


7.

g) Value Addition

 

The JGP did add value. It produced some consequential outputs, notably integrating gender into NDP 11 and Vision 2036, and raising the profile of gender, both within the UN and nationally. The programme has shown that UN agencies can work together harmoniously and achieve results on gender, engage partners as one, and speak with one amplified voice. It has built a strong, motivated and active UN Thematic Working Group on gender. The PUNOs have gained invaluable lessons, both positive and negative, on working together through a JGP. The JGP has laid the foundation for building a broad national coalition on gender. Through the JGP, CSO partners that have hitherto worked independent of each other have discovered synergies amongst themselves. Finally, beyond bringing stakeholders to work together, the JGP has helped amplify advocacy on gender.


Recommendations
1

A successor Joint Gender Programme should be developed.

2

Invest in the delivery capacity of Gender Affairs Department (GeAD)

3

Improve programme design significantly.

4

Strengthen accountability systems.

5

Improve advocacy and communication.

6

Stronger resource mobilization.

1. Recommendation:

A successor Joint Gender Programme should be developed.

Management Response: [Added: 2017/01/03]

Development of a successor JGP programme will be informed by a collective decision of the UN country team (UNCT) in consultation with key partners (i.e. State and Non State Actors)

Key Actions:

Key Action Responsible DueDate Status Comments Documents
Present the evaluation recommendation to the UNCT for consideration.
[Added: 2017/01/03] [Last Updated: 2017/03/17]
JGP technical working group 2017/03 Completed History
2. Recommendation:

Invest in the delivery capacity of Gender Affairs Department (GeAD)

Management Response: [Added: 2017/01/03]

The UN in Botswana through the Programme recognizes that GeAD is an important partner and facilitator of gender equality, women’s empowerment and the eradication of GBV. The UN will continue to pursue capacity development of GeAD either thorough proactive engagement or responsiveness to GeAD requests for capacity enhancement. 

Key Actions:

Key Action Responsible DueDate Status Comments Documents
Present the evaluation recommendation to the UNCT for consideration and implementation through identified programme initiatives.
[Added: 2017/01/03] [Last Updated: 2017/03/17]
JGP technical working group 2017/06 Completed History
3. Recommendation:

Improve programme design significantly.

Management Response: [Added: 2017/01/03]

If the successor programme is approved and developed, there will be stricter adherence to UN programming principles as well as adoption of best practices in terms of design of gender based programmes.

Key Actions:

Key Action Responsible DueDate Status Comments Documents
Present the evaluation recommendation to the UNCT for consideration and implementation in the next programme.
[Added: 2017/01/03] [Last Updated: 2017/03/17]
JGP technical working group 2017/06 Completed History
4. Recommendation:

Strengthen accountability systems.

Management Response: [Added: 2017/01/03]

If the successor programme is approved and developed, accountability systems for the programme will strengthened based on the lessons learnt and recommendations from the current programme. 

Key Actions:

Key Action Responsible DueDate Status Comments Documents
Present the evaluation recommendation to the UNCT for consideration and implementation assuming that a successor programme will be developed.
[Added: 2017/01/03] [Last Updated: 2017/03/17]
JGP technical working group and implementing partners. 2017/06 Completed History
5. Recommendation:

Improve advocacy and communication.

Management Response: [Added: 2017/01/03]

If the successor programme is approved and developed, advocacy and communication for the programme will strengthened based on the lessons learnt and recommendations from the current programme. 

Key Actions:

Key Action Responsible DueDate Status Comments Documents
Present the evaluation recommendation to the UNCT for consideration and implementation assuming that a successor programme will be developed.
[Added: 2017/01/03] [Last Updated: 2017/03/17]
JGP technical working group Implementing partners 2017/06 Completed This recommendation will be operationalized through development and implementation of an advocacy and communications strategy. History
6. Recommendation:

Stronger resource mobilization.

Management Response: [Added: 2017/01/03]

If the successor programme is approved and developed, stronger resource mobilization for the programme will strengthened based on the lessons learnt and recommendations from the current programme. All key partners will be required to uphold their financial commitments and be innovative in resource mobilization especially in a UNMIC.

Key Actions:

Key Action Responsible DueDate Status Comments Documents
Present the evaluation recommendation to the UNCT for consideration and implementation assuming that a successor programme will be developed.
[Added: 2017/01/03] [Last Updated: 2017/03/17]
JGP technical working group Implementing partners 2017/06 Completed The UN has a Joint Resource Mobilization and Partnership Strategy that can be utilized to inform the process History

Latest Evaluations

Contact us

1 UN Plaza
DC1-20th Floor
New York, NY 10017
Tel. +1 646 781 4200
Fax. +1 646 781 4213
erc.support@undp.org