Rights, Empowerment and Cohesion (REACH) for Rural and Urban Fijians (REACH) Mid Term Evaluation Report

Report Cover Image
Evaluation Plan:
2018-2022, Fiji
Evaluation Type:
Mid Term Project
Planned End Date:
05/2018
Completion Date:
05/2018
Status:
Completed
Management Response:
Yes
Evaluation Budget(US $):
20,000

Share

Document Type Language Size Status Downloads
Download document REACHMidTermEvaluationReport.pdf report English 3314.08 KB Posted 334
Download document TOR_REACH_Mid Term Evaluation Report.docx tor English 80.14 KB Posted 355
Title Rights, Empowerment and Cohesion (REACH) for Rural and Urban Fijians (REACH) Mid Term Evaluation Report
Atlas Project Number: 00083209
Evaluation Plan: 2018-2022, Fiji
Evaluation Type: Mid Term Project
Status: Completed
Completion Date: 05/2018
Planned End Date: 05/2018
Management Response: Yes
UNDP Signature Solution:
  • 1. Governance
Corporate Outcome and Output (UNDP Strategic Plan 2018-2021)
  • 1. Output 2.2.3 Capacities, functions and financing of rule of law and national human rights institutions and systems strengthened to expand access to justice and combat discrimination, with a focus on women and other marginalised groups
SDG Goal
  • Goal 16. Promote peaceful and inclusive societies for sustainable development, provide access to justice for all and build effective, accountable and inclusive institutions at all levels
SDG Target
  • 16.3 Promote the rule of law at the national and international levels and ensure equal access to justice for all
Evaluation Budget(US $): 20,000
Source of Funding: Project
Evaluation Expenditure(US $): 25,455
Joint Programme: No
Joint Evaluation: No
Evaluation Team members:
Name Title Email Nationality
Welmoed Koekebakker Consultant welmoedk@gmail.com
GEF Evaluation: No
Key Stakeholders: UNDP; Ministry of Women, Children and Poverty Alleviation; Legal Aid Commission; Donor Government of Japan.
Countries: FIJI
Lessons
Findings
1.

4. Findings

4.1 Achievement of Outputs

EQ: Is the Project on track to achieve the stated outputs?

4.1.1 Project outputs by 2017: in majority well achieved

The REACH project achieved the majority of its targeted outputs by the end of 2017. The project managed to catch up with the delays experienced in 2016 largely as a result of Tropical Cyclone Winston. By 2017, the project implemented 6824 outreach missions, covering 677 communities. 

Overall outputs are well on track. Several essential project outputs exceed targets to a significant extent. See Table 1, below.

4.1.2 Three mobile service delivery unit (buses) in operation Mobile service delivery and awareness raising on the rights are innovatively being conducted in partnership with the concerned public service delivery agencies. Three mobile units (buses) that were commissioned in April 2017 function as ‘mobile offices’, travelling to remote areas of Fiji providing opportunities for communities to develop their understanding and seek services provided by the Ministry of Women, Children and Poverty Alleviation, the Legal Aid Commission, the Human Rights and Anti-Discrimination Commission and other public sector agencies in Fiji.

4.1.3 Number of outreach missions and use of the buses Since the beginning of the REACH project, in total 677 communities have been reached including the maritime areas. Since the buses commenced operations, 424 communities have been reached in Central, Western and Northern Divisions for awareness training and services. Out of these, the REACH bus was used in 250 communities, or 59% of the REACH missions. Community visits reached by 4WD vehicles (where the REACH bus could not be used - non-accessibility): 174 (41%)

4.1.4 Districts covered Between July 2015 and December 217, REACH managed to reach 113 districts out of 196 districts in Fiji (58%), in all 14 provinces and Rotuma. Outputs exceed the target of “50% of districts covered”. The map of “REACH project – Districts covered” shows the reach-out of the project (See page iv). Remarkably, a significant part of the most unreached maritime areas have been covered. This reflects the overall project approach of “reaching the furthest behind first”. The map also shows that there is a great need for continuation of REACH activities in the districts still uncovered. Moreover, in districts already covered there is a need for deepening the REACH strategy in 2018 and, if possible, beyond 2018.

4.1.5 Awareness raising Up to December 2017, 13446 people in remote areas attended awareness sessions provided by REACH. These outputs exceed the targets to a significant extent - by over 50% - as the target was to reach 8800 people up to December 2017. 

4.1.6 Service Delivery By December 2017, 17110 people in remote areas were provided services by the REACH project. These outputs exceed the targets to a significant extent (outputs are more than double the targets) as the target was 7500 by December 2017.

4.1.7 Strategies and communication tools The REACH target was to develop 5 major “communication products, strategy documents of toolkits” by December 2017. These tools are expected to be gender responsive. This output target was fully reached: the project (with project stakeholders) developed a set of 7 awareness raising videos on service delivery in three languages (Fiji vernacular, Hindi and English) for outreach activities; a Toolkit, a Legal Aid Commission Strategic Plan, banners, a project video, and factsheets. The Toolkit constitutes an excellent project management tool to be used for all provider-level participants in the REACH project.

4.1.8 Training, strategic planning The project output in terms of training has not been fully in line with what was envisaged. The number of staff trained in relation to “strategic planning and awareness raising skills” was 99 in 2016 (target: 100)30 and 42 in 2017 (target: 100). The underlying justification is that it was considered a priority in 2017 to (after the delays incurred in particular due to TC Winston) fully focus on the outreach activities and the support to the bus operations, rather than on trainings . The workshops generally use a mechanism to monitor training effectiveness (measuring perceived increase of awareness). Some of the workshops have been highly appreciated by participants, as communicated to the MTE. Recommendation for 2018: Implement the training targets. Implement two trainings: 1) On Gender - see below, and 2) Strategic Lessons Learned workshops, end of 2018, for each of the 4 districts.

4.1.9 Participating women and men Participation of women was equal to participation of men in the awareness raising activities (50%) in the period 2015-2017. In the delivery of services, the project reached more women than men: 55%. This is in line with the overall project focus on reaching out to women and vulnerable groups. 


Tag: Rural development Gender Equality Gender Mainstreaming Human rights Justice system Local Governance Communication Knowledge management Partnership Service delivery Social cohesion Capacity Building Advocacy

2.

4.2 Progress towards Outcomes

EQ: What progress towards the outcomes has been made? EQ: What is the quality of the outputs produced thus far?

The REACH Project’s intended Specific Project Outcome is: to strengthen and promote sustainable democratic governance by adopting a peaceful, socially cohesive and socially inclusive approach. The project will support key government agencies in becoming more effective and accountable in their provision of a system of justice and the rule of law, at the same time promoting the rights of women and youth. Effective, accountable and strengthened key government agencies that are promoting peace building, social cohesion, inclusiveness and gender equality through their provision of awareness raising and delivery of services.

4.2.1 Quality of outputs One of the Evaluation Questions is on the quality of outputs. There are different ways to answer this question. As elaborated in the chapter on Methodology, this Evaluation focuses on Outcome in the results-chain ‘input-output-outcome-impact’, trying to catch credible linkages between the output of the action and intended changes. Quality of outputs is, consequently, not perceived as the “sum of quality properties of distinct outputs”, but importantly “quality of outputs” refers to the ability of outputs to contribute to outcome, in other words to the ‘junction’ between output and outcome. In this logic, “contribution to Outcome” or “progress towards Outcome” refers precisely to the quality of outputs.

4.2.2 MTE Recommendation on Outcome indicators The project does not yet have robust indicators for ‘strengthened key government agencies’ Recommendation: Develop robust key qualitative indicators, jointly with the MWCPA and LAC, in 2018, for the intended outcome of “Effective, accountable and strengthened key government agencies that are promoting peace building, social cohesion, inclusiveness and gender equality through their provision of awareness raising and delivery of services”. These indicators should be: - ‘owned’ and monitored by the key government agencies MWCPA and LAC; and - include composite indicators; qualitative-quantitative; provider-recipient indicators. They may be used in the Final Evaluation of the REACH project and the proposed RRF for a possible REACH Phase II.

4.2.3 Perceived impact: service provider perceptions In the perception of REACH stakeholders the REACH project has impact in many ways. Dr. Josefa Koroivueta, Permanent Secretary of the Ministry of Women, Children and Poverty Alleviation: “The REACH project has impact on the service providers themselves: it gives an opportunity to work together and reach out together; for some of them it is the first time to go to a rural area and experience rural village life; they get experience in public speaking (…) and it changestheir mindsets. There is also institutional impact: the platform for a comprehensive follow-on to REACH – a “REACH-Plus” - is already there”. Mr. Rupeni Fatiaki, Director Social Welfare, Ministry of Women, Children and Poverty Alleviation: “The REACH project has impact: it has improved our services. Before, cases could be pending a long time, now we see the people face to face and we make a commitment, we assure them that we will follow up on their case. (…) We are developing a new tool, a “Start-to-Finish Tracker” that may help them” . “As an institutional impact of REACH, the Ministry has decided to integrate the REACH approach in its system and in its regular funding. Now we schedule quarterly 2-3 weeks visits to Rotuma”.Several interviewees during the Mid-Term Evaluation report that REACH, in their perception, has impact on a more client centered approach in service delivery: “REACH covers places where we have never ever set foot. People tell us: “this is the first time for us to see a government officer at our doorstep, in the village”. The kind of communication is totally different. If they come here we have 3 minutes for them, but if we go there they have time to explain, we can put it in context and make a better decision. We are putting ourselves in their shoes.

In an interview with the Legal Aid Commission, the Mid-Term Evaluation asked: what are, in your view, indicators of impact of the REACH project? How do you know you can attribute the increased demand for legal aid to the REACH project? The Legal Aid Commission mentioned the following indicators of impact on the beneficiaries: 1. People mentioning that because of REACH they apply for legal aid, 2. Statistical evidence: a significant increase in legal aid applications right after the start of REACH 3. Evidence that the legal aid provided through REACH has helped people in tackling the underlying problems, e.g., domestic violence 4. Other people in the community observe this, and also apply for legal aid.

After a REACH meeting, sometimes the advisory councellor brings people applying for legal aid. The LAC also mentioned several indicators of impact on the service providers: 1. “For us it is an eye-opener. It was the first time for me to travel to these villages. I saw that there are so many people in need”. 2. “We get insight in the problems of the women. Separate women’s meetings are more effective as we get more insight in what are the real problems of the women”, 3. “I learned to accept different responses; sometimes people are rough. We have to be patient”. 

The MTE found several cases evidencing that it is precisely the integrated approach of the REACH project that makes it possible to serve the most vulnerable people. One of the members of a REACH mission found: “…we had one-to-one conversations with women’s leaders and women’s groups members; we were able to discuss development plans, future projects and community health projects. So the REACH outcome goes beyond its objectives”.


Tag: Effectiveness Human rights Justice system Rule of law Results-Based Management Service delivery Peace Building Social cohesion

3.

4.2.4 Perceived impact: beneficiary perceptions

The Mid-Term Evaluation spoke with beneficiaries of REACH missions attending awareness sessions and applying for service delivery and some beneficiaries of former REACH missions. Many expressed their appreciation for the information and services delivered. Four elderly women in Naqalotu, Kadavu, all single, were happy that they can apply for social pension as they would never have the opportunity to go to Vunisea. Four young women in Naqalotu, Kadavu, needed a birth certificate: important because they want to apply for job and go to school; in the REACH meeting they heard about women’s rights for the first time. A couple in Tavuki, Kadavu, was married by BDM during the REACH mission: they had never married because they can’t afford the travel to Vunisea, a boat trip costing 70 dollar per person. In Nasolo, Western Division, 3 women, in their fourties, heard information about women’s rights for the first time. In Drasa, one of the participants will use the information she received at the REACH meeting for further development of the Melanesian women’s group. Another lady in the same meeting, advised by the Women’s Department at the Ministry, is going to register her women’s organization “Kula” (Bird).

The Mid-Term Evaluator interviewed several REACH participants during and after the presentations, asking: Did you understand the presentation? The answers were mixed: in the Western and Central Division participants generally answered that they did understand the presentations, and they mentioned at least one issue that is important for them; in Kadavu several women were unable toanswer the question. This may be related to various factors (being ‘further behind’, remoteness, information gap, set up of the meeting, noise, etc) – the evidence is too limited to draw any conclusions at this stage.

One of the women interviewed in a REACH meeting was skeptical. “I am tired of coming to the Social Welfare Office, they keep telling me that my application has gone to Suva and then they ask me to bring the same documents that I gave before, and then they say that my application got lost … every time it costs me 30 dollar and 4 hours, I have already spent more than 100 dollars on travel …”. This lady was fortunate that she is now assisted by a Social Welfare Officer who reports that she is determined to do her best to get this application moving. The MTE also received some highly positive feedback. Lalita lives in Matbani village, right at the seaside. “The bus came here in the village. I didn’t know whether I’m entitled to receive a social pension. After I learned from the bus people that I have a right to a social pension I went to Rakiraki with my birth certificate. After 2 months there was no message so I went again and they arranged my social pension. If the bus hadn’t come I would not have received the pension. Thank you to the Government of Japan.”


Tag: Effectiveness Impact Gender Equality Women's Empowerment Human rights Programme Synergy Service delivery Peace Building Social cohesion Social Protection Advocacy SDG Integration

4.

4.3 Key contributing factors and appropriateness of approach

This Mid-Term Evaluation identified 8 factors pre-eminently contributing to the achievement of project outputs. They are basically the essential components of the REACH project approach, contributing to its effectiveness. They are coherent and mutually strengthening and all of these should be integrated in the project design in a possible Phase II.

This chapter answers four interrelated Evaluation Questions: - What factors have contributed to the status of achieving or not achieving intended outputs? - What factors are contributing to effectiveness or ineffectiveness? - What is the appropriateness of the Project approach? - Has the UNDP partnership strategy been appropriate and effective?

4.3.1 Reaching the unreached first As shown in the map of districts covered by the REACH project (page iv) a significant part of the most unreached areas has been covered: 58% of all districts. This reflects, and is in line with, the overall project approach of “Reaching the furthest behind First”. This approach is appropriate, as the REACH intervention logic, Theory of Change, objectives, target groups and activities are coherent and consistent. The focus on those unreached is an essential component of the project. Without the mobile service delivery approach, without the huge efforts to organize coordinated REACH service delivery missions in the most remote areas of Fiji, the targets would not have been met.

4.3.2 Integrated approach The collaboration between Government Ministries, Departments and statutory bodies has been a key element of the ‘integrated approach’ of the project that has precisely contributed to its effectiveness. The REACH approach has also been called “holistic” as REACH combines several approaches and components that are interrelated and mutually reinforcing. There is a wide range of examples of projects worldwide on coordinated service delivery (one-stopstops Vietnam, single window services in Cambodia and Indonesia, mobile services in Australia, Canada, Azerbeidjan etc) but it is quite unique for a project to be able to integrate social services, civil registration (BDM) and legal aid services in one coordinated mobile outreach approach. The level of collaboration between stakeholders in the REACH project is also remarkable, comparing positively with experiences elsewhere.

Horizontal and vertical integration The integrated approach applied by the REACH project also includes working with the traditional structures in place. The REACH project ensures coordinating with the local leaders and local traditional structures, including the Turaga ni Koro, the Roko Tui, the Bose Vanua (meeting for traditional chiefly leaders), the Bose Va Koro (Village council meeting), the Bose ni Tikina (District council meeting). These are the local structures where the local people seek assistance and the REACH project seeks to complement these rather than putting new systems in place. The REACH approach is also ‘integrated’ in the sense of building on ongoing commitments and activities at the Ministry. For example, the Ministry (MWCPA) identified the Women’s Expo as an opportunity to promote the project message and deliver services to the 600 women participants, spread information to the 11,000+ Expo visitors and strengthen the synergy between project partners.This decision has been appropriate in view of effectiveness (number of women reached – see the two separate activity reports) and efficiency (cost sharing). The integrated approach of not developing a parallel structure within the Ministry (MWCPA) - working with the existing project staff in the Departments of Women and Social Welfare, not creating a separate REACH team within the Ministry, is also perceived as appropriate, as this enables maximum ownership, integrating of lessons learned and practices, and contributing to the long term sustainability of the project.

4.3.3 Partnership strategy

The partnership strategy has been appropriate and effective. The project was designed with two key stakeholders, the Ministry of Women, Children and Poverty Alleviation and the Legal Aid Commission. The expansion of partnerships (BDM, FHRADC, Fiji Police Force) has been identified as a need and a potential as the project evolved, and the project has been able to actually materialize these partnerships, with more partnerships ‘on the ground’ as the need arose (e.g., the Ministry of Education, as observed during the Kadavu mission). The stakeholders interviewed during the Mid-Term Evaluation are highly committed to the REACH project. For the stakeholders the REACH project provides an ‘added value’ as ‘reaching out to those unreached’ is in their mandate but they lack the financial means to do so. The partners report good working relations, teamwork during missions, opportunities to learn from each other. The timely and accurate reports and project updates to stakeholders are appreciated. The partners have mobilized relevant expertise. An indicator of the level of mutual trust reflecting good working relations may be in the sharing of sensitive project information. The level of Capacity Development for stakeholder MWCPA has been less than anticipated as explained in par. 4.1.8 above. Capacity Development for LAC has effectively contributed to the (draft) LAC Strategic Plan. Ensuring continued good collaboration, ownership, shared understanding of approaches, and synergies between partners at all levels is a challenge. The outreach missions are to some extent an excellent way to get to know each other and strengthen ‘team building’. This is also mentioned in the responses to the written MTE surveys. The project has organized “lessons learned workshops” that have contributed to mutual understanding. Regular Board meetings are important. The aim for REACH Phase II is that the existing partnership with the Ministry of Women, Children and Poverty Alleviation, and Legal Aid Commission be further strengthened, with the potential to formally include the Human Rights and Anti-Discrimination Commission; to be expanded with additional service providers when the need arises.


Tag: Gender Mainstreaming Communication Innovation Integration Partnership Policies & Procedures Results-Based Management

5.

4.4 Challenges

The REACH project has faced many challenges as the project evolved, and in retrospect it can be concluded that most of them have been addressed well and in a timely manner. Four examples:

1. The broad scope of the project appeared a challenge for focus on transformational results. The Project Board decided to narrow the scope, removing potential overlapping activities with other support programmes such as the Fiji Access to Justice Programme.

2. Data collection for baseline analysis and coordination with other programmes was identified by the PM as a challenge. All stakeholders adhered to the importance of evidence based results.

3. Tropical Cyclone Winston resulted in implementation delays. The 2017 Board Meeting decided to extend the date of project completion to 31 December 2018.

4. The organisation of missions to the Eastern Division (maritime) has been highly challenging. Intensive coordination is required by UNDP. Weather conditions in relation to local travel by boat is again a challenge. Safety and security precautions are observed.


Tag: Challenges Efficiency Human and Financial resources Monitoring and Evaluation Programme/Project Design Project and Programme management Coordination Data and Statistics

6.

4.5 Mainstreaming Gender, Rights-based approach

4.5.1 Mainstreaming Gender

“Gender, culture and tradition” is one of the key challenges of the project. How does the REACH project manage to work through the traditional local village structures and remain faithful to the very objectives of the project: Rights, Empowerment and Cohesion, and Gender Mainstreaming?

- In terms of quantitative outputs, the focus on women and gender mainstreaming has been highly effective. - In terms of a qualitative contribution to awareness and service delivery to women, the project has made progress and gained a rich body of experience from which to draw lessons learned. - In the last phase of REACH (2018), there is a need for a more robust methodology on strengthening a gender perspective in the REACH project using qualitative indicators.

4.5.2 The numbers In the awareness raising activities of the REACH project in the period 2015-2017, participation of women was equal to participation of men (50%). In the delivery of services, the project reached more women than men: 55%. The project has been effective in reaching out to women. What do these figures signify? Has the participation of women helped them to actually receive the services? Is it possible, in the remaining project period (2018) and in a possible REACH II, to increase women’s ‘qualitative participation’, through a stronger gender sensitive approach? One of the answers to this question is addressed in the pilot S2F. See Chapter 3.7

4.5.3 Integrated in project cycle The Mid-Term Evaluation undertook a gender assessment of the Results Framework, Intended outcomes, Specific project outcome, Key Result Areas, Project objectives, Intended Outputs, Strategy, Intended results, Change pathway, Partnerships strategy, Synergies, M&E, Data collection, Project reporting, Mission reports, Communication, Visibility, Quality Assurance. The MTE concludes that ‘gender’ is fully integrated in the RRF, the project cycle, M&E. The challenge is at the implementation level: ‘how’. The project has already really thought through how to approach women in the best way, including taking into account what are the specific needs of women, the way to invite women, timing of the mission, form of presentations, the topics of presentations (highlighting issues in the interest of women, e.g., LAC focusing on family law), communication materials, seating, large number of women involved in the outreach missions making women feel more at ease, etc.

4.5.4 Challenges and best practices This MTE identified some gender-related challenges during implementation of field missions. See table 2, Chapter 4: “Field missions (2): challenges observed by the Mid-Term Evaluation”. The MTE also observed how some of these challenges are mentioned during mission briefings; sometimes they are discussed in advance missions with the roko and turaga ni koro, and they are mentioned in mission debriefings.

The MTE also observed several practices that work well, such as: - Separating sevusevu from service delivery, using different spaces, separate meetings for women: this seems to be conducive both for servicing/ empowering women and respecting sevusevu (compare Tavuki, Kadavu and Nakavika, Central Division). - Ms. Selai Korovusere, Director Women, Department of Women, Ministry of Women, Children and Poverty Alleviation: “We coordinate with women’s groups registered with us, we tell them that the REACH team is coming”. - In Kadavu the local nurse volunteered to help with BDM applications as women were queuing up - Recommendation: involve women’s groups/female leaders/service providers. So far, project focus on quantitative approach: women’s participation in numbers. Successful. Need to move on to qualitative approach: effective participation, increase in awareness, increase and quality of service delivery, empowerment, including client orientation of service deliverers. Continue to integrate gender mainstreaming in all aspects of the project: design, tools, methods, BTOR, M&E, reporting, visibility: document challenges, address challenges - Strengthen the methodology to assess qualitative impact. - Strengthen the gender capacity of the stakeholders. - Address the challenges (par. 4.4), e.g., include women’s groups in field mission preparations.

4.5.5 Recommendations The MTE organized a Gender discussion with the REACH Deputy Programme Manager and REACH coordinators. The meeting resulted in 4 Recommendations:

1. Training (continued) on how to focus on gender equality - UNDP with key stakeholders - for villagers: a) separate trainings for women (including at special events e.g. Women’s Expo), and b) separate training for men, including Turaga ni Koro, 2. Make ‘gender’ part of all aspects of REACH missions: a) Advance, b) During, c) Post 3. Make ‘gender’ a mandatory section in the BTOR report; 2 qualitative questions: - What did we see as a challenge in integrating a gender focus in this REACH mission? - What do we see as a possible solution, within out competence? 4. In August 2018, a meta-analysis of BTOR reports on Gender in REACH. Expected outcome: Best practices, to be incorporated in proposal REACH II.


Tag: Gender Equality Gender Mainstreaming Women's Empowerment Human rights Justice system Project and Programme management Results-Based Management Service delivery Capacity Building Disabilities Advocacy

7.

4.6 To what extent has UNDP assistance contributed to outputs?

UNDP is in charge of the overall project management in a comprehensive way and as such the UNDP contribution to the project has been fundamental to the achievements of the project. 

The overall management of the REACH project by UNDP has been very good. This conclusion by the independent Mid-Term Evaluation was confirmed in interviews with project stakeholders.

Day-to-day management and programme management support UNDP has set up management arrangements for the REACH project in an accountable and transparent manner. The UNDP Project Manager is in charge of the day-to-day management and decision-making for the project on behalf of UNDP. UNDP provides strategic guidance and programme management support to ensure procurement, financial and human resource management and reporting is in accordance with UNDP rules and procedures and in accordance with contractual arrangements with the donor and key stakeholders where applicable.

Reporting, transparency and relations with stakeholders A key role for UNDP is to ensure that project donors, development partners and stakeholders are kept fully informed of project’s progress. The transparency and accountability to the donor and key stakeholders is perceived as very good. The Embassy of Japan commented to the Mid-Term Evaluation that the Embassy is highly satisfied with the reports on the REACH project: “They are very good, precise and well-written, in comparison to other implementing partners”


Tag: Effectiveness Efficiency Human rights Justice system Human and Financial resources Oversight Quality Assurance Results-Based Management Technical Support

8.

4.7 Mid-Term observations on Relevance, Efficiency, Effectiveness, Impact, Sustainability

A more comprehensive evaluation against the DAC/OECD criteria is expected to be part of the Final Project Evaluation and is beyond the scope of this MTE. “This MTE will make observations on the DAC/OECD questions as and where relevant in the context of the Evaluation Questions.

4.7.1 Relevance

The project is relevant in view of: - its contribution to the realization of the very values and rights detailed in the Constitution of Fiji, - its contribution to the SDGs 5 and 16, - its contribution to specific needs identified in Fiji, - the fact that service providers interviewed by this MTE perceive the project as highly relevant, - the fact that beneficiaries in remote areas interviewed by this MTE during REACH missions perceive the project as highly relevant. 

4.7.2 Efficiency

Reaching the most vulnerable and remote people comes with a cost. This is a value-based dimension of the project. The project implementation is efficient in view of:

- sufficient evidence that the project is implemented in line with UNDP guidelines on efficiency,

- evidence that the project followed the procurement guidelines stipulated in the contract with the Government of Japan and the UNDP internal guidelines,

- considerations of cost-effectiveness have played a role throughout the project implementation, This Mid-Term Evaluation did not observe any evidence of lack of cost-effectiveness. Ultimately, there is no way to assess the cost-effectiveness of the project other than in immediate relation to the overall impact of the project.


Tag: Vulnerable Effectiveness Efficiency Impact Relevance Sustainability

Recommendations
1

Draw a realistic plan for the use of the mobile service delivery units (buses); follow up technical issues.

2

Arrange a longer-term mission planning; site selection to be coordinated wherever possible, and Plan B for risk factors.

3

Address implementation challenges identified in this MTE (see Table 2)

4

Clarify the issue of allowances and solve any misunderstandings related to allowances, the sooner the better

5

Develop a robust timely and appropriate Management response to address the reported challenge related to the ‘inability to cope with the increased demand’, in line with the modalities identified in the Risk Log. (See paragraph 4.4.8)

6

Include qualitative indicators in BTOR format.

7

Include gender reporting in BTOR format.

8

Undertake a meta-analysis of BTOR reports on lessons learned, and integrate these in the design of REACH Phase II.

9

Identify whether there is a need for technical support to MWCPA in 2018 in view of current priorities (ref. 1.1.4, 1.1.5, 1.3.1, 1.3.2); implement in line with needs assessment, if needed, with support of UNDP

10

Hold a strategic lessons learned workshop at the end of 2018, possibly as part of Final Evaluation, in 4 districts.

11

Deepen the gender approach, increase awareness, increase the quality of service delivery, empowerment, client orientation of service deliverers.

12

Continue to systematically integrate ‘gender’ in all aspects of the project.

13

Strengthen the methodology to assess qualitative impact.

14

Strengthen the gender capacity of the stakeholders.

15

Develop a gender action plan for 2018:

1. Training on how to focus on gender equality, a) UNDP with key stakeholders, b) at village level: separate trainings for women (including at events e.g. Women’s Expo), and separate training for men.

2. Make ‘gender’ part of all aspects of REACH missions: a) Advance, b) During, c) Post.

3. Make ‘gender’ a mandatory section in the BTOR report; 2 qualitative questions on challenges observed in addressed ‘gender’ in this REACH mission, and possible solutions.

4. In August 2018, a meta-analysis of BTOR ond Gender in REACH.

16

Implement the pilot and develop recommendations to be included in design REACH Phase II.

17

Develop key indicators, MWCPA and LAC with UNDP (See Ch. 4.2): include composite; qualitative and quantitative; provider and recipient indicators.

18

Identify lessons learned on client-centred service delivery.

19

6. Recommendations

6.1 Follow-up 2018 EQ: Are there any follow-up actions to be taken or any necessary adjustments, including if indicated the reorientation of the Project?

RECOMMENDATIONS for 2018, to UNDP and all stakeholders, unless mentioned otherwise: 

1. Field mission planning and implementation: - To all stakeholders, in particular MWCPA: Draw a realistic plan for the use of the mobile service delivery units (buses); follow up technical issues - Arrange a longer-term mission planning; site selection to be coordinated wherever possible, and Plan B for risk factors. - Address implementation challenges identified in this MTE (see Table 2) - To UNDP, MWCPA, LAC: Clarify the issue of allowances and solve any misunderstandings related to allowances, the sooner the better.

2. To the management of the MWCPA and LAC: Develop a robust timely and appropriate Management response to address the reported challenge related to the ‘inability to cope with the increased demand’, in line with the modalities identified in the Risk Log. (See paragraph 4.4.8) 

3. Field mission reports, Monitoring, Research for evidence based policy making (Output 2.1.3). - Include qualitative indicators in BTOR format. - Include gender reporting in BTOR format. See paragraph on gender.- Undertake a meta-analysis of BTOR reports on lessons learned, and integrate these in the design of REACH Phase II.

4. Capacity Development for stakeholders: - To MWCPA: Identify whether there is a need for Technical Support to MWCPA in 2018 in view of current priorities (ref. 1.1.4, 1.1.5, 1.3.1, 1.3.2); implement in line with needs assessment, if needed with support of UNDP, - Gender trainings: see below - To UNDP and project stakeholders: Strategic Lessons Learned workshops, end 2018, possibly as part of Final Evaluation, 4 districts. Expected Outcome: inputs for design REACH Phase II

5. Gender: - Deepen the gender approach, increase in awareness, increase and quality of service delivery, empowerment; client orientation of service deliverers. - Continue to systematically integrate ‘gender’ in all aspects of the project - Strengthen the methodology to assess qualitative impact. - Strengthen the gender capacity of the stakeholders. - Address the challenges (Table 2 chapter 4.2) - Gender action plan for 2018:

1. Training on how to focus on gender equality, a) UNDP with key stakeholders, b) at village level: separate trainings for women (including at events e.g. Women’s Expo), and separate training for men, 2. Make ‘gender’ part of all aspects of REACH missions: a) Advance, b) During, c) Post

3. Make ‘gender’ a mandatory section in the BTOR report;

2 qualitative questions on challenges observed in addressed ‘gender’ in this REACH mission, and possible solutions.

4. In August 2018, a meta-analysis of BTOR on Gender in REACH. Expected outcome: Best practices, to be incorporated in proposal REACH II.

6. Innovation Pilot S2F Delivery Tracker: implement the pilot and develop recommendations to be included in design REACH Phase II

7. Outcome Indicators: Develop key indicators, MWCPA and LAC with UNDP (See Ch. 4.2): - ‘owned’ by MWCPA, LAC (what is a key indicator for increased capacity expected?) - include composite indicators; qualitative and quantitative; provider and recipient indicator

8. Input in design Phase II: Assess: which groups are not yet reached and why?

9. Partnerships and stakeholders: - As part of the design of a possible REACH Phase II, revisit partners’ expectations, challenges, and strengths. - Identify lessons learned on client centered service delivery.

20

6.2 Key components for a follow-on REACH Phase II EQ:

What are the key potential components, necessity and expected outputs for a follow on Project?

Expected output A “REACH Phase II” is needed to:

- consolidate the achievements of REACH I, - deepen the qualitative approach, - expand the project’s coverage (number of districts, villages), including in maritime areas, - deepen the project’s coverage in terms of people receiving services, focus on “reaching the unreached first”, reach the “most vulnerable amongst the vulnerable”, - strengthen monitoring and accountability mechanisms, using innovative approaches

21

RECOMMENDATIONS for REACH PHASE II

To all project stakeholders (UNDP, MWCPA, LAC, other) unless mentioned otherwise:

1. Project design: 8 project components essentially constitute the project’s effectiveness: Reaching the unreached first, Integrated approach, Partnership strategy, Gender focus, Evidence based approach, Commitment, Innovation and Communication strategy. These should all be prominent components in Phase II.

2. Mobile service delivery units: REACH II to expand the project’s coverage, reaching out to communities that have not yet received the services, with continued focus on “reaching the unreached first” and also reach the “vulnerable amongst the vulnerable”; and revisiting communities visited in Phase I for follow up. Reaching remote communities may require acquiring additional vehicles or boat transportation.

3. Partnership: Partnership with the Ministry of Women, Children and Poverty Alleviation, and Legal Aid Commission to be strengthened, potentially to formally include the Human Rights and AntiDiscrimination Commission, and additional service providers as the need arises and as feasible.

4. S2F Delivery Tracker system: to be implemented based on the results of the 2018 Pilot, to ensure that services are delivered and completed. 

5. Institutional strengthening of the evidence based approach - data collection, monitoring, analysis and reporting - to inform policy and practice of stakeholders contributing to the achievement of the SDG 16 and SDG 5, including training on monitoring capacity (qualitative and quantitative).

6. A Capacity Development Plan to be developed, including in-house ToT capacity with key stakeholders, including a training curriculum, and strengthened training methodology.

7. Best Practices and approaches: Study on the possible replication of approaches and best practices with similar initiatives in other countries in the Pacific region based on the analysis of the data collected from REACH Phase I and II.

8. Gender strategy - Strengthen the gender approach - Continue to integrate ‘gender’ in all aspects of the project, including in REACH missions: a) Advance, b) During, c) Post - Strengthen the methodology to assess qualitative impact, - Further strengthen the gender capacity of REACH team and stakeholders, - Training/awareness at village level, separate for women and men.

9. Rights-based approach - Include qualitative indicators on the human rights based approach, including a) on service providers/receivers, and b) addressing sensitivities (cultural, gender, other). - reach out to vulnerable groups not yet reached, and ‘vulnerabilities within vulnerabilities’ - explore additional ways to ‘give a voice’ to the people “reached by REACH” 

10. Develop a communication plan, explore effective communication tools that strengthen the project outreach to target groups and wider audience. Also strengthen the visual documentation of the project (photography). 

1. Recommendation:

Draw a realistic plan for the use of the mobile service delivery units (buses); follow up technical issues.

Management Response: [Added: 2018/08/13] [Last Updated: 2020/12/05]

Agreed. The project is working with the Ministry of Women, Children and Poverty Alleviation to learn from the first year of mobile units, and improve delivery through the same. A realistic plan for the mobile units will be developed together

Key Actions:

2. Recommendation:

Arrange a longer-term mission planning; site selection to be coordinated wherever possible, and Plan B for risk factors.

Management Response: [Added: 2018/08/13] [Last Updated: 2020/12/05]

Agreed. The project has developed a mission plan for the remainder of the project, with contingency measures included in case of force majeure

Key Actions:

3. Recommendation:

Address implementation challenges identified in this MTE (see Table 2)

Management Response: [Added: 2018/08/13] [Last Updated: 2020/12/05]

Agreed. All challenges and recommendations have been presented to partners in the Project Board meeting, and a special capacity development workshop was held with 35 government partners (implementers) to discuss remedial actions for the challenges raised. The project will work with partners to incorporate these recommendations as lessons into the new phase of the project

Key Actions:

4. Recommendation:

Clarify the issue of allowances and solve any misunderstandings related to allowances, the sooner the better

Management Response: [Added: 2018/08/13] [Last Updated: 2020/12/05]

The project has discussed the matter of allowances with government partners, and agreed to maintain the use of government rate for meal and subsistence, rather than UNDP DSA rates.

Key Actions:

5. Recommendation:

Develop a robust timely and appropriate Management response to address the reported challenge related to the ‘inability to cope with the increased demand’, in line with the modalities identified in the Risk Log. (See paragraph 4.4.8)

Management Response: [Added: 2018/08/13] [Last Updated: 2020/12/05]

Agreed. The project has consulted with government partners concerning the continued and timely availability of services that are offered through the REACH project. In order to ensure timely provision of services, the government requires the ability to track cases from application to completion. The project is therefore supporting a Start 2 Finish (S2F) Service Tracker to provide government partners with this information.

Key Actions:

6. Recommendation:

Include qualitative indicators in BTOR format.

Management Response: [Added: 2018/08/13] [Last Updated: 2020/12/05]

Agreed. Qualitative questions have been trialed with partners, and a session was held on how to record qualitative data in BTORs. There has been some challenge in securing full understanding by government partners. The project will continue to work on this to achieve this recommendation.

Key Actions:

7. Recommendation:

Include gender reporting in BTOR format.

Management Response: [Added: 2018/08/13] [Last Updated: 2020/12/05]

Agreed. At the REACH Capacity Building Workshop, the REACH project introduced the concept of gender reporting in the BTOR to government partners. Gender inclusive report writing will be fully trialed in the next REACH awareness raising and service delivery sessions.

Key Actions:

8. Recommendation:

Undertake a meta-analysis of BTOR reports on lessons learned, and integrate these in the design of REACH Phase II.

Management Response: [Added: 2018/08/13] [Last Updated: 2020/12/05]

Agreed. This process has already started, and the result of recommendations from MTE and consultations with partners will be fully incorporated in phase II of the project.

Key Actions:

9. Recommendation:

Identify whether there is a need for technical support to MWCPA in 2018 in view of current priorities (ref. 1.1.4, 1.1.5, 1.3.1, 1.3.2); implement in line with needs assessment, if needed, with support of UNDP

Management Response: [Added: 2018/08/13] [Last Updated: 2020/12/05]

Agreed. The project is engaging the Ministry of Women, Children and Poverty Alleviation to develop a capacity development plan to support financial management and sustainability (1.1.4); mobile service delivery (1.1.5); and peacebuilding, social cohesion and democratic governance (1.3.1,2).

Key Actions:

10. Recommendation:

Hold a strategic lessons learned workshop at the end of 2018, possibly as part of Final Evaluation, in 4 districts.

Management Response: [Added: 2018/08/13] [Last Updated: 2020/12/05]

Agreed. The project has started preliminary planning for a REACH Lesson Learned workshop that will coincide with the REACH final evaluation at the end of the year.

Key Actions:

11. Recommendation:

Deepen the gender approach, increase awareness, increase the quality of service delivery, empowerment, client orientation of service deliverers.

Management Response: [Added: 2018/08/13] [Last Updated: 2020/12/05]

Agreed. The project has started to build the gender approach into REACH missions, starting with a training of trainers at the REACH capacity building workshop to improve to awareness and service delivery for women and people with disabilities. The project will continue to hold focused sessions on these areas.

Key Actions:

12. Recommendation:

Continue to systematically integrate ‘gender’ in all aspects of the project.

Management Response: [Added: 2018/08/13] [Last Updated: 2020/12/05]

Agreed. To be applied during the June to Dec REACH missions after the REACH capacity building workshop.

Key Actions:

13. Recommendation:

Strengthen the methodology to assess qualitative impact.

Management Response: [Added: 2018/08/13] [Last Updated: 2020/12/05]

Agreed. To be developed later taking into consideration the timeframe to under a quality qualitative survey at community level.

Key Actions:

14. Recommendation:

Strengthen the gender capacity of the stakeholders.

Management Response: [Added: 2018/08/13] [Last Updated: 2020/12/05]

Agreed. Undertaken in the REACH capacity building workshop on May 2018.

Key Actions:

15. Recommendation:

Develop a gender action plan for 2018:

1. Training on how to focus on gender equality, a) UNDP with key stakeholders, b) at village level: separate trainings for women (including at events e.g. Women’s Expo), and separate training for men.

2. Make ‘gender’ part of all aspects of REACH missions: a) Advance, b) During, c) Post.

3. Make ‘gender’ a mandatory section in the BTOR report; 2 qualitative questions on challenges observed in addressed ‘gender’ in this REACH mission, and possible solutions.

4. In August 2018, a meta-analysis of BTOR ond Gender in REACH.

Management Response: [Added: 2018/08/13] [Last Updated: 2020/12/05]

Agreed. The project will work with Legal Aid Commission and Ministry of Women, Children and Poverty Alleviation to implement:

  1. As per recommendation 11, the project has started with trainings on gender, and will roll these out to the village level, including separate engagements for men and women.
  2. This will be incorporate immediately in planned missions in 2018, and incorporated into the planning for phase II.
  3. As per recommendation 7, this will be actioned.

As per recommendation 8, this will be actioned.

Key Actions:

16. Recommendation:

Implement the pilot and develop recommendations to be included in design REACH Phase II.

Management Response: [Added: 2018/08/13] [Last Updated: 2020/12/05]

Agreed. The project will kick-off Start 2 Finish (S2F) Service Tracker initiative in June 2018, and test the tracking system in eight locations.

Key Actions:

17. Recommendation:

Develop key indicators, MWCPA and LAC with UNDP (See Ch. 4.2): include composite; qualitative and quantitative; provider and recipient indicators.

Management Response: [Added: 2018/08/13] [Last Updated: 2020/12/05]

Agreed. The project will work closely with project partners to develop indicators that are realistic and easy to collect at community level.

Key Actions:

18. Recommendation:

Identify lessons learned on client-centred service delivery.

Management Response: [Added: 2018/08/13] [Last Updated: 2020/12/05]

Agreed. To be developed along with the S2F initiative in June.

Key Actions:

19. Recommendation:

6. Recommendations

6.1 Follow-up 2018 EQ: Are there any follow-up actions to be taken or any necessary adjustments, including if indicated the reorientation of the Project?

RECOMMENDATIONS for 2018, to UNDP and all stakeholders, unless mentioned otherwise: 

1. Field mission planning and implementation: - To all stakeholders, in particular MWCPA: Draw a realistic plan for the use of the mobile service delivery units (buses); follow up technical issues - Arrange a longer-term mission planning; site selection to be coordinated wherever possible, and Plan B for risk factors. - Address implementation challenges identified in this MTE (see Table 2) - To UNDP, MWCPA, LAC: Clarify the issue of allowances and solve any misunderstandings related to allowances, the sooner the better.

2. To the management of the MWCPA and LAC: Develop a robust timely and appropriate Management response to address the reported challenge related to the ‘inability to cope with the increased demand’, in line with the modalities identified in the Risk Log. (See paragraph 4.4.8) 

3. Field mission reports, Monitoring, Research for evidence based policy making (Output 2.1.3). - Include qualitative indicators in BTOR format. - Include gender reporting in BTOR format. See paragraph on gender.- Undertake a meta-analysis of BTOR reports on lessons learned, and integrate these in the design of REACH Phase II.

4. Capacity Development for stakeholders: - To MWCPA: Identify whether there is a need for Technical Support to MWCPA in 2018 in view of current priorities (ref. 1.1.4, 1.1.5, 1.3.1, 1.3.2); implement in line with needs assessment, if needed with support of UNDP, - Gender trainings: see below - To UNDP and project stakeholders: Strategic Lessons Learned workshops, end 2018, possibly as part of Final Evaluation, 4 districts. Expected Outcome: inputs for design REACH Phase II

5. Gender: - Deepen the gender approach, increase in awareness, increase and quality of service delivery, empowerment; client orientation of service deliverers. - Continue to systematically integrate ‘gender’ in all aspects of the project - Strengthen the methodology to assess qualitative impact. - Strengthen the gender capacity of the stakeholders. - Address the challenges (Table 2 chapter 4.2) - Gender action plan for 2018:

1. Training on how to focus on gender equality, a) UNDP with key stakeholders, b) at village level: separate trainings for women (including at events e.g. Women’s Expo), and separate training for men, 2. Make ‘gender’ part of all aspects of REACH missions: a) Advance, b) During, c) Post

3. Make ‘gender’ a mandatory section in the BTOR report;

2 qualitative questions on challenges observed in addressed ‘gender’ in this REACH mission, and possible solutions.

4. In August 2018, a meta-analysis of BTOR on Gender in REACH. Expected outcome: Best practices, to be incorporated in proposal REACH II.

6. Innovation Pilot S2F Delivery Tracker: implement the pilot and develop recommendations to be included in design REACH Phase II

7. Outcome Indicators: Develop key indicators, MWCPA and LAC with UNDP (See Ch. 4.2): - ‘owned’ by MWCPA, LAC (what is a key indicator for increased capacity expected?) - include composite indicators; qualitative and quantitative; provider and recipient indicator

8. Input in design Phase II: Assess: which groups are not yet reached and why?

9. Partnerships and stakeholders: - As part of the design of a possible REACH Phase II, revisit partners’ expectations, challenges, and strengths. - Identify lessons learned on client centered service delivery.

Management Response: [Added: 2020/12/01] [Last Updated: 2020/12/05]

Key Actions:

20. Recommendation:

6.2 Key components for a follow-on REACH Phase II EQ:

What are the key potential components, necessity and expected outputs for a follow on Project?

Expected output A “REACH Phase II” is needed to:

- consolidate the achievements of REACH I, - deepen the qualitative approach, - expand the project’s coverage (number of districts, villages), including in maritime areas, - deepen the project’s coverage in terms of people receiving services, focus on “reaching the unreached first”, reach the “most vulnerable amongst the vulnerable”, - strengthen monitoring and accountability mechanisms, using innovative approaches

Management Response: [Added: 2020/12/01] [Last Updated: 2020/12/05]

Key Actions:

21. Recommendation:

RECOMMENDATIONS for REACH PHASE II

To all project stakeholders (UNDP, MWCPA, LAC, other) unless mentioned otherwise:

1. Project design: 8 project components essentially constitute the project’s effectiveness: Reaching the unreached first, Integrated approach, Partnership strategy, Gender focus, Evidence based approach, Commitment, Innovation and Communication strategy. These should all be prominent components in Phase II.

2. Mobile service delivery units: REACH II to expand the project’s coverage, reaching out to communities that have not yet received the services, with continued focus on “reaching the unreached first” and also reach the “vulnerable amongst the vulnerable”; and revisiting communities visited in Phase I for follow up. Reaching remote communities may require acquiring additional vehicles or boat transportation.

3. Partnership: Partnership with the Ministry of Women, Children and Poverty Alleviation, and Legal Aid Commission to be strengthened, potentially to formally include the Human Rights and AntiDiscrimination Commission, and additional service providers as the need arises and as feasible.

4. S2F Delivery Tracker system: to be implemented based on the results of the 2018 Pilot, to ensure that services are delivered and completed. 

5. Institutional strengthening of the evidence based approach - data collection, monitoring, analysis and reporting - to inform policy and practice of stakeholders contributing to the achievement of the SDG 16 and SDG 5, including training on monitoring capacity (qualitative and quantitative).

6. A Capacity Development Plan to be developed, including in-house ToT capacity with key stakeholders, including a training curriculum, and strengthened training methodology.

7. Best Practices and approaches: Study on the possible replication of approaches and best practices with similar initiatives in other countries in the Pacific region based on the analysis of the data collected from REACH Phase I and II.

8. Gender strategy - Strengthen the gender approach - Continue to integrate ‘gender’ in all aspects of the project, including in REACH missions: a) Advance, b) During, c) Post - Strengthen the methodology to assess qualitative impact, - Further strengthen the gender capacity of REACH team and stakeholders, - Training/awareness at village level, separate for women and men.

9. Rights-based approach - Include qualitative indicators on the human rights based approach, including a) on service providers/receivers, and b) addressing sensitivities (cultural, gender, other). - reach out to vulnerable groups not yet reached, and ‘vulnerabilities within vulnerabilities’ - explore additional ways to ‘give a voice’ to the people “reached by REACH” 

10. Develop a communication plan, explore effective communication tools that strengthen the project outreach to target groups and wider audience. Also strengthen the visual documentation of the project (photography). 

Management Response: [Added: 2020/12/01] [Last Updated: 2020/12/05]

Key Actions:

Latest Evaluations

Contact us

1 UN Plaza
DC1-20th Floor
New York, NY 10017
Tel. +1 646 781 4200
Fax. +1 646 781 4213
erc.support@undp.org