Evaluation Detail

Final Evaluation RP 2018-2021

Evaluation Plan:: 2018-2021, RBLAC
Evaluation Type:: Outcome
Planned End Date:: 04/2021
Completion Date:: 06/2021
Status:: Completed
Management Response:: No
Evaluation Budget(US $):: 50,000

Document	Type	Language	Size	Status	Downloads
ToRs Evaluation RPD 2018-2021_FV.docx	tor	English	81.87 KB	Posted	702
Final_Report CEES_28Jun21.pdf	report	English	2165.30 KB	Posted	800

Title

Final Evaluation RP 2018-2021

Atlas Project Number:

Evaluation Plan:

2018-2021, RBLAC

Evaluation Type:

Outcome

Status:

Completed

Completion Date:

06/2021

Planned End Date:

04/2021

Management Response:

Yes

Focus Area:

1. Others

Corporate Outcome and Output (UNDP Strategic Plan 2018-2021)

1. Output 1.1.1 Capacities developed across the whole of government to integrate the 2030 Agenda, the Paris Agreement and other international agreements in development plans and budgets, and to analyse progress towards the SDGs, using innovative and data-driven solutions

Evaluation Budget(US $):

50,000

Source of Funding:

UNDP sources to be determined

Evaluation Expenditure(US $):

45,000

Joint Programme:

Joint Evaluation:

Evaluation Team members:

Name	Title	Nationality
Miguel Szekely	Evaluation Team Leader	MEXICO

GEF Evaluation:

Key Stakeholders:

Donors, Country Offices, UN Agencies, National Governments, Civil Society, Academia, IGOs

Lessons
1.	The main lessons learned identified are the following: In parallel with the alignment to the Strategic Plan, it is important that the Theory of Change has intermediate Outcomes that guide the execution of the RP. For this, it is necessary that the programs and activities have identified contributions to this level of Outcomes. If the contribution of the RP to the Outcome indicators cannot be measured, it is not possible to understand the results achieved by the RP. With the pandemic, it was demonstrated that remote work is not only feasible, but even more productive. Now it is clear that professionals can offer services without having to migrate. Now human resources can connect with other countries to work there without having to move. The Country Offices that make the most of the resources and tools available are those that know the types of RP support. The RP operation seems to be supply-driven, especially on topics where specific tools are already designed and working. However, this is not necessarily a negative issue. It is important to have a clear understanding of the RP governance, and to identify the tasks of each cluster or team towards unified objectives. In addition, solutions with an integral vision of development could be interesting for donors.
Findings
1.	In general, we find the Regional Programme 2018-2021 (RP) allowed the articulation of projects, regional interventions, activities, and service plans, as well as joint efforts of COs around three outcomes that remain relevant for the region. This, enhanced by the new lines of action of the directorate (Productivity, Inclusion, and Resilience), promoted the reestablishment of the strategic position of UNDP in the region. However, the scope and potential of the RP can be further strengthened in terms of regional development and integrative solutions. This evaluation exercise had to consider the COVID-19 pandemic context, because some actions and programs related to the RP had to be adapted in 2020. In this regard, the RP was flexible enough in adapting and responding to country offices’ needs, especially with technical support and readjustments in the programs’ operation. As mentioned before, there is a strong perception that UNDP is reinstalling its position in the region over the last two years. The majority of external stakeholders agree about UNDP promoting the generation of data and evidence-based policies; and the technical capacity related to the Regional Programme activities is also highlighted. Most stakeholders reported a very good coordination with their UNDP counterpart (Regional Hub or projects personnel); but also, internal actors report moments of disconnection between NY Office and the Regional Hub. In terms of the RP rationale, the Theory of Change is clearly defined, identifying all the required elements (development challenges, problem pathways, solution pathways, RP Outputs, Desired change, Strategic Plan Outcomes, and related SDGs). Nevertheless, the vertical logic needs to be strengthened because there is a missing level between Outputs and Outcomes. Institutionally, the RP must be aligned to the Strategic Plan’s Outcomes, but the Outputs that are the RP responsibility do not necessarily contribute to those Outcomes; thus, an intermediate level of expected results is needed. In addition, current indicators do not allow measuring the real achievement in terms of results. However, this document highlights relevant actions and results that are in line with the outcomes and SDG achievement. Also, 71% of Output measurements reached their target, and there is a recognition of strong knowledge products and public goods that contribute to public policy in the region. A very high percentage of stakeholders (91.7%) consider that the knowledge and innovations generated informed policy making in their countries. Among this innovation some case studies were identified to highlight issues that can be further investigated across the Regional Programmes: SIGOB, DATACTION (InfoSegura), EWS partnerships, and the Gender cluster transversal work. In terms of the knowledge products, at least those related to the evaluation sample of projects are highly recognized by stakeholders as useful and/or innovative. In terms of sustainability of the results- perceived as partially sustainable- the main challenge is the changes in governments priorities, but stakeholders agree upon initiatives must be built with the national counterpart, and preferably since the planning and design stages. Moreover, the RP has contributed to leverage existing CO portfolios in the achievement of planned development results by offering complementary resources and tools, and by highly valued technical assistance. However, the alignment between RP’s actions and national plans is not completely clear; thus, with the information available we cannot confirm that the RP create synergies more than complementarities. Derived from the pandemic, for the next RP there are two stages to be considered: (1) COVID Shock recovery, and (2) Growth in Post-COVID era. In this context, the support that UNDP must bring to countries and the LAC region is knowledge, research, technical assistance, and interventions with focus on productivity.

Recommendations
1	Recommendation 1: Alignment between the Outcomes and the directorate narrative for the LAC region: Productivity, Inclusion, and Resilience as Outcomes to be achieved by comprehensive activities, tools and solutions related to solid content, strategic partnerships and communications, and impeccable administration.
2	Recommendation 2: Use indicators as incentives: Define indicators in terms of what the RP wants to promote (some missing points in the last RP were: south-south cooperation, partnerships, capacity building, production of data). Indicators must enable to tell the story of results achieved.
3	Recommendation 3: Include the Strategic Plan indicators in a higher level (as impact), but specific Outcomes’ indicators for the RP must be defined, in order to measure contributions and results. If the contribution of the RP to the Outcome indicators cannot be measured, it is not possible to understand the results achieved by the RP.
4	Recommendation 4: Define targets for each indicator since the planning stage, in order to be able to measure achievement. In addition, the majority of indicators must be focused on region, sub-regions or countries, rather than on specific tools.
5	Recommendation 5: Elaborate a more exhaustive target exercise, always using the information from previous exercises (historic indicators, when applicable) to ensure that targets are relevant for the region/country/project; achievable, but yet challenging, and have a baseline defined. For targets definitions it is important to consider that in monitoring and evaluation systems, exceeded targets by 120% or more are a signal for reviewing your planning. In general, targets should be achievable, but also with an objective that drives to growth.
6	Recommendation 6: Incorporate more tools and communication mechanisms for non-Spanish speaking actors.
7	Recommendation 7: In terms of growth in post-COVID era, the RP can be a mechanism to put on the table that social protection systems must consider new elements. In general, the elements of LAC's social protection policy focus on structural poverty; and the pandemic showed that they are not emergency ready. Social protection systems were not ready for providing social assistance to households facing transitory shocks and not living in extreme poverty. The COVID-19 pandemic brought this process to a halt, and abruptly challenged both, the operation, and the design of social protection efforts across LAC. On the one hand, the crisis made it evident that it was not enough to aim at covering the structurally poor as before in a context where large sectors of the population that had advanced into the middle classes, suddenly returned to the ranks of the vulnerable and of those with incomes under what is needed to cover their basic needs. The concept of social protection needs to be updated to become dynamic so that it can service the structurally poor while at the same time being able to address the needs of the transitory vulnerable that fall below the poverty line temporarily. In this context, some examples of discussion topics that the RP can promote are the following: It is now desirable that beneficiaries’ registries include potential recipients of social assistance, and also the interoperability across systems. Some relevant actions towards these goals are: In the short-term, develop electronic beneficiary registries and management information systems, that can evolve into integrated social protection data bases. Establish protocols and agreements for sharing data with other government institutions and non-governmental organizations. Invest in the interoperability of registries, not only with the social protection systems but also with other social registries such as tax data, social security systems, civil registries, etc. Collect information that allows for assessing vulnerability and exposure to shocks in the case of potential recipients.
8	Recommendation 8: With the pandemic, it was demonstrated that remote work is not only feasible, but even more productive. Now it is clear that professionals can offer services without having to migrate. Now human resources can connect with other countries to work there without having to move. In this context, the PR can function as a broker of talent and knowledge- like a virtual Hub, in addition to the physical hub- that can help different countries. Some specific required activities are the following: Research and identify needs and skills required by other countries. Development of tools to connect the labor market across countries.
9	Recommendation 9: In the guidelines to design the RP, include the definition of the types of support available and the mechanisms of dissemination to all institutional actors. It is important to socialize the usefulness of the PR with both internal and external actors in order to explain the different types of resources, activities and support available. The analysis show that COs that make the most of the resources and tools available are those that know the types of RP support.
10	Recommendation 10: Promote feedback spaces with the Country Offices on the PR (annual meetings with progress, challenges, and possible adjustments).
11	Recommendation 11: Capitalize on the advantage of a supply-driven RP: Update the products with perceived countries' demands. For this, one mechanism may be the application of an online survey on the needs and priorities of the countries. Define a supply sufficiently specific, but also comprehensive, to have adaptability in the different sub-regions/ countries.
12	Recommendation 12: Promote the Acceleration Laboratories aligned with narrative objectives. This initiative is positively recognized, but slow in the region. These labs can improve innovation and communication towards the defined vision for the region.
13	Recommendation 13: Return to the promotion of “Communities of Practice” (virtually and RP-related), which are highly valued in COs. In addition to the shared experiences themselves, these practices can illustrate the kind of support related to the RP, and the Regional Hub can learn more about the countries needs and challenges.
14	Recommendation 14: Strengthen the coordination between NY and Panama through joint-projects, definition of unified goals, and specific mechanisms for reporting and sharing advances. Coordination mechanisms are a complex challenge that, in general, can be promoted by the establishment of rules or coordination norms, or by means of incentives. In this case, the proposal is to explore the creation of joint indicators, considering the narrative and the lines of action described in Recommendation 1. The ideal is to align the incentives so that, on the one hand, the PR contributes to achieving the institutional goals of the RBLAC; and, on the other hand, that the actions of the RBLAC enrich the efforts of the PR.
15	Recommendation 15: Use the RP as an articulating entity within the different Clusters for comprehensive responses to complex problems; to have a unified vision of development.
16	Recommendation 16: In line with Recommendations 14 and 15, develop a Matrix that clearly identifies the elements of each cluster and their connection with the RP Outcomes by level of coverage (country, rub-regional or regional). Once the matrix is?? filled out, it must be reviewed to identify and resolve duplications; identify complementarities and recognize synergies.

Final Evaluation RP 2018-2021

Share

Evaluations by Unit

Latest Evaluations

Evaluation Snap Shot

Contact us