Mid-term evaluation of the Vietnam Provincial Governance and Public Administration Performance Index (PAPI) – 00095984

Report Cover Image
Evaluation Plan:
2017-2021, Viet Nam
Evaluation Type:
Mid Term Project
Planned End Date:
12/2019
Completion Date:
01/2020
Status:
Completed
Management Response:
Yes
Evaluation Budget(US $):
20,000

Share

Document Type Language Size Status Downloads
Download document MTR_Terms of Reference.pdf tor English 220.80 KB Posted 53
Download document PAPI_MTR_REPORT_FINAL_ENG_approved.pdf report English 800.49 KB Posted 49
Title Mid-term evaluation of the Vietnam Provincial Governance and Public Administration Performance Index (PAPI) – 00095984
Atlas Project Number: 00095984
Evaluation Plan: 2017-2021, Viet Nam
Evaluation Type: Mid Term Project
Status: Completed
Completion Date: 01/2020
Planned End Date: 12/2019
Management Response: Yes
UNDP Signature Solution:
  • 1. Poverty
Corporate Outcome and Output (UNDP Strategic Plan 2018-2021)
  • 1. Output 1.1.1 Capacities developed across the whole of government to integrate the 2030 Agenda, the Paris Agreement and other international agreements in development plans and budgets, and to analyse progress towards the SDGs, using innovative and data-driven solutions
SDG Goal
  • Goal 1. End poverty in all its forms everywhere
  • Goal 17. Strengthen the means of implementation and revitalize the Global Partnership for Sustainable Development
SDG Target
  • 1.a Ensure significant mobilization of resources from a variety of sources, including through enhanced development cooperation, in order to provide adequate and predictable means for developing countries, in particular least developed countries, to implement programmes and policies to end poverty in all its dimensions
  • 17.16 Enhance the Global Partnership for Sustainable Development, complemented by multi-stakeholder partnerships that mobilize and share knowledge, expertise, technology and financial resources, to support the achievement of the Sustainable Development Goals in all countries, in particular developing countries
Evaluation Budget(US $): 20,000
Source of Funding:
Evaluation Expenditure(US $): 15,650
Joint Programme: No
Joint Evaluation: No
Evaluation Team members:
Name Title Email Nationality
Douglas Snyder International Consultant
Vu Thuy Huong National Consultant
GEF Evaluation: No
Key Stakeholders: Representatives of Vietnam Fatherland Front, implementing partner (CECODES), PAPI Advisory Boards members, central and local governments, independent experts, donors.
Countries: VIET NAM
Lessons
Findings
Recommendations
1

Evaluation recommendation 1.  Several recommendations exist for structural simplification and depth of information collected, including expanding the sample size and deepening the question layers in the sub-dimensions of Participation, Public Services, and Accountability

2

Evaluation recommendation 2. It is recommended that the Vietnam Fatherland Front (VFF) takes a more formal stand with PAPI at the central level. This could be in the form of written correspondence acknowledging the involvement of the Centre for Research and Training of the Vietnam Fatherland Front (VFF-CRT) in the PAPI process. Also, the VFF is expected to include more PAPI findings in its regular reports to the National Assembly.

3

Evaluation recommendation 3.  Support should be given to the Ho Chi Minh National Academy of Politics (HCMA) to achieve its aim to create formalized curricula on PAPI to use in its training courses, considering the deep and long-term implications of such an effort.

4

Evaluation recommendation 4. Provincial level action research and diagnostic workshops should continue

5

Evaluation recommendation 5.  At the central level, relationships should be strengthened with National Assembly (NA) and information on PAPI shared with the central agencies via a small formal workshop

6

Evaluation recommendation 6.  Members of the Office of the Government (OOG) and the Ideological Commission of the party should be invited to join the PAPI advisory board.

7

Evaluation recommendation 7. Inter-provincial sharing of learnings and best practices on policy and action plan development and implementation should be bolstered to help continue the accelerating activity among provinces.

8

Evaluation recommendation 8.  UNDP should set up an M&E system to try and directly capture the degree of implementation of policy responses to PAPI

9

Evaluation recommendation 9. PAPI’s key partners can now consider how the project can sensitively work towards greater impact in terms of policy advocacy at the central government level, while maintaining the integrity of PAPI’s research.

10

Evaluation recommendation 10. To fully represent the voice of the people and to recognize the connection of policy development from the central to provincial levels, inclusion of recently emerged thematic issues in the existing survey framework is recommended

11

Evaluation recommendation 11. For long term achievement of outcomes and to maintain the independence and reliability of PAPI’s outcomes, international funding partners will need to remain engaged until more innovative funding mechanisms may be considered.

Management Response Documents
1. Recommendation:

Evaluation recommendation 1.  Several recommendations exist for structural simplification and depth of information collected, including expanding the sample size and deepening the question layers in the sub-dimensions of Participation, Public Services, and Accountability

Management Response: [Added: 2020/01/08] [Last Updated: 2020/01/13]

We note that the bigger the sample size is, the less data variation we will have. We hope that in the next phase of PAPI, if funding allows, we will be able to expand the sample size and revamp the methodology to survey in more communes.

The PAPI questionnaire is updated annually to ensure that information collected and measured stays relevant. For indicators used directly for PAPI calibration, revisions are carefully made after several years to allow index stability. The 2018 PAPI introducing two new governance dimensions and replaced some indicators.

We note the importance of justifying quantifiable indicators by raising WHY questions more often. Given the time limit for each survey and given the fact that not all possible explanations can be anticipated to lay out for respondents to choose, we also encourage deeper qualitative studies by think-tanks and research institutions using PAPI data, while we do our best to capture more of the WHY.

Key Actions:

Key Action Responsible DueDate Status Comments Documents
1.1 Revise 2020 PAPI survey questionnaire
[Added: 2020/01/08]
Governance and Participation Team (GPT), UNDP 2020/07 Initiated
2. Recommendation:

Evaluation recommendation 2. It is recommended that the Vietnam Fatherland Front (VFF) takes a more formal stand with PAPI at the central level. This could be in the form of written correspondence acknowledging the involvement of the Centre for Research and Training of the Vietnam Fatherland Front (VFF-CRT) in the PAPI process. Also, the VFF is expected to include more PAPI findings in its regular reports to the National Assembly.

Management Response: [Added: 2020/01/13]

We are aware of the benefit of engaging further with VFF at the central level to deepen the policy advocacy of PAPI at central level. We will work further with the Consortium of CECODES and the VFF-CRT to engage the VFF and VFF-CRT more deeply as recommended.

It is encouraging that, during the briefing of the mid-term review team, the VFF-CRT representative noted the recommendation, and said he would report to VFF leaders about the suggestion that VFF could utilize PAPI data formally to report to relevant agencies, as a way to strengthen their oversight roles.

Key Actions:

Key Action Responsible DueDate Status Comments Documents
Engage senior leaders of VFF in PAPI Advisory Board
[Added: 2020/01/13]
Governance and Participation Team (GPT), UNDP Viet Nam 2019/12 Completed Mr. Nguyen Huu Dung, the Standing Vice President of VFF attended the 2019 PAPI Advisory Board. See attached meeting minutes. History
3. Recommendation:

Evaluation recommendation 3.  Support should be given to the Ho Chi Minh National Academy of Politics (HCMA) to achieve its aim to create formalized curricula on PAPI to use in its training courses, considering the deep and long-term implications of such an effort.

Management Response: [Added: 2020/01/13]

We will work further with the HCMA to advocate for the inclusion of PAPI within the Academy’s executive training courses, as we have been engaging with them over the past eight years. 

We note that curriculum development requires financial resources. We will work with the HCMA to explore potential funding resources (as we did work with the Embassy of Belgium earlier). 

Key Actions:

Key Action Responsible DueDate Status Comments Documents
Integrate PAPI in Executive training course of HCMA
[Added: 2020/01/13]
Governance and Participation Team (GPT), UNDP Viet Nam 2019/12 Completed PAPI concept and findings were integrated in session on “facilitating government” by Deputy Prime Minister Vuong Dinh Hue on 29 August 2019 at HCMA’s executive training course for executive government officials
4. Recommendation:

Evaluation recommendation 4. Provincial level action research and diagnostic workshops should continue

Management Response: [Added: 2020/01/13]

We will work further with relevant national partners to conduct action research to identify what works, what does not work and why in provinces’ efforts to improve their performance in governance, public administration and public service delivery.

We appreciate the suggestion that we should reengage with the Training Centre for Elected Representatives that we once were engaged with to guide People’s Councils on how to use PAPI data for their queries at provincial, district and commune meetings. If additional funding were available, elected representatives will be another important target for deeper diagnostics of PAPI findings so that they can use the data to substantiate their query sessions with local governments in decision-making.

Key Actions:

Key Action Responsible DueDate Status Comments Documents
Conduct at least 10 Provincial PAPI diagnostic workshops after the launching of PAPI 2019 report
[Added: 2020/01/13]
Governance and Participation Team (GPT), UNDP Viet Nam 2020/12 Initiated
5. Recommendation:

Evaluation recommendation 5.  At the central level, relationships should be strengthened with National Assembly (NA) and information on PAPI shared with the central agencies via a small formal workshop

Management Response: [Added: 2020/01/13]

We note the importance of these thematic discussions with high-level policymakers at the central level, and PAPI team members do attend such thematic policy dialogues (like discussions on SEDS, SDGs, E-Government, Vietnam Development Forums, the National Assembly thematic discussions for inputs in draft laws like the Law on Public administration, etc.).

Subject to additional funding, more high-level policy dialogues at the central level could be convened on critical issues of great interest for policymakers. This can be an additional output in the next phase of the PAPI project (after 2021).

Key Actions:

Key Action Responsible DueDate Status Comments Documents
5.1 Provide PAPI publications to National Assembly Delegates prior to its plenary session in May 2020
[Added: 2020/01/13]
Governance and Participation Team (GPT), UNDP Viet Nam 2020/05 Initiated
5.2 Continue to engage members of the National Assembly in PAPI Advisory Board meeting
[Added: 2020/01/13]
Governance and Participation Team (GPT), UNDP Viet Nam 2020/12 Completed Ms. Tran Thi Quoc Khanh, Standing Member of NA’s Committee for Science, Technology & Environment attended the 2019 PAPI Advisory Board meeting. See attached document History
6. Recommendation:

Evaluation recommendation 6.  Members of the Office of the Government (OOG) and the Ideological Commission of the party should be invited to join the PAPI advisory board.

Management Response: [Added: 2020/01/13]

We will work further with the Consortium of CECODES-VFF/CRT to engage with more representatives from the Party’s commissions, Government agencies, the NA, the VFF in the PAPI Advisory Board.  

Key Actions:

Key Action Responsible DueDate Status Comments Documents
6.1 Member of OOG to join PAPI Advisory Board
[Added: 2020/01/13]
Governance and Participation Team (GPT), UNDP Viet Nam 2019/12 Completed Mr. Ngo Hai Phan, Director General of Administrative Procedures Control Agency of OOG is officially assigned to join PAPI Board. Two of his staff attended the 2019 PAPI Advisory Board meeting. Please see attached meeting minutes.
6.2 Member of the Central Theory Council (CTC) to attend the PAPI Advisory Board meeting
[Added: 2020/01/13]
Governance and Participation Team (GPT), UNDP Viet Nam 2019/12 Completed Mr. Dinh Xuan Thao, member of CTC, former member of the NA Steering Committee attended the 2019 PAPI Advisory Board meeting
7. Recommendation:

Evaluation recommendation 7. Inter-provincial sharing of learnings and best practices on policy and action plan development and implementation should be bolstered to help continue the accelerating activity among provinces.

Management Response: [Added: 2020/01/13]

We will work further with provincial governments to foster sharing of good practices and accelerate innovative ideas between provinces through regional workshops when/if the project is further resourced.

We will also advise provincial leaders to organise such sharing and learning events from their own budgets as some provinces (like Da Nang, Quang Ninh and Binh Dinh) have already done.

Key Actions:

Key Action Responsible DueDate Status Comments Documents
Foster sharing of best practices through at least 10 PAPI 2019 diagnostic workshops
[Added: 2020/01/13]
Governance and Participation Team (GPT), UNDP Viet Nam 2020/12 Initiated
8. Recommendation:

Evaluation recommendation 8.  UNDP should set up an M&E system to try and directly capture the degree of implementation of policy responses to PAPI

Management Response: [Added: 2020/01/13]

We have collected information about provincial responses and document these in annual PAPI reports for a two-fold purpose: (1) to push non-responding provinces to take actions; and (2) to give credit for provinces with proactiveness in responding to their citizens feedback from PAPI. This has been at the results level rather than at the impact level. The current M&E system to document provincial responses in annual PAPI reports and on PAPI website at http://papi.org.vn/quyet-sach-cua-chinh-quyen-dia-phuong-ve-papi will be continued.

We will attempt to collect evidence of actual measures undertaken by provinces in some governance areas. During PAPI diagnostic workshops, provinces often report on current or future measures to help improve some of the PAPI indicators and scores. The PAPI team could check on these measures. That will also help capturing some of the good practices put in place by local governments.

We will also encourage provincial leaders to provide a link on their provincial website under the theme Provincial Governance Indices to post all diagnostic workshop materials, provincial follow-up action plans/activities and direct to websites of PAPI, PCI, PAR and SIPAP for collective learning and monitoring.

With more financial resources, as suggested in the MTR report, a more systematic M&E framework to monitor impacts of provincial action plans can be feasible, while we note that to collect information about impact required deeper analysis of contexts, attributes and thorough independent research at national and provincial levels. We understand that the M&E system will help UNDP and its national partners gather good practices to share with poor performing provinces.

Key Actions:

Key Action Responsible DueDate Status Comments Documents
Explore with HCMA to stock-taking their previously conducted PAPI action researches in 33 provinces to produce a policy brief on best practices in provincial governance and degree of provincial implementation
[Added: 2020/01/13]
Governance and Participation Team (GPT), UNDP Viet Nam 2020/12 Initiated
9. Recommendation:

Evaluation recommendation 9. PAPI’s key partners can now consider how the project can sensitively work towards greater impact in terms of policy advocacy at the central government level, while maintaining the integrity of PAPI’s research.

Management Response: [Added: 2020/01/13]

We will proactively invite co-funding partners to attend and speak at PAPI workshops that are funded by the PAPI project (like the national PAPI report launches and provincial action-research activities).

We will also share information about diagnostic workshops convened and funded by provinces to keep them informed and let them decide to partake in.  

As mentioned above, UNDP will work with CECODES to further engage VFF in policy advocacy using PAPI findings and data. Our future efforts to engage with central players and other relevant organizations who promote good governance will strengthen policy advocacy impacts.

Key Actions:

Key Action Responsible DueDate Status Comments Documents
Meet with donors to agree on PAPI team’s frequent sharing of diagnostic workshop agendas for funding partners to attend and speak at PAPI workshops in 2020
[Added: 2020/01/13]
Governance and Participation Team (GPT), UNDP Viet Nam 2020/12 Completed Australian Embassy attended PAPI workshop in Hanoi in July 2019 and in Binh Duong in August 2019.
10. Recommendation:

Evaluation recommendation 10. To fully represent the voice of the people and to recognize the connection of policy development from the central to provincial levels, inclusion of recently emerged thematic issues in the existing survey framework is recommended

Management Response: [Added: 2020/01/13]

Ø  We note that most of the informants wish us to address topical issues of concern for  Government officials and the National Assembly delegates (e.g. PPP/BOT/BT infrastructure projects and socialization/privatization of education and healthcare services). The PAPI team has tried to explore possibilities to measure BOT/BT issues in the 2019 PAPI questionnaire but hasn’t collected enough supporting materials to see the scope and scale of those projects by province.

Ø  We wish to clarify that PAPI Questionnaire is composed of two main components: questions that make PAPI indicators and the index, and questions that measure hot issues/top concerns among policymakers and the public and helps anticipate policy interventions in draft laws and regulations. We are mindful of expectations, while for sure we need to venture into more topical/sensitive issues that the Government of Vietnam and National Assembly expect to hear citizen voices (indeed we should feel inspired thanks to additional requests from state officials as such, as it proves that PAPI is a ‘forum’ for policy discussions).

Key Actions:

Key Action Responsible DueDate Status Comments Documents
10.1 Pilot Geographic Information System (GIS) in sampling of PAPI 2020 survey to improve sample representation
[Added: 2020/01/13]
Governance and Participation Team (GPT), UNDP Viet Nam 2020/11 Initiated
10.2 Provide PAPI policy paper to the Central Theory Council (CTC) for the drafting of Vietnam 2021-2030 Socio-Economic Development Strategy
[Added: 2020/01/13]
Governance and Participation Team (GPT), UNDP Viet Nam 2019/11 Completed Ms. Caitlin Wiesen, UNDP Resident Representative presented PAPI findings at the consultation workshop of CTC on 28 June 2019
11. Recommendation:

Evaluation recommendation 11. For long term achievement of outcomes and to maintain the independence and reliability of PAPI’s outcomes, international funding partners will need to remain engaged until more innovative funding mechanisms may be considered.

Management Response: [Added: 2020/01/13]

We will work closely with our current donors, including the Australian DFAT and Irish Aid to explore potential extension of their funding for PAPI towards 2026 to cover PAPI for at least another government term. We note the importance of an independent/external monitoring tool to monitor how Viet Nam progresses towards a more citizen-centric, growth facilitating governance and public administration system

Key Actions:

Key Action Responsible DueDate Status Comments Documents
Submit request for further funding to potential donors for 2021-2025 period
[Added: 2020/01/13]
Governance and Participation Team (GPT), UNDP Viet Nam 2019/12 Completed Request was submitted and discussed with DFAT

Latest Evaluations

Contact us

1 UN Plaza
DC1-20th Floor
New York, NY 10017
Tel. +1 646 781 4200
Fax. +1 646 781 4213
erc.support@undp.org