Public Education Evaluation Commission

Report Cover Image
Evaluation Plan:
2017-2021, Saudi Arabia
Evaluation Type:
Final Project
Planned End Date:
12/2019
Completion Date:
12/2019
Status:
Completed
Management Response:
Yes
Evaluation Budget(US $):
25,000

Share

Document Type Language Size Status Downloads
Download document 190925 Final TOR for the Terminal Evaluation Mission - AS.doc tor English 133.00 KB Posted 278
Download document Final_Terminal Evaluation Report_26.12.2019 (1).pdf report English 554.59 KB Posted 309
Title Public Education Evaluation Commission
Atlas Project Number: 00080865
Evaluation Plan: 2017-2021, Saudi Arabia
Evaluation Type: Final Project
Status: Completed
Completion Date: 12/2019
Planned End Date: 12/2019
Management Response: Yes
UNDP Signature Solution:
  • 1. Poverty
Corporate Outcome and Output (UNDP Strategic Plan 2018-2021)
  • 1. Output 2.1.2 Capacities developed for progressive expansion of inclusive social protection systems
SDG Goal
  • Goal 1. End poverty in all its forms everywhere
  • Goal 4. Ensure inclusive and equitable quality education and promote lifelong learning opportunities for all
SDG Target
  • 1.4 By 2030, ensure that all men and women, in particular the poor and the vulnerable, have equal rights to economic resources, as well as access to basic services, ownership and control over land and other forms of property, inheritance, natural resources, appropriate new technology and financial services, including microfinance
  • 4.1 By 2030, ensure that all girls and boys complete free, equitable and quality primary and secondary education leading to relevant and effective learning outcomes
Evaluation Budget(US $): 25,000
Source of Funding: Office Budget
Evaluation Expenditure(US $): 15,000
Joint Programme: No
Joint Evaluation: No
Evaluation Team members:
Name Title Email Nationality
Eila Heikkila Independent Evaluator eilaheikkila@hotmail.com
GEF Evaluation: No
Key Stakeholders:
Countries: SAUDI ARABIA
Lessons
1.

The terminal evaluation was conducted during a relatively short time period in December 2019. At the time of the evaluation, the Commission’s organisational restructuring and priorities’ identification was going on at the macro level, so it is obvious that staff members found it challenging to participate in the evaluation interviews. Furthermore, due to the relatively long timespan of the project  (2014 – 2019) and the many organisational and staff changes, it was not possible to find people with organisational memory to provide insights from the early phase and the evolvement of priorities of the project.

 

The Project document from 2014 was slightly vague at parts; it could be observed that the education and training evaluation system was at the early phase, but has evolved during the Project’s lifetime. The data reported and available in the project documentation was sufficient, but not informative about the various project developments, e.g. argumentation for the substantive project and budget revisions. It is a weakness that the interviews with external partners, such as the Ministry of Education, could not be included into the evaluation interviews, despite several inquiries.

 

However, the terminal evalution has been conducted with great support of UNDP and the Commission management, based on available documentation and availability of people to participate in the interviews.


Findings
1.

The Terminal Evaluation of the Project is undertaken by an external consultant at the end of the Project implementation. The aim is to provide an independent evaluation of the full Project highlighting both good progress and results achieved as well as improvement needed. The purpose of the evaluation is to perform an in-depth assessment from technical and managerial viewpoints to understand to what extent the project has successfully accomplished its objectives. The aim is to provide a picture of the Project’s achievement and challenges and to provide recommendations from an independent international perspective of possibilities for future actions.

The Project has been implemented during the time period of six years in 2014 – 2019. The original agreement was signed with the Public Education Evaluation Commission (PEEC) in 2014, complemented with the Project and Budget revisions dated in 2016, 2017 and 2018. During this time period, the Commission has undergone several organisational changes. The original UNDP agreement was signed with the Public Education Evaluation Commission (PEEC) in 2014, and the revisions with Education Evaluation Commission (EEC) in 2017, and with Education and Training Evaluation Commission (ETEC) in 2018. Several Directors have managed the Commission during the Project’s implementation reflected in new strategic priorities developed under the Project. The decision of bringing all the education and training levels under one entity of the Commission is a great achievement to support coherency and consistency of different elements and initiatives in education and training evaluation.

 


Tag: Human and Financial resources Monitoring and Evaluation Project and Programme management Education

2.

Overall, it can be argued that the Project has been relevant in supporting the capacity development of the Commission in Education and Training Evaluation. The relevance of the Project can be evidenced in the development of various elements of the education evaluation system, e.g. School evaluation, Teacher Licensing, National Testing, Curricula development, National Qualifications Framework etc. The effectiveness of the Project is evidenced with achievements in the different elements of the system and components of the Project. National reference frameworks with policies and standards have been developed for the key components of the Project. The Project activities have also supported capacity building of the education environment and community. The engagement of eduation specialist from both national and international academics and specialists have contributed to the achievements in the capacity building of the Project.


Tag: Relevance National Rule of law Education Capacity Building Institutional Strengthening

3.

However, the Project has encountered also challenges. The main challenge in the efficiency of implementation and achievement of the set objectives and outcomes is the resistance of Government’s, key implementation partners’ or education community’s approval (by-law, rule, regulation etc) of the developed frameworks under some of the key areas. The components have been consistently developed, but as a result of organisational and strategic changes, and the follow-up Project revisions, also new initiatives have been included into the Project. The focus has been rather on establishing the organisation of the Commission, but not the services provided to the beneficiaries (learners, community, society). Also, some resistance to change in education evaluation both at policy level and among the education practitioners have been reported during the evaluation. The resistance to education and training evaluation is a global phenomenon and a challence in many countries, which are establising evaluation entities and systems in education. However, the Project has contributed to supporting the education and training capacities in KSA into global education evaluation trends in collaboration with international organisations and expertise.


Tag: Effectiveness Efficiency Public administration reform Partnership Education Capacity Building

4.

At the time of the Terminal Evaluation, the Commission is undergoing identification of priorities on the macro level lasting until the beginning of 2020. The Commission has benefited greatly from the capacity development, both national and international, during the Project’s implementation. The next step is to learn from the Project’s previous achievements and challenges and build a new vision and strategy for the Education Evaluation comprising all education evaluation elements at all levels of education. The UNDP is recommended to support the Commission, based on the need for capacity building identified and expressed by the Commission, to support the sustainability of the Project’s outcomes. The following graph aims to illustrate the learning curve, which is observed in the development of the different component outputs during the full implementation time of six years of the Project.


Tag: Innovation Knowledge management Programme/Project Design Sustainability Capacity Building

Recommendations
1

It is recommended that UNDP will continue supporting the Commission with capacity-building on management and leadership in education and training evaluation. It is suggested that the support of UNDP in terms of an international consultancy would be beneficial in validating the new priorities and planning the next steps in developing education and training evaluation in KSA. The Commission new strategic priorities in each evaluation element needs to be translated into vision, strategic outcomes, strategic initiatives (projects with outputs) and action plans with timeline. Similar international evaluation entities and experts can efficiently contribute to the success of the work. This work is suggested to start based on the priorities identified by the Commission.

2

It is further recommended to provide organisational management training to operationalise the Commission’s new priorities and synergise the different departments, including evaluation of general education, vocational education and training and higher education. The recommendation is to support the Commission to cooperate with similar international evaluation entities and facilitate capacitybuilding of staff with international “twinning”, job shadowing etc. with colleagues in specific expert areas (curriculum development, school evaluation, teachers’ professionalism, NQF etc.). Capacity building at all levels of the organisation is needed, management, technical staff, service staff to steer the work towards the vision and new priorities of the Commission.

3

It is recommended to build stronger partnership cooperation and communication with the key partners (such as the Government, Board of Ministers, Ministry of Education, other Ministries, schools, other actors in evaluation in KSA, such as ADAA9 etc.). It is suggested that the Commission and its various evaluation elements agree with key implementing partners on the cooperation in a Memorandum of Understanding, which defines the area and objectives of cooperation, the roles and responsibilities, timeline and commitments to ensure the continuation of the developments. Cooperation with similar education evaluation entities internationally would further strengthen the organisational capacity to continue the work in partnership cooperation.

4

From the Project’s technical point of view, it is recommended to continue the initiative of School Evaluation to implement the developed framwork, the standard, indicators and procedures. The Government’s policy decision (Board of Ministers) is needed to secure the continuity of school evaluation in KSA. It is suggested to continue the practical work of school evaluations (foreseen to start in autumn 2020) and engage education community (schools, education professionals and other community members) to cooperate in further developing the initiative of school evaluations according to the developed framework.

5

It is recommended to support the developed K-12 Teacher’s Professional Licensing Framework to become operational. The initiative to link Teachesr’s Professional Development into the teachers’ financial compensation is a good motivator to enter or upgrade Teaching Profession. It is understood that the framework has been approved with a Bylaw to start the practical implementation among the professional community of teachers. The developed framework with possibilities for upgrading of professional levels of teachers and increading renumeration would greatly contribute to the quality of teaching and is likely to contribute to enhanching employment opportunities (also among women in early school years) in the education sector in the KSA.

6

The work of national tests is recommended to continue, as initiated by the Project. National tests are an integral part of assessing learning outcomes of the education to provide decision-makers and stakeholders a reliable measure at the national level, thus contributing to the improvement of quality of education. The results from international evaluations (such as PISA) can serve as a baseline benchmark and set the goals for the next years.

7

It is recommended to continue the work on Curriculum standards and content with the support of international expertise. There is a need to expand the developed terms of reference and standards of Curricula to all levels in general eduction. In addition, the improved quality of teaching re-orient towards 21st century skills and Learning Outcomes reflecting the skills need of the knowledge-based economy, in addition to the indicators already developed. The development of curriculum standards especially higher level (from K-12 upwards) need to reflect the skills and competences needed in the labour market and employment opportunities available in the economy of the KSA.

8

It is recommended to continue the development of National Qualifications Framework in KSA (SANQF). The NQF framwork is now developed at 10 levels. Educational Institutes are required to register their institutions as well as the qualifications/programs they are offering on a set of standards. There are two sets of standards: standards for institution, standards for the program. The evaluation system is ready for the schools and programs to register into the NQF system. The qualifications framework needs further development in formalising the structure in which learning level descriptors and qualifications are used in order to understand learning outcomes. It can be assessed that all work developed in the project; curriculum, teachers’ licensing, school evaluation and curricula development need to be aligned with the developed SANQF. For example, the curricula need to specify the learning outcomes and level in SANQF.

9

It is recommended to support the visibility of UNDP Project and the developed outcomes through different media channels and target audiences with a consistent media plan. It is suggested to continue the work of media outreach to raise awarness of the developed outcomes e.g. curriculum development, school evaluation, teacher liscensing, national qualifications framwork. It is essential to raise awareness about the importance of evaluation of education and training to quarantee the quality of the education system, teaching and learning, acquiring the learning outcomes and skills in the framework of the Vision 2030. Cooperation with other international institutions, e.g. the World Bank is suggested to ensure coordination of international support (Paris Declaration, 2005).

1. Recommendation:

It is recommended that UNDP will continue supporting the Commission with capacity-building on management and leadership in education and training evaluation. It is suggested that the support of UNDP in terms of an international consultancy would be beneficial in validating the new priorities and planning the next steps in developing education and training evaluation in KSA. The Commission new strategic priorities in each evaluation element needs to be translated into vision, strategic outcomes, strategic initiatives (projects with outputs) and action plans with timeline. Similar international evaluation entities and experts can efficiently contribute to the success of the work. This work is suggested to start based on the priorities identified by the Commission.

Management Response: [Added: 2019/12/28] [Last Updated: 2021/01/25]

UNDP is in current discussions with partners for a new PD which must include recommendations done by evaluators

Key Actions:

2. Recommendation:

It is further recommended to provide organisational management training to operationalise the Commission’s new priorities and synergise the different departments, including evaluation of general education, vocational education and training and higher education. The recommendation is to support the Commission to cooperate with similar international evaluation entities and facilitate capacitybuilding of staff with international “twinning”, job shadowing etc. with colleagues in specific expert areas (curriculum development, school evaluation, teachers’ professionalism, NQF etc.). Capacity building at all levels of the organisation is needed, management, technical staff, service staff to steer the work towards the vision and new priorities of the Commission.

Management Response: [Added: 2019/12/28] [Last Updated: 2021/01/25]

Noted, suggested recommendaiton will be taken into consideration upon drafting of new PD

Key Actions:

3. Recommendation:

It is recommended to build stronger partnership cooperation and communication with the key partners (such as the Government, Board of Ministers, Ministry of Education, other Ministries, schools, other actors in evaluation in KSA, such as ADAA9 etc.). It is suggested that the Commission and its various evaluation elements agree with key implementing partners on the cooperation in a Memorandum of Understanding, which defines the area and objectives of cooperation, the roles and responsibilities, timeline and commitments to ensure the continuation of the developments. Cooperation with similar education evaluation entities internationally would further strengthen the organisational capacity to continue the work in partnership cooperation.

Management Response: [Added: 2019/12/28] [Last Updated: 2021/01/25]

In the new PD, parnters such as  the Government, Board of Ministers, Ministry of Education, other Ministries, schools, other actors in evaluation in KSA, such as ADAA etc. will be included

Key Actions:

4. Recommendation:

From the Project’s technical point of view, it is recommended to continue the initiative of School Evaluation to implement the developed framwork, the standard, indicators and procedures. The Government’s policy decision (Board of Ministers) is needed to secure the continuity of school evaluation in KSA. It is suggested to continue the practical work of school evaluations (foreseen to start in autumn 2020) and engage education community (schools, education professionals and other community members) to cooperate in further developing the initiative of school evaluations according to the developed framework.

Management Response: [Added: 2019/12/28] [Last Updated: 2021/01/25]

It is important to engage education community (schools, education professionals and other community members) to cooperate in further developing the initiative of school evaluations according to the developed framework.

 

Key Actions:

5. Recommendation:

It is recommended to support the developed K-12 Teacher’s Professional Licensing Framework to become operational. The initiative to link Teachesr’s Professional Development into the teachers’ financial compensation is a good motivator to enter or upgrade Teaching Profession. It is understood that the framework has been approved with a Bylaw to start the practical implementation among the professional community of teachers. The developed framework with possibilities for upgrading of professional levels of teachers and increading renumeration would greatly contribute to the quality of teaching and is likely to contribute to enhanching employment opportunities (also among women in early school years) in the education sector in the KSA.

Management Response: [Added: 2019/12/28] [Last Updated: 2021/01/25]

Noted, support suggested will be included as output of new PD 

Key Actions:

6. Recommendation:

The work of national tests is recommended to continue, as initiated by the Project. National tests are an integral part of assessing learning outcomes of the education to provide decision-makers and stakeholders a reliable measure at the national level, thus contributing to the improvement of quality of education. The results from international evaluations (such as PISA) can serve as a baseline benchmark and set the goals for the next years.

Management Response: [Added: 2019/12/28] [Last Updated: 2021/01/25]

Noted, recommendation will be taken into account in new PD

Key Actions:

7. Recommendation:

It is recommended to continue the work on Curriculum standards and content with the support of international expertise. There is a need to expand the developed terms of reference and standards of Curricula to all levels in general eduction. In addition, the improved quality of teaching re-orient towards 21st century skills and Learning Outcomes reflecting the skills need of the knowledge-based economy, in addition to the indicators already developed. The development of curriculum standards especially higher level (from K-12 upwards) need to reflect the skills and competences needed in the labour market and employment opportunities available in the economy of the KSA.

Management Response: [Added: 2019/12/28] [Last Updated: 2021/01/25]

New PD should expand the developed terms of reference and standards of Curricula to all levels in general education

Key Actions:

8. Recommendation:

It is recommended to continue the development of National Qualifications Framework in KSA (SANQF). The NQF framwork is now developed at 10 levels. Educational Institutes are required to register their institutions as well as the qualifications/programs they are offering on a set of standards. There are two sets of standards: standards for institution, standards for the program. The evaluation system is ready for the schools and programs to register into the NQF system. The qualifications framework needs further development in formalising the structure in which learning level descriptors and qualifications are used in order to understand learning outcomes. It can be assessed that all work developed in the project; curriculum, teachers’ licensing, school evaluation and curricula development need to be aligned with the developed SANQF. For example, the curricula need to specify the learning outcomes and level in SANQF.

Management Response: [Added: 2019/12/28] [Last Updated: 2021/01/25]

The New PD will include thye development of the qualifications framework 

Key Actions:

9. Recommendation:

It is recommended to support the visibility of UNDP Project and the developed outcomes through different media channels and target audiences with a consistent media plan. It is suggested to continue the work of media outreach to raise awarness of the developed outcomes e.g. curriculum development, school evaluation, teacher liscensing, national qualifications framwork. It is essential to raise awareness about the importance of evaluation of education and training to quarantee the quality of the education system, teaching and learning, acquiring the learning outcomes and skills in the framework of the Vision 2030. Cooperation with other international institutions, e.g. the World Bank is suggested to ensure coordination of international support (Paris Declaration, 2005).

Management Response: [Added: 2019/12/28] [Last Updated: 2021/01/25]

Continue the work of media outreach to raise awarness of the developed outcomes

Key Actions:

Latest Evaluations

Contact us

1 UN Plaza
DC1-20th Floor
New York, NY 10017
Tel. +1 646 781 4200
Fax. +1 646 781 4213
erc.support@undp.org