Final Project Evaluation of the “Building Municipal Capacities for Project Implementation (BMCPI)”

Report Cover Image
Evaluation Plan:
2021-2025, North Macedonia
Evaluation Type:
Final Project
Planned End Date:
07/2022
Completion Date:
07/2022
Status:
Completed
Management Response:
Yes
Evaluation Budget(US $):
20,000

Share

Document Type Language Size Status Downloads
Download document TOR_International Evaluator SIDA_20Apr2022.docx tor English 89.82 KB Posted 22
Download document TOR (BMCPI) Project Final Evaluation_Final 20Apr2022.docx tor English 75.96 KB Posted 40
Download document FER BMCPI final version with Executive Summary.docx report English 856.66 KB Posted 21
Title Final Project Evaluation of the “Building Municipal Capacities for Project Implementation (BMCPI)”
Atlas Project Number: 00105769
Evaluation Plan: 2021-2025, North Macedonia
Evaluation Type: Final Project
Status: Completed
Completion Date: 07/2022
Planned End Date: 07/2022
Management Response: Yes
Focus Area:
  • 1. Governance
  • 2. Others
Corporate Outcome and Output (UNDP Strategic Plan 2018-2021)
  • 1. Output 3.3.1 Evidence-based assessment and planning tools and mechanisms applied to enable implementation of gender-sensitive and risk-informed prevention and preparedness to limit the impact of natural hazards and pandemics and promote peaceful, just and inclusive societies
SDG Goal
  • Goal 16. Promote peaceful and inclusive societies for sustainable development, provide access to justice for all and build effective, accountable and inclusive institutions at all levels
SDG Target
  • 16.6 Develop effective, accountable and transparent institutions at all levels
  • 16.7 Ensure responsive, inclusive, participatory and representative decision-making at all levels
Evaluation Budget(US $): 20,000
Source of Funding: Project funds
Evaluation Expenditure(US $): 16,054
Joint Programme: No
Joint Evaluation: No
Evaluation Team members:
Name Title Nationality
Aldo Magoga International Evaluator
Dimitrija Sekovski National Evaluator
GEF Evaluation: No
Key Stakeholders: SIDA; Ministry of Local Self-Government; selected municipalities
Countries: MACEDONIA, THE FORMER YUGOSLAV REPUBLIC OF
Comments:

UNDP will do the commissioning of the evaluation, in agreement with the Donor (SIDA)

Lessons
1.

The extraordinarily high workload (high number of procurement procedures) and staffing challenges must be resolved as early and efficiently as possible (asymmetries in workload distribution may affect overall project results). Project design needs to consider internal absorption capacities to prevent delays. Need of adaptive capacity and implementation efficiency for a more rational provision of the necessary design and review services.

MDI. The role of the procedures in trust building among municipalities; the need to simplify and digitise the MDI to overcome challenges to data collection; the identification of sustainability and impact critical factors.

FA. The project's limited scope influences the implementation of recommendations for FAs, the importance of the participatory approach and the need for comparative analysis for the selection process.

TDF. The virtually unlimited demand for high-quality technical documentation; the low quality of the technical documentation; the need for additional assistance in the project implementation; the lack of a unified country-wide approach to formulating TORs and of customisation of the technical documentation; the importance of the prior screening of available documentation, and data and the need of solid quality assurance mechanisms; the expectations of support between small and large municipalities.

COVID-19: the importance of hybrid communication for favourable implementation rates under restrictions; the online evaluation of tenders; prioritising activities during periods of limitations to accumulate results


Findings
1.

1.1.  Findings

1.1.1.    Relevance

The relevance of the BMCPI project is still very high, adequately addressing the needs of beneficiary municipalities and institutions, and well-positioned in ongoing and planned complementary projects and other initiatives, benefiting from and contributing to synergies with several other relevant interventions. The primary identified problem was not considering the consequence of many small procurements, which caused a high workload for the PT and created an asymmetry in the time/energy distribution of the PT among the different project components. The dynamic relevance has been very high in a highly challenging environment characterised by multiple uncertainties and rapid context evolution, primarily in response to the Covid-19 crisis, local elections, as well as the other ongoing economic, energy and security crises.

1.1.2.    Efficiency

Despite the complications caused by COVID-19 and the very high implementation workload, the project's overall performance is excellent. The Project is about completing the key activities and achieving the intended results and objectives within the allotted time. In June 2022, approximately 91% of the project time, the budget execution is 100%. The enthusiastic, motivated and qualified team of UNDP and the implementation mechanisms proved efficient, ensuring a constant and attentive follow-up of the activities with the stakeholders involved. But procurement workload management represented a fundamental challenge in the current BMCPI phase. The costs of the project activities and products have been in line with national standards and lower than other projects of similar size and degree of complexity known to the ET.

1.1.3.    Effectiveness

The Project’s effectiveness at the level of outputs/products is highly satisfactory. The outcome/effects level effectiveness is still early to be achieved as all three Project lines of intervention (the Municipal Development Index -MDI-, the Functional Analysis -FA-, and Technical Documentation Fund -TDF-) would require additional time to bring about more lasting changes.

MDI piloting is considered comprehensive and composed of all relevant indicators of local development, providing a preliminary ranking of municipalities regarding their development levels. The MDI development process showed challenges (among them, the complexity of the MDI, data requirements and quality assurance and data validation) that the project attempts to address through a highly participatory and dynamic process. FA of the country’s local self-government system at a more general or systemic level was developed, piloting the approach among selected municipalities and regional centres. The TDF positively tested and piloted a model financial instrument to prepare comprehensive technical documentation for the highest priority infrastructure recovery and development projects. The project selected and supported 55 calls.

Municipalities show high heterogeneity (urban-rural, differences in qualified staff) in their capacities to implement the TDF model in the future and the future, TDF will not be able to adopt a ‘one size fits all approach to providing the necessary financial and technical support (Stratification’ of TDF support)

1.1.4.    Impact

Even if the outcome/effects level effectiveness is still early to be achieved so far, the ET considers there is a viable foundation for achieving the project's intended impact and that the impact is the logical consequence of the project's sustainability and ownership. The degree of institutionalisation impact of the project is still partial, especially for municipalities with insufficient technical and specialised staff.

 

1.1.5.    Sustainability/Ownership

The overall sustainability of the benefits and effects of the project is good. The sustainability of the MDI is high, thanks to the interest shown by the municipalities. The following BMCPI projects will not support the FA, but other complementary UNDP or institutional projects could facilitate the realisation of other FA.

The institutionalisation of the TDF in the municipalities is an ongoing process, not completed by the project. The extremely high interest of the municipalities in the TDF as the proper answer to their needs represents the essential element for the sustainability of this critical component. At the same time, the conditions are related to strengthening municipal capacity in preparing TORs for design and review of technical documentation, quality assurance instruments, and fundraising for project implementation. It is also under definition the future role of the BRD (that demonstrated total commitment) to host the TDF since municipalities see an essential part of the centres for regional development and the thematic ministries.

1.1.6.    Gender perspective

All institutions count on gender policies, the implementation of which is not homogeneous between them. Even if a gender analysis to better address the need of women and people living in vulnerable situations is lacking, the Project consistently pays attention to gender and vulnerable population priorities, ensuring data disaggregation in MDI and TDF and active participation of women in all the processes, including in training and the inclusion of gender awareness aspects in the central ToR for the technical documentation for different infrastructure projects.


Recommendations
1

Recommendation 1. Finalize the formulation of an outline of the legal (step-by-step) process for adoption of Municipal Development Index - MDI.

2

Recommendation 2. Co-design and communicate a Protocol on maintaining the Municipal Development Index - MDI

3

Recommendation 3. Work toward further simplification/streamlining of the Municipal Development Index - MDI model.

4

Recommendation 4. Provide training targeting responsible municipal personnel based on the final/agreed Municipal Development Index - MDI model.

5

Recommendation 5. Work toward ‘automation’ and ‘digitation’ of the process

6

Recommendation 6. Evaluate the feasibility of supporting the implementation of the BMCPI-supported Functional Analyses with other projects or programmes of UNDP or institutions

7

Recommendation 7. Placing stronger emphasis on renewable energy and energy efficiency projects considering the energy crisis

8

Recommendation 8. Improve beneficiary involvement in Project implementation

9

Recommendation 9. MDI. Identify, according to municipal characteristics, an action sheet that allows, starting from the most relevant indicators, to achieve the coverage of the typical indicators by municipal capacities over time. Automate the updating of the MDI with the data available from national institutions

10

Recommendation 10. Evaluate options for provision of support to municipalities for identification of funding opportunities and preparation of project application (e.g., TDF, BRD, MSLG, RDCs)

11

Recommendation 11. Conduct a gender and vulnerability analysis to identify differentiated needs for the most vulnerable categories, such as women, people with disabilities, children, elders, and migrants and incorporate them across the project activities.

1. Recommendation:

Recommendation 1. Finalize the formulation of an outline of the legal (step-by-step) process for adoption of Municipal Development Index - MDI.

Management Response: [Added: 2022/09/16]

The recommendation is accepted.

Key Actions:

Key Action Responsible DueDate Status Comments Documents
1.1 Outlining of the legal (step-by-step) process through close cooperation with the relevant authorities.
[Added: 2022/09/16]
- UNDP Democratic governance Portfolio; - Ministry of Local Self-government - MLSG 2023/04 Initiated - Initiated/included in the project document for the planned second phase of the project - The recommendation is accepted and found to be relevant. In the planned second phase of the project, the PIU will hire a legal expert to support the process. History
1.2 Incorporation of the key relevant actions in the Project’s work-plan
[Added: 2022/09/16]
UNDP Democratic governance Portfolio 2023/05 Initiated - To be initiated once the previous key action is completed. - The recommendation is accepted and found to be relevant. In the planned second phase of the project, the PIU will hire a legal expert to support the process.
2. Recommendation:

Recommendation 2. Co-design and communicate a Protocol on maintaining the Municipal Development Index - MDI

Management Response: [Added: 2022/09/16]

The recommendatin is accepted.

Key Actions:

Key Action Responsible DueDate Status Comments Documents
2.1. Design, share and facilitate the adoption of the Protocol for MDI maintenance
[Added: 2022/09/16]
- UNDP Democratic governance Portfolio; - Ministry of Local Self-government - MLSG 2023/12 Initiated - At conceptualization stage/included in the project document for the planned second phase of the project. - This relevant recommendation that will be addressed by the Project as suggested, during the planned second phase of project implementation.
3. Recommendation:

Recommendation 3. Work toward further simplification/streamlining of the Municipal Development Index - MDI model.

Management Response: [Added: 2022/09/16]

The recommendation is accepted.

Key Actions:

Key Action Responsible DueDate Status Comments Documents
3.1. Hire experts and engage in further stakeholder consultations to simplify/improve MDI model
[Added: 2022/09/16]
UNDP Democratic governance Portfolio 2023/05 Initiated - At conceptualization stage, included in the project document for the second phase of the project. The key requirements will be articulated in the corresponding TOR for expert support; - The simplification and improvement of MDI will be done as part of a broader process based on expert input and stakeholder consultation aiming at ‘institutionalizing’ MDI, planned in the second phase of the project.
3.2. Finalize and promote the simplified/improved MDI model
[Added: 2022/09/16]
- UNDP Democratic governance Portfolio; - Ministry of Local Self-government - MLSG 2024/02 Initiated - To be completed once the updated MDI model is developed and validated, during the planned second phase of the project - The simplification and improvement of MDI will be done as part of a broader process based on expert input and stakeholder consultation aiming at ‘institutionalizing’ MDI, planned in the second phase of the project.
4. Recommendation:

Recommendation 4. Provide training targeting responsible municipal personnel based on the final/agreed Municipal Development Index - MDI model.

Management Response: [Added: 2022/09/16]

The recommendation is accepted.

Key Actions:

Key Action Responsible DueDate Status Comments Documents
4.1. Deliver trainings on MDI use to municipalities and other stakeholders
[Added: 2022/09/16]
UNDP Democratic governance Portfolio 2025/03 Initiated - In planning stage. The necessary requirements will be reflected in the corresponding TOR for the planned expert support in the project’s upcoming phase. - The trainings will be incorporated in the broader expert input aiming at improving and ‘institutionalizing’ MDI, as part of the planned second phase of the project.
5. Recommendation:

Recommendation 5. Work toward ‘automation’ and ‘digitation’ of the process

Management Response: [Added: 2022/09/16]

The recommendation is accepted.

Key Actions:

Key Action Responsible DueDate Status Comments Documents
5.1. Engage the necessary expertise that will ensure automation/digitation of the MDI
[Added: 2022/09/16]
UNDP Democratic governance Portfolio 2024/03 Initiated - At conceptual/planning level. Relevant activities will be initiated with the start of the Project’s upcoming phase. - The necessary expertise for ‘automation’ and ‘digitation’ of MDI will be incorporated in the broader expertise package aiming at ‘institutionalizing’ MDI, as part of the planned second phase of the project.
6. Recommendation:

Recommendation 6. Evaluate the feasibility of supporting the implementation of the BMCPI-supported Functional Analyses with other projects or programmes of UNDP or institutions

Management Response: [Added: 2022/09/16]

The recommendation is not applicable.

Key Actions:

Key Action Responsible DueDate Status Comments Documents
Not applicable
[Added: 2022/09/16]
UNDP Democratic governance Portfolio 2022/09 No Longer Applicable [Justification: This has already been completed as other UNDP projects are already implementing activities based on the BMCPI-supported Functional Analyses]
7. Recommendation:

Recommendation 7. Placing stronger emphasis on renewable energy and energy efficiency projects considering the energy crisis

Management Response: [Added: 2022/09/16]

The recommendation is accepted.

Key Actions:

Key Action Responsible DueDate Status Comments Documents
7.1. Provide increased funding opportunities for energy efficiency and renewable energy projects in the new TDF
[Added: 2022/09/16]
UNDP Democratic governance Portfolio 2024/04 Initiated - At planning stage. - The Project will prioritize energy efficiency/renewable energy projects as part of the planned calls for applications under the TDF during the planned second phase of the project.
8. Recommendation:

Recommendation 8. Improve beneficiary involvement in Project implementation

Management Response: [Added: 2022/09/16]

The recommendation is accepted.

Key Actions:

Key Action Responsible DueDate Status Comments Documents
8.1. Develop a detailed stakeholder participation plan
[Added: 2022/09/16] [Last Updated: 2022/11/08]
UNDP Democratic governance Portfolio 2022/11 Completed Completed. The plan has been outlined and incorporated in the Project’s multi-year workplan. Its timeline will be adjusted depending on the official start date of the project (currently planned for 1 December 2022) History
9. Recommendation:

Recommendation 9. MDI. Identify, according to municipal characteristics, an action sheet that allows, starting from the most relevant indicators, to achieve the coverage of the typical indicators by municipal capacities over time. Automate the updating of the MDI with the data available from national institutions

Management Response: [Added: 2022/09/16]

The recommendation is accepted.

Key Actions:

Key Action Responsible DueDate Status Comments Documents
9.1. Incorporate the IT platform for MDI in the project plans and priorities
[Added: 2022/09/16] [Last Updated: 2022/11/11]
UNDP Democratic governance Portfolio 2023/02 Initiated Ongoing. Due to delay in project start date, this is postponed. However, the idea for IT platform is already part of the concepts for the MDI-related work packages. History
10. Recommendation:

Recommendation 10. Evaluate options for provision of support to municipalities for identification of funding opportunities and preparation of project application (e.g., TDF, BRD, MSLG, RDCs)

Management Response: [Added: 2022/09/16]

The recommendation is accepted.

Key Actions:

Key Action Responsible DueDate Status Comments Documents
10.1. Include the project identification / fundraising as a function in the updated model of the TDF
[Added: 2022/09/16]
UNDP Democratic governance Portfolio 2023/04 Initiated This is underway and will continue in the second phase of the project. The project will work with the RDCs, BRD and MLSG to improve dissemination of information and provision of support for creation of new projects in a more systematic fashion.
10.2. Encourage applications of joint municipal / regional projects as part of the planned TDF calls for applications
[Added: 2022/09/16] [Last Updated: 2022/11/11]
UNDP Democratic governance Portfolio 2023/04 Initiated Ongoing. The project start date has been postponed due to delayed signing of the framework agreement. The new time will be defined depending on the official start of the project. History
11. Recommendation:

Recommendation 11. Conduct a gender and vulnerability analysis to identify differentiated needs for the most vulnerable categories, such as women, people with disabilities, children, elders, and migrants and incorporate them across the project activities.

Management Response: [Added: 2022/09/16]

The recommendation is accepted.

Key Actions:

Key Action Responsible DueDate Status Comments Documents
11.1. Preparation of gender sensitive, inclusive and integrated local development plans
[Added: 2022/09/16]
UNDP Democratic governance Portfolio 2024/04 Initiated Planned for the 2nd phase of the project; gender sensitive integrated local development plans will be prepared, and gender mainstreaming trainings will be organized for local authorities. Contractors will also be sensitized on gender issues.

Latest Evaluations

Contact us

1 UN Plaza
DC1-20th Floor
New York, NY 10017
Tel. +1 646 781 4200
Fax. +1 646 781 4213
erc.support@undp.org