- Evaluation Plan:
- 2021-2025, North Macedonia
- Evaluation Type:
- Final Project
- Planned End Date:
- 07/2022
- Completion Date:
- 07/2022
- Status:
- Completed
- Management Response:
- Yes
- Evaluation Budget(US $):
- 20,000
Final Project Evaluation of the “Building Municipal Capacities for Project Implementation (BMCPI)”
| Document | Type | Language | Size | Status | Downloads |
|---|---|---|---|---|---|
 TOR_International Evaluator SIDA_20Apr2022.docx |
tor | English | 89.82 KB | Posted | 22 |
| TOR (BMCPI) Project Final Evaluation_Final 20Apr2022.docx |
tor | English | 75.96 KB | Posted | 40 |
| FER BMCPI final version with Executive Summary.docx |
report | English | 856.66 KB | Posted | 21 |
| Title | Final Project Evaluation of the “Building Municipal Capacities for Project Implementation (BMCPI)” | |||||||||
|---|---|---|---|---|---|---|---|---|---|---|
| Atlas Project Number: | 00105769 | |||||||||
| Evaluation Plan: | 2021-2025, North Macedonia | |||||||||
| Evaluation Type: | Final Project | |||||||||
| Status: | Completed | |||||||||
| Completion Date: | 07/2022 | |||||||||
| Planned End Date: | 07/2022 | |||||||||
| Management Response: | Yes | |||||||||
| Focus Area: |
|
|||||||||
| Corporate Outcome and Output (UNDP Strategic Plan 2018-2021) |
|
|||||||||
| SDG Goal |
|
|||||||||
| SDG Target |
|
|||||||||
| Evaluation Budget(US $): | 20,000 | |||||||||
| Source of Funding: | Project funds | |||||||||
| Evaluation Expenditure(US $): | 16,054 | |||||||||
| Joint Programme: | No | |||||||||
| Joint Evaluation: | No | |||||||||
| Evaluation Team members: |
|
|||||||||
| GEF Evaluation: | No | |||||||||
| Key Stakeholders: | SIDA; Ministry of Local Self-Government; selected municipalities | |||||||||
| Countries: | MACEDONIA, THE FORMER YUGOSLAV REPUBLIC OF | |||||||||
| Comments: | UNDP will do the commissioning of the evaluation, in agreement with the Donor (SIDA) |
| Lessons | |
|---|---|
| 1. | The extraordinarily high workload (high number of procurement procedures) and staffing challenges must be resolved as early and efficiently as possible (asymmetries in workload distribution may affect overall project results). Project design needs to consider internal absorption capacities to prevent delays. Need of adaptive capacity and implementation efficiency for a more rational provision of the necessary design and review services. MDI. The role of the procedures in trust building among municipalities; the need to simplify and digitise the MDI to overcome challenges to data collection; the identification of sustainability and impact critical factors. FA. The project's limited scope influences the implementation of recommendations for FAs, the importance of the participatory approach and the need for comparative analysis for the selection process. TDF. The virtually unlimited demand for high-quality technical documentation; the low quality of the technical documentation; the need for additional assistance in the project implementation; the lack of a unified country-wide approach to formulating TORs and of customisation of the technical documentation; the importance of the prior screening of available documentation, and data and the need of solid quality assurance mechanisms; the expectations of support between small and large municipalities. COVID-19: the importance of hybrid communication for favourable implementation rates under restrictions; the online evaluation of tenders; prioritising activities during periods of limitations to accumulate results |
| Findings | |
| 1. | 1.1. Findings1.1.1. RelevanceThe relevance of the BMCPI project is still very high, adequately addressing the needs of beneficiary municipalities and institutions, and well-positioned in ongoing and planned complementary projects and other initiatives, benefiting from and contributing to synergies with several other relevant interventions. The primary identified problem was not considering the consequence of many small procurements, which caused a high workload for the PT and created an asymmetry in the time/energy distribution of the PT among the different project components. The dynamic relevance has been very high in a highly challenging environment characterised by multiple uncertainties and rapid context evolution, primarily in response to the Covid-19 crisis, local elections, as well as the other ongoing economic, energy and security crises. 1.1.2. EfficiencyDespite the complications caused by COVID-19 and the very high implementation workload, the project's overall performance is excellent. The Project is about completing the key activities and achieving the intended results and objectives within the allotted time. In June 2022, approximately 91% of the project time, the budget execution is 100%. The enthusiastic, motivated and qualified team of UNDP and the implementation mechanisms proved efficient, ensuring a constant and attentive follow-up of the activities with the stakeholders involved. But procurement workload management represented a fundamental challenge in the current BMCPI phase. The costs of the project activities and products have been in line with national standards and lower than other projects of similar size and degree of complexity known to the ET. 1.1.3. EffectivenessThe Project’s effectiveness at the level of outputs/products is highly satisfactory. The outcome/effects level effectiveness is still early to be achieved as all three Project lines of intervention (the Municipal Development Index -MDI-, the Functional Analysis -FA-, and Technical Documentation Fund -TDF-) would require additional time to bring about more lasting changes. MDI piloting is considered comprehensive and composed of all relevant indicators of local development, providing a preliminary ranking of municipalities regarding their development levels. The MDI development process showed challenges (among them, the complexity of the MDI, data requirements and quality assurance and data validation) that the project attempts to address through a highly participatory and dynamic process. FA of the country’s local self-government system at a more general or systemic level was developed, piloting the approach among selected municipalities and regional centres. The TDF positively tested and piloted a model financial instrument to prepare comprehensive technical documentation for the highest priority infrastructure recovery and development projects. The project selected and supported 55 calls. Municipalities show high heterogeneity (urban-rural, differences in qualified staff) in their capacities to implement the TDF model in the future and the future, TDF will not be able to adopt a ‘one size fits all approach to providing the necessary financial and technical support (Stratification’ of TDF support) 1.1.4. ImpactEven if the outcome/effects level effectiveness is still early to be achieved so far, the ET considers there is a viable foundation for achieving the project's intended impact and that the impact is the logical consequence of the project's sustainability and ownership. The degree of institutionalisation impact of the project is still partial, especially for municipalities with insufficient technical and specialised staff.
1.1.5. Sustainability/OwnershipThe overall sustainability of the benefits and effects of the project is good. The sustainability of the MDI is high, thanks to the interest shown by the municipalities. The following BMCPI projects will not support the FA, but other complementary UNDP or institutional projects could facilitate the realisation of other FA. The institutionalisation of the TDF in the municipalities is an ongoing process, not completed by the project. The extremely high interest of the municipalities in the TDF as the proper answer to their needs represents the essential element for the sustainability of this critical component. At the same time, the conditions are related to strengthening municipal capacity in preparing TORs for design and review of technical documentation, quality assurance instruments, and fundraising for project implementation. It is also under definition the future role of the BRD (that demonstrated total commitment) to host the TDF since municipalities see an essential part of the centres for regional development and the thematic ministries. 1.1.6. Gender perspectiveAll institutions count on gender policies, the implementation of which is not homogeneous between them. Even if a gender analysis to better address the need of women and people living in vulnerable situations is lacking, the Project consistently pays attention to gender and vulnerable population priorities, ensuring data disaggregation in MDI and TDF and active participation of women in all the processes, including in training and the inclusion of gender awareness aspects in the central ToR for the technical documentation for different infrastructure projects. |
| Recommendations | |
|---|---|
| 1 | Recommendation 1. Finalize the formulation of an outline of the legal (step-by-step) process for adoption of Municipal Development Index - MDI. |
| 2 | Recommendation 2. Co-design and communicate a Protocol on maintaining the Municipal Development Index - MDI |
| 3 | Recommendation 3. Work toward further simplification/streamlining of the Municipal Development Index - MDI model. |
| 4 | Recommendation 4. Provide training targeting responsible municipal personnel based on the final/agreed Municipal Development Index - MDI model. |
| 5 | Recommendation 5. Work toward ‘automation’ and ‘digitation’ of the process |
| 6 | Recommendation 6. Evaluate the feasibility of supporting the implementation of the BMCPI-supported Functional Analyses with other projects or programmes of UNDP or institutions |
| 7 | Recommendation 7. Placing stronger emphasis on renewable energy and energy efficiency projects considering the energy crisis |
| 8 | Recommendation 8. Improve beneficiary involvement in Project implementation |
| 9 | Recommendation 9. MDI. Identify, according to municipal characteristics, an action sheet that allows, starting from the most relevant indicators, to achieve the coverage of the typical indicators by municipal capacities over time. Automate the updating of the MDI with the data available from national institutions |
| 10 | Recommendation 10. Evaluate options for provision of support to municipalities for identification of funding opportunities and preparation of project application (e.g., TDF, BRD, MSLG, RDCs) |
| 11 | Recommendation 11. Conduct a gender and vulnerability analysis to identify differentiated needs for the most vulnerable categories, such as women, people with disabilities, children, elders, and migrants and incorporate them across the project activities. |
Management Response Documents
1. Recommendation:
Recommendation 1. Finalize the formulation of an outline of the legal (step-by-step) process for adoption of Municipal Development Index - MDI.
Management Response: [Added: 2022/09/16]
The recommendation is accepted.
Key Actions:
| Key Action | Responsible | DueDate | Status | Comments | Documents |
|---|---|---|---|---|---|
| 1.1 Outlining of the legal (step-by-step) process through close cooperation with the relevant authorities.
[Added: 2022/09/16] |
- UNDP Democratic governance Portfolio; - Ministry of Local Self-government - MLSG | 2023/04 | Initiated | - Initiated/included in the project document for the planned second phase of the project - The recommendation is accepted and found to be relevant. In the planned second phase of the project, the PIU will hire a legal expert to support the process. History | |
| 1.2 Incorporation of the key relevant actions in the Project’s work-plan
[Added: 2022/09/16] |
UNDP Democratic governance Portfolio | 2023/05 | Initiated | - To be initiated once the previous key action is completed. - The recommendation is accepted and found to be relevant. In the planned second phase of the project, the PIU will hire a legal expert to support the process. |
2. Recommendation:
Recommendation 2. Co-design and communicate a Protocol on maintaining the Municipal Development Index - MDI
Management Response: [Added: 2022/09/16]
The recommendatin is accepted.
Key Actions:
| Key Action | Responsible | DueDate | Status | Comments | Documents |
|---|---|---|---|---|---|
| 2.1. Design, share and facilitate the adoption of the Protocol for MDI maintenance
[Added: 2022/09/16] |
- UNDP Democratic governance Portfolio; - Ministry of Local Self-government - MLSG | 2023/12 | Initiated | - At conceptualization stage/included in the project document for the planned second phase of the project. - This relevant recommendation that will be addressed by the Project as suggested, during the planned second phase of project implementation. |
3. Recommendation:
Recommendation 3. Work toward further simplification/streamlining of the Municipal Development Index - MDI model.
Management Response: [Added: 2022/09/16]
The recommendation is accepted.
Key Actions:
| Key Action | Responsible | DueDate | Status | Comments | Documents |
|---|---|---|---|---|---|
| 3.1. Hire experts and engage in further stakeholder consultations to simplify/improve MDI model
[Added: 2022/09/16] |
UNDP Democratic governance Portfolio | 2023/05 | Initiated | - At conceptualization stage, included in the project document for the second phase of the project. The key requirements will be articulated in the corresponding TOR for expert support; - The simplification and improvement of MDI will be done as part of a broader process based on expert input and stakeholder consultation aiming at ‘institutionalizing’ MDI, planned in the second phase of the project. | |
| 3.2. Finalize and promote the simplified/improved MDI model
[Added: 2022/09/16] |
- UNDP Democratic governance Portfolio; - Ministry of Local Self-government - MLSG | 2024/02 | Initiated | - To be completed once the updated MDI model is developed and validated, during the planned second phase of the project - The simplification and improvement of MDI will be done as part of a broader process based on expert input and stakeholder consultation aiming at ‘institutionalizing’ MDI, planned in the second phase of the project. |
4. Recommendation:
Recommendation 4. Provide training targeting responsible municipal personnel based on the final/agreed Municipal Development Index - MDI model.
Management Response: [Added: 2022/09/16]
The recommendation is accepted.
Key Actions:
| Key Action | Responsible | DueDate | Status | Comments | Documents |
|---|---|---|---|---|---|
| 4.1. Deliver trainings on MDI use to municipalities and other stakeholders
[Added: 2022/09/16] |
UNDP Democratic governance Portfolio | 2025/03 | Initiated | - In planning stage. The necessary requirements will be reflected in the corresponding TOR for the planned expert support in the project’s upcoming phase. - The trainings will be incorporated in the broader expert input aiming at improving and ‘institutionalizing’ MDI, as part of the planned second phase of the project. |
5. Recommendation:
Recommendation 5. Work toward ‘automation’ and ‘digitation’ of the process
Management Response: [Added: 2022/09/16]
The recommendation is accepted.
Key Actions:
| Key Action | Responsible | DueDate | Status | Comments | Documents |
|---|---|---|---|---|---|
| 5.1. Engage the necessary expertise that will ensure automation/digitation of the MDI
[Added: 2022/09/16] |
UNDP Democratic governance Portfolio | 2024/03 | Initiated | - At conceptual/planning level. Relevant activities will be initiated with the start of the Project’s upcoming phase. - The necessary expertise for ‘automation’ and ‘digitation’ of MDI will be incorporated in the broader expertise package aiming at ‘institutionalizing’ MDI, as part of the planned second phase of the project. |
6. Recommendation:
Recommendation 6. Evaluate the feasibility of supporting the implementation of the BMCPI-supported Functional Analyses with other projects or programmes of UNDP or institutions
Management Response: [Added: 2022/09/16]
The recommendation is not applicable.
Key Actions:
| Key Action | Responsible | DueDate | Status | Comments | Documents |
|---|---|---|---|---|---|
| Not applicable
[Added: 2022/09/16] |
UNDP Democratic governance Portfolio | 2022/09 | No Longer Applicable |
[Justification: This has already been completed as other UNDP projects are already implementing activities based on the BMCPI-supported Functional Analyses] |
7. Recommendation:
Recommendation 7. Placing stronger emphasis on renewable energy and energy efficiency projects considering the energy crisis
Management Response: [Added: 2022/09/16]
The recommendation is accepted.
Key Actions:
| Key Action | Responsible | DueDate | Status | Comments | Documents |
|---|---|---|---|---|---|
| 7.1. Provide increased funding opportunities for energy efficiency and renewable energy projects in the new TDF
[Added: 2022/09/16] |
UNDP Democratic governance Portfolio | 2024/04 | Initiated | - At planning stage. - The Project will prioritize energy efficiency/renewable energy projects as part of the planned calls for applications under the TDF during the planned second phase of the project. |
8. Recommendation:
Recommendation 8. Improve beneficiary involvement in Project implementation
Management Response: [Added: 2022/09/16]
The recommendation is accepted.
Key Actions:
| Key Action | Responsible | DueDate | Status | Comments | Documents |
|---|---|---|---|---|---|
| 8.1. Develop a detailed stakeholder participation plan
[Added: 2022/09/16] [Last Updated: 2022/11/08] |
UNDP Democratic governance Portfolio | 2022/11 | Completed | Completed. The plan has been outlined and incorporated in the Project’s multi-year workplan. Its timeline will be adjusted depending on the official start date of the project (currently planned for 1 December 2022) History |
9. Recommendation:
Recommendation 9. MDI. Identify, according to municipal characteristics, an action sheet that allows, starting from the most relevant indicators, to achieve the coverage of the typical indicators by municipal capacities over time. Automate the updating of the MDI with the data available from national institutions
Management Response: [Added: 2022/09/16]
The recommendation is accepted.
Key Actions:
| Key Action | Responsible | DueDate | Status | Comments | Documents |
|---|---|---|---|---|---|
| 9.1. Incorporate the IT platform for MDI in the project plans and priorities
[Added: 2022/09/16] [Last Updated: 2022/11/11] |
UNDP Democratic governance Portfolio | 2023/02 | Initiated | Ongoing. Due to delay in project start date, this is postponed. However, the idea for IT platform is already part of the concepts for the MDI-related work packages. History |
10. Recommendation:
Recommendation 10. Evaluate options for provision of support to municipalities for identification of funding opportunities and preparation of project application (e.g., TDF, BRD, MSLG, RDCs)
Management Response: [Added: 2022/09/16]
The recommendation is accepted.
Key Actions:
| Key Action | Responsible | DueDate | Status | Comments | Documents |
|---|---|---|---|---|---|
| 10.1. Include the project identification / fundraising as a function in the updated model of the TDF
[Added: 2022/09/16] |
UNDP Democratic governance Portfolio | 2023/04 | Initiated | This is underway and will continue in the second phase of the project. The project will work with the RDCs, BRD and MLSG to improve dissemination of information and provision of support for creation of new projects in a more systematic fashion. | |
| 10.2. Encourage applications of joint municipal / regional projects as part of the planned TDF calls for applications
[Added: 2022/09/16] [Last Updated: 2022/11/11] |
UNDP Democratic governance Portfolio | 2023/04 | Initiated | Ongoing. The project start date has been postponed due to delayed signing of the framework agreement. The new time will be defined depending on the official start of the project. History |
11. Recommendation:
Recommendation 11. Conduct a gender and vulnerability analysis to identify differentiated needs for the most vulnerable categories, such as women, people with disabilities, children, elders, and migrants and incorporate them across the project activities.
Management Response: [Added: 2022/09/16]
The recommendation is accepted.
Key Actions:
| Key Action | Responsible | DueDate | Status | Comments | Documents |
|---|---|---|---|---|---|
| 11.1. Preparation of gender sensitive, inclusive and integrated local development plans
[Added: 2022/09/16] |
UNDP Democratic governance Portfolio | 2024/04 | Initiated | Planned for the 2nd phase of the project; gender sensitive integrated local development plans will be prepared, and gender mainstreaming trainings will be organized for local authorities. Contractors will also be sensitized on gender issues. |