Evaluation of the Piloting Phase of the Virtual Zomdu Project

Report Cover Image
Evaluation Plan:
2014-2018, Bhutan
Evaluation Type:
Final Project
Planned End Date:
12/2017
Completion Date:
12/2017
Status:
Completed
Management Response:
Yes
Evaluation Budget(US $):
12,500

Virtual Zomdu (Virtual Meeting) project evaluation Management Response

Share

Document Type Language Size Status Downloads
Download document TOR for evaluation Virtual Zomdu final.docx tor English 70.46 KB Posted 1152
Download document Virtual Zomdu Evaluation Report.docx report English 148.86 KB Posted 1244
Download document UNDP Management_Response_Virtual Zomdue Evaluation-sw.docx summary English 34.14 KB Posted 1177
Title Evaluation of the Piloting Phase of the Virtual Zomdu Project
Atlas Project Number: 00079511
Evaluation Plan: 2014-2018, Bhutan
Evaluation Type: Final Project
Status: Completed
Completion Date: 12/2017
Planned End Date: 12/2017
Management Response: Yes
Focus Area:
  • 1. Others
Corporate Outcome and Output (UNDP Strategic Plan 2018-2021)
  • 1. Output 2.1. Parliaments, constitution making bodies and electoral institutions enabled to perform core functions for improved accountability, participation and representation, including for peaceful transitions
Evaluation Budget(US $): 12,500
Source of Funding: UNDP
Evaluation Expenditure(US $): 5,000
Joint Programme: No
Joint Evaluation: No
Evaluation Team members:
Name Title Nationality
Tandin Dorji Consultant
GEF Evaluation: No
Key Stakeholders: National Assembly Secretariat and GNHC
Countries: BHUTAN
Comments:

Revision of Evaluation Plan extending to Q4 2017

Lessons
1.

Evaluation Lessons

Three key lessons can be drawn from the review of the VZ facility.

  1. In the absence of strong advocacy, even when the best of facilities are put in place, people may not necessarily use them. This has been the case with the VZ initiative. The report titled “Virtual Zomdu: Bringing Parliament to the People of Bhutan: Pilot Phase Report”  had proposed a communication plan using Facebook and Twitter, talk show on BBS TV, posters and brochures, short film on VZ and community consultations. The most important which would reach at the local level effectively are the educating and creating awareness on VZ through the Gewog Office. However, not much has been done on this front.
  2. To persuade people to embrace new initiatives that require change in habit, the agents of change must be brought in the forefront and get their commitment. In this case, the potential agents of change are the MPs. However, not much has been done to get their commitment and support and things were left at “take for granted level” by assuming that the MPs will use it as VZ platform will allow them to meet their constituents more and benefit them also. Thus, when the MPs did not move much, nothing much had happened at the grassroots.
  3. For change to take place effectively and get the commitments of the intended beneficiaries to use a facility, need to use it should be created as discussed in the section on Constraints. Many MPs did not feel the need to use this platform and consequently the facility has been underutilized

Findings
1.

4.1 Relevancy of Virtual Zomdu (VZ)

Virtual Zomdu initiative is directly linked to the 11th FYP mandate of the Ministry of Information and Communication (MoIC) which is to enable effective and efficient public service delivery and improve access to reliable & affordable ICT and media services. Further, the MoIC has critical role to “promote a green and self-reliant economy sustained by an IT enabled knowledge society guided by the philosophy of GNH”. The National priorities of the 11th FYP of improving public service delivery, motivated public servants and implementation of Government Performance System; strengthened democracy and governance and gender friendly environment for women’s participation are also related to the UNDP focus area of democratic governance which in turn is relevant to the VZ initiative

 


Tag: Relevance Gender Equality Communication Service delivery Technology

2.

The G2C and G2G initiatives of the Government  are examples of approaches towards using IT for efficient and effective public service delivery. VZ facility is another relevant initiative towards embracing technology to establish enhanced frequency of communication between MPs and the local communities so that the Representatives of the people are able to fulfill their representative, legislative and oversight functions efficiently and effectively.


Tag: Relevance Oversight Service delivery Country Government Technology Civil Societies and NGOs

3.

This initiative has become even more relevant in the 12th FYP as it has proclaimed its plan objective as “Just, Harmonious and Sustainable Society through enhanced Decentralization”. Some of the priorities to achieve “Just society” are “strengthening democracy and decentralization, promoting and gender equality”. Further, to achieve its objective of sustainable society, the 12th FYP will prioritize the effectiveness and efficiency in public service delivery.  Specifically, the relevancy of Virtual Zomdu are underscored in the National Key Result Areas (NKRAs) where some of the NKRAs are reflected as “Infrastructure, Communication and Public Service Delivery Improved; Gender Equality Promoted, Women and Girls Empowered, and Democracy and Decentralization Strengthened”.


Tag: Relevance Gender Equality Service delivery Women and gilrs

4.

The integration of VZ as one of the strategic activities to strengthen the National Assembly and National Council’s representation and outreach capacity and to create a culture of informed public discussions during the legislative process in the Strategic Plan Document of the NA and NC  also indicates the opportunities of using VZ and its relevancy to strengthening democracy.


The review of the policy and plan documents clearly indicates that the VZ is relevant to the national priorities and needs of the community and in reaching the unreached. The findings of the questionnaire survey and the consultative meetings also support the findings from the document review as Virtual Zomdu facility is seen as an important vehicle towards strengthening of democracy and contributing towards addressing gender perspective and considering human rights approach.

 


Tag: Relevance Justice system Parliament Policies & Procedures Institutional Strengthening National Institutions

5.

For instance, 83.3% of the MPs and 96% of the Gups recognizes the relevancy of VZ to the national needs and priorities. Further, over 93% all the categories of respondents who are MPs, Gups and CC operators report that the VZ initiative is relevant to the needs of the beneficiaries (MPs and local communities).

There are a small percentage of respondents (between 16% and 4%) who have disagreed on the assessment indicators and perceive VZ to be irrelevant to both national needs and priorities as well as the needs of the beneficiaries (MPs and local communities). This reasons for this disagreement are mainly because some of the MPs expressed that while the concept and intention of VZ initiative is good, considering the competing financial priorities there is a feeling that this investment could have been misplaced. The other reason is because some of the respondents felt that VZ facility could have been piloted in other CCs which not close to towns. The CCs with VZ facility which are close to towns have other priorities other than VZ facilities.
On the other hand, the large majority of the people at the local level are excited about this facility as it brings parliament closer to them. For instance, Tshering, Gup of Norbuling Gewog (Serzhong), Sarpang Dzongkhag expressed: “Virtual Zomdu will allow old people and women who cannot travel far outside their Gewog and Dzongkhag can now participate in discussions”.

 


Tag: Relevance Parliament Data and Statistics National Institutions

6.

While the majority of the respondents feel that the VZ initiative has addressed gender perspective and considered human rights approach, 16.7% of MPs and 15.9% of CC Operators feel that VZ has not addressed gender perspective. Similarly, there are also 19.4% of MPs and 20.7% of CC Operators who report that the VZ has not considered human rights approach. The arguments for this expressed during the KII and FGD is that there VZ platform does not give any special privileges to disadvantaged groups and that it is a platform for all. Therefore, this category of respondents perceives that VZ will not have any contribution towards this end.

However, there are 83.3.7% of MPs and 84.1% of CC Operators who reported that the VZ initiative has addressed gender perspective and 80.6% of MPs and 79.3% of CC Operators who expressed that VZ has considered human rights approach. For instance, Kinley Wangmo, CC Operator of Toewang Gewog in Punakha underscored that “Freedom of expression is a very important human right. In this light, the VZ initiative has contributed towards enhancing accessibility of expression of views and opinions since it is located in their Gewog and people do not have to travel far. VZ has made meeting MPs and other high level people easier and  more convenient.”  . This view is in line with Article 7.2 of the Constitution of the Kingdom of Bhutan states that “A Bhutanese citizen shall have the right to freedom of speech, opinion and expression”.


Tag: Relevance Gender Equality Human rights Vulnerable

7.

4.2.    Impact of training activities on VZ management by CC operators

UNDP has invested USD 29,666.92 in between 2015 and 2017 to build the capacity of the 93 stakeholders involved in managing the VZ facility. In phase one 47 people were trained on VZ comprising CC operators, BDBL Regional IT officers, Dzongkhag IT officers and IT officers of the NC and NA. In phase two, 46 people were trained comprising mainly of CC operators. The capacity development activity was conducted by the UNDP, National Assembly and Department of Information Technology and Telecom (DITT).
Since the CC operators were the key people trained to manage the VZ facility, the views on the relevancy, effectiveness and impact of the training. The analysis has been done by phase to see if there are any nuances in the findings reported by the participants of the training.

 


Tag: Impact Relevance Jobs and Livelihoods Technology

8.

The ratings on the ability to manage the technical aspects of the VZ facility show that there is not much difference where 61.8% and 64.5% of phase 1 and phase 2 agree that they are able to manage with confidence. Further, 52.9% of the respondents from phase 1 and 64.5% from phase 2 respectively report that they will be able to carry out the basic maintenance of the VZ facility.  This is encouraging and is an indication that they are exploring and trying to use VZ facility for other purposes to keep their knowledge and skills in touch.  


70.6% and 83.3% of the CC Operators from phase 1 and phase 2 who have responded to the survey respectively report that they apply the knowledge and skills imparted during the training indicating that they are using the VZ facility. However, 29.4% from phase 1 and 16.7% from phase 2 have rated “No” indicating that they do not apply what they have learnt during the training and implying that the VZ facility was not used. This correlates well with the utilization of the VZ facility as 58.2% of the CC Operators have reported that they have not conducted VZ even once.


Tag: Knowledge management Jobs and Livelihoods Technology Data and Statistics

9.

For instance, 56.7% (about 17) of the 30 MPs who had responded to this survey indicator view that the VZ facility managers have received the necessary capacity to manage and use the VZ facilities effectively. This corresponds closely to 54.5% of the MPs who have reported that they had used VZ at least once. Thus, this judgement is based on their experiences of using the VZ facility at least once. On the other hand, 43.3% of the MPs who had responded to other questions did not respond to this indicator implying that they did not use. This too corresponds closely with 45.5% of the MPs who had said that they did not conduct VZ even once. The analysis is that in this case all the MPs who had responded experienced using VZ feel that the CC Operators have the required knowledge and skills to organize VZ indicating that the training was relevant and effective.


Tag: Impact Knowledge management Technology Capacity Building

10.

4.3.    VZ equipment and facilities

UNDAF Outcome 4 is to integrate 75 CCs with VZ facility.   UNDP achieved more than the set target as there are 89 CCs with VZ facility.  The integration of CC with VZ had been conducted in two phases.
The analysis of the VZ equipment and facilities show that 78.8% of the MPs have rated the size of the screen that they use for VZ is big enough. Similar views have been reported by the majority of the Gups (68.2%) and CC Operators (72.4%).  When it comes to the CCs, 55.6% of those set up in phase one expressed that their screen is small as compared to only 10.8% from phase two indicating that phase two CCs are more satisfied with the size of the screen. The MPs, Gups and the CC Operators of phase two who report the size of the screen as small are mainly related to their expectation of having large groups of people in one zomdu session.

 


Tag: Challenges Effectiveness Operational Efficiency Technology

11.

4.4.    Utilization of VZ facility
On August 19, 2015 Virtual Zomdu facility was launched with Kuensel coverage highlighting the occasion as “an unforgettable moment”,  “remarkable day in the history of Bhutan”.  and VZ as a pathway to “break the barriers of geographical difficulties to reach the constituents…” . It is now over two years (about 27 months) that the some of the CCs were facilities have been installed in some CCs and as of today there are 89 CCs. Unfortunately, while the capacity development and the integration of the VZ facility in the CCs have been reported as generally good, there are major issues when it comes to the utilization of this facility. For instance, 57.6% of the MPs, 60.5% of Gups and 80% of the CC operators who have responded to the survey reported that the VZ facility is not used optimally.

Of the 54.5% of the MPs who had organized VZ at least once, only 32.1% had initiated it. Further, only 11.4% of them had organized VZ before parliamentary session. The scenario is same with the CC Operators and the Gups. For instance, of the 43.6% of the Gups who had organized a VZ at least once, 41.9% had initiated a VZ but only 17.5% agree that a VZ was organized before parliamentary session.  Similarly, of the 41.8% of the CC Operators who had organized a VZ, 25.6% had initiated it but only 21.1% had report to have organized a VZ before parliamentary session.

 


Tag: Effectiveness Parliament Country Support Platform Knowledge management Technology

12.

On the other hand, 45.5% of the MPs, 56.4% of the Gups and 58.2% of the CC Operators have reported of never having used the VZ facility.

The main reasons expressed by the MPs for not using the VZ facility was because it was not in far flung places where they faced challenges to reach and where it was really required. This meant that they had to visit the remote un-connected places and since the places connected with VZ were mostly on the way they also visited o they visited these places physically instead of using the VZ. Some other reasons were problem with timing coordination since there was clash of free time. When the community people were ready the MPs were busy or ad hoc activities oblige them to travel outside Thimphu leading to difficulty in conducting VZ. Electricity and internet fluctuation in the CCs when they planned to conduct VZ also discouraged some of the MPs. Some even expressed that they feel uncomfortable sitting on a sofa and talking to the people who had walked to participate in the VZ and who are sitting on the floor. There are also MPs who did not feel any pressing need and the urgency to use VZ facility.


Tag: Challenges Parliament Country Support Platform Knowledge management Coordination Technical Support

13.

Administration with a copy to DLG. The Dzongkhag Administration will inform the Gewog who in turn will coordinate the Zomdu by inviting the interested participants within one week. The role of the CC Operators, with support from DITT, is to book the venue on the fixed date and arrange the facilities and provide technical support. 

When it comes to time required to coordinate a VZ majority had not responded (54.2% of MPs, 65.9% of Gups and 60.5% of CC Operators). This group is those who had not used the VZ facility even once. On the other hand, majority of the respondents who had used the VZ facility reported that the time required to coordinate is short.

Overall, the findings from the qualitative and quantitative data analysis show that the VZ facility has been underutilized the cause of which is mainly absence of advocacy, internet fluctuation and absence of clear guidelines.


Tag: Coordination

14.

4.5 Benefits of VZ

The analysis of both qualitative and quantitative data showed underutilization of VZ facility by MPs and constituents. For instance, only 19 of the 36 MPs, 19 of the 45 Gups and 34 of the 82 CC operators had used this platform. The KII and FGD with MPs as well as local leaders and CC Operators also indicate low frequency of VZ utilization.  

Grounded on these premises of frequency and number of MPs and constituents who used this platform, it can be concluded that there has been very little communication using the VZ platform between the MPs and the constituents. With very little communication and interaction, the MPs will neither be held accountable nor will they be able to create awareness of the roles and responsibilities of the parliamentarians. Similarly, the women participation will not be enhanced at this rate of utilization nor will the constituents be able to give their inputs in decision making. Broadly, the MPs and the constituents have not been able to benefit much from this initiative.


Tag: Parliament Communication Country Support Platform Knowledge management Operational Efficiency Awareness raising Technical Support

15.

While the opportunities that VZ initiative offers have not been harnessed, the KII and FGD as well as the quantitative data confirm that there this platform will bring both intended and unintended benefits to the Bhutanese citizens at large.

Thus, the general conclusion is that while people have not benefitted much as desired from the VZ facility, the respondents acknowledge that VZ platform will bring immense benefits to the MPs and the constituents. The survey respondents and the participants of the KII and FGD also expressed that besides the intended benefits, this facility will also have many unintended benefits mainly by advocating this facility to other sectors also.

Furthermore, VZ initiative demands coping with change of using information technology and we can expect only incremental change as has been the case with the VZ platform at this stage with 89CCs integrated with the facility.


Tag: Country Support Platform Technology

16.

4.6 Sustainability

The VZ project has reached only 89 CC centers in 89 Gewogs. There are still 116 Gewogs to be integrated with VZ facility. The CCs are managed by BDBL while at the moment the intended users are the MPs and the constituents. The VZ initiative is based on idea of using the existing space in the CCs and other facilities. The investment has been mainly on the procurement of Videoconferencing facilities and capacity building of the CC Operators, IT Managers of NC and NA, IT Officers of all the Dzongkhags and IT Officers of BDBL Regional Offices.


Tag: Sustainability Procurement Technology Capacity Building

17.

The KII with the Honorable Speaker of the Parliament, Secretary General of NA showed, Local leaders, BDBL Dy. Managing Director and the CC Operators show that there are no sustainability plans for now. The project is still with UNDP. However, once UNDP withdraws its support, all the respondents were of the view that VZ initiative can be sustained. For instance, Honorable Speaker of the Parliament shared that “VZ is for everyone. There are immense opportunities for VZ to be used for many other purposes. With time, people will start using it mainly driven by the need to use it. For this we need to create need and open up to other sectors as much as possible so that VZ platform is optimally used”. Similarly, Dasho Sangay Duba, Secretary General of NA said: “VZ is highly relevant to the Bhutanese geographical terrain which poses accessibility challenges. This facility can be an effective pathway of strengthening democracy. Thus, budget to ensure the sustainability of VZ will not be a problem as the intention is noble and the benefits that it give are immense...”

 


Tag: Sustainability Parliament Communication Technology

18.

6 Assessment of the Results Framework of the Piloting phase

A results framework had been developed towards evaluating the piloting phase of the VZ initiative. The table below provides an over view of the review comments for each indicative activities


Recommendations
1

Advocacy gap- only few MPs and constituents who had used the existing facilities integrated in 89 CCs.

2

Need gap- The MPs as well as the constituents did not feel the need to use the VZ facility

3

Monitoring gap- There has been no monitoring after integrating the 89 CC with the facility from all the stakeholders such as UNDP, Parliament and BDBL.

4

Guidelines gap- the guidelines of using the VZ is still in its draft stage. The participants of the FGD and KII and the local level and the CC operators shared that the absence of a guidelines was one reason that no VZ had been initiated from their end.

1. Recommendation:

Advocacy gap- only few MPs and constituents who had used the existing facilities integrated in 89 CCs.

Management Response: [Added: 2017/12/21] [Last Updated: 2021/01/10]

The management has taken up the need to advocate on use of VZ facility and in the 2018 AWP of Parliament, included plans on strengthening and enhancing the use of VZ through advocacy and demand generation. The Parliament plans to include an agenda on VZ during the induction programme for new Member of Parliaments.

Key Actions:

Key Action Responsible DueDate Status Comments Documents
Discuss during the Outcome 4 validation meeting
[Added: 2018/10/09]
Parliament, UNDP 2017/12 Completed
2. Recommendation:

Need gap- The MPs as well as the constituents did not feel the need to use the VZ facility

Management Response: [Added: 2018/10/09] [Last Updated: 2021/01/10]

The management has taken up the need to advocate on use of VZ facility and in the 2018 AWP of Parliament, included plans on strengthening and enhancing the use of VZ through advocacy and demand generation. The Parliament plans to include an agenda on VZ during the induction programme for new Member of Parliaments.

Key Actions:

Key Action Responsible DueDate Status Comments Documents
Discuss during the Outcome 4 validation meeting
[Added: 2018/10/09]
Parliament, UNDP 2017/12 Completed
3. Recommendation:

Monitoring gap- There has been no monitoring after integrating the 89 CC with the facility from all the stakeholders such as UNDP, Parliament and BDBL.

Management Response: [Added: 2018/10/09] [Last Updated: 2021/01/10]

The management agrees that there has been a gap. The management will inform the Parliament on this issue and encourage Parliament to take necessary action.

Key Actions:

Key Action Responsible DueDate Status Comments Documents
To be discussed with Parliament and share this evaluation report.
[Added: 2018/10/09]
UNDP 2018/01 Completed
4. Recommendation:

Guidelines gap- the guidelines of using the VZ is still in its draft stage. The participants of the FGD and KII and the local level and the CC operators shared that the absence of a guidelines was one reason that no VZ had been initiated from their end.

Management Response: [Added: 2018/10/09] [Last Updated: 2021/01/10]

The management has taken note of this recommendation. It was discussed that management will suggest/discuss with Parliament on finalizing the guideline.

Key Actions:

Key Action Responsible DueDate Status Comments Documents
To be discussed with Parliament and share this evaluation report.
[Added: 2018/10/09]
UNDP 2018/01 Completed

Latest Evaluations

Contact us

1 UN Plaza
DC1-20th Floor
New York, NY 10017
Tel. +1 646 781 4200
Fax. +1 646 781 4213
erc.support@undp.org