Appendix 1: Questionnaires Used in this Evaluation (1)

Questions for Social Watchers

Background Information

How do you define your organization, what are its main activities, what is the profile of active members and what constituencies to you work with? 

Who do you work with nationally or regionally on issues related to the Social Summit and Women’s Conference agendas?

How would you define Social Watch? When, how and why did you join Social Watch?

 What is your expected outcome of this evaluation process?

 NGO Monitoring Process

;What monitoring and networking strategies have you developed to monitor implementation of the Social Summit and Women’s Conference commitments? What factors have contributed to, or impeded the success of these strategies?

How has the Social Watch strengthened the NGOs and/or other civil society actors, with respect to their capacity to monitor progress at national and international levels? How do you link Social Watch activities with other NGO lobbying initiatives?

 How has Social Watch been used as a mobilizing tool by grassroots organizations?

How has Social Watch contributed to the opening of political spaces for NGOs and/or other civil society actors, in particular for raising issues relating to social development and gender inequity?

Have you achieved a balance between monitoring the Copenhagen and Beijing commitments?  If not, where is emphasis placed? Does the methodology used to monitor the Social Summit and Women’s Conference agreements need to be revised?

What is Novib’s role in mobilizing interest in Social Watch activities in your country or region?  How is this different from or complementary to the Social Watch secretariat’s role in galvanizing interest?  What lessons have been learned from this experience?

 Impact on National and International Processes and Policies

What changes in political process and social development policies in support of the implementation of the commitments made in Copenhagen and Beijing have taken place in your country over the past five years?  What changes do you hope to see happen in the future? What has been the qualitative or quantitative impact of these policy changes?

To what extent have national NGOs engaged in debates with their respective governments on these social development policy issues?  How do national political environments influence your work?  What are the factors that contribute to or impede changes in current policies? What factors would ideally contribute to create an enabling environment for your activities?

What policy changes can be linked to Social Watch efforts and/or NGO advocacy efforts at large?

 Annual Report Process and Content

When did you begin contributing to the Social Watch annual report?  (Please indicate the years that you submitted reports).  What is the process used to develop the annual report and how has it evolved over time? What are the challenges to ensuring widespread and representative participation? Has this process contributed to promoting interaction between organizations and social movements?

How has the Social Watch annual report been used in your work?  Do you use the report in its entirety or selected parts?  Do you use the English version or it is translated into national languages?  How widely is it disseminated?  Who uses this report on a national, regional and international level? (government officials, NGOs, media, educational institutions?) Are activities organized parallel to its distribution to discuss the report's findings and help render the content tangible to a broader population?

How do the reports respond to the needs of your organization and citizens of your country?  Have the reports enhanced your national platforms/reports?

Are the indicators used to measure progress appropriate? Is there a balance between “qualitative” and “quantitative” evaluations?

Does the report maintain the appropriate balance between South/North and East/West? Are some regions under-represented and others over-represented? Is there a balance between country-specific and international evaluations?

 Organizational Structure

How do you define the roles of the Social Watch secretariat and Novib? How effective has the operational structure of Social Watch been for your work?

What are the roles of the coordinating and regional committees?  What is your relationship to these committees? What have been the challenges linked to Novib’s role on the Coordinating Committee, particularly given its dual role as donor and participant? What is Novib’s role in mobilizing European donors, development agencies and social movements in the Social Watch initiative?

Is there a North/South balance in Social Watch activities?

What are the major advantages of and challenges to international networking?  How can the Social Watch initiative better respond to your needs?

 Funding Strategy

How are national and regional-level Social Watch activities funded? Do you receive funds from Novib? Which departments do you relate to?  To what extent is the Secretariat involved in supporting your fund-raising efforts?

What type of technical, non-financial support has Novib provided?

How do you plan to financially sustain your Social Watch activities in the future?  Which aspects of the work is it relatively easy to raise resources and which are difficult?

 Other

What are the key lessons that you have learned through your work with Social Watch? What would you do differently in the future?

Please feel free to write additional comments/ideas that you would like to share with us.

 Questions for Secretariat

 Secretariat’s Role

 The founding of Social Watch was based on the following assumptions:

 i) the commitments made in Copenhagen and Beijing are sufficiently taken serious nationally and internationally to provide for a useful advocacy pressure point, ii) NGOs are legitimate, credible and capable actors to take up this advocacy and iii) information will be the key tool to achieve results, notable by establishing “social watch” as a self-standing, recognizable and credible source of information.” 

Are these assumptions still correct?  How have these assumptions changed over the past five years if at all?  If so, how have the Social Watch’s activities been modified?

How did Third World Institute become the Secretariat for the Social Watch initiative? 

What are the challenges to balancing your different roles as participant, advocate and coordinator?

How has your mandate evolved since 1996?  How have your activities changed according to the needs and requirements of Social Watchers?  How have the related needs of the Secretariat changed?

How many full- and part-time staff have been/are assigned to this work?  What percentage of the total organizational budget and human resources have been/are devoted to Social Watch?  

What is the annual budget of Social Watch (please provide break down by year and category)? Who are the major donors?  What efforts have been made to diversify the donor base? How are these efforts coordinated with Novib? What are the challenges to involving other donors in the Social Watch initiative? 

How are decisions concerning funding made?  Are decisions to involve Novib-funded NGOs in Social Watch coordinated with the secretariat? 

How has the method of funding the Social Watch impacted your internal functioning?

 Annual Report Production (process and content)

Describe the process of producing the Annual Report? How has changed over the past five years? What and where are the bottlenecks in the process? How can the process be further strengthened? 

How are decisions made with regard to the focus and content of each report? How are the technical briefs for NGO country reports developed? What are the strengths and weaknesses of the content of the Annual Report over the past five years?

What process has been put into place to develop indicators?  Who has been involved?  How do you know that the indicators you have established are useful?

Does the methodology used to monitor and supervise the WSSD and WCW agreements need to be revised?

Are the indicators used to measure progress appropriate? Is there a balance between “qualitative” and “quantitative” evaluations? 

Question re the Fulfilled Commitments Index?  Equity Diamond?

How gender-sensitive are the indicators that are being used?  Is it important to use gender-disagregated data?

How are country reports solicited? What mechanisms are used to mobilize and sustain NGO involvement in the Social Watch initiative? How is this different from or complementary to Novib’s role in promoting interest and funding NGOs? What factors have contributed to or inhibited a more organic participation of NGOs supported by Novib (and others) in the Social Watch process? What challenges has the Secretariat faced and what lessons have been learned from this experience?

What are the challenges to ensuring regular high quality reports, based on broad-based participation and with an impact in terms of dialogue with the state and policy reforms?  What are some of the reasons for weak national processes? What would be required to strengthen these in the future?

How are country reports reviewed for editorial and political content? How can this process be improved?

How many copies of the report are printed and in what languages?  How has this number changed over the years? How are they distributed and what are the associated costs?  What kind of indicators show that the report is being read?  Used?

What are the challenges to involving regions that are weakly represented?

What are the strengths and limitations of launching the Annual Report at the UN Social Development Commission?  What efforts have been made promote the report at the international level beyond the Social Summit process?

What feedback have your received from individuals who use these reports?

 International Advocacy Role

What impact has Social Watch made at the international level (i.e. at the UN, EU and the World Bank)?  How consistent has this been over the past five years?

How has the Secretariat’s role of developing advocacy positions and strategies at the international level evolved since the inception of Social Watch? For example, what are the challenges in leading the Development Caucus?

Over the past five years, what are the main issues and at what levels has the Secretariat focused its advocacy efforts? What are the main advocacy tools/strategies employed? What are the major gains or losses resulting from this advocacy and at what levels? What kind of advocacy effort How have the collective Social Watch-coordinated proposals/language of Social Watch been used in international documents? 

 Communications [or Public Awareness] Role

What is the media and internet strategy to inform and engage the public in the key messages from each of the Annual Reports?

How are your monitoring manuals distributed and used?

How is the Social Watch web-site being used?  What electronic conferences have you initiated?  Can annual reports be down-loaded?  If so, in what languages?

What other advocacy tools – such as position papers, public seminars, fact-sheets, etc – have been developed through your work?  How do these complement your other activities? How do these tools serve to link international and national advocacy?

What relationships has Social Watch developed with the media on national and international levels?

How is the on-line social development indicators database being used?

 Other

What are the key lessons that you have learned through your work with Social Watch? What would you do differently in the future?

Please feel free to include additional comments/ideas that you would like to share with us.

 Questions for Novib

 Executive Role

Where did the Social Watch initiative originate? How was Novib involved in creating it and what were your objectives in doing so? How were different departments involved in this process and how were their objectives different (i.e. between the advocacy and project departments)? How do these objectives fit into the overall mandate and policies of Novib as a donor and an advocate?

What were Novib’s main activities during the period of 1995-1996 when you housed the Social Watch secretariat? Why did Novib take on this executive role? How were the roles of donor and manager of the secretariat reconciled?

Why was the Third World Institute requested to play a leadership role in the Social Watch initiative? How was the transition of the Secretariat role to the Third World Institute handled?

 Donor Role

How much financial support has Novib contributed since its inception in 1995 to Social Watch?  Do you provide the Social Watch secretariat with an annual figure or are monies ear-marked – in the case of the latter please provide line items for funds (break down per year, global amount and ear-marked figures to the Social Watch secretariat, and monies to national NGOs and other groups per country and region)? Please also show the total figures as a percentage of Novib’s overall funding budget.

What type of technical, non-financial support has Novib provided for Social Watch?

How do the various program-oriented departments of Novib coordinate their Social Watch activities (i.e. to make decisions on the Social Watch project and communicate with Third World Institute and other social watchers)?  How are activities coordinated between the program departments, the special programs and your lobbying departments?

What is Novib’s role in mobilizing NGO interest in Social Watch activities?  How is this different from or complementary to the Social Watch secretariat’s role in galvanizing interest?  What challenges has Novib faced and what lessons have been learned from this experience?

What factors have contributed to or inhibited a more organic participation of NGOs supported by Novib (and others) in the Social Watch process? How are decisions made to involve Novib-funded NGOs in Social Watch and what criteria is used to do so?

How does Novib coordinate its mobilizing and funding efforts with the Social Watch Secretariat?

How has Novib’s role as a donor impacted the efficiency and efficacy of the follow-up to the Social Summit and Women’s Conference agendas? How has this role affected Novib’s position among the European donor community? What other donors have been involved in supporting the Social Watch initiative?  Has Novib promoted donor diversity?  What are the challenges to involving other donors in the Social Watch initiative?

What lessons has Novib learned from its experiences providing financial support for an initiative in which it also has vested interest?

 Advocacy and Participant Roles

What have been the main responsibilities of the Coordinating Committee?  What have been the challenges linked to Novib’s role on this committee? How has Novib sought to reconcile the inherent difficulties linked to its dual role as donor and advocate?

How has Novib’s role as an advocate evolved since the inception of the Social Watch? How has this role impacted the efficiency and efficacy of the follow-up to the Social Summit and Women’s Conference agendas? How has it affected Novib’s position among Dutch and European social movements at large?

Over the past five years, what are the main issues and at what levels has NOVIB focused its advocacy efforts? What are the main advocacy strategies employed? How has Novib’s consistent engagement with Social Watch enhanced such advocacy efforts at the national, European or international levels? What are the gains or losses resulting from this advocacy and at what levels?

How has Novib involved Eurostep in the Social Watch initiative? What are the challenges involved in this effort?

What is the budget to support Novib’s own advocacy and participant roles in Social Watch?

How has Novib benefited from Social Watch materials (annual report, guide, etc.) in its work in the Dutch and European context?

 Other

How do you envision Social Watch’s role in the future?

Please feel free to add any additional information, lessons learned, etc. that would help assist SAGE in its evaluation.

 Questions for UN Officials

How do you use the Social Watch annual reports?  How is it used in the UN system?  How effective has it been in influencing the work of the UN? What are its strengths (compared to other NGO documents)? How could it be improved?

Currently Social Watch reports are pegged to the Social Summit review process.  Social Watch is concerned about the weakness of this process and are thinking about the future and how to make this report and its analysis more visible in other arenas within the UN system? What are your suggestions of useful entry points for them?

Part of the critique of in Social Watch reports focus on economic policies of the World Bank, IMF and WTO.  Social Watch realizes that they can’t focus their advocacy solely at the UN and want to engage in policy discussions with these institutions but want to figure out how to do this through the UN.  In this regard it is important for Social Watch to understand how UN is working with the WB, IMF and WTO.  What are the possible arenas for NGOs to engage in these discussions?

How can the ideological perspective of Social Watch, which may be different from that of the UN, be given more voice?

Given the success of the UNDP Human Development Report in developing indices and data that are widely used, what are future plans for improving the measurements and coming up with new ways of how we look at the world. How can Social Watch be involved in this process.

[1] We developed four different questionnaires to guide our evaluation; these were used with: Social Watchers, the Secretariat, Novib and UN officials.