Methodology

Four sets of tables were constructed for this edition to follow up on the indicators for commitments assumed by governments at the World Summit for Social Development and the Fourth World Conference on Women. This section has five subsections. The first indicates the sources of data used and the general methodology employed. The second makes general comments on the tables, pointing out some particulars of the methodology applied and providing information on the changes introduced this year in their presentation. The third refers to the tables on Changes in the Situation of Women and Changes in Social and Defense Spending. The fourth section explains and comments on the timeframes prepared on this occasion. Finally, the fifth details the procedures used in preparing the Table of Fulfilled Commitments and the treatment of dimensions that compose it.

SOURCES AND GENERAL METHODOLOGY

This edition of Social Watch maintains the improvements incorporated last year based on the contributions, suggestions and agreements of a workshop on the use of indicators in following up on social development commitments.1 It introduces new changes in the presentation of the tables, which we feel will facilitate use by our readers.

Although the difficulties we had from the outset in obtaining and managing data persist,2 we have mantained the criteria already adopted. Thus we continue using the most recent source provided by any of the most recognised international organisations; we take as reliable the data provided by them, except where error is obvious, or where different interpretations are warranted in response to different causes.

As before, we have applied the following rules:

As regards sources:

if there are alternative sources, we chose the recognised source with the greatest authority on the subject of the data in question;

in the event that the most recent data is not included in these sources, we chose, among the other alternatives, the «secondary» sources that for previous data showed the best systematic match with the recognised sources on the subject;

when we could not apply any of the foregoing criteria, we chose the source offering the largest coverage of countries.

As regards handling of the data for calculating progress indices:

when the data available refer to an interval (eg, 1990_1994) and not to a single year, we adopted the recommendation of centering the data in the middle of the interval (in this example, 1992) for calculation purposes;

in cases where the goal is not specified with a numerical value in the commitments, we assumed the specific criteria set forth, as applicable, in the following point;

in all cases where goals refer to reductions in values of indicators, we have managed them in the same way as achievements, for the sole purpose of calculating our index. We did this because we are inclined toward a procedure that is less demanding, or more generous, in rewarding progress, while being more stringent with setbacks.

As indicated in previous editions, any progress index, like those appearing here, implies adopting a standard _that acts as a «must»_based on measurable progress. We pointed out that each of the specific indicators can evolve in different ways. We recognised that these benchmark evolutions should be provided either by specialised entities, or, in the absence thereof, inferred based on some previous research, like an analysis of time series, for example. We recognised, consequently, that while it was desirable to offer exhaustive and rigorous treatment, for the majority of the indicators associated with the commitments this was not available.

Moreover, given that, in many cases, we also lack the number of observations required to generate more accurate evolutive models, our option, irremediably, was to find a simple and understandable way to assess progress toward achievement of the goals. Thus we chose to «impose» on the indicators a simple, uniform, relatively undemanding, benchmark model for assessing changes over time or making comparisons of evolutions between countries. Under such conditions, as we have always said, the conclusions deriving from the methodology adopted here do not seek to be, and cannot be, interpreted as exhaustive or categorical assessments. They are instead an approximation or indicative guide. Our progress index thus qualifies the value observed for the variable as «ahead», «on» or «behind» schedule, with regard to the expected value according to the evolutive option adopted.

For following up on commitments we maintained this basic procedure, which provided us with an index of fulfilment that reflects the degree to which countries have progressed toward achieving proposed goals. A change this year is that instead of publishing a column in the tables indicating our progress index, we have rescaled progress indices by tranches (we converted the indices to a reference scale from 1 to 5) and represented them in the tables in a column titled «Progress and regressions». This change is in keeping with our concern to simplify reading and eliminate the false precision suggested by a numerical progress index, by means of a set of icons that reflect transformation.

Rescaling was done by translating those numerical values into categories which, according to the progress index, suggested:

1. major regressions, indicated by the icon associated with «Significantly going backwards»;

2. minor setbacks, whose negative magnitude is equivalent to the progress they should have made, indicated on the tables by the icon associated with «some regression»;

3. no significant changes, indicated by the icon associated with «standing still»;

4. progress, but not sufficiently rapid to reach the goals by the agreed dates, indicated by the icon associated with «progressing but not enough»;

5. progress leading to fulfilment of the goal or at a pace that will allow it to be fulfilled on time, indicated by the icon associated with «rapidly progressing or goals already achieved».

Additionally, using icons in the column «Progress and Regressions», countries that met their goal by 1990 are ranked into three subgroups: goal achieved in 1990; countries that achieved their goal in 1990 and are making progress; countries that achieved their goal in 1990 but show backsliding.

We also maintained the criterion of noting on the tables the value the indicators should have in 1998, or the «suggested value», so that when more recent information is available, it will be feasible, using the methodological assumptions employed, for those interested in following up on the indicator in question, to make their own comparisons and assess whether progress is being made at a pace that will make it possible to achieve the goal by the year 2000.

THE «EQUITY DIAMOND»

The national reports are accompanied by a figure that presents the situation regarding four indicators: infant mortality, literacy, Gini Index, and Gender Development Index (GDI) of the UNDP. The figure makes it possible to compare a rhombus or diamond based on the respective average data of the countries in a region (or in an economic group) with another figure based on the indicators of the country in question. On each axis, if the country's value is further from the centre than the value for the region or group, this indicates a situation better than the average of the neighbouring countries (or those in the same economic group); if it is closer, the opposite is indicated. In synthesis, the larger the country's own diamond, the better the situation measured in terms of these four indicators.

COMMITMENTS AND FOLLOW_UP

The report presents 16 charts that illustrate the evolution of countries with respect to the commitments assumed by governments at the WSSD and the FWCW. Social Watch chose 13 commitments as the most relevant among those that can be measured quantitatively. Although it has not been possible to follow up on the thirteenth _«improving the availability of economical and appropriate housing for all, in accordance with the World

Housing Strategy for the year 2000»_ because appropriate indicators were not available for a sufficient number of countries, in the other cases we have chosen one or more indicators that illustrate the commitment. The pertinence of the indicator chosen varies in each case. We chose those indicators most in line with the dimensions we were seeking to measure that were available for a sufficient number of countries.3 A table was updated for each commitment, showing the value of the indicator(s) for the year 1990 (or the closest one possible, if information is not available for that date), the value of the last year available, the value that should be reached by the year 1998 in order to achieve the goal, and the goal to be reached by the year 2000. For commitments referring to goals subsequent to the year 2000, we composed the series of values due for that lapse of time and assumed the value generated as the 2000 goal, so as to adjust all the runs to one decade.

In the case of commitments with goals not associated with concrete figures, we took those from other summits where relevant.4 If reference was to universal access, the goal was assumed to be access by 100% of the population in question.

CHANGES IN THE SITUATION OF WOMEN AND IN SOCIAL AND DEFENSE SPENDING

Two tables are provided regarding changes between 1990 and 1995_96 in the Situation of Women and in Social and Defense Spending. The first table is based on three indicators: women's life expectancy, women's literacy, and girl's net enrollment in primary school. The second is based on spending for health, education and defense as percentages of the Gross Domestic Product (GDP).

For the table on women, indicators were treated the same as in the commitment tables, but based on gender data. This table has three columns whose significance is analogous to those for «Progress and Regressions» on tables for commitments 2, 11 and 1b, but instead of the icons used there, we opted to present the categories rescaled by colours.

In the table on Changes in Social and Defense Spending, we chose to qualify the changes observed based on mean behaviour of the group of countries for each of the three indicators. Thus the rankings shown there are:

Health: reductions in spending equal to or greater than a percentage point of product are ranked as a «significantly going backwards»; reductions of up to 1% are considered «some regression»; those that did not change or changed a tenth of a percent are ranked as «standing still»; «some improvement» corresponds to increased spending on health by up to 1%; and finally, «significant progress» covers increased spending by 1% or more;

Education: spending cuts greater than or equal to 0.4% are ranked as «significantly going backwards»; cuts of lower magnitude are considered «some regression»; those where this relative datum did not change or increased up to 0.2% are considered «standing still»; increases up to 0.3_0.5% are considered «progressing» and, finally, increases exceeding one_half of one percentage point are considered «significantly progressing»;

Defense: those increasing military spending by more than one_half of one percentage point are ranked as «significantly going backwards»; increased spending by 0.3% or less is considered as «some regression»; those with no relative change or a reduction of up to 0.3% are ranked as «standing still»; those cutting spending by 0.4% to 0.5% are considered to show «some progress»; and finally those cutting spending by 0.6% or more are considered to have made «significant progress».

TIME IS LIFE

As has been the tradition in our reports, we provide a chronometer for progress made towards fulfilment of commitments. This year we have chosen to present three chronometers that correspond to adult literacy, reduction of infant mortality under five years of age, and girls' net enrollment in primary school. The corresponding goals are: reduction of the adult illiteracy rate to half the 1990 rate; reduction of the infant mortality rate under five years of age by one_third of the 1990 level or to 70 per thousand live births if this figure is lower; provision of universal access to basic education for school_age children. In the latter case, the goal is assumed to be 100% of net enrollment of girls in primary school.

The procedure applied is the same as in previous years: progress indices are calculated and they are transformed into years according to the progress reflected for the 1990_2000 tranche. In the pertinent graphs, the countries located in 1995 according to the chronometers for literacy and net girls' primary school enrollment are those that are on time for achieving the goal and, if they continue at their current pace, will reach the goal by the deadline. The ranking is analogous for those located in 1996 per the chronometer for infant mortality under five years of age.

Countries that have performed best or are ahead of the rate required are positioned between 1996 and 2000, in the case of the first two chronometers, and between 1997 and 2000 for the third.

Those that have made progress but not at the expected pace, and hence will need to make additional efforts to reach their goals by the end of the millennium, are located between 1991 and 1994 for the first two chronometers, and between 1991 and 1994 for the third.

Countries that at the date of the latest data available show the same rate as in 1990 are positioned at 1990, since they have not made progress.

Countries grouped at dates prior to 1990 have had regressions.

Finally, countries set at or to the right of 2000 are those that have fulfilled their goal in advance or had already achieved it in the year 1990.

CHART OF FULFILLED COMMITMENTS

In this edition of Social Watch, we once again publish the table indicating progress toward the goals. This year, we have extended the dimension on political will. The latter, which previously only included one indicator regarding Official Development Aid (ODA), now includes two more indicators: Reduction of Military Spending and Anti_Poverty Plans. We would have liked to include quantitative indicators on poverty and inequality, but the paucity of information for recent periods provided by international organisations prevented this.

To construct the Chart of Fulfilled Commitments we performed the following steps:

I. We took the numerical values of the progress indices appearing in the columns «Progress and regressions» of the different commitment tables: C1a _ percentage of children reaching 5th grade; C1b _ primary school enrollment ratio (net); C2 _ life expectancy; C3a _ infant mortality rate (per 1,000 born live); C3b _ under_five mortality rate; C5 _ daily calorie supply; C6 _ percentage of under_five children suffering from severe and moderate malnutrition; C7 _ percentage of population with access to health care ; C8a _ pregnancies attended for every 1,000 live births; C8b _ percentage of births attended; C10 _ percentage of children under one year fully immunized; C11 _ literacy rate; C12a _ percentage of population with access to sanitation; C12b _ percentage of population with access to safe water. Certain commitment tables were omitted because data was not available to calculate progress and regressions.

II. The indicators where grouped as follows:

C1a, C1b and C11 in the column titled Literacy and Basic Education

C3a, C3b and C10 in the column titled Child Health

C5 and C6 in the column titled Food Security and Child Nutrition

C8a and C8b in the column titled Reproductive Health

C2 and C7 in the column titled Health

C12a and C12b in column titled Access to Safe Water and Sanitation

III. The Chart of Fulfilled Commitments expresses in different colours the average value of the indicators chosen. These values correspond to:

1. Significant regression

2. Regression

3. Standing still

4. Progressing but not enough

5. Progress at a good pace or goal achieved

«Progress at a good pace or goal achieved» applies to countries that had reached the goal in 1990, those that reached it subsequently, and those that, if they keep up their current pace, will reach the goal on time or early.

«Progressing but not enough» applies to countries whose index is positive but less than required to meet the goal on time.

«Standing still» applies to countries that have not shown changes in the indicator (or have shown quantitatively insignificant changes).

«Regression» applies to countries with a drop in the index up to the negative value equivalent, in absolute terms, to the value of the progress they should have made. Those who fell behind even further are put in the category of «Significant regression».

Countries showing data on the commitment tables for less than four of the aforesaid indicators were excluded from this table.

IV. For Military Spending, we assigned the same values on a scale from 1 to 5, mentioned above, as follows: 1 _ those increasing defense spending by one_half a percentage point or more; 2 _ those increasing spending by less than 0.5%; 3 _ those with no change; 4 _ those cutting spending up to one percentage point; and 5 _ those cutting spending by more than 1%.

V. As regards the commitment of developed countries to allocate 0.7% of their gross product to Official Development Aid (ODA), since there is no plan established for fulfilment of this goal, progress or setbacks were assessed based on the variations from 1996 to 1997. This time we considered the commitment of 0.7% together with the change in real terms in this aid, giving a 5 to those allocating 0.7%, 4 to those meeting the percentage but reducing aid in real terms, 3 to countries that, while not meeting 0.7%, increased aid both in relative and real terms (we include here those OECD countries not belonging to ODA), 2 to those reducing aid in percentage terms, but increasing it in real terms, and finally 1 to those decreasing aid in both aspects.

VI. As regards Anti_Poverty Plans, we gave a 5 only to those that are implementing a plan. To countries with a plan being designed or prepared, and to those who do not have a plan but claim to address anti_poverty objectives within their government policies as a whole, no value was assigned but they are mentioned on the table.

VII. On the basis of these nine columns we obtained a simple average of the indices available for each country, which made it possible to group countries in a similar situation. The final ranking was achieved by changing the individual positions within each group by applying the following criteria:

a) For countries having the same average, the position was improved if they had less gaps in information relative to those with more gaps in information on commitment indicators appearing in the first six columns.

b) We then did a second repositioning among countries that continued to be in the same place, by rewarding greater frequency of goals achieved or progress at a good pace in the first six columns, and penalising those significantly going backwards or those with some regression.

c) Finally, for the last differentiation of countries that were still tied in the ranking, we rewarded countries with goals fulfilled and penalised countries that had fulfilled their goals in 1990 but had had setbacks.

Thus we obtained the ranking we show in the Chart of Fulfilled Commitments. According to overall progress on the selected indicators, we subdivided this into three groups: countries where the average pace of progress will allow to achieve of goals by the year 2000; countries where the average pace is too slow to achieve the goals on time; and countries whose situation is worse than it was in 1990.

1 Workshop organized by Instituto del Tercer Mundo and held in Montevideo in August 1997, with the participation of specialists from the United Nations, UNICEF, NOVIB and the social sciences team that works with Social Watch Secretariat. In addition, other modifications were adopted based on subsequent suggestions remitted by the United Nations Development Program. As we indicated then, «the final responsibility for opinions, decisions and possible errors pertains, however, to Social Watch Secretariat and not to the participants or their institutions.»

2 Like those associated with the non_homogeneity of dates for which information is available, and even the existence of significant differences between the statistics provided for the same year by different sources.

3 Those indicators and the respective tables continue to be the same as in the previous edition.

4 As in the case of achieving food security, where goals were adopted based on proposals by the World Food Conference, in 1996.