WELL - Resource Centre Network for Water, Sanitation and Environmental Health

Attempts to measure the health impact of water supplies and sanitation have a long and chequered history. Many of them have been made by amateur epidemiologists at the behest of the agencies funding the construction of the facilities, and with insufficient planning and rigour. Even some studies supervised by eminent specialists have produced almost useless or meaningless results, after taking years to complete and costing substantial sums of money. This unhappy experience led a panel of experts, convened in 1975 by the World Bank, to conclude that the Bank should not undertake any long-term longitudinal studies of the question. (1)

There were brief hopes during the 1980s, the International Water Decade, that a new technique, the case-control method, would provide a quicker, cheaper means of measuring the impact on diarrhoeal disease (2). However, several experimental studies of this type produced disappointing results, and it became clear that they suffered from similar shortcomings to studies of the more conventional design.

One review of the literature listed eight common errors found in health impact studies; one or more of these shortcomings was found in every one of the studies reviewed. (3)

Epidemiological studies depend on the intervention studied (in this case, water and sanitation) and an outcome measure (the health impact). Part of the problem is the nature of the intervention. The ideal way to measure the impact of any health intervention, the double-blind, randomised, controlled trial, is not feasible for water and sanitation. There is no placebo for a pit latrine. Moreover, the unit of intervention usually has to be the community, rather than the household. Besides, it is almost impossible to allocate water supplies and sanitation at random - ethically, politically and practically.

The principal outcome is diarrhoeal disease; by any reckoning, more than 90% of the health benefits of improved water supplies and sanitation arise from reduced diarrhoeal illness, most of it in children less than five years old. This raises other problems. Diarrhoea is caused by a wide variety of micro-organisms, transmitted by a wide range of different routes. Water supply and sanitation affect only some of these. For these reasons, well-designed water supply and sanitation interventions typically reduce diarrhoea incidence by about 25%. (4)

Confounding

An epidemiological study might well find that television ownership was associated with reduced incidence of diarrhoea, but of course this does not mean that televisions prevent the disease. Rather, people who own TVs are likely to be richer and better-educated than their neighbours, and these and other factors help to protect them from illness, causing an apparent association. This is called confounding and is a major problem in health impact studies. For example, people who own latrines may have less diarrhoeal disease, but this does not necessarily prove that the latrines are the cause.

With more than 3 million children dying of diarrhoeal disease each year, a 25% reduction is a very substantial public health benefit; however, with many other factors (education, nutrition, climate) also affecting diarrhoea rates, a percentage reduction of only 25% is extremely hard to measure reliably. Moreover, if detected, it is very difficult to attribute a reduction unambiguously to improved water and sanitation.

For these reasons, a review of the published and unpublished results of the best health impact studies of the Water Decade concluded that health impact studies are not an operational tool for project evaluation or 'fine tuning' of interventions. The results are not only unpredictable; they frequently offer no firm interpretation. (5)

Moreover, by their very nature, epidemiological studies have little power to diagnose deficiencies and suggest improvements, a normal requirement of operational project evaluations. If no health impact is found, it could be because the water and sanitation facilities are not functioning, or because they are not used correctly. Functioning and use are the first questions to ask in any evaluation of a water and sanitation project. (6) Whether or not a health impact is found, the study itself does not offer any guidance on how the project, and hence the impact, might be improved.

What we do know from the existing literature on impact studies is that in those cases where a significant health impact was found, the provision of water supply or sanitation had been accompanied by improvements in hygiene. 'Hygiene' in this context refers to practices such as the washing of hands, food and utensils, or the disposal of children's stools. It may be promoted by better access to water and sanitation, or by hygiene education. Improvements in hygiene may be reflected in increased water consumption. If no such change in behaviour results from improved water supply or sanitation, the only benefits which are likely to occur are those stemming from improved water quality; in many settings, these are relatively minor or even negligible.

Minimum Evaluation Procedure

Health improvements are only the culmination of a long chain of cause and effect. This runs from the original construction of the water supplies or sanitation facilities, through their operation and hence their use, permitting changes in hygiene behaviour and thus the prevention of disease transmission. The principle of the WHO Minimum Evaluation Procedure is to examine the intermediate links in the chain - functioning and use. Hygiene behaviour is another such link.

Instead of attempting to measure disease rates, studying patterns of hygiene behaviour has far greater diagnostic power, in terms of indicating opportunities for project improvement. Since it is further back up the causal chain, it is easier to attribute to the project intervention. It is also quicker and cheaper than epidemiological studies. It can also be done at the project design stage. This will not only help to establish a baseline yardstick against which to compare evaluation results, but also improve project design. A convenient user-friendly manual is available (7), and so is a more detailed account with case studies. (8)

World Bank (1976) Measurement of the Health Benefits of Investments in Water Supply. Report No. PUN 20. Washington DC: The World Bank.

Briscoe J, Feachem RG and Rahaman MM (1985) Measuring the impact of water supply and sanitation facilities on diarrhoea morbidity; prospects for case-control methods. Geneva: World Health Organisation, Environmental Health Division.

Blum D and Feachem R. (1983) Measuring the impact of water supply and sanitation investments on diarrhoeal diseases: problems of methodology. Int. J. Epidemiology, 12, 357-365.

Esrey SA, Feachem RF and Hughes JM. (1985) Interventions for the control of diarrhoeal diseases among young children; improving water supplies and excreta disposal facilities. Bull WHO, 63 (4) 757 - 772.

Cairncross S. (1990) Health impacts in developing countries; new evidence and new prospects. J. Inst. Water & Environ. Man. 4 (6) 571-577.

WHO. (1983) Minimum Evaluation Procedure (MEP) for Water and Sanitation Projects. Geneva: World Health Organisation.

Almedom AM, Blumenthal UJ, Manderson L. (1997) Hygiene Evaluation Procedures; Approaches and Methods for Assessing Water- and Sanitation-related Hygiene Practices. London: Intermediate Technology Publications.

Cairncross S, Kochar VJ. (1994) Studying Hygiene Behaviour; Methods, Issues and Experiences. London: Sage Publications.

WELL FACTSHEET