ABSTRACT

THE FACTS

WHAT IS HYGIENE PROMOTION

HYGIENE AND CHILDREN

WHY EVALUATE

BASELINE STUDY

PLANNING

TECHNIQUES

SAMPLING AND STATISTICAL METHODS

COMMUNITY INVOLVEMENT

STUDY TEAM

PARTICIPATORY EVALUATIONS

PRESENTING THE FINDINGS

A CASE STUDY

REFERENCES

 WELL INTRANET

Go back

WELL FACTSHEET

Evaluation of hygiene promotion

Author: Ann Maria Mooijman, December 2003

Quality Assurance: Christine van Wijk and Sandy Cairncross

Abstract

After introducing the reasons for and meaning of hygiene promotion, this fact sheet presents an overview on the evaluation of hygiene promotion programmes. Readers will also find one example and some key documents and contacts.

Introduction

Unhygienic environments place children at risk of death. Some 1.5 million deaths of under-fives a year are caused by diarrhoea¹, deaths which can be avoided if diarrhoea in this age group is prevented or appropriately treated². Diarrhoea also stunts children's physical and intellectual growth and is a huge cost factor. Overall, 2.2 million people died because of diarrhoea in 2002. Hygiene promotion is an important instrument to reduce diarrhoea.

The facts: hygiene and health

Whether hardware facilities are available or not, the best way to protect people, and in particular children, from diarrhoeal diseases is a more hygienic environment and behaviour.

In the 1990's, Steven Esrey found that:

• safer excreta disposal led to a reduction of child diarrhoea of up to 36%,

• better hygiene through handwashing, food protection and domestic hygiene brought a reduction of 33%,
• improved water supply led to a reduction of only 15-20%

He also showed the importance of synergy between technical solutions and improving behaviour.

A recent review study³ showed that washing hands with soap can reduce the risk of diarrhoeal diseases by 42–47%. Interventions to promote hand washing might save a million lives a year. The result of these studies is encouraging for governments, donors and NGOs willing to undertake more hygiene promotion programmes.

There are also other powerful arguments for improved hygienic conditions. They can lead to⁴:

• Heightened personal dignity and national pride

• Savings in health costs

• Higher worker productivity

• Better learning capacities of school children

• Strengthened tourism

• Increased school attendance, especially by girls

What is hygiene promotion?

According to UNICEF (1999), hygiene promotion is a planned approach to preventing (especially) diarrhoeal diseases through the widespread adoption of safe hygiene practices. It begins with, and is built on what local people know, do and want.

Effective hygiene promotion reduces the main risky hygiene practices and conditions for women, children and men. It does so in a measurable way, to a significant level, in a pre-set period and with available resources. This is not only important for the development of hygiene education materials, but also for the design and evaluation of the promotion programmes and facilities.

Hygiene and children

Why evaluate programmes?

In general, evaluations of hygiene promotion programmes are carried out for the following reasons:

• To show the beneficiary communities the impacts and improvements made so far and identify hurdles still to be taken (evaluation as motivator)

• To show programme managers and staff the strength and weaknesses of the programme and show where adaptations are necessary (evaluation for programme improvement)

• To show policy makers the impact of the programme with the aim to get them more involved; through allocation of more resources, expansion of the programme or the lifting of hurdles in the legislation (evaluation for advocacy purposes)

• To get objectively measured information for sharing with peer experts (evaluation as scientific justification)

• To show donors the impact of the funds provided or to raise funds for future similar activities (evaluation for fund raising and justification of the use of funds)

Because of the complexity of the different expectations and the different target groups, it is important to design a hygiene promotion evaluation systematically.

Baseline Study

A hygiene promotion programme consists roughly of three phases: 

1. an assessment and planning phase;

2.  an implementation phase; and

3.  an evaluation phase.

 Undertaking an initial assessment with a baseline study is an essential condition for the proper planning and evaluation of hygiene promotion programmes. The study helps to analyse the existing problems concerning hygiene and sanitation in the programme area, while taking into consideration the different age-groups, sexes, ethnic groups and social classes.

Typically such an analysis exists of the collection of the following information:

• Official health statistics related to sanitation- and water-related diseases

• Capacity of health and hygiene professionals

• Existing financial allocations for hygiene promotion

• Responsible professionals and policy makers

• Risky practices/behaviour

• Risky environments

• Main risk groups

• Main communications channels

• Main motivators for behaviour change

Once the above information has been collected and analysed, a proper basis for programme planning and future M&E (Monitoring and Evaluation) systems has been laid.

Planning an evaluation

The design of the evaluation evolves around the following questions:

• What is the purpose and focus of the evaluation?

• What types of information have to be collected?

• Which evaluation techniques are most appropriate and feasible?

• What is the TOR for the study team including its composition?

• What will be the sample size?

• What is the timing?

• What is the budget?

Purpose. In general, an evaluation aims to show systematically:

1. How successful the project or programme has been in promoting improvement or changing hygiene practices with the given human and other resources; and

2.  How the project or programme can be improved to overcome weaknesses detected.

In order to address the two above points it is important to define what is going to be investigated. Sometimes limitations in time, staffing or budget would require a prioritising of the issues to be evaluated. In general, information collection on the following is desirable:

• Hygiene practices. The most important information needed is on the current hygiene behaviour and the community's perception on what are 'good' and 'bad' practices. The most important practices to be studied are: (1) methods of human excreta disposal (2) hand washing, especially with soap (3) food preparation and storage (4) water source choice and protection (5) water handling and storage in the home. The locally prevailing health problems/diseases and the priority concerns of local people, and the baseline data collected, will determine which specific practices are investigated.
  
• Physical conditions: The presence and conditions of water supply, sanitation and hygiene facilities all influence to what extent people can practice better hygiene. Physical conditions such as lack of drainage or hard-to-clean latrine slabs can also bring new health risks.
  
• Variation between households and communities. Many hygiene conditions and practices have gender-specific roles and values associated with them. The same goes for age-specific practices and differences in views, conditions and practices of different ethnic and religious groups and social classes.
  
• Communication channels: the relevance of the messages to both sexes in the different social and age groups, and the effectiveness of ways to get them across such as; word of mouth, messengers, social gatherings, theatre, radio, TV.
  
• Health statistics: It should be noted that even if reliable health statistics are available, impacts will only begin to show up after a critical mass of behaviour change has been achieved for a sufficiently long time. For faecal-oral diseases, it should be kept in mind that just a small proportion of people with diarrhoea (who may not be typical) seek official medical care.
  
• Comparing the change in recorded incidence of sanitation, hygiene and water related diseases as a proportion of overall diseases (minus accidents and gynaecological treatment) can give an indication of impact. However, statistics are easily distorted by a wide range of potentially intervening factors, from breakdown of water supplies to change in availability of drugs. Hence, measuring actual conditions and practices (or their indicators) is more reliable and more useful for diagnosing weaknesses in the programme.

Evaluation techniques

Once the purpose and focus of the evaluation and information demands have been agreed, the evaluation techniques can be selected. There are a number of methods that have been developed to measure behaviour change which have been widely field-tested. They are best used in combination to check for consistency and to see whether outcomes are reliable. The most common ones are:

• Structured observation of hygiene practices, e.g. observing and recording behaviour during water collection, storage and drawing;
  
• Structured observation of proxies of hygiene behaviour, e.g. the absence of excreta in yards and on rubbish heaps as an indication of the safe disposal of young children's excreta;
  
• Questioning of the people who are most likely to know about an issue (key informants), which requires techniques to deal with uninvited interventions from others with less knowledge, who may nevertheless take over for reasons of hierarchy, e.g. husbands or mothers-in-law. Probing techniques are often needed to move from polite answers to the real practices;
  
• Focus group discussion, which involves a more open-ended, but guided discussion among a group of 6 to 10 people. Skill is needed to keep the discussion on track without dominating it, but this is a very powerful method for discovering issues which an outsider would not think to ask about.
  
• Pocket voting, where women and men in the different groups are presented with drawings that show the various options and put their vote in the bag or box underneath their own, or their family's practice. Voting is done with tokens in different colours (e.g. blue for men, red for women) in order to allow separate analysis when all votes have been cast. It can be done at some distance or behind a cloth for privacy;
  
• Microbiology, e.g. tests of stored drinking water or of fingertips to assess contamination;
  
• Product measurement, e.g. sales of latrine slabs, soap consumption. See also Almedom et al. (1997).

Sampling and statistical methods

It is often not possible to visit all communities and households. The evaluation can then be carried out in a sample. Samples should be random, that is, every community / household has an equal chance of being involved.

For a random selection, an investigator can draw slips of paper with the names concerned from a bag until the required number has been achieved. An alternative is to lay a grid over a map and choose grids, and communities/households within them, at random, e.g. with the help of a table of random figures.

In stratified sampling, differences which may affect hygiene, e.g. dry or wet environment or housing areas near and far from a protected water source, are identified first. Proportional samples are then taken from each group. For example, the programme may cover six wet and 12 dry areas, from which two wet and four dry areas are chosen at random before drawing the community sample. For sample size applies in general 'the larger the sample, the better'. However, with increasing size, the statistical value of adding each additional case drops. As large samples are costly, but too small a sample will reduce the validity, it is advisable to consult a statistician when in doubt on an acceptable sample size.

For analysis, the scores are transferred to a spreadsheet or SPSS data base (Statistical Package for the Social Sciences). Most common is frequency analysis specified by sex, age and social/cultural/economic groups.

Community involvement

Before undertaking an evaluation it is important to inform the community about the evaluation (and afterwards present the results to them). If done with care and respecting existing structures and taboos, it will strengthen the relationship between project and community. Informing beforehand may influence the evaluation outcomes, so it may be advisable to give only general information until fieldwork is complete. Teams also need training on how to deal with interference and practice probing to get beyond biased information.

Study team

The evaluation of hygiene promotion is labour intensive and in most cases would need a specially contracted professional study team. Preferably it is multi-disciplinary, equally balanced between women and men, speaking the local language and familiar with the local culture. The team leader should be a good manager, have writing skills and be able to make an assessment and analysis of the information collected. She/he should also be actively involved in the development of questionnaires, observation checklists and other tools.

In many cases, appropriate staffing will not be 'readily available'. Sometimes it is useful to use two (or more) different groups, firms or NGOs, combining the strength of each into one team. Finding staff for the fieldwork is generally easier than for doing a systematic analysis of the findings and writing-up the results. In those cases you might consider to hire an outsider for the analysis/writing task, though they should be an active team member from the beginning.

After the team has been recruited, intensive training should be undertaken before the actual evaluation can start. The aim is to sensitise the team on hygiene promotion and the programme setting, to develop a common understanding of the evaluation and the methodology and to develop a team spirit.

Participatory evaluations

Besides the conventional survey studies by a team of outsiders, it is also possible to carry out evaluations using quantifiable participatory methods.

The main advantage is that the same information directly informs all levels: the communities, including specific groups therein, such as the poor and women; the district authorities, the programme staff and management, and the national authorities and donors.

The methodology builds on an approach earlier developed and applied for water supply and sanitation programmes ((Mukherjee and van Wijk, 2003). It uses well-known participatory methods such as community mapping, matrix scoring, pocket voting, ranking and sorting. As they do not require literacy everyone can use them after some practice.

By quantifying the outcomes either directly or through scales, all information becomes comparable between communities and areas and across time. Quantification also makes it possible to carry out a statistical analysis in the same way as a conventional survey.

Presentation of the findings

Before publishing the findings it is important to share the draft version for comments with the main stakeholders. In addition to the extra know-how and feed-back, it will provide an opportunity to check their concurrence with the findings of the evaluation. This might avoid potential public disputes on the findings.

The documentation of the findings is the main output of the hygiene promotion evaluation. In general, it will consist of a complete study report with the methodology, findings, analysis, conclusions and recommendations. However, for better dissemination of the findings it is desirable to make the information more accessible for groups that normally would not read detailed technical reports, such as community members, policy makers and experts. The dissemination can be done through:

1. a separate more popular written summary report and/or brochure; and

2. verbal presentations to selected groups.

¹ Black, R., Morris, S., and Bryce, J. (2003). “Where and why are 10 million children dying each year?”, The Lancet, Vol 361, June 28, 2003.

² Jones, G., et al. (2003). “How many child deaths can we prevent this year?”, The Lancet, Vol 362, July 5, 2003.

³ Curtis, V., and Cairncross, S. "Effect of washing hands with soap on diarrhoea risk in the community: a systematic review.", The Lancet Infectious Diseases, Vol 3 nr. 5, pp 275-281, 2003

⁴ Based on the brochure “Sanitation for All”, UNICEF Division of Communication, New York, USA, January 2000.

References

Almedom, A., Blumenthal, U., and Manderson, L. (1997). Hygiene Evaluation Procedures: Approaches and Methods for Assessing Water- and Sanitation-Related Hygiene Practices, International Nutrition Foundation for Developing Countries http://www.unu.edu/unupress/food2/UIN11E/UIN11E00.HTM

Boot, M., and Cairncross, S. (1993) Actions Speak: the study of hygiene behaviour in water and sanitation projects, IRC International Water and Sanitation Centre / London School of Hygiene and Tropical Medicine

Mukherjee, N.; Wijk, C. van. (2003)
Sustainability planning and monitoring in community water supply and sanitation: a guide on the Methodology for Participatory Assessment (MPA) for community-driven development progams. World Bank, Washington
http://www.gtz.de/ecosan/download/WSP-sustainability-planning-monitoring.pdf

For links on hygiene promotion see

• http://www.wsscc.org/load.cfm?edit_id=150
  
• http://www.unicef.org/programme/wes/pubs/behav/behav.htm
  
• http://www.who.int/water_sanitation_health/Environmental_
  sanit/PHAST/phast96-11/96-11index.htm
  
• http://www.irc.nl/content/view/full/1921

 BACK TO TOP

Home > Resources > Fact sheets > Evaluation of hygiene promotion