WELL - Resource Centre Network for Water, Sanitation and Environmental Health

Poverty: Waterborne diseases cost the Indian economy 73 million working days a year.

Education: In Madagascar, 3.5 million schooldays are lost each year due to ill-health related to bad sanitation.

Child health: A study in Salvador, Brazil showed that children in households with no toilet had twice the incidence of diarrhoea than those with sanitary toilets.

Gender: 11% more girls attend school where there are working and well-maintained sanitation facilities for girls, offering proper privacy.

HIV/AIDS: Water, sanitation and hygiene can mitigate against illness. Households with a 10% increase in water use for cleaning purposes enjoy a decrease in cases of diarrhoea by 1.3%.

Environmental sustainabilty: Local people in Faisalabad, Pakistan report a huge improvement in quality of life since the installation of community sewers, with children playing outside safely.

Ashoke Chaterjee, independent communicator specialising in water and sanitation, comments, "We are not against having goals and targets, but if people start chasing figures again, without thinking about the process that they're using to get there, then we're on for another disaster." WSSCC

The Millennium Development Goals (MDGs) include the target to halve by 2015 the proportion of people without sustainable access to safe drinking water and basic sanitation. This is the basis for international calls for increased investment in the sector. More than this, the evidence shows that the provision of water and sanitation services impacts on the achievement of other MDGs (poverty, education, child health, gender, HIV/AIDS and environmental sustainability), however this fact is not being effectively used to make the sector’s case.

The time is right to make the case for levering more resources to the sector. Part of the evidence lies in how sector success is monitored, the gaps identified and the MDG links presented at national level. It then depends on how effectively this information is used to prioritise resources to sub-sectors 1 at local government an community level.

'5 billion working days a year being lost in developing countries due to hygiene related illnesses'

It is important that statements and figures on water supply and sanitation are reported for global advocacy and political reasons. However, counting taps and toilets as the main way of showing success can confuse ends with means. It pays too little regard to whether or not facilities are used or if the local incidence of diarrhoeal disease begins to fall. The difficult challenge for sector monitoring is to integrate statistical accounting with health and education sector and MDG priorities. This requires a better understanding of the interrelationship between service delivery (which includes latrine construction and counting) and its impact on poverty. Alongside statistical headlines there must be evidence-based, and situation relevant poverty reduction indicators.

Local measures of poverty

In Flores, Indonesia, communities that built new water supplies and latrines with donor funds were poor according to the national poverty standard. However, with locally set criteria, the range of poverty varied from villages with 90% poor, 10% middle income and 0% rich to 0% poor, 64% middle income and 36% rich. Nevertheless, all got the same financial support to ensure access.

The challenge is to aggregate multiple sets of village and community level data so that a district can sensibly allocate resources.

Other key global level instruments include:

Poverty Reduction Strategy Paper (PRSP) - A country specific medium-term ‘roadmap’ for achieving longer term MDG targets through policy reforms, budget restructuring ‘nationally owned’ targets and international support.

Common Country Assessment (CCA) - A shared instrument of the UN to analyse the national development situation and identify key development issues, including PRSP and MDG activity.

National Human Development Report (NHDR) - An in-depth analysis of options for fostering human development and a source of detailed disaggregated data on MDG targets.

1 For example, rural water supply and sanitation, urban water supply and sanitation, water for production and water resource management

In many countries at least three sector data collection activities run in parallel,

These activities rarely converge to assist planning and decision making about sector and sub-sector resource allocation. They also risk hiding important variations within a country, for example gender disparities within and between communities, and local interpretations of poverty. Although this information may not have a dramatic impact upon the global headlines, at a local level it can lead to disastrous decisions about where to put limited resources. For example, take the distance-based definition of access to water. If this was deemed to be 'within 200 metres' this might be unrealistic and meaningless in reality, and so make the achievement of goals impossible. Why? Because for sparsely populated areas reducing the distance to reliable and safe water from 1 kilometre to within 300 or 500 metres for more people rather than a few, may be a great improvement, whilst in a highly populated slum, it may not.

Similar problems are found in relation to global definitions of poverty. The World Bank's poverty indictor, 'living on the purchasing power of less than US$1 per day' is conceptually flawed. It fails to reflect that costs differ with location and cannot solely be expressed in monetary terms (Satterthwaite 2003). What poverty is, and who the poor are, is a local reality. Using the criteria may have valid benefits to advocacy campaigns but it will need to be used with caution at the different decision making tiers within government.

Global level monitoring highly problematic

'Monitoring the population with access to improved drinking water and facilities for improved sanitation has posed major problems. A review of water and sanitation coverage data from the 1980s and the first part of the 1990s showed that the definition of safe, or improved, water supply and sanitation facilities sometimes differed not only from one country to another, but also for a given country over time. Indeed, some of the data from individual countries often showed rapid and implausible changes in level of coverage from one assessment to the next. This indicated that some of the data were also unreliable, irrespective of the definition used.'

Global Water Supply and Sanitation Assessment 2000 Report.

Household survey results present a more accurate picture of the actual use of sources and facilities than provider estimates. Coping mechanisms and the efforts of individuals and communities to improve their situation are better exposed. Valuable qualitative information about the perceptions and preferences for service options can be obtained from the user’s perspective.

"If the campaign is focused only on the building of latrines," says Dr. Surjya Kanta Mishra, Minister in the Government of West Bengal and a former doctor and local government leader who helped launch the well-known Medinipur sanitation project "there will always be people who are not reached, people who continue to defecate in the open and who continue to pollute the water sources and spread disease. High levels of latrine coverage, therefore, are simply not good enough." – WSSCC

A definition of sanitation coverage needs to move towards a focus on no open defecation

Indications show that water supply and sanitation resource allocation is anything but consistent

A recent indicative review of practices in resource allocation in the water supply and sanitation sector concluded that:

Criteria are either not used in a systematic way or, if they are, there is poor documentation of how they are applied
There is no evidence of either internationally standardised or 'best practice' defined criteria for resource allocation, and
Each country appears to have its own unique set of categories and definitions that are unique to that country.

The main instrument for monitoring global achievements in the water supply and sanitation sector is the UNICEF/WHO Joint Monitoring Programme (JMP). This provides the official indicators for monitoring MDG targets for water and sanitation and informs policy-makers on the status of the sector through the Global Water and Sanitation Assessment Report. The 2000 report presents the way forward to combine efforts to show good progress towards the sector's aims.

Contributing to the global assessment is an onerous task involving national teams representing the different sector agencies for data collection and assessment of the urban and rural water and sanitation sector. The coverage figures returned by each country, with endorsement by their government, are based, as far as judged appropriate, on credible estimates.

The 2000 assessment used a different methodology from those used previously, in an attempt to address some of the difficulties and disparities witnessed previously. Attention shifted from reliance on provider-based estimates towards a new dimension: the view of the consumer. The data sources used include national census data, the Demographic Health Survey (DHS) and UNICEF's Multiple Cluster Surveys (MICS).

Incremental improvement

It is important to acknowledge the value of incremental improvement of facilities at a household level. For example, household A can afford a traditional pit latrine with a grass wall and no vent pipe while its neighbour, household B, has a VIP latrine. Only household B meets national coverage criteria and therefore is counted in the statistics, although household A's situation may have improved significantly.

In recent years, there has been a substantial increase in resources aimed at improving water and sanitation services in Uganda, as a direct response to the government's commitment to poverty reduction. Additional measures of service quality are reduced time and distance to collect water, regularity and quantity of supply, and equity of resource distribution, all of equal importance in determining whether policies, objectives and targets are met.

The Government of Uganda recognises that better performance monitoring would also reveal 'value for money'. The ultimate aim is to provide clear evidence that water and sanitation contributes to poverty reduction. However, improvements in national managerial performance must filter down the sub-sector level and vice-versa. This does not take account of the international community's demand for global reporting and poses an enormous challenge not least because of the complexity of existing data and reporting practices.

Currently, economic policy, operational planning, objective setting and budgeting for poverty reduction is grounded in a 3 yearly updated, multi-sectoral Poverty Eradication Action Plan (PEAP), which acts as a Poverty Reduction Strategy Paper (PRSP).

Now undergoing reform, the sector is managed through a sector wide approach (SWAp) and feeds its strategic priorities into the country's medium term expenditure framework (MTEF). All public expenditure is based on an analysis of the links between inputs, outputs and outcomes. Consistent sectoral expenditure levels have to be ensured, within given constraints. The intention is to both sustain the macro economy while maximising efficient public expenditure against the PEAP goals.

The MTEF is the interface between PEAP goals, operational working plans and resource management. The current goal for water is 100% or maximum feasible access to safe water by 2015. There is no goal for sanitation. Although monitoring indicators exist they do so in the absence of verifiable baseline information and responsibility for actual monitoring is unclear. Equally, definitions (e.g. 'access') are ambiguous. The targets leading to PEAP goals are output-based and to further complicate matters some policy documents have conflicting targets.

A medium term budgetary framework (MTBF) links in to the MTEF but only provides analysis of sector performance by physical outputs. Although a range of results orientated management (ROM) outputs and sector outcome indicators are listed, many of them are inconsistent with the performance indicators used in the past and elsewhere. Expected outcomes are not supported by how these are to be measured. Financial performance is provided but is not linked to its outputs. It is therefore difficult to derive value for money or cost effectiveness of activities over the reporting period. Given this situation and the fact that the sector does not carry out community-based evaluation of water and sanitation the only valid criterion for resource allocation is quantity of outputs.

In a sector fast steering towards consumer-orientated service provision and commercial viability, a radical rationalisation of key performance indicators, data sources and institutional responsibilities is required. This the sector is looking to do through performance measurement initiatives.

Data used for reporting against global targets often fails to connect to national planning decisions about sub-sector resource allocation or the prioritisation and focus of bilateral and multilateral cooperation. Of the numerous monitoring frameworks and opportunities that exist, most are disconnected to one another and so run on separate tracks and timeframes. National monitoring is more about reporting than a decision informing exercise.

Conforming to country based Poverty Reduction Strategy Papers (PRSP) and localised goals require sub-sector based thematic and multiple indicators and these may have little resemblance to global reporting requirements. Given this situation the onus is on global agencies and stakeholders to convince governments of the existence of global monitoring instruments and their practical application.

Where a national government has set its own monitoring criterion that does not 'fit' in to global reporting formats it must be assumed the information, including impact and success, is not reported.

The lack of clarity in national target setting and reporting contributes to patchy and complicated national monitoring systems. Dependable information could be the lever for prioritising investment decisions and equitable sub-sector resource allocation, yet it is too often not available at the right time, in the right place. The experience of the water supply and sanitation sector in Uganda (see above) illustrates some of these issues.

Over two thirds of developing countries failing to set targets

'Goals and targets mobilise national and international partners into action and help forge new alliances. They also provide a means for benchmarking and assessing progress towards human development. Policy reforms, institutional change and resource allocations often result from discussions centred on time-bound targets. Yet less than one third of developing countries routinely set specific and quantifiable national targets for reducing poverty and enhancing human development'.

Although called the 'water and sanitation sector', sanitation data often does not feature in sector policy documents. For example, data collected through annual sanitation surveys may be retained by the health ministry and may not appear in the water and sanitation sections of the country's budgetary framework. Similarly, data on school sanitation may not be readily available outside the domain of the education sector. Monitoring and evaluation activities carried out by planning and quality assurance departments may not link to those undertaken by the finance ministry for value for money purposes. Data collected at district level may never be processed into management information to inform national understanding of infrastructure functionality or coverage.

A major concern at sub-national level is data validity and how biased or unrepresentative data leads to repeated failure to engage with the vulnerable and extend services to the poor. In Tanzania, 11% of households in Lindi and 74% in Kilimanjaro (both regions comparable to provinces) used protected water sources according to surveys. The Ministry of Water and Livestock quote 34% and 48%. This shows that once such figures are aggregated, the national picture becomes very distorted.

The concept of access is also open to abuse. If piped water supplies are intermittent or pressure problems lead to repeated failure to reach the outer perimeters of a village or town, if water sources are dry or pumps broken, this cannot count as access, despite the fact that the national statistics suggest otherwise. Similarly in households with latrines, non-use by men and adolescent boys and children is widespread. If only the excreta of a third of the population, namely adult women and adolescent girls, ends up in a safe place, then effective access to sanitation cannot be claimed.

To turn this situation around requires truly demand based service provision, which many countries are far from attaining. Many professionals believe that the vicious circle of poor service delivery will continue until monitoring involves first and foremost the view and reality of the consumer or user and outcome based performance measurement.

The sustainability vaccum - The purpose of international initiatives, such as the MDGs, is to reduce poverty by tackling its many dimensions concurrently. Monitoring in the water and sanitation sector should not happen in a vacuum. It should also contribute to developing and implementing policies for poverty reduction and targeting resources . However if the emphasis remains on counting physical infrastructure over qualitative data, sustainability issues that impact upon livelihoods, will be overlooked. The challenge is to show that provision of water supply and sanitation infrastructure alone is not sufficient to ensure sustained service provision.

Making a start at the national level - A good starting point is to consider the national objectives to which water and sanitation sector programmes contribute. Resource allocation will be more effective, and the outcomes most easily measurable, the more it supports the achievement of these objectives and targets. This means:

Focusing on service delivery, rather than on physical infrastructure; this requires a greater understanding and assessment of:

Assessing key aspects of hygiene behaviour change central to the realisation of health benefits associated with improved water supply and sanitation, and establishing better practices for reporting these.

Increasing attention on serving the poor as access to water and sanitation are key indicators for poverty reduction. Relevant socio economic indicators need to be incorporated into global and national reporting.

Conduct an audit of available data to provide an opportunity to bring together data from these different sources. Review how these can be used to best advantage in order to address key national monitoring needs and reduce duplication of effort.

This information provides the starting point from which the process of discussion and negotiation for using the same indicators and definitions can take place.

Sector reform and decentralisation should lead to sub-sector and district level decision-making and coordinated inter-agency partnership and collaboration, but often this is not the case. National plans may include sub-sector resource allocation but these tend to be compartmentalised and national objectives do not always reflect local targets. How national plans are equated with local realities and levels of reporting transparency is a problem for district level managers. They have no guarantee that this will be the best way of delivering local services, or if telling the truth about coverage will help or hinder future resource allocation to the district. This makes sensible monitoring very difficult, as the risks associated with accuracy are untold.

Unpicking these issues in the midst of a district's immediate need to keep their water supply and sanitation projects on track is a huge expectation. Motivation and incentive is also a key issue. If a district believes that a rosy picture of coverage will send the message that all is well, resources may be reallocated to a district with greater need. Alternatively, if a poor picture is portrayed, the manager may risk being personally sidelined in favour of someone who could better turn the district around. Monitoring and reporting is highly sensitive and deeply political and the rules are often unspoken. Interestingly, a lack of capacity is often blamed here but at what point is the whole system questioned?

The case for changes in global reporting, and moreover national utilisation is strengthened by recent research into non-traditional monitoring of water supply and sanitation. The research shows that by starting with consumers and users, progress and impact measurement is relevant, better understood and is more easily linked to poverty and sustainability indicators. Three new monitoring approaches are gaining interest:

This social survey approach developed by the London School of Hygiene and Tropical Medicine goes beyond construction of facilities to functioning, use and hygiene behaviour. It makes it possible to validate national statistics and fills essential information gaps about actual impact on people's lives.

This approach developed by WaterAid, looks at a field survey method to measure access to water supply with equity. It links water supply monitoring to PRSPs and provides new findings about the strengths and weaknesses of statistical work.

The Water and Sanitation Program (WSP), World Bank and IRC-Delft are pioneering an approach that quantifies the results of Participatory Learning and Action (PLA) findings and uses this information to aggregate and compare data between different locations and across time.

Each new approach has feasibility, strengths and limitations. The main differences lie in how data is collected. Finding a systematic way to pass this data up the institutional line, to influence national and international decision-making, is the next challenge.

The use of monitoring instruments should enable information to be provided horizontally and vertically throughout the sector, sub-sectors and relevant other sectors. Yet this universal management tool, although a requirement of programmes and projects, fails to accelerate decisions about sector investment and resource allocation where it matters. Things may add up but they certainly do not stack up to improved access to water supply and sanitation service provision.

At one level there is a desperate need for simplicity, at another, acknowledgement that everyone has an equally - and surely connected - important agenda. This will always complicate matters. As the next international decade approaches, at the very least, three issues must be reconciled.

1. Processes for poverty alleviation are multi-dimensional. In the water and sanitation sector this means more than the construction of taps and toilets. For example, sustainable access to water supply (an MDG indicator) may require a focus on issues such as effective water conservation, leak detection, improved maintenance, rationalised irrigation and crop selection, rather than only on the construction of water facilities. Sanitation may require a concentration on demand creation and hygiene behaviours of different groups (men and boys, and women and girls) alongside latrine construction. Poverty and gender indicators cut across the sector, but the sector also cuts across other sectors, primarily health and education, and vice versa. The evidence must be articulated and used to greater effect.

2. The connection needs to be made between global monitoring instruments and how national governments allocate resources in the water supply and sanitation sector and sub-sectors. Currently data collected in this way does not seem to fulfil national government or international agency agendas. Decisions are needed to agree what is their purpose, potential and inter-connection.

3. National resource allocation methods to the sector and sub-sectors require rationalisation and direction. The use of consumer based qualitative indicators and definitions that reflect local realities, will help to allocate resources sensibly. However, when this data is aggregated up the monitoring hierarchy and vice versa, its essence is lost. This situation risks undermining the sector’s contribution to the MDGs. The fact also remains that whether the information is coming from above or below, a district needs a straightforward means of distilling the key facts, so that resource allocation can truly be targeted at the poor.

Full details of all the material used in support of this briefing note are available at www.lboro.ac.uk/well

WELL BRIEFING NOTE 7 Go Back