Effective performance Monitoring and Evaluation (M&E) is essential if countries, sectors and institutions are to know whether they are on track to achieve their objectives, and are to provide information that can be acted upon to maximise performance levels.

In the Uganda water and sanitation sector prior to 2003, the only way to assess overall sector performance was to sift through documentation relating to each of the various institutions separately. Since 2003, sector performance reporting has been significantly strengthened, with the production of three good quality annual water and sanitation sector performance reports that have been well received by government and donors.

Why improve sector monitoring and evaluation?

If effective systems of national sector monitoring and evaluation can be developed, there are many potential benefits:

easier identification of good and poor performance
strengthening of mechanisms for identifying the causes of good or poor performance
service providers more easily held accountable for their performance in a transparent manner
more focused institutional roles for assessing and acting on sector performance and a framework against which capacity building strategies and targets can be developed
more focused and better integrated performance data
integration of the 'tools' of performance measurement, eg operational monitoring, financial tracking studies, evaluation etc.
improved information for assessing the effectiveness of policy and for enabling better policy making
a more credible system for arguing for more resources for the water and sanitation sector and for allocating those resources.

WELL BRIEFING NOTE 7.1 Go Back

National sector performance monitoring and evaluation in water and sanitation in Uganda

This Briefing Note presents the Ugandan experience of strengthening water and sanitation sector monitoring and evaluation and performance reporting in terms of how this was done, what the successes and challenges have been and what lessons can be shared with other countries.

Compiled by: Julie Fisher of WEDC

Briefing Note source material by: Mike Thomson, Patrick A Okuni and Kevin Sansom

Headline lessons from Uganda

Incentives for improved national sector M&E have been enhanced in Uganda because the government is genuinely interested in reviewing sector performance
A sector monitoring and evaluation group that includes representatives from major stakeholders is key to coordinating performance M&E.
A gradual process of the transfer of responsibility for data analysis and performance reporting from consultants to sector government agencies has worked well.
A set of 'golden' performance indicators for water and sanitation enables tracking of performance against a manageable number of key indicators to inform key decisions.
National household survey data has been invaluable as a means of independent assessment and it enables the review of service levels by income group.
The use of a variety of data sources provides a more comprehensive and balanced picture.
Production of good quality annual performance reports enables key M&E information to be used for the government's annual planning and budgeting.

Performance Measurement in Uganda

The challenge

A first step in improving sector M&E is to assess what is currently measured. The overwhelming amount of data relating to water and sanitation often results in an inconsistent view of performance. Effective performance measurement should triangulate information from a variety of data sources such as:

independent household surveys of user perspectives;
the service provider's data, collated by government;
participatory assessments of coping strategies; and
in-depth studies commissioned for specific purposes.

The key question to consider in effective performance measurement are:

What to measure? How to measure it? How to collect data?

How to analyse and present data? What to do with the data?

Figure 1 shows the problem of inconsistent data for use of improved sanitation, which varies between 55% and 85% for 2003. These discrepancies were due to the reluctance of respondents to admit to not using latrines. Identifying such factors is vital to an understanding of service levels and coping strategies.

A key indicator is the type of water source used. Table 1 shows that a high percentage of poor consumers use unprotected sources. Additional studies are needed to determine service levels in low-income areas and ways to improve them.

Figure 1: Example of inconsistent data sets: sanitation coverage in Uganda

Source: MoWLE (2003)

Table 1: Drinking water source of poor and non-poor households

Key performance themes

Figure 2: Performance themes for water and sanitation

Source: Delta Partnerships (2003)

A set of performance themes (Figure 2) was developed to report on overall performance in a more focused and comprehensive way. These have been the basis for strengthening data collection and analysis systems in Uganda.

'Golden' performance indicators

It was important to identify a select list of key or 'golden' indicators to inform future resource allocation and policy making and to avoid the confusion caused by the use of too many indicators. The eight golden indicators that were selected are shown in Table 2. It is important to be explicit about the primary and secondary data sources for information related to each of these indicators.

In Uganda, these golden indicators have been cascaded down to sub-sector levels (rural water and sanitation, urban water and sanitation, water for production and water resources management), providing more analysis where it is needed.

Other key sector indicators have also been developed to ensure effective sector management, but these indicators are mainly used within each sub-sector.

Improved Analysis of Sector Performance

Developing an effective system

It is important to measure and analyse data over time, by factors such as spatial location, income group and gender. Using the golden indicators, performance league tables have been developed for Uganda to show, for example, the average cost of new water points per beneficiary in each district. This indicates where there are good operational practices and where there is scope for improvement.

Comparison of actual performance to plans is another key consideration (Table 2). Long term targets for 2015 were previously established in a Sector Investment Plan and a Poverty Eradication Plan, with annual targets aimed at meeting these.

Table 2: Target levels of performance for 'golden' indicators in Uganda

More sophisticated systems link monitoring of results to expenditure, such as Uganda's Fiscal Decentralisation Strategy. A key success is that local government is now reporting on performance against the golden indicators, and this is linked then to future resource allocation decisions.

The Fiscal Decentralisation Strategy (FDS) in Uganda

The Government of Uganda (GoU) has embarked on an ambitious programme of decentralisation where responsibility for planning, resource management and service delivery is being devoted to five local government levels - district, municipality, county, sub-county, parish and village. In summary, the FDS is the process for:

achieving transparent, needs based and poverty sensitive allocations of sector funds between local governments;
streamlining transfers to local government to remove administrative burdens while increasing transparency and efficiency; and
increasing local government autonomy with respect to planning, budgeting and implementation of national sector policy, with the eventual aim of moving to block grants.

Moving towards Integrated Performance Management

Effective sector performance management involves the collection of data, its interpretation, and most importantly, the agreement and implementation of actions to improve future performance. It not only collects and analyses M&E data, but interprets and acts on the results that are obtained. Uganda is now beginning to address these issues.

Figure 3: Summary elements of effective performance reporting and management

Figure 3 brings together the key elements for effective sector performance management. This provides evidence for better resource allocation and policy making within the sector. For example, trends showing improvements in functionality of water points may be attributed to particularly innovative methods of engaging communities in their Operation and Management (O&M). On the basis of this, national policy on O&M could be changed along with the reallocation of resources.

In-depth evaluation studies

In-depth evaluation studies investigate the causes of good or bad performance. A 2005 study focuses on the low functionality rates of water points in many parts of Uganda. The areas of investigation included:

the barriers to improved functionality and how these can be overcome, taking poverty levels into account;
the cost implications of water and sanitation O&M and how to reduce them; and
the effect of involving the community and women on functionality and how this involvement can be increased.

Value for Money (VFM) reviews

There is not yet a universally agreed definition of what constitutes a VFM study, although the broad principles associated with them are economy, efficiency, effectiveness and equity. Uganda has recently commissioned such reviews to address key issues in relation to water and sanitation schemes, and it is suggested that the themes should be based on the findings of the previous year's performance report. To ensure maximum impartiality, VFM work should be carried out by an external organisation to the sector.

Tracking studies

These are another tool used within the Ugandan water and sanitation sector, as a way of monitoring the flow of the Poverty Action Fund, identifying bottlenecks to the implementation of financial processes and making recommendations to ensure that funds are allocated to the intended activities.

Using performance data at the local level

Mechanisms should be in place to allow sharing of good operational practice with local government staff, responsible for operational decisions:

Annual workshops held for district and municipality water and sanitation officers provide a forum for sharing lessons learned, recognising good performers and best improvers, and for addressing their concerns.
'Clustering' local governments based on socio-economic and/or agro-climatic characteristics enable more meaningful comparisons to be made on performance.
Analysis of performance data can be used to lobby for increased resources and their reallocation.

Capacity Building and Institutionalisation for M&E

An incremental process of support to national sector Monitoring and Evaluation has proved effective in Uganda. Initially, the Ministry of Water, Lands and Environment (MoWLE) requested a team of WELL consultants (including WEDC, Delta Partnership and GY Associates) to draft and coordinate the 2003 water and sanitation sector performance report in conjunction with government staff, with DFID funding. Training and support of about 40 sector officials was undertaken in 2004 to collect and analyse data and report on it. Over a period of three years, responsibility was gradually handed over to sector government agencies, which by 2005, with limited consultancy support, took the lead on producing the annual report.

This three stage process has provided effective skills transfer and capacity building within the sector. It has been successful in Uganda due to the Sector Wide Approach whereby central and donor funds are pooled and dispersed through government channels. Government therefore has an incentive to ensure value for money. In addition, involvement and participation of key staff strengthened the perceived value and demand for M&E and sector reporting. These processes need to be further mainstreamed and institutionalised to have even greater effectiveness.

Lessons Learned

The Ugandan experience presents lessons for a range of stakeholders:

Sector managers

Develop a focused and balanced set of sector indicators
Prioritise performance indicators and cascade down to sub-sector levels
Identify 'primary' data sources for consistency of reporting
Set realistic yet challenging performance targets
Lobby for sector resources based on performance levels and comparison with targets
Provide a mechanism for monitoring the implementation of recommendations of key studies and reviews.

Central government

Allocate roles for data collection, analysis and reporting
Develop clear methodologies and responsibilities for in-depth and VFM performance studies
Use performance monitoring to focus VFM and in-depth evaluation studies
Identify good local performers and improvers and support the dissemination of good practices
Use data as evidence for better policy making
Link resource allocation to performance levels

Local government

Link financial and performance monitoring systems
Exchange good operational practices with other local governments

NGOs

Lobby for inclusion of equity, gender, community etc. issues in sector indicators and analysis
Contribute to qualitative and case study material in sector performance reports
Lobby for policy change and resourcing adjustments

Donors

A good development process for supporting national M&E is to use expert consultants to work with local staff to produce a good quality sector performance report, then facilitate a gradual transfer of responsibilities to government for coordination and drafting
Support capacity building of sector M&E and performance reporting particularly related to reform and poverty issues
Support methodologies for better assessment of VFM provided by sector investments

Key References

Delta Partnership, (2003). Improving the Performance Measurement Framework for the Uganda Water and Sanitation Sector - Consultative Workshop, Kampala, Uganda
Delta Partnership, (2004). Performance Measurement in the Water and Sanitation Sector: Institutionalisation Report, Uganda
Thomson T.M., Okuni P.A. and Sansom K.R., (2005). Sector performance reporting in Uganda - from measurement to monitoring and management, 31st WEDC conference paper, Kampala, Uganda.
MoWLE, (2003). Water and Sanitation in Uganda - Measuring Performance for Improved Service Delivery.
MoWLE, (2004). Water and Sanitation Sector Performance Report 2004, Uganda.
Thomson, M., (2003). Performance measurement framework - Uganda water and sanitation sector. WELL Task No.2272, Loughborough University, UK.

For further information contact:

WELL

Water, Engineering and Development Centre (WEDC)

Loughborough University

Leicestershire LE11 3TU UK

Email: well@lboro.ac.uk

Phone: +44 (0)1509 228304

Fax: +44 (0)1509 223970

Home > Resources > Fact sheets > Child survival