Proposed metrics to measure quality: Overview

While the first quality of care framework in family planning was articulated over 25 years ago and a considerable amount of work has been done since then to measure quality in the context of routine service delivery. Yet, we do not have agreed upon indicators to measure quality that can be applied uniformly across different health systems and social contexts. The work done so far reflects the types of data available in developing countries. In this paper, we have taken a slightly different approach and used a common definition of quality from the outset. Indicators of quality are required for describing the nature of family planning services and quality of care offered by a health service delivery program, and for improving clients’ experience and health outcomes. Additionally, indicators are needed for monitoring quality of care overtime in a single country (e.g., for quality improvement) as well as for comparing quality across countries (e.g., for understanding contraceptive discontinuation and unintended pregnancies). Different types of indicators are needed to serve these different needs for policy and program development. Keeping these needs in mind, we propose a new set of measures to assess quality across different levels and settings. BACKGROUND More than 25 years ago, Bruce (1990) articulated a client-centered quality of care framework for family planning. Recognizing the important role of measurement in ensuring quality improvement, many efforts since then have been made to measure quality both in the context of research and routine service delivery. The methodologies and indicators used in these efforts have been reviewed by Tumlinson (2016), and RamaRao and Jain (2016). The main approaches used for data collection include: facility surveys (e.g. SA, QIQ, MLE, PMA2020, and SPA)1, cross-sectional surveys of individual women (e.g. DHS and PMA2020), and special studies conducted to assess the relationship between quality of care and reproductive health (RH) outcomes (e.g. Koenig et al. 1997). The SA used four instruments of data collection (i.e., facility audit, provider interview, observation, and client exit interview), but the QIQ used all instruments except the provider interview. Given their cost and complexity, both SA and QIQ methodologies are no longer in Proposed metrics to measure quality: Overview Anrudh K. Jain, John Townsend, Saumya RamaRao, Population Council, New York The Population Council conducts research and delivers solutions that improve lives around the world. Big ideas supported by evidence: It’s our model for global change. popcouncil.org © 2018 The Population Council, Inc. AP R IL 2 01 8 1 Photo credit: Ashish Bajracharya 1 SA: Situation Analysis; QIQ: Quick Investigation of Quality; MLE: Measurement, Learning, and Evaluation; PMA2020: Performance, Monitoring, and Accountability 2020; SPA: Service Provision Assessment; DHS: Demographic and Health Survey. common use, except in special studies.These methodologies were ostensibly replaced by the SPA, which is designed and managed by the Demographic and Health Survey (DHS). The SPA uses the same four instruments of data collection as the SA, and it has been conducted in about 15 countries. In addition, under the Family Planning 2020 (FP2020) initiative, PMA2020 collects facility-level data through facility audits in eleven countries. Beginning with Kenya in 1989, SA was used extensively to describe the quality of family planning services in several subSaharan African (SSA) countries (Miller et al. 1991). For example, Askew et al. (1994) used these data to create over 40 indicators classified under various elements of quality. Mensch et al. (1994) used these data to describe the functioning of sub-systems of family planning in Nigeria, Tanzania, and Zimbabwe. Miller et al. (1998) listed 28 indicators for infrastructure and facility readiness, and 36 indicators for quality of care. Recognizing that there was a greater utility and lower cost in using a smaller number of indicators, the QIQ methodology developed by Tulane University reduced the number of indicators to 25 for which data were collected to describe quality in Ecuador, Turkey, Uganda, and Zimbabwe (Sullivan and Bertrand 2000). Special studies also used data collected through SA to assess the effect of targeted interventions on quality of care. For example, Costello et al. (2001) used data on 24 items collected through exit interviews to assess the effect of a provider training intervention on quality of care received by clients in the Philippines. Data collected through SA has also been used to study the effect of quality on contraceptive use, method continuation, and unwanted fertility (Mensch et al. 1997, RamaRao et al. 2003, Jain et al. 2012). Tumlinson et al. (2015) used data from the MLE (Measurement, Learning and Evaluation) project to assess the relationship between quality of care and contraceptive use in urban Kenya. These studies using facility surveys primarily tried to measure each of the six elements of quality articulated in the Bruce framework separately; some of them then also combined these elements to estimate an overall index of quality. Recently, with a focus on developing more valid indices, SPA data have been used to describe the quality of care in routine care in Kenya, Namibia, and Senegal (Wang et al. 2014) and in Ethiopia (Tessema et al. 2016). These studies used factor analysis to explore diverse elements in structure, process, and outcome indicators of quality. Mallick et al. (2017) created summary measures of quality of services and quality of care from SPA data by using three methods of combining individual indicators—simple additive, weighted additive,and principle component analysis. PMA2020 data are also being used to study the relationship between quality and contraceptive use. Tumlinson (2016), after reviewing much of this information, concluded that ‘In addition, within studies investigating the quality of family planning services there is great diversity in how quality is defined and which elements of quality of care are considered most important, with no agreed set of indicators. Inconsistent definitions of quality pose a challenge to summarizing results of studies investigating quality of care in FP programs.’ Indicators used to measure quality in the past were constrained by the availability of data collected through facility and cross-sectional surveys. The purpose of this paper is to propose metrics to measure quality that go beyond the currently available data with the anticipation that some of the data required can be collected in future studies and eventually incorporated in national health information systems. Some of these data can be incorporated in the ongoing data collection activities such as DHS, SPA, PMA2020, and Health Management Information Systems (HMIS). The indicators proposed below are based on a common definition of quality and a common framework to measure quality. It is recognized that given the complex nature of quality and health systems, any one indicator, or even a small set of indictors, is not sufficient to be used for all purposes.