World Health & Population

World Health & Population 14(2) March 2013 : 4-11.doi:10.12927/whp.2013.23270
Original research

Measuring Progress toward the Millennium Development Goals and the Missing Millions

Roy A. Carr-Hill

Abstract

The 2015 target date for achieving the Millennium Development Goals (MDGs) is fast approaching, but there is very little discussion of the validity of the indicators used to measure progress.

In particular, there has been little attention given to the problems that arise when assessments of progress are based on household surveys. These are inappropriate for obtaining information about the poorest of the poor. Typically, they omit by design those not in households because they are homeless; those who are in institutions; and mobile, nomadic or pastoralist populations; and, in practice, household surveys will typically under-represent those in fragile, disjointed or multiple occupancy households; and those in urban slums and insecure areas of a country.

Those six subgroups constitute a pretty comprehensive ostensive definition of the "poorest of the poor." Between 300 and 500 million people – mainly in developing countries – will be missed worldwide from the sampling frames of household surveys.

Introduction

Millennium Development Goals

The 2015 target date for achieving the Millennium Development Goals (MDGs) is fast approaching. Moreover, the most recent report on progress (United Nations Development [UNDP] 2012) claims that three important targets have already been reached, including reducing poverty by half from the 1990 level, and significantly improving the lives of 200 million slum dwellers, well ahead of the 2020 deadline. Already civil society, academics, governments and the UN system are thinking about what the next era of development should look like, but there is very little discussion of the validity of the indicators used to measure progress. In particular, there has been little attention to the problems that arise when, as is increasingly common in developing countries, population estimates and assessments of progress toward more than half (29 of the 54) of the indicators used for monitoring progress toward the Millennium Development Goals are based on household surveys (see Appendix 1).

Population Counts

For several decades and in some countries for centuries, populations have been counted through national, usually decennial, censuses in which enumerators go to households. Inter-censal population estimates have usually depended on reliable birth and death registration systems. In most middle- and low-income countries, however, vital registration systems have never been fully functioning (Powell 1981), and there has been a similar decline in donor interest in censuses and vital registration systems (Setel et al. 2007). Instead, there is an increasing reliance on large-scale standardized household surveys for the basic data.

Many countries run national economic and social surveys to provide detailed information on consumer prices, income and employment, and other relevant data for planning. This move away from censuses to relying on surveys raises the obvious problem that drawing a sample for a survey depends on having a sampling frame in the first place which is frequently based on the census. Clearly, any problem with the census, if used as the sampling frame for a national survey, will lead to that sampling frame being biased. But there is –strangely – little recognition of these problems.

Censuses

Population censuses have always faced problems of complete enumeration. Groups of adults have been excluded from censuses in some countries for political and/or practical reasons. Non-citizens, cultural minorities or marginalized groups, and specific categories of prisoners or rebels who object to government oversight have often been excluded for political reasons (Buettner and Garland 2008), and although less frequent and certainly more transparent, this still continues (Indigenous Portal 2011; Abbasi-Shavazi and Sadeghi 2011).

Therefore, the general problem that censuses are not necessarily complete is well understood (Carr-Hill 2009). At the same time, there is an emerging consensus as to what constitutes good census practice (United Nations 2008); and censuses that follow those UN guidelines will usually overcome many of the deficiencies in earlier censuses. The guidelines are clear but there can still be problems in practice, as described below.

Housekeeping concept: While Cinderella is a fairy tale, the exclusion of poor servants from the census count in rich households (even though they usually share some of the household food), especially in Asia, is a reality, and their personal poverty is therefore missed.

Mobile populations: In developed countries, the young, highly mobile – usually male – population is also difficult to count, especially when they live in collective households, but they are relatively well-off; in developing countries, they may well be among the poorest.

Homelessness and counting de facto rather than de jure populations: These are difficult to count, especially where there are disputes over nationality (Refugees International 2007); equally, there are several millions internally displaced in many countries, either as a result of civil war or because environmental change (e.g., floods, nuclear accidents) makes their homes uninhabitable; and, although there are periodic counts, there is no regular database anywhere.

Institutional populations: There are a number of different types of institutions (care homes, [some] factory barracks, hospitals, the military, prisons, refugee camps, religious orders and school dormitories), and there is considerable variation in opinion on whether or how they should be included in the population count. They are often simply counted as special census blocks.

Careful census reporting documents how well these groups have been enumerated, and most categories are included in estimated census population counts of developed countries but not in those of many developing countries. Moreover, in many developing countries the census enumerators are often police or other government officials, who tend to use security-based national identity cards or family registration cards to validate the citizenship status of those they are enumerating (Di 2010; Dwinosumono 2006).

Assessing Poverty

In assessing the absolute level of poverty or the absolute levels of illness, household surveys are an inappropriate instrument for obtaining information about the poorest of the poor, especially in developing countries. This is because household surveys, with rare exceptions, typically omit, by design,

  1. those not in households because they are homeless,
  2. those in institutions, including refugee camps, and
  3. mobile, nomadic or pastoralist populations.

In addition, in practice, because these individuals are difficult to reach, household surveys will typically under-represent

  1. those in fragile, disjointed or multiple occupancy households (because of the difficulty of identifying them),
  2. those in urban slums (because of the difficulty of interviewing), and
  3. certain areas of a country deemed to pose a security risk.

If one wanted a practical – as distinct from theoretical – definition of the "poorest of the poor," the above collection of six population subgroups could hardly be bettered.

Census officials, because of the difficulty of enumeration, even in developed countries, often only estimate those groups' size and location, so their members are not included in the available sampling frames for household surveys. In developing countries, these marginalized groups may not be included at all, even in the estimated population counts. The lack of recognition of these problems with the design and implementation of household sample surveys, particularly in developing countries, has meant that there has been no systematic attempt to estimate the size and distribution of the population groups "missing" from the sampling frames of national household surveys.

How Many Are Potentially "Missing" from Population Counts and from Sampling Frames of Household Surveys?

The focus here is on groups for which there are credible sources, and that are normally among the poorest. Other groups, such as economic and environmental migrants (Myers 1997), are not considered below.

The Homeless

Rather obviously, household surveys omit the homeless and street children. Estimating numbers is very difficult. Over 20 years ago, the United Nations Children's Fund (UNICEF) estimated that there were about 100 million street children (UNICEF 1989) worldwide. The figure is still commonly cited but has no evidence base (Green 1998; Hecht 1998). But, however many there are, they will not be covered by household surveys.

Institutionalized Populations

Household surveys, by definition, omit from their sampling frame those in institutions: care homes, (some) factory barracks, hospitals, the military, prisons, refugee camps, religious orders and school dormitories.

Care Homes and Hospitals: Those in hospitals and care homes will on average be poorer because morbidity is associated with poverty (Lopez 2002), although that is less true for older people. There are an estimated 20 million hospital beds worldwide (World Health Organization [WHO] 2011).

Military: The Central Intelligence Agency's (CIA) World Factbook (CIA 2011) documents 92 million military personnel worldwide (including reservists).

Prison: Those in prisons will usually be poorer, and estimates of the world's total prison population are around 9.8 million (Walmsley 2003). None of these 9.8 million will be included in the sampling frame of household surveys.

Refugees: Refugees are not considered part of any nation's population so they cannot be included in survey sampling frames nor make any contribution to survey-based estimates. However, the United Nations High Commissioner for Refugees (UNHCR 2010) publishes figures annually on numbers of officially registered refugees, internally displaced persons and stateless persons. The worldwide total is 36.5 million, but these figures do not include the large number of illegal immigrants, most of whom will not be counted in a national population and, of course, not in the sampling frames of household surveys.

Nomadic and Pastoralist Groups by World Region

The permanently mobile are usually excluded from household surveys. In particular, censuses and surveys in developing countries have difficulty enumerating nomadic/pastoralist populations, who have much less access to services (UNICEF 2007); and, while it is difficult to assess their income and wealth, and there clearly are some who are rich-in-kind (or asset rich), the majority are usually poor in all senses.

There is no reliable information available on the number of nomadic pastoralists, including sea-faring mobile communities, worldwide. The only internationally comparable source is that compiled by the International Livestock Research Institute (Thornton et al. 2003), based partly on livestock numbers, although they will include agro-pastoralists who are semi-sedentarized. These estimates suggest there are 217.5 million in pastoralist households.

The Difficult to Reach

A) Fragile and Disjointed Households

The task of the census enumerator or survey interviewer is made much more difficult when the household structure is ambiguous or undefined so that either identifying the household head and/or counting the numbers in the household is almost impossible.

B) Urban Slums

The most recent estimates from UN-Habitat (2011) are that more than a billion people are living in urban slums in developing countries, but information is rarely disaggregated according to intra-urban location, and the poorest urban populations are often simply not included in data gathering.

The few surveys conducted in those slums show sharp gradients according to income quintiles within urban populations (UN-Habitat 2003). But, given the very high levels of mobility, it would seem reasonable to assume that a substantial minority of those households in the slum areas of cities in developing country are uncounted in any census. Moreover, even where they are counted in censuses, many would (because of interviewer reluctance) in practice, be excluded from sampling frames.

C) Insecure or Isolated Areas

This will obviously vary according to context and so will be a much larger problem in specific countries. Given the security situation – or simply difficulty of transport – in many countries, it can often be difficult for the implementing institutions to carry out a fully representative survey or census.

Overall Estimates, Discussion and Conclusions

Overall Estimates

The numbers vulnerable to undercounting are summarized in columns 3 and 4 of Table 1. Not all those missing populations are poor. Estimates of the poor are assumed to include all those in prison and all refugees (and there are assumed to be at least as many unofficial refugees as official refugees); all those hospitalized in low- or low-middle-income countries (about half), but only those in the military in low-income countries (about 20%); and between 20% and 40% of nomadic populations. In addition, based on informal conversations with census officials in several countries and on the difficulties of using satellite imagery in slum areas (Schurman 2009) (because of the need to rely on key informants and the visual obscurity of some structures), the number of slum dwellers missing is estimated at between 10% and 20%.


Table 1. Estimates of population groups missing from sampling frames of household surveys worldwide
    Vulnerable to undercounting Estimated percentage poor Missing poor (millions)
    Minimum Maximum   Minimum Maximum
Pastoralists   217.5 20–40% 43.5 87.0
Institutionalized Refugees 36.5 73.0 100% 36.5 73.0
  Hospitals 20.0 40–60% 8.0 12.0
  Military 92.0 15–25% 13.8 23.0
  Prisons 9.8 9.8   9.8 9.8
Urban populations   2535      
Slum populations   827.7 10–20% 82.8 165.6
Total         194.4 370.4
Source: Hospital beds – WHO (2011); Prisoners – Walmsley (2003); Military – CIA (2011); Refugees – UNHCR (2010); Nomads – Thornton et al. (2002); Urban population proportions – CIA (2011); Slum proportions in urban areas – estimated by author from UN-Habitat (2003/2011).

 

Worldwide, the totals in the subsections above add up to between 194 and 370 million (Table 1). Moreover, the estimates do not include the homeless, those in fragile or disjointed households, or those in areas where there are security risks. Most of the homeless are probably from urban slums, so there would be double counting, but the other two categories (large, but of unknown size) are definitely additional. Estimates of between 300 and 500 million worldwide would be reasonable.

Censuses and Sampling Frames

Counting Displaced and Illegal Groups

Census organizations in developed countries use several procedures for estimating the numbers of illegal immigrants. But those procedures would not work for South–South illegal migration. Moreover, there are other omitted subgroups, often quite large: for example, scheduled castes and tribes in India (Government of India, Ministry of Social Justice and Empowerment 2012) and illegal servants in rich households.

Counting and Sampling Nomads and Pastoralists

These are difficult to count simply because they are moving. Reasonable samples have been obtained through combining local-level surveys with remote sensing of livestock (Galvin et al. 2001), but documenting change in their human population remains, on the whole, elusive.

Counting Urban Slum Populations

Any face-to-face interviewing approach will be unreliable both because of the lack of a sampling frame and because respondents will be suspicious of the reason for the questions because they are illegally resident. But even if the census organization were to make available a listing of "houseless" people, given the high levels of intra-slum mobility, this would be an unreliable sampling frame for surveys.

Solutions and Conclusions

Accurate Censuses?

International organizations should support national census organizations in developing and testing procedures for counting pastoralists and other nomads (gypsies, highly mobile workers, long-distance truck drivers, travellers, etc.) and those in urban slums (Angeles et al. 2009; Schurman 2009); and in adopting best practice from UN guidelines. But the same procedures do not solve the sampling frame problem of household surveys, because of the delays between census and survey. In the absence of any simple solution, this author (Carr-Hill 2012) has shown that it may be possible to make estimates of the missing populations. It is crucial to develop similar methods more systematically, with an agreed theoretical basis.

Measuring Progress toward the Millennium Development Goals

These problems are urgent because these data are input for more than half the indicators used for assessing progress toward the Millennium Development Goals (Appendix 1) in developing countries. In particular, the UNDP (2012) claim that the goal of halving the poverty rate has been reached cannot be taken at face value. Even if there has been no growth in the number of uncounted – which seems unlikely given the collapse of vital and registration systems – the missing population means that the reported reduction of 50% from 2 billion to 1 billion in fact reflects a reduction of 43% from 2.3 billion to 1.3 billion (and the latter figure is probably a substantial underestimate).

Population undercounting means that any social program risks ignoring the poorest of the poor. This blindness is a public scandal affecting between 300 and 500 million of the poorest people in developing countries (between 4.5% and 7.0% of the total world population). It should be addressed immediately by international and national organizations, in terms of developing and testing appropriate procedures for counting.

 


 

Appendix I. Indicators for Monitoring Millennium Development Goals that are dependent on population data


  Indicator Name
1 Proportion of population below $1 purchasing power parity (PPP) per day
1A Poverty headcount ratio (percentage of population below the national poverty line)
2 Poverty gap ratio (incidence multiplied by depth of poverty)
3 Share of poorest quintile in national consumption
4 Prevalence of underweight children under 5 years of age
5 Proportion of population below minimum level of dietary energy consumption
6 Net enrolment ratio in primary education
8 Literacy rate of 15–24 year-olds
10 Ratio of literate women to men, 15–24 years old
13 Under-five mortality rate
14 Infant mortality rate
15 Proportion of 1-year-old children immunized against measles
16 Maternal mortality ratio
17 Proportion of births attended by skilled health personnel
18 HIV prevalence among pregnant women aged 15–24 years
19 Condom use rate of the contraceptive prevalence rate
19A Condom use at last high-risk sex
19B Percentage of population aged 15–24 years with comprehensive correct knowledge of HIV/AIDS
19C Contraceptive prevalence rate
20 Ratio of school attendance of orphans to school attendance of non-orphans aged 10–14 years
21 Prevalence and death rates associated with malaria
22 Proportion of population in malaria-risk areas using effective malaria prevention and treatment measures
29 Proportion of the population using solid fuels
30 Proportion of population with sustainable access to an improved water source, urban and rural
31 Proportion of population with access to improved sanitation, urban and rural
32 Proportion of households with access to secure tenure
44 Debt service as a percentage of exports of goods and services
45 Unemployment rate of young people aged 15–24 years, each sex and total
46 Proportion of population with access to affordable essential drugs on a sustainable basis
47 Telephone lines and cellular subscribers per 100 population
48 Personal computers in use per 100 population
  Internet users per 100 population
Note: 48 indicators were defined but there were an additional 6 sub-indicators, making 54 indicators.

 

About the Author(s)

Roy A. Carr-Hill, BA, DPhil, Institute of Education, University of London, London, UK

Correspondence may be directed to: Roy A. Carr-Hill, Rose Cottage, 29 Glen Avenue, York, YO31 0XN, Tel.: +44(0)1904 426388, E-mail: roycarrhill@yahoo.com.

References

Abbasi-Shavazi, M.J. and R. Sadeghi. 2011, February. The Adaptation of Second-Generation Afghans in Iran: Empirical Findings and Policy Implications. Middle East Institute, Fondation pour la Recherche Stratégique.

Angeles, G., P. Lance, J. Barden-O'Fallon, N. Islam, A.Q.M. Mahbub and N.I. Nazem. 2009. "The 2005 Census and Mapping of slums in Bangladesh: Design, Select Results and Application." International Journal of Health Geography, 8: 32.

Buettner, T. and P. Garland. 2008. "Preparing Population Estimates for All Countries of the World: Experiences and Challenges." Committee for the Coordination of Statistical Activities, Conference on Data Quality for International Organizations. Rome, Italy; July 7–8, 2008.

Carr-Hill, R.A. 2012, "Missing Millions and Measuring Progress towards the Millenium Development Goals" Conference of African Studies Association of UK, Leeds, September 6–8, 2012.

Carr-Hill, R.A. 2009, June. Mid-Term Evaluation of the EFA Fast Track Initiative. Working Paper 3, Issues in Data and Monitoring and Evaluation. Cambridge Education, OPM and Mokoro.

Central Intelligence Agency. 2011. The World Factbook 2011. Washington, DC. Retrieved August 15, 2012. <https://www.cia.gov/library/publications/the-world-factbook/fields/2011.html>.

Di, W. 2010. "Door-to-Door Census Needs More Accountability." Global Times. September 10, 2010.

Dwinosumono, S. 2006. "The Population Census in Indonesia. Presented at the Fifth Management Seminar for the Heads of National Statistical Offices in Asia and the Pacific. Daejon, Republic of Korea. September 18–20, 2006.

Galvin, K.A., R.B. Boone, N.M. Smith and S.J. Lynn. 2001. "Impacts of Climate Variability on East African Pastoralists: Linking Social Science and Remote Sensing." Climate Research 19(2): 161–72.

Government of India, Ministry of Social Justice and Empowerment. 2012. Scheduled Caste Welfare – List of Scheduled Castes. Retrieved August 15, 2012. <http://socialjustice.nic.in/sclist.php>.

Green, D. 1998. Hidden Lives: Voices of Children in Latin America and the Carribbean. London: Cassell.

Hecht, T. 1998. At Home in the Street Children of North East Brazil. Cambridge: Cambridge University Press.

Indigenous Portal. 2011, February 11. "Northeast India: COPTAM Unpleasant Tidings Greet Census 2011." Retrieved August 15, 2012. <http://www.indigenousportal.com/Politics/Northeast-India-COPTAM-Unpleasant-tidings-greet-Census-2011.html>.

Lopez, A. 2002. "Mortality and Morbidity Trends and Poverty Reduction." In Fifth Asian and Pacific Population Conference: Selected Papers (Chapter 3). Bangkok: United Nations, ESCAP.

Myers, N. 1997. "Environmental Refugees." Population and Environment 19(2): 167–82.

Powell, N.P. 1981. Major Obstacles to Achieving Satisfactory Registration of Vital Events and the Compilation of Reliable Vital Statistics. Technical Papers of the International Institute for Vital Registration and Statistics, No 15. Bethesda MD: International Institute for Vital Registration and Statistics.

Refugees International. 2007, October. About Being Without: Stories of Stateless in Kuwait. Geneva: UNHCR.

Schurmann, A. 2009. Bangladesh Urban Health Survey: Methods and Reproductive Health Results. International Conference on Urban Health Nairobi, Kenya; October 18–23, 2009.

Setel, P., S. Macfarlane, S. Szreter, L. Mikkelsen, J. Prabhat, S. Stout and C. AbouZahr (on Behalf of the Monitoring of Vital Events (MoVE)). 2007 "A Scandal of Invisibility: Making Everyone Count by Counting Everyone." The Lancet 370(9598): 1569–77.

Thornton, P.K., R.L. Kruska, N. Henninger, P.M. Kristjanson, R.S. Reid, E. Atieno et al. 2002. Mapping Poverty and Livestock in the Developing World. Nairobi: International Livestock Research Institute.

UN-HABITAT. 2003. The Challenge of Slums: Global Report on Human Settlements 2003. New York: United Nations.

UN-Habitat. 2011. "The UN Millennium Declaration and Its Goals." Retrieved October 21, 2011. <http://ww2.unhabitat.org/mdg/>.

United Nations. 2008. Principles and Recommendations for Population and Housing Censuses. Revision Series M No 67/Revf 2; ST/ESA/STAT/SER.M/67(Rev 2). New York: United Nations.

United Nations Development Program. 2012. Progress towards the Millennium Development Goals. New York: United Nations.

United Nations Children's Fund (UNICEF). 1989. State of the World's Children 1989. New York: UNICEF.

United Nations Children's Fund (UNICEF). 2007. Somalia Multiple Indicator Cluster Survey 2006. New York: UNICEF.

United Nations High Commissioner for Refugees (UNHCR). 2010. Statistical Online Population Database. Data extracted: 20/12/2010.

Walmsley, R. 2003. World Prison Population List. London: Home Office Research Paper Findings 188.

World Health Organization (WHO). 2011. World Health Statistics 2011. Geneva: WHO.

Comments

Be the first to comment on this!

Related Articles

Note: Please enter a display name. Your email address will not be publically displayed