World Health & Population
Background: There is a lack of up-to-date, systematic reviews that critically assess the role and potential limitations of evidence-based medicine and systematic Cochrane reviews originating in developing countries.
Methods: We performed a systematic literature review of all Cochrane reviews published between 1997 and 2010 by the Cochrane Neonatal Review Group (CNRG) in the field of neuropediatrics. The main outcome parameter of our review was the assessment of the percentage of reviews that originated in developing countries and the number of reviews that provided conclusive/inconclusive data.
Results: In total, 262 reviews were performed in the field of neonatology and 112 in the field of neuropediatrics. Only a small fraction (15/262 in neonatology [7/15 conclusive] and 16/112 in neuropediatrics [9/16 conclusive]) originated in developing countries.
Conclusions: There is an ongoing need for high-quality research that addresses specific issues that are most relevant to the medical care of children in developing countries. Funding and research agencies will play a pivotal role in selecting the most appropriate research programs for the developing world.
Undoubtedly, evidence-based medicine (EBM) has contributed substantially to improving the quality of medicine in general, and in neonatology and pediatrics in particular (Davis 2006). Cochrane reviews are systematic reviews/meta-analyses of primary research in the medical and health policy fields. They are considered the highest standard in EBM. Systematic Cochrane reviews analyze the effects of interventions for prevention, treatment and rehabilitation on populations. In addition, they assess the accuracy of a diagnostic test for a given condition in a specific cohort and setting. They are published and regularly updated online in The Cochrane Library, ensuring that treatment decisions can be based on the most up-to-date and reliable evidence. Each systematic review addresses a specific medical problem of relevance in the treatment or diagnosis of specific patient cohorts.
The Cochrane Neonatal Review Group (CNRG) is one of 50 review groups within The Cochrane Collaboration, and one of the most active (Sinclair 2004). A number of examples illustrate the importance of systematic reviews in improving the delivery of medical care, for example, administration of antenatal steroids, surfactant replacement therapy, hypothermia for hypoxic ischemic encephalopathy, and probiotics to prevent necrotizing enterocolitis. Moreover, and of importance, Cochrane reviews have also contributed to identifying interventions that are ineffective or harmful, for example, administration of antenatal thyrotrophin-releasing hormone and early postnatal administration of dexamethasone (Crowther et al. 2004; McGuire et al. 2010).
Thus, the Cochrane database may prove particularly beneficial for low-income countries with limited resources. However, most published clinical research has been conducted in highly industrialized Western countries, and it remains unclear how the results gained from these randomized controlled trials (RCTs) will translate into changes in medical care in the developing world. Thus, it is important that developing countries themselves get involved in the process of generating Cochrane reviews, based on their specific medical problems and needs.
Therefore, the main purpose of this study was to assess the number of Cochrane reviews that originated in low-income countries, and whether these meta-analyses provide useful data for the clinician caring for children in these countries. The aim of this study was not to systematically assess the role and potential implications of RCTs performed in Western countries for the developing world.
Methods and Clinical Questions
Based on two previous systematic literature reviews (Girisch et al. 2012; Willhelm et al. 2012), including all reviews from the CNRG (https://neonatal.cochrane.org/) and on neuropediatrics from 1997 to 2010, we conducted a subgroup analysis. The following data were retrieved from the Cochrane review database: time and origin of publication by country. The main outcome parameters were:
- Number (percentage) of reviews that originated in developing countries
- Number (percentage) of reviews that provided conclusive/inconclusive data.
All data were retrieved from the CNRG and Cochrane database and stored in electronic form, using SPSS 19.0 (SPSS, Chicago, IL., USA).
In the field of neonatology, a total of 262 reviews were included in this study (Figure 1); five were excluded because they dealt exclusively with maternal or parental issues.
As depicted in Figure 2, the vast majority of Cochrane reviews were performed in Western, industrialized countries (Australia, North America and Europe: 247/262), while only a minority of papers originated in developing countries (15/262). Table 1 provides specific data with regard to included studies. Only seven of those 15 reviews provided conclusive recommendations (six negative, one positive). In industrialized countries, 36/247 issued a positive recommendation, 97/247 issued a negative recommendation and 114/247 were inconclusive.
|Table 1. Included Cochrane reviews in the field of neonatology (1997–2010)|
|Country||Year||First author||Title||Therapeutic recommendation|
|Malaysia||2006||Nai Ming Lai||Increased energy intake for preterm infants with (or developing) bronchopulmonary dysplasia/chronic lung disease||No studies available|
|India||2003||Sachin S. Shah||Inhaled versus systemic corticosteroids for preventing chronic lung disease in ventilated very low birth weight preterm neonates||Not recommended|
|India||2005||Nandkishor S. Kabra||Multiple versus single lumen umbilical venous catheters for newborn infants||Not recommended|
|Pakistan||2006||Shahirose S. Premji||Higher versus lower protein intake in formula-fed low birth weight infants||Insufficient data – inconclusive|
|Brazil||2009||Marcela Bottino||Interventions for prevention of neonatal hyperglycemia in very low birth weight infants||Not recommended|
|Brazil||2009||Marcela Bottino||Interventions for treatment of neonatal hyperglycemia in very low birth weight infants||Insufficient data – inconclusive|
|Uruguay||2000||Agustin Conde-Agudelo||Kangaroo mother care to reduce morbidity and mortality in low birthweight infants||Not recommended|
|Malaysia||2000||Jacqueline J. Ho||Continuous distending pressure for respiratory distress in preterm infants||Insufficient data – inconclusive|
|Malaysia||2002||Jacqueline J. Ho||Early versus delayed initiation of continuous distending pressure for respiratory distress syndrome in preterm infants||Insufficient data – inconclusive|
|Saudi Arabia||2008||Khalid M. AlFaleh||Probiotics for prevention of necrotizing enterocolitis in preterm infants||Recommended|
|Israel||2004||Karla Soares-Weiser||Rotavirus vaccine for preventing diarrhoea||Not recommended|
|Nigeria||2004||Christy A.N. Okoromah||Diazepam for treating tetanus||Insufficient data – inconclusive|
|India||2007||Prakash Vemgal||Interventions for non-oliguric hyperkalaemia in preterm neonates||Insufficient data – inconclusive|
|Malaysia||2007||Jacqueline J. Ho||Magnesium sulfate for persistent pulmonary hypertension of the newborn||No studies available|
|India||2004||Sachin S. Shah||Intraventricular antibiotics for bacterial meningitis in neonates||Not recommended|
In the field of neuropediatrics, a total of 112 reviews were included, with only 16/112 originating from developing countries (detailed information is provided in Figures 3 and 4 and Table 2). Nine reviews provided conclusive recommendations (five negative and four positive), while six were inconclusive. One report provided conditional recommendations. Of reviews performed in the Western world, 33/94 provided a positive recommendation, 11/94 a conditional recommendation, and 32/94 a negative recommendation, while some 30/94 remained inconclusive.
|Table 2. Included Cochrane reviews in the field of neuropediatrics (1997–2010)|
|Country||Year||First author||Title||Therapeutic recommendation|
|India||2006||Lakshmi Narasimhan Ranganathan||Rapid versus slow withdrawal of antiepileptic drugs||Insufficient data|
|India||2005||Lakshmi Narasimhan Ranganathan||Vitamins for epilepsy||Not recommended|
|India||2005||Kameshwar Prasad||Anticonvulsant therapy for status epilepticus||Recommended|
|India||2001||Sridharan Ramaratnam||Psychological treatments for epilepsy||Not recommended|
|India||2004||Man Mohan Mehndiratta||Plasma exchange for chronic inflammatory demyelinating polyradiculoneuropathy||Recommended|
|India||1999||Sridharan Ramaratnam||Yoga for epilepsy||Insufficient data|
|India||1997||Kameshwar Prasad||Surgery for primary supratentorial intracerebral haemorrhage||Conditionally recommended|
|China||2009||Qifu Li||Traditional Chinese medicine for epilepsy||Not recommended|
|China||2006||Daniel K.L. Cheuk||Accupuncture for epilepsy||Not recommended|
|China||2005||Ming Liu||Acupuncture for acute stroke||Insufficient data|
|China||2004||Li He||Acupuncture for Bell's palsy||Insufficient data|
|Brasil||2009||Marcio M. Oliveira||Pharmacological treatment for Kleine–Levin syndrome||Insufficient data|
|Brasil||2007||Adriana S. Ramacciotti||Dipyrone for acute primary headaches||Insufficient data|
|Chile||2002||Rodrigo A. Salinas||Corticosteroids for Bell's palsy (idiopathic facial paralysis)||Recommended|
|Chile||2000||Sergio M. Castillo||Oxcarbazepine add-on for drug-resistant partial epilepsy||Recommended|
|Thailand||2001||Rungsan Chaisewikul||Calcium antagonists as an add-on therapy for drug-resistant epilepsy||Not recommended|
In this review, we demonstrated that only a small minority of these Cochrane reviews – both in the fields of neonatology (5.7%) and neuropediatrics (14.3%) – originated in developing countries. This is of concern, for worldwide the vast majority of neonates and children are born and raised in these countries. Moreover, the recommendations issued in Cochrane reviews performed in highly industrialized countries are largely applicable to the fields of neonatology and neuropediatrics as practised in industrialized countries and will potentially exclude the majority of neonates, infants, and children being born and cared for in the developing world. However, recently, efforts (through initiatives such as the Effective Health Care Alliance and the SEA-orchid consortium) have been undertaken to disseminate knowledge from the CNRG to low- and middle-income countries to ensure that care practices are evidence-based and that scarce resources will be used and allocated appropriately (Garner et al. 2004; Henderson-Smart et al. 2007). These programs target generators as well as users and teachers of evidence in order to ultimately ensure the implementation of effective interventions (Garner et al. 2004; Henderson-Smart et al. 2007).
It is also interesting to note that a disproportionately high number of systematic Cochrane analyses originate in Australia and the United Kingdom. Although speculative, this may at least in part be attributed to the fact that these countries have played important roles in the generation and promotion of EBM. Moreover, one of the most influential centres of EBM is located in Oxford, England.
The use of systematic reviews as provided by the CNRG and Cochrane database plays an important role in disseminating the best available evidence, thus contributing to the provision of good medical care at the bedside. A substantial proportion of systematic reviews provided data with regard to the question of whether a certain intervention should or should not be performed (Tables 1 and 2). This will provide the physician at the bedside with invaluable information with regard to both optimal and unnecessary treatment modalities. However, interpretation and possibly implementation of these data should be done only in conjunction with "local modifiers," for example, the decision to use intramuscular vitamin A to prevent bronchopulmonary dysplasia (BPD) may depend on local BPD incidence rates. Moreover, vitamin A may prove a valuable tool in reducing threshold retinopathy of prematurity (ROP), seen endemically in developing countries (Darlow and Graham 2007).
However, and of note, our study also demonstrated that a substantial percentage of systematic Cochrane reviews from developing countries were inconclusive and failed to provide any recommendation with regard to a specific intervention. This is in line with previous reports on this subject (Girisch et al. 2012; Mandel et al. 2006; Sinclair et al. 2003; Willhelm et al. 2012) that have also demonstrated that a substantial number of systematic reviews from highly industrialized Western countries provide only inconclusive recommendations. These reviews usually conclude that, following an extensive literature search and appraisal, insufficient trial evidence was found to guide clinical practice. Often only a trend can be seen, or statistically significant changes can be seen for short-term outcome parameters (e.g., ventilated days) (Mariani et al. 1999) but not for long-term outcome parameters (e.g., the incidence of death or chronic lung disease at 36 weeks, intraventricular hemorrhage grade 3 or 4, or periventricular leucomalacia) (Woodgate and Davies 2001). Moreover, it is noteworthy that a recent analysis demonstrated that many apparently conclusive Cochrane neonatal meta-analyses may become inconclusive when the statistical analyses take into account the risk of random error due to repetitive testing (Brok et al. 2009).
The most common reasons for failure to generate specific recommendations in our analysis were usually given as a small number of patients, poor and insufficient methodology, and heterogeneous study populations (data not shown; for more information please refer to Girisch et al. 2012 and Willhelm et al. 2012). Although reporting clinical uncertainty and thereby generating new research questions is a fundamental driving force for EBM, clinicians at the bedside will find the lack of specific recommendations frustrating and unhelpful (Willhelm et al. 2012). However, by identifying important gaps in the evidence, Cochrane reviews have the potential to promote high-quality RCTs (e.g., in the field of perinatology, collaborative quality improvement initiatives such as the WOMBAT Collaboration in Australasia (see https://www.wombatcollaboration.net/) (Willhelm et al. 2012). This can be illustrated by the fact that several recent large RCTs in perinatal medicine in industrialized countries have been undertaken when Cochrane reviews have highlighted important areas of clinical uncertainty. Examples relevant to pregnancy and childbirth management strategies have been well described (Dodd and Crowther 2006; Willhelm et al. 2012); recent examples of neonatal interventions include a) the Benefits of Oxygen Saturation Targeting (BOOST) and (b)Pulse Oximetry Saturation Trial for Prevention of Retinopathy of Prematurity (POST ROP) (Askie and Henderson-Smart 2001) and (c) the Caffeine for Apnoea of Prematurity (CAP) trial (Schmidt et al. 2007). Moreover, clinical researchers have conceptualized a research cycle that includes systematic review and observations of the effects in practice (Henderson-Smart et al. 2003).
Given the limited financial and human resources that are available in the medical arena in the developing world, future emphasis must be on long-term outcomes that are vital to infants and children and their families, as well as to healthcare workers. Importantly, in the future the effects of interventions not only on survival, but also on long-term morbidity, must be considered (Brok et al. 2009; Willhelm et al. 2012). This change in paradigm is particularly important in perinatal medicine, as there is potential for interventions to improve short-term outcomes but also to increase the likelihood of adverse longer-term outcomes in surviving neonates (e.g., administration of systemic corticosteroids in the first few days of postnatal life improves short-term respiratory function but also increases the rate of adverse neurological effects).
Our analysis has some weaknesses. Most importantly, there are many other trials and studies (non-RCTs) that have been performed in the developing world and published in the medical literature. Given the fact that only RCTs are considered for meta-analysis by The Cochrane Collaboration, the other studies and data were not analyzed in our work. However, the publication of Cochrane reviews may serve as a surrogate parameter for overall scientific endeavours in the medical arena.
In summary, this is the first systematic analysis of the potential role and limitations of Cochrane reviews in the fields of neonatology and neuropediatrics with regard to published reviews originating in low-income countries. Our findings demonstrate a need for more high-quality research that addresses specific medical problems and issues most relevant to countries in the developing world. It will be of paramount importance that funding and research agencies support research programs that address the most relevant issues in the field of pediatrics that are most pertinent to the developing world. Moreover, developing countries have to be active participants in researching these diseases; otherwise, they will not progress. If they invest their own money, they have a say in which health problems they want to tackle.
Our findings strongly suggest an ongoing need for high-quality research that addresses specific issues that are most relevant to the medical care of children in developing countries (e.g., treatments for drug-resistant tuberculosis; pediatric versions of HIV drugs; a test to determine the effectiveness of treatment of Chagas disease; new antibiotics, in the face of increasing resistance; and vaccines that do not need refrigeration or can be given without an injection).
About the Author(s)
Sascha Meyer, MD, University of Saarland, Medical School; University Children's Hospital of Saarland, Department of Paediatrics and Neonatology, Germany
Christiane Willhelm, MS, University Children's Hospital of Saarland, Department of Paediatrics and Neonatology, Germany
Wolfgang Girisch, MS, University Children's Hospital of Saarland, Department of Paediatrics and Neonatology, Germany
Sven Gottschling, MD, University Children's Hospital of Saarland, Department of Paediatric Palliative Care, Germany
Stefan Gräber, MD, PhD, University Hospital of Saarland, Department of Biostatistics, Medical Informatics and Epidemiology, Germany
Ludwig Gortner, MD, Professor, University of Saarland, Medical School; University Children's Hospital of Saarland, Department of Paediatrics and Neonatology, Germany
Correspondence may be directed to: Dr. Sascha Meyer, University Children's Hospital of Saarland, Department of Paediatrics and Neonatology, Building 9, 66421 Homburg, Germany; Tel.: +49 (0)6841-1628374, E-mail: firstname.lastname@example.org.
Askie, L.M. and D.J. Henderson-Smart.2001. "Restricted versus Liberal Oxygen Exposure for Preventing Morbidity and Mortality in Preterm or Low Birth Weight Infants." Cochrane Database of Systematic Reviews (4): CD00107.
Brok, J., K. Thorlund, J. Wetterslev and C. Gluud. 2009. "Apparently Conclusive Meta-analyses May Be Inconclusive – Trial Sequential Analysis Adjustment of Random Error Risk Due to Repetitive Testing of Accumulating Data in Apparently Conclusive Neonatal Meta-analyses." International Journal of Epidemiology 38: 287–98.
Crowther, C.A., Z. Alfirevic and R.R. Haslam. 2004. "Thyrotropin-Releasing Hormone Added to Corticosteroids for Women at Risk of Preterm Birth for Preventing Neonatal Respiratory Disease." Cochrane Database of Systematic Reviews (2): CD000019.
Darlow, B.A. and P.J. Graham. 2007. "Vitamin A Supplementation to Prevent Mortality and Short and Long-term Morbidity in Very Low Birthweight Infants." Cochrane Database of Systematic Reviews (4): CD000501.
Davis, P.G. 2006. "Cochrane Reviews in Neonatology: Past, Present and Future." Seminars in Fetal & Neonatal Medicine 11: 111–16.
Dodd, J.M. and C.A. Crowther. 2006. "Cochrane Reviews in Pregnancy: The Role of Perinatal Randomized Trials and Systematic Reviews in Establishing Evidence." Seminars in Fetal & Neonatal Medicine 11: 97–103.
Garner, P., M. Meremikwu, J. Volmink, Q. Xu and H. Smith. 2004. "Evidence into Practice: Middle and Low Income Countries Get it Together." BMJ 329: 1036–9.
Girisch, W., C. Willhelm, S. Gottschling, L. Gortner and S. Meyer. 2012. "Role of Cochrane Reviews in Pediatric Neurology." Pediatric Neurology 46(2): 63–9. doi: 10.1016/j.pediatrneurol.2011.11.021.
Henderson-Smart, D.J., D. Osborn, N. Evans, P. Beeby and H. Jeffery. 2003. "Do We Practice Evidence-Based Care in Our Neonatal Intensive Care Units?" Clinics in Perinatology 30(2): 333–42.
Henderson-Smart, D.J., P. Lumbiganon, M.R. Festin, J.J. Ho, H. Mohammad, S.J. McDonald et al. 2007. "Optimising Reproductive and Child Health Outcomes by Building Evidence-Based Research and Practice in South East Asia (SEA-Orchid): Study Protocol." BMC Medical Research Methodology 7: 43;doi: 10.1186/1471-2288-7–43.
Mandel, D., Y. Littner, F.B. Mimouni and R. Lubetzky. 2006. "Conclusiveness of the Cochrane Neonatal Reviews: A Systematic Analysis." Acta Paediatrica 95: 1209–12.
Mariani, G., J. Cifuentes and W.A. Carlo. 1999. "Randomized Trial of Permissive Hypercapnia in Preterm Infants." Pediatrics 104(5 Pt. 1): 1082–8.
McGuire, W, P.W. Fowlie and R.F. Soll. 2010. "What Has the Cochrane Collaboration Ever Done for Newborn Infants?" Archives of Disease in Childhood – Fetal and Neonatal Edition 95(1): F2–6.
Schmidt, B., R.S. Roberts, P. Davis, L.W. Doyle, K.J. Barrington, A. Ohlsson et al. 2007. "Caffeine for Apnea of Prematurity Trial Group. Long-term Effects of Caffeine Therapy for Apnea of Prematurity." The New England Journal of Medicine 357(19): 1893–902.
Sinclair, J.C. 2004. "Evidence-Based Therapy in Neonatology: Distilling the Evidence and Applying It in Practice." Acta Paediatrica 93: 1146–52.
Sinclair, J.C., D.E. Haughton, M.B. Bracken, J.D. Horbar and R.F. Soll. 2003. "Cochrane Neonatal Systematic Reviews: A Survey of the Evidence for Neonatal Therapies." Clinics in Perinatology 30(2): 285–304.
Willhelm, C., W. Girisch, L. Gortner and S. Meyer. 2012. "Evidence-Based Medicine and Cochrane Reviews in Neonatology: Quo Vadis? Acta Paediatrica 101(4): 352–3. doi: 10.1111/j.1651-2227.2011.02559.x.
Woodgate, P.G. and M.W. Davies. 2001. "Permissive Hypercapnia for the Prevention of Morbidity and Mortality in Mechanically Ventilated Newborn Infants." Cochrane Database of Systematic Reviews (2): CD002061.
Be the first to comment on this!
Personal Subscriber? Sign In
Note: Please enter a display name. Your email address will not be publically displayed