DÄ internationalArchive27/2009Systematic Literature Reviews and Meta-Analyses

Review article

Systematic Literature Reviews and Meta-Analyses

Part 6 of a Series on Evaluation of Scientific Publications

Dtsch Arztebl Int 2009; 106(27): 456-63; DOI: 10.3238/arztebl.2009.0456

Ressing, M; Blettner, M; Klug, S J

Background: Because of the rising number of scientific publications, it is important to have a means of jointly summarizing and assessing different studies on a single topic. Systematic literature reviews, meta-analyses of published data, and meta-analyses of individual data (pooled reanalyses) are now being published with increasing frequency. We here describe the essential features of these methods and discuss their strengths and weaknesses.
Methods: This article is based on a selective literature search. The different types of review and meta-analysis are described, the methods used in each are outlined so that they can be evaluated, and a checklist is given for the assessment of reviews and meta-analyses of scientific articles.
Results: Systematic literature reviews provide an overview of the state of research on a given topic and enable an assessment of the quality of individual studies. They also allow the results of different studies to be evaluated together when these are inconsistent. Meta-analyses additionally allow calculation of pooled estimates of an effect. The different types of review and meta-analysis are discussed with examples from the literature on one particular topic.
Conclusions: Systematic literature reviews and meta-analyses enable the research findings and treatment effects obtained in different individual studies to be summed up and evaluated.
Dtsch Arztebl Int 2009; 106(27): 456–63
DOI: 10.3238/arztebl.2009.0456
Key words: literature search, systematic review, meta-analysis, clinical research, epidemiology
Every year, there is a great increase in the number of scientific publications. For example, the literature database PubMed registered 361 000 new publications in 1987, with 448 000 in 1997 and 766 000 in 2007 (research in Medline, last updated in January 2009). These figures make it clear how increasingly difficult it is for physicians in private practice, clinicians and scientists to obtain comprehensive current information on any given medical topic. This is why it is necessary to summarize and critically analyze individual studies on the same theme.

Summaries of individual studies are mostly prepared when the results of individual studies are unclear or inconsistent. They are also used to study relationships for which the individual studies do not have adequate statistical power, as the number of cases is too low (1).

The Cochrane Collaboration undertakes systematic processing and summary of the primary literature for many therapeutic topics, particularly randomized clinical studies (www.cochrane.org). They have published a handbook for the performance of systematic reviews and meta-analyses of randomized clinical studies (2). Cook et al. have published methodological guidelines for this process (3). Instructions of this sort help to lay down standards for the summary of individual studies. Guidelines have also been drawn up for the publication of meta-analyses on randomized clinical studies (4) and on observational studies (5).

Publications on individual studies may be summarized in various forms (1, 610):

- Narrative reviews
- Systematic review articles
- Meta-analyses of published data
- Pooled reanalyses (meta-analyses with individual data).

These terms are often not clearly allocated in the literature. The aim of the present article is to describe and distinguish these forms and to allow the reader to perform a critical analysis of the results of individual studies and the quality of the systematic review or meta-analysis.

Methods
The various types of systematic reviews and meta-analyses of scientific articles will be defined and the procedure will be explained. A selective literature search was performed for this purpose.

A "review" is the qualitative summary of the results of individual studies (1). A distinction is made between narrative reviews and systematic reviews (Table 1 gif ppt). Narrative reviews (A) mostly provide a broad overview of a specific topic (1, 11). They are therefore a good way of rapidly obtaining current information on research on a given topic. However, the articles to be included are selected subjectively and unsystematically (1, 11). For some time, the Deutsches Ärzteblatt has been using the term "selective literature review" for this type of review. Narrative reviews will not be further discussed in this article.

In contrast, systematic review articles (B) claim that, if possible, they consider all published studies on a specific theme—after the application of previously defined inclusion and exclusion criteria (11). The aim is to extract relevant information systematically from the publications. What is important is to analyze the methodological quality of the included publications and to investigate the reasons for any differences between the results in the different studies. The results of each study are presented and analyzed according to defined criteria, such as study design and mode of recruitment.

The same applies to the meta-analysis of published data (C). In addition, the results are quantitatively summarized using statistical methods and pooled effect estimates (Glossary gif ppt) are calculated (1).

A pooled reanalysis (D) is a quantitative compilation of original data (Glossary) from individual studies for combined analysis (1). The authors of each study included in the analysis then provide individual data (Glossary). These are then compiled in a combined database and analyzed according to standard criteria fixed in advance. This form of pooled reanalysis is also referred to as "meta-analysis of individual data".

In a prospectively planned meta-analysis (E), the summary of the individual studies and the combined analysis is included in the planning of the individual studies. For this reason, the individual studies are performed in a standard manner. Prospectively planned meta-analyses will not be further discussed in this article.

It is essential for all forms of summary—except the narrative review—that they should include a prospectively prepared study protocol, with descriptions of the questions to be answered, the hypotheses, the inclusion and exclusion criteria, the selection of studies, and, where applicable, the combination of the data and the recoding of the individual data (only for pooled reanalysis).

Types of study summaries
The procedure for the summary of the studies will now be presented (modified from [7, 10, 12, 13]). This is intended to enable the reader to assess whether a given summary fulfils specific criteria (Box gif ppt).


1. Was the question to be answered specified in advance?
The question to be answered in the review or meta-analysis and the hypotheses must be clearly defined and laid down in writing prospectively in a study protocol.

2. Were the inclusion and exclusion criteria specified in advance?
On the basis of the inclusion and exclusion criteria, it is decided whether the studies found in the literature search (see point 3) are included in the review/meta-analysis.

3. Were precautions taken to find all studies performed with reference to the specific question to be answered?
An extensive literature search must be performed for studies on the topic. If at all possible, this should be in several literature databases. To avoid bias, all relevant articles should be considered, whatever their language. Moreover, a search should be performed in the literature lists of the articles found and for unpublished studies in congress volumes, as well as with search machines on the Internet.

4. Was the relevant information extracted from the published articles or were the original data combined?
For a systematic review article (B) and for a meta-analysis of published data (C), relevant information should be extracted from the publications.

For a pooled reanalysis (D), authors of all identified studies must be contacted and requested to provide individual data. The individual data must then be coded according to standard specifications, compiled in a combined database and analyzed.

5. Was a descriptive analysis of the data performed?
In all forms of summary, it is usual for the most important characteristics of the individual studies to be presented in overview tables. Table 2 (gif ppt) shows an example of such a table, taken from a meta-analysis with published data (C) (14). This helps to make the differences between the studies clear with respect to the data examined.

6. Were the calculations of the effect estimates of the individual studies and of the pooled effect estimate presented?
How were the effect estimates of the individual studies calculated?—Systematic review articles (B) usually contain tables with the effect estimates of the individual studies. In a meta-analysis of published data (C), the effect estimates of individual studies (for example, odds ratio or relative risk, see Glossary) are either directly extracted from the publications or recalculated in a standard manner from the data in each publication (Figure 1 gif ppt). Depending on the nature of the factors and target parameters (binary, categorical or continuous variables), a logistic or a linear regression model is used to calculate the effect estimates of the individual studies in the meta-analyses of published data (C) and pooled reanalyses (D).

How was the pooled effect estimate calculated?—The effect estimates of the individual studies are combined by statistical procedures to give a common pooled effect estimate (9) (Figure 1). In meta-analyses with published data (C), two methods are mostly used to calculate a pooled effect estimate: either the fixed effect model or the random effect model (15, 16). They differ with respect to assumptions about the heterogeneity of the estimate between individual studies (see point 7). The method used should be given in the publication and justified. The effect estimates of the individual studies and the pooled effect estimates can be graphically presented in the form of so-called forest plots (Glossary; Figure 1; [14]).

In pooled reanalyses (D), the pooled effect estimates are mostly calculated by logistic or linear regression. However, the statistical analysis must adequately allow for the origin of the data sets from different studies. The results of the pooled reanalyses can be presented like the results of a single combined study (Table 3 gif ppt).

7. Were problems considered in the interpretation of pooled estimates?
Was the heterogeneity between the estimates considered?—There may be marked differences between the estimates in the individual studies. This statistical heterogeneity (Glossary) between the studies may be caused by differences in study design, study populations (age, gender, ethnic group), methods of recruitment, diagnosis, or methods of measurement (17, 18). The methodological heterogeneity between the studies can be visualized in an overview table, in which the most important characteristics of the individual studies are presented (Table 2). The heterogeneity can be formally investigated with the help of statistical tests. If there is statistical heterogeneity between the studies, the random effect model, rather than the fixed effect model, should be used for the calculation of the pooled estimate (7, 15, 16). There is, however, no clear definition as to when the statistical heterogeneity between the studies is so large that the pooled effect estimate should not be calculated (1, 19). In addition, the heterogeneity between the studies should be examined by subgroup analysis (Glossary). For example, this might involve combined analysis of only studies with the same characteristics in the study population, such as homogenous age groups, the same ethnic groups or the same histological findings. Moreover, studies with the same characteristics—such as study quality or study size—may be considered separately in subgroup analyses. This may indicate whether the effect of the corresponding risk factors (Glossary) is different in the different subgroups.

Were sensitivity analyses performed?—Like subgroup analyses, sensitivity analyses (Glossary) serve to test the stability of the pooled estimate. It is, for example, possible that the pooled effect estimate is mainly determined by one large study. If this study is excluded from the analysis, the pooled effect estimate may change. This must be borne in mind in the discussion and interpretation of the results.

Was a possible publication bias considered?—A publication bias (Glossary) can be visualized with a so-called funnel plot (Glossary) (7, 2022). Figure 2 (gif ppt) shows an example with simulated data. In the upper funnel plot (Figure 2a), there is a roughly funnel shaped distribution of the effect estimates of the individual studies around the pooled effect estimates (middle broken line). There is no publication bias here. In the lower funnel plot (Figure 2b), the small studies are missing, which in this example show no increased risk. For this reason, there is probably a publication bias, because these studies had not been published.

8. How were the results interpreted?
In the interpretation of the results, possible limitations should be discussed and considered. For example, the reliability of the results of individual studies can be limited by the inadequate quality of the individual studies—for example, by selection of the study population or from aggregated data (Glossary).

Results
The method section describes the individual steps for the extraction of the relevant points which must be considered in the systematic summary of scientific articles (Box). This checklist can also be used to analyze the quality of systematic review articles or meta-analysis.

Publications on the association between the administration of oral contraceptives and the development of cervical carcinoma were used as examples of the performance of a systematic literature review (B), a meta-analysis of published data (C), and a pooled reanalysis (D). This association has been scientifically studied for a long period.

In 1996, La Vecchia et al. published a systematic review article (B) on this topic, including six studies (23). Their overview table contained a variety of information on the individual studies. No pooled effect estimate was calculated.

In 2003, Smith et al. (14) presented a meta-analysis of published data (C) of 28 studies on the same topic. The included studies were first summarized in a descriptive overview, as is common in systematic review articles (Table 2). This table shows that the study methods were heterogenous (Glossary); for example, HPV was detected in different ways (Table 2). The heterogeneity was also formally investigated with statistical tests and various subgroup analyses were performed. In contrast to the systematic review article (B) of LaVecchia et al., pooled effect estimates were calculated with the published data (Figure 1). The effect estimates for the individual studies and the pooled effect estimates with their confidence intervals (glossary) were presented as a forest plot (Figure 1).

In 2007, a pooled reanalysis (D) was published for 24 studies on the same topic for which the original data were available (24). In contrast to the meta-analysis of published data, the pooled effect estimates were calculated from the original data and only the combined results were presented (Table 3). This kind of analysis is only possible in a pooled reanalysis, as the original data with precise information on all parameters for each participant are then available. Nevertheless, here too it is necessary to consider that the individual data (Glossary) are derived from different studies.

Discussion
Systematic review articles (B) can provide a comprehensive overview of the current state of research (1). They are also necessary for the development of S2 and S3 guidelines for formal evidence-based research (25). Meta-analyses of published data (C) are performed to calculate additional pooled effect estimates from the individual studies (1). Like systematic review articles, they are feasible whether the authors of the original articles are prepared to cooperate or not.

The calculated pooled effect estimates may be of limited validity for various reasons. Firstly, it has not been clearly defined what is the maximum order of heterogeneity between the studies which is negligible and which then allows a meaningful calculation of a pooled effect estimate (1, 19). If the individual studies are too heterogenous, a pooled effect estimate should not be calculated. Secondly, the pooled effect estimate is mostly calculated from aggregated data. Subgroup analyses and the consideration of potential confounders (Glossary) are often impossible, or only possible to a limited extent (1, 19). Thirdly, publication bias is also a problem for the meta-analysis of published data.

In a pooled reanalysis (D), potential confounders and risk factors can be more easily considered (7), as they are usually only published in an aggregated form. With the individual data, the outcome parameters, risk factors, and confounders used in the analysis can be categorized in a standard manner and properly incorporated in the analysis. Individual data can be removed in accordance with the prospective specifications in the study protocol, without it being necessary to exclude the whole study. The disadvantages of pooled reanalysis are that it demands a great deal of time and money and that it is dependent on the willingness of the authors of the individual studies to cooperate. If not all authors send their individual data, this may result in biased results.

The level of evidence of the type of summary increases from the systematic review to the meta-analysis of published data to the pooled reanalysis. It is important that all three forms of summary should be performed with high quality.

Conflict of interest statement
The authors declare that there is no conflict of interest in the sense of the guidelines of the International Committee of Medical Journal Editors.

Manuscript received on 9 June 2008, revised version accepted
on 10 March 2009.

Translated from the original German by Rodney A. Yeates, M.A., Ph.D.


Corresponding author
PD Dr. rer. nat. et med. habil. Stefanie Klug, MPH
Institut für Medizinische Biometrie, Epidemiologie und Informatik (IMBEI)
Universitätsmedizin der Johannes Gutenberg-Universität Mainz
Obere Zahlbacher Str 69
55101 Mainz, Germany
klug@imbei.uni-mainz.de
1.
Blettner M, Sauerbrei W, Schlehofer B, Scheuchenpflug T, Friedenreich C: Vergleich von traditionellen Reviews, Metaanalysen und gepoolten Analysen zur Bewertung von Risikofaktoren. Informatik, Biometrie und Epidemiologie in Medizin und Biologie 1997; 28: 148–66.
2.
Higgins JPT, Green S (Hrsg.): Cochrane Handbook for Systematic Reviews of Interventions Version 5.0.1 [aktualisiert September 2008]. The Cochrane Collaboration 2008. www.cochrane-handbook.org.
3.
Cook DJ, Sackett DL, Spitzer WO: Methodologic guidelines for systematic reviews of randomized control trials in health care from the Potsdam Consultation on Meta-Analysis. J Clin Epidemiol 1995; 48: 167–71. MEDLINE
4.
Moher D, Cook DJ, Eastwood S, Olkin I, Rennie D, Stroup DF: Improving the quality of reports of meta-analyses of randomised controlled trials: the QUOROM statement. Quality of Reporting of Meta-analyses. Lancet 1999; 354: 1896–900. MEDLINE
5.
Stroup DF, Berlin JA, Morton SC et al.: Meta-analysis of observational studies in epidemiology. JAMA 2000; 263: 2008–12. MEDLINE
6.
Sutton AJ, Higgins JP: Recent developments in meta-analysis. Stat Med 2008; 27: 625–50. MEDLINE
7.
Blettner M, Schlattmann P: Meta-analysis in epidemiology. In: Ahrens W, Pigeot I (Hrsg.): Handbook of epidemiology. Berlin, Heidelberg, New York: Springer 2005; 829–59.
8.
Egger M, Smith GD, Phillips AN: Meta-analysis: principles and procedures. BMJ 1997; 315: 1533–7. MEDLINE
9.
Petitti DB: Meta-analysis, decision analysis, and cost-effectiveness analysis. Methods for quantitative synthesis in medicine. 2. Auflage New York: Oxford University Press 2000.
10.
Friedenreich CM: Methods for pooled analyses of epidemiologic studies. Epidemiology 1993; 4: 295–302. MEDLINE
11.
Montori VM, Swiontkowski MF, Cook DJ: Methodologic issues in systematic reviews and meta-analyses. Clin Orthop Relat Res 2003; 413: 43–54. MEDLINE
12.
Meerpohl JJ, Timmer A, Antes G: Systematische Übersichten als Entscheidungshilfe für die Praxis. Wie Sie die Informationsflut sinnvoll nutzen. MMW Fortschr Med 2008; 150: 41–3. MEDLINE
13.
Khan KS, Kunz R, Kleijnen J, Antes G: Five steps to conducting a systematic review. J R Soc Med 2003; 96: 118–21. MEDLINE
14.
Smith J, Green J, Berrington de Gonzalez A et al.: Cervical cancer and use of hormonal contraceptives: a systematic review. Lancet 2003; 361: 1159–67. MEDLINE
15.
DerSimonian R, Laird N: Meta-analysis in clinical trials. Control Clin Trials 1986; 7: 177–88. MEDLINE
16.
Normand SL: Meta-analysis: formulating, evaluating, combining, and reporting. Stat Med 1999; 18: 321–59. MEDLINE
17.
Higgins JP, Thompson SG: Quantifying heterogeneity in a meta-analysis. Stat Med 2002; 21: 1539–58. MEDLINE
18.
Deeks JJ, Altman DG, Bradburn MJ: Statistical methods for examining heterogeneity and combining results from several studies in meta-analysis. In: Egger M, Smith GD, Altman DG (Hrsg.): Systematic reviews in health care: meta-analysis in context. London: BMJ Publishing Group 2001; 285–312.
19.
Blettner M, Schlehofer B, Sauerbrei W: Grenzen von Metaanalysen aus publizierten Daten bei epidemiologischen Fragestellungen. Soz Präventivmed 1997; 42: 95–104. MEDLINE
20.
Dickersin K: The existence of publication bias and risk factors for its occurrence. JAMA 1990; 263: 1385–9. MEDLINE
21.
Ioannidis JP: Effect of the statistical significance of results on the time to completion and publication of randomized efficacy trials. JAMA 1998; 279: 281–6. MEDLINE
22.
Dwan K, Altman DG, Arnaiz JA et al.: Systematic review of the empirical evidence of study publication bias and outcome reporting bias. PLoS ONE 2008; 3: e3081. MEDLINE
23.
La Vecchia C, Tavani A, Franceschi S, Parazzini F: Oral contraceptives and cancer. A review of the evidence. Drug Saf 1996; 14: 260–72. MEDLINE
24.
International Collaboration of Epidemiological Studies of Cervical Cancer: Cervical cancer and hormonal contraceptives: collaborative reanalysis of individual data for 16,573 women with cervical cancer and 35,509 women without cervical cancer from 24 epidemiological studies. Lancet 2007; 370: 1609–21. MEDLINE
25.
Arbeitsgemeinschaft der Wissenschaftlichen Medizinischen Fachgesellschaften. Erarbeitung von Leitlinien für Diagnostik und Therapie: Methodische Empfehlungen („Leitlinie für Leitlinien“, Stand Dez. 2004). www.uni-duesseldorf.de/awmf/ll/ll_metho.htm.
e1.
Brinton LA, Huggins GR, Lehmann HF et al.: Long-term use of oral contraceptives and risk of invasive cervical cancer. Int J Cancer 1986; 38: 339–44. MEDLINE
e2.
Jones CJ, Brinton LA, Hammann RF et al.: Risk factors for in situ cervical cancer: results from a case-control-study. Cancer Res 1990; 50: 3657–62. MEDLINE
e3.
Peters RK, Thomas D, Hagan DG, Mack TM, Henderson BE: Risk factors for invasive cervical cancer among Latinas and non-Latinas in Los Angeles County. J Natl Cancer Inst 1986; 77: 1063–77. MEDLINE
e4.
Ebeling K, Nischan P, Schindler C: Use of oral contraceptives and risk of invasive cervical cancer in previously screened women. Int J Cancer 1987; 39: 427–30. MEDLINE
e5.
Brinton LA, Reeves WC, Brenes MM et al.: Oral contraceptive use and risk of invasive cervical cancer. Int J Epidemiol 1990; 19: 4–11. MEDLINE
e6.
Herrero R, Brinton LA, Reeves WC et al.: Injectable contraceptives and risk of invasive cervical cancer: evidence of an association. Int J Cancer 1990; 46: 5–7. MEDLINE
e7.
WHO: Invasive squamous-cell cervical carcinoma and combined oral contraceptives: results from a multinational study. WHO collaborative study of neoplasia and steroid contraceptives. Int J Cancer 1993; 55: 228–36. MEDLINE
e8.
WHO: Depot-medroxyprogesterone acetate (DMPA) and risk of invasive squamous cell cervical cancer. The WHO collaborative study of neoplasia and steroid contraceptives. Contraception 1992; 45: 299–312. MEDLINE
e9.
Thomas DB, Ray RM: Depot-medroxyprogesterone acetate (DMPA) and risk of invasive adenocarcinomas and adenosquamous carcinomas of the uterine cervix. WHO collaborative study of neoplasia and steroid contraceptives. Contraception 1995; 52: 307–12. MEDLINE
e10.
Thomas DB, Ye Z, Ray RM: Cervical carcinoma in situ and the use of depotmedroxyprogesterone acetate (DMPA). WHO collaborative study of neoplasia and steroid contraceptives. Contraception 1995; 51: 25–31. MEDLINE
e11.
Ye Z, Thomas DB, Ray RM: Combined oral contraceptives and risk of cervical carcinoma in situ. WHO collaborative study of neoplasia and steroid contraceptives. Int J Epidemiol 1995; 24: 19–26. MEDLINE
e12.
Thomas DB, Ray RM: Oral contraceptives and invasive adenocarcinomas and adenosquamous carcinomas of the uterine cervix. The WHO collaborative study of neoplasia and steroid contraceptives. Am J Epidemiol 1996; 144: 281–89. MEDLINE
e13.
Ursin G, Peters RK, Henderson BE, d'Ablaing G, Monroe KR, Pike MC: Oral contraceptive use and adenocarcinoma of the cervix. Lancet 1994; 344: 1390–4. MEDLINE
e14.
Cuzick J, Sasieni P, Singer A: Risk factors for invasive cervical cervix cancer in young women. Eur J Cancer 1996; 32A: 836–41. MEDLINE
e15.
Madeleine MM, Daling JR, Schwartz SM et al.: Human papillomavirus and long-term oral contraceptive use increase the risk of adenocarcinoma in situ of the cervix. Cancer Epidemiol Biomarkers Prev 2001; 10: 171–77. MEDLINE
e16.
Berrington A, Jha Pk, Peto J, Green J, Hermon C, UK national case-control study of cervical cancer: Oral contraceptives and cervical cancer. Lancet 2002; 360: 410. MEDLINE
e17.
Moreno V, Bosch FX, Munoz N et al.: Effect of oral contraceptives on risk of cervical cancer in women with human papillomavirus infection: the IARC multicentric case control study. Lancet 2002; 359: 1085–92. MEDLINE
Institut für Medizinische Biometrie, Epidemiologie und Informatik, Universitätsmedizin der Johannes Gutenberg-Universität Mainz: Dr. med. Ressing, Prof. Dr. rer. nat. Blettner, PD Dr. rer. nat. et med. habil. Klug
1. Blettner M, Sauerbrei W, Schlehofer B, Scheuchenpflug T, Friedenreich C: Vergleich von traditionellen Reviews, Metaanalysen und gepoolten Analysen zur Bewertung von Risikofaktoren. Informatik, Biometrie und Epidemiologie in Medizin und Biologie 1997; 28: 148–66.
2. Higgins JPT, Green S (Hrsg.): Cochrane Handbook for Systematic Reviews of Interventions Version 5.0.1 [aktualisiert September 2008]. The Cochrane Collaboration 2008. www.cochrane-handbook.org.
3. Cook DJ, Sackett DL, Spitzer WO: Methodologic guidelines for systematic reviews of randomized control trials in health care from the Potsdam Consultation on Meta-Analysis. J Clin Epidemiol 1995; 48: 167–71. MEDLINE
4. Moher D, Cook DJ, Eastwood S, Olkin I, Rennie D, Stroup DF: Improving the quality of reports of meta-analyses of randomised controlled trials: the QUOROM statement. Quality of Reporting of Meta-analyses. Lancet 1999; 354: 1896–900. MEDLINE
5. Stroup DF, Berlin JA, Morton SC et al.: Meta-analysis of observational studies in epidemiology. JAMA 2000; 263: 2008–12. MEDLINE
6. Sutton AJ, Higgins JP: Recent developments in meta-analysis. Stat Med 2008; 27: 625–50. MEDLINE
7. Blettner M, Schlattmann P: Meta-analysis in epidemiology. In: Ahrens W, Pigeot I (Hrsg.): Handbook of epidemiology. Berlin, Heidelberg, New York: Springer 2005; 829–59.
8. Egger M, Smith GD, Phillips AN: Meta-analysis: principles and procedures. BMJ 1997; 315: 1533–7. MEDLINE
9. Petitti DB: Meta-analysis, decision analysis, and cost-effectiveness analysis. Methods for quantitative synthesis in medicine. 2. Auflage New York: Oxford University Press 2000.
10. Friedenreich CM: Methods for pooled analyses of epidemiologic studies. Epidemiology 1993; 4: 295–302. MEDLINE
11. Montori VM, Swiontkowski MF, Cook DJ: Methodologic issues in systematic reviews and meta-analyses. Clin Orthop Relat Res 2003; 413: 43–54. MEDLINE
12. Meerpohl JJ, Timmer A, Antes G: Systematische Übersichten als Entscheidungshilfe für die Praxis. Wie Sie die Informationsflut sinnvoll nutzen. MMW Fortschr Med 2008; 150: 41–3. MEDLINE
13. Khan KS, Kunz R, Kleijnen J, Antes G: Five steps to conducting a systematic review. J R Soc Med 2003; 96: 118–21. MEDLINE
14. Smith J, Green J, Berrington de Gonzalez A et al.: Cervical cancer and use of hormonal contraceptives: a systematic review. Lancet 2003; 361: 1159–67. MEDLINE
15. DerSimonian R, Laird N: Meta-analysis in clinical trials. Control Clin Trials 1986; 7: 177–88. MEDLINE
16. Normand SL: Meta-analysis: formulating, evaluating, combining, and reporting. Stat Med 1999; 18: 321–59. MEDLINE
17. Higgins JP, Thompson SG: Quantifying heterogeneity in a meta-analysis. Stat Med 2002; 21: 1539–58. MEDLINE
18. Deeks JJ, Altman DG, Bradburn MJ: Statistical methods for examining heterogeneity and combining results from several studies in meta-analysis. In: Egger M, Smith GD, Altman DG (Hrsg.): Systematic reviews in health care: meta-analysis in context. London: BMJ Publishing Group 2001; 285–312.
19. Blettner M, Schlehofer B, Sauerbrei W: Grenzen von Metaanalysen aus publizierten Daten bei epidemiologischen Fragestellungen. Soz Präventivmed 1997; 42: 95–104. MEDLINE
20. Dickersin K: The existence of publication bias and risk factors for its occurrence. JAMA 1990; 263: 1385–9. MEDLINE
21. Ioannidis JP: Effect of the statistical significance of results on the time to completion and publication of randomized efficacy trials. JAMA 1998; 279: 281–6. MEDLINE
22. Dwan K, Altman DG, Arnaiz JA et al.: Systematic review of the empirical evidence of study publication bias and outcome reporting bias. PLoS ONE 2008; 3: e3081. MEDLINE
23. La Vecchia C, Tavani A, Franceschi S, Parazzini F: Oral contraceptives and cancer. A review of the evidence. Drug Saf 1996; 14: 260–72. MEDLINE
24. International Collaboration of Epidemiological Studies of Cervical Cancer: Cervical cancer and hormonal contraceptives: collaborative reanalysis of individual data for 16,573 women with cervical cancer and 35,509 women without cervical cancer from 24 epidemiological studies. Lancet 2007; 370: 1609–21. MEDLINE
25. Arbeitsgemeinschaft der Wissenschaftlichen Medizinischen Fachgesellschaften. Erarbeitung von Leitlinien für Diagnostik und Therapie: Methodische Empfehlungen („Leitlinie für Leitlinien“, Stand Dez. 2004). www.uni-duesseldorf.de/awmf/ll/ll_metho.htm.
e1. Brinton LA, Huggins GR, Lehmann HF et al.: Long-term use of oral contraceptives and risk of invasive cervical cancer. Int J Cancer 1986; 38: 339–44. MEDLINE
e2. Jones CJ, Brinton LA, Hammann RF et al.: Risk factors for in situ cervical cancer: results from a case-control-study. Cancer Res 1990; 50: 3657–62. MEDLINE
e3. Peters RK, Thomas D, Hagan DG, Mack TM, Henderson BE: Risk factors for invasive cervical cancer among Latinas and non-Latinas in Los Angeles County. J Natl Cancer Inst 1986; 77: 1063–77. MEDLINE
e4. Ebeling K, Nischan P, Schindler C: Use of oral contraceptives and risk of invasive cervical cancer in previously screened women. Int J Cancer 1987; 39: 427–30. MEDLINE
e5. Brinton LA, Reeves WC, Brenes MM et al.: Oral contraceptive use and risk of invasive cervical cancer. Int J Epidemiol 1990; 19: 4–11. MEDLINE
e6. Herrero R, Brinton LA, Reeves WC et al.: Injectable contraceptives and risk of invasive cervical cancer: evidence of an association. Int J Cancer 1990; 46: 5–7. MEDLINE
e7. WHO: Invasive squamous-cell cervical carcinoma and combined oral contraceptives: results from a multinational study. WHO collaborative study of neoplasia and steroid contraceptives. Int J Cancer 1993; 55: 228–36. MEDLINE
e8. WHO: Depot-medroxyprogesterone acetate (DMPA) and risk of invasive squamous cell cervical cancer. The WHO collaborative study of neoplasia and steroid contraceptives. Contraception 1992; 45: 299–312. MEDLINE
e9. Thomas DB, Ray RM: Depot-medroxyprogesterone acetate (DMPA) and risk of invasive adenocarcinomas and adenosquamous carcinomas of the uterine cervix. WHO collaborative study of neoplasia and steroid contraceptives. Contraception 1995; 52: 307–12. MEDLINE
e10. Thomas DB, Ye Z, Ray RM: Cervical carcinoma in situ and the use of depotmedroxyprogesterone acetate (DMPA). WHO collaborative study of neoplasia and steroid contraceptives. Contraception 1995; 51: 25–31. MEDLINE
e11. Ye Z, Thomas DB, Ray RM: Combined oral contraceptives and risk of cervical carcinoma in situ. WHO collaborative study of neoplasia and steroid contraceptives. Int J Epidemiol 1995; 24: 19–26. MEDLINE
e12. Thomas DB, Ray RM: Oral contraceptives and invasive adenocarcinomas and adenosquamous carcinomas of the uterine cervix. The WHO collaborative study of neoplasia and steroid contraceptives. Am J Epidemiol 1996; 144: 281–89. MEDLINE
e13. Ursin G, Peters RK, Henderson BE, d'Ablaing G, Monroe KR, Pike MC: Oral contraceptive use and adenocarcinoma of the cervix. Lancet 1994; 344: 1390–4. MEDLINE
e14. Cuzick J, Sasieni P, Singer A: Risk factors for invasive cervical cervix cancer in young women. Eur J Cancer 1996; 32A: 836–41. MEDLINE
e15. Madeleine MM, Daling JR, Schwartz SM et al.: Human papillomavirus and long-term oral contraceptive use increase the risk of adenocarcinoma in situ of the cervix. Cancer Epidemiol Biomarkers Prev 2001; 10: 171–77. MEDLINE
e16. Berrington A, Jha Pk, Peto J, Green J, Hermon C, UK national case-control study of cervical cancer: Oral contraceptives and cervical cancer. Lancet 2002; 360: 410. MEDLINE
e17. Moreno V, Bosch FX, Munoz N et al.: Effect of oral contraceptives on risk of cervical cancer in women with human papillomavirus infection: the IARC multicentric case control study. Lancet 2002; 359: 1085–92. MEDLINE