| Sign In to gain access to subscriptions and/or personal tools. |
Introduction to Cumulative Meta-analysis in Dentistry: Lessons Learned from Undertaking a Cumulative Meta-analysis in Periodontology
1 Eastman Dental Institute for Oral Health Care Science, University College London, 256 Gray’s Inn Road, London, WC1X 8LD, UK; Correspondence: * corresponding author, d.moles{at}eastman.ucl.ac.uk
Improving health and well-being from the consideration of isolated studies is problematic. Systematic reviews have been developed to address this problem and may include a quantitative data synthesis in the form of a meta-analysis, or a cumulative meta-analysis. The value of systematic reviews depends greatly on the availability and quality of the results of primary research. The objective of the current project was to demonstrate the technique of cumulative meta-analysis in dentistry using data from a previously published systematic review. The process highlights an issue that some trials could not be synthesized due to the lack of reporting of measures of variation. This represents a potential source of bias. Investigators are encouraged to consider their trials as part of an information continuum and to report sufficient detail to permit the trials’ incorporation into subsequent syntheses.
Key Words: systematic review • clinical trials • periodontal diseases • meta-analysis
Improving health and well-being from the consideration of studies isolated from each other is problematic. Systematic reviews can address this problem and aid decision-making. A systematic review is a review of a clearly formulated question that attempts to minimize bias using systematic and explicit methods to identify, select, critically appraise, and summarize relevant research. Once the available data about a particular topic have been systematically collected and quality-appraised, the information is pooled for analysis. Most commonly, this will take the form of a series of ‘evidence tables’ that summarize the findings from the studies included in the review. Systematic reviews may also include meta-analyses, the statistical technique of quantitatively synthesizing the review data to give an overall estimate of the effect size and its precision (Egger et al., 2001). The standard approach for undertaking a meta-analysis is to pool all available suitable data simultaneously at a single point in time. This provides a single estimate of the effect size of interest, and an associated confidence interval as a measure of the imprecision of the point estimate. Complementary to this is the technique of cumulative meta-analysis (Lau et al., 1992). Cumulative meta-analysis is the product of performing a new meta-analysis every time a new piece of evidence emerges (Lau et al., 1995). This permits evaluation of the additional contributions made by individual studies to the cumulatively pooled results of the preceding studies. Examples of the value of these techniques can be found in the medical literature. One such example consists of 33 sequential trials, between 1959 and 1988, examining the use of intravenous streptokinase for myocardial infarction, involving the enrollment of 36,974 patients. Had a cumulative meta-analysis been performed, a consistent statistically significant benefit could have been demonstrated as early as 1973 after only 2432 patients were enrolled (Lau et al., 1992). Cumulative meta-analyses have also been used to monitor the discrepancy between the emergence of sufficient evidence of the effectiveness of interventions and the delay in the recommendation of their adoption by clinical experts. Similarly, the delay before recommending the cessation of harmful or ineffective interventions has also been examined by this approach (Antman et al., 1992). For meta-analysis to be carried out, primary research must be reported in sufficient detail for subsequent analysis. We have reviewed the reporting of randomized control trials in periodontology, and found that it was not always adequate (Montenegro et al., 2002) and does not always follow the CONSORT guidelines (Begg et al., 1996). Our previous research focused solely on the quality of the reporting of methodological aspects of randomized controlled trials and refrained from a consideration of the quality of reporting of the actual results of the trials. The aim of the current research was to illustrate the potential value of the cumulative meta-analysis approach in dentistry. However, the topic chosen as the subject for the cumulative meta-analysis also illustrates some of the difficulties and potential biases that may arise when results of trials are not reported in sufficient detail.
The starting point for the current investigation was the systematic review of the effectiveness of adjunctive systemic antimicrobials for the treatment of periodontitis undertaken by the European Federation of Periodontology (Herrera et al., 2002). The focused question of that systematic review was, "In patients with chronic or aggressive periodontitis, what is the effect of systemic antimicrobials as an adjunct to scaling and root planing (SRP+AB) compared with SRP alone, in terms of clinical outcomes?" The review was restricted to randomized-controlled trials (RCT) or controlled clinical trials (CCT) of at least 6 months’ duration, and the reviewers identified 25 articles that fulfilled their inclusion criteria. The primary outcome measures reported in the review were differences in mean change in clinical attachment level (CAL) between test and control groups, and differences in mean change in probing pocket depths (PPD). For the purpose of the current exercise, the analyses are restricted to changes in CAL, and further restricted to exclude refractory/aggressive/rapidly progressive periodontal diseases or studies in which the antimicrobial was used at sub-therapeutic levels. Thus, the outcome of interest is the difference in mean CAL change between control (SRP) and test (SRP+AB) groups, such that a positive difference favors the antimicrobial therapy while a negative result favors the control. The authors of the original systematic review supplied copies of the articles that fulfilled their inclusion criteria. They marked these with highlighter pen to indicate the data that they abstracted. They also provided access to their original computer spreadsheets containing the abstracted data. These were verified and supplemented by the addition of standard deviations (SD) for change in CAL, where these had not been reported in the original articles but where there was sufficient information to calculate them directly from standard errors (SE) and sample sizes, or where it was possible to impute them. Conventional and cumulative meta-analyses were undertaken with the use of both fixed- and random-effects models, with the ‘Meta-View’ software incorporated within the Cochrane ‘Review Manager’ program (Cochrane Collaboration, 2003). This was achieved by use of an iterative approach, in which a series of separate meta-analyses was undertaken to include all the studies available at each information step. Thus, for each occasion that information became available, the new information was incorporated into the existing dataset, and the meta-analysis was repeated to update the estimates. This was undertaken separately for patient-level means of difference in CAL change in:
For trials in which a single control group was compared with multiple test arms (e g., several different antibiotic regimes), the test arms were combined into a single intervention group with summary weighted means and SDs. This was undertaken to avoid double-counting the control patients, which would have artificially inflated the sample size. Studies cannot be incorporated into the meta-analyses if there is no information on the study variability (SD or SE). This lack of information represents a possible source of bias. We investigated the potential for bias graphically by plotting the point estimates from these studies along with the means and 95% confidence intervals from those studies in which the information was available, to see if the point estimates were consistent with the overall pattern.
All/Moderate Sites Only eight of 25 trials contained sufficient information to be incorporated into meta-analyses of the responses of all/moderate sites to adjunctive systemic antibiotics (Lindhe et al., 1983b; Magnusson et al., 1994; Berglundh et al., 1998; Flemmig et al., 1998; Palmer et al., 1998; Winkel et al., 1999; Caton et al., 2000; Sigusch et al., 2001). However, of these, two trials were based on patients with rapidly progressive disease (Magnusson et al., 1994; Sigusch et al., 2001), and one used an antimicrobial at sub-therapeutic doses (Caton et al., 2000). These three trials were therefore excluded from the meta-analyses. The remaining five trials investigated the effectiveness of a variety of antibiotic systems: long-term tetracycline (Lindhe et al., 1983b); Augmentin® (Magnusson et al., 1994; Winkel et al., 1999); metronidazole (Palmer et al., 1998); and metronidazole and amoxicillin in combination (Berglundh et al., 1998; Flemmig et al., 1998). The forest plots for the conventional and cumulative fixed-effects meta-analyses are shown in Fig. 1  .
The overall effect size for adjunctive antimicrobials in all/moderate sites from the meta-analyses is a mean difference of 0.18 mm (95% CI 0.00, 0.37; P = 0.05) in favor of the adjunctive antimicrobial. The test for heterogeneity gave a chi-squared value of 1.00, df 4, P = 0.91, showing no evidence of any difference in the effect between the different trials where the different antibiotic regimes were used (Fig. 1  ). The results were based on measurements taken from 147 patients in total up to the year 1999. The cumulative meta-analysis indicated that statistical significance at the P < 0.05 level was achieved marginally earlier in 1998 (Fig. 1).
Deep Sites
The conventional meta-analysis indicates a mean difference of 0.37 mm (95% CI 0.12, 0.61; P = 0.004). The test for heterogeneity gave a chi-squared value of 1.99, df 4, P = 0.74, indicating no evidence of a difference in the results from the different antimicrobials (Fig. 2 ). For these deeper sites, the cumulative meta-analysis indicates that a statistically significant result could not have been detected any earlier.
Potential Bias from the Inability to Include Studies that Reported Only Point Estimates of Effect Size
The technique of performing cumulative meta-analyses is established in the medical literature. However a MEDLINE search (October, 2003) failed to find any reference to the use of the technique within any dental journal. In this demonstration exercise, pooling of the available information, whether by cumulative or conventional meta-analyses, indicates a small statistically significant benefit in terms of clinical attachment gain for scaling and root planing with adjunctive systemic antibiotics over and above the gain obtained from scaling and root planing alone. However, it is not appropriate to use the limited results of the current investigation as a basis for making recommendations about the adjunctive use of systemic antimicrobials in periodontal therapy. Not all the relevant trials provided sufficient information for incorporation within the meta-analyses, and the results may be subject to bias. The magnitude of the benefit is small and difficult to place in clinical context. Further, this research made no attempt to assess the negative consequences of such therapies (e g., the potential development of antibiotic-resistant strains of bacteria). When interpreting meta-analyses, whether conventional or cumulative, it is important that one bear in mind the potential biases that may affect the results. The phenomenon of publication bias is well-documented in the literature, and this occurs when trials are published selectively, such that the published studies are not representative of all the studies that have been undertaken. For example, results that are negative or of small magnitude may be less likely to be submitted and/or accepted for publication. This has been the subject of considerable interest and debate in the medical literature, and techniques have been developed that aim to detect when this may have occurred (Egger et al., 1997; Sutton et al., 2000). However, in the absence of a true reference standard, proposed methods to detect or correct for publication bias remain controversial (Thornton and Lee, 2000).
We previously found that many authors of periodontal trials did not publish sufficient details to allow for a determination of whether their trials were randomized or not, or whether the randomization that was performed was adequate (Montenegro et al., 2002). The current exercise highlights another potential source of bias, in that not all of the trials identified in the review of Herrera et al.(2002) reported measures of the variation in the data (standard deviations, standard errors, or confidence intervals). As such, these trials could not be included in meta-analyses (Fig. 3 These foregoing considerations have important consequences for the way that clinical trials are designed and reported in dentistry. It is common for there to be multiple randomized controlled trials over time, all addressing the same or similar clinical questions. Thus, each individual trial should be viewed as forming part of a continuum and should not be designed, conducted, analyzed, and interpreted in isolation. The lessons from previous trials are enormously valuable in the design of subsequent trials, in addition to contributing to the overall cumulative knowledge base. However, a review of 26 reports of randomized trials in 5 prestigious medical journals found that, in 19 articles, the authors made no systematic attempt to set their trials’ results within the context of previous trials (Clarke and Chalmers, 1998). We would recommend the following for the design and reporting of clinical trials in dentistry:
The authors are grateful to David Herrera for providing access to the material used in the EFP systematic review. Salary support was provided by University College London (DM and IN), DE13850 and dsm (RN). DM was the recipient of a dsm-Forsyth Center for Evidence-Based Dentistry fellowship, Dental Service of Massachusetts, Inc. Received for publication April 16, 2004. Revision received December 23, 2004. Accepted for publication January 13, 2005.
Journal of Dental Research, Vol. 84, No. 4,
345-349 (2005) Related articles in Journal of Dental Research:
|
|||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
 |
|
|
 |
|
Sign In to gain access to subscriptions and/or personal tools.
TOP
ABSTRACT
INTRODUCTION
