| Sign In to gain access to subscriptions and/or personal tools. |
Detection of Disease Outbreaks by the Use of Oral Manifestations
1 Department of Biomedical Informatics, School of Medicine, University of Pittsburgh, 3501 Terrace St., Suite 339, Pittsburgh, PA 15213, USA; Correspondence: * corresponding author, mit7{at}pitt.edu
Oral manifestations of diseases caused by bioterrorist agents could be a potential data source for biosurveillance. This study had the objectives of determining the oral manifestations of diseases caused by bioterrorist agents, measuring the prevalence of these manifestations in emergency department reports, and constructing and evaluating a detection algorithm based on them. We developed a software application to detect oral manifestations in free text and identified positive reports over three years of data. The normal frequency in reports for oral manifestations related to anthrax (including buccal ulcers-sore throat) was 7.46%. The frequency for tularemia was 6.91%. For botulism and smallpox, the frequencies were 0.55% and 0.23%. We simulated outbreaks for these bioterrorism diseases and evaluated the performance of our system. The detection algorithm performed better for smallpox and botulism than for anthrax and tularemia. We found that oral manifestations can be a valuable tool for biosurveillance.
Key Words: bioterrorism • dental informatics • dental public health • early detection • oral manifestations
The mailing of letters containing anthrax in 2001 (Jernigan et al., 2001) and the emergence of Severe Acute Respiratory Syndrome in 2003 (Samaranayake and Peiris, 2004) have made the early detection of disease outbreaks a significant concern. Bioterrorist and naturally occurring outbreaks require "extreme timeliness of detection" (Wagner et al., 2001a) to safeguard public health and mitigate deleterious effects. In this study, we examine a novel data source for electronic biosurveillance (Wagner, 2006): oral manifestations. Some of the diseases resulting from bioterrorist agents cause oral manifestations that are likely to be detected in dental or medical care settings, and some (Flores et al., 2003) have advocated a role for dentists in detecting outbreaks. Oral manifestations could be used for biosurveillance, especially in combination with other data, such as signs and symptoms of respiratory infections (Chapman et al., 2004b) and over-the-counter medication sales (Goldenberg et al., 2002). Biosurveillance systems use a three-phase approach to detect outbreaks (Centers for Disease Control and Prevention, 2008). First, they gather early symptom data (Wagner et al., 2001b) from systems such as hospital medical records. Second, they aggregate those data in real time, creating an electronic signal. Third, the system issues an alarm (Duchin, 2003) when the signal deviates from the predicted (i.e., sudden spike in clinical cases). The alarm prompts public health officials to investigate and take action. Biosurveillance systems are evaluated by assessments of sensitivity, specificity, and timeliness (Buehler et al., 2004). This evaluation is made difficult by the lack of true ‘gold standards’ (i.e., data from real outbreaks). Instead, detection algorithms are challenged to detect artificial spikes in the data from simulated outbreaks (Goldenberg et al., 2002; Reis et al., 2003; Wallstrom et al., 2005). To our knowledge, this is the first study that describes a new detection algorithm based on oral manifestations of bioterrorist agents. For 4 diseases (anthrax, botulism, smallpox, and tularemia), we developed baseline frequencies for oral manifestations based on clinical historical data from the Emergency Department at the University of Pittsburgh Medical Center. We evaluated the performance of the detection algorithm using simulated outbreaks.
The project consisted of 4 phases: (1) identification of signs and symptoms in the head and neck region caused by 4 bioterrorist agents; (2) development of an algorithm to retrieve emergency department reports; (3) establishment of baseline frequencies of oral manifestations for each disease; and (4) development and evaluation of the detection algorithm. First, we selected the bioterrorist agents from a list of public health threats (Wagner et al., 2003) identified as significant by organizations such as the Defense Threat Reduction Agency, Centers for Disease Control and Prevention, and the North Atlantic Treaty Organization. Anthrax, botulism, pneumonic plague, smallpox, and tularemia appeared in all source lists. We then searched MEDLINE (1966 to present), CINAHL (1982 to present), all available years in EMBASE, and the Science Citation Index for papers describing clinical manifestations of these diseases. One author (MHTU) reviewed all articles and extracted the terms describing signs/symptoms occurring in the head and neck region (APPENDIX 1). We included only manifestations that were likely to be detected either by a dentist or a physician during a head and neck and/or oral exam. In addition, we recorded the time of onset of oral relative to systemic manifestations. Plague was omitted because of no evidence of oral manifestations.
Second, we developed a set of synonyms and variants for the clinical terms describing oral manifestations (Chapman et al., 2004a), to identify comprehensively the emergency department reports containing evidence for the diseases of interest. We drew this set from the National Library of Medicine’s Unified Medical Language System Metathesaurus, Release 2004AA. For instance, synonym-variants for the term "oral ulcer" included "mouth ulcer, mouth ulceration, oral ulceration, ulceration of oral mucosa, ulcer of oral mucosa, mouth ulcers, buccal ulceration", and "ulcer buccal". It is important to note that several diseases have similar signs/symptoms. This "overlap" does not necessarily affect the performance of our data source, since the primary purpose is to detect an outbreak early, not to identify the causal agent conclusively. Our compilation included original terms, synonyms/variants, and onset of oral manifestations relative to systemic manifestations (Table 1
Third, we developed an application in Python (V. 2.4.2, http://www.python.org/) to identify reports containing at least one term of interest. Our primary data sources were all 199,691 free-text emergency department reports stored in the University of Pittsburgh Medical Center Presbyterian Hospital’s Medical Archiving Record System from 2001 to 2003. The reports were de-identified (Gupta et al., 2004), and the date was substituted with the report’s number for the week in the year of record. Our application searched each report for the listed terms (Table 1  ). When a term was found, the application checked whether the term was negated (e.g., "lacking oral ulcers") (Chapman et al., 2001). Once the application verified a positive report (i.e., that the term was not negated), it recorded a "hit" (case of interest) for the year and week of the report. Additional terms in the same report were ignored. Subsequently, the number of hits was plotted over time in Microsoft Excel (Redmond, WA, USA). The system had a sensitivity (0.98) and specificity (0.93) for term-matching (identification) (Torres-Urquidy, unpublished material). Finally, our method used historical data (previous weeks) to forecast the number of cases for the upcoming week. The algorithm generated an alert if the observed number differed significantly from the forecast. We used a four-week moving average to calculate the expected number of cases:
where Xt is the count of cases for week t. The algorithm generates an alert if
where k is a constant that controls the sensitivity and specificity of the algorithm, and The metrics we used to evaluate detection algorithms were sensitivity, specificity, and timeliness. Because no outbreaks of the 4 diseases occurred in Pittsburgh from 2001 to 2003, we simulated their effect on our baseline data (Wallstrom et al., 2005). For each of the 4 diseases, we assumed that each case had a probability of 0.4 of visiting an emergency department with oral symptoms. Since our surveillance system uses data from only one emergency department, we used published emergency department utilization data (Pennsylvania Department of Health, 2007) to estimate that each emergency department visit had a 0.0876 probability of being captured by our surveillance system. We estimated the case-detection sensitivity for our system as 0.9 for all 4 diseases. The above values imply that 3.15% of all cases would be expected to appear in the time series generated by our system. We simulated outbreaks by randomly selecting a week and day of initial exposure. For each disease except smallpox, we assumed a uniform distribution of cases extending throughout the incubation period (anthrax, 1–7 days; botulism, 12–72 hrs; and tularemia, 3–5 days). In the smallpox simulation, we had to account for its high level of contagion. Using a four-component stochastic disease model, we selected an initial number of cases, and assumed that each patient infected 1.5 other individuals (Meltzer et al., 2001), that all patients remained in the region covered by the surveillance system, and that no intervention (e.g., quarantine) was undertaken. For all diseases, we estimated a probability of 0.0315 that a case would visit the emergency department with oral manifestations immediately upon onset and be detected by the system. Simulated cases were aggregated weekly and added to the baseline data. Because of the uncertainty about whether a case would visit the emergency department with oral manifestations (0.4), and our system would detect this event (0.9), we conducted one-way sensitivity analyses on these two probabilities. Specifically, we measured sensitivity, specificity, and timeliness when the ED visit probabilities are 0.2 and 0.6 (other parameters unchanged). We also evaluated detection performance for case-detection sensitivity values of 0.8 and 1.0, leaving the other parameters unchanged. For each disease, we simulated 25 outbreaks at random between Week 17 and Week 144. Selecting this interval provided at least 16 weeks of training data for the algorithm and 12 weeks to observe the effect of each outbreak. We simulated outbreaks of anthrax, tularemia, and botulism with 100, 500, and 1000 cases, and for smallpox with 1, 10, and 50 initial cases. For each simulated outbreak, we constructed a time series of emergency department visits, ran our detection algorithm on the series, and determined the number of weeks, if any, from the initial exposure that the algorithm produced an alert. When our algorithm did not detect an outbreak, we assumed that other methods would finally detect it in the week following the last week of the simulated outbreak. We characterized overall sensitivity of detection using receiver operating characteristic curves that display the relationship between false alarm rate (controlled by the parameter k) and sensitivity. We also used activity monitor operating characteristic curves (Fawcett and Provost, 1999) to show the relationship between false alarm rate and detection timeliness. This analysis allows public health officials to evaluate a specific surveillance system according to the potential costs of false alarms. The University of Pittsburgh’s IRB approved this study as exempt (approval #0406164).
Readers should note that no outbreaks of anthrax, botulism, smallpox, and tularemia were reported in the Pittsburgh area during the period. Nevertheless, we captured the frequency of related oral manifestations naturally occurring as recorded by physicians from 2001–2003 (Table 2  ). Out of a total of 199,691 emergency department reports, 30,233 contained at least one term of interest. As shown (Table 2), the highest term frequencies were found for anthrax (including buccal ulcers, sore throat) and tularemia (e.g., tonsillitis), both at approximately 7%, while those for botulism (dry mouth) and smallpox (enanthema) were relatively low (0.55% and 0.23%). This finding makes intuitive sense, because both anthrax and tularemia have symptoms that occur quite commonly in other diseases, such as the common cold. Inspection of the graphs displaying weekly frequencies did not reveal any obvious patterns, as shown in the detected reports with oral manifestations (Fig. 1). (End-of-year valleys are artifacts resulting from the de-identification process.)
Next, we report on the performance of the detection algorithm (Fig. 2 ). The receiver operating characteristic curves show the algorithm sensitivity as a function of false alarms per year, while the activity monitor operating characteristic curves do so for the week of detection from the beginning of the outbreak. The algorithm performed best for smallpox and botulism in detecting actual outbreaks, especially with a large number of cases. For instance, the algorithm can detect a smallpox epidemic of 50 initial cases with a sensitivity of 80%, with slightly over two false alarms per year. For botulism (1000 cases), the false alarm rate at the same level of sensitivity is about 4/yr. The sensitivity characteristics for anthrax and tularemia, in contrast, resemble the line of no discrimination, indicating poor performance of our algorithm. The activity monitor operating characteristic curves present a similar impression with regard to timeliness of detection. For instance, a botulism outbreak (500 cases) is detected within one week at a false alarm rate of 4/yr. Smallpox outbreaks take longer to detect (between 6 and 7 wks) at the same false alarm rate. For anthrax and tularemia, detection timeliness ranges between 2 and 3, and 1 and 2 wks, respectively, given 4 false alarms/yr. However, the poor sensitivity of the algorithm for the two diseases reduces its application.
The sensitivity analyses showed that the sensitivity and timeliness were impervious to changes in case-detection sensitivity and moderately sensitive to large changes in the probability of reporting oral symptoms (APPENDICES 2–5). For example, the time to detect a botulism outbreak with 500 cases and 4/yr false alarms varied from 1.03 to 0.95 and 1.11 wks when the case-detection sensitivity was changed from 0.9 to 0.8 and 1.0, respectively. However, when the probability of reporting oral symptoms varied from 0.4 to 0.2 and 0.6, timeliness changed to 0.74 and 1.48 wks.
This study characterized the frequency of oral manifestations in a population and developed a model for the detection of bioterrorist attacks by monitoring these manifestations in emergency department reports. In this model, the performance differential among different diseases may have occurred for several reasons. One major factor is that oral manifestations for smallpox and botulism occurred much less frequently (0.23% and 0.55%, respectively) in emergency department reports at baseline than those for anthrax and tularemia (7.46% and 6.91%). Any increase in corresponding oral manifestations thus had a proportionally larger effect for smallpox and botulism than for anthrax and tularemia. Second, the incubation periods for anthrax and tularemia are much longer than for the other two diseases. The uniform distribution of simulated cases over the incubation period thus "diluted" the signal compared with the diseases with a shorter incubation period. In the case of smallpox, the highly contagious nature of this disease had the effect of amplifying the initial signal rapidly, and thus may be primarily responsible for the superior performance of the algorithm. The main objective of this study was to identify certain oral manifestations and explore the feasibility of using these as a data source for biosurveillance, not to construct a definitive algorithm. As such, our model is contingent on assumptions for which, at present, little support exists. First, our assumptions about oral manifestations of bioterrorist diseases are based on a literature review, not on systematic studies conducted by oral health researchers. Future studies should focus on determining oral manifestations in detail, and how health professionals report them. Second, we do not know how likely it is that patients would seek care for oral manifestations during an outbreak. With the exception of those for smallpox, none is prodromal, making systemic manifestations equally useful for monitoring. Third, the ‘moving average’ analysis we used is only one approach for detection. More sophisticated univariate detection methodologies exist, and other factors, such as day- of-week effects and seasonality, should be utilized in refining the algorithm. Last, the lack of data from real outbreaks limited our ability to evaluate the proposed detection system. A reasonable approach in leveraging our method would be to combine the developed signal with others to improve detection performance. In conclusion, our study determined the prevalence of specific oral manifestations in a population and showed that this is a viable novel method for their monitoring. In this way, dentists and physicians could contribute to effective and efficient biosurveillance.
The authors thank Wendy Chapman, Rebecca Crowley, Melissa Saul, and Robert Weyant for their assistance. M.H.T.U. received support from CONACyT, Mexico (#167967) and NIDCR/NIH (1R21DE018548-01). G.W. was supported by a grant from the CDC (R01PH000025). This work is solely the responsibility of its authors and does not necessarily represent the views of the funding sources. Preliminary results of this project were presented as a poster at the 84th General Session & Exhibition of the IADR (2006) in Brisbane, Australia.
A supplemental appendix to this article is published electronically only athttp://jdr.sagepub.com/supplemental. Received for publication September 18, 2007. Revision received September 18, 2008. Accepted for publication September 24, 2008.
Journal of Dental Research, Vol. 88, No. 1,
89-94 (2009)
|
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
 |
|
|
 |
|
Sign In to gain access to subscriptions and/or personal tools.
TOP
ABSTRACT
INTRODUCTION
is an estimate of the standard deviation of the forecast error. We computed 
