Time series modeling for syndromic surveillance

BACKGROUND: Emergency department (ED) based syndromic surveillance systems identify abnormally high visit rates that may be an early signal of a bioterrorist attack. For example, an anthrax outbreak might first be detectable as an unusual increase in the number of patients reporting to the ED with respiratory symptoms. Reliably identifying these abnormal visit patterns requires a good understanding of the normal patterns of healthcare usage. Unfortunately, systematic methods for determining the expected number of (ED) visits on a particular day have not yet been well established. We present here a generalized methodology for developing models of expected ED visit rates. METHODS: Using time-series methods, we developed robust models of ED utilization for the purpose of defining expected visit rates. The models were based on nearly a decade of historical data at a major metropolitan academic, tertiary care pediatric emergency department. The historical data were fit using trimmed-mean seasonal models, and additional models were fit with autoregressive integrated moving average (ARIMA) residuals to account for recent trends in the data. The detection capabilities of the model were tested with simulated outbreaks. RESULTS: Models were built both for overall visits and for respiratory-related visits, classified according to the chief complaint recorded at the beginning of each visit. The mean absolute percentage error of the ARIMA models was 9.37% for overall visits and 27.54% for respiratory visits. A simple detection system based on the ARIMA model of overall visits was able to detect 7-day-long simulated outbreaks of 30 visits per day with 100% sensitivity and 97% specificity. Sensitivity decreased with outbreak size, dropping to 94% for outbreaks of 20 visits per day, and 57% for 10 visits per day, all while maintaining a 97% benchmark specificity. CONCLUSIONS: Time series methods applied to historical ED utilization data are an important tool for syndromic surveillance. Accurate forecasting of emergency department total utilization as well as the rates of particular syndromes is possible. The multiple models in the system account for both long-term and recent trends, and an integrated alarms strategy combining these two perspectives may provide a more complete picture to public health authorities. The systematic methodology described here can be generalized to other healthcare settings to develop automated surveillance systems capable of detecting anomalies in disease patterns and healthcare utilization

Potential for early warning of viral influenza activity in the community by monitoring clinical diagnoses of influenza in hospital emergency departments

<p>Abstract</p> <p>Background</p> <p>Although syndromic surveillance systems are gaining acceptance as useful tools in public health, doubts remain about whether the anticipated early warning benefits exist. Many assessments of this question do not adequately account for the confounding effects of autocorrelation and trend when comparing surveillance time series and few compare the syndromic data stream against a continuous laboratory-based standard. We used time series methods to assess whether monitoring of daily counts of Emergency Department (ED) visits assigned a clinical diagnosis of influenza could offer earlier warning of increased incidence of viral influenza in the population compared with surveillance of daily counts of positive influenza test results from laboratories.</p> <p>Methods</p> <p>For the five-year period 2001 to 2005, time series were assembled of ED visits assigned a provisional ED diagnosis of influenza and of laboratory-confirmed influenza cases in New South Wales (NSW), Australia. Poisson regression models were fitted to both time series to minimise the confounding effects of trend and autocorrelation and to control for other calendar influences. To assess the relative timeliness of the two series, cross-correlation analysis was performed on the model residuals. Modelling and cross-correlation analysis were repeated for each individual year.</p> <p>Results</p> <p>Using the full five-year time series, short-term changes in the ED time series were estimated to precede changes in the laboratory series by three days. For individual years, the estimate was between three and 18 days. The time advantage estimated for the individual years 2003–2005 was consistently between three and four days.</p> <p>Conclusion</p> <p>Monitoring time series of ED visits clinically diagnosed with influenza could potentially provide three days early warning compared with surveillance of laboratory-confirmed influenza. When current laboratory processing and reporting delays are taken into account this time advantage is even greater.</p

The Use of Twitter to Track Levels of Disease Activity and Public Concern in the U.S. during the Influenza A H1N1 Pandemic

Twitter is a free social networking and micro-blogging service that enables its millions of users to send and read each other's “tweets,” or short, 140-character messages. The service has more than 190 million registered users and processes about 55 million tweets per day. Useful information about news and geopolitical events lies embedded in the Twitter stream, which embodies, in the aggregate, Twitter users' perspectives and reactions to current events. By virtue of sheer volume, content embedded in the Twitter stream may be useful for tracking or even forecasting behavior if it can be extracted in an efficient manner. In this study, we examine the use of information embedded in the Twitter stream to (1) track rapidly-evolving public sentiment with respect to H1N1 or swine flu, and (2) track and measure actual disease activity. We also show that Twitter can be used as a measure of public interest or concern about health-related events. Our results show that estimates of influenza-like illness derived from Twitter chatter accurately track reported disease levels

Combining Free Text and Structured Electronic Medical Record Entries to Detect Acute Respiratory Infections

The electronic medical record (EMR) contains a rich source of information that could be harnessed for epidemic surveillance. We asked if structured EMR data could be coupled with computerized processing of free-text clinical entries to enhance detection of acute respiratory infections (ARI).A manual review of EMR records related to 15,377 outpatient visits uncovered 280 reference cases of ARI. We used logistic regression with backward elimination to determine which among candidate structured EMR parameters (diagnostic codes, vital signs and orders for tests, imaging and medications) contributed to the detection of those reference cases. We also developed a computerized free-text search to identify clinical notes documenting at least two non-negated ARI symptoms. We then used heuristics to build case-detection algorithms that best combined the retained structured EMR parameters with the results of the text analysis.An adjusted grouping of diagnostic codes identified reference ARI patients with a sensitivity of 79%, a specificity of 96% and a positive predictive value (PPV) of 32%. Of the 21 additional structured clinical parameters considered, two contributed significantly to ARI detection: new prescriptions for cough remedies and elevations in body temperature to at least 38°C. Together with the diagnostic codes, these parameters increased detection sensitivity to 87%, but specificity and PPV declined to 95% and 25%, respectively. Adding text analysis increased sensitivity to 99%, but PPV dropped further to 14%. Algorithms that required satisfying both a query of structured EMR parameters as well as text analysis disclosed PPVs of 52-68% and retained sensitivities of 69-73%.Structured EMR parameters and free-text analyses can be combined into algorithms that can detect ARI cases with new levels of sensitivity or precision. These results highlight potential paths by which repurposed EMR information could facilitate the discovery of epidemics before they cause mass casualties

An automated, broad-based, near real-time public health surveillance system using presentations to hospital Emergency Departments in New South Wales, Australia

BACKGROUND: In a climate of concern over bioterrorism threats and emergent diseases, public health authorities are trialling more timely surveillance systems. The 2003 Rugby World Cup (RWC) provided an opportunity to test the viability of a near real-time syndromic surveillance system in metropolitan Sydney, Australia. We describe the development and early results of this largely automated system that used data routinely collected in Emergency Departments (EDs). METHODS: Twelve of 49 EDs in the Sydney metropolitan area automatically transmitted surveillance data from their existing information systems to a central database in near real-time. Information captured for each ED visit included patient demographic details, presenting problem and nursing assessment entered as free-text at triage time, physician-assigned provisional diagnosis codes, and status at departure from the ED. Both diagnoses from the EDs and triage text were used to assign syndrome categories. The text information was automatically classified into one or more of 26 syndrome categories using automated "naïve Bayes" text categorisation techniques. Automated processes were used to analyse both diagnosis and free text-based syndrome data and to produce web-based statistical summaries for daily review. An adjusted cumulative sum (cusum) was used to assess the statistical significance of trends. RESULTS: During the RWC the system did not identify any major public health threats associated with the tournament, mass gatherings or the influx of visitors. This was consistent with evidence from other sources, although two known outbreaks were already in progress before the tournament. Limited baseline in early monitoring prevented the system from automatically identifying these ongoing outbreaks. Data capture was invisible to clinical staff in EDs and did not add to their workload. CONCLUSION: We have demonstrated the feasibility and potential utility of syndromic surveillance using routinely collected data from ED information systems. Key features of our system are its nil impact on clinical staff, and its use of statistical methods to assign syndrome categories based on clinical free text information. The system is ongoing, and has expanded to cover 30 EDs. Results of formal evaluations of both the technical efficiency and the public health impacts of the system will be described subsequently

Assessing Syndromic Surveillance of Cardiovascular Outcomes from Emergency Department Chief Complaint Data in New York City

Prospective syndromic surveillance of emergency department visits has been used for near-real time tracking of communicable diseases to detect outbreaks or other unexpected disease clusters. The utility of syndromic surveillance for tracking cardiovascular events, which may be influenced by environmental factors and influenza, has not been evaluated. We developed and evaluated a method for tracking cardiovascular events using emergency department free-text chief complaints.There were three phases to our analysis. First we applied text processing algorithms based on sensitivity, specificity, and positive predictive value to chief complaint data reported by 11 New York City emergency departments for which ICD-9 discharge diagnosis codes were available. Second, the same algorithms were applied to data reported by a larger sample of 50 New York City emergency departments for which discharge diagnosis was unavailable. From this more complete data, we evaluated the consistency of temporal variation of cardiovascular syndromic events and hospitalizations from 76 New York City hospitals. Finally, we examined associations between particulate matter ≤2.5 µm (PM(2.5)), syndromic events, and hospitalizations. Sensitivity and positive predictive value were low for syndromic events, while specificity was high. Utilizing the larger sample of emergency departments, a strong day of week pattern and weak seasonal trend were observed for syndromic events and hospitalizations. These time-series were highly correlated after removing the day-of-week, holiday, and seasonal trends. The estimated percent excess risks in the cold season (October to March) were 1.9% (95% confidence interval (CI): 0.6, 3.2), 2.1% (95% CI: 0.9, 3.3), and 1.8% (95%CI: 0.5, 3.0) per same-day 10 µg/m(3) increase in PM(2.5) for cardiac-only syndromic data, cardiovascular syndromic data, and hospitalizations, respectively.Near real-time emergency department chief complaint data may be useful for timely surveillance of cardiovascular morbidity related to ambient air pollution and other environmental events

Using Ontario's "Telehealth" health telephone helpline as an early-warning system: a study protocol

BACKGROUND: The science of syndromic surveillance is still very much in its infancy. While a number of syndromic surveillance systems are being evaluated in the US, very few have had success thus far in predicting an infectious disease event. Furthermore, to date, the majority of syndromic surveillance systems have been based primarily in emergency department settings, with varying levels of enhancement from other data sources. While research has been done on the value of telephone helplines on health care use and patient satisfaction, very few projects have looked at using a telephone helpline as a source of data for syndromic surveillance, and none have been attempted in Canada. The notable exception to this statement has been in the UK where research using the national NHS Direct system as a syndromic surveillance tool has been conducted. METHODS/DESIGN: The purpose of our proposed study is to evaluate the effectiveness of Ontario's telephone nursing helpline system as a real-time syndromic surveillance system, and how its implementation, if successful, would have an impact on outbreak event detection in Ontario. Using data collected retrospectively, all "reasons for call" and assigned algorithms will be linked to a syndrome category. Using different analytic methods, normal thresholds for the different syndromes will be ascertained. This will allow for the evaluation of the system's sensitivity, specificity and positive predictive value. The next step will include the prospective monitoring of syndromic activity, both temporally and spatially. DISCUSSION: As this is a study protocol, there are currently no results to report. However, this study has been granted ethical approval, and is now being implemented. It is our hope that this syndromic surveillance system will display high sensitivity and specificity in detecting true outbreaks within Ontario, before they are detected by conventional surveillance systems. Future results will be published in peer-reviewed journals so as to contribute to the growing body of evidence on syndromic surveillance, while also providing an non US-centric perspective

Accuracy of syndrome definitions based on diagnoses in physician claims

<p>Abstract</p> <p>Background</p> <p>Community clinics offer potential for timelier outbreak detection and monitoring than emergency departments. However, the accuracy of syndrome definitions used in surveillance has never been evaluated in community settings. This study's objective was to assess the accuracy of syndrome definitions based on diagnostic codes in physician claims for identifying 5 syndromes (fever, gastrointestinal, neurological, rash, and respiratory including influenza-like illness) in community clinics.</p> <p>Methods</p> <p>We selected a random sample of 3,600 community-based primary care physicians who practiced in the fee-for-service system in the province of Quebec, Canada in 2005-2007. We randomly selected 10 visits per physician from their claims, stratifying on syndrome type and presence, diagnosis, and month. Double-blinded chart reviews were conducted by telephone with consenting physicians to obtain information on patient diagnoses for each sampled visit. The sensitivity, specificity, and positive predictive value (PPV) of physician claims were estimated by comparison to chart review.</p> <p>Results</p> <p>1,098 (30.5%) physicians completed the chart review. A chart entry on the date of the corresponding claim was found for 10,529 (95.9%) visits. The sensitivity of syndrome definitions based on diagnostic codes in physician claims was low, ranging from 0.11 (fever) to 0.44 (respiratory), the specificity was high, and the PPV was moderate to high, ranging from 0.59 (fever) to 0.85 (respiratory). We found that rarely used diagnostic codes had a higher probability of being false-positives, and that more commonly used diagnostic codes had a higher PPV.</p> <p>Conclusions</p> <p>Future research should identify physician, patient, and encounter characteristics associated with the accuracy of diagnostic codes in physician claims. This would enable public health to improve syndromic surveillance, either by focusing on physician claims whose diagnostic code is more likely to be accurate, or by using all physician claims and weighing each according to the likelihood that its diagnostic code is accurate.</p

Rapid detection of pandemic influenza in the presence of seasonal influenza

Background: Key to the control of pandemic influenza are surveillance systems that raise alarms rapidly and sensitively. In addition, they must minimise false alarms during a normal influenza season. We develop a method that uses historical syndromic influenza data from the existing surveillance system 'SERVIS' (Scottish Enhanced Respiratory Virus Infection Surveillance) for influenza-like illness (ILI) in Scotland. Methods: We develop an algorithm based on the weekly case ratio (WCR) of reported ILI cases to generate an alarm for pandemic influenza. From the seasonal influenza data from 13 Scottish health boards, we estimate the joint probability distribution of the country-level WCR and the number of health boards showing synchronous increases in reported influenza cases over the previous week. Pandemic cases are sampled with various case reporting rates from simulated pandemic influenza infections and overlaid with seasonal SERVIS data from 2001 to 2007. Using this combined time series we test our method for speed of detection, sensitivity and specificity. Also, the 2008-09 SERVIS ILI cases are used for testing detection performances of the three methods with a real pandemic data. Results: We compare our method, based on our simulation study, to the moving-average Cumulative Sums (Mov-Avg Cusum) and ILI rate threshold methods and find it to be more sensitive and rapid. For 1% case reporting and detection specificity of 95%, our method is 100% sensitive and has median detection time (MDT) of 4 weeks while the Mov-Avg Cusum and ILI rate threshold methods are, respectively, 97% and 100% sensitive with MDT of 5 weeks. At 99% specificity, our method remains 100% sensitive with MDT of 5 weeks. Although the threshold method maintains its sensitivity of 100% with MDT of 5 weeks, sensitivity of Mov-Avg Cusum declines to 92% with increased MDT of 6 weeks. For a two-fold decrease in the case reporting rate (0.5%) and 99% specificity, the WCR and threshold methods, respectively, have MDT of 5 and 6 weeks with both having sensitivity close to 100% while the Mov-Avg Cusum method can only manage sensitivity of 77% with MDT of 6 weeks. However, the WCR and Mov-Avg Cusum methods outperform the ILI threshold method by 1 week in retrospective detection of the 2009 pandemic in Scotland. Conclusions: While computationally and statistically simple to implement, the WCR algorithm is capable of raising alarms, rapidly and sensitively, for influenza pandemics against a background of seasonal influenza. Although the algorithm was developed using the SERVIS data, it has the capacity to be used at other geographic scales and for different disease systems where buying some early extra time is critical