How to discriminate between computer-aided and computer-hindered decisions: a case study in mammography

Background. Computer aids can affect decisions in complex ways, potentially even making them worse; common assessment methods may miss these effects. We developed a method for estimating the quality of decisions, as well as how computer aids affect it, and applied it to computer-aided detection (CAD) of cancer, reanalyzing data from a published study where 50 professionals (“readers”) interpreted 180 mammograms, both with and without computer support. Method. We used stepwise regression to estimate how CAD affected the probability of a reader making a correct screening decision on a patient with cancer (sensitivity), thereby taking into account the effects of the difficulty of the cancer (proportion of readers who missed it) and the reader’s discriminating ability (Youden’s determinant). Using regression estimates, we obtained thresholds for classifying a posteriori the cases (by difficulty) and the readers (by discriminating ability). Results. Use of CAD was associated with a 0.016 increase in sensitivity (95% confidence interval [CI], 0.003–0.028) for the 44 least discriminating radiologists for 45 relatively easy, mostly CAD-detected cancers. However, for the 6 most discriminating radiologists, with CAD, sensitivity decreased by 0.145 (95% CI, 0.034–0.257) for the 15 relatively difficult cancers. Conclusions. Our exploratory analysis method reveals unexpected effects. It indicates that, despite the original study detecting no significant average effect, CAD helped the less discriminating readers but hindered the more discriminating readers. Such differential effects, although subtle, may be clinically significant and important for improving both computer algorithms and protocols for their use. They should be assessed when evaluating CAD and similar warning systems

My Imagination Versus Your Feelings: Can Personal Affective Forecasts Be Improved by Knowing Other Peoples' Emotions?

A proposed remedy for biased affective forecasts is to base judgments on the actual feelings of people (surrogates) currently experiencing the event, rather than using imagination which conjures an inaccurate vision of the future. Gilbert et al. (2009) forced people to use surrogate reports by withholding all event information, resulting in better predictions. However, in life surrogate information rarely supplants event information - can people effectively integrate both types of information into their judgments? In 5 studies, respondents predicted the impact of a health state on their own happiness. Respondents incorporated surrogate information into their judgments both in the presence and absence of event information. However, they inappropriately discounted other people’s experiences as a valid predictor of their own - particularly in the presence of event information – and imagined their happiness would be different to surrogates’ happiness. Excluding pre-existing event knowledge, changing the size of the surrogate sample, or increasing the size of the response scale, did not alter the adjustment. Although surrogate information improved affective forecasts, its influence was diminished by the presence of event information

Effects of incorrect computer-aided detection (CAD) output on human decision-making in mammography

To investigate the effects of incorrect computer output on the reliability of the decisions of human users. This work followed an independent UK clinical trial that evaluated the impact of computer-aided detection(CAD) in breast screening. The aim was to use data from this trial to feed into probabilistic models (similar to those used in "reliability engineering") which would detect and assess possible ways of improving the human-CAD interaction. Some analyses required extra data; therefore, two supplementary studies were conducted. Study 1 was designed to elucidate the effects of computer failure on human performance. Study 2 was conducted to clarify unexpected findings from Study 1

Effects of ignorance and information on judgments and decisions

We compared Turkish and English students’ soccer forecasting for English soccer matches. Although the Turkish students knew very little about English soccer, they selected teams on the basis of familiarity with the team (or its identified city); their prediction success was surprisingly similar to knowledgeable English students—consistent with Goldstein and Gigerenzer’s (1999; 2002) characterization of the recognition heuristic. The Turkish students made forecasts for some of the matches with additional information—the half-time scores. In this and a further study, where British students predicting matches for foreign teams could choose whether or not to use half-time information, we found that predictions that could be made by recognition alone were influenced by the half-time information. We consider the implications of these findings in the context of Goldstein and Gigerenzer’s (2002, p. 82) suggestion that “. . . no other information can reverse the choice determined by recognition” and a recent more qualified statement (Gigerenzer & Goldstein, 2011

Subjective Patterns of Randomness and Choice: Some Consequences of Collective Responses

Any individual's response intended to be random should be as probable as any other. However, 3 experiments show that many people's independent responses depart from the expected chance distribution. Participants responding to instructions of chance and related concepts favor the available options unequally in a similar way. Consequently, in hide-and-seek games, hiders converge on certain locations and are thereby detected beyond chance by seekers who share their preferences. People agree on salient and on nonsalient options, both of which are preferred under different instructions and even in the absence of instructions. Group responses strongly correlate under diverse, even opposing (e.g., competitive and cooperative) directions. Apparently, common default tendencies, combining random and aesthetic choices, are only somewhat modified under specific instructions. Maximal agreement with others is obtained by implementing one's own aesthetic preferences. These results broadly replicate in one- and two-dimensional tasks. Implications of the findings, their possible roots, and their connection to constructs from, e.g., game theory and subjective-complexity research, are discussed

Use of computer-aided detection (CAD) tools in screening mammography: a multidisciplinary investigation

We summarise a set of analyses and studies conducted to assess the effects of the use of a computer-aided detection (CAD) tool in breast screening. We have used an interdisciplinary approach that combines: (a) statistical analyses inspired by reliability modelling in engineering; (b) experimental studies of decisions of mammography experts using the tool, interpreted in the light of human factors psychology; and (c) ethnographic observations of the use of the tool both in trial conditions and in everyday screening practice. Our investigations have shown patterns of human behaviour and effects of computer-based advice that would not have been revealed by a standard clinical trial approach. For example, we found that the negligible measured effect of CAD could be explained by a range of effects on experts' decisions, beneficial in some cases and detrimental in others. There is some evidence of the latter effects being due to the experts using the computer tool differently from the intentions of the developers. We integrate insights from the different pieces of evidence and highlight their implications for the design, evaluation and deployment of this sort of computer tool

Terrorism, Dread Risk and Bicycle Accidents

Following the airplane attacks of September 11th, 2001 it is claimed that many Americans, dreading a repeat of these events, drove instead of flying, and that, consequently, there were extra car accidents, increasing the number of fatalities directly caused by the attacks by 1,500. After the Madrid train bombings of March 11th, 2004, Spaniards, like Americans, avoided the attacked mode of travel, but no increase in car travel or fatal accidents resulted. Here we analyze behavioral concomitants of the July 7th 2005 bomb attacks on public transport in London. We find reduced underground train travel and an increase in rates of bicycling and, over the 6 months following the attacks, 214 additional bicyclist road casualties - a 15.4% increase. Nevertheless we found no detectable increase in car accidents. We conclude that, while fear caused by terrorism may initiate potentially dangerous behaviors, understanding the secondary effects of terrorism requires consideration of the environmental variables that enable fear to manifest in dangerous behaviors

Microscopic expressions for the thermodynamic temperature

We show that arbitrary phase space vector fields can be used to generate phase functions whose ensemble averages give the thermodynamic temperature. We describe conditions for the validity of these functions in periodic boundary systems and the Molecular Dynamics (MD) ensemble, and test them with a short-ranged potential MD simulation.Comment: 21 pages, 2 figures, Revtex. Submitted to Phys. Rev.