Meaning of Missing Values in Eyewitness Recall and Accident Records

Background: Eyewitness recalls and accident records frequently do not mention the conditions and behaviors of interest to researchers and lead to missing values and to uncertainty about the prevalence of these conditions and behaviors surrounding accidents. Missing values may occur because eyewitnesses report the presence but not the absence of obvious clues/accident features. We examined this possibility. Methodology/Principal Findings: Participants watched car accident videos and were asked to recall as much information as they could remember about each accident. The results showed that eyewitnesses were far more likely to report the presence of present obvious clues than the absence of absent obvious clues even though they were aware of their absence. Conclusions: One of the principal mechanisms causing missing values may be eyewitnesses ’ tendency to not report the absence of obvious features. We discuss the implications of our findings for both retrospective and prospective analyses of accident records, and illustrate the consequences of adopting inappropriate assumptions about the meaning of missing values using the Avaluator Avalanche Accident Prevention Card

Student evaluations of teaching: teaching quantitative courses can be hazardous to one's career

Published VersionAnonymous student evaluations of teaching (SETs) are used by colleges and universities to measure teaching effectiveness and to make decisions about faculty hiring, firing, re-appointment, promotion, tenure, and merit pay. Although numerous studies have found that SETs correlate with various teaching effectiveness irrelevant factors (TEIFs) such as subject, class size, and grading standards, it has been argued that such correlations are small and do not undermine the validity of SETs as measures of professors’ teaching effectiveness. However, previous research has generally used inappropriate parametric statistics and effect sizes to examine and to evaluate the significance of TEIFs on personnel decisions. Accordingly, we examined the influence of quantitative vs. non-quantitative courses on SET ratings and SET based personnel decisions using 14,872 publicly posted class evaluations where each evaluation represents a summary of SET ratings provided by individual students responding in each class. In total, 325,538 individual student evaluations from a US mid-size university contributed to theses class evaluations. The results demonstrate that class subject (math vs. English) is strongly associated with SET ratings, has a substantial impact on professors being labeled satisfactory vs. unsatisfactory and excellent vs. non-excellent, and the impact varies substantially depending on the criteria used to classify professors as satisfactory vs. unsatisfactory. Professors teaching quantitative courses are far more likely not to receive tenure, promotion, and/or merit pay when their performance is evaluated against common standards

Student evaluations of teaching: teaching quantitative courses can be hazardous to one's career

Published VersionAnonymous student evaluations of teaching (SETs) are used by colleges and universities to measure teaching effectiveness and to make decisions about faculty hiring, firing, re-appointment, promotion, tenure, and merit pay. Although numerous studies have found that SETs correlate with various teaching effectiveness irrelevant factors (TEIFs) such as subject, class size, and grading standards, it has been argued that such correlations are small and do not undermine the validity of SETs as measures of professors’ teaching effectiveness. However, previous research has generally used inappropriate parametric statistics and effect sizes to examine and to evaluate the significance of TEIFs on personnel decisions. Accordingly, we examined the influence of quantitative vs. non-quantitative courses on SET ratings and SET based personnel decisions using 14,872 publicly posted class evaluations where each evaluation represents a summary of SET ratings provided by individual students responding in each class. In total, 325,538 individual student evaluations from a US mid-size university contributed to theses class evaluations. The results demonstrate that class subject (math vs. English) is strongly associated with SET ratings, has a substantial impact on professors being labeled satisfactory vs. unsatisfactory and excellent vs. non-excellent, and the impact varies substantially depending on the criteria used to classify professors as satisfactory vs. unsatisfactory. Professors teaching quantitative courses are far more likely not to receive tenure, promotion, and/or merit pay when their performance is evaluated against common standards

Rising verbal intelligence scores: Implications for research and clinical practice.

Evidence suggests that scores on various intelligence tests have been rising at a fast rate. To find out whether performance on the Wechsler Adult Intelligence Scale (WAIS) Vocabulary subtest has also been rising, the authors searched major psychology journals for investigations involving healthy younger and older adult participants and collected the reported WAIS Vocabulary scores. The meta-analysis shows that WAIS Vocabulary scores have been rising at the rate of 0.117/year (corresponding to 1.52 IQ points/decade) for younger adults and 0.367/year (corresponding to 4.79 IQ points/decade) for older adults. Mounting evidence suggests that raw scores on various intelli-gence tests have been rising at a fast rate over the last 7 decade

Meta-analysis: on average, undergraduate students’ intelligence is merely average

According to a widespread belief, the average IQ of university students is 115 to 130 IQ points, that is, substantially higher than the average IQ of the general population ( M = 100, SD = 15). We traced the origin of this belief to obsolete intelligence data collected in 1940s and 1950s when university education was the privilege of a few. Examination of more recent IQ data indicate that IQ of university students and university graduates dropped to the average of the general population. The decline in students’ IQ is a necessary consequence of increasing educational attainment over the last 80 years. Today, graduating from university is more common than completing high school in the 1940s. We conducted a meta-analysis of the mean IQ scores of college and university students samples tested with Wechsler Adult Intelligence Scale between 1939 and 2022. The results show that the average IQ of undergraduate students today is a mere 102 IQ points and declined by approximately 0.2 IQ points per year. The students’ IQ also varies substantially across universities and is correlated with the selectivity of universities (measured by average SAT scores of admitted students). These findings have wide-ranging implications. First, universities and professors need to realize that students are no longer extraordinary but merely average, and have to adjust curricula and academic standards. Second, employers can no longer rely on applicants with university degrees to be more capable or smarter than those without degrees. Third, students need to realize that acceptance into university is no longer an invitation to join an elite group. Fourth, the myth of brilliant undergraduate students in scientific and popular literature needs to be dispelled. Fifth, estimating premorbid IQ based on educational attainment is vastly inaccurate, obsolete, not evidence based, and mere wishful thinking. Sixth, obsolete IQ data or tests ought not to be used to make high-stakes decisions about individuals, for example, by clinical psychologists to opine about the intelligence and cognitive abilities of their clients

The Influence of Object Relative Size on Priming and Explicit Memory

We investigated the effects of object relative size on priming and explicit memory for color photos of common objects. Participants were presented with color photos of pairs of objects displayed in either appropriate or inappropriate relative sizes. Implicit memory was assessed by speed of object size ratings whereas explicit memory was assessed by an old/new recognition test. Study-to-test changes in relative size reduced both priming and explicit memory and had large effects for objects displayed in large vs. small size at test. Our findings of substantial size-specific influences on priming with common objects under some but not other conditions are consistent with instance views of object perception and priming but inconsistent with structural description views

Transparent Meta-Analysis: Does Aging Spare Prospective Memory with Focal vs. Non-Focal Cues?

Background: Prospective memory (ProM) is the ability to become aware of a previously-formed plan at the right time and place. For over twenty years, researchers have been debating whether prospective memory declines with aging or whether it is spared by aging and, most recently, whether aging spares prospective memory with focal vs. non-focal cues. Two recent meta-analyses examining these claims did not include all relevant studies and ignored prevalent ceiling effects, age confounds, and did not distinguish between prospective memory subdomains (e.g., ProM proper, vigilance, habitual ProM) (see Uttl, 2008, PLoS ONE). The present meta-analysis focuses on the following questions: Does prospective memory decline with aging? Does prospective memory with focal vs. non-focal cues decline with aging? Does the size of age-related declines with focal vs. non-focal cues vary across ProM subdomains? And are age-related declines in ProM smaller than agerelated declines in retrospective memory? Methods and Findings: A meta-analysis of event-cued ProM using data visualization and modeling, robust count methods, and conventional meta-analysis techniques revealed that first, the size of age-related declines in ProM with both focal and non-focal cues are large. Second, age-related declines in ProM with focal cues are larger in ProM proper and smaller in vigilance. Third, age-related declines in ProM proper with focal cues are as large as age-related declines in recall measures of retrospective memory

Transparent Meta-Analysis of Prospective Memory and Aging

Prospective memory (ProM) refers to our ability to become aware of a previously formed plan at the right time and place. After two decades of research on prospective memory and aging, narrative reviews and summaries have arrived at widely different conclusions. One view is that prospective memory shows large age declines, larger than age declines on retrospective memory (RetM). Another view is that prospective memory is an exception to age declines and remains invariant across the adult lifespan. The present meta-analysis of over twenty years of research settles this controversy. It shows that prospective memory declines with aging and that the magnitude of age decline varies by prospective memory subdomain (vigilance, prospective memory proper, habitual prospective memory) as well as test setting (laboratory, natural). Moreover, this meta-analysis demonstrates that previous claims of no age declines in prospective memory are artifacts of methodological and conceptual issues afflicting prior research including widespread ceiling effects, low statistical power, age confounds, and failure to distinguish between various subdomains of prospective memory (e.g., vigilance and prospective memory proper)

Small samples, unreasonable generalizations, and outliers: Gender bias in student evaluation of teaching or three unhappy students?

In a widely cited and widely talked about study, MacNell et al. (2015) examined SET ratings of one female and one male instructor, each teaching two sections of the same online course, one section under their true gender and the other section under false/opposite gender. MacNell et al. concluded that students rated perceived female instructors more harshly than perceived male instructors, demonstrating gender bias against perceived female instructors. Boring, Ottoboni, and Stark (2016) re-analyzed MacNell et al.s data and confirmed their conclusions. However, the design of MacNell et al. study is fundamentally flawed. First, MacNell et al. section sample sizes were extremely small, ranging from 8 to 12 students. Second, MacNell et al. included only one female and one male instructor. Third, MacNell et al.s findings depend on three outliers -- three unhappy students (all in perceived female conditions) who gave their instructors the lowest possible ratings on all or nearly all SET items. We re-analyzed MacNell et al.s data with and without the three outliers. Our analyses showed that the gender bias against perceived female instructors disappeared. Instead, students rated the actual female vs. male instructor higher, regardless of perceived gender. MacNell et al.s study is a real-life demonstration that conclusions based on extremely small sample-sized studies are unwarranted and uninterpretable.</ns7:p