Fabrication and errors in the bibliographic citations generated by ChatGPT

Although chatbots such as ChatGPT can facilitate cost-effective text generation and editing, factually incorrect responses (hallucinations) limit their utility. This study evaluates one particular type of hallucination: fabricated bibliographic citations that do not represent actual scholarly works. We used ChatGPT-3.5 and ChatGPT-4 to produce short literature reviews on 42 multidisciplinary topics, compiling data on the 636 bibliographic citations (references) found in the 84 papers. We then searched multiple databases and websites to determine the prevalence of fabricated citations, to identify errors in the citations to non-fabricated papers, and to evaluate adherence to APA citation format. Within this set of documents, 55% of the GPT-3.5 citations but just 18% of the GPT-4 citations are fabricated. Likewise, 43% of the real (non-fabricated) GPT-3.5 citations but just 24% of the real GPT-4 citations include substantive citation errors. Although GPT-4 is a major improvement over GPT-3.5, problems remain

Responding to the Quantitative Literacy Gap Among Students in Sociology Courses

The Integrating Data Analysis (IDA) approach to undergraduate education developed by the American Sociological Association (ASA) and the Social Science Data Analysis Network (SSDAN) has been embedded in the undergraduate sociology curriculum at Lehman College, The City University of New York (CUNY), since 2003. This study draws on student and faculty assessment data from the fall 2004 and spring 2006 semesters to evaluate the effectiveness of this initiative. The results show that students in courses with an IDA component significantly improved their performance on quantitative skills tests. Efforts to engage students in active learning through the use of computers were associated with increased student comfort and greater interest in working with data. In turn, students who were comfortable working with data exhibited especially high levels of quantitative skill. Although students who were taught a wide variety of data analysis skills reported greater interest in working with data, those who were taught a more limited range of skills achieved higher performance scores. Likewise, students who were required to complete assignments and undertake graded examinations that tested a wide range of quantitative skills had less interest in working with data but achieved greater improvements in their test scores. </jats:p

Mentoring Faculty Through a Quantitative Reasoning Professional Development Program: Why Do Faculty Participate and What Do They Get Out of It?

ABSTRACTAlthough quantitative reasoning (QR) is central to general education, many college students lack fundamental numeracy skills. In response, The City University of New York established a QR faculty development program that trained instructors across the disciplines through teaching exercises, guided discussions, hands-on activities, the development of instructional/assessment materials, and feedback from mentors and peers. Ten cohorts, 2010–2019, responded to surveys that evaluated their motives for participating and the extent to which they felt their goals were met. Faculty joined the program due to factors including their concern for students, their commitment to QR instruction, and their desire to build professional networks. Program completers reported a better understanding of QR, a greater commitment to QR instruction, increased awareness of tools and techniques (e.g., progressive pedagogies, active learning and constructivist approaches), a clearer sense of students’ needs, a commitment to assessment, and strong engagement with CUNY’s multidisciplinary, multi-institutional QR community. Overall, the perceived benefits of the program match participants’ motives for joining. Respondents’ comments suggest that faculty development for general education requires motivated participants, opportunities for networking, thoughtful discussion of readings and videos, modeling of best practices, a student-centered curriculum, sensitivity to participants’ backgrounds, adequate incentives, effective mentorship, and institutional commitment.</jats:p

Fabrication and errors in the bibliographic citations generated by ChatGPT

Abstract Although chatbots such as ChatGPT can facilitate cost-effective text generation and editing, factually incorrect responses (hallucinations) limit their utility. This study evaluates one particular type of hallucination: fabricated bibliographic citations that do not represent actual scholarly works. We used ChatGPT-3.5 and ChatGPT-4 to produce short literature reviews on 42 multidisciplinary topics, compiling data on the 636 bibliographic citations (references) found in the 84 papers. We then searched multiple databases and websites to determine the prevalence of fabricated citations, to identify errors in the citations to non-fabricated papers, and to evaluate adherence to APA citation format. Within this set of documents, 55% of the GPT-3.5 citations but just 18% of the GPT-4 citations are fabricated. Likewise, 43% of the real (non-fabricated) GPT-3.5 citations but just 24% of the real GPT-4 citations include substantive citation errors. Although GPT-4 is a major improvement over GPT-3.5, problems remain

Publishing Productivity of Sociologists at American Colleges and Universities: Institution Type, Gender, and Other Correlates of Book and Article Counts

This study examines the 2013–2017 publishing productivity of sociology faculty at six types of colleges and universities (e.g., research universities, master’s institutions, and top liberal arts colleges) based on publication counts for articles, articles in high-impact journals, books, and books from high-impact publishers. We compare the productivity of groups based on institution type, gender, academic rank, years of experience, and reputation of PhD-granting institution. Our age-cohort data suggest that differentials in productivity among institution types have diminished in recent decades. The top universities are losing ground, in relative terms, while faculty at other types of institutions are more productive now than in the past. Our results for gender are unlike those reported in previous research, revealing (1) higher productivity for women than for men across most institution types and (2) the absence of any gender differential for all institution types combined. Our data also show that book and article counts are virtually unrelated, that faculty at the top liberal arts colleges have the highest average book counts, and that there is great variation in productivity within every institution type. In general, associate professors, faculty with fewer than 17 years of experience, and faculty with doctorates from top universities are especially productive. </jats:p