Multilingual Testing Constructs: Theoretical Foundations

The field of language testing has made great strides in measuring language use. It is a monolingual construct, however, that anchors standardized language testing operations and classroom practices. Language use and performance research, see for example the MLJ special issue of 2011, demands that language testing operations also consider multilingual constructs. The present paper, generally guided by the published research as well as contributions in this special issue, examines how to expand the field’s theoretical foundations to consider integrated multilingual testing constructs and translanguaging pedagogies

Does a speaking task affect second language comprehensibility?

The current study investigated task effects on listener perception of second language (L2) comprehensibility (ease of understanding). Sixty university-level adult speakers of English from 4 first language (L1) backgrounds (Chinese, Romance, Hindi, Farsi), with 15 speakers per group, were recorded performing 2 tasks (IELTS long-turn speaking task, TOEFL iBT integrated listening/reading and speaking task). The speakers’ audio recordings were evaluated using continuous sliding scales by 10 native English listeners for comprehensibility as well as for 10 linguistic variables drawn from the domains of pronunciation, fluency, lexis, grammar, and discourse. In the IELTS task, comprehensibility was associated solely with pronunciation and fluency categories (specifically, segmentals, word stress, rhythm, and speech rate), with the Farsi group being the only exception. However, in the cognitively more demanding TOEFL iBT integrated task, in addition to pronunciation and fluency variables, comprehensibility was also linked to several categories at the level of grammar, lexicon, and discourse for all groups. In both tasks, the relative strength of obtained associations also varied as a function of the speakers’ L1. Results overall suggest that both task and speakers’ L1 play important roles in determining ease of understanding for the listener, with implications for pronunciation teaching in mixed L1 classrooms and for operationalizing the construct of comprehensibility in assessments

THE INTERSECTION OF TEST IMPACT, VALIDATION, AND EDUCATIONAL REFORM POLICY

The article addresses the intersection of policy, validity, and impact within the context of educational reform in U.S. schools, looking in particular at the No Child Left Behind (NCLB) Act (2001). The discussion makes a case that it is important to reconsider the established views regarding the responsibility of test developers and users in investigating impact given the conflated roles of developers and users under NCLB. The article also introduces the concept of social impact analysis (SIA) to argue for an expansion of the traditional conceptualization of impact research. SIA promotes a proactive rather than a reactive approach to impact, in order to inform policy formulation upfront.</jats:p

Test interpretation, test use, and pedagogical implications

Abstract Test score interpretation and use are the staple of construct validity. As such, in addition to the concern with measurement accuracy, it is imperative that the meaning of test scores and their intended use(s) be also documented. Along these lines, qualitative speech analyses are undertaken in the present paper to help in the interpretation of the dimensions underlying student performance on oral tasks. Results of these analyses yield rich information that explicate the meaning of the dimensions by delineating their specific features as manifested in the speech samples. Also discussed in the paper are the ramifications of these results for pedagogical use. Insights that linguistic accuracy and communicative skills in general, and their specific features specifically, provide for instructional material and activities are addressed. Furthermore, a case is made for curricular improvements to help learners develop well-rounded L2 abilities and to improve their use of the language for real-life communication. Finally, with regard to assessment, it is argued that generic assessment criteria do not reflect the critical features operating in a given context, and assessment practitioners are urged to study their contexts of use and to tailor their criteria according to the particulars of those contexts.</jats:p