Information Artifact Evaluation with TEDSrate

Part 7: EvaluationInternational audienceThe evaluation of systems or artifacts as “outcomes” of software engineering (SE) projects has been a focus of study in SE-related research for quite some time. In recent years, evaluating artifacts, for example, mobile applications or websites has become more important, since such artifacts play increasingly critical roles in generating revenues for businesses, and the degree of artifact effectiveness is seen as a competitive factor. With the TEDS framework/procedure a novel and comprehensive approach to systematic artifact evaluation and comparison had been presented a few years ago, whose effectiveness and analytical power in comprehensive and highly detailed artifact evaluations and comparisons was empirically shown; however, despite its demonstrated capability TEDS still proved to be time and resource consuming like other evaluation approaches before. In order to overcome these constraints and provide evaluative feedback more quickly to developers and service providers, TEDSrate, a Web-based evaluation tool employing the TEDS framework/procedure, was developed. The tool was tested with two real-world organizations, the City of Seattle Emergency Operations Center (EOC) and the Seattle Sounders Football Club. The tests suggest that the highly configurable TEDSrate tool can fully implement and administer the TEDS framework/procedure and, at the same time, provide instantaneous, cost-effective, comprehensive, and highly detailed artifact evaluations to both developers and service providers

Using Theories to Design a Value Alignment Model for Smart City Initiatives

Smart city initiatives are widely becoming part of the world agenda to address crises and to identify new initiatives for countries to manage resources while providing better living conditions for all citizens. The purpose of this study was to design a model to support the alignment of value in Smart City initiatives. To address this purpose, a systematic literature review (SLR) was conducted to find what Smart City initiatives have been addressed in empirical studies, and what dimensions and factors are linked to these initiatives. The SLR also identified the stakeholders in a Smart City, and what their roles should be linked to these initiatives. Six theories were identified and used to undergird the researcher’s understanding of the domains of Smart Cities, value and alignment. The concepts from these theories were then used with the SLR findings to design a conceptual model for Smart City initiatives. The proposed Value Alignment Smart City Model (VASC) can be used to plan or assess Smart City initiatives. The main contribution is the alignment of value amongst stakeholders to support the success of such initiatives. Further research is required to investigate adopting the model and empirically evaluate it

A meta-analysis of data collection in serious games research

Serious game analytics share many of the challenges of data analytics for computer systems involving human activity. Key challenges include how to collect data without influencing its generation, and more fundamentally, how to collect and validate data from humans where a primary emphasis is on what people are thinking and doing. This chapter presents a meta-analysis of data collection activities in serious games research. A systematic review was conducted to consider metrics and measures across the human–computer interaction, gaming, simulation, and virtual reality literature. The review focus was on the temporal aspect of data collection to identify if data is collected before, during, or after gameplay and if so what fundamental processes are used to collect data. The review found that the majority of data collection occurred post-game, then pre-game, and finally during gameplay. This reflects traditional difficulties of capturing gameplay data and highlights opportunities for new data capture approaches oriented towards data analytics. Also we identify how researchers gather data to answer fundamental questions about the efficacy of serious games and the design elements that might underlie their efficacy. We suggest that more standardized and better-validated data collection techniques, that allow comparing and contrasting outcomes between studies, would be beneficial