Looking for fraud in digital footprints: sensemaking with chronologies in a large corporate investigation

During extended sensemaking tasks people typically create external representations that integrate information and support their thinking. Understanding the variety, role and use of these is important for understanding sensemaking and how to support it effectively. We report a case-study of a large, document-based fraud investigation undertaken by a law firm. We focus on the construction and use of integrated representations in the form of chronologies. We show how these supported conjecture recording, focussing on time-periods, identifying gaps, identifying connections and reviewing interpretations. We use our findings to highlight limitations of a previous analysis of representations in sensemaking which regards this as schema definition and population. The findings also argue for search tools designed to identify date references in documents, for the support of ad-hoc event selections, and the support of linking between integrating representations and source documents

Effective ways to use nonpersonal information in healthcare: report from a workshop held at University College London 15-16 April 2004

New information technologies are being introduced in the UK National Health Service as resources for the acquisition of clinical knowledge. These are forcing working practices to adapt and are affecting and challenging perceived roles, relationships and expectations of patients and health professionals alike. Effective ways to use nonpersonal information in healthcare was a two-day workshop hosted by UCL Interaction Centre at University College London intended to provide a forum for practioners and researchers working in the area of clinical health information delivery to come together to discuss access to health information, and to consider how the various challenges and opportunities relating to electronic information provision can be managed most effectively. For the first day of the workshop, the theme for presentations and discussion was information provision for and access by health professionals. Talks were given by Julius Weinberg (City University, London), Roger Slack (University of Edinburgh) and Anne Adams (University College London). The theme for the second day was information provision and access by patients. Presentations were given by Mig Muller (NHS Direct), Jane Wilson (Whittington Hospital and Medi-notes), Andrew Herxheimer (University of Oxford) and Henry Potts (University College London). On both days, delegates formed into three groups for breakout sessions in which they discussed and reported back on: information quality and use, social and organisational context, and user requirements and training in relation to the respective daily theme (health practitioners/patients). This report summerises each of the presentations and the reports by the breakout groups

E-discovery viewed as integrated human-computer sensemaking: the challenge of 'Frames'

In addressing the question of the design on technologies for e-discovery it is essential to recognise that such work takes place through a system in which both people and technology interact as a complex whole. Technology can promote discovery and insight and support human sensemaking, but the question hangs on the extent to which it naturally extends the way that legal practitioners think and work. We describe research at UCL which uses this as a starting point for empirical studies to inform the design of supporting technologies. We report aspects of an interview field study with lawyers who worked on a large regulatory investigation. Using data from this study we describe document review and analysis in terms of a sequence of transitions between different kinds of representation. We then focus on one particular transition: the creation of chronology records from documents. We develop the idea that investigators make sense of evidence by the application of conceptual ‘frames’ (Klein et al’s, 2006), but whilst the investigator ‘sees’ the situation in terms of these frames, the system ‘sees’ the situation in terms of documents, textual tokens and metadata. We conclude that design leverage can be obtained through the development of technologies that aggregate content around investigators’ frames. We outline further research to explore this further

Exploring the importance of reflection in the control room

While currently difficult to measure or explicitly design for, evidence suggests that providing people with opportunities to reflect on experience must be recognized and valued during safety-critical work. We provide an insight into reflection as a mechanism that can help to maintain both individual and team goals. In the control room, reflection can be task-based, critical for the 'smooth' day-to-day operational performance of a socio-technical system, or can foster learning and organisational change by enabling new understandings gained from experience. In this position paper we argue that technology should be designed to support the reflective capacity of people. There are many interaction designs and artefacts that aim to support problem-solving, but very few that support self-reflection and group reflection. Traditional paradigms for safety-critical systems have focussed on ensuring the functional correctness of designs, minimising the time to complete tasks, etc. Work in the area of user experience design may be of increasing relevance when generating artefacts that aim to encourage reflection

Keeping Up With the Law: Investigating Lawyers’ Monitoring Behaviour

Purpose - We wanted to provide an enriched understanding of how lawyers keep up-to-date with legal developments. Maintaining awareness of developments in an area (known as ‘monitoring’) is an important aspect of professional’s information work. This is particularly true for lawyers, who are expected to keep up-to-date with legal developments on an on-going basis. Design/methodology/approach - We wanted to provide an enriched understanding of how lawyers keep up-to-date with legal developments. Maintaining awareness of developments in an area (known as ‘monitoring’) is an important aspect of professional’s information work. This is particularly true for lawyers, who are expected to keep up-to-date with legal developments on an on-going basis. Findings - The lawyers mostly used electronic resources (particularly e-mail alerts and an electronic tool that alerted them to changes in website content), alongside interpersonal sources such as colleagues, customers and professional contacts. Printed media such as journals and newspapers were used more rarely and usually to complement electronic and person-based resources. A number of factors were found to influence choice. These included situational relevance, presentation, utility and trustworthiness, the speed of content acquisition, and interpretation facilitated by the resource. Originality/value - Our findings enrich our understanding of lawyers’ monitoring behaviour, which has so far received little direct research attention. Our design suggestions have the potential to feed into the design of new and improvement of existing digital current awareness resources. Our findings have the potential to act as ‘success criteria’ by which these resources can be evaluated from a user-centred perspective

SensePath: Understanding the Sensemaking Process Through Analytic Provenance

Sensemaking is described as the process of comprehension, finding meaning and gaining insight from information, producing new knowledge and informing further action. Understanding the sensemaking process allows building effective visual analytics tools to make sense of large and complex datasets. Currently, it is often a manual and time-consuming undertaking to comprehend this: researchers collect observation data, transcribe screen capture videos and think-aloud recordings, identify recurring patterns, and eventually abstract the sensemaking process into a general model. In this paper, we propose a general approach to facilitate such a qualitative analysis process, and introduce a prototype, SensePath, to demonstrate the application of this approach with a focus on browser-based online sensemaking. The approach is based on a study of a number of qualitative research sessions including observations of users performing sensemaking tasks and post hoc analyses to uncover their sensemaking processes. Based on the study results and a follow-up participatory design session with HCI researchers, we decided to focus on the transcription and coding stages of thematic analysis. SensePath automatically captures user's sensemaking actions, i.e., analytic provenance, and provides multi-linked views to support their further analysis. A number of other requirements elicited from the design session are also implemented in SensePath, such as easy integration with existing qualitative analysis workflow and non-intrusive for participants. The tool was used by an experienced HCI researcher to analyze two sensemaking sessions. The researcher found the tool intuitive and considerably reduced analysis time, allowing better understanding of the sensemaking process

Using Machine Learning to Infer Reasoning Provenance from User Interaction Log Data

The reconstruction of analysts’ reasoning processes (reasoning provenance) during complex sensemaking tasks can support reflection and decision making. One potential approach to such reconstruction is to automatically infer reasoning from low-level user interaction logs. We explore a novel method for doing this using machine learning. Two user studies were conducted in which participants performed similar intelligence analysis tasks. In one study, participants used a standard web browser and word processor; in the other, they used a system called INVISQUE (Interactive Visual Search and Query Environment). Interaction logs were manually coded for cognitive actions based on captured think-aloud protocol and posttask interviews based on Klein, Phillips, Rall, and Pelusos’s data/frame model of sensemaking as a conceptual framework. This analysis was then used to train an interaction frame mapper, which employed multiple machine learning models to learn relationships between the interaction logs and the codings. Our results show that, for one study at least, classification accuracy was significantly better than chance and compared reasonably to a reported manual provenance reconstruction method. We discuss our results in terms of variations in feature sets from the two studies and what this means for the development of the method for provenance capture and the evaluation of sensemaking systems