Hepenstal, SZhang, LWong, BLWSchmidt, AVäänänen, KKristensson, POPeters, A2025-02-202025-02-202023-04-19Sam Hepenstal, Leishi Zhang, and B. L. William Wong. 2023. The Impact of System Transparency on Analytical Reasoning. In Extended Abstracts of the 2023 CHI Conference on Human Factors in Computing Systems (CHI EA '23). Association for Computing Machinery, New York, NY, USA, Article 274, 1–6. https://doi.org/10.1145/3544549.35857869781450394222http://hdl.handle.net/10292/18730In this paper, we present the hypothesis that system transparency is critical for tasks that involve expert sensemaking. Artificial Intelligence (AI) systems can aid criminal intelligence analysts, however, they are typically opaque, obscuring the underlying processes that inform outputs, and this has implications for sensemaking. We report on an initial study with 10 intelligence analysts who performed a realistic investigation exercise using the Pan natural language system [10, 11], in which only half were provided with system transparency. Differences between conditions are analysed and the results demonstrate that transparency improved the ability of analysts to reason about the data and form hypotheses.Copyright © 2023 Owner/Author. Permission to make digital or hard copies of part or all of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for third-party components of this work must be honored. For all other uses, contact the Owner/Author.46 Information and Computing Sciences4602 Artificial IntelligenceNetworking and Information Technology R&D (NITRD)Machine Learning and Artificial Intelligence1202 Building1503 Business and Management3507 Strategy, management and organisational behaviourConference ContributionOpenAccess10.1145/3544549.3585786