Reading :: Working Minds

Working Minds: A Practitioner's Guide to Cognitive Task Analysis
By B. Crandall, G. Klein, and R.R. Hoffman

I've discussed Gary Klein's work before, and specifically how much I appreciate his attitude of trust and respect toward his participants. Klein's work focuses on how experienced professionals (such as firefighters, NICU nurses, and soldiers) make intuitive decisions in high-stakes, high-pressure environments.

To research such cases, Klein needed an ecological approach that allowed him to get at situated decision making in cases in which the participants couldn't necessarily articulate their assumptions, options, or triggers. At the same time, Klein couldn't just follow firefighters around—the events he wanted to study were just too rare, and when they happen, he didn't want his team to get in the way of rescue operations.

The approach that Klein and his partners developed for such cases is called cognitive task analysis (CTA), which "helps researchers understand how cognitive skills and strategies make it possible for people to act effectively and get things done," according to the back of this book. The book is, as the subtitle states, "A Practitioner's Guide to Cognitive Task Analysis." That is, it describes CTA and the situations in which it could be useful; it offers tools and strategies for performing CTA; and it discusses how CTA brings value to the participants. In this sense, it reminds me of Beyer and Holtzblatt's Contextual Design, a similar methodology book written by consultants for practitioners (although addressing different situations with a different methodological approach).

What struck me about Working Minds, though, was that the coauthors had developed a qualitative approach within psychology. As the authors note, in psychology and human factors, analysis typically happens quantitatively; students have little qualitative research training and use "preset plays" based on common statistical tests (p.107). "However, many CTA methods generate data that do not fit easily into standard statistical approaches" (p.107), and this is a problem since "quantification typically means stripping a body of data of its contextual links and decomposing it in order to assign numerical values" (p.108). At the same time, qualitative methods emerging from sociology, anthropology, and education tend to be focused on "topics that do not have a cognitive focus, such as analysis of social processes or attitudes surrounding terminal illness" (p.108).

Faced with this disjuncture, the authors set out to develop a suitable qualitative research approach for psychology's foci. Like many qualitative research approaches, this approach is not linear, with oscillations between structuring data and identifying meaning (p.110). It involves four main steps: preparation; structure data; discover meaning; identify/represent key findings (p.111). And the analysis involves creating "an audit trail that links raw data to eventual outcomes" (p.113). That is, it looks a lot like structured qualitative case study research.

In Chapter 8, the authors "introduce a level of cognitive phenomena—referred to as macrocognition— that emerges when we shift the focus to natural contexts. These are the types of cognition that CTA methods are uniquely designed to capture" (p.131). They discuss this level of cognition in terms of purpose, prior experience, situation, challenge, tools, team members, and organizational constraints (p.132). Macrocognition, they say later, is a "collection of cognitive processes and functions that characterize how people think in natural settings," as opposed to microcognition, which is "studied using carefully controlled methods and procedures" and is supposed to investigate basic, universal features (p.136). Think here of the contrast between Klein's contextualized field interviews and Kahnemann's word problems — or the contrast between laboratory measures of executive functions and ecologically valid measures. As the authors assert, "individuals make decisions but so do teams" and "decision making often depends on artifacts" (p.136). Cognitive activity, the authors assert (citing Hutchins), is "distributed across multiple agents as part of a stream of activity" (p.157).

Overall, I found this book to be rewarding. The authors have identified a need for a qualitative methodology in psychology, oriented to decision-making; they have drawn when appropriate from qualitative traditions in adjoining disciplines; but they have also recognized the differences between those methodological orientations and the one they need. They have carefully and responsibly developed and validated an approach that works for their objectives. And they have articulated it clearly and well—the book is well organized and easy to read. The result is a good intro for practitioners, but I think it would also be suitable for a methods class (with suitable framing). If you're interested in qualitative methodology, and especially if you're wondering why someone would pursue qualitative methods instead of quantitative ones, check it out.