Wednesday, October 13, 2004

Reading :: Qualitative Inquiry and Research Design

Originally posted: Wed, 13 Oct 2004 06:54:11

Qualitative Inquiry and Research Design : Choosing among Five Traditions

by John W. Creswell

Well, it turns out I'm not crazy. I've been trying to nail down exactly what people mean by qualitative research for a while now, particularly because most of the time people simply call it "ethnography." But, I wondered, is grounded theory simply a kind of ethnography? How about case studies --- can they be considered ethnography when Yin emphatically says that they're not? How does action research relate to these two? Much of what I read about "qualitative research," especially in my home discipline of rhetoric, seems theoretically and methodologically mushed up. It turns out that other fields have this problem as well, and that is what motivated Creswell to write this book.

To the extent possible, Creswell separates, compares, and contrasts five different traditions of qualitative research: biography, phenomenology, ethnography, grounded theory, and case studies. (He also notes omissions such as action research.) Like me, Creswell loves taxonomies and comparisons and tables, and he provides plenty here. Using thoughtfully considered examples of each kind of research, he traces through design, traditions of inquiry, philosophical and theoretical frameworks, research questions, data collection, data analysis and representation, report writing, and issues of quality and verification. By the end of the book, I could articulate the differences among the traditions, see how they had grown out of different disciplines, and determine which one would be most useful for answering a given question. And Creswell's direct language, clear illustrations, and focused organization allowed me to quickly understand, apply, and navigate through the material.

I also appreciated Creswell's discussion of software for supporting qualitative research. He focuses on NUD*IST, a qualitative theory-building program designed for aiding grounded theory (and not coincidentally published by SAGE, the company that Creswell's book). Through text and diagrams, he describes how NUD*IST could be used to support each of the five qualitative approaches. He also discusses other types of software in an evenhanded manner.

Finally, the book includes a thick glossary that includes definitions by approach -- especially useful since different traditions sometimes use similar words for different phenomena, different words for similar phenomena, etc. It also includes article-length examples of each approach, something that Creswell uses to great advantage when he provides illustrations throughout the book. The one thing that bothered me about the examples was that in their subject matter -- indigence, sexual assault, violent assault -- they tended to emphasize the heartrending pathos of victimhood, and I worry that this will lead readers (especially students) to seek out those sorts of projects over the less exciting, more quotidian, but still valuable aspects of life.

Creswell's text is not going to provide the detailed information that one might get from reading more narrowly defined texts (e.g., Yin's book on case study research or Strauss and Corbin's book on grounded theory). But it provides good strong overviews. I've been looking for a good solid text to anchor my grad-level qualitative research course this spring, and I think this one is it.

Blogged with Flock

Reading :: Becoming Qualitative Researchers

Originally posted: Wed, 13 Oct 2004 07:24:56

Becoming Qualitative Researchers: An Introduction (2nd Edition)

by Corrine Glesne

I recalled this book from the library because I had seen it mentioned so frequently in the qualitative research texts I've been reading. And the title, of course, marked it as a good introductory book for the graduate course I'm planning to teach. When I picked it up from the library last week, I was surprised by how thin and how friendly it looked. And as I read the first chapter on the way back to the office (a bad habit, I know), I thought: I have found a real winner here.

Due to other obligations, I put it away a while and finally read the rest of it today on the plane to Memphis. Unfortunately, the book didn't manage to sustain my initial enthusiasm, in large part due to its contrast to Creswell's book (which I finished on the same flight). Where Creswell's book gave us a broad scope of different methods, Glesne focuses on variations of ethnography. Where Creswell's book was tightly organized, Glesne's meanders. Where Creswell's book used focused, plain language, Glesne's often seems to be a forum for creative use of language. In fact, in Glesne's second to last chapter, "Improvising a Song of the World: Language and Representation," she makes much of the fact that she takes creative writing classes and encourages her students to render their research in alternative forms of expression such as poems, plays, etc. I am not particularly interested in this sort of reporting. If you are, you really should buy this book immediately.

I do appreciate how Glesne differentiates among traditional, critical, and feminist ethnographies and how she discusses action research. The discussion of postmodernism in research is more than worthwhile. Unlike Creswell, Glesne also includes a helpful discussion of research proposals and institutional review boards. And her discussion of computer-assisted data management and analysis, though shorter than Creswell's, is broader. In her best passages, she is engaging and exciting. (Of course, in her worst passages, she is self-focused, self-indulgent, and vague about specifics.) She dedicates a considerable number of pages to writing up the report, including advice from Strunk and White (use active voice; eliminate constructions such as "it is" and "there is").

Is the book worthwhile? It doesn't represent a big time investment. But I didn't find the payoff to be as high as some of the other books I've reviewed on the list. I wouldn't suggest using it as a class "introduction," though, because I don't think its meandering style will appeal to busy students. >

Blogged with Flock

(My own personal paradigm shift)

Originally posted: Wed, 13 Oct 2004 07:27:22

This weekend I had the realization that I haven't been managing projects well. So I'm in the middle of a paradigm shift. If you had looked at my computer and handheld on Friday, you might have been struck by the elaborate Aristotelian directory structure; same with my email program, in which I have carefully stored every piece of email since spring 1999. Same with my handheld, which not only had a dozen categories for pigeonholing events and todos, but also had an elaborate directory structure in its own right. But these categories, which I had conceived as ways to manage my data and my projects, were actually fragmenting my data and projects pretty severely. I could see parts, but not the whole picture. That played havoc not just with my personal projects but also with my delegation at the CWRL.

I seriously considered project management software, like Entourage, that would give me back the whole picture. But after test-driving Entourage, I rejected it: it basically did what I was doing already, throwing things into containers. Entourage might have allowed me to scale up, but ...

But when I want something on the web, I do what everyone else does. I Google it. Or I Clusty it. The last thing I want to do is go to a category-based site like Yahoo. So why not take that search paradigm and apply it to the rest of my life?

So I took a second look at some software my wife had asked me to test earlier this month. Instead of using MacJournal for notes -- a great program that allows you to store text notes in categories -- I transferred everything to Notational Velocity, a fantastic app that allows you to find-as-you-type (kind of like how your browser tries to guess the URL as you type it in). No categories, just searching. I downloaded Quicksilver, a launcher app that allows me to find any document or app by name and launch it -- just hit ctrl-spacebar, find-as-you-type, then hit return. All documents went into my Documents folder; the elaborate file structure went away. Same with my email client, which now has only a few basic categories rather than hundreds; it has an elaborate search engine built in.

Same story on my handheld. I deleted all the categories -- they don't sync with iCal anyway, for some reason. And I rediscovered the search function on my iPaq, which is actually quite well developed.

For more elaborate project management, I now use a spreadsheet. Projects in column 1; concrete steps in column 2; delegation, if necessary, in column 3; target date in column 4. The spreadsheet transfers to my iPaq too. The concrete steps can be copied and pasted into the todo list if appropriate, but I suspect that won't have to be done very often.

I'm also beginning to read 43 Folders, a site which someone (I forget who) recently described as time management porn.

This really is a significant change for me; let's see how it works out.

Blogged with Flock

Reading :: Handbook of Usability Testing

Originally posted: Wed, 13 Oct 2004 06:26:10

Handbook of Usability Testing: How to Plan, Design, and Conduct Effective Tests

by Jeffrey Rubin

I teach a usability course nearly every year -- this time it was on accessibility and usability -- and every year I dither about what book to use. Should I use Carol Barnum's book Usability Testing and Research, a book for which I have much admiration, even though I contriibuted perhaps the most awkward recommendation statement ever? Should I instead use Dumas and Redish's A Practical Guide to Usability Testing, a venerable old tome? Or should I pick up something by Jakob Nielsen?

This year I returned to Jeff Rubin's book, which has been around for a while but still seems to have the most concise and readable discussion out there. It doesn't have the high quality of examples that Barnum has, nor the up-to-date cachet of Nielsen's work. But it does get into the guts of usability methodology, and in a more rigorous way than the others do. And since my students have a usability lab for testing websites, something with a focus on methods seemed like the best choice.

Rigor, of course, is not something I normally associate with usability testing. Contemporary usability tests are descended from orthodox experimental psychology (in fact, Rubin's training was in that area), and it used to be that a "real" usability test was essentially an experiment oriented to a product, with N>=8 for each condition, multiple conditions, etc. (e.g., the Bellcore work on SuperBook). But different encrustations have developed, the most noteworthy being that testing often occurs with relatively small Ns and often no alternate conditions at all. In describing why the small N has caught on, Rubin repeats a classic fallacy first perpetrated by Jakob Nielsen. Let's call it the Easter egg fallacy.

The Easter egg fallacy goes something like this. Any given product has a finite number of problems with it. Chances are that you won't find every single problem with a single test, since users have varying levels of experience and varying backgrounds. But you'll find a good number of them. With N=5 or 6, you'll find upward of 80% of the problems. So it's not necessarily cost-effective to run tests with larger Ns -- at some point you're bringing in people who aren't really telling you anything you don't already know. Better to run multiple iterative tests with small Ns.

I don't have anything against multiple iterative tests with small Ns, but I do have a problem with the underlying logic. The assumption is that problems are hidden in the product, like Easter eggs are hidden on a dewy green lawn. How many can we find? But problems are not simply hidden in products; problems are problems because of how the product is enacted in a given environment by given actors. When programmers say with only a little irony that "it's not a bug, it's a feature," they're onto something: problems can be shifted out of their frames, applied differently (not necessarily even idiosyncratically), and can become solutions to still other problems. I recall a study sometime back -- by Rosson, I think -- of an email feature that included a bug that let people do something they wanted to do. That bug actually did turn into a "feature" in the next release, with not much more change than a write-up in the documentation. What I am saying is that the Easter egg fallacy is a structuralist reading, and it doesn't work as well as a constructivist one.

Most usability books have a version of the Easter egg fallacy. But to Rubin's credit, he does encourage larger Ns and multiple conditions, and he does take other measures to ensure rigor (as much as you'll get in a usability test, anyway). His writing is characteristically terse and parsimonious, and that extends to his examples of test plans and test reports. For the money, this book gives a quick, readable, and surprisingly complete introduction to usability testing.

Blogged with Flock

Reading :: Writing Winning Business Proposals

Originally posted: Wed, 13 Oct 2004 05:57:09

Writing Winning Business Proposals: Your Guide to Landing the Client, Making the Sale and Persuading the Boss

by Richard C. Freed, Joe Romano

Full disclosure: I took the grad level proposal writing course from Rich Freed back in 1994. It was my first semester in the Ph.D. program at Iowa State. The book was in press, so we used a course pack that was essentially the book's manuscript. I really wasn't so sure about the course: the things that Freed called proposals didn't look much like what I had been taught proposals were supposed to be (i.e., recommendation reports). And I'm a little embarrassed to admit that I didn't really "get" the methodology or warm to it until after the semester was over.

In fact, I don't think it really sank in until I taught the undergrad version of the course a few years later. I reread the book and suddenly it clicked. Perhaps that's because I had become more theoretically sophisticated by then, or because I had grown to appreciate method. Because, even though the book is written in the sort of gladhanding voice that one associates with business guides, it is quite sophisticated in its methodology. Although the business proposal is a relatively simple genre, it requires a complex -- and, more problematically, a sustained -- argument, one that will build bridges to several different stakeholders simultaneously. That's not easy, and it's next to impossible to do well the way that students are inclined to do things: procrastinate, sit down and outline the argument in one sitting, and give it a thorough copyediting. To be fair, it's also nearly impossible using the boilerplate and models of the professional proposal writer, although they tend to have experience on their side.

Freed's take is -- as I emphasize repeatedly to my students -- that writing is not expression (i.e., something that emerges from one's head) so much as it is construction, the careful building of a sustained argument out of good building materials. The first half of Freed's book is all about generating those building materials. Writing teachers may be taken aback by those materials -- which mainly consist of forms that students fill out and Aristotelian diagrams representing methods and qualifications subarguments. But they do work, they do act as enormously useful heuristics that interact to generate a multidimensional view of the many readers one's proposal can be expected to have.

Once these building blocks have been generated, assembling the proposal involves drawing heavily and in structured ways from them. It's surprising how quickly a rough draft can be assembled. Freed discusses how to make sure the argument is "aligned" and how to develop its style, displaying in the latter discussion the willingness to bend accepted mechanics rules if the result is a more persuasive document. Throughout, he reminds us that persuasion should have an ethical component: you find the best arguments, but argument building itself is a method of inquiry, a way to determine how your abilities fit the clients' needs. If you can't make that argument well, he says, you shouldn't make it at all.

In this second edition, Freed includes new and revised material: a method for guiding collaborative criticism of proposals, an expanded discussion of Situation, and a better final overview of the process. But the second edition also has the strengths of the first one. This book taught me much of what I know about sustaining arguments; I expect that I'll continue to use it for my proposal writing classes for some time.

Blogged with Flock