Week 11 – Usability Tests and Evaluator Bias

November 6, 2011

Friess, Erin. “Discourse Variations Between Usability Tests and Usability Reports.” Journal of Usability Studies 6, no. 3 (2011): 102-16.

 

The article “Discourse Variations Between Usability Tests and Usability Reports” documents research on discourse analysis techniques to compare the language used by the end-users to the language used in the evaluators’ oral reports.  Friess conducted five rounds of formative testing involving three pairs of team members who were all novices in conducting usability tests; these team member pairs were each assigned a participant.  A team of raters read the transcripts, watched the video recordings, and “determined if any of the issues mentioned in the oral reports but not mentioned by the usability participant could be reasonably assumed by the actions of the usability participant in the think-aloud usability test” (105).  From there, 83.9% of the findings had some basis in the usability testing;  however, 65% of these were accurate findings and 34.6% were potentially inaccurate findings (106).  Both the data for the accurate and inaccurate findings came from the sound bites and from evaluator interpretation.  The discussion portion of Friess’s article comments on the gatekeeper role of the evaluators and how this powerful role could explain why differences exist between the language used by end users and the language used by evaluators’ oral reports.  Four possible explanations include the following:

 

  • Confirmation Bias in Oral Reports – The evaluators appeared to seek out confirmation for issues they had previously identified (110)
  • Bias in What’s Omitted in the Usability Reports – The evaluators at no time presented finding to the group that ran counter to a claim the evaluators made at a previous meeting (111)
  • Biases in Client Desires – The evaluators did not mention a specific participant desire for an index because the client had already specified that including an index was not an option (112)
  • Poor Interpretation Skills – The evaluators were not well-experienced in this kind of study and therefore clung to the few pieces of data that they understood well (sound-bite data) (113)

 

Honestly, I found this article to be somewhat surprising.  It bothers me that so many evaluators will orally communicate information about the participants’ tests without actually referencing the transcript or notes of the test itself.  Relying solely on memory, particularly when there might be specific biases at stake, has proven faulty time and time again.  For example, my client for my STEM brief is currently studying the consistency of positive flashbulb memories over time.  Her research indicates that as time passes, individuals continue having a strong belief in the veracity of their memories while results indicate that the consistency of those memories actually decreases.  Even though the team members conducting the tests do not experience flashbulb memory while monitoring or observing the participants, there is still the element of unreliability when it comes to presenting information from their memories rather from documented data.  The whole process of learning to interpret results accurately and without bias is a crucial component of good practice in technical communication.  This study reminds me that certain parameters need to be in place even in a semi-informal testing environment of do-it-yourself usability practice.  How could conducting experiments in this fashion hurt the credibility of technical communication as a field?  What are some safeguards to put into place to avoid such biases and discrepancies in the future with do-it-yourself usability test


Week 10 – New New Media

October 30, 2011

Levinson, Paul. “Why “New New” Media?” In New New Media, 1-16. New York: Allyn & Bacon, 2009.

In Paul Levinson’s article, “Why “New New” Media?”, Levinson discusses the widespread implications of understanding the differences not just between old media and new media but also the differences between new media and new new media.  Levinson begins by defining the five most prominent principles of new media which include:

  • You Can’t Fake Being Nonprofessional – authors are not working for a newspaper or broadcast medium
  • Choose Your Medium – people can decide which medium they prefer which complements their specific talents
  • You Get What You Don’t Pay For – media are free to the consumer and sometimes for the producer
  • Competitive and Mutually Catalytic – media are competitive and they simultaneously support each other
  • More Than Search Engines and Email – new new media are different because they allow users to customize options, create content, and add specific applications (2-3)

One key element present in both new media and new new media is that “it give[s] users the same control of when and where to get text, sound and audio-visual content” (3).  Within the scope of new new media, there are multiple categories.  Levinson acknowledges that overlap occurs between categories; however, he defines these categories “based on the services they provide and the way they provide them” (5).  These categories include 1) Print, Audio, Audio-Visual, Photographic, 2) News, 3) Social Media, 4) General vs. Specific Systems, 5) Politics and Entertainment, 6) New New Media and Governmental Control, 7) Microblogging and Blogging, and 8 ) Hardware vs. Software.  Levinson also discusses how hardware such as the iPhone has propelled and made possible the speed with which the systems previously mentioned have become available (8). 

Levinson then catalogs his own involvement with new new media experiences.  This list of “achievements” in regards to new new media establishes his own knowledge on the topic and builds his credibility as one who understands the field.  Indeed, since joining Facebook in 2004, Levinson has created a MySpace account complete with blog posts, uploaded video segments of televisions appearances to YouTube, contributed to Wikipedia articles, created three podcasts, began an independent blog on Infinite Regress, joined Digg, signed-up with Twitter, and joined Second Life (9-10).  He then explains how he organized his book based on the “order of importance of the new new media in the 2008-2009 world, followed by several chapters that address across-the-board issues pertinent to all new media” (11). 

I found Levinson’s article interesting and insightful, given that I am only somewhat familiar with the new new media he discusses.  In fact, I have only joined two of the new new media he mentions, but I am a frequent user of (or familiar with) almost all the new new media listed.   The section toward the end of his article which discusses “The Dark Side of New New Media” resonated with me, particularly because of a recent incident at Lovejoy High School.  Apparently, cyberbullying has recently driven a boy at the school to attempt suicide.  He is currently in the hospital, barely emerging from a coma, and yet students at the high school are continuing to post disgusting, hateful content on his Facebook page.  One student even posted a YouTube video this weekend titled, “How to Commit Suicide.”  We need to be cognizant of such horrific events because new new media is not going away any time soon.  Therefore, we have the responsibility as technical communicators to use the new new media is appropriate, professional, and ethical ways to achieve a positive purpose.  How can we engage effectively with new new media while setting positive examples to the younger generation of new new media users?  Additionally, how do we communicate the ethics of new new media in a practical way?


Week 7 – Incorporating Rhetorical Devices into Plain Style

October 8, 2011

Whitburn, Merrill D., Marijane Davis, Sharon Higgins, Linsey Oates, and Kristene Spurgeon. “Landmark Essay: The Plain Style in Scientific and Technical Writing.” In Three Keys to the Past: The History of Technical Communication, edited by Teresa C. Kynell and Michael M Moran, 123-30. Stamford, CT: Ablex Publishing, 1999.

Whitburn, Davis, Higgins, Oates, and Spurgeon explore the emergence of the stylistic preference of “plainness” in “Landmark Essay:  The Plain Style in Scientific and Technical Writing.”  They begin by providing background on the history of rhetoric which emphasized a style celebrating “copiousness and ornamentation” (124).  However, the shift to the plain style emerged in the wake of post-revolutionary scientists who valued “acquisitions of  . . . knowledge though observation of nature, and words . . . to approximate mathematical symbols” (125).  According to the authors of the essay, this dramatic change in style involved focused on “what not to do [rather] than what to do in  . . . writing” (126).  Rather than sacrifice all elements of rhetoric for the sake of seeking plainness, Whitburn and her co-authors suggest that several rhetorical devices should be encouraged in technical writing in order to provide clarity and emphasize specific points within a text.  For example, Whitburn includes examples of parallelism, ellipsis, polysyndeton, parenthesis, climax, and antithesis as elements of rhetoric which technical writers can employ to enhance “clarity, emphasis, and smoothness” (128).  The whole of this essays seeks to shed light on the “dogmatism” of the technical writing field  regarding elements of style (129).  Whitburn concludes the essay by establishing the value of incorporating rhetorical devices, citing value in encouraging students to be more aggressive writers, emphasizing that writing is an art, arousing reader interest, and helping with discovery of content (129).

This essay attempts to move the field of technical writing toward the “arts” end of the arts-sciences spectrum by emphasizing the value of rhetorical devices.  I find this to be an interesting idea because most (non-English-major) people assume that rhetorical devices serve only to convolute text and provide unnecessary, figurative language.  However, I am inclined to agree with Whitburn regarding the value of incorporating some aspects of rhetoric into technical communication.  Parallelism, for example, is a device which helps to balance elements in a text and is therefore just as applicable in a passage about microbiology as it is in a passage about a modern-day rendition of Hamlet.

Though I do agree with aspects of Whitburn’s argument, I also feel that her comments could be abused in the realm of technical communication.  Depending on the audience, a detailed, sensory-filled description of a scene (as depicted in the “living history interpretation” on pg. 129) might hinder the reader’s response to the work as a whole.  Additionally, if the rhetorical devices employed serve primarily to provide flavorful word-play, the ethos of the writer could be sacrificed as well.  What types of audiences would respond positively to the incorporation of rhetorical devices into technical writing?


Week 6 – “How Technical Communication Textbooks Fail Engineering Students”

October 1, 2011

Wolfe, Joanna. “How Technical Communication Textbooks Fail Engineering Students.” Technical Communication Quarterly 18, no. 4 (2009): 351-75.

 

In the article “How Technical Communication Textbooks Fail Engineering Students,” Joanna Wolfe discusses how the nature of most technical communication textbooks are inherently geared toward humanities-based styles and genres, therefore limiting the effectiveness of such strategies in the field of engineering.  Additionally, there are four main areas of technical communication textbooks which directly conflict with the knowledge and practices of engineering students.  These areas include:  active versus passive voice, citation practices, data visualization, and presentation of data and results (354).

In engineering, passive voice is necessary to focus the reader’s attention on an object instead of on the person or force manipulating the object.  Using passive voice places the emphasis on the object and can often be more concise.  We often talk about how maintaining credibility with an audience is probably the most important element to consider in technical writing; however, advocating active voice rather than passive voice in an engineering field damages ethos.  In fact, “passive voice is often a deliberate rhetorical choice for engineers who strive to present an ethos of scientific objectivity rather than personal authority in writing” (357).  This notion echoes Miller’s stance when she claims that “scientists adopt [such] conventions [as] the obvious stylistic methods meant for staying out of the way . . . [such as using] subject matter third-person . . . personification, [and] passive voice” (Miller 4).  Miller goes on to assert that we, as teachers “place a double-burden on students by urging them to be impersonal on one hand, but denying them, in the name of stylistic grace, these obvious syntactic tools on the other” (Miller 4).  Surprisingly, Miller supports this idea that we are doing a disservice to students in certain scientific fields by providing textbooks which insist on syntactical form biased toward humanistic style preferences.  This is both unfair and unhelpful to engineering students.

Another element Wolfe notes is the bias in textbooks toward humanities citation practices.  Almost all textbooks advocate for APA, MLA, or Chicago style for documentation.  However, “emphasizing author’s names, titles, and direct quotes . . . focuses readers’ attention on authors and their specific words” rather than on the content itself being discussed (358).  Again, the importance of ethos flies into the forefront of my mind.  If we continue to disseminate the premise that most citation styles in the scientific field reflect APA or MLA, we “undermine our ethos as teachers” because the premise is simply not true (359).  Various numbered citation-sequence styles are more common in engineering, computer science, chemical engineering, medicine, and physics and well as other scientific fields (358).

Data visualization and the presentation of data and results are often ignored or completely eliminated from technical writing textbooks.  I believe this tendency is, again, directly linked to the humanities influence; indeed, Wolfe notes that some textbooks advocate certain types of charts (such as three-dimensional bar graphs) based solely on the aesthetic aspect.  Sacrificing clarify for aesthetics should not occur in a field which prides itself on clarity of information.  Additionally, the lack of clear visuals is directly related to the textbooks’ “general disregard for numbers and data” (364).  Teaching engineering students how to interpret results is a foundational skill for them to use their field; it is troubling that our textbooks fail to equip them for this task, especially since the ability to interpret results “is the primary rhetorical skill that separates ‘good’ from ‘poor’ engineering writers” (365).

This failure on the part of technical communication textbooks (and technical communication classes) to sufficiently equip engineering students stands out to me in the succinct response from an engineering student after taking such class.  According to a study by Ford, one student asked, “What’s the point of taking technical writing when all you are going to do is use templates afterwards?” (352).  This troubling statement reinforces the attitude held by numerous people in the scientific realm who see technical writing purely as a skill-set geared toward learning correct format; the problem-solving aspect and understanding of audience and purpose is completely eliminated when this attitude becomes the definition of the field.  How will improving technical writing textbooks geared toward engineering students affect the mindset of how those students perceive the value of technical writing?