Update 3/29/13

This blog has been the basis for:

Update 2/21/12 On February 21, after lots of great feedback, I posted a second part on usability and project management.

In many cases, user research is not a project unto itself. Rather, it is part of a larger web-centric project effort. These larger projects often have a project manager. This person has the task of coordinating the usability effort into the broader project schedule. The project manager generally has some kind of technology management background, but is often not someone who has extensive experience with usability.

This lack of knowledge about usability isn’t inherently a problem, for this is the reason why my company and I are pulled into the project in the first place. However, based on my experiences and lessons learned, here is a “cheat sheet” of things the project manager needs to be thinking about.

“Our schedule is too tight for usability testing.”

Often, time tables are tight and stake holders may feel rushed. When stakeholders hear that a usability study may run over a period of three days, and that the report will take a few more days, they may get the impression that the usability evaluator expects all work to stop during this period. Stakeholders will express the concern to the project manager that there simply isn’t enough time for usability testing.

Recommendation: Let the stakeholder know that not only do you know that time is tight, but that the usability practitioner does as well. Work does not have to stop for the usability test to take place. Further, if the development environment is flexible enough, and ideally if there is perhaps an extra day or two between sets of participants, there may be enough time not only to make changes to a prototype, but also to test out some of the changes with the next round of participants or to adjust the test plan and drill down and zero in on key problems very rapidly.

“You’ll tell my team to scrap the site and start from scratch!”

In addition, the stakeholder may believe that inherent in any usability test comes a risk that they will be told that they should be starting from scratch.

Recommendation: In my experience, with a good development team that has a reasonably good sense of design principles, the testing often uncovers some larger changes but usually uncovers a number of problems that can be fixed with minimal impact to the development schedule. Even if the stakeholder sees some larger-scale recommendations, assuming that there are no show-stoppers, these should be cataloged and saved for a future release. All the minor tweaks and low-hanging fruit, however, could be done immediately, and the stakeholder could see a large value-add for minimal cost.

“I’ve got an incredible design team. I don’t need to have anyone evaluate it.”

I never send out any kind of formal document without having one of my staff review it. No matter how good a writer I think I am, I am 100% sure that I’ll have typos in that document that I won’t find no matter how much I try to look for them. I’ll miss them because it’s my baby. Similarly, a design team, no matter how good they are, will not be able to see issues with something they developed. A design team may also wish to do their own testing, where the designers also operate as usability test moderators.

Recommendation: Don’t assume that a design team, no matter how amazing they are, will be able to see flaws in their creation. Although someone who is an interface designer can also be a usability evaluator, I’d highly recommend that they don’t evaluate a creation that they had a large part in designing. The risk of asking biased questions is much higher, and the risk of not seeing flaws all that much greater. Ideally, someone who has had no role in the creation should be evaluating the site.

“Usability testing is too expensive.”

I sometimes hear the contention that the budget doesn’t allow for usability testing. Once figuring in not only labor hours but also other direct expenses, such as participant recruiting costs, participant incentives, and facility rental, costs may seem like they are really starting to add up.

Recommendation: There are a number of arguments to be made for the cost-justification of usability testing. Even as early as the mid-1990s there were books and articles published on this very topic (Classic revised in 2005: Cost-Justifying Usability). In addition, usability testing doesn’t have to be terribly expensive. First, calculate out what a typical usability study might cost, with an optimal number of representative users in an optimal location with optimal incentives. Look at the final number: is it really too high? If so, consider reducing the number of participants; even if not optimal, at least a fair bit of valuable information is likely to emerge from testing. Similarly, I like to produce full written reports, along with a PowerPoint summary report, but if budget (my allotted hours) doesn’t permit, I might just produce the PowerPoint summary report and make certain to at least convey the general ideas of the findings. In the more uncommon instance that the budget is larger than the base optimal projections, a usability researcher can always add what I’d consider “deluxe” options, like a highlights video from the sessions.

“We’re already doing software testing – we’ve got it covered!”

Sometimes project managers are aware that there is this testing approach called “usability testing,” but they figure it’s not much different than the software/functional testing that they are already doing. They may have spreadsheets of actions that a user is supposed to be able to do, and they need to make sure that all of the design specifications are met. While this software testing is certainly important, project managers may not realize that usability testing is focused on the success of actual users using the website, instead of just verifying that the functionality exists and is working as initially intended. What if those initial intentions and assumptions were incorrect for the expected audience group?

Recommendation: Make sure to consider the needs of the actual users of the site, not just the specifications that have been handed down to you by the stakeholders.

Did I hit all the major points a project manager should be aware of?

These are the points that I’ve thought of off the top of my head. Have I hit all the major issues a project manager should be aware of? Probably not. Please let me know of other things that I’ve missed!

Continue to Part 2

Image Courtesy of nasirkhan / Shutterstock