1995
It was January of 1995 and I had just turned 21, graduated college, and pretty much immediately found a job as a “human factors consultant” (for “web usability consultants” would not exist for a few more years). I was content for about 6 months until someone informed me of a new NASA/Boeing Information Systems partnership called the GLOBE program. The GLOBE program was created by Al Gore to get science teachers using the (then spelled out) “World Wide Web” as a tool in their classrooms. (And if you’re wondering about the Al Gore/Internet thing, I am serious – somewhere I even have a signed certificate from the then VP to prove it!)
Anyway, they were looking for people with several years of technology teaching experience to fly all around the United States and conduct two one-week workshops with science teachers in each location. My teaching experience at that point was limited to teaching a freshman honors seminar and being a CPR instructor, but I had been experimenting with the web pretty solidly since 1991 and not only had my own personal web page but had designed a corporate web presence for a usability firm. I somehow managed to make my case and got the job.
I was living out of a suitcase, home only on the weekends, but I was young and single so that was just fine with me. I loved the teaching. It was very concrete –very hands on – not focused on esoteric academic stuff but on knowledge that the attendees could use immediately. In the years that followed, the kinds of courses that I taught shifted away from how to use the web (can’t imagine too much of a corporate market for that anymore!) and more exclusively towards usability and related topics. But the courses were always classroom (board room, conference room) based. That is, until two weeks ago. Flash forward – January 2011…
The Project
A Lebsontech client asked me to propose a 4-day experiential learning program with one of their clients. Their client wanted to fund a usability study while simultaneously starting to move the company towards usability independence. I was very excited to fly to New England two weeks ago and try out this new kind of training.
Preparation
To prepare, I created a generic moderator’s guide shell and reviewed both the client’s current site and some screenshots they had sent me of the updated site, which was sitting behind a firewall. And that was it – no slides to review or reorganize or customize like most trainings that I’ve done.
Day 1
The first day started with introductions and a 45 minute summary of what we would be doing. Then we dove right in and spent the rest of the day reviewing the site, developing appropriately worded tasks, and polishing off the moderator’s guide.
Day 2
On the second day, we ran through 6 participants. I moderated the first session and two designated employees from the company took turns moderating the remainder while I observed.
Day 3
On the third day, we ran one final participant and spent the rest of the day discussing high-level findings and drafting these findings into a brief report and subsequent PowerPoint.
Day 4
On the fourth day, we had a briefing with senior leadership about the findings and discussed our high-level recommendations. That was it, and I flew home.
The learning curve
I really enjoyed this experience tremendously. It was a whole new way of training and I would love to be able to do something like this again! That said, next time I do this there are a few changes that I’d like to make:
Recruiting
In prior discussion, the client had expressed confidence in their ability to recruit participants. They were given some basics in terms of screening criteria but no detailed instruction on what is involved in finding the right participants, and they didn’t want to use a recruiting agency. So the participants were not ideal on several levels.
Next Time: The learning should be broken up into two parts. Session 1 would occur before recruiting and would involve a primer on recruiting techniques and criteria. The client can then make an informed decision about whether they are able to do the recruiting in-house or whether they’d like to use an outside participant recruiting agency.
Classroom Training
In this case, aside from the brief introduction, there was no formal classroom training. In a subsequent debrief with the staff members who participated, one woman expressed that she wished that there had been a classroom component first. Others agreed that this could have been valuable to give them a heads-up before actually diving in. I had not considered this, but in only 4 days, there would not have been enough time for really in-depth training anyway.
Next Time: Assuming that we decide to break the session into two parts, along with recruiting, and perhaps test plan development, we could include a day or two of formal classroom training. The classroom training should be open to more people than just those doing the experiential learning however, as we could use the opportunity to give a much larger set of staff an understanding of the value and function of usability research; this would help achieve credibility for those that are doing the research.
Morae
I had intended to use Morae software for the logging by the multiple observers, but as soon as we set things up, we saw that port 5555 was blocked by a firewall. This is the port that Morae uses to let the observers connect with the participant’s machine. A request to the IT department to open up this port was unsuccessful. We had to use the corporate network because the website itself was behind a staging server.
Next Time: This is another thing that should be tested in round 1. If there is an issue with a corporate firewall, then there should be time to work out with IT how to do some kind of workaround. In cases where the website is not behind a firewall, we could then choose to use a 3G (or 4G now as the case may be) mi-fi device.
Results
We collaboratively came up with recommendations to the high-level issues that we had encountered during testing, but I was not able to impart the knowledge of how to make recommendations when future issues come up. There also was not enough time to really cull through notes, come up with exact counts of issues, or review recordings; that is, to apply a more rigorous overall analysis methodology, and one which I’d use in my own user research.
Next Time: Classroom training (which developers should be asked to attend as well!) could cover usable design best practices. This would certainly help. But even with two rounds now as part of the training, there still won’t be enough time to go in-depth through notes and recordings, unless we had a round 3. If two rounds are pushing the limits of feasibility, three rounds are likely exceeding them.
So can usability really be taught in a week?
Of course one can’t teach the equivalent of years of experience in a few days! However, experiential training like this, along with perhaps some added classroom instruction, can certainly accelerate a company towards usability research independence. Back in 1995, the web couldn’t be taught in a week either (well maybe it could, there wasn’t much yet out there!). That course would accelerate teachers – give them the tools that they needed to explore further on their own – and a process like this does the same. The client’s staff should now be encouraged to immerse themselves in the discipline, perhaps join a local chapter of the Usability Professionals Association and go to some events. They should now take every opportunity to learn usability.
An effort like this can also increase the salience of the user research process and the value of usability in general within the company, and provide a framework for future efforts. If I were a decision maker at the company, I’d ask for usability consultants to work in tandem with the in-house staff until such time that the staff can truly do things on their own.
I don’t know of any usability firm that advertises this kind of experiential training. I’m going to start! It’s certainly a new and rewarding experience for me and I think a valuable experience for those who are learning as well.
Image Courtesy of Anna Grigorjeva/Shutterstock
