User Testing

I’ve neglected to post over the past few weeks, but that doesn’t mean I haven’t been keeping busy. I’ve done several informal user tests, and rather than posting my results here, I wanted to give them directly to the dev community and make sure they were searchable from within our development site. You can view them in the e-mail archive here:
User 1
User 2
Video clip of user 2 creating a user, and discussion of the “All” project
Video clip of user 2 creating roles
User 3

For my own future reference, I want to note my technical problems.

1.  Problems with my computer setup:

  • My keys are rearranged into the Dvorak layout.  I need an external keyboard so that if users want to look at the keys, they can.  (I have one, but it didn’t make it to Seattle because of my minimalistic packing.)  This also ends up being a small handicap for me if I need to type during the test.
  • External mouse would be nice, too.
  • It’s a Mac, and sometimes people are used to Windows.
  • Make sure to turn off e-mail notifier, instant messenger, any other potential interruptions.  (I did most of this…)

2. Problems with software:

  • iShowU, my screen recording software, apparently crashes when you get up toward an hour of recording.  Thus, the video that I would have had for user 3 is lost.  I should take more detailed notes, though I think I did an adequate job, and if I’m going to have long tests, take a break in the middle!  It’d be good for the users too.

3. Test structure

  • There are advantages and disadvantages to testing your friends.  I did not structure the tests very much and left them very informal, and thus I have much background noise and informal dialog to sift through when parsing the video for user 2.
  • I loosely structured some tasks on the fly, but for the most part I let my users wander and they may have benefitted from a greater sense of purpose.
  • There is also the risk that they’re saying “That interface ROCKS” because they know I designed it and they’re being nice.

Despite these obstacles, the tests were quite successful in gleaning a lot of useful reactions to DrProject current and potential interfaces.

Approve users

Before I got around to user testing, I had been continuing to create redesigns for the Approve users interface.  My current favorite is this:

Approve users mockup

The tooltip would show when you hover over “More.”

This interface does several things I like:

  • Combines existing user requests with current user requests, but has a visual distinction between the two (note the “New user” text under some users’ names)
  • Hides “extra” user information.  For some existing users, the admin will know who they are.  If they have any doubts, they can hover over More to learn more about the user.
  • Powerful handling of all requests at once.  You can approve all, deny all, or set all roles to a certain setting.  In a classroom setting, all of these features would benefit a professor trying to take care of a bunch of requests quickly (the role setting option will only provide a significant benefit if they set all of their students to the same role).

When I did the user testing, I had one user who couldn’t explain why, but he preferred the current Approve Users interface over this and one of my previous designs.  His only explanation was that the verb came first (Postpone/Approve/Deny) and that made the most sense to him.  I feel that the checkbox interface is more intuitive and requires less clicks than dropdowns.  If only I could have a fully functional implementation of each of the interfaces in question to do some real user testing.  I am thinking about trying out CogTool, which seems to take prototyping one step farther to allow things like checkboxes and dropdown menus added on to mockups of screens.  If I do that, I will need to find a new round of users for testing to get some better results.  Right now, I don’t think the Approve Users redesign merits that kind of additional time input because there are so many other parts of DrProject I’m also working to improve.

User administration

Qiyu has done a great job implementing the create users screen and I think beyond a few more tweaks and some testing, it is completed.  It’s a great improvement from before, and the changes should hopefully carry over to the Register Users screen fairly easily.  He’s also started to tackle the List users screen and has been working with Greg to nail down the requirements.

Challenges beyond the Admin Panel

Several of my concerns about DrProject usability were solidified by seeing new users wading through DrProject for the first time. One of these concerns is that there is little indication of where you are within the structure of the site. I feel that something like breadcrumbs will be the solution here. I have noticed some other project management sites use this as part of their solution to a cumbersome navigational structure. I also hope that we could make the navigation more intuitive. Users often asked, “Does this mean for the project, or for the whole site?” Finally, the biggest issue left on my plate for this summer is what to do with the “All” project. I’ve been stewing on this and paper prototyping a bit. Paula, my usability mentor, has offered to meet with me next week and I’m hoping that the “All” project will be one of the issues we have time to work on together. It will be nice to have another HCI person to work with for a change, and one I can learn a lot from, I am sure!