APS2: Fancy Schmancy Usability Testing

John Vieth:  The title of this session is Exciting Usability Testing Technique.  It shows up in the schedule as fancy schmancy usability testing.  Real quick, a bit about me, my name is John Vieth.  I am a web monkey at University of Wisconsin, Platteville, just a member of the web design team there.  I am not a usability expert.  That's a disclaimer.  This is kind of a show and tell for me.  This is a fun little technique that I came across and I haven't found anyone using this technique, surprisingly, so I thought I'd share it.  I don’t want to put myself out there as a usability guru or anything.  So that means that I welcome participation especially at the end of the session if anyone has anything to share, any input on what I'm presenting, some checks and balances to kind of help us make this idea even better that would be greatly appreciated.

Some social media links there to help you find me if you'd like to learn more.  In general, this entire presentation is on the web and at the very end, you'll see a URL where you can access it.  It's actually a live web presentation.

That said, the goal of this presentation besides showing you this technique, this little show and tell, the goal is to get you excited about usability testing.  I'm hoping that this technique will look like something fun to do and just you get really excited.

But I mean really excited like how the best presenter of all time, Steve Ballmer, CEO of Microsoft, gets people excited about things like Microsoft and developers.

So what is usability testing?  Really quick, just a real quick one slide primer on usability testing.  According to Steve Krug's "Don't Make Me Think," the second edition which is a great book, which I purchased long ago before I even knew that Steve Krug was going to be presenting at this conference.  I'm a big fan.  According to Steve, usability testing is basically just the process of observing people using your product to make sure that it's easy to use.  If you just watch people use it, I mean there's no magic recipe, that's not really complicated, you just observe and make adjustments to your product or service based on those observations.

The idea is to perform small and simple tests.  Test often.  Don't be afraid to test whenever you have a question that needs to be answered.  You only need three or four test users, so this doesn't have to be a big grand production that requires a lot of planning.  I think people make a bigger deal out of it than it needs to be.  The important thing is to keep it simple.  Keep the user testing simple so that you're more inclined to do it often, and simple means cheap.  Whoa, there he is again.  That shirt is ruined.

The problems with usability testing: On the one hand, I think people tend to over-think usability testing and make it harder than it needs to be which makes it so that people don’t do it as often and don't benefit from it.  So we want to keep it simple but on the other hand, I think that bosses expect us, the more that they hear user testing is important, the more they expect to see something concrete result from it, especially if they're engaging in management by Website Magazine, they just saw an article on Website Magazine and it says how important user testing is.  They want to see user testing and they don’t want to just hear you say oh yeah, we do it all the time.  I mean they want to see something concrete to show them that you've been doing it to support decisions.

And if we do indeed keep our user testing simple like let's supposedly the goal is keep it really simple, then documentation might suffer.  If it's really simple, you're just making some observations with a real quick ad hoc test, you might also be inclined to not really document that user test case very thoroughly so that you remember the takeaways later on.  Whoa, there he is again.  That tongue is Photoshopped.  It's got to be, it's like a serpent tongue.  It can't be real.

So the solution is this exciting usability testing technique that I came across.  Basically, the idea is to record the usability test case users' screen activity, a video of them using the application and their audio at the same time simultaneously for archival and review purposes.  I'll show you a quick example video of what I'm talking about.  I'm not going to make you watch all of this.

Hi.  My name is John Vieth and this is a demonstration.  I'm going to fast-forward here.  Hey, that's an easy to use website.  So next in my task, task number two, repeat step one except this time, find the information using the A to Z resources feature.  So I'm going back here to the website starting at the homepage and see A to Z resources.  There's a link up at the top.  There's one down in the footer.  Let's go there and it's an alphabetical index of the resources on the website.  Well, I'm looking for tuition so that's probably under T, I'll click T.  There's tuition and there we go, all the tuition information I need.

So is everybody able to see how the video is superimposed over the screen recording with opacity?  I'm not sure how well that comes across at a distance, but the idea is that you can pretty well tell what I'm looking at while I'm doing my user test, usability test.  You can definitely follow the mouse pointer and see what I'm doing and you can even, when this is setup right, track the eye movement of the user and I think it's important to see that their eye movement is following what they're actually doing in the screen recording.  They get a more real basically evaluation of what's going on there.  That is only possible with some web cam settings or whatnot that basically mirror or flip horizontally the video image that I'm about to show you how to do that.

So real quick, before I go any further, is this something that everybody is familiar with?  Okay, just trying to make sure because it's so easy and I was just really surprised to find a lot of documentation of people doing this.  I think it's a valuable technique.  So that's an example.

So what are the benefits of this?  It's cheap.  It's actually free if you use a free trial of the software that I'm about to show you which is Camtasia Studio 7.  If you decide that you like the software, it's still only $299 which I feel is not going to break the bank.  But for those of you who are like me at least previously who are in an environment where regardless of the amount of money, just getting your department to pay for something is such a laborious process, it's great to have a free technique.

There are some alternatives.  In fact, what got me initially excited about this technique is a software called Silverback and it's actually not that expensive, but it's a Mac OS only software, which is really disappointing because it's a great software.  So I went to work looking for a Windows equivalent to that and I found Morae which is also produced by TechSmith, the company that makes Camtasia but it's $1,500 a license and I just don’t think that's going to happen for the typical higher education web shop.

Another benefit is it's really easy technique.  Once you see somebody do it, it's really easy.  A really great benefit is that it archives your observations of your user test cases for posterity.  So whether you want to show them to your boss, whether you want to refer to them later, I think it's just great to archive that.  You have all that rich experience archive without having to do a lot of manual documentation.

And frankly, it just impresses your boss.  You do some usability testing, you show your boss a few test cases and they really feel like you're doing something.  I don’t want to make it seem like our jobs are all smoking mirrors but sometimes the things that we do don’t impress our bosses as being time consuming or really effective or whatever.  They're really wondering a lot of times what are we doing.  When you can show them something with video and audio and screen recording, it really substantiates what we're spending our time on.

So what do you need to do this technique?  You need a computer and you should have this.  It can be a laptop or a desktop.  You need a webcam.  It can be an external webcam or a webcam built in to a laptop.  I find that laptops are easy just because it's an all-in-one thing, you can set it down, bring a user, sit them down in front of it and if the laptop has an integrated webcam, there's less gear floating around and I think it makes the user a little less apprehensive.

You need the Camtasia Studio 7 software.  Again, you can just download a free trial; it's fully functional.  If you decide you like it, you could buy the $299 license.  I talked about the differences between laptop and PC.  They both have pros and cons.

How does it actually work?  How do you do this?  I told you what you need.  How do you do it?  So you just follow these simple steps.  First, make sure you have what you need that I just mentioned.  Install the Camtasia Studio 7 software.  Obviously, it has to be on the machine.  It's a really simple install.  Double-click, setup.exe which is being supplanted by the install.php but as far as easy software installation, you can't get any easier than setup.exe.  You just setup.exe, okay, okay, okay and it's pretty much done.

A word about resolutions, high resolutions, these days everybody's displays are really awesome and we have these big 1920x1200 displays or multiple displays or whatever.  When you do a screen recording at that high resolution then you try to present it in some other format later one on the web or wherever, quality can suffer just because things are being shrunk down so much.  So you might want to adjust your screen resolution for the purpose of the testing.  Either use a display that has a native resolution of 1024x768 or change your resolution to a nonnative resolution, like right now my laptop is running at 1024x768.  Even though that's not the native resolution, it's just going to work better for this.  It works better for presenting and connecting to a projector, a lot of things so consider doing that.

Don't get too fancy, stick to one display.  A lot of us have multi-display systems.  For this, I would just keep it as simple as possible.  Again, I think that's why a laptop works best.  It can't get any simpler than that.  If it is a desktop PC, keep it real simple, one display, not a lot of gear dangling from the machine.

One really important part of setting this up is webcam settings.  By default, if you just install the software and you use the technique I'm about to show you when you get done and you look at the video that you've created, when I'm mousing up here, I'm going to be looking over there and vice versa just because the way that webcams work by default.  Luckily, every webcam has some sort of configuration, a little utility and usually very prominent in that utility is a setting to flip that.

Here's my control and in settings or under advanced settings, there's flip image horizontally.  I have that checked.  Normally it's not checked and you just have to look for a feature like that.  Every webcam is different but they all have it.  I found that they all tend to have that.

So run the Camtasia Studio 7 software.  So for the online presentation, you can follow these steps.  The URL will be at the end and you can go through it step by step.  For the rest of this presentation, I'd like to actually do a user test case really quickly and for that, if I could have a volunteer, somebody would like to come up and somebody raise their hand.  There's a young lady right in the front there.  Thank you.  Here, I'll try to get this setup for you.

So we're going to fire up the Camtasia Studio 7 software.

Before I sit the user down, I'm going to use the record.  Let me refer to my notes here.  I want to make sure I don’t overlook something because this is really important.  So we're going to go File > New project, of course and there's a very prominent button right near the top, "Record the screen" at the top left there.  Just click that and it fires up a little utility here.  Whenever I do this, my webcam utility tends to open up.  I'm just going to close that again.  I want full screen which is on the far left of that little dock.  You can record the activity for just one application but I find keeping it simple, recording the whole screen and then having the web browser maximize works the best.

Make sure that your webcam's toggled on.  Make sure that your audio is toggled on.  You might have to experiment a little bit initially.  A lot of systems have multiple audio input sources.  You just have to experiment and figure out which is the appropriate audio input.  And hit record.

So before I bring in the user, I'm going to open up.  Bear with me here.  It really is awesome.

Record.  Camtasia warns me that the recording is about to start and that I can hit F10 when I want to stop the recording.  I'll start up my web browser, new web browser window.  Sorry about this, this will just take a second.

This happens to be the website for the department that I just finished a major redesign this past year.  So for the test case then, you sit your user down, give them a test case.  The way I do it is just on paper; it's just easier that way.  They can set it beside the computer and keep it real simple.  Don’t test too many things.  I have just three short tasks here.  And encourage the user to read the questions before they actually perform the task.  It just makes the recording more interactive and also encourage the user to be verbose when they're doing their testing and just actually make audible comments about their experience and what they think.

Female:  Like on my first trial, when I was out doing my...

John Vieth:  For the purpose of this presentation, we're going to limit the testing to just one of the three steps.

Female: So I'm going to achieve what you saw earlier, and go through A-Z. And then solve for intuition...

John Vieth:  There you go.  That was easy to find.  What a great website.

After your user has left the room, don’t even bother with the mechanics of it all while they're there.  Once they leave you can hit the old F10 that will stop the recording and then we can trim out the junk at the beginning and the end later.

So here we are in Camtasia Studio and it's given me a preview basically of what I just recorded so I'm going to hit "Save" and "Edit" down at the bottom right.  There's a prominent button down at the bottom right there.  I'm just go with the default name for our purposes.  Save.  That's saving the project file and now we're in editing mode.

The very next thing that's always going to pop up is picture-in-picture and just go with the default setting and that is "Preview the picture-in-picture track on top of the video track" as opposed to a floating window.  We want this on top because that's the whole point of the technique.  I can check that, "Don’t show this dialogue again."  I didn't check it because I wanted it to pop up in this presentation because I know that that's what you folks are going to experience.

Another really important step, very next step here, it asks us what resolution do we want our project to be.  Always choose the resolution that matches your recording dimensions because this is our project file and we can always export it at a different resolution later.  I can't imagine a good reason why you would ever want to choose a resolution other than what you recorded it at so that's what I'm going to select.

So here is my video.  On the right-hand side is my video.  On the left-hand side are basically some properties.  There are a few different tabs there.  I can access various resources.  All the businesses on the right-hand side, that's basically a preview.

And then right over here is the important part.  This is the video that was recorded and this is an interactive object that you can move around, resize and whatnot, so we're going to focus on that right away.  Just click on that right at the beginning before you do anything else.  Click on it that will reveal some handles that you can grab and then you can expand that video so it basically takes up the entire recording area.

The very next thing you should do is go over here to your picture-in-picture properties and knock that opacity for that video down to 30% and all these details are in the presentation so you don't have to take too many notes.  That's about it for that.

So let's do a quick little preview.  I hit play.  I'm going to jump ahead a little bit, a lot.

Well, I think I had the recording input.  I didn't have it set to the right mic but you get the idea.  I also overlooked another really important part of this, and that is I'm going to back up to that step in just a minute here.  We're going to assume that the user's face is kind of centered in the screen.  Right now you just see some awesome forehead there which is nice but ideally, you want the whole face in there and I'll redo that step really quickly when we're done here.

This is the sort of thing you just want to do one or two times before you actually bring in your real users, and then once you have it setup, presumably, you're going to have your system there.  You're not going to make a lot of changes, you're not going to bring the laptop right at the last second and set it up.  So it actually works pretty well.  If it is a laptop, you can even instruct the user, please don’t adjust, which that might be an argument for using an external webcam, is that they're not bothered by adjusting the laptop display.  A lot of people like it facing them at a certain angle.

So I'll jump to the part where I mentioned trimming off the junk at the beginning and the end.  You just kind of scrub your recording until you get to the part where the user is sitting down.  There's the forehead.  Right there is where I want to cut it.  There's a little "split" button right here.  It couldn't be easier.  Once you've scrubbed to the proper location, you hit that "split" button and that just splits the video into two chunks and then you can select the video that you don’t want, delete, and bring everything over to the far left to the timeline.

And then just repeat the process for the garbage at the end.  Here comes the giant man in the blue jacket.  Hit that "split" button again, delete the stuff on the end, and that's it.  There really isn't a good reason to do a lot of other processing of this because generally, you're not going to add some after effects or any kind of fancy stuff.  It's just to record and experience.

So we just go to the File menu, save project.  Give it a name.  I'm going to call it HighWeb10 and our project is saved.

So it's really important to save your project files.  You're going to do some things with these.  Probably you're going to export them to video for the web or for whatever purpose but always save your original project files because that's your original.  It's kind of like your negative from back in the print photography days.  You can always go back to that original and even change the opacity of the video if you want and all sorts of stuff so keep that original.  That's it.

I'm just going to kind of restart the process just to show you that important part.  I was rushing a little bit.  When you go to record, right before you start the recording process, they give you a nice little preview down here that shows basically when you record with the webcam what it's going to capture and it's live even though you're not recording yet.  It's a live preview.  So as I adjust the laptop display here, it's changing where my face appears and you might want to bring the user in at this point if you just want to really be sure that's captured right.

So at this point, my entire face would be captured in the recording.  So I'll close that.  That's really all there is to it.

A few more comments here.  Did that, did that, did that.

One little suggestion.  Consider using Camtasia's "Callouts" feature and I'm not going to take time to demonstrate that feature now.  A lot of you have used Camtasia and Captivate so you're familiar with the features that I'm talking about.  Even YouTube has similar features now.  I think they call the features annotations.  Consider using those.  You might open up your original project file.  If there's something that you want to show your boss or show a committee in a presentation to support a decision that you've made, you can add these callouts so that as the user is using the application when they say something that's critical or they do something that's critical to support the decision that you're making, you can add a callout at that point to bring it to the attention of others with a note about that.  It's just basically a little popup that's part of the Camtasia project.

Again, keep your originals, keep your project files.

After you've saved your project, you can use the "Produce and share" feature and we won't go into details of that, but that just gives you a whole plethora of options for creating alternative formats for your project.  You can save it in MPEG-4 format, AVI, Flash video, basically anything you would ever, I mean worse case scenario you can save it as uncompressed AVI and bring it into some other application and do some further processing, editing, etc.  So basically all the options that you need are there.

Personally, I always use the web preset when I'm using the produce and share feature.  I jump right to the web preset and I use the MPEG-4 setting.  It just makes for a nice compressed video that's easier to put out on our network file system without taking up a huge amount of space.

When you do use that feature at the same time, I want the MPEG-4 file but for my convenience, it also creates an HTML file and a Flash file and the necessary Flash files, kind of self contained folders so that I can put that folder anywhere.  Double-click on an HTML file and, bam, I've got a Flash movie basically that I can put on any website.

Done and done.  That's all there is to it.

So are you excited?  Are you excited about user testing?  Isn't that fun?  It's something that you can see and it's video and audio.  Steve Ballmer is really excited about this technique now because he's so easily excited about a lot of things.  That is available on YouTube for your enjoyment.  Steve Ballmer has a ginormous ego so he appreciates your YouTube likes and comments, so go ahead and login.  Not now.

I can take some questions right now if anybody has some questions and then, again, just a reminder that this presentation is available just as you see it here online at that URL.  I believe the URL will be included in the notes for the conference.

Audience:  I have a question.

John Vieth:  Yes.

Audience:  So on the screen resolution, the only thought that I had when you were selecting the screen resolution to use is how important it is to ask or find the screen resolution that the user uses on their actual computer.  It has made all the difference in the world for the design that we're currently working on at University of Texas at Austin.  Everything looks awesome on our screens which are about this big and then we go out to the user field and they can't see key elements of the screen and didn't even know it scrolled.  So that was just one thing I wanted to add to what you were saying.

John Vieth:  Well, I'll give you a little bit of insight in the way that I design and test.  And that is that currently, I design everything so that it works at 1024x768 and then if they have a larger display it just looks even better.  So because of that, I tend to test at 1024x768 but if you have really good reasons for testing at some other resolution, it just depends on your situation.

Audience:  For people that aren't as wise as you, that's really just important to note.

John Vieth:  Yes, those numbers change from time to time.  We're still in 1024x768 mode these days.  It wasn't too long ago that we were in 800x600 mode and we were there for a few years and I think for quite a while, we're going to be in 1024x768 mode.  Yes, sir?

Audience:  A quick question about a little bit of a novice on Camtasia.  You have two video feeds essentially.  Are you able to preview those?  I noticed because you had that height adjustment.  Are you able to get a preview of each side before doing that or you're kind of on your own as far as doing a test first and a playback?

John Vieth:  Yes.  I won't open it up and play with it now but you can imagine if you've done any video editing, there's multiple video tracks and you can even add more video tracks and more audio tracks.  I don't know why you'd want to for user testing but you could, maybe if you're building a presentation or something.  So you can use the little visibility toggle so you can toggle and turn off one of those layers and then just focus on the other one.  At that point, you could export it with just one of those video layers showing if for some reason you wanted to isolate that.  It's pretty flexible that way.

Audience:  All right, thank you.

John Vieth:  Sure.  We have another question.

Audience:  Does your online presentation if you go to that site, does it include a list of the questions that you ask to your constituents, like the things that you are looking for?

John Vieth:  Yes, and it's really just the example test that I brought with me today which isn't the best example.  I guess it's a good example just in that it's short, limited to three or four questions.  I think I've heard that said.  I think I've read that, three or four users in your testing but also just a handful of questions.  So it's an example of what you can do.  There are no rules but that's what I do.

Honestly, even with this technique, it's hard for me.  I haven't done all the usability testing I'd like to do so I'm not the most experienced but because of this technique, I hope to do more and more and hopefully, you guys will find it useful.

Any other questions?  That means I explained everything.  Here's one.

Audience:  I just remember being referenced to Morae and I noticed you tagged that there.  Can you just elaborate more on the differences and maybe some similarities and just generally what you can say about that?

John Vieth:  I'll say a few things.  I actually haven't used Morae just because it's so expensive.  I feel like, what's the point?  I think there's a free trial of that software as well.  Actually, I'm not even sure there is a free trial.  It's just not an option because of the cost, for me, and I had already stumbled across this technique because I got excited by Silverback.  So I had my technique and when I saw that Morae was $1,500, I'm like, why bother, this will get it done.

But I can tell you based on Silverback we can make some assumptions that when a product is developed especially for this purpose, you're going to get some handy features for things like remote control to make the software interact really nicely with a remote mouse or remote control of some sort.  Just basically, the interface is optimized for testing is all it is; whereas, Camtasia Studio is not intended for testing, it's intended for developing presentations and other content.  So I guess that's my thoughts on it.

It's probably going to be a nice software.  It's going to have all the features that Camtasia Studio has because it's by the same company.  I want to assume that but I would probably wager that it has all of the quality, all of the features.  If you can afford it, it's probably going to be worth it just because it's going to have those extra bells and whistles, but I like to save some money, so I probably won't ever be using that.

Any other questions?  I did explain everything really well.  Well, I hope you guys enjoyed the presentation.