Last week I went to the Etre conference at ICA London where Steve Krug gave a lecture about best practices in usability testing. The entire day was incredibly interesting and even surprising at times. Let me walk you briefly through what was discussed and how the Etre event is supported.
Entering the door of ICA I realised all the seats were taken with a great variety of people from different fields of the digital industry: marketing managers, designers, developers, UX people and even beginners. To cover everything would probably take a much longer article so I’ll only highlight parts which I personally found interesting and relevant for the audience I’m talking to.
Share different stories
The talk was designed as a one long presentation where we were allowed to stop and ask questions. This gave us an opportunity to share different stories from different fields but it also brought more to the presentation itself. Steve is a great speaker, which makes a big difference when you’re learning about the process of others.
Here Steve talked about how you must start by presenting the website you want to test as soon as you have something to show (this actually refers TDT – test driven development). The main focus was on execution, which is really important for the observer to grasp the granularity of the task and level of the engagement.
Also communication is one of the key elements where the leader should always facilitate an open conversation on the topic -to make the user focus on the required task.
An interesting note was the idea that developers like to complicate things – and it’s very true. They like all these buttons and menus and cross-linking. The entire complexity of the solution represents they skill-set to which they are truly proud of. The best developers come up with the solution where the user doesn’t have to think about and application, the developer just makes it for them (sign up with Open ID as a example) and it should simplify their life.
Then we moved to technical equipment where Steve described what works well and doesn’t cost that much. Also he introduced a couple of Internet services which we can use on our quick projects. We covered in-house testing as well as remote testing, hiring participants and finding the right target audience for your project.
There was actually one challenge which I personally came up in my practice and it was hard to keep the participants in focus where we did the testing remotely. It was really hard to make the observer / participant focus on the task as well as manage all distraction on leader or participant side. We gathered enough practical examples of how to succeed.
One, Two and Three
The following section was about applying the user testing internally: set up the date when your company has decided on a testing date and all the projects you’re working on go through the testing sessions One, Two and Three which follows “Brief over the lunch” where you as a leader facilitate conversation about what happens in the morning. What are the problems and how do you categorize them for another iteration or sprints? What is the challenge with these tasks or bugz and how to manage them and finally which one to fix first? These are the things you need to cover.
“is not good enough to do the test”
After the questionnaire we talked about when to start and why. The biggest challenge for any team I see from my perspective is timing – “is not good enough to do the test” we hear all the time. You can test when ever what ever you have a more you test more you get, and even more you might find that some part of your solution don’t work properly and you can fix it before it’s late.
Test what ever is available to you. Test old versions of the website and you’ll understand what the main issue was on the website and why the client wanted to fix them. Test competitors website – to understand what they do wrong and what you should avoid in your version / build.
Test with a two groups – we discussed that always new participants are better because they don’t have a background understanding of the application. On the other hand you have someone who works in the company and can see the benefits straightway.
After each session with these three participants, I mentioned at the beginning of the article, write down the 3 keys things you can fix. You’ll end up with nine key issues of your build / apps and what you need to fix. The problem we all deal with is money or will the client to do these amendments before launch.
Least you can do
Tweaks – don’t forget if there is time and anticipate the client agreeing or disagreeing with the tweaks, which can be part of your service / delivery. In that case you have a room to go and test / change / tweak parts you were aware of.
Practical example testing
And finally we get the real testing. We crate a test groups where each of us has a site we would like to test. I chose my personal website. Please see video below.
The task was
You are an existing customer of three.co.uk. You own blackberry almost 18 months contract and which almost finish and you decide to switch to an iPhone. As a happy customer you don’t really often go to provider website to check the services so you act almost as a new customer.
You task is a find an iPhone and compare with blackberry and make a decision to move. The video from the session is placed bellow.