What We Learned About Usability Lab and Guerilla Testing
Tricks to decide if you need a User Lab or Guerrilla Test
UX Researchers’ community is living its own Barcelona vs. Real Madrid when you ask them: “What’s your team? User Lab or Guerrilla Test? Like football conversations, these methods generate controversy and many hours of conversation. Definitely, when you need to choose a test method, you will find some rivalries.
Like Barcelona and Real Madrid, both methods are very good. At Opentrends we know that because even building a User Lab, we still do Guerrilla Testing with customers.
If you don’t know which method choose for your next test, we show you some tricks to take the right decision in your next research challenges.
Martin Belam describes guerilla testing as the art of spotting lone individuals in cafés and public spaces, asking them about your product prototype, and documenting their reaction and user experience.
Simply put, guerilla testing is the act of going to a public place, approaching people who might be willing to test your product, and generating user feedback from them. It’s relatively low-cost and simple.
Keep in mind, however, that guerilla testing isn’t as easy as going to the nearest public place and conducting your test where people are willing. You have to be strategic about your choice of location. Make sure that the place you’re holding the test is a place your target audience frequently visits.
For example, if you want to test a new mobile app designed for retail chains, your best option is conducting the guerilla test in a series of retail outlets. You may opt for a test in just one location. But the more locations you test, the more insights you get and the more feedback you can work with.
A Test Lab is a room where you evaluate in detail every single user action with specific tasks and exercises. The Lab has an adjacent room called Observer Room to analyze through a mirror (and in real time) these actions. Like in Police Stations, users are suspicious people to interrogate. Test Labs usually have technology and software to measure accions and find UX/UI mistakes in any product or digital service. At Opentrends we’ve experimented with eye tracking tests and sensors to understand crazy things: user emotions. Thus, we discover those processes where users aren’t aware of what are they doing.
Usability Test Labs are an heritage from marketing research. During years, marketers have performanced focus groups in rooms following observation techniques and using artifacts as the interrogation room we said before.
The lab, as a controlled environment for performing user testing is more artificial and the insights we get are less spontaneous than Guerrilla Test, where you can’t take up more than 30 minutes of their lives. But in a controlled environment like a Lab, you can do other Research activities and Design Thinking exercices.
Context is essential in User Testing. That’s why we put it at the top of the list only in case you don’t want to read the full article. If you choose a User Lab to do user testing, people will probably go there with a clear goal in their minds: “don’t be dumb”. They maybe will dress different as they do, will talk different as they usually do and also will behave different as they do. They will preconfigure themselves such in a way that their actions or answers won’t be focused on the test itself.
Usability lab testing might not drive the organic results you’re looking for because the test subjects are in a controlled environment. They know they’re being observed and that their reactions are being recorded.
Guerilla testing, on the other hand, delivers more candid results. The user testing is conducted in a real environment, so you can be more confident that people are making genuine reactions and comments about your product.
Make the results even more insightful by conducting guerilla tests in places that the target audience is normally found. For example, if you’re testing a flight-related app, approach people in the airport line. Or if you’re working on a food delivery app, conduct a test around office lunch hours or typical dinner time.
Some UX experts might suggest that guerilla testing works with any product, including vague concepts that you have just scribbled on a piece of paper. As a result, some researchers merely ask participants to give their feedback and print-out designs and concepts. This isn’t the right way to do it, though.
The purpose of guerilla testing is finding out how people react and respond to an actual product, even if it’s just a prototype. You won’t be able to gather deep insights by asking participants to flip through papers.
It doesn’t matter whether you already have a finished product for guerilla testing. As long as you have a prototype tester — preferable one that resembles the actual product — the more valuable feedback you’ll receive.
Election seems to be easy but there is a key factor before taking the final decision: the “fantastic” legal and provacy terms. Getting permissions and acceptances for Guerrilla Usability Tests is not always easy.
If you can’t get permissions for your Guerrilla Text on the ground, you should go to a Lab and customize it to set the most realistic context.
This is not a perfect scenario, but if you want to create a simulated context, focus on disturbing things like sounds (available online) to simulate people talking, planes taking off or ringing mobiles. Also “cheat” on users by creating an the way they’d feel: pressure them by setting up time specific tasks; invite them to do combined tasks to prevent focusing in one thing only… By reproducing environments, you will have the chance to find new needs. Pero esto requerirá un extra en la preparación del test.
Both decisions could work but always have one thing in common that may bother users and get them defensive: the signature of permission for taking pictures or/and record voice and screen. This situation could be uncomfortable for some participants. Instead of doing that, try to take notes and leave users act on their own.
If context is important, test preparation is one of the success (or failure) keys. Guess having an amazing context but fail during the preparation. Avoid big fails like this. Both tests, Guerilla or User Lab could be as difficult to prepare as reliable on its results.
For example, if you go out finding users you don't need to set up rooms, devices, software tools and other settings. But you’ll need to be a hunter and catch as many users as you need on the go. Otherwise, your Guerrilla Test will fail.
In a Test Lab, everything should be scheduled and ready for users and testers. Maybe it is artificial but it’s amazing if you don’t like improvisation. If everything is well prepared, you won’t lose time between tasks. User Test Labs are taken from Marketing research and offer safety and controlled experiments with some people observing and recording movements, facial expressions or whatever user looks at. This is something we lived during last Mobile World Congress in Barcelona, where we “eyetracked” everything.
All these gadgets and software can provide amazing insights. Of course, it doesn’t mean Guerrilla Tests don’t give us serious results. But there are some voices in Researchers’ community still believing that aren’t scientific enough. Guerrilla Test detractors think you are invading users spaces, privacy and conformability. They think labs are neutral places and users will find more issues because they will take tasks as a job and finally get a gift card or compensation.
Once compared this two testing methods, maybe you can decide better in your next user test. Both of them have their limitations and strengths. So, which is your team? Street Testers or Laboratory Rats?