How we’re conducting user research while working remotely

Spare a thought for Jennifer. During a Zoom video chat with work colleagues, she’s seen wandering into her bathroom, laptop in hand, to answer nature’s call. Realising her boo-boo, she hastily turns her laptop away and exits the call, leaving her 10 co-workers to have a good chuckle. “I saw nothing”, says one of them. “Poor Jennifer”, laments another.

At Nomensa, one of the things we love about remote user research is that it allows us to get insights from people in their own everyday environment anywhere in the world. And we’re still gathering these insights despite having to move from our offices to our lounges and kitchens and, in some cases, our bathrooms. In fact, COVID-19 is having very little impact on our research capability.

We’re all remote now

We have many years of experience supplementing our lab work with remote research, including user interviews, tree testing, card sorting and moderated and unmoderated usability testing. It allows us to connect with people in remote locations, different time zones, and with accessibility needs. It also allows us to validate qualitative research with quantitive data. And now that we’re pretty much all remote by default, this experience is proving invaluable right now.

Our user experience (UX) teams work with an array of remote research tools such as Lookback, UserZoom, TreeJack, Hotjar, dScout and others to interview people and test digital products and services from the US to Malaysia. We offer both moderated and unmoderated usability testing (or a combination of the two) to ensure we obtain the insights we need.

With remote moderated testing, the researcher is live online with the participant guiding them through a series of tasks via a shared screen. Moderated tests are useful for evaluating an existing product, testing early prototypes and observing participants working through complex tasks. It’s great for capturing those unmeasurable insights such as pauses and in-the-moment thoughts.

It’s a little trickier than being face-to-face where you can offer participants a cup of tea and make a connection. “You need to find a way to replicate that tea-making process to build a rapport,” says Joe Knowles, principal UX consultant.

Moderated remote testing is vital for reaching people who may not be able to attend a face-to-face session due to location or perhaps their ability to leave the house. This makes this type of research great for removing regional bias and for speaking to more vulnerable users or those with particular accessibility requirements.

Typically we use unmoderated testing either to obtain rapid, low cost insights to guide design work or to back-up findings from our moderated testing with quantitative data. With an unmoderated test, participants complete a series of tasks on their own, in their own time. The remote and automated nature of unmoderated testing means we can quickly get a large sample size and reach a wide variety of participants.

Working across time zones

But whatever type of user research you’re doing, it’s important to be flexible and to adapt to your participant’s individual set up. For a series of website redesign projects for Cambridge Assessment, we interviewed several dozen teachers and students across seven countries, including the US, India, Pakistan, China and Malaysia.

“We ended up working early in the morning and early in the evening with not much in between,” says Joe. “It was hugely satisfying speaking to people all over the world and getting a sense of how they approach things.”

The value of cultural awareness

Working across borders requires cultural awareness and we always make sure we do the groundwork before interviewing people. But even when people speak the same language and share similar values, there are subtleties to consider.

Dropbox had asked us to redesign the help and support section of their website for English speaking users. The issue here was what to call things. “In the US, a beginner’s guide is often called a 101,” says Brendan Lancaster, senior UX consultant.

So “what do we call stuff?” was an important consideration during their research which involved several dozen usability interviews across the UK and US and surveys involving more than 3,000 people. “The big numbers we got in the surveys gave us confidence in our findings,” says Brendan.

More representative results

Remote research allows you to get a more balanced range of responses. When researching a new intranet for NHS England and NHS Improvement, we had to find a way of capturing a wide range of views that was representative of more than 8,000 staff spread across 50 locations. We used a combination of group video calls and online engagement sessions. For the engagement sessions, we used Mentimeter which allowed staff to take part in a series of live polls and Q&As.

Josh Avern, UX consultant, says the online engagement sessions captured the views of a wider group that perhaps didn’t have the time or desire to take part in one-to-one interviews. “I think it helped to make more people feel involved in the process,” says Josh.

Accessibility research

Being able to interview remotely can be especially helpful when conducting research with people with accessibility needs and long-term health conditions. The Girlguiding charity had asked us to find out more about the experiences of this group of volunteers, their needs and the issues they face in their roles.

We carried out a series of individual video interviews with volunteers with a variety of physical, cognitive and mental health conditions. We used services such as Textrelay, a telephone service for people with hearing impairments that translates live speech into text.

“The interviewees appreciated being able to participate from the comfort of their home,” says Gabi Mitchem-Evans, UX consultant. “It also meant we could gain insight from a wider range of volunteers who otherwise may not have been able to give feedback in person.”

Analysis is key

There are many tools out there to help you conduct remote user research. But collecting data is the easy part, what you do with it is where Nomensa’s brilliant team of UXers earn their salt, says Angle Ingle, Head of UX. He says: “Research alone provides data, not insights. The key to making any piece of research a success is how you use that data to generate actionable, prioritised recommendations.”