The researcher as a facilitator of learning: Awkward Silences podcast

I was lucky enough to be a guest on the UX research podcast Awkward Silences in 2020.

Their write-up is here, and you can listen to it here.

Transcript

Erin: Hello everyone. Welcome back to Awkward Silences. Today we are here with Nicola Rushton and she is joining us from, where are you today? Madrid?

Nicola: I'm in Madrid.

Erin: And JH is here as well.

JH: Hey, how's it going?

Erin: Awesome. So today we're going to talk about the researcher in an organization as a facilitator of a team's learning. So we've talked a little bit about how researchers and research apps can fit in organizations in all sorts of different ways, but today we're really going to zero in on this idea of researcher as facilitator, which is a topic that Nicola is very passionate about. So thanks for joining us.

Nicola: Really excited to be here. Thanks for having me.

Erin: Awesome. So researcher as a facilitator, why is this a good idea?

Nicola: Yeah, this is something that I've been thinking about a lot lately throughout my own practice as a designer, and a researcher, somebody who does research. And I guess what I've been coming to lately is questioning what is the actual purpose of a researcher in a software team? I think that the point, the reason why we have a researcher there at all is so that everybody else knows about users, right? Like to allow everybody else around us to have the information that they need to make the decisions that they make on a day to day basis with all the information about users that they need. And so I guess what I've been thinking about is kind of looking at how can I as a researcher facilitate the learning of the people around me, about users. Because if my goal is to have them have the information they need, then it's almost like a teaching role. And so this is kind of what I'm getting at with the facilitator of learning.

Erin: And so how might look different practically than say... or does it look different than the traditional kind of a researcher? Maybe they get an ask and they're a central resource and they do a bunch of research and they put together a readout or a report. Does the researcher, as a facilitator, to make sure that the company at large knows the voice of the user, knows the concerns and pains of the user. Does that look different than that kind of traditional process?

Nicola: Yeah. Say What I've been experimenting with lately is running research in a really, really collaborative way. So actually looking at who are the people in the team who need this information the most about users? Like if I'm a researcher and not a designer and our researcher... So in previous roles I've been a designer and researcher, so I was one of the people who most needed that information. And then moving into a role where you are the researcher only, it's the other people around you. Like the designers the engineers, the PM's, the clients, the founders, whoever it is. They're the people who need the information.

Nicola: So looking at how can this process be as collaborative as possible so that these people around are participating in the creation of the learnings through the research. So I think a lot about this phrase that comes up in the world of teaching, which is, "Tell me and I forget, involve me and I understand." And I think if we think about times that we've learned things in a teaching environment, in a classroom scenario, I think it's often the moment that the teacher really gets you to come to the answer yourself. Those are the moments where it really sticks. So I'm sort of looking at those moments and wondering how can I bring that into my research practice, the people around me, to get them to really get it.

JH: What I really like about this premise is, it feels like the clarity of purpose of the researcher in the way you stated it is very simple, right? Make other people on the team know about our users. And then when you do that, it opens up a lot of opportunities for them to experiment and try new ways of achieving that goal. Whereas I think if you get down a level into the weeds and researchers are thinking about facilitating, it's like, "How do I facilitate the sessions or how do I create a great report", which are all important things. But when you elevate to that higher level of, "We just need to make the team understand users, how do we achieve that?" It feels like there's a lot more room for creativity and engagement. Kind of like what you were just describing them with, like the teaching model. Does that resonate with you?

Nicola: Yeah, definitely. Definitely. I love seeing people getting super creative with stuff and I think it also... I mean if that becomes the end goal, I think it allows you a little bit of leeway and freedom as well. Yeah, so I've been having this conversation with a few different people in a few different formats, about what does it look like when you're pulling engineers and other people in into research? If the goal is really to have those people learn, then I guess you're just looking at it from that angle instead.

Erin: Let's talk about who are those people. Right? Because I think when we talk about collaboration, which like many ideas in business, right? There's a continuum where taken to the extreme maybe every idea is not good or bad and so on. So collaboration, it's a good thing and it's a learning through doing is probably the best way to learn. Who needs to be involved in this learning, when? Do you have thoughts on that and what does it depend on?

Nicola: Yeah, I think it's different for every project. Product teams make this really easy. So I've been lucky enough that a lot of the work I've done has been in a product team context. But I love bringing in engineers. I think engineers are my number one kind of role group who I'm like, "Yes, please bring them into research more. They're so important." But yeah, I've also, depending on the project that I've been working on, I've been pulling in all sorts of different people. So I did some work with content people because they were really important and they were going to be creating a lot of the thing that was being influenced by the research we were doing. I've worked with a lot of startup founders, clients, did some work with legal people once. I think it's really just about who needs to learn this stuff about users the most right now?

Erin: And so are those those folks, the legal folks, the content folks, the product folks, are they asking for the research? Are they the clear stakeholders of the research in those instances, or are there other cases where it's inverted and the research is happening through some other compelling event that creates it and then there's an effort to decide who should be involved and know about this?

Nicola: Yeah, I think it's different every time. Sometimes it's really obvious. Sometimes it's like, "This client, they're asking for the research, they're really engaged, bring them in." But then, yeah, definitely other times, like when I worked with the legal people, when I worked with the content people, it was absolutely a sort of, a push from my end. But I'll actually sort of frame it as I really need some help with this. I need other faces in the room, and you would be a perfect person to do it. And kind of pulling people in that way.

Erin: Yeah. Asking people for help is the best way to get them involved and ingratiate yourself to to them in general. I love that.

Nicola: Yeah.

JH: When you say "pull them in", what does that mean? They're taking notes, they're in the sessions, you're giving them the readouts right after, what activities are they engaging in?

Nicola: Yeah, so I have a couple of clear activities that happen almost every time I do research as a group, with a group of different people. So I think really key is setting learning goals together. That's a really nice place to, I guess kick off a piece of research and get everybody in the head space. To me, learning goals are really clear, sort of answerable questions that we can come back to at the end of a piece of research and ask ourselves, "Have we learned this thing?" So really setting the intention for a piece of research.

Erin: Yeah, and I was just going to say, I think that framing is actually really important, because a research goal almost becomes tautological. Like why are we, you're not really asking the fundamental question, which is what do we hope to learn from the research? You know, the goal of the research could be to do the research, to check a box. Whereas framing it around learning implies that, "Hey, we're actually doing this research to learn something." So I think the words, that words really matter there.

Nicola: Yeah, so I would very frequently try to focus my research around risk, specifically risky assumptions very often. So I really love to run a workshop with this team, this group of people who are going to be involved in the research, whoever they may be in that particular project. And have everybody really investigate what assumptions are driving this piece of design development, piece of work, this idea at this at this time. So really looking at what are those assumptions? Often what that workshop would actually look like is having people brainstorm on the phrase, "If we're wrong that blank, this will fail." So blank being the assumption there, I like phrasing things around risk because I think it, oh you know, even bringing a little bit of fear into the hearts of people, because I think it really sort of just drives home like, "Oh we really should actually do some research on this, because if we're making assumptions here that we're not 100% sure we're right about, we kind of want to figure that out soon."

Erin: Well, and that question is such a great template for getting at what the risk is. I think if you ask a group of stake holders, "What are our assumptions?" The first reaction might be like, "None. We don't have any assumptions." But if you say, "Okay, well if we're wrong, what will fail?" That can get you to that in an indirect way, which is not so unlike you might ask a question in a usability interview to try to get to the answer you're trying to get, right? Kind of have to dance around it a little bit to get there.

JH: The other benefit of these templates, and I think in the design sprint book from the Google Ventures people, they do a good job of kind of justifying this is, it's like a helpful constraint, like it feels a little hokey sometimes. I think they do the, "How might we" when they're doing brainstorm exercises, but it kind of narrows everyone in and it makes the ideas or assumptions easier to compare because they're all kind of coming in, in the same format with the same level of focus. And I imagine this, that might be the case with this format as well. Has that been true?

Nicola: Yeah, absolutely. I love that way of phrasing it. So the next thing that I would do from here is I guess get people to order all of these assumptions by which ones are we most sure about, and least sure about? And which ones would be the riskiest if we went right about it." And then really it's a matter of looking at, "Okay, which are the ones that we're less sure about and the would be more consequential if we were wrong about?", and this becomes a focus of our research today. So really looking at their risk there and... Yeah, so this is kind of the one piece that I would always, always try to include as many different people from the team as possible. Well, not in numbers, but really try to include those people. And I think what that does is number one, it's really good for buy in.

Nicola: So they're really clear on what we're doing research on, and why. Number two, all of their voices have been heard, so it's really likely that you are going to be fixing your research on something that is somebody else's fear, worry, thing that they've been thinking about. And number three, I think it kind of creates a bit of a sense of... like a cliffhanger. So you've just spent a bunch of time kind of asking some tough questions, coming up with some big, big questions, big things we want to learn. And then now they're just waiting. So it's really good for that buy in as well.

JH: What I really love about this framing as well is it feels high leverage. I've yet to meet a team that feels like they're overstaffed on the user researcher front, right? Like we have way too many researchers. It tends to be, "I'm the only researcher for eight squads and I don't have enough time." And when you think about other ways of involving people, like having them facilitate a session, it feels lower leverage. Like you have to train them, you have to make sure that they do a good job and the session is still worthwhile. Whereas this is an exercise the researcher should be doing anyways. It doesn't take a lot of extra time to involve the whole team, as you just kind of described, and now they're bought in and now you have all this other input.

JH: So it feels, in addition to being like a great way to align everyone, it feels like a really high leverage use of a researcher's time. Whereas maybe other modes of involvement have kind of a lower return and more overhead. So just from a practical standpoint, it also seems super useful since there's usually not enough researchers to go around in my experience.

Nicola: Yeah, absolutely. And the other thing is if you're looking at trying to include people around you in research, of course, everybody else has jobs to do. Yeah? So if you're going to start sort of saying to engineering managers, "Hey, I want to pull some engineers into research this... for a night this week." It's really gonna be like, "Okay, well I can only give you a few hours of somebody's time." So yeah, I've put a lot of thought into how can you use these other people's time as efficiently as possible.

Erin: We talked about getting everyone involved in this kind of what are our assumptions exercise. Are you doing that for discovery research for usability research? Is that... Are you doing that at every stage of the product development cycle or...

Nicola: I think this makes more sense to me for the sort of discovery research than necessarily like very usability focused. I've been pretty lucky that in the majority of my career the usability research I've done has been combined with doing your more generative or discovery research. Yeah. Yeah. So we talked about doing a learning goals workshop as a nice way to involve people at first? The second thing that I would always include, I think there's a lot of different pieces. I love to run sketching workshops. I love to do use a story writing exercise and things like that, depending on of course what kind of project, but the next part is in user interviews. I just love to have people come in and take notes. It sounds really simple but I just find it such an effective way of helping other people to learn because I guess... I haven't really observed it being super effective when you do the 'send a video around afterwards' like, "Hey guys, watch this video" or sending out the link like, "You can dial in and watch the interview here."

Nicola: I just really haven't observed a whole lot of uptake on that. I think that in a world where you're an engineer and you've got 37 different juror tickets with your name on them and only six hours today, after you've got to sit into this meeting and that meeting, it's hard to ask people to devote a bunch of time to being part of research. But if you're asking them, "Hey, can you come and be in the room with me and the interviewee and be sitting here taking notes for an hour straight?" I think that's a really great way to help them to have the engagement that helps them to learn as effectively as possibly.

Erin: And do you actually then use those notes or is it, the note taking, more of an effort to engage the person and being present in the session?

Nicola: Oh, 100% I use the notes. Yeah, definitely. Definitely. And I've got so much to say about note taking. I feel like I've tried a hundred different ways of having notes taken and doing notes.

Erin: Yeah, give us the TLDR. Yeah. This is like a huge part of research.

JH: Yeah, yeah.

Erin: Yeah. Best tips, what are they?

Nicola: So my favorite way of doing it is I have the note taker on a laptop in the room with me. So whether or not it's an in person interview or a video call, I have the note taker in the room with me. Have them on a Google doc or a confluence page or whatever it is, and have them typing verbatim as much as possible everything that the person says. Which I think some people sometimes balk at if you've never tried to type for an hour of somebody speaking, it sounds like, "Geesh. That sounds pretty tough." But yeah, you'd really be surprised. It's totally possible. I've seen tons of different people do it, especially if you've given them a little bit of a template with some of the questions in it.

JH: Yeah. I was gonna ask about the template piece. Whenever I'm the note taker and we have the script and I'm trying to take notes around the questions, it kind of gets hard. Because everything... it never goes right down the path. You know what I mean? You kind of veered off and everything. So I was actually gonna ask you if you like starting with the template or just give the person a blank page. It sounds like the template is helpful.

Nicola: I do a very, very basic template. So I would have... I like tables. I would often create a table with on the left hand side, not necessarily the question but like the topic area, like the high level area. And then breaking it up and if we're doing usability or if we're clicking through something, then I would break it up by page. That's the important part.

Erin: This is hitting very close to home to me, because I've been sitting in on some sessions and doing some note taking myself and it's that issue of there's like 37 questions. This is like a big sprawling generative interview and I cannot find what they're talking about on the page, and it... so terrible. So I love the idea of the grid. But I think this is the reality of collaboration too, right? Is, you got to work with people and not everyone likes to do things the same way. So that's always fun.

JH: But to your point about why it's valuable to pull somebody in and take notes, right? It's... If you have the full recording and you send around a half hour session, not to be pessimistic, but I don't think anyone's watching it really. Or they're watching it while they check their email and they do everything else that they have to do. Maybe if you pull out 30 second or one minute clips that are really like great highlights, people will absorb those. But when you're in the room taking notes, it's like active listening on steroids. You have to pay attention. You can't have slack open, you know what I mean? You're really dialed in to capture everything. And I think it's hard to replicate that through lower stakes, "Hey read this when you have a chance", or, "Look at this video and you have a chance." And people are multitasking and just not absorbing it the same way.

Nicola: Totally. And there's been a bunch of research as well that showed how much empathy is built between people when they're actually in the room physically together. And it's massive. Yeah. So if you're doing in person interviews, it's just huge.

Erin: So yeah, we can all agree on that. How do you deal with getting with the, just the logistics, right, of getting people literally in the same room, if not remotely in the same area at the same time, when people are busy? Does a sync have any value and bringing people into research? Or is it whatever you can get at the same time, it's just worth so much more than anything you can get async, you take what you can get. How do you think about that?

Nicola: I have never done this async. So if you guys have thoughts on that, I would love to hear it.

Erin: Well, I'm thinking about the use case of, "I made a report, it's in Google drive, I'll slack a link, check it out when you have time." Right? Which we're saying is not very effective. Right? But in the reality of remote work and time zones and busy people, are there any use cases for being able to take it back to the original focus of facilitating learning about the user within an organization? Can that learning happen and an async way, ever? Or is it let's get the people we can get together, together when we can get them and that's what's going to work.

Nicola: I think it can of course, and I've seen, from working with other awesome researchers, I've seen the cool ways that other people deal with that. Right? I've seen... I know Becky White at Atlassian, in Sydney, is super, super good at this. She's created some really cool stuff there, including a choose your own adventure story. Super, really engagingly written blog posts that go out internally, stuff like that, which I find really inspiring. But in my practice I've honestly been really focusing on the collaborativeness, as the way to pass that on. And I've been really pushing, every place that I'm working, to try and do it that way and try and figure out how we can.

Erin: Cool.

JH: It feels like one might be an extension of the other, right? If you can figure out the collaborative piece in pulling people into the learning goals or note taking. Maybe it's easier to identify what resonates with folks and they're already bought into research and now you have more advocates or evangelists who will help you figure out how to make a cool blog post or a good trailer of clips, or whatever it may be, that other people can come back to that didn't participate. So maybe by getting people engaged up front you have more allies to help you figure out this kind of hard async problem that Erin was getting at.

Nicola: Yeah, absolutely. And I love this concept of having everybody around being a UX thinker. Really sort of having everybody feel that responsibility for the user experience, which is of course, driven by understanding the users. The more people around to really, really get that, really feel it in the heart, and who can tell that story, I think the better products we're gonna ship.

Erin: Let's talk about understanding users, which is obviously as big of a topic as you can get in this world. But we're talking about facilitating... The researcher as a facilitator of an organization's understanding of the user. What does that mean, "understanding the user"? Are we talking about within a given project, being able to understand if our assumptions are accurate or not? Over time, are we talking about understanding the user context more and more? What do we mean when we talk... Obviously it varies from project to project, but I'm curious to get your take on what we're really trying to get into people's heads, in an organization, about users.

JH: I think it's all of the above, and I think it's easy for us, people who are really living that user centered thinking life. I think it's easy for us to sometimes forget that not all organizations are even thinking about their users. I recently did a workshop with a company, with a software company, and they brought me in to do some work with them on just really basic, just basic user research. What does it look like? And yeah, and then at the end people were coming up to me and like, "Yeah, that was really great. It's really inspiring because we're just not thinking about users it all. Like we're... It's just not even... There's only a few people in the organization who are even having that conversation." So I think it could be anywhere, to answer your question, it could be anywhere from just having people thinking about users at all, all the way through to, yeah, what you're talking about. Really understanding the context, really understanding their goals, really understanding their world. Really getting what of their problems are we solving?

Erin: Got it. There's like a Maslov's pyramid here, where, like, start somewhere and maybe you'll evolve to... Yeah, over time. Cool.

JH: Well it's also like... Understanding users is kind of a nebulous thing, right? It's not like, "Okay we did it, we checked the box, we understand everything about our users now." Right? It's very much an ongoing thing and your knowledge is very additive. So I think what you want to maybe aim for, as an organization is like, people say, "Learning is a lifelong pursuit" or whatever, right? Understanding your users is kind of like a company long pursuit. And if you think about it more as a chart, you probably just want to make sure that your slope is always going up to some degree. Maybe the rate will change depending on initiatives or focus, right? But I think you just want to avoid plateaus where it's like, "We've learned nothing new about our users over the last three months." That seems really scary, right? So I don't know that it's binary thing. "We understand users, we don't understand users." It's more of, "Do we understand more about our users today than we did yesterday?" And if that's true, that seems like a good trend. Right?

Nicola: Yeah. Yeah. And I think stopping, stopping the learning probably is synonymous with that chart going downwards, because the world changes, right? Users change pretty quickly in the world of software.

JH: Yeah, that's a good point. It's probably, yeah, it's not a plateau. It's like you're actually, you're learning... you know less because things have changed since you last talked to them.

Erin: Well, and I think we talk about continuous research in the context of shipping product continuously a lot. But yeah, users change, contexts change, the world around them changes. And maybe as importantly, this idea of understanding users and user research seems like it ought to be a task built on humility. Right? "I don't know something and I want to know it." The worst thing you can sort of do is say, "Well, now I know the user. My work's done." You have to embrace that beginner's mindset, I guess. Forever, to continue to do the work.

JH: Yeah, I think we -

Nicola: Yeah.

JH: If you use a different example, right? Like Erin, I know you've been married for awhile, do you understand your husband? Like truly, 100%, you don't know everything about somebody, right?

Erin: Well, I always think of that old Bob Dylan quote, it's like, "I was, oh, now I'm going to butcher it but... I was older then, I'm so much younger now." Do you think the more you know, the more you know, you don't know." But you actually do know more, you're just aware of how much you don't know.

JH: Yeah, yeah. Right. But yeah, there's this certain arrogance that's almost dangerous of, "We understand our users" is almost a little bit risky.

Erin: Like, yeah. Hold the phone. Yeah.

Nicola: Yeah. I think it's just one step in front of the other. Are we still going in the right direction? Are we still going in the right direction? Are we still going in the right direction? Just keep asking that question.

Erin: Thanks for listening to Awkward Silences brought to you by User Interviews.

JH: Theme music by Fragile Gang.

Erin: Editing and Sound Production by Carrie Boyd.