---

Depending on your context and your background, the notion (or even terminology) of a playbook may be completely foreign or completely mundane, however there’s a huge potential upside in developing a Research Playbook for your organization. This post will go into detail as to what playbooks are, why you should have one, and how to get started. There’s also a call-to-action at the end to help produce an open-source playbook to set a standard across the industry.

Why you should have one

The justification for and benefits of implementing and maintaining a Research Playbook

Before we get into the details of what the playbook is itself, let’s talk about why it’s important first. The playbook itself doesn’t make as much sense or drive as much value without the proper context. Ultimately, the core value of investing in and adhering to a playbook is that it will simultaneously increase the quality of research while making the research itself easier/less costly to perform. You’re probably thinking “if you can pull that off, that sounds great”, so let’s get to it.

If nothing else, writing a playbook will start paying dividends by virtually eliminating the time taken to either search for and copy/paste from a previous similar study or to write the entire plan out from scratch every time you’re spinning up a new study. The playbook is the one go-to place to get any study off the ground, centralizing anything that would otherwise be copied from another place, and if there’s an existing version of what you’re looking for, there’s no need to rewrite the entire outline. The other (arguably larger) benefit is that you save a lot of time in troubleshooting: the plays are written with enough experience and perspective to anticipate most difficulties you may run into, so by avoiding those pitfalls, you’re making research more efficient.

Another major benefit (which is somewhat related to reducing cost) is that when new members join the team, this serves as an excellent onboarding tool. By putting all of your methods, practices, and templates in one place, you reduce the overhead associated with new members finding where things are, and you help them understand the context in which your team conducts research. This is particularly beneficial for junior team members, as they likely haven’t had exposure to all of the methods your team employs and probably haven’t seen those methods laid out with such clarity. The Playbook then becomes a teaching tool as well, and helps to reduce training/onboarding costs.

The other main benefit of a Research Playbook is it increases the quality of the research your team conducts. One of the most common threats to data quality is a lack of consistency between studies and between researchers. If every researcher is writing their own plans, writing their own screeners, running their own briefing, debriefing, and test sessions their own way, it’s virtually impossible to ensure the effects you’re seeing are a result of the research or a side-effect of all of this variance. We can easily reduce this noise by providing guidelines, training, and templates to cover the vast majority of your team’s research needs. While the Playbook will evolve over time, and will provide some degree of flexibility on a per-study basis, giving everyone the same starting point and the same instructions helps establish a baseline of quality for your research activities. Any serious variance from the play should be discussed as a team as to whether it’s appropriate and/or if the play in question needs to be altered.

If you’re still with me, you may be wondering what this thing looks like and how it works in practice, so we’ll cover those next.

What is a Research Playbook?

Borrowing a metaphor from the sporting world to aid strategic decision making.

I’m unclear who was the first to use the term “playbook” to refer to a set of guidelines and templates, but I’m assuming they’re borrowing the term from the sports world. The terminology is common in Sales and Marketing, but it’s a fairly new concept in the UX world, and especially for Research. Maybe most famously/extensively used in American Football, where a playbook contains plays that are “a close to the ground ‘plan of action’ or ‘strategy’ used to move the ball down the field”. Each player on the field needs to know their role in every single play in the playbook. This knowledge is essential to successful execution of the team’s strategy, which is implemented and directed by a play-caller (usually the coach, sometimes the Quarterback) deciding which play to use in what situation based on philosophy, individual/collective strengths, the opposing defense, and situational goals. If your team has a significant points lead, you can be more aggressive. If you need to play conservatively, or only need a small yardage gain to get a First Down, your play call will be very different.

As it turns out, we have similar situational goals and strategic considerations in User Research. Is your focus early stage generative research, or is highly tactical, evaluative research? Luckily, we don’t necessarily have to work against another team actively trying to prevent our work, but we should be aware of the capabilities of our teammates, of leadership, of the project sponsors, etc., and we should pick the best possible plan of attack (or, play) based on all of those factors. Creating a playbook will help your team by codifying what the options are and providing some insight as to what situations call for which plays.

Okay, but what actually is it?

Plays, Guides, Templates. The building blocks of consistency in research methodology.

There are many forms a playbook can take, but at its basis, it’ll have at least one thing: plays. Plays are essentially sets of instructions of how to conduct a variety of human-centered research methods. It’s up to you to decide exactly what goes into the play (how much detail you go into, how prescriptive you are, etc.), but it should at least include:

1. The name of the method itself.
2. A brief description of what the method is/entails.
3. Some discussion of when it should be used and what you should expect/hope to get out of it.
4. Details on how to plan for, execute, and analyze the results of the research.
5. Links to any examples, relevant templates, or additional resources/further reading.

When it comes to the accompanying playbook, I’ve broken things down into 3 main sections: Plays, Guides, and Templates. Plays are the methods themselves, including all of the above bullet points (what it is, when to use it, and how to plan, execute, and analyze), and form the meat of the Playbook. After all, it’s called a playbook, not a guidebook or templatebook (is that even a thing?). Guides are accompanying instructional materials that will support plays but aren’t necessarily actionable in and of themselves. Guides are more the things you’ll want to refer to or distribute amongst your team to ensure everyone is on the same page, such as the Observing Research Guide (for non-researchers). Templates Are exactly what you would think they are: things you would otherwise be copy/pasting a lot across different plays or across different studies. Templates may include Standard Briefing & Debriefing, your Participant Transportation Information, and Research Plan/Summaries.

Templates are something that also allow some amount of customization per study (such as providing specific instructions on how to get to the testing location), however with great power comes greater responsibility, and it’s up to your team to decide how much leeway there is. That’s actually a common thread throughout: the playbook is ultimately something that should be owned by the entire Research team, and can be seen as one of the department’s deliverables. The specifics of how you plan and execute each method is also something that should be up for discussion, and depend heavily on the nature of your organization. If, for example, you have a recruiting/sourcing team, any information on how to get/select participants should include what it takes to work with recruiting. Or, in some companies, the Marketing department controls all direct communication with users or the public, so adding information on how to get outbound emails approved is also important. You’ll know your company best, and experience will help uncover where more guidelines and more details are needed. Having every company start from scratch every single time seems like a lot of wasted hours, which is why we should create an open-source, community-built template.

How to get started with a Playbook

A practical approach to building and implementing a playbook

Now that you’re convinced this is something you should have and you know roughly what it might entail, you’re probably wondering how to get started. As with most things in UX and in business, the answer is obviously “it depends”, however I’d be remiss if I didn’t provide some sort of framework as to how you might go about implementing something like this. Without going into too much detail up front, roughly your strategy should be:

1. Determine all of the individuals at your organization that are in any way directly or tangentially related in conducting research (“People Who Do Research”, or PWDR in the ReOps world). This may include Product Managers, Designers, Customer Success, Marketing, Sales, Senior Leadership, and, of course, Researchers.
2. Begin engaging them in conversation about the idea of creating a playbook in order to build alignment and consensus and to gauge each party’s level of interest and ownership over a variety of parts of the project.
3. Conduct an assessment of all of the research methods currently in use or that have been recently used, any supporting structures/methods/practices, and any other indirectly related activities.
4. As a team, take a critical lens to each study and each method, assessing how rigorous the methodology was, how clean and clear the results were, and what outcomes were driven as a result of the study. The goal is to pick out the good parts and identify things that could go better.
5. Use the results of that activity to begin drafting research plays, guidelines, and templates as you extract commonalities and reduce repetition in the planning, execution, analysis of research as well as the strategic considerations and how the results are shared.
6. Put together an initial draft of the identified items in your playbook and circulate among your team, among all of the PWDR identified in Step 1, and among senior leadership in all parts of the business – they may have perspective and opinions from outside your area of influence that are valuable.
7. Iterate. A lot.
8. Once you have something that everyone is happy with, begin using it. Make sure all PWDR are using the Playbook and are comfortable in the methods listed. Not all team members need to be experts in all areas of research, but they should both be able to read and follow the playbook and also know when to ask for help and whom to ask. The Playbook will never replace proper training, but it can be an excellent tool to guide personal and professional growth.
9. Periodically (quarterly?) revisit the Playbook in order to assess how well it’s serving the team and if anything needs to be added, subtracted or modified. This should be an ongoing process and should happen fairly organically, however it’s also beneficial to dedicate time to this on a regular basis.

Again, the above outline isn’t a be-all, end-all, and it leaves a lot of details out, but this should at least get you started. However, there’s a better way than starting from scratch: you can start from a community-approved open-source playbook.

A call to arms

Let’s build an open-sourced standard/template Research Playbook

Over the past year, the ReOps Community has grown and matured significantly, and along with it, my interest in and passion for Research Operations. For a long time, I thought of this as something I was in the minority when it comes to caring about it, and I attributed it to my background/training as an engineer, constantly searching for efficiency and optimization. As it turns out, there’s an entire discipline devoted to operations (popularized first by (Software) Development Operations, and then more recently by Design Operations), and while a lot of these duties have traditionally fallen to individuals in management roles, mature organizations are building out standalone Ops teams.

In my quest to improve rigor in the User Research world, I’ve come to realize that consistency can be an impediment. As an undergraduate studying research psychology and working in Andrew Elliot’s Approach-Avoidance Motivation Group, one of the most important elements of any study was developing a protocol. Each method has its own standard set of instructions and guidelines, and each individual study applied those guidelines to create a protocol. The protocol helped us minimize bias and to ensure that, across facilitators, we were maintaining consistency. Only recently did it occur to me that we can apply the same approach and philosophy to our work as professional User/Design Researchers.

After doing a little bit of searching, I didn’t see a lot of writing about Research Playbooks, nor did I see much of anything publicly available in terms of a playbook itself, which is why I’ve decided to start an open-source playbook. On top of writing the Playbook itself, I wanted to put together an accompanying blog post going into further detail as to what a Research Playbook was, how to use them, and why your team should have one. I hope the community will work with me to make this the best baseline of a Research Playbook we can make. The Google Doc where it lives will allow anyone to comment, so feel free to leave your thoughts and to circulate with your colleagues so we can get as many perspectives and opinions as possible 💪

---

Posted on September 16, 2019 in User Experience
Tags: [user research] [research operations] [operations] [playbook] [reops]