Games User Research

Multiple times over the course of my career, I’ve been asked “what’s your research philosophy (when it comes to games)?” Although I’ve been asked this during interviews or conversations about non-games applied research roles, too, it’s the most common within games. At its core, it is focused more on your overall approach to applied games user research (GUR) rather than specific methods, for example. It’s a fun question to unpack and here are some of my top mentions (in no particular order)…

1) Balancing creative intent with player-centricity, but ultimately being data-informed
It’s an ongoing balancing act and games are meant to have friction (the classic example is Souls-likes). We want to understand primarily where the unintended friction lies, cross-reference that to the design intent, and keep the friction where it works as intended. Being ‘data-driven’ is a common buzzword in tech, but really you want to be data-informed since games are a creative-driven industry, and the blending of art and science (usually) provides the best results. In the end, GUR is only one source of feedback in the grand scheme of many inputs game devs hear from. So it’s important to make it clear within the report/deliverable/debrief that UR is not a checklist to complete, but a conversation to be had in terms of helping prioritize and whether some things are even worth actioning, and that UR should be considered as one piece of a holistic feedback pie.

2) Collaboration
I think it’s best when nearly the entire research process is collaborative aside from doing the actual research (e.g., executing/analyzing/reporting). This includes establishing the goals of a specific round of research, understanding how the designers will act on the findings, timelines, etc. And with teams or stakeholders who may be less UR-mature, I will take them along for the ride to be as transparent as possible to avoid the ‘black box’ GUR phenomenon, both during research inception and upon following up on findings to see how a team is progressing with them. Constantly having an ongoing rapport with designers or other stakeholders is paramount because, in the end, it’s our (the whole team’s) research.

3) Getting involved early and often
This is one of the most stereotypical things you’ll hear from a GUR (or any applied researcher, probably) about where they’d improve their research or where they feel they might have more impact, but it’s true. I’ve done some of my most impactful research early in development, as early as exploration before there’s even a playable prototype or clearly-defined pillars. This not only helps get a foundational understanding of the space we’re creating in, but also baseline player expectations, behaviors, and motivations that we can carry forward to 1) inform future research and 2) continue to advise devs on an ongoing basis to further promote player-centric design to help them potentially innovate and create unique things to ultimately help the project stand out. Additionally, testing individual features before they’re fully-baked, prior to larger-scale holistic testing, also helps ramp up this foundational knowledge before getting to the larger, more-pressing business questions (e.g., do they like it?!) and will lead to more representative data. The ability to anticipate issues before research happens is an important skill to have as a researcher.

4) Embedding continuous discovery into research
Project constraints (e.g., budget, timeline, etc.) can often result in you falling into a specific cadence and habits with testing. This most commonly leans towards evaluative research (e.g., usability testing) with dev teams’ desiring predictability and same-ness, and can leave little room for growing foundational knowledge that might benefit the project. If you cannot carve out time for bespoke research, finding ways to test existing assumptions and discovering unknowns, within your current research cadence, that may not be at the top of the teams’ minds will only help provide a more holistic picture of the player experience. Now enjoy this meme I repurposed for a talk I gave in 2022:

5) Pragmatism
While I have a background in behavioral science, and theory is incredibly valuable to inject into the research process (e.g., understanding flow, cognitive load, etc.), research ultimately needs to be practical and actionable. It needs to fit within the fast pace of game development timelines. Rarely will a round of research be ‘perfect’ in a researcher’s eyes but it’s effective if we’re answering the right questions and the team is acting on findings. There are times when larger, exploratory studies may be helpful but those tend to have longer horizons and not be contingent on project-specific goals or strict timelines.

6) Data triangulation across multiple sources
When possible, it’s helpful to not rely on one data source when reporting a finding or issue to a team. We can use a playtest for this example: cross-referencing various data sources, such as gameplay observations, survey feedback, interviews, and telemetry can help tell complete story about a finding or issue that will ultimately be more difficult to be ignored or dismissed by a team. Of course there will be times we will still not be 100% sure why something happened or someone felt the way they did, but be as comprehensive as possible to try and mitigate those scenarios as much as possible. If you have additional time and budget, recommending additional research to investigate the current unknown is always a potential route. A post-launch support example may be marrying behavioral telemetry with large-scale survey sentiment to get a better understanding of a current problem. However, it’s an important distinction that this is going to be easier for a researcher within a publisher or studio compared to an agency to due to internal access to data

7) GUR should have a seat at the table and not just be messengers of bad news
I’ve worked with various folks, both GUR and dev teams, who have differing philosophies on this but I am a proponent of reporting positives in terms of what we’re seeing that’s working (when we’re confident enough so teams know not to change something, for example) alongside all the bad news we’re most often associated with reporting. I am also a proponent of GUR including recommendations within reports/deliverables (within reason and not necessarily personal takes) as long as it comes with a thorough understanding of an issue, consideration into the context of where the game currently is in development, what constraints the team may have (e.g., the engine they’re working in) and what the team may or may not have already considered as alternatives. This has helped me earn trust in the past and often comes with time and rapport with a dev team. Similar to point #6, this will differ if you are within a publisher/studio or within a research agency because the latter will most often not be privy to internal knowledge and discussions, so leverage recommendations thoughtfully.

Have a standardized format for reporting issues. I have found it to be a good practice to write issues in a way that explains the full picture, such that, for example, an executive who is not closely involved with the game or product can have a full understanding. This may include a brief explanation of the current design and intent, followed by the behavior exhibited by participants, and finally the outcome of how participants interacted with design. It is important to make it clear how the intent is misaligned with how users actually perceived or interacted with the design.

Have a system for addressing and labeling the severity of issues. This allows you and your team to clearly establish the difference between an issue that needs to be fixed as soon as possible and something that was a minor nuisance. This may be some sort of labeling system, such as A, B, C, D, with A being the most critical issue, whereas D is a low priority. Similarly, you may use “Critical, Medium, and Low” or “1, 2, 3” to the same effect. For a visual example, see the picture below, which uses a “Critical, Serious, Medium, Low” severity scale (source: User Focus).

Use consistent language to communicate issues. Using a consistent language to communicate issues benefits the development team in terms of knowing what to expect and not needing to re-learn report language every time. It’s also beneficial to the research team as it helps report delivery become faster as there may be less debate about how to best communicate issues. Some useful additions I’ve found include heuristics (e.g.,
https://www.nngroup.com/articles/usability-heuristics-applied-video-games/) and AXP & Accessibility pattern language (https://accessible.games/accessible-player-experiences/).

Use mindful terminology for suggestions/recommendations. If the development team(s) you work with is receptive to recommendations or fixes, use terminology like “consider” and “might be beneficial” when communicating issues. This emphasizes that they are the design experts and you, as the researcher, are simply suggesting a solution and not saying “this is how it should be fixed and it’s the only possible way.” Generally, your main objective is to find and identify problems, and this can reiterate that.

Blame the design, not the participant. I think it’s important to emphasize that user research reports are aiming to improve the game design and that should be reflected through your writing. I’ve encountered numerous instances where developers would question whether an issue written in a report was actually an issue or if we simply had an outlier as a participant. With video games being enjoyed by so many people all over the world, one person encountering a usability issue in a lab setting translates to thousands, maybe even millions out in the wild. Therefore, try to avoid issue titles such as “participants did not understand,” but instead title it “feature X is not clearly communicated.” Not only is the former title seemingly placing blame on the participants as if they did not interact with the game correctly (spoiler: there is no wrong way to play a game), but people quickly reading through the report who only see titles will be left asking “why?” rather than knowing there is an issue with feature X. Additionally, not only is it not glance-able, but “participants did not understand” is the outcome, not the issue.

Incorporate further evidence of the issue. In addition to your written explanation, things such as a screenshot, video clip, or gif can paint a clearer picture of what the issue is and help provide more context. Which leads to the next tip…

Humanize your reporting. If your data collection allows for the recording of participants while they interact with your game or product, including some relevant clips can help development teams further visualize actual humans struggling with their design, which can help motivate them to find a solution to fix it.

Provide summaries that address how the experience is affecting the player rather than what features are not working well. Researchers generally have a good idea of the design intent before testing but, depending on the scale of the game, you may have a lot of stakeholders who have differing ideas of the intended experience should be. Further humanizing your reports by stating how participants’ motivations, behaviors, and perceptions are affected can help developers think more about it as an experience that’s impacting someone rather than a product with a set of features. It’s rarely as simple as “fixing feature X will help players feel more competent during gameplay” and a more holistic, rather than reductionist, view of the user experience can help get that point across.

Differentiate between participants in the study and the proverbial player. If you are speaking specifically about something observed during a study, make it clear that you are doing so by referring to them as “participants.” For example, “Several participants were observed struggling to navigate the mission menu.” If you are speaking hypothetically about a specific interaction or experience, use the proverbial player. For example, “Players may have difficulty navigating the mission menu due to the low discoverability of the [Continue] button.”

Implement a system to indicate if an issue has been observed and/or reported during a previous study. This may be an additional text tag, such as “reoccurring”, or some sort of iconography indicating that something has been an issue that has been observed before. This not only further validates findings from previous research but might also motivate the development team to more quickly take action if they become tired of seeing participants encounter the same issues. It’s also additional evidence that can be referenced over time to build a case for elevating the severity of an issue.

Incorporate iconography for greater legibility. Using different icons to indicate different severity levels or types of issues written in reports can help your team more quickly read through a report. For example, if you want to make the team aware of something that was not necessarily an issue but something worth noting, you might label it “Note” and/or include a pencil or notepad icon to indicate as such. Therefore, if someone is short on time and needs to skim the report for important information that is only relevant to them, they can use the icons as reference points.

For anyone interested in beginning or transitioning to a career in games user research, here are some tips and suggestions that might be useful. While obtaining a degree in a relevant discipline and lab experience is a helpful start, there are many additional steps you can take to help you break into the industry. Here are some of the ones I found to be effective: 

Play games and change how you think about them

This should be a no-brainer – you should play a lot of video games. When trying to professionally study game design and development for a living, playing many different types of games is beneficial. This can include games outside of your comfort zone, such as different genres. Even games that are just plain bad can expose you to various types of design and issues. Exposing yourself to different platforms (PC, console, mobile, handheld, VR, etc.) will help you develop a more versatile skill set as a games user researcher. If you’re passionate about video games, this likely won’t even seem like work (more like a learning experience). Comprehensive knowledge and exposure will make you a more attractive candidate for a multitude of positions and studios.

Furthermore, to learn more efficiently and make better use of your time while playing, it would be beneficial to change how you think about games and playing them. You’ll want to keep user experience at the forefront while playing at all times by thinking critically and analytically. Look for usability, flow and general design issues and inconsistencies. If a game has an oddly-designed mechanic compared to the industry standard, is it for a good reason? Or, was it just an oversight that could have been improved upon with iteration? Was the tutorial sufficient? How does the progression and balance feel? Do the controls feel intuitive? And the list goes on.

Work independently

In addition to having an academic background in a relevant discipline, it is important to collect resources to educate yourself. This can include books specifically about games user research (of which there are quite a few good ones) and psychology, which will provide a solid foundation for research skills and many relevant theories that drive the field. Check my resources page for some examples.

One way to culminate all of this collecting and learning is to start your own blog. This will allow you to apply what you have learned in a constructive way, which can show potential employers how serious you are about a career in games user research. It can also be a reference during interviews, as many will ask specifically for examples of user experience issues in games. Ultimately, your blog can highlight your strengths, help you strengthen your weaknesses, and be an excellent way to demonstrate your critical thinking, writing skills (for all those playtest reports you’ll eventually be writing), interests/individuality, and growth as a researcher.

Seek advice/feedback and/or obtain a mentor

Another way to collect resources is to become active on Twitter. Many researchers in the field have Twitter accounts and tweet useful advice/informative resources (especially the official GamesUR Twitter account) and have personal websites where they document their work. Everyone I know in the community is very approachable, receptive and happy to field questions from aspiring user researchers and anyone with a general interest in what they do. This can be a great way to seek advice and feedback.

There is also an official mentoring program through the IGDA Games User Research SIG with an impressive list of academic and industry researchers willing to discuss a wide range of topics. Find someone you think would be a good fit for you and reach out!

Volunteer/offer your services to independent game developers

Reach out to indie companies that interest you, even if you are unsure if they have user experience researchers or offer similar positions/internships. Let them know about you, your interests, what you’re looking to do, and what services you can offer them. You might get lucky and they could have an open position. Or, your pitch could open them up to the idea of adding such a position. They could offer to bring you on as a volunteer/intern, which is a low-overhead, high-reward opportunity for them. These companies can be local to you (if you’re seeking specifically hands-on experience) or not if you’re willing to work remotely. I recommend the former, if possible, because you’ll start new relationships and also develop an understanding of game development and culture in general. However, either instance will provide valuable experience.

Seek entry-level or contract positions

Many large publishers and studios have user experience researchers and often offer entry-level user research positions on a contract basis. These typically range from three months to one year. While some do not require any formal experience working in games and/or user experience, many see it advantageous that candidates have done some relevant work. Once you feel you’re up to snuff for such a position, throw an application out there!

Although these positions are not practical long-term solutions, they can be a great first formal experience in the industry. You’ll learn new things, meet new people, and (hopefully) work on awesome games. Of course, your geographical location in relation to your companies of interest is a major factor. However, even for short-term contract positions, some companies will offer some sort of relocation assistance to ease the burden of moving. It can be risky to uproot for such a position, but it can also bring great experience that will ultimately help you achieve your goals.

Continue to learn and grow

Never be satisfied. Continue to teach yourself new things, both on- and off-the-job, whether it be a new research method, way to report data, or statistical-analysis program. If you’re only experienced in qualitative research, learn/practice with quantitative data sets (or vice versa). Become familiar with industry standards as well as different types of design. Continue to gather useful resources, read books, attend conferences, and learn from others to help you hone your craft. Think of your career as an RPG with no level cap; grind the side quests for the extra XP that will help you level up to achieve long-term success during the main quest. Strive to be a multidisciplinary researcher!

Hopefully these tips are helpful to aspiring games user researchers!