Multiple times over the course of my career, I’ve been asked “what’s your research philosophy (when it comes to games)?” Although I’ve been asked this during interviews or conversations about non-games applied research roles, too, it’s the most common within games. At its core, it is focused more on your overall approach to applied games user research (GUR) rather than specific methods, for example. It’s a fun question to unpack and here are some of my top mentions (in no particular order)…
1) Balancing creative intent with player-centricity, but ultimately being data-informed
It’s an ongoing balancing act and games are meant to have friction (the classic example is Souls-likes). We want to understand primarily where the unintended friction lies, cross-reference that to the design intent, and keep the friction where it works as intended. Being ‘data-driven’ is a common buzzword in tech, but really you want to be data-informed since games are a creative-driven industry, and the blending of art and science (usually) provides the best results. In the end, GUR is only one source of feedback in the grand scheme of many inputs game devs hear from. So it’s important to make it clear within the report/deliverable/debrief that UR is not a checklist to complete, but a conversation to be had in terms of helping prioritize and whether some things are even worth actioning, and that UR should be considered as one piece of a holistic feedback pie.
2) Collaboration
I think it’s best when nearly the entire research process is collaborative aside from doing the actual research (e.g., executing/analyzing/reporting). This includes establishing the goals of a specific round of research, understanding how the designers will act on the findings, timelines, etc. And with teams or stakeholders who may be less UR-mature, I will take them along for the ride to be as transparent as possible to avoid the ‘black box’ GUR phenomenon, both during research inception and upon following up on findings to see how a team is progressing with them. Constantly having an ongoing rapport with designers or other stakeholders is paramount because, in the end, it’s our (the whole team’s) research.
3) Getting involved early and often
This is one of the most stereotypical things you’ll hear from a GUR (or any applied researcher, probably) about where they’d improve their research or where they feel they might have more impact, but it’s true. I’ve done some of my most impactful research early in development, as early as exploration before there’s even a playable prototype or clearly-defined pillars. This not only helps get a foundational understanding of the space we’re creating in, but also baseline player expectations, behaviors, and motivations that we can carry forward to 1) inform future research and 2) continue to advise devs on an ongoing basis to further promote player-centric design to help them potentially innovate and create unique things to ultimately help the project stand out. Additionally, testing individual features before they’re fully-baked, prior to larger-scale holistic testing, also helps ramp up this foundational knowledge before getting to the larger, more-pressing business questions (e.g., do they like it?!) and will lead to more representative data. The ability to anticipate issues before research happens is an important skill to have as a researcher.
4) Embedding continuous discovery into research
Project constraints (e.g., budget, timeline, etc.) can often result in you falling into a specific cadence and habits with testing. This most commonly leans towards evaluative research (e.g., usability testing) with dev teams’ desiring predictability and same-ness, and can leave little room for growing foundational knowledge that might benefit the project. If you cannot carve out time for bespoke research, finding ways to test existing assumptions and discovering unknowns, within your current research cadence, that may not be at the top of the teams’ minds will only help provide a more holistic picture of the player experience. Now enjoy this meme I repurposed for a talk I gave in 2022:
5) Pragmatism
While I have a background in behavioral science, and theory is incredibly valuable to inject into the research process (e.g., understanding flow, cognitive load, etc.), research ultimately needs to be practical and actionable. It needs to fit within the fast pace of game development timelines. Rarely will a round of research be ‘perfect’ in a researcher’s eyes but it’s effective if we’re answering the right questions and the team is acting on findings. There are times when larger, exploratory studies may be helpful but those tend to have longer horizons and not be contingent on project-specific goals or strict timelines.
6) Data triangulation across multiple sources
When possible, it’s helpful to not rely on one data source when reporting a finding or issue to a team. We can use a playtest for this example: cross-referencing various data sources, such as gameplay observations, survey feedback, interviews, and telemetry can help tell complete story about a finding or issue that will ultimately be more difficult to be ignored or dismissed by a team. Of course there will be times we will still not be 100% sure why something happened or someone felt the way they did, but be as comprehensive as possible to try and mitigate those scenarios as much as possible. If you have additional time and budget, recommending additional research to investigate the current unknown is always a potential route. A post-launch support example may be marrying behavioral telemetry with large-scale survey sentiment to get a better understanding of a current problem. However, it’s an important distinction that this is going to be easier for a researcher within a publisher or studio compared to an agency to due to internal access to data
7) GUR should have a seat at the table and not just be messengers of bad news
I’ve worked with various folks, both GUR and dev teams, who have differing philosophies on this but I am a proponent of reporting positives in terms of what we’re seeing that’s working (when we’re confident enough so teams know not to change something, for example) alongside all the bad news we’re most often associated with reporting. I am also a proponent of GUR including recommendations within reports/deliverables (within reason and not necessarily personal takes) as long as it comes with a thorough understanding of an issue, consideration into the context of where the game currently is in development, what constraints the team may have (e.g., the engine they’re working in) and what the team may or may not have already considered as alternatives. This has helped me earn trust in the past and often comes with time and rapport with a dev team. Similar to point #6, this will differ if you are within a publisher/studio or within a research agency because the latter will most often not be privy to internal knowledge and discussions, so leverage recommendations thoughtfully.