On GMing Playtests

By | January 3, 2015

Hello everyone and happy 2015!

For the first post of the year, I’m going to talk about something we learned in 2014 about running playtests.  Ray Watters has run the majority of our local public playtests, but in 2014 my day-job responsibilities finally were out-of-sync with my weekend-convention opportunities, allowing me to actually get some GMing time in.  In these situations, Ray and I would both run playtests/demos, sometimes of the same module and sometimes of different modules.  And what we discovered, once we were both doing it, is that we were both getting very different kinds of feedback.

When Ray ran playtests, we got very little negative feedback and significant positive feedback.  Nothing particularly game-changing, but definitely things to watch out for and maybe tweak.  When ran playtests, we got significant positive feedback but a lot more negative feedback.  And in my case, they were things that were system-defining.  One of the most significant was at Metatopia 2014, where we got feedback about our skill system that caused us to redefine the way skills work in Forthright Beta 3 (coming soon!).

This bugged the crap out of me.  Why were we getting such different feedback?

So we began analyzing how we ran playtests differently.  Here’s what we found:

  1. Don’t Fudge The Rules.  One of the tricks of good GMing is that when players want to accomplish something, and you’re okay with it, fudge the rules.  Ray is a very good GM, so he fudges automatically and instinctively.  One such example would be when he allowed a ritual to be performed in less than a minute, because it would be cool for the players to save the crew of a crashing airship.  Problem is, rituals are explicitly designed to not be performed that quickly to avoid the Batmage – by ensuring the playtesters had a good time, the system was not actually playtested.  If you fudge the rules, you won’t get accurate feedback on if those rules work well.
  2. Explain It All Up Front.  Another problem we ran into was that Ray would explain nearly all the mechanics ahead of time, while I would explain the mechanics in general and then drill down in the midst of play.  What we found is that Ray’s playtesters would actually interact with the system as it was intended to be interacted with, while my playtesters would be playing D&D with different mechanics.  D&D and its derivatives are the default entrypoint (and elephant in the room) for the vast majority of roleplayers and roleplaying, so if you give them a traditional RPG and don’t tell them all the ways in which is it distinctly different, they’re going to default to the default style of play.  So always make sure to let your playtesters know the distinctly different theming and style of play your game presents, to ensure they know how to interact with it.
  3. Use a Variety of Scenarios.  Ray would create and use scenarios that would ease players into the game, giving them challenges and guiding them toward how the challenge was intended to be overcome – “beginner scenarios.”  I, on the other hand, would dump players in the deep end – placing them in a scenario where their own ingenuity would need to be expressed independent of the system itself. This contributed to our significantly different feedback, but in a very positive way – we were able to identify exactly how much guidance new players would need when interacting with Forthright.  This would further inform how to write the rulebook and how to design adventures.  This must be used cautiously, though, because using “advanced” scenarios can just as likely alienate playtesters because it might be seen as an intelligence / eliteness test rather than a test of the rules.

Ultimately, you as a playtest-runner should be less concerned about whether the players have a good time and more concerned about testing whether your specific ruleset interacts with players the way you intend it to. If your players know that they’re participating in a playtest, they should hopefully expect that they’re not going to have a perfect experience.  Playtesting, in the end, is not about affirmation – it’s about making your product do what you want it to do.  And negative feedback is your very best friend when you’re playtesting.

Do you have any other tips and tricks for running playtests?  Feel free to leave them in the comments below!