Playtesting 106: On the Day

This is part 6 in a series on how to playtest games (click through to parts 1 2 3 4 5).

So it’s been a while since I’ve posted one of these, as we’ve recently taken on quite a few more people at Throw the Looking Glass and I’ve been doing a lot of work working out how to structure our studio going forward (more on this soon!). So sorry about the delay - I probably won’t be posting these quite as fast as I used to going forward, but I’ll aim for one a week.

Today, we’re going to look at what to do on the day of your playtest. This assumes you’re doing on-site testing rather than online testing (if you’re going with online, release it and see what happens!). So without further ado, let’s get started!

Step 1: The Prep

Preparation’s pretty important here. You really want to make sure you’ve got a decent space to playtest in, and that you’re able to make your testers as comfortable as you can.

Playtesting should be organised as an appointment for each specific tester - you specify a certain time for the playtester to come in, and tell them how long the test will take (as opposed to getting a whole lot of people in at once and making them wait). The next appointment should give you enough time to reset, take a breather etc. If you’re doing a number of playtests in a day, it’ll be a pretty tiring day, so it’s really good to give yourself some breathing space after each test.

Some stuff you need to make sure you do:

• Have your game in a stable state (it’s good to fix the featureset for a week or so to weed out bugs, though it almost never happens that way)
• Test all your equipment. Twice.
• Make sure any forms etc. are printed before your testers get there.
• If possible, have a backup for any systems you’re using
• Have pen and paper ready (even if you’re autorecording things, making notes is vital)
• Snacks and drinks. This is one of the most important things: you’re asking people to take time to test your game in its (possibly very buggy) state, so at least feed them while they’re there.
• Have a plan. Timetable what you’re doing when.

Now that we’ve done the prep, we’ll go onto the next part: the test itself.

Step 2: The Test

Before we go forward, I want to make a quick note about personnel: in particular, how many developers should run the playtest. On the one hand, having a high developer:tester ratio can make be intimidating to the playtesters. On the other, too few developers and you’ll miss things. Generally, having two developers at a test is a good number: one developer can look at the screen for play events, while the other can look at the tester and their reactions.

During the actual test, the most important thing is to make sure you don’t lead the player on. Unless you’ve specifically decided to teach the player something, SHUT UP and let them work it out for themselves. You should only tell them what to do if it’s patently obvious that they’re not going to work something out themselves.

Working out when and when not to help the tester is an incredibly hard skill to learn, and you’ll only learn it from experience.

Decide before the test starts exactly what you want to monitor and note down, and monitor exactly that and no more. If you start looking for other things, you’ll end up missing important data and then you’ll end up having less usable data than you need.

If you like, you can also record audio and video of the players, as this can be a really useful resource. HOWEVER, realise that going through all of the recordings takes a long time, and there’s a decent chance you’ll never get to it in a busy development cycle.

Otherwise, the actual test should be a matter of just following your plan. Your goal is to make the tests as enjoyable for the players as possible (as you probably want them to want to test your game again later), while gathering as much data as you can.

Step 3: Afterwards

Straight after the test, you should really take 15 minutes or so to go over and/or discuss what happened with the rest of your team. Work out what your gut instinct is on the most important changes to your game, and work out what you think you should change. When we look at analysis next, we’ll be examining how you can confirm these gut instincts.

Then, have a rest. Have a drink. It’ll have been a long day, so take the rest of it off!

That’s it for this time - next time, we’ll start looking at analysis.