Playtesting 104: How to Measure Quantitatively

This is part 4 in a series on how to playtest games (click through to parts 1 2 3 5 6).

So we know why we’re playtesting, we know who and where, and we know what we want to measure. But how do we do it?

Today, we tackle this question for quantitative data.

So I’m not going to mince words here: you need to record data in your game. It’s a thing you just have to do. You can sit next to the person playing and count things like number of deaths, but there’s usually a bunch of other things you want to watch in that case.

You’ve got 2 main choices: use an existing logging system or roll your own. On the side of the existing logging systems, there’s Playtomic: a great tool for recording data for Flash, HTML5, Unity and iOS.

If you’re in one of these categories, you should really use it: it gives you both raw data and analysis tools such as graphing and heat maps for free!

If you’re not in these categories, you’re probably going to have to roll your own (if anyone knows of alternatives for other programming languages, we’d love to hear about it!).

Now ideally you’ll use a database of some kind that you can query quickly, as you’ll want to record a lot of data and then be able to analyse it. However, that’s adding a whole new system to your game that you probably can’t afford to do if you’re time is short.

The best alternative is text output, and in this case I’d suggest that you output things in a standard .csv format.

Before you start, pick your headings: usually a unique playtest id, game version, what the event is, when it happened (real time, game time, level number, game type, etc.), what it happened to, where it happened, any values associated with it and so on. The value of the csv is twofold: they’re easy to open in excel and analyse; and they’re easy to concatenate and edit.

When you are recording such data, make sure you record in all of these fields that are appropriate, as you might later want to look at the data from a different angle (for instance, you might want to know where players die in a certain level after you made a certain change.

At some point, when you do online testing, you’ll need to worry about gathering data from players who are running your game on their computer.

Again, its ideal to grab this stuff straight into an online database, but until then you might need to ask them to send you the output logs (getting this right is probably something you want to do before you do widespread testing, as your return rates will probably be rather low).

That’s about it - next time we’ll be looking at qualitative data collection: how to record it in person, and how to write questionnaires.