32. How the Observation Template Improves Data Quality

Step 2. Preparation

A good observation template serves as a way of improving the accuracy of data captured in playtests. It does this in a few ways:

  • Focus - By having a clear outline of what to makes notes on, the template focuses the team on what to document. Each person on a team can often have their own idea of what is important to note, the power of aligning up-front on what should be recorded is it makes everyone on the team focus on the same agenda. I’ve seen some team members leave a playtest with only a couple of lines of text written down while others in the same playtest had pages of notes, what was each person focussed on?

  • Rigour - the other benefit of having a clear structure of what to focus on, is it ensures each person observing is recording the same, or similar, data. This adds rigour by allowing the team to compare afterwards if everyone observed the same behaviours, or maybe one observer missed an event as they were distracted, but others captured it. Using multiple observers and a clear template will add confidence to what really happened in a playtest.

  • Maximising observation time - in a playtest you want to spend as much time as possible observing the player, and as little time as possible documenting. The reason being that if you are documenting then your eyes are likely away from the player, resulting in you potentially missing data. Ideally, recording the player’s behaviour should be reduced to a tickbox or a keystroke, this vastly reduces any time where your eyes are off the player compared to writing or typing. This may not always be possible depending on your playtest’s purpose, but it is a useful goal to aim towards when designing observation templates. You may of course need to make some notes beside the pre-prepared behaviours if needed. For example, the player may have found the required item to buy, but only after searching many menus, so you may note this.

Key Takeaway

It is a worthwhile investment of your time to think carefully about the design of the observation template. A good design will clearly communicate what is to be observed, and make it easy to record the most relevant player behaviours.

Next: 33. Atomic Behaviours - Adding Granularity to your Observation Template