54. Sensemaking - Aligning on What Happened
Step 5. Sensemaking
This first stage of making sense of the playtesting findings is for the team to align on what actually happened. We will not be using the interview data at this stage, only the notes from the players’ behaviours.
Why we Align
There are several benefits to reaching alignment on what actually happened in the playtests:
It motivates change - “We all saw that …”.
Alignment reduces resistance to change. One of the main factors in resisting change is if some people on the team do not buy-in to the reasons for change. If everyone is aligned, resistance is reduced.
Alignment increases commitment. If everyone saw the same unwanted player behaviours and believes they should change, there is a higher chance they’ll follow through with the changes.
The process of reaching alignment shows that we can all see things differently. Even in a playtest where everyone on the team is watching the same player at the same time, you may find variation in what people saw. Realising that things are not as objective as they might seem can be a stark realisation for some, and can lead to positive effects such as being more open-minded than before - “Maybe things are not as I thought they were”.
How to Reach Alignment
The actual process of reaching alignment is straightforward. For each hypothesis in turn:
If everyone on the team is aligned that the hypothesis is a ‘pass’:
If everyone is aligned that the hypothesis is a pass, ask if there are any other changes that could still be made (improvements). You don’t need to discuss the details here, just make a note that the hypothesis is a pass, but there is room for improvement. We can document these potential improvements in the next phase of knowledge sharing.
If everyone on the team is aligned that the hypothesis is a ‘fail’:
Make sure to check if everyone is aligned on the hypothesis’ failure for the same reason. Make note of what specifically caused the hypothesis to fail.
If the team is not aligned on whether the hypothesis is pass or fail.
If the team have arrived at mixed results on any given hypothesis, then you need to go more granular. If you have used a granular observation sheet this will help, be specific on which part of the player’s behaviour is failing, e.g. is it equipping an item or using it as intended etc. This granularity should help the team reach alignment on whether the hypothesis is a pass or fail.
This alignment stage is important as it makes the next stage of understanding ‘why’ a bit easier, it ensures everyone is aligned on what exactly is being discussed.
This process also helps with organisational learning, everyone discussing these findings are sharing what they saw and reaching one shared account of what happened. This one shared account prevents different versions of the truth being told, an undesirable situation.
The value in having more than one observer present in a playtest is to add reliability - is what you saw the same as what your colleagues saw? In addition, if you only have one observer it’s possible that they might have an agenda to push, and might ‘see’ the data in a particular way. Even if they don’t have a hidden agenda, they will still see the data in their particular way, so having multiple observers helps avoid bias, ensuring a more balanced interpretation of the findings.
Outcome
The output of this phase will be a pass/fail assigned to each hypothesis of the playtest.
Key Takeaway
Reaching alignment on what happened is an important first step in making sense of the findings. Alignment will not only improve the likelihood of changes being made to the game, but it will also improve the team’s shared knowledge - ‘What we all know’.