Thoughts on: The things we learned on Liberty Island – Oliver and Pelletier.

Oliver, M. and Pelletier, C. (2005) The things we learned on Liberty Island: designing games to help people become competent game players. Proceedings of DiGRA 2005 Conference: Changing Views – Worlds in Play . Vancouver, BC: DiGRA

This paper discusses a fairly detailed methodology developed by Oliver and Pelletier for analysing how people learn from particular instances of game play.

It draws fairly heavily on Activity Theory as far as I can see, which while worthy, might be trying too hard to apply greater usability to something that doesn’t necessarily need it. (Of course, my understanding of Activity Theory is very shallow and wikipedia bound but I think I have the gist of it)

According to Wikipedia, Activity theory theorizes that when individuals engage and interact with their environment, production of tools are resulted. These tools are “exteriorized” forms of mental processes, and as these mental processes are manifested in tools, they become more readily accessible and communicable to other people, thereafter becoming useful for social interaction… In the study of Human-Computer Interaction and cognitive science, activity theory can be used to provide a framework for evaluating design.

In a framework derived from activity theory, any task, or activity, can be broken down into actions, which are further subdivided into operations. In a design context, using these categories can provide the designer with an understanding of the steps necessary for a user to carry out a task.

Anyway, hopefully this will make more sense as I move on. (The reason I ask whether games need to be made more usable is that this says “easier” to me and a large part of the appeal of games is the challenge of them. Of course, there’s good challenge and bad challenge, generally borne of bad design)

Overall though, the methodology that Oliver and Pelletier propose seems pretty useful in the Evaluation phase of the ADDIE process in terms of usability testing and understanding how people learn while playing games. (Primarily how they learn to play the game rather than learning anything from the game)

As they put it: “What is missing is a method that looks at the process and outcomes of play and how this relates to the design of the game text as well as the social and cultural aspect of play”

They develop this by examining:

  • Learning to use tools skillfully (both the game controller itself and objects within the game)
  • Learning about the properties of in-game objects
  • Learning about game conventions
  • Learning about spaces within the game.

They note that “with any area of expertise, it is rare (if not impossible) to find individuals who can perform skillfully and provide coherent accounts of their practice, simply because much of skilled performance is tacit – the practitioner remains unaware of what it is that enables them to be successful”

They then worked with 2 gamers (a fairly small sample really) and have them both play a level of Deus Ex, a first person shooter with stealth elements. One of the gamers also plays the training level for this game beforehand, which is made up of a series of scaffolded skill development tasks. Unsurprisingly, this player made it through the actual game level faster and more effectively than the player who didn’t.

There were a number of interesting observations nonetheless.

Analysis involved the creation of transcripts that recorded interaction with the game at the level of aim, strategy used in support of that aim and the detailed tactics or instances of interactions that made up each strategy. (In the terminology of Activity Theory, on which this analysis draws, these three levels are referred to as activity, action and operation.)

Neither of the gamers had really played FPS games before and so while they were familiar with conventions from other game genres, there were a number of FPS conventions that they weren’t able to quickly transfer to their Deus Ex experience. The training level player was aware that she could save the game on demand and so was able to experiment with trial-and-error based solutions to problems/obstacles encountered in the game to a greater degree than her counterpart.

When a player resolved a problem (and there was no reason to believe this was not just luck), this is noted as an example of learning. Finally, any things that the player was able to do without needing to learn anything new – i.e anything they had already mastered – was noted as an example of transfer. These included both simple things (like saving the game from the menu) as well as more complext things, including styles of play (such as approaching particular areas as if they were part of a platform game)”

One of the areas that I’m most interested in in the FPL project is using more structured instruction in the game environment and so their description of the training level is somewhat useful.

The training level here served to provide a structured curriculum to introduce new players to the game. Twenty five separate activities were introduced and applied; for example, learning how to access goals, how to use items (including weapons), how to move in particular ways (steathily, how to jump), as well as conventions such as information being stored in data cubes. In addition, eight separate tasks were learned that were not specified by the instructions within the game, such as the fact that the avatar cannot die in the training level and guards can hear you.”

 I think it might be in the training levels of games that I can find the type of learning that is most useful in a VET context. (Need to try to remember that this is the focus of the project)

The ability to save on demand has interesting implications for the way a learner approaches the game.

“In part, progress was driven by recent failure. The tendency to save after each obstacle is overcome meant that attention was focused on solving one problem at a time. Each failed attempt to overcome the obstacle was taken into account in new attempts to progress. This meant that play was experimental, because the consequence of failure is minimized”

The second player, who hadn’t done the training level and learnt about the ability to save on demand however was hampered in using a trial and error approach to problems by the fact that they would have to go back to the start of the level if they failed which made the risk taking a more costly option.

Oliver and Pelletier compare and contrast the knowledge that the players developed (or didn’t) in the training level as well as prior knowledge and how these transferred to their playing of the same level. They identify that “two problem areas are the strategies which could be transferred but are not and the ones which are transferred and which appear to be helpful but actually impede progress (because they are being applied in an inappropriate context, for example)

They go on to discuss the ways the players learn to play. Given the small size of the study, two players, it begs the question of whether this was influenced by the general personalities of the players themselves, however, it does still raise some interesting points.

“There are marked differences in the ways the players learn to play. The strategies developed and the reasons for this, related to their previous experiences and knowledge. In case study one, the player changed their approach when encountering new problems in a fairly sequential manner; in the second case, however, after two hours of play the strategies still failed to prepare the player for new encounters. This tells us two things:

  1. The development of strategies was strongly influenced by the experience of the training level, which enabled a repertoire of solutions to be developed in response to discrete problems, and which also ensured familiarity with a range of basic operations (such as searching bodies); and
  2. That one of the reasons why the second player failed to progress was because he did not save at regular intervals and so the consequence of failure were much greater. This impeded a trial and error evolution of strategies. (This may tell us something more generally about that player’s competence with this genre)

The paper concludes with the following observations:

“The analysis of play, above, demonstrates the value of the training level in preparing players for the main game. However it also reveals a number of shortcomings with this particular design (such as the unintended learning that took place) and that it is only a partial support, since many of the strategies that could usefully have transferred, didn’t. What was learnt in the training level was only part of what was required by the player; these experiences were combined with strategies learnt from other games in order to create a repertoire of approaches to play that led to success. Indeed, this transferred experience is probably the most significant component, since player two was able to progress through the game without training…

What this suggests for designers is:

  1. That it may be productive to design the opening of games with options that can be selected depending on the player’s previous gaming experience (understood not just in terms of quantity of experience, but also familiarity with particular genres whose influence might support or undercut the intended experience here)
  2. That it might be worth undertaking studies of this kind to assess whether their training ‘curriculum’ actually does prepare players for the game since, as demonstrated here, even in well respected games there can be differences between what was intended, what was required and what was actually learnt

Hmm, I would have thought the the widespread use of difficulty levels in games as well as adaptive difficulty (if the game senses you are struggling it makes life easier) would address the first point fairly well already.
Are they suggesting some kind of pre-test?
I agree that if you are going to include training it’s worth making sure that it is helpful but you can’t force people to learn everything you teach them, no matter what you do.

“The study also highlights the importance of establishing what the conventions that hold in this particular game are (such as cues from non-player characters that particular strategies – such as direct assault, here – are appropriate in response to behaviour rather than in anticipation of it)

This is an interesting suggestion – in game advice – you just want to avoid it being “Clippy” style annoying.

Oliver and Pelletier offer an interesting model for qualifying the experience of playing a game and examining how people learn as they play – I just worry that they might miss the point that we don’t want to make games too easy and to hold players hands all the way through. The challenge is the thing.

I had one more thought prompted by this paper – Do gamers have a more developed sense of trial-and-error problem solving than non-gamers based on their experiences that even though an obstacle may appear to be impassable, the point of the game is that there is always a way forward? Can this be taught?

I think the question of how players learn to use the controller is another question that needs serious consideration – I think controller fear is something that turns off a lot of non-gamers.

This entry was posted in activities, controllers, experience, first person learner, first person perspective, first person shooter, game design, Uncategorized and tagged , , , . Bookmark the permalink.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s