Interview with Stone Librande, Lead Designer at Maxis
Q: Tell me a bit about your background.
A: Before I went into gaming I was actually doing a lot of work in user interface design. I commercialized a technology that parameterized artwork and allowed users to quickly sift through thousands of drawings just by pulling sliders mapped to different characteristics. We found a lot of video game applications for it. Then I took a job managing a Web design team at a company called MPlayer, which was a social gaming network that was a little bit ahead of its time. But looking all the way back to my childhood, game design was always something I was interested in. Eventually I worked my way into Blizzard and from there on to Maxis to work on Spore.
Q: How was Spore’s game experience created?
A: Well there were really two pieces to that. First, there was a high-level description from Will Wright. In one case, we were asked to make a game about cells swimming in a drop of water. Then there’s the bottom-up design of the game mechanics. An important consideration in the cell game was creating the right balance of risk and reward. In any game you don’t want it to either be too hard (which would become frustrating) or too easy (which would make the game boring). But everyone’s different and we wanted Spore to have broad appeal to both casual players and hardcore gamers. The question is: How do you make an experience to fit many different tastes?
One way we approached that was by giving players opportunities to outfit their cell creatures with different pieces as they evolve. Novice players can finish the whole cell experience with just the basic creature design. You can get by while taking very modest risks, but you also won’t reap great rewards from it. But for hardcore players, there’s an opportunity to really dig into the game by experimenting with the effects of different pieces. They’re invited to take a lot more risks, and they put themselves in more danger of failing. Since the traits they pick up in the cell game effect the later stages, those players who take on a greater challenge can also put themselves at an advantage and realize a greater reward.
Q: How do you guide players’ behavior in games?
A: A lot of those ideas you learned in Psych 101 like reinforcement schedules are fundamental to game design. People are subject to the same behavioral influences as pigeons and rats. You can influence the players’ behavior by attaching a meaningful reward to the actions you want them to take. For example, say you’re designing a card game and you want players to try to collect three 3’s. You could force them to do that by making it the winning condition — there’s your reward. Or you can make people pursue that same goal less aggressively by saying that three 3’s are worth 3 points, while all other collections of cards are worth one point.
The most powerful reward you can give a player is a social reward. Intrinsic rewards are nice, but adding in a social component exploits people’s basic competitive nature. If someone else has something that you don’t have, you’ll work really hard to obtain it. There’s also a element of inclusion, of being part of an in-group that’s tied together by the game experience.
Q: You gave a presentation at last year’s Game Developers’ Conference about paper prototyping. Tell me about how your method works.
A: First of all, the paper prototype is not a representation of the actual game, and it’s not intended to be. That’s not the purpose. Instead, the point is to ask and answer one simple question about the game you’re working on. Second, it should be something that you can experiment with and iterate very quickly.
This is the prototype game used to design one aspect of Spore's cell game. Click to view full size.
So for Spore’s cell game, a key design question was figuring out the various creature parts that would be available to the player, and how they balance against one another. So I put together a board game version on paper. I wrote up a large list of parts and their abilities, going big at first so we could test a lot of scenarios and then scale it back. Players would assemble a unique cell creature using different combinations of eyes, mouths, graspers and tails. The cell pieces have different game abilities. For instance, tails allow the cell to move forward and rotate. During the game, each cell would either attempt to eat the most green food tokens (herbivore victory) or to attack and kill the opposing cell (carnivore victory).
Output from the prototype, used in designing the actual game.
We ended up with 12 parts that were given away over the course of the cell game’s five stages. We also defined the other creatures you’d encounter in each of those stages, ranging from harmless to more difficult as the player progressed through the game. That output was what made it into the final game.
Q: Why do this on paper, when you could model thousands of different scenarios in one go using a computer?
A: I run a workshop teaching this technique at the Game Developer’s Conference, and computers aren’t even allowed into the session. Building prototypes with paper fosters team interaction. As people work on it, they’ll start role-playing and getting into the characters of the game. They also develop a shared vocabulary for discussing elements of the game. If you did it with computers, everyone would just be working on their own and you wouldn’t get that kind of interaction.
Q: What works best prototyped on paper?
A: You can’t represent the full gameplay experience, that’s just not practical. A video game like Spore has a lot of physics and math, and that just can’t be done on paper. Input controllers like mice or keyboards are also really difficult to simulate. Anything that’s too complex would just be misery to test. Similarly, if a user interface designer were prototyping the front end for a database, you could show what the form elements and buttons look like but you couldn’t simulate the return of actual data. That’s just too complicated to do.
That said, when you really abstract a design problem there’s a lot that you can pull into a non-electronic prototype. In my workshop, I do an exercise where I have people build prototypes of existing video games. A few years ago one team decided to try doing Rock Band, and I was really skeptical that it would work. Surprisingly, they came up with a game that captured Rock Band’s core mechanics. There were five players, one of whom had a shuffled deck of colored index cards. He would throw out the cards in sequence, and all of the other players had to dig through their own cards and throw down matching colors. When you matched the pattern, the moderator would give you coins. If you missed, he would take coins away. Players could support one another by throwing coins to band members who were missing their beats. Even though there was no music and there were no plastic instruments, the game really captured the Rock Band feel.
This is a really amazing method. Thanks so much for taking the time to talk, Stone.
Interview with Mike Ambinder of Valve Software
Valve Software has designed top-selling games including Left 4 Dead, Half-Life, and Team Fortress. I recently spoke with Mike Ambinder, PhD, the company’s full-time experimental psychologist, to discuss the professional practices that ensure high-quality game experiences.
Q: What’s your role at Valve?
A: My job is to apply knowledge and methodologies from psychology to game design. That means performing statistical analyses, developing playtesting methodologies, conducting design experiments, a little bit of interface design, and investigating alternative hardware among other things.
Q: How can psychology guide game design?
A: Well for example, in the Left 4 Dead series there are several predetermined locations in the game called “drop points” where health items or weapons will spontaneously appear. To decide what’s dropped, where, and when we considered reward and reinforcement schedules, which are elements of behavioral psychology. You can put things on a fixed schedule so that they’ll appear at regular intervals. This makes the gameplay experience more predictable, and there can be real value in that. Or you can use a variable schedule so that you don’t know what’s going to show up or when it’ll pop in. Variable schedules can create a higher rate of engagement in the game and make the experience more enjoyable as uncertainty of occurrence can increase arousal. A large component of the gameplay in the Left 4 Dead series is the use of these variable reinforcement schedules.
Q: How is testing integrated into the design process?
A: We’re constantly playtesting. Our philosophy is to playtest as much as possible, and to start it as soon as we have a playable prototype. Of course our designers are experienced and generally make good decisions about gameplay, but we don’t want to just assume we’ve got it right. Game designs are hypotheses, and every instance of play is an experiment.
Q: What’s your standard testing method?
A: We use a variety of methods, but the most favored is direct observation of real players working their way through the game. I’m not a fan of the think-aloud protocol, in part because the constant prompting detracts from the gameplay experience and can introduce inadvertent bias, and in part because people can be really bad at explaining why they do what they do. Better to just sit back, watch, say nothing, and try to understand the player’s actions. So quiet, direct observation is our preferred method, but we combine that with player Q&As, surveys, quantitative metrics, eyetracking, and design experiments, and we’re investigating methods of measuring the player’s emotional experience during gameplay.
Q: How can eyetracking help to inform game design?
A: Generally, you want to eliminate frequent long eye movements because they lead to fatigue. For example, if the area map is in the bottom right corner of the display and your progress through that map is shown in the upper left, you’ll see the player’s eyes transiting the screen a lot. The proximity compatibility principle tells us that things which are mentally proximal should also be physically proximal, and eyetracking can tell us which things are mentally proximal. By arranging related information together, you can reduce fatigue and make the interface more efficient to use.
Q: And how can you measure the emotional experience of gameplay?
A: This is still early on, but we’re looking at biometric methods like EEGs which measure brainwaves, and EMGs which measure the electrical activity of muscles. But there are questions of their cost and efficacy. They’re also both very intrusive methods, requiring either a cap that’s wired into a machine or electrodes attached to the face. In testing you want to mimic the home experience as much as possible, and EEGs and EMGs both make it feel more like a lab environment. But new technologies are emerging that could change that. Remote detection of facial expression seems promising; these systems produce data along the lines of an EMG but only use a camera to measure muscle activity in the face.
Emotion can be viewed as a vector and measured along two scales: magnitude and valence. Magnitude describes the intensity of the emotion, while valence describes its quality (either positive or negative). You can measure the magnitude pretty reliably using something like heart rate, but understanding the valence is the tricky part. How do you know if that intense emotional response is good or bad? Of course you could just ask, but again that’s not a preferred method because people don’t describe their own experiences reliably and you’re introducing bias into the response. Context is a better basis. If someone is getting killed repeatedly, you can assume that they’re experiencing a negative emotion. However, to validate we’d love to have a system which quantifies valence in real time.
Once we can measure these qualities reliably, we can start asking what the ideal emotional experience should look like over the course of the player’s interaction with the game. Maybe that would be something like a pattern of peaks and valleys that steadily rises over time, as opposed to a prolonged burst of emotion that’s experienced all at once. That seems like a plausible theory, but we won’t know until we’ve measured it.
Q: What are some of the design elements that you’ve found make better player experiences?
A: I can suggest a few things. First, the player needs to be able to understand the experience. If you die, you need to understand why you died. If you reach a decision point, you need to understand what the implications are of taking path A or path B. The designer needs to provide a sensible environment.
Variety is also really important. Don’t give people the same monsters again and again, or force them to traverse the same levels over and over. There are obvious counterpoints to this, and the constructs of the game may dictate a lack of variety, so it’s not a hard and fast rule (none of these are), but it is something we try and emphasize. The Left 4 Dead series is a great example, because you’re always interacting with a new set of players with different skill levels and different tactics, and that will completely change the dynamic of the game. It doesn’t play the same way twice.
Third, you want to provide people with a feeling of continuous advancement. People prize rewards if they increase in perceived value. They want to feel that the required level of skill builds gradually as the game progresses.
Finally, have the player make interesting choices. Which weapon should I choose? Which armor should I take? If these decisions don’t involve meaningful tradeoffs, then you’re probably not creating an enjoyable experience.
Q: How do you foster collaboration in multiplayer games?
A: Left 4 Dead is really designed to force players to cooperate. If you go out on your own, for example, you’ll get incapacitated very quickly. The game doesn’t prevent you from doing that — it’s a choice you can exercise, but it’s inevitably a losing strategy. If you have other players near you then you can collectively put up a stronger fight, and when you fall then they can easily revive you.
Testing helped us improve collaboration in Left 4 Dead as well. In the original design, the thinking was that players would build awareness of each others’ locations just through verbal cues, speaking to one another through a headset. But it turned out that in the midst of gameplay that doesn’t work well. When a teammate fell and needed to be revived, the other players had a difficult time finding him or her. They needed another cue, so we introduced glowing outlines that appear around your teammates’ bodies, and which are visible through walls. We found that really increased the players’ situational awareness, facilitated cooperation, and created a better gameplay experience.
Q: What kinds of quantitative metrics do you use to inform design?
A: We work with tons of data. We can track any variable available in the game. We’ll take information about where people die in each level, then overlay it on an image of the level to show whether people are dying in the right places, and in the right numbers. We can examine the growth in players’ skill levels over time by any of various measures, depending upon the needs of the game’s design. That may be a fairly coarse metric such as the ratio of kills to deaths, who gets the most kills, who stays alive the longest, and so on. Or you can apply several measured in combination to satisfy a very precise definition of the ideal skill level, such as players who have a moderately high rate of kills but who win a lot and stay alive for a very long time.
I really appreciate your time. I’d wish you luck, but with these kinds of practices it really doesn’t sound like you need it.
Looking for Haiti Relief Games
Two weeks ago Farmville made a special white corn crop available for real money, and passed the proceeds on to Haiti relief efforts. In a news release, Zynga reports that they raised over $1 million. I think it’s a really amazing example of a game interface used to meet a real-world need. I also expect it’s fleeting, so if you know of any other online games that are doing something similar, please do let me know through a comment to this blog post.
Applying UX skills to game design
Nathan Verrill of Natron Baxter Applied Gaming provides a wonderful in-depth explanation of how he designed Signtific Lab, a game that facilitates prolific high-quality brainstorming around a central question. While the project was wholly a game, Nathan’s descriptions of the process steps and deliverables will sound very familiar to any UX designer: mind maps, wireframes, design comps, user testing, analytics, and so on. His background is in the design of conventional user experiences, and those same core competencies lent themselves well to the design of a successful game.
While they currently reside in different industrial families, user experience design and game design share a common parent in human-computer interaction. To the extent they differ, there are opportunities for cross-discipline learning. To the extent that they’re similar, expertise and skills transfer well from one practice to another. My own experiences applying UX skills to game design provide examples of both.
I designed my first game in 2002, when Unisys started an annual tradition of sending e-cards with embedded games to clients, employees, and partners. Each year, the in-house Web team would design and develop an original game, taking it from concept to delivery. Our first idea was for a miniature golf game that fit in with the company’s sponsorship of professional golf tournaments. While we were excited about the opportunity, none of us had made video games before. So we applied the same methods and skills that we used in the design of websites, simply because they were the only ways we knew to approach any design problem.
We started by conducting ethnographic research at a miniature golf course. Now I realize that last sentence reads like it’s meant to be facetious, but this was actually an indispensable step in understanding what makes the real-life game interesting, exciting, frustrating, funny, social, competitive, and worthwhile. For example, we discovered that the courses were often designed to tempt people who overestimate their own proficiency to attempt difficult putts which, if missed, put the ball much farther away from the hole. This in turn creates a social dynamic that can reverse the fortunes of beginners who play it safe, and skilled golfers who take greater risks.
Portion of the game wireframe, showing the putting interface
From there I designed a short wireframe, available here as a PDF. In some ways this was a traditional document, showing the core functionality while saying very little about the game’s appearance. But in other ways it was very different. The document focused on small interactions, as we were developing every interface element from the ground up instead of relying on ready made widgets like those baked into Web browsers. These were presented as atomic pieces that could be assembled to build a course, much like a pattern library. I also experimented with ways to show motion over time, and the effects of objects moving relative to one another.
Wireframe of some of the obstacles players had to face
The finished game, which you can play here, had its strengths and weaknesses. The visual presentation was fantastic and the level design was really good (owing to the efforts of Todd Horning and Mike Rosario), but it had some important usability and learnability problems (precisely the things that I should have been on top of). I think the core mistake was in describing the interface elements as individual pieces without showing how they should be put together. At the time, I reasoned that game design needed broad creative latitude and that the traditional prescriptive wireframe would have been too limiting. But it turned out that the way the pieces hang together, as with a conventional user interface, is really critical the experience of the game. For subsequent games in the holiday series my documents actually started to look more and more like Web wireframes.
Do you have examples of games you’ve designed using conventional UX methods? If so, I’d love to hear from you!
Interview: Luis Von Ahn, creator, Games With a Purpose
In 2003 Luis Von Ahn introduced The ESP Game, which challenged two players working online to independently pick the same words to describe a picture. But The ESP Game was also designed with a covert purpose: to improve search technology and the accessibility of the Web by gathering metadata about untagged internet images. Impressed by the game, Google picked it up and renamed it Google Image Labeler.
Dr. Von Ahn, a professor at Carnegie Mellon University and recipient of the Macarthur Fellowship, has since built out a collection of similar “Games with a Purpose”. I spoke with him recently to discuss the theory behind his work and his vision for how it can change the way we approach the design of user interfaces.
Q: So what are “Games with a Purpose”?
A: To the player a GWAP is for all purposes a game, but as a side effect of play it’s designed to produce useful work.
A: Well, because nobody would do it. There has to be motivation for doing work. There are a few ways you can provide that. You can pay people for work, and that’s effective but it’s also expensive. Then there are motivations that drive people to contribute to something like Wikipedia, perhaps because they believe it’s a worthwhile thing or because they like the feeling that they played a personal part in building it. But that model has failed when people have tried to apply it in other contexts, so it’s not a reliable motivator. Then there are things that people do because they enjoy them. So with GWAPs, instead of paying people with money you pay them with entertainment.
Q: How much can you accomplish by playing games?
A: On average, Americans spend 1 hour every day playing videogames. That’s over 100 billion humanhours a year. That’s a humongous opportunity, considering that it only took 7 million humanhours to construct the entire Empire State Building. And consider too that while people are spending all that time playing games they’re using their brains. If you could turn all gameplay into useful work, people would be amazingly productive.
Q: If people are just playing around, then how do you know that the results are of good quality?
A: There are a couple of tricks to that. First, you can correlate one player’s results with those of other unrelated players. For example, in the ESP game the same image will be shown to multiple players who are asked to submit tags describing it. Since those players have never met and never had the opportunity to interact, if more than one person gives the exact same answer then it’s much more likely to be a reliable tag for the image. Second, you can give players questions for which you already know all possible correct outputs, to see if they’re answering honestly. If their responses fall outside of the set of correct outputs, then you can flag them as suspicious and ignore the rest of their responses.
Q: Since you started promoting Games with a Purpose, do you feel that the use of GWAPs has progressed as you’d envisioned in the broader community of design practitioners?
A: Yes and no. I think it has been catalytic to what is today called “crowdsourcing”, which didn’t even have a name when we started. But games haven’t gotten to the point where I’d like them to be. Ultimately I’d like to see all work turned into a game (I don’t see why it couldn’t be), but we’re not there yet. That’s probably because it’s very, very hard to design a good game. Once you add in the constraint of the game producing useful work, then it becomes even harder. The potential’s there, but I think designers are just starting to figure out how exploit it.
Q: So how do you go about designing games?
A: Well first we just think about them. We think about them a lot. Then we build a prototype using just paper and pencil, and start testing it like hell. You really can’t tell whether a game will be fun or not until you test it. And if you find that it is fun then you build a simple live version and test that, revise it, and so on. And even then there’s no guarantee that it people will enjoy it. Of the games that you complete, you’ll find that some are much more fun than others.
Q: Can you talk a bit about fun?
A: Actually I’m not sure how to define the word “fun”. What really matters is whether or not people play the game. It’s a strange paradox that people will often play a game that they don’t even find enjoyable. So I prefer to sidestep philosophical questions about whether or not people are really having fun, and focus on what we can measure.
This is a spectacular direction for user experience design. Thanks so much for your time!
Webcast: Extending Game Design to Business Applications
I love games and think they have a lot to teach us about user interface design. This July, I’ll be webcasting a revised and updated version of my talk on extending the gaming experience to conventional UI’s through Rosenfeld Media and Smart Experience as a part of their Future Practice series. I debuted it last year at the IA Summit and EuroIA, and got a lot of great feedback from people who found it fun, interesting, and useful.
Details are available on the webcast’s description page. You can watch it individually, or project it in a conference room for as many people as you like on the one registration. If you decide you’re interested, be sure to use the code FERRARAWBNR to get a 15% discount.
If you’re interested in hearing something different, insightful, and kind of neat I think you’ll find this one fits the bill!
Letting crows do the work for you
Filed under: Uncategorized | Tags: crows, games, gwap, IA, klein, search, synanthropy, vonahn
Leave a Comment Let’s say you wanted to gather all the loose change within a 5-mile radius, but you lack the time to do it yourself (which seems likely). Why not enlist crows to do the scavenging for you?
That’s exactly what Josh Klein has done. For his master’s thesis at NYU Klein invented a device that, through operant conditioning, trains crows to gather coins and drop them into a slot. The device then disburses peanuts as a reward, like a vending machine. It’s a fantastically efficient arrangement, since he only needs to keep the machine stocked and the crows take care of the rest.
C. brachyrhynchos, the American crow
This is formally described as “synanthropy”, adapting animals to work within human environments. But I think it’s an awesome demonstration of an even larger strategy: getting an organism to do a useful job while pursuing an unrelated goal. The crow doesn’t share our interest in money, it’s just trying to get some peanuts.
Luis Von Ahn applies the same strategy to solve difficult computational problems, using human beings as crows. A researcher at Carnegie Mellon, he’s developed a series of “Games with a Purpose” that surreptitiously gather useful metadata while engaging players in 2-person online computer games. As with Klein’s crows, the players are pursuing their own goal — in this case to have a bit of fun. But through the games’ design, they create a byproduct that has real value. All of the games attach human meaning to data that machines can’t read.
His games are:
Games with a Purpose
- The ESP game, where two players are shown the same image and asked to suggest tags describing it, then awarded points when they match up. When more than one person has independently chosen the same tag, that implies it’s a reliable descriptor of the image. Search engines can then use those tags to return more relevant image results.
- Tag a Tune plays a musical track for both players. The players need to figure out whether they’re listening to the same track or different tracks by sending each other descriptions of what they hear. The value of the game is that in the process they’ll provide useful tags like “piano”, “airy”, and “female vocals” to indicate qualities that would otherwise be indecipherable to a computer.
- Verbosity is a password-like game that gives one player a word to describe to the other player, who has to guess it. The game provides a number of canned relationships that can be used to describe the word, like “It is a type of ___”, or “it is the opposite of ___”, or “it looks like ___”. For example, if the word were “cough” then the clues might be “it is a type of physical reaction” and “it is used to clear your airway”. This game assigns meaningful ontology relationships (known as “triples”) to words, which can be used to deepen computer understanding of human language.
- Squigl takes the images and tags created in the ESP game and asks players to trace the portion of the image in which the tagged object appears. So for a picture of a woman walking her dog that’s tagged “leash”, both players would be tracing a similar area of the picture. Points are awarded depending upon how well the areas overlap. There are several possible utilities for the data gathered, from deciding what proportion of an image is relevant to a tag to informing image-reading applications.
- Matchin simply shows both players two pictures and asks them to pick the one they like best. Each consecutive time they pick the same one, they’re awarded an increasing pool of points. This game provides data along the lines of Flickr’s “interestingness” function, adding human subjectivity to digital images.
Von Ahn compares the design of these games to the design of an algorithm. Each one ensures that its data output is correct, and has a certain level of efficiency associated with it. Like Klein’s device, they also take advantage of operant conditioning by awarding players points on the site’s leaderboard. It’s a simple currency and less tangible than peanuts, but effective because it’s promoted as a measure of prestige. Profiles even allow players to use the site as a matchmaker, pairing people who see the world in similar ways.
There are plenty of other examples of people acting as crows on the Web:
- Search logs of the most commonly submitted queries provide Web designers with a prioritized list of the things users expect to find on the site, expressed in their own words.
- Flickr users feed image-processing applications when they tag their images for personal use.
- Foreign language translations of documents are used to inform probabilistic translation applications.
In all of these cases, the people doing the work have an objective that’s unrelated to the way in which someone else makes use of their work.
User-distributed work is emerging as a central component of Web 2.0, but some jobs are too laborious, too tedious, or too unfulfilling for people to pursue them on their own merits. Learning how to use crows will be an important part of governing the role of the human being in future applications.
In Defense of Documentation
Filed under: Uncategorized | Tags: agile, design, development, documentation, process
Comments (3) At recent conferences and in discussion groups, I’ve found it’s become increasingly popular to deride design documentation as time-wasting, inscrutable, and pedantic. There’s a growing drive to jump directly into development, and manage design on the fly. Few people are providing cogent arguments to the contrary. As an IA who prides himself on the quality his documents, I feel the need to address the role of the spec in the age of Agile.
Bad documentation is real, and must die
There’s no doubt that there are some absolutely terrible documentation practices out there. If you’re writing several dozen pages of design strategy in plain prose, you shouldn’t be surprised when no one bothers to read it. A core measure of a development effort’s success is its efficiency, and documents that slow down the process should be hastily ditched. It makes no sense to compound the time lost in writing poor documentation with even more time spent reading it.
Good documentation is also real
Good documents are completed quickly, express ideas efficiently, expedite development, build trust among team members, and allow people to achieve things that would otherwise be impossible. People enjoy reading and discussing them, and teams of designers, developers, writers, and clients work as a whole to review and iterate them.
An analogy
Two years ago my wife and I decided to upgrade to an HDTV. The problem was that we were going from a 24” tube set to a 46” flat panel, and the new set wouldn’t fit our existing furniture. Even worse, the money we were spending on the TV precluded us from buying new stuff. So I decided that I’d build a TV stand myself.
"Wireframe" of our TV stand
Now you need to understand that I’m not the sort of person you might describe as “handy”. I’d never built so much as a birdhouse, and my high school didn’t even have a wood shop. I had absolutely no idea how to construct a physical object. Furthermore, my wife strictly forbade the use of any saws or sharp objects for fear I might cut off something important.
So I decided to create a physical object in the same way I would create any virtual one: I drew up a wireframe in Visio. Before picking up a hammer or even buying the wood, I first mapped out the whole vision on paper. I shared early top-level designs with my wife, we talked them through, and I modified the plans until we agreed on the approach, and then I filled in all the low-level details.
The completed TV stand
I’m pretty pleased with the result (see the picture at right). It’s not exactly Herman Miller, and it weighs about 50,000 pounds, but it meets our needs precisely and cost me just under $100 to make. And this example illustrates a few key advantages to a good documentation process over going straight to development:
- Faster completion time. The documentation itself only took two days to develop, and since everything was planned out beforehand there was no wasted development time. I got the project done in six weeks, much faster than I could have without the documentation.
- Control of design. There’s a terrific symmetry between the design on paper and the finished product. This was very important, because a lot of factors had to be balanced in the design: ensuring structural integrity, fitting all of the components in without making the stand too tall, and creating aesthetic beauty. Jumping straight in, I wouldn’t have been able to manage that balance.
- Planning around restrictions. It’s hard to build something out of wood when you can’t use a saw. Lowe’s could cut the wood for me, but they could only do straight cuts at 90° angles to the board. This was a built-in hardware limitation – not a deal-killer, but it required some creative planning. Accounting for restrictions in the documentation keeps them from becoming show-stoppers later on.
- Lower risk. Of course I could have jumped straight into the build, but that would have been much more risky. I would have wasted a lot of lumber. It could have come out looking terrible. I could have discovered halfway through that there was a much better way of doing it. Designing it on paper first was harmless, and maximized my chances of success later on.
Hmm, now what could that have to do with UI design?
By now this connection should be abundantly obvious. In fact, from the most general point of view of process there is no useful distinction between building a TV stand and building a website. Both require design and development efforts. Both involve costs and risks. Both produce an end product that needs to satisfy many needs equitably.
Quality documentation allows people to achieve things that are otherwise impossible. The human brain can only do so much on its own – we need to extend its capabilities by processing things on paper, with our hands, and through conversation. The task of designing software is often so complex that you can’t even grasp the full problem until you’ve written enough of it down.
But perhaps most critically, a good document provides for a foundation of trust to be built between team members. Developers shouldn’t have to do design – it’s a distraction from their responsibilities, and exposes them to accountability for decisions that should have been handled by someone who’s paid to make them. If the developers can trust that all foreseeable design decisions have been anticipated and built into the documentation, they’ll stick to it zealously.
What about Agile?
“Agile” is the name given to a process of rapid iterative prototyping and efficient communication. I think it’s a terrific thing, because it allows concepts to be tried out and massaged into good shape in short order. I employed a bit of Agile while building my TV stand – using scrap wood to figure out to make dowel joints and apply a stain, and grabbing whatever was within reach to get a job done.
But Agile has spawned a certain animosity toward even very high quality, efficiently produced, time-saving documentation. I don’t think that’s Agile’s intention. I think it rightfully seeks to eliminate time wasted producing documents that no one will bother reading, but perhaps its mantra has been repeated so often that it’s grown beyond its original meaning. Played out to its logical extreme, ditching documentation would lead to disaster on all but the simplest projects.
So what should we do?
We need to reestablish the role of documentation in the product development lifecycle. To do that, we first need to come to grips with the fact that what people have received in the past has been found wanting. So we need to make a few key changes:
- Crank it out. We need to be able to turn documents around faster than developers can build to them, or else we’ll become a bottleneck. Our documents should be done in the simplest form possible, with no excess ornamentation or verbosity. Ugly is efficient.
- Communicate efficiently. We do after all work in usability, so our top priority should be to keep the needs of the document’s consumers in mind. Integrate images with text effectively, speak clearly, and edit out anything that doesn’t solve a problem.
- Know your medium. We don’t need to be coders, but if we understand how HTML, CSS, scripting, and databases work we’ll be able to take advantage of the strengths of these technologies. Marshall McLuhan was right – the medium is the message.
- Make people’s jobs easier. Bad documentation adds to other people’s workloads. Good documentation lightens them. Everyone will be happy to have something that reduces the total number of decisions they need to make. Our documents should take control of everything that will affect the usability of the system.
- Produce gold. People need to believe you’ll design something that will make everyone involved look good, and bring them to a better outcome than they could hope for without us there. Get everyone’s input into the document as it’s developed, and iterate it repeatedly to build confidence in the finished product.
Interview with Eric Matthews, Creative Director of Sony Computer Entertainment Europe
Filed under: Uncategorized | Tags: eric matthews, eyetoy, playstation, sony, testing, ui design, usability, video games
Leave a Comment 
"Kung Foo" allows players to punch and kick at ninjas overlaid onscreen. (Image from Gamespot)
In 2002, Sony Computer Entertainment’s (SCE) London Studio took on the unique challenge of developing video games for a very different type of controller. The EyeToy USB camera for the PlayStation 2 projects a player’s image onscreen, where he or she can interact with overlaid objects by moving around and physically touching them.
The result was a compilation of novel games called “EyeToy: Play”, the first release for the controller. In “Wishy Washy” the players wipe soap suds off of dirty windows. “Kung Foo” has players chop and kick at hordes of miniature attacking ninjas. “Mirror Time” confounds players’ visual-motor control by having them grab at objects while their image is repeatedly flipped backward, upside down, and back again.
In a market where such models for player experience had no precedent, the EyeToy became an international hit selling millions of units. Last month I spoke with Eric Matthews, the director of creative development for Sony Computer Entertainment Europe. He works with the team at SCE London Studio, which has led innovation in the video game industry with titles including “SingStar”, “The Getaway”, and the “EyeToy: Play” series, now in its fifth iteration. We discussed user interface design, the development of the EyeToy, and the role of usability in game design.
How did the idea for the EyeToy get its start?
"Mirror Time" inverts the player's image repeatedly to confuse their sense of left, right, up, and down. (Image from Gamespot)
The idea of using a webcam as a video game interface originated from SCE’s research & development facilities in the US. Dr. Richard Marks had been experimenting with things like motion control, color tracking, and augmented reality. He had developed some prototypes using cameras, but there wasn’t yet any game experience associated with it. Phil Harrison, then the head of Sony Computer Entertainment Europe development, showed Rick’s work to the various game development studios to see if anyone could think of directions it could be taken.
Ron Festejo, who’s one of the people on our team, really wanted to look into it. From there, Rick assisted on the technical side, but it was London that made it into a game.
How did designing for the EyeToy differ from design for more conventional games?
We indentified four things to focus on.
First, you are the star of the game. You’re seeing yourself up onscreen, so there’s a bit of vanity there.
Second, it had to be as much fun to watch as it was to play. We wanted it to be a great social experience, and something where parents and kids could both engage in play together.
Third, the games couldn’t be better played on a controller. It wouldn’t be worth doing if you could only do the very same things as you could in any other game, so it needed to be something unique to the controller.
Fourth, it had to be easy for people to intuit, so you don’t need to have the rules explained to you. You couldn’t have a control scheme as complicated as you do for other games, where you’ve got sixteen buttons on a controller. And of course that has the advantage of making it more accessible to people who normally aren’t as comfortable with games.
How did you design gameplay for the EyeToy?
We started by doing a bit of competitive research, looking at the landscape and what things may have been done along the same lines.
Then we asked what is it good at, and what are its weaknesses? For one, it had a magical “wow” factor of seeing yourself onscreen interacting with the game world, which got people particularly excited. But while it could pick up big broad movements, but it was not very good at fine granular motion. Another disadvantage was that it was sensitive to noise in the background. We felt it worked well as a short-form medium, with fast play and a lot of variation in the games.
Then we ran some brainstorming sessions to see where we could take the ideas. We wanted to see how many different forms of interaction we could create, so we went really broad. From there we would say okay, we’ve brainstormed an idea for a boxing game, why is that a valuable thing? Why would you want to play that?
After that we went into a very experimental prototyping phase so we could get a playable experience onscreen quickly to see if the ideas were fun and engaging. We developed around thirty prototypes, including some wild stuff that never worked. We ended up with twelve in the final game.
Can you give an example of how one of the games was designed?
Well “Wishy Washy” evolved from an idea where you had to climb to the top of a building. I sat there with Pete Marshall, the lead programmer, and we started thinking “Okay, how would you see that?” And we said well maybe the camera’s on the inside of the building looking out through the windows at you. Then that turned into, well perhaps you’re washing the windows as you go up. Then it was, you need to clean the window on each floor before you could go to the next one. Through that process of just talking it through over a half hour late at night, an evolution of thinking turned it into a window washing game.
How did you interact with marketing?
We got marketing involved to collaborate on what games had the greatest potential and harvesting the best ideas, and getting out to look for an audience. They found that it was something parents could play with their kids, that could bring down barriers and get people moving. After the game was finished they put demonstrations of it in clubs and shopping malls for the launch, because it’s one of those things you don’t really get until you’ve played.
One of the defining moments came at The PlayStation Experience, a London event where we first showed the prototypes to the public. We showed three games and the press went nuts. Everyone from small children to hardcore Tony Hawk skaters with multiple piercings were lining up to play it. The gaming press wrote that it was something very unlike what they’d seen before, and there was a lot of excitement. At that point we started to think okay, we may have something here.
Up to then, no one had thought it was going to sell millions. Well, let me amend that – we thought it would either sell less than 100,000 or more than a million but not in between. It turns out it sold somewhere between 3 and 4 million units.
Did you do any testing of the game with users?
This is going to sound terrible, but EyeToy: Play was the first game where we did formal user testing, and that was only once the game was finished. We had done ad hoc testing using the people in the QA department, children of coworkers, things like that. But this was the first time we recruited real users and set up at a facility behind the one-way glass and so forth.
And it was an absolute nightmare. Once people were into the game they had a great time playing it, but they couldn’t get there quickly enough because the flow of the menus was too long winded. People also couldn’t figure out the right distance to stand, and then someone would walk across the room in front of the camera and inadvertently trigger something onscreen. It came to a head when the players kept accidentally cancelling the setup process for their profiles. Around then Ron Festejo got up and said “Stop the blasted thing, I can’t bare it anymore!”
So we all found the first test very tough, but it was a very valuable learning experience. From that point on, we’ve tested all of the games we develop. I’m a big proponent of the value of testing with users, and we’re building our own facility in London.
How did you develop the process for testing games?
There was no company in Europe that focused on testing video games. They were all working in consumer software or Web-based usability. And some of that’s relevant, but some of it isn’t. They had a heavy task orientation, where you’re given situation A, now show me how you’d do B, and that’s not as relevant to games. They just hadn’t had the experience of applying their methodology to a game. How do you test for fun?
So we worked with an agency that had a lot of experience in Web called AmberLight, and collaborated to create a process that we use to test all of our games.
So what is that process?
First, you decide what you want to test. That includes obvious things – the fundamental control system, how it feels and how it compares to competitors. Then there are things like signposting, navigation, how long it takes to complete objectives – primary, secondary, tertiary. We do a peer group review, and get the experts to look at it before going into testing for what we think is going to work well, and what we think might not work so well.
Then we test what’s called the first publishable, which is typically one complete level of near-publishable quality with all of the controls and visuals in place – a vertical slice of the game. This comes about halfway through the development lifecycle. We test it with 10 people playing simultaneously, but separately from one another.
Following the test we have them complete a questionnaire. We ask primarily about usability concerns, but also look at appeal to some extent. We finish the testing by holding a roundtable with those people, though we’ll sometimes do individual interviews as well.
In the end, we document all of the issues we’ve found, make a top ten list from it and say okay, these are the things we want to fix for the next time round. We’ll do testing 2-3 times in the lifecycle with 10 people each time, bringing in different demographics.
Alongside that, Mark Cerny had laid the foundations for testing at Sony in the US over the last 10 years. They’re doing a lot of data capture as well, recording how long it takes to complete each level and looking for things like death clustering, which is when everyone’s dying at the same places in the game. There’s now a strong culture of user testing at studios including Foster City and Santa Monica, and that’s been a big part of enabling them to increase the quality and relevance of the titles they produce.
And what’s the future of the EyeToy?
The PS3 now has its own USB camera — the PlayStation Eye, which was released with a game called Eye of Judgment that recognizes playing cards placed on a surface.
We’ve also continued releasing EyeToy games for the PS2. Two of them make use of an evolution of the EyeToy color tracking technology that identifies both motion and color, which gives a much deeper level of control. One is EyeToy: Play PomPom Party, which is a cheerleading game that comes with a set of pom poms, pink and green. Along with that is EyeToy:Play Hero, which comes with a green sword that you use to fight your way through the game. On screen the sword can light up, take on flames, and so forth.
And we’ve just announced EyePet, which is generating some excitement. You might want to take a look at the video.
Games as production
Filed under: Uncategorized | Tags: fold.it, games, IA Summit, interface, protein folding
Comments (1) In my presentation at the 2008 IA Summit, I discussed how many human activities can be understood as games, and benefit from adopting their characteristics. When we think of games as being specifically unproductive, we’re missing the opportunity to engage users at a level beyond what can be achieved in more conventional interfaces.
In fact games can serve as catalysts of production. Take fold.it, which is a puzzle game that challenges players to find the best ways to fold proteins. This is in fact among the most difficult problems in modern biology, as a protein can take on very different characteristics depending upon its shape. For example, mad cow disease is caused by proteins that already exist in the body, but which have been folded into irregular shapes that make them agents of the disease.
A screenshot from fold.it
People who play fold.it are actually contributing to science, because the game uses the real physical properties of the proteins as its rules. Players are awarded points for things like reducing the size of the protein efficiently, or turning certain types of molecules so they all face inward. The New York Times notes that it’s plausible that by playing this game, you could actually win a Nobel prize (even if you know nothing of biochemistry).
The real pioneer in the productive use of games, though, is Luis Von Ahn of Carnegie Mellon University. I’ll discuss his work in depth in an upcoming posting.
Subscribe to this blog