Tuesday, June 30, 2009

User Interface in Interactive Storytelling

It seems that I've gone AWOL for about a year. I don't think anyone is particularly saddened, but I have some more posts to do now that I have the time.

I have been doing research on Interactive Storytelling for the past few months in preparation for my senior honors dissertation in computer science. One of the things that I've been researching, originally for the aim of finding examples of how stories can be formalized, is semiotics. I ended up coming across a website for Algebraic Semiotics, a project in which sign systems are mathematically formalized as the basis for creating a science of user interfaces.

Having now seen the use of a mathematical application of semiotics to both the structure of a story and creating user interfaces, I was brought back to a central problem of interactive storytelling: creating seamless interaction between the user and the story that she* perceives. This "perceived story" (the way in which the user interprets what comes onto the screen) is linguistic, not spatial (my interactive storytelling engine is language-based) and must therefore be able to allow the user to give feedback appropriate to the words that appear on the screen. It's important to note that the design needs to be geared towards how the reader reads into a given sentence as far as short-term statements go; intent means little in design and nothing in aesthetics.

What this pointed me to is that the interface demands for an interactive storytelling engine are unusually high. What the user sees is the only thing that the user is going to get, because the entire story is a real-time experience. This is a lot different than say, Microsoft Word, in which the final product (in hard copy, for simplicity's sake) is independent of what the user sees on the interface. All the interface is, in the context of wanting to print a hard copy, is something on the surface that gives clues as to what you will get for a final product. In Interactive Storytelling, the output of the interface is the ultimate end.

There is an additional demand, then, on the interface to an interactive storytelling engine. There has to be a harmony between the internal state of the software and the interface in which the interface could hardly be seen as a layer of buttons and switches; perhaps not entirely—the computer could be imagining and entirely different scenario than the person is and the person could still be getting output that makes for a good perceived story; but in terms of functionality, the interface has to have some kind of direct correspondence.

In addition, the user is not only pushing buttons on the screen for some type of desired feedback, she's also pushing these buttons with her input as part of the experience of the story. The perceived story is the entirety of the user interface. Anything going on below the boilerplate is propping up this real-time surface interaction.

To put it another way, every decision the user makes has to have results that "make sense" to her (I say "makes sense" because there are some things where it may not make sense to the user because she is not thinking it through in people terms, such as if in a romance story, she punches her friend in the face and then wonders why he called her a bitch*; on the other hand, to fit my example, it should never happen that she punches him in the face and he does not react.) Notice how by "react", it means that something appropriate be displayed on the screen.

Because of the need for functional equivalence between the internal engine and the interface (if the interface is doing one thing but telling the user another, and it works the same as if it were thinking the same thing as the interface, then it is functionally equivalent to thinking the same thing as the interface), the engine has to think in terms of the interface (a.k.a. in terms of the story that the user is perceiving.) Therefore, the engine must think in the same language that the interface is using, as each language has a unique syntax and thus expresses its own set of ideas in its own way. The engine has to think in this language and then speak back in this language.

This does not necessarily mean simply "English"; books themselves have a language that defines their ideas. This means that the dramatic space that the engine explores has to be in the literary language of the story, which will make for a very different way of formulating a world. Nonetheless, it is the only way to be in line with the user's perceived story—there cannot be some hidden "spatial universe" independent of the linguistic interface as there is no tidy metaphor that exists between these conceptions of the world. Trying to translate between the ideas of the spatial (or any other space of ideas) and the space of the story's dramatic language, that which the user inputs to and sees output from, would lead to an entirely dissatisfying story.

I will say quickly that I don't mean to invalidate having no basic conceptions of space and time: there is a deep structure to drama, I believe, just as there is a deep structure to lingual syntax and so it would make sense for the computer to receive a sentence, interpret it into some sort of "semantics", come up with new "actions" and then translate said "actions" back into words; but we must be careful to remember that first and foremost the engine is producing a sentence.

I'll try to summarize all this here. The chronologically ordered set of all inputs and outputs that the user enters and perceives respectively is the entirety of the story. Therefore, the engine must think entirely in terms of these elements, as the story and the interface are one in the same. This means creating an engine that sees stories in a dramatic language and expresses them in that same language, and a giant headache over the course of the next ten months.





*Why did I use the "she" pronoun? Two reasons. 1) I refuse to write "he/she", the political-correctness police try to ignore the fact that gender seems to naturally arise in language and sometimes in far more ridiculous ways than English speakers can imagine (if you don't believe me, take a look at Swahili.) 2) Nonetheless, my projects are meant to appeal to a base that is not the traditional masculine gamer, and so I'd like to keep as much emphasis as possible on that. Just a note.

*I warned you, I'm not politically correct.

Monday, July 21, 2008

More on Agency

I forgot to write a few things in my last post, instead giving into pretentious waffling. I'll list them down best I can:


-More replayability means more outcomes, which means that there's more that the player affects. This is what I mean by the richness of the interactivity, there's true agency. If there's more agency, then it means that there is more for the player to explore, and a larger space for the player to learn.

-This also means that there is more to be understood for the system that underlies the game. If the game is mastered quickly, the challenge is simple, there is nothing for the gamer with the hungry intellect and imagination. On the other hand, a challenge that can be played over and over is symptomatic of a more mentally challenging game-world.


Hopefully, I'm stepping away from vagueries with this post; but it's early in the morning (at least for a 20 year old), so I can't make promises.

Thursday, July 17, 2008

Why Agency Matters

Let me start this post with an excerpt from Italo Calvino's Invisible Cities:

Arriving at each new city, the traveler finds again a past of his that he did not kow he had; the foreignness of what you no longer were or no longer possess lies in wait for you in foreign, unpossessed places.

Marco enters a city; he sees someone in a square living a life or an instant that could be his; he could now be in that man's place, if he had stopped in time, long ago; or if, long ago, at a crossroads, instaed of taking one road he had taken the opposite one, and after long wanering he had come to be in the place of that man in that square. By now, from that real or hypothetical past of his, he is excluded, he cannot stop; he must go on to another city, where another of his pasts awaits him, or something perhaps that had been a possible future of his and is now someone else's present. Futures not achieved are only banches of the past: dead branches.

...And Marco's answer was: "Elsewhere is a negative mirror. The traveler recognizes the little that is his, discovering much he has not had and will never have."




I'll disepense with rhetorical questions and get right into what the point of that long quote is. There's a large emphasis on the notion of "replayability" in a game. Allow more characters, more plots, more bonuses, something that accomodates more outcomes? These seem like things that are parts of replayability, or may boost it in some sense, but none get to the heart of how to get replayability and why it matters so much.

The first obvious thing is that we can enjoy a game longer if its replayable. I don't think this is the real importance behind replayability though: a really long single-player campaign or RPG can start to get quite repetitive if it goes on too long, it gets tedious. If I have to play 50 missions in an RTS campaign, I'll probably get sick of it, I'm doing the same thing over and over again. When the single player mode gets really long, the same tasks repeat themselves over again, and this really is no different than replaying the same challenge, and one that you'll eventually get sick of. If that's the case, then should we use that as the basis for replay value? It seems more likely that we're just being forced by the overarching "plot" (should we be interested, or feel the compulsion to complete the plot set out by the designers) to replay the game as many times as it sees fit.

So let's forget plot for the moment. I get sick of an RPG once I've fought too many turn-based battles, sick of a real-time strategy campaign once I've played enough missions. It's the same challenge repackaged multiple times for all intents and purposes. I'll now focus on games like Civilization, in addition to skirmish modes in RTS games, fight games, sports games, etc. These seem to have differing levels of replayability, and they must have a reason for why that is. I've written about it before; we start a game not knowing anything about it, and understand it a little better each time we play it. The result is playing to better understand a paradigm each time, and gaining new satisfactions as it goes on. That's why long after one's forgotten their copy of Grand Theft Auto, they may still be playing games of StarCraft over the internet, continuing to come up with new strategies. It doesn't take long to get fully acquainted with all the ways to fight in GTA, and missions will probably stop being amusing once the fighting ceases to be any kind of a fresh challenge. StarCraft, on the other hand, allows for much more innovation in how to overcome the opponent, there are many ways a game can go and many things to try. Games like Civilization seem to take this richness to an entirely new level. A dynamic system is there for a player to explore. The opposite of this would be a puzzle game like Myst, once you've solved all of the puzzles, there's no way to challenge yourself in replaying the game. Zero replayability.

How long we can enjoy a game is important, but what's also important is that replayability is very strong indicator of richness in a game. If it's replayable, then there's something to be done that wasn't done last time. If there are that many possibilities, then there is that much more in the game. Replayability means a game with more to explore, whether it's a new strategy, a unique level, a specific chain of events, or anything else that's new to the player. In real life, people continue to study music, science, art, philosophy, etc. because there are constantly new surprises, and the richness of these endeavors helps to create great minds.

My pomp aside, I mean to say a more daring and wishy-washy concept. Art holds in it the mysteries of life, of the self, of tradition, of the future and past, etc. These are all mysteries. Mystery and the unknown seem to define art (I say all of this taking myself with a grain of salt), and the mysteries of the unknown seem to be what are both beautiful and haunting about sights and senses that evoke emotion. The haunting concept brought forth by Italo Calvino about the infinite past and future seem to be ahead of his time in describing who we are and how we are defined. The ability to change how we experience a world and come to understand it with each runthrough is a powerful one that does justice to the concept that more exists in what we can't see than what's observable. The game world needs to be overwhelmed with possibilities in order to maintain a rich flavor, one that gives the player an ability to feel wonderment when they travel through it.

Monday, May 19, 2008

A Different Approach to Cultural Conquest

I thought of an idea ago for how to integrate culture into a Civilization-style game without making it seem forced and calculating, something that just sees to take part of the point out of the whole attention to culture in the later versions of Civilization, as well as any other games that might be taking into account such an idea.

Instead of building culture objectively through different actions and then getting a predictable result of borders expanding and other nations subverting to the influence of your own civilization, I suggest creating an emergent-art engine. This is a strange idea, but it basically goes along the lines of algorithmic art and music being created through various algorithms; you (the player) can acquire more of these by means of improving technology, productivity, trade, and openness of information; and of course, art from other countries that reaches your own. Players could then look to buy and collect these works in order to further their own goals. Perhaps one piece of art will allow them to create something that will influence other players of different taste more, or perhaps its more preferable than another one that would lead to complications. Alone, the variations may not mean anything, but if players were trading art from the beginning, strategies will emerge as to which cultures it is worthwhile to trade with, and the art itself will have more validity based on its value relative to the game being played. Also, as the algorithms become increasingly complex, the possibilities will grow as to what can be created.

Some of the preferences with which people trade will probably be based on taste or arbitrary, but a little bit of entropy never hurt a game. There also could be an interesting collection system with a marketplace that varies the benefits of such trade. I probably should have evaluated and developed the idea more before posting, but it's good to see where a discussion should lead.

The general idea however does remain: creating a system of culture linked to diplomacy, technology, trade, and information/openness. The products that emerge from these are basic when they start, but the exchange of these products leads to new avenues in creating better pieces of art, which would then allow a player to focus their time on creating specific schools of thought that generate art that puts them ahead of the game in terms of culture. The art takes on its value based on the choices of the user, leaving it partially random, but mostly based on the strategic choices of players as they try to maximize their own influence.

What does the art do though? That's important. I think one could run with this idea in many directions. The one I had in mind however was that the market-based value of the art affects its influence, and should high-value art from one player be found influential in another player's country, that player will find himself able to push more influence on the people of that player. This is all an oversimplification, but it's the best I have for now.

Now, up for discussion: What's your take on the idea? How would you run with it?

Monday, April 14, 2008

Not-So-Spatial Spaces

Let's think for a minute about how a game works. There are rules, and objects, and objectives, and we have to know the best possible way to play by the rules so we can complete the objective. How do we do that? We play with it, and we get a better and better mental model of how the game works, which in turn allows us to react faster to things in the game, and make better choices in our strategy.

For this reason of course, it's important to have a good interface. A person should be given a consistent and exact model of what is happening in the world; spatially graphics need to be consistent. A monster's avatar should increase in size consistently with each step it takes closer to you, a fixed rate of change; units in strategy games need to be represented correctly in how far apart they are from each other. It's completely rigid and mathematical.

So why am I posting about this? Anyone can see that in order to figure something out, feedback needs to be consistent. On the other hand, what does this kind of an interface suggest? It suggests that pretty much every game we play is of a consistent formula, and that we're experimenting with it until we boil the game down to pure consistent logic. Nothing wrong with that, there needs to be some basis for every system, however:

Why does our information need to be represented so spatially consistently all the time? Well, no other reason than that the logic of most games is so damned reliant on spatial factors. One of the reasons that books, plays, movies, etc. can be so imaginative is that there doesn't need to be this precise idea about space. As long as it looks feasible, we can assume that they're in a real-world situation, and their ideas don't have to be subordinate to the tedious and boring parts of reality.

We also don't always need to be loaded with information about hit-points, damage, bonuses, etc. Take the first Command and Conquer: even if there was no information on hitpoints and damage, it took about one failed assault to realize that you shouldn't send infantry against flame-based weapons or a group of undefended tanks into a field crawling with rocket launchers.

So I'm getting on a tangent away from my larger point, let's get back to it. The key information of many games shouldn't require such exact feedback on the screen. I'm not talking about first-person shooters or platformers, those are games that are about space; but for many other games, we're talking about how objects are related to one another, how one player interacts with another, alliances, etc. If we want games to move further towards concepts like story, relationships between characters, human behavior, etc; then there's no point in thinking so hard about things like space. In fact, I don't want to see such constraints on space in a game of this type because everything else risks becoming subordinate to it. On the other hand, having a space to move around in within a game makes it feel a lot more real. So, here's a proposal for the best of both worlds; take a look at this application of dynamic programming to shrink an image with 'retargeting':

http://www.milkandcookies.com/link/66481/detail/

Notice how there's no need for proper scale and distance between objects when they shrink it down. We probably wouldn't notice that so much was changed if we were just looking at the pictures casually. It preserves the exact same idea, and does a better job of it than if we scale it, where parts of the picture gradually become obscured. What I'd like to see is an RPG (or something like it) that follows this model. We can slow down or speed up the character accordingly to make sure that the player doesn't just shrink the screen to speed up movement (I think everyone would agree that that would mess with suspension of disbelief). This way, the player feels like he's moving in the real world, but we have a lot more flexibility with artistic representation of the game. What feedback on space the player is getting isn't important, this isn't a strategy game where we have to know where to place artillery and how close the enemy is to our base, we're simply traveling a world where we talk, buy, explore, and so on.

This image retargeting might be a bit of a silly example, on its own it doesn't mean much, but it also brings across a general point. The algorithm is able to preserve a sense of reality without being constrained by space. In doing so, it does a lot better of a job of making the shrunken picture convey the important information. We need to do this more often in video games, or else the important information will always have to do with space, something that really only matters if we're testing our ability to understand how things move in a space. Doing something else will bring the player's cognitions elsewhere and expand the medium. This clearly isn't off limits to video games, we know enough about computer science to make space look continuous without letting it dictate the rules of the game.

Friday, April 4, 2008

The Body

I was reading over Steven Poole's archives when I saw an article he wrote for Edge Magazine about the ways in which video games have been giving their characters real bodies for the first time. Snake, in Metal Gear Solid 3 can now can be poisoned by his food, continues to hurt until he puts a bandage on his cuts, has to eat, and heals as he rests. Unlike many characters, he seems to have a body, not just an avatar.

This is definitely an interesting step foward in games, but I was going to bring up the broader point that it seems that we have no conception of a body in video games most of the time. Instead, it's always an avatar with data stored inside of it; health, items, ammunition, an alias, but no real sense of an existence that precedes this.

And as we hurtle through the digital age, that we're thinking of ourselves more and more in terms of a conscience that floats around from place to place. After all, we spend a lot of our time on our cell phones, or on the internet, where any sense of our body being tied to our identity is gone. I oftentimes find myself looking in the mirror and then realizing "oh wait, that's me?" So is there not some social implication to playing an online game?

When we start to play a game and get engrossed, our sense of self most definitely changes. Our set of functions becomes the logistics of the game itself. We're more ethereal in our connection to what we're controlling in the game; when I play StarCraft I am as much the marines, battlecruisers, and tanks at my command as I am my body when going through my normal life. At the moment it's all I'm thinking about, and it's how i'm defining my position.

So perhaps it's worth it to think about how much most video games rely on a sense of essence. I think that there's a lot to be added to the experience of a game by enriching the player's sense of existence; give him flaws, give him an itch on his back, give him body fat and dirt under his fingernails. It's a wild guess I admit, but I think that we can add a lot of emotion to games doing this; every art form has a set of aesthetics, and this could be part of our own. I'm talking about the jump from 3d models with "hitpoints" to humans beings capable of being hurt, not damaged; growing, not gaining experience; living and breathing. From there we can also lay off of the mind/body duality: if we can connect the player to the character through a functional body, then the player can experience being the character instead of the character being a subservient representation of the person playing, and this is important if we want to bring the player completely into a different world.

It's all a wild guess, and I admit this isn't my most concise or well written entry, so send some comments and I'll be happy to discuss. My general point though is that we need to give the player a real sense of existence to move games forward, not just an aethereal essence.

Saturday, March 22, 2008

Dramatic Systems and Interactivity

The Modern Drama scholar H.L. Lucas suggested in his book "Ibsen and Strindberg" suggested that Henrik Ibsen, in his well known play The Wild Duck that there was a clear change in his beliefs and methods, turning away from his usual morally driven plot-based dramas, to an emergent plot formed out of interactions and tangents that opened up new doors; characters were no longer accountable for upholding moral imperatives imposed on the text, but instead for the well being of themselves and others.

On The Wild Duck in particular, Ibsen states that "Truth is an excellent thing, but not always. Much human happiness is based on a merciful illusion, a 'life-lie'." For those who haven't read it, the plot can be reduced quite simply. It involves a man whose father has destroyed the lives of a few, but has sheltered them and rebuilt their lives to a modest life that keeps them off of complete destitution; his son however finds out the truth and feels the need to tell it to those whom his father damaged, only causing ruin wherever he goes. In simple terms, he's an idiot.

From this I've been gathering the idea for a new project, perhaps one small enough logistically to allow some experimentation. Now, bear with me as I begin to rant: In The Sims, a game that I'm assuming anyone reading this would have a fairly decent knowledge of, we have a simulator that focuses on the basics of human relationships through short term actions but also occupies the user with paying the bills, eating, going to work, defecating, etc. etc. The Sims however is in this sense more of a dollhouse, one in which the user's imagination fills in the blanks of the story. In this project however, I would take an approach that I feel is much more suitable to this idea of psychological drama, one in which we have everyday actions facilitate deeper interactions that affect human interaction, namely the exchange of information.

This runs counter to another man in the field of interactive storytelling, Chris Crawford, who doesn't care much for spatial configurations in storytelling. In his software, The Erasmatron (a piece of software I cannot claim to have used, so feel free to correct me if I have any misconceptions to anyone who has used it), he creates a world in which spatial configurations do not play any role, but instead events are stripped down to their bare significance within a story-world. My model runs, as I have said before, counter to this by using spatial and non-dramatic factors as a lubricant for drama, and to make drama that is convincingly incidental instead of straightforward; the model would run as follows:

Within each character we have a set of personality traits that affect how they react to a given situation. Also within the storyworld is a subtext of what has happened, and what each character knows. This is somewhat tricky, in order to facilitate interesting gameplay, the right characters have to know the right thing. As we know from my synopsis of The Wild Duck above, if we start out with the idiot knowing everything and it's over in a pretty short period of time. Predisposed personal relationships may also be somewhat essential, but these perhaps could be modeled off of a combination of the subtext and personalities, with a minimum of history.

We have then a stage that serves as a series of potential triggers. There should be no need however, unlike The Sims, to guide characters, or to ensure their survival by making sure they're well fed (or making sure that they don't soil themselves in the event of a bathroom line), instead we let the characters have short-term desires to eat, to sleep, to watch TV, along with mood factors that will determine how they feel if these are not met. If they're in a bad mood, they may just react differently with others. We have a stage then that creates motion, which will cause characters to "bounce" off of one another, which will in turn cause information (and lies) to be exchanged, in addition to events that could cause turns. The data then that is mostly stored in characters besides their personality factors (which dictate how they primarily react to events), would be what "history" they have with other characters, and the information that they hold. These two things of course would affect one another in how characters further act.

This of course is a rough idea of what such a stage could be like, there may not be need to let mood so drastically affect the outcome of what a character says; but on the other hand, it's more likely that given what kind of engine we want, that we could customize what affects mood the most to allow some themes to start going. In The Wild Duck for one thing, Ibsen uses the lightning through a giant sloped window affect the mood of each theme, perhaps a day/night cycle may be appropriate. These are not part of the greater picture however, so I digress...

So one thing I haven't covered then, how do we make this into something playable? It sounds like a bunch of characters walking around going about their business. As I always feel about these things, we open up many doors for a place to allow interaction, the question is what strings can the player learn by pulling. We could have the user be a character; this was my first idea, which gave me the ad-hoc name for the project "The Thirteenth Guest", a reference to the penultimate line of The Wild Duck, basically personifying one of the protagonists as a troublemaker and a wrongdoer for his spouting of the truth and consequent upsetting of life. The challenge would be for t he user to understand what information is held by whom, and how these characters react to what they know, or may know. The gameplay then is to be able to analyze interaction and leave the right footprints in order to gain a mastery of the characters, or not. I admit that I feel this is too warlike of an objective, some extra touch of motivation could be needed.

So more likely may be to give the user a variety of scenarios that he could play through as whatever type of agent he pleases. The point of this game is more about observing behavior than any sort of "puzzle", so giving the user more options would allow him to experience this kind of gameplay in any flavor he liked. Maybe the scenario would be a sort of god-mode, and see how the same characters with the same information differ when things are set up differently or one more character comes to the party. Details again; there are too many rough edges that will become painful problems when I try to implement it, so in further posts I will try to focus more on the lasting process-intensive points than how this could turn out in implementation.