Domain Authoring Study

For my Ph.D. work I developed a method of summarizing clusters of stories (narrative plans), in order to help domain authors better understand what kinds of stories are possible. To evaluate the technique, I made a tool that allows people to configure a story world and explore it by clicking on clusters and reading example stories. You can try it yourself using this link:

https://code.rac7hel.com/domain-authoring-study-demo/

At the bottom of that page (after the Intro), you can toggle between the three different versions of the tool: The Circles and Tree versions each use a different type of cluster summary, and the Control version uses no summary.

The point of this is that it’s really difficult to anticipate exactly what stories are modeled by a narrative planning domain, even if you wrote it yourself! I asked people from all three groups whether or not specific stories were possible in the world they had just configured. My summary tools helped people answer these questions more accurately.

You can try this by clicking the “Answer 14 Questions” button at the bottom, once you’ve configured the world however you want it. After you answer all the questions, it will show you what percent you got right. (This is just for demo purposes; I’m not collecting any more data.)

Feedback and questions are always welcome. Thanks for visiting!

Salience Vectors for Measuring Distance between Stories

Narrative planners generate sequences of actions that represent story plots given a story domain model. This is a useful way to create branching stories for interactive narrative systems that maintain logical consistency across multiple storylines with different content. There is a need for story comparison techniques that can enable systems like experience managers and domain authoring tools to reason about similarities and differences between multiple stories or branches. We present an algorithm for summarizing narrative plans as numeric vectors based on a cognitive model of human story perception. The vectors encode important story information and can be compared using standard distance functions to quantify the overall semantic difference between two stories. We show that this distance metric is highly accurate based on human annotations of story similarity, and compare it to several alternative approaches. We also explore variations of our method in an attempt to broaden its applicability to other types of story systems.

Full Paper:

PDF

Dataset:

Our dataset of human assessments of story similarity is archived here!

Citation:

Rachelyn Farrell, Mira Fisher, Stephen G. Ware. Salience vectors for measuring distance between stories. In Proceedings of the 18th AAAI international conference on Artificial Intelligence and Interactive Digital Entertainment, pp. 95-104, 2022.

[bibtex]

Salience as a Narrative Planning Step Cost Function

Psychological research has demonstrated that as we experience a story several features affect the salience of its events in memory. These features correspond to who? where? when? how? and why? questions about those events. Computational models of salience have been used in interactive narratives to measure which events people most easily remember from the past and which they expect more readily from the future. We use three example domains to show that events in sequences that are solutions to narrative planning problems are generally more salient with each other, and events in non-solution sequences are less salient with each other. This means that measuring the salience of a sequence of actions during planning can serve as an efficient cost function to improve the speed, and perhaps also the quality, of a narrative planner.

Full Paper:

PDF

Citation:

Stephen G. Ware, Rachelyn Farrell. Salience as a narrative planning step cost function. In Proceedings of the IEEE Conference on Games, 2022. (forthcoming)

[bibtex]

Multi-Agent Narrative Experience Management as Story Graph Pruning

In interactive narratives, experience management is used to control the world and the NPCs a player interacts with, encouraging particular types of stories or discouraging others. The space of all stories in a narrative can be understood as a story graph, with story states as nodes and actions in the story as directed edges. In this paper, we present experience management as a graph pruning problem. Starting with the full story graph, edges representing NPC actions may be pruned until there is at most one action per NPC per state. With the full graph available, the choice of what to prune may consider all possible futures, and we can ensure that undesirable stories are not reachable. By never pruning player actions, we ensure the player may make any choice and still be accommodated in the story. When this method was used to manage the story of an adventure game, players found our technique generally produced higher agency and more-believable NPC behaviors than a control. Finally, we discuss scaling the results of this method for practical use.

Full Paper:

PDF

Citation:

Stephen G. Ware, Edward T. Garcia, Mira Fisher, Alireza Shirvani, Rachelyn Farrell. Multi-agent narrative experience management as story graph pruning. IEEE Transactions on Games, 2022. (forthcoming)

[bibtex]

Narrative Planning for Belief and Intention Recognition

Planning algorithms generate sequences of actions that achieve a goal, but they can also be used in reverse: to infer the goals that led to a sequence of actions. Traditional plan-based goal recognition assumes agents are rational and the environment is fully observable. Recent narrative planning models represent agents as believable rather than perfectly rational, meaning their actions need to be justified by their goals, but they may act in ways that are not optimal, and they may possess incorrect beliefs about the environment. In this work we propose a technique for inferring the goals and beliefs of agents in this context, where rationality and omniscience are not assumed. We present two evaluations that investigate the effectiveness of this approach. The first uses partial observation sequences and shows how this impacts the algorithm’s accuracy. The second uses human data and compares the algorithm’s inferences to those made by humans.

Full Paper:

PDF

Citation:

Rachelyn Farrell, Stephen G. Ware. Narrative planning for belief and intention recognition. In Proceedings of the 16th AAAI international conference on Artificial Intelligence and Interactive Digital Entertainment, pp. 52-58, 2020.

[bibtex]

Manipulating Narrative Salience in Interactive Stories using Indexter’s Pairwise Event Salience Hypothesis

The salience of a narrative event is defined as the ease with which an audience member can recall that past event. This article describes a series of experiments investigating the use of salience as a predictor of player behavior in interactive narrative scenarios. We utilize Indexter, a plan based model of narrative for reasoning about salience. Indexter defines a mapping of five event indices identified by cognitive science research onto narrative planning event structures. The indices—protagonist, time, space, causality, and intentionality—correspond to the “who, when, where, how, and why” of a narrative event, and represent dimensions by which events can be linked in short-term memory. We first evaluate Indexter’s claim that it can effectively model the salience of past events in a player’s mind. Next, we demonstrate that salience can be used to predict players’ choices for endings in an interactive story, and finally, we demonstrate that the same technique can be applied to influence players to choose certain endings.

Full Article:

PDF

Citation:

Rachelyn Farrell, Stephen G. Ware, and Lewis J. Baker. Manipulating Narrative Salience in Interactive Stories using Indexter’s Pairwise Event Salience Hypothesis. In IEEE Transaction on Games, vol. 12, num. 1, pp. 74-85, 2020.

[bibtex]

Playable:

Here are playable versions of each of the five Twine stories described in the article:

1. The prediction one (Section IV.2 in the article) – Here we successfully predicted people’s final choice based on the event indices of their earlier choices.

2. The first attempt at influence (Section IV.3.A, the no choices one) – We railroaded people through the beginning of the story, and expected them to choose the ending we targeted, based on what we had learned in the prediction study. But they didn’t!

3. The second attempt at influence (Section IV.3.B, the unrelated choices one) – Here we added some arbitrary choices throughout the story, in case the lack of interactivity was throwing off our game. But it still didn’t work!

4. The third attempt at influence (Section IV.3.C, the low agency choices one) – Realizing that it mattered which events people were given choices about, we engineered choices for the four key events used by our hypothesis that wouldn’t break our experimental design. I was really confident this was going to work, but once again, it did not!

5. The fourth attempt at influence (Section IV.3.D, the higher agency choices one) – Having finally really learned something, we made a slight adjustment to our experimental design that allowed us to give more “meaningful” choices for the key events, but still railroad them into our targeted ending… And IT WORKED!

Combining Intentionality and Belief: Revisiting Believable Character Plans

In this paper we present two studies supporting a plan-based model of narrative generation that reasons about both intentionality and belief. First we compare the believability of character plans taken from the spaces of valid classical plans, intentional plans, and belief plans. We show that the plans that make the most sense to humans are those in the overlapping regions of the intentionality and belief spaces. Second, we validate the model’s approach to representing anticipation, where characters form plans that involve actions they expect other characters to take. Using a short interactive scenario we demonstrate that players not only find it believable when NPCs anticipate their actions, but sometimes actively anticipate the actions of NPCs in a way that is consistent with the model.

Full Paper:

PDF

Citation:

Alireza Shirvani, Rachelyn Farrell, Stephen G. Ware. Combining intentionality and belief: revisiting believable character plans. In Proceedings of the 14th AAAI International Conference on Artificial Intelligence and Interactive Digital Entertainment, pp. 222-228, 2018.

[bibtex]

Causal Link Semantics for Narrative Planning using Numeric Fluents

Narrative planners would be able to represent richer, more realistic story domains if they could use numeric variables for certain properties of objects, such as money, age, temperature, etc. Modern state-space narrative planners make use of causal links—structures that represent causal dependencies between actions—but there is no established model of a causal link that applies to actions with numeric preconditions and effects. In order to develop a semantic definition for causal links that handles numeric fluents and is consistent with the human understanding of causality, we designed and conducted a user study to highlight how humans perceive enablement when dealing with money. Based on our evaluation, we present a causal semantics for intentional planning with numeric fluents, as well as an algorithm for generating the set of causal links identified by our model from a narrative plan.

Full Paper:

PDF

Citation:

Rachelyn Farrell, Stephen G. Ware. Causal link semantics for narrative planning using numeric fluents. In Proceedings of the 13th AAAI International Conference on Artificial Intelligence and Interactive Digital Entertainment, pp. 193-199, 2017.

[bibtex]