Human Factors in Simulation

June 11, 2013


Medical mannequin simulator (photo: US Navy)

Human factors are extremely important in simulation, as this 2009 book points out. Human-factors expertise is important both in simulating human beings and in the use of simulations by human beings.


Physical human limits (image: AnyBody public repository)

Issues involved in simulating human beings include physical and ergonomic factors, as well as human behaviour modelling (the subject of this workshop). In training simulations, it is important to fully understand the human process which is to be improved by simulator training. This can include subtle issues such team interaction, as well the more obvious factors.


Pilot landing cues? (screenshot from FlightGear)

There is also a plethora of human-factors issues in the development, design, conduct, debriefing, and debugging of simulations relating to the use of simulators by human beings. Negative training, for example, occurs when users of a training simulation learn the wrong knowledge, skills, or behaviours. This can be the result of low-fidelity representation of important decision or feedback cues, of timing delays, of incorrect or incomplete problem representations, or of other simulator design flaws. It is impossible to build a simulator with 100% fidelity, and even 99% fidelity would be prohibitively expensive. To achieve the required outcomes, where is high fidelity necessary? Human-factors expertise is essential in answering that question.

Simulator sickness affects many users of flights and vehicle simulators, and limits the potential benefits of such simulators. See this 2005 study for an overview of research in this area.

In training simulations, a variety of cognitive-psychology factors also come into play. Likewise, in decision-support simulations, it is important to understand the limits of the conclusions that can be drawn. Which results tell us meaningful things about the problem at hand, and which results simply reflect characteristics of the simulation?

Simulation is an extremely valuable tool for both training and decision support. Yet, for the best results, it is important to take into account human factors in both the design and the use of the simulation.


Vehicle simulator (photo: US Army)

– Tony

Advertisement

2 Comments | Human Factors, Simulation | Tagged: , , , , | Permalink
Posted by Tony

Human Behaviour Modelling Workshop – 16 Sept

June 3, 2013

My interest in human behaviour modelling is no secret, so it’s no surprise that I’m excited to be running a half-day workshop on the subject at this year’s SimTecT conference in Brisbane.

The workshop will cover key issues and major steps in human behaviour modelling, including practical “how to” advice, data collection issues, verification & validation, and some common pitfalls. Applications to Defence, Mining, and Health industries will be covered (depending on the participants). Some practical examples written in NetLogo will be given, although the techniques presented will be relevant to any simulation system.

Update: Unfortunately this workshop has been cancelled.

– Tony

1 Comment | Human Behaviour, Simulation | Tagged: , | Permalink
Posted by Tony

“Where’s the tea?” – Simulating human behaviour

May 3, 2013


The tea

In Douglas Adams’ famous The Hitchhiker’s Guide to the Galaxy, some of the characters discuss the replacement of Arthur Dent’s brain by an electronic one:

“‘Yes, an electronic brain,’ said Frankie, ‘a simple one would suffice.’
‘A simple one!’ wailed Arthur.
‘Yeah,’ said Zaphod with a sudden evil grin, ‘you’d just have to program it to say What? and I don’t understand and Where’s the tea? – who’d know the difference?’”

But is that true? Can simple computer models adequately simulate human behaviour?


How will a crowd of panicked humans flow down this fire escape?

In fact, it depends on the goal of the simulation. In models of pedestrian flow, work by Dirk Helbing and others has shown that quite simple models can perform very well, particularly in simulating evacuation dynamics and similar panic-driven scenarios. In these situations, simulations not much more sophisticated than simple fluid-dynamics models can reveal the benefits of, for example, zigzag designs for evacuation routes. See Helbing & Johansson, “Pedestrian Crowd and Evacuation Dynamics” (Encyclopedia of Complexity and Systems Science, 2009).

Adding more sophisticated decision-making allows us to build agent-based models of economic behaviour. Will people purchase a particular product from a particular vendor? Will vendors alter their prices up or down to match other vendors? Alison Heppenstall, Andrew Evans & Mark Birkin provide a nice example of such modelling, by simulating the spatial variability in petrol (gasoline) prices in “Using Hybrid Agent-Based Systems to Model Spatially-Influenced Retail Markets” (Journal of Artificial Societies and Social Simulation, 2006). Relatively simple models of behaviour also suffice for epidemiological models (which were discussed at the 2012 Workshop on Verification and Validation of Epidemiological Models in Washington D.C.).


Anasazi ruins, Southwest USA

One of the most well known examples of agent-based modelling using this approach is the insightful study, by Robert Axtell et al., of ancient Anasazi population dynamics in the Southwest USA. In this case, behaviour in the model was synthesised from archaeological evidence, anthropological data, and rational decision-making – households will pack up and move out if they’ve seen too many bad harvests in a row. See “Population growth and collapse in a multiagent model of the Kayenta Anasazi in Long House Valley” (PNAS, 2002), and “Understanding Artificial Anasazi” (M.A. Janssen, Journal of Artificial Societies and Social Simulation, 2009).

Related modelling methods are used in studies of land use. Will farmers switch the crops they’ve been planting? Will they fell trees in the neighbouring forest? Will they abandon farming altogether and move to the city? Alex Smajgl and others discuss approaches to such modelling in “Empirical characterisation of agent behaviours in socio-ecological systems” (Environmental Modelling & Software, 2011). Grace Villamor, Meine van Noordwijk, Klaus Troitzsch & Paul Vlek, in their paper “Human decision making for empirical agent-based models: construction and validation” (International Congress on Environmental Modelling and Software, 2012), compare the strength and weaknesses of heuristic versus optimal decision-making in models. It may be difficult to accurately capture human heuristics, but it cannot necessarily be assumed that humans will always make the “best” decision.


The emotions of fear and joy: The Rescue by John Everett Millais

The choice between heuristic and optimal decision-making in models is complicated further when the humans being modelled make decisions on emotional grounds. Stacy Marsella and his team at the University of Southern California have had considerable success in modelling human emotion. One very successful use of their approach has been the tactical language training software marketed by Alelo, which also incorporates game technology. For details, see Johnson & Valente, “Tactical language and culture training systems: Using AI to teach foreign languages and cultures” (AI Magazine, 2009). Further development of this approach is likely to have several interesting applications.

For practical purposes, then, we can simulate human brains by electronic ones. But they will not necessarily be simple.

– A significantly expanded version of this post will appear as an article in the Summer 2013 issue of the Society for Modeling & Simulation International (SCS) Magazine.

1 Comment | Human Behaviour, Simulation | Tagged: , | Permalink
Posted by Tony

Emergence, Intelligence, Networks, Agents: WEIN

December 1, 2012

Networks are ubiquitous, and often large. The WWW contains over a trillion pages. The Internet contains over 900 million hosts. Facebook has over 900 million active users. The human brain contains over 80 billion neurons. The world has over over 7 billion people.

The behaviour of such networks is determined not just by the behaviour of individual nodes, but also by the network topology. The famous six degrees of separation often hold, for example. Network dynamics and self-organization are also important.

From the nodes and links of these large networks, fascinating things emerge. Nobody expected what happened when a small internal network at CERN was scaled up to planetary size, for example. And somehow, the interactions of our billions of neurons make us intelligent.

The 5th International Workshop on Emergent Intelligence on Networked Agents (WEIN’13) will be exploring some of these phenomena at AAMAS in Saint Paul, Minnesota next May.

The WEIN 2013 Call for Papers is out, and submissions are due January 30. It is likely to be an interesting event, and one that should help to build bridges between the multi-agent system and complex network communities.


Bridging disciplines: the PLoS One Map of Science

– Tony

Leave a Comment » | Human Behaviour, Research, Simulation | Tagged: , , | Permalink
Posted by Tony

Calling the Race

November 15, 2012

The big winner from the US Presidential election has been Nate Silver and his 538 blog. On the eve of the election, Silver had calculated the following probabilities from polling data:

Silver also has a fascinating analysis of which polls were accurate, and which weren’t. In particular, polls based on calling landline telephones tended to seriously underestimate the Democrat vote. Voters without landlines are more likely to be young, urban, Black, Hispanic, strapped for cash, or some combination of the five, and all five categories are more likely to vote Democrat. There are lessons here for pollsters in other countries.

The following simple NetLogo simulation model (click on the image to run it) re-rolls Silver’s electoral dice, giving alternative outcomes – exactly the kind of simulation Silver actually did to support his 90% prediction of an Obama win:

Silver appears to take a Bayesian approach to statistics. A Bayesian has been described as “one who, vaguely expecting a horse, and catching a glimpse of a donkey, strongly believes he has seen a mule.” The legendary XKCD summarises the perspective quite elegantly:

– Tony

3 Comments | Human Behaviour, Simulation | Tagged: , | Permalink
Posted by Tony

Benefiting from Board Games

November 5, 2012

Playing the Ticket to Ride board game

I’m a big fan of board games, especially the newer German-style board games, which are far superior to the games of my youth. This book argues the benefit of modern board games for learning and teaching, highlighting some of my favourites.

Board games are more than just entertainment. According to a press release, a lecturer at the University of Tennessee has won an award for using the Ticket to Ride board game to teach operations research to students. This makes good sense, since German-style board games tend to involve complex optimisation decisions.


Some Dominion cards (photo: Shannon Prickett)

Consider a massively simplified version of the enjoyable game of Dominion, for example. There are six kinds of card: copper money (costs $0, worth $1), silver money (costs $3, worth $2), gold money (costs $6, worth $3), estates (cost $2, worth 1 point), duchies (cost $5, worth 3 points), and provinces (cost $8, worth 6 points). The real game has many other interesting cards, but even this simple parody is non-trivial.

At each turn, the player draws a hand of five cards (from a deck of initially ten), purchases a new card, and discards the hand. When the deck is empty, the discard pile is shuffled to form a new deck, so each purchased card will be “used” multiple times (which is why it’s sensible to spend $6 on a gold card worth $3). However, only the green cards are worth points – as with many German-style games, the money does not directly contribute to winning the game. On the other hand, purchasing many green cards reduces the chance of a five-card hand containing much money.

One strategy is to only purchase money or the valuable province cards. A simple simulation of the game shows that, after 50 turns, this results in an average score of 112 points. In contrast, a strategy of preferring to buy green cards gives an average score of only 39. However, “switching” from one strategy to the other does best of all, with an average score of 128 when the “switch” is made at turn 35 (and an average score of at least 120 when the “switch” is made somewhere between turns 25 and 45). In other words, winning requires optimising when the strategic “switch” is made.


Average scores, as a function of when the strategic “switch” is made. Switching at turn 35 is best.

For full-blown German-style games, the optimisation problems are more difficult. As this podcast argues, they are often in the difficult class of problem called NP-complete. These are hard enough to challenge both a human and a computer.

As I’ve said before, I also have a long-standing interest in collaborative board games, such as Arkham Horror, Pandemic, and Lord of the Rings (for fans of Arkham Horror, here is one of my custom characters, with back story and marker). Collaborative board games offer an excellent way of both exploring and teaching teamwork, and a 2006 paper by José Zagal and others explores Lord of the Rings in this context.


Reiner Knizia’s Lord of the Rings collaborative board game

Good board games generate the level of engagement that makes wargaming work. When the team is attempting to solve a difficult optimisation or decision problem (as in SCUDHunt), things can get very interesting indeed.

– Tony

2 Comments | Collaboration, Decision Making, Gaming, Groups, Simulation, Teams and Groups | Tagged: , , | Permalink
Posted by Tony

Epistemology and Simulation

September 28, 2012

Epistemology is the study of knowledge. What is knowledge exactly? Well, I’m happy (for reasons argued elsewhere) to use the definition going back to Plato, that of justified true belief. For example, I know – or at least believe – that there’s a tree growing outside my window.


The tree outside my window (my photo)

Ultimately, this belief is grounded in the way the human visual system works, and on the way in which my perception of the tree triggers remembrance of trees past. All this falls within the scope of cognitive psychology, and experimental work in this area has told us a great deal about how human perception works.


The human visual system, from Gray’s Anatomy, 1918

Is my belief true? You’ll have to judge that for yourself (although the photograph may help convince you). Is it justified? Well, that’s the domain of philosophy – am I justified in trusting my senses?

In his Confessions, Saint Augustine takes “seven and three are ten” as a touchstone of truth, and in his City of God, he writes “the man who says that seven and three are eleven, says what cannot be true under any circumstances.” I agree with him. Here again, my belief falls within the scope of cognitive psychology (and developmental psychology, since Cuisenaire rods helped convince me of this back in kindergarten).


Seven and three are ten

Is my belief true? Once again, judge that for yourself. Is it justified? Well, that’s the domain of mathematics this time (and, as an older child, I learned to prove 7 + 3 = 10 mathematically).

Shortly, I hope to attend the 5th Epistemological Perspectives on Simulation (EPOS) Conference at Trinity University (San Antonio, Texas). We will be exploring whether it is possible to know things (especially things about social phenomena) as the result of a simulation (and, if so, how). It promises to be an interesting event. Papers from two of the four past instances of this conference series can be found here (2004) and here (2008).


(Public domain photo)

– Tony

Leave a Comment » | Psychology, Simulation | Tagged: , | Permalink
Posted by Tony

Games and teamwork

August 16, 2012
I Had Trouble in Getting to Solla Sollew

I Had Trouble in Getting to Solla Sollew (book cover)

One of my favourite children’s books is I Had Trouble in Getting to Solla Sollew by the great Dr Seuss. It contains that wonderful description of teamwork:

“… ‘This is called teamwork. I furnish the brains.
You furnish the muscles, the aches, and the pains.
I’ll pick the best roads, tell you just where to go.
And we’ll find a good doctor more quickly, you know.’
Then he sat and he worked with his brain and his tongue
And he bossed me around just because I was young.
He told me go left. Then he told me go right.
And that’s what he told me all day and all night. ”

Is that really teamwork, though? There are requirements on true teamwork, and these requirements include processes that are (at least to some degree) democratic. But apart from that, what does a good team really look like?

In a 2006 paper in Simulation & Gaming, José P. Zagal, Jochen Rick, and Idris Hsi explore Reiner Knizia’s “Lord of the Rings” board game, in which a team of up to five people play the roles of hobbits intent on destroying Sauron’s ring.


Reiner Knizia’s “Lord of the Rings” board game

Zagal et al. find that this collaborative board game models four aspects of more realistic teams:

  1. There is a genuine tension between perceived individual utility and team utility.
  2. Players may, if they wish, take actions which go against group consensus.
  3. Players are able to trace outcomes (good or bad) back to player decisions.
  4. Players have varying strengths and weaknesses in the context of the game.

These characteristics make the “Lord of the Rings” board game, and similar collaborative board games, fascinating tools for exploring teamwork at its best (or at its worst), and for exploring strategies for improving team effectiveness. And they’re fun, too!

– Tony

7 Comments | Collaboration, Gaming, Groups, Human Factors, Simulation | Tagged: , , , , | Permalink
Posted by Tony