Game Theory and Uncertainty

This goes beyond just Tabletop RPGs, and is much less organized or fruitful than most of my posts. Hope you find it interesting though. I apologize for anyone who hasn't played many of the games mentioned in this post, but take it as a list of recommendations. Well, except for Puerto Rico. Fuck that game.

Prologue

This is the "Tower of Hanoi." It's a classic mathematical puzzle game you may have seen before. In case you haven't, I'll explain the rules here. If you want, you can play the game online at this website.

The player has three standing rods and a number of disks. The disks all vary in size and are stacked on the leftmost rod in order of biggest to smallest as you go up. The more disks there are, the more difficult the game. On a turn, the player can move one disk from its current rod to either of the other two rods. It slides as far down as it can go. Here's the limitations: you can never stack a larger disk on top of a smaller disk, and the disk you move must always be the top of the stack on whatever rod you're moving it from. Your goal: move the entire stack of disks over to the rightmost rod, which inevitably requires that they also be stacked in the same arrangement (since, per the limitations, they must go in order from large to small).

Give it a shot.

How'd you do? Probably pretty well, assuming a small number of disks. In fact, you likely attained the goal easily enough that the true metric of success that would interest you next is seeing if you can win in the fewest number of moves possible.

Here's the thing I want you to take away: this is a solved game. Moreover, you are fully capable of solving it yourself, most likely. The minimal number of moves required to solve a Tower of Hanoi puzzle is 2^n − 1, where n is the number of disks. There's a fairly simple algorithm for what move to make each turn so that you always follow the optimal sequence. The Wikipedia article has it written out in a few forms and includes some animations, but it's actually one that many players are able to just "intuit" on their own.

The Tower of Hanoi isn't an excellent game. It's a one-time novelty puzzle that is interesting to mathematicians and occasionally useful to neurologists when used as a test. But I am telling you about it to use as an example of a type of game that can take many, many forms, far more expansive in scope than this simple thing.

Board Game Randomness

I like board games. But seemingly most board game hobbyists are obsessed with "Eurogames," typically characterized by (among other things) intense resource management gameplay. Settlers of Catan, Ticket to Ride, Power Grid, Puerto Rico, and so on are emblematic of the style. I usually tell people I dislike this genre, but that's always a bit of a lie. I have a lot of fun playing Concordia, but moreso because I find it relaxing and easy to maintain conversation with friends during gameplay. The real reason I claim to dislike these games is because I have this horrible, futile urge to "solve" them.

I never win these fucking games. And every time I play one, I find myself methodically considering every single decision with all the brainpower I have at my disposal, both for their immediate results and in forming a long-term strategy. And when I inevitably lose, all I can think about is how to do it right next time. I go over every single last decision I made in the entire game and think about what I could have done differently, and if I'm feeling ambitious then I might even begin formulating entirely different approaches for my next attempt. And, without fail, the next time I play and try correcting my mistakes, I lose again.

I am a competitive person, and I really, really love winning board games. But in all of my favorite games, I usually also love losing, because victory was not really the primary source of fun for me. But when I'm playing Ticket to Ride, I find that there's... not really anything else to latch onto. And, although I may sound like a dingus saying this, I do feel like my brain is fucking hard-wired for this type of resource management gameplay. Which makes it all the more frustrating and unsatisfying.

Time and again, in my analyses, I return to the same conclusions: there's always an X Factor that keeps things out of my control, so what's the point of strategizing? I recently started playing the game Wingspan with my D&D group and we all had a lot of fun. The bird theme is delightful. But sure enough, I went through my perpetual cycle. And my friends seemed to agree with my main evaluation: the outcome was due more to random chance than any amount of skill. Your fate is decided entirely by the cards you draw. While there are optimal decisions about what to do with the hand you've drawn, that'll rarely save you from someone else just being luckier. Every time we played, the person who did win felt as though they didn't truly earn their victory. They couldn't credit their success to a superior strategy, because it was basically arbitrary.

Of course, good games shouldn't be solvable. After the first time or two that you've played the Tower of Hanoi, you have no reason to ever play it again. Once you've internalized the correct answer, you'll never be challenged again. So the game needs randomness, right? But what do we do when randomness is a problem?

This speaks to a greater issue with the role of random chance in games. By far my favorite piece on the subject is "The Two Types of Random in Game Design" by Youtuber Mark Brown (about 20 minutes long, I highly recommend it). I'll try to re-hash the parts relevant to what I want to say here. Basically, there are two kinds of randomness in games: input randomness and output randomness.

1. 
   Input randomness is when the random event happens before the player makes the relevant decision. An example is when you play a card game like Magic: the Gathering and draw a hand of random cards to start with, or perhaps when you play a videogame with a procedurally-generated map like Minecraft or a Roguelike. A player's skill is measured in how well they can optimize their decisions when given a hand of resources they couldn't have planned for. I love this kind of randomness when done well because I always feel like it empowers a player to use pure knowledge. You can't memorize plays because you can't anticipate the scenario. Instead, you have to truly understand the game. You master the underlying factors so that you're ready for anything the randomness throws at you. The problem occurs when the degree of randomness has a much bigger impact on the results than any decisions you could have made, such as in Wingspan or Monopoly.
2. Output randomness is when the random event happens after the player makes the relevant decision. An example is an attack roll in many tactical combat games like D&D or X-COM. First the player decides to make an attack, then a die is rolled to see if they succeed. This is the kind far more likely to piss players off and is trickier to implement wisely, but it's not worse. It basically means that, rather than figuring out the exactly perfect steps to take, you have to account for risks. Someone good at calculating for risks can still meaningfully control their own fate so long as they have the chance to make enough good decisions that the aggregate results will be equally good despite variation in individual decisions along the way. For every critical fumble you get, there should be roughly one critical hit. An example I use when playing D&D 5E is whenever I roll my Hit Dice to heal during a short rest. Because you're rolling to heal, you don't know the perfect number of HD to spend. The fear of rolling low might tempt you to just roll more dice, but spillover HP isn't saved once you hit your Max HP, and then you've wasted a Hit Die. So I calculate the odds: if my Hit Dice are d10s then the average roll is 5.5, and I add my Constitution bonus of +3 to each die. Thus, my average HD roll is 8.5. You can't rely on this average if you only roll one or two HD, but if you roll, like, ten of them then they'll trend closer to this average in the aggregate. So if I'm 60 HP away from my Max HP, I roll 7 dice (60/8.5=7.06) and voila: I will get, almost without fail, within 4 or 5 HP of my target. For every die that came up a 1 or a 2, it's okay because there's probably a die that came up as a 9 or a 10. [NOTE: This isn't actually necessary because you are allowed to roll your HD one at a time if you want. I'm just impatient, and this proves that I could still make smart decisions even if the rules didn't give you that leeway]

Randomness is used to provide variety in games. If the setup is always identical and the results of decisions are always identical, then presumably all the decisions will inevitably trend towards an identical, "ideal" strategy.

So we have a very fine needle to thread here. I don't much care for an approach to games that invites that awful temptation to "solve" them. I am inexorably drawn towards "system mastery" and I hate it. I don't like when people attempt to make "tier lists" for D&D classes and I don't love when you can't play a competitive video game without people insisting on a toxic culture of optimized builds and plays. I don't want the games I play to just turn into a Tower of Hanoi puzzle. But at the same time, I also hate feeling a lack of control over the outcome of my decisions. I want to still feel like I'm exhibiting meaningful application of skill somehow.

Fighting Game Strategy

My thoughts actually first turned to this subject when discussing with my co-writer the design of his in-development Kung Fu RPG Rivers & Lakes (which I've brought up on the blog a few times now). He wants to brand it as a "tabletop fighting game," like an analog equivalent of Street Fighter. One of its foundational mechanics is the addition of "reactions" for every single targetted action. The idea is this: during combat, on your turn, if you use your action to target an opponent with a strike or a stunt, then they get to choose how they react (typically with a strike or stunt of their own, although there are a couple options that can only be used as a reaction and never as an action). The order in which they resolve depends on the action and reaction chosen. There's a hierarchy of "priority" for every option in the game which you can learn pretty quickly (attacker wins ties).

This is how all combat works all the time. Person decides action, target decides reaction, then they resolve in order of priority. If you are fighting 100 goons, then on your turn you can use your one action to attack one goon and he'll get a reaction. But on their turn, if all 100 goons use their action to attack you then you get 100 reactions, just like how it works in a wuxia movie. Rather than the overwhelming importance of "strength in numbers" that defines D&D's combat system, you actually just look way more badass the more enemies you're fighting in this game.

Thus, the primary calculus of the game is figuring out 1) the best reaction to counter or nullify the action being used against you, and 2) the best action that will still have a meaningful effect in spite of whatever reaction may be chosen by your opponent. We have found during playtesting that this is, if you'll permit me, a fucking awesome backbone for a tabletop combat system. There's a limited number of strikes and stunts to choose from, of course. Hypothetically, once a person plays enough combat they could "solve" any given scenario if they know their opponent's strengths and weaknesses well enough. But the whole point is that the core rules are merely a chassis. Once you add on kung fucking fu atop that, the possibilities are endless.

Here's where we reach a dilemma. My co-writer, the designer of the game, was learning more about fighting game design and seeking to draw deeper inspiration from it. He was struck with an idea: what if, instead of choosing your reaction after learning the action you're being targetted with, you had to choose blindly? Imagine the attacker declaring who their target is but without saying how they're attacking. The defender chooses how they react, and the two of them simultaneously reveal. Both he and I have recently been getting much more interested in "simultaneous resolution" mechanics in various forms, but I won't get into all my ideas about that here. Here's the relevant logic:

In the current system, the target has a huge advantage over the attacker, right? They have the benefit of knowing what the attacker is going to do to them, so they can always just choose the optimal counter. He posits that this is too unbalanced. He argued in favor of the inherent strategy of Rock, Paper, Scissors. I protested.

"Oh come on. There's hardly any depth to Rock, Paper, Scissors. That's, like, the whole point. It's nearly random."

"Wrong. Rock, Paper, Scissors has tons of depth. Have you ever looked into the kind of high-level, competitive play that people will do? There's actually a lot of strategy." 

It's worth noting that he's not entirely wrong. Here's a good video about it. But I continued:

"Oh hush. It's all relative. It doesn't really have much depth, it just has more depth than you'd expect. That's not the same thing. At the end of the day, no matter how much you think about each move you make and how many moves ahead you're considering, there's still only three possible outcomes. We've all been there before. You start thinking like, 'so they'll probably throw Rock, which means I should throw Paper. But they know that I suspect they'll throw Rock, which means they'll actually throw Scissors. So I should throw Rock. Except they probably suspect that I suspect this, so they'll...' And the answer you come up with will repeat every 3 steps of the thought process. You might have thought 12 steps ahead and decided to throw Scissors and you won. But it's possible that your victory wasn't because your opponent only thought 11 steps ahead, but rather only 8 steps ahead or 4 steps ahead... or even 14 steps ahead!"

We didn't belabor the Rock, Paper, Scissors debate much further. He wanted to talk about fighting games. He claimed:

"Even if we accept that Rock, Paper, Scissors is functionally random, you can use that system and still make a good strategic game. Fighting games are really fast-paced. Unlike in Rivers & Lakes, they (basically) have simultaneous resolution."

"What are you talking about? You absolutely read your opponent's moves and decide how to counter. Every move has a windup animation!"

 "Yes, but 1) a windup animation is so quick that you might not process it fast enough for it to inform your reaction, and 2) they're often reused across multiple moves or at least made to look similar. The reality in play is that competitive fighting game players are basically guessing what their opponent is going to do and react based on that guess, not on firm knowledge."

I was skeptical. I'm not convinced this is the same degree of calculated choice that you get in Rock, Paper, Scissors. In a fighting game, you aren't just guessing. You have metagame knowledge of the kinds of moves and combos your opponent can do with their character, plus knowledge of which ones that specific player prefers to use. Your decision is informed by more than a mere guess, I think. It's certainly more meaningfully informed than in Rock, Paper, Scissors. But the important thing is that he feels this could be replicated in Rivers & Lakes.

"As the target, you shouldn't have the benefit of knowing what action is being used against you. You should have to guess."

"That feels completely disempowering to me! Their action could be anything! You'd basically be making a random decision, and you're almost certainly going to make a bad one!"

"No, because if you know your opponent's strengths and weaknesses then that can inform your guess as to what action they'll use against you. It reinforces the importance of using knowledge about your opponent. Just like you said in fighting games, it's not actually a blind guess. You know things about the other player."

We've both agreed that one of the game's best strengths is how well it simulates the "know your enemy" trope from kung fu media. It's one of my favorite things about it. But I remain unconvinced. In my view, I don't think that's enough information to empower the reacting character to make a smart decision. More importantly, I don't actually think they have an unfair advantage in the current version, where they know what the attacker is going to do before they decide how to react. In fact, I think it's actually almost perfectly balanced.

This is because there are no perfect counters. No matter what, that benefit of knowledge can't completely trump the attacker's strategy, because at the end of the day you still have to roll some dice. The target may choose the optimal reaction and still fail because even the optimal reaction might be hampered by a low roll. Meanwhile, there are lots of terrible counters. That is, situationally speaking. Every action in the game has a circumstance or two where it would be an optimal counter to something. But many of them will be useless against most actions. For example, "disarm" is a great reaction if your opponent is trying to strike you with a weapon attack (and even has priority over it! You might disarm your opponent before their attack resolves, nullifying it entirely!). But it has precisely zero benefit if your opponent tries to 1) hit you with an unarmed strike, or 2) do something completely different, like grabbing, sweeping the leg, throwing, charging, etc. We agree that there's a vast range of actions at a reactor's disposal, so some amount of information is needed for them to narrow their options. But I don't think that merely knowing the opponent's stats and fighting style is enough to go on, and my co-writer thinks it might be.

Here's the thing though: in the time since this discussion, we haven't playtested it. I could be completely wrong. Maybe the "simultaneous reveal" won't break the strategy at all, and then it'll be an improvement because it adds an awesome thrill every attack. Or maybe he is wrong about the balance and simultaneous reveal will upset a good system, but the added fun of the surprise will be a worthy tradeoff. Or maybe I'm right and it's best not to rock the boat. Here's the moral of the story:

How the fuck do you know the right balance of "decisions available" and "information to inform it"??

Oh shit, that's not a moral at all. That's just a frantic, desperate question I find myself cursed with thinking about constantly because every goddamn game I play reminds me of this problem.

Other Possible Sources of Uncertainty

RPGs are complicated. Let me return to board games for a moment. I was discouraged by Wingspan, thinking that randomness was ruining my agency, but also knowing that a game with no uncertainty is a game that cannot last. What about the board games that I do enjoy? What elements did they use to solve this problem?

Well, probably my favorite type of board/party game is the "Hidden Role" genre. Mafia/Werewolf, Secret Hitler, Among Us, Deception: Murder in Hong Kong, Town of Salem, and the like. What keeps those from being totally solvable? Well there's a bit of input randomness here and there, but mostly it has to do with 1) deliberately hidden info, and 2) the inherent unpredictability of player psychology. The distinction is pretty fuzzy, I'll admit. Technically, the outcome of any random event is also a form of "hidden info," but at least you can make some calculations before hand and you will learn the info soon afterwards. In Secret Hitler, the info that's hidden doesn't get revealed until the game ends (for the most part). And while player psychology is always a source of uncertainty in competitive games, it rings more true for games where the headspace that players occupy has less to do with resource management and number crunching and more to do with lying, convincing, and negotiating.

One of my favorite board games, and one that's apparently done a pretty good job not being solved after all these decades, is Diplomacy. It's funny because it's one of the few games I know of that really strives to merge the "game theory" of political science/international relations with the "game theory" of, y'know, hobbyist gaming. Sure enough, it's all about hidden knowledge, player-to-player negotiation, and simultaneous resolution. Because of that, never does a single goddamn random element ever need to enter the equation whatsoever. But the game remains imperfect. There is a very well-known meta at this point, and far too many trends that can only be bucked by players who are intentionally subverting them, despite knowing them to be superior strategies. The only time Italy ever lasts into the endgame is if other players took pity and allowed it, and the only time you don't see a "Juggernaut" play (i.e. Russia and Turkey alliance) is when the Russia player and/or Turkey player decide they feel like switching it up and giving everyone else a chance.

So we have randomness, fog of war, and player psychology as possible elements that add uncertainty. In video games you also have a player's "physical skill" (so to speak): their ability to actually execute their decisions. When you're playing Super Smash Brothers, it's a very different thing to know the best next move than it is to actually make that move successfully. Unfortunately, this type of skill cannot easily be tested in most tabletop contexts. True, there is the genre of "dexterity games" like Crokinole, Jenga, and Flick 'em Up. But most of these games, while quite fun and thrilling, are also not terribly deep and strategic. They often only test your dexterity, because identifying the (hypothetical) best move in a turn of Crokinole is trivial. Pulling it off, however, is maddening. "Physical skill" is a mainstay of video games because it usually revolves around things like perception, reflexes, inputting long sequences without error, coordination of multiple body movements simultaneously, fine control of movement, and sometimes even endurance.

Wow. I somehow made video games sound athletic compared to tabletop games.

Another source of uncertainty you find a lot in video games is just updates. Let's say, for the sake of argument, that you create a game that can absolutely be "solved" by dedicated players. There's a healthy range of possible challenges as well as possible decisions to make, but a smart player can learn the system well enough to always optimize their moves. There simply is a "right" answer for those who spend the time to attain system mastery. Well, what if you introduce balance patches and add new content into the game? Then, you can reset the conversation. Just when the fans think they have it figured out, you just change the playing field and make them re-calculate. Lots of competitive games include an intentionally evolving metagame as part of the design.

Of course, regular updates aren't as easy to implement in tabletop, nor can they be done forever.

Okay, sure, but what about Goddamn CHESS?

That's when it hit me. There's a game with none of those elements. No randomness, no hidden knowledge (beyond simply not knowing the future), no true negotiation or playing with incentives, no physical skill, and... well, no updates since the year 1860. And yet, that shit remains unsolved!

To be clear, it is known to be mathematically solvable. Computers will probably do this eventually. But it isn't exactly close to being done, and the human capacity to solve the game is almost certainly impossible. Despite having an identical setup every single time with a limited number of possible moves every time, with everything transparent to both players and the results of every decision always being identical... it's still not a game with one "right answer" that a player can devise just by thinking on it. It has had a continuously evolving metagame for centuries now, and is genuinely fucking thrilling to learn and get better at. Sure, the pros can see a path to victory a good 20 moves ahead, but so long as you play someone of roughly equivalent rank to your own then you get a consistently challenging experience, since you're both matched in the skill of optimizing your decisions despite the game's uncertainty.

I get that Chess isn't for everyone, and while I assume that Go also probably fits all these same traits I'm describing, that's not necessarily for everyone either. But I am simply astonished at how effectively, no, effortlessly Chess manages to accomplish that very thing I set out to look for in this post without relying on any of the proposed elements of uncertainty. No matter what, whoever wins can always say with 100% confidence that they earned their victory through superior skill at the game alone.

Why isn't this just like the Tower of Hanoi if it has no randomness or fog of war or anything? Here's my theory: it still has sufficient uncertainty, but more because of the vast range of possible board states than any other source. Sure, the early game often looks pretty similar across most games. There are only so many possible first and second moves you can make. But the possibilities quickly, and I mean quickly, multiply out of control. By the 15th move of the game, it is an almost mathematical certainty that you're looking at a unique board state, never seen by any players before (unless you're both intentionally reenacting a known tournament game for some reason).

But didn't I say earlier that too many possiblities disempowers your agency? If there's too many moves you could make and not enough info to narrow it down, then your decision is functionally random. But the key here is that, on any given turn, you still only have a handful of legal moves you can make at that moment.

Not only that, but the source of uncertainty ("what will my opponent's next move be?") is small yet constant. It happens between every single one of your decisions, but it'll never shatter every idea you had about the future of the game unless you just aren't good at planning. Thus, we have a seemingly perfect balance of too vast and uncertain a possibility space to solve the game as a whole, but a narrow enough playing field at each individual decision that you can meaningfully strategize in the moment to moment.

By this criteria, Chess is pretty much a perfect game. But you know what's weird? One of the other games I thought about is almost the opposite of Chess entirely, and yet, is also nearly perfect by this same criteria.

That's right, it's Cosmic Encounter

Cosmic Encounter is one of the most unbelievably fun board games in the world. It's complete madness. No two matches are alike, and there's so much uncertainty that you'll probably begin experiencing derealization by your third turn. And yet, it's also deeply strategic. You do have control, you can make meaningful strategic choices, you can make smart plans, and you can usually credit yourself for victory.

It has basically all of those sources of uncertainty I mentioned. Randomness, fog of war, simultaneous reveal, player psychology and negotiation, deception, a nigh-infinitely expanding possibility space, and at least 3 players competing at once (typically more). And yet, it's not just chaos. It somehow threads that fine needle I've been describing.

I think part of the reason must be that the whole game is built on a fairly solid "core chassis," not unlike Rivers & Lakes. When you teach someone the game, you start by explaining the default victory condition, basic cycle of play, and all the most commonly seen moving pieces. That alone makes for a pretty good game, almost as "purely strategic" as Chess.

Then you tell them about the alien races.

"In addition to all that, you get to pick an alien race to play as from one of two or three randomly drawn options."

"Sounds cool! What do they do?"

"They're completely overpowered."

"What?"

"All of them. Every single one. They’re all OP and they all break the game."

I don't know who came up with the idea for a game that basically boils down to, "as unbalanced as humanly imaginable, but equally unbalanced for every playstyle" but they are a fucking genius.

And yet, counterintuitively, the addition of the race powers never actually renders that core chassis irrelevant. The main cycle of play is never too disrupted, instead always remaining the primary factor in all your strategizing. The default victory condition is still the most commonly achieved one, and you're forced to keep an eye on it throughout the game. Deep understanding of how to win in the "core game" is still the best foundation for any winning strategy, and the difference in skill from there just depends on who know how to account for all the race powers the best.

Conclusion

My hypothetical ideal game is one in which I am constantly challenged to figure out the best set of decisions, and that I feel I can credibly do so in spite of however much uncertainty is necessary to keep the game from being too easily "solved." I think the two extremes would be Rock, Paper, Scissors vs the Tower of Hanoi. And the examples I've found which manage to satisfy me... well, they strike a "balance" somehow, but in completely different ways. Chess minimizes uncertainty and Cosmic Encounter maximizes it and they still both have a high degree of "strategic integrity." I think Rivers & Lakes currently has the right balance, but how do you know?

I can say this for sure though: when playtesting a game, this is possibly the most important lever you can adjust. If you aren't sure the right thing to tweak, always ask yourself: "do players have too much information or not enough information? Do they have too much uncertainty or too little?"

-Dwiz