This Trading Card Game AI Watches and Learns from You

"The goal of a good artificial intelligence isn't to win, it's to lose convincingly," Chris Woods tells me, and suddenly, years of frustration spent trying to conquer indomitable computer-controlled opponents in video games begins to make sense.

Chris Woods, the lead engineer behind the digital card game Hex: Shards of Fate, might be the most experienced developer of artificial intelligences used in trading card games (TCGs), a lifetime of work that stretches back over 20 years. That claim is oddly specific, but that's partly due to the fact that Woods is one of the first pioneers in a whole new frontier. Woods, and the rest of the team at Cryptozoic Entertainment, aren't just looking to reimagine how people play these games, but revolutionize the genre itself.

The AI in Hex isn't just playing the game, it's also watching and learning. Every one of the millions of games already played have been recorded by the AI, which is aggregating all of that information to better understand how the game is played. When certain card combinations become popular, the AI will begin to anticipate and prepare to counter them.

Like Blizzard Entertainment's wildly popular Hearthstone, Hex is a card game where players duel one another with decks constructed from a vast array of different cards. But Hex takes things one step further by blending concepts prevalent in games like World of Warcraft. Hex: Shards of Fate labels itself as a massively multiplayer online trading card game (MMOTCG), creating a digital space for players to battle while also building a rich world full of characters to meet and quests to complete. It's every bit as much of a role-playing game as it is a card game.

"It opens up an entirely new way to play," Jones says. "We were really hungry for a TCG that allowed us to do something different. Just playing other people can be a ton of fun, but going into a dungeon—that puzzle solving and creativity is really gratifying." That's where Hex: Shards of Fate's artificial intelligence is of crucial importance: In order to tell an epic story, you need a great cast of villains."

Historically, TCGs have always struggled with creating clever AI opponents. The layers of random chance and complex decision-making are a nightmare for relatively lightweight video game AIs. Many compensate by loading the AI's deck with powerful cards that aren't accessible to the player or allowing the computer to peek at what cards a player might be holding. In a sense, they cheat.

But Woods has a different vision, one that involves an AI that not only understands how to play the game, but transcends being seen as just a machine. That dream was born when, as a kid, Woods started his love-hate relationship with Sid Meier's Civilization, the famous turn-based strategy game where players reenact the rise and fall of alternate-history versions of Earth's great empires.

"I learned very quickly that, no matter what, Gandhi is going to nuke you," Woods says. He describes how every encounter with Gandhi in Civilization brought out his worst. "I'd see him appear on my screen and just be like, aw fuck!"

In those moments, Woods realized that Gandhi was more than just lines of code defining a set of logic, he was a personality capable of evoking an emotional response. "That's what I wanted for Hex," he says.

If a player were to fight a wild boar in Hex, they can assume the boar would attack without much strategy, it might go straight for their health points rather than trying to gain the advantage through careful use of its cards. An undead lich, however, is a much more intelligent monster, and players might second-guess a move knowing that the lich could be laying a trap. "Being able to mine that information from your opponent gives such a unique experience to a strategy game. We don't want to boil it down to just feeling like a numbers game," Woods says. "I think AIs have been largely created to challenge the player instead of help tell the story of the entire game."

From what I've played of Hex so far, the AI is actually quite good at believably adopting the personality of opponents you face. During one encounter with a swarm of piranhas, the board quickly filled with cheap but deadly piranha cards and I was helpless to defeat them. It quite literally felt like being overwhelmed by a swarm of vicious fish.

Understanding how the AI works in Hex is no easy feat. Essentially, Hex's AI evaluates every card active on the board and in its hand, assigning them a numerical value that is positive for its cards and negative for the human opponent's cards. The AI can then begin to evaluate possibilities by calculating whether or not a move will increase the value of the "game state", thus tipping the scales in its favor or pushing it to be more negative, presumably benefitting the player.

From there, the AI begins to calculate theoretical options, a massive tree of branching possibilities that it assigns probabilities to in order to determine what the most likely outcome will be. After evaluating each potential branch, the AI can decide whether or not a move is going to be beneficial or harmful to its chances of winning.

Though there are methods like "Monte Carlo", which runs a sample size of simulations to approximate probability, or "alpha-beta pruning", which immediately removes a branch of possibility when one of its sub-branches is found to be weaker than a previously examined branch, neither were satisfactory enough for Woods. So he decided to create his own unique algorithm instead.

For Woods, it wasn't good enough to create an AI that was smart and efficient, it also needed to be dynamic. "You never want your AI to be deterministic—people don't realize it, but they figure it out. You 'get it' and you become bored because it's no longer surprising you."

To prevent this, the team came up with basic templates for each type of AI opponent—like a beast template that would guide the logic of a wild boar. They also created "thunks", smaller subroutines in the code which gives the AI special tendencies. One of those thunks is "kneecapping", which encourages the AI to always thwart an attempt by a player to use spell cards to increase the strength of one of their troop cards.

When an AI is evaluating its choices and assigning each card a value, those values can be manipulated by certain personality quirks that Woods programs the AI to choose from. If the player is fighting an elf character who has the "sentimental" quirk, she might overvalue certain cards because she doesn't want to see those cards die. Each AI template has a list of quirks it randomly chooses between each battle, meaning a character might be sentimental during one game but carelessly aggressive in the next.

"It's all very behind the scenes," Woods says. "It's not in your vision, but at the same time it is affecting you as you play through the game."

The AI in Hex will also adapt to shifts in the way people play. Every move played in Hex is stored on a table and given a "half-life"—a decaying value that determines how relevant information becomes over time. By analyzing these trends, treating the most recent as the most important, the AI can begin to respond to popular combos or strategies.

It's a smart solution, but not one without it's problems: "The biggest flaw of this approach is whenever we release a new set [of cards], the AI is going to have a week where it's just struggling," Woods says. During that week, Woods fills the table with his best guesses of what will be popular, but it's hardly a replacement for genuine data.

With all this effort—much of it to create minutiae that a player might never pick up on—I was curious to know whether Woods believes his creation is a success. "I think it actually really has been," he says. "They're not playing a video game anymore—they're having an emotional experience."

"We don't always want to make you happy—that's boring." Woods describes the joy he gets while reading posts from players talking about the ways the AI thwarted them, often referring to it as a character rather than as the computer.

The AI in Hex: Shards of Fate is still within its earliest stages, but Woods and Jones are breathlessly excited about its future potential. The team at Cryptozoic already has plans to borrow more ideas from MMORPGs to create even more challenging and unconventional encounters. Now that the framework is all built, it's up to them to test the limits of what they can achieve.

"There's so much that Hex can do," says Jones. "There's so much invention. You're really having an experience unlike anything you've ever seen when you play."