Power and Win Rate of the Decks – Analyzing the Official “Tier List” From Iksar

This week we got a meta update from Dean Ayala (Iksar) concerning their stats on the most popular and best performing decks (source).

Standard Meta Update:

(Last Full Week, All Regions, Rank 5-Legend, Alphabetical Order)

Top 5 Population
Embiggen Druid
Galakrond Rogue
Galakrond Warrior
Highlander Hunter
Highlander Rogue

Top 5 Win-Rate:
Aggro Hunter
Combo Priest
Highlander Hunter
Mech Paladin
Quest Hunter

— August Dean Ayala (@IksarHS) January 30, 2020

Upfront I’d like to say that I assume all this data is true. No one should know what people are playing and what is winning the most more than Blizzard themselves. They have all the data, and I have no reason to assume they’d just make anything up. These are probably the stats anyone would see if they had the same data and ran the same analysis. 

But something seems off, doesn’t it? Why are the most common decks people are playing the ones which don’t seem to win the most, according to Blizzard’s data? Perhaps players don’t know what they’re doing. Perhaps their perceptions are skewed and there is a lot we don’t accurately understand about the meta. Maybe players just follow the flavor-of-the-week streamer decks. Perhaps it’s all those nasty meta reports put out by third parties that everyone flocks to, like HSReplay and Vicious Syndicate.

I have a different theory. I don’t think players are badly wrong, I don’t think Blizzard’s data is off, and I don’t think the meta reports are wrong, either. 

To examine what’s going on, I’m going to start with a statement that might sound wrong at first glance. After I say something that sounds a little crazy sounding, I’ll explain why it’s actually not crazy at all in several different ways. Ready? Here goes:

A deck’s actual win rate does NOT describe a deck’s actual power.

What I’m saying is that a deck which wins, say, 55% of the time, may not be a stronger deck than one which wins 50% of the time. This has to do with something like variance found in small sample sizes either (e.g. the YouTuber who advertises the 100% win-rate deck after logging in 10 total games). Put simply, a deck’s actual win rate isn’t the thing we should be primarily concerned about.  

Obviously that can sound a bit weird as we usually understand that a deck’s power should describe how often it wins. Decks are powerful because they win more. Where things get complicated is in understanding the context of those wins.

Issue 1: How Seriously Does the Meta Take Your Deck?

The first way to look at the matter of why win rate doesn’t necessarily describe a deck’s power concerns how seriously other people in the game are trying to beat your deck. There are some decks out there which can boost their win rate (but not their power) because no one is trying to beat them on purpose. The deck itself might be rather weak and beatable, but because people don’t select their decks, cards, or mulligan to account for this weak deck in the meta, the weak deck wins more often than it otherwise would. 

We can use an example from my past play experiences to understand this. Back when Whispers of the Old Gods came out, the Warrior class was heavily defined by Control Warrior. When you saw a Warrior on ladder, 9 times out of 10, it would be Control. However, that set also brought the release of N'Zoth's First Mate and Bloodsail Cultist, which I immediately recognized were powerful cards. So I built myself an aggressive Pirate Warrior deck, took it to ladder, and immediately smashed my way into a top 25 finish for the month with it. 

Did I do all that winning because my deck was more powerful than its competition? Possibly. I can’t say for sure whether that was true or not. What I can say was true is that no one was expecting an aggressive Warrior list. They were mulliganing incorrectly against my deck at a minimum, most of the time, and the highly-aggressive strategy capitalized on that. They were both not doing what they needed to beat me and often doing the things that would make them lose (like keeping greedy, value cards in the opener). They also weren’t playing decks specifically designed to be good against my deck either. These factors worked together to boost my win rate by a lot. 

As we can see, there are decks which might have a higher actual win rate not because they’re stronger, but because no one is trying to beat them. When a deck has easily-defined weaknesses that could be exploited, the decks actual power might be lower than its win rate suggests. These decks only manage to put up good numbers when they’re rare, because the moment they get popular, they can and will be countered, often ruthlessly. 

A truly powerful deck is that one that wins despite people taking it seriously, playing against it correctly, and trying to beat it in terms of the decks they play and the cards they play in those decks. 

To place that in a simple numbers example, if two decks sport the same actual win rate (say, 55%), but deck X is 40% of the meta and deck Y is 4%, it’s quite likely that deck X is actually stronger. When you can maintain that win rate despite people trying to take you down explicitly, that’s when you know you have a strong deck. 

On that note, decks are sometimes good not because they’re powerful in general, but because they counter a specific (and very popular) popular thing. Those counter decks sport actual win rates that are higher than their power, but can win as often as their prey is common. Decks which are truly powerful are ones that are good against a wide variety of opponents; not a narrow band of them.   

Summary 1: A deck’s win rate might be inflated because it is rare. When it’s rare, people aren’t trying to beat it in the cards/decks they play and they mulligan incorrectly against it. A deck’s win rate may also be inflated when it counters one very popular thing well, even if it could not sustain that win rate on its own merits.

Issue 2: What Field Are You Competing Against?

Another issue that arises in looking at actual win rates concerns what opponents a deck is playing against. A deck’s actual win rate may be inflated, relative to its power, if you pit it against more bad decks and fewer unfavorable matches. Some decks may literally be having an easier time of it. 

For instance. let’s say you heard that I was a boxer who had won every single one of my 1,000 boxing matches. That might make it sound like I’m an impressive boxer; perhaps even the greatest there ever was. If the next fact you learn about those matches is that I’m a 6’6”, 250 pound man who exclusively fights 10 year olds, that might shake your confidence in my abilities. I’m only winning because I’m not actually facing real competition, so my win rate doesn’t say much about my skills.  

Something similar might be happening in Hearthstone. I know that sounds a bit conspiratorial – like I’m saying Blizzard is somehow rigging matchmaking in favor of certain archetypes – but there’s something more mundane here to think about. 

Let’s say you’re sitting at the rank 5 floor. Since you cannot lose stars at that rank, you decide to mess around and play some off-meta lists. So you pick up your Mech Paladin and start jamming some games. Because people around your rank are also playing bad decks, but decks that are worse than yours, you start to win a bunch of games and rank up with a 75% win rate. However, the closer you move towards the rank 2 or rank 1 region, the stiffer your competition begins to become. All of a sudden, the Mech Paladin deck stops functioning well, so you stop playing it after a few consecutive losses. Maybe you started with a 70% win rate and stopped playing it once you hit a 50% win rate, leaving you with an overall 60% actual win rate for Mech Paladin. 

Once you hit that competitive wall, however, you swap to a stronger deck, like Galakrond Rogue. Now you start climbing again, but slower overall. So you end up hitting Legend with a 55% actual win rate for Galakrond Rogue and a 60% actual win rate for Mech Paladin.

Does that mean Mech Paladin was the better deck? Of course not. In fact, it had stopped performing in our hypothetical example, whereas Galakrond Rogue did not. It was merely the case that, when the going got tough, people switched to a different, more powerful deck. The two decks were never playing against the same field. 

Something similar might happen when looking at the frequency of decks that counter your strategy as well. 

Let’s say you start climbing from Rank 16 to Legend. Looking at some HSReplay data on deck frequency, we see that Face Hunter makes up about 9% of the decks from ranks 16-11, about 7% of the decks from 10-5, about 5% of the decks from 4-1, and finally about 4% of the decks in Legend. That might be the result of the fact that Face Hunter is a cheap deck to make (so more players in the lower ranks can afford it) or that players in lower brackets play worse decks that Face Hunter can capitalize on. 

Whatever the reason, it’s clear that counters to Face Hunter will have higher actual win rates at lower ranks and decks that get countered by Face Hunter will have lower actual win rates at those ranks as well. If people play the counters to Face Hunter in the lower-ranks (like Resurrect Priest) and then stop playing it as they leave those ranks where it’s common, this will artificially boost the actual win rate of those counter above their expected power against the meta as a whole. 

Summary 2: A deck’s actual win rate may be inflated if people stop playing weak decks as they stop performing, such as when they face stiffer competition or more counter decks. If people swap to stronger decks when the competition gets stronger, this would make the stronger decks look weaker in terms of their actual win rate, even if they would perform at times the weaker deck would not. 

Conclusion: Expected Win Rate Is a Better Measure of Power Than Actual Win Rate

What we want to know, then, is not how a deck actually performed, but rather how we should expect it to perform across the meta. The expected win rate is the measure Vicious Syndicate presents in their data reaper reports: it’s a deck’s win rate against each other deck in the field, scaled by how common those matches are. (Here’s their latest Data Reaper Report #152.)

To get a sense for the difference between expected vs actual win rate, we could consider a Hearthstone-relevant example: Casinos. 

Every game in a successful casino has a negative expected payoff for the players. In the long run, the house always wins; it’s why gambling tends to be a bad career path. However, just because the house is expected to win over time, that doesn’t mean that you can’t walk out of a casino with more money than you went in with during any given trip. 

If you were thinking about gambling, the information you’d be interested in would be the expected chance of winning while you’re there; less so the fact that someone got rich playing slots last week.  

When you look at expected win rates, like VS does, you’ll see that last week’s report suggests the best decks at legend are: Embiggen Druid, Highlander Hunter, Galakrond Rogue, Zoo Warlock, and Mech Paladin. This shows a much closer correspondence to the most common decks – as reported by Iksar – than it does to the decks with the highest actual win rate in the Tweet.

So players aren’t being too weird here: they tend to play the decks with the highest expected win rates more often, as we would expect (we sometimes play decks for reasons other than “they win” – like “they’re fun” or an overestimation of its power, but the general trend between winning and popularity should be there because winning is fun). It’s not the actual win rate data is wrong. It’s just that the actual win rate doesn’t actually tell us what we want to know.