The Plugged in Tour, Worlds, Andromeda - A Brief Look at Data

Netrunner General
61

All I’m hearing from the Gabe proponents here is how the evidence to the contrary is flawed - no actual argument of their own other than belief.

The data is there if you go and look for it. There’s skill matched data from OCTGN and there are tournament reports online from a variety of large events (US Nats, Worlds, PIT etc.) which do give an insight into the selection distribution of IDs in some cases.

At the end of the day, if you believe that not only is Gabe better but that Gabe players are superior then Gabe would be over-represented at the business end of tournaments. Show me where that is the case?

When Andy first came out people were just switching the ID and playing the same deck as before; even at that stage she was already more consistent. At that point in time HQI was a thing and RDI wasn’t, so there was more support for Gabe’s play style than anything else. Since then, nothing has really hit the environment that improves the benefit of running HQ, while plenty has happpened to facilitate other strategies. I honestly can’t see how Gabe could possibly be competitve up against Andy when running HQ is so much weaker than running anywhere else. You can make the argument that Gabe can run any server he likes, but at the end of the day if he’s not running HQ every turn then you’re playing a blank ID, in which case Andy is necessarily better.

62

Pointless discussion.

The question what identity is the best has no sense because there is no way to compare them.
You can compare perfomance of each deck in your hands and choose what you want, what is comfortable for yourself only. For most top players(according to worlds) Andy is more comfortable to play with. But it doesn’t mean that Andy is better than Gabe.
If running_bear thinks that Gabe is the best and Alexfrog thinks the same about Andromeda, there is no any obstacles for them to play with their favourite identities.

But until we have metric of identity goodness and some mechanic of their comparison it is wrong to claim that A better then G or otherwise.

1 Like
63

This is the bottom line really.

Running Bear is arguing that Andy does not have solid statistical data showing that as an indentity she is stronger than Gabe. We only have data that correlates wins and Andy over Gabe, but no proof or explanation of causation. This is True.

Alexfrog is arguing that all the results we have correlate Andy and winning which is an indication that all things being equal you are more likely to win playing Andy than Gabe at any level of skill. This is true also, though we cannot say why.

Theorycrafting this game is absurdly difficult when considering something like the identity. You can compare click efficiency of Pro Contacts and Magnum pretty easily, but identities shape deck design, not vice versa.

All that can be said with any certainty is Andy stacks the wins currently. We cannot say with any objective evidence why as we do not have statistics that detailed. We cannot say anything about Gabe either save that he does not seem to be involved in many winning scenarios. We cannot address anything further.

We cannot say Gabe is better at high levels of play because there does not seem to be any objective evidence of any kind pointing to this. We cannot say Andy is statistically better, just that she correlates to better win rates for some reason.

1 Like
64

So the argument goes like this?
(in the example, talking about Kate instead of Gabe, because Kate actually made top 8)

Kate is a better identity or just as good as Andy.

Although lots of people don’t play Kate, I have anecdotal evidence that Kate is better.

For some reason, the burden of proof is on Andy to prove herself, even though she takes the top slots?

Well, it doesn’t matter because Kate came in #2, which shows:

Andy is just easier to play, Kate requires more skill, and that is why you see so many Andy wins (not because Andy is a better identity, but because it is easier to be better with her).

2 Likes
65

I provided a detailed theoretical argument as to why I thought Gabe was better against certain archetypes, but it seems like you’re only interested in arguments that come with an excel spreadsheet attached, for some reason.

My position is actually that, on the whole, Gabe players are worse than Andromeda players. I think Andromeda was the legitimately better choice for the Worlds and Plugged-In-Tour metagame (post-Opening-Moves, pre-Second-Thoughts), and so the best players saw a slight edge in Andromeda and flocked to her. Thus, we saw her dominating the top tables, because she was in the best hands. The people who are more interested in doing their own thing than winning chose Gabe, and didn’t do as well. With the release of Second Thoughts, I initially saw a shift to big ICE decks because of Restructure, so I switched to Gabe, because I think he has a slight edge against those decks, for reasons I explained earlier. That comment was then responded to by “WHAT ABOUT THE STATS THOUGH!?!??!” about 50 times.

66

I don’t think I said anything which resembles any segment of this argument. In fact, I never even said that Gabe is better than Andromeda; just that it is unreasonable to dismiss him based on data that is riddled with selection bias, and that there are certain corp strategies that he is better equipped to handle.

67

I do think using the term “better” is rather pointless.
Andromeda is more successful and not in the sense of “more people play with her, ergo more people win with her” but in the “Andromeda wins x% of games, Gabe wins y% of games. x > y.” sense.
This could be due to a number of circumstances… about which we can speculate, but not say anything definite right now. Though I do think people who bring arguements like “according to some strong players I know” should take a step back and apply logic to the situation. What “strong players” say does actually mean less than the statistical data unless those very same players go ahead and win several tournaments with Gabe against a competitive field. We simply haven’t seen that so far, which means that arguement is totally invalid.
I am not saying that it can’t be true, I am only saying, you can’t make a claim like that right now.

Personally I think Andromeda has the stronger ability. If you see Gabe, you know what to expect. If you see Andromeda, all you know for sure is that she will see nearly half her deck and pick the better quarter. That doesn’t give the corp player any information, which is incredibly powerful imho.
But that doesn’t have to be the reason, why she is more successful right now.

68

ah, internet discussions :smile:

Let’s say Gabe in fact IS better, why would it matter if other people don’t believe? Just play what you think is strongest for yourself and be done with it.

69

We have several.

  1. Win rates. OCTGN general win rates tell the strength of an ID summed across all skill levels. Pulling out only ‘high elo’ games and looking at their win rates tells strength at high skill play, given sufficient sample size.

  2. Tournament performance. Number of ‘Top X’ placements, relative to the number of entries of that ID, gives a good indication of which IDs overperformed or underperformed. Additionally, more people playing an ID in general gives some amount of evidence that of its strength.

This is what we’ve been trying to tell you all thread, but then you simply reject the data.

70

No, you just continue to make unfounded equality assumptions about self-selecting groups. You look at the data withe the assumption that the players are about equal, and so the results are based on identity. If someone else looks at the data with the assumption that the identities are about as good, or at most one is slightly stronger than the other, they’ll conclude that the players choosing that identity are better than the players choosing the other identity. Why should we prefer your conclusion over the other persons? You have not once justified your extrapolations from the data, you just keep waving it in people’s faces and expecting it to convince them.

71

I think the pro-Andy crowd needs a little perspective here. Given the nature of this forum, people are coming here to get advice or make comments on such questions as the one posed in this thread: “If I go to a tournament, and I want to run criminal, should I bring Andy or Gabe?”. This is a good honest question, and a lot of us who appear pro-Gabe really just want a more definitive answer.

A lot of the pro-Andy talk has devolved into “Pick Andy, she wins more” without really providing any further analysis on the matter. The people forwarding such arguments have to understand that this level of analysis is just not convincing to a number of us (nor should it be), it’s just a starting point for more detailed analysis. Running_bear tried to provide reasons for why Gabe might be better in certain situations, and I find the arguments convincing enough to certainly consider taking Gabe to a tourney.

From looking at the results posted, I think there are some reasonable conclusions:

  1. If going to a large tourney, like Worlds, where the meta is relatively unknown, it seems like Andy is the better choice. This could perhaps be because she has fewer bad matchups than Gabe, and is a more “consistent” choice.

  2. In smaller local tournaments, when the meta is more well known, Gabe can be an effective choice, and should not be discounted. While Andy won more Plugged In tournaments than Gabe (12 to 7), that is not necessarily an indicator of deck performance, since the total number of decks for each identity is unknown.

Note that hypomodern essentially said the same thing like 60 posts ago, it just seems appropriate to reiterate this now to get things back on track. So instead of waving stats around, perhaps we can now get back to the discussion of which matchups seem to prefer one identity over the other? Personally I think this would be a much more constructive topic to discuss, and ultimately what most people asking the “Andy or Gabe?” question would prefer to hear.

The point that I’d like to get across to the statisticians is this; the stats themselves don’t mean much, you have to use the stats to come up with a reasonable course of action. The only course of action you seem to be supporting is “always play Andy, she’s better”. This may not be what you’re intending to get across, but it sure appears that way. If you don’t support this, I’d suggest you actually state what you do support.

2 Likes
73

I think you need a little perspective here.
There is a mountain of evidence in Andy’s favor. Then, World’s result heaped yet another pile of evidence in Andy’s favor on top of it.

It is your responsibility to accept an empirical truth, when facing a mountain of evidence in its favor. Its not my responsibility to have to convince you, the evidence speaks clearly on its own.

Science doesn’t demand that you do anything extraordinary. It simply demands that you accept the evidence when it absolutely beats you over the head.

The only course of action you seem to be supporting is “always play Andy, she’s better”. This may not be what you’re intending to get across, but it sure appears that way. If you don’t support this, I’d suggest you actually state what you do support.

I am saying that it should be incredibly clear to everyone that Andromeda has performed better than Gabe in the recent past, both on an averaged ELO level (OCTGN), and high ELO level (Worlds). Additionally, the gap wasnt a small gap, but was rather large. In terms of OCTGN win rates, Andromeda is closer to The Collective than she is to Gabe. The difference between Andromeda and Gabe (~6-7% in Spin-Cycle data), is larger than the difference between Gabe, the #2 runner, and Whizzard, the #4 runner.

What does it not mean?

This doesnt mean that the state (Andy > Gabe) will continue forever. It doesnt mean you arent allowed to play Gabe. It doesnt mean that you arent allowed to try to come up with some future Gabe deck thats better than the currently popular Andy deck.

What does it mean?

It does mean that if your goal is to come up with a Gabe deck that is the best, you should understand that there is a very high bar to overcome. The Andy deck was previously way ahead of Gabe, so there is a lot of ground to make up.
It does mean that, whatever Gabe deck you come up with, you should try a similar deck with Andy, and see if that is better.
It does mean that if you are trying to come up with a superior Gabe deck, you should expect to have to make some sort of significant change from the current norm.

74

This is patently false! Please, quote me saying something like “Andy does not win more than Gabe”, or “Andy has comparable tourney results to Gabe”, or “the data is inconclusive as to which runner identity has been more successful in the past”. I have said nothing of the sort. Rather, I’ve made arguments in light of the data you’ve presented, and you haven’t provided, or even attempted to provide, a response to those arguments. Someone isn’t “denying the data” just because they draw different conclusions from it than you do.

You talk about accepting empirical truth, but everyone you’re responding to has accepted the empirical truths that the data shows; that Andy has been winning more games, and more tournaments, than Gabe. The claim “Andy is a stronger identity than Gabe” is not an empirical truth presented by the data. Rather, it is a proposition whose truth you are inferring from the data. We are arguing that the data does not necessarily imply the truth of that proposition, and you continue to pretend like our position does not account for your data, even though it explicitly does.

76

Yes, quite frankly. That’s exactly what we’re doing. That’s what science is - you can never prove anything conclusively (although of course a single counter-example is sufficient to disprove something).

The claim here is that Andy is better than Gabe and there is a huge amount of data to corroborate that hypothesis, and not much to the contrary (apart from Kingsley’s gut feeling). I fully accept that it’s a difficult situation to model and test, because you can’t possibly control for all skill levels, all different Corp archetypes and every possible order the cards in your deck might arrange themselves into - all of which contribute to the success or otherwise of a deck.

I understand that we have self-selecting groups here. But those groups don’t self-select in a vacuum. Good players play a lot, they discuss a lot and they are informed by strategy articles on the internet. Something made all these people choose to play Andromeda over Gabe, as the effect is too large to have been coincidence.

What happened was people switched IDs without changing anything else in the deck right when Adromeda was released and realised that she was comparable if not better than Gabe with a deck that favoured his play style. So it seemed reasonable that modifying the deck to suit her own play style would be even more successful. Since then, RDI became a thing and running on HQ didn’t really gain anything worth talking about, so naturally Andromeda pulled further ahead because the newer cards enabled executing an R&D lock which is patently a strong (dare I say the strongest) strategy.

I don’t think you guys appreciate how big a deal a 6% higher win rate actually is. If you doubled the amount of data we have on Gabe (I’m guessing that must be over 50k games by now) he would have to improve his win rate by 12% in order for his life-time stats to equal Andromeda.

Obviously life-time isn’t as important as what’s happening in the environment right now, but even so it illustrates how big of an edge Andromeda actually has. We’re not talking about a tiny blip that could be statistical noise, this is a very real effect.

@Alexfrog is right, we’re not saying that Gabe is necessarily worse, but if there’s a deck out there for him that’s better than Andromeda, he hasn’t found it yet and it will odds-on be very different to how people are playing him right now.

77

The win rate difference for spin cycle games (Opening Moves release and later), between Andromeda (59.10%) and Gabe (52.66%) is 6.46%, which is about 6 sigma at this sample size. (Error is around 1%, at ~10k games?)

Compared to scientific studies, most medical studies use 2 sigma (95% confidence) to declare a finding as signifcant.
In physics, the community declared the Higgs Boson officially discovered at a confidence level of 5 sigma.

Since our data is for players as a whole across skill levels, that doesnt specifically tell us that Andy is better at high ELO, but it does tell us that Gabe has a huge gap to overcome. (Even if Gabe does get better as your skill increases, it has to get better by a TON to overtake Andromeda’s lead).
The Worlds results was another huge result in Andromeda’s favor, specifically for looking at ‘High ELO’ games. (If Worlds isnt High ELO, what is?)

78

You’re clearly beyond helping. Did you even read my post? I can’t believe how arrogant and dismissive you’re being.

79

My whole point is that you cannot reasonably attribute all of the 6% you’re talking about to the marginal edge which caused good players to flock to Andromeda. The two identities are close enough that a shift in the metagame is all it takes for Gabe’s bag of tricks to be of comparable or even higher value than Andromeda’s, but that the metagame highly favoured Andromeda during the era over which these stats were compiled. If better players are playing Andromeda, then that is more reason to place less significance on the results you’re touting, and that Alex has shoved in his ears.

81

Do you really believe that someone drawing a different conclusion than you from data is “denying reality”?

82

2 Likes
83

Depends on what you’re trying to convince me of. If you say “Andromeda has a higher win rate than Gabe”, then no, I can’t and won’t refute you. You’ve certainly beaten me over the head with that one (time and time and time again…). Then again, I never did refute this, and neither did running_bear. It’s the next step that seems to be the sticky point.

I want to ask the question “why?”, and attempt to provide a rationale for the results and a method to use this information to my advantage. I also attempted to provide an experimental basis that would attempt to highlight the discrepancy and perhaps explain it further (that seems sooooo long ago). That’s how you do science; it involves experimentation under controlled conditions, an objective test, if you will. I hope that you’re not implying that the OCTGN data is taken under controlled conditions.

You seem to be convinced that your conclusion (not hypothesis, since it’s after the fact) is justified by the uncontrolled data alone, and you are unwilling or unable to investigate it further. The fact is that there any many conclusions that could be reached from the results. You seem to think that “Andy is better than Gabe” is the only one. You never even bothered to comment on the conclusions that I posted :(.

I think this example might help explain why I’m having a hard time concluding this. Say you have 10,000 people willing to make paper airplanes (many trees died for this example). They can choose between design A and design B, but you don’t stipulate which they must make. After building the airplane, they see if it can fly 10 metres (I’m Canadian, eh?). They have a month to do so, and you don’t restrict communication between participants. Results come in, and 55% of design A made the distance, and 60% of design B. Do you conclude that

a) Design B is better.
b) There isn’t a significant difference between the two.
c) Further testing is necessary.

Everyone loves a pop quiz!

1 Like