# Splatoon’s Ranking System Is Still Broken

By Evan Miller

*February 16, 2016*

In the previous episode, I analyzed the steady-state behavior of the Splatoon ranking system and showed that it is heavily biased towards producing A and **three-quarters** of Splatoon players will maintain the rank of **half** having a rank of A or **one-third** of players ranked at

Because so many players were able to reach the highest nominal ranks, there were very large skill disparities within the A ranks, making Splatoon battles less challenging for very high skill players, and more depressing for players of moderate skill who stumbled into the upper echelons of competitive play. As the A ranks became flooded with players of lower and lower skill levels, it became apparent to players, and to Nintendo, that something needed to be done.

Unfortunately, while that “something” addressed the visible symptom of having too many “A” players, the patches thus far have failed to address the underlying cause. Purely in terms of balancing the number of players in each rank, the situation is even worse than it was before the patches.

### Version 2.0: A Cure Worse Than The Illness

To address the rampant grade inflation, last year the masters of Splat introduced two new ranks beyond the

Rank | Current Points | Reward for Victory | Penalty for Defeat |
---|---|---|---|

S | 0-39 | +5 | −4 |

40-79 | +4 | −5 | |

80-99 | +3 | −6 | |

S+ | 0-39 | +3 | −6 |

40-79 | +3 | −6 | |

80-99 | +2 | −8 |

On its face, the new reward structure appears to address the problem of having too many players at the top. Once you achieve a rank of S and reach 40 points, the penalty/reward structure requires you to win significantly more games than you lose in order to maintain your point level, and so you might expect only players with the highest skill to reach S and

But did the introduction of S and

Score | Steady state | Sum |
---|---|---|

C− | 0.002% | 0.4% |

C | 0.048% | |

C+ | 0.35% | |

B− | 1.29% | 8.53% |

B | 3.11% | |

B+ | 4.14% | |

A− | 5.52% | 22.7% |

A | 7.36% | |

A+ | 9.81% | |

S | 67.9% | 68.36% |

S+ | 0.46% |

One glance at the above table should tell you that **the Version 2.0 ranking rules flooded the game with S players**. While it achieved the goal of having **two-thirds** of players were eventually pushed into S, creating a vast skill disparity within the second-highest rank.

It’s not too hard to see the source of the problem: in addition to promotion/demotion asymmetry that caused the previous grade inflation (see previous article), the new S rank injects even more points into the system for players with 0-39 points. For these players, a single win (+5 points) more than offsets a single loss (−4 points), so this region tends to pull in recently-promoted

Not too surprisingly, Nintendo attempted to patch the patch as quickly as possible.

### Version 2.2: The Patient Will Live, But The Prognosis Isn’t Good

In an attempt to shrink the swollen ranks of S, Nintendo tweaked the reward structure in the Splatoon Version 2.2 patch, which was released in October of 2015, a mere two months after the Version 2.0 patch. The new reward table looked like this:

Rank | Current Points | Reward for Victory | Penalty for Defeat |
---|---|---|---|

S | 0-39 | +5 | −5 |

40-79 | +4 | −5 | |

80-99 | +4 | −6 | |

S+ | 0-39 | +4 | −4 |

40-79 | +3 | −5 | |

80-99 | +2 | −5 |

You can see here that Nintendo eliminated the point-injection locus at S/0-39 by balancing the reward and penalty. (In Version 2.0, it was a +5 reward, −4 penalty; with Version 2.2, it is a +5 reward, −5 penalty.) So far so good. But how does the steady state look under the new rules? We can solve for it:

Score | Steady state | Sum |
---|---|---|

C− | 0.0032% | 0.65% |

C | 0.078% | |

C+ | 0.57% | |

B− | 2.08% | 13.8% |

B | 5.02% | |

B+ | 6.7% | |

A− | 8.93% | 36.71% |

A | 11.91% | |

A+ | 15.88% | |

S | 45.71% | 48.84% |

S+ | 3.12% |

While it’s not the lopsided fiasco of the Version 2.0 patch, the grade inflation is still quite bad: over 45% of Splatoon players will end up with the rank S. In other words, the second-highest rank will roughly encompass skill level ranging from the median player all the way up to players in the 97^{th} percentile.

The skill disparities within S will tend to leave many players in limbo between A+ and S, that is, constantly being promoted to S and then demoted back to A+. The net effect is to inject a steady stream of points into the system and accelerate progress toward the steady state. Anecdotal evidence confirms the existence of these A+/S limbo players:

@SplatoonNA forever between A+ and S- :')

— Juliana Chen (@fivepaninis) February 8, 2016

@SplatoonNA Currently A+. I keep going up and down between A+ and S. Still haven't ever gotten to S+ yet.

— ET (@etthenerdling) February 8, 2016

@SplatoonNA A+ my rank always goes up and down lol

— Fly Me To The Moon✨ (@sea_salt_prince) February 8, 2016

Another effect of the Version 2.2 changes was to make S+ more accessible: whereas before, less than 0.5% of players could be expected to have the S+ rank in the steady state, the figure is now about three percent of players. Reasonable people can disagree whether this particular development is a good or bad for the chi of Splatoon, but there you have it.

### Balancing The Ranks

I will reiterate the advice to Nintendo found my previous article: the simplest way to fix Splatoon’s widespread grade inflation is not to introduce new ranks, but to eliminate the asymmetry that exists between promotion and demotion. (To be demoted, you must lose 31 points relative to your last promotion; to be promoted, you must gain 30 points relative to your last demotion.)

The steady state with a "demote at zero" rule in place is as follows:

Score | Steady state | Sum |
---|---|---|

C− | 0.016% | 2.34% |

C | 0.3% | |

C+ | 2.02% | |

B− | 6.65% | 29.33% |

B | 11.34% | |

B+ | 11.34% | |

A− | 11.34% | 34.01% |

A | 11.34% | |

A+ | 11.34% | |

S | 32.14% | 34.33% |

S+ | 2.19% |

Under this distribution, roughly a third of players will have rank B(±), a third rank A(±), and a third rank

Even with this fix in place, a glut exists at the unadorned rank of S. Whereas only 11% of players will have rank

Score | Steady state | Sum |
---|---|---|

C− | 0.02% | 2.81% |

C | 0.36% | |

C+ | 2.42% | |

B− | 7.98% | 35.2% |

B | 13.61% | |

B+ | 13.61% | |

A− | 13.61% | 40.83% |

A | 13.61% | |

A+ | 13.61% | |

S | 19.94% | 21.16% |

S+ | 1.22% |

That simple change reduces the number of S players by about 50% while preserving the character of the rest of the distribution.

In any event, Nintendo’s failed attempts to fix Splatoon’s ranking system should serve as a warning to designers of competitive multiplayer games. Like the butterfly-hurricane trope, subtle asymmetries in point systems can have massive, long-term effects on the distribution of player ranks, and by extension, the overall game experience. Before implementing the next set of changes and suffering any unintended consequences, I strongly suggest that Nintendo — as well as other designers of online multiplayer games — perform a proper steady-state analysis, perhaps with the help of a script or two.

*You’re reading evanmiller.org, a random collection of math, tech, and musings. If you liked this you might also enjoy:
*

- Evaluating Splatoon’s Ranking System
- Adventure Games and Eigenvalues
- Deriving the Reddit Formula
- Ranking News Items With Upvotes
- Bayesian Average Ratings
- Ranking Items With Star Ratings: An Approximate Bayesian Approach

*Get new articles as they’re published, via LinkedIn, Twitter, or RSS.*

*Want to look for statistical patterns in your MySQL, PostgreSQL, or SQLite database? My desktop statistics software Wizard can help you analyze more data in less time and communicate discoveries visually without spending days struggling with pointless command syntax. Check it out!*

Back to Evan Miller’s home page – Subscribe to RSS – LinkedIn – Twitter