Rating and ranking players in top level competition


#1

I don’t know if this topic has been discussed very much or if it has been implemented already in fighting game competition.

I can say that I find very interesting how other sports use ranking systems to rate and compare different competitors and teams.

One of the most prominent examples is Chess, where a rating system called Elo is commonplace and is used to establish seeds in tournaments.

Players are ranked based exclusively on their performance and given a score that can give reference for comparison in relation to other players. Their score will constantly change depending on their performance, so a player that has a winning streak will always see their rating improve. Likewise, a player’s rating can also go down for losing. The magnitude of the change depends on the difference of rating between you and your opponent.

In other words, you will gain more points by defeating a higher rated opponent rather than someone with a lower rating or a newcomer.

Many other sports have since introduced rating systems based on the same principles, to the point that it’s found in nearly every sport, all with variations and different systems.

Rating systems are of course also used in gaming, particularly online gaming. For videogames, we have the TrueSkill system on the Xbox Live. This is a great rating system for ranking results of competitive online matches. It is based primarily on the Glicko rating system, which is itself an improvement over the Elo system for chess.

So… we have these big annual tournaments for fighting games, such as Evo. I was wondering if anyone has introduced ratings and rankings for these tournaments, based on the results of top level play. I would like to know if such a thing has been attempted. Of course, rating players would require a complete log with results for the matches between top players throughout the tournaments.

The only information that is needed for rating players that were in a tournament is their (nick)names, and the outcome of each and every match (i.e. just knowing who played who and who won each match is all that is needed). That’s it! Rating systems are completely objective so specific match information is irrelevant.

Ratings are cumulative and they can add up for successive tournaments. This means that a player’s score represents results for his whole career in competition.

I think that a rating system is a cool way to compare performance and rank top players, and if it hasn’t been done already it would be great to implement overall ratings for combined annual results for a specific tournament (like Evo).


#2

Wait, what?

…you mean like RanBats? :rolleyes:


#3

Ranbats? Sort of… but on a much larger scale.

For Ranbats, a handful of players gather, and usually play one another in an informal tournament, usually for fun and training. According to their performance, there is an informal “ranking” element behind the participants, and usually the top few names enjoy web notoriety.

Using a formal rating system to rank players would be something different. Basically, my idea is to take all the results in a major tournament and rank all the players. Does anyone know if the EVO organization keeps records of all the results for their tournaments? I suppose they do and it would be great to have access to all these records.


#4

i believe what you’re asking for has been done already with SRK.com, it was called the apex system. There was even an apex 2 thing I saw not to long ago on srk but I haven’t heard much about it.

now, I remember the apex system but it was before my time as a hardcore player. I don’t know much about the original APEX but One of the problems IIRC was that people were creating fake tournaments and posting fake results in order to boost their status in the system.

also, I could see players sand bagging for others so that a specific region can hold top spots.

lastly, sometimes the best players can’t make it to a tournament. The apex thing will tell you ONLY the best players who are going to the most tournaments and playing, it wouldn’t tell you WHO the best players are which is part of the problem. Monsters pop up all over the place and sometimes monsters live away from scenes.


#5

Well, I would only be interested in working with results from major tournaments such as Evo, not just any local tournament. It doesn’t seem to me like this APEX thing used a rigorous rating system, because that would be a lot of work for that many tournaments and players.


#6

doesnt matter. in my opinion. im not a fan of ranking and seeding people for obvious reasons. come to the tourney, get randomly seeded like everyone else, play to win the pot, shut up.fuck that elitist shit, like i do this, and ive won this, and ive done that, so i shouldnt be randomly seeded like those who have done less. nah, son, you aint nobody, you play fighting games, get in line. but thats just me.


#7

The thing is, most of the O.G. greats would agree with this sentiment.

They’ve said as much, many times.

Why wouldn’t they?


#8

creating a ranking system to promote seeding is stupid. SF has always kept the format that every player runs through the bracket. Everyone starts off from the same position regardless if you’re valle or a low level player. It is not uncommon for multiple top players to be placed in the same bracket. IE sb3 where justin wong gets 17th in mvc2 and sanford got 9th. That tournament was RETARDED skill wise. When you allow seeding to take place, top players\low level players can skip brackets which they could potentially lose.

EVO has tried seeding in the past and it didn’t last. For what reasons, I can’t say. I can say that when EVO has done seeding in brackets for EVO, pretty much everyone complained about it and shat all over it.

creating a ranked system on the other hand to help identify good players is mehhhhh. If you go to tournaments long enough, you can spot a monsta.


#9

I DO NOT WANT TO PROMOTE SEEDING PLAYERS!!!

Sorry, that is not the purpose of this thread. The seeding thing was just an example for what they do in chess.

Like the other guy said, creating a ranking system to promote seeding is stupid. that wasn’t my idea.

I want to rank pro players for reference, and for fun. That is all.

I don’t see that as a problem at all. I’m thinking about a ranking that only takes competitive tournament results into account. Why rank someone who doesn’t like tournaments? I believe that the best players are the ones competing and winning major tournaments! Of course, there’s no other way to prove yourself. That’s why tournaments exist.

Player inactivity is not an issue for modern rating systems. The system would not give you a definitive rating until you have recorded at least 10 tournament matches. You would have a provisional rating until then. If one were to make a ranking based solely on yearly EVO results then only the players with at least 15 matches could enter the list, which would effectively filter it down quite a bit.

If you think that playing more matches would just give you more points, this is not the case with a rating system. This is not about accumulating points to add up to your score. Ratings systems work differently and prevent abuse. For example it’s perfectly possible to have a much better rating than someone else with much fewer matches. Only outcomes are taken into account. The system is statistically unbiased.


#10

What’s the point of that?


#11

As I mentioned earlier, rating systems are used in most sports, and their usefulness and applications expand beyond simple reference. The most obvious purpose is to ultimately establish player’s strength’s relative to each other. To know where you are in the list and to compare your position with others. There are other nifty things: the ratings can actually help predict the outcome of a match. A player with a higher rating will have a statistically higher chance of beating lower rated opposition. The more the difference in rated strength, the less is the chance of an upset.

Rating systems are useful as a measurement tool, but if you don’t like the idea then you should know that there’s no reason to take them too seriously. They’re just ratings and you shouldn’t expect much more than that. The ratings themselves are objective but the criteria used for determining what can be measured is totally subjective. A rating is just that, and there’s no reason to get riled up on who is better than the other.

Ratings are a controversial subject of frequent discussion for all sports. People often misunderstand ratings. Ratings are just estimations. Ratings ultimately don’t determine who is ‘better’, it is rather who has performed better, which may or may not mean the same to you.


#12

I believe the point is that it is for reference and for fun… Your question makes no sense.

I like this idea. Points to Blog I wrote about something similar when critiquing the idea of reimplementing Apex 2. I like rating systems in general just for fun but also I like the idea of seeding when done correctly.

Seeding should in theory make it more likely for the best person at the tournament to win. This is because when seeded the best players knock out the worst players early on. The general thought is that this just means that lower level players don’t have the opportunity to progress because they are playing Justin Wong first round. However, if two lower level players get matched early on then the player who wins progresses through the bracket further for no other reason than that they got lucky. Lack of seeding can create artificial tournament results. If the four best players play 2 games in the first round of a 256 man tournament then there is the chance that the two losers will play in losers immediately. Then the person who gets knocked out in losers (the fourth best player who could have beaten everyone else there hands down) goes 0-2 which seems ridiculous imo.

I can see how seeding is disliked in the community, especially since it seems to make it more difficult for lower level players to grab a win or two before getting knocked out. Yes fighting games are supposed to be fun. But once you have money on the line it should be seeded imo, especially when the winnings extend past 2nd place (this is where real problems begin because the 3rd best competitor can potentially be knocked out by the 2nd best competitor in round one of Losers).

One of the problems with seeding is that it is almost never done correctly. This is in part because, in the past, it has been done using a points system based solely on earning a set amount of points for tournament wins. I have not done much research into rating systems in other communities (if the chess system has adopted something then you can count on it being at least acceptable). You should not be able to grind your way to a higher seed simply by competing in every tournament under the sun. Seeding should be an accurate measurement of player ranking, which in all the fighting community systems I’ve seen it is not.

For a cool look at seeding (that isn’t very successful imo, but the bracket software is pretty good) check out Damdai’s http://www.tonamento.com.

tldr; … oops I rambled.


#13

thats retarded. There are so many good players who don’t make it to tournaments. I’ve heard so many stories about older SF players in japan who have jobs\families and can only play SF maybe once a week for a few hours. They go to the arcade and beast like a mother fucker. There lives prevent them from participating in the tournament scene but those players are highly capable of beating the best tournament players. Should they get a ranking? they certainly don’t have the time to participate in tournaments but you want to know who the best players are right?

There are a few players in FL who don’t travel but are VERY beastly and the same goes for NY. Wasn’t there an EVO GG champion from IRAN or somewhere in the middle east? how many people knew about that guy prior to EVO? and how many of his friends could potentially win EVO or place top 3?

beasts are worldwide and can’t participate in tournaments.


seedings can be rigged. Thats why it shouldn’t implemented. Very easily too. When money is on the line, you can’t expect anyone to play fair and if you leave the door open for someone to cheat, they will take the opportunity to do so.

why should players like justin, valle, marn, etc… allowed to be placed higher than others due to seeding? marn was knocked out of a tournament not to long ago by a pretty good player but that player wasn’t a top player. If marn had been seeded in that tournament, he could of dodged this loss. This is just one recent example and there are so many of them. Pretty much @ any big tournament with a lot of people entering one game, you have 1-2 top players get beat by someone who’s not a top player.

fuck seeding, if you’re going to win the tournament you go through the bracket just like everyone else.


#14

Okay, I can understand; I suppose a similar scenario would be how people like looking at character tier lists or attack attributes even if they don’t necessarily take them seriously (I don’t; assuming the game is reasonably balanced). Still, isn’t looking at tournament results basically achieving the same thing?

Also, I agree with Shoultz in I truly believe seeding/byes shouldn’t really take place at all. But I suppose if its done like now, based off of ranbat placings and such and honestly it nots too big a deal.

That’s fine that it doesn’t make sense to you…as it wasn’t directed at you. He elaborated/answered my question just fine.


#15

How about a point system that takes into account the players that participate, the amount of matches played, and the size/importance of the tourney?

You’d have local, city, region, state, national and international tournament classes, each having a set point pool that can be divided amongst the top 3 players.

Local: 5 points (3,2,1 for 1st 2nd and 3rd respectively)
City: 10 (5,3,2 points)
Region: 20 (10,7,3 points)
State: 40 (20, 15, 5 points)
National: 80 (50,20,10 points)
International: 100 (60,25,15 points)

of course, the class/type of tournament would be determined by factors like venue, turn-out, location and from where players are coming from. This would deter rhode island folks from calling their local kick-it an ‘international’ tourney just because they have a Filipino kid in their clique.

In order to get the second half of my system running, it would require a little research and backtracking. We would go back to an agreed upon year (2000?) and assign retroactive points to the winners of official tourneys to determine a base ranking system.

After establishing initial base ranks, players can start winning points from tourney play. taking a cue from SFIV, individuals would also gain additional points depending on whom they run into and defeat during their matches. I could run through EVO and play scrubs the entire way, but wouldn’t earn anything except if I won it all, but if someone were to run into known, high-level players and eek out a win, they would win some points just for taking on a stronger opponent. Those points would be determined by the rank of the player (10 points for taking out #1, 7 for taking out #2-5, 5 for taking out 5-10, 3 for 10-15, 1 for 15-25).

Just my suggestion…


#16

Yes seeding can be rigged. My point was when done properly seeding can be a good thing… That doesn’t mean we have a system that is capable of doing perfect seeding, nor does it mean that such a system can even exist.

Also, I don’t want to know who the best players are globally. I want to know who the best tournament players are. They are the ones that matter. Would it matter to the tennis community if the best tennis player in the world lived in Switzerland and only ever played their three best friends? Fuck no! You’re never going to see that guy at Wimbledon.


#17

I don’t think the advantages outweigh the disadvantages of these ranking systems, especially for SF where there is no unifying organization.


#18

shoultzula, you are now rambling pointlessly.

I’m sorry, but I simply do not get your point. You’re just talking about players who don’t go to tournaments. That has nothing to do with ranking active players. All other players cannot be measured so they’re all irrelevant. A player who never competes cannot be a known player, because he’s not in the scene. AFAIK, the community is only familiar with known players who actually compete. Quite frankly, who gives a shit about unactive and unknown players who don’t compete? Sorry, but they’re not relevant in a ranking.

Again, what’s your point?

So for example, we can rank the formerly unknown GG champion who showed up at EVO. We do not rank his unknown friends.

Daigo was seeded at last EVO.

Sorry, but you have no idea of what you’re talking about. I’m beginning to think that you don’t know how seeding works. Let’s say that Marn was seeded at that tournament. It would have been just as likely for him to meet the player who beat him, because the other guy wasn’t seeded.

Your reasoning is reminiscent of the general ignorance in the community with regards to what seeding is and why it is useful. You think that seeding only protects top players. When done correctly, seeding is always a win/win situation. It prevents you from having to play more than one top player in your bracket.

That being said, I still think that implementing seeding is a vain effort at least here because random draws effectively prevent people from moaning. This really isn’t the point of the topic.


#19

not by the communities vote of course. that was because he won the exhibition, which is stupid. he only had to play 3 other people, and we all know him getting seeded was purely for sf4 hype, and thats the truth. i dont give a fuck what anyone says. there was no reason to seed daigo other then for hype. him winning the exhibition tourney wasnt really a big deal in my opinion.

stop arguing this shit. no one here likes seeding, and no one would give a shit about a ranking system just for shits and giggles. wrong community, buddy. not to be a dick, but you’re writing paragraphs of replies to shit, and its not gonna go anywhere. when it comes to tournies, a majority of us just want to be treated equal, and there may be only a slight few that think they should be properly seeded above others, but not even wong gives a shit about seeding like he said during last years devastation stream when gootecks was crying about it.

i personally could give two shits about who is ranked what in any game. im playing to win, casuals, or tourney. your ranked first in some system for tvc, cool, dont give a shit, when you sit down, im still playing to win, and your ranking makes me no less or more nervous. maybe some kiddies would find a ranking system entertaining, so i guess you got something there, but overall, i would say a vast majority of this community could care less, unless they were forced too because it had something to do with some type of money incentive.


#20

The first half of your system is not the best way to do this. In fact it is bad idea. I am not going to use a flawed point based system. That was the cause of the the failure of the first Apex. There is clear difference between your proposal and mine in this topic. I’m reposting this quote:

In my system, you don’t gather points. You get a rating instead, an objective numerical value.

This is pretty much the only part in which I agree with you. But the question is, do these records really exist and how complete are they?