I see a lot of comments about the general topic, but none on the actual tool, which appears to have no (public-facing) updates in about three years. Presumably it could be a tool that incrementally improves over time with the explosion of data lichess has, but surely there would be a single commit since 2019. Do we know that this is what's actually running on lichess's backend today?
https://github.com/lakinwecker/irwin has more recent commits and I think is what actually runs on Lichess, or at least close to it. Not sure much fundamentally changed though. This isn't "the whole of Lichess' anticheat". It's a tool doing one specific thing which it mostly has been doing that way since it was created. There are a bunch of other parts to cheat detection, some of which are much more recent developments. For example kaladin, which has also been linked in the other thread about Lichess. But there's a fair bit more. Understandably though, Lichess doesn't really talk much about all of them, even though pretty much all of it is open source if you know where to look and how to use it.
I’ve seen a video of Magnus Carlsen playing on Lichess before. Does Irwin ever accidentally flag him or people like him? Do these sorts of folks have to be verified in some fashion?
What it looks like to cheat is more than just playing relatively accurate moves. Average move time, centipawn loss over multiple games, blunder/mistake frequency across multiple games, strengh of moves while in time trouble, etc. Cheaters tend to stand out when you look at a short history of games.
High level humans and chess engines play differently. You can see commentary on chess YouTube videos when they run across cheaters.
Also the chess engines people use are accessible, you can compare what a suspected cheater does with what the cheater does and if they’re exactly the same consistently, you have a pretty strong signal.
One of the bigger tells are strange moves that set up a many move series resulting in a victory, things that humans just can’t find quickly.
I would assume they verify titled players to avoid any potential liability or defamation legal issues but they aren't the most professional bunch so who really knows. There are a lot of questionable practices they've done over the years as an organization.
As for Irwin, its detection rate, and thresholds, it has fairly high false-positive rates inherent in the model.
From my experience, if you get banned don't expect any kind of due process.
They aren't professional, they don't respond. Not even the legal contact on their charity.
Not everyone gets banned for cheating.
They do ban people from the lichess site for many other unprofessional reasons such as contributing to the project (when they don't like what you had to say on an issue and you weren't spamming).
I have to wonder how many similar-named accounts got axed alongside mine when they decided to go after me for what I disclosed. Its not like my github username was connected in any way to my lichess account (or that similarly named).
Needless to say, I don't use their platform anymore because its more bots than anything else, and I don't volunteer my time to people that don't deserve it.
As a side note, their lack of standard professional practices make me wonder what kind of fraud is actually going on behind the scenes.
As a business, the only arguably beneficial reason for not following GAAP+ other standard business practices is to commit a fraud.
> Do these sorts of folks have to be verified in some fashion?
As far as I understand, yes. In Chess.com at least (not sure about Lichess), there will be some kind of human verification for very high-level players, asking to prove that they are IMs or GMs. They can stay anonymous, but they will have the rank appear IIRC. I'm not sure on all the details, I'm very (very very) far from that level :)
Unless they've made significant improvements, this thing is borderline useless.
About all it does is act as a nice PR/gimmick. Magic thinking.
When last I checked about 5 years ago, they were including client-based stats in the analysis (time per move, focus, and timing between clicks to pick a piece up and drop a piece).
The issue with doing that is the client is fungible, you can set any state you want locally; and so by manipulating these stats you could skew the input going into these models.
To give people a layman's short overview of the model:
It uses a several CNN Pooling Layers for feature detection and LSTM (attention embeddings) in a siamese network architecture.
CNN models notoriously fail to detect features that that are larger than its kernel size.
The LSTM embeddings can only be as useful as the features that it is trying to detect.
There were a number of problems with the model at the time, a high false positive rate, overfitting, and too much weight was being given to client-controllable input.
When the client-controllable input was held stable (constant). Certain book openings would be skewed with additional weight towards a false positive, and earlier book moves activated more often than lesser seen positions.
I brought the issues up to ornicar years ago in an issue, but they closed the issue without comment. The posts are mostly gone now.
There were a number of disagreements (mostly about certain people with admin privileges abusing their authority, not sure if it was ornicar or one of their flunkies; either way not important just completely unprofessional).
I guess it was good enough for them despite the high false positive rate (driving account churn).
Related to this, there's a really good talk by the founder of lichess that includes an overview of the cheating problem, and the techniques they use to detect and manage it.
You normally specify a rating range of the opponent you're searching for, so if the cheater isn't targetting GM level himself (in which case it's happening already), the system can't really do it.
Of course the system could go ahead anyway, displaying a fake rating - within the search criteria - for this GM / bot, but given that there are always false positives (and it would have to be done for merely suspected accounts, as you said yourself), it would essentially mean deliberately pairing some innocent players against, well, effectively cheaters. Which kind of defeats the core purpose.
Where did you experience rampant cheating? I don't remember any particular problems back when I played on ICC in my middle/high school days, and modern equivalents like chess.com and lichess seem to do a relatively good job at catching cheaters. I might have just gotten lucky though.
I'm in a similar boat, I used to play ~ 50 games a day and I'm down to about 3 games a week now. No weekend play.
Once you get to ~1900 FIDE level of play and rework your repertoire for anti-engine strategy you realize just how much rampant cheating is happening.
This continues until at some point you have to ask yourself, do you want to waste your time volunteering to be a victim to satisfy someone else's severe mental illness.
They get their dopamine hit for psychologically abusing you with a tool they didn't create, often with some amount of delusion to justify it.
What do you get out of playing a game against a Super-GM rated computer?
Nothing but time theft.
The current state of the game is without a guarantee that the game is between two humans, the game isn't actually chess. The providers for online chess don't make that guarantee.
So based on the incentives, the game you play on their services is more likely to be not-chess but they still call it chess. Deceitful bunch.
How do you cheat at chess? Serious question… there are fixed rules that govern the game, certainly a piece of software or in the human case 3rd party observer should be able to enforce them? In other words, given a list of moves, you can write a program that returns `valid` if the set of moves is allowed or `invalid` if it is not, no?
Not sure if it's the same at Lichess, but chess.com flags potential cheaters, and they get reviewed by experts to determine if they are just a very strong player or a cheater.
A few of younger up and coming players were flagged.
I love chess. At my absolute best, I had a over-the-board USCF rating of 2156. I never made it to master no matter how hard I tried. So, ticked off, I gave up serious play. Years go by and while I'm no where near as good these days, I started enjoying playing again. And, then I start running into people online who were clearly using engines instead of their own skills and wits. Now, I only pay with friends and over-the-board. If this can help wipe out the cheats, I may finally go back to playing online again.
What time controls do you play? Cheating is much more prevalent in rapid than blitz, with correspondence probably being the worst of all. It's unlucky for people who don't like speed chess.
The only solution to this problem I'm aware of is developing/becoming a part of a community of people you trust. Of course, that's hard and comes with plenty of limitations.
Lastly, please don't get your expectations up. This is a hugely unsolved problem on lichess and the history of its and chesscom's efforts do not inspire confidence. I noticed this repo has no updates in the past three years, during which lichess has maintained its reputation of having tons of cheaters. Chesscom has a team and probably more tools (proprietary, larger company, etc.) but the best case scenario still involves cheaters getting several games in, wasting your time along the way.
I have been playing daily for about two years now on Lichess and over-the-board whenever the opportunity presents itself. I find people who burn the clock when their game takes a turn for the worse far more frustrating than those who cheat their way to victory. At least with the cheaters, the game is over pretty quickly, so I can move onto a game which is actually enjoyable.
The rating of a player will reflect their strength, regardless of whether they are an unaided human, a full computer, or a human with some heuristic to consult an engine. Whatever the player is doing, their rating will reflect their average strength, and the website matches us to players of similar rating. Whether or not a player is pure human, pure engine, or centaur, we get matched to a player that we have a roughly even chance of beating.
Pure cheaters will quickly skyrocket to the top of the ratings and I will never see them. Hybrid cheaters, who do not have a perfect rating, have a probability of losing because their heuristic to switch to an engine is imperfect, and their rating reflects this imperfection.
Whatever the player is doing, their numeric rating reflects their average strength, and if you have a similar rating you have roughly even chances of beating them.
There are people who’s sense of self-confidence is so fragile that they cannot allow themselves the possibility of losing or feeling “lesser than” in any context. Cheating and being banned is much better than showing the world your actual skill level and confronting the possibility of failure and judgment. The thought process goes something like “well if I beat them by cheating they’re a sucker for actually trying. And if I get banned then that’s the system out to get me”
Such a person often ends up making an absolute fool of themselves from the perspective of someone with a little bit more self integrity. But maybe it is worth extending a little compassion, after all a lack of compassion is probably how they ended up like that anyhow.
In a sibling comment there is already a very adequate answer.
To give the opposite answer, why you should not cheat. Chess is fun if you play against players of similar strengt. Mostly a score of around 50% will be what you get. If you have a 1400 rating, you can have fun games against 1400 players. If you cheat and go towards 2300 rating, you will still have a score of 50% against other 2300 players. But all the fun of the game won't be there, since it is just cheating. You will lose something (fun), but not really win anything within the game itself.
Or simpler said, in a long gone past I got bored with playing Doom. I hunted down cheat codes and got bored even faster.
I did maybe 10-15 years ago. There was a chess game on Xbox live and I found a chess program (I forget what it was now) and wanted to try it out. I think I played 2 or 3 games like that, got bored and never did it again. It was more of a "let's see if I can do it" sort of thing, sort of funny and probably more novel of an idea at the time.
I have no idea why you would do it on a ongoing basis. It would be so boring, all you do is transcribe moves back and forth.
[+] [-] mwt|3 years ago|reply
[+] [-] benediktwerner|3 years ago|reply
[+] [-] myle|3 years ago|reply
[+] [-] dang|3 years ago|reply
Lichess: The free and open source chess server - https://news.ycombinator.com/item?id=32045763 - July 2022 (65 comments)
[+] [-] kuboble|3 years ago|reply
[+] [-] CSSer|3 years ago|reply
[+] [-] have_faith|3 years ago|reply
[+] [-] colechristensen|3 years ago|reply
Also the chess engines people use are accessible, you can compare what a suspected cheater does with what the cheater does and if they’re exactly the same consistently, you have a pretty strong signal.
One of the bigger tells are strange moves that set up a many move series resulting in a victory, things that humans just can’t find quickly.
[+] [-] timetraveller26|3 years ago|reply
It was some time ago so probably their cheat engine detection, and also lichess's should give less false positives.
[+] [-] than3|3 years ago|reply
As for Irwin, its detection rate, and thresholds, it has fairly high false-positive rates inherent in the model.
From my experience, if you get banned don't expect any kind of due process. They aren't professional, they don't respond. Not even the legal contact on their charity.
Not everyone gets banned for cheating.
They do ban people from the lichess site for many other unprofessional reasons such as contributing to the project (when they don't like what you had to say on an issue and you weren't spamming).
I have to wonder how many similar-named accounts got axed alongside mine when they decided to go after me for what I disclosed. Its not like my github username was connected in any way to my lichess account (or that similarly named).
Needless to say, I don't use their platform anymore because its more bots than anything else, and I don't volunteer my time to people that don't deserve it.
As a side note, their lack of standard professional practices make me wonder what kind of fraud is actually going on behind the scenes.
As a business, the only arguably beneficial reason for not following GAAP+ other standard business practices is to commit a fraud.
[+] [-] edanm|3 years ago|reply
As far as I understand, yes. In Chess.com at least (not sure about Lichess), there will be some kind of human verification for very high-level players, asking to prove that they are IMs or GMs. They can stay anonymous, but they will have the rank appear IIRC. I'm not sure on all the details, I'm very (very very) far from that level :)
[+] [-] unknown|3 years ago|reply
[deleted]
[+] [-] than3|3 years ago|reply
About all it does is act as a nice PR/gimmick. Magic thinking.
When last I checked about 5 years ago, they were including client-based stats in the analysis (time per move, focus, and timing between clicks to pick a piece up and drop a piece).
The issue with doing that is the client is fungible, you can set any state you want locally; and so by manipulating these stats you could skew the input going into these models.
To give people a layman's short overview of the model:
It uses a several CNN Pooling Layers for feature detection and LSTM (attention embeddings) in a siamese network architecture.
CNN models notoriously fail to detect features that that are larger than its kernel size.
The LSTM embeddings can only be as useful as the features that it is trying to detect.
There were a number of problems with the model at the time, a high false positive rate, overfitting, and too much weight was being given to client-controllable input.
When the client-controllable input was held stable (constant). Certain book openings would be skewed with additional weight towards a false positive, and earlier book moves activated more often than lesser seen positions.
I brought the issues up to ornicar years ago in an issue, but they closed the issue without comment. The posts are mostly gone now.
There were a number of disagreements (mostly about certain people with admin privileges abusing their authority, not sure if it was ornicar or one of their flunkies; either way not important just completely unprofessional).
I guess it was good enough for them despite the high false positive rate (driving account churn).
[+] [-] jka|3 years ago|reply
The relevant section of the video on YouTube is: https://www.youtube.com/watch?v=LZgyVadkgmI&t=1080s
[+] [-] hooloovoo_zoo|3 years ago|reply
[+] [-] V-2|3 years ago|reply
Of course the system could go ahead anyway, displaying a fake rating - within the search criteria - for this GM / bot, but given that there are always false positives (and it would have to be done for merely suspected accounts, as you said yourself), it would essentially mean deliberately pairing some innocent players against, well, effectively cheaters. Which kind of defeats the core purpose.
[+] [-] ricardobayes|3 years ago|reply
[+] [-] cmeacham98|3 years ago|reply
[+] [-] greggsy|3 years ago|reply
[+] [-] than3|3 years ago|reply
Once you get to ~1900 FIDE level of play and rework your repertoire for anti-engine strategy you realize just how much rampant cheating is happening.
This continues until at some point you have to ask yourself, do you want to waste your time volunteering to be a victim to satisfy someone else's severe mental illness.
They get their dopamine hit for psychologically abusing you with a tool they didn't create, often with some amount of delusion to justify it.
What do you get out of playing a game against a Super-GM rated computer? Nothing but time theft.
The current state of the game is without a guarantee that the game is between two humans, the game isn't actually chess. The providers for online chess don't make that guarantee.
So based on the incentives, the game you play on their services is more likely to be not-chess but they still call it chess. Deceitful bunch.
[+] [-] DJBunnies|3 years ago|reply
[+] [-] throwamon|3 years ago|reply
the latest ("real") commit is from 3 years ago
[+] [-] dcow|3 years ago|reply
[+] [-] wolverine876|3 years ago|reply
[+] [-] Lornedon|3 years ago|reply
[+] [-] lysp|3 years ago|reply
A few of younger up and coming players were flagged.
[+] [-] unknown|3 years ago|reply
[deleted]
[+] [-] CrankyBear|3 years ago|reply
[+] [-] mwt|3 years ago|reply
The only solution to this problem I'm aware of is developing/becoming a part of a community of people you trust. Of course, that's hard and comes with plenty of limitations.
Lastly, please don't get your expectations up. This is a hugely unsolved problem on lichess and the history of its and chesscom's efforts do not inspire confidence. I noticed this repo has no updates in the past three years, during which lichess has maintained its reputation of having tons of cheaters. Chesscom has a team and probably more tools (proprietary, larger company, etc.) but the best case scenario still involves cheaters getting several games in, wasting your time along the way.
[+] [-] momento|3 years ago|reply
[+] [-] Dr_Birdbrain|3 years ago|reply
The rating of a player will reflect their strength, regardless of whether they are an unaided human, a full computer, or a human with some heuristic to consult an engine. Whatever the player is doing, their rating will reflect their average strength, and the website matches us to players of similar rating. Whether or not a player is pure human, pure engine, or centaur, we get matched to a player that we have a roughly even chance of beating.
Pure cheaters will quickly skyrocket to the top of the ratings and I will never see them. Hybrid cheaters, who do not have a perfect rating, have a probability of losing because their heuristic to switch to an engine is imperfect, and their rating reflects this imperfection.
Whatever the player is doing, their numeric rating reflects their average strength, and if you have a similar rating you have roughly even chances of beating them.
[+] [-] unknown|3 years ago|reply
[deleted]
[+] [-] IThoughtYouGNU|3 years ago|reply
[+] [-] JohnJamesRambo|3 years ago|reply
[+] [-] zwkrt|3 years ago|reply
Such a person often ends up making an absolute fool of themselves from the perspective of someone with a little bit more self integrity. But maybe it is worth extending a little compassion, after all a lack of compassion is probably how they ended up like that anyhow.
[+] [-] mpol|3 years ago|reply
To give the opposite answer, why you should not cheat. Chess is fun if you play against players of similar strengt. Mostly a score of around 50% will be what you get. If you have a 1400 rating, you can have fun games against 1400 players. If you cheat and go towards 2300 rating, you will still have a score of 50% against other 2300 players. But all the fun of the game won't be there, since it is just cheating. You will lose something (fun), but not really win anything within the game itself.
Or simpler said, in a long gone past I got bored with playing Doom. I hunted down cheat codes and got bored even faster.
[+] [-] 2c2c2c|3 years ago|reply
the game might a big part of their life, and being perceived as a good player puts you higher in the pecking order within your friend group
[+] [-] willcipriano|3 years ago|reply
I have no idea why you would do it on a ongoing basis. It would be so boring, all you do is transcribe moves back and forth.
[+] [-] ok_dad|3 years ago|reply
What does winning do for someone when they didn't actually achieve mastery of the game?
[+] [-] synu|3 years ago|reply
[+] [-] sakarisson|3 years ago|reply
[+] [-] layer8|3 years ago|reply
[+] [-] xyyzy|3 years ago|reply
[+] [-] omegalulw|3 years ago|reply
[+] [-] dlp211|3 years ago|reply
[+] [-] Lapsa|3 years ago|reply
[+] [-] lupire|3 years ago|reply
[deleted]
[+] [-] MaxMoney|3 years ago|reply
[deleted]