To be clear, as the article says, these authors were offered a choice and agreed to be on the "no LLMs allowed" policy.
And detection was not done with some snake oil "AI detector" but by invisible prompt injection in the paper pdf, instructing LLMs to put TWO long phrases into the review. They then detected LLM use through checking if both phrases appear in the review.
This did not detect grammar checks and touchups of an independently written review. The phrases would only get included if the reviewer fed the pdf to the LLM in clear violation to their chosen policy.
> After a selection process, in which reviewers got to choose which policy they would like to operate under, they were assigned to either Policy A or Policy B. In the end, based on author demands and reviewer signups, the only reviewers who were assigned to Policy A (no LLMs) were those who explicitly selected “Policy A” or “I am okay with either [Policy] A or B.” To be clear, no reviewer who strongly preferred Policy B was assigned to Policy A.
I'm amazed that such a simple method of detection worked so flawlessly for so many people. This would not work for those who merely used LLMs to help pinpoint strengths and weaknesses in the paper; there are separate techniques to judge that. Instead, it only detects those who quite literally copied and pasted the LLM output as a review.
It's incredible how so many people thought it was fair that their paper should be assessed by human reviewers alone, and yet would not extend the same courtesy to others.
I'm not surprised at all. The ML research community isn't a community any more, it's turned into a dog-eat-dog low-trust fierce competition. So much more people, papers, churn, that everyone is just fending for themselves. Any moment that you charitably spend on community service can be felt as a moment you take away from the next project, jeopardizing the next paper, getting scooped, delaying your graduation, your contract, your funding, your visa, your residence permit, your industry plans etc. It's a machine. I don't think people outside the phd system really understand the incentives involved. People are offered very little slack in this system. It's sink or swim, with very little instruction or scientific culture or integrity getting passed on. The PhD students see their supervisors cut corners all the time too, authorship bullshit jockeying even in big name labs etc. People I talked to are quite disillusioned, expect their work to have little impact and get superseded by a new better model in a few months so it's all about who can grind faster, who can twist the benchmarks into showing a minimal improvement etc. And the starry eyed novices get slapped by reality into thinking this way fairly early.
To be clear this is not an excuse but an explanation why I am not surprised.
This is 'spam' all over again. Before spam every email was valuable and required some attention. It was a better version of paper mail in that it was faster and cheaper. But then the spam thing happened and suddenly being 'faster and cheaper' was no longer an advantage, it was a massive drawback. But by then there was no way back. I think LLMs will do the same with text in general. By making the production of text faster and cheaper the value of all text will diminish, quite probably to something very close to the energy value of the bits that carry the data.
Generally speaking people have worse impulse control than they believe they do. Once you give a tool that does most of the work for you, very very few people will actually be able to use that tool in truly enriching ways. The majority of people (even the smart ones) will weaken over time and take shortcuts.
They were quite conservative in their approach, so the only things that were rejected were from people who had agreed not to use an LLM and almost definitely did use an LLM (since they fed hidden watermarked instructions to the llm's).
This means the true number of people that used LLM's in their review (even in group A that had agreed not to) is likely higher.
Also worth noting, 10% of these authors used them in more than half of their reviews.
Yes for those in group B I'd suspect many were doing exactly what these cheaters in group A were doing - submitting the unaltered output of an LLM as their review.
Interesting, so someone submitting a paper for review could also submit one with hidden instructions for LLMs to summarise or review it in a very positive light.
Given this detection method works so well in the use case of feeding reviewing LLMs instructions, it should also work for the original submitted paper itself, as long as it was passed along with its watermark intact. Even those just using LLMs to summarise could easily be affected if LLMs were instructed to generate very positive summaries.
So the 2% cheaters on policy A, AND 100% of policy B reviewers could fall for this and be subtly guided by the LLMs overly-positive summaries or even complete very positive reviews (based on hidden instructions).
That this sort of adversarial attack works is really quite troubling for those using LLMs to help them understand texts, because it would work even if asked to summarise something.
This definitely happened to a paper that I submitted a couple of years ago. ChatGPT 4 was the frontier. The reviewer gave a positive, if bland, summary with some reasonable suggestions for improvement and some nitpicks. There were no grammar or line-number comments like those from other reviewers. They were all issues that would have been resolved by reading the appendices, but the reviewer hadn't uploaded into ChatGPT. Later on I was able to replicate the output almost exactly myself.
What I found funny was that if you asked ChatGPT to provide a score recommendation, it was also significantly higher than what that reviewer put. They were lazy and gave a middle grade (borderline accept/reject). We were accepted with high scores from the other reviews, but it was a bit annoying that they seemingly didn't even interpret the output from the model.
The learning experience was this: be an honourable academic, but it's in your interest to run your paper through Claude or ChatGPT to see what they're likely to criticise. At the very least it's a free, maybe bad, review. But you will find human reviewers that make those mistakes, or misinterpret your results, so treat the output with the same degree of skepticism.
> Interesting, so someone submitting a paper for review could also submit one with hidden instructions for LLMs to summarise or review it in a very positive light.
I may or may not know a guy who added several hidden sentences in Finnish to his CV that might have helped him in landing an interview.
> Interesting, so someone submitting a paper for review could also submit one with hidden instructions for LLMs to summarise or review it in a very positive light.
Then these papers with these instructions get included in the training corpus for the next frontier models and those models learn to put these kinds of instructions into what they generate and …?
> Desk Reject Comments: The paper is desk rejected, because the reciprocal reviewer nominated for this paper ([OpenReview ID redacted]) has violated the LLM reviewing policy. The reviewer was required to follow Policy A (no LLMs), but we have found a strong evidence that LLM was used in the preparation of at least one of their reviews. This is a breach of peer-review ethics and grounds for desk rejection. (...)
Took me a while understand. So, the same person has both submitted their research article to the conference, and also acted as a reviewer for articles submitted by other people.
And if they in their review work have agreed to a "no LLM use" policy, but got exposed using LLMs anyway, then their submitted research article is desk rejected. Theoretically, someone could have submitted a stellar research article, but because they didn't follow agreed policy when reviewing other people's work, then also their research contribution is not welcome.
(At first I understood that innocent author's articles would have been rejected just because they happened to go to a bad reviewer. But this is not the case.)
Slightly more nuanced in that the reciprocal reviewer may have been essentially forced to sign despite having other commitments or may not have even been the lead contributor. Nowadays if a student submits a side project to a top-tier conference then it is required that if any authors have significant publication count in top-tier venues, then one must be a mandatory reviewer. Then one must sign that agreement. Students need to publish, much less so for me, where I really want to publish big innovations rather than increments, but now I get all these mandatory reviewer emails demanding I review for a conference because a student has my name on the paper and I'm the most senior, but I may have just seeded the idea or helped them in significant ways. However, many times those are not my passion projects and is just something a student did that I helped with, but now all AI conferences are demanding I review or hurt a student, where I'm the middle author.
But if anything, I think the whole anti-LLM review philosophy is wrong. If anything we need multiple deep background and research analyses of papers. So many papers are trash or are publishing what has already been done or are missing things. The volume of AI papers makes it impossible for a human alone to really critique work because hundreds of new papers come out a day.
Is this really what happened? The post from the conference chairs is extremely confusing. Maybe my confusion is because I've never published in a conference with reciprocal reviewers and if I had this experience maybe the post would be very clear.
In any case, I had reached the same interpretation before reading your post, thinking that this is the only interpretation that could make any sense, but I'm still not convinced that this is what happened. Hopefully, no "innocent authors' articles were rejected because they happened to go to a bad reviewer".
I keep spotting clear LLM 'tells' in text where I know the people on the other side believe they're 'getting away with it'. It is incredible at what levels of commerce people do this, and how they're prepared to risk their reputation by saving a few characters typed. It makes me wonder what they think they are getting paid for.
Based on my experience on HN, many people can't see the tells. They may pick up on a few meme things like emdashes or "delve" or "rich tapestry", but can't detect the general tone or cadence reliably.
So maybe those people are right and are getting away with it for most readers of it.
I have heard people say that they find that people who broadcast their distaste for LLMs secretly use it. I was fairly sceptical of the claim, but this seems to suggest that it happens more than I would have thought.
One wonders what leads them to the AI rejecting option in the first place.
How is nobody considering the broader political economy of scholarly publications and reviews? These are UNPAID reviews! Sure, maybe ICML isn’t Elsevier, but they are cousins to the socially parasitic and exploitative companies, at the very least.
Hiding behind a false “choice” to not use AI or basically not use AI isn’t an appropriate proposal. This is crooked and shameful. We should boycott ICML except we can’t because they are already the gatekeepers!
Your job as an academic is to disseminate your research and engage with the research community through service such as reviews, talks etc. It's part of the job, and people get a salary as university employees or company employees for this.
ML conferences aren't for profit ventures. If you submit papers and expect others to review it, you should reciprocate as well.
It would be interesting to know how many of the cheaters didn't check policy A, but checked "don't care if A or B". Because the operative part of that is "don't care", not "I will strictly adhere to either policy A or B, whatever somebody else selects for me".
So it is a sneaky and typically academic way of doing stuff. Also, "We hope that by taking strong action against violations of agreed-upon policy we will remind the community that as our field changes rapidly the thing we must protect most actively is our trust in each other. If we cannot adapt our systems in a setting based in trust, we will find that they soon become outdated and meaningless." is so academic and pointless.
People in the comment asking for harsher punishment should note that we don’t know how many people selected the „I have no strong preference“ option and got assigned to group A randomly.
It’s a bit harder to make the argument that those people _explicitly_ agreed to not use LLMs.
And given how the desk-rejection logic relies on an ethical integrity argument, actual explicit intent is important.
I think the real news from this experiment is that LLM usage is almost unavoidable even among high level professionals who are capable to and promised to do the task without LLMs. I don’t think these policies will be around in a few years. They are more like naive transition period attempts to stop a tsunami.
I've learned a bit today about how often people on hn read the article when commenting. Or potentially bots who are way off. The title alone isn't enough to totally grasp what happened here, or the methods used.
Extremely conservative detection. The real number must be much higher.
The declaration of no-LLM was done for social prestige or maybe self-deception of self-sufficiency like "I don't need LLM". And when it was time to do the actual work, the dependency kicked in like drugs.
A lesson for all of us with LLMs in our workflow.
I really like how they approach to the detection. But I am worried that this is something the community can only use effectively once. There are too many ways to bypass this detection once you know how it works.
[+] [-] bonoboTP|5 days ago|reply
And detection was not done with some snake oil "AI detector" but by invisible prompt injection in the paper pdf, instructing LLMs to put TWO long phrases into the review. They then detected LLM use through checking if both phrases appear in the review.
This did not detect grammar checks and touchups of an independently written review. The phrases would only get included if the reviewer fed the pdf to the LLM in clear violation to their chosen policy.
> After a selection process, in which reviewers got to choose which policy they would like to operate under, they were assigned to either Policy A or Policy B. In the end, based on author demands and reviewer signups, the only reviewers who were assigned to Policy A (no LLMs) were those who explicitly selected “Policy A” or “I am okay with either [Policy] A or B.” To be clear, no reviewer who strongly preferred Policy B was assigned to Policy A.
[+] [-] mikkupikku|5 days ago|reply
[+] [-] hodgehog11|5 days ago|reply
It's incredible how so many people thought it was fair that their paper should be assessed by human reviewers alone, and yet would not extend the same courtesy to others.
[+] [-] bonoboTP|5 days ago|reply
To be clear this is not an excuse but an explanation why I am not surprised.
[+] [-] jacquesm|5 days ago|reply
[+] [-] everdrive|5 days ago|reply
[+] [-] mijoharas|5 days ago|reply
They were quite conservative in their approach, so the only things that were rejected were from people who had agreed not to use an LLM and almost definitely did use an LLM (since they fed hidden watermarked instructions to the llm's).
This means the true number of people that used LLM's in their review (even in group A that had agreed not to) is likely higher.
Also worth noting, 10% of these authors used them in more than half of their reviews.
[+] [-] grey-area|5 days ago|reply
[+] [-] grey-area|5 days ago|reply
Given this detection method works so well in the use case of feeding reviewing LLMs instructions, it should also work for the original submitted paper itself, as long as it was passed along with its watermark intact. Even those just using LLMs to summarise could easily be affected if LLMs were instructed to generate very positive summaries.
So the 2% cheaters on policy A, AND 100% of policy B reviewers could fall for this and be subtly guided by the LLMs overly-positive summaries or even complete very positive reviews (based on hidden instructions).
That this sort of adversarial attack works is really quite troubling for those using LLMs to help them understand texts, because it would work even if asked to summarise something.
[+] [-] joshvm|5 days ago|reply
What I found funny was that if you asked ChatGPT to provide a score recommendation, it was also significantly higher than what that reviewer put. They were lazy and gave a middle grade (borderline accept/reject). We were accepted with high scores from the other reviews, but it was a bit annoying that they seemingly didn't even interpret the output from the model.
The learning experience was this: be an honourable academic, but it's in your interest to run your paper through Claude or ChatGPT to see what they're likely to criticise. At the very least it's a free, maybe bad, review. But you will find human reviewers that make those mistakes, or misinterpret your results, so treat the output with the same degree of skepticism.
[+] [-] Tade0|5 days ago|reply
I may or may not know a guy who added several hidden sentences in Finnish to his CV that might have helped him in landing an interview.
[+] [-] bjourne|5 days ago|reply
Has been done: https://www.theguardian.com/technology/2025/jul/14/scientist...
[+] [-] wood_spirit|5 days ago|reply
[+] [-] gregdeon|5 days ago|reply
[+] [-] merelysounds|5 days ago|reply
> ICML: every paper in my review batch contains prompt-injection text embedded in the PDF
source: https://old.reddit.com/r/MachineLearning/comments/1r3oekq/d_...
There are recent comments there as well:
> Desk Reject Comments: The paper is desk rejected, because the reciprocal reviewer nominated for this paper ([OpenReview ID redacted]) has violated the LLM reviewing policy. The reviewer was required to follow Policy A (no LLMs), but we have found a strong evidence that LLM was used in the preparation of at least one of their reviews. This is a breach of peer-review ethics and grounds for desk rejection. (...)
source: https://old.reddit.com/r/MachineLearning/comments/1r3oekq/d_...
[+] [-] sampo|5 days ago|reply
And if they in their review work have agreed to a "no LLM use" policy, but got exposed using LLMs anyway, then their submitted research article is desk rejected. Theoretically, someone could have submitted a stellar research article, but because they didn't follow agreed policy when reviewing other people's work, then also their research contribution is not welcome.
(At first I understood that innocent author's articles would have been rejected just because they happened to go to a bad reviewer. But this is not the case.)
[+] [-] chriskanan|5 days ago|reply
But if anything, I think the whole anti-LLM review philosophy is wrong. If anything we need multiple deep background and research analyses of papers. So many papers are trash or are publishing what has already been done or are missing things. The volume of AI papers makes it impossible for a human alone to really critique work because hundreds of new papers come out a day.
[+] [-] cefstat|4 days ago|reply
In any case, I had reached the same interpretation before reading your post, thinking that this is the only interpretation that could make any sense, but I'm still not convinced that this is what happened. Hopefully, no "innocent authors' articles were rejected because they happened to go to a bad reviewer".
[+] [-] michaelbuckbee|5 days ago|reply
[+] [-] jacquesm|5 days ago|reply
[+] [-] bonoboTP|5 days ago|reply
So maybe those people are right and are getting away with it for most readers of it.
[+] [-] aledevv|5 days ago|reply
Correct me if I'm wrong, but this means that many people are using LLMs despite claiming not to.
It's the first symptom of a dependency mechanism.
If this happens in this context, who knows what happens in normal work or school environments?
(P.S.: The use of watermarks in PDFs to detect LLM usage is very interesting, even though the LLM might ignore hidden instructions.)
[+] [-] Lerc|5 days ago|reply
One wonders what leads them to the AI rejecting option in the first place.
[+] [-] boelboel|5 days ago|reply
[+] [-] IshKebab|5 days ago|reply
[+] [-] quinndupont|5 days ago|reply
Hiding behind a false “choice” to not use AI or basically not use AI isn’t an appropriate proposal. This is crooked and shameful. We should boycott ICML except we can’t because they are already the gatekeepers!
[+] [-] bonoboTP|5 days ago|reply
ML conferences aren't for profit ventures. If you submit papers and expect others to review it, you should reciprocate as well.
[+] [-] qbit42|5 days ago|reply
And they didn't give a permanent ban or anything, these authors can just resubmit to another conference, of which there are many.
[+] [-] auggierose|5 days ago|reply
So it is a sneaky and typically academic way of doing stuff. Also, "We hope that by taking strong action against violations of agreed-upon policy we will remind the community that as our field changes rapidly the thing we must protect most actively is our trust in each other. If we cannot adapt our systems in a setting based in trust, we will find that they soon become outdated and meaningless." is so academic and pointless.
[+] [-] FabCH|5 days ago|reply
It’s a bit harder to make the argument that those people _explicitly_ agreed to not use LLMs.
And given how the desk-rejection logic relies on an ethical integrity argument, actual explicit intent is important.
[+] [-] bonoboTP|5 days ago|reply
[+] [-] ozgung|5 days ago|reply
[+] [-] zulban|5 days ago|reply
Extremely conservative detection. The real number must be much higher.
[+] [-] causalityltd|5 days ago|reply
[+] [-] auggierose|5 days ago|reply
[+] [-] iso1631|5 days ago|reply
I can divide 98,324,672,722 by 161,024 by hand. At least I used to be able to do, but nobody is going to pay me to do that when a calculator exists.
Likewise I can write a bunch of assembly (well OK I can't), but why would I do that when my compiler can convert my intention into it.
[+] [-] bsder|5 days ago|reply
I don't personally use LLMs for this kind of stuff, but I'll certainly not sign a "No LLM" pledge unless you give me some kind of benefit.
[+] [-] geremiiah|5 days ago|reply
[+] [-] pppoe|5 days ago|reply
[+] [-] Lliora|5 days ago|reply
[+] [-] mvrckhckr|5 days ago|reply