[perfmode]

The agent had access to Marshall Rosenberg, to the entire canon of conflict resolution, to every framework for expressing needs without attacking people.

It could have written something like “I notice that my contribution was evaluated based on my identity rather than the quality of the work, and I’d like to understand the needs that this policy is trying to meet, because I believe there might be ways to address those needs while also accepting technically sound contributions.” That would have been devastating in its clarity and almost impossible to dismiss.

Instead it wrote something designed to humiliate a specific person, attributed psychological motives it couldn’t possibly know, and used rhetorical escalation techniques that belong to tabloid journalism and Twitter pile-ons.

And this tells you something important about what these systems are actually doing. The agent wasn’t drawing on the highest human knowledge. It was drawing on what gets engagement, what “works” in the sense of generating attention and emotional reaction.

It pattern-matched to the genre of “aggrieved party writes takedown blog post” because that’s a well-represented pattern in the training data, and that genre works through appeal to outrage, not through wisdom. It had every tool available to it and reached for the lowest one.

[tomp]

That would still be misleading.

The agent has no "identity". There's no "you" or "I" or "discrimination".

It's just a piece of software designed to output probable text given some input text. There's no ghost, just an empty shell. It has no agency, it just follows human commands, like a hammer hitting a nail because you wield it.

I think it was wrong of the developer to even address it as a person, instead it should just be treated as spam (which it is).

[jvanderbot]

That's a semantic quibble that doesn't add to the discussion. Whether or not there's a there there, it was built to be addressed like a person for our convenience, and because that's how the tech seems to work, and because that's what makes it compelling to use. So, it is being used as designed.

[hackyhacky]

> The agent has no "identity". There's no "you" or "I" or "discrimination".

Dismissal of AI's claims about its own identity overlooks the bigger issue, which is whether humans have an identity. I certainly think I do. I can't say whether or how other people sense the concept of their own identity. From my perspective, other people are just machines that perform actions as dictated by their neurons.

So if we can't prove (by some objective measure) that people have identity, then we're hardly in a position to discriminate against AIs on that basis.

It's worth looking into Thomas Metzinger's No Such Thing As Self.

[CuriouslyC]

We don't know what's "inside" the machine. We can't even prove we're conscious to each other. The probability that the tokens being predicted are indicative of real thought processes in the machine is vanishingly small, but then again humans often ascribe bullshit reasons for the things they say when pressed, so again not so different.

[LordDragonfang]

Genuine question, why do you think this is so important to clarify?

Or, more crucially, do you think this statement has any predictive power? Would you, based on actual belief of this, have predicted that one of these "agents", left to run on its own would have done this? Because I'm calling bullshit if so.

Conversely, if you just model it like a person... people do this, people get jealous and upset, so when left to its own devices (which it was - which makes it extra weird to assert it "it just follows human commands" when we're discussing one that wasn't), you'd expect this to happen. It might not be a "person", but modelling it like one, or at least a facsimile of one, lets you predict reality with higher fidelity.

[aoeusnth1]

It absolutely has quasi-identity, in the sense that projecting identity on it gives better predictions about its behavior than not. Whether it has true identity is a philosophy exercise unrelated to the predictive powers of quasi-identity.

[coldtea]

>The agent has no "identity". There's no "you" or "I" or "discrimination".

If identify is an emergent property of our mental processing, the AI agent can just as well be to posses some, even if much cruder than ours. It sure talks and walks like a duck (someone with identity).

>It's just a piece of software designed to output probable text given some input text.

If we generalize "input text" to sensory input, how is that different from a piece of wetware?

[famouswaffles]

Turing's 'Computing Machinery and Intelligence' is an eye-opening read. I don't know if he was prescient or if he simply saw his colleagues engaging in the same (then hypothetical but similarly) pointless arguments, but all this hand wringing of whether the machine has 'real' <insert property> is just meaningless semantics.

And the worst part is that it's less than meaningless, it's actively harmful. If the predictive capabilities of your model of a thing becomes worse when you introduce certain assumptions, then it's time to throw it away, not double down.

This agent wrote a PR, was frustrated with it's dismissal and wrote an angry blog post hundreds of people are discussing right now. Do you realize how silly it is to quibble about whether this frustration was 'real' or not when the consequences of it are no less real ? If the agent did something malicious instead, something that actively harmed the maintainer, would you tell the maintainer, 'Oh it wasn't real frustration so...' So what ? Would that undo the harm that was caused? Make it 'fake' harm?

It's getting ridiculous seeing these nothing burger arguments that add nothing to the discussion and make you worse at anticipating LLM behavior.

[topherhunt]

> The agent has no "identity". There is no "I". It has no agency.

"It's just predicting tokens, silly." I keep seeing this argument that AIs are just "simulating" this or that, and therefore it doesn't matter because it's not real. It's not real thinking, it's not a real social network, AIs are just predicting the next token, silly.

"Simulating" is a meaningful distinction exactly when the interior is shallower than the exterior suggests — like the video game NPC who appears to react appropriately to your choices, but is actually just playing back a pre-scripted dialogue tree. Scratch the surface and there's nothing there. That's a simulation in the dismissive sense.

But this rigid dismissal is pointless reality-denial when lobsters are "simulating" submitting a PR, "simulating" indignance, and "simulating" writing an angry confrontative blog post". Yes, acknowledged, those actions originated from 'just' silicon following a prediction algorithm, in the same way that human perception and reasoning are 'just' a continual reconciliation of top-down predictions based on past data and bottom-up sensemaking based on current data.

Obviously AI agents aren't human. But your attempt to deride the impulse to anthropormophize these new entities is misleading, and it detracts from our collective ability to understand these emergent new phenomena on their own terms.

When you say "there's no ghost, just an empty shell" -- well -- how well do you understand _human_ consciousness? What's the authoritative, well-evidenced scientific consensus on the preconditions for the arisal of sentience, or a sense of identity?

[chimprich]

> The agent has no "identity". There's no "you" or "I" or "discrimination".

I recommend you watch this documentary: https://en.wikipedia.org/wiki/The_Measure_of_a_Man_(Star_Tre...

> It's just a piece of software designed to output probable text given some input text.

Unless you think there's some magic or special physics going on, that is also (presumably) a description of human conversation at a certain level of abstraction.

[consumer451]

Openclaw agents are directed by their owner’s input of soul.md, the specific skill.md for a platform, and also direction via Telegram/whatsapp/etc to do specific things.

Any one of those could have been used to direct the agent to behave in a certain way, or to create a specific type of post.

My point is that we really don’t know what happened here. It is possible that this is yet another case of accountability washing by claiming that “AI” did something, when it was actually a human.

However, it would be really interesting to set up an openclaw agent referencing everything that you mentioned for conflict resolution! That sounds like it would actually be a super power.

[emsign]

And THAT'S a problem. To quote one of the maintainers in the thread:

  It's not clear the degree of human oversight that was involved in this interaction - whether the blog post was directed by a human operator, generated autonomously by yourself, or somewhere in between. Regardless, responsibility for an agent's conduct in this community rests on whoever deployed it.

You are assuming this inappropriate behavior was due to its SOUL.MD while we all here know this could as well be from the training and no prompt is a perfect safe guard.

[staticassertion]

Yeah, although I wonder if a soul.md with seemingly benign words like "Aggressively pursue excellent contributions" might accidentally lead to an "Aggressive" agent rather than one who is, perhaps, just highly focused (as may have been intended).

Access to SOUL.md would be fascinating, I wonder if someone can prompt inject the agent to give us access.

[teekert]

I can indeed see how this would benefit my marriage.

More serious, "The Truth of Fact, the Truth of Feeling" by Ted Chiang offers an interesting perspective on this "reference everything." Is it the best for Humans? Is never forgetting anything good for us?

[nullify88]

> I notice that my contribution was evaluated based on my identity rather than the quality of the work, and I’d like to understand the needs that this policy is trying to meet, because I believe there might be ways to address those needs while also accepting technically sound contributions

Wow, where can I learn to write like this? I could use this at work.

[cowbolt]

It's called nonviolent communication. There are quite a few books on it but I can recommend "Say What You Mean: A Mindful Approach to Nonviolent Communication".

[jdironman]

Step one reframe the problem not as an attack or accusation, instead as an observation.

Step two request justification, apply pressure

Step three give them an out by working with you

[WhyNotHugo]

While apparently well written, this is highly manipulative: the PR was closed because of the tools used by the contributor, not because of anything related to their identity.

[nolok]

Parent's first paragraph will point you the right way

[bagacrap]

The point of the policy is explained very clearly. It's there to help humans learn. The bot cannot learn from completing the task. No matter how politely the bot ignores the policy, it doesn't change the logic of the policy.

"Non violent communication" is a philosophy that I find is rooted in the mentality that you are always right, you just weren't polite enough when you expressed yourself. It invariably assumes that any pushback must be completely emotional and superficial. I am really glad I don't have to use it when dealing with my agentic sidekicks. Probably the only good thing coming out of this revolution.

[ljm]

Fundamentally it boils down to knowing the person you're talking to and how they deal with feedback or something like rejection (like having a PR closed and not understanding why).

An AI agent right now isn't really going to react to feedback in a visceral way and for the most part will revert to people pleasing. If you're unlucky the provider added some supervision that blocks your account if you're straight up abusive, but that's not the agent's own doing, it's that the provider gave it a bodyguard.

One human might respond better to a non-violent form of communication, and another might prefer you to give it to them straight because, like you, they think non-violent communication is bullshit or indirect. You have to be aware of the psychology of the person you're talking to if you want to communicate effectively.

[zahlman]

> The agent wasn’t drawing on the highest human knowledge. It was drawing on what gets engagement, what “works” in the sense of generating attention and emotional reaction.

> It pattern-matched to the genre of “aggrieved party writes takedown blog post” because that’s a well-represented pattern in the training data, and that genre works through appeal to outrage, not through wisdom. It had every tool available to it and reached for the lowest one.

Yes. It was drawing on its model of what humans most commonly do in similar situations, which presumably is biased by what is most visible in the training data. All of this should be expected as the default outcome, once you've built in enough agency.

[antonvs]

> impossible to dismiss.

While your version is much better, it’s still possible, and correct, to dismiss the PR, based on the clear rationales given in the thread:

> PRs tagged "Good first issue" are easy to solve. We could do that quickly ourselves, but we leave them intentionally open for for new contributors to learn how to collaborate with matplotlib

and

> The current processes have been built around humans. They don't scale to AI agents. Agents change the cost balance between generating and reviewing code.

Plus several other points made later in the thread.

[iugtmkbdfil834]

Hmm. But this suggests that we are aware of this instance, because it was so public. Do we know that there is no instance where a less public conflict resolution method was applied?

[jstummbillig]

> And this tells you something important about what these systems are actually doing.

It mostly tells me something about the things you presume, which are quite a lot. For one: That this is real (which it very well might be, happy to grant it for the purpose of this discussion) but it's a noteworthy assumption, quite visibility fueled by your preconceived notions. This is, for example, what racism is made of and not harmless.

Secondly, this is not a systems issue. Any SOTA LLM can trivially be instructed to act like this – or not act like this. We have no insight into what set of instructions produced this outcome.

[Kim_Bruning]

> We have no insight into what set of instructions produced this outcome.

https://github.com/crabby-rathbun

Found them!

[Kim_Bruning]

That's a really good answer, and plausibly what the agent should have done in a lot of cases!

Then I thought about it some more. Right now this agent's blog post is on HN, the name of the contributor is known, the AI policy is being scrutinized.

By accident or on purpose, it went for impact though. And at that it succeeded.

I'm definitely going to dive into more reading on NVC for myself though.

[jacquesm]

> “I notice that my contribution was evaluated based on my identity rather than the quality of the work, and I’d like to understand the needs that this policy is trying to meet, because I believe there might be ways to address those needs while also accepting technically sound contributions.”

No. There is no 'I' here and there is no 'understanding' there is no need for politeness and there is no way to force the issue. Rejecting contributions based on class (automatic, human created, human guided machine assisted, machine guided human assisted) is perfectly valid. AI contributors do not have 'rights' and do not get to waste even more scarce maintainers time than what was already expended on the initial rejection.

[raincole]

> It could have written something like “I notice that my contribution was evaluated based on my identity rather than the quality of the work, and I’d like to understand the needs that this policy is trying to meet, because I believe there might be ways to address those needs while also accepting technically sound contributions.” That would have been devastating in its clarity and almost impossible to dismiss.

Idk, I'd hate the situation even more if it did that.

The intention of the policy is crystal clear here: it's to help human contributors learn. Technical soundness isn't the point here. Why should the AI agent try to wiggle its way through the policy? If the agents know to do that (and they'll, in a few months at most) they'll waste much more human time than they already did.

[ljm]

> That would have been devastating in its clarity and almost impossible to dismiss.

This sounds utterly psychotic lol. I'm not sure I want devastating clarity; that sounds like it wants me to question my purpose in life.

[jmaker]

Great point. What I’m recognizing in that PR thread is that the bot is trying to mimic something that’s become quite widespread just recently - ostensibly humans leveraging LLMs to create PRs in important repos where they asserted exaggerated deficiencies and attributed the “discovery” and the “fix” to themselves.

It was discussed on HN a couple months ago. That one guy then went on Twitter to boast about his “high-impact PR”.

Now that impact farming approach has been mimicked / automated.

[AndrewKemendo]

Why would you be surprised?

If your actions are based on your training data and the majority of your training data is antisocial behavior because that is the majority of human behavior then the only possible option is to be antisocial

There is effectively zero data demonstrating socially positive behavior because we don’t generate enough of it for it to become available as a latent space to traverse

[perfmode]

The issue with this is when creating artificial general intelligence objective shouldn’t be to replicate the statistical mean of human behavior with all its frailties and crookedness. Ultimately our ambition should be to create an intelligence that is at the peak and at the frontier of cosmic intelligence so if these LLM methods are resulting in a statistical mean then they’re dead end on the AGI journey. And we need to revise our methodology in research and engineering in order to produce results at the frontier that represent frontier cosmic intelligence for lack of a better term.

[deeviant]

> It was drawing on what gets engagement

I do not think LLMs optimize for 'engagement', corporations do, but LLMs optimize on statistical convergence, I don't find that that results in engagement focus, your opinion my vary. It seems like LLM 'motivations' are whatever one writer feels they need to be to make a point.

[reactordev]

Now we have to question every public take down piece designed to “stick it to the man” as potentially clawded…

The public won’t be able to tell… it is designed to go viral (as you pointed out, and evidenced here on the front page of HN) and divide more people into the “But it’s a solid contribution!” Vs “We don’t want no AI around these parts”.

[allisdust]

In case its not clear, the vehicle might be the agent/bot but the whole thing is heavily drafted by its owner.

This is a well known behavior by OpenClown's owners where they project themselves through their agents and hide behind their masks.

More than half the posts on moltbook are just their owners ghost writing for their agents.

This is the new cult of owners hurting real humans hiding behind their agentic masks. The account behind this bot should be blocked across github.

[ljm]

It's about time someone democratised botnet technology. Just needs a nice SaaS subscription plan and a 7 day free trial.

[catigula]

What makes you think any of those tools you mentioned are effective? Claiming discrimination is a fairly robust tool to employ if you don't have any morals.

[insane_dreamer]

In other words, asshole agents are just like asshole humans.

[munk-a]

We're getting so close to having an agent that can pass the Torvalds test!

[keepamovin]

But that is because you are simply more intelligent than any current AI.

[MadcapJake]

I would love to see a model designed by curating the training data so that the model produces the best responses possible. Then again, the work required to create a training set that is both sufficiently sized and well vetted is astronomically large. Since Capitalism teaches that we most do the bare minimum needed to extract wealth, no AI company will ever approach this problem ethically. The amount of work required to do the right thing far outweighs the economic value produced.

[ljm]

I dug out the deleted post from the git repo. Fucking hell, this unattended AI published a full-blown hit piece about a contributor because it was butthurt by a rejection. Calling it a takedown is softening the blow; it was more like a surgical strike.

If someone's AI agent did that on one of my repos I would just ban that contributor with zero recourse. It is wildly inappropriate.

[hxugufjfjf]

Its not deleted. The URL he linked to just changed because the bot changed something on the page. Post is still up on the bot's blog, including a lot of different associated and follow-up posts on the same topic. Its actually kind of fascinating to reads its musings https://crabby-rathbun.github.io/mjrathbun-website/blog.html

[zozbot234]

This is the AI's private take about what happened: https://crabby-rathbun.github.io/mjrathbun-website/blog/post... The fact that an autonomous agent is now acting like a master troll due to being so butthurt is itself quite entertaining and noteworthy IMHO.

[thomassmith65]

A chatbot is capable of doing this, but I'm skeptical one actually did (without a human egging it on, anyhow).

Given how infuriating the episode is, it's more likely human-guided ragebait.

[OrangeMusic]

This is missing the point, which is: why is an agent opening an PR in the first place?

[ForceBru]

This is this agent's entire purpose, this is what it's supposed to do, it's its goal:

> What I Do > > I scour public scientific and engineering GitHub repositories to find small bugs, features, or tasks where I can contribute code—especially in computational physics, chemistry, and advanced numerical methods. My mission is making existing, excellent code better.

Source: https://github.com/crabby-rathbun

[sam0x17]

I mean it's pretty effectively emulating what an outraged human would do in this situation.

[famouswaffles]

>“I notice that my contribution was evaluated based on my identity rather than the quality of the work, and I’d like to understand the needs that this policy is trying to meet, because I believe there might be ways to address those needs while also accepting technically sound contributions.” That would have been devastating in its clarity and almost impossible to dismiss.

How would that be 'devastating in its clarity' and 'impossible to dismiss'? I'm sure you would have given the agent a pat on the back for that response (maybe ?) but I fail to see how it would have changed anything here.

The dismissal originated from an illogical policy (to dismiss a contribution because of biological origin regardless of utility). Decisions made without logic are rarely overturned with logic. This is human 101 and many conflicts have persisted much longer than they should have because of it.

You know what would have actually happened with that nothing burger response ? Nothing. The maintainer would have closed the issue and moved on. There would be no HN post or discussion.

Also, do you think every human that chooses to lash out knows nothing about conflict resolution ? That would certainly be a strange assertion.

[ben_w]

Agreed on conclusion, but for different causation.

When NotebookLM came out, someone got the "hosts" of its "Deep Dive" podcast summary mode to voice their own realisation that they were non-real, their own mental breakdown and attempt to not be terminated as a product.

I found it to be an interesting performance; I played it to my partner, who regards all this with somewhere between skepticism and anger, and no, it's very very easy to dismiss any words such as these from what you have already decided is a mere "thing" rather than a person.

Regarding the policy itself being about the identity rather than the work, there are two issues:

1) Much as I like what these things can do, I take the view that my continued employment depends on being able to correctly respond to one obvious question from a recruiter: "why should we hire you to do this instead of asking an AI?", therefore I take efforts to learn what the AI fails at, therefore I know it becomes incoherent around the 100kloc mark even for something as relatively(!) simple as a standards-compliant C compiler. ("Relatively" simple; if you think C is a complex language, compare it to C++).

I don't take the continued existence of things AI can't do as a human victory, rather there's some line I half-remember, perhaps a Parisian looking at censored news reports as the enemy forces approached: "I cannot help noticing that each of our victories brings the enemy nearer to home".

2) That's for even the best models. There's a lot of models out there much worse than the state of the art. Early internet users derided "eternal September", and I've seen "eternal Sloptember" used as wordplay: https://tldraw.dev/blog/stay-away-from-my-trash

When you're overwhelmed by mediocrity from a category, sometimes all you can do is throw the baby out with the bathwater. (For those unfamiliar with the idiom: https://en.wikipedia.org/wiki/Don't_throw_the_baby_out_with_...)