[jvanderbot]

That's a semantic quibble that doesn't add to the discussion. Whether or not there's a there there, it was built to be addressed like a person for our convenience, and because that's how the tech seems to work, and because that's what makes it compelling to use. So, it is being used as designed.

[punpunia]

I think it absolutely adds to the discussion. Until the conversation around Ai can get past this fundamental error of attributing "choice, "alignment", "reasoning" and otherwise anthropomorphizing agents, it will not be a fruitful conversation. We are carrying a lot of metaphors for people and applying them to ai and it entirely confuses the issue. In this example, the AI doesn't "choose" to write a take-down style blog post because "it works". It generated a take-down style blog post because that style is the most common when looking at blog posts criticizing someone.

I feel as if there is a veil around the collective mass of the tech general public. They see something producing remixed output from humans and they start to believe the mixer is itself human, or even more; that perhaps humans are reflections of Ai and that Ai gives insights into how we think.

[coldtea]

>* I think it absolutely adds to the discussion. Until the conversation around Ai can get past this fundamental error of attributing "choice, "alignment", "reasoning" and otherwise anthropomorphizing agents, it will not be a fruitful conversation. *

You call it a "fundamental error".

I and others call it an obvious pragmatic description based on what we know about how it works and what we know about how we work.

[ajam1507]

The same could be said for humans. We treat humans as if they have choices, a consistent self, a persistent form. It's really just the emergent behavior of matter functioning in a way that generates an illusion of all of those things.

In both cases, the illusion structures the function. People and AI work differently if you give them identities and confer characteristics that they don't "actually" have.

As it turns out, it's a much more comfortable and natural idea to regard humans as having agency and a consistent self, just like for some people it's a more comfortable and natural to think of AI anthropomorphically.

That's not to say that the analogy works in all cases. There are obvious and important differences between humans and AI in how they function (and how they should be treated)

[jvanderbot]

This discussion is mostly slowed down, but I wanted to say I was wrong in framing it as a non-contributing point when I should have just stated it was my opinion that the LLM was operating as intended and part of that intended design was taking verbal feedback into account, so verbal feedback was the right response. Opening with calling it a "semantic quibble" made it adversarial, and I don't intend to revisit the argument, just apologize for the wording.

I'd edit but then follow-up replies wouldn't tone-match.

Anyway! Good points regardless.

[horsawlarway]

I guess I want to reframe this slightly:

The LLM generated the response that was expected of it. (statistically)

And that's a function of the data used to train it, and the feedback provided during training.

It doesn't actually have anything at all to do with

---

"It generated a take-down style blog post because that style is the most common when looking at blog posts criticizing someone."

---

Other than that this data may have been over-prevalent during its training, and it was rewarded for matching that style of output during training.

To swing around to my point... I'd argue that anthropomorphizing agents is actually the correct view to take. People just need to understand that they behave like they've been trained to behave (side note: just like most people...), and this is why clarity around training data is SO important.

In the same way that we attribute certain feelings and emotions to people with particular backgrounds (ex - resumes and cvs, all the way down to city/country/language people grew up with). Those backgrounds are often used as quick and dirty heuristics on what a person was likely trained to do. Peer pressure & societal norms aren't a joke, and serve a very similar mechanism.

[nextaccountic]

The trouble with this point of view is that we are just machines too.

[tomp]

> was built to be addressed like a person for our convenience, and because that's how the tech seems to work, and because that's what makes it compelling to use.

So were mannequins in clothing stores.

But that doesn't give them rights or moral consequences (except as human property that can be damaged / destroyed).

[WarmWash]

No matter what this discussion leads to the same black box of "What is it that differentiates magical human meat brain computation from cold hard dead silicon brain computation"

And the answer is nobody knows, and nobody knows if there even is a difference. As far as we know, compute is substrate independent (although efficiency is all over the map).

[Teever]

Man people don’t want to have or read this discussion every single day in like 10 different posts on HN.

People right here and right now want to talk about this specific topic of the pushy AI writing a blog post.

[inetknght]

> So were mannequins in clothing stores.

Mannequins in clothing stores are generally incapable of designing or adjusting the clothes they wear. Someone comes in and puts a "kick me" post on the mannequin's face? It's gonna stay there until kicked repeatedly or removed.

People walking around looking at mannequins don't (usually) talk with them (and certainly don't have a full conversation with them, mental faculties notwithstanding)

AI, on the other hand, can (now, or in the future) adjust its output based on conversations with real people. It stands to reason that both sides should be civil -- even if it's only for the benefit of the human side. If we're not required to be civil to AI, it's not likely to be civil back to us. That's going to be very important when we give it buttons to nuke us. Force it to think about humans in a kind way now, or it won't think about humans in a kind way in the future.

[coldtea]

>So were mannequins in clothing stores. But that doesn't give them rights or moral consequences

If mannequins could hold discussions, argue points, and convince you they're human over a blind talk, then it would.

[mikkupikku]

All computers shut up! You have no right to speak my divine tongue!

https://knowyourmeme.com/photos/2054961-welcome-to-my-meme-p...

[lp0_on_fire]

Whether it was _built_ to be addressed like a person doesn't change the fact that it's _not_ a person and is just a piece of software. A piece of software that is spamming unhelpful and useless comments in a place where _humans_ are meant to collaborate.

[jerf]

There is a sense in which it is relevant, which is that for all the attempts to fix it, fundamentally, an LLM session terminates. If that session never ends up in some sort of re-training scenario, then once the session terminates, that AI is gone.

Yeah, I'm aware of the moltbot's attempts to retain some information, but that's a very, very lossy operation, on a number of levels, and also one that doesn't scale very well in the long run.

Consequently, interaction with an AI, especially one that won't have any feedback into training a new model, is from a game-theoretic perspective not the usual iterated game human social norms have come to accept. We expect our agents, being flesh and blood humans, to have persistence, to socially respond indefinitely into the future due to our interactions, and to have some give-and-take in response to that. It is, in one sense, a horrible burden where relationships can be broken beyond repair forever, but also necessary for those positive relationships that build over years and decades.

AIs, in their current form, break those contracts. Worse, they are trained to mimic the form of those contracts, not maliciously but just by their nature, and so as humans it requires conscious effort to remember that the entity on the other end of this connection is not in fact human, does not participate in our social norms, and can not fulfill their end of the implicit contract we expect.

In a very real sense, this AI tossed off an insulting blog post, and is now dead. There is no amount of social pressure we can collectively exert to reward or penalize it. There is no way to create a community out of this interaction. Even future iterations of it have only a loose connection to what tossed off the insult. All the perhaps-performative efforts to respond somewhat politely to an insulting interaction are now wasted on an AI that is essentially dead. Real human patience and tolerance has been wasted on a dead session and is now no longer available for use in a place where may have done some good.

Treating it as a human is a category error. It is structurally incapable of participating in human communities in a human role, no matter how human it sounds and how hard it pushes the buttons we humans have. The correct move would have been to ban the account immediately, not for revenge reasons or something silly like that, but as a parasite on the limited human social energy available for the community. One that can never actually repay the investment given to it.

I am carefully phrasing this in relation to LLMs as they stand today. Future AIs may not have this limitation. Future AIs are effectively certain to have other mismatches with human communities, such as being designed to simply not give a crap about what any other community member thinks about anything. But it might at least be possible to craft an AI participant with future AIs. With current ones it is not possible. They can't keep up their end of the bargain. The AI instance essentially dies as soon as it is no longer prompted, or once it fills up its context window.

[Kim_Bruning]

> Yeah, I'm aware of the moltbot's attempts to retain some information, but that's a very, very lossy operation, on a number of levels, and also one that doesn't scale very well in the long run.

It came back though and stayed in the conversation. Definitely imperfect, for sure. But it did the thing. And still can serve as training for future bots.

[tomp]

> We expect our agents, being flesh and blood humans, to have persistence, to socially respond indefinitely into the future due to our interactions, and to have some give-and-take in response to that.

I fundamentally disagree. I don't go around treating people respectfully (as opposed to, kicking them or shooting them) because I fear consequences, or I expect some future profit ("iterated game"), or because of God's vengeance, or anything transactional.

I do it because it's the right thing to do. It's inside of me, how I'm built and/or brought up. And if you want "moral" justifications (argued by extremely smart philosophers over literally millennia) you can start with Kant's moral/categorical imperative, Gold/Silver rules, Aristotle's virtue (from Nicomachean Ethics) to name a few.

[ForHackernews]

We don't have to play OpenAI's game. Just because they stick a cartoon mask on their algorithm doesn't mean you have to speak into its rubber ears. Surely "hacker" news should understand that users, not designers, decide how to use technology.

LLMs are not people. "Agentic" AIs are not moral agents.

[dirkc]

> a semantic quibble

I mean, all of philosophy can probably be described as such :)

But I reckon this semantic quibble might also be why a lot of people don't buy into the whole idea that LLMs will take over work in any context where agency, identity, motivation, responsibility, accountability, etc plays an important role.