[porphyra]

Obviously I think that AI generated undressing pictures of people, especially minors, is bad and there should be safeguards against that. But how is it different from other tools like doing it manually with photoshop? Also it has been shown that many other tools like ChatGPT and Gemini/Nanobanana can also do it with sufficiently creative prompting.

I also did scroll through the public grok feed and the AI generated bikini pics were mostly Onlyfans creators requesting their own fans to generate these pictures (or sometimes generating them themselves).

[cptaj]

You know the answer to this but I'll just say it: Its different in that it requires no skill and can be done by anyone instantaneously at scale.

You know this but somehow are rationalizing this game changing fact away.

Yes, people can draw and photoshop things. But it takes time, skill, dedication, etc. This time cost is load bearing in the way society needs to deal with the tools it has for the same reason at the extreme that kitchen knives have different regulations than nuclear weapons.

It is also trivially easy for grok to censor this usage for the vast majority of offenders by using the same LLM technology they already have to classify content created by their own tools. Yes, it could get jailbroken but that requires skill, time, dedication, etc; And it can be rapidly patched, greatly mitigating the scale of abuse.

[Y-bar]

> But how is it different from other tools like doing it manually with photoshop?

The scale of effect and barrier to entry. Both are orders of magnitude easier and faster. It would take hours of patience and work to mostly create one convincing fake using photoshop, once you had spent the time and money to learn the tool and acquire it. This creates a natural large moat to the creation process. With Groom it takes a minute at most with no effort or energy needed.

And then there is the ease of distribution to a wide audience, X/Groom handles that for you by automatically giving you an audience of millions.

It’s like with guns. Why prevent selling weapons to violent offenders when they could just build their own guns from high quality steel, a precision drill, and a good CNC machine? Scale and barrier to entry are real blockers for a problem to mostly solve itself. And sometimes a 99% solution is better than no solution.

[porphyra]

This thing with guns was a legitimate argument for banning or regulating 3D printers a few years ago though and I'm glad that we didn't end up with restrictions on that front. With affordable desktop CNC machines capable of making metal parts coming soon, I hope those won't be subject to too many restrictions also.

[Permit]

> Obviously I think that AI generated undressing pictures of people, especially minors, is bad and there should be safeguards against that.

It's not obvious to me that this is your position. What safeguards do you propose as an alternative to those discussed in the article?

[porphyra]

I am for moderation and strong penalties for users that use it in that manner. Anyone who uses grok to generate an undressing image of someone without their consent within 5 seconds should probably go to jail or whatever the penalty is for someone spending 5 hours to create revenge porn with photoshop.

But I'm not sure if the tool itself should be banned, as some people seem to be suggesting. There are content creators on the platform that do use NSFW image generation capabilities in a consensual and legitimate fashion.

[kranke155]

Grom is much much less censored on purpose. I work in image editing and outside of very few people, hardly anyone uses Grok for professional work. Nano Banana Pro is used for the most part.

But for NSFW work it dominates. It’s clearly deliberate.

[ImPostingOnHN]

If you generate CSAM, whether using LLMs, photoshop, or any other tool, you are breaking the law. This would apply if you could somehow run Grok locally.

When you use a service like Grok now, the service is the one using the tool (Grok model) to generate it, and thus the service is producing CSAM. This would also apply if you paid someone to use Photoshop to produce CSAM: they would be breaking the law in doing so.

This is setting aside the issue of twitter actually distributing the CSAM.

[Jordan-117]

The Photoshop equivalent would be "an Adobe artist does the photoshop for you and then somehow emails it directly to your target and everyone who follows them."

[drawfloat]

Drawing indecent photos of children with Photoshop is also illegal in lots of countries and any company creating them for profit would be liable.

[ekjhgkejhgk]

Exactly. And the fact that companies do it with impunity is another hint that we're living in late stage capitalism.

If an individual invented a tool that can generate such pictures, he'd be arrested immediately. A company does it, it's just a woopsie. And most people don't find this strange.

[Waterluvian]

I think intent probably matters and that this gets into the "you know it when you see it" definition realm where we debate the balance between freedom of speech and security of person. ie. just how easy Photoshop, a VCR, a DVD burner app, etc. makes it for you to crime and how much are they handholding you towards criming?

I think this is an important question to ask despite the subject matter because the subject matter makes it easy for authorities to scream, "think of the children you degenerate!" while they take away your freedoms.

I think Musk is happy to pander to and profit from degeneracy, especially by screaming, "it's freedom of speech!" I would bet the money in my pocket that his intent is that he knows this stuff makes him more money than if he censored it. But he will of course pretend it's about 1A freedoms.

[cosmic_cheese]

Friction/barrier to entry is the biggest difference. People generally didn't do things like that before due to a combination of it being a colossal waste of time and most not having the requisite skills (or will and patience to acquire said skills). When all it takes is @mentioning a bot, that friction is eliminated.

[array_key_first]

How is the atomic bomb different than me going to a foreign country and manually stabbing 500,000 people in the throat?

I would say lots of ways. And that's probably why I have a few knives, and zero atomic bombs.

[XorNot]

For one liability: the person doing the Photoshop is the one liable for it, and it was never okay to do this without consent.

[pphysch]

The obvious problem is that Grok is also distributing the illegal images.

This could be easily fixed by making the generated images sent through private Grok DMs or something, but that would harm the bottom line. Maybe they will do that eventually once they have milked enough subscriptions from the "advertising".

[simianwords]

why does it matter if grok is advertising or you are advertising? in reality there's no difference. its just a tool you can invoke.

[caconym_]

How is having cameras on every street corner that identify you based on your face and height and weight and gait and the clothes you're wearing and anything you're carrying, or the car you're driving by its license plate and make and model and color and tires/rims and any visible damage, accessories, etcetera, and taking all these data points and loading them into a database that cross-correlates them with your credit bureau data and bank records and purchase history and social media and other online activity and literally every single other scrap of available data everywhere, and builds a map of everything about you and everywhere you ever go and everything you do and have ever done, makes it trivially queryable by any law enforcement officer in the country with or without a valid reason, retains it all in perpetuity, and does all this for every single person in the country without consent or a warrant issued by a judge, different from a police department assigning an officer to tail you if you are suspected of being involved in a crime?

We are going to be in some serious fucking trouble if we can't tackle these issues of scale implied by modern information technology without resorting to disingenuous (or simply naive) appeals to these absurd equivalences as justification for each new insane escalation.

[etchalon]

"Technically, anyone can make napalm at home. What's wrong with Walmart selling it?"

[kllrnohj]

> But how is it different from other tools like doing it manually with photoshop?

Last I checked Photoshop doesn't have a "undress this person" button? "A person could do bad thing at a very low rate, so what's wrong with automating it so that bad things can be done millions of times faster?" Like seriously? Is that a real question?

But also I don't get what your argument is, anyway. A person doing it manually still typically runs into CSAM or revenge porn laws or other similar harassment issues. All of which should be leveraged directly at these AI tools, particularly those that lack even an attempt at safeguards.

[glemion43]

It circumvents basic laws in plenty of countries giving kids access to these tools.

It could easily be solved by basic age verification.

The csam stuff though needs to be filtered and fixed as this breaks laws and I'm not aware what would make it legal, lucky enough