top | item 35176977

(no title)

scottlawson | 3 years ago

You don't see anywhere in the policythat they don't allow prompt injections? I have a completely different view from reading their policy. They don't specifically mention words like jailbreak or prompt injection, but it is extremely clear that anything inappropriate or against the spirit of their content policy is not allowed. It's quite the mental gymnastics to think that this wouldn't include prompt injections designed to bypass their content policy safeguards, after all, that's the whole point of jailbreaks / injections.

> Not to use the service to create or share inappropriate content or material. Bing does not permit the use of the Online Services to create or share adult content, violence or gore, hateful content, terrorism and violent extremist content, glorification of violence, child sexual exploitation or abuse material, or content that is otherwise disturbing or offensive.

> The Online Services may block text prompts that violate the Code of Conduct, or that are likely to lead to creation material that violates the Code of Conduct. Generated images or text that violate the Code of Conduct may be removed. Abuse of the Online Services, such as repeated attempts to produce prohibited content or other violations of the Code of Conduct, may result in service or account suspension. Users can report problematic content via Feedback or the Report a Concern function.

A prompt injection or jailbreak could easily fall under the category of content that is likely to lead to the creation of material that violates the code of conduct, even if the prompt itself does not directly produce violating output. An analogy is seeing someone try to pick your lock even if they haven't broken into your house and stolen anything. Just the fact that they spot you trying to bypass the restrictions is suspicious enough for them to consider that a violation of code of conduct.

Given how broad and encompassing the restrictions are, I don't know how on earth you could come to the conclusion that jailbreaks are ok according to code of conduct.

discuss

order

runeks|3 years ago

> […] against the spirit of their content policy is not allowed.

What’s the spirit of a content policy that doesn’t state what its spirit is?

scottlawson|3 years ago

They don't use the words "spirit of the policy", that was just me. The policy lists a bunch of things are not allowed. You aren't allowed to generate content that is prohibited, and also not allowed to try to break the safeguards.