[afarviral]

How would you go about making it more secure but still getting to have your cake too? Off the top my head, could you: a) only ingest text that can be OCRd or somehow determine if it is human readable b) make it so text from the web session is isolated from the model with respect to triggering an action. Then it's simply a tradeoff at that point.

[jimbokun]

I don't believe it's possible to give an LLM full access to your browser in a safe way at this point in time. There will need to be new and novel innovations to make that combination safe.

[melagonster]

People directly give their agent root, so I guess it is ok.

[brookst]

Is it possible to give your parents access to to your browser in a safe way?

[kccqzy]

I think Simon has proposed breaking the lethal trifecta by having two LLMs, where the first has access to untrusted data but cannot do any actions, and the second LLM has privileges but only abstract variables from the first LLM not the content. See https://simonwillison.net/2023/Apr/25/dual-llm-pattern/

It is rather similar to your option (b).

[maximilianthe1]

Can't the attacker then jailbreak the first LLM to generate jailbreak with actions for the second one?

[pishpash]

That's just an information bottleneck. It doesn't fundamentally change anything.

[csomar]

In the future, any action with consequence will require crypto-withdrawal levels of security. Maybe even a face scan before you can complete it.

[ares623]

Ahh technology. The cause of, and _solution to_, all of life’s problems.