[OakNinja]

"IBM Bob is IBM’s new coding agent, currently in Closed Beta. "

Promptarmor did a similar attack(1) on Google's Antigravity that is also a beta version. Since then, they added secure mode(2).

These are still beta tools. When the tools are ready, I'd argue that they will probably be safer out of the box compared to a whole lot of users that just blindly copy-paste stuff from the internet, adding random dependencies without proper due diligence, etc. These tools might actually help users acting more secure.

I'm honestly more worried about all the other problems these tools create. Vibe coded problems scale fast. And businesses have still not understood that code is not an asset, it's a liability. Ideally, you solve your business problems with zero lines of code. Code is not expensive to write, it's expensive to maintain.

(1) https://www.promptarmor.com/resources/google-antigravity-exf... (2) https://antigravity.google/docs/secure-mode

[InsideOutSanta]

While they have found some solvable issues (e.g. "the defense system fails to identify separate sub-commands when they are chained using a redirect operator"), the main issue is unsolvable. If you allow an LLM to edit your code and also give it access to untrusted data (like the Internet), you have a security problem.

[iLoveOncall]

> If you allow an LLM to edit your code and also give it access to untrusted data (like the Internet), you have a security problem.

You don't even need to give it access to Internet to have issues. The training data is untrusted.

It's a guarantee that bad actors are spreading compromised code to infect the training data of future models.

[derektank]

A problem yes, but I think GP is correct in comparing the problem to that of human workers. The solution there has historically been RBAC and risk management. I don’t see any conceptual difference between a human and an automated system on this front

[mistrial9]

no, you have a trust problem. Is the tool assisting, or is are the tools the architect, builder, manager, court and bank?

[acessoproibido]

>If you allow a human to edit your code and also give them access to untrusted data (like the Internet), you have a security problem.

Security shouldn't be viewed in absolutes (either you are secure or you aren') but more in degrees. Llms can be used securely just the same as everything else, nothing is ever perfectly secure

[strken]

I have an issue with the "code is a liability" framing. Complexity and lack of maintainability are the ultimate liabilities behind it. Code is often the least worst alternative for solving a given problem compare to unstructured data in spreadsheets, no-code tools without a version history, webs of Zapier hooks, opaque business processes that are different for every office, or whatever other alternatives exist.

It's a good message for software engineers, who have the context to understand when to take on that liability anyway, but it can lead other job functions into being too trigger-happy on solutions that cause all the same problems with none of the mitigating factors of code.

[Eufrat]

> When the tools are ready, I'd argue that they will probably be safer out of the box compared to a whole lot of users that just blindly copy-paste stuff from the internet, adding random dependencies without proper due diligence, etc. These tools might actually help users acting more secure.

This speculative statement is holding way too much of the argument that they are just “beta tools”.

[cyanydeez]

You would think so, but you should read about how they bear proof trash cans in yellow stone.

They cant. Why? Because the smartest bear ia smarter than the dumbest human.

So, these AIs are suppose to interface with humans and use nondeterminant language.

That vector will always be exploitable, unless youre talking about AI that no han controls.

[OakNinja]

Yes. But the exploitable vector in this case is still humans. AI is just a tool.

The non-deterministic nature of an LLM can also be used to catch a lot of attacks. I often use LLM’s to look through code, libraries etc for security issues, vulnerabilities and other issues as a second pair of eyes.

With that said, I agree with you. Anything can be exploited and LLM’s are no exception.