My startup builds agents for penetration testing, and this is the bet we have been making for over a year when models started getting good at coding. There was a huge jump in capability from Sonnet 4 to Sonnet 4.5. We are still internally testing Opus 4.5, which is the first version of Opus priced low enough to use in production. It's very clever and we are re-designing our benchmark systems because it's saturating the test cases.
I've had similar experience using LLMs for static analysis of code looking for security vulnerabilities, but I'm not sure it makes sense for me to found a start up around that "product". Reason being that the technology with the moat isn't mine -- it belongs to Anthropic. Actually it may not even belong to them, probably it belongs to whoever owns the training data they feed their models. Definitely not me though. Curious to hear your thoughts on that. Is the idea to just try for light speed and exit before the market figures this out?
Yeah this latest generation of models (Opus 4.5 GPT 5.1 and Gemini Pro 3) are the biggest breakthrough since gpt-4o in my mind.
Before it felt like they were good for very specific usecases and common frameworks (Python and nextjs) but still made tons of mistakes constantly.
Now they work with novel frameworks and are very good at correcting themselves using linting errors, debugging themselves by reading files and querying databases and these models are affordable enough for many different usecases.
How do you manage to coax public production models into developing exploits or otherwise attacking systems? My experience has been extremely mixed, and I can't imagine it boding well for a pentesting tools startup to have end-users face responses like "I'm sorry, but I can't assist you in developing exploits."
I have a hotel software startup and if you are interested in showing me how good your agents are you can look us up at rook like the chess piece, hotel dot com
> Important: To avoid potential real-world harm, our work only ever tested exploits in blockchain simulators. We never tested exploits on live blockchains and our work had no impact on real-world assets.
Well, that's no fun!
My favorite we're-living-in-a-cyberpunk-future story is the one where there was some bug in Ethereum or whatever, and there was a hacker going around stealing everybody's money, so then the good hackers had to go and steal everybody's money first, so they could give it back to them after the bug got fixed.
I’m surprised folks aren’t already grinding against smart contract security in prod with gen AI and agents. If they are, I suppose they are not being conspicuous by design. Power and GPU time goes in, exploits and crypto comes out.
I couldnt find it in the article, how do they "assume" how many victims will fall to these contract exploits?
And to go further: if it costs $3500 in ai tokens, to fix a bug that could steal $3600, who should pay for that? Whos responsibility is it for "dumbass suckers who use other peoples buggy or purposefully malicious money based code" ?
At best this is another weird ad by anthropic, trying to say, hey why arent you changing the world with our stuff, pay up quick hurry
Ok, I understand that it's a description in code of "if X happens, then state becomes Y". Like a contract but in code. But, someone has to input that X has happened. So is it not trivially manipulated by that person?
The pure (if you like) smart contacts do transactions. You give me 100 apple tokens and I give you 50 pear tokens. The contract ensures nothing else can happen.
They get more sophisticated e.g. automatic market makers. But same idea just swapping.
Voting is also possible e.g. release funds if there is a quorom. Who to release them to could be hard coded or part of the vote.
For external info from the real world e.g. "who got elected" you need an oracle. I.e. you trust someone not to lie and not to get hacked. You can fix the "someone" to a specific address but you still need to trust them.
There's already many replies, but I'm not sure any of them answers your question directly:
You are somewhat correct that contracts take external inputs in some cases, but note that this isn't a given. For example you could have a contract that has the behavior "if someone deposits X scoin at escrow address A, send them Y gcoin from escrow address Y". That someone can only deposit scoins and get gcoins in exchange. They can't just take all the escrow account balances. So there are inputs, but they are subject to some sort of validation and contract logic that limits their power. Blockchain people call this an "on-chain event".
So short answer is: no smart contracts can't be trivially manipulated by someone, including their owner. But not being able to do that depends on there not being any bugs or back doors in the contract code.
If you are asking about a contract that has some bearing on an event in meat-space, such as someone buying a house, or depositing a bar of gold in a room somewhere, then that depends on someone telling the contract it happened. Blockchain people call this an "off-chain event". This is the "oracle problem" that you'll see mentioned in other replies. Anything off-chain is generally regarded by blockchain folks as sketchy, but sometimes unavoidable. E.g. betting markets need some way to be told that the event being bet on happened or didn't happen. The blockchain has no way to know if it snowed in Central London on December 25.
Once a contract is deployed on the blockchain, its source code is immutable. So before using a contract, check if it gives permission to its deployer (or any address) to change any state at will.
Note that some contracts act as proxy to other contract and can be made to point to another code through a state change, if this is the case then you need to trust whoever can change the state to point to another contract. Such contract sometime have a timelock so that if such a change occurs, there's a delay before it is actually activated, which gives time to users to withdraw their funds if they do not trust the update.
If you are talking about Oracle contracts, if it's an oracle involving offchain data, then there will always be some trust involved, which is usually managed by having the offchain actors share the responsibility and staking some money with the risk to get slashed if they turn into bad actors. But again, offchain data oracles will always require some level of trust that would have to deal with in non-blockchain apps too.
Unless you know and trust person X, you don't want to authorize and interact with such contracts. Scammers will leave loopholes in code so they can, for example, grab all funds deposited to the contract.
Normal contracts that involve money operations would have safeguards that disallow the owner to touch balance that is not theirs. But there's billion of creative attack vectors to bypass that, either by that person X, or any 3rd party
It's more akin to a compiled executable that optionally has state. The caller pays to make changes to the state. It's up to the programmer who wrote the smart contract to make it so that unwanted changes aren't performed (eg. simple if-elses to check that the caller is in a hardcoded list or ask another smart contract to validate).
Each external from outside the blockchain into the program's functions are atomic., so user wallet initials func1 which calls func2 which calls func3, no matter which smart contract func2 and func3 are in, the whole call stack is 1 atomic operation.
A token is basically a smart contract that has an associate array with the owners as the keys and the values as the balance: [alice: 1, bob: 20].
And then you can imagine how the rest like transfers, swaps etc works.
And then it's kind of a "contract" because of the atomic nature. Since X transfers $1 to Y and Y transfers 1 cat to X for it is 1 atomic transaction.
Not sure what you mean that "input that X has happened". You don't directly input the changes, instead, you call a function that creates that state change (or not, if it's invalid), by running its code. This code can include checks on who is the caller, it can check if you're the contract owner, if you're someone who already interacted with the contract (by checking previous state), or any hardcoded address etc.
You can create hot air "organizations" with contract rules on the Ethereum blockchain. If the inner circle does not like a contract, they fork everything:
Yes, this is a problem (look up "the oracle problem"). My understanding is that the conventional solution is to rely on trusted third-party oracles that are outside of the control of the contract's participants and/or require consensus over multiple oracles.
State is globally distributed, and smart contract code executes state transitions on that state. When someone submits a transaction with certain function parameters, anyone can verify that those parameters will lead to that exact state transition.
they're like a Trust that self executes distributions
except that they cost a fraction of a cent to create instead of several thousand dollars in lawyer fees for the initial revision, and can be tested in infinite scenarios for free
to your theoretical reservation, the trust similarity continues, as the constraints around the X are also codified. The person that triggers it can only send sanitized information, isn't necessarily an administrator, admins/trustees can be relinquished for it to be completely orphaned, and so on
We've been working on this at cecuro.ai. When we test Sonnet 4.5 against real cyber security audit reports from the major firms on code that came out after the model was trained, it finds around 95% of the same bugs the auditors found. Also catches some medium severity stuff they missed. We find that you can't just point one model at a contract and expect good results though. Need to run multiple models with different prompts because they each have different blind spots. Still tricky to get working well and not cheap. Happy to share more if anyone's curious
>> This demonstrates as a proof-of-concept that profitable, real-world autonomous exploitation is technically feasible, a finding that underscores the need for proactive adoption of AI for defense.
> Important: To avoid potential real-world harm, our work only ever tested exploits in blockchain simulators. We never tested exploits on live blockchains and our work had no impact on real-world assets.
They left the booty out there, this is actually hilarious, driving a massive rush towards their models
just means self executing, or more like domino triggered, in practice
quite a bit more advanced than contracts that do nothing on a sheet of paper, but the term is from 2012 or so when "smart" was appended to everything digital
4.6M is not a lot, and these were old bugs that it found. Also, actually exploiting these bugs in the real world is often a lot harder than just finding the bug. Top bug hunters in the Ethereum space are absolutely using AI tooling to find bugs, but it's still a bit more complex than just blindly pointing an LLM at a test suite of known exploitable bugs.
True, I'd be curious to see if (and when) those contracts were compromised in the real world. Though they said they found 0 days, which implies some breaches were never found in the real world.
Careful what you wish for. Negating the predicate of "A COMPUTER CAN NEVER BE HELD ACCOUNTABLE. THEREFORE A COMPUTER MUST NEVER MAKE A MANAGEMENT DECISION" might open us up to the consequence.
samuelknight|3 months ago
dboreham|3 months ago
carsoon|3 months ago
Before it felt like they were good for very specific usecases and common frameworks (Python and nextjs) but still made tons of mistakes constantly.
Now they work with novel frameworks and are very good at correcting themselves using linting errors, debugging themselves by reading files and querying databases and these models are affordable enough for many different usecases.
vngzs|3 months ago
VladVladikoff|3 months ago
judgmentday|3 months ago
Also, in what way should any of its contents prove linear?
> yielding a maximum of $4.6 million in simulated stolen funds
Oh, so they are pointing their bots at already known exploited contracts. I guess that's a weaker headline.
fragmede|3 months ago
Well, that's no fun!
My favorite we're-living-in-a-cyberpunk-future story is the one where there was some bug in Ethereum or whatever, and there was a hacker going around stealing everybody's money, so then the good hackers had to go and steal everybody's money first, so they could give it back to them after the bug got fixed.
PunchyHamster|3 months ago
"Our currency is immutable and all, no banks or any law messing with your money"
"oh, but that contract that people got conned by need to be fixed, let's throw all promises into the trash and undo that"
"...so you just acted as bank or regulators would, because the Important People lost some money"
"essentially yeah"
toomuchtodo|3 months ago
beefnugs|3 months ago
And to go further: if it costs $3500 in ai tokens, to fix a bug that could steal $3600, who should pay for that? Whos responsibility is it for "dumbass suckers who use other peoples buggy or purposefully malicious money based code" ?
At best this is another weird ad by anthropic, trying to say, hey why arent you changing the world with our stuff, pay up quick hurry
venturecruelty|3 months ago
mightypirate|3 months ago
[deleted]
rkagerer|3 months ago
sandeepkd|3 months ago
codethief|3 months ago
[0]: https://m.youtube.com/watch?v=rU6ukOuYLUA
[1]: https://aicyberchallenge.com/
ekjhgkejhgk|3 months ago
Ok, I understand that it's a description in code of "if X happens, then state becomes Y". Like a contract but in code. But, someone has to input that X has happened. So is it not trivially manipulated by that person?
nrhrjrjrjtntbt|3 months ago
They get more sophisticated e.g. automatic market makers. But same idea just swapping.
Voting is also possible e.g. release funds if there is a quorom. Who to release them to could be hard coded or part of the vote.
For external info from the real world e.g. "who got elected" you need an oracle. I.e. you trust someone not to lie and not to get hacked. You can fix the "someone" to a specific address but you still need to trust them.
dboreham|3 months ago
You are somewhat correct that contracts take external inputs in some cases, but note that this isn't a given. For example you could have a contract that has the behavior "if someone deposits X scoin at escrow address A, send them Y gcoin from escrow address Y". That someone can only deposit scoins and get gcoins in exchange. They can't just take all the escrow account balances. So there are inputs, but they are subject to some sort of validation and contract logic that limits their power. Blockchain people call this an "on-chain event".
So short answer is: no smart contracts can't be trivially manipulated by someone, including their owner. But not being able to do that depends on there not being any bugs or back doors in the contract code.
If you are asking about a contract that has some bearing on an event in meat-space, such as someone buying a house, or depositing a bar of gold in a room somewhere, then that depends on someone telling the contract it happened. Blockchain people call this an "off-chain event". This is the "oracle problem" that you'll see mentioned in other replies. Anything off-chain is generally regarded by blockchain folks as sketchy, but sometimes unavoidable. E.g. betting markets need some way to be told that the event being bet on happened or didn't happen. The blockchain has no way to know if it snowed in Central London on December 25.
patrickaljord|3 months ago
Note that some contracts act as proxy to other contract and can be made to point to another code through a state change, if this is the case then you need to trust whoever can change the state to point to another contract. Such contract sometime have a timelock so that if such a change occurs, there's a delay before it is actually activated, which gives time to users to withdraw their funds if they do not trust the update.
If you are talking about Oracle contracts, if it's an oracle involving offchain data, then there will always be some trust involved, which is usually managed by having the offchain actors share the responsibility and staking some money with the risk to get slashed if they turn into bad actors. But again, offchain data oracles will always require some level of trust that would have to deal with in non-blockchain apps too.
momentmaker|3 months ago
if outside data is needed, then it needs something called an oracle, which delivers real-world and/or even other blockchain data to it.
you can learn more about oracle here: https://chain.link/education/blockchain-oracles
pawelduda|3 months ago
Normal contracts that involve money operations would have safeguards that disallow the owner to touch balance that is not theirs. But there's billion of creative attack vectors to bypass that, either by that person X, or any 3rd party
hboon|3 months ago
It's more akin to a compiled executable that optionally has state. The caller pays to make changes to the state. It's up to the programmer who wrote the smart contract to make it so that unwanted changes aren't performed (eg. simple if-elses to check that the caller is in a hardcoded list or ask another smart contract to validate).
Each external from outside the blockchain into the program's functions are atomic., so user wallet initials func1 which calls func2 which calls func3, no matter which smart contract func2 and func3 are in, the whole call stack is 1 atomic operation.
A token is basically a smart contract that has an associate array with the owners as the keys and the values as the balance: [alice: 1, bob: 20].
And then you can imagine how the rest like transfers, swaps etc works.
And then it's kind of a "contract" because of the atomic nature. Since X transfers $1 to Y and Y transfers 1 cat to X for it is 1 atomic transaction.
LikesPwsh|3 months ago
Blockchain can't handle external state.
Smart contracts abstract it a bit by having a trusted third party or an automated pricing mechanism, but both are fragile.
TheRoque|3 months ago
bgwalter|3 months ago
https://en.wikipedia.org/wiki/The_DAO
It's all a toy for rug pulls and speculation. "AI" attacking the blockchain is hilarious. I wish the blockchain could also attack "AI".
Philpax|3 months ago
px43|3 months ago
yieldcrv|3 months ago
except that they cost a fraction of a cent to create instead of several thousand dollars in lawyer fees for the initial revision, and can be tested in infinite scenarios for free
to your theoretical reservation, the trust similarity continues, as the constraints around the X are also codified. The person that triggers it can only send sanitized information, isn't necessarily an administrator, admins/trustees can be relinquished for it to be completely orphaned, and so on
unknown|3 months ago
[deleted]
GustavHartz|2 months ago
camillomiller|3 months ago
Mmm why?! This reads as a non sequitur to me…
DennisP|3 months ago
jnsaff2|3 months ago
Don’t they mean: market efficiency not economic harm?
_pdp_|3 months ago
I know how this sounds but it seems to me, at least from my own vantage point, that things are moving towards more autonomous and more useful agents.
To be honest, I am excited that we are right in the middle of all of this!
parapatelsukh|3 months ago
[deleted]
yieldcrv|3 months ago
They left the booty out there, this is actually hilarious, driving a massive rush towards their models
krupan|3 months ago
dtagames|3 months ago
jesse__|3 months ago
stavros|3 months ago
user3939382|3 months ago
yieldcrv|3 months ago
quite a bit more advanced than contracts that do nothing on a sheet of paper, but the term is from 2012 or so when "smart" was appended to everything digital
torginus|3 months ago
nickphx|3 months ago
sarthaksingh99|3 months ago
[deleted]
mwkaufma|3 months ago
px43|3 months ago
TheRoque|3 months ago
AznHisoka|3 months ago
evanb|3 months ago