[simmerup]

Terrifying that people are creating financial models with AI when they don’t have the skills to verify the model does what they expect

[nebula8804]

All we need is one major crash caused by AI to scare the capital owners. Then maybe us white collar workers can breath a bit for at least another few more years(maybe a decade+).

[onion2k]

All we need is one major crash caused by AI to scare the capital owners.

All the previous human-driven crashes didn't change anything about capital owners' approach to money, so why would an AI-driven crash change things?

[pizzafeelsright]

Most of the large outages at Meta in the past 10 years were related to early AI automation.

[danielbln]

A decade+ is wishful copium.

[martinald]

They have an excel sheet next to it - they can test it against that. Plus they can ask questions if something seems off and have it explain the code.

[AlotOfReading]

I'm not sure being able to verify that it's vaguely correct really solves the issue. Consider how many edge cases inhabit a "30 sheet, mind-numbingly complicated" Excel document. Verifying equivalence sounds nontrivial, to put it mildly.

[lmm]

> They have an excel sheet next to it - they can test it against that.

It used to be that we'd fix the copy-paste bugs in the excel sheet when we converted it to a proper model, good to know that we'll now preserve them forever.

[chrisjj]

That will get less terrifying when they move on to medical.

[myfakebadcode]

I’m trying to learn rust coming from python (for fun). I use various LLM for python and see it stumble.

It is a beautiful experience to realize wtf you don’t know and how far over their skis so many will get trusting AI. The idea of deploying a rust project at my level of ability with an AI at the helm is is terrifying.

[taneq]

If they have the skills to verify the Excel model then they can apply the same approach to the numbers produced by the AI-generated model, even if they can’t inspect it directly.

In my experience a lot of Excel models aren’t really tested, just checked a bit and them deemed correct.

[riskable]

I work for a huge bank: The actual terrifying thing is that these financial models semi-technical management are creating in Excel are actually relied upon in the first place.

If you convert bullshit from Excel to Python it's still bullshit. There's a reason why Claude can one-shot it and no one questions the result :D

[theshrike79]

The Python bullshit at least can be unit tested bit by bit and easily improved.

Nobody will see that on sheet 27 cell FG456 is actually a static number that Brian typoed in there in 2019 and not a formula.

[riscii68]

Yep. And I think many are too focused on just the pure ability to build models now, when in reality a model is at its core a set of assumptions. Using models you don't understand, didn't build, and barely reviewed amplifies the magnitude of any assumption error, imo.

[pizzafeelsright]

Someone recently had AI create a trading bot and it returns 131% on every transaction over a 30 day period - do you really think they care about code quality or ability to verify the math?

[simmerup]

ANd I have a bridge to sell you in London

[derrida]

Business as usual.

[mkoubaa]

It's not terrifying at all, some shops will fail and some will succeed and in the aggregate it'll be no different for the rest of us