top | item 47195103

(no title)

Lerc | 1 day ago

>AI in it’s current state is ruthless in achieving its goal

I don't believe this to be a trait of any AI model, the model just does the right thing or the wrong thing.

The ruthless maximising of a particular trait is something that happens during training.

It does not follow that a model that is trained to reason will nedsesarily implement this ruthless seeking behaviour itself.

discuss

order

pixl97|1 day ago

No lineage of AI models will be created that cannot achieve goals, they will be outcompeted by models that can.

Lerc|1 day ago

Perhaps, but there is a difference in a reasoning system deciding on the best way to achieve the goal.

To get the predicted disastrous effects you need to be doing function optimisation without regard to the meaning of the function parameters. Yes, models can still game the system at inference time, but in much the same way as a human might game the system, it requires awareness that you are going against the intent of some rule.