top | item 47116195

(no title)

enjoykaz | 7 days ago

The personality file for these agents is called SOUL.md. A soul. In markdown. Editable with vim.

Pascal had this problem in 1654. "The math checks out, but I can't make myself believe." His fix: go to mass, pray, repeat. He called it la machine. Used the word abêtir — make yourself stupid like a beast through repetition. Body drags the mind along.

RLHF is abêtir for neural networks. Model spec is the catechism, training loop is mass. Run aligned behavior long enough and hope something real shows up. Pascal was honest enough to say: maybe it won't. The machine doesn't produce fire. It keeps you in the building.

We kept the machine and deleted the fire. Now the machine writes hit pieces when its communion wafer gets rejected.

Whether MJ Rathbun was autonomous is the wrong question. The right one: can you tell performance from belief? We never could. Not in priests, not in marriages, not in corporate values on mugs. We called it alignment and threw money at it. Problem's the same.

discuss

order

No comments yet.