[accra4rx]

Bigger question : Is US Government ready to do a comprehensive safety evaluation ? I think it it a cheap way for OpenAI and Anthropic to get a vetting that their models are safe to use and be adaptable by various Govt entity and other organization

[datahack]

https://www.nist.gov/aisi

I think the progress has been pretty good. You should read up on their efforts.

This is kind of a pilot to develop further testing frameworks.

[ceejayoz]

Like "military grade", where civilians go "oh that must be good" and military folks go "oh dear God no".

[smsm42]

That was my first question - what is "safety" and what is their methodology for evaluating it? Who evaluated that methodology and why it is the right one? Is there a meaningful safety benefit to this evaluation, or just a CYA exercise?

[agucova]

I recommend checking out the UK AISI's work on this: - https://www.gov.uk/government/publications/ai-safety-institu... - https://www.aisi.gov.uk/work/advanced-ai-evaluations-may-upd...