top | item 43744568

(no title)

parched99 | 10 months ago

I think Powershell is a bad test. I've noticed all local models have trouble providing accurate responses to Powershell-related prompts. Strangely, even Microsoft's model, Phi 4, is bad at answering these questions without careful prompting. Though, MS can't even provide accurate PS docs.

My best guess is that there's not enough discussion/development related to Powershell in training data.

discuss

order

fragmede|10 months ago

Which, like, you'd think Microsoft has an entire team there who's purpose would be to generate good PowerShell for it to train on.