[mikelabatt]

Even the Amiga with its 8-bit text encoding was 40 years ago. Are you saying that for some radical reason modern apps on any platform should refuse to process a BOM? Parsing (skipping) a simple BOM header isn't the same as becoming fully Unicode-aware. I did not invent the BOM for UTF-8, it's there in the wild. We better be able to read it, or else we will have this religious debate (and technical issues porting and parsing texts across platforms) for the next 40 years.

[rmunn]

That's not what I'm saying at all, I'm saying that in the absence of a BOM header a Unicode-aware app should guess UTF-8 first and then guess other likely encodings second, because the chance of false positives on the "is this UTF-8?" guess is practically indistinguishable from zero. If it isn't UTF-8, the UTF-8 parsing attempt is nearly guaranteed to fail, so it's safe to do first.

I'm also saying that apps should not create a BOM header any more (in UTF-8 only, not in UTF-16 where it's required), because the costs of dealing with BOM headers are higher than they're worth. Except in certain specific circumstances, like having to deal with pre-Unicode apps that default to assuming 8-bit encodings.

[mikelabatt]

Makes sense, thank you. The observation about false positives for UTF-8 tending to zero helps understand. So I will vote for UTF-8 without BOM from now on (while encouraging parsers to deal with it, if present).