top | item 45720973 (no title) sojuz151 | 4 months ago This means that current tokenisers are bad, and something better is needed if text rendering + image input is a better tokeniser. discuss order hn newest mr_toad|4 months ago Humans recognise two vastly different types of language input (auditory and visual). I doubt that one type of tokeniser is inherently superior.
mr_toad|4 months ago Humans recognise two vastly different types of language input (auditory and visual). I doubt that one type of tokeniser is inherently superior.
mr_toad|4 months ago