top | item 45880222 Token Embeddings Violate the Manifold Hypothesis 3 points| lukeplato | 3 months ago |arxiv.org 1 comment order hn newest verdverm|3 months ago If this interests you, https://bsky.app/profile/jefferyharrell.bsky.social has been making some super cool visualizations of Qwen3's internal representationsOne thing discovered was that 800+ tokens converged to a single point in the 2500+ dimensional unembedding spaceSome examples:https://bsky.app/profile/jefferyharrell.bsky.social/post/3m4...https://bsky.app/profile/jefferyharrell.bsky.social/post/3m5...
verdverm|3 months ago If this interests you, https://bsky.app/profile/jefferyharrell.bsky.social has been making some super cool visualizations of Qwen3's internal representationsOne thing discovered was that 800+ tokens converged to a single point in the 2500+ dimensional unembedding spaceSome examples:https://bsky.app/profile/jefferyharrell.bsky.social/post/3m4...https://bsky.app/profile/jefferyharrell.bsky.social/post/3m5...
verdverm|3 months ago
One thing discovered was that 800+ tokens converged to a single point in the 2500+ dimensional unembedding space
Some examples:
https://bsky.app/profile/jefferyharrell.bsky.social/post/3m4...
https://bsky.app/profile/jefferyharrell.bsky.social/post/3m5...