[daturkel]

This looks really cool. One thing I've wondered about with, e.g., the OpenAI API is if json is really a good format for passing embeddings back and forth. I'd think that passing floats as text over the wire wastes a ton of space that could add up, and might even sacrifice some precision in. Would it be better to encode at least the vectors as binary blobs, or else use something like protobuf to more efficiently handle sending tons of floats around?

[zh217]

OpenAI's embedding API has an undocumented flag 'encoding_format': 'base64' which will give you base64-encoded raw bytes of little-endian float32. As it is used by the official python client, it is unlikely to go away.

[eigenvalue]

I totally agree when you're talking about a bunch of embeddings at once-- that's why the document level endpoint (and the token-level embedding endpoint) can optionally return a link to a zip file containing the JSON. For a single embedding, not sure it matters that much, and the extra convenience is nice.

Edit: One other thing is that you can store the JSON in SQLite using the JSON data type and then use the nice querying constructs directly at the database level, which is nice for the token-level embeddings and document embeddings. This is built in to my project.