WingNews logo WingNews GitHub [2]

jxodwyer1's comments

jxodwyer1 | 2 years ago | on: Launch HN: Metal (YC W23) – Embeddings as a Service

Thank you so much for your questions!

- As a managed service there are some overheads. We need to auth, validate and parse the inputs, fetch the index that is getting queried as we then need to use the index’s model to generate the embeddings. Then if the index is fine tuned/customized, we need to transform the embedding to the new vector space, to then call our vector index. We then fetch the metadata of the results from the db and parse the response to send it back.

- We’ve only tested upwards of 1M vectors ~ 1500 dimensions. But, more formal testing is required here and we plan to do so. I’m particularly curious about pg_vector and how it stacks up with other players as keeping the data central is a significant upside. We started with these lower vector indices to get something out there and iterate as a startup. But, scalability is part of what we want long term.

- We see both; we’ve had to turn down a very early lead with 100M+ vectors because it would derail other engineering efforts while we were starting. We’re now much better positioned to tackle that challenge as we have all the foundations.

- We haven’t considered this, but it’s an excellent idea. We’re currently discussing this with the team.

We would love to chat more; we appreciate your questions and feedback. Always happy to riff with someone who has seen issues around these use cases, like yourself. Feel free to reach us at [email protected] !

jxodwyer1 | 2 years ago | on: Launch HN: Metal (YC W23) – Embeddings as a Service

We have some open-source tooling in the works! :) We understand that some users are sensitive to managed services, we’re starting with this, but we’re planning to open source tools to improve developer experience around information retrieval and memory.

jxodwyer1 | 2 years ago | on: Launch HN: Metal (YC W23) – Embeddings as a Service

You’re right! If you want to do that in a notebook, it’s pretty straightforward. But if you want to have it running in production, it’s a bit more complicated. Also, providing users with a gui to run these operations without a notebook has resonated with many less ml savvy users. Dilbert's boss probably didn't know much about ml... :)

jxodwyer1 | 2 years ago | on: Launch HN: Metal (YC W23) – Embeddings as a Service

Chroma is awesome <3 - We have some overlap with them as we store the embeddings. But, we provide additional operations on top of the data, such as clustering/fine-tuning. We're also looking into open-sourcing some tools in the near future!

jxodwyer1 | 2 years ago | on: Launch HN: Metal (YC W23) – Embeddings as a Service

It does compare with them, but we want to lower the barrier of entry for any developer to build features that use embeddings. So we want to give regular software engineers superpowers in providing this technology within their stack and out of the box offering the infrastructure and high-level APIs to run operations on top of the vector db.

jxodwyer1 | 2 years ago | on: Launch HN: Metal (YC W23) – Embeddings as a Service

We store the vectors, but we also provide additional operations that would require additional code/infra if you just use a vectorDB. We also have the infrastructure in place to ingest all the data, generate the embeddings (we also take raw embeddings), and provide APIs for fine-tuning and clustering. Another big difference coming soon is index versioning, allowing developers to test multiple models/embeddings.

jxodwyer1 | 2 years ago | on: Launch HN: Metal (YC W23) – Embeddings as a Service

Thank you! We’ve looked into instructor-xl, and it’s really awesome! We also accept custom embeddings, allowing developers to use whatever model they want. But we want to keep adding models to allow for better experimentation.

jxodwyer1 | 2 years ago | on: Launch HN: Metal (YC W23) – Embeddings as a Service

There’s some overlap with information retrieval for chat GPT applications. As a managed service, we handle all of the infrastructure and maintenance. Also, we support additional use cases for web applications/backends, such as clustering and fine-tuning. We’re also working on an open-source alternative to the retrieval plugin.

jxodwyer1 | 2 years ago | on: Launch HN: Metal (YC W23) – Embeddings as a Service

This is great, and that makes a ton of sense! Would you want to define + experiment with these various configurations yourself explicitly, or would you expect a system to determine this automatically? I like the concept of rolling-up chunk scores!

jxodwyer1 | 2 years ago | on: Launch HN: Metal (YC W23) – Embeddings as a Service

We don’t support this use case yet, but we could by exposing an API to update the non-filterable metadata of the records. This is a cool use case; we would love to learn more about it. Would you want to create embeddings from the product name + description and then have the other attributes returned from the search results? We are very close to supporting this; just a matter of exposing a way to update those attributes

jxodwyer1 | 2 years ago | on: Launch HN: Metal (YC W23) – Embeddings as a Service

We don’t offer this through the API, yet! You can however run clustering in the UI. We are working on exposing classification so that you can generate clusters on specific topics. We plan to offer both in the API within the next week or two!
page 1
more