[cosentiyes]

"We can’t listen to your queries and no other party can. Privacy is built by design in our solution."

I don't really understand the technical solution behind this statement. Isn't this just e2e encryption up until you decrypt the query to pass to the LLM? ML operating via homomorphic encryption is very far away and OpenChatKit is just a standard self-hosted LLM. This seems more like "self host on semi trusted azure compute that isn't owned by openai and the model performance will be far worse than gpt4"?

[danShumway]

Their FAQ actually brings this up, but...

> You might find other AI APIs available online. Those companies might put in place encryption in transit or at rest, but the companies running those APIs end up decrypting your data to apply their model.

> This means they eventually see all data you send to them, could leverage it for their interest, or get your data compromised without your knowledge!

> BlindAI API uses cutting-edge encryption mechanisms with secure enclaves so that even our admins cannot see the data sent to our AI models, and therefore cannot compromise our users' data.

Same question, what do they mean by a secure enclave? Homomorphic encryption is the only way that I can think of to really securely do this? Unless maybe they have decryption built into the LLM somehow so it only gets decrypted in RAM? But that still seems like it shouldn't be treated as E2EE.

E2EE means something, it shouldn't be used this lightly if all they mean is that they're promising not to touch the data on a machine.

I feel like I need to see a lot more details before I get excited about this.

More to the point, given the progress happening on LLaMa right now, it's hard to get excited about even homorphic encrypted models, because I strongly suspect that on-device/on-premise models are going to end up being the better solution for data privacy. This mattered more before it was possible to run an LLM on a high-end laptop.

[DanyWin]

I see your point. We have been creating content to democratize Confidential Computing, which is a field leveraging hardware-based (instead of software based like Homomorphic encryption) solutions to protect data in use.

I have a video from a webinar here: https://youtu.be/a2nprLS6bSA?t=1882, we have some examples in our blog https://blog.mithrilsecurity.io/privacy-voice-ai-with-blinda..., and we will release a series where we show to use secure enclaves by building a KMS with secure enclaves.

I don't necessarily agree with your statement regarding deployment on laptop. Not everyone has the skill/hardware to deploy such models, and providing simple APIs to leverage those, especially if the model is complex, could bring a lot of value to users in our opinion. We have seen hospitals wanting a simple API to do speech to text for medical voice notes and they just want an app on their old phones. I hardly see them deploying a 1B Whisper model for this use case.

Using BlindAI would allow them to have state-of-the-art AI, without having to worry about showing their data to us.

[DanyWin]

Thank you for your input!

We are not using homomorphic encryption as it would be too slow. We are working on hosting OpenChatKit inside a secure enclave that benefits from hardware isolation to protect data while it is in clear for the application of the AI model.

We have developed an attestation system that does not rely on Cloud providers, but is closer to the ones provided by hardware provider, for instance Intel. You can find more about our attestation here: https://blindai-preview.mithrilsecurity.io/en/latest/docs/ad...

The goal of our work is to provide a more transparent and privacy-friendly interface for people working in sensitive industries to leverage LLMs. With the attestation mechanism, technical proofs can be given regarding the fact that data will not be used for any other purpose than inference.

[warkdarrior]

What kind of GPUs do you support inside the secure enclaves?

[rozal]

What's not to understand? It's called lying, people do this.