top | item 46962518

(no title)

pmig | 20 days ago

Sure, multiple of our customers that distribute applications with a machine learning/AI component also need to distribute their models. They can use our OCI registry to distribute large images with huge layers. We specifically reworked our registry implementation to storing in-transit blobs on disk to save memory, ensuring the application doesn’t run out of memory [1].

[1] https://github.com/distr-sh/distr/pull/1478

discuss

order

a_t48|20 days ago

Is registry OOM protection the only advantage your registry has for large layers? Robotics has a need for Docker tooling that handles large layers/images gracefully. Even if you've done the "right" thing and sideloaded your ML models with some other management system, CUDA layers and such are gigantic.

Edit: looking at this, this is very adjacent to some problems w/ robotics deployments. Fleet management, edge deployment, key management. Neat.

I'd be curious about the multi-artifact support. Can I declare a manifest that binds together multiple services (or a service and an ML model?) Do you support ML models as an artifact?