Show HN: cunoFS – mount S3/AZ/GCP storage *without* FUSE, 60x faster than s3fs

[cuno]

Hello HN! We’re Dan and Vaughan, based in Cambridge, UK. We recently launched cunoFS, a filesystem for fast, POSIX object storage access. Yes, we’ve heard that “Object Storage is just not suitable for POSIX workloads” or that “workloads should be rewritten to natively support S3”. But what if it could be? And often much faster than S3-native apps? We’ve worked hard to make this possible, without the shortcomings of other approaches.

Summary here:

- Run (unmodified) applications with S3/AZ/GCP or “S3 compatible” storage, for example:

  ls s3:// az:// gs://
  ffmpeg -i s3://bkt/vid.mov -c copy s3://bkt/vid.mp4
  rsync --inplace -Wav $(pwd) az://myaz/bucket

  auto fd = open("az://myaz/bucket/file.mp4", O_RDWR);
  read(fd, buf, 1024);

- POSIX-compliant: supports symlinks, hardlinks (emulated), UIDs & GIDs, permissions, random writes, etc.

- No additional server or gateway needed. Each object is a file, each file is an object directly accessible on native S3/AZ/GCP. No scrambling of data into a funky backend format.

- Bypass FUSE and maximise performance with our ultra-fast syscall interception engine (Dynasaur [2]), or optionally run as a FUSE filesystem

- Super fast - for many benchmarks it beats EFS, FSx Lustre or even attached EBS

  - Benchmarked at 50+ Gbps per EC2 node ⇔ AWS S3, scaling to 10+ Tbps [1]

  - 60x faster than s3fs in small file benchmark (copying Linux kernel source files)

  - PyTorch dataset reads boosted up to 13x (with accelerator package installed)

- Supports virtual mount points, FUSE mount points, and direct URI access.

Pricing: cunoFS is free for personal use. You need to register for a free license after 14 days. For commercial use, we offer a 30-day free eval license.

Check out our docs [3], or download now from our website [4].

P.S. Calling for partnerships/collaborations: We’re thinking of opening up Dynasaur, our ultra-fast syscall interception engine [2]. It covers all binaries (dynamic, static and semi-static) and runs in unprivileged containers/serverless. It can virtualise Linux kernel functionality and dynamically augment it with something else, without needing special privileges. Applications include user mode networking, live process migration, memory offloading, virtualisation, analytics, user-mode kernel experimentation, and of course running other FUSE file systems faster, without the FUSE bit. Please let us know if you are interested.

Looking forward to your feedback!

[1] https://cuno.io/technology/

[2] https://cuno.io/technology-details/#The_problem_with_SHIMs

[3] https://cuno-cunofs.readthedocs-hosted.com/en/stable/

[4] https://cuno.io/download/

[enagrimm]

Sure FUSE can be a bottleneck, but I find it very hard to believe s3fs is "60x slower" because of it.

[cuno]

We've spent a lot of time identifying bottlenecks and fixing them, up and down the stack, with FUSE being just one of them, even the AWS SDK itself introduces its own set that we've addressed. cunoFS can also be used with FUSE, and we find that it is roughly half the speed of non-FUSE (but thanks to our other optimisations, this is still much faster than alternatives).

[jasonvorhe]

> - 60x faster than s3fs in small file benchmark (copying Linux kernel source files)

I mean, they even mention the benchmark they base this statement on. Sure, they could be lying but why should they?