Show HN: GitHub – I tried to build AWS S3 from scratch

[johnsmith2076]

Over the past few years I’ve been thinking about how I could build SaaS and deploy it on my own infrastructure without needing to use any cloud platforms like AWS or GCP. In this repo I document my progress on building a clone of AWS S3 that functions the same as S3 using an exclusively open-source technology stack.

[usui]

Wow, I already knew about testing-stack stuff like localstack (different purpose and unrelated to this project), but the fact this is compatible and connectible with standard AWS client libraries is cool. Imagine self-hosting your own S3! That's already a software-app-inclined homelabber's dream.

[johnsmith2076]

I'm glad you like it, compatibility with the official client liberys was a key objective for this project otherwise its just a NAS with a web interface. I've been told there are some homelab subreddits that would really like this project but I don't use that platform, feel free to post there if you think it would be of value, I will be working on this for the next few weeks

[sitzkrieg]

you might be on to something with this self hosting cloud services. this could be huge!

[Helithumper]

I think this is pretty neat. Even if you don’t use it for a SaaS company I think it’s great to try your hand at making a known product.

This may also be something fun for late-college-students to do in order to brush up on their skills and to dive deep into tech that they may use in the future.

[johnsmith2076]

Thanks, yes it's definitely not meant to be a product, I just made it because I had lots of questions about how S3 worked under the hood. I'm still finishing the documentation but a college student could just follow the commands line by line and they should end up with the same system, it wasn't easy to get running

[kennu]

One of the hardest parts to achieve in building your own S3 clone might be 99.999999999% data durability and other SLA features.

[smitty1e]

A look at the S3 documentation shows that there is a fairly ludicrous amount of functionality atop the basic Hadoopiness of it all. The .pdf User Guide weighs in at a svelte 3,374 pages.

https://docs.aws.amazon.com/AmazonS3/latest/userguide/Welcom...

[hlabrinssi]

I would love to take a look at this; an S3 alternative and open source is a magic combination. Even if it's going to be a one-time product like once.com, I wouldn't mind that at all.

[johnsmith2076]

I actually build systems like this for clients for a one-time fee (see UpWork.com link on my GH profile), but all of the set-up instructions are in the repo free of charge. Just be aware even with my notes building one of these things isn't easy