Fly.io deleted all my apps without notifying me

[lukas_b]

I noticed my app was down and went to check it out. Lo and Behold my account had been flagged by their automatic fraud detection algorithm. This is despite my account having close to $500 of credit and being active for around a year now. I was not informed about this at all. I followed the prompt to allow them to charge my credit card in order to verify that I'm not a bot and now I have control of my account again. However none of my apps were restored. Has anyone else experienced something like this in the past?

[mrkurt]

This account was not banned in error. Lukas, if you’d like me to explain publicly, I'm happy to do so.

[tptacek]

Further: we do not have an automated system that deletes flagged apps.

[lukas_b]

Maybe you could start by replying to my email? I would not have complained publicly if there had been any form of communication from you but there was none.

[nomilk]

> Has anyone else experienced something like this in the past

My account was marked as high risk and was prevented from deploying for a few hours: https://stackoverflow.com/a/76400473/5783745

But it didn't wipe out my apps. Took a couple hours to figure out what was going on and how to fix it (purchase some credit and run an esoteric and possibly unrelated command in the stack overflow answer). That (and a dozen other issues) was why I left fly. For now it's just not production-ready.

I do hope they build a successful product (as defined by happy customers, not by fund raising). The more players there are in this space, the more options we have as developers, which is good even if the path there is bumpy.

[lukas_b]

Thanks for this tip. I tried it and I keep getting the error: "Error: Your account got flagged by our internal fraud protection service" even after verifying my credit card.

[yellowapple]

Well that just evaporated the interest I had in using Fly.io. Thanks for the heads-up.

[oliwarner]

That's what makes me so deeply suspicious of posts like this. It costs nothing but a few minutes to spring up a fake account and dump on the competition.

Who knows if Lukas is legit, but they don't seem to have asked the service provider where their data is yet. Or what happened. You're not going to get those answers here.

Going straight to social media before you've found anything out stinks.

[bananapub]

you've made such a big change based on one post from a basically brand new account on HN? why?

[frafart]

I haven’t had my apps deleted but the fraud detection system flags me almost every time I interact with the platform. It’s very disruptive.

[strzibny]

This is where having Kamal config at hand is great.

Even if I would lose my VM and access, I can be up and running again pretty quickly on a different host.

I just change the IP in my config/deploy.yml

[tasuki]

You mention Kamal as if it was some kind of a ubiquitous standard. It is the first time I hear about it. In what way is Kamal better than the hundreds of alternatives?

[alexedw]

Would love to set up everything on Kamal but just feels like slightly too much management cost in maintaining a database yourself for mission critical stuff. If I'm going to use a managed DB, might as well do the same for the container?

Maybe if I just ran with sqlite and a preconfigured litestream inside the main container that'd simplify things a fair bit, but then you're immediately locked onto a single machine.

[ec109685]

And I guess offsite backups too?

[Arjuna144]

Thanks for telling. It may be a bug in their system, but the way this seems to have been handled by them is really ugly

[r0ks0n]

that bug could even be... a fly...

[altairprime]

What were your apps?

[lukas_b]

a personal vpn and a company website

[add-sub-mul-div]

Dropbox deleted all my data because I didn't log in for a long time. Guess you're supposed to check in on your completely static data occasionally. Whatever, the lesson is not to let yourself seriously rely on any of these services.

[cpach]

No matter what service provider one uses, one should always have backups and a plan B/disaster recovery. The 3-2-1 rule is as relevant as ever.

[yieldcrv]

well guess I’m keeping my current solution

I use the free tier of vercel or netlify and host all the static assets on IPFS, I keep everything pinned using filecoin for free. lol yup I use filecoin to pin IPFS CIDs

and because IPFS gateway CDNs get super unreliable, I created a js script that does a series of promises to every ipfs gateway and after one actually loads it cancels all the other promises, so that ensures availability to users and can handle spikes in traffic, without messing with the free tier

[deely3]

Why not mention the name of the company? Then post will not become flagged, I suppose.

[lukas_b]

the name is fly.io. it's in the title

[sergiotapia]

this sounds terrifying. so their "algorithm" can just nuke your entire business without any signaling?

[lukas_b]

apparently I'm not the only one.

https://community.fly.io/t/account-flagged-apps-gone/14570

https://web.archive.org/web/20240710225525/https://community...

[rizalp]

thanks for informing this, will avoid fly.io at all cost

[zacksiri]

The thing about fly.io / heroku / vercel is you are gaining convenience by giving up control over the underlying infrastructure.

When you give up control of the infrastructure you also give up the ability to decide what's right and wrong.

If you want to work with something that allows you retain that control but still have the convenience of a nice deployment workflow check out my profile.

[ec109685]

Unless you build your own data center and run your own fiber, you’re at the mercy of someone else’s policies and could be shutdown at their whim.

[bmitc]

It's my understanding that Fly.io heavily uses Elixir and has several Elixir bigshots on staff. But I only see mention of Fly.io for people are moving off of it or having major issues. I'm wondering of that's really the case and how given the tech stack.

[g15jv2dp]

How is the programming language they use at all relevant?

[taspeotis]

I forgot functional programming in Elixir meant no bugs! It's built on Erlang which halves bugs tenfold too!

So 0 bugs because Elixir / 10 = 0

QED

[strzibny]

The issue is political and has nothing to do with Elixir (which is pretty great btw). Please read the post again.