My fear of this sort of thing happening is why I don't use github or gitlab.com for primary hosting of my source code; only mirrors. I do primary source control in house, and keep backups on top of that.
It's also why nothing in my AWS account is "canonical storage". If I need, say, a database in AWS, it is live-mirrored to somewhere within my control, on hardware I own, even if that thing never sees any production traffic beyond the mirror itself. Plus backups.
That way, if this ever happens, I can recover fairly easily. The backups protect me from my own mistakes, and the local canonical copies and backups protect me from theirs.
Granted, it gets harder and more expensive with increasing scale, but it's a necessary expense if you care at all about business continuity issues. On a personal level, it's much cheaper though, especially these days.
Exactly. Techofeudal overlords can switch off all "your" stuff at any time. Always have a personal and a business disaster recovery plan including isolated backups (not synchronized replication) on N >= 2 separate services/modalities.
Options to consider for various circumstances include:
- Different object storage clouds with different accounts (different names, emails, and payment methods), potentially geographically different too
- Tarsnap (while using AWS under the hood but someone else's account(s))
- MEGA
- Onsite warm and/or cold media
- Geographically separate colo DR site, despite the overly-proud trend of "we're 100% (on someone else's SPoF) cloud now"
- Offsite cold media (personal home and/or IronMountain)
How do you distinguish a mirror from not a mirror on GitHub?
I often have my git configured to push to multiple upstreams, this means that basically all of your mirrors can be primaries.
This is a really good part about GitHub. Every copy is effectively a mirror, too, and it's cryptographically verified as well, so, you don't have to worry about the mirror going rogue without anyone noticing.
IMHO Lawyers get creative, a github account can show a ton of work activity, nda voilations, etc. Your "private repro" is just a phone call away from being a public repro.
> Granted, it gets harder and more expensive with increasing scale, but it's a necessary expense if you care at all about business continuity issues. On a personal level, it's much cheaper though, especially these days.
I don't go as far as "live mirror", but I've been advocating _for years_ on here and in meatspace that this is the most important thing you can be doing.
You can rebuild your infrastructure. You cannot rebuild your user's data.
An extended outage is bad but in many cases not existential. In many cases customers will stick around. (I work with one client that was down over a month without a single cancellation because their line-of-business application was that valuable to their customers.)
Once you've lost your users' data, they have little incentive to stick around. There's no longer any stickiness as far as "I would have to migrate my data out" and... you've completely lost their trust as far as leaving any new data in your hands. You've completely destroyed all the effort they've invested in your product, and they're going to be hesitant to invest it again. (And that's assuming you're not dealing with something like people's money where losing track of who owns what may result in some existence-threatening lawsuits all on its own.)
The barrier to keeping a copy of your data "off site" is often fairly small. What would it take you right now to set up a scheduled job to dump a database and sync it into B2 or something?
Even if that's too logistically difficult (convincing auditors about the encryption used or anything else), what would it take to set up a separate AWS account under a different legal entity with a different payment method that just synced your snapshots and backups to it?
Unless you're working on software where people will die when it's offline, you should prioritize durability over availability. Backups of backups is more important than your N-teir Web-3 enterprise scalable architecture that allows deployment over 18*π AZs with zero-downtime failover.
The amount of self-aggrandizing and lack of self awareness tells me this author is doing to do all of this again. This post could be summed up with "I should have had backups. Lesson learned", but instead they deflect to whining about how their local desktop is a mess and they NEED to store everything remotely to stay organized.
They're going to dazzle you with all of their hardened bunker this, and multiple escape route that, not realizing all of their complex machinery is metaphorically running off of a machine with no battery backup. One power outage and POOF!
The author doesn't grasp what putting all your eggs into one basket means:
> Before anyone says “you put all your eggs in one basket,” let me be clear: I didn’t. I put them in one provider, with what should have been bulletproof redundancy:
That's one basket. A single point of failure. "But it should have been impossible to fail!" Backups are to handle the "impossible" failure (in reality nothing is 100% reliable).
> I’d done everything right. Vault encryption keys stored separately from my main infrastructure. Defense in depth. Zero trust architecture. The works.
Did you? Is putting all your eggs in one basket "defense in depth"? Is total trust in AWS "zero trust architecture"?
I'm not defending AWS here; they fully deserve all the fallout they can get from this, and I do feel for the dev who lost all their stuff through AWS's fuckup. Lots of people do the same.
My current employer does the same. It's a major bank, and all of their stuff is Microsoft. Azure, SharePoint, Office, Teams, the works. I think it's foolish to trust a single foreign company with all your vital data and infrastructure, operating in q country where the government demands access to everything, but this is what everybody does now.
We trust "the cloud" way too much, and expose ourselves to these sort of fuckups.
I'll concede that I'm hugely empathetic for people that suffer data loss. The pithy aphorism about there being two types of people -- those who haven't lost data, and those who do backups -- is doubly droll because only the second group really appreciates the phrase.
But it's surprising to find people with more than a decade in IT who don't appreciate the risks here.
The timeline reveals there were 13 days from when the first signs of trouble surfaced, to when the account was deleted. So a fortnight of very unsubtle reminders to do something AND a fortnight in which to act.
(I recently learned the phrase BATNA[0] and in modern <sic> IT where it's Turtles as a Service, all the way down, it's amazing how often this concept is applicable.)
Author seems very keen to blame his part-time sysadmin rather than his systems architect. I can understand the appeal of that blame distribution algorithm, but it's nonetheless misguided.
The phrasing:
> But here’s the dilemma they’ve created: What if you have petabytes of data? How do you backup a backup?
inverts the horse & cart. If you have a petabyte of data that's important, that you can't recreate from other sources, your concern is how to keep your data safe.
If you're paying someone to keep a copy, pay (at least one other) person to keep another copy. Even that isn't something I'd call safe though.
> Me: “You’re answering like I’m Piers Morgan asking ‘Do you condemn October 7th?’ and you reply with historical complexity dating to 1948.”
Yeah...
If I'm working tickets at AWS that kind of dickishness is going to ensure that I don't do more than the least amount of effort for you.
Maybe I could burn my entire weekend trying to see if I can rescue your data... or maybe I'm going to do nothing more than strictly follow procedure and let my boss know that I tried...
I did have backups. Multi-region. Redundant. I followed AWS’s own best practices to the letter.
The only failure I didn’t plan for? AWS becoming the failure.
The provider nuking everything in violation of their own retention policies. That’s not a backup problem, that is a provider trust problem.
The reason i did not kept a local copy, was that i formatted my computer after a hardware failure, after the nurse dropped the laptop in the hospital i was on. Since i have a AWS backup, i just started with a fresh OS while waiting to get discharged to return home and redownload everything.
When i returned 6 days days later, the backup was gone.
They’ve really buried the lede here: this reads like the person paying for the account was not the post author, and AWS asked the payer (who from their perspective is the owner of the account) for information.
The lede buried under that lede is that (according to an insider?) some AWS employee accidentally wiped everything immediately (contrary to typical practice in such situations of retaining data while things get sorted out), leading to a chain of brushing-off / covering-up percolating through whatever support chain the OP was talking to.
The payer is the owner of the account? I doubt that is AWS' default stance, because their contract is with the account holder, not the payer.
Me paying your bill doesn't give me ownership of your stuff - as far as AWS is concerned, your bill is paid, and that's the extent of their involvement, everything else is between you and me.
If what he writes is true, he remained the account holder and even had a backup billing method in place - something he probably wouldn't have if he wasn't the account holder.
I don't know if he's completely honest about the story, but "somebody else paid, so we decided they are now the owner" isn't how that works.
The person paying the account is not the author, i'm.
What happen is that the person paying for the account had to settle an invoice of many thousand of dollars.
They offered me AWS gift cards,to send me electronics and they will pay for it in parts.
They lost lot of money because of crypto collapse. So i accepted their solution to pay for my OSS usage for few months.
That like if i was going to pay for your rent for 1 year. You don't pay, while i don't have to pay 3-4 years of your rent at once.
What happen, is that AWS dropped a nuclear bomb in your house, in the middle of the month .. then tell you later that it was about payment.
If they told me in the first email it was about the payer, i will have unlinked and backuped.
While that is certainly true, the idea that they can so rapidly decimate your data without the possibility to restore is still terrifying if that's your only copy of that data.
They should have to hold it for at least 90 days. In my opinion, it should be more like six months to a year.
In my mind, it's exactly equivalent to a storage space destroying your car five days after you miss a payment. They effectively stole and destroyed the data when they should have been required to return it to the actual owner.
Of course, that's my opinion of how it should be. AFAIK, there is no real legal framework, and how it actually is is entirely up to your provider, which is one reason I never trust them.
Cloud user here. If you would read your contracts, and it doesn't matter which cloud service you use, they all have the same section on Share Responsibility.
AWS is not responsible for what you do with your data. AWS, on its own, is not going to make backups for you so you can recover from a failed DB migration you did yourself. AWS cannot be held responsible for your admin fat-fingering and deleting the entire prod environment.
However, AWS is responsible for the underlying infrastructure. The website you linked clearly shows "storage" as falling under AWS's responsibility: your virtual hard drives aren't supposed to just magically disappear!
If they can just nuke your entire setup with an "Oops, sorry!", what's all the talk about redundancy and reliability supposed to be worth? At that point, how are they any different from Joe's Discount Basement Hosting?
Put your valuables into a safe deposit box. Or, buy some stocks.
Some accident occurs. You don't pay your bill, address changes, etc. You have at least two entire years to contact the holder and claim your property. After that point, it is passed to the state as unclaimed property. You still have an opportunity to claim it.
Digital data? Screw that! One mistake, everything deleted.
Physical storage providers make the same kind of mistakes all the time, accidentally emptying the wrong storage unit or trashing the wrong safety deposit box.
You have potentially stronger civil remedies for recouping on those damages, but not always.
>For years, I’ve watched developers on Reddit and Facebook desperately seeking US or EU billing addresses, willing to pay $100+ premiums to avoid MENA region assignment.
>When I asked why, a colleague warned me: “AWS MENA operates differently. They can terminate you randomly.”
This has happened to me too destroyed five years of my life no joke. Obviously it wasn’t just the set up and pipelines that took only 4 to 6 months but as a chain reaction to collapsed the entire startup. It was so unexpected. Lesson learned.
The java command line theory seems strange to me. Java has no standard command line parser in the JDK (Standard library). Apache commons cli probably comes closest to a standard, and they support --gnu-style-long-options just like everybody else. The jvm(runtime) has some long non-POSIX options, but that's not very relevant here.
Yeah, it's fishy. I never claimed the Java theory is confirmed, just that it’s what an insider told me after the fact.
They said a dry-run flag was passed in the --gnu-style form, but the internal tool expected -dry, and since Java has no native CLI parser, it just ignored it and ran for real.
Supposedly the dev team was used to Python-style CLIs, and this got through without proper testing.
If you are given only 5 days to comply with some request, that's how complicated your infra at AWS should be - so you can migrate to another provider in that time.
Just use EC2 and basic primitives which are easy to migrate (ie S3, SES)
Hi,
The infrastructure was not complex at at all, i can transfer it in 1 day.
I was hospitalized, in another city, with all the computer at home, and locked behind 2FA.
They send me is on notice on Thursday, by Monday evening, all access was revoked.
For weeks i asked for a readonly access to my data, then they could take anytime they want to verify, they refused.
And he more i ask about my data, they more they avoid to speak about it.
Think about it , you could be sick, on a trip, having jetlag, in some festival, getting married... by the time you are back online, the delay was gone.
Editorial comment: It’s a bit weird to see AI-written (at least partially; you can see the usual em-dashes, it’s-not-X-it’s-Y) blog posts like this detract from an author’s true writing style, which in this case I found significantly more pleasant to read. Read his first ever post, and compare it to this one and many of the other recent posts: https://www.seuros.com/blog/noflylist-how-noflylist-got-clea...
I’m not much of a conspiracy theorist, but I could imagine a blog post almost identical to this one being generated in response to a prompt like “write a first-person narrative about: a cloud provider abruptly deleting a decade-old account and all associated data without warning. Include a plot twist”.
I literally cannot tell if this story is something that really happened or not. It scares me a little, because if this was a real problem and I was in the author’s shoes, I would want people to believe me.
Not AI-generated. Not everyone is born writing flawless English.
If it sounds like an LLM, maybe it is because people like me had to learn how to write clearly from LLMs because English is not our first language.
I could’ve written in my native tongue, but then someone else will have complained that not how english is structured.
Also, the story is real. Just because it is well-structured doesn't mean it's fiction. Yes, i used AI to resort it, but i can assure you that no AI will generate the Piers Morgan reference.
Isn't that part of AI, simply because that's how the patterns work, and how we're taught to write in the writing classes?
BTW, I actually use the em-dash symbols very frequently myself — on a Mac and on Android, it's very easy through the standard keyboard, with Option-Dash being the shortcut on the Mac.
Not sure why you're downvoted - this was exactly my thought reading it. I spend a significant portion of my time reading human-generated and AI-generated long-form writing and I can very easily see the AI stuff.
But maybe it doesn't matter any more? Most people can't.
This is good but not really enough. You need another backup to cover the case where this backup is burned to a crisp when your house catches fire. And that second backup needs to be in another geographic region to guard against regional disasters such as meteor impact, super volcano eruption (possibly not a concern for you but it is for me), etc.
So you restored from your off cloud backup, right?
Tell me you have off cloud backups? If not, then I know its brutal, but AWS is responsible for their part in the disaster, and you are responsible for yours - which is not being able to recover at all.
This is really the longest, most self-aggrandizing sermon yet on "I stored my data on this other computer I don't own", complete with conspiracy theory and all.
Store your data on your own disks, then at least you will blame yourself, not .. Java command line parsers?
My only question is: where the hell was your support rep? Every org that works with AWS in any enterprise capacity has an enterprise agreement and an account rep. They should have been the one to guide you through this.
If you were just yolo’ing it on your own identification without a contract, well, that’s that. You should have converted over to an enterprise agreement so they couldn’t fuck you over. And they will fuck you over.
floating-io|7 months ago
It's also why nothing in my AWS account is "canonical storage". If I need, say, a database in AWS, it is live-mirrored to somewhere within my control, on hardware I own, even if that thing never sees any production traffic beyond the mirror itself. Plus backups.
That way, if this ever happens, I can recover fairly easily. The backups protect me from my own mistakes, and the local canonical copies and backups protect me from theirs.
Granted, it gets harder and more expensive with increasing scale, but it's a necessary expense if you care at all about business continuity issues. On a personal level, it's much cheaper though, especially these days.
wewewedxfgdf|7 months ago
He said, "no, it's on github".
I said no more.
burnt-resistor|7 months ago
Options to consider for various circumstances include:
- Different object storage clouds with different accounts (different names, emails, and payment methods), potentially geographically different too
- Tarsnap (while using AWS under the hood but someone else's account(s))
- MEGA
- Onsite warm and/or cold media
- Geographically separate colo DR site, despite the overly-proud trend of "we're 100% (on someone else's SPoF) cloud now"
- Offsite cold media (personal home and/or IronMountain)
cnst|7 months ago
I often have my git configured to push to multiple upstreams, this means that basically all of your mirrors can be primaries.
This is a really good part about GitHub. Every copy is effectively a mirror, too, and it's cryptographically verified as well, so, you don't have to worry about the mirror going rogue without anyone noticing.
ransom1538|7 months ago
WhyNotHugo|6 months ago
I wish there were some service which would _pull_ my public git repositories, but not allow me to delete anything without a ~90day waiting period.
nucleardog|7 months ago
I don't go as far as "live mirror", but I've been advocating _for years_ on here and in meatspace that this is the most important thing you can be doing.
You can rebuild your infrastructure. You cannot rebuild your user's data.
An extended outage is bad but in many cases not existential. In many cases customers will stick around. (I work with one client that was down over a month without a single cancellation because their line-of-business application was that valuable to their customers.)
Once you've lost your users' data, they have little incentive to stick around. There's no longer any stickiness as far as "I would have to migrate my data out" and... you've completely lost their trust as far as leaving any new data in your hands. You've completely destroyed all the effort they've invested in your product, and they're going to be hesitant to invest it again. (And that's assuming you're not dealing with something like people's money where losing track of who owns what may result in some existence-threatening lawsuits all on its own.)
The barrier to keeping a copy of your data "off site" is often fairly small. What would it take you right now to set up a scheduled job to dump a database and sync it into B2 or something?
Even if that's too logistically difficult (convincing auditors about the encryption used or anything else), what would it take to set up a separate AWS account under a different legal entity with a different payment method that just synced your snapshots and backups to it?
Unless you're working on software where people will die when it's offline, you should prioritize durability over availability. Backups of backups is more important than your N-teir Web-3 enterprise scalable architecture that allows deployment over 18*π AZs with zero-downtime failover.
See, as a case study, Unisuper's incident on GCP: https://www.unisuper.com.au/about-us/media-centre/2024/a-joi...
spiralcoaster|7 months ago
They're going to dazzle you with all of their hardened bunker this, and multiple escape route that, not realizing all of their complex machinery is metaphorically running off of a machine with no battery backup. One power outage and POOF!
gblargg|7 months ago
> Before anyone says “you put all your eggs in one basket,” let me be clear: I didn’t. I put them in one provider, with what should have been bulletproof redundancy:
That's one basket. A single point of failure. "But it should have been impossible to fail!" Backups are to handle the "impossible" failure (in reality nothing is 100% reliable).
mcv|7 months ago
> I’d done everything right. Vault encryption keys stored separately from my main infrastructure. Defense in depth. Zero trust architecture. The works.
Did you? Is putting all your eggs in one basket "defense in depth"? Is total trust in AWS "zero trust architecture"?
I'm not defending AWS here; they fully deserve all the fallout they can get from this, and I do feel for the dev who lost all their stuff through AWS's fuckup. Lots of people do the same.
My current employer does the same. It's a major bank, and all of their stuff is Microsoft. Azure, SharePoint, Office, Teams, the works. I think it's foolish to trust a single foreign company with all your vital data and infrastructure, operating in q country where the government demands access to everything, but this is what everybody does now.
We trust "the cloud" way too much, and expose ourselves to these sort of fuckups.
Jedd|7 months ago
I'll concede that I'm hugely empathetic for people that suffer data loss. The pithy aphorism about there being two types of people -- those who haven't lost data, and those who do backups -- is doubly droll because only the second group really appreciates the phrase.
But it's surprising to find people with more than a decade in IT who don't appreciate the risks here.
The timeline reveals there were 13 days from when the first signs of trouble surfaced, to when the account was deleted. So a fortnight of very unsubtle reminders to do something AND a fortnight in which to act.
(I recently learned the phrase BATNA[0] and in modern <sic> IT where it's Turtles as a Service, all the way down, it's amazing how often this concept is applicable.)
Author seems very keen to blame his part-time sysadmin rather than his systems architect. I can understand the appeal of that blame distribution algorithm, but it's nonetheless misguided.
The phrasing:
> But here’s the dilemma they’ve created: What if you have petabytes of data? How do you backup a backup?
inverts the horse & cart. If you have a petabyte of data that's important, that you can't recreate from other sources, your concern is how to keep your data safe.
If you're paying someone to keep a copy, pay (at least one other) person to keep another copy. Even that isn't something I'd call safe though.
[0] https://en.wikipedia.org/wiki/Best_alternative_to_a_negotiat...
lamontcg|7 months ago
Yeah...
If I'm working tickets at AWS that kind of dickishness is going to ensure that I don't do more than the least amount of effort for you.
Maybe I could burn my entire weekend trying to see if I can rescue your data... or maybe I'm going to do nothing more than strictly follow procedure and let my boss know that I tried...
unknown|7 months ago
[deleted]
seuros|7 months ago
The only failure I didn’t plan for? AWS becoming the failure.
The provider nuking everything in violation of their own retention policies. That’s not a backup problem, that is a provider trust problem.
The reason i did not kept a local copy, was that i formatted my computer after a hardware failure, after the nurse dropped the laptop in the hospital i was on. Since i have a AWS backup, i just started with a fresh OS while waiting to get discharged to return home and redownload everything.
When i returned 6 days days later, the backup was gone.
akerl_|7 months ago
That person wasn’t around to respond.
blargey|7 months ago
luckylion|7 months ago
Me paying your bill doesn't give me ownership of your stuff - as far as AWS is concerned, your bill is paid, and that's the extent of their involvement, everything else is between you and me.
If what he writes is true, he remained the account holder and even had a backup billing method in place - something he probably wouldn't have if he wasn't the account holder.
I don't know if he's completely honest about the story, but "somebody else paid, so we decided they are now the owner" isn't how that works.
seuros|7 months ago
What happen is that the person paying for the account had to settle an invoice of many thousand of dollars. They offered me AWS gift cards,to send me electronics and they will pay for it in parts.
They lost lot of money because of crypto collapse. So i accepted their solution to pay for my OSS usage for few months.
That like if i was going to pay for your rent for 1 year. You don't pay, while i don't have to pay 3-4 years of your rent at once.
What happen, is that AWS dropped a nuclear bomb in your house, in the middle of the month .. then tell you later that it was about payment.
If they told me in the first email it was about the payer, i will have unlinked and backuped.
floating-io|7 months ago
They should have to hold it for at least 90 days. In my opinion, it should be more like six months to a year.
In my mind, it's exactly equivalent to a storage space destroying your car five days after you miss a payment. They effectively stole and destroyed the data when they should have been required to return it to the actual owner.
Of course, that's my opinion of how it should be. AFAIK, there is no real legal framework, and how it actually is is entirely up to your provider, which is one reason I never trust them.
unknown|7 months ago
[deleted]
yardie|7 months ago
https://aws.amazon.com/compliance/shared-responsibility-mode...
You, the customer, are responsible for your data. AWS is only responsible for the infrastructure that it resides on.
crote|7 months ago
AWS is not responsible for what you do with your data. AWS, on its own, is not going to make backups for you so you can recover from a failed DB migration you did yourself. AWS cannot be held responsible for your admin fat-fingering and deleting the entire prod environment.
However, AWS is responsible for the underlying infrastructure. The website you linked clearly shows "storage" as falling under AWS's responsibility: your virtual hard drives aren't supposed to just magically disappear!
If they can just nuke your entire setup with an "Oops, sorry!", what's all the talk about redundancy and reliability supposed to be worth? At that point, how are they any different from Joe's Discount Basement Hosting?
slashdave|7 months ago
Some accident occurs. You don't pay your bill, address changes, etc. You have at least two entire years to contact the holder and claim your property. After that point, it is passed to the state as unclaimed property. You still have an opportunity to claim it.
Digital data? Screw that! One mistake, everything deleted.
akerl_|7 months ago
You have potentially stronger civil remedies for recouping on those damages, but not always.
lowbloodsugar|7 months ago
>When I asked why, a colleague warned me: “AWS MENA operates differently. They can terminate you randomly.”
Huh.
cnst|7 months ago
(I did have to look it up.)
ProofHouse|7 months ago
roncesvalles|7 months ago
hyperman1|7 months ago
seuros|7 months ago
They said a dry-run flag was passed in the --gnu-style form, but the internal tool expected -dry, and since Java has no native CLI parser, it just ignored it and ran for real.
Supposedly the dev team was used to Python-style CLIs, and this got through without proper testing.
SergeAx|7 months ago
1) Keep my backups with a different infrastructure provider
2) Never allow third parties to pay for critical services
3) The moment I am asked for "verification," the emergency plan kicks in
huksley|7 months ago
Just use EC2 and basic primitives which are easy to migrate (ie S3, SES)
seuros|7 months ago
I was hospitalized, in another city, with all the computer at home, and locked behind 2FA.
They send me is on notice on Thursday, by Monday evening, all access was revoked.
For weeks i asked for a readonly access to my data, then they could take anytime they want to verify, they refused.
And he more i ask about my data, they more they avoid to speak about it.
Think about it , you could be sick, on a trip, having jetlag, in some festival, getting married... by the time you are back online, the delay was gone.
jasonvorhe|7 months ago
S0y|7 months ago
If that's your whole infra you really shouldn't be on AWS in the first place.
cnst|7 months ago
infinitedata|7 months ago
dmead|7 months ago
tguvot|7 months ago
You can request via help site data takeout under ccpa or gdpr. Took about two weeks but i got all my data
sitzkrieg|7 months ago
alper|6 months ago
Yeah, well. The dog ate my homework.
lightedman|7 months ago
0x696C6961|7 months ago
jbrw|7 months ago
foundry27|7 months ago
I’m not much of a conspiracy theorist, but I could imagine a blog post almost identical to this one being generated in response to a prompt like “write a first-person narrative about: a cloud provider abruptly deleting a decade-old account and all associated data without warning. Include a plot twist”.
I literally cannot tell if this story is something that really happened or not. It scares me a little, because if this was a real problem and I was in the author’s shoes, I would want people to believe me.
seuros|7 months ago
If it sounds like an LLM, maybe it is because people like me had to learn how to write clearly from LLMs because English is not our first language.
I could’ve written in my native tongue, but then someone else will have complained that not how english is structured.
Also, the story is real. Just because it is well-structured doesn't mean it's fiction. Yes, i used AI to resort it, but i can assure you that no AI will generate the Piers Morgan reference.
ageitgey|7 months ago
If anything, an AI tool would have written a shorter, less rambling post.
cnst|7 months ago
BTW, I actually use the em-dash symbols very frequently myself — on a Mac and on Android, it's very easy through the standard keyboard, with Option-Dash being the shortcut on the Mac.
ritzaco|7 months ago
But maybe it doesn't matter any more? Most people can't.
saltysalt|7 months ago
dboreham|7 months ago
Kate5477|6 months ago
[deleted]
bengray|6 months ago
[deleted]
486sx33|7 months ago
[deleted]
garrickmurphy|7 months ago
[deleted]
wewewedxfgdf|7 months ago
Tell me you have off cloud backups? If not, then I know its brutal, but AWS is responsible for their part in the disaster, and you are responsible for yours - which is not being able to recover at all.
stefan_|7 months ago
Store your data on your own disks, then at least you will blame yourself, not .. Java command line parsers?
tchbnl|7 months ago
Ah, but that's still one basket.
Bratmon|7 months ago
jsiepkes|7 months ago
bitdeep|7 months ago
averrois|7 months ago
palepa|7 months ago
[deleted]
reactordev|7 months ago
If you were just yolo’ing it on your own identification without a contract, well, that’s that. You should have converted over to an enterprise agreement so they couldn’t fuck you over. And they will fuck you over.