buchanmilne's comments

You don't need to use a separate provider to address the problem here, where using a separate provider may not address the problem at all:

> "We're fully redundant across AZs, regions, and even cloud providers!" crows the engineer with a single corporate credit card backing the entire house of cards.

https://twitter.com/QuinnyPig/status/1288275701389389825

You could backup to a different account, possibly in a different region, with the same provider, as long as the other account has a different payment method. Using one provider would in most cases reduce the amount of effort to get back up and running.

The key piece would be how DNS is structured and hosted/billed.

> Amazon is AWS's biggest customer, but I've never been told to compress things with zstd while working at AWS.

You may have switched transparently to using it (e.g. for your application log files), via internal Amazon tooling (that many AWS services use).

However, this tweet wasn't about AWS customer's data, but AWS's own data.

> Maybe data in S3 is compressed? I don't know the internals of S3.

Almost every other AWS service uses S3 for storing something.

Think of EBS snapshots, DynamoDB backups, RDS backups.

And that's not even the service-internal data.

> At Shopify, we had some serious issues at the time (~2014) with either LMDB or the Go driver that we couldn't resolve after several months

Is there an issue/bug for this somewhere?

The article says:

> Why? All the crash test dummies are male. Even the “female” dummies the government requires in tests are just smaller versions of male dummies.

Wait, so are there or aren't there female dummies?

Secondly, does "All" refer to only the U.S., or Europe too? It seems not.

For example (see https://www.theaa.com/driving-advice/safety/euro-ncap)

> From 2015 Euro NCAP added a new restraint test, a full frontal impact against a rigid barrier at just over 30mph with small female dummies in the driver's seat and rear passenger seat.

and

> Improved child occupant protection (2016)

> Euro NCAP introduced two new crash test dummies representing a 6 and 10 year old child to better assess the effectiveness of restraint systems. Compatiblity with i-Size and a wider range of universal child restraints is also assessed.

https://www.autoexpress.co.uk/car-news/108721/women-at-great... seems to be a much better article, covering a lot of the misconceptions in the one from the topic.

Also:

> With men making up 49 per cent of the European population but 76 per cent of road deaths, using a male 50th percentile dummy allowed testers to ensure that assessments would relate to the largest proportion of accident victims.

You said:

> it's all very well to say "expect to lose an AZ" but during this outage it's not been physically possible to remove the broken AZ instances from multi-AZ services because we cannot physically get them to respond to or acknowledge commands

"Expect to lose an AZ" includes not being able to make any changes to existing instances in the affected AZ.

If you had instances across multiple AZs behind an ELB with health checks, then the ELB should automatically remove the affected instances.

If you have a different architecture, you would want to: * Have another mechanism that automatically stops sending traffic to impaired instances (ideal), or * Have a means to manually remove the instances from service without being able to interact with or modify those instances in any way

Does that help, or have I misunderstood your problem?

> Is there another level of patches below point releases for backported security patches

No, because its unecessary (in my experience)

> or are your options to upgrade to something that adds new features and may be backwards incompatible

It won't be backwards incompatible, otherwise they would leave the change for a new minor release.

Why would you not want a new feature (e.g. the new "clone plugin") if it gives you value now without any migration required (because of incompatibilities that may arise if all new features are left for minor versions which are reserved for incompatibilities)?

MySQL GA dates: 5.5 3 December 2010 5.6 5 February 2013 5.7 21 October 2015 8.0 19 April 2018

The average seems to be about 2.5 years between minor releases, so this doesn't seem unusual.

Usually new minor versions are only released if there are: * Changes to database behaviour defaults (e.g. default sql mode) * Deprecation or removal of features * Compatibility changes (changes to replication formats)

New features are fine, as long as they don't introduce regressions in other areas.

Even MySQL 5.7 received some good improvements in the past 18-24 months (not just bug fixes).

I feel this is a really pedantic complaint from someone who either doesn't use MySQL, or wants to hate it for a silly reason.

> Nobody expects defaults to change between patch/minor versions.

Can you provide an example of an application-side (IOW, not a DB-infrastructure-type variable) that has changed it's default in a patch version after GA in 5.7 or 8.0?

IIRC, all of the defaults changed in very early non-GA versions.

> replication options like multi-source

MySQL 5.7 supports multi-source replication: https://dev.mysql.com/doc/refman/5.7/en/replication-channels...

> spider for sharding and galera are all built in and actively maintained.

Galera is available in MySQL-compatible binaries from Percona (Percona-XtraDB-Cluster): https://www.percona.com/software/mysql-database/percona-xtra...

MySQL also has their own Paxos-compliant clustering solution (called "Innodb cluster"), first available in 5.7, but much better in 8.0:

https://dev.mysql.com/doc/refman/5.7/en/mysql-innodb-cluster...

I played with Spider in MariaDB 10.3, and it didn't satisfy the requirements we had, and it doesn't seem to have progressed significantly since then.

Yes, it's neat that MariaDB merged it and is keeping it working, but there don't seem to be many use cases where it ends up being significantly better than e.g. multi-source replication (the use cases where you're going to run out of disk, in most cases spider's un-implemented push-down join etc. make it too slow to be practical either).

> Online schema changes,

https://dev.mysql.com/doc/refman/8.0/en/innodb-online-ddl-op...

(including "instant column add")

> easy horizontal scale (huge numbers that make db2 and oracle sweat), s3 backed tables, non blocking backups,

MySQL has had "MySQL Enterprise Backup" for a long time. Percona wrote a similar implementation (XtraBackup), which works for MySQL, Percona, and MariaDB. AFAIK MariaDB copied XtraBackup as mariabackup (not criticising them for doing this, just pointing out that Percona did the original heavy lifting on this).

> flashback, versioned tables

Yep, versioned tables are a nice feature that I don't think are available in any other MySQL-like DB.

> If you go enterprise, there isn’t much out there as accessible as maxscale for transaction failover!

Not even ProxySQL ( https://proxysql.com/ )?

To be clear, I like MariaDB, but Oracle isn't doing a bad job on MySQL. MySQL 5.7 is pretty solid, and they have been improving defaults (while leaving the option to enable compat settings which are necessary when migrating a large deployment between versions), and MySQL 8.0 has some really nice improvements. For some of the best features from each, Percona (either standard Percona 8.0, which gives features like MariaDB's thread-pool, more comprehensive encryption including redo-logs etc., or XtraDB-Cluster to go with Galera instead of Innodb Cluster) offers quite a few benefits/improvements (and is contributing a lot of bugfixes/improvements upstream).

(My team runs quite a big installation, currently running Percona 5.7, and migrating to MariaDB probably won't be practical or justifiable considering replication compatibility which is a requirement to do no-downtime migrations)