When non-traditional databases became popular, I thought it was primarily driven by people short-sightedly prioritizing development time over all of the good relational database features.
Now I see things differently -- the non-traditional databases are just better at scaling horizontally, eventual consistency, and running in cloud environments than the traditional databases. They are easier to set up and use. Some of them now have pretty good relational features and schemas.
During the past decade the Oracles of the world have continued to think dismissively of new non-traditional databases, as I did at first. The non-traditional databases got better and better while Oracle kept doing a lot of the same stuff. Oracle just didn't take the competition seriously, and that's fair enough because the competition didn't deserve to be taken seriously at first -- but now it does, and it's becoming a threat to Oracle's business.
It's obvious that Amazon would want to use their own database products -- but the impressive thing is that those products, which are not very old, are already good enough to replace Oracle for a lot of use cases at very high scale.
"[N]on-traditional databases" with far better performance actually pre-date relational databases.
Relational databases solved the problem they had, of requiring much developer effort to evolve for new application needs, and re-organization of the datastore for changing performance needs. Basically, everything is a table, and you can combine/separate tables, from whatever form it currently is, to whatever form you currently need. This is also helpful for reporting (as opposed to operational applications).
But relational databases, like dynamic languages, pay a performance price for their flexibility.
Of course, every other innovation has since been added, and there's Oracle's licensing too. So it's much more complex than the base technology of relational algebra.
There are places where relational DBs are inherently better than noSQL (namely when you need fully up-to-date information on every query, no matter what) - But those situations are becoming more rare compared to situations where potentially stale data is an okay tradeoff for performance gains.
Honestly, IMO the biggest issue holding noSQL databases back is lack of good documentation/support a lot of the time. Technically, though, they're going to become the standard for most use cases soon.
Of the three DB products mentioned in the tweet, only one is a non-traditional database: DynamoDB.
Both Redshift and Aurora are traditional RDBMs, which are forked from very old codebases.
Redshift is a fork of Postgres 8.0 that was adapted for OLAP use-cases, while Aurora is a fork of MySQL (or PostgreSQL in the PostgreSQL-compatible versions) that runs on a custom storage engine.
The main reason technically-capable companies like Amazon or Netflix are moving away Oracle has nothing to do with horizontal scaling. It's the licensing costs.
Oracle is just too expensive for them, and even if you are not Amazon yourself, RDS (or any other Cloud RDBMS) would be much cheaper for you.
The non-traditional databases got better and better while Oracle kept doing a lot of the same stuff
I'm not sure this is true, you can look at the new features in 8i, 9i, 10g, 11g, 12c and see that Oracle is adding a lot of features, and some of them are very impressive. The problem is most people don't need most of them and if you want any of them you have to pay for all of them.
And the "non-traditional" databases had a pretty low bar. MongoDB just had to stay up for more than 5 minutes and not lose your data. Do they even do that yet?
> The non-traditional databases got better and better while Oracle kept doing a lot of the same stuff. Oracle just didn't take the competition seriously, and that's fair enough because the competition didn't deserve to be taken seriously at first -- but now it does, and it's becoming a threat to Oracle's business.
Yes, like traditional car companies not taking EV's too seriously, or Intel not taking mobile/ARM too seriously, etc. It seems the older and more established a company is, the thicker their forward-thinking blinders are. "We're king, no one can EVER touch us."
Shameless plug, but I think the new math that underlies global consistency is super interesting, I did a webcast with Professor Daniel Abadi about how Google Spanner differs from FaunaDB (my employer, which doesn’t require atomic clocks), you can get a link to the recording here https://www2.fauna.com/wcspannercalvin
I imagine at the scale of Amazon replacing some of the core data stores with all their existing data is a quite complex task. Just think about the amount of data you need to migrate and keep in sync during the migration and all the third party tools using such data. So while it takes long, I don't think Amazon is particularly slow doing it.
I bet Amazon uses Oracle back-office software like PeopleSoft , some Oracle ERP and/or BI Tools(Hyperion). Oracle has a stronghold on these type of systems and they only play nice with other Oracle products.
I have no clue why Larry Ellison thinks it's a good idea to take potshots at one of his own large customers, instead of talking about the benefits of Oracle's products. Very weird, and kind of childish.
Because Oracle doesn't give a crap, they know they have a captive market and are the definition of a rent-seeking company. They don't sell their products based on their merits, they do it because companies have either have no choice or are already held hostage by them.
Because people can have opinions and don't have to shill out to the highest bidder. Amazon is in a position where they don't have to rely on Oracle, and that's a good thing for them. It's also a good thing for everyone else.
This is not surprising at all. For the past couple of years, Oracle's behavior has been egregious at best.
You can pick up any of their product and find that MYSQL, MS-SQL etc are supported but 2001/03 release because of "incompatibility issues". If that is not enough Oracle's support is clueless about what are these "incompatibility issues". And given the precarious security environment we are in, everyone needs a DB which has been patched sufficiently and allows newer features like TLS 3.0 etc, you are left with no choice but to go for the only supported DB is Oracle DB. I have sat in many meetings where the CIO/CTO have seen this as an arm twisting tactic by Oracle.
I am no market expert but if Oracle keeps going down this path they will cease to exist in next 10-15 years.
They just keep buying niche enterprise applications that have entrenched non-technical users and then raise prices, drop interoperability with non Oracle and start milking.
I'm not heavily involved with databases from a development point of view, is someone able to explain why Oracle is as successful as it is?
What sets them apart from what seems like a plethora of other DB systems?
Putting aside business practices, Oracle database has features that don't show up in opensource systems, often for years.
The one I miss most is resource constraints on queries. It's nice to be able to guarantee that some queries will get more resources than others.
On the other hand, Oracle's SQL dialect is (or was, I stopped using it about 5 years ago) full of frustrating backwardness. No boolean type, so you get a mix of CHAR(1)s or INTs, depending on the prevailing DBA's opinion. And there's no serial or autoincrementing type, so for every table you wind up copying and pasting the same code over and over (create index, create sequence, insert trigger, update trigger).
The most head-scratching part is that you can find apologists for these flaws.
They are viewed as a 'safe' choice for enterprise customers in that they've been around forever, run on pretty much everything, and have product offerings that can tick off pretty much every vendor selection checkbox. They're essentially the IBM of databases (if you're not already tied to DB/2 as a 'big blue' shop or SQL Server as a Microsoft shop)
They have been around virtually forever, they had a very aggressive sales organization, and their database does generally perform better than the competition for most standard SQL workloads.
That brings back an old memory of the data warehouse being extremely overburdened during peak. Capacity constraints and over utilization kept bringing down clusters. As a last ditch effort, the data warehouse team started randomly disabling queries en masse, under the assumption that if they were actually mission critical, the user would just re-enable them again.
I knew some engineers that worked on a centralized data engineering team that served 100+ software teams, and they managed several thousand scheduled queries. I felt so bad for the guy that had pager duty that first night. He said he got a sev-2 every 12 minutes on average for 24 hours straight.
I certainly hope Amazon has fixed their data warehouse issues since then.
This is as much or more about moving away from a specific product stack - Oracle E-Business Suite (EBS), which is an aging and expensive ERP titan. The news and people who aren't dealing with Oracle don't necessarily see that clearly.
I'd bet that their Oracle Data Warehouse being referred to here isn't just about the Oracle RDBMS database technology itself, they are talking about a specific COTS Oracle Data Warehouse model that you can buy prebuilt and works as a destination for all other Oracle ERP etc. subsytems.
I say this knowing it first hand coming from a company trapped in that particular Oracle stack. It's expensive, limiting and very locked in, and has a huge footprint and requires specialists to run (as do many ERPs).
I see actually a mad scramble by the big tech vendors (Microsoft, SAP and others) trying to push their ERP solutions as cloud enablement has opened up a window of opportunity to shift and escape some of the pains of the old ways (while creating an entirely new set of problems).
I'd be more interested if Amazon sees this same opening and is trying to enter the ERP game itself by building something in house that they turn around and offer as a service via AWS. They certainly have the scale to look pull off such a thing, and they already own all the enabling technologies to build an ERP system, data warehouse and rest of the stack.
Wanted to share some insights on the data warehouse industry here from a co-founder of intermix.io.
According to db-engines.com, Oracle has declined 9% since July 2016. Amazon Redshift and Google BigQuery have grown over 50% in the same period, while Snowflake and PrestoDB (although each are 10x smaller than Redshift and BigQuery) have grown over 60%.
This is happening because enterprises are shifting to the cloud. When they go to the cloud they are shifting -away- from on-prem warehouses like Oracle, Teradata, and Vertica. Enterprises choose Amazon or Google depending on which cloud platform they adopt.
Companies launched after 2010 were born in the cloud (and thus never used Oracle since Oracle does not have a cloud) and are more likely to choose Snowflake due to ease-of-use (even as Snowflake is more expensive than Redshift).
What does this mean?
Consider that we are still in the early phases of cloud adoption. 32% of enterprises are in the cloud, rising to 52% by 2022. At the same time, over half of enterprises said they will use more than one cloud (hybrid cloud) within the next 10 years.
Amazon Redshift is the warehouse of choice for Enterprises on AWS. Snowflake is eating up the mid and SMB markets. prestoDB solves awesome problems for interactive and exploratory analysis for all. BigQuery is used by GCP customers.
These trends indicate that Oracle will struggle to grow revenues and margins, as they are relegated to serve the (still large but shrinking) portion of the market that chooses on-prem, and pursue aggressive rent-seeking of an aging install base (the Java mess is an unrelated but telling example of this strategy).
To reverse this trend, Oracle must find a way to serve cloud customers. That probably means acquisitions.
The story goes that Amazon was one of Oracle’s biggest customer, and they have been for a long time.
When Oracle entered the cloud they started making fun of Amazon, “If their cloud and database offerings are so great, how come they use us to do the actual heavy lifting? That’s because our stuff is rock solid and theirs isn’t”
This pissed off Bezos, the sky broke and a voice foretold, NO MORE ORACLE!
This isn’t about nosql vs sql, this is about how oracle is lead by an out of touch, megalomaniac selling a subpar product, that you can literally get for free. I hope this gives other companies the kick they need to dump Oracle and MSSQL too.
Many Oracle installs used to rely on the apps that ran on them like ERPs and the like. Oracle bought many of those companies (e.g. Peoplesoft)
With cloud availability and many of the features of high end DBs commoditized, and the new strain of apps being cloud / SaaS, Oracle's available market has dried up.
Oracle has been threatened for a while, hence their desperate run to the cloud. Like IBM, though, they have enough fat to survive a very, very long winter; and, as cornered animals go, they are among the most vicious.
Personally, I don’t expect to see Oracle die in my lifetime. They will keep cannibalizing other companies to stay alive, probably forever.
[+] [-] twblalock|7 years ago|reply
Now I see things differently -- the non-traditional databases are just better at scaling horizontally, eventual consistency, and running in cloud environments than the traditional databases. They are easier to set up and use. Some of them now have pretty good relational features and schemas.
During the past decade the Oracles of the world have continued to think dismissively of new non-traditional databases, as I did at first. The non-traditional databases got better and better while Oracle kept doing a lot of the same stuff. Oracle just didn't take the competition seriously, and that's fair enough because the competition didn't deserve to be taken seriously at first -- but now it does, and it's becoming a threat to Oracle's business.
It's obvious that Amazon would want to use their own database products -- but the impressive thing is that those products, which are not very old, are already good enough to replace Oracle for a lot of use cases at very high scale.
[+] [-] hyperpallium|7 years ago|reply
Relational databases solved the problem they had, of requiring much developer effort to evolve for new application needs, and re-organization of the datastore for changing performance needs. Basically, everything is a table, and you can combine/separate tables, from whatever form it currently is, to whatever form you currently need. This is also helpful for reporting (as opposed to operational applications).
But relational databases, like dynamic languages, pay a performance price for their flexibility.
Of course, every other innovation has since been added, and there's Oracle's licensing too. So it's much more complex than the base technology of relational algebra.
[+] [-] Derek_MK|7 years ago|reply
Honestly, IMO the biggest issue holding noSQL databases back is lack of good documentation/support a lot of the time. Technically, though, they're going to become the standard for most use cases soon.
[+] [-] unscaled|7 years ago|reply
The main reason technically-capable companies like Amazon or Netflix are moving away Oracle has nothing to do with horizontal scaling. It's the licensing costs. Oracle is just too expensive for them, and even if you are not Amazon yourself, RDS (or any other Cloud RDBMS) would be much cheaper for you.
[+] [-] jacques_chester|7 years ago|reply
[+] [-] gaius|7 years ago|reply
I'm not sure this is true, you can look at the new features in 8i, 9i, 10g, 11g, 12c and see that Oracle is adding a lot of features, and some of them are very impressive. The problem is most people don't need most of them and if you want any of them you have to pay for all of them.
And the "non-traditional" databases had a pretty low bar. MongoDB just had to stay up for more than 5 minutes and not lose your data. Do they even do that yet?
[+] [-] grumpydba|7 years ago|reply
[+] [-] emmelaich|7 years ago|reply
To whit, de-normalised tables for the sake of "efficiency".
It's pretty difficult to under normalise a two column table -- which is what K/V is.
[+] [-] dgemm|7 years ago|reply
[+] [-] cronix|7 years ago|reply
Yes, like traditional car companies not taking EV's too seriously, or Intel not taking mobile/ARM too seriously, etc. It seems the older and more established a company is, the thicker their forward-thinking blinders are. "We're king, no one can EVER touch us."
[+] [-] AareyBaba|7 years ago|reply
[+] [-] jchanimal|7 years ago|reply
[+] [-] dstroot|7 years ago|reply
Oracle sued my company a few years back for license compliance issues - I vowed never to run their stuff again and rip it out wherever I find it.
[+] [-] Dunedan|7 years ago|reply
I imagine at the scale of Amazon replacing some of the core data stores with all their existing data is a quite complex task. Just think about the amount of data you need to migrate and keep in sync during the migration and all the third party tools using such data. So while it takes long, I don't think Amazon is particularly slow doing it.
[+] [-] danso|7 years ago|reply
[+] [-] adrr|7 years ago|reply
[+] [-] mathattack|7 years ago|reply
[+] [-] czardoz|7 years ago|reply
[+] [-] snuxoll|7 years ago|reply
[+] [-] thisisit|7 years ago|reply
From Wikipedia: "Note that there was no v1 of Oracle Database, as Larry Ellison, "knew no one would want to buy version 1."
https://en.m.wikipedia.org/wiki/Oracle_Database#Releases_and...
[+] [-] devwastaken|7 years ago|reply
[+] [-] thisisit|7 years ago|reply
You can pick up any of their product and find that MYSQL, MS-SQL etc are supported but 2001/03 release because of "incompatibility issues". If that is not enough Oracle's support is clueless about what are these "incompatibility issues". And given the precarious security environment we are in, everyone needs a DB which has been patched sufficiently and allows newer features like TLS 3.0 etc, you are left with no choice but to go for the only supported DB is Oracle DB. I have sat in many meetings where the CIO/CTO have seen this as an arm twisting tactic by Oracle.
I am no market expert but if Oracle keeps going down this path they will cease to exist in next 10-15 years.
[+] [-] narrator|7 years ago|reply
[+] [-] Nerada|7 years ago|reply
[+] [-] jacques_chester|7 years ago|reply
The one I miss most is resource constraints on queries. It's nice to be able to guarantee that some queries will get more resources than others.
On the other hand, Oracle's SQL dialect is (or was, I stopped using it about 5 years ago) full of frustrating backwardness. No boolean type, so you get a mix of CHAR(1)s or INTs, depending on the prevailing DBA's opinion. And there's no serial or autoincrementing type, so for every table you wind up copying and pasting the same code over and over (create index, create sequence, insert trigger, update trigger).
The most head-scratching part is that you can find apologists for these flaws.
[+] [-] blihp|7 years ago|reply
[+] [-] zjaffee|7 years ago|reply
[+] [-] org3432|7 years ago|reply
[+] [-] unknown|7 years ago|reply
[deleted]
[+] [-] mathattack|7 years ago|reply
[+] [-] saosebastiao|7 years ago|reply
I knew some engineers that worked on a centralized data engineering team that served 100+ software teams, and they managed several thousand scheduled queries. I felt so bad for the guy that had pager duty that first night. He said he got a sev-2 every 12 minutes on average for 24 hours straight.
I certainly hope Amazon has fixed their data warehouse issues since then.
[+] [-] sixdimensional|7 years ago|reply
I'd bet that their Oracle Data Warehouse being referred to here isn't just about the Oracle RDBMS database technology itself, they are talking about a specific COTS Oracle Data Warehouse model that you can buy prebuilt and works as a destination for all other Oracle ERP etc. subsytems.
I say this knowing it first hand coming from a company trapped in that particular Oracle stack. It's expensive, limiting and very locked in, and has a huge footprint and requires specialists to run (as do many ERPs).
I see actually a mad scramble by the big tech vendors (Microsoft, SAP and others) trying to push their ERP solutions as cloud enablement has opened up a window of opportunity to shift and escape some of the pains of the old ways (while creating an entirely new set of problems).
I'd be more interested if Amazon sees this same opening and is trying to enter the ERP game itself by building something in house that they turn around and offer as a service via AWS. They certainly have the scale to look pull off such a thing, and they already own all the enabling technologies to build an ERP system, data warehouse and rest of the stack.
That's my pet theory at least!
[+] [-] btown|7 years ago|reply
https://www.nasdaq.com/symbol/orcl/after-hours-chart
The “lawnmower” keeps on mowing: https://news.ycombinator.com/item?id=18323166
[+] [-] adrr|7 years ago|reply
[+] [-] paul94133|7 years ago|reply
According to db-engines.com, Oracle has declined 9% since July 2016. Amazon Redshift and Google BigQuery have grown over 50% in the same period, while Snowflake and PrestoDB (although each are 10x smaller than Redshift and BigQuery) have grown over 60%.
This is happening because enterprises are shifting to the cloud. When they go to the cloud they are shifting -away- from on-prem warehouses like Oracle, Teradata, and Vertica. Enterprises choose Amazon or Google depending on which cloud platform they adopt.
Companies launched after 2010 were born in the cloud (and thus never used Oracle since Oracle does not have a cloud) and are more likely to choose Snowflake due to ease-of-use (even as Snowflake is more expensive than Redshift).
What does this mean?
Consider that we are still in the early phases of cloud adoption. 32% of enterprises are in the cloud, rising to 52% by 2022. At the same time, over half of enterprises said they will use more than one cloud (hybrid cloud) within the next 10 years.
Amazon Redshift is the warehouse of choice for Enterprises on AWS. Snowflake is eating up the mid and SMB markets. prestoDB solves awesome problems for interactive and exploratory analysis for all. BigQuery is used by GCP customers.
These trends indicate that Oracle will struggle to grow revenues and margins, as they are relegated to serve the (still large but shrinking) portion of the market that chooses on-prem, and pursue aggressive rent-seeking of an aging install base (the Java mess is an unrelated but telling example of this strategy).
To reverse this trend, Oracle must find a way to serve cloud customers. That probably means acquisitions.
[+] [-] sys_64738|7 years ago|reply
[+] [-] nojvek|7 years ago|reply
When Oracle entered the cloud they started making fun of Amazon, “If their cloud and database offerings are so great, how come they use us to do the actual heavy lifting? That’s because our stuff is rock solid and theirs isn’t”
This pissed off Bezos, the sky broke and a voice foretold, NO MORE ORACLE!
[+] [-] stanislavb|7 years ago|reply
[+] [-] ec109685|7 years ago|reply
[+] [-] toyg|7 years ago|reply
[+] [-] erik_landerholm|7 years ago|reply
[+] [-] 1024core|7 years ago|reply
[+] [-] kapauldo|7 years ago|reply
[deleted]
[+] [-] chris_wot|7 years ago|reply
How many folks still actually use Oracle? And how many are trying to get rid of them?
[+] [-] r00fus|7 years ago|reply
With cloud availability and many of the features of high end DBs commoditized, and the new strain of apps being cloud / SaaS, Oracle's available market has dried up.
[+] [-] toyg|7 years ago|reply
Personally, I don’t expect to see Oracle die in my lifetime. They will keep cannibalizing other companies to stay alive, probably forever.
[+] [-] Axsuul|7 years ago|reply
[+] [-] oh-kumudo|7 years ago|reply