You can add Splunk to the list of companies with a similar clause. As a Splunk competitor it makes sales a bit harder initially (we can show our product's numbers, but nothing to compare them against), but if you can convince customers to set up a head-to-head proof-of-concept of their own, well, they tend to figure out why Splunk doesn't want you publishing benchmarks...
How do potential customers react to "These are our numbers. We would compare them with Splunk's, but their license forbids publishing benchmark results"?
Can you give your name ? I'm fighting against our Splunk decommission project because the big boys tell us we can just use ELK, to which we reply that it means months of devs to reproduce Splunk abilities, to which they reply human cost is invisible but license cost is a sore point for the board...
I love Splunk, it works so well after data is ingested so... who are you if you're better?
Can one provide a benchmarking suite that anyone can execute without posting the results of the test? Thus allowing others to run the test themselves easily but not putting you on the hook for the result?
One of my personal bugbears is the DeWitt Clause for Datomic, especially because knowing the performance profile of Datomic is very important for understanding whether your app will be a good fit for it given some of its peculiarities.
I think someone from Oracle would be more informed on that matter. JK. On a more serious note, who would dare to displease a multibillion corp with hundreds of lawyers (without being backed by a similar co & lawyers)?
> Oracle also inserted a clause in their terms of use that boiled down to the fact that one can’t publish benchmarks without getting an explicit approval from Oracle.
This feels horrible and would make me look away from any software that has such a clause. Then again, i use very little proprietary software in place and when i don't, it's mostly due to someone else choosing it for a project and me just needing to bite the bullet.
Though in regards to databases, i'm not sure why you'd fork over the cash and use something proprietary, unless you're trying to get rid of any sort of liability on your own end. Then again, i'm pretty sure that you could also find someone to offer support for your PostgreSQL or MySQL/MariaDB deployment, if you wanted to waste money (or did anything so interesting where such support would be warranted).
> Some cloud vendors permit you to benchmark their service but require reciprocity: you must make the benchmark reproducible and allow benchmarking of your own service or tool in response.
This is a bit better in comparison.
Though licenses in general puzzle me. For example, MongoDB is licensed under SSPL so anyone who offers it as a cloud service would have to open source their entire infrastructure: https://www.mongodb.com/licensing/server-side-public-license
The whole enforcement angle feels like it would probably impact an individual who benchmarks databases instead of reading bunches of legalese more, for example, than it would impact a larger company that could "figure things out".
Having only cursory experience with Oracle databases (as in install and run some queries and that's it), is there any advantage to them over MariaDB or PostgreSQL? Better development experience, easier to tune or no tuning necessary, anything that makes it worth over the free database servers?
As someone who spent some time in Oracle land: MariaDB can't hold a candle to it, but postgresql comes close.
Some things better in Oracle vs Postgres (and I might be dated on my postgres knowledge):. The active/active failover story of Oracle is better with RAC. Auto vacuum horror stories don't exist in Oracle.
Also pro oracle: The 'enterprise' ecosystem is better. Everything enterprisey integrates with oracle, postgresql is still a toss up.
But at the end of the day, I still vastly prefer postgresql. The endless list of weird idiosyncracies and weird limitations in oracle makes you always feel a bit dirty, compared to the relatively clean syntax of postgresql. In oracle land, it is common to wait 1 major version before using new features, because they are unstable when released.
And dealing with oracle support is hell with an additional bonus of pain. They take months for a simple bug fix. They won't admit a bug exists, then call you at 3AM and give you a patch written 2 years ago.
Oracle licensing is a game for advanced poker players. It will be expensive. Then you negotiate, walk away with a 40% discount, making it more expensive than competitors, and find out later it was still a bad deal. They'll interprete standard words like CPU in a slightly different way in their licenses, and finding out in an audit will cost you a lot. Licensing is a never ending drain on your time, and you will loose their games in the end.
As someone who worked with Oracle DB for a quite some time.
Stay as far away form it as possible. It only exists to milk already 'captured' companies, and all competitive niche advantages it had were slowly taken over by postgres.
About 10 or 15 years ago, OracleDB was years and years ahead of other databases. They had better replication, they had a better query optimizer, they had better storage management. OracleDB was the big thing you wanted in a business for a reason.
However, by now, MariaDB and especially PostgreSQL have caught up so much that this edge is gone and it feels like they are just siphoning money from companies who have invested in their big oracle cluster years ago. I do veto any new oracle-first or oracle-only development at work.
Interesting. As a SaaS vendor, we do not allow performance testing of the production system. Because, you know, just casually saturating production resources can become very iffy for strange and unexpected reasons. And you will always be able to saturate a system, or a subsystem of the subsystem of the system.
However, we have provided bigger customers, or customer willing to pay for it, with performance testing environments. We have, however, usually survived into the curiosity phase - "just how much to I have to throw at this thing to break it?".
Looking at the language, almost all of them allow you to run benchmarks since it's phrased as "you may not publish benchmark results"; it doesn't forbid to actually run them. Never mind that MS-SQL, Oracle, etc. are not SaaS vendors of course.
To be honest, if a cloud vendor has technical problems with someone running a few benchmarks then that would make me very wary of said cloud vendor. What's the difference between a "benchmark" and "using all resources I paid for" anyway?
There's a really big difference between "don't performance test on our hardware that you're sharing with other tenants" and "don't performance test on our software no matter whose hardware it's running on".
If you want to benchmark for internal reasons you don't publish the results and nobody knows. If you want to make a service to the community, run your benchmarks, download Tor and publish the results anonymously. I don't see what the big deal is?
Is this only limited to marketing claims where you post it on your company's website?
Its mainly related to MSSQL and Horracle. Horracle will just use their legal team (which is bigger than their engineers and developers) to bludgeon you over benchmarks.
Oracle isn't making an attestation about performance that the benchmark addresses, the benchmark aims to make statements about Oracle for some other reason, and that's important: Rights to free-speech generally end when they cause harm, and bearing full costs in the defense of false benchmarks is certainly harm.
Look at it this way: The clause aims to prevent purchases (entering into agreement with Oracle) under false pretenses. Oracle sells software to solve business problems, not you-need-a-paper-that-makes-Oracle-look-bad problems, and I think vendors are wise to protect themselves from that.
On the other hand, if you actually bought Oracle to solve a problem, and it didn't do that, you're still free to make those benchmarks and sue the shit out of Oracle with them, and this agreement can't by itself prevent the benchmarks from reaching the public record at that point.
> Is this only limited to marketing claims where you post it on your company's website?
If your company makes X and your company website contains a benchmark saying Oracle is slower than X, you're not just making a statement that you observed Oracle was slower than X, you're also making an attestation that the benchmark is a fair representation of both Oracle and X. And judge and jury are going to be wondering if it's as fair as you say, or if it's unfair as Oracle says.
Now, if you're a university and you don't make X, you might be able to argue that even if it's unfair, it was done in good-faith, and judge and jury may believe that, but Oracle will ask, if you truly believed X was fair, why didn't you get our feedback before publishing? and you'd better have a good answer to that.
On the other hand, if you choose to be anonymous, you may be able to avoid the judge and jury, but the community has to wonder who you are, whether you are motivated by a relationship to a company or product that competes with Oracle, or an impatient researcher who can't meet the standard of professional publishing. The community will wonder, but they have lots of other things to wonder about too, so they probably will not wonder for very long. So what's the point? Techdudes already know what they think of Oracle, and nobody who writes code talking or Oracle thinks that Oracle was chosen for its benchmarks, so who is this anonymous benchmark for?
Honestly, the presence of that clause screams to me "this app sucks and we'll sue you if you tell anyone how badly". That may not be the case whatsoever, but my first assumption is that they're trying to hide terrible performance.
While I would prefer if this clause was not a thing, I also understand why it exists even for great products.
It is surprisingly difficult to reproduce many workload benchmarks and quite easy to engineer a benchmark that misrepresents real-world database performance. There are tools that exist to generate optimally pathological workloads that target specific database implementations, while looking completely reasonable and innocuous. It doesn't even need to be a bad faith benchmark by a competitor, there is a high probability that the person configuring the environment does not know how to do it correctly and/or optimally.
The DeWitt Clause is a defense against the unfortunate pervasiveness of incompetent and/or bad faith benchmarking. Companies have a well-founded reason to not trust third parties to do a good job of representing the performance of their product.
Author of the blog post in question here. Let me clarify: they shouldn't be there because they're not OSI-approved, right? Just wanna get your point here.
(While I understand that BSL/SSPL lack certain liberties, I deemed it okay to mark them as "open source" for the purposes of this post.)
If Oracle ever wants to be the good guys just once, I have an idea for them that's right in their wheelhouse. Step 1: buy grsecurity's kernel hardening patches. Step 2: put said patches in the publicly released UEK source. Step 3: wait for grsecurity to refuse to give them future patches. Step 4: sue grsecurity for imposing further restrictions on the exercise of rights granted by the GPL.
They try to be the good guy. Their free tier is quite extensive (24GB of RAM, 4 ARM vCPUs and ±2 AMD cores, a several hundred GB of storage), good enough to run quite a decent personal cluster on, probably to lure in businesses for their AWS-style cloud services which are as ridiculously expensive as their competition.
However, just like AWS, Azure, and GCloud, their admin UI is complicated, slow, frustrating and full of invented acronyms and quirks.
Oracle Virtualbox is pretty good, and free for personal use. They also make patches available for their Oracle Linux kernel - the UEK, and in a better format than RedHat.
floren|3 years ago
car_analogy|3 years ago
xwolfi|3 years ago
I love Splunk, it works so well after data is ingested so... who are you if you're better?
raverbashing|3 years ago
The only number you need to compete with Splunk is a smaller price tag. I'd bet a lot of people would switch solutions and not look back
larkinrichards|3 years ago
unknown|3 years ago
[deleted]
dwohnitmok|3 years ago
casion|3 years ago
The performance of it depends heavily on a variety of factors which may it or may not apply to you.
Like a lot of software, the devil is in the details.
andrenotgiant|3 years ago
It seems like more of a threat stance to various partners and ecosystem players than anything else.
bluestreak|3 years ago
igorlukanin|3 years ago
KronisLV|3 years ago
This feels horrible and would make me look away from any software that has such a clause. Then again, i use very little proprietary software in place and when i don't, it's mostly due to someone else choosing it for a project and me just needing to bite the bullet.
Though in regards to databases, i'm not sure why you'd fork over the cash and use something proprietary, unless you're trying to get rid of any sort of liability on your own end. Then again, i'm pretty sure that you could also find someone to offer support for your PostgreSQL or MySQL/MariaDB deployment, if you wanted to waste money (or did anything so interesting where such support would be warranted).
> Some cloud vendors permit you to benchmark their service but require reciprocity: you must make the benchmark reproducible and allow benchmarking of your own service or tool in response.
This is a bit better in comparison.
Though licenses in general puzzle me. For example, MongoDB is licensed under SSPL so anyone who offers it as a cloud service would have to open source their entire infrastructure: https://www.mongodb.com/licensing/server-side-public-license
And yet i don't think that Digital Ocean is: https://www.digitalocean.com/products/managed-databases-mong... (or maybe they offer the older non-SSPL version).
The whole enforcement angle feels like it would probably impact an individual who benchmarks databases instead of reading bunches of legalese more, for example, than it would impact a larger company that could "figure things out".
redwood|3 years ago
RedShift1|3 years ago
hyperman1|3 years ago
Some things better in Oracle vs Postgres (and I might be dated on my postgres knowledge):. The active/active failover story of Oracle is better with RAC. Auto vacuum horror stories don't exist in Oracle.
Also pro oracle: The 'enterprise' ecosystem is better. Everything enterprisey integrates with oracle, postgresql is still a toss up.
But at the end of the day, I still vastly prefer postgresql. The endless list of weird idiosyncracies and weird limitations in oracle makes you always feel a bit dirty, compared to the relatively clean syntax of postgresql. In oracle land, it is common to wait 1 major version before using new features, because they are unstable when released.
And dealing with oracle support is hell with an additional bonus of pain. They take months for a simple bug fix. They won't admit a bug exists, then call you at 3AM and give you a patch written 2 years ago.
Oracle licensing is a game for advanced poker players. It will be expensive. Then you negotiate, walk away with a 40% discount, making it more expensive than competitors, and find out later it was still a bad deal. They'll interprete standard words like CPU in a slightly different way in their licenses, and finding out in an audit will cost you a lot. Licensing is a never ending drain on your time, and you will loose their games in the end.
Xelbair|3 years ago
Stay as far away form it as possible. It only exists to milk already 'captured' companies, and all competitive niche advantages it had were slowly taken over by postgres.
tetha|3 years ago
However, by now, MariaDB and especially PostgreSQL have caught up so much that this edge is gone and it feels like they are just siphoning money from companies who have invested in their big oracle cluster years ago. I do veto any new oracle-first or oracle-only development at work.
tetha|3 years ago
However, we have provided bigger customers, or customer willing to pay for it, with performance testing environments. We have, however, usually survived into the curiosity phase - "just how much to I have to throw at this thing to break it?".
Beltalowda|3 years ago
To be honest, if a cloud vendor has technical problems with someone running a few benchmarks then that would make me very wary of said cloud vendor. What's the difference between a "benchmark" and "using all resources I paid for" anyway?
josephcsible|3 years ago
jamesblonde|3 years ago
Nextgrid|3 years ago
Is this only limited to marketing claims where you post it on your company's website?
SahAssar|3 years ago
igorlukanin|3 years ago
noasaservice|3 years ago
unknown|3 years ago
[deleted]
geocar|3 years ago
The big deal is that it's slanderous.
Oracle isn't making an attestation about performance that the benchmark addresses, the benchmark aims to make statements about Oracle for some other reason, and that's important: Rights to free-speech generally end when they cause harm, and bearing full costs in the defense of false benchmarks is certainly harm.
Look at it this way: The clause aims to prevent purchases (entering into agreement with Oracle) under false pretenses. Oracle sells software to solve business problems, not you-need-a-paper-that-makes-Oracle-look-bad problems, and I think vendors are wise to protect themselves from that.
On the other hand, if you actually bought Oracle to solve a problem, and it didn't do that, you're still free to make those benchmarks and sue the shit out of Oracle with them, and this agreement can't by itself prevent the benchmarks from reaching the public record at that point.
> Is this only limited to marketing claims where you post it on your company's website?
If your company makes X and your company website contains a benchmark saying Oracle is slower than X, you're not just making a statement that you observed Oracle was slower than X, you're also making an attestation that the benchmark is a fair representation of both Oracle and X. And judge and jury are going to be wondering if it's as fair as you say, or if it's unfair as Oracle says.
Now, if you're a university and you don't make X, you might be able to argue that even if it's unfair, it was done in good-faith, and judge and jury may believe that, but Oracle will ask, if you truly believed X was fair, why didn't you get our feedback before publishing? and you'd better have a good answer to that.
On the other hand, if you choose to be anonymous, you may be able to avoid the judge and jury, but the community has to wonder who you are, whether you are motivated by a relationship to a company or product that competes with Oracle, or an impatient researcher who can't meet the standard of professional publishing. The community will wonder, but they have lots of other things to wonder about too, so they probably will not wonder for very long. So what's the point? Techdudes already know what they think of Oracle, and nobody who writes code talking or Oracle thinks that Oracle was chosen for its benchmarks, so who is this anonymous benchmark for?
unknown|3 years ago
[deleted]
kstrauser|3 years ago
jandrewrogers|3 years ago
It is surprisingly difficult to reproduce many workload benchmarks and quite easy to engineer a benchmark that misrepresents real-world database performance. There are tools that exist to generate optimally pathological workloads that target specific database implementations, while looking completely reasonable and innocuous. It doesn't even need to be a bad faith benchmark by a competitor, there is a high probability that the person configuring the environment does not know how to do it correctly and/or optimally.
The DeWitt Clause is a defense against the unfortunate pervasiveness of incompetent and/or bad faith benchmarking. Companies have a well-founded reason to not trust third parties to do a good job of representing the performance of their product.
unknown|3 years ago
[deleted]
unknown|3 years ago
[deleted]
jenny91|3 years ago
igorlukanin|3 years ago
(While I understand that BSL/SSPL lack certain liberties, I deemed it okay to mark them as "open source" for the purposes of this post.)
unknown|3 years ago
[deleted]
chmod775|3 years ago
I'm sure they are at least purchasing some modern-day 'indulgences' by - for instance - donating food to starving north korean elites?
josephcsible|3 years ago
jeroenhd|3 years ago
However, just like AWS, Azure, and GCloud, their admin UI is complicated, slow, frustrating and full of invented acronyms and quirks.
throwamon|3 years ago
https://www.upi.com/Top_News/World-News/2020/10/07/North-Kor...
fragmede|3 years ago
raverbashing|3 years ago
native_samples|3 years ago
unknown|3 years ago
[deleted]
GorillaWarfare|3 years ago
jkaplowitz|3 years ago
igorlukanin|3 years ago
unknown|3 years ago
[deleted]