Good for them and all, and I know they're still kind of stealth mode, but this article doesn't say much to me.
It's not that I doubt they can deliver, I just want to see some numbers. The one graph I could find (http://www.clustrix.com/wp-content/uploads/2010/04/clustrix-...) really doesn't say much, and to be honest, I can't even really tell how I'm supposed to be reading it.
You don't go and build the best sports car in the world and then tell everyone "it goes really fast", you show videos of it screaming around a curvy mountain road. I want my curvy mountain road video.
They were supposed to launch on May 4, but the story seems to have got out a day earlier. There should be more details coming, starting from their official launch at Web 2.0 tomorrow.
I thought the holy grail of the web scale is being able to start with little (startups cannot pay 80K for a licence), then painlessly scale the thing. What they offer seems more like a better price option for large enterprise solutions - licence is (supposedly) cheaper than the cost of scaling. And that's cool, ok. But, personally, what I would like to be able to do is to launch a small application with a DB holding, say, 10K records and then scale it up to billions of records investing money when I actually have them. Or not.
Agreed. We purchase an Oracle driven enterprise app that would probably be the target market for this. We would hold billions of records if it was economical to do so - the problem is that 80K is a lot and it is probably cheaper to pay me to do 3hrs a week maintenance/backup.
The other area we might have used this is somewhere we have about 40 billion database entries (several TB's of data) and growing. However, like Google et al, once you get to that scale it's not worth paying for a proprietary service because cheap/comodity hardware on a FOSS stack is a lot cheaper.
However, there is almost certainly a market here for "enterprise" type businesses (you know, the sort of grey faced telecoms-style place). The problem is supplanting Oracle...
* There is a physical limit to ACID scalability because of synchronization latency and network throughput maximums. KV stores don't have this issue.
* Will developers still have to implement KV caching layers for database data? I suppose the real question is: does the system accelerate "hot" data where normal RDBMS sharding schemes would otherwise not?
* Does the linear scalability also apply to OLAP-type activity where queries are ran against large datasets?
Oracle RAC allows multiple machines to synchronize access to a shared disk. Each node in a RAC cluster locks individual disk pages and brings the data to the node for processing. Only one node is working on a query a time, while synchronizing access to resources at a physical level.
Clustrix is fundamentally different. We compile the query down into query fragments and evaluate relevant portions of the query at the node which holds the relevant data. We bring the query to the data.
You can find more specifics on our architecture in our white paper:
Clustrix is all about concurrency and parallelism. While we didn't specifically focus on OLAP, our model allows us to do very well for these types of workloads because we break the query apart and let individual nodes work on their portion of the query.
While we deliver our product as an appliance, it's all based on commodity hardware.
If you had an infinitely scalable database, would you still want to put a caching layer in front of it? Most people would rather not deal with the added complexity.
In a typical sharding scheme, one would choose a partition key (typically some primary key, e.g. user id) and use that as a distribution strategy. At such granularity, hot spots are a common problem. In Clustrix, we have independent distribution/partitioning for each table/index, down to the row level. To get hot spots is if you contend on a single row -- but that's really an indication of something going terribly wrong in the data model.
ACID (transactional properties) and KV (data model) are orthogonal concepts. Our systems scales exceptionally well and supports a wide range of data models.
I'm not sure the hardware is so custom - the pictures on the Clustrix site look very much like generic Supermicro x86 servers with a Clustrix logo stuck on them.
The "OLTP landscape chart" raises red flags. Matching functionality of Oracle in 4 years and $18m is an extraordinary claim.
So this raises the question as to what exactly can it do? The white-paper covers straight select, insert, sort and a an inner join which is pretty good already, but it doesn't go beyond that.
The complete MySQL API compatibility is a bold statement.
If they implemented the whole thing themselves (I would say that should take more than 4 years in a couple-of-guys setup) then maintaining that compatibility must be/will be a nightmare. If you sold me a product as MySQL-compatible and I find that some bizarre and undocumented side-effect of MySQL implementation that my code relies on is missing, you will have to emulate that side effect.
If, on the other hand, they based their implementation on MySQL code, then this raises some licensing questions (MySQL is GPL so they will have to make their changes public). They may have separated the code somehow so that this is less of an issue, though it adds a bit of uncertainty to the whole setup.
there was this programming language i remember reading about a while ago that was basically about moving computation around to different nodes to bring the "function to the data" as opposed to the other way around -- which is the exact same thing that the white paper says clustrix is doing.
anyways, with any sort of non-trivially related data, it becomes a continuous graph optimization problem to minimize the number of nodes you hit on any given "query" (ie: keep data that tends to be accessed together on the same nodes or die), and can easily see how though it may seem like an elegant solution, that you can very quickly run into scary problems that really are just a giant tangled mess.
anyways, distributed joins aren't that hard. select and insert are solved by the key-value/consistent hashing dbs. even mysql cluster gives you all that stuff these days (the whitepaper mentions non-trivial multi-master joins)
Hey, they're looking for a "Technical Support Engineer - 6AM Shift":
Required Skills / Duties:
...
- Able to lift 80 lbs several times in a single day. Our servers weigh between 50 and 140 lbs depending on configuration.
My company might have bought this 6 months ago assuming it provably works as advertised. We ended up sharding (which I'd gladly pay $80k to have avoided) and going Cassandra for future apps.
Whitepaper seems to imply all nodes are connected to all other nodes, using infiniband. Infiniband's point-to-point afaik, so it'd be interesting to know how they plan to do that with hundreds of nodes.
Exadata does use SSDs now, in the form of Sun's flash memory cards (though they still are backed by rotational disk). You're right, however that this and Exadata look to attack the same problem in a similar way. If we believe Clustrix about the "100s of nodes" rather than the 20 they show data for, it would likely out-perform Exadata by a bit too.
Congrats Sergei and the Clustrix team! Scalable databases are a really hard problem to solve. It's great to see products that push the boundaries of what's possible.
How is this comparable to fathomdb in terms of what it provides on a business level. (not how i.e. fathomdb leverages 'the cloud' where as this seems to be using specialized hardware?)
[+] [-] leif|16 years ago|reply
It's not that I doubt they can deliver, I just want to see some numbers. The one graph I could find (http://www.clustrix.com/wp-content/uploads/2010/04/clustrix-...) really doesn't say much, and to be honest, I can't even really tell how I'm supposed to be reading it.
You don't go and build the best sports car in the world and then tell everyone "it goes really fast", you show videos of it screaming around a curvy mountain road. I want my curvy mountain road video.
[+] [-] bertove|16 years ago|reply
[+] [-] unknown|16 years ago|reply
[deleted]
[+] [-] qq66|16 years ago|reply
[+] [-] snitko|16 years ago|reply
[+] [-] ErrantX|16 years ago|reply
The other area we might have used this is somewhere we have about 40 billion database entries (several TB's of data) and growing. However, like Google et al, once you get to that scale it's not worth paying for a proprietary service because cheap/comodity hardware on a FOSS stack is a lot cheaper.
However, there is almost certainly a market here for "enterprise" type businesses (you know, the sort of grey faced telecoms-style place). The problem is supplanting Oracle...
[+] [-] alanh|16 years ago|reply
[+] [-] rbranson|16 years ago|reply
* Custom hardware sounds expensive.
* This does not solve CAP issues.
* There is a physical limit to ACID scalability because of synchronization latency and network throughput maximums. KV stores don't have this issue.
* Will developers still have to implement KV caching layers for database data? I suppose the real question is: does the system accelerate "hot" data where normal RDBMS sharding schemes would otherwise not?
* Does the linear scalability also apply to OLAP-type activity where queries are ran against large datasets?
* What are they doing that Oracle RAC isn't?
[+] [-] sergei|16 years ago|reply
Clustrix is fundamentally different. We compile the query down into query fragments and evaluate relevant portions of the query at the node which holds the relevant data. We bring the query to the data.
You can find more specifics on our architecture in our white paper:
http://www.clustrix.com/wp-content/uploads/2010/04/clustrix-...
Clustrix is all about concurrency and parallelism. While we didn't specifically focus on OLAP, our model allows us to do very well for these types of workloads because we break the query apart and let individual nodes work on their portion of the query.
While we deliver our product as an appliance, it's all based on commodity hardware.
If you had an infinitely scalable database, would you still want to put a caching layer in front of it? Most people would rather not deal with the added complexity.
In a typical sharding scheme, one would choose a partition key (typically some primary key, e.g. user id) and use that as a distribution strategy. At such granularity, hot spots are a common problem. In Clustrix, we have independent distribution/partitioning for each table/index, down to the row level. To get hot spots is if you contend on a single row -- but that's really an indication of something going terribly wrong in the data model.
ACID (transactional properties) and KV (data model) are orthogonal concepts. Our systems scales exceptionally well and supports a wide range of data models.
[+] [-] DenisM|16 years ago|reply
Their advertised setup is a small number of nodes interconnected with infiniband. Partitioning problem doesn't apply here.
[+] [-] forkqueue|16 years ago|reply
[+] [-] DenisM|16 years ago|reply
So this raises the question as to what exactly can it do? The white-paper covers straight select, insert, sort and a an inner join which is pretty good already, but it doesn't go beyond that.
[+] [-] bertove|16 years ago|reply
[+] [-] boris|16 years ago|reply
If they implemented the whole thing themselves (I would say that should take more than 4 years in a couple-of-guys setup) then maintaining that compatibility must be/will be a nightmare. If you sold me a product as MySQL-compatible and I find that some bizarre and undocumented side-effect of MySQL implementation that my code relies on is missing, you will have to emulate that side effect.
If, on the other hand, they based their implementation on MySQL code, then this raises some licensing questions (MySQL is GPL so they will have to make their changes public). They may have separated the code somehow so that this is less of an issue, though it adds a bit of uncertainty to the whole setup.
[+] [-] lzimm|16 years ago|reply
anyways, with any sort of non-trivially related data, it becomes a continuous graph optimization problem to minimize the number of nodes you hit on any given "query" (ie: keep data that tends to be accessed together on the same nodes or die), and can easily see how though it may seem like an elegant solution, that you can very quickly run into scary problems that really are just a giant tangled mess.
anyways, distributed joins aren't that hard. select and insert are solved by the key-value/consistent hashing dbs. even mysql cluster gives you all that stuff these days (the whitepaper mentions non-trivial multi-master joins)
holla
[+] [-] Maro|16 years ago|reply
Required Skills / Duties: ... - Able to lift 80 lbs several times in a single day. Our servers weigh between 50 and 140 lbs depending on configuration.
Sounds more like a bodybuilder position to me =)
http://www.clustrix.com/about/careers
[+] [-] lsb|16 years ago|reply
If you give people a few TB of memory, of course database reads are going to be fast.
[+] [-] DenisM|16 years ago|reply
[+] [-] mattmaroon|16 years ago|reply
[+] [-] unknown|16 years ago|reply
[deleted]
[+] [-] DenisM|16 years ago|reply
Whitepaper seems to imply all nodes are connected to all other nodes, using infiniband. Infiniband's point-to-point afaik, so it'd be interesting to know how they plan to do that with hundreds of nodes.
[+] [-] josephruscio|16 years ago|reply
It's been used in high-performance compute clusters with thousands of individual nodes.
[+] [-] jbellis|16 years ago|reply
[+] [-] drosenthal|16 years ago|reply
[+] [-] coffeemug|16 years ago|reply
[+] [-] shaunxcode|16 years ago|reply
[+] [-] bkrausz|16 years ago|reply
[+] [-] apphacker|16 years ago|reply
[+] [-] sown|16 years ago|reply
I'm getting bored at my current job. I think if I could port our enterprise solution to cassandra it would help in an interview?
[+] [-] ck2|16 years ago|reply
[+] [-] einarvollset|16 years ago|reply