It's very refreshing to see here that "attack" is not used in the way that one might expect from just the headline, meaning "a possibly unwarranted criticism that I didn't like or found unfair, or that I am taking personally".
I am endlessly impressed with how antirez responds to any critique of Redis that I've ever seen. He's always taken it as a positive, and looked for the truth in the critique, rather than searching for something to be wrong and try to discredit the critique.
My opinion of him and the Redis project increases further every time.
Really? I hadn't seen the original posts before clicking on the this one and I assumed this was some kind of security breach...I hadn't heard of Aphyr before but just assumed it was some kind of netsec (white or black hat) group. I actually skimmed the OP's first paragraphs several times because I didn't understand what was going on.
That said, I agree that DB reliability should be taken with the same rigor as net security...but I was kind of under the impression that it already was, in that DBs are pretty serious business. Also, "attack" has the connotation of, well, an "attack"...here, some of the failures happen in regular business operations, which is a problem different from when the system is under "attack".
But at least the OP took the criticism graciously. When I read what the case actually was, I then worried that the OP was having a bunker mentality.
Yes, you can recover from lost acknowledgements by asking for the transaction ID from postgres before committing--or by making up your own flake ID and writing it to a table. Given a queue with at-least-once delivery (which includes, say, durable storage on the client), you can check for the presence of that ID at a later time and re-apply the transaction to recover from network errors safely.
The transaction ID does wrap around, so there's a time limit depending on your transaction throughput. You can also ask for certain transactional properties on rows, though this won't allow you to recover in all (most?) cases.
Yet more tangentially related: it is an instance of the Byzantine Agreement problem, which is unsolvable in general: no finite protocol guarantees consistent state in the presence of packet loss.
Redis is one of those things I both love and love to hate.
I've had good results using Redis as a lock server, but I live in (perhaps misplaced) fear of a client hanging or crashing leaving a lock stranded. Not that this is really Redis's problem.
Hello, you can easily mount a lock that auto-releases itself after some timeout using the new (2.6.13) extended SET command (see http://redis.io/commands/set) or simply a Lua script.
hmm, i'm not sure how it could be better worded, but since antirez already links to this, i had thought you were posting a response to antirez's comments
I'm frustrated that when the HN editors deduped the original story, they apparently deleted ALL the instances, leaving only this one. I wanted to read the discussion on the subject of Aphyr's research, not Antirez' response.
It looks bad, HN. We all know that VMWare is litigious as (try looking up benchmarks sometime.) But to (presumably) cave so quickly and effortlessly suggests... well, I'm not sure.
The other possibility is that Aphyr yanked them himself, probably under duress (or else there'd just be an 'update' at the bottom of the research's page.) Aphyr, is this what happened? I figure you probably can't talk freely if so, but say something.
1) I no longer work for VMware, but Pivotal. Redis is open source and copyright is of the original guys that wrote the code: I, Pieter Noordhuis, other contributors.
2) I posted the link to the original article in the first very lines of my reply. Actually thanks to my reply the exposure the Aphyr research had about Redis is the greatest, compared to the other data stores mentioned. I publicly said thank you to Aphyr on Twitter, and posted its blog post.
It is extremely unlikely that any pressure was put on the HN admins by VMWare or anyone else to get stories scrubbed. It's almost as unlikely that VMWare gives a shit about stories about Redis.
brown9-2|12 years ago
Legion|12 years ago
My opinion of him and the Redis project increases further every time.
danso|12 years ago
That said, I agree that DB reliability should be taken with the same rigor as net security...but I was kind of under the impression that it already was, in that DBs are pretty serious business. Also, "attack" has the connotation of, well, an "attack"...here, some of the failures happen in regular business operations, which is a problem different from when the system is under "attack".
But at least the OP took the criticism graciously. When I read what the case actually was, I then worried that the OP was having a bunker mentality.
pygy_|12 years ago
In the PostgreSQL evaluation[0], Aphyr noticed that, if a packet confirming a transaction is dropped, the client ends up in a deadlock.
Does PostgreSQL keep a record of the past transactions, and their success or failure. If so, is it possible to query it?
[0] http://aphyr.com/posts/282-call-me-maybe-postgres
aphyr|12 years ago
The transaction ID does wrap around, so there's a time limit depending on your transaction throughput. You can also ask for certain transactional properties on rows, though this won't allow you to recover in all (most?) cases.
praptak|12 years ago
Glyptodon|12 years ago
I've had good results using Redis as a lock server, but I live in (perhaps misplaced) fear of a client hanging or crashing leaving a lock stranded. Not that this is really Redis's problem.
antirez|12 years ago
jamwt|12 years ago
https://github.com/jamwt/dreadlock
It will release the lock when the client dies (disclaimer: I wrote it).
Or you can go whole hog and use zookeeper + ephemeral nodes. More robust but quite a bit more complex.
nutmeg|12 years ago
keeran|12 years ago
http://en.wikipedia.org/wiki/CAP_theorem
lacksconfidence|12 years ago
undoware|12 years ago
It looks bad, HN. We all know that VMWare is litigious as (try looking up benchmarks sometime.) But to (presumably) cave so quickly and effortlessly suggests... well, I'm not sure.
The other possibility is that Aphyr yanked them himself, probably under duress (or else there'd just be an 'update' at the bottom of the research's page.) Aphyr, is this what happened? I figure you probably can't talk freely if so, but say something.
antirez|12 years ago
1) I no longer work for VMware, but Pivotal. Redis is open source and copyright is of the original guys that wrote the code: I, Pieter Noordhuis, other contributors.
2) I posted the link to the original article in the first very lines of my reply. Actually thanks to my reply the exposure the Aphyr research had about Redis is the greatest, compared to the other data stores mentioned. I publicly said thank you to Aphyr on Twitter, and posted its blog post.
So I really don't understand your theories here.
aphyr|12 years ago
HN stories on my original posts are still there, as far as I can tell. They just never hit frontpage.
tptacek|12 years ago
unknown|12 years ago
[deleted]
JulianMorrison|12 years ago
contingencies|12 years ago
aphyr|12 years ago
http://aphyr.com/posts/287-asynchronous-replication-with-fai...