top | item 9288056

(no title)

mjpt777 | 11 years ago

While one thread is in the action of rotation other producers return right away from the offer. The possibility of starvation occurs if the same thread each time it retried, that buffer has advanced to the next rotation. If adding 100 byte messages to a 128 MB buffer that is greater than a 1 in a million chance on each rotation. To have this continue then the probabilities have to be multiplied for the number of times you expect it to happen. So for ultimate starvation that gets crazy very very quickly ;-) Do you see it as more likely than that?

discuss

order

danbruc|11 years ago

Not really, all these things are pretty unlikely. But if you deploy enough installations and run them with enough load you are pretty much guaranteed to see all the issues, even if only rarely. There will be someone simply killing no longer needed threads rendering a log buffer unusable from time to time. Someone will from time to time send relatively huge messages making the starvation scenario more likely. The thread won't of course starve forever but just hitting the rotation twice will make for a pretty heavy outlier in the latency chart.

And just in case I wasn't clear about that, I am looking at all of this from a pretty theoretical point of view - I have at best a very rough idea what this will be used for in the real world and what scenarios are important and likely and what just doesn't matter. Actually maybe even the thing I just wrote is nonsense because nobody in your target audience will simply kill threads, but then again developers are notoriously good at bending and ignoring rules.

mjpt777|11 years ago

Thanks for the feedback. I'll adjust my terminology to be more precise as it is the right thing to do. Doing things in the open is a great way to learn. I'm not shy of airing my laundry :-)