When I was a senior at Swarthmore College, Herb Wilf came over from U Penn to teach a course in combinatorial algorithms. I was encouraged to attend.
He claimed that choosing a subset of k integers at random from {1..n} should have a log in its complexity, because one needs to sort to detect duplicates. I realized that if one divided [1..n] into k bins, one could detect duplicates within each bin, for a linear algorithm. I chose bubble sort because the average occupancy was 1, so bubble sort gave the best constant.
I described this algorithm to him around 5pm, end of his office hours as he was facing horrendous traffic home. I looked like George Harrison post-Beatles, and probably smelled of pot smoke. Understandably, he didn't recognize a future mathematician.
Around 10pm the dorm hall phone rang, one of my professors relaying an apology for brushing me off. He got it, and credited me with many ideas in the next edition of his book.
Of course, I eventually found all of this and more in Knuth's books. I was disillusioned, imagining that adults read everything. Later I came to understand that this was unrealistic.
Love anecdotes like this! But admittedly I feel a bit lost, so please forgive my ignorance when I ask: why does choosing a subset of k integers at random require deduplication? My naive intuition is that sampling without replacement can be done in linear time (hash table to track chosen elements?). I’m probably not understanding the problem formulation here.
The article, and many of the responses, are hinting at the fact that bubblesort is an example of an anytime algorithm. This is a wide class of algorithms which provide a correct answer after some amount of time, but provide an increasingly good answer in increasing amounts of time short of the completion time. This is a super valuable property for real time systems (and many of the comments about games and animations discuss that). The paper that introduced me to the category is "Anytime Dynamic A*" [0], and I think it's both a good paper and a good algorithm to know.
Anytime algorithms are great for robotics planning, for example. A plan does not have to be perfect to be useful, especially when it can be refined further in the next timestep. And the robot cannot act out the plan instantaneously, so by the time one is close to the point where a non-ideal segment would be, one has had many timesteps to refine/optimize it. But robot could start moving right away.
I used bubblesort on purpose in a game project. Specifically, to sort sprites in an NES game back to front, lazily, spending as few CPU cycles as possible. Bubblesort on the very small list (a dozen objects max), and early exit after the first swap. It eventually completes, and that was just fine. It's tiny, incredibly simple, and somewhat resilient to the list changing from frame to frame as objects spawn and despawn. Each partial sort makes some progress no matter what.
A few other algorithms would have fit the bill just as well, but bubblesort is perfectly adequate, so that's what will likely ship. More complex algorithms end up losing out due to greater initial overhead or larger ROM size.
I've used bubblesort when simulating LEO satellite constellations, calculating which satellite is closest to a location. I used one single backwards pass of bubblesort, so O(n) every k timesteps to bring the closest to the head of the array, then every timestep just do one backwards bubblesort pass over the first few in the array. Given satellites move smoothly, if you initialize right (a few full passes at the start to get the closest few at the front) and get the constants right so a satellite outside the front few in the array can't have moved far enough to become closest without being promoted to the front few by a periodic full pass, then you always maintain the closest at the front of the array very cheaply. And this has the advantage of also being very simple to code.
On 8-bit and 32-bit microcontrollers (e.g. 8-bit AVR, 32-bit ESP8266/ESP32), Insertion sort is 6X faster than Bubble Sort on random data. I have tested this up to about N=1000.
Both Insertion sort and Bubble sort are O(N^2). Both are stable sorts. Insertion sort consumes only about 10-20 bytes more flash memory than Bubble sort. It's hard to think of situations where Bubble sort would be preferred over Insertion sort.
Shell sort is vastly superior if you can afford an extra 40-100 bytes of flash memory. (It's not too much more complicated than Insertion sort, but sometimes, we don't have 100 extra bytes.) It is O(N^k), where k ≈ 1.3 to 1.5. As soon as N ⪆ 30, Shell sort will start clobbering Insertion sort. For N ≈ 1000, Shell sort is 10X faster than Insertion sort, which in turn is 6X faster than Bubble sort. Unfortunately Shell sort is not stable.
Comb sort has a similar O(N^k) runtime complexity as Shell sort. But it seems slower than Shell sort by a constant factor. Comb sort is also not stable. I cannot think of any reason to use Comb sort over Shell sort.
Quick sort is not much faster than Shell until about N ≈ 300. Above that, the O(N*log(N) of Quick sort wins over the O(N^k) of Shell sort. But Quick sort is not stable.
Merge sort is stable and runs in O(N*log(N)), but it consumes an extra O(N) of RAM, which may be impossible on a microcontroller. You may be forced back to Insertion sort for a stable sort.
There is stable in-place merge sort, it runs in O(n*log(n)^2). It is about 3 times more complex than shell sort. I implemented it here https://github.com/thomasmueller/bau-lang/blob/main/src/test...
(most sort algos you mentioned above are in the same direcory btw)
You didn't mention heap sort. A simple implementation, which doesn't do any method calls just like shell sort (also next to the merge sort above) is about twice as complex than shell sort.
For me it's lsb radix, Yeah I know it only works on numbers, but much younger me independently invented it when slinging 3480 mainframe tape as a grave shift operator. The company had invested in mainframes early and by the time I had got there was was slightly disfunctional, they still had mainframe operators and we would run the nightly batch jobs to process orders. While they had a hard drive(the ramac) they never liked to update their programs to use it, so every major step of the process would read a tape and write a new tape(they used the tapes sort of like a massively inefficient version control, so the process could be restarted at any point) at the end of the night we would have to file a couple hundred tapes back in the library. As I hated randomly seeking through the library and was bad at ad hock sorting I put together a manual sorting routine so the numbered tapes could go back in order. What ended up working best for me I found out later was the good ol' LSB radix sort and I have a soft spot for it to this day.
I read this all the time from other people, but for me, Selection sort is the easiest to remember and implement. My next easiest would be Insertion sort.
Bubble sort doesn't click for me easily. I think it's because the terminating condition seems uglier than Selection sort or Insertion sort. I always have a little voice in the back of my mind, "Is this outer loop guaranteed to terminate?"
As others have said, it is easy enough for a child in the 80s, with only a
BASIC manual to come up with it. Been there, done that. Didn't even had a name for it.
Later I read a magazine explaining several algorithms and found the name of what I had implemented.
For the curious, the ZX Spectrum microdrive listed files on the cartridges by order found on tape. I wanted to display it in alphabetical order like the "big" computers did.
I felt this comment in my soul. I’ll never understand it: I’ve written thousands of lines of code (as a hobbyist) to solve all sorts of problems I’ve run into and yet always seem to struggle to wrap my mind around the core algorithms any real developer should be able to handle easily. This is why I’ve never pursued programming as a career.
The only use I've seen is incrementally sorting large arrays during brute-force search of said arrays, since that is approximately free and brute-force search is pretty efficient and fast on modern CPUs. Set a "sorted" flag if/when the array is eventually sorted.
The idea was that the vast majority of arrays in a large set are not searched often enough to justify the cost of sorting them and sorting is an expensive operation if you are computing on a deadline. You also don't always know which ones will be heavily searched ahead of time. Using bubblesort, only the heavily accessed arrays end up sorted but as a side-effect of search rather than having separate heuristics to decide when/what to sort.
Yeah, the article beat me to the gamedev example. Bubble sort being able to always "soft sort" on every iteration makes it the easiest to suspend and resume when you have a lot of other work to do, and when sorting is low priority.
Also, general wisdom to be mindful of data sizes and cache coherency. O(NLogN) vs. O(N^2) doesn't mean much when you're only sorting a few dozen items. Meanwhile, O(N) space can have drastic performance hitches when reallocating memory.
Not only in best case. Haven't seen this elsewhere, and know only few people who know that, so, a kind of a puzzle: what are the conditions when bubblesort is always O(n)?
Reminds me of an interview I had a while ago. The interviewer in all seriousness asked me to code up a sorting algorithm on the whiteboard. He was more of a business person than technical so was probably thinking of insertion, selection and bubblesort.
I said sure, quicksort, mergesort or radixsort?
He just said "okay, let's skip to the next question". :)
For small sets, or small-ish sets when you are coding quick, don't have a convenient standard library sort to hand, and are prioritising correctness over absolute performance.
Though in reality almost never: you almost always have a convenient built-in sort that is as quick & easy to use (likely quicker & easier), and in circumstances where the set is small enough for bubblesort to be just fine, the speed, memory use, or other properties of what-ever other sort your standard library uses aren't going to be a problem either.
As others have pointed out, sometimes it is useful for partial sorts due to the “always no less sorted than before at any point in the process (assuming no changes due to external influence)” property.
wrt:
> If you make each frame of the animation one pass of bubblesort, the particles will all move smoothly into the right positions. I couldn't find any examples in the wild,
There are hundreds of sort demos out there, both live running and on publicly hosted videos, that show the final positions by hue, getting this effect. Seems odd that they couldn't find a single one.
EDIT: actually, I can't find any of the rainbow based sort demos I was thinking of, a lot of promising links seem dead. I take back my little moan!
If you need a stable sort, can't be bothered finding a massive oversize library to link to, and only need to sort a relatively small number of objects on a system that's resource-constrained, I'm guessing?
I'm surprised that the simple, ~80 lines version of stable-in-place merge sort (see link in the above comments) is not more widely known. It is O(n log n log n) and not all that hard to implement.
When the array is almost sorted. Bubble sort complexity is linear + inversions so if the inversions are low (the more sorted the array the lower the number of inversions), bubble sort is close to a linear pass.
When sorting eigenpairs of a dense matrix, usually tou end up with a Schur decomposition. The basic operation that you can do is swap two adjacent eigenvalues on the diagonal, so bubblesort is a natural candidate.
In all your big-O analyses, remember: n = 3 more often than you think. n = 12 a lot more often than you think. If that's your case, there's nothing wrong with bubble sort unless you have very tight performance constraints.
Worse, big-O always hides a constant factor. What's bubblesort's constant? What's quicksort's? It wouldn't surprise me if, for small enough n (2 or 3, and maybe a bit higher), bubblesort is actually faster.
Note well: I have not actually benchmarked this.
Also note well: Determine what your n is; don't assume that it's either large or small.
For the downvoters, he's referring to this instance when (then) Senator Obama jokingly referenced bubble sort during this Google event: https://www.youtube.com/watch?v=koMpGeZpu4Q
It was one of the many viral moments during Obama's original campaign where he seemed cool and in touch.
A: For small arrays. I would add: particularly if you need a stable sort algorithm, which is either complex (Block Sort) or uses O(n) space (Merge Sort).
There is stable in-place merge sort [1], which is O(n*log(n)^2) and not that complex or hard to implement (about 80 lines, and that includes the ~15 lines of binarySearch, which you might need anyway).
bubble sort is sometimes used in information retrieval use cases for reranking top k based on some signals, especially specific to a user profile. I feel heap sort comes up as well, yet neither are necessarily the most efficient.
Syzygies|2 months ago
He claimed that choosing a subset of k integers at random from {1..n} should have a log in its complexity, because one needs to sort to detect duplicates. I realized that if one divided [1..n] into k bins, one could detect duplicates within each bin, for a linear algorithm. I chose bubble sort because the average occupancy was 1, so bubble sort gave the best constant.
I described this algorithm to him around 5pm, end of his office hours as he was facing horrendous traffic home. I looked like George Harrison post-Beatles, and probably smelled of pot smoke. Understandably, he didn't recognize a future mathematician.
Around 10pm the dorm hall phone rang, one of my professors relaying an apology for brushing me off. He got it, and credited me with many ideas in the next edition of his book.
Of course, I eventually found all of this and more in Knuth's books. I was disillusioned, imagining that adults read everything. Later I came to understand that this was unrealistic.
refibrillator|2 months ago
itemize123|2 months ago
addaon|2 months ago
[0] https://cdn.aaai.org/ICAPS/2005/ICAPS05-027.pdf
jononor|2 months ago
hwayne|2 months ago
amilios|2 months ago
zeta0134|2 months ago
A few other algorithms would have fit the bill just as well, but bubblesort is perfectly adequate, so that's what will likely ship. More complex algorithms end up losing out due to greater initial overhead or larger ROM size.
jeltz|2 months ago
cubefox|2 months ago
mhandley|2 months ago
bxparks|2 months ago
Both Insertion sort and Bubble sort are O(N^2). Both are stable sorts. Insertion sort consumes only about 10-20 bytes more flash memory than Bubble sort. It's hard to think of situations where Bubble sort would be preferred over Insertion sort.
Shell sort is vastly superior if you can afford an extra 40-100 bytes of flash memory. (It's not too much more complicated than Insertion sort, but sometimes, we don't have 100 extra bytes.) It is O(N^k), where k ≈ 1.3 to 1.5. As soon as N ⪆ 30, Shell sort will start clobbering Insertion sort. For N ≈ 1000, Shell sort is 10X faster than Insertion sort, which in turn is 6X faster than Bubble sort. Unfortunately Shell sort is not stable.
Comb sort has a similar O(N^k) runtime complexity as Shell sort. But it seems slower than Shell sort by a constant factor. Comb sort is also not stable. I cannot think of any reason to use Comb sort over Shell sort.
Quick sort is not much faster than Shell until about N ≈ 300. Above that, the O(N*log(N) of Quick sort wins over the O(N^k) of Shell sort. But Quick sort is not stable.
Merge sort is stable and runs in O(N*log(N)), but it consumes an extra O(N) of RAM, which may be impossible on a microcontroller. You may be forced back to Insertion sort for a stable sort.
thomasmg|2 months ago
You didn't mention heap sort. A simple implementation, which doesn't do any method calls just like shell sort (also next to the merge sort above) is about twice as complex than shell sort.
JKCalhoun|2 months ago
somat|2 months ago
bxparks|2 months ago
Bubble sort doesn't click for me easily. I think it's because the terminating condition seems uglier than Selection sort or Insertion sort. I always have a little voice in the back of my mind, "Is this outer loop guaranteed to terminate?"
hmng|2 months ago
For the curious, the ZX Spectrum microdrive listed files on the cartridges by order found on tape. I wanted to display it in alphabetical order like the "big" computers did.
ExtremisAndy|2 months ago
tzs|2 months ago
anothernewdude|2 months ago
another_twist|2 months ago
strken|2 months ago
vlovich123|2 months ago
jandrewrogers|2 months ago
The idea was that the vast majority of arrays in a large set are not searched often enough to justify the cost of sorting them and sorting is an expensive operation if you are computing on a deadline. You also don't always know which ones will be heavily searched ahead of time. Using bubblesort, only the heavily accessed arrays end up sorted but as a side-effect of search rather than having separate heuristics to decide when/what to sort.
johnnyanmac|2 months ago
Also, general wisdom to be mindful of data sizes and cache coherency. O(NLogN) vs. O(N^2) doesn't mean much when you're only sorting a few dozen items. Meanwhile, O(N) space can have drastic performance hitches when reallocating memory.
nick__m|2 months ago
pieter3d|2 months ago
observationist|2 months ago
jaw0|2 months ago
and every new hire got taken to the whiteboard to learn about sort algorithm performance: bubblesort is O(n) in the best case.
and in our codebase, the data being sorted fit that best case (the data was already sorted or almost sorted).
avmich|2 months ago
Findecanor|2 months ago
sureglymop|2 months ago
I said sure, quicksort, mergesort or radixsort?
He just said "okay, let's skip to the next question". :)
zitterbewegung|2 months ago
dspillett|2 months ago
Though in reality almost never: you almost always have a convenient built-in sort that is as quick & easy to use (likely quicker & easier), and in circumstances where the set is small enough for bubblesort to be just fine, the speed, memory use, or other properties of what-ever other sort your standard library uses aren't going to be a problem either.
As others have pointed out, sometimes it is useful for partial sorts due to the “always no less sorted than before at any point in the process (assuming no changes due to external influence)” property.
wrt:
> If you make each frame of the animation one pass of bubblesort, the particles will all move smoothly into the right positions. I couldn't find any examples in the wild,
There are hundreds of sort demos out there, both live running and on publicly hosted videos, that show the final positions by hue, getting this effect. Seems odd that they couldn't find a single one.
EDIT: actually, I can't find any of the rainbow based sort demos I was thinking of, a lot of promising links seem dead. I take back my little moan!
unknown|2 months ago
[deleted]
aappleby|2 months ago
ErroneousBosh|2 months ago
thomasmg|2 months ago
another_twist|2 months ago
CyLith|2 months ago
AnimalMuppet|2 months ago
Worse, big-O always hides a constant factor. What's bubblesort's constant? What's quicksort's? It wouldn't surprise me if, for small enough n (2 or 3, and maybe a bit higher), bubblesort is actually faster.
Note well: I have not actually benchmarked this.
Also note well: Determine what your n is; don't assume that it's either large or small.
caycep|2 months ago
sdsd|2 months ago
It was one of the many viral moments during Obama's original campaign where he seemed cool and in touch.
beeforpork|2 months ago
thomasmg|2 months ago
[1] https://github.com/thomasmueller/bau-lang/blob/main/src/test...
kazinator|2 months ago
https://en.wikipedia.org/wiki/Shellsort
Shellsort can be regarded as an improvement over either Bubble Sort or Insertion Sort.
13415|2 months ago
opensourcemaxi|2 months ago
pilord314|2 months ago
You can also do something like a calendar queue with bubble sort for each bin.
LorenPechtel|2 months ago
And while I've never hit a case I would think it would have merit with data known to be pretty close to properly sorted.
lucraft|2 months ago
When I was playing The Farmer Was Replaced and needed to implement sorting, I just wrote a bubble sort. Worked first time.
unknown|2 months ago
[deleted]
pestatije|2 months ago
whateveracct|2 months ago
JSR_FDED|2 months ago
jhallenworld|2 months ago
6Roman6|2 months ago
[deleted]