Tangentially, in his linked page "hall of shame" https://jon-cld.s3.amazonaws.com/test/ahall_of_fshame_SSIMUL... , he argues humans prefer row A to row C. Personally in most cases I prefer row C. It may have less color accuracy, but is often a lot sharper, which in my eyes stands out a lot more.
Did you read the sentence right after he makes that claim?
> At Cloudinary, we recently performed a large-scale subjective image quality assessment experiment, involving over 40,000 test subjects and 1.4 million scores.
This isn't about your, my or Jon Sneyer's individual opinions on these photos, it's the consensus of 40,000 test subjects. And yeah, I agree with you that for a significant number of photos I disagree with the consensus. But given that we're both posting on HN we're both also likely the kind of people who like to prove other people wrong by actively looking for and overvaluing counterexamples, so also keep that in mind. Because to be honest, for a large number of photos I also agree with the consensus.
Psychovisual image quality tests are difficult and complicated. Selection of the population, methods, guiding text, motivation, etc. can lead to strange things happening.
As an example of the difficulties an academy leading corpora TID2013 has a reversal in quality between categories 4 and 5 -- highest quality images seem to get slightly worse quality ratings than the next highest quality images.
I observed a leading image quality laboratory to produce substantially different results for the same codec when the person conducting the experiments changed.
There are hard opinions if images should be reviewed one image pixel to one monitor pixel, or if images should be zoomed like they are in practical use, particularly on mobile.
Should zooming be allowed? Should people be allowed to move closer to the monitor if they want as part of the viewing? etc. etc.
It's also comparing two overly compresses images with encoders which produce very different kind of artificats (bluriness vs. blocks of wrong colors). Which one is better is always going to be subjective (or at least application-specific). Really, as long as the model rates both as shit it is not necessarily a bad model.
vanderZwan|3 years ago
> At Cloudinary, we recently performed a large-scale subjective image quality assessment experiment, involving over 40,000 test subjects and 1.4 million scores.
This isn't about your, my or Jon Sneyer's individual opinions on these photos, it's the consensus of 40,000 test subjects. And yeah, I agree with you that for a significant number of photos I disagree with the consensus. But given that we're both posting on HN we're both also likely the kind of people who like to prove other people wrong by actively looking for and overvaluing counterexamples, so also keep that in mind. Because to be honest, for a large number of photos I also agree with the consensus.
JyrkiAlakuijala|3 years ago
As an example of the difficulties an academy leading corpora TID2013 has a reversal in quality between categories 4 and 5 -- highest quality images seem to get slightly worse quality ratings than the next highest quality images.
I observed a leading image quality laboratory to produce substantially different results for the same codec when the person conducting the experiments changed.
There are hard opinions if images should be reviewed one image pixel to one monitor pixel, or if images should be zoomed like they are in practical use, particularly on mobile.
Should zooming be allowed? Should people be allowed to move closer to the monitor if they want as part of the viewing? etc. etc.
account42|3 years ago