Instead of trying to smooth the entire image, it reduces blocking artifacts by finding plausible coefficients that reduce discontinuities at block boundaries (jpegs are encoded as a series of 8x8 blocks). "jpeg2png gives best results for pictures that should never be saved as JPEG", but knusperli works well on normal photographic content, since it only tries to remove blocking artifacts.
> A JPEG encoder quantizes DCT coefficients by rounding coefficients to the nearest multiple of the elements of the quantization matrix. For every coefficient, there is an interval of values that would round to the same multiple. A traditional decoder uses the center of this interval to reconstruct the image. Knusperli instead chooses the value in the interval that reduces discontinuities at block boundaries. The coefficients that Knusperli uses, would have rounded to the same values that are stored in the JPEG image.
> Instead of trying to smooth the entire image, it reduces blocking artifacts by finding plausible coefficients that reduce discontinuities at block boundaries
This is essentially the same thing, but using slightly different regularizers. In both cases, an image is found that whose jpeg compression is identical to the given one, by selecting the "most regular" image among the large family of such images. Jpeg2png minimizes the total variation over the whole image, and knusperly only along the boundaries of the blocks. The effect is quite similar at the end.
I was going to mention Knusperli as well. It's worth mentioning that the "no more artifacts" claim in the title depends heavily on what you understand to be an artifact. With modern, good quality JPEG encoders (e.g. mozjpeg), blocking only appears when the file is heavily compressed, or after a lot of generation loss. A decoder like this can't recover information that isn't there, obviously. So at more moderate levels of compression, where there's no noticeable blocking but the information inside the blocks is distorted or missing, it's not likely to help you a lot.
(As a citation, I can only offer the fact that I've frequently tried to clean up JPEG images I've found online, and Knusperli is usually my first step. It's never a sufficient step, though.)
That reminds me of a quote about chlorine trifluoride:
"...It can be kept in some of the ordinary structural metals—steel, copper, aluminum, etc.—because of the formation of a thin film of insoluble metal fluoride that protects the bulk of the metal ... If, however, this coat is melted or scrubbed off, and has no chance to reform, the operator is confronted with the problem of coping with a metal-fluorine fire. For dealing with this situation, I have always recommended a good pair of running shoes."
This is a poor-mans version of the deblocking filter which you find in recent video standards, but jpeg was to early to get it.
The advantage of those over this is that the encoder knows that the decoder is going to run the filter, and it can use that in its perceptual model to choose an encoding which looks most like the original after deblocking. Video codecs also use the deblocked version as reference frames. So you see these artifacts in mpeg-2 video and jpeg, but not more recent ones.
I'd call newer formats to have poor-man's deblocking, because they literally apply blur to blocky images, instead of reconstructing an optimal gradient.
In newer codecs you can notice that sometimes blocks of 8 or 16 pixels get blurred with a 4px-wide blur, which is barely enough to cover up the edges, but not enough to mask square-ish shapes underneath.
And one day, a law enforcement officer will use it on a bad quality JPEG image and one of the innocent persons whose photos were used to train the NN will be arrested.
This strikes me as conceptually similar to the problem of printing floating point numbers. A naive algorithm will convert the binary floating point to decimal and print that out, with potentially many trailing digits that are more or less garbage. The more human-friendly algorithms figure out the shortest/simplest decimal that would be rounded to the same binary floating point quantity, and prints that instead.
You and the rest of this thread's participants may be interested to know that the problem was solved a while ago; you want the "Dragon4" algorithm from "How to Print Floating-Point Numbers Accurately" [0]. Indeed, the problem is difficult, and having implemented the algorithm, I can confirm that it has earned its reputation. Numeric algorithms are hard and the only reward is accuracy.
"jpeg2png gives best results for pictures that should never be saved as JPEG. Examples are charts, logo's, and cartoon-style digital drawings."
Interesting value proposition.
There may be equivalents for other misused file types/formats. The obvious one is the inverse case of photos saved as GIFs, but I imagine audio and text (eg. PDF malapropisms?) are possible too.
I just tried it with a text heavy screenshot, a common scenario where the JPEG encoding produces a poor result with lots of speckle around each letter.
The result is very good. The speckling is effectively eliminated without smearing the letters.
I d say the issue is the prpgram here, a compressor program might have heuristic to detect png-favored images vs photos. I cannot imagine we couldn't detect the easy cases reliably (because I'd say often images are more in a category or another). Crop the image to 64x64, try png and if the result is sufficiently small continue, else convert to jpg ?
This is really cool, but in other comments I'm reading that good JPEG encoders and decoders depend on knowing the other.
That you can design a better JPEG encoder for a known reference decoder (which decoded version best matches the original), and you can likewise design a better JPEG decoder for a known reference encoder.
But now that there are both encoders and decoders trying to be more efficient and non-blocky respectively... it feels like a mess. Even more so because a decoder never knows which encoder was used, and an encoder never knows which decoder.
Is there any solution to this mess? Is there a best practice here? Since most JPEG's are probably viewed on the web, do all browsers share the same decoder, and so effort is best spent on the encoder? Or even while encoder improvements are made specifically for the algorithms browsers use, can improvements to decoders still be made that remain improvements, instead of e.g. "overfitting" and actually making things worse?
This just feels like a bit of an "improvements race" on both sides which leaves me a little dizzy in trying to understand it...
The obvious improvement is to use a better format than JPEG. Maybe something that can include information about the encoding process so the decoder can use that instead of guessing. Or maybe even a non-lossy format so we can focus more on data compression and less on visual compression.
For the first image I much prefer the jpeg decoded image to the de-noised one. The de-noised one just looks like it was sent through a low-pass/median filter and loses contrast, detail, and texture. If you flip between the full-sized images the jpeg2png looks like vaseline on the lens.
The naming is also odd. I don't imagine it would do well with text (which are the sorts of images I associate with png) that was jpeg encoded then decoded with it.
Yes, this is definitely an interesting question. Earlier this year I implemented a version of trellis quantization for some compression experiments that I've been tinkering with in my spare time. My code considers more possibilities than just rounding to the nearest quantization step when it deems that the bit rate savings in after entropy encoding may be worth the additional image loss (e.g., it might even completely zero a coefficient). That would violate this decoder's assumption that the original DCT coefficient must have been within a limited range around the quantization step.
I know that mozjpeg [1] features trellis quantization for JPEG encoding. I wonder how this decoder would do with that?
[+] [-] Scaevolus|5 years ago|reply
Instead of trying to smooth the entire image, it reduces blocking artifacts by finding plausible coefficients that reduce discontinuities at block boundaries (jpegs are encoded as a series of 8x8 blocks). "jpeg2png gives best results for pictures that should never be saved as JPEG", but knusperli works well on normal photographic content, since it only tries to remove blocking artifacts.
> A JPEG encoder quantizes DCT coefficients by rounding coefficients to the nearest multiple of the elements of the quantization matrix. For every coefficient, there is an interval of values that would round to the same multiple. A traditional decoder uses the center of this interval to reconstruct the image. Knusperli instead chooses the value in the interval that reduces discontinuities at block boundaries. The coefficients that Knusperli uses, would have rounded to the same values that are stored in the JPEG image.
[+] [-] enriquto|5 years ago|reply
This is essentially the same thing, but using slightly different regularizers. In both cases, an image is found that whose jpeg compression is identical to the given one, by selecting the "most regular" image among the large family of such images. Jpeg2png minimizes the total variation over the whole image, and knusperly only along the boundaries of the blocks. The effect is quite similar at the end.
[+] [-] bscphil|5 years ago|reply
(As a citation, I can only offer the fact that I've frequently tried to clean up JPEG images I've found online, and Knusperli is usually my first step. It's never a sufficient step, though.)
[+] [-] locofocos|5 years ago|reply
That's the funniest thing I've read in a while.
[+] [-] Lanrei|5 years ago|reply
"...It can be kept in some of the ordinary structural metals—steel, copper, aluminum, etc.—because of the formation of a thin film of insoluble metal fluoride that protects the bulk of the metal ... If, however, this coat is melted or scrubbed off, and has no chance to reform, the operator is confronted with the problem of coping with a metal-fluorine fire. For dealing with this situation, I have always recommended a good pair of running shoes."
[+] [-] ajb|5 years ago|reply
The advantage of those over this is that the encoder knows that the decoder is going to run the filter, and it can use that in its perceptual model to choose an encoding which looks most like the original after deblocking. Video codecs also use the deblocked version as reference frames. So you see these artifacts in mpeg-2 video and jpeg, but not more recent ones.
[+] [-] pornel|5 years ago|reply
In newer codecs you can notice that sometimes blocks of 8 or 16 pixels get blurred with a 4px-wide blur, which is barely enough to cover up the edges, but not enough to mask square-ish shapes underneath.
[+] [-] 0-_-0|5 years ago|reply
[1] Implemented for a custom compressor: https://hific.github.io/
[+] [-] lazyjones|5 years ago|reply
[+] [-] leeoniya|5 years ago|reply
https://github.com/TianZerL/Anime4KCPP
[+] [-] IgorPartola|5 years ago|reply
[+] [-] wtallis|5 years ago|reply
[+] [-] Kednicma|5 years ago|reply
[0] https://lists.nongnu.org/archive/html/gcl-devel/2012-10/pdfk... or use your search engine
[+] [-] jandrese|5 years ago|reply
[+] [-] webmaven|5 years ago|reply
Interesting value proposition.
There may be equivalents for other misused file types/formats. The obvious one is the inverse case of photos saved as GIFs, but I imagine audio and text (eg. PDF malapropisms?) are possible too.
[+] [-] jandrese|5 years ago|reply
The result is very good. The speckling is effectively eliminated without smearing the letters.
JPEG Encoded, 25% quality: http://jubei.ceyah.org/overcompressed/crop-jpg.png
JPEG2PNG: http://jubei.ceyah.org/overcompressed/crop-converted.png
[+] [-] makapuf|5 years ago|reply
[+] [-] dmitriid|5 years ago|reply
[+] [-] crazygringo|5 years ago|reply
That you can design a better JPEG encoder for a known reference decoder (which decoded version best matches the original), and you can likewise design a better JPEG decoder for a known reference encoder.
But now that there are both encoders and decoders trying to be more efficient and non-blocky respectively... it feels like a mess. Even more so because a decoder never knows which encoder was used, and an encoder never knows which decoder.
Is there any solution to this mess? Is there a best practice here? Since most JPEG's are probably viewed on the web, do all browsers share the same decoder, and so effort is best spent on the encoder? Or even while encoder improvements are made specifically for the algorithms browsers use, can improvements to decoders still be made that remain improvements, instead of e.g. "overfitting" and actually making things worse?
This just feels like a bit of an "improvements race" on both sides which leaves me a little dizzy in trying to understand it...
[+] [-] IgorPartola|5 years ago|reply
[+] [-] ris|5 years ago|reply
[+] [-] karmakaze|5 years ago|reply
The naming is also odd. I don't imagine it would do well with text (which are the sorts of images I associate with png) that was jpeg encoded then decoded with it.
[+] [-] KarlKemp|5 years ago|reply
[+] [-] etaioinshrdlu|5 years ago|reply
[+] [-] a_e_k|5 years ago|reply
I know that mozjpeg [1] features trellis quantization for JPEG encoding. I wonder how this decoder would do with that?
[1] https://github.com/mozilla/mozjpeg
[+] [-] ur-whale|5 years ago|reply
Denoiser in ray tracing are quite a success, and this strikes me as quite similar.
In fact, now that I think of it, you probably don't even need a GANN, since you can create an infinite training set for quasi-free.
[+] [-] ache7|5 years ago|reply
[+] [-] pents90|5 years ago|reply
[+] [-] crazygringo|5 years ago|reply
Things like the canonical test image being a sexy "come hither" look, coming from a Playboy centerfold, definitely don't help.
And it's not enough to say "well the rest of the centerfold isn't shown". Come on. We all know the source of the image.
And "tradition" is far less important than CS being a welcoming environment for all people.
Computer science is a profession. Let's use images that are appropriate for a professional setting.
[+] [-] enriquto|5 years ago|reply
[+] [-] unknown|5 years ago|reply
[deleted]
[+] [-] rotskoff|5 years ago|reply
[+] [-] pkulak|5 years ago|reply
[+] [-] unknown|5 years ago|reply
[deleted]
[+] [-] FriendlyNormie|5 years ago|reply
[deleted]