Anime2Sketch: A sketch extractor for illustration, anime art, manga

[CobrastanJorji]

I think I'm missing something. I get the Sketch-to-Photo synthesis that this is based on. It's really weird, neat stuff. But as a layman, I'm having trouble seeing the difference between the result of this anime-to-sketch synthesis and what I'd expect to get out of a simple edge detection. Is the difference that it's more clever about which details to ignore?

[fsloth]

I only dabble in graphics - but generallly simple edge detection needs really uniform tonality and no textures in the input to work well. Look for example in the more "sketchy" examples how the linework that "looks right" is extracted from a quite noisy input. Also, in the top example where there are houses, the contrast difference that gets extracted to linework is lower than in the character areas.

So, the flat shaded images with explicit black outlines - yes, it's likely there is much difference with edge detection. But when the image has lots of different contrasts and tonalities this looks much more impressive.

[mattigames]

Not actually, here is what most common edge detection methods produce (last one is Anime2Sketch): https://i.imgur.com/nt0D1ef.png

[greatgoat420]

I was actually going to ask if someone had done a comparison with edge detection.

[wj]

This looks like a great tool to generate some Pokémon and Beyblade coloring pages for my kids. We went through everything in Google image results many moons ago.

[swsieber]

I really want to see how this performs on Octonauts stills

[fireattack]

Just a heads up, you should use higher quality (or better, just use PNG) for the output.

The default Image.save quality is very low to a point that the JPEG artifact is more prominent than the line art themselves.

L91 @ data.py: image_pil.save(image_path, format='PNG')

[forgotpwd16]

Then someone can use https://github.com/taivu1998/GANime to recolor them.

[slazaro]

Do it iteratively one after the other to see if after a while the results are unrecognizable from the originals. Like those experiments that translated a text between languages to create gibberish.

[gibolt]

This feels like a tool with lots of business cases.

Studios may be able to accelerate digitalization and colorization.

The ability to convert stills to a fillable outline or repurchase for labels/marketing/branded coloring books (or apps) could be worth some money to those with a large content library.

[Cloudef]

Looks more like shaded art to unshaded lineart rather than sketch. Sketches are usually way more messy, like a blueprint for the final product.

[dagmx]

This is actually pretty impressive, and I can see it being really useful if it can generate clean line art from Animation roughs.

It would be really interesting to see this in OpenToonz or the like.

[0x426577617265]

Semi off-topic – is there a tool to turn a picture into an drawing? I sometimes see websites where people have created a avatar of their headshot that looks ‘toonish’.

[knicholes]

I believe you're looking for something like https://github.com/taki0112/UGATIT.

[camillovisini]

How WSJ stipple drawings are made: https://youtu.be/sZzP9PQJXLs

[swframe2]

Also checkout u-2-net: https://github.com/xuebinqin/U-2-Net There is a variant that can turn images into line drawings.

[throw_m239339]

Interesting. I wonder how it fares with 3D renderings? I'm a Blender user and unfortunately, Blender "Toon Shading" capabilities are not very good compared to say Cinema 4D.

[jp555]

There are lots of very good techniques to do toon shading in Blender. Here's a few:

BEER NPR (MALT renderer) - https://blendernpr.org/beer/

Lightning Boy shader 2.0 - https://www.youtube.com/watch?v=8fHZcnTFYEI

Procedural hatching and manga shaders - https://gumroad.com/l/tcKOI

[mushufasa]

How is this technically different from photoshop filters? https://design.tutsplus.com/tutorials/sketch-photoshop-effec...

[asutekku]

Photoshop just detects edges and does thus ends up detecting both sides of a drawn line or change in shading for example. This does not appear to show such artefacts.

[karmasimida]

The project is much cleaner, and from the link you posted, the picture is noisy.

[pjgalbraith]

I wonder if this can be used for comic book inking. It looks like they have an example of that.

Typically the workflow is pencil drawing -> cleaned up ink drawing (japanese animation uses a similar process too). If this can speed up that process it could save a lot of time.

[ekianjo]

Does not work as well as advertised :) I think the author clearly cherry picked their examples.

[mkesper]

Can you provide some counter-examples, probably as issues in the repo?

[jakearmitage]

Does anyone know a similar model that transforms normal images into Western Comic Book style? I've seen it a lot for Anime/Manga, but never for that classic style of 90's comic books.

[ZephyrBlu]

I'm not super familiar with Deep Learning, but based on the fact this is effectively extracting edges and the ConvTranspose2d layers I'm guessing it's some sort of Convolutional Neural Net?

[A4ET8a8uTh0]

It is pretty neat. Off-topic, what anime is that test from?

[bla15e]

Vinland Saga

https://youtu.be/7U7BDn-gU18

[androng]

What could we use this for? The immediate thing that comes to mind is making a coloring book. I’m wondering if I could use it to make something original

[zakki]

If I want to use this program do I have to have a good GPU in my computer to run this program or I just need to install the required software?

[lostgame]

I believe that this will not be too GPU-intensive, but that will of course depend on the input resolution of the video.

[knicholes]

The training is what requires a good GPU. For inference, a CPU should be fine.

[offtop5]

Is there any way for someone to post a Google collab notebook with this.

I think this would be pretty cool if it would support any picture or video.

[istorical]

can anyone show what happens if you feed it regular video / photo?

[chias]

Nothing very good.

https://i.imgur.com/LfZI0QD.png