Show HN: Offline sketch to image geneartor in a whiteboard

[arctanx]

I'm excited to share my indie product, https://drawing.pics

In this post, I'll share all the technical details behind it.

So, what exactly is DrawingPics? It's an image-to-image and sketch-to-image generator that runs in a whiteboard, allowing you to draw your draft directly on the canvas and easily generate the desired image.

How does it work from a technical perspective? TLDR: Miniconda + Diffusers + Electron + Excalidraw

One of the most important aspects of creating this tool was finding a stable diffusion locally inference solution. There are many solutions available, such as https://github.com/leejet/stable-diffusion.cpp and https://github.com/apple/ml-stable-diffusion. I tested the C++ version, but the inference speed was very slow, and Metal GPU support still had issues (you can find relevant issues in their repo). Ultimately, I decided to use python to run it because PyTorch is mature and MPS support is well-established. And I chose Miniconda, because it can create a small, isolated Python environment to run the program.

The AI model should run in the background so that we can continuously produce images while you draw. We need to find an RPC method to enable communication between the Python process and Electron's Node.js process. The easiest way is to run a Python HTTP server, but the memory usage is too high. We need a more lightweight solution, so I used xmlrpc for memory efficiency, although there might be better alternatives that I'm unaware of. The AI inference part is handled by diffusers, which is great, but I had to apply some custom patches to make it work in this situation. This can be a bit challenging if you're not familiar with Python.

For the frontend, I initially used a low-level canvas library and tried to implement a drawing pad from scratch. However, it had too many details, so I chose a more mature option: Excalidraw. It's fantastic, with the only shortcoming being limited API support.

Finally, I combined all these technologies in Electron, ensuring they work smoothly on both the main process and the renderer process.

Ok, Is DrawingPics free to use? 80% of the project is free to use. I only charge a lifetime license for the effort I've put in, as I've spent 4 months on it and made numerous improvements based on user feedback. Currently, only Mac is supported. Windows support will be added later.

[xnx]

I here a lot of people really like Stable Diffusion for Krita (https://github.com/Acly/krita-ai-diffusion). It's pretty easy to set up, powerful, local, and totally free.

[arctanx]

Krita is a great drawing app, but the Krita SD plugin is not easy to set up.

I think DrawingPics installation is more friendly, I wrote a blog about it https://docs.drawing.pics/index/getting-started/drawingpics-...

[phlsa]

Unfortunately, the model download keeps failing for me. That being said, it looks promising and I love how simple the setup looks.

[arctanx]

Sorry for that, could you please provide screenshots or description in here: https://drawingpics.canny.io/feedback So that I can investigate the issue

[ffitch]

I wish someone came up with a rough sketch to neat pretty sketch generator, that’d be more practical for whiteboarding.

[3abiton]

I think you can do that with Krita + SD (with specific sketch model?)

[arctanx]

This idea is really cool, I'll try :)

[michaelmior]

> The easiest way is to run a Python HTTP server, but the memory usage is too high. We need a more lightweight solution, so I used xmlrpc for memory efficiency

Doesn't XML-RPC operate over HTTP?

[arctanx]

It does, but I used Flask previously for comparison, so I came to that conclusion.

[artur_makly]

Congrats on the launch.. this will inspire more young budding artists (kids especially) .. will email you from @visualsitemaps.com for more ideas and questions.

[mgkimsal]

Minor typo: "Check You Email" should be "Check Your Email". Congrats on launch.

[arctanx]

Thx, fixed now.

[acheong08]

I’m confused. How is this different from ControlNets?

[arctanx]

The Controlnets model is too large(maybe 2GB? extra download), which means it requires more memory. The line control model that I used in drawingpics is about 300MB, which is suitable for light users

[playingalong]

The pricing link in hamburger doesn't open for me.

Chrome Android

[endofreach]

"Write reply to this HN post"-prompt fail?