Show HN: Visual Sudoku solver in the browser

[taiters_]

Hello!

I recently wanted to learn a bit about computer vision. Initially, I wanted to build something which could solve a jigsaw puzzle, but figured I should start with something (much) simpler, so I've built this instead.

This is a visual Sudoku solver which runs in the browser. It works by using OpenCV to identify and process the Sudoku grid, passing this to a simple ML model to identify the digits, and then solving the puzzle with a backtracking algorithm. The ML model was trained on the TMNIST data set using a model built with Keras, also a completely new area to me.

It's far from perfect, and doesn't like non perfectly lit or overly warped puzzles, but the main goal here was learning, which I did, a lot.

As this was primarily a learning project, I've tried to document my approach as much as possible, which can be found in this Python notebook:

https://github.com/Taiters/sudoku-solver/blob/main/notebooks...

I used Python while exploring OpenCV and training the models etc, and eventually ported this over to web (OpenCV.js + Tensorflow.js) to get something I could actually share with people.

Feel free to have a dig around the source or play around with the solver!

Github: https://github.com/Taiters/sudoku-solver

[giza182]

I think it would be super helpful to have an example puzzle there so one can quickly try it out without having to find a sudoku puzzle elsewhere

[taiters_]

Yeah that's good idea, thanks for the suggestion. I even went out and bought a puzzle book while building this, so it's definitely not something most people have around.

[rlexsu]

You can also just grab 1 sudoku from nytimes for testing (not paywalled) https://www.nytimes.com/puzzles/sudoku

[iamflimflam1]

I made a similar thing a while back - works really well.

https://www.atomic14.com/2020/07/25/browser-ar-sudoku.html

You can see it in action here: https://sudoku.cmgresearch.com/

(assuming it still works - haven't tried it for a while).

[taiters_]

I tried the blur threshold approach you mention in your post, and it's giving much better results!

Great post btw, very detailed.

[taiters_]

Nice! Yeah the puzzle detection works really well in your one. I'll need to have a read through your write up to get some pointers.

[hermannj314]

I undertook this exact project last year but stopped after writing a solver and playing with OpenCV to do edge detection. I also bought a Sudoku puzzle book for making a data set to work with.

So of course I am glad to see this and thank you for writing this up. Now I'm reinvigorated to throw some time at this project again.

As an aside, a great link I encountered during my project regarding building the fastest Sudoku Solver (conversation): https://codegolf.stackexchange.com/questions/190727/the-fast...

[Sanctor]

Here are a couple attempts trying to solve an NYT hard sudoku puzzle: https://ibb.co/gtJghBP https://ibb.co/9nvHT1P

1) The half transparent green font color is really hard to read over a white background.

2) It fails to recongize already filled in squares and tries to change them. The source is computer generated and the image is pretty clear. It seems like a best case scenario for digit recongnition, nevertheless it fails.

3) It constantly re-evaluates while I'm trying to hold the puzzle up to the camera with varying degrees of success. Too fast for me to get a screen grab when I see a successful attempt among all the failures.

Cool project though! I was excited to try it :)

[taiters_]

Thanks for the feedback!

I've recently made a change to try to improve results (Specifically the thresholding step when it's processing the image).

For me it's now finding the grid a bit more consistently, though your mileage may vary.

I've noticed the digit recognition still struggles a bit when looking at a screen though. Perhaps I need to get a bit more data for training the model.

[NKosmatos]

Hey, thanks for sharing this! I was thinking of creating a visual minesweeper solver as a web app and for sure I’m going to check your notebook ;-)

ps1. As commented, I don’t have a sudoku available to try this. ps2. Yes, it’s cheating but we’re doing it for learning.

[httpsterio]

off-topic but I thought you might find this interesting or helpful :)

generally, when adding post scriptums, p.s. notes, it's common to add a p for each following note. So, p.s., p.p.s. etc :)

[zeograd]

I love the idea but couldn't get it to work.

When I click the snapshot button from the /viewer URI, I end up on /image and can only download the photo I took or go back.

What is the intended workflow?

[taiters_]

I've uploaded a quick recording here (Seems Youtube forced it to be a "Short"): https://www.youtube.com/shorts/rdZ25pyqqmA

So ideally it should "solve" a sudoku puzzle when there's one in the frame. Then you can use the snapshot button (taking you to /image) to save and download the solution.

Though if it's not solving it (Which is definitely not uncommon in less than perfect lighting etc), I can see the workflow being unclear.

[piyushtechsavy]

I would be good if you can also allow upload of an image with camera.