[psuter]

We built a barely working version of that during a hackathon a while back: https://github.com/chesseye/chesseye (README has a link to a video).

It takes a lot of shortcuts, works with just the right lightning, etc., but worked great as a proof-of-concept :)

We got away with not identifying the pieces by just detecting the color, assuming the game started from the initial position, and assuming only legal moves (the whole game is unambiguous using these assumptions).

It's all old-school computer vision with hand-written features, and I'm pretty confident there is tons of low-hanging fruit, but who has the time.

[pkacprzak]

If I understand correctly (watched the video) you start from the initial position and track moves that are made in order to update the position to reflect the move, is this correct?

[psuter]

That's correct. If you're curious about the architecture: the vision part detects a chessboard, then corrects the perspective and restricts the image to just the square of the board, then looks at each square and has some simple thresholds to decide if it's occupied by a piece, and of what color. From there, the camera is treated as a black box sensor that continuously streams two 64-bit masks, for where it thinks it sees white and black pieces. There is a second program (controller) that turns that stream into a stream of chess positions (and a Unix pipe in between). The sensor is faulty of course and the controller has logic for ignoring bits from the mask where there cannot possibly be pieces etc.

[novalis78]

That’s very cool, thanks for sharing!