[casylum]

Take a look at this. It sounds exactly like what you are describing.

https://www.modalai.com/products/voxl-2-mini?variant=4424638...

Maybe this is something you could develop on instead of building your own.