I came to say that this will probably make our fan constantly spin until I saw that this is already in the TODO list of the project.
Every client side feature such as background blur or face detection will make my (or every other person's I know) fans spin constantly so I will never use that feature.
I don't know if there's a potential software way to solve this problem or do we have to wait until everyone's using an m2 or i8 cpu?
I'm not hip to the js world. Is there a common pattern for offloading the compute heavy parts of a js workload to a server somewhere? I'd totally just leave my pc running so it could do the heavy lifting for my mobile devices if that meant better battery life.
If there's going to be waste heat, I'd rather it go into the building that I pay to heat.
A cursory glance shows the output of this project is simply bounding boxes for multiple hands, while the handpose project you mentioned is tuned for a single hand but it provides the key points for fingers and joints. I sense that handpose is probably far more useful for anyone going beyond a prototype.
Author here, this is a known issue and being worked on for handtrack.js 2.0.
The face/head errors is likely an artifact of transfer learning (handtrackjs is finetuned on the an object detection model trained using the coco object detection dataset which contains a person category).
Current approach being explored to expand the training data to include faces/heads such that the model better discriminates this from hands.
There are also size and speed optimizations being explored (smaller models such as efficientdet, quantization).
It also recognizes my ceiling as a hand when there's no hand in front of the camera. Not enough negative training data?
To satisfy the curiosity of VoidWhisperer: the score was 0.868. And it also sometimes loses track of a hand, in my case quite reliably when it's flat and seen from the side.
WebXR is a JS API that, among other things, allows you to track hands using the device's capabilities. Handtrack.js is a good alternative for browsers that are not compatible with WebXR.
I used handtrack.js a few months ago to write an in browser theremin. It's not perfect but it allowed me to write the whole thing in about three hours and only kind of blew up my 2011 mbp.
Hi, handtack.js author here,
I am exploring some size and speed optimizations which might help with this (smaller models such as efficientdet, quantization). Will share once something is ready.
How is something like this addressed in ML? In traditional software development there are tools like automated tests & static analysis to prevent your customers running into these types of embarrassing issues.
[+] [-] ofrzeta|5 years ago|reply
Every client side feature such as background blur or face detection will make my (or every other person's I know) fans spin constantly so I will never use that feature.
I don't know if there's a potential software way to solve this problem or do we have to wait until everyone's using an m2 or i8 cpu?
[+] [-] gibolt|5 years ago|reply
Although, if this is meant for prototypes, it doesn't matter if you get to validate your idea's efficacy
[+] [-] JosephRedfern|5 years ago|reply
[+] [-] __MatrixMan__|5 years ago|reply
If there's going to be waste heat, I'd rather it go into the building that I pay to heat.
[+] [-] kesor|5 years ago|reply
[+] [-] enjoylife|5 years ago|reply
[+] [-] m3sh|5 years ago|reply
[+] [-] amelius|5 years ago|reply
[+] [-] vykthur|5 years ago|reply
The face/head errors is likely an artifact of transfer learning (handtrackjs is finetuned on the an object detection model trained using the coco object detection dataset which contains a person category).
Current approach being explored to expand the training data to include faces/heads such that the model better discriminates this from hands.
There are also size and speed optimizations being explored (smaller models such as efficientdet, quantization).
[+] [-] tgv|5 years ago|reply
To satisfy the curiosity of VoidWhisperer: the score was 0.868. And it also sometimes loses track of a hand, in my case quite reliably when it's flat and seen from the side.
[+] [-] VoidWhisperer|5 years ago|reply
[+] [-] devxpy|5 years ago|reply
[+] [-] mywacaday|5 years ago|reply
[+] [-] SftwreEngnr|5 years ago|reply
[deleted]
[+] [-] rufus31415|5 years ago|reply
[+] [-] oksurewhynot|5 years ago|reply
https://github.com/msmedes/theremin
[+] [-] vykthur|5 years ago|reply
[+] [-] po1nter|5 years ago|reply
[+] [-] bathtub365|5 years ago|reply
[+] [-] villgax|5 years ago|reply
[+] [-] unknown|5 years ago|reply
[deleted]