I watched Wayne Coyne of the Flaming Lips do something similar with some kind of "I don't know what" controller, it was some kind of input in his microphone stand. As he moved it around, the sound and projection changed.
I remembered learning about similar MIDI controllers when I was in school.
Imogen Heap created a set of gloves that transform finger flexing and wrist movement into midi signals you can use in whatever way your performance software allows.
bigiain|4 days ago
https://mimugloves.com/gloves/
https://www.youtube.com/watch?v=vq52kT6YY-0