Control Linux based distros using hand gestures using OpenCV, GTK, Mediapipe(github.com)
github.com
Control Linux based distros using hand gestures using OpenCV, GTK, Mediapipe
https://github.com/flying-pizza-69/GestureX
9 comments
In the screenshot they’ve mapped fist to “shutdown now” which seems to either indicate high confidence in the gesture recognition capabilities, or a good sense of humor about the whole thing.
Some people want to live life on the edge. Expert mode would be a gesture that nukes your installation
Suggestion: middle finger mapped to "run last command with sudo".
For those who don't know, this works in many shells:
sudo !!I would prefer looking up towards the ceiling combined with a deep breath, followed by muttering a four-letter word.
Are you by chance interested in a command named after the four-letter word, which, when invoked, automatically fixes typos in the last run command and reruns it: https://github.com/nvbn/thefuck
Starting in the early aughts and extending into the release of the Wiimote there were a handful of projects like this, largely for Mac. Never amounted to much more than a proof of concept most of the time but I always found the concept exciting. I had a lot of hope for Magic Leaps initial stand-alone hand tracking device before they pivoted to AR
I love this, I can't tell if it's really true or not
I feel like a few gestures to lock or switch workspaces would be really ideal.
I feel like a few gestures to lock or switch workspaces would be really ideal.
I worked on something very similar to this a couple years ago. My main issue with it why I didn't go beyond the PoC stage was that raising my hands up to the camera soon became a pain (I use a laptop), esp as I have it at eye level. It also had a hard time keeping up with my motions, so most of the time it was getting a blur.
I had a overhead-ish camera on a stick, that flopped up from the laptop screen back, to point straight down at the keyboard. Alternately, a phone clip-on fish-eye lens was low-res but ok, iirc. Kludgy little mirrors were iffyer, but there's a commercial one for iPad I didn't try.
Blur wise, perhaps it was 30 fps? I was mostly ok with 60 fps. Would have liked 4k resolution for markers and precision though.
More problematic was integrating high-latency "wait for the camera frame and then for mediapipe" with low-latency "fast keypress response". Drove me towards replayable event streams, and thus custom apps. But I was using it for custom XR-ish apps anyway.
Running high-res mediapipe etal on multiple cameras kept a thinkpad doorstop laptop warm even before starting XR. Keyboard feel got a bit toasted. But TFA doing gestures-only fixed-poses low-res single-camera is far lighter.
Blur wise, perhaps it was 30 fps? I was mostly ok with 60 fps. Would have liked 4k resolution for markers and precision though.
More problematic was integrating high-latency "wait for the camera frame and then for mediapipe" with low-latency "fast keypress response". Drove me towards replayable event streams, and thus custom apps. But I was using it for custom XR-ish apps anyway.
Running high-res mediapipe etal on multiple cameras kept a thinkpad doorstop laptop warm even before starting XR. Keyboard feel got a bit toasted. But TFA doing gestures-only fixed-poses low-res single-camera is far lighter.
It's a pretty old system; had it over 10 years now, so I didn't expect too much re the blur. I'll likely revisit after I've upgraded to something more modern. Maybe by then I'll have the camera position figured as well.
Love to get into the XR space myself.
Love to get into the XR space myself.
What is, exactly, a "smile hand gesture"?
By comparing the list to the icons, it would be what I would call a "Finger Gun[1]"
1: https://en.wikipedia.org/wiki/Finger_gun
1: https://en.wikipedia.org/wiki/Finger_gun
similar but different, I've used MediaPipe to create Eyeroll, a WebExtension that lets you scroll up and down webpages by blinking green your eyes.
https://github.com/bjesus/eyeroll/
https://github.com/bjesus/eyeroll/
What? Not using LLMs or an expensive cloud based API?
> This project utilizes the hand gesture recognition model provided by the Google MediaPipe project.
The rest is just some glue.
The rest is just some glue.
Quite pleasant, eh? I hope we see more projects without them.