Header logo is

Estimating Human Handshape by Feeling the Wrist

2020

Miscellaneous

hi


Hand gesture recognition has been widely studied for several applications, including sign language and touchless user interfaces. Sensing approaches for recognizing gestures range from cameras and sensorized gloves to electromyography and mechanomyography. Somewhat surprisingly, a human who places a finger on the inner wrist of another person can learn to perceive different handshapes, and in particular transitions between handshapes. Could this tactile sensing approach work for automatic gesture recognition? As proof of concept, we secured a finger-shaped biomimetic tactile sensor (SynTouch BioTac) to the palmar surface of a human wrist to gather wrist contour information. Typically used for robotic manipulation and surface characterization, this sensor outputs 19 spatially distributed finger pad deformations, DC and AC pressure, and DC and AC temperature. A user performed five gestures (the numbers 1 to 5 in American Sign Language, ASL), five times each with their dominant hand while BioTac data were collected from their wrist. We trained our model on 60% of the collected data, leaving the other 40% for testing. Using statistical features and ensembles of classifiers, we obtained a preliminary accuracy on the test set of 90%. Our short-term goals are to collect more data and classify the results considering the temporal evolution of the gestures. Our long-term goals are to more deeply investigate which sensing modalities included in the BioTac provide the most meaningful information for this application, to achieve similar results with a simpler wearable sensor, and to expand recognition to the entire range of nearly 40 ASL handshapes.

Author(s): Maria-Paola Forte and Eric M. Young and Katherine J. Kuchenbecker
Year: 2020
Month: September

Department(s): Haptic Intelligence
Bibtex Type: Miscellaneous (misc)
Paper Type: Work in Progress

Address: Leiden, The Netherlands
How Published: Work-in-progress poster presented at EuroHaptics
State: Published

BibTex

@misc{Forte20-EHWIP-Handshape,
  title = {Estimating Human Handshape by Feeling the Wrist},
  author = {Forte, Maria-Paola and Young, Eric M. and Kuchenbecker, Katherine J.},
  howpublished = {Work-in-progress poster presented at EuroHaptics},
  address = {Leiden, The Netherlands},
  month = sep,
  year = {2020},
  doi = {},
  month_numeric = {9}
}