Fingerspelling Sign Language Using a Hand Model

Initializing live version
Download to Desktop

Requires a Wolfram Notebook System

Interact on desktop, mobile and cloud with the free Wolfram Player or other Wolfram Language products.

This Demonstration shows how to translate a line of text into American Manual Alphabet fingerspelling using a simple 3D animated hand model. Select the message to translate from the pull-down menu, then play the animation. Increasing the number of frames per pose and slowing down the time playback gives a smoother animation.

Contributed by: Stewart Dickson and Shawna Martell (August 2012)
Open content licensed under CC BY-NC-SA


Snapshots


Details

To translate a line of text into three-dimensional time-varying language, this animated hand model was imported from an X3D file as XML. We derived the dimensions of the hand model from a combination of Defense Advanced Research Projects Agency (DARPA)-sponsored Digital Human published research as well as our own anthropometric study. We employ a method of traversing the X3D XML Document-Object Model (DOM) tree that is very general purpose. It is identical to traversing an animation scene graph in a high-performance 3D animation package.

Note, however, that XML in itself is intrinsically dangerous. For example, the Exponential Entity Expansion is a block of XML code that is both well formed and valid according to the rules of an XML schema, but which when executed recursively expands to overflow memory on the XML parser. We circumvent these difficulties here by pre-importing X3D externally from this notebook, using the necessary Read[ ] or ToExpression[ ] to convert XML string-valued fields, which have similar vulnerabilities for Mathematica.

This version imported 27 hand gesture poses, one for each letter of the alphabet plus a "neutral" hand pose. Double letters are separated by a "bounce" pose built on the double-letter pose. The poses were built in another notebook, which contained 23 sliders—one to set each finger joint rotation—within a Manipulate, plus a user interface to save the current pose into a file.

The Demonstration inserts a "bounce" pose between double letters in the input message.

References

[1] Web 3D Consortium. "VRML 2.0 Sourcebook: X3D Examples Archive." (Jul 6, 2012) www.web3d.org/x3d/content/examples/Vrml2.0Sourcebook.

[2] J. Yang and E. Peña Pitarch, "Kinematic Human Modeling," Technical Report No. VSR-04.02, The Virtual Soldier Research (VSR) Program, The University of Iowa, 2004. web.archive.org/web/20050323085750/http://www.digital-humans.org/Report2004/Documents/01-KinematicHumanModeling.htm.

[3] American Sign Language University (William Vicars). "Sign Language: Dr. Bill's ASL Fingerspelling and Handshape Art." (Jul 6, 2012) www.lifeprint.com/asl101/fingerspelling/abc-gifs/index.htm.



Feedback (field required)
Email (field required) Name
Occupation Organization
Note: Your message & contact information may be shared with the author of any specific Demonstration for which you give feedback.
Send