IBM develops virtual deaf interpreter

HURSLEY, England (AP) – Here’s a productive twist on the animated characters known as avatars that carry out fantasies in virtual computer worlds. IBM Corp. researchers have developed an avatar that can translate spoken words into sign language.

The technology is meant for use in the real world and could be useful when human interpreters for the deaf are unavailable, or when a conversation is sensitive. Imagine sitting through a lecture and seeing a digital character projected on a screen behind the speaker, interpreting the speech in real time.

The IBM team and advocates for the deaf caution that the system is still a prototype. It also works only with British Sign Language for now because it was created at an IBM research centre in Hursley, England.

But the system, known as SiSi (for “Say It Sign It”), is expected to move out of the labs and into commercial products. It is designed to be baked into other vendors’ deaf-accessibility products and expanded to include other countries’ sign languages.

SiSi works by using speech recognition to convert a conversation into text. From there, SiSi translates the text into the gestures used in sign language and animates a customizable avatar that carries them out.

Among its many uses, the Say it Sign it software could eliminate the need for human sign language interpreters at meetings and conferences, says IBM.

By Paul McDougall
InformationWeek
September 13, 2007 02:15 PM

IBM (IBM) says it has developed a computer program that will automatically convert the spoken word to sign language that can be animated on a display screen by a digital avatar.The company said in an announcement Thursday that its SiSi (Say it Sign it) software uses speech recognition technology to convert the spoken word to text, which is then used by the host computer to control the movements of an screen avatar gesturing in British Sign Language.

IBM researchers envision a number of uses for the technology. It could be employed by business conference organizers to project a signing avatar on a display screen for the benefit of hearing impaired attendees. SiSi would also eliminate the need for them to hire human sign language interpreters.

SiSi could also be loaded on a portable computing device to allow individuals to more easily communicate with deaf family members. Conceivably, the technology could also be used to add sign language capabilities to avatars in virtual reality environments like Second Life. IBM has yet to disclose how it plans to commercially market SiSi.

An estimated 55,000 deaf or hearing impaired U.K. citizens communicate using British Sign Language, which differs significantly in a number of respects from American Sign Language. IBM hasn’t said if it’s developing a similar system for ASL, but the company now appears to have the fundamental technologies in place to do so.

Officials at advocacy groups for the deaf are welcoming the breakthrough. “It is encouraging to see that mainstream research is contributing to the objective of a more inclusive society,” said Guido Gybels, director of new technologies at the U.K.’s Royal National Institute For Deaf People, in a statement.