[PS-2.20] Sign Language and Immersive Environments: Challenges, Solutions and Opportunities

Montagud, M. . 1 , Orero, P. 2 , Nuñez, J. A. 1 , Matamala, A. 2 & Fernández, S. 1

1 i2CAT Foundation
2 Universitat AutònomaBarcelona (UAB)

The integration of accessibility services (including subtitling, audio description and sign language interpreting) in immersive environments, including Virtual Reality (VR) and 360º contents, still needs to be explored. This presentation will consist of discussing associated challenges and potential solutions to enable truly inclusive and personalized (multi-screen) immersive experiences, with a focus on the integration of sign language interpreting. On the one hand, the different presentation modes, assistive solutions and personalization features being considered in the H2020 project ImAc (http://www.imac-project.eu/) will be discussed. On the other hand, the potential and suitability of adopting the contributions of the H2020 project VR-Together (http://vrtogether.eu/) within the accessibility arena will be explored. In particular, the technological contributions of VR-Together not only allow overlaying a squared 2D video window with a signer in immersive environments, but also seamlessly embedding avatars or 3D real-users? representations, with a photo-realistic quality. This allows perceiving the sign language interpreter as an additional character of the environment, and not like a "simple and probably intrusive add-on". The developed technology in both projects also enables the real-time integration of remote interpreters through networked communications. Demos from both projects will be also showcased.