Extended-Personal Reality: augmented recording and transmission of virtual senses through artificial-Intelligence

EXPERIENCE’s aim is to create a new Extended-Personal Reality, which allows users to automatically record their own virtual reality (VR) environments. Along with recording visual and auditory experiences, the project will aim at creating a device which also records psychophysiological patterns, which will in turn elicit psychological, cognitive, neurophysiological and behavioural responses. To achieve this, a novel lightweight wearable device will be designed, embedded with sensors for brain and autonomic regulation monitoring, multiple cameras for a 3D view, hand kinematics, and environmental noise monitoring.

Neurobehavioral and multimodal neuroimaging data obtained from this device will be used by advanced Artificial Intelligence algorithms to create the VR environments, which can be shared with others carrying the wearable device and letting them relive the experiences of the creator.

The EXPERIENCE system will, among other aspects, be used for identifying novel approaches for the treatment of affective disorders by producing the Extended-Personal Reality environments with artificially modified emotional states.

EXPERIENCE will also be explored as a diagnostic tool where behavioural responses/performance on VR-based neurocognitive tasks will be compared in affective disorder patients to normative behavioural data retrieved from healthy comparison groups.

EXPERIENCE is expected to have various interdisciplinary impacts, such as

1. increasing communication possibilities with embedded emotional states between people,
2. reducing stigma around mental disorders,
3. improving clinical diagnostics, evaluation, treatments, and the stratification of psychiatric disorders, and
4. improving access to and reducing costs of mental health treatments, thereby decreasing the loss in productivity caused by mental health disorders.

The project was awarded a complementary grant, implying that the beneficiaries of projects funded by the same call should cooperate and provide their results to each other. Therefore, the development of EXPERIENCE will be done in collaboration with three other EU projects: SONICOM, TOUCHLESS and CAROUSEL.