We present a marker-less AR/DR system that can replace the arm of the user with a virtual bionic prosthesis in real time including finger tracking. For this, we use a mixed reality HMD that provides the user with a stereo image based on video-see-through (VST). We apply chroma-keying to remove the user’s arm from each captured image and input reconstructed background information into the removed pixels. Before rendering the prosthesis model into the image, we re-target motion capture data of the user’s hand to the kinematic skeleton of the prosthesis to match the current hand pose. This system opens new research possibilities on self- and other-perception of bionic bodies. In a first evaluation study of the system, we propose that users perceive the virtual prosthesis model as a part of their body (i.e., that they experience a sense of ownership). We tested this assumption in a laboratory study with 27 individuals who used the system to perform a series of simple tasks in AR with their prosthesis. We measured body ownership and other measures with self-reports. In support of the hypothesis, users experienced a sense of body ownership. Also, a feeling of self-presence is induced during the task, and participants rated the overall experience as positive.

The number of individuals using some form of bionic technology that is merged to their bodies (e.g., prostheses, exoskeletons) is likely to increase in the future. Such Embodied Digital Technologies (EDTs) will affect the psychological processes underlying social interaction, perception, and stereotyping, especially when users and non-users of these technologies meet and need to coordinate in public. Therefore, understanding the psychological processes underlying the use and the perceptions of embodied technologies is important for designing such devices for smooth coordination. However, today’s limited availability of both, technology and their users limits the possibilities for conducting psychological studies in this area. We thus suggest employing Mixed Reality (MR) to simulate wearing those technologies and to simulate encounters between users for studying associated processes. We call our system TechnoSapiens to emphasize the merging of human bodies with technology. We describe major technological requirements, propose a psychological framework to assess the system’s capabilities and lay out the current state-of-the-art in the respective fields.