Research

Project Summary

Current technical sensor systems offer capabilities that are superior to human perception. Cameras can capture a spectrum that is wider than visible light, high-speed cameras can show movements that are invisible to the human eye, and directional microphones can pick up sounds at long distances. The vision of this project is to lay a foundation for the creation of digital technologies that provide novel sensory experiences and new perceptual capabilities for humans that are natural and intuitive to use. In a first step, the project will assess the feasibility of creating artificial human senses that provide new perceptual channels to the human mind, without increasing the experienced cognitive load. A particular focus is on creating intuitive and natural control mechanisms for amplified senses using eye gaze, muscle activity, and brain signals. Through the creation of a prototype that provides mildly unpleasant stimulations in response to perceived information, the feasibility of implementing an artificial reflex will be experimentally explored. The project will quantify the effectiveness of new senses and artificial perceptual aids compared to the baseline of unaugmented perception. The overall objective is to systematically research, explore, and model new means for increasing the human intake of information in order to lay the foundation for new and improved human senses enabled through digital technologies and to enable artificial reflexes. The ground-breaking contributions of this project are (1) to demonstrate the feasibility of reliably implementing amplified senses and new perceptual capabilities, (2) to prove the possibility of creating an artificial reflex, (3) to provide an example implementation of amplified cognition that is empirically validated, and (4) to develop models, concepts, components, and platforms that will enable and ease the creation of interactive systems that measurably increase human perceptual capabilities.

 

Research Direction beyond the Project

In our group we conduct research at the crossroads of human computer interaction, media technology, and ubiquitous computing.

Our research interests are in challenges that pose hard questions for basic research, but at the same time have a clear application to specific domains or impact on society.


The overall research question is: how can we enhance human abilities through digital technologies. In a recent IEEE Computer article we have outlined this vision of amplifying the human mind through digital technologies [1].

With increasing automation and with autonomous systems becoming ubiquitous, the challenges for keeping the human in the loop and in the center becomes even greater and more important. We have suggested a novel interaction paradigm “Intervention user interfaces” to reflect this [2].

Specific Technologie and Topics

Topics that we work on include:

    • physiological sensing for human computer interaction (EMG, EEG)
    • interfaces that adapt to the user’s cognitive load
    • measuring and understanding cognitive load in the context of interactive systems
    • interaction with automated and autonomous systems
    • interaction with big data and interactive exploration of complex data
    • augmented reality (AR) and its applications
    • virtual reality (VR) and interaction in virtual reality, e.g. haptics in VR
    • emotions in user interface
    • accessibility of ubiquitous computing systems
    • wearable computing user interfaces
    • automotive user interface in the context of automated driving and with a focus on secondary tasks

References

[1] A. Schmidt, “Technologies to Amplify the Mind,”  (PDF) in Computer, vol. 50, no. 10, pp. 102-106, 2017. doi: 10.1109/MC.2017.3641644

[2] Albrecht Schmidt and Thomas Herrmann. 2017. Intervention user interfaces: a new interaction paradigm for automated systems. (PDF)  interactions 24, 5 (August 2017), 40-45. DOI: https://doi.org/10.1145/3121357