The MindSee project aimed to design and develop an application for scientific information seeking that exemplified the fruitful symbiosis between modern Brain-Computer Interface (BCI) technology and human-computer interaction in a real-world application context.
This research tool leveraged a substantial amount of information, comprising over 50 million scientific documents from the following scientific databases: Web of Science, Digital Libraries of the Association of Computing Machinery (ACM), the Institute of Electrical and Electronics Engineers (IEEE), and Springer.
The project sought to support the scientific information seeking process by also harnessing implicit signals from users. Both cortical and physiological activity were considered, evaluating how they could impact the MindSee system’s ability to present increasingly relevant information to the participants’ research.
MindSee proposed the fusion of electroencephalography (EEG) with peripheral physiological sensors (e.g., electrodermal response, facial electromyography, and ocular behavior such as fixations and pupillometry) and explicit contextual information, such as user-selected data.

This symbiotic system enabled the “prediction” of user intentions and needs based on the processing of information derived from the analysis of cognitive and physiological signals. These implicit data were utilized to feed a machine-learning algorithm, and as a result, the interface could implement symbiotic adaptations (e.g., displaying new relevant data, modifying the ranking of proposed results, and reducing information density to avoid comprehension difficulties). Real-time estimates of implicit signals were used to complement explicit inputs from the keyboard and mouse in this application, which was tested in real-world scientific information-seeking tasks, where user information exploration was guided by computer co-adaptations, customized uniquely for each user.

References:

Jacucci, G., Barral, O., Daee, P., Wenzel, M., Serim, B., Ruotsalo, T., Pluchino, P., Freeman, J., Gamberini, L., Kaski, S. and Blankertz, B. (2019). Integrating neurophysiologic relevance feedback in intent modeling for information retrieval. Journal of the Association for Information Science and Technology, 70(9), 917-930.

Pluchino, P., Gamberini, L., & Jacucci, G. (2019). User experience and perceived usability of traditional and multimodal interfaces for scientific information seeking. In Human-Computer Interaction. Design Practice in Contemporary Societies: Thematic Area, HCI 2019, Held as Part of the 21st HCI International Conference, HCII 2019, Orlando, FL, USA, July 26–31, 2019, Proceedings, Part III 21(pp. 445-458). Springer International Publishing.

Gamberini, L., & Spagnolli, A. (2017). Towards a definition of symbiotic relations between humans and machines. In Symbiotic Interaction: 5th International Workshop, Symbiotic 2016, Padua, Italy, September 29–30, 2016, Revised Selected Papers 5 (pp. 1-4). Springer International Publishing.

Spagnolli, A., Conti, M., Guerra, G., Freeman, J., Kirsh, D., & van Wynsberghe, A. (2017). Adapting the system to users based on implicit data: ethical risks and possible solutions. In Symbiotic Interaction: 5th International Workshop, Symbiotic 2016, Padua, Italy, September 29–30, 2016, Revised Selected Papers 5 (pp. 5-22). Springer International Publishing.

Gamberini, L., Spagnolli, A., Blankertz, B., Kaski, S., Freeman, J., Acqualagna, L., Barral, O., Bellio, M., Chech, L., Eugster, M. and Ferrari, E., Negri, P., Orso, V., Pluchino, P., Minelle, F., Serim, B., Wenzel, M., & Jacucci, G. (2015). Developing a symbiotic system for scientific information seeking: the mindsee project. In Symbiotic Interaction: 4th International Workshop, Symbiotic 2015, Berlin, Germany, October 7-8, 2015, Proceedings 4 (pp. 68-80). Springer International Publishing.

Pluchino, P., Gamberini, L., Barral, O., & Minelle, F. (2014). How semantic processing of words evokes changes in pupil. In Symbiotic Interaction: Third International Workshop, Symbiotic 2014, Helsinki, Finland, October 30-31, 2014, Proceedings 3 (pp. 99-112). Springer International Publishing.

Jacucci, G., Spagnolli, A., Freeman, J., & Gamberini, L. (2014). Symbiotic interaction: a critical definition and comparison to other human-computer paradigms. In Symbiotic Interaction: Third International Workshop, Symbiotic 2014, Helsinki, Finland, October 30-31, 2014, Proceedings 3 (pp. 3-20). Springer International Publishing.