University of Padova


Spatial Cognition in Mediated Environments

Project Description

Place is an essential feature of our day-life. First of all, we are always present in a place; secondly, place hosts artifacts we exploit to carry on our activities. In particular, the HTlab promotes a vision of place as “a set of resources organized and transformed by the involvement in human action at any certain moment” (Spagnolli, Gamberini, 2005).

The meaning of place, thereby, is defined by the relationship between people and the place resources (which can be objects as well as other social actors), developed through the actions performed. Therefore, the use of an ethnographic, observational approach to the study of place, in order to investigate how people construct place by actions.
Currently, the Htlab is particularly interested in verbal and non-verbal resources mobilized to identify the position of an object in a mediated environment. The bridge connecting communication and cognition has already been conceptualized by Talmy (1983), who asserts that “language schematizes space, selecting certain aspects of a referent scene” while disregarding the remaining aspects (pp.225). Consequently, the ambitious aim of this project will be to study spatial cognition through the studies of reference communication.

Setting of the experiment with images projected over a large screen Setting of the experiment with images projected over a floor

How do we relate with the place we inhabit? Currently this project points at analyzing people’s preference of behaviours in identifying objects’ location presented in virtual images projected on screens.
Our focus of attention will be to:

– POINTING: Kita (2000) defines pointing as a non-linguistic communication act through which people can indicate a place, an object or a direction. Its function is to shift the interlocutor’s attention towards a specific space or object.

– FRAMES OF REFERENCE: Levinson underlines that place is cognitively categorized in different ways according to the speaker’s culture (2003). This different construction of place emerges from the coordinates utilized to describe it (intrinsic, relative, absolute).

– INDEXICALS: Hanks (1992) defined indexical expressions (e.g. this, that) as codifiers of a relation between the referent and its context. Differently from the frames of reference deictic relations are not characterized by coordinates, but by the relation between the referent and the ground of the verbal reference.

– PERSPECTIVES: Tversky (1996) posed a milestone in the spatial cognition research providing evidence about the effects of different visual perspectives on linguistic spatial production. Survey (the view of an environment from the top, e.g. a map), Route (the representation of an environment from the inside) and Gaze (the view of the environment from an external fixed point) result in different frames of references produced by speakers to describe them.

Image used in the experiment Image used in the experiment

The Htlab contribution to the existing studies on spatial reference and cognition is mainly based on the new kinds of environment proposed, virtual ones. These new places not only extend participants’ possibilities of actions beyond the physical area, but also host new forms of collaborations.

Some of the Htlab research issues are:

– coordination of verbal and non-verbal contributions in describing place;
– mediated pointing and verbal communication in spatial reference;
– factors affecting spatial cognition;
– collaboration and spatial reference.

Work Description

EXPERIMENT 1: Actions investigated: POINTING
This research investigated the occurrences of mediated pointing in identifying digital objects on a computer screen with an arrow controlled by the mouse.

EXPERIMENT 2: Actions investigated: POINTING + FRAME OF REFERENCE
In this study, conducted at the University of Padova, we tested the interactions between different types of pointing (no pointing, natural pointing, mediated pointing using the mouse), types of task (locative vs descriptive) and types of co-presence (two participants in the same room or in different rooms).

EXPERIMENT 3: Actions investigated: PERSPECTIVES + FRAME OF REFERENCE
In this work, still in progress, participants were required to answer some questions about the location of objects represented in a virtual image. These referents differed according to the perspectives in which they were displayed.

Results

EXPERIMENT 1: The main outcomes highlighted that mediated pointing can be considered as a genuine gesture, because users who are familiar with this device utilize it as a natural extension of their body.
Related paper: Spagnolli A., Gamberini L., Renò L. (2006). Mediated pointing and the corporeal field. Intellectica, 41-42: 55-73.

EXPERIMENT 2: The main results are summarized below:
– absolute frame of references (nord-sud-ovest-est) were rarely used;
– indexicals are rarely produced without a type of pointing of any kind;
– frames of reference are preferred over indexicals;
– mediation seems not to affect spatial reference;
– frames of reference were mainly produced in the description task while indexicals in the locative task.
Related Paper: Spagnolli A., Gamberini L. (2006). Multimodal spatial reference in mediated environments: Users’ preferences and the pragmatics of pointing and talking. In CHI ’06 Extended Abstracts on Human Factors in Computing Systems (Montrèal, Quèbec, Canada, April 22 – 27, 2006). CHI ’06. ACM Press, New York, NY, 1349-1354.

EXPERIMENT 3: In progress!!

HTLab Publications

– Renò, L. A. (2005). Presence and Mediated Spaces: a Review. PsychNology Journal, 3 (2): 181-199.
Spagnolli A. , Gamberini L. (2005). A Place for Presence. Understanding the Human Involvement in Mediated Interactive Environments, PsychNology Journal 3, (1): 6-15.

– Spagnolli A., Gamberini L. (2006). Multimodal spatial reference in mediated environments: Users’ preferences and the pragmatics of pointing and talking. In CHI ’06 Extended Abstracts on Human Factors in Computing Systems (Montrèal, Quèbec, Canada, April 22 – 27, 2006). CHI ’06. ACM Press, New York, NY, 1349-1354.

– Spagnolli A., Gamberini L., Renò L. (2006). Mediated pointing and the corporeal field. Intellectica, 41-42: 55-73.

References

– Sotaro, K.(2000). Gesture and the Process of Speech Production: We think, therefore we Gesture. Language and Cognitive Processes, 15 (6), 593-613.

– Levinson, S.C.(2003). Space in Language and Cognition. Explorations in Cognitive Diversity. Casmbridge: Cambridge University Press.

– Hanks, W.F. (1992). The Indexical Ground of Deictic Reference. In A. Duranti and C. Goodwin (E ds), Rethinking Context: Language as an Interactive Phenomenon (pp.43-76). Cambridge: Cambridge University Press.

– Taylor, H.A., & Tversky, B. (1996). Perspective in Spatial Descriptions. Journal of Memory and Language, 35 (3), 371-391


© 2019 Università di Padova - Tutti i diritti riservati