How do decision makers interact with geospatial data? An eye tracking study

Within large engineering projects, data of all types (usually geospatial) is disseminated from a variety of sources to different teams, organisations, and stakeholders. Interactive web mapping is an example of the advances in tools and technology that have allowed effective presentation of such information across a variety of different sectors, including within engineering. One of the key areas of spatial data utilisation concerning such projects in the UK are Environmental Impact Assessments (EIAs). For large projects (involving development) to be submitted to the relevant authorities for approval, data must be sourced, collected, analysed, and then presented in a format that can be annotated, assessed, and subject to decision-making; the EIA process allows for the creation of a mandatory Environmental Statement, through a collaborative approach across multiple disciplines (Saarikoski, 2000). Spatial data is the backbone of such submissions and is integral to the EIA process; many stakeholders, therefore, perceive web maps as a critical element in the design, implementation, and construction of large engineering projects (Gharehbaghi and Scott-Young, 2018). Little is known, however, about how users interact, interpret, and collaborate with these online maps. An understanding of how different users and decision makers interact, process, and rely on information it gives an indication of what the user depends on to make decisions using spatial data. In turn, this can specify more relevant needs in terms of data specification, access and presentation which will help streamline decision making using EIAs in the engineering sector. The aim of this study is to understand participants’ interaction with EIA geovisualisations when making decisions, allowing categorisation and examination of their use of mapped data layers. This will explore whether current decision support tools meet the data and cognitive needs of the individual, including supporting how they interact and select data to come to their decision in multidisciplinary projects. Such work is part of an ongoing PhD, from which previous work has focused on how certain groups of participants rely on map elements to make decisions and how this relates to their background and the task given (Hepburn and Fairbairn, 2019; Hepburn, Fairbairn and Barr, 2020; Hepburn et al., 2021). Eye tracking allows measurements that reveal the location, object, and order of interactions with map-based information during a defined task (Carter and Luke, 2020). The ‘eye mind link’ (Just and Carpenter, 1980; Rayner, 2009; Rayner and Reingold, 2015) connects the gaze of the viewer with points and zones of graphical stimulus, and eye tracking offers a usable and reliable tool to assess the distribution of visual attention. When eye tracking is the main data collection technique for usability studies, it is necessary to adopt a mixed methods approach since individual metrics may not fully explain the decisions being made by the participants. In this study, surveyed data on decisions is collected alongside the eye tracking metrics and background information on each participant, along with observations based on screen recordings obtained during the eye interactions. Metrics collected include fixation (the point at which someone is looking) frequency, count and length for defined areas of interest, whilst mouse clicks and navigation were also recorded. This study was rigorously designed based on previous eye tracking experiments, the context of the research, and the resources available. Participants were employees at the engineering consultancy Atkins, the industrial sponsor on this project. The Tobii X2-30 fixed eye tracker was used, recording at 30 Hz (times per second the position of the eyes is registered by the eye tracker). The stimuli were presented on a 13” laptop screen and resolution of 1920 x 1080. Importantly, the study was not conducted in a controlled lab environment, but in the participants’ usual working environment, meaning the distractions of everyday office life were present. This allows this study to mimic the actual situation in which individuals make decisions. Data was collected at a variety of Atkins offices in the UK from December 2019 to March 2020. In total 35 participants took part, with 33 recordings usable and available for analysis. The participants volunteered from a variety of job roles (analyst, consultant, technician, senior), from a variety of departments (ecology, geospatial, architecture, infrastructure). All participants regularly dealt with geospatial data and were familiar with working on a diverse range of projects requiring EIAs. Background information and a signed ethical agreement were collected from all participants prior to taking part in the study. The background profile contains a variety of information, including job role, department, experience, and the nature of decision-making (collaborative, individual or mixed). Mapped EIA scenarios were designed and styled based on existing geospatial infrastructure in Atkins. In total three Abstracts of the International Cartographic Association, 3, 2021. 30th International Cartographic Conference (ICC 2021), 14–18 December 2021, Florence, Italy. https://doi.org/10.5194/ica-abs-3-109-2021 | © Author(s) 2021. CC BY 4.0 License.s of the International Cartographic Association, 3, 2021. 30th International Cartographic Conference (ICC 2021), 14–18 December 2021, Florence, Italy. https://doi.org/10.5194/ica-abs-3-109-2021 | © Author(s) 2021. CC BY 4.0 License.

The aim of this study is to understand participants' interaction with EIA geovisualisations when making decisions, allowing categorisation and examination of their use of mapped data layers. This will explore whether current decision support tools meet the data and cognitive needs of the individual, including supporting how they interact and select data to come to their decision in multidisciplinary projects. Such work is part of an ongoing PhD, from which previous work has focused on how certain groups of participants rely on map elements to make decisions and how this relates to their background and the task given (Hepburn and Fairbairn, 2019;Hepburn, Fairbairn and Barr, 2020;Hepburn et al., 2021).
Eye tracking allows measurements that reveal the location, object, and order of interactions with map-based information during a defined task (Carter and Luke, 2020). The 'eye mind link ' (Just and Carpenter, 1980;Rayner, 2009;Rayner and Reingold, 2015) connects the gaze of the viewer with points and zones of graphical stimulus, and eye tracking offers a usable and reliable tool to assess the distribution of visual attention. When eye tracking is the main data collection technique for usability studies, it is necessary to adopt a mixed methods approach since individual metrics may not fully explain the decisions being made by the participants. In this study, surveyed data on decisions is collected alongside the eye tracking metrics and background information on each participant, along with observations based on screen recordings obtained during the eye interactions. Metrics collected include fixation (the point at which someone is looking) frequency, count and length for defined areas of interest, whilst mouse clicks and navigation were also recorded.
This study was rigorously designed based on previous eye tracking experiments, the context of the research, and the resources available. Participants were employees at the engineering consultancy Atkins, the industrial sponsor on this project. The Tobii X2-30 fixed eye tracker was used, recording at 30 Hz (times per second the position of the eyes is registered by the eye tracker). The stimuli were presented on a 13" laptop screen and resolution of 1920 x 1080. Importantly, the study was not conducted in a controlled lab environment, but in the participants' usual working environment, meaning the distractions of everyday office life were present. This allows this study to mimic the actual situation in which individuals make decisions. Data was collected at a variety of Atkins offices in the UK from December 2019 to March 2020. In total 35 participants took part, with 33 recordings usable and available for analysis. The participants volunteered from a variety of job roles (analyst, consultant, technician, senior), from a variety of departments (ecology, geospatial, architecture, infrastructure). All participants regularly dealt with geospatial data and were familiar with working on a diverse range of projects requiring EIAs. Background information and a signed ethical agreement were collected from all participants prior to taking part in the study. The background profile contains a variety of information, including job role, department, experience, and the nature of decision-making (collaborative, individual or mixed). Mapped EIA scenarios were designed and styled based on existing geospatial infrastructure in Atkins. In total three scenarios were created to simulate decision making situations. This paper focuses on Scenario B, which asked participants to rank four routes for the laying of a cable in an environmentally sensitive area on the Isle of Wight, UK from least to most suitable. This cable led from a mast to specified outlet sites. During data collection, responses were accessed through ESRIs Survey123 platform, with links to a StoryMap which presented the data in a familiar format for the participants. The participants were able to click, zoom and pan on the interactive maps.
First the participants were asked to select their basemap preference; Open Street Map (OSM) or satellite imagery. 48% of participants chose satellite imagery compared to 52% who chose OSM. During analysis it was concluded that there was no relationship between job role, grade, or team and the basemap chosen. When comparing the basemap choice to various eye tracking metrics including length of time, number of fixations, length of fixations, visit time and click count, there was also no difference between the two groups of OSM and satellite imagery. This suggests that decision makers should have the flexibility to choose a basemap which meets their own cognitive requirements, and that it does not impact the decision being made. The use of basemap choice to define participant groups has been eliminated, although future analysis will investigate how it relates to the decisions made by the participants. Observations were made in the data collection and from the eye tracking screen recording, from these different types of interaction can be defined. With regards to layer investigation, participants either turned each layer on and off individually, or looked at multiple layers together. Participants fell into one of these two distinct groups. Similarly with legend interactions, participants either examined the data in direct order (from top to bottom), or randomly. It was interesting to see these distinct interaction behaviours and further work will explore these.
Participants were then asked to rank the four cable route options using their basemap choice and very basic data, including mast site, proposed routes, rivers, waterbodies, and buildings. The third decision made by the participants was another ranking of the routes, this time using a variety of common data layers usually found in EIAs to aid their decision. The same layers were presented as in the first ranking, but participants had the opportunity to interact with these existing layers alongside additional EIA data layers. Interaction was much lower with the existing layers (already turned on), compared to the variety of layers which could be activated. The existing layers included important information for the participant, including the location of the proposed cable line, which were critical for the decision-making process. Overall, only 28% of participants interacted with the existing layers, compared to 80% of the participants who interacted with the optional layers. This demonstrates that the choice of these existing layers when designing a decision support tool can have an impact on interactions, as existing layers are unlikely to be interacted with as much as the optional counterparts. Of the existing layers, on average 6.67% were interacted with, compared to an average of 60.8% of the additional data layers. Figure 1 shows that there was much more variability and range in the number of layers interacted with from the additional set compared to the existing set. It is evident from box plots below ( Figure 1) that this data is skewed due to the high percentage of participants choosing to not interact with the data for existing and additional layers, 72% and 20% retrospectively. Therefore, their usage was 0% despite receiving the same information and task as the other participants. Further analysis will investigate the characteristics of these participants as passive interactors with the data and compare to their counterparts who actively interacted with the data. Overall, the findings indicate that map users have different interaction techniques. Understanding more about these techniques and how they influence the decisions being made is the next step. It is interesting that participants interact with existing layers less than the optional layers available. Using the background information collected about each participant will help group the participants to understand the relationship between job role, department, and experience with interaction; giving the ability to improve decision making tools within multidisciplinary projects.