BECOMING FAMILIAR: HOW INFRASTRUCTURE ENGINEERS BEGIN TO USE COLLABORATIVE VIRTUAL REALITY IN THEIR INTERDISCIPLINARY PRACTICE

SUMMARY: The design community has recently shown increased interest in using virtual reality (VR) in project review contexts. While single-user headsets currently attract most VR-related attention, room-like immersive VR environments can help facilitate design team engagement and shared exploration of projects. However, to date relatively little research concerns how large-scale VR environments are used in and adapted for professional practice. To address this gap, we set up a bespoke portable VR display system called 3D-MOVE in a major UK construction office to investigate how project team members used and evaluated collaborative VR processes. Over a three-month period, we conducted ten video-recorded VR sessions to observe how engineering professionals familiarize themselves with VR in order to help inform its deployment in practice. The study results show that emergent discussions about design models and questioning of design-related assumptions dominated all observed sessions, even though they were staged as technology demonstrations; which supports the social aspects of large-scale collaborative VR processes. However, before participants could focus on design review, they had to familiarize themselves with the VR technology and time required to do so varied depending on the complexity of the VR configuration. As the participants engaged with the VR environment, they reflected on their processes, requirements and expectations and provided feedback for improving the VR experience. Articulating this familiarization with collaborative VR can inform its deployment with respect to minimizing the learning curve and any distractions or discomfort associated with its use while maximizing the aspects of value-added collaborative engagement. Additional considerations concerning content, interactivity and logistics emerged as necessary to address before VR technologies can become standard practice.


INTRODUCTION
Virtual reality (VR), while not a new technology, has recently gained a renewed attention as an effective and engaging means of visualizing building and infrastructure design projects, which suggests its potential for improving communication through shared visualization (Whyte and Nikolić 2018).This growing interest in VR corresponds with a marked rise in relatively low-cost consumer market virtual and augmented reality products such as Oculus Rift, HTC Vive, Google Cardboard, and Microsoft HoloLens, among others.The use of large projection-based collaborative VR systems in practical design and construction project settings has gained less attention, though there are documented benefits for collaborative decision-making, such as in design reviews (e.g.Dunston et al. 2011;Liu 2017;Whyte and Nikolić 2018).Potential reasons for their lower uptake include the relatively high costs of these systems (Philpot et al. 2017), their spatial requirements and relative immobility, and their general perception as difficult to use (Zaker and Coloma 2018).
Thus, to facilitate the uptake of collaborative VR processes among design and construction project teams, these obstacles must be understood and addressed.To this end, we installed a large-scale portable VR system in the central offices of a large infrastructure practice.The novelty of the study lies in extending experimental usertesting studies using similar systems in academic settings by placing the collaborative VR technology within an organization to observe the process of technology uptake.Specifically, the study asks what happens when collaborative VR system is placed in a professional setting.How do project team members begin to use such technology?What do participants perceive as helpful or difficult when using such VR system?What measures would be required to integrate these systems into professional interdisciplinary review practices?
To address these questions, we first review the literature on this topic and the work on collaborative VR, particularly within design and construction contexts.Then, we discuss our method of observing project teams invited to view their project models in collaborative VR, to investigate how novice users engage with VR technology.Participants were asked to evaluate the usability of the VR system and identify further use-cases and relevant information needs to facilitate the integration of VR into their work practices.Our findings showed three emergent themes: engagement with large-scale VR technology, engagement with design models and the value of VR supported collaboration.We then conclude the paper with implications for research and practice.

COLLABORATIVE VR
Since the term was coined in the 1980s, VR has aimed to provide users with a compelling experience of being immersed in a simulated virtual environment within which they can intuitively interact.Early technology-focused definitions described VR systems as special purpose hardware components, which carried certain limitations for the social science research that sought to make comparisons across VR systems.Recently, a more widely adopted definition of VR focuses on users experience and differentiates VR from other display systems through one of its defining factors: presence.In this context, presence can manifest as a sense of inhabiting virtual environments, and of being together with other users (Biocca et al. 2003).
Presence, as the "perceptual illusion of non-mediation" (Lombard and Ditton 1997), is typically associated with the sensory features of a VR system.Hence, the assumption is that the desktop-type non-immersive system are less likely to induce a sense of presence when compared with full sensory, visually immersive systems, such as head-mounted displays.Yet, when conceptualized through its physical and social components (Ijsselsteijn et al. 2000), the term presence offers an even more complex view of virtual environments in which users may simultaneously experience the sense of being both physically located somewhere and being together with someone respectively.Thus, a VR medium can offer a high degree of one experience, but not the other.
In the context of VR, presence thus seems to be a more variable state (Otto 2002) that is affected by other sensory and representational factors.For example, head-mounted displays (HMDs) provide full visual immersion and therefore a high degree of physical presence, but not a sense of social presence due to the lack of communicative signals with other users, or the "nearness of communicative partners" (Ijsselsteijn et al. 2000).In other words, users wearing HMDs are typically alone in the virtual world, although the use of avatars with gestures and facial expressions in VR settings can begin to establish social presence for remotely connected participants (Bente et al. 2008;Nguyen and Duval 2014).Others note that HMD environments can also affect the a sense of presence or task performance due to a typically disembodied experience, or lack of one's own body representation (Pan and Hamilton 2018;Philpot et al. 2017;Sanchez-Vives and Slater 2005), as well as increase a sense of discomfort of a tethered experience, including the motion sickness.
By contrast, collaborative, multi-user or room-like virtual environments can afford a degree of both physical and social presence.In this paper, we focus on a particular VR system that accommodates co-located participants.It is important to note that we do not consider collaborative virtual environments more broadly defined as digital spaces that gather multiple, often remote participants, such as online games (e.g.World of Warcraft) or virtual communities (e.g.Second Life).Specifically, we focus on room-like projection-based display systems that allow multiple users to physically inhabit a shared space while surrounded by virtual information that increases a sense of social presence (Kuhlen and Hentschel 2014).We refer to these types of systems as collaborative VR.

VR IN DESIGN AND CONSTRUCTION
The growing appeal of VR in design and construction stems from its ability to simulate aspects of the built environment in a visually more compelling manner.Typically, these systems allow users to dynamically navigate the virtual environment, to switch between different viewpoints and perspectives, and to interact with the displayed information in real time.Many researchers have investigated a range of VR configurations for an array of individual and collaborative task types.Single user VR, such as head-mounted displays for example, seem to lend themselves well to tasks related to understanding individual user behavior and preferences (Heydarian et al. 2015), safety scenarios training (Buttussi and Chittaro 2018), interior design (Kaleja and Kozlovská 2017), and other tasks that rely on user's position and rotation tracking functionality.
The motivation for using VR is linked to the growth of collaborative practices in the design, engineering and construction disciplines that increasingly focus on developing, simulating and reviewing 3D information models.Not only do these models hold semantic data, but they also empower project stakeholders to visualize design projects within specific usability or performance scenarios, and in turn provide more meaningful feedback.Moreover, the growing adoption of building information modeling (BIM) tools centers on the ability for users to interact with the 3D models (Tse et al. 2005).But while many BIM-related tools support design authoring and information sharing processes, they are not necessarily a platform for user-centered collaborative tasks such as those found in the design review process (Plume and Mitchell 2007;Shiratuddin and Thabet 2011).
Depending on the task at hand, the ability to easily visualize and interact with design models is critical in providing information and stakeholder feedback (Castronovo et al. 2013).These requirements are particularly relevant for tasks that rely on adequate spatial understanding, in which the characteristics of a display medium, such as viewing perspective (e.g.object-centered vs. viewer-centered) and viewing scale (e.g.monitor vs. large screen), can affect the way users understand information (Whyte and Nikolić 2018;Paes et al. 2017).Therefore, visualization depends not only on the representation of content (e.g.3D, 2D), but also on a given display medium (e.g.monitor, large screen or head-mounted displays) and the ability to manipulate models to gain meaningful feedback (e.g. through interactive content features and user interfaces).
The value of using large-display collaborative VR systems in design and construction settings lies is in their ability to support direct communication channels and understand user reactions to and experiences within virtual spaces (Christiansson et al. 2011;Dunston et al. 2010;Nykänen et al. 2008).Several studies have demonstrated how virtual mock-ups shown using immersive and semi-immersive projection displays can promote conversations that are valuable for problem solving (Dunston et al. 2007;Messner 2006;Whisker et al. 2003).Moreover, these studies show the importance of establishing social presence for multiple users involved in collaborative tasks to achieve common goals (Wienrich et al. 2018).And while they can be difficult to coordinate, co-located or face-to-face sessions have advantages over remote alternatives for their greater communication channel "bandwidth" (Bassanino et al. 2014), in which non-verbal cues and gesture-driven communication are key in mutual engagement as indirect measurements of social presence (Germani et al. 2012).The importance of social presence is supported by studies that examine the socio-emotional dimensions of human communication in collaborative problem solving tasks.Building on the notions of common ground, group awareness and interpersonal trust, establishing social presence can inform the development and deployment of technologies to mediate collaboration.
The use of large display immersive VR for design review can affect collaboration by facilitating project stakeholder interactions and enhancing client engagement (Tutt and Harty 2013).Liston (2000) investigated how various modes of visualizing information can affect how design teams spend their time in design meetings, and found that 4D CAVE environments can help construction teams to quickly identify and address issues by changing their interaction dynamics and focusing their discussions on explaining, rather than describing information.This is also evident in participatory design practices where problems are seldom clearly defined, and thus require the engagement of both experts and non-experts at key stages in evaluating proposed solutions.Dunston et al. (2007) highlighted the advantages of large display VR systems for performance-based and user-centered approaches to design, including increased participant interaction with design processes and deeper evaluation of space functionality.Experiments and user tests have similarly confirmed the value of using large display collaborative VR systems for design review and construction planning (Castronovo et al. 2013;Shiratuddin et al. 2004;Whisker et al. 2003), for on-site safety training (Sacks et al. 2013), for computational fluid dynamics modeling (Kuhlen and Hentschel 2014), and for explorative analysis of 3D data (Laha et al. 2012).
However, a broader question concerning the use of collaborative VR lies in how to effectively engage teams in reviewing project information (Kim et al. 2013) and anticipate issues when planning such processes (Mastrolembo Ventura et al. 2019).At the same time, as collaborative VR presents opportunities to engage a broad range of users with digital prototypes, industry uptake of such systems has remained sporadic compared to single-user wearable VR.Since the inception of CAVE tehcnology in the 1990s, the promise of lower-cost, commodity-based VR systems has responded to the steadily increasing demand for scalable VR solutions that leverage existing workflows in practical settings.
To understand how project teams incorporate collaborative VR into their daily work practices, the 3D Mobile Visualization Environment (3D-MOVE)a lightweight, rear-projected collaborative visualization environment (Parfitt and Whyte 2014) (see FIG. 1)was placed in the central offices of a large infrastructure practice.The goal was to understand the processes and considerations for VR uptake within the organization in terms of technology engagement, perceived usability and utility of such a system.This study contributes to the current knowledge on how collaborative VR such as 3D-MOVE can engage project team members in reviewing large infrastructure project models in practical settings, while also identifying specific usability aspects and considerations towards integrating VR into standard practices.

METHOD
To investigate how project teams use and perceive VR technology in their organizational practice, we set up the three-screen rear-projected VR system (3D-MOVE) in the major project's central offices location (FIG.1).This research perspective emphasizes the situated character of design and construction practice, as realized in social, perceptual and material interaction through practitioners' ongoing experience in the contexts of practice.To help bring collaborative VR more into the mainstream practice, the low-cost, scalable, and easy-to-use approaches for projection-based VR systems are based on commodity components and existing practices wherever possible.

FIG. 1: The 3D-MOVE in use in design review in the major project's System-Wide offices
We used video-based observations of users' interactions in VR, augmented with post-experience feedback questionnaires, informal conversations and field notes.The rationale behind this approach builds on the methodological principles of video-based studies prioritizing the situated and interactional accomplishment of practical action (Heath et al. 2010), and the broad positioning of the study.The video-based analysis methods provide relevant research means to understand in depth how collaborative VR is used for performing design-related activities (Maftei 2015;Maftei and Harty 2015;Tutt and Harty 2013).Working with the Innovation Team in the project who organized project teams' visits to the central offices to use the 3D-MOVE, the researchers gathered data from ten sessions over five days during the three-month period (Table 1).The sessions were primarily organized as technology demonstrations, but using the models from the current projects the teams were working on.The number of users in each of the sessions ranged from three to eight.Augmenting direct observation with video recordings draws on the tenets of video-based studies of workplaces and of interactions (Heath et al. 2010;Heath and Luff 2008), which recognize and emphasize the potential of using video to examine how the talk, gestures, visible conduct and the use of artefacts becomes relevant to the accomplishment of a social and cooperative activity.Using video in research offers practical advantages to investigate at granular levels how the participants perform their actions when (re)viewing the design, how they interact with and around the technology, and how they orient themselves to the technology and to other participants' actions.

VR setup and training
The 3D-MOVE system was located in the project's central, 'System-Wide' offices and involved the physical frame setup of the screens, positioning and aligning of the projectors and fine-tuning of the display settings and testing the system.The final setup featured three rear-projected screens (2.8m x 2.1m) attached to a single Windows workstation with a stereo-capable OpenGL graphics adaptor.Maintaining the single-computer design for the multiscreen system allows the continued use of the familiar Windows desktop and applications, leveraging end-user computing skills and familiarity with existing commercial applications.The system used three DLP Optoma EH505 active stereo projectors, each outputting 5000 ANSI lumens displayed at XGA resolution (1024x768).Though the setup had 3D-stereoscopic capability, the 3D-MOVE was primarily used in non-stereo as a generally preferred mode by the users for the remaining time.Based on the initial comments and informal feedback from the users, 3Dconnexion SpaceMouse as a commodity interactive device used for content and model navigation was replaced with a, what was perceived as more intuitive, x-box controller.Initial one-day training was conducted to enable the project staff to 1) operate and use the VR system (e.g.turn the system on and off, load the existing simulations, use the navigation and input devices); and 2) develop the VR content using the documented workflow, which outlined the steps to bring a design model from Bentley Microstation into Unity 3D game engine and subsequently load and display it using MiddleVR -a plugin to streamline customization of display and interactions configurations depending on the display system characteristics and input devices.
Written instructions and video demonstration were developed, which allowed engineers to prepare their own models for use in VR.While the research team offered continuous support in developing the VR-ready models, several project team members championed the development of virtual prototypes, as well as mobilization of other project participants around informal technology and design demonstrations.Lastly, all project models contained geometry only, with no textures or other associated data.

Data collection and analysis
This research primarily collected and analyzed a total of 7.40 hours of audio-video recordings and observations of ten sessions with project participants experiencing, testing, and discussing the use of VR.Various team members from at least two projects attended the VR sessions (Table 1), where only the members of the Innovation Team and the CAD Solution Expert attended more than one session.Most of the VR sessions were organized following the formal design review meetings, and although the teams viewed the VR models from the respective projects, the sessions were unstructured in a sense that there was no pre-defined agenda for what to look in a model.Hence, the research was bound to the practicalities of collecting the empirical data around the participants' spontaneous ways of using the technology and engaging with their work.These interactions were recorded with one video camera mounted on a tripod facing the projected area of the 3D-MOVE, at approximately 2.10m height.
The process of operationalizing the empirical data in the study draws on thematic analysis around detailed observations of participants' interactions with the technology and the model content as the unit of analysis.Thematic coding was done through iterative review of the whole data until reaching the saturation of analytical categories, revealing three themes.This analytical process involved repeated examination and categorization of the data set and the research team further reviewed the preliminary themes before developing the final analytical categories.The transition from early to detailed analysis was supported by developing a set of working tables centralized as an open ended Excel spreadsheet used as a way to capture the observations, index the hours of data and saturate the analytical categories, while preserving manageable access to empirical examples instantiating the overarching themes.Following recommendations of undertaking video-based studies and video analysis methods (e.g.Heath et al. 2010), the analytical process was advanced through organizing collections of short instances selected from the video data to illustrate the analytical themes.The analysis draws on these thematic collections of illustrative short fragments of data, which were thoroughly examined to capture the minutiae of interactions emerging in the VR sessions.Following these sessions, the participants were also given questionnaires to evaluate the usability of VR system characteristics, including the value of three screens; screen size; realism of size and scale of displayed objects; ease of navigation and orientation; the value of VR for understanding the design; for engaging in team discussion; and VR usefulness for design reviews.The responses to the open-ended questions about the specific challenges, suggested features and other potential use scenarios were analyzed in conjunction with the video data to address the research interest around the overall usefulness of VR and its overall potential to be adopted in practice.

RESULTS
Given the early stage of VR implementation in practice, the thematic analysis of the video observations revealed three themes characterizing the emergent users' behaviors transitioning from: 1) engaging with the technology to 2) engaging with the design models, and to 3) evaluating the VR usability and utility for interdisciplinary design reviews.The first two themes emerge from direct and observable interactions in 3D-MOVE, revealing how the users initially orient to the technology to then quickly shift their focus to the model content, while the third theme draws from the participants' reflections both in action and after the VR experience on the usability of collaborative VR in practice.These thematic aspects characterizing the processes emerging in the 3D-MOVE are further unpacked and illustrated in more detail in the following sections.

Theme 1 -Engaging with the technology
This theme presents a range of instances how project teams experience the immersive environment when introduced to it.Selected instances of video data detailed in the next section illustrate the transitions from: 1) becoming comfortable to use the 3D-MOVE under the guidance and instructions provided by the research team; to 2) learning to navigate and orient in the model, before shifting the focus on the displayed design.These instances are illustrated in more detail below.

Experiencing the technology
This aspect illustrates specific user-technology interaction instances, revealing how the system's characteristics affect users' sense for navigation, orientation in the model, and immersion (e.g.discomfort due to higher navigation speed, limited viewing options to look up and down, or instance of physically stepping into the screens).A pattern of interaction where the participants learn to navigate the model as a group emerged across the 3D-MOVE sessions.Episode 1 illustrates an instance where one of the participants guides his colleague to navigate to a certain point in the model, directing through pointing and commenting to slow down, while the other is trying to adjust the speed and the movement (FIG.2).Even though all the sessions were in non-stereoscopic mode, this instance illustrates that some users may experience the strong sense of being inside the simulated model, and the processes of making sense of the environment by learning to orient inside the two types of spacesthe virtual space of the 3D model, and the physical boundaries of the 3D-MOVE space.

Engaging in an active exploration of the model
Once the participants felt confident with the technology, their focus shifted to the displayed content where they began to discuss the design, even in informal sessions.Episode 3 (FIG.4) illustrates an interesting shift from exogenous (standing outside the 3D-MOVE) to endogenous (entering the 3D-MOVE space) design inquiry, where the participant (Designer 1), initially standing outside the VR space makes general comments about coming to see the technology, but within the first ten minutes of the session the dynamic of the conversation changes through both verbal and bodily behavior, where he enters the VR space and starts noticing and discussing the model in light of an earlier formal design review session, pertaining to design decisions that impacted construction costs (E4, FIG. 4) Now, we'll say we checked this on a 3D! Designer 1 (10:55): We, as designers, had to put these barriers for health and safety reasons.And we told them [the client] immediately, but they just didn't absorb it.
The participants' interaction with the model in full-scale connected to a greater degree of engagement with their actual design work in progress.Following up on a formal design review performed earlier in the same day, these designers noticed that the VR model conveyed more clearly the reasoning behind safety-related design decisions and associated costs that the client questioned.More specifically, the participants commented that the fences designed to protect the electricity wires were clearly needed for the health and safety reasons in operation, and "this way" of simulating the model "makes it obvious".Moreover, the designers noted that using VR during their design review discussion with the client would have saved time in demonstrating the need for the protection fences compared to their standard document-based process.

Theme 2 -Engaging with the design models
This theme illustrates the processes where participants inadvertently begin to engage with the design by navigating the model to review, identify and discuss issues in a collaborative VR.The theme indicates that VR can support particular ways of performing design review through 1) triggering participants to notice design issues that may challenge their previous understanding and expectations about the designs, and 2) enabling engaged and focused discussions among the team members.

Challenging assumptions: identifying design and model issues
Episode 4 bellow illustrates an instance where a group of participants attended a VR session primarily to experience the technology, but soon identified a specific design issue with the vents shafts and the size of rooms needed to accommodate specific equipment.Navigating across the multiple levels of the asset, the participants' discussion alternated between recognizing intended design aspects (e.g.location and number of elements, dimensions of openings, heights etc.), and questioning represented objects and unexpected issues.
Episode 4 (E4, 20th November 2015, clip 4) Engineer 3 (11:07): Where is the lifting hatch?I can't remember whether… Oh, there?Is it?Engineer 4 (11:13): Is it the lower?Engineer 1 (11:20): No, just turn it around a bit ((pointing and showing rotation with his arms)) 'Cause I want to look at that! ((points and draws attention to some other design issue)) ((The team shifts the focus on other design aspects)) Engineer 1 (12:10): I think the hatch is here!((pointing in the model)) Engineer 2 (12:13): This one?Engineer 1 (12:14): The red one, yes.Engineer 1 (12:19): Why can't we see through it?Engineer 2 (12:20): But is it glass?Or it has a lid, I think.Engineer 4 (12:23): This is when the Perma comes and all hatches are closed up Engineer 3 (13:30): Are you sure?Is it a hatch or is it a vent shaft?Engineer 4 (13:35): Well, it could be, I am not sure, but there's a few of them ((the team continues to examine the model by navigating various levels in the design)) Engineer 1 (17:45): Can we go one more floor down?((Engineer 2 continues to navigating the model)) Engineer 1 (17:55): So we got to have a shaft going down in here as well!It's very interesting.Engineer 4 (18:00): It should be.
This episode, akin to an informal design review activity triggered uncertainty around the identity of a component -"Are you sure?Is it a hatch or is it a vent shaft?"-and it consequently raised awareness on the need to comply with design requirements -"So we got to have a shaft going down in here as well!"

Social presence through collaborative discussion and engagement
A dominant thematic aspect occurring in collaborative VR emerged around participants' togetherness in making sense of the design, discussing and imagining potential changes and alternative solutions, negotiating between disciplines and planning further steps as a group through drawing conclusions informing the design development.
The VR environment triggered team discussions covering a range of planned, but also spontaneously noticed design issues.These discussions unfolded through a combination of both verbal and rich bodily behavior related to distinct ways of identifying, describing and addressing design issues in the model, by employing deictic gestures (i.e.direct references to identity/ spatial location of design components in relation to the context and from the perspective of the participants) and directly referring with 'that', 'this', 'here'.An instance presented below (Episode 5, FIG. 6) represents a tendency observed across all sessions for the teams in the 3D-MOVE to engage in a design discussion.Here, the participants' discussion was configured around indicating changes to the design, assessing established dimensions and making decisions to adjust the design to comply with site features.This led various disciplines to confront their perspectives on the suggested changes: That should be at that level, that should go at this level, and this, should be here!Engineer 1 (17:50): But how much is the gate?Design Lead (17:52): The opening?It's 5 meters.Engineer 1 (17:58): What about this fence?Design Lead (18:02): If this is at that level ((points towards the model to indicate the height)) and this is here ((points the element in the model with his pen)), and the fence is 2.6 meters, then there needs to be a step.I mean the construction won't be easy, but the design is not easy either.
At the same time, emerging interactions in the process of reviewing the design in VR indicated the need for other material artifacts and means of seeing and communicating design issues to explain or augment the VR model.In this example, the participant overlaid a piece of paper on the virtual model as a way to explain design aspects regarding its shape and size.These means of expressions were used to communicate participant's design intention (in this case the need to design the fence in compliance with the steps) in a visual way.
Once the participants identified potential design issues, they tended to follow up with imagining alternative design solutions and discuss these as a group (Episode 6).This instance illustrates an exchange between the designer and the engineer analyzing the possible outcomes of design alternatives before confirming the solution.
Episode 6 (E6, November 26th 2015, clip3) Engineer 1 (00:05): […] I assume that Thames Water might appreciate a bit of storage.Design Lead (00:13): I think the thing is, in any case, You have to, even if you want to reduce this ((starts to point toward a pipe in the model)) to show a bit of storage, it would still have to be at that level ((points towards the model)) Engineer 1 (00:26): I know what you mean.((steps closer to the screen and starts to point design issues the model)) Because there's lower chamber for storage here, that!((points towards the left side of the front screen)) (00:34): But basically, we've said this, but I think the only thing that Thames Water are interested in is the flow rate.And we haven't calculated one, just used the model.Design Lead (00:47): Exactly, but the flow rate, if you're increasing the pressure ((points in the model)) Engineer 1 (00:48): But we won't increase it, this is a 200mm pipe ((lowers his body towards the pipe)), and the water just goes bum in the sewer!I think there's a very huge rate for the size of the site!Design Lead (01:08): That's true.Engineer 1 (01:13): So this, the solution that you have got here, is better than the higher level.This is my opinion.
Towards the end of the session, the team comments on the perceived usefulness of VR in enabling them to see what was not possible in other review sessions using other media: Engineer 1 (04:22): Ok, we'll work on the comments.I think that [the review session] is done.(04:25): It was good to see everything we couldn't see on meeting the other day!
In short, the second theme highlights patterns of team and design interactions in collaborative VR where the participants' assumptions about the design were challenged even after doing formal design reviews elsewhere, and where they collectively engaged in discussions to question, propose options and negotiate solutions.These interactions were exemplified in: 1. Embodied ways of referring to the model to better express anticipated consequences of proposed changes, or to give a better sense of the design components dimensions; 2. Combined use of verbal references to other design representations (e.g.drawings) and non-verbal, gestured and embodied reference to the VR model (imagining and showing dimensions through hand and bodily gestures like pointing, lowering down, or arms movement to show width or height of elements) to overcoming challenges regarding the design scale and dimensions; and 3. Use of external artifacts (e.g.pen and paper) as a way to overcome the inability to directly measure objects in the VR models and better explain and communicate identified design issues (e.g.details, dimensions); and importantly, record conclusions regarding further steps in developing the design (paper note).

Theme 3 -Value of collaborative VR for interdisciplinary design reviews
Participants, both in the observed sessions and post-session surveys commented on the potential of using the collaborative VR in future deign reviews, as well as its value in supporting other project activities and phases, such as project development for example.Participants specifically identified the following use cases that can benefit from using the collaborative VR, including the client engagement to reduce time and effort to communicate and explain design decisions; design coordination; reviewing health and safety requirements; and constructability and 4D simulations among other.
Throughout the VR sessions, the participants commented on the capabilities of the large display VR technology to clarify and communicate design.Episode 7 (FIG.7) presented below captures insights drawn on participants' early experience of their VR model after a formal design review that took place earlier that day.The participants specifically indicated the potential of VR to help them convey the design-related issues to the client more clearly and thus eliminate, or reduce the time typically required to address client's questions through standard paper-based forms and protocols.
(14:41)"We had design comments … on how to move things around, such as these fences here" ( 14:46) FIG. 7: Episode 7 (E7;November 20th, 2015;clip1, 14:41-14:46) Designer 1 (14:33): So we had design comments (…) on how we could move things around, such as these fences here ((pointing towards the right side screen)) Designer 2 To create more access for the ladder ((showing with his palm)) Designer 2 (14:50): But the main issue that we got was that we had a query whether we need to have these Armcos here ((both participants pointing to the right side screen to indicate the cables protection elements)) The reason we do this is 'cause these cables coming up, to protect them Now what we have to do, just for this, we have to go through and create a schedule of comments, and spend time and money doing responses, coming back to them, push something they might not interpret in this way, whereas if we would have done it in here, it just wouldn't have been, the comment wouldn't have been made there.(15:34): So, as a consequence, there is any associated cost to this, and time, and I imagine it will be something offset, by the time that's wasted, closing up loops.(15:50): That [VR] is very impressive.
Drawing on this experience of simulating their model in the VR environment, the participants also indicated the potential of VR for design interface coordination: Designer 1(12:30): Yes, we would use it [VR] for when we have our interface design meetings, between the concrete and steel designers and the electrical for example, we should all be standing here [in VR] Another example of participants' reflection on the usability and usefulness of VR is illustrated in the Episode 8 below (FIG.8).Engineer 1's gestures serve to further reveal and support spatial understanding of an issue that was otherwise impossible to be noticed on a plan viewin this case the U shaped upper part of the gate:

Post-session feedback on the usability and usefulness of the 3D-MOVE
In addition to the informal conversations after the observed VR sessions, a total of 32 participants also completed post-session survey on their experience and perceived value of specific VR system and model characteristics.The overall ratings indicate generally a positive attitude and acceptance of such technology (FIG.9), where the realism of scale and size was seen as valuable throughout.It is worth restating that stereoscopic viewing, though available, was disabled after initial comments that it was distracting.User tracking was also excluded to reduce the potential for developing motion sickness and discomfort given the multi-user set up.Reflecting on the usability and utility of the 3D-MOVE, Table 2 offers a summary of participants' feedback on the three areas for using collaborative VR in practice, such as additional use scenarios and project types, as well suggested changes and improvements for an increased VR usability in design reviews., November 20th, 2015, clip 2, 20:35-21:33)

DISCUSSION AND CONCLUSIONS
The early-stage user engagement with the collaborative VR environment described in this study supported the social value of such environments in that it consistently engaged team members in design discussions.However, before they could focus on the design model, participants required time to familiarize themselves and become comfortable with the VR system.This familiarization process can be visualized in stages (see FIG. 11), in which for all the observed sessions, the participants took time to: 1) learn how the VR environment was configured; 2) engage with the model using verbal and non-verbal cues and challenge assumptions through conversations with other team members; and 3) reflect on the usability of the experience and suggest ways to improve and expand the technology through solicited and unsolicited feedback.For this study, we did not quantify the time users spent familiarizing with the system, but the relatively short duration of VR sessions suggests a quick transition in focus from technology to design, which could be explained by the way VR was configured.

FIG. 11: Engagement arc of observed stages of engagement during collaborative VR sessions
For example, the VR system used in this study was configured in a straightforward manner, featuring a non-stereo, non-tracked single egocentric view of the model that allowed only basic navigation options, such as walking or flying.Even then, the viewing scale of the VR system and its relatively fast movement speeds affected participants' experiences in learning how to orient themselves and navigate within the model, with respect to small spaces.While it was generally easy for the participants to acclimate to these parameters during their session periods, we assume that a more complex VR configuration (e.g. one with additional object interactivity, data filtering or dynamic view switching capabilities) would require users more time to acclimate, and thus additional training beyond such discrete review sessions (FIG.12).During a formal design review session period, what is deemed to be optimal time that users should spend in order to become familiar with the VR system can vary depending on the session objectives, format and user aptitude.Thus, this system complexity and the additional time it may require can inform expectations around early deployment of collaborative VR for design reviews, particularly if the goal is to maximize engagement with the design process and minimize user awareness of the mediating technology.For project managers, this suggests that more complex VR functionality should be balanced with optimized model scope and size (Liu et al. 2014), particularly for less-experienced users.Otherwise, the time required to orient to new VR technology may compromise the time spent on engaging with the design process, thus reducing the overall effectiveness of the VR-supported design review.Moreover, reflecting on technology use during and after VR sessions can offer suggestions for both improving user experience and directing greater attention to design reviews (Mastrolembo Ventura et al. 2019).In this way, as users become comfortable navigating digital environments, the boundaries between the physical and virtual space can be rendered almost unnoticeable.

FIG. 12: Conceptual representation of time the users may spend becoming familiar with VR during a review session depending on the level of their experience with and the complexity of VR configuration
Observations of all sessions consistently confirmed a social aspect of room-like VR in that participants engaged quickly and organically in exploring and discussing the digital model.And while all VR sessions were set up as VR technology demonstrations, users who interacted with the VR model inevitably began to notice and question various design attributes, such as location and size of various design elements and safety-related issues through a combination of both verbal cues (e.g.'that' 'this', 'here') and physical actions (e.g.pointing, gesturing, moving).This aspect of reviewing design projects in collaborative VR environments can foster more complex group interactions and faster decision-making abilities (Bassanino et al. 2014;Sallnäs 2005).At the same time, the informal nature of the observed sessions (i.e. the lack of a specific agenda) limited the ability to ascertain to what extent reviews conducted using VR affected design outcomes.
In several instances, the teams noted design issues to follow up on after the VR session, which suggests the value in adding the ability to modify designs during reviews, to annotate or record discussions, or to otherwise note issues discussed in VR environments.In addition, given that to date the teams' formal design reviews largely rely on desk-based tasks and design documents, participants identified the need to leave an audit trail of design review sessions.This suggests a requirement of additional features to make collaborative VR more applicable, such as bidirectional data flows between native model applications and VR environments, added object interactivity and options for dynamic and multi-view displays (Lather et al. 2018).Currently, support for displaying BIM models directly in VR environments is improving through a number of new industry plugins, which reflects the need for multi-modal setup capabilities for design reviews to allow teams to view and change model information concurrently.Without such support, there is a risk that design engagement and discussions promoted by collaborative VR may continue to fall short without mechanisms to capture these discussions.
Large displays and corresponding viewing scales were seen as particularly valuable VR attributes, especially for scenarios that involved communicating design decisions to stakeholders (e.g.clients and contractors), but also in the design of both large and small-scale projects (e.g.tunnels and tight spaces).Due to the many overhead system components typically involved in infrastructure projects, the ability to look up in a VR environment was also seen as necessary, either through a tilt-view option or configuring the VR system with a projected roof.Overall, participants rated the usability of this collaborative VR environment favorably, with navigation generally perceived as easy.At the same time, while participants tended to praise the collaborative VR environment and expressed their willingness to use it again for design reviews, a more challenging issue emerged during informal conversations concerning the logistics of conducting formal design reviews in such a space.For this study, the VR display was set up in the central project office, which participants identified as a challenge for the project's physically distributed teams who attend formal design reviews at other locations.In addition to being able to capture the outcomes of the discussion, ensuring that team members have convenient access to VR technology is an important prerequisite for planning the deployment of collaborative VR systems, particularly for large projects run by distributed teams.
As considerations for introducing collaborative VR into practice straddle usability, logistics and overall support for users, they can be clustered according to the groups of practitioners likely to play a steering role in the VR deployment (Table 3).While project team members that directly use VR systems-including trade engineers, design leads and design managers-may raise the usability-related issues, other actors involved in the deployment process, including project managers, innovation teams and change managers should also consider the broader aspects of VR system accessibility and resource-level support to ensure the acceptance and adoption of such systems.In summary, this work contributes to the discourse on early stage practical uptake of collaborative VR systems by showing how practitioners evaluated the usability of a large-scale collaborative VR and perceived value of such a system for design review.Understanding perceived usability is the first step towards a more robust VR effectiveness evaluation in terms of the design outcomes of mediated design reviews.And while the viability of a collaborative VR system may in part depend on the scientific rationale behind its application (Kim et al. 2013), the conversation with the practitioners revealed contextual considerations for project and change managers in planning sustained use of VR systems after they are introduced.In this sense, this study also contributes to the broadly recognized need to involve design and construction practitioners in testing VR systems to inform their adoption and use in daily practice.In turn, this involvement can open longer-term pathways for developing implementation plans and strategic business models for adopting VR in practice.
Collaborative VR environments can give project teams a powerful agency in questioning, evaluating and justifying design decisions.Depending on user aptitude and the configuration of a given VR environment, initial familiarization with VR technology can quickly facilitate deep engagement with design content.The articulation of this familiarization process has implications for engineering and design practitioners in terms of informing the use of VR and accounting for challenges during its implementation.Thus, this study also informs the development of collaborative VR approaches for design and engineering organizations, both by suggesting adjustments for their usability and by highlighting the role research can play in their deployment in professional settings.At the same time, while participants in this study were generally supportive of the VR environment, their early-stage engagement revealed broader logistical and managerial considerations that should be addressed in order to facilitate its regular and sustained use.As practitioners become more comfortable with using VR technology, the next step is to investigate the organizational and managerial considerations required to implement collaborative VR approaches in professional settings and to observe how VR technology can influence and shape decisionmaking practices in the engineering and design community.

Episode 5 (
E5, November 26th 2015, clip1) Engineer 1 (16:20): Shall I have a go?Design Lead (16:21): Actually I have a list ((takes his paper list)) ((The Design Lead starts pointing toward various components in the design)) (17:40): FIG. 6: Episode 5 (E5, November 26th 2015, clip1, 18:50) Design Lead (18:51): If we have a step, you have to work with the fence to make it smoother.Engineer 2 (19:02): Let me see, the electrical wires ((points with his arm and fingers)) this is one circuit, and so if you have steps it might not work?Design Lead (19:09): Err, I mean I appreciate that.I'll see if there's something they [the design team] can address easily.Design Lead (28:40):I mean the construction won't be easy, but the design is not easy either.
[the cables] But this [VR] makes it obvious.So, a client reviewing would not have made the comment.(15:10):

FIG. 9 :
FIG. 9: Participants' ratings of the 3D-MOVE characteristicsBoth the responses to open-ended questions and the participants' comments during the sessions indicate that the technology is most useful for engaging the client and other disciplines to primarily communicate design more clearly.For example, one participant pointed that:"It [the 3D-MOVE] is very useful for CRL reviews at the moment about rail tracks and MEP, if they can go to GATE with it.And all they want to know is about access and maintenance, stuff like this, you know, for them to get a good idea about."(BIM Manager, 10th December 2015, clip 1)

Table 1 :
Data collection overview with the series of observations and participants Project eng.; Certification eng.; Electrification eng.; CAD eng.; Engineer; CAD Lead; IT Eng.; CAD solution expert

Table 2 :
Summary of participants' feedback on the 3D-MOVE usability and further technology improvementsFor example, for clash checking, the participant recommended adding specific functionality features, such as the ability to query dimensions(Engineer, 20th November 2015, clip 2, 20:35).Similarly, other adjustments were suggested to allow access to properties about components (e.g.materials) and annotate, or mark reference elements in the model (FIG.10,Episode E9)."…for example, you could say OK, you need to go and fix THIS one!" ((pointing))

Table 3 :
Areas of considerations for deploying collaborative VR by the groups of practitioners