# TOWARDS USERS' OPTIMAL AND PLEASURABLE EXPERIENCE IN SMART ENVIRONMENTS

EDITED BY : Mi Jeong Kim, Xiangyu Wang and Inhan Kim PUBLISHED IN : Frontiers in Psychology

#### Frontiers eBook Copyright Statement

The copyright in the text of individual articles in this eBook is the property of their respective authors or their respective institutions or funders. The copyright in graphics and images within each article may be subject to copyright of other parties. In both cases this is subject to a license granted to Frontiers. The compilation of articles constituting this eBook is the property of Frontiers.

Each article within this eBook, and the eBook itself, are published under the most recent version of the Creative Commons CC-BY licence. The version current at the date of publication of this eBook is CC-BY 4.0. If the CC-BY licence is updated, the licence granted by Frontiers is automatically updated to the new version.

When exercising any right under the CC-BY licence, Frontiers must be attributed as the original publisher of the article or eBook, as applicable.

Authors have the responsibility of ensuring that any graphics or other materials which are the property of others may be included in the CC-BY licence, but this should be checked before relying on the CC-BY licence to reproduce those materials. Any copyright notices relating to those materials must be complied with.

Copyright and source acknowledgement notices may not be removed and must be displayed in any copy, derivative work or partial copy which includes the elements in question.

All copyright, and all rights therein, are protected by national and international copyright laws. The above represents a summary only. For further information please read Frontiers' Conditions for Website Use and Copyright Statement, and the applicable CC-BY licence.

ISSN 1664-8714 ISBN 978-2-88966-260-9 DOI 10.3389/978-2-88966-260-9

#### About Frontiers

Frontiers is more than just an open-access publisher of scholarly articles: it is a pioneering approach to the world of academia, radically improving the way scholarly research is managed. The grand vision of Frontiers is a world where all people have an equal opportunity to seek, share and generate knowledge. Frontiers provides immediate and permanent online open access to all its publications, but this alone is not enough to realize our grand goals.

#### Frontiers Journal Series

The Frontiers Journal Series is a multi-tier and interdisciplinary set of open-access, online journals, promising a paradigm shift from the current review, selection and dissemination processes in academic publishing. All Frontiers journals are driven by researchers for researchers; therefore, they constitute a service to the scholarly community. At the same time, the Frontiers Journal Series operates on a revolutionary invention, the tiered publishing system, initially addressing specific communities of scholars, and gradually climbing up to broader public understanding, thus serving the interests of the lay society, too.

#### Dedication to Quality

Each Frontiers article is a landmark of the highest quality, thanks to genuinely collaborative interactions between authors and review editors, who include some of the world's best academicians. Research must be certified by peers before entering a stream of knowledge that may eventually reach the public - and shape society; therefore, Frontiers only applies the most rigorous and unbiased reviews.

Frontiers revolutionizes research publishing by freely delivering the most outstanding research, evaluated with no bias from both the academic and social point of view. By applying the most advanced information technologies, Frontiers is catapulting scholarly publishing into a new generation.

#### What are Frontiers Research Topics?

Frontiers Research Topics are very popular trademarks of the Frontiers Journals Series: they are collections of at least ten articles, all centered on a particular subject. With their unique mix of varied contributions from Original Research to Review Articles, Frontiers Research Topics unify the most influential researchers, the latest key findings and historical advances in a hot research area! Find out more on how to host your own Frontiers Research Topic or contribute to one as an author by contacting the Frontiers Editorial Office: researchtopics@frontiersin.org

# TOWARDS USERS' OPTIMAL AND PLEASURABLE EXPERIENCE IN SMART ENVIRONMENTS

Topic Editors: Mi Jeong Kim, Hanyang University, South Korea Xiangyu Wang, Curtin University, Australia Inhan Kim, Kyung Hee University, South Korea

Citation: Kim, M. J., Wang, X., Kim, I., eds. (2020). Towards Users' Optimal and Pleasurable Experience in Smart Environments. Lausanne: Frontiers Media SA. doi: 10.3389/978-2-88966-260-9

# Table of Contents

*04 Editorial: Towards Users' Optimal and Pleasurable Experience in Smart Environments*

Mi Jeong Kim, Xiangyu Wang and Inhan Kim


Jihoon Kim and Ju Yeon Kim

*45 Biomorphic Clothing Sculpture Interface as an Emotional Communication Space*

Young Sun Yoo, Minjin Cho, Jung Sun Eum and Seon Ju Kam

*56 Immersive Experience Model of the Elderly Welfare Centers Supporting Successful Aging*

Eun J. Lee and Sung J. Park

*70 Effective University Facility Management Plan Proposal Reflecting the Needs of the Main Users*

Min Soo Kim and Jun Ha Kim


Jingoog Kim and Mary Lou Maher

*106 Neurophysiological Correlates of User Experience in Smart Home Systems (SHSs): First Evidence From Electroencephalography and Autonomic Measures*

Laura Angioletti, Federico Cassioli and Michela Balconi

*115 Developing Design Solutions for Smart Homes Through User-Centered Scenarios*

Mi Jeong Kim, Myung Eun Cho and Han Jong Jun


# Editorial: Towards Users' Optimal and Pleasurable Experience in Smart Environments

Mi Jeong Kim<sup>1</sup> , Xiangyu Wang<sup>2</sup> and Inhan Kim<sup>3</sup> \*

*<sup>1</sup> School of Architecture, Hanyang University, Seoul, South Korea, <sup>2</sup> Australasian Joint Research Centre for Building Information Modelling, Curtin University, Bentley, WA, Australia, <sup>3</sup> Department of Architecture, Kyung Hee University, Yongin, South Korea*

Keywords: smart environment, human factors, optimal experience, pleasurable interaction, human-computer interaction

**Editorial on the Research Topic**

**Towards Users' Optimal and Pleasurable Experience in Smart Environments**

# INTRODUCTION

The transition from the pre-digital to the digital age has been acknowledged as a fundamental step that changed people-environment relations in contemporary society, with relevant implications and important psychological outcomes in our daily lives (Stokols, 2018). Smart environments are one of the key elements that characterize such transition. The smart environment that incorporates interactivity is an important research theme in the architectural domain. Consequently, the challenges involved in developing intelligent environments that can support a comfortable lifestyle have attracted extensive research attention. Living spaces in which a wide range of smart devices are integrated into building components can support occupants' activities and extend their capabilities. Such integration has affected occupants' cognitive experiences related to their surrounding environments. To provide readers, a view of recent developments in this research area, this special issue focuses on the application of a human-centered approach in the design of smart environments ensuring occupants' optimal and pleasurable experiences. After a rigorous review by experts on the submitted articles, 12 papers are included in this special issue. These articles can be organized into three categories as follows.

Edited and reviewed by: *Giuseppe Carrus, Roma Tre University, Italy*

> \*Correspondence: *Inhan Kim ihkim@khu.ac.kr*

#### Specialty section:

*This article was submitted to Environmental Psychology, a section of the journal Frontiers in Psychology*

Received: *15 July 2020* Accepted: *14 September 2020* Published: *20 October 2020*

#### Citation:

*Kim MJ, Wang X and Kim I (2020) Editorial: Towards Users' Optimal and Pleasurable Experience in Smart Environments. Front. Psychol. 11:583663. doi: 10.3389/fpsyg.2020.583663*

# CONCEPTUAL MODELS AND SCENARIOS FOR SMART ENVIRONMENTS EMPHASIZING EXPERIENCE

In implementation of smart environments, the conceptual models are essential to identify the factors critical to ensuring optimal interactivity, as well as the potential scenarios that effectively create pleasurable experiences. To this end, Kim and Maher presented three conceptual metaphors for designing smart environments—device, robot, and friend—to frame new ways for the interaction design of smart environments. The metaphorical design framework can support designers in creating novel interaction experiences by facilitating the formation of a common mental model for new interactive designs. Further, Kim et al. investigated the ways to control and adapt technology to fulfill the daily needs of users and developed a framework of smart home services from the perspective of supporting user experience. Since the success of smart homes fundamentally depends on their adoption and use by people in the context of daily life, they developed design solutions for smart homes through user-centered scenarios. Moreover, Lee and Park proposed an immersive experience service model for elderly welfare centers by analyzing the health benefits of immersive experience technologies and related services. They showed that the use of these technologies, such as virtual reality and mixed reality, could mitigate physical, and spatial constraints by immersing users into the desired environment and would thus contribute substantially to maintaining the health of the elderly. Interestingly, the potential of smart environments is not limited to the architectural domain but is applied to the clothing area in terms of interfaces to smart spaces. From this perspective, Yoo et al. argued that clothing combined with digital technology can be a wearable interface that adapts to environmental changes and increases biological limits as an emotional communication space. They proposed a conceptual biomorphic clothing sculpture interface, as a knowledge base that using the parametric design methodology from the 3D form development of architecture.

# PROPOSING SMART HOME, FACILITY, INDUSTRY, AND CITY BASED ON USERS' NEEDS

Several types of smart environments are available for areas such as the home, a facility, an industry, and a city. To implement pleasurable smart environments, users' needs should be identified and understood in detail not Only from a developer or a designer's perspective but ALSO from that of those individuals. For this purpose, Lee and Kim critically reviewed smart homes for older adults using an evaluation framework comprising four categories: wellness, independence, acceptance, and design. They asserted that it is important to understand older adults' cognitive and emotional aspects and not just emphasize the efficiency of smart homes. Unlike the general studies on university facilities that have been focused mainly on space management, Kim and Kim proposed an effective university management plan that reflects the needs of the main users—students—to improve their satisfaction, which would lead to the optimal smart environment for them. Further, by focusing on newly developed intelligent devices for assisting elderly people in China, Meng et al. investigated the opportunities and the challenges for the country's elderly care industry in offering a smart environment based on occupants' needs and preferences. Their findings may enable industry stakeholders to make better decisions regarding smart elderly care services. Lastly, An et al. proposed a needs map for service design in smart cities, which can be applied to various stages of service development, since they consider the inclusion of actual service users in the service design a key factor in determining the sustainability of services for smart cities. They demonstrated the feasibility of the proposed needs map by applying it to the service design for the smart city experience zone in Daejeon, Korea.

# EMPIRICAL STUDIES FOCUSING ON INTERACTIVITY AND ENGAGEMENT

Researchers often conduct empirical studies to explore the effects of variables on specific systems or to identify the practicability of proposed ideas in defined areas. Thus, empirical studies that focus on the interactivity and the engagement in smart environments should be performed to ensure that the user experience is optimal and pleasurable. In this regard, Angioletti et al. adopted a neuroscientific multimethodological approach to define the possible effects of smart home systems on user experience and provided evidence for neurophysiological correlates of user experience in such systems. Their study forms the basis for understanding users' responsiveness of these systems in terms of their cognitive and emotional engagement while interacting with a complex system. In addition, Cho and Suh explored spatial color efficacy in an eye-tracking study by examining different applications of the same color combination in a retail interior environment to determine whether these cause different emotional responses, which, in turn, influence viewers' perception of luxury and their intention to stay. Their results provide useful guidelines to designers in planning retail stores to achieve the desirable level of interactivity with users. Similarly, Kim and Kim conducted an eye-tracking experiment to evaluate the effects of background music on the perceived atmosphere of a service setting. Their findings indicate that the differences in scan paths and locations between slow tempo music and fast tempo music change over time. Further, Han et al. focused on smart accessibility to develop a design process of a model that integrates geospatial data from various sources to present user-customized universal design information. They emphasized that the process of communication with users is a key issue and that providing information services using a geospatial database is as important as improving physical accessibility.

# TOWARD PLEASURABLE, OPTIMAL SMART ENVIRONMENTS

This Research Topic specifically focuses on research and case studies that illustrate opportunities and challenges related to human-environment interactive experiences enabled through smart services and technologies. This collection contains new studies that provide cognitive insights on understanding occupants' needs and experiences which might motivate others in this field to develop new cognitive paradigms. All the proposed methods, models, scenarios, and applications in these studies contribute to the current understanding of the smart environment paradigm and are likely to inspire further research on addressing the challenges related to creating pleasurable, optimal smart environments.

# AUTHOR CONTRIBUTIONS

All authors listed have made a substantial, direct and intellectual contribution to the work, and approved it for publication.

# FUNDING

This work was supported by the National Research Foundation of Korea (NRF) grant funded by the Korea government (MSIT) (No. NRF-2019R1A2C1087344).

# REFERENCES

Stokols, D. (2018). Social Ecology in the Digital Age. London: Academic Press.

**Conflict of Interest:** The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Copyright © 2020 Kim, Wang and Kim. This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) and the copyright owner(s) are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.

# Smart Accessibility: Design Process of Integrated Geospatial Data Models to Present User-Customized Universal Design Information

Sooyeon Rosie Han<sup>1</sup> \*, Seunghyun Yoon<sup>2</sup> and Suyeon Cho<sup>2</sup>

<sup>1</sup> Urban Computing Research Lab, Konnektus, Seoul, South Korea, <sup>2</sup> IT Convergence and Security Lab, Pluxity Co., Ltd., Seoul, South Korea

Environmental accessibility information measured by universal design guidelines does not exist in a form that can be effectively implemented as a geospatial database. Thus, this study explored the design process of a smart accessibility data model that integrates geospatial data with environmental accessibility information in a mixed indoor/outdoor environment. First, a typology of accessibility information integrated with universally designed geospatial information was identified, and a field experiment was conducted to observe components of the built environment and environment-behavior interactions during travel in a sightseeing area of Seoul, South Korea. The analysis found that each user group, namely people with mobility impairments, visual impairments, or hearing impairments, had different barriers, and facilitators in the environment. Certain barriers for one group could work as facilitators for another, and vice versa; also, some components previously classified as facilitators failed to actually fulfill that function. Additionally, the user groups demonstrated different prioritization of spatial attributes. The findings of the field study were organized in the data model as priority information and weighted values according to user group. The smart accessibility data model developed in this study has implications for designing user-customized multimodal systems, such as wayfinding services and web-based maps, that are useful to everyone, regardless of their ability or age. Furthermore, it increases the users' decision-making power to plan a trip and can exert invisible pressure inducing physical improvements in the areas that lack accessibility, through visually displaying accurate information on the physical environment.

Keywords: accessibility, barrier-free, geospatial, information model, universal design, user-customized, vulnerable groups, environmental behavior

# INTRODUCTION

Vulnerable groups are more strongly affected by environmental factors that determine an individual's degree of independent life and equitable rights to social participation (Iwarsson and Ståhl, 2003). Various attempts have been made all over the world to improve environmental accessibility, from developing instruments such as guidelines and checklists to measuring actual

Edited by:

Xiangyu Wang, Curtin University, Australia

#### Reviewed by:

Yuchun Huang, Tatung University, Taiwan Ginette Wessel, Roger Williams University, United States

\*Correspondence: Sooyeon Rosie Han braverosie.han@gmail.com

#### Specialty section:

This article was submitted to Environmental Psychology, a section of the journal Frontiers in Psychology

Received: 12 October 2019 Accepted: 12 December 2019 Published: 14 January 2020

#### Citation:

Han SR, Yoon S and Cho S (2020) Smart Accessibility: Design Process of Integrated Geospatial Data Models to Present User-Customized Universal Design Information. Front. Psychol. 10:2951. doi: 10.3389/fpsyg.2019.02951

**7**

environmental conditions, at the local, federal, and national levels, and in collaboration with the private sector. However, since these instruments are not developed with no consideration for the design of a wayfinding system, the information collected is not in a form that can be effectively implemented as a geospatial database, which plays a key role in developing service platforms. Existing geospatial services, such as webbased maps and navigation applications, are thus intended primarily for people without disabilities, as they are based on geometric location data only, which cannot confirm the access, mobility, and safety of vulnerable groups. The few studies dealing with environmental accessibility in geographic information systems (GISs) concentrate only on people with visual impairments (Marsh et al., 2000; Ran et al., 2004; Farcy et al., 2006; Hunaiti et al., 2006) and people who use wheelchairs (Hunaiti et al., 2006; Karimi et al., 2014). They do not adequately provide user-customized information due to the limited source of data relying on governmental standards, focus mainly on outdoor data (e.g., pedestrian accessibility) although the environment is intimately connected by outdoor spaces and indoor spaces, and omit the process by which information should be stored mapped, and managed. Therefore, this study defines the concept of geospatial information for all, identifies typology of environmental accessibility, and finally presents a smart accessibility data model that integrates geospatial data with user-customized universal design information in a mixed indoor/outdoor environment, based on an empirical understanding of complex environment-behavior interactions.

### RESEARCH METHODS

**Figure 1** shows a general overview of the current study's research method, outlining the primary steps required to design the smart accessibility data model.

Ten global guidelines or relevant research papers were reviewed to define the concept of universally designed geospatial information and to identify a typology of universally designed geospatial information, listing both geospatial data and attribute data. Universal design guidelines have been applied at all different scales, from electronic products to transportation systems. This study focused on environmental accessibility in the built environment at the local, federal, and national levels, in collaboration with the private sector, to obtain diverse environmental components.

To fill a gap in knowledge about environmental accessibility information drew in the previous stage and to examine the interaction process between components of the built environment and users with special needs, an observation experiment was conducted in a sightseeing area of Seoul. Significant differences according to user groups were found, addressed in the smart accessibility data model by priority information and values weighted by user group. This study took a microscopic approach to identify individuals' purposedriven mobility process and behavioral characteristics, directly observing and monitoring the reaction and behavior of a subject in particular locations. Observation is an effective method for vividly describing and analyzing subjects' behavior or attitude and has been considered an effective data collection method when the nature of a phenomenon would be difficult to identify through questionnaires alone (Whyte, 2001; John, 2006; Gehl and Svarre, 2013). Behavior-centered observation focuses on the actions and reactions of the observed subject in a physical environment, whereas environment-centered observation focuses on changes in the physical environment based on the detailed evidence that appears there. During observations, it is important to carefully analyze changes in activity caused by the environment, as well as changes in the physical environment from the observer's perspective. However, because it is difficult to control the intervention of unwanted external variables in a natural environment at the discretion of the observer, and because thorough observation of the selected samples over a long period of time is required, it is difficult to quantify and generalize the results of observations. This study thus focused on exploring in detail and qualitatively analyzing the development of a particular situation, which cannot be generalized, based on the physical characteristics of the subjects. Furthermore, to grasp causes of specific behaviors, unstructured interviews were conducted right after the experiment. To verify validity through detailed observation, more than two observers were placed on site for each type of subject.

The final step of the study was data modeling. Unified Modeling Language (UML) class diagrams were adopted to formalize this study's smart accessibility data model. UML is a standard visual modeling language that has been used for the analysis, design, and implementation of software-based systems (Borges et al., 2001; Lisboa-Filho et al., 2010). The class diagram is the main building block of object-oriented modeling in UML, containing object classes, their attributes, and the relationships among objects. In a few significant studies, researchers have integrated information about environmental accessibility with a data model. However, the studies were limited in that the information pertained only to particular groups, such as people with visual impairments or people using wheelchairs due to mobility impairments (Chen et al., 2015; Kasemsuppakorn et al., 2015); in that they incorporated only certain types of environmental information, such as outdoor pedestrian accessibility (Yairi and Seiji, 2007; Laakso et al., 2013); and in that they presented the information model at a conceptual level only, instead of implementing actual field-based information. This study designed a smart accessibility data model intended for all people based on universally designed geospatial information in a mixed indoor/outdoor environment.

# TOWARD "GEOSPATIAL INFORMATION FOR ALL"

# The Concept of Universal Design in Geospatial Information

Universal design was first defined as "simply a way of designing a building or facility at little or no extra cost so it is both attractive and functional for all people, disabled or not" (Mace, 1985).

Since then, the concept has been more broadly described by the European Institute for Design and Disability (EIDD) in terms of the "Design for All" philosophy, which aims to realize accessibility, convenience to all members of society, and responsiveness to evolving diversity in the built environment, everyday objects, services, culture, and information (EIDD, 2004). Similarly, the Health and Places Initiative (HAPI) presents seven principles of universal design: equitable use, flexibility in use, simple and intuitive use, perceptible information, tolerance for error, low physical effort, and size and space for approach and use (HAPI, 2014). Other related concepts include barrier-free and inclusive design, which share a focus in designing everything to meet the needs of all, regardless of ability or age.

This study focused on information describing built environment in universal design approach, pursuing the goal of "geospatial information for all" (**Figure 2**). Geospatial information is data referenced to a place (i.e., a set of geographic coordinates) that can be gathered, manipulated, and displayed in a GIS (Folger, 2010). The information includes both geometric location data and attribute data, or descriptive information about the properties of events, features, or entities associated with a place. Existing geospatial information does not incorporate environmental accessibility information as attribute data, which leaves vulnerable groups unsure of their access, mobility, and safety. This is because environmental accessibility information, which is established and measured by the standards and guidelines of universal design, is not collected in a form that can be effectively implemented as geospatial attribute data. To provide geospatial information for all, information about environmental accessibility must be included, and further, geospatial information should be provided in the form of the "smart accessibility data model" described in section "Smart Accessibility Data Model."

# Identifying a Typology of Environmental Accessibility Information

Several guidelines and standards for environmental accessibility have been established to compile requirements and measure environmental conditions at the local, federal, and national levels, in collaboration with the private sector (**Table 1**). Each guideline is categorized based on its place of application,

#### TABLE 1 | Example guidelines and standards of universal design.

fpsyg-10-02951 December 26, 2019 Time: 16:35 # 4


such as streets or buildings, and each category includes features designed for people with visual impairments or wheelchair users with mobility impairments. The World Tourism Organization (UNWTO) provides an accessibility guideline for tourist attractions worldwide with the goal of realizing "Tourism for All," so that there may be no grievances regarding accessibility or mobility in use of tourism resources (World Tourism Organization, 2013; World Tourism Organization and Fundación ACS, 2015).

Based on examination of the guidelines and standards outlined in **Table 1**, types of spatial and attribute data that needed to be built as a future data model were classified into two lists, for outdoor and indoor environments (**Table 2**). Spatial data for both lists included components of the built environment, such as entrances and vertical access (i.e., stairs, elevators or lifts, and escalators). Each component includes attribute data containing information about accessibility. For example, an entrance consists of a door and a ramp. Doors include attribute data such as the type and width of the door, the direction of opening, the height of threshold, and the type of knob, whereas ramps include attribute data such as the width, gradient, and texture.


# INTERACTION BETWEEN THE BUILT ENVIRONMENT AND SPECIAL USER GROUPS

#### Field Experiment

fpsyg-10-02951 December 26, 2019 Time: 16:35 # 5

Because the dynamic and comprehensive nature of environmentbehavior interactions is difficult to capture through literature review, a field study was conducted in Seoul. The purpose was to classify attribute data by user group to provide usercustomized universal design information; this was done by observing the significant differences stemming from various interactions between the building components and the user groups with special needs.

The locations chosen for this study were situated in an underground shopping mall in Seoul. To focus on interactions during continuous travel from indoor to outdoor space, locations with many connecting spaces were selected, such as subway stations, department stores, the city hall, and public squares (**Figure 3**). The physical environment was studied to map all physical conditions or variables in the targeted space that could influence people's actions, and this study was analyzed along with a floor plan analysis and the field experiment. Prior to the experiment, the floor plan and components of the space and their relationships were examined to investigate the environmental features from the perspective of universal design. The attribute information of the targeted locations was then outlined based on the lists for indoor and outdoor environments derived from analysis of existing guidelines.

The City Hall station was assigned as the starting point and Exit 18 of the underground shopping center was assigned as the arrival point. Participants were encouraged to move and explore freely between these two points, with the restrooms, nursing room, and shops assigned as required stops. For average pedestrians without any physical impairments, the total travel distance is 500 m; the travel time is approximately 15 min at a normal pace when the purpose of traveling is simply to move from the starting point to the arrival point. We speculated that there would be differences in methods of access and travel routes based on the characteristics specific to different user groups. Considering that the subway station and underground shopping center are a connected space for pedestrians, and to elicit a natural flow of movement, no time limitation was set for the study subjects to complete the field experiment. The average travel time spent on the observation experiment was approximately 44 min per person, ranging from 32 to 70 min. The experiment was conducted for 10 days, from September 17 to 28, 2018, during non-rush hours in the morning and afternoon to avoid heavy pedestrian traffic.

# Target Group

All people benefit from universal design, not only those with disabilities. However, this study's experiment targeted people with special needs because the goal was to extract not only geometric location information, which is also accessible to people without disabilities, but also environmental accessibility information, which can change based on physical attributes or needs. To address the full variety of special needs and

FIGURE 3 | Area and environmental features of the field experiment.

complete the list of attribute information present in domestic and international guidelines, which have thus far concentrated only on wheelchair users and people with visual impairments, user groups were roughly categorized into three groups: people with mobility impairments, people with visual impairments, and people with hearing impairments. People with mobility impairments included those who travel in wheelchairs and those who push baby strollers. Elderly people, who manifest complex physical attributes, were excluded from the experiment group, as their needs can be addressed by aggregating the attributes of each group. Attributes and assistive device features by subject type are presented in **Table 3**. To fully understand the characteristics of each user group, all subject types had physical attributes that were distinct from others in the same user group, and each type had more than two subjects. A total of 18 participants (eight males and 10 females; four wheelchair users; four people with babies in strollers; six people with vision impairments; and four people with hearing impairments) volunteered to take part in the experiment. The mean age of the participants was 35, ranging from 27 to 50 years. To verify validity through detailed observation, more than two observers were placed on site for each type of subject. Additionally, to exclude habitual traveling conditions that occur in familiar places, subjects were limited to those who had never been to the target area.

#### Activity Mapping

A researcher conducted a one-to-one follow-up observation of a subject and recorded the observation on the prepared map. The results of follow-up interviews were recorded on the map. Mapping of observation points at a certain location is called activity mapping. The effectiveness of this research method is well-established and it is often employed in urban studies to understand the close relationship between user behavior and the physical elements of the environment (Appleyard, 1969; Gehl and Svarre, 2013). The observation and documentation in activity mapping focuses mainly on the physical access behaviors of the subject on the pathway from the starting point to the destination, such as gaze movements and directional changes at a stop. To ensure the collection of natural and realistic data, the researcher maintained a certain distance from the subject during the observation and documentation, to keep the subject from becoming conscious of the researcher. A sample map and observation documentation for one of the experiment subjects among total thirty-six cases is shown in **Figure 4**.

# Observation Results and Insights

The observation experiment showed significant differences in the needs of each user group and their interactions with the environmental elements. First, each group had a different focus: for people with a mobility impairment, physical accessibility was the greatest issue; for people with a visual impairment, orientation was the greatest issue; and for people with a hearing impairment, comprehensibility was the greatest issue.

Second, each user group had different environmental barriers and facilitators, and the same spatial information that functioned as a barrier to one group could function as a facilitator to another (**Table 4**). For example, whereas the sidewalk curb is a barrier to access for people with a mobility impairment, it is a facilitator for people with a visual impairment, as it helps to distinguish a sidewalk from the road. Similarly, overpasses are a barrier for people with mobility impairments but safer paths than crosswalks for people with visual impairments. Conversely, elevators functioned as a facilitator for people with mobility impairments, but were avoided by people with visual impairments due to difficulty in operation and evacuation. If a building had even a single flight of stairs, people with visual impairments preferred to use the stairs.


TABLE 4 | Barriers and facilitators for each user group.


Third, the importance of attribute data differed by group, even with respect to the same spatial information (**Table 5**). For example, when information on the entrance was provided, door width was the most important detail for the group with mobility impairments, whereas opening type was the most important for the group with visual impairments. Similarly, in TABLE 5 | Prioritization of attribute data by user group.

fpsyg-10-02951 December 26, 2019 Time: 16:35 # 8


the case of restrooms, accessibility and location were the most important pieces of information for the group with mobility impairments, whereas gender separation and orientation was the most important for the group with visual impairments.

Fourth, analysis of some observed points revealed misconceptions regarding some facilitators. For example, a Braille sign for gender marking was installed on the entrance of a restroom, but it was practically impossible for the group with visual impairments to locate, especially those with complete vision loss. Those with partial vision loss were able to locate the Braille sign, but the presence of the Braille was unnecessary because they were able to distinguish a men's restroom from a women's restroom based on the color and shape of the signs. Similarly, wheelchair lifts, which were initially classified as facilitators for people who travel in wheelchairs, were found to have limitations due to potential fall risk, the psychological burden of requesting someone's help to operate the lift, and unwanted attention from other pedestrians when getting on the lift. Therefore, it is imperative to reflect the barriers and facilitators for each user group and the priority of attribute data when designing a data model.

#### SMART ACCESSIBILITY DATA MODEL

#### Data Network

The data model in this study was designed based on the topologically connected network linking the pedestrian path network of the outdoor environment to the corridor path network of the indoor environment (**Figure 5**). The pedestrian path network connects to pedestrian zones, such as parks and squares, by transition spaces; to buildings by entrances; and to train stations or bus stops by modality linkage. In other words, transition space connects open outdoor locations where it is difficult to define a specific entrance; entrances connect outdoor locations to indoor locations, or connect between indoor locations with a specific entrance area; and modality linkage connects places where the modality of movement changes, such as from walking to taking public transport. Similarly, the corridor path network in a building connects rooms or restrooms through entrances.

#### Data Structure

The data structure for providing user-customized universal design information is divided into universally designed geospatial data and user data (**Figure 6**). The universally designed geospatial data encompass not only location data, geometry data, and unique ID, which can identify the space by its purpose, but also accessibility, safety, and weighting information. Location data are divided into indoor data and outdoor data. When a subject is outdoors, geographic coordinates and altitude are displayed based on GPS information; when a subject is indoors, location information is provided using relative positioning based on the floor plan. Geometry data define GPS points that can be either two-dimensional (x, y) or three-dimensional (x, y, z). Environmental accessibility information includes attribute data extracted from the aforementioned typology of accessibility such as threshold height and braille sign. The data comprise safety information (referring to user data) and

weighting information to evaluate the prioritization of attribute data by user type.

**Figure 7** shows the complete smart accessibility data model with inheritance relationships and associations. The model is divided into two top-level classes: Pedestrian Accessibility and Building Accessibility. The Pedestrian Accessibility class is inherited by the child classes Pedestrian Path, Pedestrian Zone, Transition, and Modality Linkage. In addition to typical attribute data, such as geometric type and latitude and longitude, the Pedestrian Path class includes attribute data for pedestrian path type, surface material, gradient, width, presence of a Braille block and curb, and weight type. Moreover, the Pedestrian Path class has relationship with Pedestrian Crossing, Curb, Stairs, Lift, and Elevator. The Pedestrian Crossing class has special attributes, such as presence of an acoustic signal, because this is important information for people with visual impairments. The Curb class can be associated with both the Pedestrian Path and Pedestrian Crossing classes due to level changes in the crosswalk area. The Stair class has attributes for the presence of a handrail and Braille sign and the number of steps, which is important information for people with visual impairments, and also ramp width and gradient, which is important for people with mobility impairments. The term "lift" is interchangeable with "elevator" in some countries; however, in the field experiment area of this study, the lift is an assistive device, distinct from an elevator, installed next to a staircase to aid the mobility of wheelchair users. The field experiment found that the wheelchair lift limited in functioning as a facilitator due to safety issues; therefore, it was not classified with a plus sign in weight type.

Type information in attribute data is defined as enumerations (**Figure 8**). For example, pedestrian path type is enumerated according to sidewalk, mixed road, and walkway, in that the risk level varies according to user type. Weight type includes user types such as visually impaired, mobility impaired, and hearing impaired, with plus and minus signs referring to information in the enumerations of barriers and facilitators (**Figure 9**). The weight value plays an important role in providing usercustomized information because barriers and facilitators differ by user type. For example, certain barriers for people with a mobility impairment (e.g., a curb or pedestrian overpass) function as facilitators for people with a visual impairment; and even within the same spatial component, the priority of attribute data differs by user type, as captured from the field experiment.

Han et al.

**16**

FIGURE 9 | Weight type information for barriers and facilitators.

#### DISCUSSION AND CONCLUSION

fpsyg-10-02951 December 26, 2019 Time: 16:35 # 12

This study was driven by three research aims. The first was to identify a typology of environmental accessibility information as geospatial attribute data by analyzing national universal and barrier-free design guidelines and standards. The second was to explore the different information needs and interaction methods of various user groups with the environment through an observation experiment. The analysis found that each user group, namely people with mobility impairments, visual impairments, or hearing impairments, had different barriers and facilitators in the environment. Certain barriers for one group could work as facilitators for another, and vice versa; also, some components previously classified as facilitators failed to actually fulfill that function. Additionally, the user groups demonstrated different prioritization of spatial attributes. The final research aim was to design a method to store and present information about environmental accessibility in a systematic way, resulting in smart accessibility data model that integrates geospatial data with user-customized universal design attribute data in a mixed indoor/outdoor environment. It should be noted that only static information was included as attribute information in the aggregated data of universal design. Dynamic information, such as construction status or floor condition based on the weather, was excluded.

To supplement and complete the existing universal design guidelines, which have accumulated data concentrating mainly on wheelchair users and people with visual impairments, this study's experiment included participants pushing a baby stroller and people with a hearing impairment. In future, the range of participants can be extended to many other user types. Conversely, it is possible to develop an optimal approach by conducting more in-depth research on a particular user group.

The process of communication with users is a key issue, but faces limitations in displaying numerous pieces of attribute information using the conventional method. For example, people with visual impairments have limited access to visual maps; people with hearing impairments have a limited literacy and have limited access to voice assistance services. Thus, it is imperative to conduct a follow-up study on a multimodal system that provides

#### REFERENCES


information in the optimized way for users based on the smart accessibility data model presented in this study.

Providing information services using a geospatial database is as important as improving physical accessibility itself, which can take a relatively long time. It not only increases the users' decision-making power but also visually displays accurate information on the physical environment, and can exert invisible pressure inducing physical improvements in the areas that lack accessibility.

### DATA AVAILABILITY STATEMENT

All datasets generated for this study are included in the article/supplementary material.

#### ETHICS STATEMENT

Ethical review and approval was not required for the study on human participants in accordance with the local legislation and institutional requirements. The patients/participants provided their written informed consent to participate in this study.

#### AUTHOR CONTRIBUTIONS

All authors contributed to the conception and design of the study, and manuscript revision, performed the content analysis and field study, and read and approved the submitted version of the manuscript. SH wrote the first draft of the manuscript. SY and SC wrote sections of the manuscript.

#### FUNDING

This research was supported by the Ministry of Culture, Sports and Tourism (MCST), and the Korea Creative Content Agency (KOCCA) through the Tourism Research and Development Program 2019.


networks. EAI Endorsed Trans. Collab. Comput. 1:e5. doi: 10.4108/cc. 1.2.e5


Annual Conference on Pervasive Computing and Communications, 2004, Orlando, FL.


**Conflict of Interest:** SH was employed by the company Konnektus. SY and SC were employed by the company Pluxity Co., Ltd.

Copyright © 2020 Han, Yoon and Cho. This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) and the copyright owner(s) are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.

# A Critical Review of Smart Residential Environments for Older Adults With a Focus on Pleasurable Experience

#### Li Na Lee<sup>1</sup> and Mi Jeong Kim<sup>2</sup> \*

*<sup>1</sup> Department of Software and Information Systems, University of North Carolina at Charlotte, Charlotte, NC, United States, <sup>2</sup> School of Architecture, Hanyang University, Seoul, South Korea*

Various smart services and technologies have been developed to support older adults' well-ness, make their daily tasks easier, and enhance their overall quality of life. When people grow older, older adults inevitably experience a significant decrease in their physical, cognitive, and sensory capabilities, which makes them develop negative attitudes toward technology. In this regard, this study highlights that older adults require not only usable and practical spaces but also smart residential environments that can fulfill them emotionally. Research on smart environments for this population should consider the hedonic and experiential factors of interacting with technology, such as fun, fulfillment, play, and user engagement. This study aims to provide a comprehensive review of smart residential environments to support positive aging and pleasurable user experience in the architecture domain. For this critical review emphasizing the pleasurable smart environment, an evaluation framework was developed, consisting of four categories: well-ness, independence, acceptance, and design. Through an extensive analysis of selected papers in the architecture domain, it was found that studies on the smart home tend to focus on utilitarian factors, such as usability, monitoring physical experiences, and simulating energy efficiency, and rarely mention psychological well-ness. Smart environments should be designed to not only emphasize efficiency, effectiveness, and satisfaction but also to engage older adults and provide them positive experiences. As various smart technologies continue to evolve and integrate into smart living spaces, it is important to understand older adults' cognitive and emotional aspects and make the smart environment a more comfortable place for them.

Keywords: smart environment, smart home, aging in place, older adults, pleasurable experience

# INTRODUCTION

A "silver tsunami" is on its way. Silver tsunami refers to the rapid aging of the population and, in particular, of the baby boomer generation. This demographic shift has moved the focus of researchers, designers, health care providers, and policymakers from ascertaining ways to extend the lifespan to ways to improve the quality of life. The aging population will face attitudinal, environmental, and institutional changes in their later life (Bartels and Naslund, 2013). Living arrangements are dynamic because of changes associated with life events, such as widowhood,

#### Edited by:

*Seungkwang Shon, Dongshin University, South Korea*

#### Reviewed by:

*Ning Gu, University of South Australia, Australia Hyun Joo Kwon, Pusan National University, South Korea*

> \*Correspondence: *Mi Jeong Kim mijeongkim@hanyang.ac.kr*

#### Specialty section:

*This article was submitted to Environmental Psychology, a section of the journal Frontiers in Psychology*

Received: *18 October 2019* Accepted: *29 December 2019* Published: *24 January 2020*

#### Citation:

*Lee LN and Kim MJ (2020) A Critical Review of Smart Residential Environments for Older Adults With a Focus on Pleasurable Experience. Front. Psychol. 10:3080. doi: 10.3389/fpsyg.2019.03080*

retirement, and changes in living environment. Age-related changes prevent older adults from actively participating in activities that are essential for their well-being. Owing to the rapid increase in the number of older adults, numerous smart technologies are designed to promote their well-being including various health applications to support physical activities as well as sensor-based networks for monitoring activity—and to help these individuals stay in touch with family and friends (Morris et al., 2013; Yang et al., 2013; Lloret et al., 2015; Liu et al., 2016). Further, various studies are being conducted on the positive impact of smart technology on living environments, routine life activities, and maximizing the independence of older adults (Horgas and Abowd, 2004).

The concept of smart home has developed rapidly since the early 1990s, and various studies have been conducted in this research field. The smart home (also termed the intelligent home, aware home, and adaptive house) is a residence equipped with technologies that include sensors, wired and wireless networks, and intelligent systems (Bitterman and Shach-Pinsly, 2015). Over the past decade, smart home technology has increasingly targeted people having reduced capabilities due to age or disability. It was initially focused on increased security and energy savings, whereas, recently, the scope of its use has gradually varied to enhance the overall quality of life. However, the application and use of such technology in living spaces is limited. Moreover, few studies have considered older people's understanding of challenges or barriers that may occur in smart living and their acceptance of this concept. Their acceptance of assistive homebased technology is dependent on the complex relationships between cognitive and emotional components (Lê et al., 2012). We believe the aspects considered in designing smart residential environments for older adults in later life should include engagement and positive affect. Smart residential environments for older adults should be designed not only with an emphasis on providing efficiency, effectiveness, and satisfaction but also on engaging them and providing positive experiences. In this study, we focus exclusively on the living environment for older adults to promote pleasurable experiences. A pleasurable experience is defined in this study as one that allows aging occupants to have considerable amount of fun while living in a smart home and interacting with a variety of smart devices.

A variety of studies on the smart environment for older adults have been published in research disciplines, such as electrical engineering, information technology (IT), computer science, gerontology, biomedicine, and robotics, since the early 1990s (Bitterman and Shach-Pinsly, 2015). Although the smart environment involves the space in which human beings reside, the research areas of architecture and town planning have been less involved with smart home research. Research on the smart environment in the architecture domain tends to focus on utilitarian factors, such as ensuring usability, monitoring physical experiences, and increasing energy efficiency, and rarely considers users' emotional well-ness. The question is no longer only whether smart technologies are efficient, effective, or usable but how well these are able to engage older adults and provide them a positive and pleasurable experience as regards smart living. A smart environment in which older adults can experience emotional pleasure is required. To make smart homes more acceptable, by turning resistance into appreciation and providing pleasurable and positive experiences, technological understanding should entail spatial recognition. This study critically reviews the literature on smart environments that were published in the architecture research domain, using an evaluation framework consisting of four categories: wellness, independence, acceptance, and design. This evaluation framework of smart environments for older adults has enabled a critical review of the selected papers. A Harvey Ball has been used to analyze the value of the smart environment of each paper based on the four categories indicated in the evaluation framework. As various smart technologies continue to evolve and integrate into smart living spaces, it is important to understand older people's cognitive and emotional aspects and make the smart environment a more comfortable place for them.

# PLEASURABLE SMART RESIDENTIAL ENVIRONMENTS FOR OLDER ADULTS

Various types of living environments are available for older adults. Most older adults prefer to live independently in a familiar home setting as long as possible (Eckert et al., 2004; Boldy et al., 2011). This lifestyle choice is known as "Aging in Place (AIP)." Assisted living facilities (ALFs) are characterized as a housingand-services setting to maintain safer, healthier living conditions (Horgas and Abowd, 2004). AIP enables older adults to live in a familiar environment and maintain current social networks and social interactions with close family members Further, their mental functions remain healthy through interactions with their friends in that environment. In addition, a comfortable, familiar environment provides them emotional stability. In the case of ALFs, price may be burden, but tailored services can lead to a more convenient life (Horgas and Abowd, 2004). Providing a safe, secure, and comfortable living environment is important to improve the well-being and happiness of older adults (Perez et al., 2001; Sabia, 2008). Thus, understanding the functions needed to support well-being based on their patterns of living and behavioral characteristics is critical (Costa-Font et al., 2009).

Many studies have emphasized the need to use information and communications technology (ICT) technologies (Deen, 2015), to provide appropriately designed living environment suitable for older adults fitted with embedded sensors and voiceactivated services (Ding et al., 2011). The development of ICT helps users to take control over smart technologies in the home (Kerbler, 2016). This idea is generally known as ambient intelligence or the innovative and smart environment. Such an environment combines modern computing, networking, and smart and innovative devices by helping users to communicate with homes and other users through special interfaces in general. Within the context of the smart environment, numerous sensors are connected to the individual's home. These sensors can measure physical and physiological functions and monitor all activities, and they provide the user real time warnings on devices that malfunction. A smart home is an environment that adopts ICT to collect and share information, analyze and monitor residents' behavioral patterns, and improve residents' quality of life (Courtney, 2008; Balta-Ozkan et al., 2013). It is clear that smart technology has a positive effect on overall life because it helps older adults to easily perform essential activities in the space with minimal energy. However, caution needs to be exercised regarding the spaces in which all the functions are automated since they can make older adults frailer by limiting their movements. Deeper consideration is required on the approach to be adopted to automate intelligent systems in such ways that help older adults remain physically and mentally active. A balance needs to be found to provide the right environment without being too intrusive.

Currently, older adults are being exposed to technologies and increasing their experience as regards using interactive technologies with the assistance of the younger population. For information access using technology, a typical human being needs perceptual, motoric, and cognitive capabilities. However, as people age, their cognitive, physical, and sensory capabilities tend to deteriorate (Abegaz, 2014). Frequently mentioned concerns are high cost and privacy implications. Additionally, older adults may believe that smart home technologies are difficult to control or impractical to use (Peek et al., 2014). Participants in preimplementation studies also expressed concerns regarding the burden it may put on their children in their role as caregivers (i.e., causing workload or worrying) and the possible negative effects on their personal health (Rush et al., 2013; Lee and Coughlin, 2015; Peek et al., 2017). Usually, older adults are afraid of innovation and modern technology; they suffer from so-called technophobia (Sponselee et al., 2007; Booker, 2011) because they are not yet ready to change their mindset toward technology. Older adults who did not grow with current technologies have difficulty accepting smart technologies (Peek et al., 2014). Researchers should carefully examine the ways in which they can help older adults realize the fact that modern technology can help them to be safe within their homes, with independence.

Many studies have been conducted on the smart environment, smart home, and smart technologies designed for older adults or people with disabilities. According to Demiris and Hensel (2008), studies on using smart technologies consider a restricted number of settings based on communities or laboratories, in which the suggested technological innovations demonstrate a high level of feasibility. This study shows that 71% consider technologies for functional monitoring, 67% for safety monitoring, 47% for physiological monitoring, 43% for cognitive support or sensory aids, and 19% for monitoring security, whereas only 19% focus on ways to increase social interaction. The research conducted was directly related to testing and analyzing technological advancements, devices, and sections in the systems of innovative built living environment, that are connected with ensuring control and safety, identifying the activities of users, sending reminders, and evaluating their physiological functions. It should be noted that an insufficient number of studies are related to the analysis of activities and needs of people according to their physical or mental abilities (Cesta et al., 2007; Pecora and Cesta, 2007). One of the needs that older adults have is being independent. Therefore, it is a matter of crucial concern to take into account the perceptions of older adults in terms of smart home technologies since the latter are the tools of improvement needed for quality aging. To provide a pleasurable smart environment, a human-oriented approach should be adopted, rather than a technology-oriented one.

Smart spaces are needed that can provide older adults with pleasurable experiences beyond just usable and practical spaces. We believe that the smallest space design retrofit can have a potentially lifesaving impact. Simple design choices can provide older adults a safer living environment in the same living space to which they are accustomed, and hence enable them to live longer. For this purpose, not only technological interventions but also the roles of design professionals are important. For older adults, a house is not merely a living space. It has diverse sensory and emotional experiences, such as memories, temperatures, smells, and familiarity with spaces. Emotions are a very important part of human life, because they have power to influence the way we make decisions, evaluate risks, solve problems, focus our attention, find something interesting, and categorize information. Such spaces that affect these emotions may make the users want to continuously stay there, and the positive emotions they receive from the space will play a significant role in promoting the well-being of older adults.

# METHODOLOGY

#### Selection

Research on smart homes and smart environments is being conducted to support well-ness for older adults. Most such studies have focused on support of those over the age of 60, using personalized smart services that enable monitoring and tracking via wearable or implanted sensors. We primarily conducted search through Google Scholar, using terms that included multiple ways of describing older adults (e.g., "older adult"; "senior"; "elder") and smart homes, such as "smart home," "smart living," "assistive," "smart technology," "health smart," "intelligent living," "intelligent building," "smart environment," and "smart technologies to support healthy aging." We selected papers in the architecture domain only. We examined other sources, including reports, websites, and relevant newspaper articles, to conduct a critical review of smart environments focused on older adults' well-being. A variety of studies on the smart environment for older adults have been published in research disciplines, such as electrical engineering, IT, computer science, gerontology, biomedicine, and robotics. However, we found it relatively difficult to find a substantive amount of formal literature about smart home and smart technology for older adults in the architecture domain. Since the studies on smart environments for older adults published in architectural journals are very scarce, this study included all studies on smart environments even though some did not have older adults as their targets. Among articles on smart environments in the architecture domain, we reviewed 50 publications since 2000, and 30 of these articles were excluded because these are cases focusing on only technical aspects or simulations. This review paper focuses more on design aspects than on technological solutions. Finally, we selected 20 papers for the purpose of identifying factors that should be considered in developing a pleasurable smart environment for older adults.

# Contextual Analysis

We developed a contextual analysis framework for the first stage of the critical review, aiming to extract principal factors that focused on a smart environment to enable pleasurable experience for older adults (**Table 1**). For each selected paper, we focused on analyzing the following aspects: the demographic considered, the technological features used, the smart environment context focused on, the user experience provided, the methods used in the research, and the ultimate purpose of the research.

As a result, providing a healthy, safe, and secure environment and monitoring activities are primary part of smart environment research in the selected paper. Many studies show that saving energy is often an additional benefit for older adults. Whereas, some studies focus exclusively on older adults, the remaining tend to frequently mention older adults because of the characteristics of the studies although they did not specifically target older adults. The proportion of studies using the selfreporting method is higher than that of exploratory studies. Most studies on designing smart environments were conducted on individual spaces; however, a few studies have been extended to the community and environmental level. This evaluation framework was established by gathering all the components considered in selected papers on the smart environment. These components were then categorized into four main categories related to the pleasurable experiences of older adults connected to the smart environment that supports AIP. This analysis is a basic work to extract factors comprising the evaluation framework in section Evaluation framework.

#### Evaluation Framework

We developed the evaluation framework by analyzing components of smart environments through the contextual analysis and extracting factors to provide pleasurable experiences in smart environments to enable older adults to live independently and to promote a sense of overall wellbeing in them (**Table 2**). The categories for evaluation framework were in divided into four: Well-ness, Independence, Acceptance, and Design. Through the contextual analysis, we found that new technologies make space smarter and advance independence to promote the well-ness of older adults (Vacher et al., 2011; Lattanzio et al., 2014).

The ultimate goal of the smart environment commonly referred to in all the selected papers is to promote the wellbeing of older adults. We identified the critical factors of a pleasurable smart environments focused on preserving physical, psychological, and environmental well-being. For a pleasurable smart environment, providing a secure, and safe environment to promote the physical and emotional health of older adults is essential. Environmental well-ness involves considering the interaction with not only the natural environment but also the social environment. It is difficult to find smart spaces developed for the purpose of providing fun and interesting experiences to older adults in the papers selected. However, the aspect of fun must be considered to promote emotional well-ness for a pleasurable smart environment.

Older adults strive to maintain their independence and autonomy at the end of life. They prefer to maintain a greater degree of personal independence with help from family, friends, or caregivers. For older adults to control their own lives independently, it is necessary to automate the spaces in which they live (Demiris, 2008, 2009; Helal et al., 2008), particularly when they are too physically frail and too impaired cognitively and hence unable to manage life independently. Automated spaces designed with the idea of affordance can help them to understand the use and function of the smart living environment (Norman, 1999). In the process, it is necessary to support their physical and cognitive functions and to provide usable, affordable options.

The attitude of older adults in adopting smart technologies is also discussed as an important factor for them to enjoy well-being while leading independent lives (Courtney et al., 2008; Pal et al., 2018). Various studies are attempting to understand the characteristics of older adults and apply them to design. To develop a pleasurable smart environment for them, it is important to understand key factors that influence their acceptance of smart technologies. Further, to induce older adults to accept smart technologies, studies must consider factors evoking positive emotions during the use of smart technologies and appliances, providing sustainable smart environments, giving benefit from the use of smart technologies and providing experiences that consider users' needs (Hargreaves and Wilson, 2017).

The aforementioned factors are crucial in considering the design of a future smart residential environment. Designs that demonstrate effective grasp of the characteristics of the elderly can provide pleasurable experiences. The use experience of the elderly in smart homes depends on how the smart home is designed (Eggen et al., 2017). The design should not only meet the needs of the individual using the space but should demonstrate a detailed understanding of the relationships between the individual, community, and environment. The evaluation framework for smart environments for older adults required a critical review of selected papers.

# CRITICAL REVIEW OF SMART ENVIRONMENT FOR OLDER ADULTS

#### Well-Ness

The overall goal of smart home research is to enable older adults to live independently at home as long as possible for promoting their physical, psychological, and environmental well-ness. In this section, the following four critical aspects for promoting well-ness in smart environments are included: Safety, Health, Interaction, and Fun.

The smart home concept was originally developed with a focus on improving security and energy saving (Chen et al., 2010). In the previous decade, the aim of smart home technologies has gradually expanded to include the purposes of assisting people with disabilities, older adults, and those with reduced

#### TABLE 1 | Contextual analysis of smart environment factors from each paper.


*(Continued)*

#### TABLE 1 | Continued



capabilities to enrich the living environment, improve comfort and facilitate well-being (Stefanov et al., 2004; Demiris et al., 2009; Ding et al., 2011). Smart homes increase domestic comfort, convenience, security, and leisure as well as reduce energy use through optimized home energy management (Hargreaves et al., 2018). Chen et al. (2010) state that safety, health, sustainability, and convenience are suggested as being the four major goals in the development of an intelligent living space policy. Lee et al. (2013a) analyses the efficiency of a smart service for bedrooms aimed at supporting the older adults' behavioral needs. The focus of the smart service patterns is on emergency, convenience, and health preservation. In particular, one of the needs of older adults is a service associated with emergencies and management of health in the smart home. Kim et al. (2009) studied older adults who need technologies to assist their daily activities at home. The research included ranking of the categories covered by the technological advancements according to their value, and the older adult participants referred to security and safety as the top value. Aging causes a wide range of physical problems and impairments that create an urgent need in new technological systems for older adults that would facilitate safety and maintenance of health under the conditions of the smart environment. In this sense, improving physical health, security, and energy savings are considered important factors when designing a smart environment.

Although these health and safety aspects are important for enriching aging life, happiness is also an essential concept of well-being within the emotional well-ness of older adults. The contribution of technological systems toward improving the quality of life for older adults should focus on social and fun aspects as well. Behr et al. (2010) stated that older adults' active participation in social activities and the establishment of their sense of belonging as a social member have important effects on successful AIP. Home features hardwired with smart communication technology can be used to encourage their social connectedness and prevent those living independently from feeling isolated. In this regard, Mahmood et al. (2008) conducted many pilot studies to find the answers to the questions: How do older adults perceive the use of technology? What factors influence their perceptions about using communication and monitoring technology? According to those studies, smart technologies take into consideration direct communication with friends and family, which facilitates the improvement of health and supports the emotional balance of older adults. Park and Kim (2018) introduced a semi-supervised named entity recognition system for extracting execution targets from natural language commands. They focused on voice-activated human–interface systems in a smart home. They show the possibility that because smart appliances can understand the user's language and react immediately, they can conduct simple conversations with older adults living alone at home. This prevents older adults from feeling alone and isolated. This system can not only enhance the usability of smart technology but also arouse interest in users. Although many studies still emphasize health and safety as the key features of a smart home, we should not overlook the importance of social and fun aspects.

#### Independence

Considering automation capabilities within the smart environment to help older adults who are physically frail and cognitively impaired is important to enable them to live independently and to improve their well-being. In this section, the following four critical aspects for supporting their independence in the smart environment are included: Automation, Affordance, Physical support, and Psychological support.

To support the independence of older adults, smart home systems provide automation capabilities that allow them to have control over their living environment and monitor it. According to Yu et al. (2019), in a smart home for older adults, the technologies of unobtrusive sensors can be applied, which facilitate providing health care services and help in evaluation of their daily activities. Thus, it is possible to gather data regarding them with no intrusion into their daily life and routine (Wickramasinghe et al., 2017). This approach can maintain the privacy of older adults in their living space, and their normal everyday life remains the same. Several devices are integrated into smart homes, in particular, for video surveillance, intrusion detection, entertainment, smoke and fire detection, and health monitoring. Many of these devices use different communication protocols with various levels of abstraction that are incompatible with each other (Kim et al., 2017). Kim et al. (2017) proposed an extensible OSGi-based architecture to ensure effective integration of different smart services and devices. These smart sensing technologies demonstrate the possibility of providing physical and psychological support by collecting information about the occupants' behaviors and predicting the behavior patterns in smart environments (Spataru and Gauthier, 2014). Jalal et al. (2013) proposed human activity recognition methodology from recognized body parts of human depth silhouettes to monitor services at smart homes.

Sensing technology and solutions for managing smart devices are required to automate everything in the house and make the living space smart. However, using smart systems can be an extra burden for older adults. It is important to help older people learn to use smart systems independently by informing them about system functions. Many studies (Park, 2008; Darby, 2010; Cho et al., 2013; Maher and Lee, 2017) have considered the idea of affordances to provide a clear perception of possible interactions between householders and artifacts in smart environments. Smart technologies should be designed that allow older adults to communicate with technologies easily, rather than having to learn complex technical languages and commands to support their independent living (Hargreaves et al., 2018). Cho et al. (2013) explained that if the functional and physical environment is well-designed, it provides affordance of smart workplaces, and hence, it is possible to use smart spaces more effectively, since users' understanding of space use increases. Smart appliances presenting in spaces are mapped to behavioral patterns suggested by participants. For example, lamps are automatically turned on, if sensors attached to doors detect that the doors are opened. If smart services are customized to match older adults' behavioral patterns, it is possible to support their physical and psychological independence. The idea of Lee et al. (2013a) is to introduce a smart service on the basis of connecting the behavior of older adults within a space. It should be a service that is customized for a particular smart home and behavior of individual adults and should not be limited to physical assistance alone but should include psychological and social support. Creating and customizing intelligent environment services is important to provide physical and psychological support for older adults.

#### Acceptance

Many older adults believe that their independence can be facilitated by their use of smart home technologies, yet these conditions often do not translate into a willingness to accept smart home technology (Courtney et al., 2008; Demiris, 2009). In this section, the following four critical aspects (Positive experience, Sustainability, Perceived usefulness and benefits, and Need finding) are discussed to understand the use and acceptance of smart technology by older adults.

Most previous smart home research has explored the technical challenges of delivering smart domestic environments (Cook, 2012). The majority of this work has not focused on users and their requirements. The recent research demonstrates a growing interest of older adults in devices for smart spaces (Mennicken et al., 2014; Wilson et al., 2015), but determining the aspects older adults like and the solutions suitable for them are issues to be resolved. To design the smart environment in which users may be interested and that they would find acceptable, it is important to properly understand the use and concept of each space constituting a smart house and arrange the smart technologies depending on users' needs. Older adults should evaluate the features and effects of living in a smart environment based on how they perceive and understand it. Cho et al. (2013) identified the key concept and attributes for designing smart workspaces around activities conducted in such spaces based on users' needs and preferences. The approach is unique in that it shifts its focus from the elder people to the pre-elder people. The latter have characteristics that differ from those of older adults. The future development of smart services should be considered by focusing on the needs, technical dispositions, and preferences of pre-elder people. Hargreaves et al. (2018) identified when the users are motivated to use smart technology through in-depth interviews by conducting a longitudinal study. Darby (2010) also pointed out that learning from user experiences is significant in designing a smart environment. According to Lee et al. (2013b), ensuring customization of suitable services for smart homes is complicated because IT developers work on the basis of their own understanding instead of taking into consideration the needs, psychology, and behavior of the actual residents. As a result, residents do not experience the expected level of satisfaction or quality of life on using smart home services. Therefore, performing a need finding process when designing smart environments for older adults is important for providing a satisfactory experience.

According to Chen et al. (2010), users' acceptance appears to be affected by perceived usefulness and perceived enjoyment. Hargreaves et al. (2018) identify that the task of learning how to use smart technologies is demanding and time-consuming. They point out that older adults still do not understand the benefits of smart technologies. No matter how helpful smart systems can be for their daily life, these systems are useless if older adults do not use them. Efforts should be made to make them fully understand the benefits of using these systems. Hargreaves et al. (2018) indicate that a clear understanding of technological and human factors is necessary to design for smart living. In addition, it is difficult to validate the total effect of diverse smart technologies, since smart homes have often been studied in laboratory settings, and hence, they have been rarely applied to the real world. They study to facilitate the design of intelligent space based on an assessment of user needs. It is necessary to study how smart spaces can be accepted by older adults. Chen et al. (2010) introduce the technology acceptance model used to evaluate the complexity and the dynamics of users' perceptions. It emphasizes that perceived enjoyment has positive effects on users' acceptance. Older adults demonstrate a positive attitude toward adapting to new technologies in their residential environment. However, their preferences as regards the control methods for technological systems showed limitations in their adaption to new detailed techniques. One of the desires older adults expressed concerning technological system control was an easy interface that resembles that of television and a remote control pad (Kim et al., 2009). This result showed that the design of technological smart systems for older adults should incorporate easy, user-friendly control without any complicated menus. It is important to provide positive experiences in the smart environment. In this sense, design, and planning considerations could be suggested on the basis of the understanding of older adults.

Smart home devices designed to make homes more sustainable could allow the aging to maintain independence (Skjølsvold and Ryghaug, 2015). Some studies associate architectural design with the choice for materials from sustainable sources, indoor air quality, energy efficiency, and productivity (Barbosa et al., 2016). A smart heating control system is one of the smart devices to which users can have easy access. It is already actively used in many homes because it has the advantages of low price and ease of use. The advantage it offers of saving energy through a simple operation positively affects users, leading them to accept the system without any sense of repulsion (Dimitrokali et al., 2015). Barbosa et al. (2016) studied the application of effective technologies of interior design that can change the living space based on the introduced improvements in environment sustainability combined with the principles of green building. A popular current tendency is using smart interior design with flexible furniture and movable walls in compact apartments. Smart interior design techniques enable saving on building resources and materials, which consequently leads to the reduction in energy required for heating, lighting, and air conditioning. It is possible to transform the living space rather quickly with the use of RoboWalls electric motors that can be controlled with instructions given through a computer interface, smartphone application, or direct voice commands. Such techniques may be an adoptable solution to support independent living for older adults.

#### Design

Active research on smart technology and integration of devices into the living environment started as far back as the late 1990s. Regarding the future design and development of smart environments, it is clear that users need to be better accounted for or actively drawn into the design and development process. In this section, the following four critical aspects are included: Human-centered approach, Individual level, Community level, and Environmental level.

The importance of understanding the needs of older adults has been mentioned several times in section acceptance. In addition to understanding these needs, it is necessary to examine the ways in which such understanding can be applied in space when designing a smart environment. Kymäläinen et al. (2017) presents human-centered co-design process with users as an approach for studying intelligent environments. They introduce a variety of design methodologies, such as persona, user scenario, and paper prototypes, to understand usage situation. The design is evaluated through observations, focus group discussion, and interviews. This is the human-centered approach typically used in designing user experiences in the field of human–computer interaction, which can actively reflect needs of users who directly use intelligent spaces, leading to increased acceptance of smart homes. It suggests that it is important to understand users' needs to increase their acceptance. By using the persona called Alice, it introduces a multifaceted design process and challenges, to provide an appropriate design to the persona. Such a designoriented study should be actively conducted in the field of architecture. Chien and Wang (2014) pointed out that smart technologies should be customized according to user's needs and preferences as part of the human-centered approach. This approach will help to identify how smart technologies are being used and can be integrated into existing buildings in an effective way (Darby, 2010). Adopting this approach is necessary to make users consider the designs and implementation of smart technologies.

The benefits of home automation to a society could be so much more if smart homes were scaled into fully connected smart communities (Behr et al., 2010; Li et al., 2011). Smart city technologies such as smart mobility management tools, smart transportation tools, smart energy grids (Darby, 2010), and etc. can make living environment more effective and efficient. A smart environment should not only help older adults with the activities necessary for life in convenient ways but also provide them with multisensory experiences so that visual, aural, tactile, olfactory, and gustatory senses are stimulated appropriately (Clements-Croome, 2005). The built environment for older adults needs designs that can increase contact with natural light or the external environment. When designing smart environments, it is necessary to pay attention to the activities in cooperation with the community and environment surrounding human beings, rather than focusing simply on the living space used by humans. To understand how to create more productive environments, it is important to understand how humans use spaces. How spaces are designed have a significant effect on air distribution, acoustic quality, natural ventilation, and the amount of daylight (Clements-Croome, 2005), which in turn have a major effect on the fundamental quality of life for humans. In addition, spaces can change a user's patterns of life and behaviors. Various experiences felt in spaces depend on the aesthetic, functional, and emotional properties of the spaces. Cooperation of various fields is essential in developing a smart environment to improve the quality of human life. Living in smart homes has social and economic implications and, therefore, should involve IT specialists, engineers, architects, city planners, and designers covering fields of psychology, sociology, and ethics, and a dialogue and close collaboration should be maintained simultaneously with academia, industry, and policymakers. Thus, establishing a good-quality scientific experimental platform to conduct interdisciplinary research on issues related to smart homes is essential.

#### Summary

Through the critical review of the 20 selected articles, the development of a pleasurable smart environment is complicated and involves multiple factors, such as physical, cognitive, psychological, and environmental factors. The relationships between these factors need to be explored in depth. In this research, we developed an evaluation framework for smart environments to enable AIP. Through the application of this framework, each selected paper in the field of smart environments was evaluated from a balanced perspective for critical review. The results are as shown in **Table 3**. The Harvey ball was used to interpret the degree to which each factor is mentioned in each paper. In case the ball is filled with black completely, it means that a specific factor is dealt with as an important factor in the paper (Han and Kim, 2018; Lee et al., 2019). Both researchers conducted a thematic analysis to identify major issues while reviewing the selected papers. Since each ball was chosen according to the researcher's subjective opinion, they discussed the elements related to each paper in depth in a debriefing session and finally reached a consensus. A critical analysis of selected smart environment studies identified the types of fields in which studies are being conducted and the fields that require more in-depth studies in the architecture domain. The analysis revealed that studies on smart environments are being increasingly conducted to support the well-ness of older adults. However, studies to support emotional well-ness in a smart environment for AIP are scarce.

Relatively many studies have been conducted to understand the characteristics of the manner in which older adults lead their daily life within a space. Several studies considered ways to integrate various sensors effectively into their living space for automation. Many studies refer to the importance of protecting and supporting the health and safety of older adults but rarely mention ways to support methods of improving positive psychological states. Because studies on a smart environment often aim to facilitate the physical independence of older adults in their living space, it was confirmed that factors related to independence are mentioned relatively evenly in the selected papers. It was also found that these studies understand the importance of understanding the personal tendencies and characteristics of older adults and applying them to the design when designing a smart environment but lack in researching how smart spaces can be expanded and connected to a smart community and smart city. **Table 3** shows that many studies are missing the need for space intended for the emotional satisfaction of older adults. These missing factors can be the necessary future direction of smart environment research for older adults to realize the vision of AIP to support pleasurable experiences.

# CHALLENGES AND DESIGN ISSUES OF PLEASURABLE SMART ENVIRONMENTS FOR OLDER ADULTS

We identified a future research direction through the critical review using the evaluation framework. The selected papers were reviewed on the well-ness, independence, acceptance, and design aspects. In this process, we identified the challenges and design issues that arise in providing a pleasurable smart environment for older adults.

# Smart Environment as a Friend to Promote Emotional Well-Ness

Under the Well-ness dimension, many studies tend to design a smart environment to monitor the behavior of older adults, to


*Harvey ballsrepresent "very poor," "poor," "average," "good," and "very good," respectively.*

Smart

evaluate their health status, or to protect them from the risks occurring in the living space. Unobtrusive sensors are installed in various places to monitor the living environment. The strength of the smart environment is that it automatically tracks older adults' behaviors and provides the information necessary for them (Kim et al., 2013). However, it requires many sensors and cameras for detecting their behaviors and location continuously to provide this function. We would emphasize the importance of smart systems that can mitigate the emotional loss experienced by older adults, and not smart systems that only support their functional status. Smart appliances or sensors can be utilized to further reveal design factors in the space and encourage older adults to interact with living environments to provide them hedonic and pleasurable experiences. Aging adults can be affected by their living environment. For example, Amazon Echo and Alexa have shown in many studies the possibility of actively incorporating voice-activated sensors in the lives of older adults, including such cases as inducing them to listen to music or to control the thermostat and reminding them of important events. Such sensors can be integrated into a space to induce or initiate certain behaviors of these adults (Forster and Van Walraven, 2007; Miller et al., 2011, 2014).

In addition, many papers deal with health and safety as an important issue, whereas few studies consider social or entertaining factors for older adults in the smart environment. Many other research disciplines pointed out that active participation and engagement in social activities are critical for maintaining a good quality of life for older adults. It is important to consider the ways in which the smart environment should be designed to promote the emotional well-ness of older adults. The quality of life of an individual is increasingly influenced by interconnections with others (Hirsch et al., 2000). For this reason, several technologies, and robotics have been developed to enhance communication between older adults and their families, relatives, friends, nurses, and doctors to support social interaction. Wang et al. (2014) indicate that aging adults overcome loneliness, anxiety, and depression when communicating with companions. The failure of developers of new and innovative technologies to put into consideration the emotional and social aspects of older adults in their technologies can be a cause for many missed opportunities. A smart space does not stop at facilitating smooth communication with others, and the space itself can play the role of a friend. The conversational agent in the smart living environment can promote social relationships for older adults, acting as a friend or family, not simply as a machine. Since older adults regard human relationships as important, the smart environment should be designed to simulate or stimulate the role of a friend or family.

# Smart Environment to Support Emotional Independence

The selected papers for this review paper present that smart homes and environments can be an option to assist older adults to live independently and maintain an acceptable quality of life (Jacelon and Hanson, 2013). Through the review process, we found that many studies focus on investigating ways for supporting the independent life of older adults. Nevertheless, many studies tend to focus on designing an assisted environment to support the basic life and physical functions of older adults, but often exclude their cognitive or psychological aspects (Mann and Milton, 2005; Hong et al., 2012). The smart environment for older adults should be designed so that it can allow them to lead a satisfying life in the space by monitoring their behaviors and monitoring their emotions. Understanding the relationships between human emotions and interactive systems would have a positive impact on social, cognitive, physical, and other human behaviors. The integration of affective computing and intelligent interfaces is one big opportunity for proving them emotional support (Kuderna-Iulian et al., 2009; Luneski et al., 2010). More improvements of the smart environment for older adults will be realized with the interpretation of sensor data on their emotional status by using affective computing that considers facial expressions, gestures, or speech output of the users (Tao and Tan, 2005).

The smart space that can detect emotional changes in older adults becomes able to provide them personalized information. A personalized adaptive space can be defined as an intelligent space in which the space learns patterns of usage for each individual user and adapts its behavior to that person in a non-trivial way (Jameson, 2008; Surie et al., 2013; Schmidt and Braunger, 2018). The smart space can satisfy users' needs autonomously by recognizing and inferring their behavioral and emotional patterns and can make decisions for older adults when appropriate. They gradually show a passive attitude in overall life owing to physical and mental constraints. The smart environment for them should play a role in drawing their positive emotions. The smart environment should be designed to provide proactive feedback to them generically instead of waiting for their responses. The smart technologies should recognize discomfort so that the user does not feel negative emotions when living in the smart environment. Providing proactive feedback helps older adults know factors to which they should pay attention, and such feedback includes suggestions and helps them make decisions.

# Challenges to Understanding the Needs of Older Adults Related to Smart Environments

Smart homes and the smart environment should be introduced to older adults with careful considerations of their strengths and potential risks. Financial accessibility or affordability should be considered. In the environment in which older adults reside themselves, a small risk may come up as a big problem. Older adults are less likely to feel the necessity of building smart systems by investing a large amount of money owing to ambiguous fears about whether they will use such systems for a sufficiently long period that justifies such expense or about whether they will be able to use these systems properly (Gunge and Yalagi, 2016). To understand their needs and provide a smart environment appropriate for them, it is necessary to understand them correctly (Haines et al., 2007). They face additional challenges because as people grow older, their cognitive, physical, and sensory abilities change, causing older adults to show different attitudes toward technology. Researchers easily overlook that these adults have heterogeneous characteristics (Courtney et al., 2008). They introduce elements such as bigger prompts, high contrast, and simplified interactive functions as design solutions for older adults. These factors could assist older adults to have smooth interaction, but also could reduce their interest. Currently, they are being exposed to technologies and increasing their experiences with the assistance of the younger population. It is increasingly important to understand when, how, and why older adults are engaged.

Technical and psychological accessibility can be addressed by fully investigating the views and needs of older people when implementing smart homes (Lê et al., 2012). Nonetheless, another problem related to implementing the smart environment for them is that very few empirical experiments have been conducted on this issue (Brush et al., 2011). Many studies still depend on the self-report methodology to ascertain older adults' evaluation of the smart environment. The main challenge of self-report methods is that these rely on users' recollection and self-interpretations. People tend to provide responses that are more positive or more frequent than in reality. It is necessary to investigate the problems that can occur while older adults are residing in the smart space themselves.

# Role of Architecture Domain in Designing Smart Environments

Various smart technologies are continually evolving. Architecture or housing is the basic space for human life. Smart homes should prioritize a comfortable and pleasant space for people to live, instead of aiming at housing where smart technologies are installed. People's lifestyles are constantly changing, and accordingly, the buildings in which they reside, are also changing. In addition, smart technologies in the buildings should be changed according to changes in people's lifestyles and housing structures. The smart environment should be able to support the interaction between people and space. It is necessary to conduct research concerning the skills needed for older adults and the design of a pleasant space for them to live (Labonnote and Høyland, 2017).

People in the past usually slept in the bedroom, watched TV in the living room, and worked in the study. However, thanks to the ICT devices and smart technologies that allows the overcoming of space restrictions, people can work in bed and watch TV, and even search for news in the bathroom. With the introduction of these smart devices, people are living in the age in which the boundary of space disappears. To have older adults adapt to smart environments surrounded by technology, they should not be required to engage in a complicated process for using that technology. Thus, in the architecture field, it would be necessary to conduct a study to propose ways for the design of a new space for older adults (Lê et al., 2012).

# CONCLUSION

Our society will change, and older adults will increase. They will inevitably be surrounded by smart services and technologies in their living environment, regardless of their wishes in this regard. Given that the proportion of older people is increasing more rapidly than that of the younger generations, research related to aging, and smart technology will be needed in various fields in preparation to meet the needs of this aging population. Many studies are developing technologies and environments necessary for extending the lives of older people. Through this analysis, we realized the need for smart technologies that will allow older adults to live well, rather than technologies that will prolong their life. The four dimensions we proposed are widely applicable to the smart environment research on older adults. The main findings derived through this framework are as follows:


Because this study reviewed a selected list of journal articles published in the field of architectural studies, one limitation is that it excluded the views on smart environments covered in other research fields. It should be noted that it is difficult for this study to address an integrated, synthesized overview of the current state of smart residential environments for older adults since it did not conduct a systematic review through meta-analysis. However, this study would provide readers the perspectives on the current status of research related to the smart environment in the architecture domain. We identified research challenges and design issues through a critical review of selected papers with the objective of enabling the well-being of older adults through a pleasurable smart experience. The details covered in the evaluation framework are critical factors that should be considered in providing this type of environment. These details can be widely used across stages, from the beginning stage of understanding the target user to the design stage and the system evaluation stage.

#### DATA AVAILABILITY STATEMENT

All datasets generated for this study are included in the article/supplementary material.

#### REFERENCES


# AUTHOR CONTRIBUTIONS

LL composed this study, designed the framework, and completed the quantitative analysis. MK provided supervision throughout the research and contributed substantially to the analytical part of the research.

### FUNDING

This work was supported by the National Research Foundation of Korea Grant funded by the Korean Government (NRF-2019R1A2C1087344).


physical world for sustainability. Sustainability 10:3725. doi: 10.3390/su10 103725


**Conflict of Interest:** The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Copyright © 2020 Lee and Kim. This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) and the copyright owner(s) are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.

# Fixation Differences in Spatial Visual Perception During Multi-sensory Stimulation

#### *Jihoon Kim1 and Ju Yeon Kim2 \**

*1 Department of Advertising and Public Relations, University of Alabama, Tuscaloosa, AL, United States, 2 Department of Interior Architectural Design, Soongsil University, Seoul, South Korea*

The mood and atmosphere of a service setting are essential factors in the way customers evaluate their shopping experience in a retail store environment. Scholars have shown that background music has a strong effect on consumer behavior. Retailers design novel environments in which appropriate music can elevate the shopping experience. While previous findings highlight the effects of background music on consumer behavior, the extent to which recognition of store atmosphere varies with genre of background music in sales spaces is unknown. We conducted an eye tracking experiment to evaluate the effect of background music on the perceived atmosphere of a service setting. We used a 2 (music genre: jazz song with slow tempo vs. dance song with fast tempo) × 1 (visual stimuli: image of coffee shop) within-subject design to test the effect of music genre on visual perception of a physical environment. Results show that the fixation values during the slow tempo music were at least two times higher than the fixation values during the fast tempo music and that the blink values during the fast tempo music were at least two times higher than the blink values during the slow tempo music. Notably, initial and maximum concentration differed by music type. Our findings also indicate that differences in scan paths and locations between the slow tempo music and the fast tempo music changed over time. However, average fixation values were not significantly different between the two music types.

#### Keywords: visual modality, multi-sensory, eye tracking, fixation, music type

# INTRODUCTION

Music can have diverse effects on consumer behaviors, including emotion (Bitner, 1992; Tansik and Routhieaux, 1999; Eroglu et al., 2005; North et al., 2016), purchasing (North et al., 1999; Alpert et al., 2005), brand attitude and loyalty (Chebat et al., 2001; Grewal et al., 2003), subjective time recognition and time of stay in a store (Holbrook and Anand, 1990), and other behavioral variables (Turley and Milliman, 2000). In the past, consumers automatically accepted the spaces given to them based on the products they chose to purchase. However, people have come to consider emotional experience clues (e.g., store atmosphere) to which they are exposed when selecting products. That is, modern consumers tend to select spaces based on atmosphere rather than merely accepting the spaces that stores create.

#### *Edited by:*

*Mi Jeong Kim, Hanyang University, South Korea*

#### *Reviewed by:*

*Seung Hyun Cha, Hanyang University, South Korea Deuckhang Lee, Nazarbayev University, Kazakhstan*

> *\*Correspondence: Ju Yeon Kim kjy@ssu.ac.kr*

#### *Specialty section:*

*This article was submitted to Environmental Psychology, a section of the journal Frontiers in Psychology*

*Received: 15 October 2019 Accepted: 17 January 2020 Published: 05 February 2020*

#### *Citation:*

*Kim J and Kim JY (2020) Fixation Differences in Spatial Visual Perception During Multi-sensory Stimulation. Front. Psychol. 11:132. doi: 10.3389/fpsyg.2020.00132*

In some cases, atmosphere can have a greater effect on a purchase decision than the product itself. Demoulin (2011) found several spatial environmental elements closely related to atmosphere: view, lighting, noise, music, scent, and temperature. Background music is another element of atmospheric impact (Milliman, 1986; Duncan Herrington, 1996). Lusensky (2010) found that background music was the most important element in the experience of a space, and Kim (2016) found that consumers were often exposed to background music, second in frequency only to visual stimuli. Furthermore, visual memory tends to last 1–2 s, whereas auditory memory tends to last 4–5 s, suggesting that humans remember auditory stimuli better than visual stimuli (Finney et al., 2001).

Experimental studies about emotional response to store spaces emerged several decades ago. Using eye tracking to study visual attention, Yarbus (1967) found that scan paths and frequency of attention to a given image varied with task type. More recently, DeAngelus and Pelz (2009) used eye tracking to measure fixation number, fixation duration, and fixation saccade. Greene et al. (2012) extracted fixation data relative to behavioral intention to determine changes over time. These scholars and others have confirmed that eye movement depends on the relative perceived importance of areas in a space. Posner et al. (1980) examined shifts in visual attention among the structures of a visual system. Through visual attention experiments that manipulated attention using signal positioning, they developed a model for attention that incorporated vision and judgment.

Scholars have shown that background music has a strong effect on consumer behavior (Bitner, 1992; North et al., 1999; Chebat et al., 2001; Alpert et al., 2005). While previous findings highlight the effects of background music on consumer behavior, the extent to which recognition of store atmosphere varies with background music type in sales spaces is unknown. To explore this phenomenon further, we analyzed quantitative fixation data for visual stimuli relative to auditory stimuli (i.e., different music genres). In the current study, we investigated (1) eye movement across space by music type, (2) space scan paths by music type, and (3) time between initial concentration and maximum concentration on the space by music type.

# LITERATURE REVIEW

#### Cross-modal Sensory Effects on Visual Attention

An important goal for retail managers and marketers is to provide memorable and unique experiences for customers. Multisensory cues can help stores stand out from competitors and attract customers by delivering a pleasant shopping experience (Verhoef et al., 2009; Hagtvedt and Brasel, 2016). Thus, retailers strive to enhance sensory inputs and use emerging digital technology to attract consumers and improve the shopping experience. Especially impactful when used in various combinations, sensory stimuli are basically cues related to vision, audition, olfaction, haptics, and gustation (Spence et al., 2014). Each contributing to subjective interpretations of the world, the five senses enable humans to perceive objects and phenomena in their environment (Calvert, 2001).

While vision is generally assumed to be the dominant sensory mode in retrieving external information (Reda et al., 2013), recent findings in marketing literature suggest that other sensory modalities can moderate visual processing (Hultén, 2011). In particular, sound can impact the primary visual cortex (Shams and Kim, 2010). Neuroscience scholars have found that several sensory brain areas are convergence zones where inputs from different sensory modalities interact, blend, and influence each other (Stein and Stanford, 2008; Murray et al., 2016). For example, multisensory stimuli (e.g., background music heard while finalizing a transaction at a store) can be perceptually clustered together, affecting visual attention, spatial awareness, and emotional state (Driver and Spence, 2000). In this regard, the auditory and visual systems are particularly interlocked. Moreover, sensory cues can non-consciously influence consumer decision making, changing their attitudes and behaviors (Togawa et al., 2019).

Numerous findings support the notion that auditory input during product interaction can strongly influence the perception and evaluation of products (Lv et al., 2020). Compared to an isolated visual stimulus, multisensory cues (i.e., visual-auditory combinations) provide extra assistance in product evaluation. Although scholars have identified how various qualities of store atmosphere might influence customers (Verhoef et al., 2009), the effect of multisensory input from technological advancements on consumer behavior needs further clarification.

#### The Effects of Music on Consumer Behavior

Scholars have recognized the effects of using background music to influence consumer behavior in various settings and situations (North and Hargreaves, 1996; Chebat et al., 2001; Alpert et al., 2005). Findings have confirmed that background music offers emotional responses such as pleasure and arousal (Spence et al., 2014).

Scholars have considered the influence of background music on consumers, including the presence or absence of music (Kellaris and Cox, 1989), liked or disliked music (Spangenberg et al., 1996), and manipulating specific components of musical structure. Furthermore, scholars have examined the effects of background music on a wide range of variables, including shopping time (Holbrook and Anand, 1990; Yalch and Spangenberg, 2000), total amount of purchasing (North et al., 1999; Alpert et al., 2005), and overall shopping experience (Dubé and Morin, 2001). More recently, scholars have examined how background music influences consumer perceptions and evaluation of foods. Some have focused specifically on how background music influences the taste of wine (Areni and Kim, 1993) and how background music influences beer taste (Carvalho et al., 2016).

However, research on the effects of background music type (i.e., tempo) on consumer behavior is relatively sparse. Addressing other aspects of background music, scholars have found that quiet (vs. loud) airline cabin noise increased the intensity of sweet solutions and decreased the intensity of umami-flavored solutions (Wang et al., 2015). With regard to tempo, however, few scholars have examined its effect on customer behavior. The way background music tempo influences physiological reactions through the neurophysiology system remains unclear.

# Justification of the Eye Tracking Experiment

People frequently do not choose between alternative courses of behavior; rather, they depend on cognitive abilities that are fairly well developed (e.g., awareness and recognition). Eye movement is one of the quickest and most frequent human actions, allowing people to gather numerous bits of information *via* eye fixations on a perceptual field. Outside that perceptual field, however, detailed information is likely to be missed. Moreover, people are normally unaware of the specific eye movements they make when performing a task. Eye movement is closely associated with covert attention. Covert attention plays a key role in information processing and decision making, not only by acting as a selection device but also by executing and maintaining central processing activity to ensure that decisions occur fast and accurately. Consequently, eye movements not only reflect selective attention but also the intensity and nature of the central cognitive processes in information perception, evaluation, and memory, providing real-time information about these ongoing processes that cannot be obtained otherwise. Thus, measuring where the eye is focused or how the eye is moving can help us understand how individuals view objects.

Data collection using eye tracking experiments does not depend on participant reports or memory. When questioned about their actions, participants might not remember their behavior. They might be unaware of what they did due to forgetting or lack of awareness. By examining data, visualizations, and replays, we can determine causes of behavior without relying on fallible human memory. Eye tracking analysis can lead to discoveries that would be considerably more difficult to uncover using other methods. Eye tracking shows how quickly and for how long customers notice features, key content, and brands and can reveal, through the accumulation of minute changes, where participants spent the most time looking.

Scholars have shown that the human visual system is essential to information searching and decision making (Lee and Ahn, 2012). Because the human visual system automatically completes various functions that are important to goal-directed behavior, visual information has a vital impact on shopping experiences (e.g., product evaluation and purchase). However, few scholars have investigated the role of visual processing using eye tracking devices.

# MATERIALS AND METHODS

#### Design

To test the hypotheses, we conducted a two-condition, randomized, within-subject experiment. The independent variable was music genre (i.e., jazz song with slow tempo vs. dance song with fast tempo), and we measured visual attention using eye tracking technology.

#### Stimuli

We conducted a field survey to select the music genres for auditory stimuli. We visited eighty cafés in Seoul, South Korea, and classified the kinds of music they played using 10 genres (i.e., new age, dance, rock, ballade, old pop, electronic, world music, jazz, classic, and hip hop; reference). We chose jazz (Music 1) and dance (Music 2) for our auditory stimuli because these two genres had the highest frequency scores (*N* = 109: jazz = 37; dance = 29; ballade = 23; old pop = 14; no music = 3; electronic = 2; new age = 1; and rock, world music, classic, and hip hop = 0). We used each genre with beat per minute (BPM) to create background music for the experiment (Music 1: jazz song with 88 BMP vs. Music 2: dance song with 137 BMP).

Across different auditory stimulus conditions (i.e., tempo and genre), we collected quantitative data about eye movement and fixation using eye tracking equipment. First, for visual stimuli, we selected a cafe space that participants would recognize as a setting where they might make a purchase. We used a camera (i.e., EOS 600D Crop Body; film size: 22 mm; full frame body: 35 mm; lens: EF-S 10–18 mm, F 4.5–5.6; focal length: 10 mm) to photograph this space and create images for the experiment. We configured the indoor space (approximately 8,000 × 6,000 mm) so that the person placing an order at the counter and the person seated at a table to the side were at adult eye level. As a result, the image was a close approximation of a human field of view (FOV). We also used Adobe Photoshop CS6 to enhance image quality.

#### Participants and Procedure

We recruited 40 undergraduate students [20 males and 20 females; mean age of 23.2 years (SD = ± 2.32)] enrolled at a private university in Seoul, Korea, in exchange for extra credit. We invited participants to the lab after confirming that they did not have any vision impairment that might limit their ability to see a computer screen (i.e., naked or corrected vision above 0.5). We prohibited mascara and color lenses during the experiment to maximize the quality of our eye tracking data. Upon arrival to the lab, participants signed informed consent forms approved by the Institutional Review Board (IRB) and then sat down at computer stations.

The distance between the monitor and each participant was roughly 650–700 mm to ensure accurate and consistent eye tracking calibration. Before the experiment, we ran a quick calibration program to prepare the eye tracking camera. On their computer screens, participants watched a white dot move to four different areas. We performed the calibration test up to two times and excluded participants who did not pass. Participants who did pass then read the following instructions on the screen: "Imagine you are visiting a coffee shop. Please look around the café and think how the atmosphere of the space and the music feels." Because we used a within-subject study design, each participant looked at the image while listening to Music 1 and Music 2; however, the order in which they heard the two auditory stimuli was random. The first multisensory stimulation lasted 1 min. After a 1-min rest time, the second multi-sensory stimulation began. The experimental session lasted a total of 3 min and took place in a dark room so that the participants could focus on their task with maximum concentration.

#### Dependent Measure

We measured the real-time eye movement of the participants. We used eye tracking equipment (i.e., iMotions: SMI-REDn, 30 Hz) to determine the impact of auditory stimuli on visual perception of the physical environment pictured in the image. We extracted raw fixation data using BeGaze 3.7 (SMI). We set the rate of recording and storage to 30 Hz (i.e., 30 data per second).

#### RESULTS

#### Effective Rate of Raw Data

From the total sample (*N* = 40), we excluded six participants (3, 11, 14, 23, 30, and 38) who failed to exceed the criterion of 80% eye tracker accuracy and one participant (1) due to technical problems with raw data. Therefore, we analyzed the data from 33 participants. The mean effective rate was 90.9% (SD = ±4.13) (see **Table 1**). In the experiment, we set the fixation time for the visual stimuli to 120 s (i.e., 60 s with Music 1 and 60 s with Music 2). Therefore, we analyzed 3,600 (120 s × 30 Hz) fixation data for each participant.

#### Raw Data Extraction

Using the eye tracking data, we extracted values for fixations, saccades, and blinks, the primary components of human eye movement. Fixations occur when the eyes stop scanning and hold in one place for a time while saccades are the rapid eye movements that occur between fixations, allowing the fovea to shift from one point of interest to another. When plotted in chronological order, fixations and saccades constitute a scan path. Fixation count refers to the number of fixations that occurred in an area of interest every 0.03 ms, indicating how long an individual spent looking at a particular part of the image on the screen (i.e., degree of interest). Frequency refers to how many times per second the eye tracker registered the position of the eyes (i.e., raw data divided by fixation count). Blinking refers to the involuntary act of shutting and opening the eyelids. During each blink, the eyelid blocks the pupil and cornea from the illuminator, resulting in missing raw data points at particular *x*-*y* coordinates.

Fixation and saccade frequencies did not differ significantly between Music 1 and Music 2. Fixation frequency for Music


1 (12.29) and Music 2 (12.22) was close. However, blink frequency for Music 2 (12.29) was twice the frequency for Music 1 (6.28). The fixation and saccade frequency and mean values were not significantly different between Music 1 and Music 2, but the blink frequency and mean values for Music 2 were two times those of Music 1 (see **Table 2**). In addition, although the fixation mean values were similar, the fixation standard deviation for Music 1 was higher than Music 2.

#### Analysis of Eye Gaze Pattern Differences

Paired sample *t*-tests for fixation (*t* = 0.26, *p* = 0.82) and frequency (*t* = 0.10, *p* = 0.92) indicated no significant difference between Music 1 and Music 2. However, analysis of variance (*s* 2 ) for fixation revealed that the standard deviation for Music 1 (*s* 2 = 15,804) was more than two times Music 2 (*s* 2 = 6294.78). Therefore, although the mean values were similar, fixation values differed across participants by music type (see **Table 3**). Moreover, although fixation values differed across the participants, changes in fixation over time and the order of fixations did not. Therefore, we analyzed individual fixation data in more detail.

We compared raw fixation data, count, and frequency for Music 1 and Music 2. According to the raw data, the number of participants (*n* = 18) who showed more fixation during Music 1 was higher than the number of participants (*n* = 15) who showed more fixation during Music 2. In terms of fixation count, the number of participants (*n* = 17) with a higher index during Music 2 was higher than the number of participants (*n* = 14) with a higher index during Music 1. Two participants had the same fixation count during Music 1 and Music 2. The number of participants (*n* = 14) with higher average frequency for Music 1 was the same as the number of participants (*n* = 14) with higher average frequency for Music 2 (see **Figure 1**). Five participants had the same average frequency during Music 1 and Music 2.

TABLE 2 | Average count and frequency of raw eye tracking data for all participants.


TABLE 3 | Mean, population variance (σ<sup>2</sup> ), and standard variance (*s*<sup>2</sup> ) for raw eye tracking data by music type.


#### Differences in Initial Concentration and Maximum Concentration

We defined the time at which the top 5% index value for high fixation started as initial concentration and the time at which the index value for high fixation peaked as maximum concentration. Results of paired *t*-tests for average initial concentration (*t* = 1.46, *p* = 0.15) and average maximum concentration (*t* = 0.23, *p* = 0.82) revealed no significant difference between Music 1 and Music 2.

On average, initial concentration began at 12 s during Music 1 and 15 s during Music 2. Based on population variance (*σ*<sup>2</sup> ) and sample variance (*s* 2 ) values among the participants, initial concentration during Music 1 was 1.5 times faster than during Music 2. Maximum concentration began at 36 s during Music 1 and 37 s during Music 2, and differences among the participants were not significant between Music 1 and Music 2. Although differences in average initial concentration were not large, differences in variance indicate different initial concentrations among the participants.

The number of participants (*n* = 22) with earlier initial concentration during Music 2 was twice the number of participants (*n* = 11) during Music 1. The average initial concentration was between 10 and 15 s depending on music type, and the difference in average initial concentration between Music 1 and Music 2 was at least 5 s. Maximum concentration also differed between Music 1 and Music 2. The number of participants (*n* = 19) with an earlier maximum concentration during Music 1 was higher than the number of participants (*n* = 14) during Music 2. Maximum concentration differed among the participants (see **Figure 2**).

We analyzed initial concentration frequency at five-second intervals. In the first interval, the number of participants (*n* = 8) with initial concentration during Music 2 was higher than the number of participants (*n* = 7) during Music 1. In the second interval, the number of participants during Music 1 (*n* = 13) was more than three times the number of participants (*n* = 4) during Music 2. After 20 s, the number of participants who began concentrating during Music 2 was higher. We then analyzed maximum concentration frequency at 5-s intervals. We found no difference in the number of participants in the first two intervals. However, at 20 s, the number of participants (*n* = 4) during Music 1 was higher than the number of participants (*n* = 2) during Music 2. The difference between Music 1 (*n* = 7) and Music 2 (*n* = 1) was larger at 45 s (see **Figure 3**). At all other times, the number of participants during Music 2 was higher than or equal to the number of participants during Music 1.

#### Differences in Scan Paths and Locations

To determine areas of the screen in which participants showed interest, we divided the visual stimulus into a 12 × 12 grid (see **Figure 4**). We used color spectra to indicate relative degrees of visual interest across the Gridded Area of Interest (AOI).

We divided areas of interest into three time ranges – early period (T1: 0 ~ 20 s), middle period (T2: 20 ~ 40 s), and late period (T3: 40 ~ 60 s) – and compared degrees of interest between Music 1 and Music 2. Analysis of grids during Music 1 revealed that exploration of the entire image was not intensive in T1 and that the primary area of interest was e-g' (i.e., the employee making coffee). The distribution of eye movement was primarily in the left half of the image. In T2, areas of exploration were more spread out to the left and right of the primary area of interest in T1. In T3, the area of interest shifted from e-g' to g-e' (i.e., product display). Between 10 (i.e., early period; highest frequency of initial concentration) and 45 s (i.e., late period; highest frequency of maximum concentration), the area of interest shifted from human objects to purchasable objects.

Analysis of grids during Music 2 revealed that initial concentration began in T1 and that frequency decreased thereafter (see **Figure 4**). As we found during Music 1, the primary area of interest was e-g' in T1 and began to shift in T2. In T3, maximum concentration occurred on e-g' and d-c' (i.e., upper wall of the cafe where the coffee menu is visible).

#### DISCUSSION

Traditionally, store managers have focused on interior store aesthetics for visual stimuli. To a lesser extent, they have started to pay attention to auditory stimuli. Previous findings have shown how cross-modal sensory stimuli can affect consumer behavior (Togawa et al., 2019). Recently, store marketers have increasingly used multisensory cues in their stores to create positive and emotional experiences for customers. Exploring the effect of multisensory stimuli on consumer behavior offers a more holistic perspective on the atmospheric features to which customers are likely to be exposed in a retail environment.

In the current study, we analyzed differences in visual exploration and fixation by music type. First, the average fixation values did not differ significantly between Music 1 and Music 2. One possible explanation is that because visual cues trigger our dominant sense, auditory cues might not exert a strong influence on eye movement. However, according to analysis of variance, the fixation values for participants during Music 1 were at least two times higher than during Music 2, and the blink values during Music 2 were at least two times higher than during Music 1. Comparing frequencies of eye movement between the raw data and the count values, we found that the areas across which eyes moved were larger during Music 2 (i.e., fast tempo), but that fixation frequency was higher during Music 1 (i.e., slow tempo). This analysis is meaningful because objects in scenes are generally coded for visual attention because they provide external information (Navalpakkam and Itti, 2005). Through their visual attention, people prioritize these perceptual objects based on their relative perceived importance. These visual attention patterns are reflected in eye movement. Thus, we inferred covert cognitive processes from eye tracking data. Moreover, in our analysis of data, rather than average differences by music type, we examined relative mean values between fixation and count. In addition, because fixation patterns varied across music type and participants, we had to consider each factor separately.

Second, initial and maximum concentration differed by music type. Regardless of music type, maximum concentration began after at least 40 s on average. Initial concentration was earlier during Music 1, and variance among participants was larger during Music 2. These findings suggest differences in the way the participants responded to music. The average maximum concentration was 36 s during Music 1 and 37 s during Music 2, and the earliest maximum concentration was 35 s. When we divided the time into 5-s intervals and examined frequency of initial concentration across the participants, we found that initial concentration began in the early period for both types of music. However, during Music 1, frequency of initial concentration was high at 10 s (39%) in T1. During Music 1, the highest frequency of maximum concentration was 45 s (21%), but a high frequency also emerged at 20 s (12%) in T1.

Third, our findings indicate that differences in scan paths and locations between Music 1 and Music 2 changed over time. When we divided the viewing time into T1, T2, and T3, we found that eyes moved to the menu or products on the wall in T1. In T2, scan paths differed between music types. During Music 1, scan paths dispersed across the entire image so that eyes landed on both humans and products. During Music 2, scan paths dispersed across the entire image in T3, later than during Music 1. Synchronized stimuli, resulting in cross-modal effects (i.e., visual and auditory), present different routes for perceiving objects in the scene. Auditory cues, enabling cross-modal sensation, further explain how individuals view objects. The results show how auditory cues might interact with visual cues. As sensory marketing research shifts attention from single sensory systems to cross-modal sensory systems, scholars need better insight into the way various sensory systems might interact with each other.

In the current study, we analyzed differences in eye movement and fixation during multi-sensory stimulation (i.e., screen image and background music). Although differences in average eye-movement values between music types were not large, the results are meaningful because we were able to extract the characteristics of visual perception and degree of interest in spaces using eye tracking. Our findings should guide practitioners of neuro-marketing to consider multi-sensory stimulation when designing spaces where consumers are likely to purchase products.

#### Limitations of the Study

The limitations of the current study open pathways to future research. First, we exclusively recruited college students. Scholars should consider using wider, more random sampling in order to increase generalizability. Second, we focused on two different genres of music: jazz music with slow tempo and dance music with fast tempo. Given the variety of existing genres, scholars should consider examining the potential impact of other types of background music. Third, we conducted this study conducted

in a laboratory setting rather than investigating consumers in their normal shopping environments. For visual stimuli, we selected a cafe space that participants would recognize as a setting where they might make a purchase. Then we created images for the experiment. Participants in a real-setting experiment might have responded to the questionnaire differently. Follow-up field research could more fully reveal the effect of background music on the perceived atmosphere of a service setting. Finally, scholars should qualitatively examine how consumers move their eyes across a visual field and where they place their attention in the field of view.

#### CONCLUSION

Sensory information (e.g., sights, sounds, smells) is typically stored in and retrievable from a sensory memory location. Sensory memory intensifies with the repetition of sensory information, serving as a buffer for external sensory stimuli. The perception of sensory attributes occurs when the process satisfies the condition. Attention then increases with the activation of a particular sensory modality (e.g., visual or auditory). Nearly 70% of the sensory receptors in humans are allocated to vision, making it the most important element in the human sensory system. Thus, understanding how humans process visual sensory information and how it interacts with other sensory information is important. To investigate the effect of background music on visual perception of a physical environment, we conducted an eye tracking experiment in a service setting.

Although we did not find any significant differences between the slow tempo and fast tempo conditions with regard to visual perception and interaction, the individual differences between groups by music type are meaningful. Results show that the blink values during the fast tempo music were at least two times higher than the blink values during the slow tempo music. Moreover, eye movement was more active when the background music tempo was relatively fast, indicating that external auditory information might have activated the visual cortex. We also investigated how differences in scan paths and locations between

#### REFERENCES


the slow tempo music and the fast tempo music changed over time. Our findings provide an empirical basis for examining changes in the way consumers respond to multi-sensory stimuli.

Applying Yarbus (1967) to the current study, we shed further light on differences in visual perception during multi-sensory stimulation in a service setting. We measured visual attention using eye tracking technology and compared various components, including fixation, saccade, and blink. Our findings suggest that a qualitative research method (e.g., interviews) could capture the interaction or joint effect of multi-sensory cues on visual perception.

# DATA AVAILABILITY STATEMENT

The datasets generated for this study are available on request to the corresponding author.

# ETHICS STATEMENT

The studies involving human participants were reviewed and approved by IRB Office, Soongsil University. The patients/ participants provided their written informed consent to participate in this study.

#### AUTHOR CONTRIBUTIONS

JK and JYK contributed to the conception and design of the study. JK organized the database and analyzed and interpreted the data under the supervision of JYK. All authors contributed to the writing and editing of the manuscript.

# FUNDING

This work was supported by the National Research Foundation of Korea Grant funded by the Korean Government. This project number is 2017R1A2B2007276.


**Conflict of Interest:** The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

*Copyright © 2020 Kim and Kim. This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) and the copyright owner(s) are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.*

# Biomorphic Clothing Sculpture Interface as an Emotional Communication Space

*Young Sun Yoo, Minjin Cho, Jung Sun Eum and Seon Ju Kam\**

*Department of Clothing and Textiles, Kyung Hee University, Seoul, South Korea*

This study found that clothing, combined with digital technology, can be a wearable computer that adapts to environmental change and increases biological limits, as well as an emotional space to which art and design are applied, and that promotes mutual communication. A user-friendly interface is proposed for those pursuing a new artistic experience by building a conceptual algorithm for biomorphic clothing sculpture modeling that can be used to create generative designs simply by entering moderated mediator variables. Biomorphic clothing sculpture refers to clothing that is inspired by life in nature and modeled in three dimensions. As the human body and buildings are 3D forms, the parametric design method used frequently in architecture is implemented here and used as a tool to explore attributes and representations of 3D modeling in clothing sculpture design. On the basis of a literature review, this study presents knowledge-based data for biomorphic clothing sculpture that can predict generative design outcomes, and through a case study, identifies the mediator variables and attributes of the parametric process related to biomorphic clothing sculpture modeling. The knowledge-based data on biomorphic clothing sculpture and biomorphic clothing sculpture modeling mediator variables discovered during the research are applied to the 3D modeling process as visual data and presented as the conceptual interface of biomorphic clothing sculpture.

#### *Edited by:*

*Mi Jeong Kim, Hanyang University, South Korea*

#### *Reviewed by:*

*Han Jong Jun, Hanyang University, South Korea An Hyun Joo, Hansung University, South Korea*

> *\*Correspondence: Seon Ju Kam kamjoo@empas.com*

#### *Specialty section:*

*This article was submitted to Environmental Psychology, a section of the journal Frontiers in Psychology*

*Received: 14 October 2019 Accepted: 15 January 2020 Published: 07 February 2020*

#### *Citation:*

*Yoo YS, Cho M, Eum JS and Kam SJ (2020) Biomorphic Clothing Sculpture Interface as an Emotional Communication Space. Front. Psychol. 11:117. doi: 10.3389/fpsyg.2020.00117*

Keywords: biomorphic design, clothing sculpture, interface, parametric design, biophilia

#### INTRODUCTION

The recent development of digital technology that crosses the boundary of virtuality and reality has facilitated the transformation and expansion of the concept of space, blurring the lines not only of architecture but also of modeling and formative arts. Moreover, computing technology previously used by designers or architects with expertise is now provided as a service in the form of algorithms on platforms built for design purposes, enabling users to obtain emotional pleasure by creating new sculptures simply by entering moderated data. The parametric design method can be used to create forms that apply structural order and principles that appear in nature based on mathematical algorithms and can automatically modify the features of model components merely by adjusting form-generation figures based on the user's intuition. Architecture and fashion share the attribute of being a cultural and individual tool of esthetic expression, as both create a 3D form that protects humans. Clothing is defined as a space that covers the human body and, with the mobility of digital humans, is interpreted as an architectural space. Combined with digital technology, clothing can become a wearable computer that adapts to environmental change and increases biological limits as well as an emotional space that is applied to art and design and promotes mutual communication between the artist and the audience.

The biophilia hypothesis suggests that humans feel comfortable in nature from the perspective of evolutionary biology and has become the basic concept of biophilic design (Söderlund and Newman, 2017). Many studies have shown that biophilic design helps humans recover their health and relieves mental stress (Ulrich, 1984; Joye, 2011; Söderlund and Newman, 2017), engaging them and improving creativity (Öhman and Mineka, 2001). Joye and Van Locke (2007) conducted an experiment on stress relief provided by nature and found that simplified natural landscape images and geometrically transformed natural patterns are more effective at reducing stress than real natural environments such as forests. Findings include that elements of biomorphic architecture such as plants, or environments that include plants, are visually appealing and can induce a positive esthetic response (Joye, 2006). Thus, visual elements inspired by living matter in nature stimulate the natural esthetic senses of humans and encourage creativity to produce other artistic objects. Moreover, the advancement of an evolutionary system of computer technology based on a nature-friendly design paradigm promotes design methods based on computing technology that can easily create new geometric forms rooted in nature's biological structure and elaborate, mysterious biological functions. The objective of this study is to explore parameters required by biomorphic clothing sculpture (BCS) modeling conceptual algorithm to create generative fashion design simply by entering moderated data into parametric design algorithms using biomorphism as a knowledge base and using these to build an interface for people seeking new emotional communication for pleasure.

# LITERATURE REVIEW

Through a literature review, this study verified the applicability of the parametric design process in architecture to clothing sculpture and presented knowledge-based data to be used in the BCS interface and analytical items to be used in the case study, exploring parameters by examining the design approaches of biomorphic architectural form and parametric form generation methods.

#### Clothing Sculpture as Architecture

As clothing has a 3D form and many parts that are connected based on the human body form, it is akin to a 3D architectural form, in which multiple elements are integrated into a structure. Moreover, the structural building method, details about connecting one part with another, and various physical properties used in architecture are similar to style, cutting and sewing methods, details, and fabrics used to make clothing. The 2006 *Skin + Bones: Parallel Practices in Fashion and Architecture* exhibition held at the Los Angeles Museum of Contemporary Art involved world-renowned architects and fashion designers. The exhibition displayed works in the two fields around the theme of new software materials and technology. Sculptor Zaha Hadid mentioned in an interview that both architecture and fashion exist for the well-being of users and that the basic principle of design is to explore emotional fragments in visual language. She claimed that both buildings and clothing are thus attractive 3D objects with functionality and esthetics, as they create style and pattern that reflects the paradigm of the times (Jury, 2008).

Building information modeling (BIM) is a technology that models not only design data for buildings but also all related information using a computer—from design to management and destruction of a building (Arayici, 2008). BIM is applied as a mediator variable in the parametric design process and helps solve problems in architectural design in which elaborate and diverse 3D forms must be developed. Kwon and Jun (2014), in a study of the design of framework materials for *hanoks* (traditional Korean houses), claimed that parametric design can use information on the architectural elements that form a building as various parameters to easily transform and adjust forms. In other words, they used the formative principles and geometric shapes of *hanoks* as a knowledge base and proposed a parametric design process based on how the materials are bonded as the mediator variable. They claimed that by altering the measurement of just one material in digital space, various other materials could also be altered easily and automatically, which enabled them to promptly assess the effect of modifying the material measurements. Based on this, they adapted the parametric design method used in architecture to the study of the BCS interface and used parameters in the knowledgebased modeling process as exploratory tools.

The advancement of new materials and technology, as well as 3D printing, has provided a completely new definition of tailoring. 3D modeling is becoming an innovative design process, as it uses scanning technology to design fashion items that perfectly fit the human body form. Farahi (2017) called for research on clothing made from flexible materials that produces harmony with the human body using geometry and proposed a study to develop materials that resemble human skin, based on anatomist Karl Langer's idea that topological lines could be mapped onto the human body to represent the lines of skin tension. The scope of the application of research on BCS using the parametric design process with biomorphic inspiration as the knowledge base can be expanded in association with the development of 3D printing fashion design.

# Biomorphic Design

Biomorphic design, biophilic design, and biomimicry are all terms based on the concept of biomorphism. Biomorphism is a compound of "bio," a complex term for life, lively phenomena, and biotics, and "morphism," the collective form or composition of an organ or part (Alloway, 1975). Sculptor Jean Arp is referred to as the father of biomorphism (Greenbaum, 1994). Edward B. Henning, curator of the Cleveland Museum of Art, argued that Arp's wood relief, *Forest*, influenced Joan Miro's early experimental work in biomorphism, *The Hare* (Henning, 1979). Biomorphism is based on Henri Bergson's philosophy of vitalism, which had a significant effect on French esthetics in the twentieth century. Bergson interpreted that the driving force for the evolution of "*élan original*" is "*élan vital*," or dynamism and claimed that various images created by the movement of life are immaterial and comprised of the continuity of time that changes every hour. Intuition is perceived as shared intellect and instinct (Khandker, 2013; Manfred, 2016). Bergson's philosophy influenced expressional characteristics of biomorphic art such as organic form, abstraction of vital movement, continuity of the object of expression, and automatism. Works by biomorphic artists followed. Arp referred to his sculptures as "concretion"; they expressed life in abstract, organic volumes of round shapes depicting the abundance and fullness of the growth and proliferation of nature (Rider, 2015). Miro identified a large tree, a blade of grass, an animal or snail, lizard, or small insect with life that has the same value as the entire universe. He freely modified this relationship through surrealist automatism and expressed abstract life forms that drift across the canvas (Adamowicz, 2012). Henry Moore detected the order of nature from primitive art filled with vitality and newly interpreted living organisms (Chryssovitsanou, 2013). Influenced by Miro and Arp, William Baziotes turned the growth of protozoa that drift in the ocean into abstract, wobbly lines and light expressed as vibration (Naves, 1995). Gaudi's La Sagrada Familia is also considered a typical biomorphic building, as its pillars, which support the ceilings within, were inspired by the vitality and dynamics of trees. Gaudi designed this solid structure by imitating not only the external and natural form of trees but also their natural power (Joye, 2006).

Biophilic design that emerged with the popularization of biophilia in the 1980s and biomimicry, which emerged for sustainability, has inherited the attributes of biomorphic art that originated in modernism in the early twentieth century (Messenger, 2004). Edward O. Wilson, the progenitor of sociobiology, coined the term "biophilia" as a compound of "bio" and "philia" in 1979 to mean "living things have love of the natural environment." Based on this meaning, he proposed biophilia as a formula whereby humans become one with living nature and achieve creative evolution as they are moved and healed (Joye, 2011).

Biomimicry refers to the biomimetics of the basic structure, principles, and mechanisms of living things and biomaterials in nature (Messenger, 2004). The term is derived from the Greek "*bio*" and "*mimicry*" and refers a design approach that mimics nature's models, systems, and elements to solve complicated human problems. Janine Benyus, while not coining the term "biomimicry," contributed to its generalization *via* her book *Biomimicry: Innovation Inspired by Nature*, which explains the early principles of biomimicry. She perceived nature as the object of the model, measure, and mentor and presented sustainability as the purpose of biomimicry (Symeonidou, 2019). Biomorphic design shares the concept of biomimicry when it uses natural motifs for the purposes of human welfare and healing—the fundamental concepts of biophilia. Hence, this study discusses biophilia along with the concept of biomorphic design. Cases of biomimicry include the Eiffel Tower, whose pattern and fractal form is inspired by high bone structure; the BionicMotionRobot (Festo, 2017), which mimics the movements of life; the high-speed railway in Japan, which reduces noise caused by aerodynamics inside tunnels by drawing inspiration from the silence of the kingfisher's beak and owl's feathers (Khaliq et al., 2014; Kennedy and Marting, 2016; Hayes et al., 2019); and Zimbabwe's Eastgate Center, which reduces energy requirements by employing a ventilation system that imitates the structure of a termite nest (Ramzy, 2015). Thus, biomimicry—which mimics nature's biological systems or structures—is becoming a solution in multiple fields, from everyday life to industry.

Joye (2006) analyzed the attributes of biomorphic expressions in architecture as visual and structural sharing of nature's models; imitation of natural patterns integrating biological patterns, textures, and sculptures; and emotional expressions of the evolution of trees or plants dominated by fractal geometry. He claimed that these attributes could become the grammar of form generation, producing creative and artistic outcomes. Fibonacci spirals and phyllotaxis patterns were earlier applied to the golden ratio and the Canon of Proportions in *The Vitruvian Man* and are still inspiring architects today (Symeonidou, 2019). Previous studies of biomorphic design combined with technology mentioned images such as birds' nests, honeycomb, wings, twigs, radiolarians, onion cells, cabbage sections, human bones, spider webs, wavy wood, seashells, sponges, dripping water, shoals of fish, and *Sepia pharaonis* cuttlefish as motifs of biophilic emotions. These are applied to parametric methodology and are becoming sources of inspiration for modeling of furniture or buildings (Messenger, 2004; Vincent and Garcia, 2009; Mirkia et al., 2018; Symeonidou, 2019). **Figure 1** illustrates types of contemporary biomorphic design approaches derived from Bergson's theory of vitalism and characteristics of biomorphic art and biophilia.

#### Parametric Design

Parametric design is a computer representation method that creates forms as design attributes and turns them into parameters as geometrical elements (Barrios Hernandez, 2006). In a parametric model, parameters can be altered to find new alternative solutions to a problem (Lotfabadi et al., 2016). Parametric design includes two modeling techniques. One is a more elaborate parametric modeling method that integrates a knowledge-based system, so that all elements are modified in terms of size and location through parameters of the relationship. This has the benefit of enabling the designer to predict the outcome of the parametric design. The other is a design method based on form generation, which programs designers' ideas and related designs by entering them directly into a computer (Barrios Hernandez, 2006). The parameter refers to a certain factor or element that serves as a boundary or can be quantified. It is applied as a figure that can be quantified, but it may also include a functional standard such as light or structural resistance or a conceptualized esthetic standard used by designers to develop their own creative ideas. The parametric process is generated as algorithms *via* which a designer's ideas materialize. The generated algorithms can create

and modify various scenarios by adjusting the original parameters such as the shape's location, scale, and angle depending on the concept; or the user may obtain output that satisfies their intentions through simulation (Abdullah and Kamara, 2013). The outcomes of parametric design may vary depending on the design procedures, design ideas, and tools or software used.

The process of clothing sculpture modeling involves parametric procedures related to the human body. Clothing is not only worn by a 3D human body but has various design components related to form, which makes it difficult to operate using a conventional digital design process alone. Therefore, there is a need for research on a parametric design process algorithm that can model curves and complicated forms. With information obtained from nature, the geometric principles and esthetics of the parametric design process form generation technique may lead to creative design outcomes in the BCS parametric design process, as the tool can bring comfort and pleasure to people today, whose everyday lives are linked to digital devices.

Parametric form generation is related to the generative principles of fractals, tessellation, and Voronoi. Fractal geometry was introduced in the 1970s when Benoit Mandelbrot declared that most biological forms have a fractal structure and was classified as a new geometric concept characterized by self-similarity, repeatability, and endless irregularity (Stamps, 2002; Vaughan and Ostwald, 2010; Rian and Asayama, 2016). In a fractal, shapes of different sizes are in a geometric form, but the location of the shapes or rotation of the forms is depicted in different states and magnifications (Stamps, 2002). The magnification of a fractal is not accurate, unlike in Euclidean geometry where magnifications are fixed by precise numbers, representing the complexity of the fractal pattern (Joye and Van Locke, 2007). As an example, Vaughan and Ostwald (2014) pointed out that the form of a tree from a distance is similar to the stem structure and leaf tissues of the same tree and claimed that a fractal structure is characterized by consistent complexity or irregularity of scales that intersect. Sedrez and Pereira (2012) stated that fractal geometry is depicted as rough, holey, or broken objects and is generated by a recurrent process in which the initiator and generator are repeated infinitely. Thus, a part is similar to or the same as the whole or has self-similarity that consistently follows the initial form in which everything is similar. In the generator, lines and faces, crooked lines, and non-linear forms can be used, and this generator is used in the algorithm to depict the final form. Even in the same algorithm, generators can be varied to represent different forms. In other words, the generative principle of the fractal algorithm is a feedback algorithm in which certain figures are entered, and the results are calculated through a fractal equation, which is then replaced according to the results (Rian, 2018). Accordingly, fractal algorithms are used in science, engineering, and medicine to explore and model non-linear and complicated shapes, as well as in creative concepts of art (Rian and Asayama, 2016).

The Voronoi diagram defined by mathematician Voronoi in the early twentieth century received attention as a very useful diagram that can explain phenomena and structures in nature. It is used in the form generation process of architecture, jewelry, and industrial design, as well as in various other fields such as computer engineering, meteorology, geology, topography, archeology, medical science, molecular chemistry, and ecology (Kim and Kim, 2006; Zheng et al., 2011; Fantini et al., 2016). The Voronoi diagram is an outcome of specific space division whereby, starting from a discontinuous point referred to as the "site," all cells are comprised of points that are closer to the generation seed (Lautensack, 2008; Malinauskas, 2008; Phillips, 2014). All intersections that are not empty are collected to form a Voronoi diagram limited by the domain and used to model the space from a series of calculated points (Sainlot et al., 2017). With these characteristics, the Voronoi diagram is positioned as a generative mechanism of digital space combined with computer programming, explaining the focus on 3D space generation of Voronoi in the parametric design process of clothing sculpture. Clothing is comprised of a regular system following the curves of the human body that constantly evolves based on changes in the design components and is characterized by self-organization in which spontaneous patterns are formed. Through self-organization, a single pattern proliferates repeatedly, which seems to be an evolutionary process. 3D Voronoi that materializes through the reciprocal self-organization technique can be seen in natural phenomena that are easily observed, cell division, patterning in animals, and natural environments.

Tessellation is used to refer to faces that are filled horizontally when generated by the connection of Voronoi points. As the edges of the tessellation are used only to define the shape of each Voronoi cell, they can be used in the same sense as the Voronoi diagram. However, the current study considers Voronoi in terms of 3D form generation and tessellation in terms of 2D patterns. The attributes of each Voronoi cell generation point are defined as solid or empty, and related cells have the same attributes as the first generation point (Chow et al., 2007). These attributes are also related to the pattern generative principles of tessellation. The infrastructure of many patterns has three regular tessellations: triangles, squares, and regular hexagons. Squares, which are most easily applied, are popular (Sarhangi, 2012), but triangles are most commonly used, as they are effective in expressing simplicity and flexibility because the vertices are located on the same plane, unlike other polygons (Werghi, 2011). Tessellation, in which small, similar parts gather into an aggregation to form a huge whole, is prolific in the natural world and can be observed through an electron microscope; it thus serves to express the important grammar of parametric design, in which the forms of the natural world are computed.

### RESEARCH METHOD FOR THE BIOMORPHIC CLOTHING SCULPTURE CONCEPTUAL PARAMETRIC DESIGN PROCESS

This study aims to develop the conceptual interface applied by the categorization of biomorphic inspired images and 3D printed clothing design components as revealed through case studies as procedural parameters. In this study, parametric design methodologies and processes are used as a modeling method for the BCS interface. Based on the esthetics and colors of biomorphic art covered in the literature review, form generation approaches used in biomorphic architectural design, and parametric design process theories, this study proposes a conceptual parametric design process for the BCS interface as an emotional communication space. To prove the practicality of the conceptual interface, 126 biomorphic design cases and 124 3D printing clothes cases are collected and categorized into parameters. The representative examples of the case analysis parameters are illustrated as figures drawn or presented in the form of 3D modeling instances to demonstrate the results. **Figure 2** shows the conceptual parametric design process for the BCS formed as a figure with reference to the parametric design process of architecture.

# CASE STUDY

#### Knowledge Database Case Study: Biomorphic Inspiration Images

As the knowledge database for the BCS interface, this study used biomorphic inspiration images from which generative design outcomes were predicted based on the findings of the literature review. The selected images were those created as 3D artistic forms or designs that referred to the biomorphic design form generation approach. Keywords such as "biomorphic shape," "biomorphic form," and "biomorphic structure" were used in the search, and one was selected only when similar forms were repeated for the same concept, ultimately resulting in the collection of a total of 126 cases. The cases were classified based on the form generation approach (**Figure 1**) to biomorphic architectural design summarized in the theoretical background and analyzed in categories such as "realistic expression of natural patterns," "abstraction of natural forms," "imitation of natural structures," and "evolutionary morphological variation." "Realistic expression of natural patterns" is a category that imitates nature and embodies

animals and plants, reproducing them by instilling design opinions, rather than by merely imitating natural objects, thereby adapting the vitality of nature in the form of animals and plants and expressing it in similar natural form. "Abstraction of natural forms" is a method that expresses the mobility and naturalness of form represented by non-linear, free curves in the abstracted form of nature. Gentle curves and vivid curved surfaces are used to express visual changes and depth. "Imitation of natural structures" is a category of form represented by imitating the interaction and interconnectivity of nature, which takes the form of self-organization and is expressed similarly in cell structure. "Evolutionary morphological variation" is a structure similar to cell division and evolution that turns out to be a variation of a heterogeneous and unrealistic form. **Figure 3** summarizes the major cases of biomorphic images categorized into these four types.

#### Case Study of Parameters Moderating the Biomorphic Clothing Sculpture

A case study was conducted of 3D-printed clothing sculptures with BCS characteristics to explore parameters used in BCS modeling during the parametric design process of the BCS interface. The cases were those released as 3D-printed clothing. Keywords such as "3D printing fashion," "3D printing designer," "3D printing wearable fashion," "3D biomorphic fashion," "3D nature-inspired fashion," and "3D biomimicry fashion" were used in the search, and a total of 124 cases were collected. The cases were analyzed based on the theory of biomorphic design expression characteristics and parametric design form generation principles. Three components of ordinary clothing were used as mediator variables, and finally, parameters were moderated to "silhouette," "surface texture," and digital color related to BCS composition. The results of the case study were applied as parameters in the BCS interface design, and the results are presented as photos with a focus on those that best represent the attributes of each mediator variable. When the results could not be explained by the photos collected, BCS modeling algorithms were established using a rhinoceros and a grasshopper, and exemplar photos are used to present the case.

#### Silhouette

BCS is a 3D structure with a *Z*-axis instead of a flat 2D plane, and its application to fashion design requires consideration of the shape of the human body form. Usually, a design is comprised of the structure of design principles that create the surface effect, and the principle of BCS is organically related to the human body and materialized complex, integrated esthetic order. Thus, of the internal lines and silhouette that are the formative components of general clothing, only the silhouette was used as a mediator variable. The general clothing silhouette was classified by shape based on letters of the alphabet such as H and X, or subdivided into I, X, A, and Y, as well as by shapes of objects such as an arrow, hourglass, tulip, tunic, balloon, cocoon, princess, empire, or mermaid. The BCS in the case study turned out to be a more exaggerated silhouette than that of general clothing, perhaps owing to the 3D printing material and texture. Therefore, this study divided the most fundamental silhouettes of H and X into symmetrical typical and asymmetrical atypical and categorized them into four types. As the result of a case analysis based on the above, they were again subdivided into basic silhouette (s.), "shoulder exaggerated s.," "hip exaggerated s.," and "shoulder, hip exaggerated s.," with a focus on the horizontal width of the shoulders, waist, and hips. To achieve these distinctions, the user can make diverse and easy choices regarding procedures for form generation and processing in the interface. Accordingly, the user can choose from a total of 16 silhouette options. **Figure 4** shows the categorization of BCS silhouette types; the most common case is 3D-printed clothing in the typical X silhouette.

#### Surface Texture

Materials of BCS applied to the parametric design methodology were classified with a focus on the surface texture that occurs by adjusting the measurements of pattern size or thickness in the parametric form modeling process. In other words, BCS surface textures that might vary depending on unevenness or density of pattern size were categorized as "embossed," a type that shows differences in fabric surface thickened with embossing

or pattern; "lacey," which has a netted or laced surface"; "furry," a texture of long fur like animal fur; or "complex," in which multiple textures are mixed together. **Figure 5** illustrates typical cases of 3D-printed clothing sculptures for each of the four surface texture categories. Photos are also presented of examples embodied by adjusting the parameters of the BCS form algorithms developed to explain the clear distinctions among textures.

#### Digital Color

Digital color directly perceives light through color and is physically and emotionally differentiated from general color perceived through the process of light absorption and reflection. Interface color is based on the attributes of light and may appear not only in collage images but also in multimedia or video clips; it is digital color that expresses emotional symbols for user communication.

The interface has immaterial color characteristics that are non-fixed, amorphous, and non-constructive given a perception of continuous scenes or flow of time and space in which multiple faces are connected. Based on these color characteristics of the interface, this study used the color emotions and dominant colors that appear in artworks in the background of biomorphic art. YR and DK tones are used in biomorphic expressionist art to express artists' subjective emotions and responses based on objects and events, rather than objective facts. BCS colors used in the interface are mainly digital colors classified into acid colors using medium brightness and high chroma with high attractiveness, which excludes achromatic colors; blending colors evolving from the phenomenon in which two or more colors are physically mixed; blur color that forms a gradation in which three color elements change according to irradiation or tones disappearing; and chrome color with metallic expressions added to produce achromatic glossy material or a metallic feel of silver and platinum. **Figure 6** summarizes typical cases of 3D-printed works of each type as

well as the four types of digital color extracted from the cases of biomorphic expressionist artwork.

#### CONCLUSION: PROPOSAL FOR THE BIOMORPHIC CLOTHING SCULPTURE CONCEPTUAL INTERFACE

This paper presents a conceptual BCS interface that uses biomorphic design as a knowledge base using parametric design methodology from 3D form development of architecture. Rather than focusing on the technical aspects of 3D modeling, this study focused on developing an interface as a user-friendly playground with clothing sculpture modeling as the intermediary. A literature review and case study were conducted to gather knowledge-based data on interface and mediator variables for the parametric design process, and biomorphic design image categories were created using knowledge-based data that can predict biomorphic forms. The study also categorized silhouette, surface texture, and digital color as parameters required for building 3D BCS forms and determined the subordinate attributes of each mediator variable.

**Figure 7** illustrates the conceptual task pad of the BCS interface, which is divided into four parts: A, B, C, and D. There are five parts including the saved biomorphic inspiration image database. The images supported by the biomorphic inspiration image database category are saved in the database and can be loaded for use. The biomorphic inspiration image database is categorized, and thus, it is easy to save and process images. Data are saved in blob (binary) format. A collection of research images is provided, to which users can add more. A is comprised of four categories: File, Edit, Window, and Help. In File, the user can load an image file or save a created image. The user can perform basic tasks with the loaded image such as selecting, enlarging, and reducing it in the toolbox B or Edit tab. Specific image modification is carried out in C. C is a toolbox comprised of form generation mediator variables. The BCS interface changes the names of the parameters "silhouette" and "digital color," derived from the case study, to "formation" and "variation color," to make it easier for users to access them. In formation, detailed form characteristics of BCS for adjusting the shape are given as numbers. Multiple parameters can be selected, and there is no separate order of application. Moreover, detailed settings are possible for each mediator variable, and the values must be entered as numbers or ratios. Surface texture is determined through four techniques

#### REFERENCES


extracted from the case study, and the selected technique creates expressive technique algorithms through mathematical operations that adjust thickness or pattern size, and form generation methods of fractals, tessellation, and Voronoi, thereby altering the surface. In terms of color variation, types of colors extracted from the case study were used as mediator variables. Colors other than the proposed types can be added by users. Moderated images as parameters are generated as 3D objects according to parameters such as image, form, color, and technique, selected on the D monitor. **Figure 8** demonstrates the procedures for using the BCS 3D modeling interface.

This study is significant in that it used biomorphic design with various concepts and ambiguous emotions as knowledgebased data to predict the results of the parametric process as well as the forms of clothing sculpture interworked with fashion that changes with time as parameters through a case study. The results of this study will be provided as a user-friendly interface that is accessible to users simply by clicking on the option tool comprised of mediator variable items and will be thoroughly reviewed *via* future case studies.

In addition, this study was focused on the development of the conceptual interfaces; thus, the paper does not include descriptions of computational terms for modeling cases. Instead, we are preparing a further paper focusing on the modeling case of clothing sculpture, which will include a description of the modeling process in computational terms.

#### DATA AVAILABILITY STATEMENT

The datasets generated for this study are available on request to the corresponding author.

#### AUTHOR CONTRIBUTIONS

All authors listed have made a substantial, direct and intellectual contribution to the work, and approved it for publication.

#### FUNDING

This work was supported by the Ministry of Education of the Republic of Korea and the National Research Foundation of Korea (NRF-2016S1A5A2A01927666).


**Conflict of Interest:** The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

*Copyright © 2020 Yoo, Cho, Eum and Kam. This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) and the copyright owner(s) are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.*

# Immersive Experience Model of the Elderly Welfare Centers Supporting Successful Aging

Eun J. Lee<sup>1</sup> and Sung J. Park<sup>2</sup> \*

<sup>1</sup> Department of Architecture, Keimyung University, Daegu, South Korea, <sup>2</sup> Department of Architectural Engineering, Keimyung University, Daegu, South Korea

This study investigates the application of immersive experience services to leisure facilities for the successful aging of the elderly. In the past, the social image of the elderly was that of passive beings who needed care due to physical and cognitive decline associated with biological aging. However, the concept of "successful aging" actively highlighting the positive aspects of aging and trying to promote longer and healthier life has started to acquire importance in recent years. In this context, elderly welfare centers can be described as facilities that encourage learning, leisure, and social activities of the elderly with impaired physical and cognitive functions. The use of recent immersive experience technologies such as virtual reality and mixed reality (MR), in order to mitigate physical and spatial constraints and provide an immersion into the desired environment and situation, could contribute substantially to the health of the elderly. However, the application of immersive technologies is concentrated on the provision of entertainment, education, and medical facilities. The number of previous studies on the immersion experiences of the elderly is limited, and the connection between immersion experiences and various services and programs that promote successful aging at elderly welfare centers requires further research. This study analyzes the function and space of the elderly welfare centers for successful aging through the review of previous studies and classifies immersion technology categories based on the review of the relevant literature. The study analyzes the health benefits of immersive experience technologies and related products and services and proposes an immersive experience service model for the elderly welfare center. The results of the study could provide a valuable input for the spatial application of immersive experience technologies for successful aging in the future.

Keywords: successful aging, elderly welfare center, the elderly, immersive experience, immersive experience service, immersive technology

# INTRODUCTION

The rapid growth of the elderly population is a global trend. Korea has become an aged society in 2017, after only 17 years as an aging society. The proportion of people aged 65 or older currently exceeds 15 percent, with the elderly population expected to reach 10 million people by 2026, making Korea a post-aged society (Statistics Korea, 2018). Korea's entry rate to aging society is considerably

#### Edited by:

Xiangyu Wang, Curtin University, Australia

#### Reviewed by:

Daejin Kim, Iowa State University, United States Andreas Ihle, Université de Genève, Switzerland

> \*Correspondence: Sung J. Park sjpark@kmu.ac.kr

#### Specialty section:

This article was submitted to Environmental Psychology, a section of the journal Frontiers in Psychology

Received: 13 October 2019 Accepted: 03 January 2020 Published: 11 February 2020

#### Citation:

Lee EJ and Park SJ (2020) Immersive Experience Model of the Elderly Welfare Centers Supporting Successful Aging. Front. Psychol. 11:8. doi: 10.3389/fpsyg.2020.00008

**56**

faster than in other developed countries such as the United States (73 years), Germany (40 years), and Japan (24 years) (OECD, 2005). Population aging is generally perceived as an increase in older (over 65) dependent populations that need to be cared for financially and medically in terms of national and social aspects. However, with the continued improvement of the educational and economic levels of the elderly, there is a growing interest in "successful aging," a term which highlights the positive aspects of aging and involves a longer and more active life. Successful aging is the process of proactively coping with the fear of inevitable declines associated with an old age; this is done by maintaining high levels of physical, mental, and social activity and acting as an active rather than passive member of society. This new perception of aging requires a new paradigm different from the existing focus on the medical and welfare systems.

Recent studies suggest that "immersion experience" technology using information and communication technology (ICT) and virtual reality (VR) can promote the health and well-being of the elderly (Yamagami et al., 2007; Angelini et al., 2015). Immersive experience can contribute significantly to successful aging by inducing positive experience and facilitating communication of the user through a new or familiar virtual content. However, the existing immersion technology applications are largely provided for the purpose of entertainment, education, or therapy. They have limited application measures to the housing and hospital used to house and care of the elderly. The elderly welfare centers that provide leisure activities, disease prevention, and social interaction need to find ways to also provide these and other cost-effective services from the perspective of successful aging.

The purpose of this study is to develop the immersive experience service model that can be applied at the elderly welfare centers via linking the main functions of the elderly welfare center for successful aging with the immersive experience technology that can support it. This study analyzes the possibility of the positive immersive experience for the elderly and suggests an interaction and application method based on the programs supported by the elderly welfare centers as well as spatial needs. The immersive experience service model developed here suggests new developments in the field of immersive experience content and healthcare services.

### RESEARCH SCOPE AND METHODOLOGY

Several research steps have been taken in this study (**Figure 1**). Firstly, the concept of successful aging and the characteristics of immersive experience were analyzed through literature review. Secondly, the study analyzed the function and program space at the elderly welfare centers from the perspective of successful aging and reviewed the literature focused on identifying the positive effects of immersive technology for the elderly. Here, we analyzed the immersive experience technology and related products and services for the elderly. The literature reviewed included journal articles, extended abstracts, and conference proceedings, using databases such as ScienceDirect, ERIC, SpringerLink, and EMBASE. Three search keywords were used: "elderly," "immersive technology," and "immersive experience."

Thirdly, we derived service contents and related immersion skills based on the characteristics of the immersion experience and the function of the elderly welfare center. Fourthly, the immersive experience service composition system and a service platform were established, and the contents of the immersive experience service-based service platform were derived based on the functions of the elderly welfare centers. And finally, an immersive experience service model is proposed, taking into account the spatial function of the elderly welfare centers.

## SUCCESSFUL AGING AND IMMERSIVE EXPERIENCE

#### Successful Aging

"Aging" is a progressive degeneration process that is accompanied by a decline in biological function and an increase in mortality with age (Hayflick, 2000). With the development of medical technology and educational level, elderly health care has begun to focus on understanding not only biological aging but also social and psychological aging and on seeking ways to support the multidimensional needs and requirements of the elderly. In particular, as social myths about aging gradually shift from negative to positive, "successful aging" has emerged as the subject of neo-aging and has been studied in various definitions and measures over the past decades (Rowe and Kahn, 1998; Nusselder and Peeters, 2006; Ouwehand et al., 2007; Chodzko-Zajko et al., 2009). A theory of successful aging has been established to describe conditions meeting social and biomedical needs of the elderly. The former includes satisfaction with life, psychological well-being, social functioning, and so forth, whereas the latter emphasizes physical and cognitive functionality. Havighurst (1963) defined successful aging as social adaptation from a social psychology perspective, stating its importance for social stability and the flexibility of personality and relationships in this process. On a social side, successful aging implies that emotional attachment and self-esteem stem from personal experiences and memories, controllable circumstances, or circumstances not necessarily influenced by environmental characteristics (Baltes and Baltes, 1990) and that the relaxation of mind through involvement with nature is one of the main factors of being happy in old age (Bhatt, 2015). Successful aging on the biomedical side, on the other hand, implies minimizing the physiological degradation and increasing active life expectations through the prevention of chronic diseases (Chodzko-Zajko et al., 2009). Other successful aging factors include longevity, active (healthy) life, disease-free life, absence of chronic disease, reduced depression and loneliness, and high physical and mental functionality (Nusselder and Peeters, 2006; Ouwehand et al., 2007).

In contrast, Rowe and Kahn (1998) noted that aging should be considered in a variety of ways, rather than in one aspect. This can be explained by distinguishing between the usual aging and successful aging. Usual aging refers to helpless old age with no specific disease or dysfunction, but with risk factors

for disease, while successful aging can overcome or delay the universal aging with appropriate control of the environment and individual efforts. Thus, aging may be different depending on the environment or individual ability; therefore, supportability should be considered in individual and diversified aspects.

Recent studies have categorized the concept of successful aging into five different areas, namely, disease prevention and health, physical and cognitive functioning, psychological characteristics, social relations, and productive activities (Rowe and Kahn, 2015). Learning is often emphasized as a successful aging factor (Duay and Bryan, 2006), and research shows that participation in educational and creative activities in old age can have a positive impact on aging (Gallistl, 2018). The purpose of this study is to support successful aging in the five categories listed above. These are described below in more detail (**Table 1**).

Disease prevention is the prevention of risk factors for diseases and disorders. Preventive support is particularly important for older people because they may experience chronic or non-specific


illnesses including multiple medically unexplained symptoms (MMUS). Physical and cognitive functioning describes the potential for the elderly people to live independently and is also a prerequisite for successful aging. The psychological characteristics involve acknowledging aging and embracing various aspects of one's life, looking forward to life, and finding a meaning. Productive activities include a positive view of knowledge and skills and, in particular, educational activities that, along with self-development, can be a motivation for a new life goal (Vaillant, 2008). Finally, maintaining social relationships with others is related to the adaptability to the environment and provides an opportunity for an improved well-being. In other words, successful aging should strive to expand the individual's capacity and help maintain better functions, as well as to support active relationships and participation in productive roles rather than adapting to the existing society.

#### Characteristics of Immersive Experience

Recent advances in media technology have resulted in the development of the new display and communication devices that can simulate environments which are very similar to reality or simulate a complete virtual reality (Jeong, 2019). The user can then maximize their presence and immersion in the displayed environment and experience a natural and intuitive interaction with the 3D structure. The result is an immersive user experience and improved interaction technologies that provide an opportunity to better serve the elderly who are vulnerable to the information society and digital devices. The ultracompact devices or smart devices that are required in modern society cannot meet the physical, cognitive, and emotional needs of the elderly due to the small displays and the need for continuous operation. Immersive technology supports 3D virtual content and natural communication and helps to improve social interactions and physical functions by allowing the user to be

TABLE 2 | Characteristics of immersive experience and technology (based on study by Jeong, 2019).


immersed in the desired environment (Whitlock et al., 2012; Wiemeyer and Kliem, 2012; Wu et al., 2013).

Immersion means lifelike vividness of the user surroundings and provides various immersion experiences by maximizing reality and minimizing physical constraints. The immersive experience can display characteristics such as interactivity, virtual simulation, information processing, and virtual presence. The details of the immersion experience characteristics are shown in **Table 2** (Jeong, 2019).

Immersive experience promotes user-centered information transfer and exchange while leading to satisfactory results by allowing interaction with the environment as an active agent (Sauter, 2003). In other words, it satisfies the desire for empathy going beyond the sensory stimulation of the individual and has a positive effect on cooperativeness and social exchange. As a result, immersive experience has the potential to become a tool for achieving a physical, emotional, and social satisfaction. Based on all of this, it is essential to discuss the use of immersive experience in the context of health and welfare services provision.

#### ELDERLY WELFARE CENTERS AND IMMERSIVE TECHNOLOGY

#### Elderly Welfare Center Supporting Successful Aging

The elderly welfare centers in Korea are leisure facilities for the elderly, providing leisure activities, health care, and social activities for the elderly. They are similar to the elderly welfare centers in Japan and the multipurpose senior centers in the United States, as well as Altentagesstätta in Germany (Lim et al., 2014). The goal of the elderly welfare centers is to provide a comprehensive service for the elderly in different communities, for them to be able to live comfortable and independent lives (Park et al., 2013). To this end, the Elderly Welfare Act proposes operating and facility standards for these centers through the Elderly Welfare Center Enforcement Rules. The operation of the elderly welfare centers is defined as a provision of life and health counseling for the elderly, provision of employment information, rehabilitation or functional reinforcement training, and provision of programs to improve education including liberal arts. Additional detailed programs and services should be planned, taking into account the characteristics of each community and audience.

The elderly welfare centers are a community-oriented elderly welfare service network supporting the creation of local communities and the multidimensional welfare of the elderly following the current direction of elderly welfare practice which agrees with the "aging in place" (AIP) concept. AIP is a strategy for successful and active aging, meaning aging with people who are familiar in familiar areas. However, this strategy is also used in the expectation that community care is costeffective in mitigating financial burdens (Vasunilashorn et al., 2012). Accordingly, this study found similarities between the direction of successful aging and the ultimate goal of the elderly welfare centers. The functions of the elderly welfare centers were categorized into leisure and education, function recovery, counseling and employment, and social interaction promotion functions.

An elderly welfare center usually has a floor area of 500 m<sup>2</sup> or more and at least one essential facility such as an auditorium, an office, a counselor office, a restaurant, or a program room. As of 2018, there were 385 elderly welfare centers in Korea, with an average total floor area of 2,500 m<sup>2</sup> , which is about three times larger than that in Japan (Ministry of Health and Welfare, 2019). Compared to other elderly leisure welfare facilities, the ratio is high in scope, number, and number of users. However, since the 2011 amendment to the Elderly Welfare Act, there have been no significant changes in the provision of the programs in these centers (Hong and Yoo, 2017).

This study derives spatial composition based on the function of elderly welfare centers for successful aging (Park et al., 2013; Shin, 2016; Hong and Yoo, 2017; Kim et al., 2017). **Table 3** summarizes the association between the function of an elderly welfare center and successful aging. Depending on the function, an elderly welfare center can consist of up to 10 spaces for hobbies, physical activities, education, medical care, rehabilitation, counseling and employment, refection, and rest. Each space is classified into a detailed set of functions, based on the intended program of use. Based on the analysis of the spaces related to successful aging, we found an association between disease prevention (5), physical and cognitive functions (7), psychological characteristics (4), and productive activities (3) and social relationships (5).

# Immersive Experience Technology for the Elderly

A recent innovative immersion technology implementation included sophisticated computer graphic technology, network technology for massive data transmission, and a humancentered interaction solution that delivers five senses (Cummings and Bailenson, 2016). Immersion technology

#### TABLE 3 | Functions and space composition of the elderly welfare center for successful aging.


"•" means there is a connection between rows and columns.

is a computer-based device to improve user attention and concentration, in both physical and cognitive respects. It is made by combining technologies from various fields (Keshner and Kenyon, 2004). More specifically, immersive experience technology includes technologies supporting multimodal interaction to immerse the user in a virtual platform which is the result of a computer combined with the real physical environment. This study analyzed the characteristics of the immersive experiment and contents related to immersion technology through advanced research on technology for the elderly (**Table 4**). Previous studies focused on analyzing the elderly's interaction with virtual information and objects or techniques for immersing in information and content provided. Experimental studies related to the immersion experience of the elderly mainly focus on the contents of the Internet and smartphone use. There is insufficient empirical research on the immersion technology utilization and immersion experience. Immersion technology and immersion experience for the elderly can be classified into interactive video game-based (IVGB) games, robots based on artificial intelligence (AI) or natural language understanding (NLU), and immersive content using augmented and mixed reality. The results of previous studies show that immersion experiences are related to physical, mental, and social functions of the elderly, especially the IVGB training games that improve sensory and motor skills, perception, and cognition (Whitlock et al., 2012; Lai et al., 2013) and form relationships with the grandchildren (Wollersheim et al., 2010). The AI robots used for social care promote social exchange and emotional relaxation (Libin and Cohen-Mansfield, 2004; Wada and Shibata, 2006), and the use of immersive content can enhance life satisfaction and family bonds (Cornejo et al., 2013).

However, the results of our analysis results indicate that research on immersion technology for the elderly is immature and is mostly based on experiments focused on the Internet and smartphone usage. It is important that immersion technology induces interaction with virtual reality, virtual information, and virtual objects and that it functions to immerse the user in the information or content provided.

Hence, we can say that there is a need for the adoption of major new technologies such as visual reality (VR), augmented reality (AR), AI, NLU, and holographic reality (HR), in order to enhance the immersive experience for the elderly.

Recently, the immersive experience services have been successful at providing products and services in various fields, such as entertainment, education, and medical care. Moreover, with the advancement of major technologies, these are also being used in a more complex manner. This study investigated trends in immersive technology-based products and services that support successful aging. Priority was given to the immersive technologybased products where the customer base or target audience was elderly or where the product and service objectives could function for successful aging. The specific case analysis referred to the information provided by the official homepage of the product and service provider, based on the fact that the official homepage is the best medium to provide clear and detailed information on the relevant product. The products and services based on immersive experience technology for the elderly are presented in **Table 5**.

Virtual reality and AR technologies focus on health benefits that could be achieved through the provision of an immersive positive experience with nature. Examples include a lamp to project virtual sunshine designed by Leslie Nooteboom, a design engineer in England, and an artificial skylight to provide the

#### TABLE 4 | Summary of previous studies on immersive experience for the elderly.


(Continued)

#### TABLE 4 | Continued

fpsyg-11-00008 February 7, 2020 Time: 15:20 # 7


"•" means there is a connection between rows and columns.

experience of sunshine and sky proposed by Komorebi and CoeLux, a company for developing light systems. The startup company Rendever in the United States provides a step-bystep solution for education and recuperation of the elderly community, as well as a platform that allows users to experience a virtual natural environment. Maplewood Senior Living Center in the United States, which receives Rendever's services, is actively adopting VR/AR technology, including a creation of an immersive "Skylounge."

The AI/NLU technology can provides services to the elderly in everyday life and emergencies, based on natural language recognition for smooth communication and learning from the collected data. In a related case, Japan recently developed an animal-shaped robot as a solution to address the isolation and loneliness of the elderly; the robot provides user benefits such as communicating with animals and being comforted without having to look after a real pet (Banks et al., 2008). Paro, developed by the National Institute of Advanced Industrial Science and Technology (AIST) in Japan, is a seal-shaped robot that consists of four main senses: AI-based and NLU-based vision, hearing, balance, and touch. The assessment of the benefits of its use showed that the psychological status of the elderly was improved to a level similar to that of living with puppies (Wada and Shibata, 2007).

Other examples include Sony's Aibo and Omeron's NeCoRo, which have similar functions and purposes. In addition, KIA's Real-time Emotion Adaptive Driving (READ System), presented at the Consumer Electronics Show (CES) in 2019, could recognize the driver's vital signs and control the five senses in the vehicle to optimize the internal space of the vehicle considering a driver's emotions and situation in real time. In other words, current AI technology can adjust cameras and various sensors controlling music, temperature, lighting, and aroma based on user emotions and surrounding conditions.

#### TABLE 5 | Product and services used in immersive experience technology.


Finally, HR technology is a major technology that can maximize visual immersion and create virtual objects that can interact with the physical environment in 3D (Buckley, 2011). Advances in screenless display technologies have led to the development of projector-based hologram tables and hologram projectors. The "HealthVoyager," an immersive medical observation system at Boston Children's Hospital, can provide medical care in 3D for young patients and caregivers. In addition, the HoloLens, developed by Microsoft, also provides 3D virtual objects that can interact with users in real time. Medical science is exploring how to use HoloLens to improve our understanding of telemedicine and care (Sirilak and Muneesawang, 2018). As such, the development and use of immersive experience technology suggest that environmental elements such as color, texture, and temperature/humidity can function as immersive tools. This makes it essential to be used to support our aging society.

#### IMMERSIVE EXPERIENCE SERVICE MODEL IN THE ELDERLY WELFARE CENTER

#### Framework of Immersive Experience Service

The immersive experience service is an ecosystem-based industry where contents, a service platform, network, and a device are all combined. The service requires an open platform to implement the related technologies. The service platform consists of components with individual functionalities, such as "input" to collect data, "process" to informationize and process the data, "store" to store the collected data and processing results to reuse them, and "output" to understand and share the characteristics of the results data. **Figure 2** presents the framework of the immersive experience service proposed in this study.

The framework of the immersive experience service for successful aging informationizes the input data, synthesizes the processed result, and sends it to the device that realizes the service according to the program content, in order to support successful aging. The user can experience immersion via the provided service. Components and content of the immersive experience service platform for successful aging are as follows. The input data include biometric information, environmental information, user-specified information, and so forth. The data input from a user or an administrator, such as program information provided by a welfare facility or user-specific matter, is user-specified information. The input data are collected for the analysis or analyzed according to predefined principles and then visualized or natural language processed to produce a result. Each collected datum and result are archived for reuse and serviced through a connected output device.

#### Immersive Experience Service Platform for Successful Aging

**Table 6** shows an immersive experience service platform for successful aging.

A total of 20 services were identified in this study. The input component data, according to each content item, are biometric

data, such as user health information and emotional state; environmental data, such as temperature/humidity, direction, light, and pressure information; and user-specified information, such as the support program and structure, elderly learning level, and parental contact information. In particular, as successful aging is a multidimensional and complex concept, it is important to continuously manage the knowledge of experts and facility personnel in each field such as individual counseling contents, physical and cognitive ability levels, and family relationships, in order to support various characteristics of elderly welfare center users. That is, it is important to provide a customized immersive service through real-time update and input of userspecified information. Information processing can be divided into two groups: VR/AR/HR graphic processing for visualizing analysis based on each data and AI/NLU program for recognizing and expressing natural language with a predetermined learning pattern. When the service is ready to deliver via each processing method, it can be outputted to the connected VR/AR/HR/AI/NUL device.

We presented psychologically immersive experience services as a series of examples. From the perspective of successful aging, it is important to support psychological traits to provide a controllable environment and qualities that remind us of good experiences and memories (Baltes and Baltes, 1990; Bhatt, 2015). To this end, the immersive experience service implements a VR graphic based on personal experiences or memories and can provide access to a virtual environment (e.g., a childhood home) through a VR device (S.10 and S.12). Furthermore, it can provide an immersive environment that allows interaction with virtual nature elements (sunlight, sky, animals, plants, etc.) and can be adjusted through AR lighting and HR projector through an intuitive AR/HR graphic process (S.8 and S.10). Finally, vocal and facial expression pattern information is analyzed by the AI and NLU programs to provide a customized indoor environment through the autonomous control device (S.11).

# Immersive Experience Service Model by Space of an Elderly Welfare Center

Based on the derived immersive experience service, this study proposes an immersive experience service model by space of an elderly welfare center for successful aging, taking into account the functionality of an elderly welfare center. **Table 7** shows an immersive experience service model based on an elderly welfare center.

Immersive experience content and services are provided through successful aging and functionality of the center. As mentioned above, these can be applied to up to 10 spaces in these centers. Additionally, a user can experience immersion such as presence, interactivity, virtual simulation, and information processing depending on a service provided. To apply this service in each space, the lobby and educational spaces need to support customized virtual experience activity for each program based on VR/AR technology. It is also important to create an immersive multimodal participation-based learning environment by sharing the results with the elderly in other communities. The physical

#### TABLE 6 | Immersive experience service platform for successful aging.


activity space should support enjoying the virtual activities together with the other elderly by using social VR. In an autonomous space, such as a fitness room, it is necessary to learn the behavioral patterns for fall and have virtual coaches for exercise equipment. In a medical space, remote treatment through a hologram is needed due to the absence of a specialist, and a medical note service is provided using a virtual object to help understand more about the health condition. The rehabilitation space needs to provide an autonomous environment adjustment, such as providing and recovering VR content step by step, in order to improve cognition and physical function, and light and temperature/humidity according to the emotional state. In the counseling and employment space, individual counseling and employment support should be provided based on the learning ability and aptitude of the elderly, and it will be important to build confidence in job activities following retirement, through virtual job experience. The elderly welfare centers providing lunch should develop a method to provide information in an easy way using an AR application, such as analyzing the health conditions and diet-related information of an elderly person to provide customized meals and health-related information through food photos.

The resting area is a center of immersive experience that can provide up to nine services. It can provide a dynamic nature immersion environment, such as a virtual sky, sunlight, HR/AR, and natural creatures (plants, birds, dolphins, and so forth), for psychological well-being and relaxation. Lastly, the public space in these centers is a space where exchange and movement occur. As the elderly welfare center program is conducted in different places

#### TABLE 7| Immersive experience service model according to the elderly welfare center.


"O" means thereis a connection between rows and columns.

depending on the individual timetables, it will be important to support navigation by providing an AR navigation tool that can guide the route in real time. In addition, an AI social robot that recognizes and expresses natural language can be used as a primary means of communication and provision of information.

#### DISCUSSION

This study proposed the immersive experience service model through linkage with immersion technology, considering the function and space composition of the elderly welfare center for the successful aging of the elderly. The architectural field has emphasized the improvement of physical environment or physical support focused on an evidence-based design, a universal design, and a barrier-free design. The purpose of this study was to explore the elderly welfare center environment from a new perspective by linking immersive experience skills that improve life satisfaction and social interaction, as a way to support mental and social functions important in successful aging. In addition, the immersive experience service discussed in this study can alleviate the physical and spatial constraints of the elderly and can function positively in successful aging by providing immersion into the desired environment and situation. Thus, it has sufficient value to support its application in the elderly welfare centers. The results of this study are summarized as follows.

First, successful aging is a process of actively coping with deterioration and impairment of old age, and can be defined as disease prevention, high level of physical and cognitive function, psychological stability, productive activity, and social relationships. Therefore, it is necessary to prepare support measures for successful aging in a comprehensive manner through linkage with local communities and by applying technology to efficiently provide this. In particular, the elderly welfare centers, as facilities used for the leisure and education of the elderly, recovery of skills, counseling and employment, and social interaction, must compose programs and provide relevant services to support successful aging. In this study, we analyzed the spatial composition of the elderly welfare centers by function to support successful aging by classifying these into hobbies, physical activity, education, medical, rehabilitation, counseling, employment information, refection, rest, and common spaces.

Second, the analysis of the previous studies shows that the effects of the immersion experience skills of the elderly are improved through improved physical balance and fatigue levels (Lai et al., 2013; Wu et al., 2013; Calogiuri et al., 2018), attention and memory (Zelinski and Reyes, 2009), and reduced depression, improved social skills, and stress reduction (Wu et al., 2013). This study derived the main technologies supporting the immersive experience of the elderly. These included VR, AR, AI, NLU, and HR. The immersive experience should be provided with the contents according to the functions of the user and the space as the main technologies are combined. Based on the results of our review examining the product and service trends of immersive experience technology, VR and AR technology include natural immersion product development, virtual training and rehabilitation content, and a path-finding application that induces a positive experience with nature. In this case, there are AI social robots for reducing the depression of elderly people through emotion recognition of facial expression and biosignal analysis and an autonomous control system for natural language communication. HR technology is developing products such as projectors and wearable devices to implement holograms, and medical finding systems and telemedicine are being increasingly used.

Third, the immersive experience service consists of content, platform, network, and device, and the service platform includes components of input, processing, storage, and output functions. This study constructed an immersive experience service platform for successful aging, with the input variables including biological information, environmental information, and personal information of elderly people. Processing collects and analyzes information about the input data and then visualizes or uses natural language processing to produce the result. The storage maintains the generated results for reuse, with the output being a device that presents the generated results to the outside world. The elderly are provided with the immersive experiences of presence, interactivity, virtual simulation, and information processing characteristics depending on the elderly welfare center's available space and programs.

Finally, we derived 20 service content items considering the goals of successful aging and functions of the elderly welfare centers. We also suggested an immersive experience service model based on the functionally classified space of the elderly welfare center. The service content for each space is as follows. The hobbies and educational spaces support virtual experience activities according to the program and provide a multiparticipatory learning environment that shares the results with the elderly of other communities. In the physical activity space, virtual activities for each group should be performed according to the level of physical function, and in the case of an autonomous activity space without a monitor, a fall prediction system and a hologram virtual coach will be required for the safety of the elderly. As medical spaces in the centers are not manned, it will be important to support a good flow of communication with the elderly, such as delivering medical results using virtual objects during remote medical care and providing medical treatment using holograms such as HoloLens. Rehabilitation spaces need to support step-by-step learning content for cognitive and physical function training and provide a customized environment based on the physical and emotional states of the elderly. The counseling and employment space will provide consultation and employment information by analyzing the elderly learning data and aptitudes, as well as providing virtual job experiences and virtual education. Food and beverage spaces will need to provide food based on individual diets and other food-related information using AR applications, and rest spaces could support psychological well-being through the interaction with animals and plants using a virtual natural environment. Finally, public spaces could provide a real-time route search and an AR navigation according to the individual program timetables and also provide information on facilities for the elderly welfare centers and support natural communication using social AI robots.

# CONCLUSION

fpsyg-11-00008 February 7, 2020 Time: 15:20 # 13

This study sought to apply immersive experience technology to the successful aging of the elderly. The immersive experience service was derived by linking the survey contents with the recently released immersive technology. The service of the elderly welfare centers for successful aging was presented as 20 applications methods by space, according to the functional classification of the elderly welfare centers. The study proposed the use of an immersive experience service system based on the characteristics of the immersive experience, according to service content. This study is a novel Contribution To The Field of elderly friendly architecture and construction. By subdividing the immersive experience service system according to the services provided by the existing elderly welfare centers, this study contributes to the improvement of applicability of the immersive experience service system to the real-life elderly welfare centers. Finally, this study provides a direction in which to expand the application of immersive experience technology and services for the elderly, including the entertainment, education, and medical fields. However, it should be noted that the immersive experience service model proposed in this study deals with the current program content and is limited to the currently developed technology. A further study is required to analyze the existing quantitative data in a clear and rigorous manner (including conducting in-depth interviews with the users) and to broaden the scope of the search criteria to alleviate the limitations on the

#### REFERENCES


use of immersion technology for the elderly. The ultimate goal of this study is to apply immersion technology that can interact with programs for each space in the physical environment, such as at the elderly welfare centers, assuming that the experience of immersion and the characteristics of the technology can function positively toward successful aging. This can be of value for the development of the elderly welfare centers.

# DATA AVAILABILITY STATEMENT

All datasets generated for this study are included in the article/supplementary material.

# AUTHOR CONTRIBUTIONS

EL and SP conceived and designed the research. EL collected and analyzed the information by reviewing the literature and investigating the case, and providing a substantial contribution to writing of the manuscript. SP provided the insights about the outline in the process of research progress, and also consistently examined, modified, and supplement the manuscript. Both authors interpreted the results and the meaning of the research, and approved the final manuscript.

# FUNDING

This study was funded by the Korea Research Foundation grant. Project number: NRF = 2018R1C1B6008735.


Hayflick, L. (2000). The future of ageing. Nature. 408:267. doi: 10.1038/35041709



**Conflict of Interest:** The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Copyright © 2020 Lee and Park. This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) and the copyright owner(s) are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.

fpsyg-11-00219 February 14, 2020 Time: 18:55 # 1

# Effective University Facility Management Plan Proposal Reflecting the Needs of the Main Users

#### Min Soo Kim and Jun Ha Kim\*

Department of Housing and Interior Design, Kyung Hee University, Seoul, South Korea

#### Edited by:

Mi Jeong Kim, Hanyang University, South Korea

#### Reviewed by:

Hyun-Bo Seo, University of Seoul, South Korea Daniel Castro, Georgia Institute of Technology, United States Baabak Ashuri, Georgia Institute of Technology, United States

> \*Correspondence: Jun Ha Kim junhakim@khu.ac.kr

#### Specialty section:

This article was submitted to Environmental Psychology, a section of the journal Frontiers in Psychology

Received: 14 October 2019 Accepted: 31 January 2020 Published: 18 February 2020

#### Citation:

Kim MS and Kim JH (2020) Effective University Facility Management Plan Proposal Reflecting the Needs of the Main Users. Front. Psychol. 11:219. doi: 10.3389/fpsyg.2020.00219 As the low birthrates in Korea intensify, the average school-age population is showing great reduction and the resources used for admission to the university are gradually decreasing as well, due to the continuing economic downturn. Therefore, in order for the university to remain competitive and keep up with the fast paced changes, it is very significant that they induce qualitative growth of university education and facilities through extensive and thorough analysis on the university facility management (UFM) for students who are the main users of university facilities. However, research done on university facilities in Korea has been focused mainly on space management, while the effective UFM reflecting in-depth opinions on the user class has been quite inadequate. Therefore, in order to improve student satisfaction and improve the efficiency of UFM, it is crucial to understand the importance and status of the UFM items. This way, an initial plan for improvement of UFM considering the priorities that actively reflect the opinions of the students is prepared in this research. For the UFM items, the eight classifications of UFM, which can be evaluated by users, and the details of the classifications are used for this research. For the UFM strategies, the first 176 performance indicators (PIs) are collected, consolidated, and deleted. Finally, eight UFM strategies are derived. In order to find out which UFM items need more focus on, importance–performance analysis (IPA) is conducted. The priority of management is determined by where each factor is located on the grid. Additionally, multiple linear regression analysis is conducted to examine the effect of the importance value on the UFM items on the importance value on the UFM strategies. Finally, the average values of importance for the strategies of UFM of two groups are compared and analyzed. As a result of the stages listed above, this research attempts to provide basic data on preparing the UFM plan. Therefore, it is possible to apply the method that reflects the needs of actual users in preparing future UFM plans throughout the research methods proposed in this research.

Keywords: university facility management, UFM items, UFM strategies, importance–performance analysis, multiple linear regression analysis

# INTRODUCTION

fpsyg-11-00219 February 14, 2020 Time: 18:55 # 2

Universities are the institutions that play a central role when it comes to driving technological development and social changes as the infrastructure for performing various functions such as education, research, and community service. As we enter this era, the role of universities and social responsibility to foster the level of human resources required by society along with national competitiveness are expected to increase (Kim et al., 2006; Hassanain, 2008; Akinyode, 2014).

Despite the importance of the various roles that universities hold, most Korean universities are currently faced with a serious crisis (Jeon, 2009). As the low birthrates in Korea intensify, the average school-age population is showing great reduction and the resources used for admission to the university are gradually decreasing as well, due to the continuing economic downturn. In fact, in 2015, the number of universities across the country was 390, including 51 national and public universities, and 339 private universities. However, the number of colleges and universities decreased to 384, with 49 national and public universities and 335 private universities in 2018. As a result, not only is competition among domestic universities intensifying, but additionally global competition among universities are showing greater rivalry, and efforts to secure qualitative competitiveness of university education and facilities are becoming a major required factor.

Therefore, in order for the university to remain competitive and keep up with the fast paced changes that will be occurring in the future, it is very significant that they induce qualitative growth of university education and facilities through extensive and thorough analysis on operation and management for students who are the main users of university facilities (Reynolds and Cain, 2006; Kim et al., 2018).

However, research done on university facilities in Korea has been focused mainly on space management, while the effective university facilities management reflecting in-depth opinions on the user class has been quite inadequate. Unlike other countries, in Korea, the research that is performed based on the factors that influence the learning outcomes of university facilities are not active, which leads to the need of users in the field to obtain higher education (Shin and Kim, 2012). Under all of these circumstances, applying facility management (FM) strategy to the university facilities is a strong requirement (Shin and Kim, 2012). FM in universities generally mean that it provides a suitable environment for education and research purposes which is the main objective of the university. By operating and maintaining the university facilities in an optimal state, it reduces the operating costs by optimizing maintenance activities. In other words, university facilities, unlike general facilities, consist of facilities that require diverse functions, such as basic education facilities, research facilities, and support facilities. The need for research on efficient management is even higher.

Therefore, in order to improve student satisfaction and improve the efficiency of university FM (UFM), it is crucial to understand the importance and status of the UFM items. This way, a plan for improvement of UFM considering the priorities that actively reflect the opinions of the students can be prepared in this research (**Figure 1**).

# BACKGROUND

# University Facility

University facilities are the basic hardware when it comes to university research, education, and academic activities (Lee and Cho, 2008). It is also a spatial background in which various functions, such as university's main education, research, and student's rest, are performed. It has already been verified through many previous studies that these university facilities, just like other educational facilities, perform a great influence on the quality of education provided in the university and the satisfaction of students as the main users (Kim et al., 2006; Hassanain, 2008; You and Lee, 2008). Association of higher educational facilities in the United States indicates that universities can effectively achieve their predetermined goals and objective by the systematic FM performance measure (Appa, 2007). University facilities are largely divided into four facilities: basic education facilities, support facilities, research facilities, and attached facilities according to Article 4 of the Korean College Operational Regulations (**Table 1**). These university facilities are composed of various functions such as educational function, research function, and support function. The facility can tend to be complex and used by various users, so it follows with various requirements for maintenance. Accordingly, in order for efficient management to take place, it is necessary to prepare an effective management plan in consideration of the users' position.

#### University Facility Management

The International FM Association (IFMA) defines FM as a profession that encompasses multiple disciplines to ensure

#### TABLE 1 | Classification of university facilities.

fpsyg-11-00219 February 14, 2020 Time: 18:55 # 3


functionality, comfort, safety, and efficiency of the built environment by integrating people, place, process, and technology. Another definition comes from the 29 European countries, where they defined FM as an integration of processes within an organization to maintain and develop the agreed services which support and improve the effectiveness of its primary activities (EN15221-1: 2006 FM – part 1: terms and conditions). UFM is an integrated service activity that is comprehensive and needs long-term planning and management for the purpose of minimizing costs, maximizing utility, and flexible maintenance. This can be done by optimizing fixed assets that are used and held for university purposes. For example, physical services can be done in order to ensure that a building's air conditioning is operating effectively or safely. In depth, physical services are a series of actions to confirm whether a building's heating and cooling are operating effectively or safely (Kim, 2016).

Moving on, intangible services refer to a service that checks whether a building is kept clean and is well supervised by a contractor or a manager. FM is frequently used similarly for space management or asset management, however, FM can tend to be limited when it comes to commercial assets that are a bit more complex to manage and operate or have more of a broader spatial scope.

Furthermore, the university's FM is directly related to the financial aspects, and the facility maintenance management routinely inspects, maintains, and repairs damaged facilities so that they can preserve the function of the completed facility and also enhance the convenience and safety of users. The legal definition of facility maintenance management is focused on corrective maintenance in the comprehensive FM domain. Yet, the term "FM" in this research covers the academic content of FM, which is beyond its legal meaning. Additionally, other studies pointed out that the necessity for preparing and making systematic, active, and effective future FM plans (Shon et al., 2003; Cho and Lee, 2008; Ryu and Lee, 2008; Yun et al., 2009). Therefore, in this research, "FM" is referring to the abovementioned corrective maintenance in reactive manner and also the wide ranging activities performed for the efficient use of the facility done by effectively planning and managing that prevents future accidents or inefficiencies.

# Satisfaction on UFM

The satisfaction research on the educational facility has been actively conducted all around the world (Kim, 2019). One factor that all of these studies have in common is that satisfaction is measured from a variety of perspectives. Astin (1993) categorized the limited satisfaction level by estimating the students "satisfaction in a specific field, such as surveys of students" perceptions, the professor's lecture scores, and administrative services. Yorke et al. (2000), on the other hand, provides satisfaction to the United Kingdom university students by labeling them by the areas of curriculum organization, teaching and learning, library, computer training facilities, computer facilities, student service, school environment, restaurants and lounges, student council activities, and self-development opportunities. Ruben (1995) classified satisfaction as quality of teaching, quality of administrative service, and quality of teaching and learning. Lastly, LeBlanc and Nguyen (1999) evaluated satisfaction by analyzing it into functional value, epistemological value, social reputation, justice value, economic value, and social value.

# MATERIALS AND METHODS

### UFM Items Selection

This research conducted a literature review of Korean Standard (KS) and previous studies to establish a classification of UFM items that can be evaluated by users. Our research tools are based on the previous research (Shin et al., 2015) that adopted "FM KS FM Service Standard Specification (KS S 1004-1: 2011)" and the "KS FM Standard Specification Based on KS FM Standard Specification." The eight classifications of UFM, which can be evaluated by users, and the details of the classifications are used for this research. The total number of users consists of 57 male (39.6%) and 87 female college students. The classification and details of FM survey tools used are shown in **Table 2**. There are eight classifications of university facilities management survey tools set up in this study, starting with "Ensuring the Safety of the Facility" and seven more areas.

#### UFM Strategies Selection

In order to derive the strategy of UFM, performance indicator (PI) data are collected. In-depth analysis of previous researches used to identify management tasks carried out in universities. Using this data, previous research data related to domestic and foreign PIs are investigated and analyzed.

First, 13 papers and conference proceedings related to FM service evaluation index are collected. After that procedure, PIs of basic UFM services are analyzed, and then PIs for domestic and foreign university facilities are analyzed. In order to narrow down the scope of UFM strategies to these collected PIs, PIs with high similarity are consolidated, followed by two Delphi surveys of UFM experts (**Table 3**).

fpsyg-11-00219 February 14, 2020 Time: 18:55 # 4


(Continued)

#### TABLE 2 | Continued

fpsyg-11-00219 February 14, 2020 Time: 18:55 # 5


Determining the necessity of integrating and modifying the PIs, which are the raw data, in order for finally selecting the strategy of the UFM, the first 176 PIs are collected/consolidated and deleted. Finally, eight UFM strategies are derived (**Figure 2**).

A total of 176 pieces of original data of PIs related to UFM are collected, excluding items related to the exterior of the building, parking, site, landscaping, construction, moving, etc., which are out of scope in this research. For a case of high similarity, 44 PIs are determined through the integration process.

An expert group consisted of UFM practitioners at Korean universities were asked to investigate the importance of the extracted PIs, Delphi survey. A panel of experts was selected under one condition; at least over 5 years professional experiences in a UFM. Thirty-three UFM experts (23 male and 10 female) participated to evaluate the importance of PIs and this survey is conducted using the five-point Likert scale. The data collected in the first and second rounds are analyzed using SPSS 20. Based on the mean value of the importance, 20 PIs scored above average are extracted. Subsequently, an additional second Delphi survey is conducted to select final UFM strategies based on the importance of the PIs (**Table 3**). According to a three-point scale ("1" = not


fpsyg-11-00219 February 14, 2020 Time: 18:55 # 6

necessary, "2" = important but not essential, "3" = essential), the content validity ratio (CVR) is also applied to ensure the adequacy of eight UFM strategies and the equation 1 was used as follows:

$$\text{CVR} = (n - N/2)/(\text{N/2})\tag{1}$$

where n is the number of experts indicating a UFM strategy is "three-essential" and N is the total number of experts on the panel. All eight UFM strategies got a CVR of 0.33 or higher indicating that at least 66% of the panel (22 out of 33 experts) rates the items to be essential. There are three lagging PIs (#1, 2, and 3) that are output oriented, relatively easy to measure but hard to improve, on the other hand, there are five leading PIs (#6, 7, 9, 10, and 18) that are input oriented, relatively easy to improve but hard to measure accurately.

#### Importance–Performance Analysis

In order to consider the relationship between two different factors such as the importance and the performance, importance– performance analysis (IPA) method is used (Martilla and James, 1977). This research is to examine the average value through satisfaction and importance survey on 35 university facilities management items, and then to select the items to input management manpower and cost through IPA. As shown in **Table 2**, the users' overall satisfaction level is 3.26 and the importance level is 3.99, indicating that Gap (B−A) 0.73 shows a lower level of satisfaction compared to the importance of overall management items.

Therefore, as shown in **Figure 3**, it is fundamental to adjust the direction toward improvement of the satisfaction of the five management items in the second quadrant. This can be done by adjusting and redistributing resource inputs to the five management items in the fourth quadrant. In detail, the items with low satisfaction to importance, five items that need to be improved by urgently inputting human and material resources are "secure passage space" (#3), "voice and braille guide (#20)," ensuring evacuation space in side of the building "(#2)," ensuring evacuation space around the building "(#4)," establishing a control system "(#5)." On the other hand, the five items with high satisfaction to importance that have an excessive input of human and material resources are "obstacle removal management for various activities" (#18), "maintenance of guidance systems" (#19), "securing field of vision" (#6), "continuous maintenance of facility performance" (#12) and "providing artificial lighting environment" (#28).

### RESULTS

#### Multiple Linear Regression Analysis of UFM Items for UFM Strategies

In order to identify the UFM items that affect the strategy of UFM, multiple linear regression analyses are conducted in order to find out how an UFM item is numerically related to the UFM strategy, examining the effect of the importance value on the UFM items on the importance value on the UFM strategies (**Table 5**). The tolerance limit of all variables are >0.1 and the VIF value is <10, so it is judged that there is no problem on the collinearity. The overall explanatory power (R 2 ) of the regression equation is 15.9–36.8%. The following table shows the UFM items that affects the eight UFM strategies (**Table 4**).

Among the entire UFM items, the items that affect the "operation cost-related activities" of UFM and have high standardized coefficient beta are "water quality management (0.266)" and "artificial lighting management (0.261)." This is related to energy such as water resources and electric energy, and it can be seen that students think there is a high correlation between energy consumption and operating cost. In connection with the entire UFM items, the items that alter the "building O&M-related activities" and have a high standardized coefficient beta are "securing facility performance standards manual (0.290)," "water quality management (0.222)." Through this, it can be concluded that college students believe that building O&M-related activities cost can be reduced by management plans, for instance, securing management manuals and strengthening water quality management.

The items that influence the "public expenses" and have high standardized coefficient beta are "temperature control system (0.208)," "security authentication system (0.229)," and "individual equipment for heating and cooling operation (0.227)." It shows us that students tend to think that temperature control systems and heating and cooling operation systems can possibly reduce the public expenses by controlling the usage. With the introduction of security systems, the public expenses are expected to be scaled down by saving the input of human resources. The items that are involved with the change of the "facility acquisition adequacy" and have high standardized coefficient beta are "finishing material quality control (0.275)," TABLE 4 | Multi-linear regression of UFM items for UFM strategies.

fpsyg-11-00219 February 14, 2020 Time: 18:55 # 7


"water quality management (0.274)," "integrated control system (−0.229)," and "individual equipment for heating and cooling" (0.197). A possible conclusion from this result, as the contents of securing the facility itself are about the interior space where students mainly live, is that it is related to quality control of finishing materials. Another thing that can be spotted is that that the heating and cooling control determines the appropriateness of securing the indoor space.

The items that affect the "space allocation adequacy" and have high standardized coefficient beta are "finishing material quality (0.462)." Considering that the main spaces used by students are lecture halls and rest spaces, the use of appropriate finishing materials can be used as a criterion for judging the suitability of students for space allocation. The items that modify the "energy goal management" and have high standardized coefficient beta are "finishing material quality (0.282)" and "individual equipment for heating and cooling." To sum it up, the operating equipment of heating and cooling machinery, which is directly related to energy, influences the energy goal management. Likewise, the correlation between fpsyg-11-00219 February 14, 2020 Time: 18:55 # 8


the quality control of the interior finishing material and the energy consumption is immense. The items that affect the "safety management" and have high standardized coefficient beta are "securing evacuation space (0.286)," "water quality management (0.320)," "maintenance for rest (−0.266)," and "security equipment installation (0.213)." Safety management is one of the most significant aspects of UFM strategies and it can be seen that the management of crime prevention and security system can make a difference when discussing the matter of safety for our students. Securing the evacuation space is also another factor that is directly related to their safety in case of an emergency. There is a negative correlation with the management items related to the rest space and as shown, the management items related to rest are not associated with the psychological stability of students. The last items that affect the "manpower procurement" and have high standardized coefficient beta are "sensor detection and automatic light off management (0.268)" and "security equipment installation (0.244)." Regarding securing manpower, students seem to think that manpower resources can be diminished by using automatic extinguishing systems and security equipment.

In conclusion, UFM items that influence the eight UFM strategies the most are "water quality management" (that has the positive impact on the four strategies), "operating expenses-related activities," "bldg. O&M-related activities," "facility acquisition adequacy," and "safety management." Based on these results, in order to prioritize UFM items in the future, it is mandatory to manage the water quality from a higher level as well as managing cost-related items, facilities, and safety. The followings are the UFM items that commonly have an effect on the strategy of UFM (**Table 5**).

#### Importance Differences in the Strategy of UFM Between Users and Managers

It is commonly assumed that the problem of UFM is not being operated efficiently. However, issues are caused when FM is conducted without accurately reflecting the needs of users of university facilities. Therefore, this research examines the difference in the strategy of UFM between users and facility managers (**Table 6**). As a result of examining the importance gap between users and facility managers, all areas show negative values. In general, facility managers tend to guess that the strategy of UFM is more critical than students who use facilities. The biggest difference among them is "public cost-related activities (electricity, gas, and water resource usage fees)." This indicates that for facility managers, cost reduction is much more important than for students. In reality, it can be seen that they respond more sensitively toward the costs incurred in operating the facility. The second contrast among them is "manpower procurement." In other words, it means that it is more of a priority to the manager who directly manages it, than the students who use the facility. On the other hand, the item with the least difference is shown as "safety management." This is the most important FM goal in both groups along with both groups identifying safety as their top priority. Among the three items related to costs, there are differences that are higher than the average cost in "building O&M-related activities" and "public expenses-related activities." This indicates that managers see the cost reduction as the main strategy of FM compared to users who value convenience in simply using facilities.


TABLE 5 | UFM items commonly affecting UFM strategies.

#### DISCUSSION AND CONCLUSION

fpsyg-11-00219 February 14, 2020 Time: 18:55 # 9

University facilities should support the provision of academic and rest functions through proper management of these facilities. This enables students to perform various activities such as study and rest. However, as mentioned above, the current domestic UFM does not adequately reflect the needs of the facility users (students). It is mainly focused on oldfashioned methods, such as reactive maintenance, where the problems are dealt with when they occur. This happens due to the lack of effective FM strategies for university facilities. Based on the satisfaction and importance data of the current facilities management items, it is surely necessary to prepare an effective university facilities management plan. Based on the results of this research, the following conclusions are reached.

First, through the extensive review of the literature and two rounds of Delphi survey, 35 UFM items and 8 UFM strategies have been extracted. Those items and strategies are used in the survey for the main users of university facilities.

Second, in order to find out which UFM items need more focus on, IPA is conducted. The answers on the importance and the performance of each factor are graphically displayed on the two-dimensional grid. The priority of management is determined by where each factor is located on the grid. Respondents are asked about the questions such as "How important is this feature with respect to other features? How well did the feature perform?"

Third, in exchange for identify UFM items that affect the strategy of UFM, multiple linear regression analysis is conducted to examine the effect of the importance value on the UFM items on the importance value on the UFM strategies.

Fourth, after figuring out the difference in the importance of users and facility managers who actually run university facility, the average values of importance for the strategy of UFM of two groups are compared and analyzed.

As a result of the four stages listed above, this research attempts to provide basic data on preparing the UFM plan. Therefore, it is possible to apply the method that reflects the needs of actual users in preparing future UFM plans throughout the

#### REFERENCES


Astin, A. (1993). Diversity and multiculturalism on the campus. Change. 25, 44–52.


Jeon, S. (2009). A Study on the Classification System of University Facilities and Space. Master's Thesis, Kongju University, Gongju.


research methods proposed in this research. Since UFM is unique from the general FM for other types, the research focuses on university users' opinions and utilizes expert's opinions through the Delphi survey to show the direction how the UFM should be heading to the future.

However, since this research is conducted on the limited spatial scope of university facilities, it was not able to cover the spatial scope of the entire university campus where students stay, such as dormitories and student restaurants (places that affect the overall quality of university education). Further research on the supporting facilities is required in the future. Therefore, in-depth research should be conducted on all facilities on university campuses to further investigate users' satisfaction, importance factors, and differences in perceptions of facility managers to prepare comprehensive UFM plans. Through such supplementary work, it is highly suggested to prepare a UFM plan that can cover the spatial scope of the entire university. As a result, it will be possible to establish an effective university management plan that can maximize the satisfaction of the main users, students.

#### DATA AVAILABILITY STATEMENT

The datasets generated for this study are available on request to the corresponding author.

#### AUTHOR CONTRIBUTIONS

MK conducted the related background study, collected the data, and analyzed them. JK designed and directed the entire research.

#### FUNDING

This work was supported by the National Research Foundation of Korea Grant funded by the Korean Government (NRF-2018R1D1A1B07049281).


fpsyg-11-00219 February 14, 2020 Time: 18:55 # 10


**Conflict of Interest:** The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Copyright © 2020 Kim and Kim. This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) and the copyright owner(s) are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.

# Necessity of the Needs Map in the Service Design for Smart Cities

#### Seyun An<sup>1</sup> , Sungwhan Kim<sup>2</sup> \* and Soyeon Kim<sup>3</sup>

*<sup>1</sup> Department of Industrial Design, College of Construction Environment and Design, Hanbat National University, Daejeon, South Korea, <sup>2</sup> Institute for Basic Science, College of Engineering, Hanbat National University, Daejeon, South Korea, <sup>3</sup> User Centered Smart Cities Research Cluster, Hanbat National University, Daejeon, South Korea*

Recently, the issue of how to involve actual service users in the service designs for Smart cities has grown in importance, as this is a key factor in determining the serviceability and sustainability of developed services. Reckless participation of users can make the service design process inefficient and cause a cost increase. So, in this study, we propose the needs map which can be applied to various stages of service development and help service designers to make efficient and reasonable decisions without being too time-consuming. The proposed needs map is a schematization of theoretical concepts for Smart city planning in accordance with the framework of service-technology-infrastructure-management (STIM). Through the needs map, various urban problems, statistical data, and users' needs in a smart city are classified and analyzed concept by concept, and these focused analysis results suggest a proper direction to researchers in the service development plan. To demonstrate experimentally the feasibility of the needs map, we applied it in practice to a service design for the Smart city experience zone located in Daejeon, South Korea.

#### Edited by:

*Mi Jeong Kim, Hanyang University, South Korea*

#### Reviewed by:

*Seungyeon Choo, Kyungpook National University, South Korea Fan Qiang Qiang, Northeastern University, China*

> \*Correspondence: *Sungwhan Kim sungwhan@hanbat.ac.kr*

#### Specialty section:

*This article was submitted to Environmental Psychology, a section of the journal Frontiers in Psychology*

Received: *14 October 2019* Accepted: *29 January 2020* Published: *25 February 2020*

#### Citation:

*An S, Kim S and Kim S (2020) Necessity of the Needs Map in the Service Design for Smart Cities. Front. Psychol. 11:202. doi: 10.3389/fpsyg.2020.00202* Keywords: smart city, needs map, service design, living lab, user centered design

# INTRODUCTION

#### Study Background and Objective

The smart city is a future high-tech city in which various information services can be provided through devices whenever and wherever by using Information Communication Technology (ICT), and is a concept that has gradually evolved from Information Cities and U-Cities. When the term "Smart city" is mentioned, rather than a simple compound of two words "Smart" and "City" in some sense it means a process of making existing cities smart. PAS 180 announced by BSI (British Standards Institution) defines "Smart city" as the effective integration of physical, digital, and human systems in the built environment to deliver a sustainable, prosperous, and inclusive future for its citizens (British Standards Institution BSI, 2014).

Recently, the global interest in the Smart city is growing. In India, the Modi Government that was inaugurated in 2014 declared its Smart city policy as one of its core missions, and it designated 100 mart cities nationwide as of 2018. China also announced the construction plan of 500 smart cities until 2020. As of the end of 2017, about 500 smart cities were under construction in China, and they occupied 50% of smart cities under construction in the world. Among them, the Chinese government selected officially 277 cities as a model of smart sights. In 2019, the Korean Ministry of Land, Infrastructure, & Transport (MOLIT) announced the "3rd Comprehensive Plan on Smart Cities (2019∼2023)" which integrated policies and projects from government departments. It is a mid-to long-term master plan for the construction and proliferation of smart cities and the creation of innovative ecosystems, and the enhancement of global initiatives. In this period, the Korean Ministry of Public Administration & Security (MOPAS) also held a Smart City Joint Workshop with the World Smart Sustainable Cities Organization (WeGO), the World Bank, and the International Telecommunications Union (ITU), and 19 countries including India, Malaysia, and Kyrgyzstan participated in the workshop. Furthermore, local authorities and central government departments are continuing the efforts for new projects for smart cities.

Technology adoption is indispensable in developing Smart cities and there are two approaches to applying cutting-edge technologies to making existing cities smart. One is the techdriven method (TDM) and the other is the human-driven method (HDM) (Kummitha, 2018). In TDM, under the leading of government officials or policy makers, new services for Smart cities are developed by service providers or private companies equipped with technologies, in many cases for their business. Meanwhile, HDM requests the active participation of the citizens so that technologies and services for them are developed according to their actual needs. In many development processes of Smart cities in the world, however, TDM has been the most common method used. An enormous number of innovative services for Smart cities have been proposed by researchers from all over the world, with vague hopes that these services will help to solve various problems confronting large cities (Kummitha, 2018). However, only a small number of them have been put to practical use and the rest have disappeared from (local) communities right after their commodification, due to their impracticality to communities. This has occurred because researchers did not consider what is actually needed by the cities, but simply imitated the existing products to focus on and apply specific technologies (Oktaria and Smart City Services, 2017). Even though Korea has an advanced Information & Communication Technology (ICT) environment, developing its Smart cities also leaned toward fragmentary technological developments.

In recent years, the issue of how to involve actual service users in the service designs for Smart cities has grown in importance, since this is an important factor in determining the serviceability and sustainability of developed services (Kummitha, 2019). In the service design for Smart cities, user involvement is required in each stage of finding issues, defining terminologies, developing services, and evaluating results, although reckless participation of users can make the service design process inefficient and cause a cost increase. For instance, Living Lab is an open methodology service aimed at addressing and confronting problems in a (local) community through the direct participation of various interested parties (Living Lab Guide, 2019). Experts or facilitators encourage discussion between (or with) participants in Living Lab meetings, but in practical Living Lab cases the individuality and diversity of people's experiences and preferences detracts from the Lab's ability to efficiently draw useful insights. In consideration of limited budget and time, the determination of the scope or direction of discussion before direct user participation in a Living Lab prevents rambling discussion and keeps participants focused. In fact, if the issues are clear and do not cause major conflicts of interests, it may be enough for experts to determine issues and derive solutions from indirect user participation (such as questionnaires, various statistical data, etc.). This suggests the necessity of a new methodology which should be efficient at user-involvement and have a practical application in real cities. In this paper, we want to develop a new framework through which miscellaneous needs, interests, or requirements from users and citizens are categorized and interpreted in the functional terminology for understanding real cities. This indirect method for user involvement may help, e.g., designers to have definite insights on users' needs, and these insights can be applied to the service development.

To understand the urban structure and design the urban space, many structural factors and functional concepts have been proposed by previous researches and widely accepted. Given the service design or space planning for Smart cities, user involvement without consideration of these characteristic features through which the system of cities can be understood results in vacuous debates, and also does not guarantee the sustainable serviceability of developed services. So, we are interested in devising a new method based on these theoretical features to provide a fundamental yardstick for service designers to use at each stage of the service design process for smart cities. In this paper, we propose the needs map as a schematization of theoretical concepts for Smart city planning in accordance with the framework of the service-technology-infrastructuremanagement (STIM). This needs map can help designers to analyze and confront urban issues from various essential perspectives and find some insight either in the Living Lab process or before it starts. Through such common insight about interested issues, co-creation between users, service designers, government officials, etc. would be possible. Also, the users' needs within various social fields with urban issues can be closely understood through the needs map and reflected in the service development. That is to say, the proposed needs map is a methodology not for user participation but for smart city-tailored filtering and efficient utilization of a variety of users' needs in actual cities. To demonstrate the feasibility of the proposed needs map, we will provide an application of a Needs Map to the service design for the Smart city experience zone located in Daejeon, South Korea.

#### Study Method and Scope

In this study, we will form the conceptual framework for the needs map from the viewpoint of services for the Smart city, and it will be utilized to develop services for the Smart city experience zone to verify its necessity for the Smart city service design. The study method and scope are organized as follows.

We will take three steps in this study. In the first step, key concepts for the needs map framework are considered through the literature study. In particular, we will seek to understand essential notions or entities for the Smart city service design using previous works that contain ISO SMART CITY CONCEPT MODEL GUIDELINE, and by consulting experts. In the second step, the needs map, which will consist of four layers, will have key entities appropriately located on it, with each layer being divided into four sides of service, technology, infrastructure, and management. This needs map provides useful information about what should be considered in designing specific services. Finally, the proposed needs map will be experimentally applied to service designs for the smart city experience zone to demonstrate the usability or feasibility of the needs map.

Indeed, the needs map is quite utilizable in the discovery step of the service design process based on Design Thinking, in which user's surveys and studies on the situation are performed. Service completeness can also be improved by double-checking concepts missed in the service realization process. By using the needs map, researchers can thoroughly investigate extensive urban issues or users' needs in a relatively short time through cooperation with interested parties. Eventually, the needs map will help planners to survey vastly diverse urban experiences and develop sustainable services to reflect the purpose of the user-centered smart city.

# EXPLANATION FOR THE NEEDS MAP

#### STIM Framework of the Needs Map

The eventual purpose of this study is to develop services suitable for Smart cities by finding refined needs, not from the simple classification of users and operation managers, but from the analysis of a variety of raw needs from interested parties. We hope that the proposed needs map will play such a methodological role. We organized the needs map on the basis of the STIM framework. U-City emphasized technologybased infrastructure planning (Lee and Leem, 2008), but Smart city is centered around space planning and being a peopleoriented service (An, 2016; Korea Agency for Infrastructure Technology Advancement, 2017b). This STIM notion has evolved into the ICTs-based U-city which focused on service and technology. Through this paper, the "user-centered" concept is assimilated into the STIM framework, and eventually into the needs map.

For the needs map, we take the perspective of Service-Technology-Infrastructure-Management for the Smart city design. Service is based on the various needs of users themselves. To detect the human motion and state—since Technology should support two-way communication with citizens as users of the urban space and management—various technologies such as cameras, sensing technologies, interface technologies, GPS, and telecommunication are required. Specifically, Infrastructure includes physical facilities equipped with technologies just mentioned to realize services into the real space. On the other hand, Management refers not just to service operation, but to the concreteness and realizability of the business model for sustainable services. In the STIM framework, Infrastructure and Management are related for function, Service and Management for contents, Infrastructure and Technology for interface, and Service and Technology for interaction. To enable these to be embodied in real cities, SW, HW, and System designs are required.

# Four Layers on the Needs Map

The needs map is composed of four rectangular layers based on the STIM framework. In fact, users' needs can be analyzed through various forms (for instance, questionnaires, user behaviors, visit frequency to websites, etc.). In order to reflect these needs to the service design development for Smart cities, they should be interpreted and categorized in a standard manner. Going outward from the center, the terms on the tiers become specific and detailed in the perspective of the STIM embedded model. For the needs map shown in (**Figure 1**), a round multilayer structure is used to emphasize the hierarchical relation between layers and the horizontal one between terminologies on a layer. The explanation of the layers follows.

Layer 1 is based on the STIM embedded design model. It is a promotion process for planning, designing, and operating a Smart city and is a multi-layer design model with Service-Technology-Infrastructure-Management layers which are on four rectangular sides of Layer 1, respectively.

Layer 2 is for the functional aspect of the STIM embedded design model. The service sector of Layer 2 is composed of Contents and Interaction. The technology sector represents Interface and Interaction. The infrastructure sector includes Function and Interface. The management sector consists of Function and Contents. Entities on Layer 2 are defined as follows: Contents are a variety of information and details realized ultimately in the perspective of the software; Interaction is the entity for realizing the user-centered design in terms of interaction between service environment and user; and Interface is the medium and protocol for communication between thing to thing or thing to human, and should be considered for practical realization of technology models for the Smart city. Function is to help physical Infrastructure and systematic Management to play a consistent and efficient role.

While Layer 2 is for functional approach of the STIM model, Layer 3 represents practical entities for design and realization in the real space. Each entity on Layer 3 can be used as the detailed goal which should be solved to realize the service.

Layer 4 is composed of the data indicators that will be used to draw miscellaneous insights required for the service development of the Smart city. Sixteen statistical indicators and their details appear on Layer 4. In fact, these indicators are used in national annual surveys as statistical social indicators. Each indicator will become a region and survey subject to which practical services for the Smart city should be provided. For instance, the field of residential transportation is an important indicator which the future housing or urban model should provide as a service for the Smart city, and furthermore the supply and demand of housing, the residential status, the convenience of transportation, etc., should be checked.

# Functional Entities on the Layers

In this study, the needs map is defined as a visualized output on which essential entities for the Smart city service are linked into the STIM framework according to their functional meanings. In fact, before obtaining these functional entities on the layers, many important concepts for the Smart city were discussed and extracted through several expert advisory councils in line with the advent of the Smart city over the U-city (see **Table 1**). To derive fundamental concepts for Smart cities and test beds, we held workshops and conferences for experts in which discussions between experts were made through brainstorming methods

based on design thinking<sup>1</sup> . In addition, an advisory conference of relevant experts was held to better understand the experiencetype test bed for the Smart city and for establishing its design process. Experts for the conference included professors and chief researchers from the fields of urban engineering, construction and design, etc. In the advisory conference, contents for the service, technology, infrastructure, and operation fields were discussed to determine core elements and directions of the Smart city.

In this paper, on the basis of these fundamental concepts for the Smart city, we discreetly selected functional entities through careful literature studies which are arranged on the layers in accordance with their practical functions. These entities are chosen to accomplish the purpose of the user-centered Smart city, and so their physical and systematic perspectives are continuously deliberated to incarnate them in sustainable services on (smart city) experience zones. Quantitative social indicators or statistical indicators also are considered to address various urban issues. These entities are distributed on layers of the needs map which form in stratum. Entities on a higher layer are more specific than those on a lower one.

To reflect the various needs of users, the service sector in the needs map can show reasons or purposes for why users require and use the service. So the service sector should contain contents, their role and goal, and communication and cooperation with various related parties. In particular, contents for the Smart city should include "living" (for supporting the living convenience and safe life of the citizens), "work" (for activating the local economy of production and enhancing the work efficiency of the related parties), "entertainment" (for activating citizen participation in the local community by specializing local cultural asset), and "mobility" (for securing the convenience, approach, and comfort in pedestrian and transportation methods).

As described earlier, technology supports two-way communication with citizens as users in the urban space and management. So, contents for the technology sector should include sensing and interface technologies for motion detection and network processing and security for information exchange.

Infrastructure includes physical facilities to realize services into the real space, and so the infrastructure sector should consider contents which are related to terms such as "built," "movable," and "variable."

<sup>1</sup>We held five conferences (workshops and expert advisory meetings) for three months from April to September 2017 to derive the need map concept.


Management implies the concreteness and realizability of the business model for sustainable services, and so the management sector should include contents such as governance establishment, private cooperation, public participation, facility operation-management, and maintenance of contents and information.

**Table 2** shows the detailed explanation of structural elements of the needs map.

#### Mathematical Explanation for the Needs Map

The needs map in **Figure 1** provides a way to understand users or citizens' needs for service developments in cities. Speaking in detail, all behaviors which users perform in a city are analyzed and grouped into needs through the needs map, with individual services developed from extracted user needs. For example, in the next section, image analysis devices were used for the behavior analysis of the users visiting the Doan District which is a new town in Yuseong-gu, Daejeon, South Korea, and user needs were discovered from the analysis result. In this example, the amount of data was relatively small and was able to be manually dealt with. This local study shows effectively the usefulness of the needs map in service developments for Smart cities. However, to apply the needs map to Smart cites as a whole requires planners to maneuver the immense amount of data which is constantly updated. Needs from variously interested persons who are men, women, residents, visitors, officials, etc. are also complicated and extensive. This shows another reason why the needs map is necessary. The needs map proposes a framework model through which user behaviors are classified, managed, and utilized for needs extraction and further service developments for smart cities.

The classification of such large-scale data requires several filtering processes of data sets. In algorithms for Big Data Analysis, a filter or kernel is given in the form of a matrix and, in fact, the filtering process is a convolution of the filter with a data set. So to design filters appropriate to the purpose is very important in data classification. A needs map should permeate these filter matrixes. This filter design for the need map will be one of our future studies (see **Figure 2**).

#### EXPERIMENTAL APPLICATION OF THE NEEDS MAP TO THE SMART CITY EXPERIENCE ZONE

In this section, we show the feasibility of the needs map in **Figure 1** to develop services for a Smart city experience zone. To do so, we take the following three steps. First, by using statistical social indicators we obtain overall spatial information related to entities on Layer 4. This spatial information helps to understand the current status of the experience zone. In particular, the environment and security issues in the experience zone drawing high interests of users are intensively discussed. Second, based on the pabulum obtained in the first step, a place in the experience zone to which services will be practically provided is chosen and perambulated. The reconnaissance of the chosen region based on the needs map helps to evaluate the adaptability of the practical space design and the service application. We obtain baseline data from the investigation of physical environment and people's behaviors in the reconnaissance for service developments. Third, the purpose and direction for services which will be provided to the experience zone are set up. Those services are practically realized in the chosen space.

#### The 1st Step of the Needs Map Application: to Understand the Experience Zone Through Layer 4

In this step, as services developed for smart cities are intrinsically used by a large number of citizens in public places, the environment issue in the experience zone is considered.

#### TABLE 2 | Explanation of functional entities on the needs map.


FIGURE 2 | Data classification process using filters.

According to statistics of social indicators released by Daejeon Metropolitan City (2018) 25.1% of its residents consider the garbage problem as something that should be quickly addressed, with 22.9% reporting on the air pollution and stench problem. Furthermore, "the destruction of green area" (8.6%), "the air pollution and stench" (5.8%), and

"the pollution of river water" (4.7%) show high rates of increase compared to 2016 and such statistics indicate that they are urgent problems. As for the security issue, one in two citizens in Daejeon think that "the proliferation of security cameras" (48.9%) is indispensable, and after the CCTV expansion "the reinforcement of community police patrol" (18.7%) and "the maintenance of streetlights" (12.2%) follow.

### The 2nd Step of the Needs Map Application: Field Investigation at the Experience Zone Through Layer 4

The Doan district is an area in Doan New Town and Spa Tourist Site (Wonsinheung-dong, Oncheon 1-dong and Oncheon 2-dong) in Yuseong-gu, Daejeon. Yuseong Spa Station of Daejeon Metro Line 1 is located at the Doan district and two avenues (Doan-daero and Gyeryong-daero) cross on this district so that accessibility through public transportations is quite convenient. Also, the Doan district is close to accommodations, shopping towns, and large-scale apartment complexes. Since the user experience elements can be maximized, the Doan district with many infrastructures was finally selected as the experience zone after an advisory board discussion.

In this study, image analysis devices (WS-6210 Infrared Thermal Camera) were used for the behavior analysis of the users visiting the Doan district. The field investigation was conducted on November 19, 2017, from 16:30 p.m. to 18:30 p.m. for 2 h. The user behavior survey was conducted only to check the frequency of behaviors and did not infringe on personal portrait rights (see **Figure 3**).

In the behavior analysis, user's behaviors were separated into 11 pedestrian activity types and the frequency of each behavior type was counted. The behavior types are classified into essential activity (waiting, passage, purchase), selective activity (entertainment/amusement, food/beverage, exercise), and social activity (gathering, conversation, smoking, cultural performance/event). Peculiar features which can be identified visually such as gender, age group, vehicle possession, riding on a motorcycle or bicycle, carrying out a criminal act, or using a mobile phone, or other smart devices were also recorded (Oh and Lee, 2013).

The image data extracted through the image analysis device was coded as "1" for each user's behavior per case using the Microsoft Excel program. In multiple responses, the sum of the cases may be less than the sum of each survey. For example, if four men walked together, they coded four for their gender and one for walking. This is to determine the universal behavior of the user group. A total of 124 cases were extracted through the image analysis. Males visit more frequently, with 92 visits compared to females with 76 visits.

The result of the user behavior analysis shows that the following services are helpful and useful to users who visit the Doan district (An and Kim, 2018):


#### The 3rd Step of the Needs Map Application: Establishment of Services for the Experience Zone

In this step, the entities on the needs map were utilized to develop the smart view service. The developed smart view service can also be installed as an independent street stand based on the module type according to the necessary functions, but our aim is to develop the design plan which is utilizable together with the existing infrastructures. By using the needs map, each smart view function can be described as follows (see **Figure 4**).

#### 1) Service

Based on items in the service sector of the needs map, the Smart View service was carefully planned. To reflect various needs for user groups we considered the content component, the communication and cooperation within diverse interested parties, and the concept of their role and goal. The content component includes living (supporting the living convenience and safe lives of the citizens), work (activating the local economy of production and enhancing the work efficiency of the related parties), entertainment (activating citizen participation in their local community by specializing local cultural assets), and

FIGURE 3 | User behavior analysis using image analysis devices.



mobility (securing the convenience, approach, and comfort of pedestrian and transportation methods), etc.

Smart view is a smart streetlight which is modularized to have an assembled structure. It plays a basic role of not only a streetlight but also a security camera. Additionally, it has the environmental monitoring function to measure temperature, humidity, micro dust, and ultraviolet light, and provides public and local information through a kiosk (see **Tables 3**, **4**).

#### 2) Technology

Based on items in the technology sector of the needs map, the technological components of Smart view were carefully planned. To support the two-way communication and management with the citizens in the urban space, sensing and interface technologies were adopted to detect human movement and state, network, and processing for various information exchanges, and security. In

#### TABLE 4 | Main functions of Smart View according to actual service types.


TABLE 5 | Functions of smart view according to Technology.


TABLE 6 | Functions of smart view according to infrastructure.


particular, the connectability with other hardware, software, and services already used in the city was taken into consideration (see **Table 5**).

#### 3) Infrastructure

Since Infrastructure includes the physical form to realize services into the real space, Smart View was designed in the TABLE 7 | Functions of smart view according to management.


consideration of the fixed, movable, or variable functional elements. Through modularization, it can be designed into an independent kiosk according to the purpose of necessity, and it can also be designed in the application form to be connected to existing public facilities, such as a streetlight or a bus stop (see **Table 6**).

#### 4) Operation Management

Management means not just the service operation, but also the concreteness and possibility of the business model for sustainable services. Thus, the governance establishment for private cooperation and citizen participation, and the management, operation and maintenance of various contents and information were considered. The integrated control to operate all information is important in the view of management, and so the role of the integrated control center is quite essential to information operation for providing the Smart View service and actual management of Smart View (see **Tables 7**, **8**).


#### TABLE 8 | Key features of smart view.

#### CONCLUSION

In this paper, we recognized the problems of Smart city planning focused on fragmentary technology developments and proposed the needs map as a schematization of theoretical concepts for the Smart city planning in accordance with the framework of the service-technology-infrastructuremanagement (STIM). This study has important implications for architects, designers, planners, and policy makers who seek to develop creative Smart city services. We confirmed that researchers can thoroughly investigate extensive urban issues or users' needs in a relatively short time through cooperation with interested parties by using the needs map. Researchers can develop or analyse Smart city services through each of the layers and entities on the need map. In other words, service completeness can be improved by double-checking concepts missed in the service realization process.

Furthermore, the users' needs within various social fields with urban issues can be closely understood through the needs map and reflected to the service development. Applying the needs map to not local but whole smart cites requires the management of an immense amount of data which is constantly updated. Needs from variously interested people who are men, women, residents, visitors, officials, etc. are also complicated and extensive. This demonstrates another reason why the needs map is necessary. The needs map proposes a new methodology through which user behaviors are classified, managed, and utilized for needs extraction and further service development creation for Smart cities. The classification of such large-scale data requires several filtering processes of data sets. In algorithms for Big Data Analysis, a filter or kernel is given in the form of a matrix and, in fact, the filtering process is a convolution of the filter with a data set. So, to design filters appropriate to the purpose is very important in data classification. The needs map should permeate these filter matrixes. This filter design for the need map will be our future study. Through our future study, we can analyse the current status of smart services in urban spaces and suggest directions for improving the Smart city services.

FIGURE 5 | Modeling image of smart view.

To demonstrate the feasibility of the proposed needs map, we provided an application of the needs map to the service design for the Smart city experience zone located in Daejeon, South Korea. We propose the necessity of Smart View on this experience zone (see **Figure 5**). Smart View is equipped with service functions to address problems raised by the analysis result. However, this application does not verify the sustainability of Smart View in the manuscript because it was not manufactured as a real product. The manufacture of Smart View requires time and money. In fact, it is our longterm future works to demonstrate the usefulness of the needs map and the sustainability of services developed through the needs map.

#### DATA AVAILABILITY STATEMENT

All datasets generated for this study are included in the article/supplementary material.

#### AUTHOR CONTRIBUTIONS

SA conceived of the presented idea and developed the theory. SA and SoK performed the computations and verified the analytical methods. SuK helped carry out the Mathematical concepts for analyzing associations between 4 layers of Needs Map. All authors provided

#### REFERENCES


critical feedback and helped shape the research, analysis, and manuscript.

#### FUNDING

This research was supported by the research fund of Hanbat National University in 2018.


**Conflict of Interest:** The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Copyright © 2020 An, Kim and Kim. This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) and the copyright owner(s) are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.

# Conceptual Metaphors for Designing Smart Environments: Device, Robot, and Friend

Jingoog Kim\* and Mary Lou Maher

Department of Software and Information Systems, The University of North Carolina at Charlotte, Charlotte, NC, United States

A metaphor is a design tool that can support designers in forming and exploring new design concepts during the process of designing. Digital technologies embedded in built environments provide an opportunity for environments to be more intelligent and interactive. However, most architectural concepts associated with smart environments such as smart homes and intelligent buildings tend to focus on how advances in technology can improve the quality of the residential environment using automation and not on how people interact with the environment. We posit that conceptual metaphors of device, robot, and friend can open up new design spaces for the interaction design of smart environments. We present three metaphorical concepts that can frame new ways of designing a smart environment that focuses on interaction rather than building automation, each of which have distinct HCI techniques.

#### Edited by:

Mi Jeong Kim, Hanyang University, South Korea

#### Reviewed by:

Henri Achten, Czech Technical University in Prague, Czechia Dor Abrahamson, University of California, Berkeley, United States

\*Correspondence:

Jingoog Kim jkim168@uncc.edu

#### Specialty section:

This article was submitted to Environmental Psychology, a section of the journal Frontiers in Psychology

Received: 16 November 2019 Accepted: 28 January 2020 Published: 17 March 2020

#### Citation:

Kim J and Maher ML (2020) Conceptual Metaphors for Designing Smart Environments: Device, Robot, and Friend. Front. Psychol. 11:198. doi: 10.3389/fpsyg.2020.00198 Keywords: metaphor, embodied interaction, human-computer interaction, interaction design, smart environments

# INTRODUCTION

New technologies allow our built environment to become intelligent and interactive. Embedding computation in physical environments changes our environment from a static to an interactive space. There are many concepts related to emerging new technologies in architecture such as building automation, smart homes, adaptive buildings, intelligent buildings, and interactive architecture. However, there are few studies about a theoretical and methodological framework to understand and expand the design space of interactive designs in a built environment. Therefore, designing smart environments requires a new foundation to guide in the conceptualization of novel designs using concepts that emerge from human–computer interaction (HCI). Using metaphors is a design technique that can frame new design spaces for interactive designs and support designers in creating novel interaction experiences. Metaphorical references can also assist users in perceiving affordances of novel designs. We look to metaphorical design to provide a common mental model for both designers and users in the transition from traditional to smart environments.

Interaction between an occupant and an interactive system in a built environment relies on embodied interaction, an approach to HCI designs emphasizing everyday experience as a foundational concept for HCI. As computers are increasingly embedded in physical environments, embodied interaction has expanded to ubiquitous computing with the development of new technologies. Understanding embodied interactions that can be adapted to smart environments thus is essential in order to identify new design spaces for smart environments. We claim that

**91**

characterizing embodied interaction with the conceptual metaphors can provide a basis for designing smart environments.

In this paper, we present three conceptual metaphors as a basis for characterizing smart environments: device, robot, and friend. We provide a review of existing embodied interaction designs that can be adapted to smart environments from the perspective of these three metaphorical concepts to expand on the ways in which each metaphor is distinct and enables the creation of new designs that provide a consistent mental model for designers and users. We discuss how each conceptual metaphor frames new designs for smart environments through educational experiences. Finally, we show how framing a specific design with each of the conceptual metaphors leads to different interactive experiences.

### METAPHORS IN HCI

Metaphors are widely used in HCI as a vehicle for representing and developing designs. A metaphor is a mapping process from a familiar object to an unfamiliar object, and it provides the framework to familiarize an unknown concept through a mapping process. The role of a metaphor in HCI is to facilitate developing, maintaining, and learning the conceptual foundation of the interactive design as well as orienting the user with it (Saffer, 2005). Using metaphors involves the exploration and expression of an idea that is integral to design generation and innovation (Brady, 2003). In this perspective, metaphors can be used as a tool in the design process to understand new topic areas or as a means to create new ideas about familiar subjects. They enhance our perception by transforming our sense of reality (Ricoeur, 1991), and new metaphors can create a comprehensive conceptual system (Lakoff, 1993). Metaphors can also assist in engaging the designers' existing mental models. A mental model is an organized collection of data that acts as a representation of a thought process (Marcus, 1992, 1995). Mental models refer to analogs of real-world processes, including some other aspects of real-world processes (Gentner and Gentner, 2014). Analogical reasoning, an inference method in design cognition (Gero and Maher, 1991), is a method for developing designs that can lead to unexpected discoveries. In conceptualizing a new interactive system, a metaphor can be a useful tool for establishing a common mental model for designers. We claim that the positive impact of metaphors in HCI can be beneficial in conceptualizing smart environment. In this paper, we describe how metaphorical design enables the conceptualizing of a smart environment design in which different metaphors lead to new conceptual spaces.

The most well-known metaphor in HCI is the desktop metaphor, which represents the user interface in a way that is similar to interactions concerning certain objects, tasks, and behaviors encountered in physical office environments. Despite the desktop metaphor still being predominant in the personal computing environment, it shows problems and limitations (Moran and Zhai, 2007; Antle et al., 2009; Houben, 2011; Jung et al., 2017) in being adapted into recent interaction designs (e.g. tangible interaction, speech interaction, and mixed reality) since it focuses on the personal computing and visual interface design. While much of the research done on metaphors in user interface design has been focused upon the use of metaphors in the design of visual communication elements of the graphical user interface (GUI) and in understanding users' mental models of such interfaces (Voida et al., 2008; Antle et al., 2009; Houben, 2011), some researchers have pointed out the limitations of the desktop metaphor and proposed alternative metaphors (Abowd and Mynatt, 2000; Moran and Zhai, 2007; Antle et al., 2009; Jung et al., 2017). They showed several dimensions (e.g. context, modality, materiality, and affordance) of alternative metaphors as a systematic strategy to emphasize a new interactive form which can be conceptualized with metaphorical mapping. A smart environment provides potential design spaces that are yet to be fully explored and understood. We posit that new forms of smart environment can be characterized by comprehensive metaphors that can uncover potential design spaces for a smart environment by providing a common mental model.

# SMART ENVIRONMENTS AND EMBODIED INTERACTION

Smart environments are associated with recent architectural concepts based on new technologies. From the HCI perspective, a smart environment is based on embodied interaction involving physical movements of occupants and spatial aspects of the environment. In this section, we describe architectural concepts associated with smart environments and embodied interaction that can be adapted to smart environments.

#### Smart Environments

Smart environments reflect recent architectural phenomena that embed computation in built environments, providing dynamic spaces to support a range of humanistic functions. There are many architectural concepts associated with smart environments: intelligent buildings, building automation, sentient buildings, smart homes, responsive architecture, adaptive buildings, kinetic architecture, and interactive buildings. These architectural concepts reveal different perspectives of smart environments such as purposes, functions, building components, and interactivity.

The concepts of intelligent buildings, building automation, and sentient buildings are associated with automation in a built environment. An intelligent building is a building using integrated sensor systems that maintain optimum performance by automatically responding and adapting to the operational environment and user requirements (Callaghan, 2013). Building automation is a system for monitoring and controlling mechanical, security, fire and flood safety, lighting, and HVAC systems in a building (Achten, 2014). A sentient building is a sensor-driven monitoring and controlling system that has an internal representation of the building from which it derives control strategies (Mahdavi, 2004; Achten, 2014). These concepts mostly focus on automation for environmental comfort, safety, security, privacy, energy use, and efficiency. The smart home is a part of building automation that applies to residential buildings (Achten, 2014). Although the smart home is a similar concept to building automation, it provides

more opportunities for occupants to actively control their environment through handheld devices connected with the home environment. In other words, smart homes involve not only automation but also direct interaction to support ease of control in the living environment and occupants' routine activities. Responsive architecture, adaptive buildings, and kinetic architecture are associated with environmental comfort and sustainability. These concepts typically refer to smart facade systems that automatically react to internal and external environmental conditions with physical movements of building elements (Achten, 2014; Fortmeyer and Linn, 2014).

The architectural concepts associated with smart environments provide opportunities to improve the quality of the residential environment including environmental comfort, ease of control for living, security, better building performance, and sustainability. However, most concepts focus on automation rather than how people interact with the built environment. We look to conceptual metaphors and metaphorical design that can provide a new approach to rethinking the smart environment for novel interactive experience designs.

#### Embodied Interaction

Interaction between a user and a system in a built environment is strongly associated with embodied interaction since it depends on the user's physical body movements and spatial aspects of the environment. Embodied interaction is interaction between computer systems and people that involves our physical bodies for the interaction in a natural way, for example, gestures. Dourish (2001) described that embodied interaction is about "how we understand the world, ourselves, and interaction comes from our location in a physical and social world of embodied factors." Embodiment leverages users' body movements to facilitate interaction with a computational system embedded within a space or physical object which involves tangible computing, social computing, mixed reality, responsive environments, pervasive computing, and physical computing (Antle et al., 2009). Embodied interaction emphasizes a way to combine perspectives of tangible interaction and social computing (Dourish, 2001). Recent embodied interaction designs expand to new technologies such as wearable computing, kinesthetic sensations, full-body interaction, multimodal interaction, and conversational embodied agents (Hobye and Löwgren, 2011).

Embodied interaction is also associated with metaphorical design. Lakoff and Johnson (2008) argue that abstract concepts rely on metaphorical extensions of embodied schemata, which are mental representations of recurring dynamic patterns of bodily interactions that structure the way we understand the world. Embodied schemata based on recurring patterns of bodily experience facilitate reasoning about abstract concepts in metaphorical thinking. An embodied metaphor is a mapping process between a source domain of embodied experiences and a target domain of an abstract concept. Therefore, an embodied metaphor extends embodied schemata to structure and understand abstract concepts (Antle et al., 2009; Bakker et al., 2012).

Rompay et al. (Van Rompay and Hekkert, 2001; Van Rompay et al., 2005) investigated the effect of embodied schemas in product design with several experiments based on the theory of metaphor developed by Lakoff and Johnson (1999, 2008). Rompay et al. examined the relationship between embodied schemas (e.g. inside–outside), the product expressions (e.g. secure–insecure), and design properties (e.g. size, material) through experiments that ask subjects to rate product expressions associated with embodied schemas and/or design properties of various designs (e.g. different shapes of chairs). The results showed that product expressions are associated with the same underlying embodied schemas in the spatial and material features of products. That means bodily experiences can structure our understanding of products and product characteristics can be derived from schemata that affect a product's expression in a consistent manner. In this perspective, embodied schemas, a basis for embodied metaphors, can provide guiding principles for designing physical products, and designers can use embodied metaphors as a design tool.

While Rompay et al. focused on how the properties of product design can be associated with embodied schemas, there are several studies that investigated the effect of embodied metaphors in interaction designs. Antle et al. (2008, 2009) presented an interactive musical sound-making environment, Sound Maker, for learning about abstract sound concepts. They used the ontological metaphor "music is body movement" to map bodily movement to the sound parameters of volume, tempo, and pitch. From the comparative study of the same interactive system implemented with and without an embodied metaphor, they provided evidence that an embodied metaphor supports the creation of systematic relationships between specific user actions and specific system actions.

Bakker et al. (2012) have presented a tangible system, MoSo Tangibles, for learning about abstract sound concepts (pitch, volume, and tempo). This is a similar approach to that of Sound Maker in terms of using embodied metaphors for mapping bodily movements to sound parameters. MoSo Tangibles uses multiple tangible artifacts and embodied schema for hand movements, while Sound Maker uses a single artifact for full-body interaction. This case was revealed to elicit a set of embodied metaphors that children may use in their reasoning about abstract concepts related to sound parameters.

Abrahamson and Trninic (2011) used embodied interaction based on spatial metaphors of bodily experience for mathematical learning. The interactive display called Mathematical Imagery Trainer measures the height of the user's hands to give visual feedback for rational-number problem solving. The results of the study showed that spatial embodied metaphors facilitate mathematical problem solving.

Hemmert and Joost (2016) used embodied metaphors for interactions with mnemonic objects in live presentations. They focused on the use of container metaphors for bimanual and spatial interactions in the context of live presentations (i.e. topics are picked up, and one goes through a series of points and comes to a conclusion). The user can activate a topic by picking up the corresponding object. Each topic consists of multiple points, and the user can activate a point by walking up to its position on stage. This case reveals the use of embodied metaphors for a multimodal interaction.

Löffler et al. (2016) applied conceptual metaphors to the realm of colors for tangible interaction design (e.g. Important is a dark color). They investigated color-to-abstract mappings to tangible interaction design: whether colors can substitute haptic object characteristics (i.e. the size, weight, or temperature of tangibles) when conveying abstract meaning. The results suggested that color can replace haptic attributes in metaphoric mappings.

The studies on embodied interaction and embodied metaphor show the potential and effect of embodiment on designing physical products. The insights from the studies indicate that embodied metaphors can provide guiding principles for physical products and can extend to smart environments. However, the studies on embodied schemas focus on physical design properties rather than interactivity, and the studies on embodied metaphors for interactivity tend to use metaphors for mapping body movements while focusing on a specific tangible and gesture interactions. We build on this foundation to introduce metaphorical concepts that can guide the designer to explore the design space of embodied interaction technologies to enable a common mental model for the designer and the user in a smart environment.

# CONCEPTUAL METAPHORS AND HCI FRAMEWORK FOR CHARACTERIZING SMART ENVIRONMENTS

We present three metaphorical concepts as a basis for characterizing smart environments through analogical reasoning: device, robot, and friend (Kim et al., 2018). Interaction between a user and a computer as a phenomenon can be a basis for a new metaphor in the sense that metaphors in HCI provide familiar mental models of how the computer can be used. In order to identify new metaphors for smart environments, we focused on the role of the user and the role of the system when performing a task. We categorized three different perspectives of embodied interaction for the potential conceptual metaphors: performing tasks by direct control, performing tasks by automation, and performing tasks by assistance. Direct control for performing a task represents the fact that users actively control an activity, environment, and information with an embodied interaction (e.g. touch, gesture, and tangible). In this case, the user initiates and leads the interactions between the user and the system. We established the device metaphor for direct control to emphasize the characteristics by which a user performs a task using an interactive design like a device. The performing task by automation represents that an interactive system actively performs tasks detecting external conditions and analyzing data without human control. In this case, the interactive system initiates the interaction between the user and the system. We use the robot metaphor to highlight the characteristics of automation and autonomous features. The performing task by assistance represents that an interactive system actively intervenes in certain activities done by the users (e.g. personal assistance using AI technologies). In this case, both the user and the system can initiate and lead the interactions between them. We use the friend metaphor for the performing task by assistance for emphasizing the characteristic of a human-like manner for natural interaction.

A conceptual metaphor for HCI is associated with interaction styles and modalities (Satzinger et al., 2011). For example, the desktop metaphor reflects direct manipulation where users interact with objects on display screen menus and WIMP interfaces. The document metaphor involves browsing and entering data in electronic documents such as browsing, WWW, hypertext, hypermedia, forms, and spreadsheets. The dialog metaphor carries on a conversation with speech or natural language. In order to characterize a smart environment with the presented metaphors, we adapted the concepts of interaction type, interface type, and affordance as a framework for representing HCI techniques.


There are numerous embodied interaction designs that use different modalities and interaction techniques. As recent embodied interaction expands to new technologies such as wearable computing, kinesthetic sensations, full-body interaction, and conversational embodied agents (Hobye and Löwgren, 2011), many of the recent embodied interaction designs tend to use multimodal interaction.

# CHARACTERIZING SMART ENVIRONMENTS WITH CONCEPTUAL METAPHORS

As a basis for characterizing smart environments, we review a range of existing embodied interaction designs through the perspectives of HCI techniques (i.e. interaction type, interface type, and affordance) and our three conceptual metaphors. Our selection of designs is based on a sample of the literature using a search for research publications in Google Scholar. Our search terms include multiple ways of describing embodied interaction (e.g. "embodied interaction," "tangible interaction," "embodied metaphor," "multimodal interaction," "mid-air gesture," "haptic," "public display," "full-body interaction," "embodied AI," etc.). We also sampled interaction designs by searching for websites and commercial products using terms such as "embodied interaction," "tangible interaction," "personal assistant," and "multimodal interaction" to include in our review of implemented embodied interaction designs. In this review, we included examples of existing embodied interaction designs categorized into embodied types by decreasing the scale of physical involvement in the users' interactions: full body (e.g. mid-air gesture, interactive wall), tangible using physical objects [e.g. tangible user interface (TUI), organic user interface (OUI)], kinesthetic (e.g. haptic, touch), and conversational (e.g. conversational embodied agent, AI agent). There were no specific exclusion criteria for the results of articles

and commercial products, but we avoided including multiple similar designs in a single category. In this review, we examine 24 existing designs (**Figure 1**) for alignment with three metaphorical concepts, embodied types, and interaction types.

**Figure 2** shows the percentages of designs categorized by our three metaphorical concepts: 50% of device (12 designs), 21% of robot (5 designs), and 29% of friend (7 designs). Most designs in the device metaphor are wearable and have multi-touch displays, a TUI, an OUI, and mid-air gesture-based public display. Most designs in the robot metaphor use everyday objects (e.g. smart lock, shape-changing bench, shape-changing wall, and robotic suitcase). Most designs in the friend metaphor are different types of personal AI assistants using speech, screen, hologram, and physical robot types of interfaces.

**Figure 3** shows the percentages of designs categorized by embodied types: 29% of full body (two designs of device, four designs of robot, one design of friend), 29% of tangible (seven designs of device), 17% of kinesthetic (three designs of device, one design of robot), and 25% of conversational (six designs of friend). The results indicate that kinesthetic and tangible types are mostly associated with the device metaphor, the full-body type is related to the robot metaphor, and the conversational type is associated with the friend metaphor.

**Figure 4** shows the percentages of existing designs categorized by interaction types: 23% of instructing (four designs of device, one design of robot, one design of friend), 23% of conversing (six designs of friend), 27% of manipulating (seven designs of device), 12% of exploring (one design of device, one design of robot, one design of friend), and 15% of sensing (four designs of robot). Two out of 24 designs use two interaction types, and thus, the total number of interaction types in the chart is 26. Most embodied interaction such as TUI, touch, and mid-air gesture rely on instructing and manipulating types of interaction. However, the analysis shows that exploring, sensing, and conversing types of interaction are not a small portion of embodied interaction. The embodied interaction designs that use those types of interaction reflect recent emerging technologies that use multimodal interaction. The analysis also indicates that instructing and manipulating types are mostly related to the device metaphor, the sensing type of interaction is related to the robot metaphor, and the conversing type of interaction is strongly associated with the friend metaphor. Exploring types of interaction can be applied for all three metaphorical concepts.

**Figure 5** shows the emergence of existing designs categorized by three metaphorical concepts. While the designs based on the device metaphor gradually increase from 2005, the designs based on the robot and friend metaphors mostly appear from 2012 and 2013. The designs based on the robot and friend metaphors tend to combine emerging technologies with embodied interaction using sensing and conversing types of interaction with a multimodal interface type. From this perspective, we assume that the robot and friend metaphors can be beneficial in providing a common mental model and uncovering new designs spaces for future embodied interaction designs.

**Table 1** shows the mapping of interaction types, interface types, and affordances onto the metaphorical concepts, as realized in the 24 embodied interaction designs. In terms of interaction style and modality, the device metaphor is strongly associated with direct manipulation, the robot metaphor involves automated features, and the friend metaphor reflects conversational interaction and a human-like manner. The distinct characteristics of each of these metaphors lead to different types of interactions, interfaces, and affordances.

This analysis of existing designs has identified how each metaphor is associated with specific interaction types, interface types, and affordances that can be a guide to future embodied interaction designs. In this section, we describe examples of existing embodied interaction designs associated with each metaphor and how each metaphor structures a smart environment with embodied interaction by interaction types, interface types, and affordances.

#### Smart Environment as a Device

A smart environment with embodied interaction is a device on a much larger scale, extending the concept of a smartphone or smart appliance to an interactive design. This metaphor represents performing tasks through users' direct control. It encompasses providing better service, performance, and ease of control by using an interactive design as a device. This metaphor emphasizes multiple purposes and operations in a single device. Thus, it provides insight into how users can control information, activity, and environment with an interaction. The device metaphor represents direct manipulation which involves instructing, manipulating, and exploring types of interaction. It encompasses the interface types TUI, mid-air gestures, and OUI. The interfaces for this metaphor use GUI elements and physical shapes of the interface for the affordance and the signifier.

**Figure 6** shows examples of the device metaphor for smart environments. Cube (family of the arts), shown in **Figure 6A**, is a tangible home controller that users can rotate, shake, spin, tap, and swipe to control different parts of the home such as lighting, heating, and cooling. This is an example of the device metaphor being applied to control an environment using tangible interaction, and Cube looks like an actual device. Cube uses the manipulating type of interaction through physical actions such as rotating, shaking, and tapping the cube. The interface type of Cube is a combination of GUI, TUI, and appliance. Cube uses a GUI similar to existing mobile apps such as a clock screen with a battery icon, a music icon, and a dial graphic. This familiar GUI helps users to understand the usage of Cube. The type of technique is a tangible interaction to manipulate Cube using physical actions. In this case, the tangible interaction is used to control multi-functions in a place, and it can be compared to controlling multi-functions by a smartphone and voice commands through a smart speaker (e.g. Amazon Echo, 2014)<sup>1</sup> . This type of platform is an appliance used as a smart home device. The affordances and signifiers are icons/graphics (digital) and the surfaces/shape of Cube (physical). For example, a surface of Cube shows a dial type of controller graphic, and the dial makes the user rotate Cube (e.g. turning up the volume). Another

<sup>1</sup>Amazon Echo (2014). — Alexa Speaker. Available online at: https: //www.amazon.com/all-new-amazon-echo-speaker-with-wifi-alexa-darkcharcoal/dp/B06XCM9LJ4 (accessed February 28, 2020).

**Figure 6B** is an interactive public display that uses mid-air gesture interaction (Walter et al., 2014). A public display and the use of mid-air gestures are analogous to a mobile device in the sense that they provide experiences similar to mobile interaction such as with smartphones and tablet PCs. Touch interaction on mobile devices can be transferred to mid-air gesture interaction in a similar way (e.g. tap of touch to dwelling of gesture), and the use of GUI and presentation of information in mobile devices can be transferred to a public display in a similar way. The interactive public display using mid-air gestures uses instructing types of interaction (e.g. selection gesture and confirmation gesture). The interface types are mid-air gestures and a shareable display. The affordances and signifiers are mirror images of the user and the shape of items.

The last example is an interactive floor display (Camba et al., 2018), shown in **Figure 6C**. This example presents a larger scale of the device metaphor. The interactive concrete tiles with LEDs

provide a natural visualization such as pedestrian navigation, advertising, and entertainment. In this example, the building as a device can use an entire floor of the building or multiple spaces as a user interface. The interaction between users and device is also expanded as walking through the information rather than touching/manipulating a device in a space, but is still analogical (e.g. touching/pressing button vs. walking tiles). The interaction floor display uses exploring types of interaction by allowing the users to walk on concrete tiles; users explore physical spaces on the floor to get information. The interface types are OUI, touch (foot), and physical objects (LED concrete tiles). The interactive floor uses various graphics such as figures, text, and logos. The graphics are presented when the walking action is provided as input. In this case, the affordances are guiding and walking, and the affordance signifiers are graphics and the positions of the highlighted tiles. For example, presenting arrow graphics for navigation purposes induces users to follow the arrows. Some highlighted tiles that are a bit further away from a user make the user look at the tiles or walk through the tiles.

assistant (e.g. beckoning, dancing)


FIGURE 6 | Embodied interaction designs in device metaphor. (A) Tangible controller (family of the arts), (B) interactive public display (Walter et al., 2014), (C) interactive floor display (Camba et al., 2018).

In the device metaphor, the examples use instructing, manipulating, and exploring interaction types. A significant characteristic of the device metaphor is manipulatable/direct interaction, and this characteristic is associated with instructing, manipulating, and exploring types of interaction. For the interface type, the examples use some generalized GUI or graphics to help users' understanding even if they use different techniques and platforms. The examples show a combination of digital affordances and physical affordances. The digital affordances tend to be generalized affordances such as icons, buttons, and graphics to take advantage of familiar affordances. On the other hand, the physical affordances are unique and are associated with the size and shape of the physical platform. The interactive floor is a good example for a combination of physical affordances and digital affordances.

#### Smart Environment as a Robot

A smart environment with embodied interaction is a robot in the sense that it is an intelligent machine capable of performing tasks without explicit human control. An interactive system based on this metaphor reacts to external changes autonomously to perform tasks. This metaphor emphasizes automation, autonomous decision making, artificial intelligence, and the physical actions of a robot. It reflects learning and adapting through interaction with users and sensors. The robot metaphor involves exploring and sensing interaction types for automated features that detect users and analyze data to take a system action. It includes the interface types of robot, appliance, ambient device, and OUI with automation. The affordances for this metaphor are physical movements of the interface (e.g. automatic unlocked doorknob for opening door) and automated information changes (e.g. automatic activated screen for face ID).

**Figure 7** shows examples of the robot metaphor with different scales and purposes in a smart environment based on embodied interaction. The smart door lock (Goji Smart Lock, 2014)<sup>2</sup> , as shown in **Figure 7A**, is a small-scale version of the robot metaphor using a physical object and represents automation and autonomous features of a smart home, automatically detecting occupants and opening the door. The building component, the smart door lock, is a robot used to manage home security in this case. The smart door lock uses the sensing type of interaction so that the door lock detects a user when the user comes close to the door. The interface types are physical object, GUI, and appliance. Although the door lock presents some text and icons (GUI), the interface mostly depends on sensors for the major function of security and the mechanical components as a robot. The digital affordance is the lock icon (open/close), and it allows the user to manually control the door lock. The door lock is a round type of knob, and it is used as a physical affordance to turn the knob.

A shape-changing bench (Grönvall et al., 2014) called coMotion is a public bench that is installed in public spaces, as shown in **Figure 7B**. The bench, as a robot, changes shape by itself by detecting people sitting on it and their body movements to encourage their social communication. The shape-changing bench uses two types of interaction: sensing and exploring. Basically, the interaction between the user and the bench is that of sensing. For instance, the bench detects a sitting user and then changes its shape. In another case, the bench changes its shape when the user changes his/her sitting position. However, if the user explores the shape of the bench to find a more comfortable position, the interaction type would be that of

<sup>2</sup>Goji Smart Lock (2014) Indiegogo. Available at: http://www.indiegogo.com/ projects/400022/fblk (accessed February 28, 2020).

FIGURE 7 | Embodied interaction designs in robot metaphor. (A) Smart door lock (Goji Smart Lock, 2014), (B) shape-changing bench (Grönvall et al., 2014), (C) robotic suitcase (Travelmate Robotics, 2018)<sup>3</sup> .

exploring in this case. The interface types are physical objects and shareable. The bench uses only a physical object with sensors and mechanical elements. Therefore, there are no visual interfaces and affordances. The physical affordance is the shape of the bench, such as the high and low positions of the seat. The shapes allow users to decide their sitting position and posture.

The robotic suitcase (Travelmate Robotics), **Figure 7C**, shows an everyday object that is mapped to a physical robot. As a mobile robot, the suitcase follows the user by detecting the user's location. The robotic suitcase also reacts to the user's gestures such as beckoning him/her to come closer and gesturing at him/her to turn around. The interaction type of the robotic suitcase is that of sensing by detecting the user's location and body movement. The interface types are physical objects and mobile, which is an automatic system that consists of sensors and mechanical components. The affordance and signifier is the LED lighting that indicates the moving direction.

In the robot metaphor, the examples depend on the sensing type of interaction and the robot type of interface, since the major characteristic of the robot metaphor is a physical automation system for achieving certain functions. However, the interaction type can be that of exploring when the purpose/function of the design involves more user actions. The affordances in the robot metaphor are physical and depend on the embodied interactions used and their shapes.

#### Smart Environment as a Friend

A smart environment with embodied interaction is a friend in the sense that the interaction centers around advising and supporting the users. Interactions based on this metaphor have the role of supporting activities within a specific context. The primary characteristic of this kind of interaction is an open-ended structure and outcome, which are characterized by meandering interaction. This metaphor represents performing tasks with assistance regarding the supporting users' activities. This metaphor emphasizes conversation and the perspective of a friend being offered. It reflects research in personification, effect, and artificial intelligence as well as co-learning over time. The friend metaphor represents conversational interaction, which involves conversing and exploring types of interaction that reflect a human-like manner. It includes the interface types of speech and multimodal. Affordances and signifiers for this metaphor differ from traditional affordances and signifiers that mostly focus on the shape of a familiar object as a visual cue. Instead, this metaphor uses speaking manner (e.g. instructional speech, emotional speech, speech tone, speech pace, and speech pitch), facial expressions (e.g. angry face, crying face), and the actions of a virtual assistant (e.g. waving hello, dancing, and drinking coffee).

There are many home assistants termed smart speakers nowadays with the increasing use of AI technologies, and they support users' everyday activities like setting timers, providing news, and playing music in a human-like manner, such as using voice interaction, for instance, as depicted in **Figure 8A**. In this case, the AI agent is mapped to a friend that supports the user's

<sup>3</sup> Travelmate Robotics (2018). Travelmate Robotics. Available at: Available at: https://travelmaterobotics.com/ (accessed February 28, 2020).

<sup>4</sup> Gatebox (2019) 株式会社 株式会社 トップページ- Gatebox. Available at: https:// www.gatebox.ai (accessed February 28, 2020).

routine activities using natural language in a human-like manner, although most smart speakers use voice interaction in a humanlike manner, where a human-like manner can involve other types of interaction such as virtual character, text (chatbot), image, etc. Amazon Echo (Amazon Echo Alexa speaker) is a smart home device, and the user interacts with the virtual assistant called Alexa through voice conversation, like for instance, "Alexa, what's the weather report for tomorrow?" The interaction type of Amazon Echo is that of conversing, and the interface types are speech and appliance. In this case, affordances are the voice responses of Alexa such as communicating information and instructions by speaking.

The Viewpoints AI installation, **Figure 8B**, is a participatory interactive installation that allows humans to collaborate with a virtual agent to improvise movement-based performance pieces together in real time (Jacob and Magerko, 2015). The virtual agent analyzes participant movements and improvises responses based on past interactions with people. In this case, the AI agent is mapped to a friend to act as a dance partner and cocreative performer. The interaction type of this example is that of exploring, which explores the movements of a virtual agent, including physical and spatial movements. Although it uses the exploring type of interaction, the interaction is analogous with conversing in terms of communicating through dancing and cocreating performance. The interface types are mid-air gestures and shareable display. The affordances are bodily movements of the virtual agent (e.g. mimicking the user's dance, changing the tempo of the dance, and suggesting a new dance based on the user's movements).

Gatebox (Gatebox 株式会社 ), displayed in **Figure 8C**, is a virtual home assistant that uses a hologram and is an example of virtual character–type human-like manner. Unlike the first two examples, the virtual character of Gatebox uses a combination of speech interaction and whole-body actions like an actual person. In this case, the bodily actions of the virtual character can play an important role as an affordance, for instance, having teatime with the user. In addition, the virtual character can respond to and send a text message to the user. Thus, it affords more complicated users' actions, emphasizing emotional relationships between users and the virtual assistant even if it provides functions similar to those of Amazon Echo. Gatebox uses a conversing type of interaction. However, it also uses a more emotional human-like manner based on the interface type, while Amazon Echo uses only speech interaction with functional purposes in a human-like manner. Gatebox uses mixed reality with a hologram as an embodied interface, and the technique used is a multimodal interface including speech and display. Gatebox involves many different affordances since it uses a visual and emotional assistant. The virtual assistant can gesture and speak emotionally to induce a user action. Some examples of these gestures are greeting gestures, pointing icons/information, running to the user, etc. The gestures make the user greet the assistant, look at the information, and take an action according to that information. Examples of emotional talking through speech and text message are "come home early" and "hmmm. . .." Talking makes the user respond or take an action.

In the case of the friend metaphor, the interaction type should be conversing since the essential characteristic of the friend metaphor is a natural dialog. In the examples given, speech is the common interface type for conversing interaction. However, gestures can be also considered for an interface type such as dancing. Another aspect of the interface type is visual elements such as the visual assistant of Gatebox. Although Gatebox uses mixed reality with a virtual character, it can be expanded to other types of interface such as wall/floor display and 3D projection. In addition, both of the examples are home device types of appliances. However, potential designs can be embedded into building components such as walls, windows, and doors. Lastly, the use of visual assistants shows potential affordances that involve the emotional aspect of a human-like manner. Further research is needed for identifying potential affordances.

# FRAMING NEW DESIGNS OF SMART ENVIRONMENTS USING CONCEPTUAL METAPHORS

#### Educational Scenarios

In order to explore the effect of the conceptual metaphors in design scenarios, we used the conceptual metaphors in educational settings. We focused on two perspectives of conceptual metaphors: how people adopt the metaphors as a mental model from the user's perspective and how designers apply the three metaphors to develop their conceptual design from the designer's perspective. In this section, we describe the factors that influence adopting the metaphors for new designs and example designs that the students created using the metaphors.

#### Users' Perspective on the Use of Conceptual Metaphors: A HCI Course

In order to study users' mental model on the use of three conceptual metaphors, we created and conducted a class activity, the Design by Metaphor learning activity, for a HCI course that introduces and provides experience in concepts and methods for HCI. The course is an active learning course that encourages learning through class activities. We developed a class activity to explore Design by Metaphor in three courses: 2019 summer (15 students), 2019 fall for undergraduate students (88 students), and 2019 fall for graduate students (43 students). We introduced the role of metaphors in HCI and examples of metaphors in HCI before conducting the class activity. We also introduced the three conceptual metaphors with example designs and design elements associated with each metaphor. The class activity is to identify signifiers, affordances, and interaction modalities in a given interactive design and describe which metaphor could be used to describe the design. For the activity, the students were given the three metaphors (device, robot, friend) and 18 examples of HCI designs in a template file including the definition of the three conceptual metaphors and a picture of each of the 18 example interaction designs. The 18 example designs for the activity include various types of interface and interaction (i.e. GUI, TUI, embodied, mobile, touch, mid-air gesture, speech, AR,

VR, wearable, robotic, AI assistant). We selected the examples to represent different types of interaction, interface, modality, material, and appearance of design elements.

In the activity submissions, the students showed distinct patterns for mapping between the metaphors and design examples. The device metaphor was predominantly mapped to examples of GUI, mid-air gesture, and TUI. The robot metaphor was predominantly mapped to examples of physical movements of an artifact responding to users' movements (e.g. a kinetic facade, an interactive wall, a travel suitcase) and a car infotainment. The friend metaphor was predominantly mapped to various AI assistants (e.g. Amazon Echo, Pillo, Gatebox). These patterns are consistent with the metaphorical framework that we identified from the review of existing embodied interaction designs shown in **Table 1**. In other words, the device metaphor is strongly associated with direct manipulation, the robot metaphor involves automated features and physical movements, and the friend metaphor reflects conversational interaction and a humanlike manner.

Another interesting pattern is that some design examples were mapped to multiple metaphors. In these cases, the design examples mostly use multimodal interaction. For example, most students mapped either the robot metaphor or the friend metaphor to a personal assistant using GUI, touch, and speech interaction (e.g. Google Assistant app). Some students selected both the robot and the friend metaphor when mapping specific design factors to each metaphor. That means specific design factors such as the shape of the physical/digital design and the interface type affect selecting the metaphors.

We also identified the themes that the students described in mapping a design example to one of the metaphors. We identified three emerging themes from the answers to the question: why did you select a specific metaphor for the signifiers, affordances, and modalities you found in the design example? The three emerging themes are: signifier and affordance, interaction modality, and purpose of design. The students mostly focused on signifier and affordance and interaction modality in the device metaphor. That means when the metaphor is device, they focused on what the interface looks like and how it is controlled as a device. In the robot metaphor, the students mostly focused on interaction modality. In other words, they focused on the sensing type of interaction with automated features and physical movements of the robot metaphor. For the friend metaphor, the students focused on Purpose of Design associated with the concept of human-like manner and Interaction Modality associated with conversational interaction.

#### Designers' Perspective on the Use of Conceptual Metaphors: An Interaction Design Studio Course

In order to study how designers apply the three metaphors to develop their conceptual design, we used the three metaphors in an interaction design studio course. The course is a studio approach to teaching topics in interaction design. Aspects of interaction design taught in the studio include: gesturebased interaction, tangible interaction, large public display interaction, tabletop interaction, multi-touch tablet interaction, and human–robot interaction. The topic of the design studio for the semester was metaphorical design for human–building interaction, and the design project focused on designing interactive buildings. The design project was a team project, and each team consisted of three to four students. During the semester, the students developed their design concepts based on the conceptual metaphors, implemented the prototype, and evaluated the prototype. We introduced metaphorical design and examples of metaphors in HCI at the beginning of the semester. We also introduced the three metaphorical concepts for HBI with examples, and the students performed precedent studies associated with each metaphorical concept during the first 3 weeks. After that, we assigned one of the metaphors (device, robot, or friend) that the students were asked to use when developing their design in their team project. There were six design teams, and we assigned each metaphor to two teams: two teams for the device metaphor, two teams for the robot metaphor, and two teams for the friend metaphor. The students used the assigned metaphorical concept and interaction type, interface type, and affordance in the design process as a framework for framing their conceptual design. In this section, we describe three design concepts that the students created and how the students applied each metaphor to develop their design concept.

The design concept influenced by the device metaphor is a smart door for a professor's office that responds to visitors when the professor is absent. This design concept expanded the concept of an answering machine to an office door as a device. To be specific, when a student knocks on the door, the door greets the student and informs him/her that the professor is out for the day by playing a recorded voice, followed by prompting the student to leave a message with the student's contact information. Once the student leaves a message, the door system immediately sends the student's message to the professor by mobile app. The HCI techniques used for this design concept are:


This design concept presents a good example of metaphorical design in terms of mapping an existing concept of a device to a building component through analogical reasoning. The design team identified a novel function from an answering machine, and the answering machine was transferred to an office building. The design concept also shows a novel behavior for potential smart environments through the affordances and signifiers. The affordances and signifiers are a combination of physical and digital affordances that represent the characteristics of a door and an answering machine. As a result, designing the interaction model was strongly influenced by the metaphorical reference.

The design concept influenced by the robot metaphor is an interactive door for a classroom that provides class information. The interactive door detects students and displays the class information (e.g. class name, class schedule, number of students in the classroom) as a student approaches the door through emojis and messages. The emoji is a major design factor that indicates the class status. For example, when a student arrives at the door, the student can see a lit-up emoji with a neutral face and a small message "5 min left until the class database 101." The HCI techniques used for this design concept are:


fpsyg-11-00198 March 13, 2020 Time: 17:19 # 12

• Affordance/signifier: emojis for indicating the class status

The design team focused on two aspects to adopt the robot metaphor: the automated sensing and the visual interface for the system. The automated features in the design are detecting users, collecting spatial data, and displaying the building information to increase the users' awareness on their activities and the building status. While the automated features represent the characteristics of the robot metaphor, the emoji visualization makes the design look more like a typical robot.

The design concept influenced by the friend metaphor is a smart speaker, an office assistant called Pluto, for a meeting room. The smart speaker, which has a 3D-printed alien shape, is a voiceactivated recording system for a meeting room. It detects the users' presence in the room and responds to the users' speech and records a group conversation or individual presentation, and then it exports a text transcript to an email or a Google Drive folder. The used HCI techniques for this design concept are:


The design concept is similar to existing smart speakers such as Amazon Echo. However, this design shows an example of an embedded AI assistant in a built environment focusing on the meeting activities and the meeting space. That means the smart speaker has a clearer role as an office assistant for the meeting room, while existing smart speakers play more universal roles, mostly focusing on the smart home. Another aspect in adopting the friend metaphor in this design concept is the shape of the smart speaker. The smart speaker was designed as a 3Dprinted alien shape, and the ears of the alien include LED lights to indicate that the alien is hearing the users' voice. As a result, the combination of speech interaction, the physical shape of the design, and the signifier make the design more like a friend through personification of the interactive system.

# Framing a Single Product That Is Developed for Each of the Metaphorical Concepts

The three metaphorical concepts we present provide distinct design spaces based on the characteristics of each metaphorical concept and the HCI techniques associated with each metaphorical concept. The metaphorical concepts provide a conceptual framing for a design, and the HCI techniques provide a technical framing that facilitates the exploration of the design space in a practical way. Each metaphorical concept thus can guide a design to different concepts for the same purpose or the same function through the metaphorical framework. In this section, we describe how a specific design for smart environments can be conceptualized differently by the different metaphorical concepts.

The smart home and energy conservation are common examples for smart environments. Many smart home apps and smart devices provide similar functions for energy saving such as managing room temperature, but they use different interactions and interfaces. We describe how each metaphorical concept can frame designing a smart room for energy conservation as an example. We apply the same design goals and functions to the three metaphorical designs for the smart room design. The design goals are saving energy and satisfying environmental comfort. The functions are managing energy consumption, controlling room temperature, controlling lighting, and controlling shading.

Smart room as device. The device metaphor for smart environments provides the conceptual space for easy control in the built environment. In order to achieve the design goals and the functions, this conceptual space specifies the instruction type of interaction and includes possible interface types focusing on how to realize easy control. To be specific, information visualization, sharable, display, and mobile are possible interface types to achieve the function of managing energy consumption. For the function of controlling room temperature, lighting, and shading, GUI, TUI, touch, mid-air gesture, and mobile are possible interface types. While interface types frame a technical solution, affordances and signifiers guide detailed designs. In the device metaphor, metaphorical references provide visual and physical cues for affordances and signifiers, and the use of metaphorical references makes the design more like a device supporting a user's mental model. For instance, a dashboard design, icons, switches, dials, buttons, and sliders can be applied to the information visualization for indicating the energy consumption as a digital affordance. Vibrating blinds, flickering bulbs, beep sounds, and different colors of LED indicator on the wall can indicate a certain energy status and afford a user action to save energy as a physical affordance. The affordances associated with the device metaphor also help to design user actions (e.g. gesturing up/down, a dial shape of TUI, and tapping the wall) for controlling room temperature, lighting, and shading. These digital/physical affordances based on the interface types help to achieve the design goal of energy saving supporting the function of managing energy consumption and the design goal of satisfying environmental comfort supporting direct manipulation based on the user's preference.

Smart room as robot. The robot metaphor for smart environments captures the design space for autonomous features of smart environments. This conceptual space can expand the given functions, managing energy consumption and controlling room temperature/lighting/shading to collecting environmental data, tracking energy consumption history, and analyzing the user's pattern. These expanded functions then are transferred to specific system actions based on the sensing and exploring types of interaction. The expected system actions enable a personalized setting for providing comfort and optimizing environmental conditions for saving energy such as automatic changes of

blinds and temperature. The physical system actions make the smart room a robotic building component. In this case, the design does not emphasize affordances since it does not require intentional user actions. For the metaphorical design using the robot metaphor, the focus is on the automated features and identifying novel functions/behaviors that can be automated by the system using additional sensors.

Smart room as friend. The friend metaphor provides the design space for personified smart environments that support occupants' personal activities. This metaphorical reasoning can be applied to a smart room (e.g. personified room) or a building component (e.g. personified door) through the use of a virtual character. The interaction types associated with the friend metaphor are conversing and exploring. In this case, the user saves energy and controls the environment through conversational interaction with the personified room. For example, a virtual character recommends opening/closing window blinds based on changes of external conditions, and the user confirms the change. Another example is that the virtual character gives energy saving tips based on the user's behavioral patterns. In this case, the smart room encourages a user's behavioral changes in the use of energy and increases the user's awareness. Selecting interface types for the friend metaphor depends on the visibility of the virtual character. Speech and/or display for chatting can be considered for an invisible virtual character, and AR and speech can be possible interfaces for a visible virtual character. For the personification of the environment, the smart room design can use different metaphorical references for the role of the friend (e.g. energy expert, building manager, buddy, colleague, and roommate). The role of the friend influences the design of the virtual character and the signifiers for the affordances (e.g. speaking manner, gestures, and actions).

# CONCLUSION

Metaphors provide a common perspective for characterizing new designs in HCI. Metaphors assist in forming a common mental model for new interactive designs that support designers in creating novel interaction experiences. Embodied interaction is an integral aspect of smart environments due to the scale of the environment. As recent embodied interactions increasingly expand to include new technologies with multimodal interaction, smart environments provide potential design spaces that are yet to be fully explored and understood. Characterizing smart environments with embodied interaction by using conceptual metaphors can be a new foundation for a theoretical and methodological framework to understand and discover the potential design spaces for smart environments. In this paper, we focus on how people interact with the built environment, while many architects focus on how advances in technology can improve the quality of the built environment using automation.

We present three metaphorical concepts that enable new ways of designing smart environments: device, robot, and friend. The device metaphor represents performing tasks through the users' direct control; the robot metaphor emphasizes automation, autonomous decision making, and the physical actions of a robot; and the friend metaphor represents performing tasks using human-like assistance for supporting users' activities.

A critical review of existing embodied interaction designs using the three metaphorical concepts with HCI techniques provides a framework for characterizing design spaces for smart environments. We reviewed 24 existing embodied interaction designs that represent various interaction modalities for embodied interaction. The analysis shows that each metaphorical concept refers to distinct interaction types, interface types, and affordances, which creates distinct design spaces for smart environments. We found that the device metaphor is associated with instructing, manipulating, and exploring types of interaction. These interaction types involve TUI, mid-air gesture, wearable, and OUI as interface types. The affordances and signifiers for the device metaphor use generalized GUIs and physical shapes of an interface to make the designs look like a physical device.

While the device metaphor focuses on direct manipulation, the robot metaphor focuses on automated features using exploring and sensing types of interaction. The interface types associated with the robot metaphor are robot, appliance, ambient device, and OUI with automation. Since these interface types rely on a sensor-based system in a physical space, the affordances and signifiers for the robot metaphor involve physical movements of occupants and/or architectural components rather than graphics and physical shapes.

The friend metaphor reflects conversational interaction and a human-like manner based on the personification of the interactive system, while the device and the robot metaphor reflect functional aspects of the artifact. The interaction types for the friend metaphor thus include conversing and exploring. The primary interface type for the conversing type of interaction is speech, but it can be expanded to many other interface modalities with dialog features and applications such as appliances and wearable devices. The affordances and signifiers for the friend metaphor are unique and interesting in the sense that it uses speaking instruction, emotional speech, tone, pace, pitch of speech and actions of a virtual assistant associated with physical or graphical shapes. Identifying new affordances and signifiers for the friend metaphor is a topic for research on novel interaction designs for smart environments.

The educational experiences in which we presented the three metaphorical concepts show the effect of the conceptual metaphors on recognizing affordances in smart environments and on designing new smart environments. We focused on how people adopt the metaphors as a mental model from the user's perspective and how designers apply the three metaphors to develop their conceptual design from the designer's perspective. From the user's perspective, the students showed distinct patterns on mapping the metaphors to design examples that are consistent with the metaphorical framework identified from the review of existing embodied interaction designs. That means the metaphorical concepts can provide a sharable mental model for new designs. We also identified the influential factors in adopting each metaphor: signifier and affordance and interaction modality in the device metaphor, interaction modality in the

robot metaphor, and purpose of design and interaction modality in the friend metaphor. These findings support the effect of the metaphorical design framework on users' perspective, but further research is needed to collect statistically significant data in a laboratory study in order to verify the effect of the metaphorical framework. From the designer's perspective, the conceptual designs that the students created show how designers use the metaphorical framework to develop a design concept for smart environments. The design applying the device metaphor actively used a metaphorical reference (i.e. answering machine), a building component (i.e. door), and physical affordance (i.e. knocking). The design applying the robot metaphor focused on automated features and the visual representation of the system. The design applying the friend metaphor showed a unique role of a friend in a specific environment (i.e. office assistant), an appearance of the personified object (i.e. alien), and the signifiers for the affordances of a virtual assistant (i.e. LED indicator on the ears and the speaking manner). These results show different aspects of metaphorical design when applying the metaphors and framework for a specific design. The limitation of these results is in the scale (20 participants) and context (design education) of the study. Further research is needed to conduct a laboratory study of the three metaphors that recruits more participants and studies the effect across several design tasks.

We explored how each metaphorical concept can conceptualize a specific design differently through framing a smart room design for energy conservation. The exploration of a smart room design using the three metaphorical concepts shows that each metaphorical concept facilitates: achieving the design goals by providing a distinct conceptual space, selecting interaction types based on each metaphorical concept as a guide to explore possible interface types, and guiding the designer toward specific affordances and signifiers based on metaphorical references when designing the interactions, user actions, and interfaces. In framing the smart room design, the device metaphor provided the conceptual space for easy control to achieve the design goals, and the interaction type and the interface type focused on how to realize easy control. Physical affordances in the device metaphor involved physical building components to make the smart room a device, and the combination of digital and physical affordances influenced the design of user actions using the metaphorical references.

#### REFERENCES


While the conceptual design in the device metaphor actively used metaphorical references for designing user actions, the conceptual design in the robot metaphor focused on identifying system actions associated with the automated features. The conceptual space for autonomous features facilitated expanding and transferring the given functions to automated functions, for example, controlling the room temperature by tracking/analyzing the occupants' patterns of behavior in the room. This metaphorical design thus provides a basis for identifying novel automated functions and behaviors of the system. The friend metaphor provided the design space for personified smart environments. In the metaphorical design using the friend metaphor, the role of the friend provided a basis for considering the interaction types and interface types. The metaphorical references for the role of the friend influenced the design of the affordances, and this metaphorical design facilitated identifying novel personalized and friendly functions and behaviors for the interaction.

We conclude that the metaphorical concepts presented can frame new design spaces that lead to a shared mental model and novel interaction designs for future smart environments. The metaphorical concepts can be a design tool and an educational tool for designing smart environments. The contribution of this paper is a review of existing embodied interaction designs from the perspective of three metaphorical concepts and a metaphorical design framework that enables novel approaches to conceptualizing interactivity in smart environments as a device, robot, or friend.

### AUTHOR CONTRIBUTIONS

JK composed this study, designed the framework, and completed the analysis. MM provided supervision throughout the research and contributed substantially to the analytical part of the research.

# FUNDING

This research received no specific grant from any funding agency in the public, commercial, or not-for-profit sector.

digital tabletop," in Proceedings of the Fifth International Conference on Tangible, Embedded, and Embodied Interaction, New York, NY: ACM, 93–100.



**Conflict of Interest:** The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Copyright © 2020 Kim and Maher. This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) and the copyright owner(s) are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.

# Neurophysiological Correlates of User Experience in Smart Home Systems (SHSs): First Evidence From Electroencephalography and Autonomic Measures

#### Laura Angioletti, Federico Cassioli\* and Michela Balconi

#### Edited by:

Mi Jeong Kim, Hanyang University, South Korea

#### Reviewed by:

Fali Li, University of Electronic Science and Technology of China, China Xiaoyu Xia, The Military General Hospital of Beijing PLA, China

> \*Correspondence: Federico Cassioli federico.cassioli@unicatt.it

#### Specialty section:

This article was submitted to Environmental Psychology, a section of the journal Frontiers in Psychology

Received: 14 November 2019 Accepted: 24 February 2020 Published: 19 March 2020

#### Citation:

Angioletti L, Cassioli F and Balconi M (2020) Neurophysiological Correlates of User Experience in Smart Home Systems (SHSs): First Evidence From Electroencephalography and Autonomic Measures. Front. Psychol. 11:411. doi: 10.3389/fpsyg.2020.00411 Research Unit in Affective and Social Neuroscience, Department of Psychology, Catholic University of the Sacred Heart, Milan, Italy

Home automation brings together technology, engineering, and user experience (UX). Within this framework, even neuroscience could be a valuable discipline to explore UX. For the first time, in the present work, some distinguishing effects of domotics on users' cognitive and emotional behavior are highlighted by using the neuroscientific approach. In order to define possible effects of a smart home system (SHS) on UX, a neuroscientific multimethodology was adopted with the purpose of recording and confronting the neural activity (electroencephalography, EEG) and autonomic system responses of 19 individuals during a resting state (RS) baseline and the exploration of five different tech-interaction areas in a domotic environment. EEG findings showed a generalized neural activation reflected by alpha band activity while participants were exploring the tech areas confronted with the RS. The delta band was mainly present in temporo-central compared to frontal and parieto-occipital areas and was interpreted as a higher emotional activation related to the whole UX. This effect was found for the sixth tech-interaction area (i.e., bedroom) compared to the RS, and it is supposed to represent an enhanced emotional response and integration processing toward a higher multisensory interactive area. Regarding autonomic activity, an increase in heart rate (HR) was found for the bedroom area compared to the RS, thus showing a specific effect on physiological indices in this engaging tech area. The present research constitutes the first attempt to understand the user responsiveness to SHS, in terms of cognitive and emotional engagement, by adopting a neuroscientific perspective. Some high-value benefits derived from this approach will be described in light of the neurophysiological results.

Keywords: electroencephalography, autonomic measures, domotics, home automation, smart home systems

# INTRODUCTION

fpsyg-11-00411 March 17, 2020 Time: 16:36 # 2

# Home Automation

Home automation, also known as "domotics" (from the Latin domus, "home"), is an interdisciplinary science, which, by using informatic, engineering, and user experience (UX) insights, delivers ways to improve the quality of life inside and through the habitation (Navarro-Tuch et al., 2019). This research field has been defined as an extension of ubiquitous computing aiming at the development of ambient intelligence for smart environments (Aarts and De Ruyter, 2009; Cook, 2009). These systems are generally associated to adjectives like "smart" or "intelligent" thanks to their capacity to help and meet some user needs with a certain degree of automation (Kanemura et al., 2013). Drawing a sharp line between traditional houses and domotics is not possible: however, there are several peculiarities that help differentiate the two concepts. Smart homes are positively typified by flexibility, in the sense that modifying the configuration of the system is effort- and cost-free even after its installation, and by multifunctionality in terms of capacity to operate and control several different functions via one single device. These mentioned proprieties lead to easiness to implement innovative features, without adding new devices and also simplifying the wiring scheme and therefore enhancing security (ENEA Report: Elia and Santini, 2011). Domotics systems not only receive orders and store information; they can also discover patterns of consumption, make inferences, and therefore optimize processes. Consequently, the use of these technologies enables guaranteeing the avoidance of waste and repetition, facilitating and assisting user intentions, but also adjusting the system to new technologies and users' specific needs.

# Neuroscience Studies on Smart Home Systems

Historically, researchers have been using qualitative methods (e.g., interviews) to explore in-depth users' opinions regarding smart home systems (SHSs). Also, valuable evaluative studies have been conducted. They tend to focus on relations between users and SHSs, adopting a psychosocial perspective and collecting precious information on "how smart homes work in practice (routines, meanings, technology, and knowledge), including relational aspects and functionality" (Gram-hanssen and Darby, 2016). To fully understand the impact and the quality of the interaction between the user and an SHS, self-report methods are needed but may be not enough, because subjects provide only explicit and conscious information. In this sense, neuroscientific tools offer high-value insights exploring implicit neurophysiological mechanisms with real-time cognitive and emotional response data with a good balance between ecological and internal validity, also thanks to new instrumentations featuring wearable and wireless technologies (Mihajlovic et al., 2015). Neuroscience literature on home automation has previously focused on applying brain-controlled systems allowing people to control environment functions (Babiloni et al., 2009; Cincotti et al., 2010; Aloise et al., 2011). Other studies have explored the brain–computer interface (BCI), a system that is able to recognize patterns of electrical activity in the brain, through high-density electroencephalography (EEG) devices, therefore creating a new communication channel with the environment (Babiloni et al., 2007). However, to our knowledge, there may be a literature gap when bringing together neuroscience, UX, and home automation. In fact, no previous research has explored and studied the interaction between users and home automation from a neuroscientific cognitive and emotional perspective.

The present study aimed at investigating users' neurophysiological correlates by considering EEG and autonomic responses (biofeedback) during interactions with a domotics environment. Specifically, EEG data allow the collection of information on brain activity, which ensures a deeper understanding of both cognitive and emotional processes (Aftanas et al., 2001; Balconi et al., 2015; Khushaba et al., 2013). Regarding autonomic activity, the data offer information about arousal and engagement levels; for example, heart rate (HR) activity modulations are linked to positive or negative emotions (Van't Wout et al., 2006; Vanutelli et al., 2017a,b).

The use of a multimethodological neuroscientific approach allows exploration of the impact of both explicit (conscious correlates) and implicit (unconscious correlates) levels of users' home automation system (Balconi et al., 2015). Specifically, in this study, the implicit and unconscious correlates were explored by using central and peripheral indices. Moreover, this approach may help in providing interesting insights on the UX as highlighted in previous exploratory contributions on emotional domotics (Angioletti and Balconi, 2019; Navarro-Tuch et al., 2019). For instance, EEG cortical oscillations can be informative of emotion processing, and different levels of investigation are possible. Firstly, regarding lateralization, the valence-specific hypothesis argues that both hemispheres process emotion, but each hemisphere is specialized for valence-specific emotion: with the left hemisphere more dominant for positive emotions and the right hemisphere for negative emotions (Ahern and Schwartz, 1979; Davidson, 1992). Secondly, different brain regions play specific roles related to the emotional process: (i) frontal and prefrontal cortex activation have been related to cognitive control over emotional stimuli and emotional behavior (Balconi et al., 2015); (ii) temporo-central areas have been involved in sensory processing (Kayser and Logothetis, 2007) and are a part of a more extended neural network responsive to the environment and social stimuli (Acevedo et al., 2014); and (iii) parieto-occipital areas are recruited by emotional visual stimuli, in particular when these stimuli are highly arousing (Lang et al., 1998).

During the interactions between an SHS and users, we expect to see a higher general activation compared to the resting state (RS) (baseline) because of higher involvement and resource allocation processes. Specifically, we expect to observe both brain and autonomic responses to an SHS in terms of: (i) a general decreased alpha brain power (increased activity) due to the increase of individuals' attentional engagement (Dimberg and Petterson, 2000; Davidson, 2002; Balconi et al., 2014); (ii) different levels of delta band activation in response to SHS ascribed to emotion-related information processing and caused by the novelty of the stimulus and pleasantness

Angioletti et al. Neurophysiological Correlates of UX in SHS

experience for the user (Balconi and Lucchiari, 2006); and (iii) a concomitant increase of individuals' HR response, showing emotional arousal and a certain level of engagement (Benedek and Kaernbach, 2011).

### MATERIALS AND METHODS

#### Participants

Nineteen healthy subjects (Mage = 25.05, SDage = 3.05, age range: 18–27, nmale = 7) were recruited for the study. Inclusion criteria were normal, or corrected to normal, visual acuity and age between 18 and 38 y3ears. All participants voluntarily took part in the experiment after being informed about the study aims expressed by the informed consent. This research was conducted following the principles and guidelines of the Helsinki Declaration and was approved by the Ethics Committee of the Department of Psychology of the Catholic University of the Sacred Heart.

#### Procedure

The experiment took place in a home automation environment in Milan (Italy), which was the show loft owned by the tech company Duemmegi S.r.l., a developer and seller of domotics systems. Participants were asked to explore and interact with the home automation systems by using heterogeneous functions around the environment. The points of interaction were situated in the following areas of the house: hall, kitchen, living room, bathroom, and bedroom (**Figure 1**). The rationale for this selection depends on two aspects: (i) these areas constitute the environments present in the domotic living space and (ii) in each area, there was the installation of main devices producing effects testable by neuroscientific measures.

Each of these interactions was activated by pre-set commands on a smartphone app. The equipment was fully provided by the research team, with a short previous briefing where subjects were shown how to use the app and the interface. After the participant issued the command, each tech-interaction area produced a specific feedback eliciting novel sensory stimuli. Every interaction had a certain level of complexity in terms of response provided by the SHS. For example, the first ones consisted in a simple output presented by the system after the request of the user (e.g., turning the radio on). During the task, the SHS responses started to become more sophisticated, involving and connecting more than one device together, providing a full environmental reaction. The total exploration time for all the areas was about 30 min. The phase of exploration was preceded by the recording of a 120 s baseline in a silent zone, with subjects facing a white wall without any particular stimuli present. The first point of interaction (hall area) consisted in turning the light and the radio on. The second one (kitchen area) involved the activation of the kitchen with a hidden stove, an oven, and a wine container appearing from a normal desk. The third one (living room area) consisted in activating a multimedia projector on a screen that appeared on the wall only after the interaction started. The fourth one (bathroom area) involved the activation of a series of features such as: activation of specific lights for a shaving or makeup

session, chromotherapy, and activation of a smart television. Finally, the fifth one (bedroom area) consisted in a "home bedtime mode" activation, where the blinds were closed and the home secured by locking the front door and turning off the gas and lights. The growing activation of devices in each area engaged participants in interacting with the environment. Each area involved more than one sense and provided intrinsic pleasantness in individuals, by striving to fulfill domestic needs. During the exploration, individuals' neural activity and autonomic responses were monitored using EEG and biofeedback measures.

#### Neuroscientific Measures EEG

Electroencephalography measures were collected via a 15 channel EEG system (LiveAMP, Brain Products, Munich, Germany) with electrodes positioned over Fp1, Fp2, F3, Fz, F4, T7, T8, C3, Cz, C4, P3, Pz, P4, O1, and O2 (**Figure 2A**), adopting the 10/20 system of electrode placement (Jasper, 1958). An ElectroCap was used for signal recording. Data were acquired with a frequency band between 0.01 and 40 Hz and a sampling rate of 500 Hz. The electrode impedance for each individual was monitored before data collection and was kept <5 k. Portions of data that presented artifacts were removed in order to increase the specificity. Ocular artifacts (blinks and eye movements) were corrected using an eye movement correction algorithm via a regression analysis in combination with artifact averaging (Sapolsky, 2004). Finally, a standard independent component analysis (ICA) analysis was applied. The EEG data were bandpass-filtered (0.1–40 Hz, 48 dB/octave roll-off), and frequency power data were computed by fast Fourier transformation (FFT) for standard frequency bands: delta (0.5–4 Hz), theta (4–8 Hz), alpha (8–12 Hz), and beta (14–20 Hz) (Keil et al., 2001).

#### Biofeedback

Autonomic measures were collected with a Biofeedback 2000xpert system with radio module MULTI (Schuhfried GmbH, Mödling, Austria) positioned on the participant's hand. The device was connected to a computer with Bluetooth. The system, via a sensor (4 mm diameter Ag/AgCl electrode) attached to the

volar surface of the middle phalanges of the forefinger of the non-dominant hand (**Figure 2B**), was able to measure peripheral parameters. The recorded indices were: HR, skin conductance level (SCL), skin conductance response (SCR), pulse volume amplitude (PVA), and blood volume pulse (BVP). Specifically, for HR index, inter-beat intervals of the electrocardiogram were converted to HR in terms of beats per minute (bpm), scoring peak acceleration during the various experimental conditions. Both SCR and SCL were measured in µS: values of SCR were manually scored and defined as the largest increase in conductance during the domotics interactions. For SCL, the level of conductance as average during the conditions was considered. Moreover, for PVA and BVP, the pressure changes within the probe in the fingertip were transmitted to a personal computer (PC), where signals were band-pass-filtered (0.3–30 Hz), amplified, and stored. We considered the average of PVA variations during the home automation interaction.

#### Data Analysis

For EEG data, four repeated measures multivariate analysis of variances (MANOVAs) were separately applied to the dependent measure of frequency bands (delta, theta, alpha, beta). We considered three regions of interest (ROIs) composed and obtained by averaging the electrodes in the following way: frontal (Fp1, Fp2, F3, and F4), temporo-central (C3, C4, T7, and T8), and parieto-occipital (P3, P4, O1, and O2). Also, the lateralization, in terms of the left and the right hemisphere was considered [left frontal (Fp1, F3), right frontal (Fp2, F4), left temporo-central (C3, T7), right temporo-central (C4, T8), left parieto-occipital (P3, O1), and right parieto-occipital (P4, O2)]. Analysis was carried out with the following within factors: ROI (three: frontal, central– temporal, and parieto-occipital), lateralization (two: left and right hemisphere), and area (six: baseline, hall, kitchen, living room, bathroom, and bedroom). Regarding autonomic measures, a set of four repeated measures MANOVAs was conducted for each index (HR, SCL, SCR, PVA, and BVP) as well with area (six: baseline, hall, kitchen, living room, bathroom, and bedroom) as a within factor. For all ANOVA tests, degrees of freedom were corrected by the Greenhouse–Geisser epsilon when appropriate. Post hoc analysis (contrast analysis for ANOVA, with Bonferroni corrections for multiple comparisons) was successively applied. The size of statistically significant effects has been estimated by computing partial eta squared (η 2 ) indices.

# RESULTS

# EEG

#### Alpha Band Activity

As shown by MANOVA for the alpha band, a main effect for area was found [F(1,90) = 6.09, p ≤ 0.001, η <sup>2</sup> = 0.28]. Pairwise comparisons revealed higher levels of alpha activity for the baseline condition (for all comparisons, p ≤ 0.001) compared to other areas. No other significant effects were found for other areas. Significant results are reported in **Figure 3A**.

#### Delta Band Activity

As shown by ANOVA for the delta band, a main effect for ROI was found [F(1,37) = 7.46, p ≤ 0.001, η <sup>2</sup> = 0.27]. Pairwise comparisons revealed higher levels of delta activity in temporo-centrale (TC) regions (for all comparisons, p ≤ 0.001) compared to other ROIs.

Moreover, a significant interaction effect ROI × area was found [F(1,182) = 6.33, p ≤ 0.001, η <sup>2</sup> = 0.26]. Pairwise comparisons revealed higher levels of delta activity in area 6 compared to baseline in TC [F(1,18) = 6.61, p ≤ 0.001, η <sup>2</sup> = 0.26]. No other significant effects were found for other areas. Significant results are reported in **Figure 3B**.

No significant effects were found for other EEG bands (theta and beta) and for lateralization factor.

#### Biofeedback

As shown by MANOVA for HR, a main effect for area was found [F(1,17) = 5.54, p = 0.031, η <sup>2</sup> = 0.24]. Pairwise comparisons revealed lower levels of HR for the baseline condition compared to the sixth area (bedroom) [F(1,17) = 5.54, p = 0.18; η <sup>2</sup> = 0.24]. No other significant results for autonomic measures were found for other areas. Significant results are reported in **Figure 4**.

### DISCUSSION

This study provided new insights on the neurophysiological correlates of UX inside a domotic environment using neuroscientific tools. Specifically, participants explored an SHS show loft composed of five main tech-interaction areas while their central (electrophysiological) and autonomic (peripheral) system activity were collected. Tech-interaction areas coincided with the show loft's ambients (hall, kitchen, living room, bathroom, and bedroom), and every interaction area had a certain level of complexity in terms of the response provided by the SHS: indeed, in the last areas, the SHS responses started to become more sophisticated, involving and connecting more than one device together, providing a full environmental activation.

This increasing level of complexity was detected in the present sample by EEG and autonomic modulation activity. The intrinsic relationship between these two different levels of measures will be elucidated below.

Firstly, at the cortical level, an increase of alpha band oscillations was mainly found for the baseline condition compared to the other tech areas, and this result could be interpreted as a mechanism of attentional suppression when participants were at rest, followed instead by a cognitive generalized brain activation during tech-interaction area exploration. This result is in line with our expectations and with previous basic research on alpha band activity describing a decrease of alpha power with higher cortical excitability (Foxe and Snyder, 2011). Moreover, according to Ray and Cole (1985), alpha oscillations are lower for tasks requiring attention to the environment (e.g., our tech-interaction areas), than for those not requiring such attention (e.g., RS). This finding could suggest that in general, a domotic space and these tech-interaction areas have required relevant attentional demands to participants and engaged them at a cognitive level.

Recently Navarro-Tuch et al. (2019) integrated a classical methodology for emotion elicitation, that is, the International Affective Picture System (IAPS; Lang et al., 1990), in an experimental domotic environment, and theoretically discussed the emotional domotics general system and components. According to this model, UX depends on two components: the physiological and emotional variables that can be measured by multiple wearable devices such as physiological and cortical sensors. Although their experimental results will be discussed in future research articles, the authors stated that they mainly focused on physiological (autonomic) variables and facial expression analysis. On the other hand, the present preliminary findings may suggest that, in order to analyze a holistic UX in an SHS, the integration of neural information deriving from cortical oscillations' functional meaning can be highly informative, firstly for understanding whether users are focusing their attention on the target and secondly to verify the coherence between autonomic responses and high-level complex reactions.

Secondly, an increase of delta power was found in temporocentral brain regions compared to anterior and posterior ROIs

during the whole experimental phase. This effect might be interpreted as an emotional activation, perhaps related to the intrinsic pleasantness of the experience inside the domotic show loft (Grandjean and Scherer, 2008; Balconi et al., 2009). Indeed, regarding the cortical origin of delta waves during cognitive processes, it has been hypothesized that these low-frequency oscillations are associated with motivational and emotional states involving prefrontal brain structures (Knyazev, 2007). Previous basic research highlighted that the delta band depends on activity of motivational systems and participates in salience detection of emotional stimuli (Knyazev, 2007). In addition, its modulation was also found to be related to the arousing power of stimuli regardless of the valence. Therefore, it may be responsive to motivational and attentional significance of relevant emotional cues per se (Basar, 1999; Balconi et al., 2009; Balconi and Pozzoli, 2009). With reference to the ROI, anterior modulations of delta power are thought to mirror cognitive load related to emotional information processing, while delta increase over more posterior areas was proven to occur following stimulations that have an emotional positive/appetitive connotation (Balconi and Lucchiari, 2006; Balconi et al., 2009).

Moreover, this temporo-central delta band manifestation was significantly higher in the sixth tech-interaction area compared to the RS. The nature of the delta band as a marker of emotional relevance might confirm that this last area with its features was the most emotionally engaging. Nonetheless the role of the delta band during cognitive tasks has been previously associated with cortical inhibition of the sensory afferences that interfere with internal focus (Fernandez et al., 1995; Harmony et al., 1996, 2009; Harmony, 2013). Before, delta oscillations were shown to be implicated in the synchronization of brain activity with autonomic functions, in motivational processes associated with both reward and defensive mechanisms, in higher emotional involvement, and in cognitive processes related to attention and the detection of motivationally salient stimuli in the environment (Knyazev, 2009, 2012). Thus, a possible explanation might be that in the present study, delta activity in temporo-central brain regions reflected a condition of (i) emotional focus on specific external engaging and interacting environments and (ii) attentional orientation for novel and partially unexpected stimuli (characterizing the sixth tech-interaction area) (Fernández et al., 1998; Balconi and Lucchiari, 2006; Grandjean and Scherer, 2008). Otherwise, another possible alternative explanation could be that during the feedback provided by the sixth area, an intersensory integration involving sensory cortices in temporo-central brain regions occurs (Kayser and Logothetis, 2007). Still, given that this is the first study exploring the role of delta in complex dynamic environments, before coming to strong conclusions, caution is needed.

It is worth noticing that we did not find significant results for the high-frequency beta band and theta band, perhaps because these cortical oscillations reflect more the controlled emotion cognitive appraisal (e.g., goal conduciveness and task/goal relevance; Grandjean and Scherer, 2008), which is possibly less consistent in this exploratory study compared to the higher automatic emotional reactivity derived from delta and autonomic findings. Moreover, no significant differences were found for lateralization in the present study; therefore, no valence-specific effect can be stated by this first preliminary evidence. However of great interest would be the deepening of possible brain lateralization effects in users who explore and live in an SHS.

Lastly, HR activity was greater while participants were exploring the different tech areas compared to the baseline condition. Once again, this effect was mainly significant for one specific space of the show loft, the bedroom. This domotic area was characterized by the activation of different scenarios involving the sensory system: indeed, lights turned off slowly, and a vocal sound alerted the person on the activation of the security alarm.

In neuroscience literature, autonomic parameters are considered sensitive performance measures, and according to the "doctrine of autonomic space," Backs et al. (2005) suggested a relation between increased sympathetic activation over time and the number of executive processes involved in the situation. Also, previous research suggested an association between HR increase and cognitive demand (Kramer, 1991; Roscoe, 1992; Backs and Seljos, 1994; Veltman and Gaillard, 1998; Brookhuis and De Waard, 2001; Wilson, 2002). Thus, it is plausible that users sensitively detected heightened cognitive load while entering the bedroom and invested additional cognitive resources to maintain a given level of arousal as demands increased. In line with this, HR increase was previously linked to an emotional arousal, involvement, or stress condition (Benedek and Kaernbach, 2011; Balconi and Vanutelli, 2015; Vanutelli et al., 2017a,b). On the whole, HR increase was interpreted as a confirmation of generalized activation at the physiological level related to the domotic environment's cognitive and emotional effects compared to a resting condition. The qualitative observation of a higher HR for all areas compared to the baseline might confirm areas' emotional intrinsic pleasantness engaging the participants. Regarding the effect found for the bedroom (sixth tech-interaction area), a significant HR increase was found concurrently with higher delta in TC brain regions, perhaps as a possible marker of motivational–emotional activation related

to a noticeable difference featuring this area compared to other areas, that is, the vocal human sound on safety measures.

To conclude, the present study provides initial evidence about the integration of cortical and autonomic measures in healthy individuals during their UX inside a domotic environment and while interacting with a complex SHS. The interaction with different multisensory tech-interaction areas induced in participants a cortical and autonomic modulation related to increased cognitive processing and emotional engagement (specifically in temporo-central brain areas). Also, specific techinteraction areas were found to be more engaging than others, and this could be due to their multisensory nature able to augment UX, since the effect was present only in the most complex areas. Regarding the use of these two measures (EEG and autonomic indices), they are independent on a functional level (Balconi et al., 2015), since for instance, HR is more representative of the autonomous emotional impact derived from the areas.

Despite its several advantages, this study is not without limitations, and it is necessary to be cautious with the interpretation of present results that, so far, constitute initial experimental evidence in the field. Indeed, the benefits of exploiting a multimethodology involving central and autonomic measures were discussed, although self-report scales and questionnaires were not directly related to the neurophysiological measures. Two main limitations of this study regard the absence of information derived from self-report questionnaires and subjective feelings participants experienced during their experiments. In the future, it would be necessary to collect information on the individuals' subjective feelings in order to correlate them with EEG and autonomic measures. Moreover, no specific interaction effect was found for the gender variable, though our sample size should be increased – specifically for male participants – to generalize present results to a wider population. Lastly, this study did not take into consideration the comparison with people of different ages, although previous studies highlighted interesting positive attitudes toward SHSs so far by means of qualitative measures (Demiris et al., 2004; Meulendijk et al., 2011).

Nevertheless, future research could deepen these issues by directly comparing, for example, different developmental ages until elderly samples to investigate if (and how) the cognitive and emotional responses related to these complex systems change over time. Also, supplementary neuroscientific measures such as eye-tracking systems or functional near infrared spectroscopy

#### REFERENCES


(fNIRS) could be applied to explore the relation between ocular behavior and hemodynamic neural responses. Finally, explicit and subjective measures should be considered and interpreted together, to explore the interplay between covert and overt responses, as well as the role of individual factors, such as the technology's degree of familiarity, gender, and age digital divide, but also personality components, including locus of control and motivational factors.

# DATA AVAILABILITY STATEMENT

The datasets generated for this study are available on request to the corresponding author.

### ETHICS STATEMENT

The studies involving human participants were reviewed and approved by the Ethics Committee of the Department of Psychology of the Catholic University of the Sacred Heart. The patients/participants provided their written informed consent to participate in this study.

# AUTHOR CONTRIBUTIONS

MB and LA contributed to the conception and design of the study. FC and LA wrote the first draft of the manuscript. MB, FC, and LA contributed to the manuscript revision and read and approved the submitted version of the manuscript.

# FUNDING

This research received funding from the Catholic University of Milan, Milan, Italy, D1.1 2019.

#### ACKNOWLEDGMENTS

The authors kindly thank Monica Simone for her support in collecting the experimental data and the tech company Duemmegi S.r.l. (Milan, Italy) for their availability in affording the domotic show loft.

dimension. Neurosci. Lett. 303, 115–118. doi: 10.1016/s0304-3940(01)017 03-7




**Conflict of Interest:** The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Copyright © 2020 Angioletti, Cassioli and Balconi. This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) and the copyright owner(s) are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.

# Developing Design Solutions for Smart Homes Through User-Centered Scenarios

Mi Jeong Kim<sup>1</sup> \*, Myung Eun Cho1,2 \* and Han Jong Jun<sup>1</sup>

<sup>1</sup> School of Architecture, Hanyang University, Seoul, South Korea, <sup>2</sup> Construction Research Institute, Hanyang University, Seoul, South Korea

The success of smart homes is fundamentally dependent on their adoption and use by people in the context of daily life. This study investigated ways to control and adapt the technology to fulfill user daily needs, which are the active drivers of smart housing technology adoption. A framework of smart home services was developed by focusing on the practicability of each variable from the perspective of supporting user experience. By developing scenarios based on previous studies, we identified residents' behaviors and intentions regarding smart home technology and its use. Their issues were identified through the health problems and daily activities identified in the scenarios, and customized smart home services were developed for each resident based on technical solutions, space requirements, and design solutions. The main strength of this study is the adoption of user-centered methods to build a rich picture of the issues that affect households and the community related to the design, installation, and use of technology.

#### Edited by:

Seungkwang Shon, Dongshin University, South Korea

#### Reviewed by:

Rongrong Yu, Griffith University, Australia Bok Hee Yoo, University of Ulsan, South Korea

#### \*Correspondence:

Mi Jeong Kim mijeongkim@hanyang.ac.kr Myung Eun Cho mecho@hanyang.ac.kr

#### Specialty section:

This article was submitted to Environmental Psychology, a section of the journal Frontiers in Psychology

Received: 30 November 2019 Accepted: 12 February 2020 Published: 20 March 2020

#### Citation:

Kim MJ, Cho ME and Jun HJ (2020) Developing Design Solutions for Smart Homes Through User-Centered Scenarios. Front. Psychol. 11:335. doi: 10.3389/fpsyg.2020.00335 Keywords: smart home, user-centered scenario, design solution, framework for smart home service, context

# INTRODUCTION

The development of information and communication technologies (ICTs), Internet of things (IoT), big data, and artificial intelligence (AI) has started changing people's daily lives recently. Computing and information processing are spreading into daily life since these are increasingly being embedded in environments and artifacts invisibly. A new paradigm of human–computer interaction (HCI) is the integration of humans and humans, humans and objects, and objects and objects, and organically connecting them. This new technological paradigm is expected to cause significant changes in various fields, but it is predicted that the future information technology (IT) environment will be developed around the home (De Silva et al., 2012; Krishna and Verma, 2016; Borsekova et al., 2017). Noury et al. (2003) introduced the concept of a "health smart home" as a variation of smart homes with a focus on assistive technologies for the independence of the elderly and patients in housing (Noury et al., 2003). Many studies have developed techniques for specific groups of users, such as those with dementia, those vulnerable to falls, and those who would require emergency help. Numerous research projects have implemented a variety of prototypes of smart systems, which include sensors, algorithms, and intelligent devices (Das et al., 2002; Mihailidis et al., 2008; Krishna and Verma, 2016).

Existing smart home research has focused on technology development related to intelligent housing that can demonstrate new possibilities for the use of advanced technologies. These studies

initially focused on home automation and networking technologies that facilitate remote control of electrical, lighting, and heating appliances (Arunvivek et al., 2015). Intelligence has recently become augmented and pervasive (Ricquebourg et al., 2006). Current research on smart homes has emphasized collection of contextual information about the domestic environment and its residents and provision of customized, automated supports (Singh et al., 2014). These studies focus on technology adoption and emphasize the need to provide user-friendly interfaces but regard the user as a passive agent and are essentially not focused on the user perspective. In other words, technology adoption was not based on clear user-centered understanding. For example, a home telehealth service, which incorporates ICT into the medical industry, will save medical costs for seniors who need chronic disease and health care and ensure independent living. Users perceived it as potentially useful, but in practice, they often refuse biosignal measurements and daily life monitoring through various sensing systems, such as cameras (Peek et al., 2014; Cimperman et al., 2016). This phenomenon is due to the introduction of technology without an in-depth understanding of its users.

Recently, the necessity of conducting smart home research in a more user-centered manner has been suggested, on realizing that technology development cannot achieve substantial results in other IT fields without a user-centered vision. The overall success of smart homes is fundamentally dependent on people's adoption and use of this concept in the context of everyday life, regardless of the eventual form in which they adopt it. This study investigated ways to control and adapt the technology to fulfill users' daily needs, which are the active drivers of smart housing technology adoption. To this end, we developed a framework of smart home services that focuses on the practicability of each variable from the perspective of supporting user experience. To validate this framework, this study captures each variable's implications for smart home services through the proposed framework and proposed smart home services and solutions tailored to each individual based on the complex context of daily life. In particular, this study does not focus on smart technologies, but rather on the services in which they are installed and used. The analysis method used has been developed with an interest in determining the ways in which smart technologies can be used based on the user situation and needs.

#### RELATED WORKS

#### Development of Smart Homes

Berlo and Allen (1999) described a smart home as "a working environment which includes the technology to allow the devices and systems to be controlled automatically." Emphasis is placed on intelligent dwellings with automatic control, including for lighting, climate, appliances, and security systems, such as access control and alarm systems. As home networking has developed with the availability of high-speed internet technology, such as asymmetrical digital subscriber line technology (ADSL), the smart home concept has been expanded by installing sensors in objects used daily and by enabling interworking with mobile devices. Recently, Balta-Ozkan et al. (2013) defined a smart home as "a residence equipped with a high-tech network, sensors and devices, and features that can be remotely monitored, controlled, and provide services that respond to the needs of its inhabitants." The key to smart dwellings is the ability to automatically control dwelling facilities and devices from outside the dwellings. New technologies such as AI and the IoT can analyze the living patterns of residents and enable communication and information collection between smart devices, objects, and humans (Orwat et al., 2008; Arunvivek et al., 2015). Many of the new technologies that use various sensing systems, such as motion sensors and video cameras, are being developed to the extent that they can automatically support the user's contextual awareness without the need to directly manipulate devices (Mann et al., 2001; De Silva et al., 2012).

Research on smart homes has been conducted in various fields, but thus far, most of these are in engineering and technical sciences domain (Wilson et al., 2015). For these studies, the goal of smart homes is to improve the quality of life of residents through automated devices, to enable them to live a safe, healthy, comfortable life independently (Gracanin ˇ et al., 2011). For example, MIT AgeLab has developed a technology-based home service that integrates into everyday life to improve well-being and safety. It developed this service after evaluating residents' attitudes and needs related to various aspects, such as daily activities, social activities, mobility, safety, and nursing (Agelab, 2017). Further, a multidisciplinary team at the Georgia Institute of Technology, under its "Aware Home" project, constructed a three-story house to test and evaluate the engineering design of smart homes and identified users' habits and behavior models through footprint detection technology. It also proposed techniques for fostering lasting bonds and social exchange between family members (Cory et al., 1998). Marikyan et al. (2019) emphasize energy consumption management and healthcare needs of aging users in terms of the services and context-led aspects that smart homes provide (Flynn et al., 2016). By enabling residents to monitor and control their energy supply against demand, they propose a novel and profound solution that reduces energy use and promotes environmental sustainability (Balta-Ozkan et al., 2014; Bhati et al., 2017).

Health smart home provides next-generation medical care for seniors by enabling their family and carers to remotely monitor the health of seniors through technology (Orr et al., 2006). Pervasive computing applications can be useful for predicting falls based on changes in gait. Intelligent devices in the home, from cell phones to furniture, picture frames, kitchen utensils, and toilets, are used to motivate residents to manage their diet, take medications, or continue exercising (Hudson and Cohen, 2003). Moreover, telemedicine technologies that connect patients with clinicians to monitor physiological signals, such as heart rate, through wearable devices or devices attachable to clothing or skin, or to manage chronic diseases at home, are becoming increasingly common (National Research Council et al., 2004). Thus, the future of computing for homes lies in creating a healthy, intelligent, interactive living environment (Do and Jones, 2012). Innovations in technology should be used to improve individual

lives and develop human potential. A common concern of all age groups—not just the elderly or patients—is whether they would be able to live comfortably in their homes. Therefore, the research on smart homes needs to be extended by considering ways to improve the well-being of the middle-aged and younger age groups, thus moving beyond the present elderly- and patientoriented research.

# Users and Acceptance of Smart Home Technology

Technology developers and researchers claim that advanced, applied knowledge will make our lives more comfortable. Their purpose is to support the daily lives of residents through technologies, such as those for energy management, security, monitoring, and detecting incidents (Yu-Ju et al., 2002; Gracanin ˇ et al., 2011). Despite this broad range of potential and assumed benefits of technology adoption, if we focus only on technological features, the technology can disappear before they are even incorporated into our lives (Cook, 2012). Thus, smart home research requires a sustained, systematic understanding of users because adopting smart technologies and incorporating these in everyday life are important for the success of smart homes (Haines et al., 2007). For the elderly with chronic or health disabilities, home telehealth services are expected to improve the quality of life in the home, reduce medical expenses, and provide independent living (Onor et al., 2008; Choi et al., 2018). These services include access to personal health information or records, remote patient monitoring, and chronic disease management. However, the elderly, the target population of smart technologies, do not understand new IT-based solutions and concepts and face special challenges in using these solutions (Cimperman et al., 2013).

In particular, home monitoring technologies are designed to support safe and independent living at home (Mihailidis et al., 2008). Monitoring technologies, such as systems for emergency response, fall detection, and health and physiological monitoring, provide a customized residential environment that tracks and records autonomously. However, research indicates that many users do not accept these technologies and have a high rate of device abandonment (Lund and Nygård, 2003). Their nonacceptance and non-usage may be regarded as the failure of smart home designs and operational procedures (Fisk, 1998). Therefore, for the successful realization of smart homes, it is critical to understand the factors that potential users consider important and necessary, and then decide on acceptable technologies and functions, rather than being concerned with technological performance in isolation. Courtney (2008) stated that our society sometimes neglects or ignores privacy as it stresses the need for technology. Demiris (2004) raised concerns regarding the use of technology in homes, such as privacy violations of older people, anxiety regarding the use of unfamiliar technology, and unnecessary surveillance. In particular, recent advances in home telehealth services include the transfer, management, and analysis of personal health data, which leads to concerns regarding security problems (Cimperman et al., 2016). Similar to other types of technology, smart home technology is only effective if the user accepts it and integrates it into daily life (Cimperman et al., 2013). Understanding users who are willing to adopt IT is important in IT design and implementation. Thus, this study attempts to answer the following questions: Who are the potential users of smart homes? What are the smart home technologies that these users need in their daily lives?

# A FRAMEWORK FOR CONSTRUCTING SMART HOMES SERVICES

This study proposed a framework to provide a structured way of understanding smart home services. The usefulness of frameworks is described in terms of three concerns: space, technology, and users. **Figure 1** shows the framework for configuring smart home services. Unlike research that has focused on technical issues, the framework seeks to identify and integrate cross-cutting relationships based on understandings of smart homes and users. In particular, the framework's focus is on multimodal interactions between users and smart homes that integrate space and technology. The space dimension focuses on HCI aspects, including user experience (UX), whereas the technology dimension emphasizes users' perception and acceptance of technology. Intelligent computing and architecture are integrated to create new responsive and interactive environments. This environment is constantly connected to the network, where residents can interact with neighbors in the community to which they belong, and provides various residential services that are necessary and appropriate for residents. The proposed framework will help designers, architects, engineers, and researchers alike to explore and develop smart homes in a more expanded, integrated perspective. The framework of health smart home services, established on a framework constructed by Kim et al. (2014), is extended from a user and multidimensional perspective.

#### Users

Smart homes must provide information and services tailored to the user's situation. This is because residents in homes live differently, in accordance with their own characteristics and physical ability. Basically, people desire to gain increased comfort and convenience through the smart home, but the degree and method of realizing this goal can vary from individual to individual. This dimension has three categories related to user preference and experience: characteristics, lives, and physical ability. It is important to understand the characteristics and health of the user. For example, when a designer develops smart homes to assist individuals with dementia, the user's needs need to be understood in detail not from the designer's perspective, but from that of those individuals (Orpwood et al., 2005). To design a smart home that best meets users' needs, we need to understand their lives in considerable depth. Crabtree and Rodden (2004) argued that smart home design might be informed by attending to the routines of the home; thus, the routines of the inhabitants' everyday lives should be explicated. Groups of users will differ in terms of their wants, needs, and use of technology, and may require different design solutions. According to research results

related to user characteristics, the younger the age, the higher the education level, and the higher the income, the more the use of technology, such as the internet and smartphone. Further, the greater the desire to continue living in the place where they live and the higher the intimacy with technologies, the greater the acceptance of smart housing technology (Peek et al., 2014).

The younger generation generally has more exposure to, and experience in using, technology than the older generation. Mihailidis et al. (2008) compared opinions and differences between generations to identify differences in technology preferences and acceptance among users. The higher the level of technical anxiety, the more hesitant the use of smart home technology; computer anxiety was found to inhibit the use of smart home technology by the elderly in particular (Holden and Karsh, 2010). The use of technology is related to issues such as security, privacy, and trust as well as practical and ergonomic concerns with user-friendliness. These issues present critical design challenges related to the interaction between the user and the smart home. People do not live alone, but rather, they share the same network connected to the community space and coexist and cohabit with others at the same time. Therefore, the concept of smart home service extends to the community and is required not only in personal life but also in community life. In this framework, users control and regulate technology more efficiently and become masters of their lives and smart homes.

#### Space: Residential Setting

Domestic settings are different from work-oriented settings; further, smart home settings are unlikely to be purpose-built. Therefore, it is essential to understand housing in terms of activities, quality of life, and subjective happiness in the evolutionary context rather than in the contexts of work and efficiency. This dimension has four categories: appliances, household objects and furniture, sensor and actuators, and infrastructure. Rodden and Benford (2003) argued that home settings should be understood through diverse approaches that are concerned with functional forms of household artifacts and devices as well as interactive environments and technological infrastructures. Emphasizing the nature of space integrated with technology, Do proposed that a smart living environment is interactive and has sensors and actuators as part of the building, to support living in the forms of things that think, spaces that sense, and places that play (Do and Jones, 2012). To understand the context of home, different methods need to be adopted for collecting diverse aspects of the context and these aspects should be combined to create the complete context for the domestic dwelling. For example, ethnographic studies can highlight domestic routines, whereas longitudinal studies can identify the ways in which technologies are used (Crabtree et al., 2004). Ethnographic techniques can be developed to identify how people actually live with technology: the social, cultural and historical context; the participants' experience of aging and ill-health; factors that matter to them; technologies in their home and life; and their capabilities to operate and interpret technologies.

Interactive technologies need to be manifested within the residential settings. Information appliances, such as internet fridges, perform a single function or cluster of functions with wireless networking (Gaver and Martin, 2000). Among interactive household objects, a reminder display creates a context for remembering, thus tempering the elderly's memory loss, and a notice board can be combined with communication capabilities (Mynatt et al., 2000). The medication alarm system not only informs the time of taking the medication but also collects data on medication. Pressure and motion sensors tagged on slippers track weight, steps taken, or falls and send these

data to medical staff (Hindus et al., 2001). Some furniture could act as interactive settings through sensors that detect actions, such as the DiamondTouch table and augmented cupboard (Crabtree and Rodden, 2004). A software platform can be applied to furniture or mirrors to provide a radio and weather and health information, or to automatically adjust the illumination according to the surrounding environment through sensors (Sponselee et al., 2008). Through such augmented appliances, objects, and furniture, residential settings can be transformed into interactive environments that effectively assist residents to live healthily at home. Reliable control methods are essential for efficiency in system operation, and thus, the design of HCI is a crucial component of intelligent settings in daily life. This includes designing computing diffused into homes to be unobtrusive, intuitive, and reliable to act in expected ways.

#### Smart Technology

It is the age of ubiquitous and pervasive computing. The use of ICT is essential for smart dwellings because it changes daily lives in residences in meaningful, fundamental ways. ICT distributed in rooms, devices, and systems (i.e., lighting, heating, and ventilation) is aware of people's activities and needs. This dimension has three categories: ubiquitous computing, AI, and IoT. IoT connects sensors, devices, actuators, radio frequency identification tags, laptops, and mobile phones to share network resources in conjunction with each other (Krishna and Verma, 2016). The technology helps in energy management systems and supports access to devices and remote monitoring of embedded devices (Chatzigiannakis et al., 2015; Li et al., 2016). Advanced AI not only collects occupants' data but also applies visual and sensory-based tracking systems to identify them based on facial expressions and emotion recognition (Mano et al., 2016). Visualbased tracking systems, such as cameras, can monitor the status of occupants in the smart home. An AI-based IoT framework provides a continuous monitoring system of living patterns of residents through various sensors attached to the human body and in the environment to avoid health hazards and provide customized health care services accordingly (Mann et al., 2001).

Perceptual capability that is aware of the inhabitants and their needs is emphasized to provide customized and situated aids, and embedded intelligent components are used for context awareness. To establish intelligent infrastructures, various sensors need to be embedded in the fabric of the environments, which support drawing inferences from contextual information. Mihailidis et al. (2008) proposed home monitoring systems that are targeted toward specific home support goals: personal emergency response systems; automated fall detection; activity of daily living monitoring; environmental controls, such as lights, heating, and ventilation; and health monitoring, such as heart rate monitoring and detection of sudden changes in a person's lifestyle patterns that may indicate changes in health, using sensors located in the environment (Mihailidis et al., 2008). Many studies have investigated monitoring of residents' daily activities and physiological health conditions and described the state of the art of sensors, algorithms, and tracking devices in smart systems (Noury et al., 2003; Orwat et al., 2008; Marikyan et al., 2019). The smart technology framework monitors user mobility patterns and ensures a high level of functionality that preserves privacy and complementation of user data.

# RESEARCH METHODOLOGY

In this study, technologies and their use are considered in the context of the spaces of the home and community, and the networks of family and social relations linked to these technologies. We developed scenarios based on the results of the previous studies to build a rich picture of how people actually live in smart homes (Cho and Kim, 2017, 2018; Cho et al., 2018). Smart home design is an interrelationship between different disciplines. The importance of collaboration between specialty fields for solving the problems of existing smart home technologies and applying these solutions is becoming apparent. To find ways to effectively incorporate a comprehensive design for smart homes, 12 experts from various fields, such as IT developers, researchers, architectural designers, and employees from the IT services industry, the medical industry, and business enterprises, were invited to analyze and comment on scenarios. The scenario is fictional but based on the actual ethnographic account of the problems that people experience in residential dwellings. We encapsulated typical features and behavior of homogeneous subsegments of the target population. In designing the user experience, the most important trigger for effective decision-making is in-depth knowledge of users. The target groups of this study are the retired elderly in their 60s and 70s who live in their homes, those in their 40s and 50s who are actively socializing with their children, and those in their 30s who are growing rapidly. Three questions were provided for reflection: (1) What is important and what are the problems for the residents in each situation presented in the scenario? (2) How can space or technology improve their lives? (3) What are the smart home services that should be provided for residents in each situation?

#### Scenarios

By developing scenarios, we identified residents' behaviors and intentions as regards smart home technology and use. In particular, the scenarios focused on predicting the skills that people of different ages would need and use based on their health, work, and daily life experiences. A technology is always developed around standard users, and hence, the situation may not be suitable for other users. Eisma et al. (2004) suggested that developing technologies need to be built for diverse user groups and long-lasting relationships with them should be established. This article reports on an extensive study to engage with the cultural and contextual issues surrounding the use of new technologies by diverse users. The technical experiences of users and their physical ability and health status are important variables to consider in developing a smart home service. In this regard, previous studies (Cho and Kim, 2017, 2018; Cho et al., 2018) provide evidence that users' needs and expectations differ according to age. Hence, as shown in **Table 1**, the scenarios we consider have three components: personal characteristics, physical ability and health condition, and daily home life and activities. Personal characteristics include age

#### TABLE 1 | Six user scenarios.

fpsyg-11-00335 March 19, 2020 Time: 17:11 # 6


and gender. Physical ability and health conditions as well as daily home life and activities are based on the activities of daily living (ADLs), such as basic activities and instrumental activities, which help in understanding the physical health and daily activities of the user and in providing appropriate information and services. The first two scenarios in **Table 1**

consider a man and a woman, both aged 65 years. The next two involve a man, aged 52 years, and a woman, aged 45 years; both have children. The last two scenarios consider a man and woman aged 31 years and 38 years, respectively. These three types of scenarios are used to represent the age groups of 60– 70, 40–50, and 30–40 years. However, further validation needs to be conducted for a broader population to generalize this study's results.

# Proposed Smart Home Service According to Context-Based Solutions for Scenarios

The residents' living problems were identified through their health problems, activities, and daily routines presented in the scenarios, and customized smart home services needed for each resident were developed based on technical solutions, spaces, and design solutions.

#### Identifying Problems Through Health Status and Daily Life Analysis

On analyzing the first scenario, that of elderly people in their 60s and 70s, experts pointed out the health problems caused by physical aging and the difficulties of performing household work, such as basic cleaning and washing, and basic purchasing activities such as for food items and daily necessities. They expressed concern about the elderly individuals' lack of activity, the disconnection of their social relationships, and their difficulty in interacting with their children. The difficulty of attending hospital for regular medical care was also highlighted as a problem. It was also found that elderly people lack experience in using modern technology and their fear of devices could make technology adaptation difficult.

By analyzing the second scenario, which considered middleaged people in their 40s and 50s, experts identified that in terms of health, middle-aged people were coping with stagnation, lack of strength, fatigue, and low motivation. It was perceived as a problem that this group lacked the time required to solve the problems of lack of rest and regular exercise and inability to have individual time and to engage in hobbies and community activities. The experts also identified that middle-aged individuals found it difficult to take good care of their health, since they led busy lives, and that they also found it difficult to communicate with their adolescent children.

In the third scenario, related to unmarried singles in their 30s, experts recognized mental health issues, such as loneliness and stress, as critical problems. In addition, irregular activities, such as skipping meals and inadequate sleep, were thought to be difficult factors in daily life. They pointed out that singles found cooking and cleaning bothersome and regard their home as a sleeping space. The problem is that their loneliness can increased because they do not know, or interact with, others in the neighborhood. Further, singles do not engage in enjoyable sports and hobbies. The experts identified issues and needs in each scenario, as shown in **Table 2**.

#### Solution of Technologies and Spaces for Smart Homes

To develop solutions, we considered the technical and spatial aspects of the problems. We summarized the technical parts into functions and devices and divided the spatial parts into unit households and communities within the complex (see **Table 3** for details).

For the elderly in their 60s and 70s, the technical solution most mentioned was the need for continuous monitoring through sensors to identify everyday patterns and cope with crisis situations. In addition, technology acceptance was a problem, although they need technical help for managing chronic disease and regular treatment. The aspect of most interest for proposing spatial solutions was social exchange, and therefore, various community spaces within the complex were suggested, such as meeting spaces, exercise facilities, and paths for outdoor walks.

For those in their 40s and 50s, who are unable to take good care of themselves owing to a busy schedule, it is useful to continuously measure blood pressure and sugar levels and pulse rate as a technical solution, collect data, and provide health counseling, management, and exercise guidance. A smart home



X Family interaction X Social implications

X Re-education about new technology

X Leisure and exercise X Family connectivity

#### TABLE 3 | Technical and spatial solutions.

fpsyg-11-00335 March 19, 2020 Time: 17:11 # 8



training room that has a spatially assisted exercise console was proposed. In addition, to build harmonious family relationships, a smart family room that combines the necessities of space and technical solutions was proposed. This is a space equipped with smart technologies, such as context awareness and augmented intelligence, and is a space specifically set aside for family-friendly programs, which expands the existing family room concept. For rest and self-time, a rest area, such as a sauna, a library, and a wine bar in the complex or a family rest area, is proposed.

In the case of singles in their 30s, when they leave their workplaces late, they often cannot clean or perform housework, and hence, through technology, it is necessary to monitor and control the house from other places. It was suggested that they can easily interact with each other through apps by forming social networks with others of similar ages in complexes and meet when needed for engaging in shopping, hobbies, and sports together. Spatial solutions are also required for a variety of activity spaces to share social activities, walks, hobbies, etc., and promote social life. For this generation, it is difficult to solve problems simply by providing space, and thus, it is necessary to develop various programs that can activate meetings and continue social exchanges, and various meeting apps that induce participation in the complex. This approach would resolve social disconnection and isolation of these individuals.

For people of all ages, the space for health measures, proper treatment, and customized exercise based on such treatment must be provided. Therefore, it is desirable to establish a professional nursing space in the complex to enable health measurement and data management for each individual and to provide simple rehabilitation treatments or exercise guidance in connection with community hospitals.

#### Customized Smart Home Services

Various smart home services can be presented based on problems and solutions identified in the three scenarios. In this study, smart home services are classified into five categories: basic daily life support, health care and management services, environment services, psychological well-being services, and social relationship enhancement services.

Basic daily life support is a service that helps residents with basic daily activities, such as household chores, shopping, and meal preparation. Specifically, this category includes services to perform household chores, such as laundry, washing dishes, and ironing; to enable shopping for, and delivery of, household essentials; and to prepare meals and side dishes to suit occupants' needs. Devices, such as network information appliances and AI robots, can be used for automated data collection and storage of records of purchasing experiences. Based on user's purchasing patterns, information and services (e.g., regular store automatic purchase) can be provided.

Health care and management services include hospitalbased disease management, physiological measurement, health counseling, and exercise guidance services. Various sensors or devices in the house check and manage occupants' health status of blood pressure and diabetes. Biomedical information collected along with health measurements through sensors or measuring instruments is provided in connection with community hospitals, and video-based medical consultations

#### TABLE 4 | Customized smart home services.

fpsyg-11-00335 March 19, 2020 Time: 17:11 # 9


The color shades show more important services for each age group.

and medicine prescriptions are provided. It also provides health counseling based on residents' health information and appropriate exercise guidance.

Environment services include security and safety, energy management, and cleaning services. It is provided to prevent, or cope with, safety accidents, such as theft and fire occurring in a house, or to reduce energy consumption, such as for lighting and heating. Cleaning services, such as for washing dishes, managing laundry, and cleaning the house, would be especially useful for seniors and singles.

Psychological well-being services are important for residents' self-esteem, development, and happiness. Smart devices and internet education, technology installation and management, and smart entertainment services are included. The technology installation and management service is aimed at overcoming the frustration caused by inability to use technology and at improving usability along with technical education. It is related to establishing and installing the initial environment. It is a service that installs it for those who are not familiar with the new technology and helps in case of malfunction or failure. This can reduce the burden perceived on using unfamiliar technologies. Residents' favorite entertainment such as videos, music, and games are managed, and customized information is provided in conjunction with AI.

Social relation enhancement services include those for communication, social connection, exercise, and hobbies in the community. Communication services are available not only in smartphones but also in intelligent objects, appliances, and walls with networks and platforms installed. Various channels and convenient interfaces promote interaction with family and friends at other locations. It provides information related to community facilities frequently used by residents in the area, supports various social activities, sports, and hobbies of residents through public spaces within the complex, and develops applications for program development, communication, and reservation of meeting space. The community program aims to eliminate the negative feelings of loneliness and to encourage residents to experience the happiness of being together by letting them participate and actively communicate in various fields, such as watching movies, reading, walking, and biking.

The services proposed by the smart home can be extended to various domains. Depending on residents' particular situation, some services may be more important than others. Proposed smart home services are shown in **Table 4**. For the elderly, basic daily life support, health care and management, and social relation enhancement services are more useful than other services. Single individuals would find psychological wellbeing and social relation enhancement services more useful than others, whereas middle-aged people would need various services in all areas.

#### DISCUSSION AND CONCLUSION

The purpose of this study is to develop and apply smart home services based on user-centered understanding for more practical and effective smart home implementation. The main strength of this study is the use of qualitative, user-centered methods to build a rich picture of the issues related to the design, installation, and use of technology that affect

households and the community. This approach allowed us to develop detailed ideas and proposals to improve the daily lives of residents. Another strength of our study is its interdisciplinary nature—we incorporate the perspectives of those from various fields, including IT developers, researchers, architectural designers, and employees from the IT services industry, the medical industry, and business enterprises. Their diverse backgrounds allowed us to combine practical experience with theoretical approaches.

#### Prospective Users and Challenges

One of the main goals of the early smart homes was to provide assistive services for people with disabilities and for the elderly. Recent trends in smart home research have highlighted healthcare services, and thus, activities and health-related assistance have become the most important goals of smart homes. However, the boundaries of smart home users and services need to be expanded. The dissemination of smart homes may be limited if they do not focus on actual needs. Smart technology is not exclusive, and innovations in technology should be used to improve individual lives and develop human potential, rather than being limited to the elderly or to patients. It is a common concern for people of all ages who want to live comfortably in their homes. The development of modular, affordable smart home technologies enables their incorporation into existing ones as well as newly built homes. The number of potential users will grow and may be women, men, and children of various generations living in homes. The challenge should be to gather data on a wider variety of residents and to be interested in their needs and their use of technology.

### Customization and Adaptation

The lives of residents in homes are not as repetitive and predictive as designers believe. People vary, and existing domestic environments are organic and dynamic. Technology may not be used in the way designers intended. As a result, it should be recognized that there is a need to consider the different situations of residents and try to solve the problem from various directions. It is desirable to focus on the benefits that technical assistance would provide and to identify the use and placement of appropriate technologies in the context of day-to-day life for a healthy and happy home. Tailored solutions for residents need to be provided in smart homes. The way in which the elderly, middle-aged, and singles live, and their state of health are inevitably different. Thus, the technologies, spaces, and services they need are very different. Different groups of users may require different design solutions in terms of aspects such as households, generations, and cultures. The important thing is to determine their needs, and then apply and adapt the demanded and preferred technologies for everyday life. There should be no social barriers to the adoption of smart homes, such as obstacles to providing effective, tailored services. Concerns regarding loss of control, reliability, privacy, trust, and irrelevance often make it difficult for individuals to accept smart home technology. We should understand the environment in which users can adopt technology and investigate whether and how smart home technologies may be effectively incorporated into the domestic context. The solution needs to be scalable, sustainable, and sociotechnical.

#### Social Interaction and Support

Most services are applicable only to single smart homes, and sometimes to single rooms. Some research projects are devoted to location detection and do not implement any practical service for residents. We consider that if these limited systems with a few features are used, the smart home dream we imagine will not materialize. Without proper services and utilities, their widespread utilization cannot be achieved. The service area of smart homes should be expanded to satisfy residents. It is necessary to provide various services in connection with complexes and communities where residents live. Similar to the concept of telehospital/telemedicine service in conjunction with local hospitals, more new service networks might emerge that will connect homes for information sharing. The network serves as a platform for easy home access to services that are frequently used by residents, such as libraries, sports facilities, and welfare centers in communities. Future smart homes will promote the integration of all possible services into traditional homes. These homes will provide almost all the essential services, such as communication, medical, energy, public facilities, entertainment, and security services. In this study, we proposed a service that can be used by extending the concept to the community, but additional research is required to make these services costeffective, efficient, and acceptable.

#### Knowledge Sharing and Collaboration

The research on smart homes constitutes an interdisciplinary domain. The architecture of a smart home depends on other branches, such as technologies, spaces, and services. Smart homes benefit from improvement and diffusion through the integration of these sectors. While research on smart homes is typically conducted in the engineering, technology science, and design domains, there is increasing interest in various sectors ranging from healthcare, services, and economics to energy. In this study, we identified the need for knowledge sharing and collaboration in related fields. Many other methods are being developed in individual projects or research environments, and hence, smart home residential service chains lack effective integration and information sharing. It is time for an integrated approach to smart homes that focuses on users.

#### DATA AVAILABILITY STATEMENT

Publicly available datasets were analyzed in this study. This data can be found here: http://www.khousing.or.kr/.

#### AUTHOR CONTRIBUTIONS

MK and MC composed this study, designed the framework and the methodology part of the research, and completed the qualitative analysis. MK and HJ provided supervision throughout the research.

# FUNDING

fpsyg-11-00335 March 19, 2020 Time: 17:11 # 11

This work was supported by the National Research Foundation of Korea (NRF) grant funded by the Korea Government (MSIT)

#### REFERENCES


(NRF-2019R1A2C1087344). This research was supported by the Basic Science Research Program through the National Research Foundation of Korea (NRF) funded by the Ministry of Education (2019R1I1A1A01043066).



**Conflict of Interest:** The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Copyright © 2020 Kim, Cho and Jun. This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) and the copyright owner(s) are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.

# Spatial Color Efficacy in Perceived Luxury and Preference to Stay: An Eye-Tracking Study of Retail Interior Environment

Ji Young Cho<sup>1</sup> \* and Joori Suh<sup>2</sup>

<sup>1</sup> Department of Housing and Interior Design, Kyung Hee University, Seoul, South Korea, <sup>2</sup> School of Architecture and Interior Design, University of Cincinnati, Cincinnati, OH, United States

Color is a significant interior element with the power to influence emotions and behaviors in a particular environment. Numerous studies have investigated the impact of a single color on emotion; however, the collective emotional, cognitive, and behavioral effect created by combinations of colors applied to a space has not been thoroughly investigated. In this study involving both a survey as well as eye-tracking technology, we explored shaping the concept of spatial color efficacy by examining different applications of the same color combination in a space to determine whether they may cause different emotional responses, thereby impacting viewers' perception of luxury and intention to stay. A total of 26 interior design students at a university in Korea participated in the study. An environment simulating a hypothetical retail store was developed using a 3D rendering program, and six variations of spatial applications were created for each high luxury color combination and low luxury color combination to be used as stimuli. While viewing the images, participants were asked to identify which image looked most luxurious and in which space would they most want to stay. Results show the following: (a) the same color combination, if applied differently in a physical environment, can create different emotional responses, thereby affecting perceived luxury and preference to stay; (b) even a low luxury color combination can enhance perceived luxury and preference to stay depending on the spatial application; (c) gaze bias exists when selecting the most luxurious space and stating preference to stay as shown in the high correlation between dwell time and choice; in addition, differences in emotional response across images were also observed in the variations of pupil sizes measured during viewing various applications; (d) dark colors used in large amounts of surface were perceived as more luxurious than light colors when the same color combination was applied; and (e) appropriate contrast among colors was more influential in preference to stay than extreme or minimal contrast. Results expand the understanding of human behavior in relation to spatial color efficacy based on the spatial color combination and potential decision-making process in a retail setting.

Keywords: color, interior design, luxury, preference, retail

#### Edited by:

Mi Jeong Kim, Hanyang University, South Korea

#### Reviewed by:

So-Yeon Yoon, Cornell University, United States Graciela Lucía Tonello, Universidad Nacional de Tucumán, Argentina

> \*Correspondence: Ji Young Cho jcho@khu.ac.kr

#### Specialty section:

This article was submitted to Environmental Psychology, a section of the journal Frontiers in Psychology

Received: 30 November 2019 Accepted: 07 February 2020 Published: 31 March 2020

#### Citation:

Cho JY and Suh J (2020) Spatial Color Efficacy in Perceived Luxury and Preference to Stay: An Eye-Tracking Study of Retail Interior Environment. Front. Psychol. 11:296. doi: 10.3389/fpsyg.2020.00296

# INTRODUCTION

fpsyg-11-00296 March 28, 2020 Time: 18:59 # 2

Color is a significant interior element with the power to influence emotions, such as pleasure and arousal; cognition, such as meaning making; and behaviors, such as intention to approach or avoid, in a particular environment (Spence et al., 2014). A retail store environment is a significant marketing point to deliver a brand's intended image and message (Cho and Lee, 2017), and the perception of luxury is known to yield pleasure and positive emotion (Yani-de-Soriano and Foxall, 2006). Although numerous studies on the impact of color on emotion and behavior have been conducted, most studies have involved a single color, and few researchers have attended to the effect of the combination of several colors (Deng et al., 2010). Moreover, the question of human emotional, cognitive, and behavioral responses to variations of spatial application, such as on walls, floors, or furniture, has not been clearly answered. If applied differently in a space, will a color combination yield different emotional responses? What spatial application of color results in a high level of perceived luxury (PL) or preference to stay (PS)? The term efficacy refers to "the power to produce a desired result or effect" (Efficacy, 2020). In this research, we define the term spatial color efficacy as "the power of the ambiance created by a threedimensional space, where a certain color combination is applied, to produce a desired emotional, cognitive, and behavioral effect." Spatial color efficacy could range from highly effective to not at all effective.

The contemporary culture of emphasizing individual identity has resulted in the tendency to prefer higher-quality products that express distinctiveness, elitism, and uniqueness. The concept of luxury has been studied in a wide range of disciplines, including marketing, branding, consumer science, and design. Although the meaning of luxury may vary per individual and culture because it is multidimensional, literature shows that PL consists of the following five dimensions: conspicuousness, uniqueness, quality, hedonism, and extended self (Vigneron and Johnson, 2004). Conspicuousness is associated with wealth and elitism; uniqueness refers to exclusiveness and rarity; quality means well-crafted and superior; hedonism refers to glamorous and stunning; and extended self means being successful and powerful (Vigneron and Johnson, 2004, p. 487). In hedonic dimensions, Dubois and Laurent (1994) stated that the key elements of luxury are the pleasure, satisfaction, and excitement experienced when purchasing luxurious products.

Recently, the concept of luxury has migrated from the traditional understanding of exclusivity, the privilege of royalty, and inaccessibility to the contemporary notion of an accessible medium for self-expression, reflecting a "democratic right to happiness" (Chandon et al., 2016, p. 299). In view of the growth of the culture of contemporary millennials and their penchant for multidimensional experiences, opportunities for self-exploration, and novelty, a deeper understanding of luxury experience in terms of emotional stimulation and sensorial discoveries becomes crucial (de Kerviler and Rodriguez, 2019). When a luxury brand provides experience with emotional, sensorial, and intellectual stimulation, such experience broadens one's self-concept and sense of self (de Kerviler and Rodriguez, 2019). The implication is that the concept of luxury has been expanded to customers' longterm emotional well-being. Previous studies have shown that the feeling of luxury can draw positive emotion, such as pleasure and arousal; therefore, designing a retail space that expresses luxury would benefit retailers in that creating such a space for customers would make them feel inclined to stay (Cho and Lee, 2017).

In fact, while online shopping continues to grow, customers who patronize brick-and-mortar stores still look for meaningful emotional experiences that cannot be provided online (Kim and Lee, 2016). Kotler (1973) argued in his seminal article "Atmospherics as a Marketing Tool" that sensory qualities of the retail space influence buyers' perception of the atmospheric quality of the space, and the affective state of buyers may eventually impact purchase probability. Offline stores are no longer merely spaces for selling products; instead offline retailers have discovered the need for strategies to provide sensuous and experiential spaces where customers desire to stay longer. The physical environment of a store affects the emotional experience of consumers and in turn influences their behavior and attitude (Donovan and Rossiter, 1982; Middlestadt, 1990; Kim and Lee, 2016). Important in causing emotional, cognitive, and behavioral responses, a physical environment where a service takes place is called a servicescape, which is "a composite of three dimensions: ambient conditions; spatial layout and functionality; and signs, symbols, and artifacts" (Bitner, 1992, p. 65). With the surge in the development of interactive art, highly visual digital culture, and high-end technology, designers have become responsible for thoughtful visual control in their proposal of emotional and sensorial stimuli in spatial design.

When entering a space, people make emotional and cognitive evaluations based on the information they receive from the physical spatial environment (Rye et al., 2016). Whether the emotional or cognitive state occurs first is debatable and unresolved, with many researchers arguing the two perspectives (Lin, 2004). In an extensive review of the cognition–emotion debate, Lin (2004) concluded that in evaluating a physical environment, such as a servicescape, cognitive processing in organizing the perceptual image of the environment occurs before affective processing, followed by the other cognitive processing, which is a more concrete evaluation of the environment. Finally, behavioral response, such as approach or avoidance, proceeds. This implies that sensory stimuli in a retail store are first perceived by a customer and affect her or his emotional state of pleasure–arousal. The emotional state influences the customer's cognitive evaluation of the environment, that is, their perception of friendliness, luxury, or PS, which can lead to the customer's behavioral responses of approach or avoidance and make a purchase. Typically, an interior physical environment is understood as structural interior components, such as ceiling, floor, walls, columns, or stairs; and non-structural spatial elements, such as color, materials, light fixtures, and decorative items that shape the spatial image (Cho and Suh, 2020). One of the main sensory channels used by humans is sight; thus, color, size, and shape are the main visual dimensions of an atmosphere (Kotler, 1973). Among these, color is the most expressive in its properties, and the immediate nature of its delivery of information accounts for the perception of color

playing an important role in retail stores, shaping the image of the overall interior space, impression, and meaning (Söker, 2009; Kim and Kim, 2017). Color creates psychological as well as physiological effects, causing various emotional feelings as well as value judgments and intention to buy (Baker et al., 2002; Ou et al., 2004). Many researchers who have focused on the atmospheric aspects of color in relation to the enhancement of the shopping experience have tried to draw broad conclusions about the effect of a single color. To illustrate, the majority of studies on a single hue have dealt with the impact of colors with contrasting color temperature (e.g., cool vs. warm) or those with different wavelengths (e.g., long vs. short) on arousal or pleasure; but because the interior space where people reside is not typically monochromatic, understanding the impact of color combination on emotion requires additional attention.

Studies using color combination are limited and still in their early stages. Color Image Scale by Japanese psychologist, Kobayashi (1990), includes an extensive set of three-color combinations associated with certain adjectives, such as calm or vivid. Kobayashi provided 1,170 three-color combinations associated with 180 adjectives based on surveys and questionnaire, but the combinations are provided in palettes and the detailed research procedures and findings were not reported. The research of Yoon and Wise (2014) about the affective experience of color combination based on the color image scale is meaningful in that they used multiple combinations of color as visual stimuli applied to a threedimensional physical environment. In a study of the impact of color on the perception of store luxury, emotions, and store preference, Cho and Lee (2017) found that color combinations perceived as high luxury (HL) impact pleasure, arousal, and store preference more than those perceived as low luxury (LL). The reason for the dearth of studies on color combination may be the result of the complexity involved in selecting appropriate colors and the difficulty in producing visual stimuli to test the impact of color on emotion by controlling other attributes.

In assessing the human response to environmental stimuli, pleasure and arousal have been the two significant emotional dimensions believed to influence behavior (Baker et al., 2002). In order to understand how pleasure and arousal change depending on spatial applications of a color combination, eye-tracking technology can be an effective tool to measure eye movements like fixations (stops) and saccades (jumps or moves). Eye-tracking technology provides researchers with objective data on the viewer's subjective emotions and cognitive processes occurring in the brain, facilitating understanding of human gaze behavior. The eye-tracking technique allows a researcher to identify where and how long a person looks in the targeted area (dwell time) as well as the time-based sequence of eye movement from one location to another. Previous studies on gaze behavior and visual attention to stimuli have provided useful insights into understanding the relationship between viewers' tendency to look at certain stimuli and their preferences. In eye-tracking research, longer fixation in a particular area of interest (AOI) is typically considered indicative of greater interest and engagement (e.g., Just and Carpenter, 1976; Poole et al., 2005). AOI can be created "a priori or post hoc as geometric or free-form shapes around products, items, or any other section of the image the researcher designers to analyze" (Huddleston et al., 2015, p. 568). Dwell time, also called visit duration, is defined as the "sum (all fixations and saccades within an AOI for all selected participants)/number of selected participants" (SensoMotoricInstruments, 2017, p. 243). Total visit duration, which is the same as dwell time, can be interpreted as a measure of "cognitive processing (thought) through attention" (Huddleston et al., 2015, p. 568).

Eye-tracking data can be used to interpret the existence of gaze bias in a given task. The gaze bias effect refers to the tendency to look longer at certain preferred stimuli, and an individual's gaze behavior is known to be closely related to a viewer's preference and decision making (Shimojo et al., 2003; Saito et al., 2017). Gaze bias is also involved in viewing three-dimensional spatial visual stimuli and spatial decision making (Wiener et al., 2012). Meanwhile, pupillometric data reflect the brain's cognitive and emotional processes (Granholm and Steinhauer, 2004), especially the arousal dimension. The diameter of the pupillary aperture of the eye tends to increase in an arousal state and decrease in an unpleasant state (Hess and Polt, 1960). Some more recent studies reported that pupil size increases in emotionally arousing materials irrespective of hedonic liking (Bradley et al., 2017). Likewise, eye-tracking technology may provide useful information about subtle differences in emotional responses during a spatial search by demonstrating how eye movement and pupil size are influenced by spatial visual stimuli where color combinations are used. In addition, because two factors influence visual attention—the top–down factor (individual traits) and the bottom–up factor (physical characteristics of stimuli) (Wedel and Pieters, 2008)—the identification of participants with similar demographic characteristics is necessary.

In summary, available literature suggests the following:

First, previous researchers investigating the role of colors on emotion have mainly used a single color, not color combinations. Only a small number of them have investigated the impact of color combinations on emotion or preference; therefore, more studies using color combinations are necessary. Second, an understanding of the way one color combination may change emotional and cognitive response when applied in a space is needed. Because many interior spaces one encounters and uses consist of several colors, examining users' responses to color combinations is worthwhile. Third, the impact of a single color combination applied to a physical environment on the perception of luxury has rarely been studied, and thus requires attention. With one color combination, whether the perception of luxury or intent to stay differs depending on where the color is applied, such as floor, wall, or furniture, is not well known. Fourth, using eye-tracking technology may reveal users' tendencies in gaze behavior with regard to how spatial color combinations are applied. The association between gaze behavior and verbal response indicating the most luxurious space and PS may reveal the user's behavior underlying the designation of a pleasurable experience. To our knowledge, this study is a first attempt to use eye-tracking technology to investigate the role that the

spatial application of a color combination plays in affecting spatial color efficacy leading to the perception of luxury and the intent to stay.

Therefore, in this exploratory study, we hypothesized the following:

H1: Spatial color efficacy and spatial applications of a color combination: When applied differently in a threedimensional space, a color combination will create different emotional and cognitive responses. This phenomenon will be observed in verbal response about the selection of PL and PS as well as the dwell time and the pupil size in eyetracking data during free viewing and the selection tasks.

H2: Spatial color efficacy and PL: Some spatial applications of the same color combination will be more effective than others in affecting the viewer's PL in a retail space. Dwell time in the selection task will correlate with the selection of PL, and the pupil size will correlate with PL.

H3: Spatial color efficacy and PS: Some spatial applications of the same color combination will be more effective than others in affecting the viewer's PS in a retail space. Dwell time in the selection task will correlate with the selection of PS, and the pupil size will correlate with PS.

H4: Depending on how a color combination is applied in a space, even a LL color combination will create positive emotional and cognitive responses, influencing PL and PS.

**Figure 1** provides a summary of a conceptual framework of spatial color efficacy in this study. It shows that spatial color efficacy was investigated in two aspects: the effectiveness of spatial color on the perception of luxury and the effectiveness of spatial color on the intention to stay (preference). Spatial color applications consisting of two conditions of stimuli (HL color combination vs. LL color combination) were examined. Regarding emotional state, arousal was measured through the pupil size, and pleasure was measured through the dwell time. Regarding cognitive state, two dimensions of PL and PS were measured through the selection tasks. As a potential behavior, PS may be a critical indicator that leads to future purchase behavior. This framework is designed to clarify the understanding of human behavior in relation to spatial color efficacy based on the spatial color combination and potential decision-making process in a retail setting.

# MATERIALS AND METHODS

#### Experiment Material Development

In order to understand the spatial color efficacy on PL and PS, we first identified a HL color combination and a LL color combination. Then for each category, we created six different variations to examine how the participants' emotional responses differ depending on where each color was applied. The color selection process was as follows.

#### Step 1: Selection of HL and LL Color Combinations

Color combinations conveying HL and LL were selected from previous research, in which one of the authors of the current paper had participated. Cho and Lee (2015) developed a hypothetical retail store and applied eight color combinations, having identified one HL and one LL color combination from their pilot study of a survey that asked 116 American and Korean college students to select HL and LL images. Cho and Lee (2017) confirmed in a survey of 218 adults that (a) the selected HL color combination was perceived as more luxurious than the selected LL color combination, and (b) the degree of perception of luxury increases pleasure and arousal, which ultimately increases store preference. In the current study, we used the same HL and LL color combinations that Cho and Lee (2017) used in their research as our initial base color combinations.

#### Step 2: Stimuli Development

The HL and LL combinations were applied to an image of a hypothetical retail store, developed with a 3D-rendering program, Sketch UP, and REVIT. The hypothetical store featured floor, wall shelves, mirror, ceiling, three display tables, a cashier's

desk, and two chairs. The size of the store was typical for a retail store, with 3.3 m in height, 18 m in depth, and 9.1 m in width. Appropriate ceiling and wall lighting for product display was applied to achieve a natural, typical look for a store. White tones were applied to ceiling. Three color combinations were applied as follows: Hue 1 to the floor (FL), Hue 2 to the main wall and furniture (WF), and Hue 3 to the recessed wall and seating (RS). In addition, to make the wall shelves visible, Hues 2 and 3 were applied to the shelves on the main wall and recessed wall to provide contrast. Thus, six variations of spatial applications were generated by applying Hues 1, 2, and 3 to the FL, WF, and RS in different ways. **Table 1** provides a summary of a hypothetical store space and the visual stimuli (HL and LL combinations) used in the experiment.

# Participants

A total of 26 interior design majors at a university in South Korea voluntarily participated in the study. Interior design majors were selected because they are particularly qualified to participate in this study because they can be more sensitive to colors than laypersons as they have knowledge of color theory; in fact, the literature shows that designers and laypersons are known to differ in the way they look at buildings (Gifford et al., 2002). The research was announced to one interior design program in which one of the authors teaches, and students voluntarily signed up for the experiment. This study was conducted ethically with the approval of the Institutional Review Board (IRB).

# Experiment Setting and Procedure

The experiment was conducted in a quiet area in a small conference room with a desk and four chairs but no window in order to maintain same temperature, humidity, and light so that participants could focus on the experiment without disruption. During the experiment, participants sat in front of a 24-inch monitor (i.e., 537.6 mm × 296.5 mm without frame) equipped with an eye tracker placed approximately 65 cm from the monitor following the recommendations of the manufacturer. The luminance of the monitor screen was equalized in order to maintain identical brightness of the screen for the experiment. The eye-tracking device was the Remote Eye-Tracking Device Professional (RED) by SensoMotoric Instruments (SMI). The principal investigator and an assistant monitored and administered the experiment. The researchers sat next to the participant, looking at the laptop where the experiment program was installed. The eye-tracking data were stored with 60 Hz, which yields a total of 1,200 pieces of raw gaze data during 20 s per each stimulus. The experiment procedure was as follows. First, the calibration and validation process was conducted. In order to check the accuracy of data collection and precision, we maintained the deviation of tracking data below 0.5◦ on the X, Y axis. Second, an image was provided as a test for participants to become familiar with the setting. Then the experiment assistant explained the procedure to the participant.

The procedures of the main experiment were as follows:

(1) Participants looked at each image in the HL group displayed on a computer screen for 20 s per image.


The same procedure was repeated with the LL stimuli. Total duration for the experiment for each participant varied from 10 to 13 min because the verbal response times varied per individual.

At each step, the participants' eye movements were recorded to determine whether gaze bias existed during the tasks and how the participants' tendency to look at certain images related to their choices. Their eye activity data—dwell time and pupil size were collected using the eye tracker and analyzed using SPSS; the frequency of verbal responses regarding the most luxurious image and the image of the space in which they most wanted to stay were counted. The measures of gaze behavior (dwell time) and arousal (pupil size) were examined during free viewing and during the two tasks in which they identified their PL and PS. **Figure 2** shows the experiment procedure in a diagram format.

# RESULTS

# Accuracy and Precision of Data Check

Checking the accuracy and precision of the obtained eyetracking data is essential to maintain the quality of the data. Accuracy denotes "the difference between true and recorded gaze direction," and precision denotes "how consistent calculated gaze points are, when the true gaze direction is constant" (Holmqvist et al., 2012). The two criteria examined in the eye-tracking data were the tracking ratio and the degree of eye deviation. The tracking ratio is the "[n]umber of non-zero gaze positions divided by sampling frequency multiplied by run duration, expressed in percent" (SensoMotoricInstruments, 2017, p. 347); that is, it shows how much gaze data were collected from the participants. The higher the tracking ratio, the more real gaze data are collected. The tracking ratio can reveal the accuracy and reliability of the data, and more than 80% is considered reliable (Shin and Shin, 2013). Precision can be explained with the degree of eye deviation. Calibration X deviation means deviation on X for eye data (SensoMotoricInstruments, 2017); thus, one participant can produce four deviation degrees: right eye X deviation, right eye Y deviation, left eye X deviation, and left eye Y deviation.

#### TABLE 1 | Basic structure of a hypothetical store and HL, LL visual stimuli.

Basic structure of a hypothetical retail store space

fpsyg-11-00296 March 28, 2020 Time: 18:59 # 6

Developed and reported in Cho and Lee (2015, 2017)

#### HL and LL Hue Selection Image 1 was perceived as the most luxurious

and 5 the least in Cho and Lee (2015). Image 1 was perceived as more luxurious than 5 in Cho and Lee (2017) Based on the study results noted above, 1 and 5 were identified as HL and LL, respectively. Below are the color codes of used hues.

Stimuli Set 1. HL group Newly developed specifically for the current study, six variations of spatial applications were created from the HL color combination.

study, six variations of spatial applications were created from the LL color combination.

The recommended degree is less than 0.5◦ . After removing data from two participants with less than 80% tracking ratio and the other four with more than 0.5◦ of eye deviations, a total of 20 participants' data were selected and analyzed (15 females and 5 males). As a result, the proportion of dismissed participants was 23%, that is, 6 of the original 26.

#### Spatial Color Efficacy Observed by Verbal Response Perceived Luxury

First, participants' verbally delivered responses about PL were analyzed in terms of the frequency of responses and the reason for selection. **Table 2** includes a summary of the participants' responses regarding the group of HL images, consisting of a three-hue combination: dark brown, brown, and ivory.

Among the six images, E was chosen as the most luxurious space by seven participants. It featured a dark brown FL, brown WF, and ivory RS. Reasons for selecting E included dark color and a harmonized look with a nice contrast of colors; participants said they would feel calm and comfortable in this setting. In contrast, C and D were chosen as the least luxurious image by eight participants each. C featured an ivory FL, brown WF, and dark brown RS; D featured a brown FL, ivory WF, and dark brown RS. Participants stated that in C they would feel uncomfortable because of its excessively dark wall and display shelves lacked contrast, and the dark wall and bright floor created too much contrast; in D they said they would feel distracted and uncomfortable because of the lack of harmony among colors and the excessively bright color used on the wall. When comparing spatial color used in E (most), and C and D (least), E had a relatively larger amount of dark color used on both floor and main walls; C and D had a stronger contrast between the wall and the floor.

The LL color group featured brown, orange (in medium dark shade), and green (in medium dark shade). **Table 3** offers a summary of images selected as most luxurious, the frequency of responses, and reasons for selection.

Among the six images, Q was chosen as the most luxurious space by six participants. Q featured a green FL, brown WF, and orange RS. Reasons for selecting Q included its use of dark color on the walls and a small amount of green used in the space. P and R were also highly rated as the most luxurious space with five votes each. P featured a brown FL, orange WF, and green RS; R featured an orange FL, brown WF, and green RS. Reasons for selecting P and R included contrast and harmony among colors as well as the dark color used in the space, which created a luxurious look. In contrast, as the least luxurious image, O was selected by seven participants; it featured an orange FL, green WF, and brown RS. Participants stated that O lacked calmness because of the excessive amount of green used in the space. T was also frequently selected as least luxurious with six votes; it featured a green FL, orange WF, and brown RS. Respondents also mentioned excessive use of green as the reason for lack of luxuriousness; they stated that green seemed more appropriate for a children's space and felt more casual than luxurious. When comparing the spatial color used in Q, P, and R (most luxurious), and O and T (least luxurious), Q, P, and R featured relatively larger amounts of dark color used, such as on the floor or the main wall; but in O and T, only the RS was a dark color.

Response results indicate that green applied to the wall in an interior space diminishes the perception of luxury, and brown walls facilitate the perception of luxury. Orange was not mentioned as a reason for luxuriousness.

In summary, participants' responses regarding PL changed according to different ways of application of a color combination in a physical environment. Throughout the two sets of experiment results, the common features of PL emerged as follows. When same color combinations are applied, people tended to feel (a) high PL when a relatively large amount of the space was of a brown with darker tone; (b) high PL when a brown hue with darker tone was applied on the floor than on the wall or other interior elements; (c) low PL when the floor is lighter than the walls or other interior elements; (d) high PL when a lighter color appears on a recessed wall and a darker color appears on the main wall than vice versa; and (e) low PL resulting from too much contrast between colors or lack of contrast.

#### Preference to Stay

Participants' verbally delivered choices of PS were analyzed in terms of the frequency of responses and reasons for selection. **Table 4** is a summary of the frequency of images chosen for PS in the HL group.

Among the six images, A was chosen by seven participants as the space in which they would most prefer to stay. A featured a brown FL, dark brown WF, and ivory RS. Reasons for selecting A included nice brown colors, which felt calm and comfortable. B was also highly regarded by six participants as the space in which they would most prefer to stay. It featured a dark brown FL, ivory WF, and brown RS. Participants responded that a bright color yields a comfortable environment; a bright wall and dark floor seem to highlight products displayed well. In contrast, a space in which participants least preferred to stay was image F, featuring an ivory FL, dark brown WF, and brown RS. Participants reported that F featured too dark walls and light floor with too much contrast but a lack of contrast on the main and recessed walls, making them feel pressure from the dark walls and uncomfortable from the light floor to use as a space.

**Table 5** presents a summary of the frequency of images chosen for PS in the LL group.

Among the six images, P was chosen as the space where participants would most prefer to stay with seven votes. P featured a brown FL, orange WF, and green RS. Reasons for selecting P included its nice harmony among colors with green as an accent, making them feel comfortable to stay. Q was also highly ranked as a place they preferred to stay, chosen six times and featuring a green FL, brown WF, and orange RS. Participants stated that the brown walls made them feel comfortable staying. As the space in which they least preferred to stay, T garnered eight votes and featured a green FL, orange on WF, and brown RS. Participants reported that the bright orange wall and green floors were too distracting and colorful, making them feel uncomfortable about staying. The other space identified

#### TABLE 2 | Frequency of image chosen for PL and reason for selection in HL.


#### TABLE 3 | Frequency of image chosen for PL and reason for selection in LL.


Note: Total amount of vote is more than the number of students as some students voted two images.

as the one in which they least preferred to stay was O, featuring an orange FL, green WF, and brown RS. The commonalities between T and O included brown applied only in small amounts on the RS, and green and orange as the main colors in the space. Participants stated that T and O seemed like children's spaces, such as playgrounds and elementary schools, not spaces for adults.

In summary, one color combination applied in different ways in a physical environment can change the participants' responses regarding PS. Throughout the two sets of experiment results, the common features of PS emerged as follows. When same color combinations are applied, (a) people tend to prefer a proper level of contrast, not too much contrast, or lack of contrast between colors; and (b) the use of similar tones of two distinctive colors in the majority of the space is unlikely to be preferred.

# Spatial Color Efficacy Captured by Eye-Tracking While Viewing Collection of Six Images

#### Free Viewing and Gaze Behavior

In order to understand varying degrees of spatial color efficacy in variations of spatial application of the same color combination,

#### TABLE 4 | Frequency of image chosen for PS and reason for selection in HL.


TABLE 5 | Frequency of image chosen for PS and reason for selection in LL.


participants' emotional responses were examined using dwell time and pupil size.

When analyzing gaze behavior in viewing the collection of six images together, we set up each image as an AOI and examined the dwell time and the pupil size. Following recommendations from the SMI manual, we used a dwell time longer than 0.08 s (default value) for analysis. We calculated the dwell time for each image during (a) free viewing (FV) of the stimuli without a specific task given; (b) search for a space with PL; and (c) search for a space where participants preferred to stay (PS). **Figure 3** presents screen captures of heat maps of eye-tracking data. **Figure 4** presents a summary of the dwell time and the pupil size when viewing spaces considered HL and LL.

The heat maps show "gaze patterns over the stimulus image visualized as a colored map" (SensoMotoricInstruments, 2017, p. 101). They show data distribution and different intensity of

the visual observation (Špakov and Miniotas, 2007). They are generated based on fixation hits: blue indicates less hits, while red means most hits in the three-color coding system. **Figure 3** shows that, during free viewing of HL, participants' visual attentions were more on images A and B than other images, and these two were chosen as the spaces where participants would most prefer to stay. In addition, during free viewing of LL, participants' visual attentions were more on images P, the space where participants would most prefer to stay, than other images.

**Figure 4** shows that the dwell time and the pupil size when considering PL as well as PS tended to be clearly different across six images. To illustrate, in HL compared to the other two occasions (FV and the search for PL), in the search for PS, the dwell time for B was relatively long, and dwell times for C and F were relatively short. In addition, among LL images, compared to the other two occasions (FV and the search for PL), in the search for PS, the dwell time for P was relatively long, and dwell times for S and T were relatively short, indicating that more gaze bias exists in preference decisions than in decisions of perception of luxury or free viewing. **Figure 4** also shows that the distribution of the pupil size resembles that of the dwell time for determining PL. In contrast, the distribution of the pupil size does not resemble that of the dwell time during FV.

In terms of HL, the image with the longest dwell time during FV was A, followed by B and E. In fact, participants also cited A as the space where they would most prefer to stay, followed

by B and E, in the same order. When determining PL, the image with the longest dwell time was B, followed by E and A. B was actually the image listed as the second most luxurious space. When determining PS, the image with the longest dwell time was B followed by E and A. In fact, B was the image listed as the second most preferred space with just a one vote difference from A. Participants spent the longest time on B when searching for the most luxurious space as well as the most preferred space, but the selection of E as the most luxurious space and the selection of B as the space where they would most prefer to stay may indicate that people debated their decision among A, B, and E. In contrast, dwell times for C and D, which were selected as the least luxurious space, were the shortest during the choice of PL. Likewise, F, selected as the space where participants would least prefer to stay, had the shortest dwell time during the choice of PS. The results indicate that some of the images are more visually engaging to the viewers than others.

In the viewing of the LL group, the image with the longest dwell time during FV was P, which was actually selected as the space where most would prefer to stay. During the search to determine PL, the image with the longest dwell time was Q, which was selected as the image most luxurious. During the search to determine PS, the image with the longest dwell time was P, which was actually selected as the space where most prefer to stay. Compared to the other two occasions, in the search for PS, the dwell time for P was notably long, indicating that depending on the spatial application, a space developed based on a low luxurious color combination can be visually appealing and engaging.

In addition, in order to examine any statistically significant relationships among dwell times and choices for PL and PS, a correlation analysis was conducted. The results are reported in **Table 6**.

The results show that positive correlations were found between the dwell time and the choice of PL in images A, B, C, D, and E. In addition, positive correlations were found between the dwell time and the choice of PS in images A, B, C, D, and E as well as in the dwell time in free viewing and their choice of PL in images A, D, E, and F. No correlation was found, however, between the dwell time in FV and the choice of PS.

Among LL stimuli, the results of the correlation analysis show that positive correlations were found between the dwell time and the choice of PL in all six images. In addition, positive correlations were found between the dwell time and the choice of PS in images O, P, Q, R, and S and between the dwell time in free viewing and their choice of PL in images P, R, and S.

#### Pupil Size Differences

In order to determine whether the participant's arousal state changes during viewing different spatial applications of the same color combination, we also examined pupil size changes. According to the literature, the pupil size can be an indicator of emotional response, arousal, and strong interest (Sirois and Brisson, 2014; Bell et al., 2018; Mathôt, 2018); therefore, the pupil size for each AOI was calculated during FL, PL, and PS.

In order to understand whether the pupil size differs across images, a paired sample t-test was conducted. The results showed that during the choice of PL, the pupil size while viewing B (M = 4.01, SD = 0.52) and C (M = 3.27, SD = 1.73) was different but at a level only marginally statistically not significant [t(19) = 2.07, p = 0.052].

During the choice of PS, a significant difference in the pupil size was found between A (M = 4.03, SD = 0.55) and D (M = 3.08, SD = 1.62); t(19) = 2.15, p = 0.04; and between A (M = 4.23, SD = 0.70) and F(M = 3.92, SD = 1.04) at a level marginally not statistically significant [t(19) = 2.00, p = 0.059]. During the choice of PS, A was actually the image of the space where participants most preferred to stay, and F the least, which means the pupil size differed when looking at images of spaces where participants most and least preferred to stay. The pupil size was larger when participants looked at the images of spaces where they most preferred to stay than least.

Among LL stimuli, during the choice of PL, a significant difference in the pupil size was found between O (M = 4.15, SD = 0.48) and T (M = 2.61, SD = 2.01); t(19) = 3.26, p = 0.004; between P (M = 4.12, SD = 0.60) and T; t(19) = 3.42, p = 0.003; between Q (M = 4.19, SD = 0.51) and T; t(19) = 3.33, p = 0.003; and between S (M = 4.00, SD = 1.09) and T; t(19) = 2.63, p = 0.016. The pupil size was larger when participants looked at the images of most luxurious than least. During the choice of PS, the pupil size differed between O (M = 3.95, SD = 1.05) and T (M = 3.27, SD = 1.74) and between P (M = 3.82, SD = 1.41) and T but at a level only marginally not statistically significant [t(19) = 2.02, p = 0.059, and t(19) = 2.02, p = 0.057 each]. In fact, T was selected as the image of the space that participants least preferred to stay as well the second least luxurious. Q was most luxurious, and P was the image of the space where participants most preferred to stay. Although not at a statistically significant level, data showed that the pupil size differed when looking at images of spaces where participants most and least preferred to stay.

In summary, how research results supported hypotheses appears below:

H1: Spatial color efficacy and spatial applications of a color combination: When applied differently in a threedimensional space, the same color combination will create different emotional and cognitive responses. This phenomenon will be observed in verbal response about the selection of PL and PS as well as the dwell time and the pupil size in eye-tracking data during free viewing and the selection tasks. → Supported

H2: Spatial color efficacy and PL: Some spatial application of the same color combination will be more effective than others in affecting the viewer's PL in a retail space. The dwell time in the selection task will correlate with the selection of PL, and the pupil size will correlate with PL. → Partially supported. The dwell time and the selection correlated, but the pupil size did not correlate with PL.

H3: Spatial color efficacy and PS: Some spatial application of the same color combination will be more effective than others in affecting the viewer's PS in a retail space. The dwell time in the selection task will correlate with the selection of

HL Dwell time during PL Dwell time during PS A B C D E F A B C D E F Choice of PL A 0.946\*\* B 0.879\*\* C 0.852\*\* D 0.451\* E 0.678\*\* F 0.266 Choice of PS A 0.625\*\* B 0.803\*\* C 0.590\*\* D 0.718\*\* E 0.790\*\* F − LL Dwell time during PL Dwell time during PS O P Q R S T O P Q R S T Choice of PL O 0.927\*\* P 0.906\*\* Q 0.879\*\* R 0.775\*\* S 0.933\*\* T 0.919\*\* Choice of PS O 0.794\*\* P 0.819\*\* Q 0.860\*\* R 0.787\*\* S 0.796\*\* T −

TABLE 6 | Correlation analysis between dwell time and response for HL and LL.

\*Correlation is significant at the 0.05 level. \*\*Correlation is significant at the 0.01 level (two-tailed).

PS, and the pupil size will correlate with PS. → Partially supported. The dwell time and the selection correlated, but the pupil size did not correlate with PS.

H4: Depending on how a color combination is applied in a space, even a LL color combination will create positive emotional and cognitive responses, influencing PL and PS. → Supported

#### DISCUSSION

#### Spatial Color Efficacy on Perceived Luxury and Preference to Stay

Our findings in this research demonstrate that spatial color efficacy in various applications of one color combination could vary; furthermore, this research revealed that spatial color efficacy resulting from spatial application of one color combination may vary in affecting PL as well as PS. We also demonstrate that, depending on spatial application, even a LL color combination can enhance positive emotional response leading to PL and PS. These results address the limitation of relying only on color combination swatches in spatial design and suggest vigilant attention to detailed aspects of spatial color efficacy in designing retail interior spaces. In fact, subtle changes in the spatial application of a color combination can create compelling pleasurable experience and may influence the customers' perception and behavior. Therefore, when designing a retail interior space, generating variations of spatial application of initial color combinations as a trial run and assessing the detailed atmospheric quality created by each spatial application will be a meaningful exercise. A palette of three colors may appear as a unitary color combination, but innumerable possibilities of potential spatial variations are embedded, which could lead to diverse emotional and cognitive responses. To our knowledge, this is the first study of the relationship between PL and PS and the application of variations of one color combination in a retail environment, in particular, using eyetracking technology.

When a color combination was applied differently on the floor, main or recessed wall, and furniture, certain tendencies were observed; and thus, we provide the following recommendations for designers and marketers to achieve high levels of PL and PS:


Then what are the possible reasons behind such tendencies? The participants' preference for darker color on surrounding walls and floor may be explained by Stenglin's (2008) binding theory, according to which feeling too unbound by a lack of vertical enclosure causes insecurity, and an extreme level of binding created by spatial enclosure can evoke claustrophobic responses. In contrast, an appropriate level of spatial enclosure can "produce comfort zones of security or freedom" (p. 425). A sense of enclosure and a sense of privacy created by spatial enclosure (Ching, 1996) are important affective dimensions that cause the feeling of comfort and security in a space. Although binding theory does not address the relationship between the affect created by applications of color and spatial enclosure, depending on the property of color, spatial planes could appear approaching or receding and thereby affect the visual emotional perception of the enclosure of a space. One result of this study regarding viewers' preference for dark brown on the floor aligns with Rasmussen's (1964) explanation of "impression of gravity" (p. 219) that can be achieved by using a brown hue on a floor.

Participants' preference for similar hues in varying tones (e.g., brown and dark brown in image A) in a majority of spaces can be explained by the similarity-based model of color relationship suggested by Deng et al. (2010). In an experiment in which participants were asked to choose up to seven colors for seven components of a shoe design, Deng et al. (2010) found that people used relatively small numbers of color palettes. They also found that people like to achieve overall visual congruence by using colors relatively close or exactly matching in the CIELAB color space, and with congruency as a background, people use different hues as a salient figure. Deng et al. (2010) argued that the use of similar colors can increase figural goodness, coherence, unity, and easy perception with less cognitive load, a tendency more strongly supporting the influence of a visual coherence perspective than an optimal arousal perspective on aesthetic preference. When interpreting the similarity-based model of color relationship in the three-dimensional interior space, we understand that people may like background colors of similar hues with varying tones (lightness) and one distinctive accent color. In the current research, participants did not prefer spaces of distinctive hues with similar tones (e.g., orange and green) used in the majority of spaces as observed in image O or T, aligning with the results of the study by Deng et al. (2010) on the infrequent use of colors of the same degree of lightness with hues or saturations of considerable difference. In addition, the finding of the preference of participants in the current study for a proper level of contrast instead of extreme contrast or lack of contrast between the wall and the floor can be explained from the perspective of the congruence model because too much contrast makes the overall space seem disunified and complex.

# Gaze Bias

This research supports the gaze bias theory, addressed by many previous researchers. The results of this study demonstrate that viewers' gaze behavior is related to their preference or choice. The tendency toward viewing certain images longer than others existed in the selection tasks. The dwell time in the task of searching to determine PL correlated with the choice of luxurious space; and the dwell time in the selection task of indicating PS also correlated with the choice of space in which participants preferred to stay. This finding aligns with those in previous studies on gaze bias and preference (Shimojo et al., 2003; Saito et al., 2017). Even though the pupil size did not correlate to the point of statistical significance with the dwell time or the selection, the tendency showed that patterns in the dwell time and the pupil size have some similarity in distribution. This result suggests that depending on the application of the colors, a single color combination can create different emotional and cognitive responses; therefore, some of the applications can appear more attractive than others and visually engaging for the viewers.

#### Implications, Limitations, and Future Research

This study provides several implications to designers, educators, and researchers. First, findings from this research suggest a useful guideline for designers in planning retail stores to achieve desirable levels of luxury. Second, this research provides a recommendation to design educators who teach color theory and environmental psychology to incorporate both color combinations and spatial application. Color use in a space differs from that on a product or an object because people are positioned inside three-dimensional spatial elements. In understanding the human emotional and cognitive response to color in a space, consideration of multidimensional phenomena created by the application of colors in a physical space is crucial. In a three-dimensional space, the overall proportion and composition of multiple components—walls, floors, and objects—are perceived along with colors. Some elements are

under feet; others, overhead; and still others, in the front or on the side. The surfaces of these colored components in relation to the body can alter visual perceptions. Sometimes students learn about color theory only on small color swatches, and they do not delve into the possibility of diverse applications of one color combination in three-dimensional space and the emotional, cognitive, and behavioral responses generated from such application. Developing software that could generate multiple variations of spatial color applications for students to view or a software program using augmented reality (AR) to apply color combinations to real space and see the effect easily would be of considerable help in nurturing the perceptual understanding and knowledge of color application for the pleasurable experience of space. Third, this research provides insight to researchers on the use of eye-tracking technology on the study of impact of color on emotion and behavior. As one of the physiological measurements, eye-tracking technology can be a strong link to reveal behaviors underlying the decisionmaking process.

Three limitations are present in this study. First, the number of participants was small and limited to interior design students in one country. Students may not have the economic and financial capacity to afford luxury goods, so some gap may exist between actual luxury consumers and members of the participant group. However, as millennials, born between 1980 and 2000, occupy a substantial market in Korea (Ahn, 2019) and overseas (Lee, 2019) and the concept of luxury expands to highlight self-concept and experiential quality instead of exclusiveness or inaccessibility (Chandon et al., 2016; de Kerviler and Rodriguez, 2019), a study with college students as participants is still meaningful. The reason for engaging interior design major students as participants was to study individuals with similar characteristics who would reveal the gaze behavior of a particular group. Based on the findings of this study, future studies could expand to participant groups with diverse demographic backgrounds to achieve generalization. Second, in the current study, only two color combinations were used for the study of spatial color efficacy. Incorporating more diverse color combinations as stimuli in experiments will be helpful in observing gaze behavior and identifying commonalities of the space perceived as luxurious and the one in which people preferred to stay; this information can be developed into a spatial color efficacy theory in interior design. Third, we did not include any questions on purchase intention in this study because no specific product was designated.

Future research that includes purchase intention will be helpful to reveal the link between verbal and physiological response and purchase behavior. In addition, as spatial color efficacy was revealed, in order to more systematically investigate attributes that contribute to PL and PS, a study using

#### REFERENCES

Ahn, J. G. (2019). The life of YOLO who shows in SNS: Millennials buying luxury goods in The Korea Economic Daily, Hankyung. Available at: https:// www.hankyung.com/economy/article/2019112268111 (accessed November 23, 2019).

choice modeling theory will be meaningful. Choice modeling theory will allow researchers to examine how and what attributes among those found in this research—such as contrast among colors, darkness in tones, and similarity-based color of relationship—influence participants' decision making for PL and PS. Moreover, examining participants' own meaning of luxury was beyond the scope of this research. Linking participants' perception of luxury and verbal responses could reveal attributes contributing to PL and PS. Because both topdown factors (individual characteristics) as well as bottom-up factors (physical attributes of stimuli) are important in visual attention and environmental psychology, future researchers should systematically incorporate both factors.

#### DATA AVAILABILITY STATEMENT

The datasets generated for this study are available on request to the corresponding author.

#### ETHICS STATEMENT

The studies involving human participants were reviewed and approved by Kyung Hee University. The patients/participants provided their written informed consent to participate in this study.

# AUTHOR CONTRIBUTIONS

JC developed the study concept, collected the data, conducted the data analysis, drafted and reviewed the manuscript, and supervised the research. JS developed the conceptual framework, interpreted the data, drafted the manuscript, and extensively reviewed and edited the whole manuscript. All authors contributed to manuscript revision and read and approved the submitted version.

#### FUNDING

This work was supported by a grant from Kyung Hee University in 2016 (KHU-20161386).

# ACKNOWLEDGMENTS

We acknowledge the research participants. We also acknowledge Edward Lim at Varison Co., Ltd for providing advice in using BeGaze, the analysis software of SMI RED.


investigate consumer behavior. Front. Psychol. 9:1655. doi: 10.3389/fpsyg.2018. 01655


**Conflict of Interest:** The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Copyright © 2020 Cho and Suh. This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) and the copyright owner(s) are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.

# Opportunities and Challenges for Chinese Elderly Care Industry in Smart Environment Based on Occupants' Needs and Preferences

Qingfeng Meng<sup>1</sup> , Ziming Hong<sup>1</sup> , Zhen Li<sup>1</sup> , Xin Hu<sup>2</sup> , Weixiang Shi<sup>3</sup> \*, Jun Wang<sup>2</sup> and Kai Luo<sup>3</sup>

<sup>1</sup> School of Management, Jiangsu University, Zhenjiang, China, <sup>2</sup> School of Architecture and Built Environment, Deakin University, Geelong, VIC, Australia, <sup>3</sup> Australasian Joint Research Centre for Building Information Modelling, School of Built Environment, Curtin University, Bentley, WA, Australia

#### Edited by:

Mi Jeong Kim, Hanyang University, South Korea

#### Reviewed by:

Mikyung Kim, Chungbuk National University, South Korea Minah Lee, Kunsan National University, South Korea

\*Correspondence: Weixiang Shi brandonshi163@gmail.com

#### Specialty section:

This article was submitted to Environmental Psychology, a section of the journal Frontiers in Psychology

Received: 11 November 2019 Accepted: 24 April 2020 Published: 04 June 2020

#### Citation:

Meng Q, Hong Z, Li Z, Hu X, Shi W, Wang J and Luo K (2020) Opportunities and Challenges for Chinese Elderly Care Industry in Smart Environment Based on Occupants' Needs and Preferences. Front. Psychol. 11:1029. doi: 10.3389/fpsyg.2020.01029 New developments in intelligent devices for assisting elderly people can provide elders with friendly, mutual, and personalized interactions. Since the intelligent devices should continually make an important contribution to the smart elderly care industry, smart services or policies for the elders are recently provided by a large number of government programs in China. At present, the smart elderly care industry in China has attracted numerous investors' attention, but the smart elderly care industry in China is still at the beginning stage. Though there are great opportunities in the market, many challenges and limitations still need to be solved. This study analyzes 198 news reports about opportunities and challenges in the smart elderly care industry from six major Chinese portals. The analysis is mainly based on needs assessment for elderly people, service providers, and the Chinese government. It is concluded that smart elderly care services satisfy the elders' mental wants and that needs for improving modernization services are the most frequently mentioned opportunities. Also, the frequently mentioned challenges behind opportunities are intelligent products not being able to solve the just-needed, user-consumption concept and the ability to pay, which is the most frequently mentioned challenge. The results of this study will enable stakeholders in the smart elderly care industry to clarify the opportunities and challenges faced by smart elderly care services in China's development process and provide a theoretical basis for better decision making.

Keywords: smart elderly care, intelligent environment, opportunities and challenges, news reports, content analysis

# INTRODUCTION

At present, China has become the country with the largest number of elderly people in the world. According to the latest statistics from the National Bureau of Statistics, the number of elderly people over the age of 60 in China has been increasing in recent years. In 2013, it exceeded 200 million, accounting for only 14.9%, and it reached 240.9 million in 2017, breaking the record by 17%. By the end of 2018, the population aged 60 and over was 249.49 million, accounting for 17.9%. Among them, the population aged 65 and over was 166.58 million, accounting for 11.9%. Compared with

that at the end of 2017, the proportion of the elderly population continues to rise. With the aging of the population, the state has paid more and more attention to the development of the old-age industry, issued a series of policy support, gradually improved the social security system for the elderly population, fully opened the development of the aged care service market, vigorously prospered the old consumer market, and actively promoted the community-based in-home elderly care, the combination of medical care and elderly care and other models.

The smart elderly care industry is a product under the background of the "supply-side structural reform" and a recipe for solving the imbalance of China's elderly care industry structure. Smart health and elderly care refer to the integration of information technology and products such as health care electronics, Internet of Things, cloud computing, big data, and mobile Internet for collecting data such as human signs and home environment; realizing family, community medical institutions, health care services and information interconnection, and analytical processing between professional medical institutions; and providing intelligent, personalized, and diversified products and services to meet the increasingly urgent health needs of the people (Yang et al., 2016).

Wisdom for the elderly plays an important role in improving the quality of life of the elderly and has attracted the attention of many investors and government departments. The smart elderly care industry grows rapidly since the state council published the guideline for promoting the development of "Internet+" in July of 2015. The Premier of the state council pointed out that the smart elderly care industry needs to be improved as soon as possible, in the Report on the work of the Chinese government. In the early 21st century, the smart elderly care service was employed in the United States and European developed countries, and now it is in a mature stage. For example, as early as 2007, Europe states established the Active and Assisted Living (AAL) Research and Development Program to promote the smart elderly care industry (Courtney et al., 2008; Panek and Zagler, 2008). Over the last 10 years, the AAL program has focused on improving the well-being of older adults through the use of adapted digital technology. The AAL forum also attracts potential investors and buyers looking for commercially viable solutions (Cotten et al., 2016; Siegel and Dorner, 2017). Since 2008, the smart elderly care industry in the United States has been committed to bridging the digital divide between technology and health care so that people can live healthier lives. However, the development of China's smart elderly care system is still in its infancy, and many factors are affecting the development of the smart elderly care industry, such as the residents' needs and preferences and the service level of product service providers. Some of these factors may positively promote the development of a smart elderly care environment, and some may become obstacles in the development of the smart elderly care industry.

To more effectively promote the healthy and sustainable development of the smart elderly care industry in China, we need to clarify the opportunities and challenges faced by the smart elderly care industry in China's development process. First, the domestic industry practitioners are keen to understand these opportunities and challenges to formulate effective property management strategies, which is crucial for them to obtain competitive opportunities especially in the initial stages of the industry development. This will also guide the development of China's smart elderly care market. The development of China's smart elderly care system is still in its infancy, and many factors are affecting the development of the smart old-age environment, such as the residents' needs and preferences and the service level of product service providers. Some of these factors may positively promote the development of a smart elderly care industry, and some may become obstacles in the development of the smart elderly care industry. With the aging of the population, China's government has paid more attention to the development of the smart elderly care industry. The government has published a series of policies that support the industry: "Deeply develop the elderly care industry," "Deeply promote innovation-driven development strategy," and "Deeply integrate "Internet+" and smart elderly care industry." The smart elderly care industry in China is just beginning to burst, so there are lots of opportunities and challenges in the industry. Therefore, it is very important to find out all opportunities and challenges in the smart elderly care industry of China overall. The findings in this paper will guide the industry in the right direction.

This paper uses content analysis methods to analyze the news reports about the smart elderly care system from June 2018 to June 2019. It comprehensively summarizes and analyzes the opportunities and challenges of China's smart old-age environment in the development process. The research findings will not only provide valuable implications to the industry practitioners but also help the government formulate appropriate strategic property management strategies to ensure the healthy development of this sector.

The chapters of this paper are arranged as follows: the second chapter will summarize and analyze the current status of China's smart elderly care; the third chapter introduces the research methods of this paper; the fourth chapter summarizes the opportunities and challenges that we can understand through the analysis of China's smart old-age environment in the development process; the fifth chapter analyzes the existing obstacles and gives corresponding countermeasures and suggestions; and the sixth chapter is the summary of this paper.

# ANALYSIS OF THE CURRENT SITUATION OF SMART ELDERLY CARE IN CHINA

This study focuses on smart care service for Chinese elderly people. This analysis exposes a better understanding of the most popular topics in the smart elderly care industry, including the status of the elderly care service and smart community of elderly people in China. By depicting a clear picture of related topics, the theoretical background of this study is provided.

First of all, for the old-age care and old-age environment of the elderly in China, aging at home, usually co-residing with family members, is the most popular living arrangement for older adults in China (Silverstein et al., 2006). The conventional Chinese cultural pattern of filial piety, which obliges family members to assist their elders, has contributed to the close connections

between different generations (Mao and Chi, 2011). The Chinese government has also enacted laws (e.g., the Elders' Protection Law) to maintain this traditional family support system to care for older people financially, physically, and emotionally (Wu et al., 2005). In China's small-scale peasant society, the family is the foundation of society and is the most basic unit. At present, with the continuous development of the economy and the continuous improvement of people's living standards, more and more elderly people choose to support the elderly in some oldage communities. With the further development of the market economy, social changes, and Chinese-style "family care" model, a new model consisting of "community care" and "institution care" has been produced (Zhu et al., 2014).

Secondly, with the gradual maturity of technologies such as artificial intelligence and big data, the industry tends to be informal, convenient, and intelligent. For a time, those who are entering are talking about "intelligence and old age," and more technology and energy are left in old age. On the smart hardware, the design and use of smart devices in China has emerged in China's old-age community. Regarding the status quo of China's smart elderly care, China is changing from an agricultural society to an informational society to achieve the "modernization" found in the Western world (Economy, 1998). With the rapid development of information and technology, the smart retirement community is also a fast-growing thing. Moen and Erickson (2001) designed the smart elderly community as an age-homogenization facility that provides residents with a variety of services and facilities to meet their changing needs in an environment. Jenkins et al. (2002) suggest that the smart elderly community life can benefit residents by improving health status and social interaction levels. In 2012, China began to implement the concept of a "smart city" which focused on the care of the elderly using technology (Heng et al., 2014). In 2017, the three ministries and commissions have announced two demonstrations of smart health elderly care application pilots. At present, there are 79 smart health elderly care demonstration enterprises, 130 smart health elderly care demonstration streets (townships), and 29 smart health elderly care demonstration bases.

Besides, some scholars have analyzed the motivations for the development of smart old-age care. Chou (2010) used the first random sample from large countries, guided by a comprehensive model, and used a study of urban–rural comparison methods to study the willingness of 20,255 elderly people to go to elderly care and found that in urban and rural areas, only 20 and 17% of seniors are willing to do so. For both economic and sociocultural reasons, researchers have pointed to the need to offer more support for family care and to expand and strengthen non-institutional forms of long-term care, such as home health and community-based care. This study has provided additional evidence of the need to build a long-term care system that is consistent with China's economic and sociocultural conditions and that meets older adults' needs and preferences. Hu (2010) relied on the demographic structure to analyze the aging trend and, by comparing with the foreign treatment methods, determined the necessity of establishing an intelligent community in China. He proposed that the intelligent elderly community can not only meet the ideal status of the elderly in the country and reduce the burden on workers but also point out a new feasible way for aging China. Therefore, it is very important to establish a Chinese smart elderly community.

The existing research on opportunities and challenges for smart elderly care industry in other countries is mainly on smart technologies and smart equipment. Habib et al. (2014) introduced the challenges and open issues of smartphone-based solutions for fall detection and prevention. Suryadevara and Mukhopadhyay (2012) introduced a wireless sensor networkbased home monitoring system for wellness determination of the elderly and analyzed the opportunities of a wireless sensor network in the smart elderly care industry. Geman et al. (2015) presented some challenges and opportunities in Ambient Assisted Living (AAL) for disabled and elderly people addressing the various state-of-the-art and recent approaches particularly in artificial intelligence, biomedical engineering, and body sensor networking. Foreign researchers have less research on opportunities and challenges for the development of smart elderly care industry. So it is of great significance to find out the opportunities and challenges for the smart elderly care industry.

# RESEARCH METHODS

News coverage contains valuable information, and analyzing news coverage provides a feasible way of exploring the various aspects of a phenomenon through excavating the story behind it and the perception of field experts (An and Gower, 2009). This method can clearly understand the attitudes of frontline personnel and related practitioners to the research objects and evaluate the news content as professional journalists and also explore the complex interrelationship between a thing and its complicated use environment. This method has been applied to many fields such as the humanities and social sciences.

This paper uses quantitative analysis and qualitative analysis research methods to explore the opportunities and challenges of smart old-age development in China from news reports. There are several reasons for using this method. First of all, because smart elderly care is used in the life of the elderly, it is an important development direction for China to enter the Internet of Things and the "Internet+" era, and it is also one of the specific applications of the continuous development and progress of artificial intelligence (AI). This has led to a large number of reports from the mass media in the country about this news. According to the Code of Ethics of the Society of Professional Journalists, news coverage is accurate, fair, and thorough. A careful study of relevant news reports found that most of the articles reported were some annual smart old-agerelated seminars, on-site interviews organized by smart oldage care communities, and on-the-spot interviews with smart equipment manufacturers. The interviewees were also related industries' professionals (government officials, academics, elderly users, service operators, product developers, etc.). Therefore, the information reported in the news reports used to explore the challenges of smart elderly care development in China is credible and valuable. Besides, the use of traditional survey methods (interviews or questionnaires) takes a lot of time and effort

and is not efficient. As China's smart elderly care system is still in its infancy, the use of smart elderly care products in oldage services is insufficient, the elderly are less likely to accept new things in terms of learning acceptance, and the number of professional staff engaged in elderly care smart equipment services is small. The direct interviews with direct workers took a lot of time and effort and were very inefficient. On the contrary, the use of content analysis to analyze news reports can obtain results more objectively and reliably, and it takes less time and resources and is more efficient. Also, since the development of China's smart old-age care is in its infancy, news reporting research on this issue is very time sensitive and insufficient in research and exploration. Therefore, based on the comprehensive analysis of news reports, it is possible to further explore through interviews, questionnaires, etc., providing valuable value for the entire study. Analyzing news reports has been used in several published studies, confirming its usefulness and robustness in understanding certain issues.

Content analysis is an approach often used to analyze news coverage (Calloway et al., 2006; Qu et al., 2009). It was also used in this research to gain insights from the identified news reports. It is a systematic and objective research method for making valid inferences from collected data to describe and quantify specific phenomena (Downe-Wamboldt, 1992). Its suitability to analyze news coverage has been confirmed in the historical studies of An and Gower (2009), for example. Compared with traditional methods (e.g., interviews and questionnaire surveys), the content analysis provides more objective and reliable results (based on real and "mute" evidence). Meanwhile, it consumes less time and resources (Hu et al., 2017). Nimrod (2009) analyzed benefits to elders in the elderly community by using newspaper coverage analysis and content analysis. Mercille (2014) examined the mainstream media coverage to explore the housing bubble issue in Ireland. The successful explorations of these related studies indicated that news coverage analysis is suitable for such papers. Data collection (identification and collection of news reports on the development opportunities and challenges of smart aging in China) and data analysis (qualitative analysis and quantitative analysis of collected news reports) are two key steps, as shown in **Figure 1**.

The news gathering process includes the following:

#### Determine the Source of News Reports

The Internet provides a convenient way to get news stories. It has the advantages of large news reports, high keyword search efficiency, and low search cost. Since most of the news reports about China's only old-age community have been published or reprinted on major portals, the news reports used in this study are all from China's major portals. Other sources (e.g., newspapers and magazines) were not used as it is too inefficient to collect and search related news coverage in this way and as they often involve access charges (Hu et al., 2019).

The list of "Top Sites in China" released by the Alexa web ranking service provider facilitated the identification of the major Chinese portal websites used in this study (Alexa, 2019). Alexa lists the top sites in each country/territory, and its reliability and usefulness are well accepted (Jowkar and Didegah, 2010). This study screened the top 30 websites ranked by the Chinese Alexa website in 2019. Since some websites are special, websites such as shopping websites and video websites, it is impossible to find suitable news reports. Therefore, we have filtered six portals to more accurately search for the news reports corresponding to the research content (**Table 1**). These websites are news websites, or their news reports are one of the important parts of their websites.

# Retrieving News Reports About China's Smart Retirement Community

As shown in **Figure 1**, the article search is performed using a search engine in the selected portal to control the search scope to each portal. Since most news reports are in Chinese, the keywords searched are in Chinese. There are many different keywords in the news report that can represent the same meaning, such as smart elderly care, smart elderly care, high-tech elderly care, opportunities, challenges, obstacles, malpractices, and deficiencies. To get a comprehensive search result, from April to July 2019, we used multiple Chinese phrases to search on each portal. For example, we use "Smart Aging Opportunity" and "Smart Elderly care Advantages" as keywords to search for news reports on the sina.com website or the "Smart Care Challenge" as a keyword to search for news reports on the 163.com website. A total of 3,534 articles (**Table 1**) were obtained.

# Formulate Selection Criteria and Choose News Reports

We develop selection criteria for news stories to determine which news stories can be used. To be objective, we select 200 latest news reports on the smart elderly care industry. The articles reported are some annual smart old-age-related seminars, on-site interviews organized by smart old-age care communities, and on-the-spot interviews with smart equipment manufacturers. The interviewees are also related industries' professionals (government officials, academics, elderly users, service operators, product developers, etc.). First, we determine the similarity between the article and the topic under study by evaluating the topic and content of the news. Secondly, considering that China's smart devices are in the development stage, the product update progress is very fast, so we only select news reports in the past year. Thirdly, the articles selected must focus on an identified obstacle or opportunities and must mention three or more factors of obstacles and opportunities in the development of the industry. Besides, there are duplicate articles between different portals that need to be screened to ensure the quality of the selected news stories and the accuracy of the analytical data. Finally, in the 3,534 news reports, 198 news reports most relevant to the research topic were screened out. After several discussions, we finally choose 198 news reports out of the original 200 because two of them do not meet our standard. There were 58 in 2018 and 140 in 2019. They are seminars and presentations on the stakeholders of smart devices, interview articles on actual use of customers, and the development of smart old-age communities in China.

Qualitative analysis and quantitative analysis were used. Qualitative analysis involves identification and coding, which

divides the barrier into different groups. This process includes the selection of analysis units, coding, grouping by category, analysis, and credibility assessment. First, the analysis unit is the basis for reporting analyses, and the most appropriate analysis unit is whole interviews or observational protocols (Graneheim and Lundman, 2004). In this study, the analysis unit is the content sentence mentioned in each article about "Smart Care for the Elderly" and "Smart Care for the Elderly." Second, coding and grouping categories can be conducted based on either predetermined systems/frameworks or analysis of collected data. Since different news reports have different words describing the same opportunity or challenge, based on similar analysis units, we will summarize and encode a short sentence. The analysis units with the same meaning are assigned to the unified obstacle group, and the identified obstacle groups are further divided into different categories. Third, we use Microsoft Excel to build a database and enter the barriers and categories identified by the barriers. Therefore, this process can modify inappropriate opportunities and challenges group names and groupings. Based on the qualitative analysis, quantitative analysis is used to count the number of obstacle groups, and the frequency with which the obstacles are mentioned is calculated.

# RESULTS

As shown in **Tables 2**, **3**, 198 news reports in this paper are coded according to the form of "Category – opportunities and challenges–unit of analysis," and the content analysis unit is classified, to cover all the contents of the news reports, and finally from the content analysis unit coding table based on the news report. Because of the article space limitation, the table does not show the full content analysis unit and coding results.

**Table 4** shows the 14 opportunities identified and their frequencies based on the content analysis. Among them, the highest frequency is to improve the level of institutional modernization services, accounting for 59.1%. For the elderly, the main opportunity for smart retirement is to meet the spiritual and


health needs of the elderly, accounting for 56.8%. The satisfaction of physiological needs and safety needs is relatively low.

**Table 5** shows the 17 challenges identified and their frequencies based on the content analysis. Among them, the most frequent challenge is that smart product services are not practical and cannot solve just-needed concepts, accounting for 59.8%. The elderly have insufficient market education and have a vague concept of smart elderly care, a conservative concept of consumer consumption, and a limited ability to pay. These two issues are also relatively high, both above 30%. However, existing policy constraints, lack of policy support, lack of a unified administrative agency, and other challenges occur less frequently.

These opportunities and challenges are divided into three categories: senior user needs, product service delivery, and government promotion. In terms of opportunities, the opportunities for smart retirement communities mainly face the needs of elderly users and the provision of products and services. Among the 14 opportunities, 11 are related to the needs of elderly users and product services. On the challenge side, 9 of the 17 challenges were related to product service, accounting for 52.9%.

# DISCUSSION

Through statistical analysis of the collected news reports, we identify the opportunities and challenges in the smart elderly industry in China. The following discussion will be based on three dimensions, including elderly user demand, product service provision, and government promotion, to analyze the challenges and opportunities.

# The Dimension of Elderly User Demand

Satisfying the spiritual needs and the health needs of elders is the most important opportunity that the smart elderly care industry faces in the development of China. In the meantime, insufficient market education, the vague concept of the smart elderly care industry, poor acceptance to new things, negative consumer behavior, and the issues of ability to pay are the most important challenges that the smart elderly care industry faces in the development of China. There is enough evidence that the smart elderly care system can improve the quality of elders' life by using advanced technologies, such as health monitoring device, elderly care devices, remote medical treatment, and noninfectious chronic disease (NCD) management cloud platform. On the other hand, the smart elderly care system can satisfy


#### TABLE 3 | The content analysis coding table of challenges in the industry.


TABLE 4 | Opportunities for Chinese elderly care industry in the smart environment.


mental needs with VR/AR technology and remote care system. Intelligent devices can provide personalized interaction and service to release loneliness of elders who have no children compared to the traditional elderly care system.

Serviceability of smart elderly care products needs to be improved, though responses and reviews of smart elderly care products are quite good. Furthermore, elders still have a lot of trouble in using these products. Compared to the young generation, elders show a negative attitude to new things and have a relatively poor learning ability. So their willingness to accept smart elderly care products is not strong. Besides, smart elderly care products are expensive in general while the consumption behavior of most elders is relatively conservative. There are conflicts between the consumption behavior of most elders and the high price of smart elderly care products. Therefore, it results in many challenges in the smart elderly care industry.


The manufacturers in China nationally promote the concept of smart elderly care and provide elders with a free trial to motivate the willingness of elders. In the process of national promotion, the initially targeted elders are suggested to be the elders who already have the willingness to accept intelligent devices, and then the products can be promoted within the elders' community with positive responses and reviews. At the same time, the manufacturers need to create and update smart elderly care products and service for families, communities, and institutions, such as wearable devices for health management system and health monitoring system and robots for home services or institution services. The manufacturers can also provide rental service in some pilot cities for elders who want to experience the products but have relatively limited ability to afford the products.

# The Dimension of Product Service Offering

Improving the modernization of elderly care service and the efficiency of management in the elderly care system is the most important opportunities in the development of the smart elderly care industry. The following are the most important challenges for the industry: The smart products are not practical in many cases, the supply chain is long and fragmented, the innovation system is old, and there is a lack of effective integration in the industry. There is no doubt that smart devices improve the quality of elderly care service through the use of lots of advanced technologies, such as big data, AI, 5G, and Internet of Things. They help elderly care services to improve accuracy and efficiency. Besides, they also assist elderly care institutions by deeply finding out the demands of their customers through collecting data and analyzing the data collected by smart devices. Analyzed data will assist managers in the institutions to make the right decisions.

However, the smart elderly care industry still faces lots of problems that need to be resolved. Firstly, many smart products provided by elderly care institutions cannot satisfy the personalized demands of elders because the products still need lots of behavior data of elders to be trained. Secondly, many smart products still have trouble in data connection because manufacturers use different interface standards. So it is very hard to effectively use and share the data. The smart elderly care industry involves the IT industry, medical treatment industry, health industry, and elderly care industry. So it is urgent to set up an open standard to link those industries. The industry still has lots of issues on medical device access, medical and health data security, and development of integration of medical institutions and care institutions. Most elderly care institutions are not willing to cooperate with manufacturers in daily health management and NCD management. The manufacturers should investigate elders' demand before developing smart products, and the surveys should categorize and divide these products into as many units. Smart products should be updated and modified through a full life cycle. The manufacturers should carefully consider elders' willingness, the ability of acceptance, and affordability because the purpose of smart products is to improve the quality of elders' life. The industry needs to set up several pilot smart elderly care communities or institutions to gain more experience in a practical environment. In the next few years, the smart elderly care system should be involved in smart cities to resolve Information Isolated Island issues. All the data from the medical system, elderly care system, and other systems should be connected in an efficient way to support the smart elderly care system.

# The Dimension of Government Promotion

Assisting China's government to make scientific decisions and supporting China's government to supervise the elderly

care institutions are the most important opportunities in the development of the smart elderly care industry. The existing policy constraints are the most important challenges in the development of the smart elderly care industry. China's government has announced the "Action Plan for The Development of Smart Elderly Care Industry" (2017–2020). In the action plan, it is mentioned that the industry should fundamentally grow to a comprehensive industry system and produce several smart elderly care brands which are impressed in elders. However, in reality, smart elderly care systems are difficult to promote in most cities in China because the action plan is too general to execute in cities of China. Many constraints and limitations in policies and regulations need to be resolved soon in every province of China. In the meantime, the smart elderly care industry is still in the beginning stage, and so China's government just selects several developed cities to promote the smart elderly care system. In other words, it is not a real nationwide promotion in China. Lack of policies and regulations for the local government reduces the motivation of the local government for promoting the industry. So it is highly recommended that China's government should promulgate adequate policies and regulations for the smart elderly care industry. Besides, standards and guidelines for the industry are also foundations for promoting the smart elderly care system.

In the future, the Chinese government will continue to vigorously strengthen the development of informalization around the state. The government will suggest that smart elderly care institutions need to provide elders with smart products for improving the quality of smart elderly care. The industry will be partly subsidized by the government for promoting smart products. The manufacturers and service providers need to fund a forum for bringing the government, manufacturers, and service providers together around the state to find out smart solutions and to engage with other people in the market. The forum is suggested to be cofinanced by the government and manufacturers. The specific aim is to foster the emergence of manufacturers, service providers, and systems for elders living well at home and in institutions. The elderly users should provide more reviews on smart products while enjoying the smart service to support the development of the smart elderly care industry in China. Each year, the forum issues a call designed to solve the negative reviews which help promote the elders to be positive on smart products and a call designed to reflect the opportunities and challenges in the development process of the smart elderly care industry. Through our cooperation, the smart elderly care industry in China will be on the right track.

# CONCLUSION

Whether it is from the top design, rigid demand, or scientific and practical convenience, smart elderly care will be an important force in the future elderly care industry. At present, there is a lot of challenges and shortcomings in the initial stage of the development of the smart old-age environment in China, bringing various opportunities. It is of great significance to clarify these opportunities and shortcomings to promote the development of China's smart elderly care.

This paper mainly analyzes 198 news reports of China's six major portals, explores the opportunities and challenges of smart elderly care development in China, and identifies 14 opportunities and 17 challenges. These opportunities and challenges are divided into three categories: senior user needs, product service delivery, and government promotion. According to the frequency of opportunities and challenges reported in the new coverage analysis, the most significant advantage relating to these three groups is to improve the level of institutional modernization services, and the most significant barrier relating to these three groups is that smart product service is not practical and cannot be used to solve just-needed concepts.

The research results of this paper are beneficial to China's smart aged care service providers and China's smart old-age care industry in many aspects. First, the domestic industry practitioners are keen to understand these opportunities and challenges to formulate effective property management strategies, which is crucial for them to obtain competitive opportunities especially in the initial stages of the industry development. Besides, this will also guide the development of China's smart elderly care market.

Future research, such as case studies, interviews, and questionnaire survey, is needed to examine individual barriers more closely and develop practical means for their amelioration and elimination.

# DATA AVAILABILITY STATEMENT

All datasets generated for this study are included in the article/supplementary material.

# AUTHOR CONTRIBUTIONS

QM, XH, and JW devised the project, the main conceptual ideas, and proof outline. QM, ZH, and WS developed the theory and analyzed the current situation of smart pension in China. ZL and KL verified the analytical methods. QM and XH encouraged ZH and WS to investigate quantitative analysis and qualitative analysis research methods. QM and ZL supervised the findings of this work. All authors discussed the results and contributed to the final manuscript.

#### FUNDING

This work was supported by the National Natural Science Foundation of China (nos. 71971100, 71671078, 71501084, 71771127, and 71602083), the Social Science Fund of Jiangsu Province (nos. 19GLB005 and 19GLB018), the Key Project of Philosophy and Social Science Research in Colleges and Universities in Jiangsu Province (no. 2018SJZDI052), sponsored by the Qing Lan Project of Jiangsu Province, and the Youth Backbone Teacher Training Project of Jiangsu University.

# REFERENCES

fpsyg-11-01029 June 2, 2020 Time: 20:41 # 10


**Conflict of Interest:** The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Copyright © 2020 Meng, Hong, Li, Hu, Shi, Wang and Luo. This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) and the copyright owner(s) are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.