Mobile QR Code QR CODE

  1. ( Robotics Lab, Hyundai Motor Company/Uiwang, Korea {simon_shim, juhee.park}@hyundai.com, mecjin@gmail.com)
  2. ( School of AI Convergence, Sungshin Women’s University / Seoul, Korea {jkim, 20211366}@sungshin.ac.kr)
  3. ( School of Business Administration, Sungshin Women’s University / Seoul, Korea {swhwang, 20210540}@sungshin.ac.kr)
  4. ( Graduate School of Management of Technology, Sungkyunkwan University / Suwon, Korea leehee@skku.edu)
  5. († These authors contributed equally to this work)



CPS, Virtual spaces, Robot services, SLR, TSA

1. Introduction

A robot is a complex machine that can be programmed to automate tasks and operations so they require minimal human effort [1]. On a large scale, the primary role of a robot is to assist with tasks and activities. In particular, the manufacturing sector has emerged as the primary field in which robotic systems are widely available and autonomous to the point where most manufacturing processes rely on them [2]. In addition to their widespread use in that sector, the COVID-19 pandemic of the past several years has inspired remote human activities and services via robots, as well as virtual spaces collectively referred to as the metaverse [3]. Prime examples are contactless delivery robots, including delivery of packages and food, where robots can significantly assist humans.

Moreover, the substantial popularity of metaverses such as Decentraland and Roblox proves that such virtual environments can serve not only as spaces for games or entertainment but also for general human interaction and communication. It should be noted that virtual environments have been studied as a part of a cyber-physical system (CPS), which focuses on the interaction between cyber and physical spaces. However, it will be necessary to implement appropriate scalable platforms and real-time streaming technologies to support the virtual space.

While the use of robots significantly impacts the processes of manufacturing and product design, recent advancements have allowed development and design of robot bodies and systems in virtual spaces. For instance, NVIDIA launched a virtual development environment called the Omniverse platform [4], which enables robot design and body simulations in all virtual spaces. Because of the advantages of its graphic processing unit technologies, NVIDIA intends to expand its platform’s capabilities into manufacturing, building, and autonomous environments.

When considering robots on the Omniverse platform as shown in Fig. 1, the use of virtual space leads to increased potential for robot development. However, since only a handful of technical and business examples of robots exist in virtual space, it is necessary to scrutinize the main robot trends and considerations within virtual spaces in academic and industrial arenas. In particular, to extend our studies to a wider range of robotics and virtual environments, we must proceed with them based on a CPS.

This paper investigates current trends and considerations for robotics in virtual environments. We adopted two approaches for this review: a systematic literature review (SLR) and a technical standards analysis (TSA). The SLR explores the significance of specific topics in selected research articles from academic journals [5]. In this study, we approach the SLR using multiple research-paper search engines, and we selected a few publications that satisfy all our requirements as detailed later. We selected multiple standards documents for the TSA, including standards for virtual reality data formats, protocols, interfaces, and robotics safety. We also investigated future directions for standards. Using these two approaches, we were able to analyze directions in R&D and applications of academic and industry perspectives to CPS-based robotics. Patent analysis is a possible consideration in an extensive review. However, we believe technological advances and regulatory impacts are more closely related to research articles and technical standards, rather than patents. Moreover, the metaverse and virtual space, including the CPS, are active research topics in the academic communities, and technical standards serve as coordinators for various industrial patents competing with each other. Hence, we did not consider patent analysis.

This paper continues as follows. Section 2 explores the main trends and directions in robotics and the metaverse. In Section 3, we propose our comprehensive approach encapsulating both the SLR and the TSA. SLR research results are described in Section 4, and TSA results are described in Section 5. Section 6 discusses the factors to be considered for robot services in virtual environments. Finally, Section 7 concludes this paper with possible directions for future work.

Fig. 1. An example from NVIDIA’s Isaac Sim.
../../Resources/ieie/IEIESPC.2024.13.1.69/fig1.png

2. Concept, Main Trends, and Directions in Robotics and Virtual Space

The concept of virtual space comes from the cyber-physical system, which defines the system between cyberspace and physical space [6]. By constructing these two spaces interconnected under a system shown in Fig. 2, the CPS has both control and access, depending on the direction in which the spaces communicate. Even if both directions are permitted access and control, the direction determines whether these features are either passive or active. For instance, cyberspace can engage in the active control and access from cyber- to physical space, whereas the physical environment can engage in passive control and access from physical spaces to cyberspaces.

In deployment of a real service, cyberspace is implemented and operates on a server that integrates devices, including user interfaces [6]. Note that cyberspace considers the method of virtual space simulation for user access. In other words, CPS-based services and models focus on interaction, integration, simulation, and the platform. From the robotics perspective, safety and assistance with a CPS are other key research topics due to the integration of hardware from the physical space in the virtual space.

Due to the constitution of a CPS, various forms of cyberspace are considered part of the CPS model, including the metaverse, which has shifted the fundamental concepts of other services. For instance, Hyundai Motor Company proposed the Metamobility concept initiative, which interconnects the metaverse, mobility services, and robotics for brand new services and mobility devices, as depicted in Fig. 3 [7].

Recent literature has also suggested that the metaverse can provide significant insight into various research domains, including automotive and mobility research [8]. In their suggestions for the multidimensional service concept and influences on existing fields, the examples of Metamobility and the metaverse in the automotive field and ICT have shown how vital CPS-based robotic systems are, and that significant potential exists in this field.

Fig. 2. A Simple Diagram of a CPS [6].
../../Resources/ieie/IEIESPC.2024.13.1.69/fig2.png
Fig. 3. An Example of Hyundai Metamobility Concepts at CES 2022.
../../Resources/ieie/IEIESPC.2024.13.1.69/fig3.png

3. Comprehensive Approaches: SLR and TSA

This section considers a comprehensive approach with the SLR and standards analysis in parallel. In academic research and reviews of selected topics, the SLR has emerged as a widely known approach to literature review research [9-11]. In our work, the SLR began by setting a research question to answer various aspects encountered in CPS-based robot services. The SLR needed to explicitly explore the issues and problems involved in real-world experiments and implementations based on the perspectives of industry professionals. Moreover, since industry is not just for in-depth academic and technical knowledge, but is also for profit and customer satisfaction, the SLR process must include practicality.

To that end, we adopted the TSA and the SLR in parallel. As shown in Fig. 4, the TSA follows the procedure outlined in steps 1 through 6. First, in steps 1 and 2, the TSA formulates the research question(s) and searches standards documents from leading standards organizations such as the ISO. In Step 3, the selection process is conducted, where standards documents are carefully chosen based on the search results and industry reputation. In steps 4 through 6, we reviewed the selected standards documents and analyzed technical items, main contents, and fundamentals. After the analysis, we found key insights and the technical directions for standardization.

For our TSA process, we initiated the selection within IEEE, ISO, and UL documents owing to their popularity and credibility in the industry as well as the technical feasibility of CPS-based robot services. To visualize the SLR and TSA processes, Fig. 4 shows the step-by-step procedure of each one. Note that both SLR and TSA results are discussed holistically.

Fig. 4. Comprehensive SLR and TSA Approaches in Parallel.
../../Resources/ieie/IEIESPC.2024.13.1.69/fig4.png

4. Systematic Literature Review

4.1 Research Questions

For our SLR, the research questions we set are listed in Table 1. All research questions were selected to investigate trends and insights into using robots or robotic platforms in virtual environments.

Table 1. Research Questions & Goals for the SLR.

Question

Goal

RQ 1. What do the trends of robot services in virtual environments look like?

Identify the research interests of this specific subject.

RQ 2. What types of robots are used frequently in a CPS-based robot service?

Identify the robot types that the majority of research focuses on.

RQ 3. What kinds of research methodologies dominate robot services in virtual environments?

Comprehend the research approach, and find out the maturity in this field.

RQ 4. To what extent are academic studies software-oriented?

Confirm the main dependency of research on HW or SW.

RQ 5. What areas of academic research are dominant?

Identify the research focus and direction.

4.2 Credentials and Requirements

In this subsection, we illustrate the details of the credentials and requirements for each step in the SLR. First, we needed to address the selection of the literature database engine in Step 2 of the SLR. Because of our subject and the relevant technical advancements that have been made, we had to select a literature database with a significant academic reputation in the engineering and technical areas, particularly in the electrical and electronic fields. We also had to consider significant academic publishers on a global scale. After considering those two conditions, we selected the following literature databases: IEEExplore, ACM Digital Library, SpringerLink, and ScienceDirect. Note that all those databases have good reputations in robotics and device research. For Step 3, we considered technical limits of, and constraints on, each search engine, along with relevant policies and regulations dealing with heavy data traffic and API abuses.

Table 2 explores an overview of, and the technical issues in, each database. Common issues with all literature databases are limits on the search list and prevention of repeated access from a specific address. Choosing proper search keywords and minimizing web traffic are significant aspects of SLR automation.

Because we wanted to select primary literature from the collection, we focused on collecting as much subject-related matter as possible. Thus, we used the following search terms:

· (virtual OR VR OR augmented OR AR OR metaverse OR CPS) AND robot AND service AND scenario

This search string was designed to return robot and robot services in virtual environments, including virtual and augmented reality and the metaverse. We also ensured that this string was applied across the literature databases. Regarding web collection in the automation, we followed the terms and conditions of each literature database to utilize the available SDK (IEEE Xplore and ScienceDirect), and manually collected the data if no SDK was available.

For Step 4 in the SLR, we defined our filtering credentials as listed below. Note that for Title Keyword we applied two-step filters, first with (1) and next with (2).

· Range of Years: 2007-2022

· Terms in Publication Names: Engineering, Computer Science, Business, Service, Computing, Information, Communication, Network, Intelligence, Interaction, Artificial, Software, IT, Interface, Application, Tech, Data, Cyber, Scenario

· Exclusions from Publication Name: Pain, Education, Biosys, Aerospace, Thermal, Social, Agricul, Ocean, Nuclear, Culture, Cultural, Safety, Pathology, Petro, Responsible

· Exclusions for Publication Type: Review, Survey, Tutorial, Editorial, Reference, Abstract, Discussion, Communication, Course, Encyclopedia, Book, Chapter, proceeding

· Inclusions in Title Keyword: (1) Robot, robot, Machine, machine, Device, device, (2) Virtual, CPS, Cyber, Augmented, Metaverse, VR, AR

Table 2. Research Questions & Goals for TSA.

Question

Goal

RQ 6. What is the data structure like in the link between virtual and real spaces for robot services?

Identify the data format differentiation between virtual and real spaces.

RQ 7. How can a digital framework contribute to robot services in a virtual space?

Identify a framework through a complex analysis of the virtual elements.

RQ 8. Via what interface can the virtual and physical spaces be integrated?

Identify interface specifications between the virtual and physical worlds.

RQ 9. What would be the safety guidelines for either virtual or physical robots?

Identify considerations when connecting a CPS from robot-related safety requirements for the entire service area.

4.3 SLR Result and Analysis

Our steps in the SLR approach are shown in Fig. 5. Through steps 3, 4, and 5, the number of papers selected from the literature was narrowed down to 36. In Step 5, we reviewed the abstract and main contents of each paper for validation, because several of the papers that remained after Step 4 did not focus their research on CPS-based robot services at all. A list of the 36 papers and their publications is in Table 3. Note that the Title Keyword filtering step eliminated many studies due to the high volume of non-robot or non-device-related services and scenario research in the results.

Table 3. Selected Papers and their Publications.

No

Author

Title

Year

Publication

Type

1

Felipe Carvalho, Alberto Raposo, Ismael Santos, Mauricio Galassi

Virtual reality techniques for planning the offshore robotizing

2014

2014 12th IEEE International Conference on Industrial Informatics

Conference

2

Susanne Stadler, Kevin Kain, Manuel Giuliani, Nicole Mirnig, Gerald Stollnberger, Manfred Tscheligi

Augmented reality for industrial robot programmers: Workload analysis for task-based, augmented reality-supported robot control

2016

2016 25th IEEE International Symposium on Robot and Human Interactive Communication (RO-MAN)

Conference

3

Vincent Autefage, Damien Magoni, John Murphy

Virtualization toolset for emulating mobile devices and networks

2016

MOBILESoft '16: Proceedings of the International Conference on Mobile Software Engineering and Systems

Conference

4

Rui Li, Marc van Almkerk, Sanne van Waveren, Elizabeth Carter, Iolanda Leite

Comparing Human-Robot Proxemics Between Virtual Reality and the Real World

2019

2019 14th ACM/IEEE International Conference on Human-Robot Interaction (HRI)

Conference

5

Hakan GENÇTÜRK, Uğur YAYAN

Development of Augmented Reality Based Mobile Robot Maintenance Software

2019

2019 Innovations in Intelligent Systems and Applications Conference (ASYU)

Conference

6

Alexander Arntz, Sabrina C. Eimler, H. Ulrich Hoppe

"The Robot-Arm Talks Back to Me" - Human Perception of Augmented Human-Robot Collaboration in Virtual Reality

2020

2020 IEEE International Conference on Artificial Intelligence and Virtual Reality (AIVR)

Conference

7

Gustavo Caiza, Pablo Bonilla-Vasconez, Carlos A. Garcia, Marcelo V. Garcia

Augmented Reality for Robot Control in Low-cost Automation Context and IoT

2020

2020 25th IEEE International Conference on Emerging Technologies and Factory Automation (ETFA)

Conference

8

Doris Aschenbrenner, Danielle van Tol, Zoltan Rusak, Claudia Werker

Using Virtual Reality for scenario-based Responsible Research and Innovation approach for Human-Robot Co-production

2020

2020 IEEE International Conference on Artificial Intelligence and Virtual Reality (AIVR)

Conference

9

Gabriele Bolano, Arne Roennau, Ruediger Dillmann

Planning and Evaluation of Robotic Solutions in a Logistic Line Through Augmented Reality

2020

2020 Fourth IEEE International Conference on Robotic Computing (IRC)

Conference

10

Daniel L. Marino, Javier Grandio, Chathurika S. Wickramasinghe, Kyle Schroeder, Keith Bourne, Afroditi V. Filippas, Milos Manic

AI Augmentation for Trustworthy AI: Augmented Robot Teleoperation

2020

2020 13th International Conference on Human System Interaction (HSI)

Conference

11

Oscar A. Aguirre, Juan Carlos Ñacato, Víctor H. Andaluz

Virtual Simulator for Collaborative Tasks of Aerial Manipulator Robots

2020

2020 15th Iberian Conference on Information Systems and Technologies (CISTI)

Conference

12

Alexander Arntz, Agostino Di Dia, Tim Riebner, Sabrina C. Eimler

Machine Learning Concepts for Dual-Arm Robots within Virtual Reality

2021

2021 IEEE International Conference on Artificial Intelligence and Virtual Reality (AIVR)

Conference

13

Khoa C. Hoang, Wesley P. Chan, Steven Lay, Akansel Cosgun, Elizabeth Croft

Virtual Barriers in Augmented Reality for Safe and Effective Human-Robot Cooperation in Manufacturing

2022

2022 31st IEEE International Conference on Robot and Human Interactive Communication (RO-MAN)

Conference

14

Christian Kowalski, Anna Brinkmann, Sandra Hellmers, Conrad Fifelski-von Böhlen, Andreas Hein

Comparison of a VR-based and a rule-based robot control method for assistance in a physical human-robot collaboration scenario

2022

2022 31st IEEE International Conference on Robot and Human Interactive Communication (RO-MAN)

Conference

15

Matt Schmittle, Anna Lukina, Lukas Vacek, Jnaneshwar Das, Christopher P. Buskirk, Stephen Rees, Janos Sztipanovits, Radu Grosu, Vijay Kumar

OpenUAV: a UAV testbed for the CPS and robotics community

2018

ICCPS '18: Proceedings of the 9th ACM/IEEE International Conference on Cyber-Physical Systems

Conference

16

Yuanzhi Cao, Tianyi Wang, Xun Qian, Pawan S. Rao, Manav Wadhawan, Ke Huo, Karthik Ramani

GhostAR: A Time-space Editor for Embodied Authoring of Human-Robot Collaborative Task with Augmented Reality

2019

UIST '19: Proceedings of the 32nd Annual ACM Symposium on User Interface Software and Technology

Conference

17

Uwe Gruenefeld, Lars Prädel, Jannike Illing, Tim Stratmann, Sandra Drolshagen, Max Pfingsthorn

Mind the ARm: real-time visualization of robot motion intent in head-mounted augmented reality

2020

MuC '20: Proceedings of Mensch und Computer 2020

Conference

18

Maya Dimitrova, Aleksandar Krastev, Roman Zahariev, Eleni Vrochidou, Christos Bazinas, Tsvete Yaneva, Elena Blagoeva-Hazarbassanova

Robotic Technology for Inclusive Education: A Cyber-Physical System Approach to Pedagogical Rehabilitation

2020

CompSysTech '20: Proceedings of the 21st International Conference on Computer Systems and Technologies

Conference

19

Lucia Cascone, Aniello Castiglione, Michele Nappi, Fabio Narducci, Ignazio Passero

Waiting for Tactile: Robotic and Virtual Experiences in the Fog

2021

ACM Transactions on Internet Technology (TOIT), Volume 21, Issue 3

Journal

20

Lorenzo Parenti, Serena Marchesi, Marwen Belkaid, Agnieszka Wykowska

Exposure to Robotic Virtual Agent Affects Adoption of Intentional Stance

2021

HAI '21: Proceedings of the 9th International Conference on Human-Agent Interaction

Poster

21

Julius Von Willich, Andrii Matviienko, Sebastian Günther, Max Mühlhäuser

Comparing VR Exploration Support for Ground-Based Rescue Robots

2022

MobileHCI '22: Adjunct Publication of the 24th International Conference on Human-Computer Interaction with Mobile Devices and Services

Conference

22

Olivia Herzog, Niklas Forchhammer, Penny Kong, Philipp Maruhn, Henriette Cornet, Fritz Frenkler

The Influence of Robot Designs on Human Compliance and Emotion: A Virtual Reality Study in the Context of Future Public Transport

2022

ACM Transactions on Human-Robot Interaction (THRI), Volume 11, Issue 2

Journal

23

S. Senthilkumar, P. Suresh

Analysis of Virtual Robot Movements for Performance Improvements in Personal Communication

2016

Wireless Personal Communications

Journal

24

Qingmeng Tan, Yifei Tong, Shaofeng Wu…

Modeling, planning, and scheduling of shop-floor assembly process with dynamic cyber-physical interactions: a case study for CPS-based smart industrial robot production

2019

The International Journal of Advanced Manufacturing Technology

Journal

25

Hwaseop Lee, Yee Yeng Liau, Siku Kim…

Model-Based Human-Robot Collaboration System for Small Batch Assembly with a Virtual Fence

2020

International Journal of Precision Engineering and Manufacturing-Green Technology

Journal

26

Joachim MichniewiczGunther Reinhart

Cyber-physical Robotics - Automated Analysis, Programming and Configuration of Robot Cells based on Cyber-physical-systems

2014

Procedia Technology

Journal

27

Omar AdjaliManolo Dulva HinaAmar Ramdane-Cherif

Multimodal Fusion, Fission and Virtual Reality Simulation for an Ambient Robotic Intelligence

2015

Procedia Computer Science

Journal

28

Biyu ZhuAiguo SongSong Li

Research on 3D Virtual Environment Modeling Technology for Space Tele-robot

2015

Procedia Engineering

Journal

29

Ioan Stefan SacalaMihnea Alexandru MoisescuSimona Iuliana Caramihai

Cyber Physical Systems Oriented Robot Development Platform

2015

Procedia Computer Science

Journal

30

Awais AhmadAnand PaulHangbae Chang

Smart cyber society: Integration of capillary devices with high usability based on Cyber-Physical System

2016

Future Generation Computer Systems

Journal

31

Nikolaos NikolakisVasilis MaratosSotiris Makris

A Cyber Physical System (CPS) approach for safe human-robot collaboration in a shared workplace

2018

Robotics and Computer-Integrated Manufacturing

Journal

32

Romulo Gonçalves LinsPaulo Ricardo Marques de AraujoMarcio Corazzim

In-process machine vision monitoring of tool wear for Cyber-Physical Production Systems

2019

Robotics and Computer-Integrated Manufacturing

Journal

33

Pierre A. Akiki, Paul A. Akiki, Yijun Yu

EUD-MARS: End-user development of model-driven adaptive robotics software systems

2020

Science of Computer Programming

Journal

34

Yong PanChengjun ChenJun Hong

Augmented reality-based robot teleoperation system using RGB-D imaging and attitude teaching device

2021

Robotics and Computer-Integrated Manufacturing

Journal

35

Chengxi LiPai ZhengCarman K. M. Lee

AR-assisted digital twin-enabled robot collaborative manufacturing system with human-in-the-loop

2022

Robotics and Computer-Integrated Manufacturing

Journal

36

David Fernandez-Chaves, Jose-Raul Ruiz-Sarmiento, Javier Gonzalez-Jimenez

Robot@VirtualHome, an ecosystem of virtual environments and tools for realistic indoor robotic simulation

2022

Expert Systems with Applications

Journal

Fig. 5. Steps in the SLR Approach.
../../Resources/ieie/IEIESPC.2024.13.1.69/fig5.png

RQ 1. What do the trends of robot services in virtual environments look like?

We observed a steady increase in the number of publications when we analyzed the selected literature over the years\textcolor{blue}{.} Note that fewer studies were published in 2021 and 2022 than in the previous years due to the impacts of COVID-19. Fig. 6 shows this pattern, indicating the growing interest in robot services in virtual environments.

Fig. 6. Increases in the Number of Publications.
../../Resources/ieie/IEIESPC.2024.13.1.69/fig6.png

RQ 2. What types of robots are used frequently in CPS-based robot service research?

We also paid significant attention to the robot types in the research; for virtual reality, researchers aligned with their studies robot types based on mobile or industrial (i.e., fixed) status [12]. According to Fig. 7, in mobile robots, legged robots are of main interest in the research, but they attracted less interest than the other two. Also, "Other" robot types or "No specific type" were often considered because some robot research focuses were decoupled from the physical size or type of robot.

Fig. 7. Trends in the Types of Robots.
../../Resources/ieie/IEIESPC.2024.13.1.69/fig7.png

RQ 3. What kinds of research methodologies dominate robot services in virtual environments?

As shown in Fig. 8, most research on virtual space robot services has followed primary approaches: quantitative, qualitative, and conceptual. Within the quantitative approach, we also identified inferential, simulation, and experimental categories [13]. Since CPS-based robot services are recent topics of interest, current research approaches are more geared toward conceptual and simulation approaches than other approaches, particularly as seen in the number of publications on those types, which increased from 2020 to 2022. Note that inferential approaches, including case studies, were relatively few due to a scarcity of real cases. By contrast, the qualitative approach, which includes user interviews, has been slowly increasing in popularity for human-robot interaction purposes.

Fig. 8. Trends in Research Approaches.
../../Resources/ieie/IEIESPC.2024.13.1.69/fig8.png

RQ 4. To what extent are academic studies software-oriented?

Fig. 9 confirms that most research papers considered both hardware and software. By contrast, only a small portion focused on software, creating software-based structures and designs such as a virtual environment and a CPS-based robot platform software architecture. Note that there are no ``HW only'' papers due to our research focus (i.e., papers had to include software).

Fig. 9. Trends in Software-oriented Research.
../../Resources/ieie/IEIESPC.2024.13.1.69/fig9.png

RQ 5. What areas of academic research are dominant?

Fig. 10 shows the overall results of the research focuses listed in Section 2. The dominant research areas are human-robot interaction, integration, and simulators. In other words, researchers concentrated on the human experience and interaction with robots in both physical and virtual spaces. Note that safety and assistive areas were less dominant than the others, given their importance and significance.

Fig. 10. Trends in Research Focus.
../../Resources/ieie/IEIESPC.2024.13.1.69/fig10.png

5. Technical Standards Analysis

This section describes the analysis conducted and insights gained into the technical standards associated with robots in virtual space. Note that this formal process involves thoroughly analyzing and investigating each standards document. We secured standards documents, meeting records, and memorandums, and searched the key index for rapid analysis. Technical standards have the advantages of saving resources, building technologies from the ground, and adopting recent advancements [14]. They also provided our studies' research direction, roadmap, and practical requirements. Thus, we approached the TSA procedure in a manner as identical as possible to the approach we used for SLRs shown in Fig. 4. One exception is that for credibility, we researched the existing literature to identify the necessary standards documents instead of searching for documents with an internet search engine [15-17].

5.1 Research Questions

Following the steps shown in Fig. 4, our research questions for the TSA were as follows. All questions were for trends and insights into robots or robotic platforms in virtual environments.

5.2 The Selected Standards

For this paper, we searched for several standards documents. Given the good reputation and wide range of industry adaptations, we selected primary standards from three organizations: the International Standards Organization (ISO), the IEEE, and Underwriter Laboratory (UL). These are well-known standards organizations for various fields, including electronics, wireless communications, and safety. In the selection process, we explored the following technical standards areas and identified the following standards documents.

· Technical & Interface (RQ 6 & RQ 8):

ISO/IEC 23005: Information technology-media context and control

IEEE P2888: Interfacing Cyber and Physical World

· Service & Operation (RQ 7):

ISO 23247: Automation systems and integration-Digital twin framework for manufacturing

· Safety (RQ 9):

UL 3300: Outline of Investigation Helps Advance Safety of Consumer, Service, and Education Robots

Note that IEEE P2888 is currently at the working group level and is still in progress. UL 3300 is also continually updating documents.

5.3 Main Contents and Focus

Since we selected four standards, we describe each, and provide our research findings for each RQ.

RQ 6. What is the data structure like in the link between virtual and real spaces for robot services?

ISO 23005 is the next generation of MPEG-V media context and control standards [18]. In other words, future media include interactions between the physical and virtual worlds via ISO standards, which define the basic data information and format for transmission between real and virtual worlds. Regarding our specific subject matter, this standard is suitable for CPS-based robot services since it deals with virtual space.

As shown in Fig. 11, we can identify the information structure for access and control of real robots in a virtual world. For example, if we want to send physical robot information to a virtual world, we must format the access information and scope for transmission. Since this standard also considers the transmission between two virtual worlds, the same data formatting can be applied, but it must be a unanimous decision made under consensus. Although there is no known commercial case of communication between virtual or metaverse universes, communications between Roblox and Decentraland-as an assumption-could be a possible example of transceiving information between two virtual worlds [18]. The primary definitions of the information include control, sensory effects, virtual world objects, and interaction-device-specific data format [18].

Fig. 11. The Basic Structure Interconnecting Physical and Virtual Worlds [18].
../../Resources/ieie/IEIESPC.2024.13.1.69/fig11.png

RQ 7. How can a digital framework contribute to robot services in virtual space?

ISO 23247 starts from the idea of automated operation of the manufacturing process in a digital twin [19]. The main inspiration is that each process, entity, and element can be defined within the digital twin framework and can be created to be observable. The primary reference models are based on the entity and the domain, as shown in Fig. 12. In Fig. 12, manufacturing the digital twin framework is considered part of the entity's task range. Each domain is defined as a high-level system and a sub-system for entity cases. In the framework, three elements (user, digital twin, and device communication) are connected, and their outcome is linked to the manufacturing domain.

Fig. 12. Basic Structure of the Domain Level of a Digital Twin Framework [19].
../../Resources/ieie/IEIESPC.2024.13.1.69/fig12.png

In detail, the standards define the basic principles and requirements of applying digital twins within the manufacturing sector, and they reference architecture models for domain and entity. They also explain the basic properties of observable manufacturing elements for identifiers, characteristics, schedules, status, locations, reports, and relationships with technical requirements of information sharing between entities from the reference model.

RQ 8. Via what interface can the virtual and physical spaces be integrated?

IEEE P2888 defines interface specifications for devices and contents in cyber- and physical spaces [20]. Aside from ISO 23005 containing data parameters and formatting, this standard covers the overall interfaces based on applications and scenarios. The IEEE Computer Society sponsors it, and ongoing working groups have been formed. The main active specifications in progress are sensors, actuator interfaces, digital synchronization, and holographic visualization. Note that IEEE P2888 standards documents show the main interfaces between spaces include sensors, actuators, digital synchronization, and holographic contents.

RQ 9. What would be the safety guidelines for either virtual or physical robots?

UL 3300 defines the standards for functional safety of service, communication, information, education, and entertainment robots [21]. It focuses on robot safety, and contains a wide range of topics such as operation types, the range of robot types, electric power, and sizes [21]. Although it does not directly discuss robotic services in the virtual world, the standards still consider a hardware robot's safety risk to always be present. The standards were released in 2021, but updates and amendments to the specifications are ongoing, and an updated version may soon be released.

5.4 Summary of the TSA

The overall results of the TSA are shown in Fig. 13, which identifies the scope and areas for each robot and virtual space-related standards specification. We recognize the main goals, and focus on certain items besides the SLR results. We also determine the current progress and future direction of each standard.

Fig. 13. Overview Diagram of Robot-related Technical Standards.
../../Resources/ieie/IEIESPC.2024.13.1.69/fig13.png

In particular, each standard contains the potential necessary components of robot-virtual space integration, such as data format, process, interface, and safety. In other words, it is crucial to carefully consider various perspectives to develop even a simple robot–virtual-space service.

The TSA also provided in-depth insights that allowed us to realize we should consider various aspects, including communication links, the framework, and safety guidelines to prevent potential issues from arising. In particular, the results show that CPS-based robots must consider interfaces between the two spaces to provide users with comprehensive experiences with robot services. Even if robot services solely operate in the metaverse or a virtual space setting, we must consider the possibility that hardware robotic devices can be integrated into those environments as extensive services in which industry standards must be followed.

6. Main Considerations for Robot Services

This paper analyzed the CPS and convergence of virtual and physical worlds, along with SLR research conducted mainly on academic papers plus industrial standards that reflect the requirements of manufacturers and users. Research has shown that the directions embodied in academics and the situations in which many HW manufacturers have expressed their opinions on the technology are highly related to a CPS in the industry standards. Therefore, the following three considerations should be noted when developing research related to CPS-based robot services.

First is the purpose of a CPS-based robot service. With an explicit definition and design for its purpose, a CPS-based robot service can increasingly be perceived as a feasible item. The acceptance of robots and related technologies is already high in industry, and the adoption rate for robots is also increasing in service industries. Therefore, in this context, where the adoption of robots in the physical world is expanding, it is necessary to clearly explain the incentives to use robots in the virtual world to create value and relate them to the physical world. Since a robot costs more than a smartphone, a CPS-based robot service can reduce such capital expenditures and can become a catalyst for creating new added value.

Another consideration is the CPS-based robot service model. Robots have emerged as a vehicle for realizing business models that are not businesses themselves. They are expected to serve as main channels to collect user data while performing services, and implementing predefined services with minimal errors. In this way, even if various technologies are grafted into a CPS and robots, the structure and substance of the targeted service will always be the critical success factor in clarifying service outcomes.

Finally, it is necessary to consider the infrastructure and ecosystem of the robotics industry. For example, the explosive supply of automobiles was not limited to the automobile itself but also involved the expansion of gas stations followed by road infrastructure and oil refinery technology. This proves the ecosystem's importance. When robots become universal, ecosystems-for example, third parties such as Apple's App Store-will emerge. Therefore, defining the prerequisites for CPS robot services is necessary in order to achieve growth and to gain a broad customer base. For example, ultra-low latency is an essential precondition for an authentic connection. Sensor and motor technologies are also essential to secure highly accurate control. Lastly, market maturity level is the main factor in generalizing these technologies.

7. Concluding Remarks

This paper used both SLR and TSA to investigate CPS-based robot services. First, our main contribution is an approach that identifies new technology and service areas that can be used to design and develop robots with low cost and high degrees of freedom. Our approach also focused on academic research and industry trends via literature reviews and standards documents. Finally, we also provided suggestions for future work and directions for robotics in virtual environments. This paper proves the possibility of constructing a new business ecosystem by combining the concept research of the CPS with hardware and software technology in robotics. Therefore, there is a need for further research from both perspectives to prove the business feasibility.

Our work needs to extend the business model for future work since the business model has become the focal point for running a service. Analysis should also be conducted on whether CPS-based robot services can define specific business customers and targets, and whether they can establish customized profit and operation models. Data collected through new hardware, such as that featured in the early days of smartphones, proves the potential value of the market that physical robots will create within the CPS concept, just as the growth of the cloud and network technologies led to the emergence of the digital goods market. The framework of this study can also be appreciated for its validity.

We also need to exploit the feasibility and applicability of a service by implementing a virtual environment and real hardware robots. In particular, it is necessary to analyze feasibility through an advanced network environment, artificial intelligence, and robot control technology to see if transactions in the CPS virtual world can technically add value to the physical world.

ACKNOWLEDGMENTS

This research was supported by Hyundai NGV and National Research Foundation Korea (NRF) grant funded by the Ministry of Science and ICT (MSIT) of Korea, grant number 2021R1F1A1063690.

REFERENCES

1 
Li, Shufei, Pai Zheng, Sichao Liu, Zuoxu Wang, Xi Vincent Wang, Lianyu Zheng, and Lihui Wang. "Proactive human-robot collaboration: Mutual-cognitive, predictable, and self-organising perspectives." Robotics and Computer-Integrated Manufacturing 81 (2023): 102510.DOI
2 
Pedersen, Mikkel Rath, Lazaros Nalpantidis, Rasmus Skovgaard Andersen, Casper Schou, Simon Bøgh, Volker Krüger, and Ole Madsen. "Robot skills for manufacturing: From concept to industrial deployment." Robotics and Computer-Integrated Manufacturing 37 (2016): 282-291.DOI
3 
Metaverse, AI, And Robots: 5 Tech Trends That Defined The Year 2022, India Times, Dec 2022,URL
4 
Omniverse Platform Overview, NVIDIA,URL
5 
Kitchenham, Barbara. "Procedures for performing systematic reviews." Keele, UK, Keele University 33, no. 2004 (2004): 1-26.URL
6 
Akanmu, Abiola A., Chimay J. Anumba, and Omobolanle O. Ogunseiju. ``Towards next generation cyber-physical systems and digital twins for construction.'' J. Inf. Technol. Constr. 26, pp. 505-525, 2021.URL
7 
Hyundai Motor Shares Vision of New Metamobility Concept, ‘Expanding Human Reach’ through Robotics & Metaverse at CES 2022, Press Release, Hyundai Motor Group, 2022,URL
8 
Kim, Joon Young, and Jeong Min Oh. "Opportunities and Challenges of Metaverse for Automotive and Mobility Industries." In 2022 13th International Conference on Information and Communication Technology Convergence (ICTC), pp. 113-117. IEEE, 2022.DOI
9 
den, J.; Eikebrokk, T. R. "Implementing IT Service Management: A systematic literature review." Int. J. of Info. Management. 2013, 33, 512-523.DOI
10 
Turner, M.; Kitchenham, B.; Brereton, P.; Charters, S.; Budgen, D. "Does the technology acceptance model predict actual use? A systematic literature review." Info. and Software Tech., 2010, 52, 463-479.DOI
11 
Tian, Wulin, and Carlo Caponecchia. "Using the functional resonance analysis method (FRAM) in aviation safety: a systematic review." Journal of Advanced Transportation 2020 (2020): 1-14.DOI
12 
Ben-Ari, Mordechai, Francesco Mondada, Mordechai Ben-Ari, and Francesco Mondada. "Robots and their applications." Elements of robotics (2018): 1-20.DOI
13 
Kothari, Chakravanti Rajagopalachari. Research methodology: Methods and techniques. New Age International, 2004.URL
14 
Role of standards: A guide for small and medium-sized enterprises, working paper, United Nations Industrial Development Organization, 2006.URL
15 
Wang, Yuntao, Zhou Su, Ning Zhang, Rui Xing, Dongxiao Liu, Tom H. Luan, and Xuemin Shen. "A survey on metaverse: Fundamentals, security, and privacy." IEEE Communications Surveys & Tutorials (2022).DOI
16 
Shao, Guodong. "Use case scenarios for digital twin implementation based on iso 23247." National Institute of Standards: Gaithersburg, MD, USA (2021).URL
17 
Robo Business announces full conference agenda, The Robot Report, Sep. 2022DOI
18 
ISO/IEC 23005-1:2020 Information technology - Media context and control - Part 1: Architecture, 2022, ISOURL
19 
ISO 23247-2:2021, Automation systems and integration - Digital twin framework for manufacturing - Part 2: Reference architecture, 2021, ISOURL
20 
Yoon, Kyoungro, Sang-Kyun Kim, Sangkwon Peter Jeong, and Jeong-Hwan Choi. "Interfacing cyber and physical worlds: Introduction to IEEE 2888 standards." In 2021 IEEE International Conference on Intelligent Reality (ICIR), pp. 49-50. IEEE, 2021.DOI
21 
UL 3300, Outline of Investigation Helps Advance Safety of Consumer, Service and Education Robots, 2021, UL StandardsURL
Soomin Shim
../../Resources/ieie/IEIESPC.2024.13.1.69/au1.png

Soomin Shim received his B.A. and M.A. in Business Administration from Indiana University, Bloomington in 2008, and 2010. He is currently a senior research engineer at Robotics Lab, Hyundai Motor Company. Before joining Hyundai, he was with KT Economics & Management Research Institute and LG Electronics. His research interests include robot business, robot-friendtly building infrastructure, technical standardization and marketing.

Joon Young Kim
../../Resources/ieie/IEIESPC.2024.13.1.69/au2.png

Joon Young Kim received the Bachelor of Science and Ph.D. degrees from the School of Electrical and Computer Engineering at Purdue University, West Layette in 2010 and 2015. He is currently an assistant professor at the School of Artificial Intelligence (AI) Convergence, Sungshin Women’s University. Before joining Sungshin Women’s University, he was with Hyundai Motor Company as a senior research engineer in South Korea. His research interests include spectrum analysis, Software Defined Radio (SDR)-based implementation, robotics and virtual reality platform integration, vehicle communication, infotainment system, and mobility service.

Seong Wook Hwang
../../Resources/ieie/IEIESPC.2024.13.1.69/au3.png

Seong Wook Hwang received his PhD in Industrial Engineering and Operations Research from the Pennsylvania State University, University Park, PA, USA. He is currently an Assistant Professor of the Department of Business Admini-stration at Sungshin Women’s University, Seoul, South Korea. He has performed research in service and transportation operations management and location-allocation problems for alternative vehicles. His current active research includes electric vehicle routing problems in the smart grid for future energy systems applications.

Jeong Min Oh
../../Resources/ieie/IEIESPC.2024.13.1.69/au4.png

Jeong Min Oh is currently pursuing B.S in Artificial Intelligence and Intelligent IoT at Sungshin Women’s University, Korea since 2021. Her current research interests include AI system, machine learning-based emotion recognition, and IoT platform.

Bo Kyung Kim
../../Resources/ieie/IEIESPC.2024.13.1.69/au5.png

Bo Kyung Kim is currently pursuing B.A. in Business Administration and Big Data Science at Sungshin Women’s University, Korea since 2021. Her current research interests include business analysis, and service scenario.

Ju Hee Park
../../Resources/ieie/IEIESPC.2024.13.1.69/au6.png

Ju Hee Park received the Bachelor's degree in electronic engineering from Ajou University, South Korea in 2015. She is currently a robotics strategy engineer at Robotics Lab, Hyundai Motor Company. Before joining Hyundai, she worked at Renault Samsung Motors as project manager, engine diagnosis SW engineer and HMI validation leader from 2015 to 2021. Her research interests include Service Robot Commercialization and Robot Product & Service Planning.

Dong Jin Hyun
../../Resources/ieie/IEIESPC.2024.13.1.69/au7.png

Dong Jin Hyun received the B.S. degree from the School of Mechanical and Aerospace Engineering, Seoul National University, Seoul, South Korea, in 2006, the M.S. degree in mechanical engineering from the University of Michigan, Ann Arbor, MI, USA, in 2007, and the Ph.D. degree in mechanical engineering from the University of California, Berkeley, CA, USA, in 2012.,He was a Post-Doctoral Associate in mechanical engineering, Massachusetts Institute of Technology (MIT), Cambridge, MA, USA, in 2013. Since 2013, he has been associated with the Hyundai Motor Company, Uiwang, Gyeonggi, South Korea. He is presently serving as a Vice President at the Hyundai Motor Company, and the Head of Hyundai Motors’ Robotics Lab. His research interests include dynamics and control of legged locomotion, human-robot interaction for wearable robots, and biomechanics.

Hee Sang Lee
../../Resources/ieie/IEIESPC.2024.13.1.69/au8.png

Hee Sang Lee received the Ph.D. degree in industrial and systems engineering from the Georgia Institute of Technology, Atlanta, GA, USA. He is currently a Professor with the Department of Systems Management Engineering and the Graduate School of Management of Technology, Sungkyunkwan University. His current research interests include innovation management, strategic technology management, and management science. He has published articles in several journals in his field of research, including Management Science, European Journal of Operational Research, and Scientometrics. He has participated in different national and international congresses, such as International Conference on Management of Technology, and the Institute for Operations Research and the Management Sciences.