Deep Learning Approach for Enhanced Object Recognition and Assembly Guidance with Augmented Reality

Boon Giin Lee, Xiaoying Wang, Renzhi Han, Linjing Sun, Matthew Pike, Wan Young Chung

Research output: Chapter in Book/Conference proceedingConference contributionpeer-review

Abstract

In an effort to enhance the efficiency and precision of manual part assembly in industrial settings, the development of software for assembly guidance becomes imperative. Augmented reality (AR) technology offers a means to provide visual instructions for assembly tasks, rendering the guidance more comprehensible. Nevertheless, a significant challenge lies in the technology’s limited object detection capabilities, especially when distinguishing between similar assembled parts. This project proposes the utilization of deep learning neural networks to enhance the accuracy of object recognition within the AR guided assembly application. To achieve this objective, a dataset of assembly parts, known as the Visual Object Classes (VOC) dataset, was created. Data augmentation techniques were employed to expand this dataset, incorporating scale HSV (hue saturation value) transformations. Subsequently, deep learning models for the recognition of assembly parts were developed which were based on the Single Shot Multibox Detector (SSD) and the YOLOv7 detector. The models were trained and fine-tuned, targeting on the variations of the positions of detected parts. The effectiveness of this approach was evaluated using a case study involving an educational electronic blocks circuit science kit. The results demonstrated a high assembly part recognition accuracy of over 99% in mean average precision (MAP), along with favorable user testing outcomes. Consequently, the AR application was capable of offering high-quality guidance to users which holds promise for application in diverse scenarios and the resolution of real-world challenges.

Original languageEnglish
Title of host publicationIntelligent Human Computer Interaction - 15th International Conference, IHCI 2023, Revised Selected Papers
EditorsBong Jun Choi, Dhananjay Singh, Uma Shanker Tiwary, Wan-Young Chung
PublisherSpringer Science and Business Media Deutschland GmbH
Pages105-114
Number of pages10
ISBN (Print)9783031538292
DOIs
Publication statusPublished - 2024
Event15th International Conference on Intelligent Human Computer Interaction, IHCI 2023 - Daegu, Korea, Republic of
Duration: 8 Nov 202310 Nov 2023

Publication series

NameLecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)
Volume14532 LNCS
ISSN (Print)0302-9743
ISSN (Electronic)1611-3349

Conference

Conference15th International Conference on Intelligent Human Computer Interaction, IHCI 2023
Country/TerritoryKorea, Republic of
CityDaegu
Period8/11/2310/11/23

Keywords

  • Assembly Tasks
  • Augmented Reality
  • Object Detection
  • Object Recognition

ASJC Scopus subject areas

  • Theoretical Computer Science
  • General Computer Science

Fingerprint

Dive into the research topics of 'Deep Learning Approach for Enhanced Object Recognition and Assembly Guidance with Augmented Reality'. Together they form a unique fingerprint.

Cite this