Loading...
Object locating dataset for human-robot collaboration
Taylor, Devin McCarthy, Mark Kalithasan, Kubeshavarsha Gracia, Fabian
Taylor, Devin
McCarthy, Mark
Kalithasan, Kubeshavarsha
Gracia, Fabian
Citations
Altmetric:
Files
Loading...
DevinTaylorPoster.pdf
Adobe PDF, 397.66 KB
Loading...
DTaylorAbstract.pdf
Adobe PDF, 103.49 KB
Other Names
Location
Time Period
Advisors
Original Date
Digitization Date
Issue Date
2025
Type
Abstract
Poster
Poster
Genre
Keywords
Human-robot collaboration,Spatial relationships,Object detection,Robotics
Subjects (LCSH)
Citation
Taylor, D., McCarthy, M., Kalithasan, K., Gracia, F., Dao, M., & Yan, F. Object locating dataset for human-robot collaboration. -- FYRE in STEM Showcase, 2025.
Abstract
Humans and robots interact more closely as time marches forward. Whether it is general AI or any other learning module, robots need a database in which new information is processed, learned, and built upon. In particular, the future of human-robot collaboration (HRC) depends on computational understanding of spatial relationships. This study allows for more natural commands during such interactions, like “to the right of” instead of exact coordinates, and improves reasoning for unfamiliar objects. To build a dataset to apply these algorithms, we have chosen 56 household objects that can be sorted into various categories of affordance, the possible actions that can be done to an object for a desired usage outcome. Affordance is helpful when the robot is dealing with unfamiliar objects. We took various images, where each image sets a unique scene of multiple objects to be used as a framework for assigning spatial relationships. The analysis includes three main phases. First, the red-green-blue (RGB) and depth components of these images are used in the Faster Region-based Convolutional Neural Network (RCNN) for object detection, which outputs both objects’ bounding boxes and labels. Secondly, the bounding box coordinates are passed to a connected Bayesian Neural Networks (BNN) to classify the spatial relationships of said objects. Finally, these relations are integrated with a robot through a large language model (LLM) to create fluid communication between the user and the robotic companion. This has applications in the kitchen, offices, and general spaces, where one can ask, “Grab the ___; it is next to the ___” and the robot will understand exactly one’s intention. Using techniques like BNNs and LLMs, we push boundaries of robotic spatial understanding through deep learning.
Table of Contents
Description
Poster and abstract presented at the FYRE in STEM Showcase, 2025.
Research project completed at the Department of Mathematics, Statistics and Physics.
Research project completed at the Department of Mathematics, Statistics and Physics.
Publisher
Wichita State University
Journal
Book Title
Series
FYRE in STEM 2025