top of page
outcome_edited_edited.jpg

Human-Robot Collaborative Brick Stacking System

07.2023 - Stuttgart, Germany

Integrative Technologies and Architectural Design Research (ITECH)

Master of Science

Seminar Project (Human Robot Collaboration)
 

ABSTRACT

ITECH_officialLogo_edited_edited_edited_

This project aims to equip robots with vision capabilities, enabling real-time responses to their environment. This development has significant potential in architecture and construction, where uncertainties often present major challenges. The research explores the integration of computer vision, ROS2, and robotic manipulation to enable a collaborative system where a human and a robotic arm work together to complete a brick stacking task. The goal is to allow a human to place bricks freely, and have the robot perceive the placement and autonomously continue the stacking operation.

#Linux, #ROS2, #C++, #Python, #RealSense, #ComputerVision, #ArUcoMarkers #RaspberryPi, #TCP / UDP, #Grasshopper

hrc workflow_edited.jpg
human robot collaboration system architecture.JPG
robosight.gif

System Architecture

The system is divided into three main modules:

  1. Perception Module

    • A RealSense depth camera captures RGB-D images of the working area.

    • Each brick is tagged with an ArUco marker to facilitate detection and pose estimation.

    • The system uses OpenCV's ArUco library to detect markers and estimate their 6-DOF poses relative to the camera.

  2. Transform & Calibration Module

    • ROS2's tf2 is used to define the transform tree from the camera frame to the robot's base frame.

    • A static transform is published between the camera and the robot using tf2_static_transform_publisher.

    • Optional hand-eye calibration can be conducted to enhance precision.

    • With the full transform chain (marker -> camera_link -> base_link), the system calculates the marker's pose relative to the robot.

  3. Motion Planning & Control Module

    • The marker's position in the robot's base frame is used as the target pose for the end effector.

    • MoveIt2 is used to perform inverse kinematics and motion planning.

    • Collision avoidance, workspace constraints, and pose feasibility are considered in the planning phase.

    • Final trajectories are sent to the robot via FollowJointTrajectory interface in ROS2.

bottom of page