Deformable Object Manipulation
This page collects my work on the manipulation deformable objects, from cable perception and grasping to current research on mobile manipulators and co-manipulation. The most recent activities are connected to the SiMOD project.
Mobile manipulation of deformable objects
Project Overview
Within the SiMOD project, I work on perception and manipulation pipelines for deformable objects using mobile manipulators.
One representative setup combines a UR5e arm mounted on a Neobotix MPO-500 mobile base, with an Intel RealSense D435i installed near the gripper. This enables target detection, environment perception, grasp planning, and execution in cluttered and partially occluded scenarios.
My role
This software work is primarily mine, excluding third-party perception models.
- Perception and manipulation pipeline integration
- Grasp execution and task-level software logic
- System testing in simulation and on the real platform
- Extension toward physically interactive manipulation behaviors with force feedback
Technical focus
- Deformable object detection and localization
- RGB-D perception for target estimation
- Grasping of bags and “soft” objects
- Reactive manipulation under interaction constraints
- Integration between perception, motion, and execution control
Co-manipulation and interaction strategy
A key aspect of this line of work is co-manipulation of large deformable objects with multiple manipulators.
The control logic is not just about tracking a shared path. One robot acts as the leader and autonomously decides the reference motion, while the second robot follows through force sensing and passive force-control behavior. This makes the system more tolerant to uncertainty in object deformation and interaction constraints, and it is especially useful when geometric shape alone is not enough to coordinate the task reliably.
Related public posts
DLO perception and grasping foundations
Project Overview
This work started from my internship and master’s thesis and laid the foundations for later manipulation projects.
I developed a ROS-based pipeline for handling deformable linear objects such as cables and ropes on planar surfaces. The workflow combines image-based detection, depth-based pose reconstruction, grasp execution, and validation in simulation and on the real system.
What I developed
- ROS software pipeline for the full manipulation workflow
- Integration between perception, 3D estimation, and manipulator execution
- Simulation in CoppeliaSim for validation of grasp configurations
- Real-world grasping tests on cable-like objects
Method summary
- RGB image processing for DLO detection
- 3D pose reconstruction using stereo or depth information
- Grasp and move routine for deformable linear objects on planar surfaces
- Sim-to-real comparison between modeled and physical behavior
Note on perception models
The cable segmentation module was based on state-of-the-art deformable linear object perception methods rather than a perception network developed by me. References are below.
Tools and platforms across this line of work
- ROS1 and ROS2
- CoppeliaSim
- RGB-D perception
- Universal Robots manipulators
- Mobile bases and integrated mobile manipulators
- Force-aware interaction control