Our goal

A robot arm you can instruct in plain language, one that picks, clears and hands over everyday objects reliably, in clutter and next to people, without retraining for every new task.

We build towards this in layers. Fast, reliable grasps make it possible to work in clutter. Working in clutter and around people makes real deployments possible. Language and memory then let the robot take on whole tasks, not just single picks.

Open-vocabulary, language-driven grasp planning on our UR5e testbed (2× speed).
  1. 01GraspWhere to hold an object
  2. 02ClearHow to reach it in clutter
  3. 03ShareWorking beside people
  4. 04ReasonUnderstanding the whole task
01

Grasp

Reliable grasps, in milliseconds

Everything else depends on a grasp that works the first time. Sampling-based and end-to-end learned grasp planners are often slow, hard to repeat and fragile outside the lab. We design grasp planners that are fast, explainable and suited to the gripper actually on the robot.

QuickGrasp: segmented surfaces and selected antipodal grasp points on a bottle and a banana point cloud
QuickGrasp finds grasp points directly on segmented object surfaces.

QuickGrasp: lightweight antipodal grasp planning

Instead of searching the full 6-DoF space of gripper poses, QuickGrasp estimates grasp points on the object surface. A soft region-growing step segments even curved surfaces into planes, and an optimisation-based quality metric enforces indirect force closure. It outperformed the widely used Grasp Pose Detection (GPD) baseline in simulation and on a UR5 with a Robotiq gripper.

QuickGrasp antipodal grasp planning demo

ICRA 2025 Paper · arXiv

Learned three-finger grasp planning, executed on the UR5e (2× speed).

Grasp planning for underactuated three-finger grippers

Most learned grasp planners assume a two-finger parallel-jaw gripper. We introduced an SE(3) + triangle grasp representation, in which each vertex of a triangle is a fingertip, and trained a network to predict three-finger grasps directly from RGB-D images. Grasp quality matched the GraspIt! planner, and the predictions transferred from simulation to the real robot.

20 s → 15 ms grasp inference compared with GraspIt!

Learned three-finger grasp planning
Three-finger grasp planning with GraspIt!

J. Intell. Robot. Syst. 2025 Paper · Indian patent application 202441057093

Slip correction: as the bottle fills and gets heavier, the gripper detects slip at the fingertips and tightens its grasp (2× speed).

Holding on: force control and slip correction

Heavy-duty grippers like our three-finger Robotiq use non-backdrivable worm gears. They can only cap motor current, so grip force jumps suddenly and cannot be eased off. We added force-sensing pads to each fingertip and closed the loop on fingertip force. This lets the robot hold a soft sponge ball without crushing it.

The same sensors detect slip: a sudden spike in the force gradient (more than eight times the local average) signals that the object is moving, and the gripper tightens until it stops.

IEEE IICAIET 2024 Paper

02

Clear

Reaching a target buried in clutter

In practice the object a robot needs is rarely sitting alone. Getting to it means deciding which other objects to move, and in what order, without toppling the pile.

ClutterNav: gradient-guided search for 3D clutter removal
UR5e with three-fingered gripper above a pile of white blocks hiding a black target block
Simulated clutter, height map, point cloud and per-object segmentation

ClutterNav: deciding what to remove next

ClutterNav treats clutter removal as a sequential decision problem. A removability critic, learned from demonstrations, estimates how much the pile would be disturbed by removing each object. Integrated gradients then estimate how each object affects access to the target. Balancing the two, the robot picks the next best object to remove and updates its view of the scene after every removal.

Near human-like removal sequences, in real time, with no hand-written heuristics

ICRAE 2026 arXiv

03

Share

Working safely beside people

Factories, labs and recycling lines are shared spaces. A useful manipulator has to keep working while people move around it, rather than stopping every time someone comes close.

Real-time perception and motion planning for human-robot collaboration

User-study trial: the UR5e re-plans around a person working at the same table (left), with the robot's view of the scene (right).

Real-time perception and planning for a shared workspace

We fuse live human tracking with occupancy-grid motion planning on a UR5e. As the person moves, the robot re-plans its trajectory around them and carries on with its task, avoiding collisions without halting.

87% fewer collisions in a 16-person user study (mean 4.44 → 0.56 per trial)

A person and a UR5e arm working at the same table

ICRM 2025 Paper

Proactive collaboration on a packaging task: the robot tracks the person's progress and does its share at the right moment (2× speed).

From avoiding people to anticipating them

Avoiding collisions is only the start. In a collaborative packaging task, the robot tracks the person's arm in 3D, estimates how far through each step they are, and predicts what will be needed next. An agent then splits the task between person and robot, so the robot picks and places its items as the person finishes theirs, instead of waiting to be told. When something goes wrong, the system detects it and asks the operator how to recover.

Ongoing work with Accenture Labs

04

Reason

From a single instruction to a whole task

The last layer turns a grasping system into an assistant. The robot must work out what an instruction means for the objects in front of it, then carry a multi-step task through to the end, even when it cannot see everything.

UR5e arm with a three-finger gripper and RGB-D camera above a table of objects and a bin
Real-robot setup for long-horizon, language-driven pick-and-place.

Open-vocabulary, language-driven grasp planning

Combining open-vocabulary image segmentation with LLM reasoning lets the robot work out what to grasp, and how, from a natural-language request. It needs no fixed list of object categories, and it can explain its choices. The clip at the top of this page shows it running.

Language-driven grasp planning with open-vocabulary segmentation and an LLM

IEEE STPEC 2025 Paper

RB-VLA: vision-language-action with memory

Today's vision-language-action models react to what they currently see and lose track of progress in long tasks. RB-VLA keeps a compact belief about task history and object interactions. The vision-language model is queried once per task for intent, and a diffusion policy acts on intent and belief together.

  • +52.5% success on multi-stage pick-and-place vs. π0
  • +37.5% success on stacking vs. π0
  • Up to 5× lower inference latency

IROS 2026 arXiv

RB-VLA architecture: a reasoning VLM produces intent, a recursive belief estimator tracks task state, and a diffusion model outputs robot actions
RB-VLA: the VLM reasons once at the start, the recursive belief tracks task progress, and a diffusion policy produces actions.

Applications and supporting work

Human-robot battery pack disassembly (3× speed).

Application

Lithium-ion battery recycling

Spent lithium-ion battery packs have to be taken apart before their materials can be recovered. This is slow, varied and hazardous work for people. Our semi-autonomous system shares the job: an operator handles steps that need judgement, and the robot grasps, cuts and extracts cells and electrode stacks. It brings together the grasping and shared-workspace work above.

ICEMPS 2024 Paper · Indian patent application 202341051709

Robotic rebar placement

Measuring how hard rebar placement is for a robot's motion planner, in terms of geometry, space and computation, as a step towards automating construction sites.

ISARC 2026 Paper

Motion planning for redundant arms

Voxel-based A* search combined with redundant inverse-kinematics solutions for optimal manipulator motion.

CK 2025 Paper

Accenture

This research is supported by the Accenture – IIT Madras Centre of Excellence.