Gagan Khandate

gagank at cs dot columbia dot edu

I am a PhD student Computer Science at Columbia University . Fortunate to be advised by Matei Ciocarlie my research primarily focuses on the question of "How can we get our robots to be as dexterous as humans in the real world?".
To this end, I have worked on developing reinforcement learning methods for highly dexterous manipulation. Recently, I'm interested in achieving diverse dexterous skills from videos of humans performing such tasks.

Previously, I worked as a Research Engineer at Systemantics where I primarily developed controls, kinematics and motion-planning firmware for serial and parallel collaborative industrial manipulators. I received my master's from Columbia University in Mechanical Engineering with a focus on Robotics and recieved my bachelor's from Indian Institute of Technology Madras also in Mechanical Engineering with a minor in micro-electronics.

GitHub  /  Google Scholar  /  LinkedIn

profile photo


  • NEW! [2024-1] Our work on Value Guided Exploration has been accepted at ICRA 2024
  • NEW! [2023-10] Our robot hand demonstrating dexterity with tactile sensing selected to TIME Magazine - Best Inventions of 2023 list.
  • [2023-04] Our work on dexterous manipulation with tactile sensing featured on Columbia Engineering News. See article .
  • [2023-04] Excited to share that our work on sampling-based exploration for learning dexterous manipulation skills is accepted to RSS 2023.
  • [2023-01] I will be interning at Meta Reality Labs for the upcoming summer.
  • [2022-05] Excited to be spending summer at Amazon Robotics AI, Boston.
  • [2022-02] Paper on in-hand manipulation accepted to ICRA 2022
  • [2021-05] Invited talk on using tactile sensing for in-hand manipulation at ViTac Workshop in ICRA 2021
  • [2021-03] Guest lecture on Model Predictive Control in our course on Robot Learning
  • [2020-05] Paper on snake locomotion accepted to ICRA 2020
  • [2019-11] Lecture discussing a seminal paper on vision based grasping in our course on robot learning.


project image

RxR: Rapid eXploration for Reinforcement Learning via Sampling-based Reset Distributions and Imitation Pre-training

Gagan Khandate*, Tristan Luca Saidi*, Siqi Shang*, Eric Chang, Johnson Adams, Matei Ciocarlie
Submitted to Autonomous Robots - RSS 202 Special Issue, 2024
arxiv / website /

We present a method for enabling Reinforcement Learning of motor control policies for complex skills such as dexterous manipulation. We posit that a key difficulty for training such policies is the difficulty of exploring the problem state space, as the accessible and useful regions of this space form a complex structure along manifolds of the original high-dimensional state space. This work presents a method to enable and support exploration with Sampling-based Planning. We use a generally applicable non-holonomic Rapidly-exploring Random Trees algorithm and present multiple methods to use the resulting structure to bootstrap model-free Reinforcement Learning. Our method is effective at learning various challenging dexterous motor control skills of higher difficulty than previously shown. In particular, we achieve dexterous in-hand manipulation of complex objects while simultaneously securing the object without the use of passive support surfaces. These policies also transfer effectively to real robots. A number of example videos can also be found on the project website:


project image

Value Guided Exploration with Sub-optimal Controllers for Learning Dexterous Manipulation

Gagan Khandate*, Cameron Mehlman*, Xingsheng Wei*, Matei Ciocarlie
International Conference on Intelligent Robots and Systems 2024, 2023
arxiv / website /

Recently, reinforcement learning has allowed dexterous manipulation skills with increasing complexity. Nonetheless, learning these skills in simulation still exhibits poor sample-efficiency which stems from the fact these skills are learned from scratch without the benefit of any domain expertise. In this work, we aim to improve the sample-efficiency of learning dexterous in-hand manipulation skills using sub-optimal controllers available via domain knowledge. Our framework optimally queries the sub-optimal controllers and guides exploration toward state-space relevant to the task thereby demonstrating improved sample complexity. We show that our framework allows learning from highly sub-optimal controllers and we are the first to demonstrate learning hard-to-explore finger-gaiting in-hand manipulation skills without the use of an exploratory reset distribution.

project image

Sampling-based Exploration for Reinforcement Learning of Dexterous Manipulation

Gagan Khandate*, Siqi Shang*, Eric Chang, Tristan Luca Saidi, Johnson Adams, Matei Ciocarlie
Robotics: Science & Systems, RSS, 2023
arxiv / website /

In this paper, we present a novel method for achieving dexterous manipulation of complex objects, while simultaneously securing the object without the use of passive support surfaces. We posit that a key difficulty for training such policies in a Reinforcement Learning framework is the difficulty of exploring the problem state space, as the accessible regions of this space form a complex structure along manifolds of a high-dimensional space. To address this challenge, we use two versions of the non-holonomic Rapidly-Exploring Random Trees algorithm; one version is more general, but requires explicit use of the environment’s transition function, while the second version uses manipulation-specific kinematic constraints to attain better sample efficiency. In both cases, we use states found via sampling-based exploration to generate reset distributions that enable training control policies under full dynamic constraints via model-free Reinforcement Learning. We show that these policies are effective at manipulation problems of higher difficulty than previously shown, and also transfer effectively to real robots.

project image

On Feasibility of Learning Finger-gaiting In-hand Manipulation using Intrinsic Sensing

Gagan Khandate, Maximillian Haas-Heger, Matei Ciocarlie
IEEE International Conference on Robotics and Automation (ICRA), 2022
arxiv / video / website /

In this work, we use model-free reinforcement learning (RL) to learn finger-gaiting only via precision grasps and demonstrate finger-gaiting for rotation about an axis purely using on-board proprioceptive and tactile feedback. To tackle the inherent instability of precision grasping, we propose the use of initial state distributions that enable effective exploration of the state space.

project image

Automatic snake gait generation using model predictive control

Emily Hannigan, Bing Song, Gagan Khandate, Maxmillian Haas-Heger, Ji Yin, Matei Ciocarlie
IEEE International Conference on Robotics and Automation (ICRA), 2020
arxiv / video /

In this paper, we use Model Predictive Control (MPC) with iLQR to automatically generate effective locomotion gaits via trajectory optimization. An important advantage of our method is that it can be applied to generate gaits under different environment dynamics such as dry friction or viscous friction while also enabling obstacle avoidance.

project image

SBP-Guided MPC to Overcome Local Minima in Trajectory Planning

Emily Hannigan, Bing Song, Gagan Khandate, Maxmillian Haas-Heger, Ji Yin, Matei Ciocarlie
Optimal Planning and Control: Fusing Offline and Online Algorithms Workshop, ICRA, 2019
arxiv / video /

Non-linearity of robot dynamics can cause failure of trajectory optimization methods (ex. iLQG) due to local optima. On the other hand, Sampling Based Planning (SBP) methods such as Rapidly-exploring Random Trees (RRT) are inherently robust to presence of local optima but often generate in-efficient trajectories. We propose combining these two classes of methods to retain the strengths of each. We use RRT trajectory as initialization for iLQG and overcome local optima while producing an efficient trajectory.

project image

Algorithmic Gait Synthesis for a Snake Robot

Gagan Khandate, Emily Hannigan, Maximilian Haas-Heger, Bing Song, Ji Yin, Matei Ciocarlie
Toward Online Optimal Control of Dynamic Robots: From Algorithmic Advances to Field Applications Workshop, ICRA, 2019
arxiv /

In this work, we study the use of deep reinforcement learning for control of snake robots. While prior work on control of snake robots primarily uses open loop control backed by sinusoidal gaits (serpenoid curves), we demonstrate the use of deep reinforcement learning (PPO) for generating snake gaits under different environments. We also compare our method with other classes - model predictive control (MPC) and sampling based planning (SPB). Our results show that the gaits generated by model-free deep reinforcement learning are comparable (sometimes better) to MPC in terms of efficiency and energy comsumption.