Gagan Khandategagank at cs dot columbia dot edu
I am a PhD student Computer Science at Columbia University . Fortunate to be advised by Matei Ciocarlie my research primarily focuses on the question of "How can we get our robots to be as dexterous as humans in the real world?". Previously, I worked as a Research Engineer at Systemantics where I primarily developed controls, kinematics and motion-planning firmware for serial and parallel collaborative industrial manipulators. I received my master's from Columbia University in Mechanical Engineering with a focus on Robotics and recieved my bachelor's from Indian Institute of Technology Madras also in Mechanical Engineering with a minor in micro-electronics. |
|
News
|
Pre-prints
|
|
RxR: Rapid eXploration for Reinforcement Learning via Sampling-based Reset Distributions and Imitation Pre-trainingGagan Khandate*, Tristan Luca Saidi*, Siqi Shang*, Eric Chang, Johnson Adams, Matei Ciocarlie Submitted to Autonomous Robots - RSS 202 Special Issue, 2024 arxiv / website / We present a method for enabling Reinforcement Learning of motor control policies for complex skills such as dexterous manipulation. We posit that a key difficulty for training such policies is the difficulty of exploring the problem state space, as the accessible and useful regions of this space form a complex structure along manifolds of the original high-dimensional state space. This work presents a method to enable and support exploration with Sampling-based Planning. We use a generally applicable non-holonomic Rapidly-exploring Random Trees algorithm and present multiple methods to use the resulting structure to bootstrap model-free Reinforcement Learning. Our method is effective at learning various challenging dexterous motor control skills of higher difficulty than previously shown. In particular, we achieve dexterous in-hand manipulation of complex objects while simultaneously securing the object without the use of passive support surfaces. These policies also transfer effectively to real robots. A number of example videos can also be found on the project website: https://sbrl.cs.columbia.edu |
Publications
|
|
Value Guided Exploration with Sub-optimal Controllers for Learning Dexterous ManipulationGagan Khandate*, Cameron Mehlman*, Xingsheng Wei*, Matei Ciocarlie International Conference on Intelligent Robots and Systems 2024, 2023 arxiv / website / Recently, reinforcement learning has allowed dexterous manipulation skills with increasing complexity. Nonetheless, learning these skills in simulation still exhibits poor sample-efficiency which stems from the fact these skills are learned from scratch without the benefit of any domain expertise. In this work, we aim to improve the sample-efficiency of learning dexterous in-hand manipulation skills using sub-optimal controllers available via domain knowledge. Our framework optimally queries the sub-optimal controllers and guides exploration toward state-space relevant to the task thereby demonstrating improved sample complexity. We show that our framework allows learning from highly sub-optimal controllers and we are the first to demonstrate learning hard-to-explore finger-gaiting in-hand manipulation skills without the use of an exploratory reset distribution. |
|
Sampling-based Exploration for Reinforcement Learning of Dexterous ManipulationGagan Khandate*, Siqi Shang*, Eric Chang, Tristan Luca Saidi, Johnson Adams, Matei Ciocarlie Robotics: Science & Systems, RSS, 2023 arxiv / website / In this paper, we present a novel method for achieving dexterous manipulation of complex objects, while simultaneously securing the object without the use of passive support surfaces. We posit that a key difficulty for training such policies in a Reinforcement Learning framework is the difficulty of exploring the problem state space, as the accessible regions of this space form a complex structure along manifolds of a high-dimensional space. To address this challenge, we use two versions of the non-holonomic Rapidly-Exploring Random Trees algorithm; one version is more general, but requires explicit use of the environment’s transition function, while the second version uses manipulation-specific kinematic constraints to attain better sample efficiency. In both cases, we use states found via sampling-based exploration to generate reset distributions that enable training control policies under full dynamic constraints via model-free Reinforcement Learning. We show that these policies are effective at manipulation problems of higher difficulty than previously shown, and also transfer effectively to real robots. |
|
On Feasibility of Learning Finger-gaiting In-hand Manipulation using Intrinsic SensingGagan Khandate, Maximillian Haas-Heger, Matei Ciocarlie IEEE International Conference on Robotics and Automation (ICRA), 2022 arxiv / video / website / In this work, we use model-free reinforcement learning (RL) to learn finger-gaiting only via precision grasps and demonstrate finger-gaiting for rotation about an axis purely using on-board proprioceptive and tactile feedback. To tackle the inherent instability of precision grasping, we propose the use of initial state distributions that enable effective exploration of the state space. |
|
Automatic snake gait generation using model predictive controlEmily Hannigan, Bing Song, Gagan Khandate, Maxmillian Haas-Heger, Ji Yin, Matei Ciocarlie IEEE International Conference on Robotics and Automation (ICRA), 2020 arxiv / video / In this paper, we use Model Predictive Control (MPC) with iLQR to automatically generate effective locomotion gaits via trajectory optimization. An important advantage of our method is that it can be applied to generate gaits under different environment dynamics such as dry friction or viscous friction while also enabling obstacle avoidance. |
|
SBP-Guided MPC to Overcome Local Minima in Trajectory PlanningEmily Hannigan, Bing Song, Gagan Khandate, Maxmillian Haas-Heger, Ji Yin, Matei Ciocarlie Optimal Planning and Control: Fusing Offline and Online Algorithms Workshop, ICRA, 2019 arxiv / video / Non-linearity of robot dynamics can cause failure of trajectory optimization methods (ex. iLQG) due to local optima. On the other hand, Sampling Based Planning (SBP) methods such as Rapidly-exploring Random Trees (RRT) are inherently robust to presence of local optima but often generate in-efficient trajectories. We propose combining these two classes of methods to retain the strengths of each. We use RRT trajectory as initialization for iLQG and overcome local optima while producing an efficient trajectory. |
|
Algorithmic Gait Synthesis for a Snake RobotGagan Khandate, Emily Hannigan, Maximilian Haas-Heger, Bing Song, Ji Yin, Matei Ciocarlie Toward Online Optimal Control of Dynamic Robots: From Algorithmic Advances to Field Applications Workshop, ICRA, 2019 arxiv / In this work, we study the use of deep reinforcement learning for control of snake robots. While prior work on control of snake robots primarily uses open loop control backed by sinusoidal gaits (serpenoid curves), we demonstrate the use of deep reinforcement learning (PPO) for generating snake gaits under different environments. We also compare our method with other classes - model predictive control (MPC) and sampling based planning (SPB). Our results show that the gaits generated by model-free deep reinforcement learning are comparable (sometimes better) to MPC in terms of efficiency and energy comsumption. |