Datacast Episode 33: Domain Randomization in Robotics with Josh Tobin
The 33rd episode of Datacast is my conversation with Josh Tobin - the founder of a stealth machine learning startup and previously deep learning & robotics researcher at OpenAI. Give it a listen to hear about his background in mathematics, his time as a CS Ph.D. student at UC Berkeley, his wide-ranging research in robotics at OpenAI, the importance of working on the right problems for junior researchers, his initiative with Full Stack Deep Learning, and much more.
Josh Tobin is the founder and CEO of a stealth machine learning startup. Previously, Josh worked as a deep learning & robotics researcher at OpenAI and as a management consultant at McKinsey. He is also the creator of Full Stack Deep Learning, the first course focused on the emerging engineering discipline of production machine learning. Josh did his Ph.D. in Computer Science at UC Berkeley, advised by Pieter Abbeel.
Show Notes
(2:02) Josh studied Mathematics at Columbia University during his undergraduate and explained why he was not set out for a career as a mathematician.
(3:55) Josh then worked for two years as a Management Consultant at McKinsey.
(6:05) Josh explained his decision to go back to graduate school and pursue a Ph.D. in Mathematics at UC Berkeley.
(7:23) Josh shared the anecdote of taking a robotics class with professor Pieter Abbeel and switching to a Ph.D. in the Computer Science department at UC Berkeley.
(8:50) Josh described the period where he learned programming to make the transition from Math to Computer Science.
(10:46) Josh talked about the opportunity to collaborate and then work full-time as a Research Scientist at OpenAI - all during his Ph.D.
(12:40) Josh discussed the sim2real problem, as well as the experiments conducted in his first major work "Transfer from Simulation to Real World through Learning Deep Inverse Dynamics Model".
(17:43) Josh discussed his paper "Domain Randomization for Transferring Deep Neural Networks from Simulation to the Real World", which has been cited more than 600 times up until now.
(20:51) Josh unpacked the OpenAI’s robotics system that was trained entirely in simulation and deployed on a physical robot, which can learn a new task after seeing it done once (Read the blog post “Robots That Learn” and watch the corresponding video).
(24:01) Josh went over his work on Hindsight Experience Replay - a novel technique that can deal with sparse and binary rewards in Reinforcement Learning (Read the blog post “Generalizing From Simulation").
(28:41) Josh talked about the paper "Domain Randomization and Generative Models for Robotic Grasping”, which (1) explores a novel data generation pipeline for training a deep neural network to perform grasp planning that applies the idea of domain randomization to object synthesis; and (2) proposes an autoregressive grasp planning model that maps sensor inputs of a scene to a probability distribution over possible grasps.
(32:27) Josh unpacked the design of OpenAI's Dactyl - a reinforcement learning system that can manipulate objects using a Shadow Dexterous Hand (Read the paper “Learning Dexterous In-Hand Manipulation” and watch the corresponding video).
(35:31) Josh reflected on his time at OpenAI.
(36:05) Josh investigated his most recent work called “Geometry-Aware Neural Rendering” - which tackles the neural rendering problem of understanding the 3D structure of the world implicitly.
(28:21) Check out Josh's talk "Synthetic Data Will Help Computer Vision Make the Jump to the Real World" at the 2018 LDV Vision Summit in New York.
(28:55) Josh summarized the mental decision tree to debug and improve the performance of neural networks, as a reference to his talk "Troubleshooting Deep Neural Networks” at Reinforce Conf 2019 in Budapest.
(41:25) Josh discussed the limitations of domain randomization and what the solutions could look like, as a reference to his talk "Beyond Domain Randomization” at the 2019 Sim2Real workshop in Freiburg.
(44:52) Josh emphasized the importance of working on the right problems and focusing on the core principles in machine learning for junior researchers who want to make a dent in the AI research community.
(48:30) Josh is a co-organizer of Full-Stack Deep Learning, a training program for engineers to learn about production-ready deep learning.
(50:40) Closing segment.
His Contact Information
His Recommended Resources
“Thinking Fast and Slow” by Daniel Kahneman