By Julia Schwarz
Ben Eysenbach, an expert in reinforcement learning, has joined the computer science faculty as an assistant professor.
Reinforcement learning is a type of machine learning that uses rewards to teach an autonomous agent — think of a robot or an AI chatbot — to master tasks without detailed instructions. Imagine asking a robot to clean your room. Should it start by picking up clothes, cleaning windows or making the bed? With reinforcement learning, the robot interacts with its environment and learns from feedback to make smart decisions, just like a human would.
Reinforcement learning can even reveal new solutions that humans hadn’t considered, according to Eysenbach. Playing games like chess or Go, for example, reinforcement learning methods frequently beat human opponents.
“At a high level, reinforcement learning is about discovering new strategies,” said Eysenbach.
Eysenbach’s interest in reinforcement learning developed after he first did work in computer vision as an undergraduate at MIT. He ended up somewhat frustrated, he said, because while computer vision detects things in the world, it doesn’t tell us what to do with that information. “It can tell you whether it looks like rain,” he said, “but not whether you should wear a raincoat.”
While working at Google Research after graduating from MIT, he discovered that reinforcement learning could answer some of his questions. “Reinforcement learning,” he said, “allows you to not just understand the world, but also figure out how to make decisions.”
Much of his work, Eysenbach said, is drawing connections between different areas of machine learning and reinforcement learning with the goal of designing better, simpler methods. For example, his doctoral dissertation at Carnegie Mellon borrowed ideas from information theory and probabilistic modeling and applied them to reinforcement learning to simplify how human users teach autonomous agents.
Eysenbach focused on a specific type of reinforcement learning which uses data rather than rewards. So instead of having human users provide feedback through rewards, Eysenbach’s algorithms ask humans to provide a single image that shows the desired outcome. If you want a robot to set the dining room table, for example, you can simply show it a picture of how you want the table to look. This method is already common in other areas of machine learning, according to Eysenbach. Applying it to reinforcement learning makes the task of teaching a robot much simpler for human users.
Eysenbach joined the Princeton faculty in September 2023 after completing his doctoral degree. In spring 2024 he co-created and co-taught an introduction to reinforcement learning class with Mengdi Wang, associate professor of electrical and computer engineering. The subject, he said, hadn’t been offered in the computer science department for several years. And in fall 2024 he will teach an independent work seminar on unsupervised reinforcement learning.
Looking ahead, Eysenbach is eager to scale up reinforcement learning methods, enabling autonomous agents to tackle ever more complex tasks. “How can we design reinforcement learning algorithms that can discover how to do everything and build anything?” he asked.