Q&A: Boston Dynamics on Atlas’s Parkour Skills

Yesterday’s video from Boston Dynamics showing a pair of Atlas robots doing parkour together is already up to nearly 3 million views, and for good reason. The company continues to push forward the state of the art for dynamic bipedal…

Yesterday’s video from Boston Dynamics showing a pair of Atlas robots doing parkour together is already up to nearly 3 million views, and for good reason. The company continues to push forward the state of the art for dynamic bipedal robots, now by mixing in perception as well as upper-body maneuvers that humanoid robots find particularly challenging. A behind-the-scenes video and blog post provided an uncharacteristic amount of detail about the process that Boston Dynamics goes through to make videos like these, but we still had questions. And happily, Boston Dynamics had answers!

Here’s the new Atlas parkour video, if you missed our post yesterday: For more details from the experts, we spoke with Scott Kuindersma, the Atlas team lead at Boston Dynamics, and Ben Stephens, the Atlas controls lead, via email. IEEE Spectrum: Can you describe some of the constraints that Atlas is operating under, and how brittle its behaviors are? For example, can it handle changes in friction, and can it adapt autonomously if different sequences of movements are required? Scott Kuindersma and Ben Stephens: The ability to adapt behaviors to a range of circumstances is a key design principle for Atlas, so for an activity like parkour, we frequently test the robot by making changes to the geometry of the course. Atlas is also able to deal with things like feet sliding to some extent. We run subsets of these behaviors on wood, mats, asphalt, grass, and surfaces with grip texture without explicitly telling the robot that the friction and ground compliances are different. But there are of course limits—parkour on ice probably wouldn’t work. (Spot, which is used in a wide range of commercial environments, has more explicit mechanisms for detecting slip events and automatically changing its control response to cope with different types of surfaces). Atlas’ control system also provides some flexibility in reordering move sequences, whether these sequences are provided ahead of time (as was the case here) or if they are generated online as the output of a planning process. The idea behind Atlas’ behavior libraries is that they can be reused in new environments. Spectrum: It’s very impressive to see Atlas using more upper body for dynamic maneuvers. To what extent will Atlas continue to use human-ish motion for dynamic mobility, as opposed to motions that could be more optimized for unique robotic capabilities? Kuindersma and Stephens: We’re interested in creating behaviors that take full advantage of the hardware even if the resulting motion is not perfectly humanlike. That said, the incredible breadth and quality of human motion remains a source of inspiration for us, particularly in cases like parkour where the coordination and athleticism on display motivates useful hardware and software innovation. Spectrum: You mentioned in your blog post that the robot has no spine or shoulder blades, which places some limitations on what it can do. After several iterations of Atlas, how much bioinspired design do you think is the right amount? Kuindersma and Stephens: When building robots like Atlas, there’s always a long list of engineering tradeoffs that shape the final design. The current robot has evolved over several generations of humanoids at Boston Dynamics and represents a good tradeoff between size, range of motion, and strength-to-weight ratio. When our work identifies physical limits of the machine, that becomes useful information to our design team. In some cases, limitations can be improved through incremental upgrades. But for new robot designs, we have to make strategic decisions about how the limitations of the current machine conflict with what we want the robot to do over the next few years. These decisions are primarily motivated by our technical goals and experimental analyses and less so by human performance data. Finding and operating at the limits of the robot hardware is part of the motivation for doing things like parkour. Spectrum: Last we heard, Atlas was not using machine learning in these contexts. When you’re teaching Atlas new behaviors, how exactly do you do that? Kuindersma and Stephens: The behaviors Atlas performs during parkour can be expressed as optimization problems that compute strategies for coordinating forces and motion over time. We use optimization both to design the behaviors in Atlas’ library offline and to adapt and execute them online. This programming strategy works well when you can describe what you want as a tractable optimization problem, but not all tasks are like that. For example, machine learning becomes an essential tool for programming behavior in cases where detailed solutions are hard to write down (e.g., vision-dominant manipulation tasks). We’re excited about opportunities to solve problems by leveraging the strengths of both approaches going forward. Spectrum: At this point, is Atlas more constrained by hardware or software? If you want Atlas to do something new, what draws the line between impossible and not? Kuindersma and Stephens: Finding and operating at the limits of the robot hardware is part of the motivation for doing things like parkour. But if we consider a longer term vision for what we want robots like Atlas to do, there is a lot of opportunity for software innovation using the existing hardware. We will continue to improve on both fronts. Over the past seven years, Atlas’ behavior has evolved from walking up stairs and moving cardboard boxes to the running, flipping, and dancing you see today. We’re excited to see where the next seven years will take us. Atlas humanoid robots Boston Dynamics Evan Ackerman Evan Ackerman is a senior editor at IEEE Spectrum. Since 2007, he has written over 6,000 articles on robotics and technology. He has a degree in Martian geology and is excellent at playing bagpipes. Type Feature Topic Artificial Intelligence Magazine Fast, Efficient Neural Networks Copy Dragonfly Brains An insect-inspired AI could make missile-defense systems more nimble Frances Chance 30 Jul 2021 12 min read Getty Images/Richard Penska/500px In each of our brains, 86 billion neurons work in parallel, processing inputs from senses and memories to produce the many feats of human cognition. The brains of other creatures are less broadly capable, but those animals often exhibit innate aptitudes for particular tasks, abilities honed by millions of years of evolution. Most of us have seen animals doing clever things. Perhaps your house pet is an escape artist. Maybe you live near the migration path of birds or butterflies and celebrate their annual return. Or perhaps you have marveled at the seeming single-mindedness with which ants invade your pantry Looking to such specialized nervous systems as a model for artificial intelligence may prove just as valuable, if not more so, than studying the human brain. Consider the brains of those ants in your pantry. Each has some 250,000 neurons. Larger insects have closer to 1 million. In my research at Sandia National Laboratories in Albuquerque, I study the brains of one of these larger insects, the dragonfly. I and my colleagues at Sandia, a national-security laboratory, hope to take advantage of these insects’ specializations to design computing systems optimized for tasks like intercepting an incoming missile or following an odor plume. By harnessing the speed, simplicity, and efficiency of the dragonfly nervous system, we aim to design computers that perform these functions faster and at a fraction of the power that conventional systems consume. Looking to a dragonfly as a harbinger of future computer systems may seem counterintuitive. The developments in artificial intelligence and machine learning that make news are typically algorithms that mimic human intelligence or even surpass people’s abilities. Neural networks can already perform as well—if not better—than people at some specific tasks, such as detecting cancer in medical scans. And the potential of these neural networks stretches far beyond visual processing. The computer program AlphaZero, trained by self-play, is the best Go player in the world. Its sibling AI, AlphaStar, ranks among the best Starcraft II players. Such feats, however, come at a cost. Developing these sophisticated systems requires massive amounts of processing power, generally available only to select institutions with the fastest supercomputers and the resources to support them. And the energy cost is off-putting. Recent estimates suggest that the carbon emissions resulting from developing and training a natural-language processing algorithm are greater than those produced by four cars over their lifetimes. It takes the dragonfly only about 50 milliseconds to begin to respond to a prey’s maneuver. If we assume 10 ms for cells in the eye to detect and transmit information about the prey, and another 5 ms for muscles to start producing force, this leaves only 35 ms for the neural circuitry to make its calculations. Given that it typically takes a single neuron at least 10 ms to integrate inputs, the underlying neural network can be at least three layers deep. But does an artificial neural network really need to be large and complex to be useful? I believe it doesn’t. To reap the benefits of neural-inspired computers in the near term, we must strike a balance between simplicity and sophistication. Which brings me back to the dragonfly, an animal with a brain that may provide precisely the right balance for certain applications. If you have ever encountered a dragonfly, you already know how fast these beautiful creatures can zoom, and you’ve seen their incredible agility in the air. Maybe less obvious from casual observation is their excellent hunting ability: Dragonflies successfully capture up to 95 percent of the prey they pursue, eating hundreds of mosquitoes in a day. The physical prowess of the dragonfly has certainly not gone unnoticed. For decades, U.S. agencies have experimented with using dragonfly-inspired designs for surveillance drones. Now it is time to turn our attention to the brain that controls this tiny hunting machine. While dragonflies may not be able to play strategic games like Go, a dragonfly does demonstrate a form of strategy in the way it aims ahead of its prey’s current location to intercept its dinner. This takes calculations performed extremely fast—it typically takes a dragonfly just 50 milliseconds to start turning in response to a prey’s maneuver. It does this while tracking the angle between its head and its body, so that it knows which wings to flap faster to turn ahead of the prey. And it also tracks its own movements, because as the dragonfly turns, the prey will also appear to move. The model dragonfly reorients in response to the prey’s turning. The smaller black circle is the dragonfly’s head, held at its initial position. The solid black line indicates the direction of the dragonfly’s flight; the dotted blue lines are the plane of the model dragonfly’s eye. The red star is the prey’s position relative to the dragonfly, with the dotted red line indicating the dragonfly’s line of sight. So the dragonfly’s brain is performing a remarkable feat, given that the time needed for a single neuron to add up all its inputs—called its membrane time constant—exceeds 10 milliseconds. If you factor in time for the eye to process visual information and for the muscles to produce the force needed to move, there’s really only time for three, maybe four, layers of neurons, in sequence, to add up their inputs and pass on information Could I build a neural network that works like the dragonfly interception system? I also wondered about uses for such a neural-inspired interception system. Being at Sandia, I immediately considered defense applications, such as missile defense, imagining missiles of the future with onboard systems designed to rapidly calculate interception trajectories without affecting a missile’s weight or power consumption. But there are civilian applications as well. For example, the algorithms that control self-driving cars might be made more efficient, no longer requiring a trunkful of computing equipment. If a dragonfly-inspired system can perform the calculations to plot an interception trajectory, perhaps autonomous drones could use it to avoid collisions. And if a computer could be made the same size as a dragonfly brain (about 6 cubic millimeters), perhaps insect repellent and mosquito netting will one day become a thing of the past, replaced by tiny insect-zapping drones! To begin to answer these questions, I created a simple neural network to stand in for the dragonfly’s nervous system and used it to calculate the turns that a dragonfly makes to capture prey. My three-layer neural network exists as a software simulation. Initially, I worked in Matlab simply because that was the coding environment I was already using. I have since ported the model to Python. Because dragonflies have to see their prey to capture it, I started by simulating a simplified version of the dragonfly’s eyes, capturing the minimum detail required for tracking prey. Although dragonflies have two eyes, it’s generally accepted that they do not use stereoscopic depth perception to estimate distance to their prey. In my model, I did not model both eyes. Nor did I try to match the resolution of a dragonfly eye. Instead, the first layer of the neural network includes 441 neurons that represent input from the eyes, each describing a specific region of the visual field—these regions are tiled to form a 21-by-21-neuron array that covers the dragonfly’s field of view. As the dragonfly turns, the location of the prey’s image in the dragonfly’s field of view changes. The dragonfly calculates turns required to align the prey’s image with one (or a few, if the prey is large enough) of these “eye” neurons. A second set of 441 neurons, also in the first layer of the network, tells the dragonfly which eye neurons should be aligned with the prey’s image, that is, where the prey should be within its field of view. The model dragonfly engages its prey. Processing—the calculations that take input describing the movement of an object across the field of vision and turn it into instructions about which direction the dragonfly needs to turn—happens between the first and third layers of my artificial neural network. In this second layer, I used an array of 194,481 (214) neurons, likely much larger than the number of neurons used by a dragonfly for this task. I precalculated the weights of the connections between all the neurons into the network. While these weights could be learned with enough time, there is an advantage to “learning” through evolution and preprogrammed neural network architectures. Once it comes out of its nymph stage as a winged adult (technically referred to as a teneral), the dragonfly does not have a parent to feed it or show it how to hunt. The dragonfly is in a vulnerable state and getting used to a new body—it would be disadvantageous to have to figure out a hunting strategy at the same time. I set the weights of the network to allow the model dragonfly to calculate the correct turns to intercept its prey from incoming visual information. What turns are those? Well, if a dragonfly wants to catch a mosquito that’s crossing its path, it can’t just aim at the mosquito. To borrow from what hockey player Wayne Gretsky once said about pucks, the dragonfly has to aim for where the mosquito is going to be. You might think that following Gretsky’s advice would require a complex algorithm, but in fact the strategy is quite simple: All the dragonfly needs to do is to maintain a constant angle between its line of sight with its lunch and a fixed reference direction. Readers who have any experience piloting boats will understand why that is. They know to get worried when the angle between the line of sight to another boat and a reference direction (for example due north) remains constant, because they are on a collision course. Mariners have long avoided steering such a course, known as parallel navigation, to avoid collisions These three heat maps show the activity patterns of neurons at the same moment; the first set represents the eye, the second represents those neurons that specify which eye neurons to align with the prey’s image, and the third represents those that output motor commands. Translated to dragonflies, which want to collide with their prey, the prescription is simple: keep the line of sight to your prey constant relative to some external reference. However, this task is not necessarily trivial for a dragonfly as it swoops and turns, collecting its meals. The dragonfly does not have an internal gyroscope (that we know of) that will maintain a constant orientation and provide a reference regardless of how the dragonfly turns. Nor does it have a magnetic compass that will always point north. In my simplified simulation of dragonfly hunting, the dragonfly turns to align the prey’s image with a specific location on its eye, but it needs to calculate what that location should be. The third and final layer of my simulated neural network is the motor-command layer. The outputs of the neurons in this layer are high-level instructions for the dragonfly’s muscles, telling the dragonfly in which direction to turn. The dragonfly also uses the output of this layer to predict the effect of its own maneuvers on the location of the prey’s image in its field of view and updates that projected location accordingly. This updating allows the dragonfly to hold the line of sight to its prey steady, relative to the external world, as it approaches. It is possible that biological dragonflies have evolved additional tools to help with the calculations needed for this prediction. For example, dragonflies have specialized sensors that measure body rotations during flight as well as head rotations relative to the body—if these sensors are fast enough, the dragonfly could calculate the effect of its movements on the prey’s image directly from the sensor outputs or use one method to cross-check the other. I did not consider this possibility in my simulation. To test this three-layer neural network, I simulated a dragonfly and its prey, moving at the same speed through three-dimensional space. As they do so my modeled neural-network brain “sees” the prey, calculates where to point to keep the image of the prey at a constant angle, and sends the appropriate instructions to the muscles. I was able to show that this simple model of a dragonfly’s brain can indeed successfully intercept other bugs, even prey traveling along curved or semi-random trajectories. The simulated dragonfly does not quite achieve the success rate of the biological dragonfly, but it also does not have all the advantages (for example, impressive flying speed) for which dragonflies are known. More work is needed to determine whether this neural network is really incorporating all the secrets of the dragonfly’s brain. Researchers at the Howard Hughes Medical Institute’s Janelia Research Campus, in Virginia, have developed tiny backpacks for dragonflies that can measure electrical signals from a dragonfly’s nervous system while it is in flight and transmit these data for analysis. The backpacks are small enough not to distract the dragonfly from the hunt. Similarly, neuroscientists can also record signals from individual neurons in the dragonfly’s brain while the insect is held motionless but made to think it’s moving by presenting it with the appropriate visual cues, creating a dragonfly-scale virtual reality. Data from these systems allows neuroscientists to validate dragonfly-brain models by comparing their activity with activity patterns of biological neurons in an active dragonfly. While we cannot yet directly measure individual connections between neurons in the dragonfly brain, I and my collaborators will be able to infer whether the dragonfly’s nervous system is making calculations similar to those predicted by my artificial neural network. That will help determine whether connections in the dragonfly brain resemble my precalculated weights in the neural network. We will inevitably find ways in which our model differs from the actual dragonfly brain. Perhaps these differences will provide clues to the shortcuts that the dragonfly brain takes to speed up its calculations. This backpack that captures signals from electrodes inserted in a dragonfly’s brain was created by Anthony Leonardo, a group leader at Janelia Research Campus.Anthony Leonardo/Janelia Research Campus/HHMI Dragonflies could also teach us how to implement “attention” on a computer. You likely know what it feels like when your brain is at full attention, completely in the zone, focused on one task to the point that other distractions seem to fade away. A dragonfly can likewise focus its attention. Its nervous system turns up the volume on responses to particular, presumably selected, targets, even when other potential prey are visible in the same field of view. It makes sense that once a dragonfly has decided to pursue a particular prey, it should change targets only if it has failed to capture its first choice. (In other words, using parallel navigation to catch a meal is not useful if you are easily distracted.) Even if we end up discovering that the dragonfly mechanisms for directing attention are less sophisticated than those people use to focus in the middle of a crowded coffee shop, it’s possible that a simpler but lower-power mechanism will prove advantageous for next-generation algorithms and computer systems by offering efficient ways to discard irrelevant inputs The advantages of studying the dragonfly brain do not end with new algorithms; they also can affect systems design. Dragonfly eyes are fast, operating at the equivalent of 200 frames per second: That’s several times the speed of human vision. But their spatial resolution is relatively poor, perhaps just a hundredth of that of the human eye. Understanding how the dragonfly hunts so effectively, despite its limited sensing abilities, can suggest ways of designing more efficient systems. Using the missile-defense problem, the dragonfly example suggests that our antimissile systems with fast optical sensing could require less spatial resolution to hit a target. The dragonfly isn’t the only insect that could inform neural-inspired computer design today. Monarch butterflies migrate incredibly long distances, using some innate instinct to begin their journeys at the appropriate time of year and to head in the right direction. We know that monarchs rely on the position of the sun, but navigating by the sun requires keeping track of the time of day. If you are a butterfly heading south, you would want the sun on your left in the morning but on your right in the afternoon. So, to set its course, the butterfly brain must therefore read its own circadian rhythm and combine that information with what it is observing. Other insects, like the Sahara desert ant, must forage for relatively long distances. Once a source of sustenance is found, this ant does not simply retrace its steps back to the nest, likely a circuitous path. Instead it calculates a direct route back. Because the location of an ant’s food source changes from day to day, it must be able to remember the path it took on its foraging journey, combining visual information with some internal measure of distance traveled, and then calculate its return route from those memories. While nobody knows what neural circuits in the desert ant perform this task, researchers at the Janelia Research Campus have identified neural circuits that allow the fruit fly to self-orient using visual landmarks. The desert ant and monarch butterfly likely use similar mechanisms. Such neural circuits might one day prove useful in, say, low-power drones. And what if the efficiency of insect-inspired computation is such that millions of instances of these specialized components can be run in parallel to support more powerful data processing or machine learning? Could the next AlphaZero incorporate millions of antlike foraging architectures to refine its game playing? Perhaps insects will inspire a new generation of computers that look very different from what we have today. A small army of dragonfly-interception-like algorithms could be used to control moving pieces of an amusement park ride, ensuring that individual cars do not collide (much like pilots steering their boats) even in the midst of a complicated but thrilling dance. No one knows what the next generation of computers will look like, whether they will be part-cyborg companions or centralized resources much like Isaac Asimov’s Multivac. Likewise, no one can tell what the best path to developing these platforms will entail. While researchers developed early neural networks drawing inspiration from the human brain, today’s artificial neural networks often rely on decidedly unbrainlike calculations. Studying the calculations of individual neurons in biological neural circuits—currently only directly possible in nonhuman systems—may have more to teach us. Insects, apparently simple but often astonishing in what they can do, have much to contribute to the development of next-generation computers, especially as neuroscience research continues to drive toward a deeper understanding of how biological neural circuits work. So next time you see an insect doing something clever, imagine the impact on your everyday life if you could have the brilliant efficiency of a small army of tiny dragonfly, butterfly, or ant brains at your disposal. Maybe computers of the future will give new meaning to the term “hive mind,” with swarms of highly specialized but extremely efficient minuscule processors, able to be reconfigured and deployed depending on the task at hand. With the advances being made in neuroscience today, this seeming fantasy may be closer to reality than you think. This article appears in the August 2021 print issue as “Lessons From a Dragonfly’s Brain.” Keep Reading ↓ Show less