Oral-History:Howie Choset
Howie Choset
Howie Choset was born in New York and grew up on Long Island. Choset's early interest in motion led him to pursue robotics. He earned a dual bachelor's degree from the University of Pennsylvania in 1990 in computer science and engineering in 1990 from the Moore Schooli and in entrepreneurial management in from the Wharton School. He did his graduate work in mechanical mngineering at Cal Tech, earning a master's and Ph.D. in 1991 and 1996, respectively. After graduation, he joined the faculty at Carnegie Mellon University, where he continues today a a Professor of Robotics, directing the undergraduate robotics minor and teaching an undergraduate course on Robotics.
During his undergraduate years he worked under Ruzena Bajcsy at the GRASP lab working on vision systems, and at Caltech he worked with Joel Burdick on the the robotic exploration of three-dimensional space. At Carnegie Mellon, his reseach has centered on medical robots, control design, and multi-agent path planning . Choset has researched and designed snake robots, co-founded Medrobotics (which utilizes snake robots for minimally invasive surgery) with Marco Zenati and Alon Wolf, and founded a second company which manufacturers modular robot actuators.
In this interview, Choset recounts his career in robotics and his contributions to the field. He describes his early interest in motion which led him to study robotics, and his early collaborations with Ruzena Bajcsy and Joel Burdick which helped guide his career. He speaks about his research at Carnegie Mellon, especially his work on medical robots, his contribution to the development of the snake robot, and his various industrial and student collaborations, and the practical applications that arose from it. Additionally, he reflects on the evolution of the field of medical robotics, and provides advice to young people interested in the field.
About the Interview
HOWIE CHOSET: An Interview Conducted by Peter Asaro, IEEE History Center, 15 September 2014.
Interview #693 for Indiana University and IEEE History Center, The Institute of Electrical and Electronics Engineers, Inc.
Copyright Statement
This manuscript is being made available for research purposes only. All literary rights in the manuscript, including the right to publish, are reserved to Indiana University and to the IEEE History Center. No part of the manuscript may be quoted for publication without the written permission the Director of IEEE History Center.
Request for permission to quote for publication should be addressed to the IEEE History Center Oral History Program, IEEE History Center, 445 Hoes Lane, Piscataway, NJ 08854 USA or ieee-history@ieee.org. It should include identification of the specific passages to be quoted, anticipated use of the passages, and identification of the user. Inquiries concerning the original video recording should be sent to Professor Selma Sabanovic, selmas@indiana.edu.
It is recommended that this oral history be cited as follows:
Howie Choset, an oral history conducted in 2014 by Peter Asaro, Indiana University, Bloomington Indiana, for Indiana University and the IEEE.
Interview
INTERVIEWEE: Howie Choset
INTERVIEWER: Peter Asaro
DATE: 15 September 2014
PLACE: Chicago, IL
Early Life and Education
Howie Choset:
Well, thank you for your interest in my career and history of my field. I know a lot of people.
Peter Asaro:
<laughs> Great. So we usually start by asking where you were born and where you grew up and went to school.
Howie Choset:
Okay. So my name is Howie Choset. I was born in New York, and I grew up on Long Island and went to John F. Kennedy High School. When I was in high school I entered a variety of science fair competitions, and the one that’s most notable was when I was a senior in high school, I entered the Duracell Battery Contest, and I was one of the national finalists for building a little robot that had a sonar sensor and it would go around and avoid objects.
Peter Asaro:
And that was your first robot that you ever built or…
Howie Choset:
So it’s not clear, you know, what’s a robot? So it’s hard to answer, “What’s the first robot I ever built?” But since I was in junior high school, I used to build little electronic controlled cars using shoe boxes as the body and motorized wheels to drive them around. So maybe I would say around seventh, eighth grade is when I started building robots.
Peter Asaro:
Great. And so did you realize at that time that that’s the kind of career you wanted to have?
Howie Choset:
I realized at a very young age, maybe as early as five, that I was always interested in motion, how to make things move. I was always interested in how cars moved on the highway, how animals moved. And then probably by the time I was in the seventh or eighth grade I knew I wanted to go to program that motion. I can’t remember exactly when I realized that was a robot, but certainly at an early age I wanted to work on robots.
Peter Asaro:
Great. And then where did you go to undergraduate?
Howie Choset:
So I went to college at the University of Pennsylvania where I majored in Computer Science and Engineering. And I had a second major in Business from the Wharton School.
Peter Asaro:
And who did you work with while you were there?
Howie Choset:
When I was at the University of Pennsylvania I worked with Ruzena Bajcsy in the GRASP Lab. In working with her, her staff and her students, I learned a fair bit about robot vision.
Peter Asaro:
And taking up the business degree, did you have aspirations to have a robotics company at some point?
Howie Choset:
So when I applied to graduate school, I wasn’t sure. “Did I want to become a professor or become an entrepreneur and start my own company?” I chose the path of academia. I wanted to become a professor. At least become a professor at first. However, 10 years ago I did start my first company. The name of the company is called Medrobotics. They make a small, surgical snake robot for minimally invasive surgery. That robot was invented in my group and then with the support of some local Economic Development funds, most notably the Pittsburgh Live Science Greenhouse, we were able to kick the technology over the fence from a university lab to a company. And at that point we were able to attract investors, in fact, enough investment where eventually we built a robot that operated on a person. So I don’t believe that as a university professor I could’ve singlehandedly built a robot that would then operate on a person. Had to be through a company. And the people in this company did all the hard work, not only building the robot, making it operating room compliant and safe for a person, but also doing all the necessary documentation that it can be allowed to operate on a person. So in my view, this is a good story of, well, me starting a company, and then second, when it was a right point in time to kick the technology over from the university to the commercial side.
Peter Asaro:
And when you were an undergraduate at the GRASP Lab, what was it like to work there? What time of period was that? What was going on there?
Howie Choset:
How’d you know I worked in the GRASP Lab?
Peter Asaro:
Isn’t that what you said? Or…
Howie Choset:
I didn’t say it.
Peter Asaro:
Oh, you worked with Ruzena Bajcsy though.
Howie Choset:
Bajcsy. But you inferred correctly.
Peter Asaro:
Yeah.
Howie Choset:
So I was quite fortunate to have had the opportunity to work in the GRASP Lab with Ruzena Bajcsy’s group. Working there was great. I would say mainly because of the people. They’re just smart, hard-working people who are willing to teach me new math and new programming tricks as I needed them. Most notably was a woman named Helen Anderson who was a staff member in the GRASP Lab. She not only taught me good programming and some robot vision, but I also got to learn a little bit about writing and how to express your ideas from her.
Peter Asaro:
And that’s where you started working on vision systems; is that right?
Howie Choset:
When I was working in the GRASP Lab, I worked solely on robot vision, because that was my job. It wasn’t my interest or my first choice interest. However, I’m incredibly grateful for that opportunity, because I think if I had been left to my own devices, that’s an area of robotics I otherwise would not have explored on my own.
Graduate work at Cal Tech
Peter Asaro:
And then did you decide immediately to go on to graduate school or where did you go?
Howie Choset:
So I decided my freshman year of college, when I was at the University of Pennsylvania, that I wanted to go to graduate school. I really love the academic environment. I like how people come together and brainstorm on new ideas. And then the rigor it takes to prove those ideas. So for that reason I wanted to go to grad school. Eventually I went to the California Institute of Technology, Caltech. In Pasadena, California. My advisor there was Joel Burdick, and it’s worth noting that when I went to graduate school, my major was Mechanical Engineering. That was a change from Computer Science to Mechanical Engineering. That was a very hard and difficult change for me, because all of the math and background that graduate mechanical engineers have, I didn’t have. But I have to say it’s been an incredibly-- well, excuse me. I have to say it was an incredibly worthwhile experience, because it really set the pace and the tone for the kind of work that my research group does today.
Peter Asaro:
And what was it like working with Joel Burdick?
Howie Choset:
Working with Joel Burdick was great. He’s just a phenomenal advisor. Well, he’s very down-to-earth, he’s very kind. But he’s also incredibly smart. He brings an incredible perspective. He was able to direct me where to go. But on the flip side, he gave me incredible amounts of autonomy. So I would say I was well supported and well directed. Or what’s the word? So I was well supported, yet I had great autonomy.
Peter Asaro:
Great. And what was your thesis project?
Howie Choset:
My thesis project was entitled “The Hierarchical Generalized Voronoi Graph.” At the time I was interested in getting high degree of freedom systems, like a snake robot, to explore an unknown three-dimensional space. However, that problem was way too hard. So Joel and I focused on just having a point explore a three-dimensional space. Exploring a two-dimensional space isn’t that hard. When things go into three dimensions, there’s all these subtleties and nuances that one doesn’t normally expects to have as a difficult problem, and that’s mainly because we are two-dimensional thinkers by our very nature.
Peter Asaro:
And were you working with some theoretical models or do you have an actual snake robot that you were working with?
Howie Choset:
So at the time, I was not working with a snake robot to explore a three-dimensional space, because we didn’t have such a snake robot. So what my thesis work did was derive the theory for how one could explore math and then use that math for future excursions into three-dimensional environments, only using line of sight or sensor data. When it came time for me to demonstrate the ideas for my thesis, I ended up using a mobile robot to construct a special planar case of the three-dimensional structure that my thesis work prescribed.
Peter Asaro:
And when was the first time you got to work with an actual snake robot?
Howie Choset:
The first snake robot I worked with was the one that Joel Burdick and Greg Chirikjian, then his grad student, now a professor at Johns Hopkins University, built. Before I was working on my thesis project, I developed local, sensor-based planners that allowed Greg’s snake robot to avoid nearby obstacles when going from, say, a start position to a goal location.
Peter Asaro:
Okay. And when was that work?
Howie Choset:
The work I did in avoiding local obstacles, and I should acknowledge, was with – let me say again. The work that I did in local, sensor-based planning for snake robots, was also with a Japanese researcher Nobuaki Takanashi . And this is work that we did in 1993.
Peter Asaro:
So when did you finish your Ph.D.?
Howie Choset:
I finished my Ph.D. from Caltech in 1996. Actually, it was March 3, 1996.
Peter Asaro:
Okay. Were there any other notable projects or collaborations during your graduate work?
Howie Choset:
During my graduate work, I was fortunate to work with a lot of other students who had strong math backgrounds. The two people who stand out the most are Andrew Lewis, who’s now a professor of maths at Queens College in Canada. And the other is Jim Ostrowski, who works for a small company in the Bay area.
Peter Asaro:
Great. And did you go on for a post-doc after your Ph.D. or…?
Howie Choset:
I’m going to answer a slightly different question.
Peter Asaro:
Sure.
Howie Choset:
During my time as a Ph.D. student, I was fortunate enough to be selected to go on a program that the National Science Foundation sponsored where I worked in a Japanese lab. Part of that program included my visits to other Japanese labs. And one of those visits, I became friends with Keiji Nagatani. He’s now a professor at Tohoku University, but he served as my post-doc from 1998 to 2000.
Peter Asaro:
Great. And what kind of projects were you working on in Japan?
Howie Choset:
When I was working in Japan, I worked on a manipulation project. Really it was how to have a single finger sense objects as the fingertip rolls on the corner of flat faces. The project that Keiji Nagatani and I eventually worked on, what was called topological localization. It was using the embedded topology, you know, the understanding of, “Here are junctions in your environment. Here are edges that connect your junctions.” How to reason at that level on how to make a robot localize itself.
Peter Asaro:
Great. And did you develop other collaborations while you were in Japan?
Howie Choset:
None that lasted.
<laughter>
Peter Asaro:
Great. And where did you go after that post-doc?
Howie Choset:
So Japan was a summer program.
Peter Asaro:
A summer program.
Howie Choset:
Okay. After I graduated from graduate school, I worked in Japan in a separate, in a second, lab, on some computational geometry, some fundamental math. And then I went to work for a small company called Nomadic Technologies. They make mobile robots. While I was there I developed algorithms to cover known spaces.
Teaching and Research at Carnegie Mellon
Howie Choset:
After Nomadic Technologies, I started my faculty position at Carnegie Mellon.
Peter Asaro:
Okay. And that would’ve been 2000 or so or…?
Howie Choset:
I started my faculty position at Carnegie Mellon in 1996.
Peter Asaro:
’96. Okay. So what kind of work have you been doing since you were at Carnegie Mellon?
Howie Choset:
So I arrived at Carnegie Mellon 18 years ago, and I’ve been lucky to have had 13 amazing Ph.D. students, 28, 29 master’s students, and easily 150 undergraduates work in my lab. We do a variety of things, but one way to organize them is by snake robots and gates that make snake robots work, medical robots, manufacturing and multi-agent path planning. So snake robots represent the core of a lot of what my group is doing. We build the mechanisms themselves, which is challenging, because you want to build something that’s small, yet maneuverable and strong. And of course as low-cost as possible. We also worry about the algorithms. The motion control. How do you coordinate these internal degrees of freedom to get the robot to go in some general direction that you care? Another important problem is now that you can make the robot go in whatever desired direction you care, where does the robot go? We call that path planning. Finally, to close the loop, there’s a problem of estimation. This is sometimes called the SLAM problem, simultaneous localization and mapping. How do we know that we’re where we want to be? How do we know we’re executing the right controller? Those were the kinds of problems that the snake robot work inspired.
So let’s take medical robotics, the second big area of research my group undertakes. So we built a small, surgical snake robot for minimally invasive surgery. The benefits of minimally invasive surgery can be quite profound. It’s less pain on the patient. The person can heal faster, and the person can return to normal life more quickly, lower hospital stays. Another side benefit for minimally invasive surgery is that I think medical care gets disseminated to more people that otherwise would not have received medical care. Because now you don’t have to be in a center of medical excellence. You can perhaps do more things maybe in an office visit. The problem with minimally invasive surgery is there’s only two kinds of tools. One of them is a rigid laparoscope which means you can access only points from a line of sight from the incision. The other kind of mechanism are endoscopes. They’re flexible, but they buckle easily. A small surgical snake robot inherits the benefits of both worlds. It’s both rigid and flexible at the same time. So if you want to do a heart operation, instead of cracking the chest, we can make a small incision in your solar plexus, make a 25 millimeter turn one way, a 25 millimeter turn the other way, and voila, we’re behind the heart.
Now, as proud as I am of this technology, I also have to stress this is really a team sport. So for one thing, my then post-doc, he’s now a professor at the Technion, his name is Alon Wolf, he was a medical robotics expert who came to my lab and really got me reinvigorated into medical robotics. My clinical collaborator, his name is Marco Zenati, at the time he was a cardiac surgeon at the University of Pittsburgh, but now he’s a professor at Harvard, he’s the one who was able to look at what the snake robots looked like then, blur his eyes just a little bit, extrapolate as to where they can go and then give Alon and me the feedback that we needed initially to really iterate on this design. So Alon and I spent, you know, quite a long time scratching our heads, trying to figure out how to make this, you know, build a small surgical snake robot. It’s hard. How do you get all that power into a small volume? Most of the people in this community at that point in time were trying to develop novel actuators, MEMS, electropolymer actuated muscles, shape memory alloys. And that work is great; it just wasn’t quite ready for prime time in making a robot work. And then one day I sat down. I just literally sat down and said, “I’m going to figure this out.” And 20 minutes later, half the solution came to me and then I told Alon my half, he gave me the second half, and voila, we had the small surgical snake robot. Marco was then able to give us feedback, and he was aggressive in getting us into the operating room. We started operating on pigs, and the mechanism just iterated and iterated.
Medrobotics and the Snake Robot
Howie Choset:
And then at a certain point I was very proud to say that Marco Zenati, Alon Wolf and I, we co-founded this company called Medrobotics. And I don’t think we could’ve started this company without the assistance of Jim Jordan. He is the Chief Investment Officer of an organization called the Pittsburgh Life Science Greenhouse. The Pittsburgh Life Science Greenhouse looks at small companies and university research and helps them understand the true market potential of their technology, gives them some money, but also makes lots of introductions to potential investors and to potential CEOs. And it was Jim Jordan’s vision that really got us to articulate where this surgical snake robot can go. And with that vision, and with our great work ethic, we were able to attract a CEO from Boston. I have to… <laughs> I wanted to say Boston Dynamics, but it was from Boston Scientific.
Peter Asaro:
Hm.
Howie Choset:
So with Jim Jordan’s help we were able to attract a CEO from Boston Scientific, and then there the company was able to take off. So we were able to attract first 12 million dollars in investment, and that brought us to the first in-human operation where I was fortunate enough to be present. So I got to watch my snake robot operate through a small incision on this woman’s heart. And she received a diagnostic which otherwise would’ve required a full sternotomy, 12 to 14 days she would’ve been in the hospital. Except this time she went home the next day. Very little suffering. That company today is led by Samuel Straface, and he’s the CEO of Medrobotics, and he’s just wonderful. The breadth of knowledge that he has on the medical device field, on what it takes to make systems go. He just knows everything. And he just has a way of being able to talk with people in all sorts of disciplines. You know, brings it all together. And I’m proud to say that the company has received its European CE mark. So this means they’re able to operate in Europe with this robot. And we’ve performed on the order of 15 to 20 operations in the last month, since we’ve obtained this certification, on a base of neck or base of throat operation procedures which otherwise would’ve required an invasive incision or bad chemotherapy or the patient just would’ve been told, “We’re really sorry. There’s nothing we can do for you.”
Peter Asaro:
So what was the big breakthrough that you had that you figured out and put together with Alon in that 20 minutes?
Howie Choset:
The breakthrough that we had in designing the small surgical snake robot was how we can get all of the actuation off board the robot. So there’s no motors in the robots, and we’re able to marionette that robot using four cables. So literally using four fishing wires and six motors that you can buy from RadioShack, we were able to build a mechanism that can follow an arbitrary curve up to a curvature limit in a three-dimensional space.
Peter Asaro:
Hm. And you can control the point along the snake at which the curve curves?
Howie Choset:
The way the physician or the user is able to control the robot is he has a joystick and that joystick steers where the head of the snake goes. Almost as if it’s a ball floating in space. You tell it where to go, and it’s going to go. The rest of the snake robot is what supports that ball. So it’s a follow-the-leader mechanism. You aim where the tip of the snake goes, it goes, and the rest of the snake follows.
Multi-agent Systems
Peter Asaro:
Okay. So tell me a bit about your work on multi-agent systems.
Howie Choset:
So snake robots have many, many degrees of freedom. In the parlance of motion planning, that means they live in high dimensional configuration spaces. We’re interested in other systems that have many, many degrees of freedom. So one such system would be a multi-agent system. Something where you have, say, 100 robots, each of which have to go from a current location to some goal location. And what we want to do is we want to find optimal paths but do that in a short amount of time. Right now with multi-agent systems, there are pretty much two extremes on a spectrum. You can either plan in a coordinated space, in other words in this multi-dimensional search space, which becomes computational intractable shortly after seven or eight robots, or you can have every robot plan a path through itself, and when robots start to come into collision you create traffic control laws that get them to avoid each other. That’s nice, except one, you’re not guaranteed or you probably won’t get an optimal solution. But even worse, you may find yourself in a situation called deadlock, where this robot’s waiting for that robot, who’s waiting for that robot, coming all the way around the cycle. Everybody’s waiting for everybody else to make a move and no one’s going to make the move because they’re constantly waiting. It’d deadlock. Everyone’s deadlocked. So there’s this sort of unbreakable tradeoff. What we were able to do was find something in the middle. And the way this works is we have every robot plan for itself. So every robot plans its own path. But only when they come within proximity of each other and only the robots that come within proximity of each other. We locally expand their search spaces to account for the two or three robots that are in contact with each other. Worst-case scenario, you get into a bottleneck where everyone piles into the same bottleneck. However, on average, we were able to demonstrate that you get computationally tractable solutions, and again, using the words from motion planning, we get slower than exponential growth in the number of robots.
Peter Asaro:
And have you been able to reapply that to the snake robots or…
Howie Choset:
We have not been able to reapply that algorithm directly to snake robots to make snake robots find paths more easily for each other, but what we have been able to do was apply an algorithm to a group of snake robots, if you want a group of snake robots to find paths in some location from one place to another.
Applications of Snake Robots Beyond Medicine
Peter Asaro:
And have you been developing other kinds of snake robots apart from the non-invasive medical design?
Howie Choset:
So a lot of times people ask me or make the comment, “Oh, it’s just a snake robot. It’s one kind of snake robot.” But no, no, no, no. Eskimos have 28 words for snow, whereas to us it’s like snow and slush. And the same thing is true of the snake robots. To me the medical snake robot, which is very small, is a completely different beast than, say, the snake robot we have for locomotion. The locomoting snake robot is used for urban search and rescue. Most recently I’ve been using the medical – excuse me.
Most recently I’ve been using the locomoting snake robot for archaeology. So I’ve been to Egypt, and last month I was in Turkey. The challenges for archaeology are just like search and rescue, except everyone’s been dead for 5,000 years, so there’s no rush. So you can sit and take it easy. We’ve also been developing locomoting robots for inspection of nuclear reactors. So a locomoting snake robot is one that uses its internal degrees of freedom in order to propel itself forward. It locomotes. The reason why you would want a locomoting snake robot is if you need to go a very long distance beyond which conventional arm, even if it’s a fixed-base snake robot, can otherwise reach. You would also need a locomoting snake robot if you have all sorts of mobility challenges, just threading through tightly packed volumes and getting to locations that people, machinery, otherwise can’t access. That’s the locomoting snake robot. So we have the small snake robot. Let’s say we have the medium size locomoting snake robot, and we built a large snake robot – I know, small, medium, large – for manufacturing in confined spaces.
Peter Asaro:
So tell me about the archaeological work. Did you go explore some pyramids or…
Howie Choset:
So we’ve only gone on two archaeological trips. The first one was in Egypt. In Egypt we first sent our snake robots into these caves off the coast of the Red Sea. These caves are very small and they were built by the ancient Egyptians. The ancient Egyptians, they used to carry their ships from the Nile to the Red Sea, but they didn’t carry the ships in their full form. They built the ships out of modules. It was nice and modular. So they, like, carry pieces of their ships. They’d assembled the ships, sailed to probably Somalia, traded, traded, traded, came back, disassembled the ships, and instead of carrying them back to the Nile, they stored them in these little caves so the bad guys won’t steal their sea vessels. The following year they’d come, get the ships, reassemble them, and the process repeats. Now, a lot of those caves are unstable for people to enter, so the hope was that we would be able to send our snake robots in and find evidence of other ships. Unfortunately, we found nothing. And the main reason why we found nothing was that our robots weren’t good enough. Every time we’d do a search and rescue training or we did this archaeology, we’d find a challenge that we just couldn’t handle. In this particular case, we had trouble mediating the dirt. In other words, the snake robots slid a lot, and we also had trouble going up steep inclines. Last month we went to Turkey where we went into these burial mounds, and there we had fewer challenges that the robot couldn’t overcome, but again, we didn’t find anything. But this time we found nothing, because there was nothing there.
Peter Asaro:
Uh-huh. Okay.
Howie Choset:
I also forgot to say something I should’ve pointed out. We also sent our snake robots into these caverns that were underneath the Great Pyramid.
Peter Asaro:
Oh.
Howie Choset:
And again, we found nothing, but we also ran out of time. Not because we didn’t find nothing.
Peter Asaro:
Okay. And for the manufacturing in confined spaces, what are some of the applications that the snake robot is particularly useful for?
Howie Choset:
So the snake robot is good at getting into a tight spot and then, say, applying – actually, I’d rather not talk about the manufacturing in confined spaces in detail. Let’s strike that.
Peter Asaro:
Okay.
Howie Choset:
Yeah, I think the sponsor may not like me talking about it. Yeah, I’m sorry about that.
Peter Asaro:
Sure.
Other Robotic Models
Howie Choset:
I also should acknowledge, we do build other robots too, I should say. So my research group is quite good at building a variety of mechanisms. One of them, I should acknowledge, is born out of our snake robot work. And that is we’re getting very good at building modular systems. So our current snake robot can be put together without tools. You know, just using your hands, you can screw them together as if they were Lego. But what’s more impressive than the mechanical connection is the underlying software and electronics that makes this happen. And as a result of that, we’re now building other kinds of modules that we can put together. So we built a tank tread module. We’re building a legged system, again, out of our modules. But other robots that we’ve built, we’ve built a small mobile robot that’s yea big, this high off the ground, but it is very fast and it can maneuver over objects many times its size. The idea is that a lot of times when we would take our snake robot to do some search and rescue training, I couldn’t help but wonder, “Could an articulated wheeled vehicle do just the same?” And that’s the answer to this problem. We built another robot, or excuse me.
We’re in the process of building another robot that we’re going to be able to throw in the air, and then using its wheels it’ll be able to ride itself and then land wheel down. My interest there is, one, being able to take a running jump and land on a platform, but I’m also interested eventually in building a spider robot where the spider can run and then jump and then land on a vertical surface. My research group, this is in collaboration with Jon Luntz at the University of Michigan, and Bill Messner, who’s now at Northeastern. We built what’s called an actuator array. Imagine you have a checkerboard, but each checkerboard has a pair of wheels where one wheel rotates this way, the other wheel rotates this way, but there’s beads on these wheels. So these wheels collectively can provide a planared, vectored force in any direction. So now you have an array of these, you could put a box down on top of it and then you can manipulate the box, spin it around and send it off. It’s sort of the next best thing in conveyor belts. I’m trying to think what other robots we’ve built or we’re in the process of building. Because I know I’m going to think of one more and then be embarrassed that we didn’t discuss it. But I guess we’ll have to leave it at that.
Peter Asaro:
So you mentioned spider robots. Do you actually look to biological models as you construct these systems, or are they mostly engineering focus?
Howie Choset:
So a common question we get with our snake robots is are we biologically inspired? Is this biologically inspired robots? And the answer is “yes.” But I think just saying “yes,” doesn’t do the question justice. We’re biologically inspired on multiple levels. So first we look at the snake, we build a snake robot. But then we just used our engineering intuition, our underlying math, to make the snake robot work as well as we possibly could. In doing so, there are times we can return the favor to biology and start explaining to biologists how real snakes work. However, this illumination’s going both ways. So I work with a physicist and a biologist. His name is Dan Goldman at Georgia Tech. And he and his crew collect all sorts of data on snakes and they draw great conclusions and they posed some questions which were able to demonstrate or disprove on this snake robot. So the robot is being used as a template to answer some of Dan’s questions. But at the same time, Dan looks at the snake, looks at the snake robot and says, “Hey, if you just did something a little bit differently, can we make the snake robot work better?” And what we’re doing now is we’re developing algorithms that can take the data we get from the snake, biological snake, sieve out the core principals, and use those principals to make the snake robot maneuver in ways that our raw engineering intuition alone could not have.
Breakthroughs in Snake and Modular Robotics
Peter Asaro:
And over the course of your career, what have been the kind of big breakthroughs in snake robotics and modular robotics?
Howie Choset:
So it’s kind of hard for me to point to a breakthrough, because a lot of the work that we do is happening continually, in pieces and pieces and pieces. So the only time where I thought, “Aha,” you know, it was such a tremendous jump forward, was when we conceived of the medical snake robot mechanism in those 20 minutes. However, there’s just a lot of hard work, a lot of ingenuity, a lot of, “Let’s try this,” a lot of, “Let’s think this out, reason things out that happen in a very piecemeal…” This is what science and engineering development is. I’m very proud of a series elastic robot that we have. I’m very proud of the modular snake robot that we have. I’m very proud of the geometric mechanics, most notably the work that Ross Hatton, my Ph.D. student, he’s now a professor at Oregon State, was able to derive in our work. So I guess to say I’m proud of everything, but I can’t point to that, other than building mechanisms, that one critical breakthrough. Maybe there is none. You know, that’s one of the reasons why robotics is great. It’s both multidisciplinary, meaning that we have mechanical engineering, electrical engineering, computer science, as well as psychology and other fields, but it’s also interdisciplinary. The computer science affects the electronics. I have to make tradeoffs between the mechanical engineering and the computer science and so forth. We all have to wear each other’s hats in order to make the robot work.
Funding Sources
Peter Asaro:
And what have been some of the sources of funding over your career for your research?
Howie Choset:
Well, first, it’s never enough. So I’ve been very lucky to have attracted funding from the National Science Foundation, the Office of Naval Research, the Army, and DARPA. I’ve also been fortunate to receive funding from industry. However, I don’t know if industry wants me to report where my funding’s coming from. So out of respect, I’ll just say industry.
Peter Asaro:
And are there differences in the kinds of the work or the ability to share the results of your research depending on where the funding is coming from?
Howie Choset:
With my industrial research… Actually, I’m going to answer a different question and I’ll get back to your question. In all the projects in which my group works, we always work with a stakeholder in that field. So when we’re doing our medical robotics, Marco Zenati with a cardiac surgeon. When we were doing search and rescue, Robin Murphy, who’s a roboticist, but also Sam Stover, a rescue worker. You know, one of them is one of us; one of us is one of them. When we’re doing, when we used to do car painting, we had Jake Braslaw from Ford. Again, he’s worked with us nonstop. So I have this, I’ve been very lucky, to have attracted collaborators who were very aggressive in getting me into their worlds and very willing to come and sit and hang out in my world. And because of that, we end up developing joint results. Joint results lead to joint publications. So using industry as an example, I have co-authors who are my industrial collaborators. Since they’re co-authors, they have to go through whatever process they have to go through in order for the paper to be published. So that is the only hoop through which I have to pass. And I’ve never had a problem passing through that hoop.
Evolution of Medical Robotics
Peter Asaro:
Great. And medical robotics is becoming an increasingly hot topic and emerging area. How do you, having been in the area for a long period, how do you see that kind of historical trajectory of medical robotics over your career?
Howie Choset:
I believe the futuristic medical robot isn’t going to look like a robot. Right now, we have this notion of using conventional robots, robot arms, maybe robot snakes, converting them into something that’s of medical use, and then going from there. I believe what we’re going to do is we’re going to see more implements, more tools, become roboticized, become more intelligent. And what this is going to do is it’s going to offload a lot of the burden from surgeons to perform procedures. And we’re seeing this trend already. Take for example 40 years ago if you had something wrong with your large intestines, you underwent surgery, they looked at your large intestines and said, “Hey, you have this terrible inflammation. It’s Crohn’s Disease, it’s colitis. Here’s your prednisone,” and off you go. Today you have a colonoscopy. A non-surgeon performs the procedure. A non-surgeon performs a minimally invasive ‘cedure. In fact, colonoscopy is so minimally invasive and so low-cost it’s now become a diagnostic, not just a therapeutic tool. And we have non-surgeons performing the colonoscopy. In fact, we now have medical students performing colonoscopy. So I see a trend in where more specialists, people who have not had the same access to resources and perhaps medical training, being able to perform what was once surgical level care but outside of the centers of medical excellence. In regional hospitals. In office visits. Even in the field. And these procedures are going to be done by more intelligent tools, but by med techs, physicians, specialists. Where I think surgeons are going to have a role, because we’re always going to need surgeons, is they’re going to do the more complex procedures, and I envision that surgeons and specialists are going to form new ways to collaborate to invent a whole host of therapies otherwise we couldn’t get. So it will have the benefit of a surgeon but with the invasiveness of a specialist.
Peter Asaro:
So in terms of the research side of medical robotics, have you seen more research over the previous decades coming from medical conferences or medical instruments and devices conferences, or has most of the research come, you know, conferences like this in robotics? Or has it been very interdisciplinary all along?
Howie Choset:
I think the development of medical robotics and medical devices has been a little bit fragmented. So medical devices are not necessarily robots, and they’re still coming out. The more we can do with a device the better off we are. You know, if we can get around using the robot, that’s probably a good thing, because it’s a simpler mechanism. In terms of the development of medical robotics, we have industrial development and we have academic development. Some of that is cooperative. You know, Intuitive surgical does a very good job reaching out to the academic community and either sponsored research or directive research in the hopes of bringing some of those ideas back to their robot to make a better system. Other companies, which are smaller, don’t have the resources or perhaps the time to reach out to academic work. So the smaller companies still aren’t necessarily directly collaborating with academics. However, the smaller companies are hiring the academics. In fact, I believe the future innovations are going to come from small companies and perhaps some applied, very applied, academic units, because they’re small, they’re still maneuverable. They don’t have the inertia. And that’s where I think most of the change is going to happen.
Collaborations and Students
Peter Asaro:
Great. Are there any other collaborations or students who have gone on to do work in robotics that you’d like to discuss?
Howie Choset:
Let’s see. Did I get all my collaborators? Yeah. I should mention Vijay Kumar a little bit. So I’ve been very lucky to have a host of collaborators and friends in the robotics community. I used to work a lot with Al Rizzi on the interface between controls and path planning, and he and I did a lot of work together on car painting. He ultimately left academia and went to work for Boston Dynamics, where he has an incredibly successful career. I do a fair bit of interacting with Vijay Kumar on the multi-agent work. He and I bounce ideas off of each other. Our students talk to each other, and we serve on each other’s student thesis committees, and we’ve been able to generate some nice ideas in the multi-agent path planning world.
Peter Asaro:
Great. And what’s it been like working at – oh.
Howie Choset:
Oh, sorry. I’m sorry. I forgot. Also, I do a fair bit of work with Robin Murphy on search and rescue robotics. The truth is, I would not have done anything in search and rescue robotics if it hadn’t been for Robin.
Peter Asaro:
And that’s been primarily the snake.
Howie Choset:
With the snakes, yeah.
Peter Asaro:
Now, what’s it been like at Carnegie Mellon to be a robot <inaudible>?
Howie Choset:
Well, I have one other I have to say.
Peter Asaro:
Oh, sure.
Howie Choset:
Okay. This is one body of work we didn’t touch on, which I think is worth mentioning. So one of the thing – excuse me. One area of work on which we’re trying to expand the benefit of minimally invasive surgery is recreating the situational awareness for physicians that is lost when you go from an open procedure to a minimally invasive one. In minimally invasive surgery, you’re essentially operating through straws, and that’s all you see. What we’ve been doing is in the past starting with the work of Keiji Nagatani, who was my post-doc, now professor at Tohoku University, we did a lot of simultaneous localization and mapping, SLAM, for mobile robots. And we were able to transfer and then build upon the SLAM results to do SLAM in the body. We call it body SLAM. And the reason why this is an important topic is that we can simultaneously map out the anatomy, as well as localize the robot to the anatomy when doing minimally invasive surgery. So the goal of this work would be imagine you’re watching the movie The Matrix and you want to be able to spin the heart around and get this vantage point, or you spin it this way to get that vantage point when doing your minimally invasive surgery. And I’m very proud to say that my collaborators and friends Russ Taylor and Nabil Simaan. Russ is at Johns Hopkins University, Nabil is at Vanderbilt University. The three of us together have a very strong project in what we’re calling concurrent situational awareness, where we’re not only trying to improve the situational awareness for the physician, but we’re also trying to improve it for the robot so the robot can do more things for the situational awareness. And we have a very successful collaboration that – and we just recently got funded through the National Robotics Initiative to advance this work. So in the next four years you’re going to see great things coming out of this group.
Peter Asaro:
I just interviewed Russ. What’s he like as a collaborator?
Howie Choset:
Well, let’s see. The fact that I’m hesitating means I have to come up with the right set of words. And the reason why is Russ is an incredibly special person. He is your best friend, and he can be thick-headed, and sometimes it’s great that he’s thick-headed. But he is thoughtful, kind, and supportive. He’s just a wonderful person to deal with. But there’s always that exterior you have to break. However, one thing about robotics is it’s a team sport. And the personalities among Nabil Simaan, Russ Taylor and me just match perfectly. So Russ and I can argue like cats and dogs and then two minutes later an observer would never even know we were just arguing. Because that’s the kind of dynamic we were able to develop. Russ has this incredible breath of knowledge that he brings to the table and he’s quite humble about it too. You know, he is the father of medical robotics. He’s the one who started all this. So I find it an honor and a privilege to be working with him and to hear him say kind things about the work that my group does. Nabil Simaan is probably the nicest person in robotics. He’s just a nice guy. To spell his name is N-A-B-I-L, S-I-M-A-A-N. Nabil Simaan, he, I think, is the most up and coming researcher in medical robotics. I think his group generates the most number of neat medical devices, as well as algorithms to control them. He’s just a wonderful academic, because he’s theoretically grounded, and at the same time he can interact and work with industry. And the three of us just have this nice rapport and complementary skill sets that allow this collaboration to work. In fact, what’s unique about this collaboration is we’re at different universities and yet we talk directly to each other’s students. You know, as if his students are mine and vice versa.
Environment at Carnegie Mellon
Peter Asaro:
Great. And so what’s the environment at Carnegie Mellon like?
Howie Choset:
Carnegie Mellon University is a special place, especially for robotics. It’s a small school, but we’re big on robotics. So you have that small school feel, but yet that critical mass, more than critical mass, to work on robotics. I would say my collaborators, colleagues, my friends, in the department, although we all have varied interests, we all have a deep respect and a deep understanding of the other interests. In fact, I would say take a step further. We have a deep understanding of what we don’t know about each other’s interests. So when I have a problem with vision I can go talk with Yaser, Marcel, or Abhinav, depending upon the particular problem I have with vision. If I have a problem with singular value decompositions, I talk with my vision friend Fernando. Why would I have problems with singular value decompositions? I’m now using the analysis the vision people have been using for years in our locomotion work. In fact, machine learning is very, very big at Carnegie Mellon. My collaborator, Jeff Schneider, he and I have been working together for seven years on applying machine learning techniques, techniques that were developed to understand how galaxies form or for drug discovery. We’re now building upon those techniques to control snake robots. And just having these world-class researchers who, around you, the expertise is amazing. But the next thing is, is the spirit of collaboration is just very high at Carnegie Mellon in general, and in the Robotics Institute. Finally, people just work hard. And knowing that your colleague is working very hard, in some ways is inspiring, but it’s also relaxing, because you’re not doing all the work.
Peter Asaro:
<laughs>
Howie Choset:
The other thing about Carnegie Mellon that I also should stress are the students are phenomenal. We get amazing students who are theoretically grounded and yet want to and are very good at making robots work. And these are students at all levels. We have the best master’s, Ph.D. students, but our undergraduate population is also very unique. With undergraduates, my lab, we actually employ probably the greatest number of undergraduates. Probably the highest number of undergraduates. And my undergraduates go off and do amazing things after they leave my lab, mainly because they were amazing and I just was able to give them an environment in which they can grow just a little bit more. But my undergraduates publish papers, they’ve had their work appear on the cover of USA Today. You know, they just are great. Through the work with these undergraduates I’ve been able to introduce a new course, an undergraduate robotics course that’s a pleasure to teach. And recently with my colleagues Reid Simmons, most notably, at Carnegie Mellon, we prescribed a new major in robotics. So there are very few places where you’d have enough undergraduates to be so interested in robotics that it can justify my teaching courses in robotics. Because that’s what I really want to teach. As well as have access to a talent pool to further advance the robots that we’re developing.
Peter Asaro:
And then there any other specific students that you want to mention or who are still working in robotics or have gone on to do other robotics things?
Howie Choset:
So let’s go in order. My first student Ercan Acar works at Intel on microcontroller automation. My second student – I have 13 students. I don’t know if I want to go through all the students. All of my Ph.D. students either work in robotics or they work in a field, let’s say, automation, that required a robotics background in order to enter. Half my students are working in academia, and the other half are working in industry. One such student, Sarjoun Skaff, he started a toy robotics company, and now it’s been morphed into a robotics company that patrols warehouses and supermarket aisles to take automatic inventory. I just had another one of my students go start a company in the Bay area that has pilots flying over farm fields and using vision algorithms to figure out where you should put the fertilizer or place the water, what part of the field needs attention. And that company’s just doing phenomenally well.
Peter Asaro:
What’s the name of that company?
Howie Choset:
TerrAvion, T-E-R-R-A-V-I-O-N. Two of my students went to work for my medical device company, Medrobotics. So that was nice, at least, to keep the talents local with me. And one of my students works for MITRE, which is sort of in between industry and academia. But I have to say, I’ve been very fortunate to have these great, hard-working people who are also just humble as can be. And I might add, socially and emotionally better balanced than I’ll ever be. You know, work in my group. And it’s been quite fortunate. I recently started a second company, and two, excuse me, three of my students and one of my staff members work in that company.
Peter Asaro:
And can you tell us about that company?
Howie Choset:
That company is making a modular robot actuator which one can put together in different configurations to make a robot that will be robust, good enough, for real use, field use or industrial use.
Peter Asaro:
Great. Is there anything we missed, didn’t cover, that you want to add?
Howie Choset:
Let’s see. I don’t know. I’ll think of things afterwards, of course. I’m proud to say that the work with Dan Goldman’s group and my group is going to result in our first publication in Science in two weeks.
Peter Asaro:
Congratulations.
Howie Choset:
Yeah, that’s pretty great. And again, it’s one of those things that we couldn’t have done it without him, he couldn’t have done it without us. Although I think we needed him more than he needed us. Anything else? Carnegie Mellon’s great. Robotics is great. No, I’m good.
Advice to Young People
Peter Asaro:
So the question we end with is what’s your advice to young people who are interested in a career in robotics?
Howie Choset:
Well, first, whenever you give a talk, make sure the videos work before your talk. Because if you’re going to be interviewing, giving a presentation, you have to make a really good first impression. I don’t. So that’s one. It’s very easy for an older person to tell a younger person, “Go do what your heart says. Go do what you like.” You know, I got that advice then and it’s hard to follow that advice, because I think younger people either don’t know what their heart’s telling them, or at the very, very least, they are afraid to try something new. So my advice is, of course, follow your heart. But if you’re not sure about that, whatever you do, try and find something else to do too. I’m very fortunate that Ruzena Bajcsy forced me to learn robot vision. I’m very fortunate that when I was in college, some of my friends got me to practice taekwondo. These are things I normally wouldn’t have done. It’s different from robotics. Just try and do something else too. And follow your heart.
Peter Asaro:
Great.
Howie Choset:
I wish I had better advice for young people. Oh, I have another piece of advice for young people. Okay. So two pieces of advice for young people. One, make sure your presentations work. Okay. My other piece of advice to young people is don’t be afraid to admit you don’t know something. Often we’re asked a question and we feel like if we don’t answer it, we’re a failure. It’s okay to say, “I don’t know.” It’s okay to say, “Hey, let me think about that and I’ll get back to you.” That’s fine. I think what may trouble some young people is they think they know it all and then they’re asked a question that they don’t know the answer to, and that just rocks their world. Get over that. We know very little as it is and it’s okay not to know something, because that will be the opportunity to go learn something new. Okay. I’m done with the advice.
Peter Asaro:
Great. That was a great idea. Thank you very much.
Howie Choset:
Thank you for your interest.
