Oral-History:George Pappas

About George Pappas

George Pappas was born in New York to Greek parents. At five he returned to Greece, where he completed primary education before returning to the United States for his undergraduate and graduate education. He studied Computer and Systems Engineering at Rensselaer Polytechnic Institute, earning his BS and MS in 1991 and 1992, respectively. In 1998 he received his Ph.D. in Electrical Engineering and Computer Science from the University of California, Berkeley. Pappas is currently the Joseph Moore Professor and Chair of the Department of Electrical and Systems Engineering at the University of Pennsylvania, as well as holds positions in the Departments of Computer and Information Sciences, and Mechanical Engineering and Applied Mechanics. He is also a member of the GRASP lab and the PRECISE Center. Prior to his current appointment at UPenn, he served as the Deputy Dean for Research in the School of Engineering and Applied Science.

Pappas' research interests include control theory, hybrid and embedded systems, control systems, UAVs, and distributed robotics. For his work he has received several awards and honors, including the Antonio Ruberti Young Researcher Prize, the George S. Axelby Award, and the NSF PECASE.

In this interview, George Pappas discusses his work in robotics, in particular his work at RPI, UC Berkeley, and Penn. He describes the environment of his education and workplaces, and the various collaborations and influences he experienced, as well as the problems and projects he researched. Additionally he reflects on the evolution of robotics throughout his career, its future applications and challenges, and impact on society.

About the Interview

GEORGE PAPPAS: An Interview Conducted by Selma Sabanovic with Peter Asaro, IEEE History Center, 23 May 2011.

Interview #742 for Indiana University and IEEE History Center, The Institute of Electrical and Electronics Engineers Inc.

Copyright Statement

This manuscript is being made available for research purposes only. All literary rights in the manuscript, including the right to publish, are reserved to Indiana University and to the IEEE History Center. No part of the manuscript may be quoted for publication without the written permission of the Director of IEEE History Center.

Request for permission to quote for publication should be addressed to the IEEE History Center Oral History Program, IEEE History Center, 445 Hoes Lane, Piscataway, NJ 08854 USA or ieee-history@ieee.org. It should include identification of the specific passages to be quoted, anticipated use of the passages, and identification of the user. Inquiries concerning the original video recording should be sent to Professor Selma Sabanovic, selmas@indiana.edu.

It is recommended that this oral history be cited as follows:

George Pappas, an oral history conducted in 2011 by Selma Sabanovic with Peter Asaro, Indiana University, Bloomington Indiana, for Indiana University and the IEEE.

Interview

INTERVIEWEE: George Pappas
INTERVIEWER: Selma Sabanovic with Peter Asaro
DATE: 23 May 2011
PLACE: Philadelphia, PA

Early Life and Education

Q:

Could you tell us a little bit first about where you were born, and some of your first educational experiences, where you were educated?

George Pappas:

Yes, I was born in New York, but my parents were from Greece, and when I was 5 years old we went back to Greece, and that’s where I attended high school. But then I was essentially a dual citizen and decided actually to come back to the U.S. to Rensselaer Polytechnic for my undergraduate, and then moved on to Berkley for my Ph.D.. I had a one year visit, basically to Harvard while I was at – during my Ph.D., then I completed my Ph.D. 1998 and I’ve been at Penn pretty much ever since.

Q:

So what did you do when you were at RPI?

George Pappas:

So at RPI, I did my undergraduate then up – when I was about a junior, I decided that this area sort of control automation robotics is the area that I would like to do my further studies. And at the time, there was a very, very, very well known internationally renowned center on sort of robotics for space exploration. It was the time of the space shuttle and the robotic arms of the space shuttle. So NASA had Center on Space Robotics, and that was a draw for me. So it was lots of great people at RPI at the time. So then that’s – I did my Master’s there in the – in sort of a robotics area there, not necessarily space robotics. And based on my Master’s experience, at RPI, I decided to sort of move on to a Ph.D. which then I did at UC Berkley.

Q:

And who were the people you worked with at RPI?

George Pappas:

So my advisor at RPI was Professor Kostas Kyriakopoulos. So he was a faculty at RPI at the time, and then when I graduated with my Master’s he moved on to become a faculty in Greece in the area of robotics and so on. I guess, perhaps what was interesting is why did I choose to do control and robotics in the beginning. And part of the reason I’m in this field, is because it was a very difficult course in controls at the sophomore level. It was a notorious course at RPI, and I was – I took it as a challenge to do well, and there was – when I was taking the final, there was fire alarm, and I had completed – and this was in the middle of the exam. And because it was a fire alarm, they said, “Whatever you have written, we’ll take into consideration, exam is over.” And I had pretty much completed the exam in an hour and a half, and the professor said, Joe Chow, who is now the Associate Dean there, told me, “Well you have two options. You can either control in robotics or you can open a Greek Diner.” So being Greek, I said, “Okay, well, I’m not going to do the diner.” So I decided to do robotics instead, and I think that worked out well for me. So...

Q:

So what did you work on for your Master’s?

George Pappas:

I was working – at the time there was a lot of attention on the so called, parallel parking problem. That was the early, very early ‘90’s, late ‘80’s and there was a challenge of how to take cars which cannot move sideways, and how can you design algorithms that we can do parallel parking automatically? So I think because that presents a challenge, if direct – if robots are omnidirectional, obviously you can move in all directions. But if they’re not, then the question was how to design algorithms for parallel parking. So that was actually my Master’s project. And I went actually to my first conference which I – was a very empowering actually experience, because I felt that – I just want to keep on going to conferences forever. So how do I do that? I go get a Ph.D., and then based on my experience with the parallel parking problem, I was looking at other faculty around the country that were working on the problem and one of them was Professor Shankar Sastry at UC Berkeley. So I decided to apply there and that’s how I ended up at UC Berkeley. Of course, my Ph.D. had nothing to do with parallel parking but that’s how I got into UC Berkley, and that’s why I decided to actually go there.

UC Berkeley

Q:

So what did you work on for your Ph.D.? What kinds of...

George Pappas:

So at the beginning of my Ph.D. I thought I would work on the parallel parking and they were also very good researchers there doing – focusing on parallel parking and the mathematics behind and the algorithms behind parallel parking. But at the same time there was sort of a new wave of research that was taking place in the area of hybrid systems and hybrid control which is basically how we connect high level planning with say, low level control and high level reasoning and low level control, and one being traditionally more discreet or logic based then – or knowledge based on so on. And the other one being much more continuous, and then there was an issue of how do you bring these two worlds together. Half of the agenda was more on the CS, computer science front. And the other half was on more electrical engineering, signal processing, filtering sensors. So the challenge was how do we bring these two things together, and that was a main research agenda at Berkeley at the time. There was a so called path project at the time which was looking at sort of automation at highways, how do you get cars to basically talk to each other in order to reduce the spacing between cars. And a similar – I started working on a similar project on air traffic management. How do you sort of think about aircraft to talk to each other and do conflict resolution on the fly, on their own without necessarily having a central air traffic controller doing the negotiation between different aircraft. So that has a roots in a robotics, collision avoidance, conflict resolution is a problem that has been addressed a lot in the robotics literature. So there were a lot of things that I think we could offer to areas such as air traffic control, and that’s still remains to be the case. So from that I ended up going much more towards this are of hybrids systems of how to on one side have logic based switching whether you should use this controller, or that controller, or this maneuver or that maneuver. So how to supervise basically low level controllers or low level maneuvers in a way to achieve a higher level goal. So that was in some sense, my topic or the direction, at least of my Ph.D. thesis.

Q:

What were you doing simulations? Or were you working with actual robots?

George Pappas:

I was not working with actual robots. I was working mostly on the algorithms, and also the how to verify and prove that algorithms are correct. And the unique challenge there is that there’s people that for example, prove that their – say, conflict resolution works well, and then there’s people that verify computer science for example whether some logic based switching is reasonable. For example, if the obstacle or the air craft is within 10 meters, or 10 minutes of flight time away, then you do this evasive action and so on. And then there’s this issue of how do you ensure that if you put everything together. You verify that the logic works, you verify the controller works, but when you put the two things together, you actually ended up getting crashes. In fact, if you look at the recent grand challenge, a lot of the crashes, a lot of the cars that didn’t work was basically because of the interactions of the software logic with the controllers. So that interaction between the software logic and the controllers is what I ended up actually working on. Not so much from an experimental standpoint but much more from a frame work standpoint, like how do we reason, how do we form all this, how do we model, and how do we verify properties of the integrated system.

Q:

And you mentioned one of the applications that you were thinking about was the air traffic management?

George Pappas:

Yes.

Q:

Were there others?

George Pappas:

Yes. So during my Ph.D., I had looked mostly at air traffic management. And then since we – since I became a professor at Penn with my colleagues here at Penn, we also looked at bio-molecular networks. So for example, you can have the same kind of mixed discreet behavior in biology that for example, concentrations of certain bio-molecules can rise and then if they cross a certain threshold, then a certain gene activates and that triggers another continuous biochemical process. And I think there’s a lot of work in that direction. And then a big emphasis that I’ve looked at while I was here was in some sense in the area of what I would call, multi-robot system. So just like you have multi-agent systems in air traffic transportation. We have lots of agents, lots of aircraft trying to negotiate for partly common goals like safety and partly individual goals like going to your destination. You have the same issue in multiple robots where you can have a common goal like cover a particular area and then the question becomes how do you do that type of collaboration among individual robots to achieve a certain common goal. So I think a lot of the emphasis of the work that we do here has been based a lot on the sort of the multi-robot systems. And there I – we do simulation work but then with collaborators in the lab for example, Professor Vijay Kumar, we’re also looking at experimental versions in collaboration of the work. But the paradigm of hybrid systems are mixing things to – is very broadly applicable now even now to – in the energy space, and legged robots where you have sort of bouncing and so and so, it applies broadly.

Q:

Who were some of the people that you worked with while you were at Berkeley?

George Pappas:

My advisor was, Shankar Sastry, who had enormous impact on both my personality as a scientist and as a person, as well. But they were also, it was a really great, fantastic time, it was sort of a golden age of – in my area at the time, there’s just – there was just a tremendous wealth of talent synchronizing at that – those two three years in the period of ’97 to 2000. So Claire Tomlin, who is also now faculty at Berkeley, she was there at the time. John Lygeros, who is now in Zurich, was there at the time. Joao Hespanha who is now at the U.C. Santa Barbara. Magnus Egerstedt who is at Georgia Tech. Dattaprabodh Godbole, who’s now a leader at Honeywell, actually working on autopilots and so on. Karl Johansson, who is now at the Royal Institute of Technology in Stockholm. There was just a tremendous – Yi Ma, who is now at Urbana. There was just a tremendous wealth of talent. And I think that was one of the great things I really enjoyed at Berkeley that there was this sort of environment. It was not just individual or two, but there was actually an environment and you kind of felt that you just plugged into the environment just fantastic things happen. And you try to sort of emulate a little bit that culture here which we’ve tried to do.

Q:

Besides the people, what are some of the other important aspects of that, what you call is the environment?

George Pappas:

Yeah, so I think – so environment is both people having really great people around you. And the other one is – was a little bit – it was a culture of collaboration which was very important. So it was not a culture where everybody was trying to sort of close off their ideas. Everybody was really sharing ideas. Everybody was sharing problems. Everybody felt that they could go to the coffee place and really think about a problem and/or its solution. And it was, let’s try to write a paper together. Let’s try to address this big issue. So that was one – this openness of the culture of collaboration. I felt this way. The other one was this culture of being – thinking ambitious and being, thinking about big problems. And so I think that was... everybody was intellectually ambitious. Everybody wanted to change the world and whether they do that or not is a second thing, but that kind of culture if you want to think big about to impact the world, I think that was another important thing. And the third one, which is also very important in my opinion, is this culture of mentorship. That it is not just your own success, but it’s also the success of your students. And I think that is something that you create this culture where you’re happy with everybody else’s successes and every – and your student’s successes and you kind of celebrate, your students doing well, rather than just the research accomplishment. So you also celebrate the fact that you’re taking a person from here through the Ph.D. and then they become – and I think that is very important because at the end of the day you can do fantastic research but we are in the education business. And I think, that kind of – your graduate studies and the Ph.D. program is where really education and research mixed together and I think that was really celebrated in a very strong way there.

Q:

Could you give an example of how – so what are – is something that would display that kind of mentorship role?

George Pappas:

So I think it was very, very – so when, just to give you an example, when people were going out on the job market for an academic position, I felt that everybody was extremely well prepared for the academic position. And that – and it was not just a matter of... attending a meeting and going to some meeting where they would say, “Okay this is what you need to do for the interview and so on.” It was actually the culture. And by the culture, what I mean by that is that, for example, the fundings agencies would come to Berkeley or a lot of visitors would come to Berkeley and you would be exposed to all these people. And – or when funding agencies would come and they say, “This is where the future of the field is heading towards.” And there would be these meetings, you would actually actively participate in these meetings. So when you came out, you didn’t just come out as a student that was simply had a research accomplishment, but you came out as a student who that had a research accomplishment, but also had networked with a lot of very important people in the field and also had networked with people in agencies that were pointing to where the future is heading. And you also had networked with people that had very strong views about where research and education go. So you kind of inherit a little bit that culture and you’re being very well prepared, even though you don’t – it’s not part of necessarily of a formal process of mentorship. It’s part of basically just being there and participating. And I think, of course, your advisor is very important in creating that environment where if visitors come, I remember, I was there for a month and a visitor, very famous visitor came and my advisor asked me, “Well don’t you present something.” And you get constantly, basically into that culture or funding agency representative would come from NSF or DOD and then you would actually go for coffee with them to discuss your own views, even though you’re very young about where the future is heading. And you get basically into this mindset of what’s coming up next. What’s coming – which is not necessarily the more traditional advising paradigm, which is okay, here’s your problem, solve it. You get your Ph.D. and you move on. So I think that I felt was very – that has really affected the way I think as a researcher and also behave and operate as a researcher ever since basically

Interesting Questions and Projects

Q:

You mentioned that everyone was interested in these big problems. What were some of the big problems at the time?

George Pappas:

I think at the time at least from control or robotics is – some of the issues were, I mean, at least that I was involved were how to think about large scale infrastructure systems and control for them. So highway automated, highway systems at the time was a very big project at least at Berkeley. Air traffic transportation was also a very big problem. So those were the ones that at least, that were within my intellectual neighborhood. And over time, I think then it has transitioned much more to sort of multi agent robotics, and then it moved a little bit towards sensor networks and so on. But that was after my time at Berkeley. When I was at Berkeley the automated highway project was a very, very large project which had lots of graduate students and so on. And so that was very unique, yeah.

Q:

Do you have any idea where the prominence of the project is? So was it from – go ah – I was wondering is it for funding? Or from somebody’s idea? Or what was the – where did...

George Pappas:

Yeah, so, I guess the idea, and the leader here is really Professor Pravin Varaiya, who has had another impact in my own personal development. I mean, the basic idea is that infrastructure’s not going to change. We’re not going to dramatically change or double the number of highways. So the question’s how can we be smart about the existing highways. And I think then ideas from automation, robotics and control can really help in increasing efficiency of the infrastructure and the idea was to take existing vehicles and add some intelligence on top. So that they can actually have much closer spacing, spacing that humans cannot do on their own, but you really need computers to sort of to monitor the spacing and control it to very tight and increase highway capacity by some factor of three or four. So I think that was the vision, and then the funding came predominantly from the California transport – Department of Transportation, Caltrans, and there’s the Path Project and that involves also collaboration along on one side the robotics, the control automation people but also the transportation engineers and so on. So that was also another unique feature that it brought all these communities together. Now, the project matured technologically and it went to, I think, in sometime in 1996 there was a rough demonstration Highway 5 at UC San Diego, where they were actually able to show a platoon of vehicles doing some kind of like a soft train, with very tight spacing and so on. So technological viability and visibility was demonstrated, then from that point on the question is, okay do you transition that in real highways and so on. And of course, the answer is, you never really transition the full blown idea. So I think what has happened, of course, is a lot of components of that big vision have been already introduced in a lot of our vehicles. So for example, if you see what’s happening in the automotive industry, a lot of the features they talk about is adoptive cruise control, or intelligent cruise control, or safety warnings, and a lot of those – a lot of that technological feasibility really goes back to large projects like that which, of course, were not implemented at full scale by say, transportation departments, but a lot of the components are making their way into the automotive industry right now. And I think, Volvo, I believe last year, or last month is introducing some of the spacing, actually controllers, in their next line up of Volvo cars.

Q:

What were some of the interesting questions in terms of robotics and control?

George Pappas:

I think collision avoidance, conflict resolution, how to so there were two layers of planning there. So there was the layer of planning of how do I go from here to New York, which highways do I take, how do I make lane changes? How do I enter the highway? How do I exit the highway? So this was the more – is the more discreet level of planning and control, and how do you make potentially changes to the plan if something happens with failures and so on. So that was one layer, and the other layer was the more low level, which is how do you keep your distance to the car in front? What kind of information do you exchange with cars in front and behind? And what is the information infrastructure that you have to share with the robotic cars in this kind string of vehicles in order to sustain the kind of desired spacing. Do you exchange position information? Do you exchange velocity information? Relative velocity? Relative acceleration? What is the level of – what’s the level information and how often do you need transmit in order to have very tight spacing to do this. And then the question is, if a car wants to enter or exit the highway, then you how do you break the platoon? How do you get out and so on? So that was on the automated highway side. I think on the similar kind of problems arise in the air traffic control arena where the principle at least at the time, was to do free flight, which was that every air craft would sort of choose its own path... without using the sort of the grid, the virtual grid that’s on top of the United States and sort of navigating because now you have GPS, you can pretty much fly anywhere, and then the big challenge there is conflict resolution. If two aircrafts or three or four, especially as you get closer to airports get in sort of head on collisions, or any type of collision, then what are the algorithms by which you sort of avoid collisions, resolves conflicts and so on. And I think that is very much a problem that people in robotics have looked at with robots and multiple robot settings, or robot avoiding obstacles, of how to avoid obstacles and so on. So there was a lot of exchange at the time between the air traffic control and the robotics industry. The problem with the air traffic control industry in compared to the automotive industry is that it’s highly regulated and it was a lot more – it still is a lot more challenging to get sort of technological advances into the regulated FAA system as is. Whereas in the automotive industry is a little bit easier to get – not the full blown agenda but parts of it.

Q:

Were you able to see any of your work implemented by the FAA?

George Pappas:

No, that’s very hard but you can – what you can – there were pieces that were – that was a DARPA SEC program which was taking some of the conflict resolution work that was happening at Berkeley and they were able to demonstrate it in some experimental flights with just two aircraft. So again, technological viability were shown, but to make that sort of full blown, full scale... project that the FAA, that was a much harder. And, of course, now with President Obama Next Gen. initiative, a lot of those ideas maybe not at a full blown free flight agenda are sort of resurfacing as how to make it at least more intelligent and how to help air traffic control advisors make intelligent decisions and reduce their workload while making airlines a bit more efficient, reduce fuel and so on.

Q:

And so was this all still work that you were doing during your Ph.D. or...?

George Pappas:

Yes, and a lot of this was of course, in collaboration with a lot of the people that I’ve already mentioned and a lot of it is, in some sense, still going on by some of the collaborators and so on yeah.

UPenn

Q:

So what did you do after your Ph.D.?

George Pappas:

Yeah, so when I came to Penn as an assistant professor, Penn of course is a very... internationally renowned place for robotics. So it was very clear to start working a little bit on what, where the new challenges, not necessarily in air traffic or in automated highway systems, but actually in robotics itself. And it was clear that – I mean, at least there were two major themes evolving. One was going from a single robot to multiple robots. I mean that transition was sort of starting in early 2000 and in some sense it’s still happening now. And then the other one is to sort of look at connections between robotics and biology. And I think, there’s multiple dimensions in that front. So then the – going from a single robot to multiple robots was very well aligned in some sense with the problems of intelligent cars on a highway, and intelligent aircraft. So I think that was with a different focus that was a new set of problems that we decided to work on while at Penn. And a lot of us at Penn actually in the GRASP Lab are working on different aspects of the multi robot coordination problem. And so more specifically the problems where things as – robots are different, they typically have a team of robots. They’re smaller, so they’re more disposable, more unreliable, but compared to air traffic control, they typically have a common mission. So they wanted to cover it, or they want to cover a particular area, or they want to collect information, say over a search site and so on. And then you have many of them, and the question then is how do you on one side coordinate? So the coordinate is who does what, who talks to who, and what kind of information against they exchange with each other. And then the ques – that’s the higher level. The lower level is how do they go out and they do, say trajectory planning, control, and so on. So that they can achieve the common team mission, and how do they go from a team objective to individual objectives is the kind of main challenge in that topic. And I think that has brought robotics in some sense much closer to areas such as communications and networking and formation flow and so on. And signal processing and I think that was intellectually very appealing and so on. The other dimension that was happening is that with there is growing interface between robotics and biology on many fronts. One is that people look at bio-inspired robotics, and even that is – has multiple dimensions. So you can look at bio inspired robotics from a locomotion perspective. For example Dan Koditschek is working on how to look at cockroaches and make sort of legged robots with very robust locomotion systems, so on and so on. Then there is bio-inspired robotics even at the coordination level. So for example if you look at a team of ants, how do they cooperate to achieve a common goal, like searching for food and how can those principles be translated in the multi robot agenda. So that was - that’s another dimension. And then there is a third one with a bit more exotic and a bit more forward thinking is that – I mean, if you look at robots, the way we build robots right now is basically out of mechanical and electrical parts. And so, we program them to sort of achieve the – to get these anthropomorphic robots, or the cars, the intelligent cars, of the – iRobot, Roomba and so on. But as we understand a little bit biology better, at both the organism level, but also at the cellular level, I mean, people are starting to explore also how to build, sort of robots at the cellular level. So rather than thinking of silicone as a substrate by which you build robots, now you’re looking at basically building cells that you can program cells to act like biosensors, bio-actuators, and bio-computation and they can perhaps, go inside your body and send certain chemicals and then possibly interact with the environment in a way that typical robots would do in the mechanical world, but now we’re starting to think about if you like robotic biology, making biology even more robotic and so on. And that’s a bit more forward thinking but that’s another thing that has evolved over my experience here at Penn.

Q:

And in doing – and this is research that you were doing or research that was being done by various people in the lab?

George Pappas:

Yeah, so I think a lot of this research has been done with a lot of my – the members of my group. A lot of my students, my former post docs, and former students, but also in collaboration with a lot of the other groups in the GRASP lab. And I think, as I was saying before, at GRASP lab, we feel we do have a little bit of the open collaborative culture as that we saw at Berkeley, both among different groups that bring different expertise. So one can bring the algorithmic, one can bring the experimental, or one could be a little bit more bio-oriented, or one could be a little bit more mechanical, one could be more electrical and so on. And – but also with a mentorship culture, and I think that we really try to.... the success of our students is basically our own success. So we try to promote that open culture also to our students so that they could collaborate with each other among different groups and kind of pursue all of these things. So obviously we contribute different parts and different pieces, but I think the important thing is – as a collective accomplishment among my group, but also among other – with other groups as well.

Students and Collaborations

Q:

Who would you say are your closest collaborators?

George Pappas:

For a lot of the multi robot work, it would be Vijay Kumar on the more robotics and experimental side as well as algorithmic side. And then on the algorithmic side it would be Professor Ali Jadbabaie, who is working on more of the mathematics of multi-agent systems. On the biology side, again a lot of this work has been with Professor Vijay Kumar, but also with some biologists. The biologist are from the cellular level at Penn, it’s Harvey Rubin in the School of Medicine, but also... other folks at the organism level so I’m the P.I. for biologically inspired projects which involves both lots of engineers, robotic engineers, but also lots of biologists. And the idea there is to look at biologically inspired principles of cooperation. And how, for example, transitioning a lot of the ants work into, for example, collaboration in ants to collaboration of robots. And there the interesting thing is even when you get different biologists together, even different biologists don’t really – the ants people don’t really talk to the dolphins people and so on. So there’s a lot of interesting intellectual discussion happening even on the biology side. And then you have interesting interaction between cooperation principles in biology and cooperation principles in engineering. What’s different, what’s common, and what makes sense on one side? What makes sense on both sides and so on. But of course, most of the collaboration is really with your own students and your own postdocs, who have now – I mean, some of them. Of course, here but a lot of them are now on to faculty positions elsewhere.

Q:

Who were some of your early students who’ve become faculty?

George Pappas:

Paulo Tabuada is now at UCLA, is an associate professor at UCLA. In the area of multi-robots systems one of the prime works on flocking of – when you look at sort of the movies, like Lion King and you see lots of lions flocking, so a lot of the flocking work is based on – at least some of the algorithmic guarantees that we have shown is based on the work with Ali Jadbabaie, with my former post doc Herbert Tanner, who is now at the University of Delaware. A lot of the work on human sort of, I would call it more – so in other vision – another challenge that I see going forward is how would we interact with robots at a very high level of obstruction. Especially as robots get into broad society and you have the – everybody interacting with robots, we cannot be programing robots using C or Java, we have to actually be able to speak to a robot. And that would be the way that, for example, you would want your kids or your parents or your grandparents to potentially interact with a robot. So the question then is how would you speak to a robot. And I think that area of human robot interaction and higher level interaction mechanisms that would make robotics a bit – penetrate more in broader sight are very important for the field and there we have done some work with Hadas Kress-Gazit who she’s now at Cornell University. And she’s following up that agenda of, kind of the Night Rider, how would you speak to a robot and the robot would actually do what – how would you understand what you’re saying. On one side and then how would you execute exactly what you’re doing. So I think that’s another main topic that I see going forward that Hadas has really, I think pioneered. But then there’s a lot of other collaborators Agung Julius, he’s now faculty at Rensselaer. He has really worked on the interface between robotics and biology and how to – and collaboration with Selman Sakar, who just one this award who is now actually postdocing at MIT. There’s – I have – there’s quite a few members that I feel very proud and blessed that they were actually members of my group.

Robot Applications

Q:

So you mentioned a lot of different types of projects looking both at biological direction or the multi-robot. Were there particular applications that those were related to?

George Pappas:

Yeah, so... I tend to work mostly on the how to think about robots, rather than building them. But my thinking is always in collaboration with people that actually build them. So there are some projects for which we had to demonstration, for example, in the multi-robot space, we had to develop algorithms that – others than develop robots that flew them. In the work we did for sort of talking to robots, we applied it to the DARPA Urban Challenge in the semifinal run to see that the algorithms would actually work in that setting. Again, we didn’t... use our algorithms during the run, but we actually post challenge – after the challenge concluded, we actually took a semifinal run and – so we always ground our algorithmic developments on real problems. In fact, I don’t like to ground our algorithmic developments on other people’s algorithmic developments but I really wanted to ground them on real. And then, a lot of the real applications – typically, excuse me, in collaboration with others. Because I don’t consider myself to be a very good experimentalist, but there are much better people on that in the GRASP lab but I do interact a lot with them. So, yeah.

Q:

So what inspires the particular real world problems or applications that you think of?

George Pappas:

I tried to talk to experimentalist, as well as a lot of the industry people, as well as people in the field that use robotics, for example in the case of – in the Department of Defense funding, I tend to talk to a lot of people that actually use this systems rather than people that develop these systems as to what do they see as problems. And from that I usually draw motivation about the problems that I work on. And the problems I work on are usually on the algorithmic side, however, I usually draw motivation from people in industry that say that this is a big problem for us. To me, that’s very important to sort of acknowledge the importance of the problem and that there is sort of a market for the solution of such a problem out there. But I also need to make sure that that is a problem that would lead to – that’s the market side, but then, we are all in the Ph.D. training business. So at the same time, you have to take that problem and select parts of it that would lead to good Ph.D. thesis and good careers for the students. Not all problems that come from users, or from – or from industry necessarily are good academic problems, and vice versa. So, and I think that – that is usually a challenging part. Identifying problems that have importance in a broad setting, but also academic interest, so that if people address them, they can go out, write nice papers, and present in conferences, and on side, convince your academic community that this is a very interesting, elegant problem. But at the same time, it’s at least motivated by a problem of importance rather than identifying an intellectual hold and going and covering it without necessarily identifying whether that hole has a need first and so on. And I – so basically a lot of it comes in discussion with industry, in discussion with.... with... funding agencies. A lot of these industrial workshops where they basically clearly say, “Well, you know, we need ways to, say, interact with robots – say at higher levels.” Now of course, how you translate that down to a more technical challenge, is a challenge in its own right. But I think it points you in the right direction. Because identifying the right direction, I think, to me is half of the battle. Because after – lots of faculty are very intelligent, and lots of students are very intelligent. I think it’s really pointing them in the right direction that is the – half of the battle. Because then you know that their talents will address the problems that are in front of them. But it’s really pointing them in the right direction. And then the right direction also means a problem that has long term professional growth for the students, which is another dimension one has to consider as well.

Future Challenges of Robotics

Q:

In terms of the big problems that robotics faces which do you see are the ones that – if we really made progress on them could really advance robotics in a big way?

George Pappas:

Yeah, so I think since we already mentioned, I think this issue of what I call, human robot interaction is very important. It is perhaps, under attended, it’s a difficult problem to even formulate, and I think there we should be talking on the other side of the human robot interaction which is potentially our friends in cognitive science and so on. And I think that’s very important as an area. The interactions with biology, I think will be strong at multiple levels, whether that’s from the biomechanical level, all the way to the bio-organizational level. Or to the cellular level where we think of cells as robots and how we try to program cells the way we try to program robots. I think the – however, I think one of the major problems in robotics that I think is - they are from the beginning and is not been really addressed in a satisfactory way. And is that robotics is in some sense, very well aligned with what I would call the science of integration. And the science of integration basically is that if you look what a robot is, it is essentially, you have a legged robot or you have a automotive car or an air craft, or an unmanned area vehicle, or manipulator, and then at some point, you have sensing, you have actuation, and you have computing. And how do we integrate sensing actuation, computing, physics, software in a way that it is fast and rapid and reusable, and modular, is in some sense not well addressed. What part of robotic science is platform dependent? What part of robotics is platform independent? I think that’s – what are the platform abstractions like the way the TCP was able to be an abstraction for communications, where people could develop applications for networking, and then people can make networking faster. So what are the parts of planning algorithms say that would work for both cars and aircraft. And what is the part of robotics that really depends on the physical particular aspects of this robot or that robot? And I think my concern is that every time we have a new robot, we are doing a lot of vertical integration, we have a robot, we build Kalman filters, we build controllers and then we build the hierarchy all the way up. And then if we go to a different robot, we do exactly the same thing. And how do we move laterally, accomplishments from one platform to another? Is very important for enabling, what I would call the science of intelligent machines. Because it’s not clear, what part of the science is platform dependent, and what part of it is platform independent. And I think that to me, is at the center of robotics in the intellectual center of robotics. It needs to be addressed by a larger percentage of our community.

Q:

Do you think the...

George Pappas:

And I – in fact, if I can add, I think there, there are also opportunities to discuss in other sciences that have achieved these type of... hierarchical or cross layer. So networking is a discipline that was able to come up with an architecture that has served them well. In the area of computer science, people can talk about modularity and reusing software components without necessarily rewriting the code from scratch all the time. Now, of course, there are unique challenges to robotics and that is that this robot may have two legs or four legs, or be flying. So at some level they have to be different, and you also are dealing with an environment that is uncertain, and I think whereas in a lot of these other domains, the environment is very well controlled. But I think that is the central question that needs to be addressed.

Q:

Do you think it’s going to be in the end just more through standards or through the open source like the ROS or...?

George Pappas:

I mean, standards are important and they identify interface but I think if you do standards too quickly, it can also be a problem. And so, ideally what I would like to see is the following thing. I mean, you buy a robot, you and you can go on the web and download a Kalman filter and a controller and within a day, you have it working quickly. So you shouldn’t have to write three papers and have two students working on two years to do that. So if we can do that, then we will know what part of robotic development is platform dependent, and what part of robotics is platform independent.

Q:

Do you think one of those robot operating systems is a wave in the right direction?

George Pappas:

I think that will help in the – in making a lot of the integration, software integration challenges more robust, and faster and better, and will be more available. Now, whether the – what is probably missing in the – is that when should I use this Kalman filter versus that Kalman filter. And I think that type of – so one should enrich such projects with some kind of a process by which I know that if I download this piece of – this module from the robotic operating system, I know that it will work in these situations for these types of robots, as priori, as I download it. And that type of type checking I think is important to have some kind of faith that what I’m downloading will work. Otherwise you will quickly integrate it and you will test whether it works or not. But if you want to have any faith, or confidence that it will work well, during the rapid design time, I think. So I think that is really important to quickly assemble intelligent systems or re – quickly reconfigure intelligent systems as the – say the environment changes or some sensor fails or some actuator is installed on the robot, I think. So I think that to me is a central question that I see, that everybody deals with but nobody addresses it head on in some sense, yeah.

Personal Contributions to Robotics

Q:

What do you think is – are some of your most important contributions to robotics?

George Pappas:

My students. I think, I actually believe that very strongly. But – and so, if I look at.... I’m just going to name, say three. So I think the work by Herbert Tanner on the flocking, I think and – is a fantastic contribution. I think the work by Hadas Kress-Gazit, she’s working on sort of linguistic control of robots. And how do you take English sentences and you translate that down to a robotic plans. If you say, for example if you say something like, “Go behind the building and look for survivors,” and how does a robot take such a sentence and translate that down into a trajectory and knows what behind and what building. I think that type of automation, at higher levels, connecting higher levels of reasoning, and communication to low level execution I think is important, and it is related to this science of integration of how do you go from high level, potentially platform independent plans, to low level platform dependent executions. And then I would say a lot of the work on the multi-robot systems where you can view robots not as – not so much as robots that move things from here to there, or they go somewhere, but actually you think of robots as extra communication agents where a robot – you have a team of robots and a lot of them for example can spread over an area to say, provide a communication infrastructure network so that one robot can talk with another robot via other robots. So robots need to – in a distributed way reconfigure to route information from one robot to another. I’m just naming three, this was by the Ph.D. work of Michael Zavlanos. But I think, I’m very proud of my students, and for every student that came through – sort of come up with such a story.

Q:

Three is the magic number.

George Pappas:

Three is the magic number. Okay. Good.

Q:

Have you had public response to your work? And what has it been?

George Pappas:

Yeah...

Q:

This might be a difficult question, because you mentioned a number...

George Pappas:

And I understand, yeah.

Q:

Of things that have to do with actual applications that are very imaginable.

George Pappas:

Most of the kind of the.... the public awareness of this type of work, usually comes from the angle of the Departments of Defense funding. And as such, it typically comes from what I would call search and rescue type missions, or – so... I think there has been press releases and so on and outlets in the media that have looked into this. But a lot of the times you have – it really comes from with a kind of military tone which I – sometimes I have very mixed feelings because... it’s not necessarily focusing so much on the scientific aspects of the work, which of course could be very broad and very dual use and so on. But really looks at it from a military view point. So I think others in the GRASP lab have had, of course, tremendous success, because they usually the public awareness comes not so much from the thinking about robots, but actually the application of robots in the domain. So I think in that setting of course, Vijay Kumar and Dan Koditschek and so on, and others have a lot more public.... exposure of their work, in some sense. I don’t know if that answers it but...

Robotics and Society

Q:

Definitely. I mean, how do you envision if you do the social relevance of robotics research?

George Pappas:

There’s two aspects of it. I mean, robotics is always is b – I would like to decompose the robotics in two parts. One is the technology of robots. And I think there, when you look at the technology of robots, you are really looking at the robotic platforms. You’re looking at whether this is an automated car, whether is a vacuum, automated vacuum cleaner, or is a manufacturing manipulator and so on. I think there, the social relevance is much more – its impact on industry, social welfare, economy, and so on. Jobs created, and I think there some – I think robotics may very well be in the sort of... I would call it the IBM to Microsoft transition of the late ‘70’s where there was hardware, garage style hardware. But now, we’ve sort of ROS-like investments and serious operating systems and with building sort of an application layer on top of robotics one could see sort of a reemergence and resurgence of lots of applications of robotics and lots of the platforms of robotics. So I think that – I think we could be sort of in the middle of a transformative period for robotics from the platform point of view, I think. And then there’s what I would call the science of robotics. And the science of robotics is everywhere. I mean, it really is everywhere. It’s in the unmanned vehicles that Northrop Grumman has. It is going to be actually in green buildings. It’s going to be – it’s already in next generation automotive systems and so on. So I think that’s not necessarily – I think one of the challenges in robotics is that in the broad perception it has very much this anthropomorphic image. And that is a good thing and a bad thing. So when you say a robot to somebody, they know exactly what you mean, but bad thing is that they know exactly what you mean. And I think when – and I think a robotics needs to sort of claim a lot of the intellectual successes that have happened in domains that are not necessarily anthropomorphic which are sort of areas such as unmanned vehicles, potentially automotive systems, air traffic transportation and so on which use a lot of the components of the science of robotics, but they’re not necessarily applied to what people perceive as robotic platforms. So one has to basically view it at both angles.

Q:

Why do you think there’s a rising focus on human robot interaction? This may be just a personally interesting question to me.

George Pappas:

I think part of the reason why there is a need is because I do think that the level of automation robotic platforms has moved up – let’s say in the ‘70’s you would be doing simple robotic, go from point A to point B, and then and so on. And I think now we are basically able to have levels of intelligence which is basically, “Go to New York.” And I think now you’re getting at a level of intelligence, or that machines are getting at a level of intelligence that is a lot easier to basically be at a communication level with a human. And I think with the broad public, you can have a lot of people programing it using C, of going to point A to point B. But as we get robots more and more and more intelligence, what has happened over the 20 years, I think now we are at a stage, where I believe, robots and humans can actually really interact because the level of automation robotics has really increased to the point that we can actually plan at a very high level obstruction, a high level of obstruction that can actually be very close to the level that humans interact with other humans. So I think that is, to me, at least why – so it’s really the progress that has been made at the level of increasing robotic autonomy that we can actually now start thinking about human robot interactions the way that would be meaningful for humans to actually use robots and program robots the way they would speak to another colleague or another friend or family member. So I think that wa – that’s why I think investments are needed in that area so that robots really penetrate broader society. And I think that could be a tremendous way that – by which not just robotics, but broadly automation and intelligence is much more pervasive in our homes, and offices, and so on.

GRASP Lab

Q:

You directed the GRASP lab for a while. Could you tell us a little bit about that experience and how you think the lab has changed through time or...?

George Pappas:

Yeah, so I was the director of the GRASP lab from – for three years – from 2005 to 2008. And the emphasis I put as a director is to help collaborative research efforts. So I think the main emphasis was to sort of look at collaborative research opportunities and increase the collaboration within the lab, now that it was already... very high, but it was actually more going out and getting a lot of research grants and proposals and so on. And of course, that was very well aligned with what is really happening in the funding agency. So – which is transitioning from sort of a single P.I. to multiple P.I. funding model in some sense. So in other – so that was one major objective. The other one which I’m sort of proud of is that we started the Master’s in Robotics. Which is the second master’s – was at that time was the second master’s, which was a very focused program in robotics. And it was very unique because areas such as robotics are very hard to do from any particular discipline, whether that’s electrical engineering, mechanical engineering, or computer science. And so there were lots of students that basically said that, “I want to do robotics, I don’t want to do computer science. I don’t want to do electrical, I really want to do robotics.” So there was a clear educational need there. And so, we started the program which is administered by the lab and it exposes students to sort of the right balance of electrical engineering, mechanical engineering and computer science. Bringing in sort of AI and control, and computer vision, and mechatronics and so on. And it’s now satellites to – we’re still in growth mode, but 40 to 50 students, and I think that is another way by which the lab can have impact. Because at the end of the day we – as I was saying before we are an educational institution and our goal is to not do research or education in isolation, but rather couple it too. And such programs enable you to take the research excellence and sort of convert that into educational...

George Pappas:

Yeah, so I think the industry is very important for any educational program in my opinion. Especially for a master’s more targeted program. And especially for a program that’s new. I think that’s very important. Students, you have to look at it from the students type, they come to a program, it’s a new program, they want to know that there future is both interesting, exciting and secure. And – so they, so I think we...

George Pappas:

Well in three ways, either they go to what you would call robotic industries like iRobot and so on. Or they would go to other industry that may be broadly robotics. They’re interested in the science of robotics, not necessarily the platforms of robotics or they could be sort of the Lockheed Martins and so on. And then, a lot of them also continue, they’re inspired and they continue on to do Ph.D. either at Penn or elsewhere. So I think – and we have been doing extremely well with the program. So that was another... aspect that I focused on during... being the director. And the other one was just broadly making the world a little bit more aware, the academic world in particular more aware about what’s happening here, by having very distinguished speakers and so on. And I think that is – that’s to me very important education for everybody. I mean, the faculty, the students, because you can take courses, and you can do research but you also have to network your community with the external community so that they have a very important sense of what’s happening externally. So – and I think, now the lab has grown tremendously over the past five, six, seven years. Also in the – while Vijay Kumar was the director, as well as, Costas, and I think now, right now we may be producing actually the most Ph.D. students than any other institution in the area of robotics in the United States.

Q:

How many do you have?

George Pappas:

We have about 70 to 80 Ph.D. students and I believe CMU has a similar number.

Q:

But CMU has a lot more faculty.

George Pappas:

A lot more faculty, yes. Yes. But if you look at it from educational standpoint and if you think your impact is through your students, then I think – I agree. So if you think that impact – CMU’s a larger place and they have much more – many more researchers, but when it comes to students, and getting students out there, at least Ph.D. students, we are about comparable right now.

Advice to Young People

Q:

So since we’re – unless Peter has some other question that he’d like to ask, since we’re at the student’s should we do the closing statement?

Peter:

Sure.

George Pappas:

Closing statement.

Q:

Our general closing statement is this question, so if – do you have some advice that you would give to young people who are interested in robotics?

George Pappas:

So my advice intellectually would be focus on hard problems. And there are certain hard problems that have been with us from the beginning and there are certain hard problems that emerge. So, for example, I would call the science of integration as a problem that has been us from the beginning. And I think their hard problems like human-robot interaction, like multi-robot systems that are emerging or connecting to biology and so on. The other one is you should have one leg in robotics, and one leg somewhere else. Be it interdisciplinary and that, that has it’s – of course it sounds great, everybody’s doing it, but other – people that really do it know that it’s very challenging. You have two communities to make happy, there are metrics of success in different communities is very different but the reward can be very high, both intellectually, but also from a career standpoint. So I think that – so whether you want to do robotics in biology, or whether you want to do robotics in networking, or whether you want to do the sort of – those types of interactions or robotics in cognitive science, I think that’s where some of the bigger problems as robots become more and more intelligence – more intelligent, we will have to embrace more and more disciplines outside robotics which focused more on the platforms but as we go higher in intelligence we’ll have to talk to cognitive scientist. As we go bi – more cellular we’ll talk to biologist and so on. So you have to be intellectually open and in some sense brave to make that transition. And that may delay your Ph.D. it may be difficult at the beginning, you may feel that you’re slowing down. But I think in the long term, it pays off. Third, I think is to pick your advisor well. I think this is something I have learned. I think a lot of students pick projects well, but I’m not so sure they pick advisors as well as they pick projects. They’re extremely excited because they really want to do this great project which is – and so on. But being with an advisor, and I’ve been on the – being with an advisor is a very special relationship. The mentoring you will receive from a great advisor is by far the best lesson you will – the most important lesson of your life potentially, professional life. And I think, so I think there one has to look at advisors that have very good track records of advising and so on. And I think that’s something that I would think – I think that student’s should be – when they make graduate applications they should also – they should be looking how great the projects are. But they should also have tried to identify or estimate how good the advisor has been or will be and so on. Because you could be at a great place, in a great project but if your advisor – if that chemistry is not good, and if that advisor is not looking at your own interests as the metric of success, then there could be conflicts and so on. I don’t know what else.

Q:

How would you pick a good advisor? Especially, when you’re a student and you’re in the program?

George Pappas:

Yeah, so I think for – I think for more established research, researchers and so on, you could sort of look at their track record and whether they.... whether they – a lot of their former students are in academia, or industry leaders or whatever. Depending on what your goals are you should be looking. Now that’s a little bit unfair for more es – young faculty. But for more established faculty that’s one thing. And for established faculty another thing you can do is you can look at their students and you can potential ping them, and ask them for, “How was it working with X, Y, Z and so on?” And I think that could provide some meaningful advice. Not everybody’s going to have the same response but that could be the mechanisms that you could pursue and so on. Now that’s not fair for young faculty that are just starting and they may or may not have a track record. So I think there you’re just going to have to go with a gut feeling. Sometimes it could be – this could be the most exciting advisors because they’re full of energy. They have lots of ideas, they just came out of their Ph.D. or postdoc, and so on. So I think there, you’re just going to have make a little bit of a judgment call, and you could also see a little bit the surrounding culture. So for example if that person is in a center, or a lab, where there is that type of a culture then that type of culture will sort of spill over from one person to the other. Another thing you can look at people’s collaborative credentials. So if you see that they write papers with others, then you can have a good sense that these people are open. What – if you see that it’s basically my student and the faculty advisor and all their papers are that, then it will be more of a pairwise relationship between you and your advisor. So depending on what your goals are, I think you can sort of try to look at hints of that when you pick an advisor. I think picking the right advisor for me, especially was extre – can be completely life changing experience, and so, I was very fortunate to have tremendous advising, advisors and mentors in my life.