Oral-History:Frederic Kaplan
About Frederic Kaplan
Frederic Kaplan was born in Paris, France. He graduated with an engineering degree from the Ecole Nationale Superieure des Telecommunications in 1997 and went on to earn a Ph.D. in Artificial Intelligence from Université Pierre et Marie Curie (Paris VI) in 2000. He then joined the Sony Computer Science Laboratory in the robotics department from 1997 to 2006, where he helped develop the AIBO robot in Japan. Following this, he worked as a researcher for six years in the Rolex Learning Center at EPFL. In 2012 he joined the teaching staff at EPFL as Professor and Chair of Digital Humanities, and became director of the newly founded Digital Humanities Laboratory (DHLAB). Kaplan is also co-founder of the OZWE, a 2008 startup company which designs and manufactures interfaces, electronic products, and digital publications, and Executive Director of Bookapp.com, a company he helped launch in 2010 with Laurent Bolli and Cristiana Freitas. Currently, he is directing the "Venice Time Machine" project in collaborations with Ca'Foscari University in Venice and the Venice State Archives, as well as other museum and exhibition projects. Additionally, he is chief editor of Frontiers in Digital Humanities and co-directs the Digital Humanities book collection at EPFL Press.
Kaplan's research interests include innovative interfaces, digital humanities, artificial intelligence, robotics, developmental systems, multi-agent systems, language acquisition, and gesture interaction. His work in robotics includes contributions to several projects, such as the AIBO robot, Wizkid, and QB1, as well as more than a hundred scientific papers, six books, and about ten patents.
In this interview, Kaplan outlines his career in robotics, starting from Sony all the way to his position at EPFL. He discusses his contributions to various robotics projects, including the Sony AIBO robot, and the design influences from biology, psychology, and art. Commenting on his interactions and collaborations with Luc Steels and other roboticists, his experience with his startup and the robot-art movement, and the future challenges of the field, he also provides advice for young people interested in a career in robotics.
About the Interview
FREDERIC KAPLAN: An Interview Conducted by Peter Asaro, IEEE History Center, 18 July 2011.
Interview #725 for Indiana University and IEEE History Center, The Institute of Electrical and Electronics Engineers Inc.
Copyright Statement
This manuscript is being made available for research purposes only. All literary rights in the manuscript, including the right to publish, are reserved to Indiana University and to the IEEE History Center. No part of the manuscript may be quoted for publication without the written permission of the Director of IEEE History Center.
Request for permission to quote for publication should be addressed to the IEEE History Center Oral History Program, IEEE History Center, 445 Hoes Lane, Piscataway, NJ 08854 USA or ieee-history@ieee.org. It should include identification of the specific passages to be quoted, anticipated use of the passages, and identification of the user. Inquiries concerning the original video recording should be sent to Professor Selma Sabanovic, selmas@indiana.edu.
It is recommended that this oral history be cited as follows:
Frederic Kaplan, an oral history conducted in 2011 by Peter Asaro, Indiana University, Bloomington Indiana, for Indiana University and the IEEE.
Interview
INTERVIEWEE: Frederic Kaplan
INTERVIEWER: Peter Asaro
DATE: 18 July 2011
PLACE: Lausanne, Switzerland
Early Life and Education
Q:
So if you could just start by telling us where you were born and where you grew up and went to school.
Frederic Kaplan:
Yes, I was born in Paris; went to school in my neighborhood, like in the Fifth District, Latin Quarter. And then I went to an engineering school called the Ecole Nationale Superieure des Telecommunications. So it's the telecommunications school in France. I stayed there three or four years, specialized in artificial intelligence. And I wanted to do a Ph.D. at that point. I was kind of clear I wanted to do research in AI or robotics. That's a kind of childhood dream, anyway. But I was not so sure where to do it. I mean, I was planning to go to university.
Sony Computer Science Lab
Frederic Kaplan:
And then I got the opportunity of being involved in the setting up of the Sony Computer Science Lab in Paris, which was the only fundamental research lab that was out of Japan at that time. And when I discovered the lab, it was – I mean, we should remember it was a time where Sony was not doing computer science. I mean, there was no VAIO computers or – I mean, Sony was really well known more for hi-fi product. And I was not so sure about what it would mean to do research in computer science at Sony. But I met Luc Steels, who was the director of that lab, and he was someone I was – I knew from my textbook. I mean, he was with Rodney Brooks, one of the founders in new AI, embodied intelligent paradigm. I mean, really shifting from the traditional symbol of processing, reasoning kind of artificial intelligence to much more collective, embodied approaches. And so I was very lucky to meet him by chance, and I started I think as the first French employee in the freshly created Sony Lab. And I stayed there 10 years, basically.
Q:
What year did you start there?
Frederic Kaplan:
When? I started there in 1997. And started by doing my Ph.D. on multi-agent system and the evolution of language. I was modeling the origin of language. That was the theme of the laboratory. I mean, can we do with computer simulation on how language evolving? Can we tell what's happening when you introduce newcomers in a large population of speakers? And that was a very exciting time, where it was a branch of artificial life which was more focusing on simulation of biological processes, but we were doing that for cultural processes and we were – there was just a couple of labs in the world doing that. And so that was the main topic of my Ph.D., already very interdisciplinary in a sense that there was linguists, paleontologists – very, very exciting time. And the project that was the main project for my Ph.D. was actually creating a population of robots that had to agree on a common language. The name was Talking Heads experiments. Basically we had, in various places in the world, couples of robots that could look at a whiteboard with geometrical figures, and they had to agree on names for describing what they were seeing, typically like a red triangle, they would use like “kofuba’ or like an invented word for that red triangle. And the other robot would maybe not know that word, and would look where the first robot was looking – kind of joint attention – and make guesses and use the word in its own term in return. And we did an experiment that lasted something like six months – so kind of really big, real-time experiments where a population of something like several thousand robots were exchanging bodies – they were not like seven thousand physical bodies; they were like 20 or so different physical bodies, but they were swapping bodies, and we were simulating this creation of a new language among robots. And very interesting things were observed, like polysemy, for instance, between words. We had situation where one words like "bozu pie"– I think that was the – it was a word used for large object in a sense that – for let's say half of the population, robots were looking at the whiteboard, and when they were seeing an object with a lot of pixels, a big area, they would use "bozu pie" as saying, "This is a big object – bozu pie." And the other half of the population were using that word but for a wide object. They were just looking at the horizontal dimension of the object. And what was interesting was that this ambiguity was staying in the population because there was very, very few long and thin object. And so when one robot was saying, "Oh, this is bozu pie," the other one would say, "Oh yeah, that object, it's bozu pie too," although it was kind of two different notion. And so when we discovered that, we introduced a long and thin object, and then the ambiguity between these robots were revealed, and there was kind of reorganization when one word was used for wide object and a new word was recruited for big object, essentially. And what was funny is that although these robots, they have nothing in common with us – I mean, they have very simple body; their perception system was very different – but that type of ambiguity between large and wide object is something that emerged, and it's something that is actually happening in natural languages. Like in French, typically we use "large" is a sense of wide object, whereas in English "large" is for big or large surfaces. So it's intrinsically ambiguous in the evolution of language, and that reappeared in the experiment itself. And so that was very encouraging, seeing that in the future – so that was like 1999 – we would be able to say something about cultures based simply on artificial robotic experiments. And it's in some way the topic I've been pursuing and I'm still pursuing today with various type of other experiments. But that's one of my big convictions; that is, that human sciences are going to gain a lot from the arrival of new kind of experimental methodology involving AI and robotics in some cases. So that was my Ph.D. project. I published a book based on that.
Sony AIBO
Frederic Kaplan:
And by the time I was finishing, I got the chance of being involved in the beginning of the AIBO project, which was the Sony robotic dog. It was – at the beginning, that as really like rather secret project inside Sony, only done in a Japanese lab. And they had the first – they came to us in Paris and showed us the first prototypes. And because I was in some way by chance becoming specialist of language and robots, I got invited to stay longer in the Japanese lab in Tokyo, and I started to go very regularly to that, essentially to learn about this first prototype and being able to bring back one of this first prototype to Europe. That was actually a whole adventure to pass through customs with that thing. Anyway, but so I went to work directly with the Japanese team building the robot, and that was really, really exciting, because it was a time where like a couple of people in Tokyo were just bringing together all the necessary components that you had to do to build a robot, that at that time nobody else had done any four-legged robot as convincing as the AIBO was. It was not yet called the AIBO, but.
Q:
And who were those people?
Frederic Kaplan:
So, Masahiro Fujita was the chief architect and project manager on this project, and that was like – he's an amazing man. I mean, he's really – I've rarely seen all my life someone mastering so many engineering techniques and being capable of supervising such a complex project as building a robot like this one. And he was heading a team where they were very patiently, prototype after prototype, making that thing real. And the spirit behind that team was Toshi Doi, who was a very important engineer in Sony. He was one of the makers of the CD at that point, and he was the person convinced at that time at Sony that robotics would become something as big as the computer game industry, essentially. That was the idea, that Sony was making a lot of money with PlayStation, and they thought there would be an opportunity for doing the same in the new market that would be like real-life, essentially, and the PlayStation of the future in their idea would be a robot. So there was really a sense of pioneering something really new in that team, and I have really, really good memories of spending time with engineers there. And there were no – almost no documentation. I mean, you had to learn most of the things just by oral history and go sit by each engineer to understand how the vision system was working, how the operating system – because it was like a custom Sony operating system underneath – etcetera, etcetera. And so I learned all that. So I was just lucky to be at the right moment at the right time.
But came back with the first AIBO prototype in Europe, and started in Paris then to develop all the interaction part and linguistic part on that robot. And so we started to do experiments a bit, I would say, naively. I mean, we were trying to teach the robots some words about everyday objects. I mean, the whole idea on the AIBO was that it had to be entertaining in the long run. It was obviously entertaining when you were seeing it – I mean, it was moving very convincingly. But the whole idea was can we make that device stay interesting for more than a couple of minutes? I mean, can it stay interesting for several days, for several weeks, for several years? That was really the challenge. And the underlying hypothesis we were making is that it's going to be by teaching, it's going to be by spending time with the robot. It's going to be in a situation where you would see how the robot change based on your teaching that you're going to have really a value from that robot. Our objective would be – I mean, typically Sony was making like hi-fi system. If Sony would come to you and say, "Hey, I take your old hi-fi system and I give you the latest Sony hi-fi system," most people would say yes. Our objective with the AIBO research was to say that if someone come to your AIBO and say, "Hey, I give you the latest AIBO with all the new features," you say, "No. I want to keep my old AIBO because, hey, it has all this unique life trajectory, all this time we've been spending together. That's absolutely more valuable than the couple of new features you just added to your new product."
So that was the kind of underlying assumption. So we tried to work on making the robot know about surrounding, making it understanding its everyday environment; and, on something very concrete, making it recognize everyday objects that you would show it. So our typical day at that time was like you would go in the morning, you would show objects, like a ball, a small plastic cucumber, and you would say, "Ball, ball, ball," and "cucumber." And then you would ask, "What is it?" And it would say, "Oh, cucumber for the ball." And so you would start it again. And doing that every day, because in some way, there was no way of cheating. I mean, you had to show that ball on every lighting conditions, on every type of backgrounds, so that progressively it will learn. So that was kind of working, but very, very – it was very exhausting to teach. At some point I was lucky to have internship, and so they were doing the – <chuckles> – this teaching job for me. But still it was difficult. And we started to understand why it was difficult, and one of the reasons was that it was very difficult to share attention with the robot. The robot has a lot of different behaviors. He was attracted by different things. And so in many ways, when I was saying the word "ball," typically it would – when you would look at the actual data it was using to try to induce the relationship between the word and its perception, you would discover actually that it was not at all the – they were nothing in common. I mean, there wasn't like a – it would look at the wall when it would hear the word ball, and then the data was so corrupted in a machine-learning sense, that making sense of doing statistical inference on that was worthless. So we say, "Oh yeah, but that's obvious," because in fact what we were showing to the robot had no value for the robot.
We have in some way to get our robot interested in what we're showing to him. And so we started by building some triggering of attention based on movement, typically. Like I remember one summer I built an algorithm which was the idea of, okay, the robot is intrinsically interested by movement, and then if it sees like a reoccurring movement it will build a model out of that – kind of a very bottom-up approach, in which we were trying to push things in the world of the robot. It was really the idea that the robot has its own world, and that there's not necessarily an intersection with your world. And we were trying to make that intersection concrete. It's really the notion of "Umwelt" which was introduced by the Warner School in Germany 60 years ago – this notion of what is relevant in an environment for a given species. And for the robot, it was clear that this intersection was not sufficiently – it could be – I mean, if we would program the robot to be interested in the ball, then okay, the ball would exist; you could name it. But if you're not programming the robot in advance for being interested in a specific object, then it was very difficult to have the robot interested.
So we tried different things, and after couple of years or so, I started to work with one of my colleague, Pierre-Yves Oudeyer, who just finished his Ph.D. at that time, on the new idea. It was that we could – that the main problem was a motivation problem. The robot was not interested in what we were showing – in the object we were showing. So we had to work in some way not in the perception level, but one step deeper, I would say, at the motivation level. We had to have our robot intrinsically interested in some aspect of the environment, and based on that we would be able to teach words for the thing it would be interested in – that would be easy. But the difficult part was that generic, intrinsic motivation system, how we will call it. And there were a couple of other people working on that idea at that time, but that was kind of still a new idea in the sense that most of the research was on the reinforcement learning, with the idea of an extrinsic motivation. Typically many people were teaching robots what was good or what was bad by giving treats for good things and saying, "No, bad!" for bad things. And so implicitly, these robots, they could never go beyond this type of shaping that was really coming from an external point of view. And first that was making the old learning dependent on the teacher, and also it was – in some sense it's very difficult to tell what's easy to learn for a robot and what's not easy to learn.
So all our idea was to say, "We should build an architecture where the robot is modeling how it is learning." That's not so difficult in some way. I mean, a robot can predict – can learn to predict the effect of its action. I mean, if it's moving its arm, it could say, "I think my arm is going to be at that position in a couple of seconds." And then it could see whether it was at that position, whether it hit an object or not, etcetera, etcetera. And based on that, he has an error. If the error is small, then that means he's predicting very well what's happening to him. If the error is high, it means he has no clue whatsoever what's happening to him. And typically if you would look like at the television with snow on the television, I mean, that's just unpredictable. And many things in the world are unpredictable.
So, the whole idea was to say what the robot should try to do is find the things which are not completely understood, not with a very small error, not impossible to understand with high error, but things which can learn; which basically means things in which – situations in which the error is going down. So, it would measure the derivative of the error curve and look for situation where the error is going down. That means a situation in which he's learning. So the whole idea is to say we built a detector inside the robot that detects the condition of learning, and then we reinforce the behavior of the robot based on that. That means that the robot will actually decide the action that optimize the learning. That means that in any situation, you will say, "What can I do now to learn the most?" So I will avoid things I already know. I will avoid things that I know, as far as I can see, that I can't learn anything because it's too complex. I will focus on what we call progress niches, situations which are optimal for learning at a given point. Because obviously, imagine a situation – imagine two situations which seems to be promising for learning. If you focus on one, progressively you will learn everything you can on that and reach a plateau, and then do not receive reward anymore, and then focus on the other one, etcetera.
So that was the very generic principle, that was actually discovered – I mean, it came as a discovery really by deduction. But then once we in some way discovered that, we discovered that actually many other people have said something very similar really quite a long time ago, I mean, about the search for curiosity, the search for intrinsic motivation, the balance between easy challenge and difficult challenge – actually many different domains there were ideas like that. We had just rediscovered that in our naïve exploration. And so we built this kind of generic learning engine and tried to apply to it to many, many things. Like we had the robot learning how to walk based with that system. That means that he will just make random movement at the beginning, and discover at some point somewhere in the movement we're actually managing to go in one direction, typically going backward. And then our robot would go for one hour backward, until – I mean, going backward was not surprising at all anymore, and then he would discover that a small shift in his parameter would go a bit on the left, and he will start to go on the left, on the right for another hour; and typically on the third hour discover that he still had not tried to go forward, and that if he go forward, then there are other things to do – typically exploring spaces, sensory motor spaces, that could be as simple as the walking spaces or as complex as learning how to use object. And we did a whole series of experiments where we basically almost reproduced some of the Piaget steps on developmental scales showing how robots could learn to apply generic schemata on particular kind of objects without having any internal notion of the schema, or something like that. I mean, just by progressively developing a developmental pattern, a developmental sequence that would just emerge about as the result of the robot cooperating with its environment.
And that was very exciting, again, because it's – we were showing experimentally that the body had an importance that was as high as the software. I mean, typically the software inside that robot was always the same. We build this generic engine, and then we were just coupling a body attached to the engine. And imagine just you had a robot with slightly longer legs, then the size of these legs will actually change the whole developmental sequences because some type of walking would be learned before others. Then some other type of skills would be built on top of that. So we were showing that in fact you could code behavior into very concrete morphological dimensions, and that in some way understanding developmental pattern had to be done in that way, not as a kind of program in the DNA that is just unfolding itself, but that actually the size of the finger, the size of head, the global morphological consideration, may influence later on some other aspect.
Biological and Psychological Influences
Q:
To what extent did you draw on biological and psychological models in the actual design of your architecture?
Frederic Kaplan:
So we were never – I mean, we read a lot on psychological model. We were definitely aware of the literature and – but in some way I think we always tried to have an engineering approach, in the sense that we were trying to think of a problem as problems to receive solution, and that's how we were creating the algorithms, not trying to be biologically inspired for the sake of biologically inspired. I'm personally not convinced about imitating nature. I'm much more convinced by trying to understand what the real problem are, find original solution, and then discovering surely that indeed these some way are present in the brain or in other animals, or give inspiration of what to look for. But I always see engineering as rather autonomous process of discovery that is then having a lot of influence and create tools for thought for all the other disciplines. And situation where you just look at animals, what they are doing, or just look at the internal structure and then try to mimic that, that's actually not so many example in the history of technology where this had significant influence. So it's a bit, I mean, radical, and many other researchers are exactly defending the opposite view. But my feeling is that it's by trying to ask the right question and build the machinery using as many possibility as you want that usually makes the most interesting discovery, that then can be applied or give ideas when you look at animals or look at biological system or look at cultural systems. Yeah.
Q:
You mentioned the size of the body affects how you learn interaction and things. What other aspects of the body are crucial to learning, especially for perception and language?
Frederic Kaplan:
It's – your body is shaping all your skills, in a sense that – especially when you see development as a sequence, as a cascade of events. Then it's just like built one on top of another. So it's your relationship with the world that serves as the basic material, and then that serves metaphorically to learn other things later on. So it's really the – everything is embodied. There's no notion of abstraction. There's no notion of internal representation or symbolic representation or whatever. I mean, everything is based on the initial sensory motor relationship that you have with the world, and this relationship is what's used to actually build all the abstract thought that one may have later on. I mean, there's actually a very strong argument for that based on all the work that has been done on metaphors, or the work of George Lakoff, in particular, including his work with Nunez on mathematics. I mean, he's showing that even what may seem the most abstract or disembodied thing as mathematics is strongly linked with basic sensory motor skills, and that – it's essentially based on that. So that's also why – I mean, I've been basically starting from question about language and going back to question about joint attention, how typically would a one-year-old child learn, and stepping back to a three-month child. I mean, we've been just going back and back and back to more fundamental sensory motor skills, because in some way, if we don't understand them, jumping immediately to a higher-level skill is very unlikely to work. It's actually an intuition that Turing had in his famous 1950 article on computing, on the Turing test. I mean, he was saying that if actually a machine would ever succeed the Turing test, it has to learn like a baby. And that was just a note at the end, but it's actually fundamental and you have to take that very seriously in the sense that learning like a baby is not like mimicking how a baby is learning; it's coming from this very unstructured world in utero to building skills, which all builds on one on top of another, which are highly embodied. And luckily, I mean, you can do that now with robotics, and that is one of the main contributions robotics can do, is making the body viable – I mean, making it something you can change, making it something – you can have a situation where the software stays stable and the body change, and then understanding for the first time the influence of the body on some specific processes. No other sciences can do that, because they can just say that people that have different type of body have different skills or whatever, but they can't use this very precise trigger on the body structure as viable, and that's one of the main potential of robotics, yeah.
Luc Steels
Q:
What was it like working with Luc Steels?
Frederic Kaplan:
That was extraordinary years. I mean, working with Luc Steels is – I mean, he was really like a master for me, I mean, in terms – in the way he was writing, in his very large approach to science, in the way he was using arts also as part of the research process. I mean, half of the things we were doing were in museums, because there was places where we were just trying new things. And he had like a mix of fundamental research and ambition to be in the real world and ambition to contribute to problems larger than just the AI problems. That was amazing to me. And he was also – he had, I would say, a very trustful relationship with his students. I mean, in the sense that he was not using the students as slaves for doing his own research. I mean, he has his own research in some way, and he was really letting us grow and becoming independent scientists. So that was great years, definitely. In – what made me – so I stayed ten years at Sony.
What made me change essentially among other things is that Sony decided to stop the AIBO in ‘96 – in 2006, after seven years of commercial exploitation. And although, I mean that was the – absolutely great to be in the only company manufacturing entertainment robots for all this time. Although it was much less interesting, I mean, to be in the company which was robot, not make anymore. And so I thought that was a good time to move and I wanted, also, to do things maybe more concrete, more in the real world, more applied. And paradoxically, because all this work in the industry was actually very, very fundamental in terms of type of research and we were dealing with absolutely a strong belief amateur research question. And I thought, “Okay, I wanna do a things more applied. Let’s go back to the academia,” and I went to a EPFL on the invitation of Pierre Dillenbourg to join, like, a growing lab.
EPFL
Frederic Kaplan:
It was the CAF Laboratory, which at first may’ve been like a big shift because actually it was not in s – a robotic lab per se. I mean, it was a pedagogical lab where the idea was, “Can we make furniture, new kind of computers, that would – that we could introduce in learning environment as tools for students to learn?” And, yeah – and I was really thinking at that time that the future of robotics would not necessarily be small robot like the AIBO. I mean, although the AIBO – that was a success in some way. I mean, it also showed that, I mean, it was probably not the only way to have robotics coming into the home. And my hypothesis, which is still my hypothesis now, is that the wonderful technology we were building for robotics, especially for human/machine interaction for understanding context, for being able to act in the real world, which no other computer can do, then we can just translate them and apply them to things that we would not call robots when you see them but – which, in fact, would integrate through robotic technology. So robotic objects in a way.
And I started exploring that in that new context at – in Switzerland at EPFL, building interactive tables, moveable furnitures with lot of very exciting project. Also working very closely with industrial designers that were knowing how to make these objects. And one of the projects, which was very important for me, was the trial to make a robotic computer. Really, my conclusion was that robotic – that computer science had stayed very conservative, in terms of interfaces, for quite a long time – with the mouse and keyboard. And we were – at that point, there were no multi-touch yet – I mean, when I started robotic type of experiments – but we were feeling that there was com – that new type of interfaces were growing and I really thought that robotic had a role to play.
And so with the help of Martino d'Esposito who was an industrial designer I met, actually, during the Sony years when we were – we worked on building a robot playroom <laughs> with new objects and I was really impressed, I mean, of the creativity of that industrial design could have in terms of building things. We started to think about what could be the shape of a robotic computer. Essentially, we got inspired a bit by the iMac publicity, where you have, like – it’s an old iMac publicity where someone is passing in the street and the iMac is following that person and start to have a purely non-verbal exchange, just by moving its head. And all of that’s just like marketing. That was showing the force of non-verbal communication and what you could do if you had a computer that would move a bit. Not move in the house but just, like, be essentially a robotics screen. That was the guiding ideas.
And based on that, we got lucky to be chosen for an exhibition at the museum of modern art in New York. And that really launched the project because we had to produce in something like six or eight months an actual robot prototype that would stay there with – three month in an exhibit. And so we worked like <laughs> tremendously for all that – run j – I mean, passing from what was just like a drawing on the paper to actually an actual working prototype. And that first prototype called a "Wizkid" was really the starting point of interesting adventure, which is with the exploration of the space of robotic computers. So that was like a screen with a camera in front of it and then a robotic arm so it could, like, move in different directions. And at the museum, people would just pass by and it word turn to touch faces, turn towards them. Then you have a gesture interaction system where they could select thing on the screen at a distance. Go – like in kind of augmented reality system. And it was interesting because the Museum of Modern Art was an art museum. There was no, like, scientific explanation whatsoever so it had to be very fluent in terms of – very intuitive. Although at that time, nobody has done gesture interface of that sort. I mean, it was – you started to have the first multi-touch screens and people were kind of understanding that maybe there’s something to do, like, on surfaces but, like, gesturing in the air was very new. Now, people are more used to that but – and still all these visitors at the Museum of Modern Art, they got the idea quite rapidly. I mean, they were there, got very attracted by the robot, gesturing some were understanding, some were not.
OZWE
Frederic Kaplan:
But still, we were very, very impressed by how well that was working. And so based on that, we decided to create a company because I felt – I mean, the project benefit a lot from the EPFL ecosystem so that we could go to that level of an actual working prototype in a museum. But to go to the next level, like industrial production, even like small series, it was, like, yet another step to take. So we created a company named OZWE, O-Z-W-E to commercialize that product. We tried, as much as possible, to make it more looking like a sports car than a gentle friend, because there was, like, obviously a price problem and so we thought then that we should make it serious. It should be like – this is not like a toy because we can't price it like a toy. That was already a problem with AIBO that was actually priced – looking like a toy and priced like a computer. We thought, we need to be priced – unfortunately because of Swiss quality and all that, we need to be priced slightly upper so we need to have a design that goes with it. And so we came up with the second design and another name called QB1, which was much more like monolithic and looking really like a sculpture when you would see it from the – from an external point of view. And then, that sculpture was actually animated in a sense that it would turn towards you and you could have a whole relationship with it. It was very, very exciting. We also built our own 3-D camera. There was no, like, off the shelf 3-D camera at that point and so we built a new system based on post infrared, where basically the – well the – this device could have depth and it could actually in – see and perceive not only faces and object but actually the distance. And that was changing completely all the vision problem I was dealing with for 15 years, making them so much differently. I mean, so that means the – we had a computer that was a gesture-based computer that was really understanding its environment. I mean, he was understanding who was there. He was understanding what type of object was there. And so we started to build applications for that new settings, like what I call real-scale media. I mean, for me, this is a new type of media which are not like immersive media, like your smartphone or your tablet or books or television, where you go look at the screen and, you know, basically, leaving the space to go to that particular fictional space or virtual space. Robots make you stay in the same space. I mean, when you interact with a robotic dog, it can, comes to you. I understand the relationship between you and the dog and then you see that I’m interacting with it. We stay in the same space. We’re not – it’s not like you were reading a book and you basically go somewhere else.
So that notion of real-scale media, which is intrinsic to robotic – that was really what we were trying to push with that device. And showing, I mean, what it could mean for some different type of application, I was really personally convinced that the number of application you could do is as big as you – what you can do now on an iPhone or whatever. I mean, there’s a market there that if you extracted from robotics, which is as big in fact and which is things happening in the real world at real scale. So we started for music, for various type of augmentation. Typically, you’re in your kitchen, you’re finishing tomato sauce that – on your pasta, the tomato sauce is finished you just show it at a distance to the device. It recognize the shape and the type of sauce it is, put it automatically on your next shopping list, eventually suggest to you that maybe you should try the arrabbiata sauce the next time, et cetera and then you put it to the bin. I mean, it gets basically – it goes in the fabric of our everyday interaction. It’s much less interrupting than going to a smartphone or switching the computer, whatever. I mean, all these things are still things that are blocking our everyday interaction. Here it has smooth as another person in a room where you say, “Hey, by the way, remember this... Launch that music... Have a look at this,” this type of thing. So we produced that computer. We sold it before everybody else. I mean, before Microsoft came with its Connect system. I mean, they were, like, they were starting 2009 so that was really, probably, the first computer sold with that technology. It got the attention of many big groups actually and we – and among the first to buy – there were a couple of real, like, customers but really there was more the big groups that were interested. So Logitech bought one of – and quite soon after, Samsung got interested and also integrated that technology in their lab. Of course, you never know what happens when they <laughs> start integrating with a big industrial lab and I’ve been working at Sony and so I know how hard it is actually, even if you actually successfully bring an object like that into a line of product into a big company. But we were very – like, happy of that – that at least some people were getting the idea and we’ve been continuing demonstrating it, showing it and that was really a landmark. And still, I mean...
Q:
How many of them do you think you sold?
Frederic Kaplan:
Oh, I think we never went like – something like 30, you know, something. I mean, not so much – clearly, I mean, still, like, a very small production in many ways. It’s...
Q:
What do they cost?
Frederic Kaplan:
They were costing $10,000. So – which is a reasonable price, in terms of when you start a product like that, obviously for the big market. I mean, that was not at all competitive. But – so that was like – that was – yeah, that was an important step and...
Q:
Well, let’s go back a little bit.
Frederic Kaplan:
Can I interrupt you a second?
Q:
Sure.
<off topic conversation>
"Design and the Elastic Mind"
Q:
So I was just going to ask you was that the "Design and the Elastic Mind" show moment?
Frederic Kaplan:
Yeah, "Design and the Elastic Minds"
Q:
That was a great show.
Frederic Kaplan:
Yeah, many interesting things were at that exhibit. I mean, it was stuff with interesting things. So...
Q:
Yeah, I got to see it.
Frederic Kaplan:
...that was like a great opportunity and so to – yeah, to show things in the making and things that were making bridges between design and science. And we were really, I mean, Martino was like an industrial designer and I was like a computer scientist or something, so.
Challenges for Robotics
Q:
Since we’re limited on time, I’ll ask the important ones first and then I’ll jump back to some of the historic ones. So what would you say, on the road ahead, are the biggest technical challenges for robot objects and or collaborative learning with robotics?
Frederic Kaplan:
I think it will – I mean, the big thing is the integration of robotic technology in everyday life. And something which is changing now is the fact that, well, technology becomes available, it becomes available as kits or as element which promise – of people to probably tinker kind of new system of some sort. And a big, big new thing is the cloud, now. I mean, the fact that you can benefit from unlimited computer power, you can aggregate the learning on the experience of thousands of cameras, robots. We can – we start seeing that with, like, Google Goggles for instance. That's a start that should do really, really interesting result on the problem – on the very problem that I was tackling, like, 10 years ago when I was trying to teach my own robotic dog the name of an object. I mean, if you have, like, all the cameras in the world, all the systems that cont – that actually collaborate in sharing their databases of objects, you can start and you have, like, unlimited processing power. That type of problems becomes possible. So viewing the robots really as interfaces to the cloud, like what are computers now. Computers turn now to be just interfaces to the cloud. What counts is the possibility of reusing the same data over different interfaces. Robots are one of them and they have the possibility of acting a specific way in the world and they have the possibility of capturing some elements specifically in the world. A good example I think is how – I mean, you know what, for instance, Google Street View, which is in some way type of robotic technology. I mean, we have this perception technology. Well, you put a Roomba or, like, a robotic element in your house. You put the similar type of camera on top of it and you have almost turned, a search engine for the home, which means that I could then just type and find back an object at my place, having that type of system, just like monitoring where are the things. These are typically the type of frontier where the mobility’s important. Real-scale aspect are important. The fact of – I think the possibility of dealing with the mass of the real world is important. This is – this was not possible on small system because of the computing power. Now, with the fact that there’s basically a single computer, which is a world-wide computer, we’re gonna have robotic interface attached to it and this is, for me, the future of robotics.
Advice to Young People
Q:
What’s your advice to young people interested in careers in robotics?
Frederic Kaplan:
Well, there is obviously, I mean, very interesting academic journey. I mean, many difficult problems to solve, many interesting Ph.D. to be done, et cetera. For the one who wants to contribute to the real world, it’s sometimes more difficult because there’s not so many successful robotic company out there. I mean, there’s a couple but it’s not yet, I mean, that always anticipated boom where robotic is gonna be like computer science. So – well, I think it will come from small companies that would know how to actually trigger that power of the cloud and be smart in doing some full robotic interfaces – but maybe simple – and find the right equilibrium between what it cost and the benefit it gives, and that would really come from startup, I think. And so the obviously interesting, yeah, opportunities for young people who are more interested more in the, like, real world real application thing, not so much in the academia side, to go and find one of those startup and then try the adventure <laughs> with them. Of course, with – I mean, there will be a one percent of success only, among all the startups, as usual. But among them, they’re gonna be, I mean, the future key players in this new type of media which are robotic media.
Challenges of Startups
Q:
You’ve done a startup yourself. What are the big challenges for starting up your own robotics company?
Frederic Kaplan:
Well, the challenges is just listening to the market, the market, the market, the market and trying to forget about your dream robot that you wanna make. I mean, that’s fine for academia in many ways. You may have, like, big things to solve and challenge – big challenge you want – big problems you want to explore. But once you’re in the market, you should listen to the market all the time. That means trying really to solve the problems people really have. And be completely agnostic on forms. I mean, a robot should have the form to perform its function. There’s something very tricky when you start a startup – is that the press, for instance, always over-evaluate. I mean, you got a lot of attention all the time because the press is always interested in – about the future of robotics and always interest in about the fact that there’re gonna be small humanoids hanging around. But when you look at the history of successful robotic product, it’s always been product and very pragmatic, dealing exactly with a well-identifying problem and solving it. And part of the difficulty when you start a robotics startup is being pragmatic and saying, “Well, of course I would love to do that special type of robot. Of course even the press encourage me to do that. I will get a lot of attention if I do that.” But a couple of years later, “Oh, well.” I will just be out of business because there’s no market for that. So – well, just listening to the market and being, of course, inventive, but inventing pragmatically <laughs>.
Other Museum Projects
Q:
You mentioned you did some other museum work when you were in the Sony lab. Is that right?
Frederic Kaplan:
Yeah.
Q:
What were some of those projects?
Frederic Kaplan:
Oh, we worked with the Protocol Labratorium. Essentially, there was, like, extension of this robotic culture idea. I mean, the idea that you can have robots creating their own cultural phenomena, interacting with one of them, having this special thing where what they’ve been creating is more than the sum of all the robots, in a sense that if you suppress possibly the robots one by one from that community and you replace them with new robots that don’t know anything, then they still learn from that language, typically – which is something nobody is actually possessing. And that was of course, a fascinating idea as a – from an artistic point of view and from a, also, a philosophical point of view, I mean, what are our place? I mean, what is the difference between what make us human? Is that culture but what – or what? These stupid machines seem to have culture also. Does that mean that maybe animals have culture too? And et cetera, et cetera – so that type of question. And so we’ve been exploring that in different settings, essentially as extension of the talking head experiments, as different type of dialogues between machines, about things. There was a system where people were just, like, getting a snapshot of their eye and then there was the subject of conversation of the robots. We had other systems where people can go and change what was on white boards and the robot would talk about it. And so – and what was interesting is that all these system were connected with one another. I mean, there was one system in Antwerp, one system in Cambridge, one system in Tokyo and that was, like, the whole idea of installation that goes beyond installation – that are like kind of phenomena for which you have, like, a point of view. And I like a lot that idea and that’s the place where a museum can play a role showing that, yeah, it’s more than just showing one thing. It’s showing something which is happening at a world-wide level and that’s actually the difficult thing for people to understand – is how we shifted in 20 years to a situation where technology is world-wide. There a single machine which is a world-wide computer, accessible from anywhere and robots, what they do, is at a world-wide level. And so artistic installation, they should have this world-wide aspect. I mean, they should be in communication with other artistic installation, trying to grasp this globalization, which we still not having completely realized what it means.
Art and Robotics
Q:
And what do you think is the relation between art and robotics? You hinted at it there.
Frederic Kaplan:
I mean, artists and scientists have similar views in the sense that they want to say something. They want to demonstrate something and they have, in some way, to convince or to point about specific things happening in the world. They have a variation in criteria which are different, I mean, in the sense that scientist are judged by their peer and have to publish and where the artist have to be recognize in other ways. But apart from that, the questioning and even sometimes the methods are – there are a lot of similarities. So I would not, like, say, like, meeting between art and science and blah, blah, blah. I would just say they are like similar concern, similar type of discourses, different validation processes. But apart from that, it’s purely, permeable and it’s about saying something about the current society and saying something about its future. I mean, on that sense, it’s – there’s a lot of motivation, I would say, which are the same. That’s why, I mean, it’s so easy to make, in some way, a transition between good artistic installation and good scientific work. I mean, it’s – as long as things are done seriously. I mean, that’s why.
Collaborations
Q:
You mentioned a few but who are some of the collaborators you’ve had over the years?
Frederic Kaplan:
Oh, so many! So most – I mean, so after I worked with Luc Steels, I worked with one of his students which was Pierre-Yeves Oudeyer and we worked a lot together and build a lot of system. Most, actually, of our papers are cosign. I worked with Verena Hafner, a German woman. We worked for several years together. She’s now in Berlin also doing robotics. Arriving at EPFL, I worked with Pierre Dillenbourg on the pedagogical aspect and integration of technology into school orchestration, that notion or that – actually the teacher is like of orchestrating real time what’s happening in the classroom and that technology should be there to support the teacher, not to replace him. That’s a strong idea from Pierre, what we’ve been exploring together. And all my Ph.D. students here and my colleagues. It’s a huge team <laughs> and so I won’t start naming them because it’s gonna be too much – and Martino d'Esposito on the industrial design part.
And now my new venture I would say or my new topic of exploration is done with Laurent Bolli, who is a graphic designer, a graphic industrial designer and we work on book as machines. I mean, I’m convinced actually that the other "machinique" revolution that will come in the coming years is linked with books in some way. I mean, books have been going on the first mechanization process with the printing press and then offset et cetera, et cetera. Going into more regular forms, it has had tremendous impact on what we are, on the – on how our eyes are reading, on our sensory motor space. I mean, we know, I mean, that we’ve been profoundly changing the way we see our world. Not only in terms of ideas but in terms of reading skills and so in terms of our relationship, a very intimate relationship, with the world, since the arrival of books. And what’s happening now is a full mechanization of the book. I mean, it happened to the maps, for instance. Map are an interesting connective tool that you use for coordination re – there’s a history of map which is very similar to the history of books. I mean, there was just – they were hand-produced and they’ve been mass produced. Then they’ve been imprint, but made from digital data. I mean, all the digital part actually were done for the preparation and then they was just, like, printing the map. Okay? And now, they’ve been transitioning to full digital media like Google Earth, where you can zoom, you can have, like, other aspect. And what is interesting in that transition is not so much whether it’s more powerful. It’s that before, with a printed map, you can do – you had to learn how to use it but you can do also many other things with it. I could wrap something with a map or I could use it to protect myself from the rain or I – I mean, basically, interaction with the map was conventional but not mechanized. By the time it actually become fully digital, all – what you can do with Google Earth typically is absolutely predefined. I mean, there’s a lot of thing you can do but, outside of that, you can’t do anything. This is what’s happening to the book now.
It’s a kind of second revolution – or third or whatever but – to the book. I mean, when you – we start to have, to a situation where the book is fully mechanized from the production process, but its usage is still very free. I mean, you have, like with paper book, a lot of freedom in what you do with it. And you come to a situation where it got fully mechanized. That means that you can do interactions much more interesting with it, much more rich like a robot. I mean, like a full complex computer system. But also, it means that interaction become regulated, become part of the machine. That mean it has integrated the second part of its – not only this production rule but its usage rules. So this is basically what is currently happening. It’s a long process. I mean, people don’t always look at, well, whether the Kindle is more interesting than the iPad. I mean we’re not talking about that. I mean, typically, after the printing press, for 50 years, the book that were produce, they were exactly the same than the old medieval manuscript and they were just copying for 50 years. Nothing changed in terms of the user. But the system, in terms of machines, and some of the way the book was produce, was changing and that had a tremendous impact on our cultures. So one of my topic now is really seeing book as machines. And, again, translating all the thing we’ve been learning about robotics, about sensory motor skills, about embodiment with the topic of books. It’s something which is not yet discussed a lot and that’s why it’s interesting me and I’ve been starting talks with Laurent Bolli – a new venture on that called "Bookapp," where – book applications where we explore that both from a research point of view and now from a very pragmatic point of view at adapting existing books and seeing what will happen with that. So it’s – robotics leads to everything <laughs>.
Q:
Great. Anything else you’d like to add?
Frederic Kaplan:
No, I think that’s fine.
